build runs very slow when export numbers are high

[waterlike86]

i found that the function function_tables_and_exports takes very long when the export numbers are high.
most of the time is spent in function check_all_implemented to check for missing functions.
in our project where the number of exports is > 70000 this function can easily take up to 15 mins or more

maybe we can make this check parallel? and possibly improve the execution logic.

[waterlike86]

cc @kripken @awtcode

[kripken]

I'd try to speed this up with early exiting in is_already_implemented - avoiding the string search might help. Or perhaps we should do a single search up front for all patterns function X, and then the check can be much faster using that.

[waterlike86]

i first subtracted the 2 arrays, remove all which are already in all_implemented and looped through the rest to perform the string search. this still took half the time which is close to 10mins.

1 more question, do we need this check in dynamic linking? perhaps its not that useful when its a side/main module?

[awtcode]

One suggestion is to copy the larger array into a map so that all the accesses will be in constant time and instead of looping thru the arrays and logging the elements one by one, just store all of them into a string and log them at one go.

[waterlike86]

@kripken i opened a PR which helps with this

[stale]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.