Track build artifacts in DAG #2246
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This allows to do quicker re-compile option validation by not requiring to access the disk and check all candidate erlang files for option changes. This also opens up the way to compilers that produce more than one artifact per file.
The propagation was confusing source files and artifacts; the artifact ordering was flipped, and the tagging non-mandatory (aside from as edges), which made things hard to identify when plugins for compilers are used.
now all sources have at least one edge with 'artifact' label, let's ignore it when choosing whether to compile in parallel or sequentially
Also fixes artifacts left on filesystem, and produces less filelib:last_modified() checks.
fix parallel compilation
tsloughter
approved these changes
Apr 3, 2020
|
Yeah part of it there is that this should sooner or later let me (in the next phase) go in and drop some of the beam file reading that is taking place to instead rely on the artifact tracking in the DAG. I think @max-au had a few optimizations he wanted to add on top of it, but ended up digressing into rewriting a minimal form of his own compiler there; we'll re-sync sooner or later to get to a more steady state. |
- topsort of an app does a full analysis of all apps to come up with their app-level DAG - each extra_src_dir has its own analysis run - each extra_src_dir run is simulated as being an app - we therefore end up doing a lot of useless analysis for single app runs This patch notes runs with 1 or 0 apps and skips doing the topsort analysis on them, which speeds things up further.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This allows to do quicker re-compile option validation by not requiring
to access the disk and check all candidate erlang files for option
changes. This also opens up the way to compilers that produce more than
one artifact per file.
This is not exactly optimal on all fronts but should still be faster than the old thing.
Related to #2200