Switch to EPP analysis covering all apps at once when compiling #2236
Conversation
This commit is a transition point that makes some assumptions about new callbacks for the compiler modules, which will likely not hold when it comes to making epp be able to read and handle multiple applications at once (so it can resolve parse transforms and behaviours across OTP apps). As such, a follow-up commit is going to be completing that one and changing its API in incompatible ways; if you find this commit during a bisect, it's probably not a good one in which to run a bisection.
While leaving the old one in place, prep the ground for new analysis phases for the DAG work. The new DAG functions are added, but not hooked in yet. Fixes to the EPP handling have also been added due to issues in resolving include_lib information
This moves the old DAG stuff out from the main code paths and substitutes it with the new epp-based DAG work. Also fixes previous work for integration tests: - consider ptrans from erl_opts in app ordering - display proper app compilation info messages due to reordering
|
Oh another gotcha: I didn't get rid of all the old ass code to parse attributes in |
|
Managed to get an OTP-18 version going... the problem is that I use |
- Can't deal with := in map type declarations - maps:update_with didn't exist back then so we reinvent it for the few months until we deprecate OTP 18
Reduces dependency verification time for large codebases.
rebar_compiler_dag: implement single-pass pruning
|
FWIW I've added a contribution from @max-au to this branch which optimizes the DAG purging further and improves incremental build performance, and includes a short fix we found to prevent dangling edges in prior builds from triggering newer rebuilds that no longer have these dependencies. |
THat was code written on a bus and rarely executed, but still the function called didn't exist for real.
This continues the Compiler refactor sequence in #2200 and switches code analysis to be done with EPP, which can understand macros and conditionally expand paths.
At the same time, this PR expands things to analyze all project Apps as one unit, which in turn allows properly detecting cases such as behaviours, parse transform, or include files changing in other applications forcing a re-build of the local ones.
The one limitation is that EPP won't easily allow the case where a public include file includes private include files, and the public include file is included by a foreign module; in such a case changes to the private headers won't be seen outside of their original app.
The apps are sorted in proper build order according to the new DAG, and should allow for interesting parallelism options down the line as well.
Preliminary tests on larger code bases seem to imply that this slightly slows down the initial build but significantly speeds up the following partial builds.