[rush] Feature idea: split build and test into separate tasks

[elliot-nelson]

Summary

For projects with long (60+ second) unit test phases, allowing the task runner to split building and testing into separate phases would potentially increase parallelism and lower overall build time.

Details

Say you have dependency chain A --> B --> C, where each project takes 60 seconds to compile and then another 60 seconds to run jest. If I run rush build --to C on my MacBook, it can execute a "maximum of 16 simultaneous processes", but it's going to do 1 heft build at a time for 6 minutes total.

But, if the task runner had the ability to split each project into two phases (something like: heft build-only and heft test-only), it could interleave them, for a total build time of 4 minutes (give or take).

  A (build)  --->  B (build) ---> C (build) --> C(test)
              \               \
               ->  A (test)    -> B (test)

If a test phase fails for a project, you could retroactively mark the corresponding build "failed" in the appropriate data structure, and either cancel/abort any dependent builds that are already running, or let them finish but preemptively mark them skipped. (Generally unit test failures still let any other projects that aren't dependent on the failing project build, so you wouldn't exit rush build altogether.)

I'm not sure how a project would tell rush that it's OK to split up its build and test phases. I'd like to say "if a project has build and test scripts it's OK", but that would potentially be a big breaking change, since existing projects could be configured in any way. (For example, many projects in our monorepo set both build and test to heft test --clean, etc.). Also, you might want to leave alone regular developer's habits of running "rushx build" or "rushx test".

Probably the safest way to introduce this feature is to add a rush-project.json setting, where you can specify yourself what rushx commands represent building and testing:

// Enable parallel tests in the task runner for this project, using
//   rushx build-only
// and
//   rushx test-only
// for the two phases
{
    "parallelTests": {
        "buildCommand": "build-only",
        "testCommand": "test-only"
    }
}

Standard questions

Question	Answer
@microsoft/rush globally installed version?	N/A
rushVersion from rush.json?	N/A
useWorkspaces from rush.json?	Yes
Operating system?	Mac
Would you consider contributing a PR?	Yes
Node.js version (node -v)?	12.17.0

[elliot-nelson]

Actually, thinking about this more, I'm probably making it too build-specific.

"rush build" is just code for "rush, run rushx build in every project, tracking dependencies for me". So maybe the customization I want is actually at the command-line.json level -- a way to say that "rush build" is an alias for "rush build-only test-only".

(but, we still need rush to understand that build-only and test-only are two phases, one dependent on the other, and that the build-only phase blocks the build-only phase of dependent projects but the test-only phase does not.)

[elliot-nelson]

(Linking so I don't lose them: was chatting with Pete and he pointed me at #2299 and #2300, I'll read up on those and possibly close this issue as a duplicate!)

[elliot-nelson]

All of my ideas are already present in #2300, so closing!