chore(test): Run ram_blowup_regression only on CI#9717
chore(test): Run ram_blowup_regression only on CI#9717
ram_blowup_regression only on CI#9717Conversation
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Test Suite Duration'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.
| Benchmark suite | Current: af2f824 | Previous: 8fd9446 | Ratio |
|---|---|---|---|
test_report_zkpassport_noir-ecdsa_ |
3 s |
2 s |
1.50 |
test_report_zkpassport_noir_rsa_ |
2 s |
1 s |
2 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @TomAFrench
|
I think this means this test will never run, as it currently doesn't run in CI: Lines 1 to 6 in 910cd93 |
|
Maybe it could only run in nightly? Not sure how that runs but I know there's something that runs nightly from which we get fuzz failures? |
|
We run this test in CI on the master branch. We just disable it for PRs to make them quicker to merge. I'd like to lean into just using nextest profiles if we want to vary how tests are run under certain circumstances. |
|
Okay thanks, I'll use nextest to pick a profile locally 👍 |
|
I do agree that |
Description
Problem*
Executing integration can be slow, in particular
ram_blowup_regressiontakes up a disproportionate amount of time.Summary*
Changes the
nargo_cliintegration tests to excluderam_blowup_regressionunless theCIenv var is set.Additional Context
Say I want to have an idea whether commenting out some code in an SSA pass breaks any of the integration or regression tests. I can run the following command, which runs almost 700 tests; way less than the total (over 5000). With this PR, this takes just over 15 seconds on my laptop.
cargo test -p nargo_cli --test execute forcebrillig_false_inliner_i64_minHowever if I include the
ram_blowup_regressions, this goes up to almost 2 minutes, which means I'm less likely to run the tests:(It is not exactly clear why it's so slow, because if I
cdinto the directory and usecargo run -p nargo_cli -- execute --forcethen it's fast).Documentation*
Check one:
PR Checklist*
cargo fmton default settings.