Repository navigation
[rocjitsu][ci] Update benchmark workflow for the sim perf dashboard - #13663
Conversation
RattataKing
commented
Oct 16, 2026
- Bump the benchmark corpus pin
- Dashboad v1 is still in the progress, updated benchmark workflow will collect data first and present partial data to match the existing dashboard's format
- ISSUE ID: [rocjitsu] Collect longer benchmark candidates and local runtimes for V1 #12611
2b49048 to
5904288
Compare
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
Pre-commit check failed⛔ pre-commit failed Please run locally:
This repo uses |
|
🎉 All checks passed! This PR is ready for review. |
kuhar
left a comment
There was a problem hiding this comment.
Can we remove the unused scaffolding so these interfaces show what is actually exercised?
- In
emulation/rocjitsu/website/dashboard/scripts/dashboard-data/tests/test_benchmark_result.py:5,63,85,importlib.utilandmanifest()are unused, andcontext()is consumed only forsource_subject. Simplifying that fixture also removes unused revision/context fields that look as though they are being checked. - At
.github/workflows/rocjitsu-benchmarks.yml:44, themachineandsource_commit_messagejob outputs have no downstream consumers; their step outputs already feed the converter locally. - In the test file at line 235, the removed-script-name blacklist mostly records intermediate implementation choices. Assertions about the intended publication artifacts and validation would protect the contract more directly.
kuhar
left a comment
There was a problem hiding this comment.
I think it would help to make these helper names and test fixtures more explicit:
- In
emulation/rocjitsu/website/dashboard/scripts/dashboard-data/prepare-benchmark-result.py:29,33, use names such asparse_timestampandbuild_published_run: the former parses a datetime, and the latter builds a whole run, whereas the dashboard contract uses “result” for a per-test record. - In
emulation/rocjitsu/website/dashboard/scripts/dashboard-data/tests/test_benchmark_result.py:15, nameROOTasREPO_ROOTto identify which root the seven-parent traversal targets. - In the same test file at line 104, explain that the unusual indentation and trailing whitespace deliberately detect reserialization. The publication tests require the archived raw JSON to preserve those bytes.
kuhar
left a comment
There was a problem hiding this comment.
I think it would help to make these remaining names more explicit:
- In
emulation/rocjitsu/website/dashboard/scripts/dashboard-data/tests/test_benchmark_result.py:411, renametest_workflow_shell_and_removed_pathsto something liketest_workflow_shell_and_publication_policy; its body now checks shell syntax, publication policy, dispatch inputs, and checkouts, with no removed-path assertions. - In that test file at lines 18–21, use
SOURCE_SHA(orROCJITSU_SHA) andCORPUS_SHAforSHAandCORPUS. With separate workflow and Tensile revisions in the fixture, identifying each revision at its declaration makes the provenance assertions easier to follow. - In
emulation/rocjitsu/website/dashboard/scripts/dashboard-data/prepare-benchmark-result.py:23, a name such asencode_json_byteswould make the format and return type ofencodedclear at its hashing and file-writing call sites.
kuhar
left a comment
There was a problem hiding this comment.
Left an inline reply and 3 new comments
kuhar
left a comment
There was a problem hiding this comment.
A few comments, plus an inline reply
kuhar
left a comment
There was a problem hiding this comment.
Left an inline reply and a new comment
TheRock Submodule Bump ActivityNewest first
|