ci: register new Python tests automatically at commit time

[teamleaderleo]

Summary

A new tests/test_*.py fails the preflight guard until someone pastes a [[test]] block into tests/test-execution.toml. This happens even when ci-guards.yml already runs the file and the validator has already worked out the lane. #14148 hit exactly this.

After this change, adding a test and its ci-guards.yml step is all a commit needs:

• python3 scripts/ci/validate_test_execution_registry.py --write registers every unregistered test that a workflow already runs, on linux-guard, then validates as before.
• The pre-commit hook (scripts/git-hooks/pre-commit, installed by ./scripts/setup.sh) runs --write whenever a commit stages a test or workflow file, and stages the updated registry. It leaves the registry alone if the registry has unstaged edits. It never blocks a commit, because CI still validates.
• A test that no workflow runs is not registered. Someone has to decide where it executes. CI still fails on it with the paste-in hint, and the hint now also mentions --write.

Testing

• python3 tests/test_ci_test_execution_registry.py passes. Two new tests check that --write:
  • registers a test a workflow runs, the result validates, and a second run is a no-op
  • leaves a test no workflow runs untouched
• End-to-end run of the hook: I committed a new test plus its ci-guards.yml step in a throwaway clone with the hook installed. The hook printed registered new tests in tests/test-execution.toml, the commit included the registry entry, and validate_test_execution_registry.py --base-sha HEAD~1 passed.
• validate_test_execution_registry.py --write on current main changes nothing, and the registry is valid.

Checklist

• I tested the change locally
• I added or updated tests for behavior changes

🤖 Generated with Claude Code

https://claude.ai/code/session_01RSxoKWiT1D35DDmEYxk59f

---

Generated by Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

---

Summary by cubic

Automates registration of new Python tests in tests/test-execution.toml so adding a test and its ci-guards.yml step is all a commit needs, instead of the preflight guard failing until someone pastes in a [[test]] block.

• validate_test_execution_registry.py --write registers every unregistered test a workflow already runs, then validates as before.
• The pre-commit hook runs --write when a commit stages test or workflow files and stages the updated registry; it skips if the registry has unstaged edits and never blocks a commit.
• Tests no workflow runs are left unregistered for a person to place; CI still fails on them with the hint now also mentioning --write.

^{Written for commit d0a766c. Summary will update on new commits.}

Summary by CodeRabbit

• New Features
  • Added automatic registration for unlisted tests when a workflow already runs them. Tests without a clear workflow placement remain unregistered for manual review.
  • The pre-commit hook can sync the test registry when relevant test or workflow changes are staged.
• Bug Fixes
  • Registration errors are reported, and the validation command exits with a failure status when registration cannot be completed.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 14fb2b70-9d82-4e1d-b56c-b324c582d630

📥 Commits

Reviewing files that changed from the base of the PR and between f70a62b and d0a766c.

📒 Files selected for processing (3)

• scripts/ci/validate_test_execution_registry.py
• scripts/git-hooks/pre-commit
• tests/test_ci_test_execution_registry.py

 _________________________________________________________________________________________________
< Errare Humanum Est, Perseverare in Debugging. To err is human, to persist in debugging, divine. >
 -------------------------------------------------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}