Repository navigation
fix(OMN-14081): make pyproject.toml test-selector trigger content-aware - #1393
Conversation
The governed change-aware test selector escalated EVERY omnibase_core PR to the full 40-shard suite (~22.5 min, 40 runner slots) with full_suite_reason=test_infrastructure, because `pyproject.toml` was a bare path-prefix trigger in test_selection_adjacency.yaml with zero content inspection. Since core is the foundational released package, nearly every PR touches pyproject.toml for a version bump or entry-point registration, so nearly every PR escalated. ENABLE_SMART_TESTS=true is live, so this was production CI behavior. Changes (detect_test_paths.py + test_selection_adjacency.yaml + ci.yml): - Content-aware pyproject.toml classification via a base-vs-head TOML diff (classify_pyproject_dependency_relevant). Narrow ONLY when the change is confined to metadata-only [project] keys (version, entry-points, scripts, urls, description, classifiers, license, authors, ...). Escalate on any change to the dependencies array, [project.optional-dependencies], [dependency-groups], [build-system], [tool.*] (incl. pytest/coverage config), requires-python, or any unrecognized key. - FAIL-CLOSED by construction: missing base/head content, a TOML parse failure, or no --base-ref supplied all escalate. It is an allow-list of SAFE keys, not a block-list of dep tables — unknown keys escalate. - Remove .github/workflows/ from test_infrastructure_paths (align to omnibase_infra). A workflow-only change carries zero test-code delta, is exercised on the PR's own CI run, and is backstopped by the unconditional merge_group -> main full suite (root CLAUDE.md Rule #4). scripts/ci/ is KEPT (conservative — it houses the selector itself). - Wire --base-ref (from github.event.pull_request.base.sha) into ci.yml's detect-changes step so the classifier can read the base pyproject.toml. Proof on the real version-bump PR #1388 (0.46.5 -> 0.46.6): BEFORE: is_full_suite=true, reason=test_infrastructure, 40 shards AFTER: is_full_suite=false, narrowed Typical case (1-module fix + version bump + lockfile): 40 shards -> 1 shard. Control (real dependency add): still escalates (fail-closed preserved). Adds 24 tests (classification unit tests, compute_selection escalation paths, and CLI end-to-end tests against a real tmp git repo). Full local gates green: ruff format+check, mypy --strict, pre-commit (97 hooks). Parent epic OMN-13969 (WS-T selective-testing revival).
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 45 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Summary
Closes OMN-14081 (parent epic OMN-13969, WS-T selective-testing revival).
omnibase_core's governed change-aware test selector escalated every PR to the full 40-shard suite (~22.5 min, 40 runner slots) with
full_suite_reason=test_infrastructure, becausepyproject.tomlwas a bare path-prefix trigger intest_selection_adjacency.yamlwith zero content inspection. Since core is the foundational released package, nearly every PR touchespyproject.tomlfor a version bump or entry-point registration, so nearly every PR escalated.ENABLE_SMART_TESTS=trueis live, so this was production CI behavior — the single biggest demand-reduction lever on core's queue.Changes
pyproject.tomlclassification (detect_test_paths.py): a base-vs-head TOML diff (classify_pyproject_dependency_relevant) narrows only when the change is confined to metadata-only[project]keys (version,entry-points,scripts,urls,description,classifiers, license, authors, …). It escalates on any change to thedependenciesarray,[project.optional-dependencies],[dependency-groups],[build-system],[tool.*](incl.[tool.pytest.ini_options]/[tool.coverage]),requires-python, or any unrecognized key.--base-refsupplied all escalate. This is an allow-list of SAFE keys, not a block-list of dep tables — an unknown/new pyproject key escalates by default. Shared-module / test-infra / threshold escalation is unchanged..github/workflows/fromtest_infrastructure_paths(align to omnibase_infra, which omits it). A workflow-only change carries zero test-code delta, is exercised on the PR's own CI run, and is backstopped by the unconditionalmerge_group→mainfull suite (root CLAUDE.md Rule TKT-003: Validate Import Resolution and Eliminate Unused Dependencies #4).scripts/ci/is KEPT (conservative — it houses the selector itself). This addresses the merge_group-trim workflow-only PR case.--base-ref(fromgithub.event.pull_request.base.sha) intoci.yml's detect-changes step so the classifier can read the basepyproject.toml.Before / after proof (real version-bump PR #1388,
0.46.5 -> 0.46.6)is_full_suite=true, reason=test_infrastructure, 40 shardsis_full_suite=false, narrowedDominant real-world case (1-module fix + version bump + lockfile): 40 shards → 1 shard (
tests/unit/cli/).Control — a real dependency add: still escalates (fail-closed preserved).
Tests
Adds 24 tests: classification unit tests (version bump, entry-point, metadata, real dep add, optional-deps, dependency-groups, build-system, uv.sources, requires-python, pytest-config, malformed/missing → fail-closed);
compute_selectionescalation paths (relevant True/False/None, shared-module non-suppression, ignored-when-absent); and CLI end-to-end tests against a real tmp git repo (--base-ref+git showchain).Local gates green:
ruff format+ruff check,mypy --strict,pre-commit(97 hooks),pytest tests/unit/scripts/ci/(87 passed).dod_evidence
test_infrastructure, NEW=narrowed).pytest tests/unit/scripts/ci/— 87 passed.Evidence-Source: OCC#3664
Evidence-Ticket: OMN-14081