Skip to content

fix(OMN-15263): supply DEV_REDPANDA_ADVERTISE_HOST in every compose-render fixture + generalize the required-env coverage gate to all four - #2504

Merged
jonahgabriel merged 1 commit into
devfrom
jonah/omn-15263-compose-render-env
Jul 27, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
jonah/omn-15263-compose-render-env

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jul 27, 2026 •

Copy link
Copy Markdown
Collaborator

Ticket

OMN-15263 — OMN-15173's DEV_REDPANDA_ADVERTISE_HOST :? fail-fast breaks every compose-render test; the required-env coverage gate is blind to 3 of the 4 render fixtures.

Root cause

#2495 (OMN-15173) switched docker/docker-compose.infra.yml to the compose :? fail-fast form for DEV_REDPANDA_ADVERTISE_HOST. That change is correct and stays. Redpanda's command: block is interpolated by docker compose config regardless of --profile, so every render that layers the base infra file now exits non-zero unless the var is set — and #2495 added it to exactly one of four render fixtures. The other three build their env hermetically (env= replaced, not inherited).

Blast radius, measured on omnibase_infra#2502 (run 30290604529, job 90060230679): 12 failures — 1 judge + 1 prod + 10 stability-test — all subprocess.CalledProcessError from docker compose config, cascading Tests (Split 2/15) → CI Tests Gate → Test-Failure Ratchet Gate → the required CI Summary. Reproduced identically on #2500 on different hosted runners. Invisible locally: the render tests skip where docker compose is absent, so a clean full local suite was honest but blind.

Changes

1. Supply the var in the three hermetic render fixtures (test_prod_runtime_compose_render.py, test_judge_compose_render.py, test_stability_test_runtime_compose_render.py), value localhost, # kafka-fallback-ok — test fixture, matching #2495's own fixture. The compose fail-fast is byte-unchanged — docker/ is not touched by this PR at all. The wrong fix here would be handing compose a :-localhost default back; that resurrects exactly the silent off-host regression OMN-15173 removed.

localhost specifically, not a synthetic hostname: prod and stability both assert the base command: was actually overridden by the overlay ("localhost:19092" not in redpanda_command, "100.109.203.94:49092" not in ...). A non-localhost dummy would silently defeat that leak detector.

2. The actual defect — generalize tests/ci/test_compose_required_env_coverage.py. That gate exists precisely to catch "a :? var was added to compose but not to the fixture" (OMN-5240) and did not fire, because FIXTURE_FILE was hardcoded to one of four call sites. It now:

  • checks every registered render fixture, parametrized so a failure names the specific fixture and the specific var;
  • computes each fixture's supplied set as module-level env dict keys (AST) ∪ vars defined in the --env-file files that fixture passes to compose — judge legitimately gets 4 of its vars from docker/judge.env.example, so a naive dict-only check would have been a false positive on day one;
  • fails closed on discovery: any tests/integration/**/*compose_render*.py module not in RENDER_FIXTURES is RED. A fifth render fixture cannot silently escape coverage the way three just did;
  • carries an explicit intentionally_unset hatch, used by exactly one entry: test_dev_runtime_compose_render.py must not set the var, because its OMN-15173 counter-test proves the unset render fails. The hatch is itself gated — test_intentionally_unset_vars_are_justified requires the var to still be :?-required in compose (stale exemptions go RED), requires a real reason string, and requires the exempted fixture to actually mention the var.

3. Counter-test for OMN-15173 — test_dev_advertise_host_keeps_fail_fast_form. Reverting compose to ${DEV_REDPANDA_ADVERTISE_HOST:-localhost} would also turn every render green, while restoring the regression. That shortcut is now RED, and it is static, so it fires on hosts with and without Docker.

4. tests/integration/docker/test_docker_integration.py: the render env dict is lifted out of the test body into module-level COMPOSE_CONFIG_RENDER_ENV so the gate extracts all four fixtures uniformly. Pure move — same keys, same values.

Seams

  • compose :? var name → fixture dict key. Matched field-by-field by the gate itself, for all 5 registered fixtures, not by two independent suites. The gate is the cross-boundary regression test for this seam.
  • fixture → --env-file. The gate parses each fixture's own --env-file arguments and reads those files, so it tracks drift instead of hardcoding a list; a fixture pointing at a non-existent env file is an assertion failure.
  • module-level dict, by AST. Only module-scope dict literals with str keys count — a dict nested in a test body is never silently credited as coverage, which is the shape that let the docker-integration fixture look covered.

Verification

RED-first (all captured before/after, artifacts in the session scratchpad):

Proof Result
New gate with fix (1) reverted 3 failed — test_all_required_compose_vars_in_fixture[prod|judge|stability], each naming DEV_REDPANDA_ADVERTISE_HOST and its fixture
New gate after fix (1) 8 passed
Old gate vs pristine dev @ 6bc1e7ff 43 required / 0 missing → old gate PASSES while CI is red — the mechanism defect, demonstrated
Simulated new :? var in compose, no fixture touched 5 failed (all fixtures) — ticket acceptance criterion 3
Compose reverted to :-localhost 2 failed — test_dev_advertise_host_keeps_fail_fast_form + the stale-exemption check
Unregistered *compose_render*.py added 1 failed — fail-closed discovery

Real docker compose config renders on a Docker host (non-mutating config only — no up, no lane touched), using the exact fixture env dicts imported from these modules:

prod       without_fix  exit=1 RED   required variable DEV_REDPANDA_ADVERTISE_HOST is missing a value
prod       with_fix     exit=0 GREEN
judge      without_fix  exit=1 RED   required variable DEV_REDPANDA_ADVERTISE_HOST is missing a value
judge      with_fix     exit=0 GREEN
stability  without_fix  exit=1 RED   required variable DEV_REDPANDA_ADVERTISE_HOST is missing a value
stability  with_fix     exit=0 GREEN
dev-lane   unset        exit=1 RED   required variable DEV_REDPANDA_ADVERTISE_HOST is missing a value
dev-lane   explicit     exit=0 GREEN advertises 198.51.100.50:19092

The last two rows are the counter-proof: the OMN-15173 runtime fail-fast still fires for a real dev-lane bring-up path after this fix, and an explicit value is still honored verbatim.

Gates — run on .200 per rule 11a, patch-transferred with per-file sha256 verified equal on both hosts before the run (no vacuous green from the edit-locality trap):

  • ruff check + ruff format --check over tests/ci, tests/integration/docker, tests/integration/infra — clean
  • mypy tests/ci/test_compose_required_env_coverage.py — clean
  • pre-commit run --files over all 6 changed files — clean, no hook modified a file
  • pytest gate + all 4 render fixtures — 8 passed, 22 skipped; pytest docker-integration + kafka-env unit — 10 passed, 15 skipped

Stated plainly, not counted as a pass: the 12 render tests skip on .200 and on this Mac (no docker compose — this is the exact blindness that let the regression land). Their authoritative GREEN is the .201 real docker compose config render table above plus the dedicated hosted-runner Compose Required-Env Coverage (OMN-5439) job (run 30296123866, job 90081728592) exercising all 5 parametrized fixtures. Corrected 2026-07-27T19:4xZ by independent verification: an earlier revision of this line claimed the green came from this PR's own Tests (Split 2/15). That is false — the governed selector emitted {"selected_paths":["tests/ci/"],"is_full_suite":false} (Detect Changes job 90082641865), so Tests (Split 1/1) ran tests/ci/ only (975 passed) and none of the five changed tests/integration/** render modules were collected on their own PR. That selector gap is recorded as a second instance on OMN-15245. A skip was never counted as a pass.

Notes

Refs: OMN-15263, OMN-15173 (#2495), OMN-5240, OMN-15234 (#2502), OMN-15255 (#2500), OMN-13969/13970

Evidence-Source: OCC#5166
Evidence-Ticket: OMN-15263

…ender fixture + generalize the required-env coverage gate to all four

OMN-15173 (#2495) switched docker-compose.infra.yml to the compose `:?`
fail-fast form for DEV_REDPANDA_ADVERTISE_HOST and added the var to exactly
one render fixture. Redpanda`s `command:` block is interpolated by
`docker compose config` regardless of `--profile`, so every layered render
started exiting non-zero on hosted CI: 12 failures in Tests (Split 2/15),
cascading to CI Tests Gate -> Test-Failure Ratchet Gate -> the required
CI Summary, on any PR whose selector escalated to the full suite.

Fix (1): supply the var (render-only `localhost`) in the three hermetic
render fixtures - prod, judge, stability-test. The compose fail-fast is
UNCHANGED; nothing regains a silent default.

Fix (2), the actual defect: tests/ci/test_compose_required_env_coverage.py
exists to catch "a `:?` var was added to compose but not to the fixture"
(OMN-5240) and did not fire, because FIXTURE_FILE was hardcoded to one of
four render fixtures. It now checks every registered fixture (env dict keys
union `--env-file` contents), fails closed on any unregistered
`tests/integration/**/*compose_render*.py` module, and carries an explicit,
justified `intentionally_unset` hatch for the dev fixture that must NOT set
the var (its OMN-15173 counter-test proves the unset render fails).

Also adds test_dev_advertise_host_keeps_fail_fast_form: giving compose a
`:-localhost` default back would turn every render green while restoring the
off-host regression OMN-15173 removed. That shortcut is now RED, on hosts
with or without Docker.

tests/integration/docker/test_docker_integration.py: the render env dict is
lifted from the test body to module-level COMPOSE_CONFIG_RENDER_ENV so the
gate extracts it the same way as the other three. Pure move, same values.

Evidence: RED (fix 1 reverted) = 3 parametrized gate failures naming
DEV_REDPANDA_ADVERTISE_HOST and each fixture; GREEN after = 8 passed. Real
`docker compose config` renders on a Docker host: prod/judge/stability all
exit 1 without the var and exit 0 with it; dev lane still exits 1 when the
var is unset and honors an explicit value. Old gate re-run against pristine
dev: 43 required / 0 missing - it passed while CI was red.
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 21 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 455a5465-e53a-495b-a091-46a056f3a9dd

📥 Commits

Reviewing files that changed from the base of the PR and between 6bc1e7f and 1e9b7eb.

📒 Files selected for processing (6)
  • tests/ci/test_compose_required_env_coverage.py
  • tests/integration/docker/test_docker_integration.py
  • tests/integration/infra/test_dev_runtime_compose_render.py
  • tests/integration/infra/test_judge_compose_render.py
  • tests/integration/infra/test_prod_runtime_compose_render.py
  • tests/integration/infra/test_stability_test_runtime_compose_render.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-15263-compose-render-env

Comment @coderabbitai help to get the list of available commands.

jonahgabriel added a commit to OmniNode-ai/onex_change_control that referenced this pull request Jul 27, 2026
#5166)

* evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#2504

* evidence: OCC companion self-bind for #5166

---------

Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8001 ] unreachable — preflight short-circuit (no models available)


Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical findings No
blocked CRITICAL findings found Yes
degraded All models unavailable (infra) No (pilot)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8468/OMN-8524)

@jonahgabriel
jonahgabriel merged commit b1bda59 into dev Jul 27, 2026
165 of 175 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-15263-compose-render-env branch July 27, 2026 19:38
jonahgabriel added a commit that referenced this pull request Jul 27, 2026
…ector (#2505)

* fix(OMN-15245): fail-closed changed-test coverage in the governed selector

The governed selector could drop a test file the diff itself changed. Two
recorded live instances, both green CI runs that never collected the changed
test modules:

  * OMN-15218 / #2493 -- a scripts/ + tests/scripts/ diff selected
    ["tests/unit/"] (22053 tests, none of them the 47 new ones).
  * OMN-15263 / #2504 -- a six-test-file diff selected ["tests/ci/"]
    (run 30296123866, Detect Changes job 90082641865); the five changed
    tests/integration/** modules were never collected on the PR that
    existed to repair them.

Invariant: any CHANGED path under tests/ is covered by the emitted selection
(its own directory at minimum). Narrowing may add tests, never drop one the
diff touched. Applied last in _resolve() so it sees every other mapping.

Also:
  * scripts/** now maps to tests/scripts/ + tests/unit/scripts/ -- the two
    families that actually exercise scripts/. Previously scripts/ produced no
    selection at all and fell through to the blanket tests/unit/ fallback.
  * New CHANGED_TEST_UNNARROWABLE full-suite escalation: a changed test module
    directly under tests/ has no containing directory below tests/ itself.
  * UNRUNNABLE_TEST_PREFIXES documents the families the pytest job structurally
    cannot run (tests/integration/docker/ is --ignored by both pytest steps and
    has its own gate in docker-build.yml; tests/chaos/ and tests/performance/
    are marker-deselected). Selecting them cannot make them run and would make
    pytest exit 5 when one is the sole selected path.
  * Consumer seam: prepush_smart_tests.sh filters tests/integration/ out of its
    pytest invocation (it also passes --ignore=tests/integration), so the new
    integration selections cannot wedge a push on exit 5.

RED-first, exists-but-wrong (not absence): the new tests fail 14/14 against the
pre-fix selector, including both recorded replays; the hook seam test fails 2/2
when the filter pattern is wrong rather than missing.

* chore(OMN-15245): re-trigger CI after Evidence-Source: OCC#5190 landed in the PR body

---------

Co-authored-by: jonahgabriel <jonahgabriel@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant