Skip to content

fix(OMN-15779): classify seam-graph observations production-application vs test-harness-in-src (R1 audit) - #2043

Merged
github-actions[bot] merged 1 commit into
devfrom
jonahgabriel/omn-15779-r1-production-vs-harness-split
Aug 10, 2026
Merged

github-actions[bot] merged 1 commit into
devfrom
jonahgabriel/omn-15779-r1-production-vs-harness-split

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Why

RSD lane-2 audit finding R1 (2026-08-10, [lane:rsd-corrections-15767]): the corpus-wide producer_send/consumer_subscribe counts this ticket's own extractor produces (OMN-15779's symbol-resolution landing, #2041) mix real inter-service application seams with test substrate that is shipped inside a scanned src/ tree rather than under tests/ — e.g. omnibase_core's event_bus/testing/contract_event_bus_substrate.py (18 hits) and runtime/transport/runtime_transport_conformance.py (6 hits). "Production" in the prior derivation meant non-self-hit, not inter-service application seam. The finding explicitly states a second split is required before any gate or golden-generation work (OMN-15756, OMN-15767) consumes these numbers.

This PR is that precursor split — required before OMN-15767 (golden generation) or OMN-15756 (the PR-time gate) can honestly consume corpus counts.

What changed

src/omnimarket/seams/models/model_seam_graph.py:

  • ModelSeamGraphCodeObservation.is_test_harness — a @computed_field derived from source_path (always correct, never a constructor input that could drift or be forgotten). Matches event_bus/testing/**, runtime/transport/*conformance*, and defensively tests/** (already excluded upstream in every real caller today — no discovery root passed by node_seam_graph_compute includes a tests/ dir, but the classification holds if one ever did).
  • ModelSeamGraphObservationKindSummary + ModelSeamGraphV1.observation_summary — per-kind total/test_harness/production_application counts, computed (not constructor input), sorted by kind, omitting zero-observation kinds. Replaces the ad hoc manual grep/count a prior verification pass produced by hand with a durable, code-computed value any gate/golden consumer can read directly.

No change to extraction logic itself — purely additive classification on top of the existing code_observations.

RED-before / GREEN-after

9 new tests in tests/test_seams_extraction.py (TestTestHarnessSourceClassification, TestObservationSummarySplitsHarnessFromProduction), all failing against the unmodified model (AttributeError: 'ModelSeamGraphCodeObservation' object has no attribute 'is_test_harness' / same for observation_summary on ModelSeamGraphV1) — confirmed RED by running against origin/dev's unmodified model before writing the implementation. All 9 pass after.

Verification

  • Targeted: tests/test_seams_extraction.py + tests/test_seams_registry_contract.py + tests/test_seams_registry_match_integration.py + node_seam_graph_compute/node_seam_match_compute node tests — 109 passed, 0 failed.
  • mypy --strict src/omnimarket/seams/ — clean.
  • ruff format / ruff check --fix — clean.
  • Full local suite (pytest tests/ -q -m "not kafka") — 16,377 passed, 0 failed, 76 skipped, 9 deselected, 1 xfailed.
  • Governed pre-push selector (OMN-13973) escalated to full suite (changed_test_unnarrowable — shared model diff) — 15,232 passed, 0 failed, 47 skipped, 3 deselected, 1 xfailed. Gate passed, push allowed.
  • pre-commit run --all-files-equivalent (--files on the two touched files) — all hooks passed.

Live corpus reproduction (real 3-repo scan, omnimarket this branch, omnibase_infra 487cb8c2, omnibase_core c031bec0 — the same sibling SHAs the R1 audit finding cited):

kind total self-hit test-harness-in-src production-application
producer_send 38 0 16 22
consumer_subscribe 30 0 8 22
ref_pin 1 1 0 0

24 harness hits total (18 event_bus/testing/contract_event_bus_substrate.py + 6 runtime/transport/runtime_transport_conformance.py) — exact match to the audit finding's numbers. Determinism re-verified: two in-process runs over the identical pinned tree produce byte-identical model_dump_json() output.

One disclosed discrepancy: at the identical sibling SHAs the prior derivation cited, this pass's consumer_subscribe total is 30, not the previously-recorded 29 (producer_send=38 and ref_pin=1 both match exactly). Same extraction code, same trees — flagged rather than silently reconciled; full per-observation breakdown available in the PR discussion on request.

Evidence-Ticket: OMN-15779
Evidence-Source: OCC#6338

…on vs test-harness-in-src (R1 audit)

Corpus-wide producer_send/consumer_subscribe counts (OMN-15779's own
symbol-resolution landing) mix real inter-service application seams with
test substrate shipped inside a scanned src/ tree rather than under
tests/ (omnibase_core's event_bus/testing/contract_event_bus_substrate.py,
runtime/transport/runtime_transport_conformance.py). RSD lane-2 audit
finding R1 (2026-08-10) flagged that a second split is required before
any gate or golden-generation work consumes these numbers.

Adds ModelSeamGraphCodeObservation.is_test_harness (computed field,
derived from source_path -- always correct, never a constructor input
that could drift) and ModelSeamGraphV1.observation_summary (per-kind
total/test_harness/production_application counts), replacing the ad hoc
manual grep a prior verification pass produced by hand with a durable,
code-computed value.

RED-before: 9 new tests, all failing against the unmodified model
(AttributeError -- field/computed property did not exist). GREEN after.
109 targeted seam tests pass, mypy --strict clean, full local suite
(16,377 passed, 0 failed, 76 skipped, 9 deselected, 1 xfailed) clean.
Reproduced live against the real 3-repo corpus (omnimarket, omnibase_infra
487cb8c2, omnibase_core c031bec0): 24 harness hits (18 event_bus/testing,
6 runtime/transport conformance), exact match to the audit finding --
production_application is 22/22 for producer_send/consumer_subscribe.

Evidence-Ticket: OMN-15779
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 13 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: add86560-ccbb-4546-90fc-6fd251e6736e

📥 Commits

Reviewing files that changed from the base of the PR and between b8d79f1 and 9fc5445.

📒 Files selected for processing (2)
  • src/omnimarket/seams/models/model_seam_graph.py
  • tests/test_seams_extraction.py

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

✅ Architectural Review — PASSED

Errors: 0
Warnings: 0

What this checks

Rule Description
ARCH-TOPIC-001 No hardcoded Kafka topic strings in handler code
ARCH-DI-001 No event_bus=None bypass
ARCH-DI-002 No direct Handler instantiation outside workflow_runner/adapters
ARCH-DI-003 No reinvented DI containers
ARCH-TOPIC-002 contract.yaml topics follow `onex.{cmd

No architectural violations found.

Static architectural lint — no model inference (OMN-14176).

@github-actions

Copy link
Copy Markdown

✅ Hostile Reviewer — PASSED

Blocking findings (critical/error): 0
Total findings: 0
Models succeeded: qwen3-review,qwen3-review-b


Gate semantics

Verdict Meaning Blocks merge?
passed >=2 models succeeded, no critical/error findings No
blocked CRITICAL or ERROR findings found Yes
degraded Fewer than 2 models succeeded (infra unavailable/timeout) Yes (OMN-15110)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)

jonahgabriel added a commit to OmniNode-ai/onex_change_control that referenced this pull request Aug 10, 2026
…imarket#2043 (#6338)

* evidence(OMN-15779): author OCC companion for OmniNode-ai/omnimarket#2043

OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head 9fc54451b6b2d64d499f445f869f0416aa43807a.

* evidence(OMN-15779): self-bind OCC#6338 + rebind contract_sha256

* fix(OMN-15779): restore mutated Append-Only-Gate receipt, add round-qualified evidence for OCC#6338

The OCC Evidence-Source autobind bot reused the static path
drift/dod_receipts/OMN-15779/dod-occ-evidence-admissibility-validator/command.yaml
(already merged twice, by OCC#6329) instead of a round-qualified name,
tripping the OCC Append-Only Gate (a merged receipt file must never be
edited in place). Same bot bug already diagnosed and worked around on the
OMN-15763 stream (PR #2039 / OCC#6327).

Fix: restored the shared path byte-identical to origin/dev, moved the
fresh (real, non-fabricated) evidence for omnimarket PR #2043 to
dod-occ-evidence-admissibility-validator-pr-2043/command.yaml.

Evidence-Ticket: OMN-15779

---------

Co-authored-by: omnimarket-bot <bot@omninode.ai>
Co-authored-by: Jonah Gray <jonah@omninode.ai>
@github-actions
github-actions Bot merged commit e3caf0f into dev Aug 10, 2026
154 of 164 checks passed
@jonahgabriel
jonahgabriel deleted the jonahgabriel/omn-15779-r1-production-vs-harness-split branch August 10, 2026 23:31
github-actions Bot pushed a commit that referenced this pull request Aug 11, 2026
…schema_version bump for computed-field wire shape (#2049)

* fix(OMN-15854): seam-graph third split (validator-runtime/tooling) + schema_version bump for computed-field wire shape

Splits the seam-graph extractor's production-application bucket into
{validator-runtime-tooling, inter-service-application} so a downstream
gate/golden generator no longer reads validator conformance scaffolding
(8 omnibase_core/validation/**/runtime_*.py files, 16 of 22
producer_send + 16 of 22 consumer_subscribe production-application hits)
as genuine inter-service Kafka application traffic.

- ModelSeamGraphCodeObservation gains is_validator_runtime_tooling
  (computed_field, glob: validation/ dir segment + runtime_*.py filename,
  both required -- non-weakening against runtime_local.py and
  runtime_log_event_bridge.py, which start with runtime_ but sit outside
  validation/).
- ModelSeamGraphObservationKindSummary gains validator_runtime_tooling
  and inter_service_application int fields (additive; production_application
  kept for continuity, now == the sum of the two new fields).
- schema_version bumped seam-graph/v1 -> seam-graph/v2: PR #2043 already
  added computed fields (is_test_harness, observation_summary) that
  serialize into model_dump()/model_dump_json() without a version bump;
  this PR resolves that drift in the same change that adds its own new
  computed fields, rather than compounding a second silent drift. Zero
  external deserializers today (cross-repo grep, see PR body).
- Fail-closed default: an unmatched source_path defaults to
  inter_service_application (un-suppressed) -- tooling-bucket exclusion
  requires a positive glob match, mirroring the existing
  is_test_harness convention.

Tests: pin the 8 tooling files' classification, pin all 12 named genuine
inter-service sites as NOT reclassified, pin the schema_version literal,
pin the serialized field sets (top-level, observation, summary row), and
reproduce the ticket's exact 22 -> 16+6 split per kind end-to-end.

OMN-15854

* chore(OMN-15854): retrigger CI after OCC companion #6355 merged

OCC companion onex_change_control#6355 merged (65fe0c63) after the
mischaracterized "stale-poller" blocker was corrected (verifier
refutation) -- root cause was pre-commit fail-closed on missing
ci:ready label (OMN-15731), not a reject-skip-token race. Label
applied, pre-commit ran clean, companion merged. This empty commit
mints a fresh CI run so OCC Companion Merged Gate and CI Summary
re-evaluate against the now-merged companion. Per RETRIGGER LAW this
is a push-minted fresh run (build cost), not a rerun.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant