Skip to content

docs(OMN-11513): 2026-06-04 sanctioned-live --agent evidence — first full PASS + OMN-12664 post-closure 404 observation - #2166

Merged
jonahgabriel merged 3 commits into
devfrom
clone45/omn-11513-evidence-2026-06-04-agent-runs
Jun 4, 2026
Merged

jonahgabriel merged 3 commits into
devfrom
clone45/omn-11513-evidence-2026-06-04-agent-runs

Conversation

@clone45

@clone45 clone45 commented Jun 4, 2026 •

Copy link
Copy Markdown
Collaborator

What this adds

Contractor evidence port (same shape as merged #1886), under evidence/OMN-11513/:

  • proof-of-life/2026-06-04-93adf56-live-agent-pass/ — first full sanctioned-live --agent PASS from the contractor environment (generate → validate → register → invoke, SentimentClassifier), stability lane, transport=bus_backed (SEA chore(contracts): backfill ModelTicketContract for 50 done tickets [OMN-8637 wave 44] #206 readiness artifact). Generation served at ladder tier 1 (local Qwen, per SEA chore(contracts): backfill ModelTicketContract for 50 done tickets [OMN-8637 wave 43] #205); cloud tier not attempted. Classified diagnostic in the packet (missing: runtime identity, topic offsets, projection state).
  • proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/ — forced-escalation run (local tiers made unreachable via the documented SEA_LOCAL_INFERENCE_BASE_URL seam) to exercise the cloud tier after OMN-12664 moved Done. Observation on SEA dev 93adf56, 2026-06-04 16:09Z: escalated to cloud_gemini, runtime round-trip live, failed with the same two-URL 404 signature as the 2026-06-03 filing — registered …/v1beta/openai/chat/completions, called …/v1/chat/completions. Typed SeaGenerationError; full log in run.log. The packet's Interpretation boundary section is explicit that this seat cannot distinguish fix-not-deployed from fix-not-on-live-path.
  • integration-test-passes/2026-05-30-bret-obj1-token-registry-stability.md — Objective 1 evidence authored 2026-05-30, ported now (was sitting uncommitted in the contractor clone).
  • repros/OMN-12587-asyncio/ — the F1 asyncio repro script referenced from the OMN-12587 thread (fix verified in SEA chore(contracts): backfill ModelTicketContract for 50 done tickets [OMN-8637 wave 41] #203; retained as regression reference).
  • EVIDENCE_MANIFEST.md — rows for the above + rows for the already-merged 2026-05-26/2026-05-29 files that were missing from the manifest + a 2026-06-04 addendum section.

Verification thread

OMN-12587 (a comment citing these paths follows once this PR has a number). Umbrella: OMN-11513.

Process disclosure

The local omniclaude Plan Gate hook flagged this branch (ticket-pattern branch, no docs/plans/ file). This is a docs-only evidence port with no implementation changes, matching the merged precedent #1886, so the manifest edit was applied by script rather than adding a plan file for a docs port. Flagging here for transparency — happy to add a plan file if that is preferred for evidence ports going forward.

No secrets: run logs verified free of key material before commit.

Summary by CodeRabbit

  • Documentation
    • Extended evidence manifest time window through 2026-06-04 and added an addendum capturing status notes and an evidence gap
    • Added integration-test notes and multiple proof-of-life packets documenting sanctioned and forced-escalation runs, event chains, and readiness evidence (including observed two-URL 404 signature)
  • Tests
    • Added a reproducible test that exercises asyncio event-loop handling to validate a prior delegation failure mode

@coderabbitai

coderabbitai Bot commented Jun 4, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 55b9babc-3bee-4480-b041-899b4d92889d

📥 Commits

Reviewing files that changed from the base of the PR and between f32f913 and adce219.

📒 Files selected for processing (13)
  • contracts/OMN-11513.yaml
  • contracts/OMN-12664.yaml
  • drift/dod_receipts/OMN-11513/dod-deploy-assessment-pr-1739/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-cross-repo-test/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1323/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1607/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1608/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1886/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-2166/command.yaml
  • drift/dod_receipts/OMN-11513/dod-omni-home-pr-145/command.yaml
  • drift/dod_receipts/OMN-11513/dod-omnibase-infra-pr-1735/command.yaml
  • drift/dod_receipts/OMN-11513/dod-omnibase-infra-pr-1739/command.yaml
  • drift/dod_receipts/OMN-11513/dod-stability-runtime-proof/command.yaml
✅ Files skipped from review due to trivial changes (8)
  • drift/dod_receipts/OMN-11513/dod-stability-runtime-proof/command.yaml
  • drift/dod_receipts/OMN-11513/dod-omnibase-infra-pr-1739/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1607/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1608/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-cross-repo-test/command.yaml
  • drift/dod_receipts/OMN-11513/dod-omnibase-infra-pr-1735/command.yaml
  • drift/dod_receipts/OMN-11513/dod-occ-pr-1886/command.yaml
  • contracts/OMN-12664.yaml

📝 Walkthrough

Walkthrough

This pull request extends the OMN-11513 evidence bundle with updated manifest references, a new integration test stability document, two 2026-06-04 live-agent runs (one pass, one forced-escalation), a deterministic asyncio repro test for OMN-12587, and multiple contract and drift receipt updates. All changes are documentation, evidence artifacts, or tests.

Changes

OMN-11513 Evidence Collection

Layer / File(s) Summary
Evidence manifest index update
evidence/OMN-11513/EVIDENCE_MANIFEST.md
Time window extended through 2026-06-04; references added for new integration-test, proof-of-life, repro, and addendum entries.
Integration test evidence: SEA Token and Registry Stability
evidence/OMN-11513/integration-test-passes/2026-05-30-bret-obj1-token-registry-stability.md
Objective 1 evidence with environment binding metadata, registry/served model ID verification, 7-task delegation regression analysis (two runs + aggregate), escalation behavior, OMN-12438 max_tokens observation, cross-references to OMN-12434/OMN-12438/OMN-12502, and exit checklist.
Proof-of-life: Sanctioned live-agent pass run (2026-06-04)
evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/event-chain.json, kafka-readiness-evidence.json, overlay-evidence.json, proof-packet.md
Post-closure --agent run with event chain, Kafka readiness probe, overlay metadata, and a proof packet documenting PASS, reproducibility steps, packet field coverage, artifacts, and interpretation addendum.
Proof-of-life: Forced-escalation diagnostic run (2026-06-04)
evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/event-chain.json, kafka-readiness-evidence.json, overlay-evidence.json, proof-packet.md
Forced-escalation --agent run documenting method for forcing escalation, observed SeaGenerationError, recurring two-URL 404 signature mismatch, packet capture status, and noted evidence gaps when delegation events are dropped.
Asyncio reproduction test for OMN-12587
evidence/OMN-11513/repros/OMN-12587-asyncio/test_publish_and_wait_in_running_loop.py
Deterministic pytest exercising StabilityDelegationBus.publish_and_wait inside a running asyncio event loop using an in-memory _FakeTransport, validating successful completion and correlated response.
Contracts: added/updated dod_evidence entries
contracts/OMN-11513.yaml, contracts/OMN-12664.yaml
Added dod_evidence entries referencing PR #2166 and an OCC observation for OMN-12664 with command-based checks.
Drift receipts: contract_sha256 updates and new receipt
drift/dod_receipts/OMN-11513/...
Updated contract_sha256 across multiple receipt command.yaml files and added a new receipt for dod-occ-pr-2166 capturing probe stdout and verification metadata.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰 The rabbit hops through manifests and logs so spry,

Finds registry breadcrumbs and proof-of-life nigh,
A forced run, a passing run, and a loop made tame,
Tests and receipts all tagged by name,
Hooray — the evidence garden grows another bright sprig!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly and specifically describes the main additions: sanctioned-live agent evidence with a PASS result and a post-closure OMN-12664 404 observation, matching the substantial new evidence files and manifest updates in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch clone45/omn-11513-evidence-2026-06-04-agent-runs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

without a live broker. This is the same injection seam the shipped tests use.
"""

async def start(self) -> None: ...
"""

async def start(self) -> None: ...
async def close(self) -> None: ...
async def close(self) -> None: ...
async def arm_response(
self, *, response_topic: str, correlation_id: str
) -> None: ...
) -> None: ...
async def publish(
self, topic: str, key: bytes | None, value: bytes, headers: object = None
) -> None: ...
@clone45
clone45 force-pushed the clone45/omn-11513-evidence-2026-06-04-agent-runs branch from f32f913 to bba4f12 Compare June 4, 2026 16:45

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/proof-packet.md (1)

36-42: ⚡ Quick win

Add language specifier to fenced code block.

The error message code block should specify a language identifier for proper syntax highlighting and documentation tooling compatibility.

📝 Suggested improvement
-```
+```text
 model registry drift: inference effect reported model-not-found for
 served_model_id 'gemini-2.5-flash'
 at https://generativelanguage.googleapis.com/v1beta/openai/chat/completions;
 error_message="Client error '404 Not Found' for url
 'https://generativelanguage.googleapis.com/v1/chat/completions'"
</details>

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
@evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/proof-packet.md
around lines 36 - 42, The fenced code block in proof-packet.md lacks a language
specifier; update the triple-backtick opening for the error message block to
include a language identifier (e.g., text) so syntax highlighters and tooling recognize it—locate the fenced block that begins with and change it to

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/proof-packet.md`:
- Around line 36-42: The fenced code block in proof-packet.md lacks a language
specifier; update the triple-backtick opening for the error message block to
include a language identifier (e.g., ```text) so syntax highlighters and tooling
recognize it—locate the fenced block that begins with ``` and change it to
```text while keeping the existing block content unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1a1caa3b-8988-4d61-bee8-da7dc1c13d71

📥 Commits

Reviewing files that changed from the base of the PR and between 4675c31 and f32f913.

⛔ Files ignored due to path filters (2)
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/run.log is excluded by !**/*.log
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/run.log is excluded by !**/*.log
📒 Files selected for processing (11)
  • evidence/OMN-11513/EVIDENCE_MANIFEST.md
  • evidence/OMN-11513/integration-test-passes/2026-05-30-bret-obj1-token-registry-stability.md
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/event-chain.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/kafka-readiness-evidence.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/overlay-evidence.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-forced-escalation/proof-packet.md
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/event-chain.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/kafka-readiness-evidence.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/overlay-evidence.json
  • evidence/OMN-11513/proof-of-life/2026-06-04-93adf56-live-agent-pass/proof-packet.md
  • evidence/OMN-11513/repros/OMN-12587-asyncio/test_publish_and_wait_in_running_loop.py

@jonahgabriel
jonahgabriel force-pushed the clone45/omn-11513-evidence-2026-06-04-agent-runs branch from bba4f12 to adce219 Compare June 4, 2026 17:37
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jun 4, 2026
Merged via the queue into dev with commit a06dfdb Jun 4, 2026
36 checks passed
@jonahgabriel
jonahgabriel deleted the clone45/omn-11513-evidence-2026-06-04-agent-runs branch June 4, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants