Skip to content

feat(OMN-13366): add deterministic acceptance evidence - #1299

Merged
jonahgabriel merged 1 commit into
devfrom
jonahgabriel/omn-13366-deterministic-acceptance
Jun 20, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
jonahgabriel/omn-13366-deterministic-acceptance

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jun 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Implements OMN-13366 P1 deterministic acceptance for verifiable delegation classes.

  • Declares score_source: deterministic_acceptance and deterministic acceptance_authority for code_generation, test, and validator_generation in task_class_contracts.v1.yaml.
  • Extends quality-gate result evidence with score_source, acceptance_version, corpus_hash, validator_or_artifact_hash, acceptance_command, actual_score, pass, and failure_cases.
  • Makes verifiable task classes with deterministic DoD use deterministic acceptance as the pass/fail authority; non-verifiable task classes keep existing heuristic authority.
  • Extends node_generation_consumer corpus acceptance results with the same deterministic evidence bundle for validator-generation corpus gates.
  • Adds focused OMN-13366 tests and updates corpus/DTO identity coverage.

Evidence-Ticket: OMN-13366
Evidence-Source: OCC#2831

Verification

Latest rebased head: 3b7316d53b5d260464802c0655c934be11892bd3 on origin/dev 2ce4b9672e30fa0647b87de5a042b31f2bf18a7c.

  • uv run pytest tests/unit/delegation/test_quality_gate_deterministic_acceptance_omn13366.py tests/unit/nodes/node_generation_consumer/test_corpus_acceptance.py -q -> 17 passed.
  • uv run pytest tests/unit/delegation tests/unit/nodes/node_generation_consumer -q -> 497 passed.
  • uv run ruff check src/ tests/ && git diff --check -> passed.
  • LC_ALL=C LANG=C uv run pytest -m "not integration and not kafka" -q -> 7554 passed, 35 skipped, 97 deselected.
  • OMNIBASE_INFRA_PATH="$OMNI_HOME/omnibase_infra" LC_ALL=C LANG=C uv run pre-commit run --all-files -> passed after rerun; first run only normalized generated dep-health evidence, which was restored.

Earlier full uv run pytest -q on the pre-rebase head also ran: 7576 passed, 75 skipped, 4 failed. Failures were external/local-environment, not OMN-13366 paths:

  • 3 Kafka tests could not use local Kafka/topic metadata on localhost:19092.
  • 1 node_pr_polish unit assertion captured the local LC_ALL=C.UTF-8 warning from the fake uv subprocess; the broad unit/non-Kafka run passed with LC_ALL=C LANG=C.

Rollback

No rollback applied. Change is isolated to deterministic acceptance evidence/result semantics and task-class declarations.

@coderabbitai

coderabbitai Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds deterministic acceptance evidence to quality gate results: task class contracts gain score_source, acceptance_authority, and evidence_fields metadata; ModelQualityGateResult and ModelCorpusAcceptanceResult gain hash/command/score fields; handler_quality_gate.py computes and spreads evidence into all result paths; a compat shim redirects imports; and a new test suite validates the full deterministic/heuristic authority behavior.

Changes

Deterministic Acceptance Evidence

Layer / File(s) Summary
Contract declarations and DTO evidence fields
src/omnimarket/configs/task_class_contracts.v1.yaml, src/omnimarket/models/delegation/wire/model_quality_gate.py
YAML contracts add score_source, acceptance_authority, and evidence_fields to code_generation, test, and a new validator_generation task class. ModelQualityGateResult gains score_source, acceptance_version, hash/command fields, actual_score, pass_ (aliased to "pass"), and failure_cases, plus alias-aware model config.
Corpus acceptance evidence generation
src/omnimarket/nodes/node_generation_consumer/corpus_acceptance.py
ModelCorpusAcceptanceResult gains the same evidence fields and alias config. New helpers compute stable SHA-256 corpus/handler hashes and the acceptance command string. A _result_with_evidence(...) helper fills all evidence fields and computes actual_score; both result-construction call sites delegate to it.
Quality gate handler: deterministic authority wiring
src/omnimarket/nodes/node_delegation_quality_gate_reducer/handlers/handler_quality_gate.py
Adds module-level constants for acceptance version/source/verifiable task types and helper functions to compute hashes, derive deterministic scores, and decide authority applicability. delta() now computes acceptance_evidence before result construction and spreads it into all three result paths (deterministic-pass, heuristic-failure, final-pass). Legacy fail_category switches from enum members to string literals.
Compat import redirect
src/omnimarket/nodes/node_delegation_quality_gate_reducer/models/model_quality_gate_result.py
Switches ModelQualityGateResult and EnumQualityGateCategory imports from omnibase_core.models.delegation.wire to omnimarket.models.delegation.wire.model_quality_gate, with an updated compatibility note.
Tests: compat assertions and deterministic acceptance behavior
tests/test_delegation_dto_omnibase_compat_imports.py, tests/unit/delegation/test_quality_gate_deterministic_acceptance_omn13366.py, tests/unit/nodes/node_generation_consumer/test_corpus_acceptance.py
Compat tests add OMNIMARKET_OWNED_DTO_IMPORTS for ModelQualityGateResult and assert module ownership. New deterministic acceptance tests cover verifiable pass, fractional actual_score on failure, deterministic-over-heuristic precedence, heuristic fallback for non-verifiable contracts, and YAML authority field assertions. Corpus acceptance tests expand to assert evidence fields, aliased pass, and failure_cases.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • OmniNode-ai/omnimarket#619: Modifies the same handler_quality_gate.py contract-driven deterministic/heuristic DoD handling and quality-gate evaluation logic that this PR extends with acceptance evidence generation.
  • OmniNode-ai/omnimarket#675: Both PRs modify the same deterministic vs. heuristic check evaluation structure in handler_quality_gate.py, including float-equality handling in the deterministic check path.
  • OmniNode-ai/omnimarket#1032: Directly overlaps with this PR's change of legacy fail_category assignment from EnumQualityGateCategory enum members to "pass"/"fail_heuristic" string literals in handler_quality_gate.py.

Poem

🐇 A bunny hops through hash and score,
Computing evidence, a deterministic floor.
SHA-256 for corpus, artifact in tow,
actual_score computed, pass_ set to glow!
No more heuristic guessing in the gate —
The rabbit stamps: deterministic. Accept the fate! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat(OMN-13366): add deterministic acceptance evidence' directly and clearly summarizes the main change: adding deterministic acceptance evidence, matching the PR objectives and the substantial modifications across config, models, handlers, and tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonahgabriel/omn-13366-deterministic-acceptance

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
github-actions Bot enabled auto-merge June 20, 2026 01:28
@github-actions

github-actions Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

✅ Architectural Review — PASSED

Errors: 0
Warnings: 0

What this checks

Rule Description
ARCH-TOPIC-001 No hardcoded Kafka topic strings in handler code
ARCH-DI-001 No event_bus=None bypass
ARCH-DI-002 No direct Handler instantiation outside workflow_runner/adapters
ARCH-DI-003 No reinvented DI containers
ARCH-TOPIC-002 contract.yaml topics follow `onex.{cmd

No architectural violations found.

Powered by node_pr_review_bot

@github-actions

github-actions Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

✅ PR Review Bot — CLEAN

Findings: 0
Run ID: ``

Note: Reviewer models are running as stubs (OMN-7969–OMN-7972 pending). Architectural findings from Phase 1 (arch-lint) are the authoritative signal until model handlers land.


Verdict semantics

Verdict Meaning
clean No blocking findings
risks_noted Findings noted; threads passed verification
blocking_issue MAJOR/CRITICAL findings not resolved — do not merge

Powered by node_pr_review_bot — FSM pipeline with multi-model review fan-out

@github-actions

github-actions Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical/error): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8524) and do not block merge. Error: cli_review exited non-zero (all models failed or binary error)


Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical/error findings No
blocked CRITICAL or ERROR findings found Yes
degraded All models unavailable (infra) No (pilot)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)

@jonahgabriel
jonahgabriel force-pushed the jonahgabriel/omn-13366-deterministic-acceptance branch 3 times, most recently from 8943119 to 92e3531 Compare June 20, 2026 02:19
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jun 20, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Jun 20, 2026
@jonahgabriel
jonahgabriel force-pushed the jonahgabriel/omn-13366-deterministic-acceptance branch from 92e3531 to 3b7316d Compare June 20, 2026 02:30
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jun 20, 2026
Merged via the queue into dev with commit 630099f Jun 20, 2026
75 checks passed
@jonahgabriel
jonahgabriel deleted the jonahgabriel/omn-13366-deterministic-acceptance branch June 20, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant