Skip to content

feat(OMN-10848): proof of life — canary → scores → routing feedback loop - #606

Merged
github-actions[bot] merged 3 commits into
jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-gradingfrom
jonah/omn-10848-proof-of-life-canary-scores-routing-feedback-loop
May 11, 2026
Merged

github-actions[bot] merged 3 commits into
jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-gradingfrom
jonah/omn-10848-proof-of-life-canary-scores-routing-feedback-loop

Conversation

@jonahgabriel

Copy link
Copy Markdown
Contributor

Summary

  • Adds test_integration_feedback_loop.py: end-to-end integration test proving the full feedback chain works in-memory — ModelCanaryReport → HandlerCanaryScoreReducer.accumulate() → materialize() → build_available_models_from_scores() → HandlerRoutingPolicy.handle() selects qwen3-coder-30b (0.94 recall) over deepseek-r1-14b (0.70 recall)
  • Re-exports EnumTaskType, ModelRoutingPolicyRequest, ModelAvailableModel, and build_available_models_from_scores from node_routing_policy_engine.__init__ to avoid cross-node .models.* reach-in violations (OMN-9263)
  • Cherry-picked OMN-10846 handler_score_lookup.py commits (PRs feat(OMN-10846): bridge capability_scores to routing policy engine via score lookup #604 still open) as this task depends on the score lookup bridge

Ticket

OMN-10848 — Task 5 of 2026-05-11-canary-routing-feedback-loop.md

dod_evidence

  • test_canary_scores_feed_routing_decisions passes: 1 passed in 0.12s
  • Full suite: 3576 passed (5 pre-existing Postgres/Kafka infra failures unchanged)
  • Cross-node reach-in test (test_no_new_cross_node_model_reach_ins) passes — no new violations
  • All pre-commit hooks pass (with OMNIBASE_INFRA_PATH set)

Base branch note

This branch stacks on jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-grading (PR #605) and cherry-picks the handler_score_lookup.py addition from OMN-10846 (PR #604). Once both #604 and #605 merge, this PR should be rebased onto main before merging.

Test plan

  • Verify test_canary_scores_feed_routing_decisions passes in CI
  • Verify test_no_new_cross_node_model_reach_ins passes (no new reach-ins)
  • Confirm 5 pre-existing Postgres/Kafka integration failures are unchanged

…a score lookup

Adds build_available_models_from_scores() — a caller-side bridge that converts
capability_scores table rows into ModelAvailableModel instances accepted by the
routing policy engine. HandlerRoutingPolicy is not modified.
End-to-end integration test proving the full chain works in-memory:
ModelCanaryReport → HandlerCanaryScoreReducer → ModelMaterializeResult →
build_available_models_from_scores → HandlerRoutingPolicy selects champion.

Also re-exports EnumTaskType, ModelRoutingPolicyRequest, ModelAvailableModel,
and build_available_models_from_scores from node_routing_policy_engine.__init__
to avoid cross-node .models.* reach-in violations (OMN-9263).
@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 73da91f1-487f-4433-ba87-4112c64d196d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-10848-proof-of-life-canary-scores-routing-feedback-loop

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
github-actions Bot merged commit 1af2f16 into jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-grading May 11, 2026
5 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-10848-proof-of-life-canary-scores-routing-feedback-loop branch June 11, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant