Repository navigation
feat(OMN-10848): proof of life — canary → scores → routing feedback loop - #606
Merged
Conversation
…a score lookup Adds build_available_models_from_scores() — a caller-side bridge that converts capability_scores table rows into ModelAvailableModel instances accepted by the routing policy engine. HandlerRoutingPolicy is not modified.
End-to-end integration test proving the full chain works in-memory: ModelCanaryReport → HandlerCanaryScoreReducer → ModelMaterializeResult → build_available_models_from_scores → HandlerRoutingPolicy selects champion. Also re-exports EnumTaskType, ModelRoutingPolicyRequest, ModelAvailableModel, and build_available_models_from_scores from node_routing_policy_engine.__init__ to avoid cross-node .models.* reach-in violations (OMN-9263).
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
github-actions
Bot
merged commit May 11, 2026
1af2f16
into
jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-grading
5 checks passed
This was referenced May 11, 2026
jonahgabriel
deleted the
jonah/omn-10848-proof-of-life-canary-scores-routing-feedback-loop
branch
June 11, 2026 17:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test_integration_feedback_loop.py: end-to-end integration test proving the full feedback chain works in-memory —ModelCanaryReport→HandlerCanaryScoreReducer.accumulate()→materialize()→build_available_models_from_scores()→HandlerRoutingPolicy.handle()selectsqwen3-coder-30b(0.94 recall) overdeepseek-r1-14b(0.70 recall)EnumTaskType,ModelRoutingPolicyRequest,ModelAvailableModel, andbuild_available_models_from_scoresfromnode_routing_policy_engine.__init__to avoid cross-node.models.*reach-in violations (OMN-9263)handler_score_lookup.pycommits (PRs feat(OMN-10846): bridge capability_scores to routing policy engine via score lookup #604 still open) as this task depends on the score lookup bridgeTicket
OMN-10848 — Task 5 of
2026-05-11-canary-routing-feedback-loop.mddod_evidence
test_canary_scores_feed_routing_decisionspasses:1 passed in 0.12stest_no_new_cross_node_model_reach_ins) passes — no new violationsOMNIBASE_INFRA_PATHset)Base branch note
This branch stacks on
jonah/omn-10847-populate-routing_outcomesquality_score-from-canary-grading(PR #605) and cherry-picks thehandler_score_lookup.pyaddition from OMN-10846 (PR #604). Once both #604 and #605 merge, this PR should be rebased onto main before merging.Test plan
test_canary_scores_feed_routing_decisionspasses in CItest_no_new_cross_node_model_reach_inspasses (no new reach-ins)