feat: normalize and edit model groups - #838
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| for agent_id, raw_payload in conn.execute("SELECT agent_id, payload FROM agent_pool"): | ||
| payload = json.loads(raw_payload) | ||
| group_name = payload.pop("group_name", "") | ||
| if not group_name: | ||
| continue | ||
| group_name = canonical_group_name(group_name) | ||
| conn.execute("INSERT OR IGNORE INTO model_group (group_name) VALUES (?)", (group_name,)) | ||
| conn.execute( | ||
| "INSERT OR REPLACE INTO model_group_member (agent_id, group_name) VALUES (?, ?)", | ||
| (agent_id, group_name), | ||
| ) | ||
| conn.execute( | ||
| "UPDATE agent_pool SET payload = ? WHERE agent_id = ?", | ||
| (json.dumps(payload, ensure_ascii=False), agent_id), | ||
| ) |
There was a problem hiding this comment.
📝 Info: Migration mutates agent_pool while scanning it
_migrate_legacy_groups iterates a SELECT ... FROM agent_pool cursor and runs UPDATE agent_pool inside the loop on the same connection (orchestrator.py). The scan is by rowid and only the non-key payload column is updated in place, so no rows move, insert, or delete, and a re-visit would be idempotent since group_name is already popped. Safe in practice; .fetchall() up front would remove the hazard. The single-row migration test does not exercise multi-row scanning.
Was this helpful? React with 👍 or 👎 to provide feedback.
| async function load() { | ||
| const res = await fetch("/admin/state"); | ||
| state = await res.json(); | ||
| await refreshModelGroups(); |
There was a problem hiding this comment.
📝 Info: load() aborts entirely if model_groups fetch fails
refreshModelGroups() is awaited early in load() with no try/catch (admin.py). If /api/v1/model_groups errors, the whole load path — agents, analytics, readiness, trace — is skipped and the console stays blank. This follows the existing awaited-fetch pattern, but adds another hard dependency up front. A later language switch also calls renderModelGroups() guarded only by state.agents.length (admin.py); if that refresh failed, state.modelGroups is undefined and .map throws.
Was this helpful? React with 👍 or 👎 to provide feedback.
| event.preventDefault(); | ||
| const groupName = els.modelGroupName.value.trim(); | ||
| const memberIds = Array.from(els.modelGroupMembers.selectedOptions).map(option => option.value); | ||
| const exists = state.modelGroups.some(group => group.group_name === groupName.replaceAll("-", "_").toLowerCase()); |
There was a problem hiding this comment.
📝 Info: Client group-name canonicalization matches server
saveModelGroup picks POST vs PATCH by comparing groupName.replaceAll("-","_").toLowerCase() to stored canonical names (admin.py). Because the input pattern forbids whitespace and consecutive separators, this matches the server's canonical_group_name (contextual_orchestrator/model_group.py:53-66) for every pattern-valid input, so the exists check stays reliable.
Was this helpful? React with 👍 or 👎 to provide feedback.
| "capability_coverage": { | ||
| capability: sum(capability in agent.tags for agent in members) | ||
| for capability in sorted(MODEL_CAPABILITIES) | ||
| if any(capability in agent.tags for agent in members) | ||
| }, |
There was a problem hiding this comment.
📝 Info: capability_coverage relies on discovery tags
capability_coverage counts members whose tags contain each MODEL_CAPABILITIES name (orchestrator.py). It is non-empty only for discovered agents, since agent_from_discovered injects bare capability strings into tags (contextual_orchestrator/model_discovery.py:297-302). Hand-configured agents (coding/reasoning) yield {}, matching the contract test.
Was this helpful? React with 👍 or 👎 to provide feedback.
* feat(routing): add operator-managed model groups * fix(api): enforce model group create semantics * fix(routing): preserve eligibility and REST contracts * docs(groups): remove ephemeral model example * fix(groups): route advertised aliases end to end * ci: hourly OpenCode maintenance agent routed through this gateway (#835) * ci: add hourly OpenCode maintenance agent routed through this gateway The scheduled job boots the contextual-orchestrator gateway with the five org provider secrets seeded into the process-local KV registry (bootstrap transport only), auto-discovers chat-capable models, assigns the ox-alpha measured-routing group across OpenRouter + OpenCode Zen aliases, then runs the pinned OpenCode CLI pointed at http://127.0.0.1:8000/v1 with model 'ox-alpha' so the agent's own traffic exercises group routing. The agent works the PR queue (review -> fix -> recheck -> merge), root-causes failing checks, and advances docs/product-technical-gap-baseline.md when the PR queue is empty. COPILOT_GITHUB_TOKEN is not used; the existing review-agent key scheme is untouched. * fix(ci): remove ephemeral model binding * fix(ci): install gateway and grant branch writes * fix(groups): preserve internal dispatch default * fix: isolate model group routing evidence * fix: preserve conduct semantics for model groups * ci: minimize hourly loop permissions and installs * fix: measure streamed group routing * feat: route model groups across modalities (#837) * feat: normalize and edit model groups (#838) * fix: preserve capability routing contracts * release: v0.2.0 — model groups, cost-aware discovery, changelog baseline Bump 0.1.0 -> 0.2.0 and add the canonical Keep-a-Changelog file with the 0.1.0 baseline and the 0.2.0 additions (operator-managed model groups, measured group routing, OpenCode Zen discovery + free-tier classification, Strix B105 root-cause remediation). * fix(discovery): tolerate null modality arrays * docs: specify model group product and technical contracts * release: align v0.2.0 changelog and lock metadata * feat: stream orchestrated reasoning summaries * fix: keep free reasoning streams fail closed * fix: preserve free routing evidence and analytics * fix: distinguish failed Responses streams in analytics * fix: pin structured free judge to selected agent * fix: lock container dependencies and virtual capabilities * fix: keep free passthrough on zero-cost models * fix: prune removed routing measurements * refactor: remove unreachable responses passthrough branch * fix: route virtual models across media capabilities * fix: retain reset candidate routing rows * fix: preserve Responses instructions in workflows * test: align model-group missing-member code with canonical agent_not_found #831 unified worker-agent not-found errors on agent_not_found; the model-group CRUD contract now asserts that same canonical code. * fix(api): model-group creation returns canonical agent_not_found for unknown members * ci: route hourly OpenCode loop through auto * fix: reject unsupported orchestrated structured output * docs: correct Responses stream options error * fix: retrieve URL-encoded model identifiers * fix: measure free capability and failover routing * fix: resolve model group review findings * fix: close model group integration gaps * fix: align provider inventory and session cache scope * fix: harden compose secrets and repeated reasoning summaries * fix: contain binary response disconnects * fix: ground Zen free discovery in structured costs * fix: keep group judge within allowed members * fix: preserve free catalog evidence across reloads * docs: assign unique model-group ADR number * fix: preserve catalog capability evidence * fix: preserve declared Bytez endpoint capabilities * fix: validate batch model identity at ingress * fix(api): normalize missing model group errors * docs(prd): align product bets with model groups * fix(discovery): filter chat-only Bytez transports * docs(adr): reserve model-group decision identifier * feat: replace routing heuristics with measured evidence ledgers (#847) * feat: replace routing heuristics with measured evidence ledgers Remove DOMAIN_HINTS/COMPLEX_HINTS keyword tables; route via eligibility contracts, declaration priority/capability fit/cosine affinity over operator-declared metadata, and measured intra-group quality then EWMA tokens-per-second. Add structured fail-closed triage gas with content-hash verdict caching and real-time fast-mlsirm judging on direct routes that feeds a Beta-Bernoulli quality ledger with in-budget failover. ADR 0027 + doctoring APA 7 references (Jacobson 1988; Gelman et al. 2013; Karpukhin et al. 2020; Ong et al. 2024; Chen et al. 2023; Zheng et al. 2023; Jeon et al. 2021). Gap baseline added at docs/product-technical-gap-baseline.md. * fix: keep routing evidence units and capability boundaries honest * fix(admin): tolerate unavailable model-group state * docs(loop): require PRD and measured web capacity * fix(admin): remove retired policy hint metric * fix(routing): validate evidence before mutation * fix(discovery): remove model-name free inference guidance * fix: complete #834 model-group persistence on the normalized agent-pool schema - model_group/model_group_member relations compose with main's normalized agent_pool (no JSON shadow); save() maintains membership, load_all() restores group_name via join. - Legacy payload promotion reads agent_pool_legacy_payloads during the migration window and drops it after promoting group names. - DB-naming gate now extracts SQL from AST string constants so prose in comments can no longer produce false identifier violations. - Batch runner signature aligned (messages, mode, model) with the merged LocalBatchBackend; stream-route fake accepts the merged caller kwargs. * test: case-sensitive DDL pattern so prose cannot fake identifiers * docs: refresh protected-main gap evidence * docs: record exact provider regression suite * docs: correct exact provider head * fix(admin): make guidance customer-actionable * fix(admin): remove internal terms from customer copy * docs(ui): record customer-copy screenshot audit * test(ci): lock hourly orchestrator loop contract * fix: replace synthetic admin status with truthful empty states * docs: refresh exact-head remediation baseline * fix(stream): cancel orchestration after Responses disconnect * test: align boundary contracts with current routing * docs: complete public boundary docstrings * docs: refresh exact-head product gap queue * test: enforce complete public docstrings * ci: enforce protected exact-head merge loop
1 participant
Summary
model_groupandmodel_group_memberrelationsVerification
pytest -q tests/test_agent_pool_db.py tests/test_admin_contract.py tests/test_analytics_runtime.py tests/test_model_group.py(28 passed)Design boundary