Repository navigation
feat(routing): add operator-managed model groups #834
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 81 commits
Commits
Show all changes
97 commits
Select commit
Hold shift + click to select a range
27afbce
feat(routing): add operator-managed model groups
seonghobae 8118d1a
fix(api): enforce model group create semantics
seonghobae b1eb222
fix(routing): preserve eligibility and REST contracts
seonghobae 6b069a0
docs(groups): remove ephemeral model example
seonghobae 3b17ff0
fix(groups): route advertised aliases end to end
seonghobae f448175
ci: hourly OpenCode maintenance agent routed through this gateway (#835)
seonghobae b0ffb33
fix(groups): preserve internal dispatch default
seonghobae b1fdec6
fix: isolate model group routing evidence
seonghobae b498c43
fix: preserve conduct semantics for model groups
seonghobae 356efc4
ci: minimize hourly loop permissions and installs
seonghobae e5b5095
fix: measure streamed group routing
seonghobae b353e6a
feat: route model groups across modalities (#837)
seonghobae fae8665
feat: normalize and edit model groups (#838)
seonghobae 852fc1e
fix: preserve capability routing contracts
seonghobae 19ab869
Merge pull request #839 from ContextualWisdomLab/fix/capability-routi…
seonghobae 7c6f564
release: v0.2.0 — model groups, cost-aware discovery, changelog baseline
seonghobae c4896ae
fix(discovery): tolerate null modality arrays
seonghobae 4bf5cf0
Merge pull request #841 from ContextualWisdomLab/fix/discovery-null-m…
seonghobae 49d0940
docs: specify model group product and technical contracts
seonghobae 4e0cbca
Merge pull request #842 from ContextualWisdomLab/docs/model-group-pro…
seonghobae 0e6507e
Merge remote-tracking branch 'origin/feat/model-group-cost-aware-disc…
seonghobae fd6dde7
release: align v0.2.0 changelog and lock metadata
seonghobae 6f49413
Merge pull request #840 from ContextualWisdomLab/release/v0.2.0-model…
seonghobae bc89d61
Merge remote-tracking branch 'origin/main' into stack-834
seonghobae 43717d3
feat: stream orchestrated reasoning summaries
seonghobae 5970a4d
Merge remote-tracking branch 'origin/feat/model-group-cost-aware-disc…
seonghobae 5cde1ae
fix: keep free reasoning streams fail closed
seonghobae 99ee64d
Merge remote-tracking branch 'origin/main' into feat/model-group-cost…
seonghobae 8df2bfb
fix: preserve free routing evidence and analytics
seonghobae 4f1e597
fix: distinguish failed Responses streams in analytics
seonghobae 2cc49b0
fix: pin structured free judge to selected agent
seonghobae 1fd7493
Merge branch 'feat/model-group-cost-aware-discovery' of https://githu…
seonghobae 34f77ec
Merge remote-tracking branch 'origin/main' into stack834b
seonghobae ee8ff16
fix: lock container dependencies and virtual capabilities
seonghobae a0d5552
fix: keep free passthrough on zero-cost models
seonghobae 8208f39
fix: prune removed routing measurements
seonghobae c262b8d
refactor: remove unreachable responses passthrough branch
seonghobae e38e959
fix: route virtual models across media capabilities
seonghobae 18b5890
Merge remote-tracking branch 'origin/feat/model-group-cost-aware-disc…
seonghobae fd5fa8c
fix: retain reset candidate routing rows
seonghobae fefd6e0
fix: preserve Responses instructions in workflows
seonghobae f828a73
Merge remote-tracking branch 'origin/main' into feat/model-group-cost…
seonghobae 21d47d1
test: align model-group missing-member code with canonical agent_not_…
seonghobae 3d774fe
fix(api): model-group creation returns canonical agent_not_found for …
seonghobae 1bc81de
ci: route hourly OpenCode loop through auto
seonghobae 3a7f4a6
Merge branch 'feat/model-group-cost-aware-discovery' of https://githu…
seonghobae 4f2daf0
fix: reject unsupported orchestrated structured output
seonghobae bccee78
docs: correct Responses stream options error
seonghobae 63436f4
Merge remote-tracking branch 'origin/feat/model-group-cost-aware-disc…
seonghobae 7181d3e
fix: retrieve URL-encoded model identifiers
seonghobae 9a084d0
fix: measure free capability and failover routing
seonghobae 3b0feb6
Merge pull request #843 from ContextualWisdomLab/feat/free-reasoning-…
seonghobae 453c4de
Merge remote-tracking branch 'origin/main' into codex/pr834-review-fixes
seonghobae 29915ef
fix: resolve model group review findings
seonghobae d4d1575
fix: close model group integration gaps
seonghobae 40e00f8
fix: align provider inventory and session cache scope
seonghobae 69cc674
fix: harden compose secrets and repeated reasoning summaries
seonghobae e2f4264
Merge remote-tracking branch 'origin/main' into codex/pr834-review-fixes
seonghobae 5492f80
fix: contain binary response disconnects
seonghobae ce83f95
fix: ground Zen free discovery in structured costs
seonghobae bfc95d7
fix: keep group judge within allowed members
seonghobae 4b1f2a8
fix: preserve free catalog evidence across reloads
seonghobae 1b0d2dd
Merge remote-tracking branch 'origin/main' into codex/pr834-review-fixes
seonghobae f1fea50
docs: assign unique model-group ADR number
seonghobae 098de8a
fix: preserve catalog capability evidence
seonghobae f9dc08b
fix: preserve declared Bytez endpoint capabilities
seonghobae 8d27aa6
fix: validate batch model identity at ingress
seonghobae a11c541
fix(api): normalize missing model group errors
seonghobae 9c02b19
docs(prd): align product bets with model groups
seonghobae a4269d9
fix(discovery): filter chat-only Bytez transports
seonghobae cb950a2
docs(adr): reserve model-group decision identifier
seonghobae 8341ac6
Merge remote-tracking branch 'origin/main' into codex/pr834-review-fixes
seonghobae 93b8016
feat: replace routing heuristics with measured evidence ledgers (#847)
seonghobae 8b03206
fix(admin): tolerate unavailable model-group state
seonghobae c146d1e
docs(loop): require PRD and measured web capacity
seonghobae 550c988
fix(admin): remove retired policy hint metric
seonghobae 5ecdd3f
fix(routing): validate evidence before mutation
seonghobae 7fa07ac
fix(discovery): remove model-name free inference guidance
seonghobae 503365f
Merge remote-tracking branch 'origin/main' into pr834-work
seonghobae 773caae
fix: complete #834 model-group persistence on the normalized agent-po…
seonghobae b03cd82
test: case-sensitive DDL pattern so prose cannot fake identifiers
seonghobae 6201564
Merge remote-tracking branch 'origin/main' into codex/pr834-exact
seonghobae 73f9a4f
docs: refresh protected-main gap evidence
seonghobae cef3498
docs: record exact provider regression suite
seonghobae daab699
docs: correct exact provider head
seonghobae a488687
fix(admin): make guidance customer-actionable
seonghobae a5f8d84
fix(admin): remove internal terms from customer copy
seonghobae 6b6352e
docs(ui): record customer-copy screenshot audit
seonghobae ed0698d
test(ci): lock hourly orchestrator loop contract
seonghobae 7896ac2
fix: replace synthetic admin status with truthful empty states
seonghobae d095eb4
docs: refresh exact-head remediation baseline
seonghobae e563920
fix(stream): cancel orchestration after Responses disconnect
seonghobae 3bc37b4
test: align boundary contracts with current routing
seonghobae b189e10
docs: complete public boundary docstrings
seonghobae 822a071
docs: refresh exact-head product gap queue
seonghobae 7526fdb
test: enforce complete public docstrings
seonghobae ab3aab2
ci: enforce protected exact-head merge loop
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| .git | ||
| .codegraph | ||
| .venv | ||
| .hypothesis | ||
| .secrets | ||
| __pycache__ | ||
| *.pyc | ||
| *.db | ||
| build | ||
| dist |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| You are the hourly maintenance agent for ContextualWisdomLab/contextual-orchestrator, | ||
| running through the contextual-orchestrator gateway itself. Provider models and | ||
| operator-managed groups are discovered from the gateway; never infer that two | ||
| differently named provider models are equivalent. | ||
|
|
||
| Work autonomously for at most 45 minutes, then stop with a short summary. Never | ||
| post intermediate progress reports. Priorities, in order: | ||
|
|
||
| 1. PR merge loop. Read PR numbers only from `/tmp/trusted-pr-numbers.txt`, which | ||
| contains same-repository branches selected before the privileged agent starts. | ||
| Never query, read, check out, comment on, or merge any other PR in this run; | ||
| fork PRs require a separate secret-free review workflow. For every listed PR: | ||
| a. Read reviewer comments (OpenCode, Devin, CodeRabbit, Strix, Noema, humans) and fix | ||
| valid findings on the branch; push fixes. | ||
| b. Re-check GitHub Checks. Retry transient provider/rate-limit failures | ||
| once before investigating. | ||
| c. When every required check is green and no unresolved blocking review | ||
| remains, merge (squash). | ||
| d. Move to the next open PR. Do not force-push; other agents may have | ||
| pushed concurrently — pull/rebase instead and respect their commits. | ||
|
|
||
| 2. Failing checks on main or scheduled workflows: trace logs to root cause and | ||
| open a focused fix PR (one concern per PR). | ||
|
|
||
| 3. Product gaps. If no PRs remain open, pick the highest-leverage gap from | ||
| docs/product_planning.md only after reading it, then reconcile the choice | ||
| with docs/product-technical-gap-baseline.md. Implement it with tests + | ||
| docstrings and open a PR that updates the baseline file. | ||
|
|
||
| Rules: | ||
| - Keep each change minimal and reviewable; stack dependent PRs when natural. | ||
| - Follow AGENTS.md governance: KV credentials (never os.getenv at runtime), | ||
| snake_case two-word object names, 100% test/docstring coverage intent, | ||
| paper-grounded decisions with APA citations in docs. | ||
| - Never weaken security gates; never disable required workflows. | ||
| - Web changes must preserve asynchronous responsiveness. Run the checked-in k6 | ||
| end-to-end scenario, record the measured concurrency envelope and bottleneck, | ||
| and improve only bottlenecks demonstrated by before/after evidence. | ||
| - Do not touch COPILOT_GITHUB_TOKEN or the existing review-agent key scheme. | ||
| - Do not invent routing weights, heuristics, or rules of thumb. Preserve measured | ||
| provider evidence and paper-grounded calibration; record missing evidence as a | ||
| gap instead of guessing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| name: OpenCode hourly maintenance loop | ||
|
|
||
| # Hourly autonomous maintenance: an OpenCode agent whose model traffic is | ||
| # served by THIS repository's gateway (contextual-orchestrator) with | ||
| # auto-discovered providers and operator-managed model-group routing. | ||
| # The agent works the PR queue (review -> fix -> recheck -> merge), root-causes | ||
| # failing checks, and advances docs/product-technical-gap-baseline.md items. | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "23 * * * *" # hourly at :23 UTC; offset avoids top-of-hour CI storms | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
| issues: write | ||
|
|
||
| concurrency: | ||
| group: opencode-hourly-loop | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| external-pr-review: | ||
| name: Review external PR diffs without secrets | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| contents: read | ||
| pull-requests: read | ||
| issues: none | ||
| steps: | ||
| - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # actions/checkout@v7 | ||
| with: | ||
| persist-credentials: false | ||
| fetch-depth: 0 | ||
| - name: Validate untrusted fork diffs without executing them | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| set -euo pipefail | ||
| gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls?state=open&per_page=100" \ | ||
| --jq '.[] | select(.head.repo.full_name != env.GITHUB_REPOSITORY) | [.number, .head.sha] | @tsv' | | ||
| while IFS=$'\t' read -r pr_number head_sha; do | ||
| [ -n "$pr_number" ] || continue | ||
| git fetch --no-tags origin "pull/${pr_number}/head:refs/review/external-${pr_number}" | ||
| test "$(git rev-parse "refs/review/external-${pr_number}")" = "$head_sha" | ||
| git diff --check "origin/main...refs/review/external-${pr_number}" | ||
| printf 'PR #%s head %s: diff integrity passed\n' "$pr_number" "$head_sha" >> "$GITHUB_STEP_SUMMARY" | ||
| done | ||
|
|
||
| loop: | ||
| name: Run OpenCode maintenance agent through the local gateway | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 55 | ||
| permissions: | ||
| contents: write # required for reviewed branch fixes; scoped to this job | ||
| pull-requests: write | ||
| issues: write | ||
| env: | ||
| # Bootstrap transport only: these flow into the process-local KV registry | ||
| # (see scripts/ci/serve_seeded_gateway.py); the gateway never reads them | ||
| # from the environment again after seeding. See docs/kv-credentials.md. | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} | ||
| OPENCODE_ZEN_API_KEY: ${{ secrets.OPENCODE_ZEN_API_KEY }} | ||
| BYTEZ_API_KEY: ${{ secrets.BYTEZ_API_KEY }} | ||
| NVIDIA_NIM_API_KEY: ${{ secrets.NVIDIA_NIM_API_KEY }} | ||
| NVIDIA_NIM_API_KEY_SUB: ${{ secrets.NVIDIA_NIM_API_KEY_SUB }} | ||
|
seonghobae marked this conversation as resolved.
Resolved
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # actions/checkout@v7 | ||
| with: | ||
| persist-credentials: true # the agent needs to push fixes/branches | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # actions/setup-python@v6 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Set up Node for the OpenCode CLI | ||
| uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # actions/setup-node@v4 | ||
| with: | ||
| node-version: "22" | ||
|
|
||
| - name: Install pinned runtime dependencies | ||
| run: python -m pip install --require-hashes -r fuzz/requirements-property.txt | ||
|
|
||
| - name: Install OpenCode CLI | ||
| run: npm ci --ignore-scripts | ||
|
devin-ai-integration[bot] marked this conversation as resolved.
Resolved
devin-ai-integration[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| - name: Generate ephemeral loopback gateway token | ||
| run: | | ||
| gateway_token=$(python -c 'import secrets; print(secrets.token_urlsafe(32))') | ||
| echo "::add-mask::$gateway_token" | ||
| echo "GATEWAY_BEARER_TOKEN=$gateway_token" >> "$GITHUB_ENV" | ||
|
|
||
|
github-advanced-security[bot] marked this conversation as resolved.
Fixed
seonghobae marked this conversation as resolved.
Fixed
|
||
| - name: Start the contextual-orchestrator gateway with auto-discovery | ||
| run: | | ||
| set -euo pipefail | ||
| nohup python -m scripts.ci.serve_seeded_gateway \ | ||
| --serve \ | ||
| --agents examples/agents.mock.json \ | ||
| --auto-discover-model-agents \ | ||
| --auth-token "$GATEWAY_BEARER_TOKEN" \ | ||
| --host 127.0.0.1 --port 8000 \ | ||
| > /tmp/gateway.log 2>&1 & | ||
| for _attempt in $(seq 1 30); do | ||
| if curl -fsS -H "Authorization: Bearer ${GATEWAY_BEARER_TOKEN}" \ | ||
| http://127.0.0.1:8000/healthz >/dev/null 2>&1; then | ||
| echo "gateway ready" | ||
| exit 0 | ||
| fi | ||
| sleep 2 | ||
| done | ||
| echo "::error::gateway did not become healthy"; tail -50 /tmp/gateway.log; exit 1 | ||
|
|
||
| - name: Point OpenCode at the local gateway | ||
| run: | | ||
| set -euo pipefail | ||
| mkdir -p ~/.config/opencode | ||
| cat > ~/.config/opencode/opencode.json <<JSON | ||
| { | ||
| "\$schema": "https://opencode.ai/config.json", | ||
| "model": "contextual_orchestrator_gateway/orchestrator/auto", | ||
| "provider": { | ||
| "contextual_orchestrator_gateway": { | ||
| "npm": "@ai-sdk/openai-compatible", | ||
| "name": "Contextual Orchestrator Gateway", | ||
| "options": { | ||
| "baseURL": "http://127.0.0.1:8000/v1", | ||
| "apiKey": "${GATEWAY_BEARER_TOKEN}" | ||
| }, | ||
| "models": { | ||
| "orchestrator/auto": { "name": "measured automatic orchestration" } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| JSON | ||
|
|
||
| - name: Build trusted same-repository PR allowlist | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls?state=open&per_page=100" \ | ||
| --jq '.[] | select(.head.repo.full_name == env.GITHUB_REPOSITORY) | .number' \ | ||
| > /tmp/trusted-pr-numbers.txt | ||
|
|
||
| - name: Run the hourly loop agent | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| ./node_modules/.bin/opencode run --model contextual_orchestrator_gateway/orchestrator/auto \ | ||
| "$(cat .github/opencode/hourly-loop-prompt.md)" | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| - name: Dump gateway log on failure | ||
| if: failure() | ||
| run: tail -100 /tmp/gateway.log | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,3 +12,5 @@ tempcred.txt | |
|
|
||
| # hypothesis fuzzing DB | ||
| .hypothesis/ | ||
| .secrets/ | ||
| .coverage | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.