docs(gap-baseline): record the sidecar preflight max_tokens root cause - #921
Merged
Conversation
Fulfills the reference added in .github#1436's code comment (Devin flagged it as a missing baseline entry when it merely pointed at a not-yet-written one). Records the exact-evidence trail (downloaded strix-reports artifact from this repo's own PR #912 run) that this repo's PRs cannot fix directly since the sidecar is central-.github-owned infrastructure.
|
Warning Review limit reachedNext included review available in 51 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthrough기술 격차 기준선 문서에 2026-08-30 검증 결과를 추가했다. Changes기술 격차 기준선
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: 🔵 Low · up to This documentation-only change has no direct production impact, but it currently contains an unsupported claim about the next scheduler pass and a contradictory statement about the gateway timeout, which could mislead future troubleshooting. It is mergeable with explicit owner follow-up to correct those statements. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.) ✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…ar fix .github#1436 merged (admin bypass, structurally deadlocked check — evidence on the PR). Re-queued opencode-review/noema-review/strix on this PR plus #911/#920 as the live canary: the specific max_tokens 502 symptom is confirmed fixed, but noema-review still failed with a distinct signature (bytez discovery 500 + preflight finding zero passing routes). Records what's confirmed, what's still open, and the working hypothesis (concurrent-run rate-limit contention) pending a clean re-observation.
Devin flagged (on #921) that ModelClient._response_content returns successfully for any string content, including "", so the entry's "reasoning consumes the budget, content comes back empty" narrative doesn't match the code, and the generic error message quoted implies a narrower condition (non-string/absent content, reasoning falsy) than originally claimed. Verified against the code and the original strix-reports artifact (no raw provider payload was ever captured — sanitized by design), corrected the entry to state what's actually evidenced (budget mismatch reproduces, matching it fixes it) versus what was an unverified hypothesis, and noted the fix's own tests target a different, stricter function (the launcher's own preflight content check) that is unaffected by this correction.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
seonghobae
pushed a commit
to ContextualWisdomLab/.github
that referenced
this pull request
Aug 30, 2026
…al reasoning latency Exact-evidence reproduction (Strix run 33306775025 on ContextualWisdomLab/contextual-orchestrator#921, job 99244624298): the routing probe marked a DeepSeek NIM route "ready" in 18s, then this identical end-to-end gateway request against that same healthy route was cut off by curl's own --max-time bound at exactly 30.0s -- "gateway preflight request could not reach the local sidecar" was that timeout, not a real connectivity failure. 30s is inconsistent with real reasoning-model completion latency and with this org's own stated accuracy-over-speed review policy; this job already budgets 120 minutes overall. Raised the bound to 120s so a genuinely healthy, slower route isn't rejected by an arbitrarily tight preflight check. Verified RED before GREEN: the new test (test_gateway_preflight_curl_timeout_tolerates_real_reasoning_latency) fails against the pre-fix 30s bound (confirmed by stashing the fix and re-running just that test) and passes after restoring it. Full suite: 1884 passed, 1 skipped, 25 subtests passed.
…evidence Downloaded and inspected the actual strix-reports artifact instead of continuing to speculate: the real causes were (1) noema-review/ opencode-review having zero visibility into per-route preflight rejection reasons, and (2) the gateway preflight's 30s curl timeout cutting off a route the routing probe had just proven healthy in 18s. Both fixed and RED/GREEN-tested in ContextualWisdomLab/.github#1440.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…gling "above" Devin caught it on #921: the previous edit deleted the paragraph stating the rate-limit hypothesis while the replacement text still said "the hypothesis above," leaving nothing for that reference to point to. Restated it inline.
…ned 8/27, gateway check broke 8/29) Investigated whether .github's opencode-review verdict-checker itself was defective, since it's been failing org-wide for days. It isn't: git history shows it was a rubber stamp (always exit 0) until 8/27, when it was correctly rewritten to require a real matching review. Two days later, 8/29's "exercise exact gateway readiness" commit introduced the end-to-end gateway check that shipped with the max_tokens:16 bug (#1436 fixed today). The now-strict checker collided with a newly-broken dispatch path, not a checker design flaw. Confirmed via #1246: last real opencode-agent review was 8/23, none since despite the head moving forward repeatedly.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
1. Softened "will resolve on the scheduler's next pass" to not assert an outcome that hasn't been verified. 2. Removed the false implication that the routing probe's 10s per-candidate timeout says anything about the separate gateway curl timeout completing in time -- especially now stale since that curl timeout was raised 30s->120s in .github#1440 after being found too tight for real reasoning-model latency.
seonghobae
pushed a commit
that referenced
this pull request
Aug 30, 2026
Split the 44/46-failure count into its schedule vs workflow_dispatch components (43 schedule: 42 failure + 1 cancelled; 3 workflow_dispatch: 2 failure + 1 skipped) instead of the looser "44 of 46 scheduled runs" phrasing, and independently confirmed the corroborating http_status_500 against the primary job log (contextual-orchestrator PR #921, job 99243631744, 2026-08-30T10:26:47Z) rather than only the paraphrase in the existing gap-baseline entry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Gs7KmNvH75nxz1sL8mKjw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fulfills a reference
ContextualWisdomLab/.github#1436added inscripts/ci/contextual_orchestrator_review_sidecar.sh's code comment, pointing here for the exact-evidence trail behind that PR's fix. Devin's review on #1436 correctly flagged that the comment named this entry before it existed — this PR is that entry.Records the root cause of the org-wide
opencode-review/noema-review/strixfailure signature every open PR has been showing at sidecar boot: the sidecar's routing probe already proved anvidia_nimDeepSeek route healthy at a 4096-token budget, but a separate end-to-end gateway check re-tested the same route withmax_tokens:16— too small for a reasoning model, which returnsreasoningwith nocontent, misreported by the server as502 invalid_structured_output.This repo's PRs can't fix the sidecar directly (it's central
.github-owned infrastructure); this is a documentation-only PR recording the evidence trail.Verification
python3 -m pytest tests/test_hourly_opencode_loop_contract.py— 1 passed (the gap-baseline doc contract)git diff --check— cleanGenerated by Claude Code
Summary by CodeRabbit