feat(hindsight): route LLM via ChatGPT subscription, qwen fallback - #79
Merged
Conversation
qwen3.6-35b backend errors/timeouts on retain/recall. Point hindsight at litellm alias `hindsight-default` -> chatgpt/gpt-5.4-mini (native chatgpt/ provider, Codex OAuth device flow) falling back to qwen. - bump litellm to v1.88.1 for chatgpt responses-bridge fixes (#26219) - mount writable /chatgpt-tokens, set CHATGPT_TOKEN_DIR; auth.json is rewritten on token refresh Host steps required before deploy: - mkdir /mnt/spool/apps/config/hindsight/litellm/chatgpt-tokens, chown 568:568 - litellm config.yaml: add `hindsight-default` model alias and router_settings fallback to qwen3.6-35b-think-general - first chatgpt/ request prints device-code login URL to container logs; complete in browser within 15 min Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
qwen3.6-35b backend erroring/timing out on Hindsight retain/recall. ChatGPT Business subscription now available — route through litellm's native
chatgpt/provider (Codex OAuth) with qwen as fallback.Changes
v1.83.14-stable→v1.88.1— old pin has thechatgpt/provider but misses responses-bridge fixes (fix(chatgpt): preserve responses routing and recover empty output BerriAI/litellm#26219)CHATGPT_TOKEN_DIR=/chatgpt-tokens+ writable token volume (auth.json rewritten on auto-refresh)qwen3.6-35b-think-general→ litellm aliashindsight-default, so provider swaps stay config-onlyModel choice:
chatgpt/gpt-5.4-mini— highest Business quota (60–350 msgs/5h), sized for extraction workloads.gpt-5.5-instantrejected by Codex backend for ChatGPT-account auth.Host steps before deploy (config outside repo)
mkdir /mnt/spool/apps/config/hindsight/litellm/chatgpt-tokens && chown 568:568it/mnt/spool/apps/config/hindsight/litellm/config.yaml: addhindsight-default→chatgpt/gpt-5.4-minitomodel_list; addrouter_settings: fallbacks: [{hindsight-default: ["qwen3.6-35b-think-general"]}]docker logs -f hindsight-litellmwithin 15 minVerification
./scripts/validate-stack.sh hindsightpassesresponse_format— litellm allowlist gap, [Bug]: ChatGPT responses allowlist drops text, so response_format/json_schema never reaches ChatGPT Codex backend BerriAI/litellm#24356)🤖 Generated with Claude Code