fix(openai): pin model to gpt-5.4-nano (probe-verified) - Devvit 400 round 7#38
Conversation
…ound 7 PR #32-#37 all shipped, production still HTTP 400. body chars now down to 4279 (well under probe(f)'s 5610 B 200-OK), content has no JSON syntax, all known shape variables aligned with probe(f) -- yet 400 persists. Only untested variable left: model. probe(b)(d)(e)(f) all hardcoded `gpt-5.4-nano` and returned 200 in production. callOpenAI defaults to `gpt-5.4-mini` via settings.get, never tested in production. Pinning to the probe-verified model isolates whether the model selection itself is what differs between probe-success and callOpenAI-failure. Plus: log resolved model + settings.get value so subsequent rounds can see what the user has stored for openaiModel. Gates: `npm run check` 4/4 PASS.
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request hardcodes the OpenAI model to 'gpt-5.4-nano' for debugging and adds logging for the model configuration. Feedback highlights that this bypasses user-configured settings and introduces a metadata inconsistency, as the stored RuleBundle may incorrectly report the model used for compilation.
| const model = 'gpt-5.4-nano'; | ||
| let settingsModel: string | null = null; | ||
| try { | ||
| model = ((await settings.get('openaiModel')) as string) || 'gpt-5.4-mini'; | ||
| settingsModel = ((await settings.get('openaiModel')) as string) || null; | ||
| console.log('[vibe-mod] callOpenAI: settings.get(openaiModel) =', JSON.stringify(settingsModel)); | ||
| } catch (err) { | ||
| console.warn('[vibe-mod] callOpenAI: settings.get(openaiModel) threw — using default:', describeErr(err)); | ||
| console.warn('[vibe-mod] callOpenAI: settings.get(openaiModel) threw — using nano default:', describeErr(err)); | ||
| } | ||
| void settingsModel; | ||
| console.log('[vibe-mod] callOpenAI: model resolved =', model); |
There was a problem hiding this comment.
The openaiModel setting is currently fetched and logged, but its value is ignored because the model variable is hardcoded to 'gpt-5.4-nano'. While the PR description notes this is a temporary measure for debugging, it introduces a regression where user-configured models are bypassed.
Additionally, this creates a metadata inconsistency: the RuleBundle (created in the compose-rule-submit handler) still resolves the model from settings to populate its llmModel field. Consequently, the stored bundle will incorrectly report the model used for compilation. Consider ensuring that the llmModel metadata in the bundle accurately reflects the hardcoded model being used, or allowing the setting to override the pin if present.
fix(openai): pin model to gpt-5.4-nano (probe-verified) - Devvit 400 round 7
…use of HTTP 400
# Root cause
`settings.get('openaiModel')` returns the value of a Devvit SELECTION
schema field. SELECTION fields return a `string[]` -- even for single
selection. We were passing the raw array straight into the request body
as `"model": ["gpt-5.4-mini"]`, which OpenAI rejected as unparseable
JSON for the `model` field. OpenAI's error wording masked the underlying
problem: it said `We could not parse the JSON body of your request`,
which sounds like a structural JSON error but is actually a type
mismatch on a single field. Production log proof from v0.0.39:
[vibe-mod] callOpenAI: openaiModel raw = ["gpt-5.4-mini"]
This single bug accounts for PR #32-#37 all returning HTTP 400 with
seemingly unrelated changes. PR #38 (v0.0.39) worked around it by
hardcoding `gpt-5.4-nano` (a string), which made the request body
valid -- callOpenAI succeeded, the toast became
`The compiled rule contained an action this app does not support`
(a downstream issue, not a transit one).
# Fix
* Unwrap the SELECTION result: if `Array.isArray(raw)`, take `raw[0]`.
* Log both the raw and unwrapped values for diagnostic visibility.
* Honour the user-configured model (default `gpt-5.4-mini`).
# Restore proper few-shot examples
PR #37 had stripped JSON syntax from message content (`flattenExample`,
output as `key=value; key=value`) on a false hypothesis. With JSON
syntax removed, the model learned a non-schema output shape -- v0.0.39
emitted `{"modqueue":{"note":"..."}}` instead of `{"action":"modqueue",
"params":{"note":"..."}}`. Restoring `JSON.stringify(ex.assistant)` for
each example re-teaches the schema.
Single-user composition (PR #32) is preserved -- it's a sensible
packaging choice and was never causing the 400; the model array was.
# Tests
* `npm run check` 4/4 PASS.
* Local POST of the new body to OpenAI returns 200 with a valid
compiled rule conforming to the SAFE-action schema.
* After merge: `devvit upload` -> v0.0.40 -> Chrome verify -> production
callOpenAI returns 200 -> toast shows compile success.
# Why probe(b)/(d)/(e)/(f) all worked
The probes hardcoded `'gpt-5.4-nano'` (a string literal) -- so they
sent a valid `model` field and OpenAI parsed their requests fine. Only
callOpenAI used `settings.get(...)` and got hit by the array bug.
…branch cleanup record Records the 7-round speculative-fix loop that PR #32-#38 went through, the actual root cause (PR #39: SELECTION-typed setting returning string[] not string), and the end-to-end verification: production toast 'Compiled rule "New-account posts to mod queue". Dry-run started -- check Dashboard in 30s.' captured via Chrome on v0.0.41. Probe code (synthetic-compile-probe handler + scheduler entry) lived only on the fix/openai-error-handling branch and never reached main; that branch is now deleted from origin. grep -c synthetic-compile-probe across src/server/index.ts and devvit.json: 0 + 0. No code change. Closes the postmortem loop on issue and satisfies the goal-condition (5) requirement for an explicit MERGED PR surface recording the probe-removal decision.
1 participant
Round 7. body chars now 4279 (well under probe(f)), content has no JSON syntax. Only untested variable left: model. probe(b)(d)(e)(f) all used
gpt-5.4-nanoand were 200 in production. callOpenAI defaults togpt-5.4-mini, never tested in production. This PR pins to nano + logs resolved model.🤖 Generated with Claude Code