fix(openai): Uint8Array body + shrink few-shot to 1 example (Devvit 400 round 5)#36
Conversation
…Devvit 400 round 5) PR #32 (single message), #33 (source ASCII), #34 (drop reasoning_effort + verbosity), #35 (eliminate `\n` from content) all shipped. Production v0.0.36 still returns HTTP 400 "We could not parse the JSON body". Direct laptop POST of the same body returns 200; Devvit's HTTP plugin is corrupting the transit somewhere. Two-axis change in one PR: 1. Body as Uint8Array (not string). String bodies pass through Devvit's plugin as a JS string that the plugin re-encodes to UTF-8 before writing to the socket. Large stringified-JSON bodies appear to corrupt during that re-encode. Uint8Array bypasses it: bytes are final, plugin only streams them. Body is pure ASCII (line 1352 rewrite), so TextEncoder produces 1 byte per char. 2. Few-shot truncated to 1 example. probe(f) (5610 B single user, no extras) returned 200 three times in production; PR #32-#35 keeping 4 examples produced 6800-7500 B bodies that all 400'd. Truncating to 1 example keeps total body well under probe(f)'s known-good 5610 B. Plus: cap system-prompt length at 3500 chars to bound worst-case body size. Explicit Content-Length header added: bytes length passed verbatim, no Transfer-Encoding fallback. Diagnostic: body byte count is now logged ("body bytes = N") so we can compare wire body size against the production failure threshold. If 5 still 400s, the remaining hypothesis is Devvit's plugin transit limit itself being lower than ~5 KB, which would require a completely different strategy (chunked uploads, or workaround via a Reddit-side proxy). Gates: `npm run check` 4/4 PASS.
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refines the callOpenAI function to improve reliability when communicating with the OpenAI API. Key changes include truncating the system prompt to 3500 characters, limiting few-shot examples to one, and sending the request body as a Uint8Array to avoid encoding issues. The reviewer raised concerns regarding the silent truncation of the system prompt and the manual setting of the Content-Length header, suggesting better error handling for the former and caution regarding the latter due to potential compatibility issues.
| const exampleLine = (ex: (typeof FEW_SHOT_EXAMPLES)[number], i: number) => | ||
| `EXAMPLE ${i + 1} INPUT: ${collapse(ex.user)} OUTPUT: ${JSON.stringify(ex.assistant)}`; | ||
| const SHORT_PROMPT_CHARS = 3500; // hard cap on system prompt; rest of body stays small | ||
| const systemShort = collapse(VIBE_MOD_SYSTEM_PROMPT).slice(0, SHORT_PROMPT_CHARS); |
There was a problem hiding this comment.
The silent truncation of the system prompt at 3500 characters is risky. If the prompt grows in the future, this could cut off critical instructions (such as the clarification mode logic or the JSON schema definition) at the end of the string. This would lead to unpredictable LLM behavior or parsing errors. Consider adding a check to log a warning or throw an error if the collapsed prompt exceeds this limit, rather than silently slicing it.
| body: asciiSafeBody, | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| 'Content-Length': String(bodyBytes.byteLength), |
There was a problem hiding this comment.
Manually setting the Content-Length header is generally discouraged when using fetch. In many standard-compliant environments (like browsers or Node.js with undici), Content-Length is a forbidden header name and attempting to set it manually will throw a TypeError. While this may be a necessary workaround for the current Devvit HTTP plugin issues, it could cause failures if the environment becomes more standard-compliant or when running in different execution contexts (e.g., unit tests). Since bodyBytes is a Uint8Array, most fetch implementations will automatically calculate and set the correct Content-Length.
fix(openai): Uint8Array body + shrink few-shot to 1 example (Devvit 400 round 5)
…d 6) PR #32-#36 all shipped, production still 400 from Devvit transit. v0.0.37 sent body bytes=4401 (smaller than probe(f)'s 5610 B which was 200), so size is not the constraint. Remaining variable: content character composition. probe(f) had content = `'a'.repeat(5500)` (no JSON syntax characters). All our shipped fixes had content containing inline `JSON.stringify(ex.assistant)` which produces many `\"` `\\` escape sequences when re-stringified by the outer body wrapper. Hypothesis: Devvit's transit corrupts bodies with high `\"` density in the content field. Eliminate the variable: serialize few-shot examples as plain English with `=` and `;` separators instead of `{}:,"`. The content string now contains zero `{`, `}`, `[`, `]`, `:`, `,`, `"` characters from our prompt data. Implementation: - New `flattenValue` recursively serializes any value (string/number/bool/ array/object) to plain English: arrays as `a or b or c`, objects as `key=value key=value`, strings whitespace-collapsed. - `flattenExample` walks each few-shot example's `assistant` field through flattenValue producing `EXAMPLE OUTPUT id=r_xxx; name=...; on=onPostSubmit; ...`. - Outer body shape unchanged: model, response_format, messages (1), max_tokens. - Body sent as string (not Uint8Array) since PR #36 byte body didn't help. Prompt fidelity: - Model still learns rule schema from the system prompt (unchanged from PR #33). - response_format: { type: 'json_object' } forces strict JSON output. - Local POST returns 200 with valid compiled rule: `{"id":"r_new_account_modqueue","name":"New account to mod queue", "sourceNL":"...","on":["onPostSubmit"],"when":{...},"then":[...]}` Gates: `npm run check` 4/4 PASS.
1 participant
Round 5. Previous rounds: PR #32 single message, #33 source ASCII, #34 drop features, #35 flatten content -- all still 400 in production despite local 200. This round: (a) body as Uint8Array bypasses Devvit's string-body re-encode path; (b) few-shot truncated to 1 example so total body fits well under probe(f)'s known-good 5610 B. CI 4/4 PASS. Logs body-byte-count for measurement.
🤖 Generated with Claude Code