fix: coerce null chat completion content to empty string at the normalization boundary

[sakibsadmanshajib]

Summary

The deploy of merge commit 2df3afd (PR #1163) failed its SDK replay
against the demo box (run 32879931588):

tests/chat-completions/chat-completions.test.ts:36
"Chat Completions > returns a valid chat completion via SDK"
AssertionError: expected 'object' to be 'string'
Expected: "string"
Received: "object"

Correction to the field named in the incident report. The failing
assertion is typeof response.choices[0].message.content, not
response.model as first reported. Line 36 of the test is:

expect(typeof response.choices[0].message.content).toBe("string");

response.model is asserted separately at line 51 in the very next test
in the same file (model field shows Hive alias not provider handle),
which passed. typeof null === "object" in JavaScript, so "Received:
object" for a content assertion means the wire value was a bare null,
not an actual object.

Root cause

Not PR #1163. That PR touches only apps/edge-api/internal/anthropic/,
adding request-side fields (Thinking, TopK, User,
ParallelToolCalls, ThinkingBlocks, ThinkingConfig) to that package's
own OAIRequest/OAIMessage structs. Those structs are private to the
anthropic package: grep -rn "anthropic\." apps/edge-api/internal/inference/*.go
returns no hits outside a comment. The OpenAI-compatible chat-completions
path (apps/edge-api/internal/inference) has its own separate
ChatCompletionResponse/ChatCompletionMessage types and never touches
the anthropic package's types. No shared struct, no cross-package
marshaling effect.

The real cause is the free-pool router, PR #1115 (hive-free, four
load-balanced provider members: OpenRouter dots-3-note-preview:free,
Google gemini-flash-latest, and two Groq gpt-oss-20b keys) plus its
retry fix PR #1155 (dispatchWithRetry, which lets a request land on any
member on retry). The replay's HIVE_TEST_MODEL is hive-free
(deploy-demo-box.yml), so this plain "Say hello" / max_tokens: 256
request could land on any of the four. The failing test ran for 50.8s
versus 1-7s for its four sibling tests in the same run, consistent with a
reasoning-capable member (Groq's gpt-oss-20b or Gemini's thinking-capable
flash model) spending its entire token budget on hidden reasoning before
hitting the length limit, returning finish_reason: "length" with
message.content omitted from the JSON entirely.

Go's ChatCompletionMessage.Content is *string. Unmarshaling a response
missing that key leaves it nil; normalizeChatCompletion in
apps/edge-api/internal/inference/chat_completions.go re-marshaled that
nil pointer straight back out as JSON null, which every OpenAI SDK
(this test uses the real openai npm package) treats as an unconditional
string.

This exact class of failure was already documented in
deploy-demo-box.yml's own comments for a different alias
(deepseek-v4-flash: "returned message.content as object/null/string
across probes"), which is why HIVE_TOOLS_MODEL was pinned away from it.
hive-free was never given the same treatment because its tool-free
smoke test uses no max_tokens pressure test and had not yet hit a
reasoning-heavy member in CI.

This gateway is provider-blind by design. No pool member's response
shape should leak to a client through the OpenAI-compatible surface.
OpenAI's own contract makes content nullable only alongside
tool_calls/function_call; every other case must be a string. That is
where this fix lives.

Fix

normalizeChatCompletion (apps/edge-api/internal/inference/chat_completions.go)
now coerces a nil, tool-free message.content to an empty string
before marshaling the response, via a new coerceNullContent helper. A
genuine tool-call message with content: null is left untouched, since
that shape is spec-correct per OpenAI and is exercised by the existing
"passes tools through" replay test.

Why no existing test caught this pre-merge

packages/sdk-tests only runs against a live deployed box (sdk-replay
job, post-merge), never in CI unit/integration tests, because it needs a
real LiteLLM + provider round trip. The Go unit-test suite for
normalizeChatCompletion (usage_clamp_test.go) had fixtures for
zero-completion-token clamping but none exercising an upstream response
with content entirely absent from the JSON. Added two new unit tests in
chat_completions_null_content_test.go that assert on the actual
marshaled wire bytes (not just the Go struct), which would have caught
this before any live deploy:

• TestNormalizeChatCompletion_NullContentCoercedToEmptyString:
  reproduces the missing-content upstream shape and asserts the
  outgoing JSON has "content":"", never null.
• TestNormalizeChatCompletion_NullContentPreservedWithToolCalls: asserts
  a genuine tool-call message's content: null is left alone.

Buglog entry

To be appended to .wolf/buglog.jsonl on main via a separate
buglog-only PR, per .claude/rules/openwolf.md:

{"date":"2026-08-25","error_message":"AssertionError: expected 'object' to be 'string' at tests/chat-completions/chat-completions.test.ts:36 (response.choices[0].message.content)","root_cause":"hive-free free-pool member (PR #1115/#1155) returned message.content omitted (null) after burning its max_tokens budget on hidden reasoning; normalizeChatCompletion re-marshaled the nil *string as JSON null instead of coercing it, leaking a non-OpenAI-contract shape to every SDK client","fix":"apps/edge-api/internal/inference/chat_completions.go: normalizeChatCompletion coerces nil, tool-free message.content to an empty string; tool-call messages with null content are left untouched per the OpenAI contract","tags":["free-pool","chat-completions","normalization","sdk-replay","hive-free"]}

Test plan

• go build ./apps/edge-api/...
• go vet ./apps/edge-api/...
• New unit tests pass:
  go test ./apps/edge-api/internal/inference/... -run 'NullContent|NormalizeChatCompletion' -v
• Full apps/edge-api suite green, no regressions:
  go test ./apps/edge-api/... -count=1 -short
• Next deploy's sdk-replay job green against the live demo box
  (this PR cannot verify that itself; the orchestrator confirms on
  merge)

Reproduction: static, from the actual failing-run log
(gh run view --job 97907530300 --log) plus static code reading. Live
reproduction against the demo box was not attempted directly (no API key
available to this agent; another agent was separately reported to be
investigating an unrelated Cloudflare Tunnel issue on the same box). The
box itself answered a plain reachability probe (401 on /v1/models
with no auth) during this session, so it was not down at the time of
investigation.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[coderabbitai]

Warning

Review limit reached

Next included review available in 56 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6b00eee8-9f77-40b2-983d-b3d8fdd9fd6f

📥 Commits

Reviewing files that changed from the base of the PR and between 2df3afd and f466d03.

📒 Files selected for processing (2)

• apps/edge-api/internal/inference/chat_completions.go
• apps/edge-api/internal/inference/chat_completions_null_content_test.go

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}