Repository navigation
fix(combo): derive session stickiness key from Responses API .input, not just .messages (#7270) - #7277
Conversation
There was a problem hiding this comment.
Code Review
This pull request resolves issue #7270 by introducing the normalizeStickinessMessages helper, which normalizes request bodies to support session stickiness across both Chat Completions (.messages) and OpenAI Responses API (.input) formats. Corresponding unit tests have been added to verify this behavior. The review feedback correctly points out that the helper fails to handle an array of strings (string[]) for the input field, which would silently disable stickiness, and suggests updating both the implementation and the unit tests to handle and verify this case.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if (Array.isArray(input) && input.length > 0) { | ||
| return input as Array<{ role?: string; content?: unknown }>; | ||
| } |
There was a problem hiding this comment.
If the input field is an array of strings (e.g., string[]), which is a valid format for the Responses API, normalizeStickinessMessages currently returns it directly cast as Array<{ role?: string; content?: unknown }>. This causes deriveMessageHash to fail to find any message with role === "user" (since strings do not have a role property), resulting in a null hash and silently disabling session stickiness. Checking if the first element is a string and mapping the array to message objects resolves this issue.
if (Array.isArray(input) && input.length > 0) {
if (typeof input[0] === "string") {
return input.map((content) => ({ role: "user", content }));
}
return input as Array<{ role?: string; content?: unknown }>;
}| const fromInputString = stick.normalizeStickinessMessages({ input: "plain string turn" }); | ||
| assert.deepEqual(fromInputString, [{ role: "user", content: "plain string turn" }]); |
There was a problem hiding this comment.
Add a unit test case to verify that normalizeStickinessMessages correctly handles and normalizes an input array of strings (string[]).
const fromInputString = stick.normalizeStickinessMessages({ input: "plain string turn" });
assert.deepEqual(fromInputString, [{ role: "user", content: "plain string turn" }]);
const fromInputStringArray = stick.normalizeStickinessMessages({
input: ["hello", "world"],
});
assert.deepEqual(fromInputStringArray, [
{ role: "user", content: "hello" },
{ role: "user", content: "world" },
]);|
Thanks for the thorough root-cause writeup and the regression test that drives the real One gap before we merge: Could you:
Everything else here looks solid — the |
…vation (cheaper-inference#7270) normalizeStickinessMessages()'s Array.isArray(input) branch cast the array straight through, so a Responses-API `.input` array of PLAIN STRINGS (each string shorthand for a user message) never matched deriveMessageHash's `role === "user"` lookup and the key stayed null — the same fail-open bug cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to {role: "user", content: item}, mirroring the string-item handling already established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem. Adds a regression test case (unit-level normalizeStickinessMessages assertion + round-robin re-pin case) so the fix's own suite covers this shape. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
0f55956
into
cheaper-inference:release/v3.8.49
|
Thanks @alltomatos! Merged into release/v3.8.49 — validated in the combined merge-train, all gates green. Great catches (Next16 async params, credential-tuple dedupe). |
…not just .messages (cheaper-inference#7270) (cheaper-inference#7277) * fix(combo): derive session stickiness key from Responses API .input, not just .messages (cheaper-inference#7270) * fix(combo): map bare-string .input array items in stickiness key derivation (cheaper-inference#7270) normalizeStickinessMessages()'s Array.isArray(input) branch cast the array straight through, so a Responses-API `.input` array of PLAIN STRINGS (each string shorthand for a user message) never matched deriveMessageHash's `role === "user"` lookup and the key stayed null — the same fail-open bug cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to {role: "user", content: item}, mirroring the string-item handling already established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem. Adds a regression test case (unit-level normalizeStickinessMessages assertion + round-robin re-pin case) so the fix's own suite covers this shape. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…not just .messages (cheaper-inference#7270) (cheaper-inference#7277) * fix(combo): derive session stickiness key from Responses API .input, not just .messages (cheaper-inference#7270) * fix(combo): map bare-string .input array items in stickiness key derivation (cheaper-inference#7270) normalizeStickinessMessages()'s Array.isArray(input) branch cast the array straight through, so a Responses-API `.input` array of PLAIN STRINGS (each string shorthand for a user message) never matched deriveMessageHash's `role === "user"` lookup and the key stayed null — the same fail-open bug cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to {role: "user", content: item}, mirroring the string-item handling already established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem. Adds a regression test case (unit-level normalizeStickinessMessages assertion + round-robin re-pin case) so the fix's own suite covers this shape. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com> --------- Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Problem
Combo session stickiness is a silent no-op for the entire OpenAI Responses API
(
/v1/responses) surface. A single conversation is re-ordered across accounts everyturn — regardless of the strategy (round-robin / random / strict-random / weighted /
priority) or the "session stickiness" toggle — while the same conversation over
/v1/chat/completionssticks correctly.Reported in the Discord (EN) support channel: "I used round robin/random/strict random
and enabled session stickiness … it still throws the chat between accounts (the same
chat, not a new one) … It's OpenAI responses API."
Root cause
The stickiness key is derived exclusively from
body.messages:applySessionStickiness(open-sse/services/combo.ts:1208,:2634) is called withbody.messages.deriveMessageHash(open-sse/services/combo/sessionStickiness.ts) guards onArray.isArray(messages)and returnsnull(fail-open, silent) otherwise.Responses-API requests carry their turns in
.input(a plain string or an array ofmessage items) and never populate
.messages. So the key resolves tonull, the stickylookup/binding are skipped, and every turn is re-ordered as if stickiness were disabled.
Format translation (
.input→.messages) happens per-target, insidehandleSingleModel/handleChatCore, i.e. after combo target ordering has alreadybeen decided — so it cannot help the stickiness key derivation upstream.
Fix
Derive the key from a normalized view of the request that covers both wire shapes,
rather than teaching
deriveMessageHashabout a single format:normalizeStickinessMessages(body)insessionStickiness.tsreturns a{role, content}[]view from.messages(Chat Completions, takes precedence), or.input— handling both the plain-string and the array-of-message-items forms of theResponses API. Returns
nullwhen neither carrier yields turns (fail-open, unchanged).handleComboChatweighted/priority/random path andhandleRoundRobinCombo) now passnormalizeStickinessMessages(body)instead of theraw
body.messages.deriveMessageHashand the sticky store are untouched; the change is confined to thekey's input. Chat Completions behavior is preserved (
.messagesstill wins), and thefail-open path for turnless bodies is preserved.
Verification
New regression test (fails on the unfixed code, passes after the fix). It drives the
real
handleComboChatwith a round-robin combo and an.input-shaped Responsesbody:
node --import tsx/esm --test tests/unit/combo-stickiness-responses-input-7270.test.ts # 1..4 — # pass 4 # fail 0Before the fix, 3 of 4 failed — the
.inputconversation rotatedconn-A → conn-B → conn-C …(set size 3) instead of pinning, and the normalized-key assertions had nohelper to call. After the fix:
normalizeStickinessMessagescovers.messages,.input(array),.input(string),precedence, and the empty→null case.
Neighbouring stickiness suites — full pass, no regressions:
node --import tsx/esm --test \ tests/unit/combo-session-stickiness.test.ts \ tests/unit/combo-rr-session-stickiness-3825.test.ts \ tests/unit/combo-disable-session-stickiness.test.ts # 1..35 — # pass 35 # fail 0Lint + typecheck clean:
Diff summary
open-sse/services/combo/sessionStickiness.ts— addnormalizeStickinessMessages()(normalizes
.messages/ Responses-API.inputstring|array →{role,content}[]).open-sse/services/combo.ts— bothapplySessionStickinesscall sites pass thenormalized view instead of
body.messages.tests/unit/combo-stickiness-responses-input-7270.test.ts— new regression test.Closes #7270.
Thanks to the reporter and to whoever wrote the issue's precise pipeline trace — it
pinpointed the
.input/.messagesgap exactly.