Skip to content

fix(combo): derive session stickiness key from Responses API .input, not just .messages (#7270) - #7277

Merged
diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.49from
alltomatos:fix/7270-stickiness-responses-input
Jul 18, 2026
Merged

diegosouzapw merged 2 commits into
cheaper-inference:release/v3.8.49from
alltomatos:fix/7270-stickiness-responses-input

Conversation

@alltomatos

Copy link
Copy Markdown
Contributor

Problem

Combo session stickiness is a silent no-op for the entire OpenAI Responses API
(/v1/responses) surface. A single conversation is re-ordered across accounts every
turn — regardless of the strategy (round-robin / random / strict-random / weighted /
priority) or the "session stickiness" toggle — while the same conversation over
/v1/chat/completions sticks correctly.

Reported in the Discord (EN) support channel: "I used round robin/random/strict random
and enabled session stickiness … it still throws the chat between accounts (the same
chat, not a new one) … It's OpenAI responses API."

Root cause

The stickiness key is derived exclusively from body.messages:

  • applySessionStickiness (open-sse/services/combo.ts:1208, :2634) is called with
    body.messages.
  • deriveMessageHash (open-sse/services/combo/sessionStickiness.ts) guards on
    Array.isArray(messages) and returns null (fail-open, silent) otherwise.

Responses-API requests carry their turns in .input (a plain string or an array of
message items) and never populate .messages. So the key resolves to null, the sticky
lookup/binding are skipped, and every turn is re-ordered as if stickiness were disabled.

Format translation (.input → .messages) happens per-target, inside
handleSingleModel/handleChatCore
, i.e. after combo target ordering has already
been decided — so it cannot help the stickiness key derivation upstream.

Fix

Derive the key from a normalized view of the request that covers both wire shapes,
rather than teaching deriveMessageHash about a single format:

  • New normalizeStickinessMessages(body) in sessionStickiness.ts returns a
    {role, content}[] view from .messages (Chat Completions, takes precedence), or
    .input — handling both the plain-string and the array-of-message-items forms of the
    Responses API. Returns null when neither carrier yields turns (fail-open, unchanged).
  • Both combo call sites (handleComboChat weighted/priority/random path and
    handleRoundRobinCombo) now pass normalizeStickinessMessages(body) instead of the
    raw body.messages.

deriveMessageHash and the sticky store are untouched; the change is confined to the
key's input. Chat Completions behavior is preserved (.messages still wins), and the
fail-open path for turnless bodies is preserved.

Verification

New regression test (fails on the unfixed code, passes after the fix). It drives the
real handleComboChat with a round-robin combo and an .input-shaped Responses
body:

node --import tsx/esm --test tests/unit/combo-stickiness-responses-input-7270.test.ts
# 1..4 — # pass 4 # fail 0

Before the fix, 3 of 4 failed — the .input conversation rotated conn-A → conn-B → conn-C … (set size 3) instead of pinning, and the normalized-key assertions had no
helper to call. After the fix:

  • a sessionless Responses-API conversation re-pins to its turn-1 connection across 5 turns;
  • distinct Responses-API conversations still spread across connections (spreading guard);
  • normalizeStickinessMessages covers .messages, .input (array), .input (string),
    precedence, and the empty→null case.

Neighbouring stickiness suites — full pass, no regressions:

node --import tsx/esm --test \
  tests/unit/combo-session-stickiness.test.ts \
  tests/unit/combo-rr-session-stickiness-3825.test.ts \
  tests/unit/combo-disable-session-stickiness.test.ts
# 1..35 — # pass 35 # fail 0

Lint + typecheck clean:

npx eslint --suppressions-location config/quality/eslint-suppressions.json \
  open-sse/services/combo/sessionStickiness.ts open-sse/services/combo.ts \
  tests/unit/combo-stickiness-responses-input-7270.test.ts   # 0 problems
npm run typecheck:core                                        # clean

Diff summary

  • open-sse/services/combo/sessionStickiness.ts — add normalizeStickinessMessages()
    (normalizes .messages / Responses-API .input string|array → {role,content}[]).
  • open-sse/services/combo.ts — both applySessionStickiness call sites pass the
    normalized view instead of body.messages.
  • tests/unit/combo-stickiness-responses-input-7270.test.ts — new regression test.

Closes #7270.

Thanks to the reporter and to whoever wrote the issue's precise pipeline trace — it
pinpointed the .input/.messages gap exactly.

@alltomatos
alltomatos requested a review from diegosouzapw as a code owner July 15, 2026 11:47

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request resolves issue #7270 by introducing the normalizeStickinessMessages helper, which normalizes request bodies to support session stickiness across both Chat Completions (.messages) and OpenAI Responses API (.input) formats. Corresponding unit tests have been added to verify this behavior. The review feedback correctly points out that the helper fails to handle an array of strings (string[]) for the input field, which would silently disable stickiness, and suggests updating both the implementation and the unit tests to handle and verify this case.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +226 to +228
if (Array.isArray(input) && input.length > 0) {
return input as Array<{ role?: string; content?: unknown }>;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the input field is an array of strings (e.g., string[]), which is a valid format for the Responses API, normalizeStickinessMessages currently returns it directly cast as Array<{ role?: string; content?: unknown }>. This causes deriveMessageHash to fail to find any message with role === "user" (since strings do not have a role property), resulting in a null hash and silently disabling session stickiness. Checking if the first element is a string and mapping the array to message objects resolves this issue.

  if (Array.isArray(input) && input.length > 0) {
    if (typeof input[0] === "string") {
      return input.map((content) => ({ role: "user", content }));
    }
    return input as Array<{ role?: string; content?: unknown }>;
  }

Comment on lines +111 to +112
const fromInputString = stick.normalizeStickinessMessages({ input: "plain string turn" });
assert.deepEqual(fromInputString, [{ role: "user", content: "plain string turn" }]);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add a unit test case to verify that normalizeStickinessMessages correctly handles and normalizes an input array of strings (string[]).

  const fromInputString = stick.normalizeStickinessMessages({ input: "plain string turn" });
  assert.deepEqual(fromInputString, [{ role: "user", content: "plain string turn" }]);

  const fromInputStringArray = stick.normalizeStickinessMessages({
    input: ["hello", "world"],
  });
  assert.deepEqual(fromInputStringArray, [
    { role: "user", content: "hello" },
    { role: "user", content: "world" },
  ]);

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks for the thorough root-cause writeup and the regression test that drives the real handleComboChat — this is exactly the right fix for #7270 and the precedence rule (.messages wins, then .input string, then .input array) matches how the rest of combo.ts already treats these two wire shapes.

One gap before we merge: normalizeStickinessMessages()'s Array.isArray(input) branch casts the array straight through, but the Responses API also allows .input to be an array of plain strings (each string is shorthand for a input_text user message) — this codebase already special-cases that exact shape in open-sse/utils/responsesInputNormalization.ts (normalizeCodexResponsesInputItem, typeof itemValue === "string"). I confirmed with a quick probe that normalizeStickinessMessages({ input: ["some text"] }) returns the raw string array unchanged, so deriveMessageHash never finds a role === "user" entry and the key stays null — the same fail-open bug this PR is fixing, just for a narrower wire shape. gemini-code-assist's automated review on this PR flagged the same thing independently.

Could you:

  1. Extend the array branch to map bare-string items to { role: "user", content: item } (or similar) before returning, and
  2. add a test case for that shape (both a direct normalizeStickinessMessages assertion and ideally a round-robin re-pin case like the ones you already have)?

Everything else here looks solid — the .messages path is untouched byte-for-byte so existing Chat Completions sessions keep the exact same key (no cache/affinity reset), and I verified no interaction with #7269 (that one only touches the executor-level input sanitizer, which runs after combo target ordering has already happened). Once the string-array case is covered we're good to merge as-is.

…vation (cheaper-inference#7270)

normalizeStickinessMessages()'s Array.isArray(input) branch cast the array
straight through, so a Responses-API `.input` array of PLAIN STRINGS (each
string shorthand for a user message) never matched deriveMessageHash's
`role === "user"` lookup and the key stayed null — the same fail-open bug
cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to
{role: "user", content: item}, mirroring the string-item handling already
established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem.

Adds a regression test case (unit-level normalizeStickinessMessages assertion
+ round-robin re-pin case) so the fix's own suite covers this shape.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit 0f55956 into cheaper-inference:release/v3.8.49 Jul 18, 2026
5 checks passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Thanks @alltomatos! Merged into release/v3.8.49 — validated in the combined merge-train, all gates green. Great catches (Next16 async params, credential-tuple dedupe).

HouMinXi pushed a commit to HouMinXi/OmniRoute that referenced this pull request Aug 2, 2026
…not just .messages (cheaper-inference#7270) (cheaper-inference#7277)

* fix(combo): derive session stickiness key from Responses API .input, not just .messages (cheaper-inference#7270)

* fix(combo): map bare-string .input array items in stickiness key derivation (cheaper-inference#7270)

normalizeStickinessMessages()'s Array.isArray(input) branch cast the array
straight through, so a Responses-API `.input` array of PLAIN STRINGS (each
string shorthand for a user message) never matched deriveMessageHash's
`role === "user"` lookup and the key stayed null — the same fail-open bug
cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to
{role: "user", content: item}, mirroring the string-item handling already
established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem.

Adds a regression test case (unit-level normalizeStickinessMessages assertion
+ round-robin re-pin case) so the fix's own suite covers this shape.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…not just .messages (cheaper-inference#7270) (cheaper-inference#7277)

* fix(combo): derive session stickiness key from Responses API .input, not just .messages (cheaper-inference#7270)

* fix(combo): map bare-string .input array items in stickiness key derivation (cheaper-inference#7270)

normalizeStickinessMessages()'s Array.isArray(input) branch cast the array
straight through, so a Responses-API `.input` array of PLAIN STRINGS (each
string shorthand for a user message) never matched deriveMessageHash's
`role === "user"` lookup and the key stayed null — the same fail-open bug
cheaper-inference#7270 fixed, just for this narrower wire shape. Map bare-string items to
{role: "user", content: item}, mirroring the string-item handling already
established in responsesInputNormalization.ts's normalizeCodexResponsesInputItem.

Adds a regression test case (unit-level normalizeStickinessMessages assertion
+ round-robin re-pin case) so the fix's own suite covers this shape.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>

---------

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(combo): session stickiness is a silent no-op on the OpenAI Responses API (.input never normalized to .messages)

2 participants