Skip to content

fix: allow opting out of response_format strict mode (#483) - #486

Merged
robert-j-y merged 1 commit into
mainfrom
devin/1777126021-issue-483-structured-outputs-strict
Apr 28, 2026
Merged

fix: allow opting out of response_format strict mode (#483)#486
robert-j-y merged 1 commit into
mainfrom
devin/1777126021-issue-483-structured-outputs-strict

Conversation

@robert-j-y

@robert-j-y robert-j-y commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #483 by adding a new structuredOutputs.strict setting that lets users opt out of response_format.json_schema.strict per chat model.

Problem

src/chat/index.ts hardcoded strict: true whenever a JSON schema response format was used:

response_format:
  responseFormat?.type === 'json'
    ? responseFormat.schema != null
      ? {
          type: 'json_schema',
          json_schema: {
            schema: responseFormat.schema,
            strict: true, // <-- hardcoded, no override
            name: responseFormat.name ?? 'response',
            ...(responseFormat.description && { description: responseFormat.description }),
          },
        }
      : { type: 'json_object' }
    : undefined,

There was no provider-setting, no call-site option, and no way for a user to opt out. For models whose downstream providers don't implement strict json_schema (e.g. moonshotai/kimi-k2.6 routed through Parasail/Venice/Io Net), OpenRouter returned HTTP 404 "No endpoints available matching your guardrail restrictions and data policy" because the strict flag eliminated every eligible endpoint.

Fix

Added a new optional structuredOutputs?: { strict?: boolean } field to OpenRouterChatSettings. When present, its strict value is forwarded to response_format.json_schema.strict. When omitted, the SDK continues to default to true (backward compatible).

Before

const model = openrouter.chat('moonshotai/kimi-k2.6');
// SDK always sends strict: true → 404 if provider doesn't support it

After

// Default behavior unchanged — strict: true
const defaultModel = openrouter.chat('moonshotai/kimi-k2.6');

// Opt out for providers that don't support strict mode
const relaxed = openrouter.chat('moonshotai/kimi-k2.6', {
  structuredOutputs: { strict: false },
});

// Explicit opt-in still works
const explicit = openrouter.chat('moonshotai/kimi-k2.6', {
  structuredOutputs: { strict: true },
});

Implementation notes

  • src/chat/index.ts:145 — replaced the hardcoded strict: true with this.settings.structuredOutputs?.strict ?? true.
  • src/types/openrouter-chat-settings.ts — added the structuredOutputs?: { strict?: boolean } setting alongside other top-level chat settings.
  • src/completion/index.ts — verified there is no response_format.json_schema branch (the legacy completion path forwards responseFormat directly without setting strict), so no mirror change was needed.
  • Existing should pass responseFormat for JSON schema structured outputs and should use default name when name is not provided in responseFormat tests in src/chat/index.test.ts continue to assert strict: true and pass without modification — the default is preserved.

Tests

Added e2e/issues/issue-483-response-format-strict-option.test.ts with 8 cases covering:

  1. Default (no setting) → strict: true (backward compat)
  2. structuredOutputs: { strict: false }strict: false
  3. structuredOutputs: { strict: true }strict: true
  4. structuredOutputs: { strict: undefined } → default true
  5. structuredOutputs: {} (empty object) → default true
  6. Strict override preserves name and description
  7. No response_format emitted when call has no responseFormat, regardless of structuredOutputs
  8. responseFormat: { type: 'json' } (no schema) still emits json_object, no strict, no json_schema

Verified the failing-on-main subset of these tests reproduce the bug before applying the fix (2/8 fail on main, all 8 pass on this branch).

Human review checklist

Verified against the diff at HEAD:

  • Default behavior is preserved: chat(model) with no structuredOutputs setting still emits strict: true (existing tests in src/chat/index.test.ts are unmodified and continue to pass; default-case test in the new file confirms it explicitly).
  • No type assertions / as casts in the new test file (per repo no-casting rule).
  • The public type addition (OpenRouterChatSettings.structuredOutputs) is the right shape — option lives on the chat settings, mirrors reasoning / plugins placement, and is documented with TSDoc.
  • No existing test in src/chat/index.test.ts was modified — git diff --merge-base origin/main -- src/chat/index.test.ts is empty.
  • Changeset is minor (new public setting = small feature).

Checklist

  • I have run pnpm stylecheck and pnpm typecheck
  • I have run pnpm test and all tests pass (421 passed)
  • I have added tests for my changes (e2e/issues/issue-483-response-format-strict-option.test.ts, 8 cases — all pass; the two strict: false cases failed on main as expected before the fix)
  • I have updated documentation (TSDoc on the new structuredOutputs setting)

Changeset

  • I have run pnpm changeset to create a changeset file (minor)

Co-Authored-By: Robert Yeakel <robert.yeakel@openrouter.ai>
@robert-j-y
robert-j-y merged commit 82e8014 into main Apr 28, 2026
2 checks passed
@robert-j-y
robert-j-y deleted the devin/1777126021-issue-483-structured-outputs-strict branch April 28, 2026 19:09
@github-actions github-actions Bot mentioned this pull request Apr 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

response_format.json_schema.strict is hardcoded to true, breaks routing for models whose providers don't support strict mode

1 participant