Skip to content

fix(auxiliary): avoid failed DeepSeek requests before structured tasks - #113111

Closed
fangliquanflq wants to merge 2 commits into
NousResearch:mainfrom
fangliquanflq:fix/auxiliary-deepseek-structured-output
Closed

fangliquanflq wants to merge 2 commits into
NousResearch:mainfrom
fangliquanflq:fix/auxiliary-deepseek-structured-output

Conversation

@fangliquanflq

@fangliquanflq fangliquanflq commented Sep 16, 2026 •

Copy link
Copy Markdown

What does this PR do?

DeepSeek auxiliary tasks now select json_object before dispatch instead of first sending an unsupported json_schema request. This removes one deterministic HTTP 400 and retry from every structured title, kanban, goal-judge, and plugin call routed to DeepSeek while retaining the existing rejection retry for unknown provider behavior.

Symptom

A structured auxiliary call routed to DeepSeek first fails with This response_format type is unavailable now, then succeeds only after Hermes retries without the structured-output field.

Impact

Every affected task pays for an avoidable failed request, extra round-trip latency, and a misleading error-like log entry. The feature eventually succeeds, but the cost repeats for every structured auxiliary call.

Bug Cause

Trigger: agent/auxiliary_client.py / _build_call_kwargs always forwards a caller's json_schema response format.

Causal chain:

  1. Title generation or another structured auxiliary caller supplies an OpenAI json_schema response format.
  2. The auxiliary request builder forwards that format without consulting the resolved provider and model capabilities.
  3. DeepSeek rejects the request with HTTP 400, and the recovery ladder sends a second request without the field.

Why it is wrong: DeepSeek's native API supports json_object but not OpenAI's json_schema variant, so the first request is guaranteed to fail.

Working sibling / contrast: The existing rejection ladder recovers by removing the format entirely, proving that prompt-compliant JSON is sufficient; DeepSeek also accepts json_object directly.

Ruled out: This is not a malformed schema or task-specific title bug. The direct provider comparison accepts the same request with json_object and rejects the json_schema variant, and every structured caller shares the same auxiliary request builder.

Fix

  • Select the response format centrally for every auxiliary route, including fallback routes.
  • Preserve json_schema only when model metadata positively identifies support; otherwise use json_object.
  • Treat native DeepSeek as JSON-object-only by default while allowing an explicit per-model supports_structured_output override.
  • Keep the reactive rejection retry as protection for stale or inaccurate capability data.

Related Issue

Closes #113064

Type of Change

  • Bug fix (non-breaking change that fixes an issue)

Changes Made

  • agent/auxiliary_client.py - choose a supported structured-output format after each route resolves.
  • agent/models_dev.py - expose JSON-schema capability selection and support supports_structured_output model overrides.
  • tests/agent/test_structured_output_rejection_retry.py - cover pre-dispatch DeepSeek format selection.
  • tests/agent/test_models_dev.py - cover the explicit capability override and DeepSeek default.

How to Test

  1. Route title generation to DeepSeek and start a short chat.
  2. Confirm the first request uses response_format.type=json_object and succeeds without the structured-output rejection retry.
  3. Automated (102 passed):
scripts/run_tests.sh tests/agent/test_structured_output_rejection_retry.py tests/agent/test_models_dev.py

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) - or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys - or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows - or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide - or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior - or N/A

Screenshots / Logs

N/A - automated coverage exercises the request payload selection without exposing credentials or session content.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/deepseek DeepSeek API area/config Config system, migrations, profiles labels Sep 17, 2026
@alt-glitch

Copy link
Copy Markdown

This was generated by AI during triage.

Related: #107963 (open) targets the same wasted DeepSeek json_schema 400 with a different mechanism (remember rejected routes per process and start at json_object next time) versus this PR's proactive capability lookup in agent/models_dev.py. Reviewers may want to compare the two; issues #113064, #102849, #84976, #88830 all report the symptom.

@fangliquanflq

Copy link
Copy Markdown
Author

Thanks for the cross-reference. I compared the current diffs: #107963 reacts after a schema rejection, retries with json_object, and memoizes the endpoint/model for the process, so a cold route still pays one failed request. This PR selects the format before dispatch from the resolved model capability, defaults native DeepSeek to json_object, permits an explicit per-model override, and retains the existing rejection fallback for stale capability data. The overlap is real, but the mechanisms and cold-route behavior differ; this note does not identify a correctness gap that requires a code change here.

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks @fangliquanflq. The problem this PR targets landed on main via #113966 (5cc8177), which handles the whole class (primary and fallback auxiliary paths, provider capability up front, per-route memo) in one change. Closing as superseded by the landed fix; the issue is closed with the same reference.

@teknium1 teknium1 closed this Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/deepseek DeepSeek API type/bug Something isn't working

Projects

None yet

3 participants