Skip to content

[Fix] Resolve tool argument types through top-level anyOf/oneOf/allOf - #36626

Merged
Fridge003 merged 3 commits into
sgl-project:mainfrom
JustinTong0323:xinyuan/toplevel-anyof-tool-args
Aug 28, 2026
Merged

Fridge003 merged 3 commits into
sgl-project:mainfrom
JustinTong0323:xinyuan/toplevel-anyof-tool-args

Conversation

@JustinTong0323

@JustinTong0323 JustinTong0323 commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Tool schemas that compose subschemas with a top-level anyOf/oneOf/allOf (legal JSON Schema / OpenAI spec, e.g. discriminated-union arguments) break most SGLang tool-call parsers: nested object/array/scalar arguments are returned to the client as JSON-encoded strings instead of typed values, or (glm45/glm47 streaming) as invalid JSON.

"parameters": {
  "type": "object",
  "oneOf": [
    { "type": "object",
      "properties": { "kind": {"const": "acme"},
                      "payload": {"type": "object", "properties": {"value": {"type": "string"}}} },
      "required": ["kind", "payload"] },
    { "type": "object", "properties": { "kind": {"const": "other"} }, "required": ["kind"] }
  ]
}

With this schema, calling acme(kind="acme", payload={"value":"hello"}) yields payload == "{\"value\": \"hello\"}" (a string) on the qwen3_coder parser in both streaming and non-streaming modes, and on the glm47 parser when streaming.

Root cause

Every affected detector navigated parameters["properties"][arg] only at the top level. With a top-level combinator there is no properties key, so argument-type inference failed and each parser fell back to treating values as strings:

  • qwen3_coder: _get_arguments_config returns the whole schema (with oneOf/type keys) as the param map → every value returned raw, both modes. Scalars are also affected ("count": "7", "verbose": "True").
  • glm45/glm47: streaming decides the value type when <arg_value> opens; with no schema info it defaults to string and JSON-quotes the value. Non-streaming survives only because of a json.loads fallback.

A second, pre-existing bug sits on the same path: glm45/glm47 streaming decides whether to emit the outer closing } via _last_arguments.endswith("}"). When the last argument is object/array-typed, the trailing } belongs to the nested value, so the outer object is never closed and streamed arguments is invalid JSON (masked live only when the model emits trailing whitespace inside </arg_value>).

Fix

  • New shared get_schema_properties() in function_call/utils.py: returns top-level properties, descending into anyOf/oneOf/allOf branches when absent (first branch wins on duplicate keys, matching oneOf preference order).
  • Adopted in every detector with a parse-path property lookup: glm45, glm47, qwen3_coder, step3, minicpm5, hunyuan, spark25, minimax_m2, minimax_m3, dots, poolside_v1, mimo, kimik2 (1–3 lines each).
  • glm45/glm47 streaming finalize: close the outer arguments object whenever a parameter was started, instead of the endswith("}") heuristic.

Two deliberate behavior notes:

  • minicpm5: for top-level-combinator tools the allowed-props filter was previously bypassed (empty property set); it now activates, so undeclared arguments are dropped as they already are for flat schemas.
  • qwen3_coder: a declared-but-empty properties: {} keeps its old semantics (no spurious "param not defined" warnings).

kimik3_structural_tag is intentionally untouched: it builds grammars and already degrades gracefully to AnyTextFormat with top-level combinators.

Test plan

  • New registered tests: TestGetSchemaProperties (4) + TestTopLevelCompositeToolSchema (6: glm45/glm47 streaming with oneOf schema, flat-schema streaming brace cases, qwen3_coder stream/non-stream) in test_function_call_parser.py, and TestMinimaxM3TopLevelOneOf (1: scalar coercion under oneOf) in test_minimax_m3_detector.py. Every detector test was verified red on unpatched main; cases that also pass on unfixed code (glm non-streaming, structure-driven minimax streaming) were deliberately dropped.
  • Full test_function_call_parser.py + test_minimax_m3_detector.py (270 tests): no regressions.
  • GPU A/B on 2×H200, 6 schema shapes (oneOf nested object, anyOf, allOf, array/scalar coercion, flat control, multi-tool mixed) × stream/non-stream, asserting decoded argument types and finish_reason=tool_calls:
model / parser baseline this PR
Qwen/Qwen3-Coder-30B-A3B-Instruct + qwen3_coder 2/12 12/12
zai-org/GLM-4.5-Air-FP8 + glm45 7/12 12/12

CI States

Latest PR Test (Base): ✅ Run #33100877731
Latest PR Test (Extra): ❌ Run #33100877307
Latest PR Test (AMD ROCm 7.2): ❌ Run #33100877334

Tool schemas may legally compose subschemas with a top-level
anyOf/oneOf/allOf instead of declaring properties directly (e.g.
discriminated-union arguments). Detectors navigated
parameters["properties"] only at the top level, so argument types
could not be inferred and nested object/array/scalar values came back
as raw strings (qwen3_coder in both modes) or JSON-quoted strings
(glm45/glm47 streaming).

- add shared get_schema_properties() that descends into
  anyOf/oneOf/allOf branches (first branch wins on duplicate keys)
- adopt it in all detectors with parse-path property lookups:
  glm45, glm47, qwen3_coder, step3, minicpm5, hunyuan, spark25,
  minimax_m2, dots, poolside_v1, mimo, kimik2
- glm45/glm47 streaming: always close the outer arguments object at
  finalize; the endswith("}") shortcut misfires when the last value
  is a nested object, leaving streamed arguments as invalid JSON
@JustinTong0323

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Aug 27, 2026
@JustinTong0323

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

1 similar comment
@JustinTong0323

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

…view follow-ups

- minimax_m3: _get_child_schema navigated top-level properties only;
  with a top-level anyOf/oneOf/allOf the param schema resolved to None
  and non-streaming parsing corrupted nested object args
- qwen3_coder: restore exact semantics for a declared but empty
  properties: {} (avoid spurious 'param not defined' warnings)
- tests: minimax_m3 oneOf detect/stream cases, glm4 flat-schema
  streaming brace case, comment wording
Drop cases that also pass on unfixed code (glm non-streaming has a
json.loads fallback; minimax nested-tag parsing is structure-driven)
and rewrite the minimax_m3 case around scalar coercion, which is the
path that actually breaks without schema resolution
@JustinTong0323

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

@Fridge003
Fridge003 merged commit 0665102 into sgl-project:main Aug 28, 2026
288 of 347 checks passed
Refefer added a commit to Mad-Labs-HQ/sglang that referenced this pull request Sep 26, 2026
Qwen3.8-Flash-Next keeps the XML tool-call envelope but drifts back to its
native JSON payload under long context, emitting

    <function=script><parameter=arguments>{"body": "..."}</parameter>

instead of <parameter=body>. The call reaches the client as
{"arguments": "{\"body\": ...}"} and strict clients reject it with
`missing field body`. Measured at 353 such calls in one agent session.

Unwrap only on unambiguous evidence: an `arguments` parameter the tool does
not declare, holding a JSON object whose keys are ALL declared. The schema is
read through a new _get_declared_properties() rather than
_get_arguments_config(), whose $ref fallback returns the schema object itself
and would treat `type`/`required` as parameter names. A strict JSON decoder
refuses NaN/Infinity so an unwrapped value cannot become unparseable to a
non-Python client. Values pass through untouched -- re-running
_convert_param_value would turn the literal string "null" into None.

Port from madlabs/pennyroyal-v2.5.0 (ed6e799) onto systemone-prod:

* Upstream has no equivalent: nothing in function_call/ unwraps an
  `arguments` envelope, and the only upstream change to this detector since
  the merge-base is sgl-project#36626 (types resolved through anyOf/oneOf/allOf).
* The pennyroyal detector carried third-party wrapper/tool-name validation
  that is not on this branch, so the hunks were re-applied by hand to
  upstream's detector. They land at the same two points: the parameter loop
  in detect_and_parse, and the parameter-close emission in
  parse_streaming_increment. The logic is unchanged.
* One deliberate difference: _get_declared_properties resolves the map with
  upstream's get_schema_properties (sgl-project#36626) instead of reading only the
  top-level `properties`. "Declared" now agrees with the map that types
  parameters, so a tool whose parameters sit in top-level anyOf/oneOf/allOf
  branches is unwrapped when every payload key is declared in some branch,
  and is still left alone when any branch declares `arguments` itself. A
  $ref-only schema still has no declared properties, as before.

The pennyroyal test file also held that third-party wrapper suite, so this
adds a fresh test_qwen3_coder_detector.py with only the envelope tests (60
cases across chunk widths None/1/7/10000) plus 12 new cases for union
schemas. Without the detector change, 24 of the 72 fail -- every case that
expects an unwrap; the rest pin behaviour that must stay stock.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Refefer added a commit to Mad-Labs-HQ/sglang that referenced this pull request Sep 26, 2026
Port of upstream PR sgl-project#21829 (still open). Stock qwen3_coder withholds a
<parameter> value until its closing tag, so a tool call that authors a file
arrives as one delta after seconds of dead air. On a 7,459-char body at 16-char
increments: 3 deltas, max 7,708 chars -> 472 deltas, max 18 chars.

NOT a straight port. Upstream streams UNDECLARED parameters too ("unknown
schema, treat as string"), which here is exactly the stray `arguments`
envelope the previous commit repairs -- and once `"arguments": "` is on the
wire the unwrap can never run, because the streaming path cannot retract what
it has already sent. _should_stream_param therefore requires a parameter the
tool publishes in `properties` AND declares string-like. Non-string types
still buffer, since their JSON encoding is not prefix-stable under
re-serialization.

Port from madlabs/pennyroyal-v2.5.0 (c327975) onto systemone-prod:

* Upstream still has no incremental streaming for this detector: sgl-project#21829 is
  not on this branch, and there is no _should_stream_param or equivalent
  anywhere in function_call/. So there was nothing of upstream's to reconcile
  against; the declared-only semantics above are kept exactly as in
  pennyroyal. If sgl-project#21829 lands upstream later, its "undeclared -> string"
  rule must not be taken: a mutation run giving _should_stream_param that
  rule fails 22 of the tests here, including every streaming envelope
  unwrap.
* The hunks are byte-identical to pennyroyal apart from line wrapping. They
  were re-applied by hand because upstream's detector lacks pennyroyal's
  third-party wrapper validation; they sit at the same places (state in
  __init__, reset on <function=, close-out when a terminator arrives, and
  the no-terminator branch that starts or continues a stream).
* Upstream's parameter branch is not gated on being inside a tool call the
  way pennyroyal's was. That is harmless here: outside a function
  current_func_name is None, which matches no tool, so
  _should_stream_param says no and the stock buffered path runs.
* "Declared" goes through _get_declared_properties, which since the
  previous commit resolves top-level anyOf/oneOf/allOf branches via
  upstream's get_schema_properties (sgl-project#36626). A string parameter declared
  only in a union branch therefore streams; the buffered path types it
  from the same map, so both paths still agree on the value.

Tests: pennyroyal's 86 streaming cases (chunk widths 1/3/13/500/10000,
byte-at-a-time prefix invariance, values containing `<`, a literal
</parameter>, embedded XML, unicode, empty and newline-only payloads,
non-string params keeping their type, and the envelope-under-streaming
composition), plus a union-schema streaming case and 10 cases of an explicit
"undeclared parameter arrives in one piece" test. Without the streaming
change only the two "does it actually stream" cases fail; the rest are
invariants the buffered path satisfies trivially and exist to hold the
streaming path to it. The upstream TestQwen3CoderDetector and
TestTopLevelCompositeToolSchema suites, and the Responses-API streaming
cases that drive qwen3_coder, still pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
efschu pushed a commit to efschu/htsglang that referenced this pull request Sep 26, 2026
…cks, chronologisch)

Grundlage: Präsenz-Scan aller 230 27B-Commits seit 76f8deb gegen diesen Baum
(Stichprobe der hinzugefügten Zeilen je Commit); die 94 fehlenden minus die bewusst
anders gewählten Formen (76e87ac/4ae11ababd -> S3 form.calibration_identity;
479f6ec/d7f588e017/d0fba8955f/34892e3017 -> S2 NF-Formen; 7f81f09/3c14481318 ->
S4/S7a; 3dbb790 line_gate_27b (Werkzeug, Schritt 9); 8604d13 W100-by-name
(Nutzer: bleibt aus); 6545e2c flashinfer-Pin in pyproject (Image-Frage, nicht Baum)).
Liste: 92bbccb eb5d044 829ebd0 431fcbc ef4d11f 6816062 a233e50 2cc593c f0c8451 87cc4fb c529777 31f2dbe c50085a 3301a96 036b368 e1d1fe9 03c68af 6dddc2e 06932b5 87389c4 58a7490 f85ac55 fc64aa5 5aa24dd 97c0e9a 159333c d9f1532 f3c685b 8550655 e50fb59 db2c2ef f09dc0d c255e10 51b810e 28a55a2 34965fc ff3d9cc 340a018 bee5e10 67b6352 fdade85 ed6630d f1c9a43 b434831 517f26d 0b6b60b a40837f 644de86 aff2b7c 197b701 856024b 238512a 9738626 b857a22 1f8c24d d294b3e 810239d b429dfd e714c95 9efd974 3d63e0a d3cfcf3 fee6134 7985b56 49a14e9 fc45706 19c720e 5306bee 6f1235a c98eaa3 93bc802 328349e f9fb3a2 572af73 94fa8b4 d342caa 2fd7d7e 3ebbb96 871d55f 78c2f16 3babf51 196f6a8 e70af54 22eccfc

Inhalt: Upstream-Ports (sgl-project#33758 sgl-project#37818 sgl-project#36738 sgl-project#33459/sgl-project#30096 sgl-project#34446 sgl-project#36267 sgl-project#33778 sgl-project#34859
sgl-project#36415 sgl-project#35255/sgl-project#36638 sgl-project#39858/sgl-project#40259 sgl-project#31417 sgl-project#34892 sgl-project#32225 sgl-project#30832/sgl-project#36626 sgl-project#39574 sgl-project#29579
sgl-project#31468 sgl-project#32575 sgl-project#31648); xsn409/410-Wake-Verdikte; Vision-Linie V1-V3b + xsn438
(SGLANG_WEG2_VISION_FLIP_URGENT); D-Planer L6 (159333c); DFLASH-Window-Pool
sync-frei, PLAN_SYNC_FREE, D-Kollektive (vocab-argmax, a2a-Merge, deferred rebuild),
#DGAP/D_DEFER_SEQ_LENS_CPU; Mamba-Anker Raster 4096 + Per-Path-Cap + Inner-Release;
P-TRIM (--p-trim-end-anchor); FP8 uniform Marlin; ModelOpt/NVFP4 RadixArk; GGUF G1-G6
+ F1/F2; native-mixed sgl-project#38 (sm_8x W4A8, sm_12x CUTLASS/W4A16); RC1-Capture-Set; sgl-project#49
Agent-Turns; dynchunk (--p-chunk-policy, --p-chunk-dynamic-min-tokens).

Auflösungen (Gabel -> Form, Grund):
- L6 d_operating_point_rows: 27B (d) "Token-Vektor auf jeder Position aus der Kapazität"
  nur bei TP-symmetrischem D (Profil d_layout paged_dcp); sonst NF-sgl-project#1293-Pin + NF-Anker-
  Klausel. mamba_ssm_dtype aus EARLY_READ_FACTS nur bei Profil early_read_flags.
  Overhead-Kalibrierung liest mit form.CalibrationIdentity statt LineIdentity.
- RC1 Capture-Set: neuer RecordKey-Term d_capture_set (qwen27b), Leser
  CalibrationIdentity.d_max_running_requests; nextflash unverändert.
- URC Carrier-Hold: 27B _weg2_carrier_hold entfällt (S2 NF-Rotation), Inner-Release und
  Per-Path-Cap bleiben (Env, Default aus; 27b.env setzt sie).
- scheduler_pp_mixin/overlap_utils/batch_result_processor: NF H49/H58 und 27B #PGAP/#DGAP
  komponiert (beide Instrumente getrennt schaltbar).
- schedule_policy: P-TRIM-Kurzschluss vor NF H63-Fold/QSA-Korn; sgl-project#36415 Hoist + NF
  computed_input_len.
- gdn_backend sgl-project#33778: 27B-strided-Verify; NF-Ring flacht beide Layouts ab.
- flashinfer_backend: RC9-Datei + NF-Form-A-Waiver (27B-intern mehrfach gegabelt).
- checkpoint_census: GGUF-Leser + NF exclude_segments (PLE) in einer Aggregation.
- xchg_manifest: FLAT_SEGMENTS in beiden (dst/src) NF-Breitenbedingungen ausgenommen.
- vram_peak_window: NF-Kumulativ-Peak liest über den 27B-Fast-Read.
- FP8: 8c86eb8-Rest nachgezogen (private Workspace-Registry entfernt, wie RC9).
- argv_d: vision= an allen drei Aufrufstellen (inkl. NF --d-only).
- census_checkpoint_decision (W161) jetzt für beide Profile aktiv.

Gates: py_compile aller geänderten Dateien; ruff F821/F811 ohne neue Funde gegenüber
dem Vorgänger (PendingSeqLensCpu ist String-Annotation wie in RC9); dup_defs_gate 0 neu.
Tests: 73 portierte Testdateien, Lauf nach dem Ruhefenster (Boot aktiv).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants