[Fix] Resolve tool argument types through top-level anyOf/oneOf/allOf - #36626
Merged
Fridge003 merged 3 commits intoAug 28, 2026
Merged
Conversation
Tool schemas may legally compose subschemas with a top-level
anyOf/oneOf/allOf instead of declaring properties directly (e.g.
discriminated-union arguments). Detectors navigated
parameters["properties"] only at the top level, so argument types
could not be inferred and nested object/array/scalar values came back
as raw strings (qwen3_coder in both modes) or JSON-quoted strings
(glm45/glm47 streaming).
- add shared get_schema_properties() that descends into
anyOf/oneOf/allOf branches (first branch wins on duplicate keys)
- adopt it in all detectors with parse-path property lookups:
glm45, glm47, qwen3_coder, step3, minicpm5, hunyuan, spark25,
minimax_m2, dots, poolside_v1, mimo, kimik2
- glm45/glm47 streaming: always close the outer arguments object at
finalize; the endswith("}") shortcut misfires when the last value
is a nested object, leaving streamed arguments as invalid JSON
Collaborator
Author
|
/tag-and-rerun-ci |
Collaborator
Author
|
/rerun-failed-ci |
1 similar comment
Collaborator
Author
|
/rerun-failed-ci |
…view follow-ups
- minimax_m3: _get_child_schema navigated top-level properties only;
with a top-level anyOf/oneOf/allOf the param schema resolved to None
and non-streaming parsing corrupted nested object args
- qwen3_coder: restore exact semantics for a declared but empty
properties: {} (avoid spurious 'param not defined' warnings)
- tests: minimax_m3 oneOf detect/stream cases, glm4 flat-schema
streaming brace case, comment wording
Drop cases that also pass on unfixed code (glm non-streaming has a json.loads fallback; minimax nested-tag parsing is structure-driven) and rewrite the minimax_m3 case around scalar coercion, which is the path that actually breaks without schema resolution
Collaborator
Author
|
/rerun-failed-ci |
Fridge003
approved these changes
Aug 28, 2026
4 of 5 tasks
5 tasks done
3 of 5 tasks
3 of 4 tasks
5 tasks done
Refefer
added a commit
to Mad-Labs-HQ/sglang
that referenced
this pull request
Sep 26, 2026
Qwen3.8-Flash-Next keeps the XML tool-call envelope but drifts back to its
native JSON payload under long context, emitting
<function=script><parameter=arguments>{"body": "..."}</parameter>
instead of <parameter=body>. The call reaches the client as
{"arguments": "{\"body\": ...}"} and strict clients reject it with
`missing field body`. Measured at 353 such calls in one agent session.
Unwrap only on unambiguous evidence: an `arguments` parameter the tool does
not declare, holding a JSON object whose keys are ALL declared. The schema is
read through a new _get_declared_properties() rather than
_get_arguments_config(), whose $ref fallback returns the schema object itself
and would treat `type`/`required` as parameter names. A strict JSON decoder
refuses NaN/Infinity so an unwrapped value cannot become unparseable to a
non-Python client. Values pass through untouched -- re-running
_convert_param_value would turn the literal string "null" into None.
Port from madlabs/pennyroyal-v2.5.0 (ed6e799) onto systemone-prod:
* Upstream has no equivalent: nothing in function_call/ unwraps an
`arguments` envelope, and the only upstream change to this detector since
the merge-base is sgl-project#36626 (types resolved through anyOf/oneOf/allOf).
* The pennyroyal detector carried third-party wrapper/tool-name validation
that is not on this branch, so the hunks were re-applied by hand to
upstream's detector. They land at the same two points: the parameter loop
in detect_and_parse, and the parameter-close emission in
parse_streaming_increment. The logic is unchanged.
* One deliberate difference: _get_declared_properties resolves the map with
upstream's get_schema_properties (sgl-project#36626) instead of reading only the
top-level `properties`. "Declared" now agrees with the map that types
parameters, so a tool whose parameters sit in top-level anyOf/oneOf/allOf
branches is unwrapped when every payload key is declared in some branch,
and is still left alone when any branch declares `arguments` itself. A
$ref-only schema still has no declared properties, as before.
The pennyroyal test file also held that third-party wrapper suite, so this
adds a fresh test_qwen3_coder_detector.py with only the envelope tests (60
cases across chunk widths None/1/7/10000) plus 12 new cases for union
schemas. Without the detector change, 24 of the 72 fail -- every case that
expects an unwrap; the rest pin behaviour that must stay stock.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Refefer
added a commit
to Mad-Labs-HQ/sglang
that referenced
this pull request
Sep 26, 2026
Port of upstream PR sgl-project#21829 (still open). Stock qwen3_coder withholds a <parameter> value until its closing tag, so a tool call that authors a file arrives as one delta after seconds of dead air. On a 7,459-char body at 16-char increments: 3 deltas, max 7,708 chars -> 472 deltas, max 18 chars. NOT a straight port. Upstream streams UNDECLARED parameters too ("unknown schema, treat as string"), which here is exactly the stray `arguments` envelope the previous commit repairs -- and once `"arguments": "` is on the wire the unwrap can never run, because the streaming path cannot retract what it has already sent. _should_stream_param therefore requires a parameter the tool publishes in `properties` AND declares string-like. Non-string types still buffer, since their JSON encoding is not prefix-stable under re-serialization. Port from madlabs/pennyroyal-v2.5.0 (c327975) onto systemone-prod: * Upstream still has no incremental streaming for this detector: sgl-project#21829 is not on this branch, and there is no _should_stream_param or equivalent anywhere in function_call/. So there was nothing of upstream's to reconcile against; the declared-only semantics above are kept exactly as in pennyroyal. If sgl-project#21829 lands upstream later, its "undeclared -> string" rule must not be taken: a mutation run giving _should_stream_param that rule fails 22 of the tests here, including every streaming envelope unwrap. * The hunks are byte-identical to pennyroyal apart from line wrapping. They were re-applied by hand because upstream's detector lacks pennyroyal's third-party wrapper validation; they sit at the same places (state in __init__, reset on <function=, close-out when a terminator arrives, and the no-terminator branch that starts or continues a stream). * Upstream's parameter branch is not gated on being inside a tool call the way pennyroyal's was. That is harmless here: outside a function current_func_name is None, which matches no tool, so _should_stream_param says no and the stock buffered path runs. * "Declared" goes through _get_declared_properties, which since the previous commit resolves top-level anyOf/oneOf/allOf branches via upstream's get_schema_properties (sgl-project#36626). A string parameter declared only in a union branch therefore streams; the buffered path types it from the same map, so both paths still agree on the value. Tests: pennyroyal's 86 streaming cases (chunk widths 1/3/13/500/10000, byte-at-a-time prefix invariance, values containing `<`, a literal </parameter>, embedded XML, unicode, empty and newline-only payloads, non-string params keeping their type, and the envelope-under-streaming composition), plus a union-schema streaming case and 10 cases of an explicit "undeclared parameter arrives in one piece" test. Without the streaming change only the two "does it actually stream" cases fail; the rest are invariants the buffered path satisfies trivially and exist to hold the streaming path to it. The upstream TestQwen3CoderDetector and TestTopLevelCompositeToolSchema suites, and the Responses-API streaming cases that drive qwen3_coder, still pass. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
efschu
pushed a commit
to efschu/htsglang
that referenced
this pull request
Sep 26, 2026
…cks, chronologisch) Grundlage: Präsenz-Scan aller 230 27B-Commits seit 76f8deb gegen diesen Baum (Stichprobe der hinzugefügten Zeilen je Commit); die 94 fehlenden minus die bewusst anders gewählten Formen (76e87ac/4ae11ababd -> S3 form.calibration_identity; 479f6ec/d7f588e017/d0fba8955f/34892e3017 -> S2 NF-Formen; 7f81f09/3c14481318 -> S4/S7a; 3dbb790 line_gate_27b (Werkzeug, Schritt 9); 8604d13 W100-by-name (Nutzer: bleibt aus); 6545e2c flashinfer-Pin in pyproject (Image-Frage, nicht Baum)). Liste: 92bbccb eb5d044 829ebd0 431fcbc ef4d11f 6816062 a233e50 2cc593c f0c8451 87cc4fb c529777 31f2dbe c50085a 3301a96 036b368 e1d1fe9 03c68af 6dddc2e 06932b5 87389c4 58a7490 f85ac55 fc64aa5 5aa24dd 97c0e9a 159333c d9f1532 f3c685b 8550655 e50fb59 db2c2ef f09dc0d c255e10 51b810e 28a55a2 34965fc ff3d9cc 340a018 bee5e10 67b6352 fdade85 ed6630d f1c9a43 b434831 517f26d 0b6b60b a40837f 644de86 aff2b7c 197b701 856024b 238512a 9738626 b857a22 1f8c24d d294b3e 810239d b429dfd e714c95 9efd974 3d63e0a d3cfcf3 fee6134 7985b56 49a14e9 fc45706 19c720e 5306bee 6f1235a c98eaa3 93bc802 328349e f9fb3a2 572af73 94fa8b4 d342caa 2fd7d7e 3ebbb96 871d55f 78c2f16 3babf51 196f6a8 e70af54 22eccfc Inhalt: Upstream-Ports (sgl-project#33758 sgl-project#37818 sgl-project#36738 sgl-project#33459/sgl-project#30096 sgl-project#34446 sgl-project#36267 sgl-project#33778 sgl-project#34859 sgl-project#36415 sgl-project#35255/sgl-project#36638 sgl-project#39858/sgl-project#40259 sgl-project#31417 sgl-project#34892 sgl-project#32225 sgl-project#30832/sgl-project#36626 sgl-project#39574 sgl-project#29579 sgl-project#31468 sgl-project#32575 sgl-project#31648); xsn409/410-Wake-Verdikte; Vision-Linie V1-V3b + xsn438 (SGLANG_WEG2_VISION_FLIP_URGENT); D-Planer L6 (159333c); DFLASH-Window-Pool sync-frei, PLAN_SYNC_FREE, D-Kollektive (vocab-argmax, a2a-Merge, deferred rebuild), #DGAP/D_DEFER_SEQ_LENS_CPU; Mamba-Anker Raster 4096 + Per-Path-Cap + Inner-Release; P-TRIM (--p-trim-end-anchor); FP8 uniform Marlin; ModelOpt/NVFP4 RadixArk; GGUF G1-G6 + F1/F2; native-mixed sgl-project#38 (sm_8x W4A8, sm_12x CUTLASS/W4A16); RC1-Capture-Set; sgl-project#49 Agent-Turns; dynchunk (--p-chunk-policy, --p-chunk-dynamic-min-tokens). Auflösungen (Gabel -> Form, Grund): - L6 d_operating_point_rows: 27B (d) "Token-Vektor auf jeder Position aus der Kapazität" nur bei TP-symmetrischem D (Profil d_layout paged_dcp); sonst NF-sgl-project#1293-Pin + NF-Anker- Klausel. mamba_ssm_dtype aus EARLY_READ_FACTS nur bei Profil early_read_flags. Overhead-Kalibrierung liest mit form.CalibrationIdentity statt LineIdentity. - RC1 Capture-Set: neuer RecordKey-Term d_capture_set (qwen27b), Leser CalibrationIdentity.d_max_running_requests; nextflash unverändert. - URC Carrier-Hold: 27B _weg2_carrier_hold entfällt (S2 NF-Rotation), Inner-Release und Per-Path-Cap bleiben (Env, Default aus; 27b.env setzt sie). - scheduler_pp_mixin/overlap_utils/batch_result_processor: NF H49/H58 und 27B #PGAP/#DGAP komponiert (beide Instrumente getrennt schaltbar). - schedule_policy: P-TRIM-Kurzschluss vor NF H63-Fold/QSA-Korn; sgl-project#36415 Hoist + NF computed_input_len. - gdn_backend sgl-project#33778: 27B-strided-Verify; NF-Ring flacht beide Layouts ab. - flashinfer_backend: RC9-Datei + NF-Form-A-Waiver (27B-intern mehrfach gegabelt). - checkpoint_census: GGUF-Leser + NF exclude_segments (PLE) in einer Aggregation. - xchg_manifest: FLAT_SEGMENTS in beiden (dst/src) NF-Breitenbedingungen ausgenommen. - vram_peak_window: NF-Kumulativ-Peak liest über den 27B-Fast-Read. - FP8: 8c86eb8-Rest nachgezogen (private Workspace-Registry entfernt, wie RC9). - argv_d: vision= an allen drei Aufrufstellen (inkl. NF --d-only). - census_checkpoint_decision (W161) jetzt für beide Profile aktiv. Gates: py_compile aller geänderten Dateien; ruff F821/F811 ohne neue Funde gegenüber dem Vorgänger (PendingSeqLensCpu ist String-Annotation wie in RC9); dup_defs_gate 0 neu. Tests: 73 portierte Testdateien, Lauf nach dem Ruhefenster (Boot aktiv).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tool schemas that compose subschemas with a top-level
anyOf/oneOf/allOf(legal JSON Schema / OpenAI spec, e.g. discriminated-union arguments) break most SGLang tool-call parsers: nested object/array/scalar arguments are returned to the client as JSON-encoded strings instead of typed values, or (glm45/glm47 streaming) as invalid JSON.With this schema, calling
acme(kind="acme", payload={"value":"hello"})yieldspayload == "{\"value\": \"hello\"}"(a string) on theqwen3_coderparser in both streaming and non-streaming modes, and on theglm47parser when streaming.Root cause
Every affected detector navigated
parameters["properties"][arg]only at the top level. With a top-level combinator there is nopropertieskey, so argument-type inference failed and each parser fell back to treating values as strings:qwen3_coder:_get_arguments_configreturns the whole schema (withoneOf/typekeys) as the param map → every value returned raw, both modes. Scalars are also affected ("count": "7","verbose": "True").glm45/glm47: streaming decides the value type when<arg_value>opens; with no schema info it defaults tostringand JSON-quotes the value. Non-streaming survives only because of ajson.loadsfallback.A second, pre-existing bug sits on the same path: glm45/glm47 streaming decides whether to emit the outer closing
}via_last_arguments.endswith("}"). When the last argument is object/array-typed, the trailing}belongs to the nested value, so the outer object is never closed and streamedargumentsis invalid JSON (masked live only when the model emits trailing whitespace inside</arg_value>).Fix
get_schema_properties()infunction_call/utils.py: returns top-levelproperties, descending intoanyOf/oneOf/allOfbranches when absent (first branch wins on duplicate keys, matching oneOf preference order).glm45,glm47,qwen3_coder,step3,minicpm5,hunyuan,spark25,minimax_m2,minimax_m3,dots,poolside_v1,mimo,kimik2(1–3 lines each).endswith("}")heuristic.Two deliberate behavior notes:
minicpm5: for top-level-combinator tools the allowed-props filter was previously bypassed (empty property set); it now activates, so undeclared arguments are dropped as they already are for flat schemas.qwen3_coder: a declared-but-emptyproperties: {}keeps its old semantics (no spurious "param not defined" warnings).kimik3_structural_tagis intentionally untouched: it builds grammars and already degrades gracefully toAnyTextFormatwith top-level combinators.Test plan
TestGetSchemaProperties(4) +TestTopLevelCompositeToolSchema(6: glm45/glm47 streaming with oneOf schema, flat-schema streaming brace cases, qwen3_coder stream/non-stream) intest_function_call_parser.py, andTestMinimaxM3TopLevelOneOf(1: scalar coercion under oneOf) intest_minimax_m3_detector.py. Every detector test was verified red on unpatched main; cases that also pass on unfixed code (glm non-streaming, structure-driven minimax streaming) were deliberately dropped.test_function_call_parser.py+test_minimax_m3_detector.py(270 tests): no regressions.finish_reason=tool_calls:qwen3_coderglm45CI States
Latest PR Test (Base): ✅ Run #33100877731
Latest PR Test (Extra): ❌ Run #33100877307
Latest PR Test (AMD ROCm 7.2): ❌ Run #33100877334