Skip to content

Add Harmony format parsers for GPT-OSS models#50

Merged
waybarrios merged 1 commit intomainfrom
feature/harmony-parsers
Feb 10, 2026
Merged

Add Harmony format parsers for GPT-OSS models#50
waybarrios merged 1 commit intomainfrom
feature/harmony-parsers

Conversation

@waybarrios
Copy link
Copy Markdown
Owner

Implements support for GPT-OSS models that use the Harmony response format (closes #38).

Harmony is a multi-channel format with control tokens that separates reasoning, tool calls, and user-facing responses into distinct channels:

  • analysis channel for internal reasoning
  • commentary channel for tool calls (addressed with to=functions.{name})
  • final channel for the user response

Example of a tool call in Harmony format:

<|channel|>commentary to=functions.get_weather
<|constrain|>json
<|message|>{"location": "San Francisco", "unit": "celsius"}
<|call|>

What was added:

  • HarmonyToolParser (registered as "harmony" and "gpt-oss") that extracts tool calls from commentary channel blocks and final content from the final channel. Supports both non-streaming and streaming modes.

  • HarmonyReasoningParser (registered as "harmony") that extracts reasoning from analysis channel blocks. Multiple analysis blocks are concatenated. Streaming tracks channel transitions and emits reasoning vs content deltas accordingly.

  • convert_tools_to_typescript() utility that converts OpenAI JSON Schema tool definitions to the TypeScript namespace format expected by Harmony models:

namespace functions {
  // Get weather for a location
  type get_weather = (_: {
    location: string,
    unit?: "celsius" | "fahrenheit"
  }) => any;
}
  • 41 tests covering tool parsing, reasoning parsing, TypeScript conversion, edge cases (unicode, malformed JSON, incomplete calls, mixed channels), and streaming behavior.

  • CI updated to include the new tests in the test-matrix job.

New files:

  • vllm_mlx/api/harmony_tools.py
  • vllm_mlx/tool_parsers/harmony_tool_parser.py
  • vllm_mlx/reasoning/harmony_parser.py
  • tests/test_harmony_parsers.py

Modified files:

  • vllm_mlx/tool_parsers/init.py
  • vllm_mlx/reasoning/init.py
  • .github/workflows/ci.yml

Usage:

vllm-mlx serve <model> --enable-auto-tool-choice --tool-call-parser harmony --reasoning-parser harmony

Test plan:

  • All 41 new tests pass locally
  • All 120 existing parser tests still pass (no regressions)
  • ruff and black checks pass
  • Imports verified

@waybarrios waybarrios force-pushed the feature/harmony-parsers branch from 59ca7d2 to 49bbcd1 Compare February 9, 2026 04:18
@waybarrios waybarrios self-assigned this Feb 9, 2026
@waybarrios waybarrios added the enhancement New feature or request label Feb 9, 2026
@waybarrios waybarrios merged commit 3cea624 into main Feb 10, 2026
7 checks passed
@TomLucidor
Copy link
Copy Markdown

Update with the latest code: vllm-mlx serve: error: argument --tool-call-parser: invalid choice: 'harmony' (choose from auto, mistral, qwen, llama, hermes, deepseek, kimi, granite, nemotron, xlam, functionary, glm47)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GPT-OSS Support/Guide

2 participants