[gpt-oss] Add model_identity to system message retrieval for harmony chat template

[lyuwen]

Purpose

When using GPT-OSS, model_identity passed in from chat_template_kwargs is not being used to adjust model's identity. This PR is to pass the model_identity from chat/completion to the system message formatter.

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[gemini-code-assist]

Code Review

This pull request aims to pass the model_identity from chat_template_kwargs to the system message formatter for the harmony chat template. However, the current implementation has a critical bug where it does not handle the case where request.chat_template_kwargs is None, which will lead to an AttributeError and crash the server. I've provided a suggestion to fix this issue.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[yeqcharlotte]

@lyuwen could you share some examples in your test plans. what's the rendered messages before and after the change?

[lyuwen]

For example, if I run this before the patch:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
MODEL_NAME = "gpt-oss-120b"
response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "What are you?"}],
    extra_body={
        "chat_template_kwargs": {"model_identity": "You are DeepSeek developed by MoonShot.", "reasoning_effort": "high"},
    }
)
print("Reasoning:")
print(response.choices[0].message.reasoning_content)
print("Final:")
print(response.choices[0].message.content)

The output would be :

Reasoning:
The user asks "What are you?" Probably just a simple question. We need to respond as ChatGPT, giving a brief description. Should be concise, friendly.
Final:
I’m ChatGPT, an AI language model created by OpenAI. I can understand and generate human‑like text, answer questions, help with writing, brainstorm ideas, explain concepts, and more. Think of me as a virtual assistant you can chat with about a wide range of topics.

But the expected behavior is the model assuming the provided identity and respond with something like the following:

Reasoning:
The user asks "What are you?". The system messages: the user wants a brief answer. Should comply. The assistant is DeepSeek developed by MoonShot. So respond with explanation. Simple.
Final:
I’m an AI language model called **DeepSeek**, created by MoonShot. I’m designed to understand and generate text, answer questions, help with tasks, and hold conversations across a wide range of topics. Think of me as a virtual assistant that can provide information, brainstorm ideas, explain concepts, and much more—all through natural language interaction.

Upon investigation, below are the reason this parameter is not taking effect - the model_identity passed in from api was not passed to the system message builder.

vllm/vllm/entrypoints/openai/serving_chat.py

Lines 1763 to 1768 in 73a484c

	sys_msg = get_system_message(
	reasoning_effort=request.reasoning_effort,
	browser_description=None,
	python_description=None,
	with_custom_tools=request.tools is not None,
	)

And system message just used the default model identity:
https://github.com/openai/harmony/blob/ec7606df9e87e3d0a1fec9f50928c1e407f0c438/python/openai_harmony/__init__.py#L181-L184

[lyuwen]

The model_identity parameter itself is already in the vLLM code base, it just wasn't used.

vllm/vllm/entrypoints/openai/parser/harmony_utils.py

Lines 87 to 96 in 7cab92f

	def get_system_message(
	model_identity: str | None = None,
	reasoning_effort: Literal["high", "medium", "low"] | None = None,
	start_date: str | None = None,
	browser_description: str | None = None,
	python_description: str | None = None,
	container_description: str | None = None,
	instructions: str | None = None,
	with_custom_tools: bool = False,
	) -> Message:

So the prompt with chat template should look like

<|start|>system<|message|>You are DeepSeek developed by MoonShot.
Knowledge cutoff: 2024-06
Current date: 2025-12-09

Reasoning: medium

# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>user<|message|>What are you?<|end|><|start|>assistant

But without model_identity passed in, it used the default:

<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.
Knowledge cutoff: 2024-06
Current date: 2025-12-09

Reasoning: medium

# Valid channels: analysis, commentary, final. Channel must be included for every message.<|end|><|start|>user<|message|>What are you?<|end|><|start|>assistant

[chaunceyjiang]

/cc @yeqcharlotte PTAL.

[chaunceyjiang]

Suggested change

	model_identity=(request.chat_template_kwargs or {}).get("model_identity"),
	model_identity=(request.chat_template_kwargs or {}).get("harmony_model_identity"),

WDYT? @yeqcharlotte @lyuwen

[lyuwen]

"model_identity" seems more straightforward since it aligns with the GPT-OSS's original chat template. Using "harmony_model_identity" would need a bit more explanation in the vllm docs.
Pasted from the chat template:

{#-
--
In addition to the normal inputs of `messages` and `tools`, this template also accepts the
following kwargs:
- "builtin_tools": A list, can contain "browser" and/or "python".
- "model_identity": A string that optionally describes the model identity.
- "reasoning_effort": A string that describes the reasoning effort, defaults to "medium".
#}

Maybe the additional ones suggested in #30873 could use a prefix since it could be harmony specific?
Anyway, either works for me.

[seunggil1]

It appears that apart from reasoning_effort, the remaining fields are specific to the Harmony format.

I think it might be beneficial to add a harmony_ prefix to these keys (e.g., harmony_model_identity). My concern is that even users familiar with the Harmony format might easily overlook this feature unless they carefully read the vLLM documentation.

[seunggil1]

I realized I had opened a duplicate PR (#30873) addressing the same issue, as I missed this one initially. I'll be closing mine in favor of this one.

However, I'd like to suggest an improvement based on my implementation. Instead of supporting only model_identity, it would be better to read other arguments supported by get_system_message from chat_template_kwargs as well. This provides greater flexibility for future use cases without needing additional code changes.

The test plan remains the same as described in this PR.

Suggested change

	chat_template_kwargs = request.chat_template_kwargs or {}
	sys_msg = get_system_message(
	model_identity=chat_template_kwargs.get("model_identity"),
	reasoning_effort=request.reasoning_effort,
	start_date=chat_template_kwargs.get("start_date"),
	browser_description=chat_template_kwargs.get("browser_description"),
	python_description=chat_template_kwargs.get("python_description"),
	container_description=chat_template_kwargs.get("container_description"),
	instructions=chat_template_kwargs.get("instructions"),
	with_custom_tools=should_include_tools,
	)