feat: add configurable thinking output format support for vLLM

[AyRickk]

Related to #5992

---

Summary by cubic

Adds configurable thinking output format support for vLLM by parsing custom tags in streamed responses, while preserving vLLM’s native reasoning_content during streaming. Models using tags like … or … now emit proper “thinking” and “assistant” chunks.

• New Features
  • Add VllmOptions: thinkingOpenTag and thinkingCloseTag to configure custom thinking tags (both required).
  • Implement ThinkingTagExtractor to split streamed content into thinking vs. regular content, with partial-tag handling and flush on stream end.
  • Override vLLM streaming to emit “thinking” chunks first and pass through assistant chunks; use direct SSE to preserve reasoning_content. Behavior is unchanged when options are not set.
  • Add unit and integration tests; update vLLM provider and reference docs with configuration examples.

^{Written for commit d99b93c. Summary will update automatically on new commits.}

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

1 out of 2 committers have signed the CLA.
✅ (AyRickk)[https://github.com/AyRickk]
❌ @continue Agent
Continue Agent seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[AyRickk]

I have read the CLA Document and I hereby sign the CLA

[AyRickk]

recheck

[continue]

Added documentation for the new thinking output format configuration feature:

Changes made:

1. vLLM Provider Documentation ()

   • Added new "Thinking output format" section explaining the two supported formats (standard reasoning_content field and custom tags)
   • Included configuration examples for both YAML and JSON formats showing how to use thinkingOpenTag and thinkingCloseTag
   • Added link to vLLM's reasoning outputs documentation

2. YAML Reference ()

   • Documented thinkingOpenTag and thinkingCloseTag properties in the models section
   • Added example configuration showing vLLM with custom thinking tags
   • Included cross-reference to vLLM provider docs

3. JSON Reference ()

   • Documented thinkingOpenTag and thinkingCloseTag properties for config.json
   • Added cross-reference to vLLM provider docs for examples

The documentation maintains the existing level of detail and focuses on practical usage of the new feature.

[continue]

Added integration tests in core/llm/thinkingTagIntegration.vitest.ts to verify ThinkingTagExtractor works correctly with BaseLLM's streaming functionality.

The tests cover:

• Extraction of thinking content from single and multiple chunks
• Handling of partial tags at chunk boundaries
• Flush behavior when stream ends
• Multiple thinking blocks in a single stream
• Custom tag format support (e.g., <reasoning>, [THINK])
• Interaction with native thinking role chunks
• Correct behavior when thinking tags are not configured

These integration tests complement the existing unit tests for ThinkingTagExtractor by verifying the end-to-end behavior when the extractor is integrated with the actual streaming pipeline.

[cubic-dev-ai]

1 issue found across 4 files

Prompt for AI agents (all 1 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="core/llm/index.ts">

<violation number="1" location="core/llm/index.ts:1412">
Custom provider streaming now suppresses assistant tool-call messages with empty `content`, so downstream tool executions are lost.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[cubic-dev-ai]

1 issue found across 8 files

Prompt for AI agents (all 1 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="core/llm/index.ts">

<violation number="1" location="core/llm/index.ts:1413">
Tool-call/usage-only assistant chunks are dropped because they are now yielded only when `content` is non-empty, preventing tools from ever executing.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[cubic-dev-ai]

1 issue found across 7 files

Prompt for AI agents (all 1 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="core/llm/index.ts">

<violation number="1" location="core/llm/index.ts:1412">
Tool/function-call chunks are dropped because new guard filters out assistant messages whose content is an empty string, so streaming tool calls no longer reach the caller.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[RomneyDa]

@AyRickk see comments, I think we could merge if

• functionality is siloed to the VLLM class
• use a single thinkingTagName not duplicate tag configs (unless it would be normal for them to be different in VLLM, I'm not sure)
• Update config yaml schema as in comment, and in config-yaml package (or it will not work for yaml configs)
• On first glance I wonder if the tag parsing logic could be significantly more concise, maybe worth another look/approach?
• Follow up on/clean up test/docs commits. This is a new feature and we are tweaking when it runs (i.e. we might not run for community PRs) but I think in this case it made sense

[cubic-dev-ai]

1 issue found across 2 files (reviewed changes from recent commits).

Prompt for AI agents (all 1 issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="core/llm/thinkingTagIntegration.vitest.ts">

<violation number="1" location="core/llm/thinkingTagIntegration.vitest.ts:30">
Tests duplicate the production `_streamChat` logic inside `MockVllm`, so none of these cases exercise the real vLLM streaming implementation—regressions in `Vllm._streamChat` would still pass.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

[AyRickk]

@RomneyDa is it ok ?