Skip to content

Add DeepSeek-V4-Flash-GGUF to Studio with none/high/max reasoning - #6908

Merged
danielhanchen merged 4 commits into
mainfrom
add-deepseek-v4-flash-studio
Jul 7, 2026
Merged

danielhanchen merged 4 commits into
mainfrom
add-deepseek-v4-flash-studio

Conversation

@danielhanchen

Copy link
Copy Markdown
Member

Summary

Adds unsloth/DeepSeek-V4-Flash-GGUF as a default selectable model in Studio, wired with the model's recommended decoding defaults and its three tier reasoning control (none / high / max).

Changes

  • studio/backend/core/inference/defaults.py: add unsloth/DeepSeek-V4-Flash-GGUF to DEFAULT_MODELS_GGUF and DEFAULT_MODELS_STANDARD so it appears in the model picker.
  • studio/backend/assets/configs/inference_defaults.json: add a deepseek-v4 family with temperature 1.0, top_p 1.0, top_k -1, min_p 0.0, repetition_penalty 1.0, matching the official generation_config.json (do_sample: true, temperature: 1.0, top_p: 1.0). Register the deepseek-v4 match pattern ahead of the more general deepseek patterns.
  • studio/backend/core/inference/llama_cpp.py: in detect_reasoning_flags, surface high alongside max for deepseek-v4 / deepseek4 model ids. The official encoder accepts reasoning_effort in {high, max} (plus off via enable_thinking), but the shipped template only branches on max, so a literal scan alone yields only max. This exposes the full none/high/max ladder. Scoped strictly to deepseek-v4, so other hybrid templates are unaffected.
  • studio/backend/tests/test_safetensors_capability_advertise.py: two regression tests (deepseek-v4 advertises ['high', 'max']; a non deepseek-v4 hybrid template stays ['max']).

Reasoning wiring

DeepSeek-V4 exposes reasoning through chat_template_kwargs, matching the vLLM and SGLang recipes:

Studio selection chat_template_kwargs Rendered
None {enable_thinking: false} no reasoning, no preamble
High {enable_thinking: true, reasoning_effort: "high"} reasoning on, no preamble
Max {enable_thinking: true, reasoning_effort: "max"} reasoning on plus the maximum effort preamble

This reuses the existing enable_thinking_effort reasoning style end to end (detect_reasoning_flags -> reasoning_effort_levels -> _request_reasoning_kwargs), so no frontend changes are needed.

Testing

pytest studio/backend/tests/test_safetensors_capability_advertise.py passes 19/19, including the two new tests.

Adds unsloth/DeepSeek-V4-Flash-GGUF as a default selectable model with the
recommended decoding defaults (temperature 1.0, top_p 1.0 from the official
generation_config.json) and its three tier reasoning control. The high/max
ladder is surfaced for deepseek-v4 model ids and flows through the existing
enable_thinking_effort reasoning style via chat_template_kwargs, so no
frontend changes are needed.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for the DeepSeek-V4 model, including default inference configurations, adding DeepSeek-V4-Flash-GGUF to the default model lists, and injecting the 'high' reasoning effort level during capability detection. The feedback suggests improving the model family detection logic in llama_cpp.py by replacing simple substring checks with segment-based matching to prevent potential false positives with future model names (e.g., 'deepseek-v40').

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +614 to +618
normalized_id = (model_identifier or "").lower()
if (
"deepseek-v4" in normalized_id or "deepseek4" in normalized_id
) and "high" not in effort_levels:
_wanted = set(effort_levels) | {"high"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Avoid using simple raw substring checks like "deepseek-v4" in normalized_id to detect model families, as this can lead to false positives in the future (for example, if a model named deepseek-v40 or deepseek40 is released). Instead, split the repository name into segments using delimiters like dashes, underscores, and dots, and verify if the tag appears as a whole segment.

        normalized_id = (model_identifier or "").lower()
        repo_name = normalized_id.split("/")[-1]
        segments = re.split(r"[-_.]", repo_name)
        is_deepseek_v4 = "deepseek4" in segments or any(
            segments[i] == "deepseek" and segments[i + 1] == "v4"
            for i in range(len(segments) - 1)
        )
        if is_deepseek_v4 and "high" not in effort_levels:
References
  1. When statically detecting model types or tags from repository IDs or names, avoid simple raw substring checks to prevent false positives (e.g., matching 'deepseek-v40' or 'deepseek40'). Instead, split the repository name into segments using delimiters like dashes, underscores, and dots, and verify if the tag appears as a whole segment.

danielhanchen and others added 2 commits July 7, 2026 13:08
…rt, render tests

Match deepseek-v4 on whole repo-name segments so a future deepseek-v40 or
deepseek40 cannot false-match the synthetic 'high'. In _request_reasoning_kwargs,
emit enable_thinking when a named effort level is sent without it, so the
newly exposed High mode renders thinking-on over the API (the UI already sent
it explicitly). Add a none/high/max render-path test file (jinja behind
importorskip) with a lone-high regression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants