Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
</div>

## 📣 News and Discussions
- [04/25/2026][**DeepSeek V4 Flash**](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) We now support finetuning `deepseek-ai/DeepSeek-V4-Flash`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml) and [guide](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/guides/llm/dsv4-flash.md).
- [04/22/2026][**Qwen3.6-27B**](https://huggingface.co/Qwen/Qwen3.6-27B) We now support finetuning `Qwen/Qwen3.6-27B`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5/qwen3_6_27b.yaml).
- [04/20/2026][**Qwen-Image**](https://huggingface.co/Qwen/Qwen-Image) We now support finetuning `Qwen/Qwen-Image`, thanks to [@harshareddy832](https://github.com/harshareddy832). Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/diffusion/finetune/qwen_image_t2i_flow.yaml).
- [04/16/2026][**Qwen3.6 MoE**](https://huggingface.co/Qwen/Qwen3.6-35B-A3B) We now support finetuning `Qwen/Qwen3.6-35B-A3B`. Check out our [recipe](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_6_35b.yaml).
Expand Down
19 changes: 1 addition & 18 deletions docs/guides/llm/dsv4-flash.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,27 +48,10 @@ A new in-tree `HuggingFaceStorageReader` recognizes `F8_E8M0` and `F8_E5M2` dtyp

## Launch Training

Two recipes ship under [`examples/llm_finetune/deepseek_v4/`](https://github.com/NVIDIA-NeMo/Automodel/tree/main/examples/llm_finetune/deepseek_v4):

- [`deepseek_v4_flash_validate.yaml`](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_validate.yaml) — single-node 8 × A100-80G infrastructure validation on a 4-layer truncated harness exercising the full attention zoo (`compress_ratios=[0, 0, 4, 128]` → SWA / SWA / CSA / HCA), `num_hash_layers=2`, `pp_size=2`, `ep_size=4`. Use this first to confirm the environment and checkpoint paths before scaling out.
- [`deepseek_v4_flash_hellaswag.yaml`](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml) — HellaSwag finetune recipe. The yaml header documents how to scale `num_hidden_layers` and `ep_size` for the full 43-layer multi-node run.
A ready-to-use recipe ships at [`examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml`](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml). The yaml header documents how to scale `num_hidden_layers` and `ep_size` for the full 43-layer multi-node run.

NeMo Automodel supports several ways to launch training — via the Automodel CLI with Slurm, interactive sessions, `torchrun`, and more. For full details on all launch options (Slurm batch jobs, multi-node configuration, environment variables, etc.), see the [Run on a Cluster](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/launcher/slurm.md) guide.

### Quick infrastructure validation (single node, 8 × A100-80G)

Run the 4-layer validate harness to confirm forward / backward / PP / DCP health end-to-end before launching the full schedule:

```bash
PYTHONPATH=/path/to/Automodel:$PYTHONPATH automodel \
examples/llm_finetune/deepseek_v4/deepseek_v4_flash_validate.yaml \
--nproc-per-node 8 \
--model.config.pretrained_model_name_or_path=/your/local/dsv4-flash \
--model.config.name_or_path=/your/local/dsv4-flash
```

The validate yaml uses `num_hidden_layers=4` with `compress_ratios=[0, 0, 4, 128]`, exercising SWA / SWA / CSA / HCA on every forward, and `num_hash_layers=2` so both stage-0 layers are hash-routed.

### Standalone Slurm Script

Below is a standalone Slurm script example for the HellaSwag recipe. Before running it, ensure your cluster environment is configured following the [Run on a Cluster](https://github.com/NVIDIA-NeMo/Automodel/blob/main/docs/launcher/slurm.md) guide. Then submit the job:
Expand Down
91 changes: 91 additions & 0 deletions docs/model-coverage/llm/deepseek-ai/dsv4-flash.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# DeepSeek V4 Flash

[DeepSeek V4 Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) is DeepSeek's latest fine-grained Mixture-of-Experts language model. It uses a 43-layer all-MoE backbone with 256 routed experts plus one shared expert per block, top-6 routing, and a hybrid per-layer attention zoo (SWA / CSA / HCA) selectable through `compress_ratios`. The first `num_hash_layers` blocks use a hash-clustering gate, and every block maintains `hc_mult=4` Hyper-Connection streams mixed via a learned col-norm-first Sinkhorn router.

:::{card}
| | |
|---|---|
| **Task** | Text Generation (MoE) |
| **Architecture** | `DeepseekV4ForCausalLM` |
| **Parameters** | fine-grained MoE, 256 routed + 1 shared expert |
| **HF Org** | [deepseek-ai](https://huggingface.co/deepseek-ai) |
:::

## Available Models

- **DeepSeek-V4-Flash**

## Architecture

- `DeepseekV4ForCausalLM`

## Example HF Models

| Model | HF ID |
|---|---|
| DeepSeek V4 Flash | [`deepseek-ai/DeepSeek-V4-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) |

## Example Recipes

| Recipe | Description |
|---|---|
| [`deepseek_v4_flash_hellaswag.yaml`](https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml) | SFT — DeepSeek V4 Flash on HellaSwag with pipeline parallelism |


## Try with NeMo AutoModel

**1. Install** ([full instructions](../../../guides/installation.md)):

```bash
pip install nemo-automodel
```

**2. Clone the repo** to get the example recipes:

```bash
git clone https://github.com/NVIDIA-NeMo/Automodel.git
cd Automodel
```

:::{note}
The full 43-layer schedule requires a multi-node run; see the recipe yaml header for `ep_size` / `pp_size` guidance. See the [Launcher Guide](../../../launcher/slurm.md) for multi-node setup.
:::

**3. Run the recipe** from inside the repo:

```bash
automodel --nproc-per-node=8 examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml
```

:::{dropdown} Run with Docker
**1. Pull the container** and mount a checkpoint directory:

```bash
docker run --gpus all -it --rm \
--shm-size=8g \
-v $(pwd)/checkpoints:/opt/Automodel/checkpoints \
nvcr.io/nvidia/nemo-automodel:26.02.00
```

**2.** Navigate to the AutoModel directory (where the recipes are):

```bash
cd /opt/Automodel
```

**3. Run the recipe**:

```bash
automodel --nproc-per-node=8 examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml
```
:::

See the [Installation Guide](../../../guides/installation.md) and [LLM Fine-Tuning Guide](../../../guides/llm/finetune.md).

## Fine-Tuning

See the [Fine-Tune DeepSeek V4 Flash](../../../guides/llm/dsv4-flash.md) guide and the [Large MoE Fine-Tuning Guide](../../../guides/llm/large-moe-finetune.md).

## Hugging Face Model Cards

- [deepseek-ai/DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash)
2 changes: 2 additions & 0 deletions docs/model-coverage/llm/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ NeMo AutoModel supports the [AutoModelForCausalLM](https://huggingface.co/transf
| Qwen / Alibaba Cloud | [Qwen3-Next](qwen/qwen3-next.md) | `Qwen3NextForCausalLM` |
| DeepSeek | [DeepSeek](deepseek-ai/deepseek.md) | `DeepseekForCausalLM` |
| DeepSeek | [DeepSeek-V3](deepseek-ai/deepseek-v3.md) | `DeepseekV3ForCausalLM`, `DeepseekV32ForCausalLM` |
| DeepSeek | [DeepSeek V4 Flash](deepseek-ai/dsv4-flash.md) | `DeepseekV4ForCausalLM` |
| Mistral AI | [Mistral](mistralai/mistral.md) | `MistralForCausalLM` |
| Mistral AI | [Mixtral](mistralai/mixtral.md) | `MixtralForCausalLM` |
| Mistral AI | [Ministral3 / Devstral](mistralai/ministral3.md) | `Mistral3ForConditionalGeneration` |
Expand Down Expand Up @@ -96,6 +97,7 @@ qwen/qwen3-moe
qwen/qwen3-next
deepseek-ai/deepseek
deepseek-ai/deepseek-v3
deepseek-ai/dsv4-flash
mistralai/mistral
mistralai/mixtral
mistralai/ministral3
Expand Down
Loading