[Model] Remove native LongCat implementations by xianbaoqian · Pull Request #53811 · vllm-project/vllm

xianbaoqian · 2026-08-26T00:01:54Z

Mutually agree with the LongCat team.

Depends on:

Make LongcatFlashConfig self-consistent without building the model huggingface/transformers#48899 - config hard codes default for num_hidden_layers which might disagree with num_layers
https://huggingface.co/meituan-longcat/LongCat-Flash-Chat/discussions/15 - checkpoint is missing model_type so the Transformers code cannot load with AutoConfig

Purpose

Remove the native LongCat-Flash Chat, Lite, and MTP implementations and route the supported causal-LM architectures through the Transformers modeling backend, following the migration pattern from #53608 and #53615.

This change:

routes LongcatFlashForCausalLM and LongcatFlashNgramForCausalLM to TransformersMoEForCausalLM;
removes the native model files, native MTP integration, and the now-unused n-gram CUDA operator;
teaches the generic MoE fuser to load LongCat's nested router projection and preserve zero-expert routing;
handles the public Chat checkpoint's missing model_type without executing its stale remote modeling code;
preserves LongCat's dual-attention layer accounting and the Lite compilation/cudagraph safeguard;
documents that Chat uses the official Transformers implementation while Lite still requires --trust-remote-code for its custom n-gram wrapper.

Duplicate-work checks found no open PR for LongCat removal or migration to the Transformers backend. Open LongCat feature PRs are unrelated and the generic LongCat tool parser is intentionally retained.

This PR was prepared with AI assistance. The human submitter must review and understand every changed line before marking it ready for review.

Test Plan

.venv/bin/pre-commit run --from-ref origin/main --to-ref HEAD
.venv/bin/python -m pytest tests/models/transformers/fusers/test_moe.py tests/transformers_utils/test_config.py -k 'moe_fuser or longcat' -q
.venv/bin/python -m pytest tests/models/test_registry.py -k LongcatFlash -q
.venv/bin/python -m pytest tests/config/test_model_arch_config.py -k LongCat -q

Before marking ready:

Run a LongCat generation/evaluation comparison on GPU.
Build the removed CUDA extension paths in a CUDA development environment.
Complete human review of every changed line.

Test Result

Pre-commit: passed, including Ruff, mypy, clang-format, Markdown, and repository policy hooks.
MoE fuser and LongCat config tests: 19 passed.
LongCat registry tests: 2 passed.
LongCat architecture regression test against the public Chat checkpoint: 1 passed.
Both public Chat and Lite configs resolved to TransformersMoEForCausalLM; meta-device construction matched all LongCat MoE and MLA blocks.
Chat resolved through the built-in Transformers config without trusted remote code. Lite resolved through its remote n-gram config with --trust-remote-code.
GPU model evaluation: not run because no GPU is available in the development environment; this draft must not be marked ready until results are added.
CUDA build: not run because CUDA/CMake toolchains are unavailable in the development environment.

mergify · 2026-08-26T00:02:32Z

Documentation preview: https://vllm--53811.org.readthedocs.build/en/53811/

Route LongCat-Flash models through the Transformers backend and retain the compatibility handling needed by Chat and Lite. Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

DarkLight1337 · 2026-08-27T05:43:39Z

I am concerned this will conflict with the WIP model PRs. Maybe wait for them to be merged first?

xianbaoqian · 2026-08-27T08:23:30Z

I am concerned this will conflict with the WIP model PRs. Maybe wait for them to be merged first?

#48553 Do you mean this one?

DarkLight1337 · 2026-08-27T08:26:13Z

No, I mean Qwen3.8-Next and GLM5.3-Next. They touch a lot of files

mergify · 2026-08-31T17:08:34Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @xianbaoqian.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

hmellor · 2026-09-01T14:25:37Z

Why is this necessary? This config is already registered in Transformers

hmellor · 2026-09-01T14:31:01Z

+class LongCatModelArchConfigConvertor(ModelArchConfigConvertorBase):
+    def get_num_hidden_layers(self) -> int:
+        return self.hf_text_config.num_layers


What reads this?

hmellor · 2026-09-01T14:50:17Z

+        hf_linear = (
+            hf_gate.get_submodule(self.gate_linear_name)
+            if self.gate_linear_name
+            else hf_gate
+        )


This could be unified to:

hf_gate_weight = hf_gate.get_submodule(self.gate_weight_location)

In the case where the weight belongs directly to the gate self.gate_weight_location will be "" and get_submodule will just return hf_gate

IMO the gate_linear_name -> gate_weight_location is playing a big part in clarity. I spent the first 5 minutes trying to figure out where another linear layer was coming from

hmellor · 2026-09-01T15:06:19Z

+                num_decoder_layers = (
+                    self.text_config.num_layers
+                    if self.text_config.model_type
+                    in ("longcat_flash", "longcat_flash_ngram")
+                    else self.text_config.num_hidden_layers
+                )


I would prefer to store the layers_name from pipeline_parallel rather than use model specific attributes like this

hmellor · 2026-09-01T15:37:02Z


        # kv_lora_rank indicates that this is an MLA model
        if getattr(text_config, "kv_lora_rank", None) is not None:
+            num_attn_module = (


We can continue to use num_hidden_layers as the ground truth for "how many attention modules are in this model". Then we can infer this number from num_hidden_layers / len(layers)

Conflicts resolved: - CMakeLists.txt, csrc/libtorch_stable/{ops.h,torch_bindings.cpp}, vllm/_custom_ops.py: keep main's vocab_parallel_embedding op, drop the LongCat ngram_compute_n_gram_ids kernel along with its sources. - docs/models/supported_models.md, vllm/model_executor/models/registry.py: keep both sides (LongCat entries plus main's NanbeigeForCausalLM). - tests/transformers_utils/test_config.py: keep both imports. - vllm/model_executor/models/transformers/base.py: take main's side. Its refactor keys attention by AttentionFuser.layer_index, which already handles LongCat's two attention sublayers per decoder layer, so the extract_layer_index(prefix, num_attn_module) workaround is superseded. - vllm/model_executor/models/longcat_flash{,_mtp,_ngram}.py: stay deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor · 2026-09-17T11:44:11Z

+    ) -> tuple[str, torch.dtype | None, str] | None:
        """Matches `topk(score(linear(x)))`, `score` being `softmax`/`sigmoid`.

        Returns the scoring function and the dtype the router computes in."""


This docstring should be updated

mergify · 2026-09-24T12:39:17Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @xianbaoqian.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify Bot added documentation Improvements or additions to documentation ci/build labels Aug 26, 2026

[Model] Remove native LongCat implementations

093ffab

Route LongCat-Flash models through the Transformers backend and retain the compatibility handling needed by Chat and Lite. Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Xianbao QIAN <xianbao.qian@gmail.com>

xianbaoqian force-pushed the remove-longcat-native branch from 41443b9 to 093ffab Compare August 26, 2026 10:09

xianbaoqian marked this pull request as ready for review August 27, 2026 05:39

xianbaoqian requested review from AndreasKaratzas, DarkLight1337, Harry-Chen, LucasWilkinson, ProExpertProg, WoosukKwon, hmellor, houseroad, mgoin, robertgshaw2-redhat, tlrmchlsmth, yewentao256, youkaichao and ywang96 as code owners August 27, 2026 05:39

claude Bot reviewed Aug 27, 2026

View reviewed changes

mergify Bot added the needs-rebase label Aug 31, 2026

hmellor reviewed Sep 1, 2026

View reviewed changes

mergify Bot removed the needs-rebase label Sep 17, 2026

hmellor reviewed Sep 17, 2026

View reviewed changes

hmellor mentioned this pull request Sep 17, 2026

[Transformers backend] Fuse MoE routers that score after the top-k #57410

Merged

mergify Bot added the needs-rebase label Sep 24, 2026

aoshen02 mentioned this pull request Oct 5, 2026

[Model] LongCat-Flash: scale MLA norms while loading and drop the post-load sweep #60080

Merged

6 tasks done

Conversation

xianbaoqian commented Aug 26, 2026 • edited by hmellor Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

mergify Bot commented Aug 26, 2026

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

DarkLight1337 commented Aug 27, 2026

Uh oh!

xianbaoqian commented Aug 27, 2026

Uh oh!

DarkLight1337 commented Aug 27, 2026

Uh oh!

mergify Bot commented Aug 31, 2026

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

mergify Bot commented Sep 24, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xianbaoqian commented Aug 26, 2026 •

edited by hmellor

Loading