[Feature][Config] Support application-directed Mamba checkpoint token and input processing

[nicholaskh-ai]

Purpose

Implements RFC #55697.

Part 1 of 3 for Application-Directed Mamba Prefix Checkpointing.

Hybrid GDN/Mamba architectures (e.g. Qwen3.5 35B-A3B) exhibit high state memory footprint (128KB-512KB per layer) compared to pure Attention. In catalog attribute extraction, multi-candidate verification, and agent workflows where 1 common product/document prefix is shared across multiple concurrent candidate queries, heuristics cannot safely determine application semantic boundaries (such as text prompt vs multimodal image boundaries).

This PR introduces explicit application-directed checkpointing at the input layer:

• Adds --enable-mamba-checkpoint and --mamba-checkpoint-token (defaults to <|mamba_checkpoint|>) CLI & engine options.
• Automatically sets mamba_cache_mode = "align" when checkpointing is enabled.
• Dynamically registers <|mamba_checkpoint|> as a special token in the HF renderer.
• Parses and strips the checkpoint marker in InputProcessor while adjusting multimodal placeholder offsets.
• Propagates mamba_checkpoint_position into Request.

Follow-up PRs:

• Part 2: Scheduler coordination, checkpoint boundary chunk splitting, and unready state machine.
• Part 3: Batched two-phase grouped prefill execution for hybrid GDN/Mamba (up to 7.6x speedup on L40S).

Duplicate-work Check

• PR [Bugfix][Core] Retain align-mode mamba grid and decode-end checkpoints #53803 addresses align-mode grid retention for EAGLE, not application-defined checkpoint markers.
• PR [Model Runner V2] Automatic prefix caching for GDN hybrid models (mamba_cache_mode=all) with MTP speculative decoding #54637 targets automatic prefix caching for all-mode MTP in MRV2, not application-directed markers in V1.
• No open PR addresses explicit control-token-directed checkpoint boundaries for hybrid Mamba models.

Test Plan

• Unit tests for HF renderer tokenizer registration:
  pytest tests/renderers/test_hf.py -k "test_ensure_mamba_checkpoint_token" -v
• Unit tests for InputProcessor marker extraction and multimodal placeholder offset shifting:
  pytest tests/v1/engine/test_input_processor_trace_replay.py -k "test_extract_mamba_checkpoint" -v
• Unit test for Request data structure propagation:
  pytest tests/v1/test_request.py -k "test_request_copies_session_id" -v

AI assistance was used to prepare this change. The human submitter is responsible for reviewing the changed code and test results.

Generated with Devin

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Signed-off-by: nizhang1 nizhang1@coupang.com

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @nicholaskh-ai.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[nicholaskh-ai]

CC @mgoin @youkaichao

Hi reviewers, this is Part 1 of the implementation stack for RFC #55697. It introduces the application-directed Mamba checkpoint token and InputProcessor parsing/multimodal alignment without altering model execution.

Whenever you have a moment, we would appreciate your review and adding the ready label to kick off CI. Thanks!

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @nicholaskh-ai.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork