[hmellor]

Overview

Split from #60823: deletes the vendored Inkling processor and builds Inkling's multimodal inputs with upstream Transformers InklingProcessor.

Related PRs:

• [Chore] Port more vendored configs and processors to upstream Transformers #60823: the larger unvendoring PR this is split from; Inkling has been dropped from it.
• [Chore] Use upstream Transformers DeepseekV4Config #61392 and [Chore] Use upstream Transformers MiniMaxM3VLConfig #61813: the same split for DeepSeek-V4 and MiniMax-M3.

No other open PR touches vllm/models/inkling/common/mm_preprocess.py or the vendored processor.

Claims

• Deletes vllm/transformers_utils/processors/inkling.py (504 lines). InklingProcessingInfo.get_hf_processor loads upstream InklingProcessor from the checkpoint, which ships in every Transformers version vLLM supports (>= 5.16.1).
• Images: the upstream image processor patchifies, but vLLM still does the 2x long-edge upscale with PIL LANCZOS, because upstream defaults to no rescale and torchvision LANCZOS does not match the reference.
• Audio: the upstream feature extractor computes the mel features and InklingProcessor._extract_dmel_bins (private) quantises them to dMel ids. The dMel parameters now come from the checkpoint's preprocessor_config.json instead of being copied from the vLLM config.
• The multimodal kwarg names are unchanged (input_audio_features stays), so the model code and the Rust frontend, which sends Inkling audio under that key, are untouched. The only output difference is that the dMel ids are int32 instead of float32 holding the same integers; the audio tower casts to int32 anyway.
• No numerics change: processor outputs and greedy decode are bit-identical to main.

Validation

Processor outputs on CPU, thinkingmachines/Inkling-NVFP4, main vs this PR: 4 images (224x224 to 1920x1080, and 37x501), 3 audio clips at 16 kHz (1 s to 31 s), one 44.1 kHz clip, and a mixed image + audio prompt. Prompt token ids, pixel_values and num_patches are identical; the dMel ids have identical values.

Greedy decode, thinkingmachines/Inkling-Small-NVFP4 on 2x B200 (TP2), transformers 5.19.0, FlashInfer autotuning off, 128 tokens with logprobs:

Prompt	main run 1 vs run 2	main vs this PR
Text only	identical	identical
1 image (x2 prompts)	identical	identical
2 images	identical	identical
1 audio (x2 prompts)	identical	identical
Image + audio	identical	identical

"Identical" means the same prompt token count, text and logprobs (max |Δ logprob| = 0). With FlashInfer autotuning on, two main launches already differ from each other, so it was turned off.

vllm serve thinkingmachines/Inkling-Small-NVFP4 --tokenizer-mode inkling --trust-remote-code \
  --tensor-parallel-size 2 --max-model-len 32768 --limit-mm-per-prompt '{"image": 2, "audio": 1}' \
  --kernel-config '{"enable_flashinfer_autotune": false}'

pytest tests/renderers/test_inkling.py tests/models/multimodal/processing/test_common.py -k "inkling or Inkling"
# main and this PR: 48 passed (includes 3 test_processing_correctness cases for Inkling-NVFP4)

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[hmellor]

/ci run

[github-actions]

✅ Triggered Buildkite CI #95352 for commit 1297b09ae210.