[Tokenizer] Validate caller token IDs against the vocab and active multimodal markers

[rodamani]

Motivation

Caller-supplied input_ids are not validated against the vocabulary before multimodal expansion (_validate_input_ids_in_vocab was removed from TokenizerManager as a dead helper). Out-of-range or negative IDs are not rejected at admission, and a caller can place a processor's multimodal marker ID in a prompt that supplies no media.

Modifications

• TokenizerManager._validate_generation_input_ids checks each normalized prompt against the text vocabulary (hf_text_config.vocab_size), allowing only the active processor's placeholder IDs for media actually supplied with that prompt (per batch item). Empty prompts, embedding overrides, session continuation and batch metadata keep their existing contracts.
• Processors expose get_input_token_ids_for_data (base, LLaVA, inkling).
• Warmup synthetic IDs are bounded to the vocabulary (otherwise warmup on models with a vocabulary under 65536 would trip the new check).
• The optional SGLANG_MM_STRIP_PROCESSOR_INPUT_IDS payload reduction that was originally bundled here is split into its own PR.
• Tests: test/registered/unit/managers/test_tokenizer_input_contracts.py (new), test_tokenizer_manager_rid_cleanup.py, test/registered/unit/entrypoints/test_server_warmup.py.

Relationship to open PRs: #31908 wires a plain [0, vocab_size) check into _validate_one_request, which does not know about processor-registered multimodal markers (it would reject valid placeholder IDs outside the text vocabulary, or accept a marker with no supplied media) and does not bound warmup IDs.

Accuracy Tests

CPU: 55 passed, 23 subtests passed across the three files. Porting note: one test needed scheduler.model_config on its stub because main's _process_and_broadcast_mm_inputs now reads requires_mm_token_modalities.

Compatibility

Requests whose input_ids are all inside [0, vocab_size) (or are the active processor's placeholders for media supplied with that prompt) behave as before. Requests with negative or out-of-vocabulary IDs, or with a multimodal marker but no matching media, now get a ValueError (400) at admission instead of reaching the model. No new flags; text-only and embedding requests are unaffected.

Speed Tests and Profiling

Adds a Python pass over caller-supplied input_ids at admission (text-only requests have no media lookups). Not separately benchmarked.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #36767748375
Latest PR Test (Extra): ❌ Run #36767747692
Latest PR Test (AMD ROCm 10): ❌ Run #36767748350

[rodamani]

/tag-and-rerun-ci