[1/N] Harden Transformers modelling backend multi-modal path

[hmellor]

Fixed

• Text-only prompts crashed multimodal models with KeyError: "Modality 'image' not found". apply() ran _apply_vision whenever the model supported images, not when items were present; the mm_token_type_ids is None guard never fires because HF returns an all-zero tensor. Repro: BAAI/Emu3-Chat-hf + a plain text prompt
• Token-ID prompts had special tokens added twice on re-tokenisation: Gemma3 [2] became [2, 2], so ID and text prompts disagreed
• Memory profiling allocated 1 audio placeholder for N audio items. get_dummy_text emitted audio_token * n with no separator, so the run-length scan merged them. Verified: 3 items produced 1 placeholder, now 3
• Audio and image field configs overwrote each other. Both branches of _get_mm_fields_config mapped every key in hf_inputs, so whichever ran last took ownership. Keys are now attributed to the modality that produced them, derived from each sub-processor's model_input_names
• Removed a redundant all-True is_embed mask allocated per audio placeholder

Changed

• Adjacent audio placeholders now raise instead of silently collapsing into one. No HF-generic per-item audio token count exists to split the run with. Stopgap: text_replacement_offsets will fix it properly

Tests

• Added test_processing_correctness_transformers. The backend was excluded from _test_processing_correctness, so it had no cached-vs-uncached coverage at all. This caught the first two bugs
• Added adjacent-placeholder, text-only, and modality-partition cases. Both existing multi-item tests put a separator between placeholders, so the merge path was untested

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[DarkLight1337]

Let's get this fix in first, though I have a comment

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #82867 for commit fff4dd11aaa1.