[CI][Mamba] Use well-conditioned prompts for FlashInfer ReplaySSM parity tests

[ZJY0516]

Overview

Fix the persistently failing test_replayssm_flashinfer_decode_matches_baseline[v1-nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4] in the (H100) ReplaySSM E2E job. The failure comes from an ill-conditioned prompt, not ReplaySSM: the FlashInfer parity tests now use a prompt that decodes confidently past the replay window. Test-only change.

Claims

• The FlashInfer ReplaySSM decode (MRV1 + MRV2) and spec-decode parity tests no longer depend on a near-tie in "Once upon a time, in a small village,".
• The Triton/Nano tests keep the original PROMPTS; no non-test code changes.

Validation

Local, 1x H200 (SM90, same as the CI H100), FlashInfer 0.7.0.post1.

Before (reproduces CI):

pytest v1/e2e/test_replayssm_decode.py::test_replayssm_flashinfer_decode_matches_baseline -k v1
FAILED ...[v1-...NVFP4] - AssertionError: Test1:
  Matched tokens: [2156, 1486, 1261, 6552, 8401, 8876, 52206, 2274, 20127, 1317, 3346, 1046]
  baseline: ' there was a young girl named Maya who loved to read. a young girl ...'  {' a': -0.68, ...}
  replayssm_flashinfer: ' there was a young girl named Maya who loved to read. apart from ...'  {' apart': -3.51, ...}

After:

Test	Result
test_replayssm_flashinfer_decode_matches_baseline[v1-...]	passed
test_replayssm_flashinfer_decode_matches_baseline[v2-...]	passed
test_replayssm_flashinfer_spec_decode_matches_baseline	passed

Prompt stability across 5 separately started engines (3 baseline, 2 ReplaySSM; same config as _check_replayssm_parity, CUDA graphs on), compared pairwise (10 pairs) with the check_logprobs_close rule:

Prompt	Pairs failing	Min top-1/top-2 margin
"Once upon a time, in a small village," (old)	2/10, including ReplaySSM vs ReplaySSM	0.0 (token 1)
"The capital of France is"	0/10	0.0–0.5
"It was the best of times, it was the worst of times," (new)	0/10, exact match in all 10	≥ 1.25

Details

Root cause. The NVFP4 Nemotron-3.5 checkpoint does not produce bitwise-identical logits across engine instances, even with an identical baseline config: three baseline-only engines disagreed on prompt 1 from the first generated token (" there was" vs " there lived", margin 0–0.12). The " there was" branch falls into a degenerate repetition ("...loved to read. a young girl named Maya...") with a nearly flat next-token distribution, where run-to-run noise pushes one side's token out of the other side's top-5. Two identically configured ReplaySSM engines fail check_logprobs_close against each other the same way, and baseline-vs-ReplaySSM divergence is within baseline-vs-baseline divergence, so ReplaySSM numerics are not the cause. I also checked that vLLM's ring-tracker advance matches the FlashInfer checkpointing_ssu flush rule (prev_k + T > MAX_WINDOW).

Fix. Add FLASHINFER_PROMPTS, replacing the fragile prompt with "It was the best of times, it was the worst of times,", which decodes all 32 tokens confidently and so still exercises the 16-token window flush/replay. The spec-decode parity test uses the same model and prompts and is switched too. test_replayssm_flashinfer_mtp only checks output length and is unchanged.

Limitations / follow-up. The underlying run-to-run non-determinism of the NVFP4 baseline (likely MoE kernel/autotune selection) is not addressed here.

Duplicate check. Searched issues/PRs for test_replayssm_decode, replayssm_flashinfer_decode_matches_baseline, and related keywords: none found. Of the open PRs touching this test file (#46239, #47576, #48792, #49847, #49887, #50380, #52962, #55688), none changes these prompts or addresses this failure. #55688 restructures the file's constants and may need a trivial rebase.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[ZJY0516]

/ci run --allow-stale

[github-actions]

✅ Triggered Buildkite CI #93482 for commit c4e8aa357ea8.

⚠️ This PR is 34 commits behind upstream main. Running CI at your own risk because --allow-stale was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest main, then rerun /ci run on the latest PR commit.

[vllm-agent]

CI selector (shadow): 1 test steps (1 jobs) instead of 2 (2 jobs)

Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works.

Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.

steps (jobs)	Today's rules	Selector	Would skip	Would add
NVIDIA, CPU and others	2 (2)	1 (1)	1 (1)	0 (0)
AMD mirrors	0 (0)	0 (0)	0 (0)	0 (0)

Selector would run (1)

• replayssm-e2e

Would skip (today's rules run them) (1)

• v1-others-cpu

Would add (today's rules do not run them) (0)

none

AMD mirrors: would skip (0)

none

AMD mirrors: would add (0)

none

_{1 changed files · base e58f8dde7f · head f118eb5d6a · Python record: build 93039 at 1e5d0ea888 · kernel record: table a982c81 (build 93415), map a982c81 · not counted: 10 build steps, 5 A100 steps the generator no longer emits, 0 optional steps the selector would also run}

[ZJY0516]

/ci run --allow-stale

[github-actions]

✅ Triggered Buildkite CI #93538 for commit f118eb5d6a60.