Conversation
There was a problem hiding this comment.
Code Review
This pull request ensures deterministic benchmark dataset generation by sorting the available tokens from the tokenizer vocabulary and resetting the random seeds (both random and numpy.random) during ImageDataset.load to prevent processor initialization from affecting the global RNG state. It also adds corresponding unit tests to verify these behaviors. There are no review comments to address, and we have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Summary
Motivation
Seeded random-image
bench_servingruns were not fully reproducible across fresh client processes. With the same--seed 777and identical sampled image-size statistics, two runs reported different token totals:total/text/vision = 4611/48/4563vs4609/44/45654610/47/4563vs4610/47/4563This makes strict SGLang/vLLM random-image comparisons unreliable even when the nominal benchmark arguments match.
Validation
test_benchmark_datasets_api.py: 40 passed, 3 subtests passedThis only changes benchmark workload generation; it does not touch the serving hot path or claim a runtime performance change.
CI States
Latest PR Test (Base): ❌ Run #29570655883
Latest PR Test (Extra): ❌ Run #29570655646