Repository navigation
Add --constrained and --engine-samples for the engine perf branches - #10
Merged
Merged
Conversation
Ports the example server's side of two open vLLM changes, from the fork's constrained-vocab (vllm#58216) and diffusion-samples (vllm#58438) branches: - --constrained sends diffusion_constrained on every label read, so the engine runs the unembedding, sampler and self-conditioning over the labels only. - --engine-samples sends a fixed sample count as one diffusion_samples request with a seed, and reads one choice per draw. --max-samples caps the count. Both are opt-in, unlike the fork's example server, because a stock vLLM rejects seed on a diffusion request. The read path now returns one distribution list per draw, and diagnostics.timing gains "requests". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the server side of two vLLM changes that are still open. Both come from the fork branches that the djev-spark image builds on:
--constraineddiffusion_constrained)label_massis always 1--engine-samplesdiffusion_samples)samples: Ngoes out as one request that fans into N seeded noise draws, instead of N separate requests.--max-samplescaps NBoth flags are off by default. The fork's example server turns them on, but stock vLLM returns 400 when a diffusion request sets
seed. Stock vLLM ignoresdiffusion_constrained.The patch is the fork's example-server diff, which applied cleanly, with the two defaults flipped.
one_read,one_read_continuationandread_manynow return one distribution list per draw.diagnostics.timinggainsrequests.Measured on a GB10 with the djev-spark stack, flags on against off, same engine: +8–9% req/s with 1 sample, +33–50% with 4 samples.
This PR does not overlap #9, which changes only the chat and tool-choice path.
🤖 Generated with Claude Code