Skip to content
Merged
Changes from 7 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
3d3c556
fix batch invariant warning
WindChimeRan Feb 21, 2026
0a0ea10
add pre-commit
WindChimeRan Feb 21, 2026
f42bf0e
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Feb 21, 2026
aa879f6
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Feb 22, 2026
6e3d53f
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Feb 22, 2026
899d1e8
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Feb 24, 2026
98d2996
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Feb 27, 2026
bb3e4b2
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 1, 2026
48eeee6
update all env var consumers
WindChimeRan Mar 1, 2026
4801f60
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 1, 2026
0a26ebc
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 2, 2026
09c4dc9
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 2, 2026
aff083e
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 2, 2026
e0f34c8
fix tests for env var
WindChimeRan Mar 3, 2026
4da8190
Merge remote-tracking branch 'fork/fix/register-batch-invariant-env' …
WindChimeRan Mar 3, 2026
f86edc3
Merge remote-tracking branch 'origin/main' into fix/register-batch-in…
WindChimeRan Mar 3, 2026
ddb04a1
Merge remote-tracking branch 'origin/main' into fix/register-batch-in…
WindChimeRan Mar 4, 2026
b7454e9
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 4, 2026
aead361
fix legacy kernel test
WindChimeRan Mar 4, 2026
b688ae3
Merge remote-tracking branch 'fork/fix/register-batch-invariant-env' …
WindChimeRan Mar 4, 2026
463a60f
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 5, 2026
8b59cb2
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 6, 2026
c4ac7c3
Merge branch 'main' into fix/register-batch-invariant-env
yewentao256 Mar 6, 2026
152853d
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 7, 2026
b6b9efa
fix fa_utils
WindChimeRan Mar 7, 2026
a331e79
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 7, 2026
51aa06a
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 8, 2026
95670b3
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 10, 2026
b4b8d85
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 21, 2026
0fdc21e
Merge branch 'main' into fix/register-batch-invariant-env
WindChimeRan Mar 22, 2026
e9da71a
fix fp8 utils merging conflict
WindChimeRan Mar 22, 2026
0843198
Merge remote-tracking branch 'origin/main' into fix/register-batch-in…
WindChimeRan Mar 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions vllm/envs.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
VLLM_TARGET_DEVICE: str = "cuda"
VLLM_MAIN_CUDA_VERSION: str = "12.9"
VLLM_FLOAT32_MATMUL_PRECISION: Literal["highest", "high", "medium"] = "highest"
VLLM_BATCH_INVARIANT: bool = False
MAX_JOBS: str | None = None
NVCC_THREADS: str | None = None
VLLM_USE_PRECOMPILED: bool = False
Expand Down Expand Up @@ -484,6 +485,9 @@ def _get_or_set_default() -> str:
["highest", "high", "medium"],
case_sensitive=False,
),
# Enable batch-invariant mode: deterministic results regardless of
# batch composition. Requires NVIDIA GPU with compute capability >= 9.0.
"VLLM_BATCH_INVARIANT": lambda: bool(int(os.getenv("VLLM_BATCH_INVARIANT", "0"))),
# Maximum number of compilation jobs to run in parallel.
# By default this is the number of CPUs
"MAX_JOBS": lambda: os.getenv("MAX_JOBS", None),
Expand Down