Skip to content

Updated Python executable path in NIXL KVBench workloads - #709

Merged
amaslenn merged 2 commits into
NVIDIA:mainfrom
Bohatchuk:main
Oct 16, 2025
Merged

amaslenn merged 2 commits into
NVIDIA:mainfrom
Bohatchuk:main

Conversation

@Bohatchuk

Copy link
Copy Markdown
Contributor

Summary

Due to the changes implemented in NIXLBench container (ai-dynamo/nixl#845) (Python venv is part of PATH) and to support further integration compatibility with NIXLBench container updated Python executable path for NIXL KVBench and NIXL KVBench CTPerf workloads.

Test Plan

Real-platform runs of NIXL KVBench CTPerf and NIXL KVBench w/ POSIX and UCX backends on internal clusters (both aarch64 and x86_64 archs):

  • uv run cloudai run --system-config conf/system/system.toml --test-scenario conf/test_scenario/kvbench_ctperf.toml
name = "NIXL.KVBench.CTPerf"

[[Tests]]
id = "NIXL.KVBench.CTPerf"
name = "KVBenchCTPerf"
description = "NIXL KVBench CTPerf"
test_template_name = "NixlPerftest"
num_nodes = 2
time_limit = "00:45:00"

  [Tests.cmd_args]
  docker_image_url = "urm.nvidia.com#.../nixlbench:...-latest"
  subtest = "sequential-ct-perftest"
  num_user_requests = 10
  batch_size = 1
  num_prefill_nodes = 1
  num_decode_nodes = 1
  prefill_tp = 4
  decode_tp = 4
  isl_mean = 10000
  isl_scale = 3000
  model = "deepseek-r1-distill-llama-70b"
  • uv run cloudai run --system-config conf/system/system.toml --test-scenario conf/test_scenario/kvbench_posix.toml
name = "NIXL.KVBench.POSIX"

[[Tests]]
id = "NIXL.KVBench.POSIX"
name = "KVBenchProfile"
description = "NIXL KVBench w/ POSIX backend"
test_template_name = "NIXLKVBench"
num_nodes = 1
time_limit = "00:10:00"

  [Tests.cmd_args]
  docker_image_url = "urm.nvidia.com#.../nixlbench:...-latest"
  model = "./examples/model_deepseek_r1.yaml"
  model_config = "./examples/block-tp1-pp16.yaml"
  backend = "POSIX"
  num_requests = 1
  source = "file"
  num_iter = 16
  page_size = 256
  filepath = "/data"
  • uv run cloudai run --system-config conf/system/system.toml --test-scenario conf/test_scenario/kvbench_ucx.toml
name = "NIXL.KVBench.UCX"

[[Tests]]
id = "NIXL.KVBench.UCX"
name = "KVBenchProfile"
description = "NIXL KVBench w/ UCX backend"
test_template_name = "NIXLKVBench"
num_nodes = 2
time_limit = "00:10:00"

  [Tests.cmd_args]
  docker_image_url = "urm.nvidia.com#.../nixlbench:...-latest"
  model = "./examples/model_deepseek_r1.yaml"
  model_config = ["./examples/layer-tp1-pp1.yaml", "./examples/block-tp1-pp16.yaml"] 
  backend = "UCX"
  source = ["memory", "gpu"]
  op_type = ["READ", "WRITE"]

amaslenn
amaslenn previously approved these changes Oct 16, 2025
@amaslenn
amaslenn merged commit f3725a7 into NVIDIA:main Oct 16, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants