Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ keywords = ["llm", "genai", "inference", "nvidia", "distributed"]

[workspace.dependencies]
# Published separately by https://github.com/ai-dynamo/aisimulate.
aisimulate-core = { version = "=0.12.0-dev.2" }
aisimulate-core = { version = "=0.12.0" }

# Local crates
dynamo-runtime = { path = "lib/runtime", version = "1.5.0" }
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ classifiers = [
]

dependencies = [
"aisimulate==0.12.0.dev2; python_version >= '3.11' and python_version < '3.14'",
"aisimulate==0.12.0; python_version >= '3.11' and python_version < '3.14'",
"aiperf==0.12.0",
"matplotlib",
"networkx",
Expand Down
2 changes: 1 addition & 1 deletion container/deps/requirements.aisimulate.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
# SPDX-License-Identifier: Apache-2.0

# Published AISimulate wheel staged into planner and framework runtime wheelhouses.
aisimulate==0.12.0.dev2
aisimulate==0.12.0
11 changes: 7 additions & 4 deletions container/templates/wheel_builder.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -635,16 +635,19 @@ COPY deploy/inference-gateway/sidecar/ /opt/dynamo/deploy/inference-gateway/side
{% if target == "planner" or (target == "runtime" and framework in ("vllm", "sglang", "trtllm")) %}
COPY container/deps/requirements.aisimulate.txt /opt/dynamo/container/deps/requirements.aisimulate.txt

# AI Simulate is released separately as an abi3 wheel. Stage the exact published
# wheel consumed by ai-dynamo instead of rebuilding it from vendored source.
# Download only this distribution; runtime images own dependency installation
# through their requirements files and local wheels.
# AI Simulate is released separately as an abi3 wheel. Its public PyPI artifact
# is a small resolver sdist because the full wheel is too large for that registry;
# stage the actual wheel directly from NVIDIA's package index. Download only this
# distribution; runtime images own dependency installation through their
# requirements files and local wheels.
RUN --mount=type=cache,id=uv-root-{{ context.dynamo.uv_version }},target=/root/.cache/uv,sharing=shared \
export UV_CACHE_DIR=/root/.cache/uv && \
source ${VIRTUAL_ENV}/bin/activate && \
python -m pip download \
--only-binary=:all: \
--no-deps \
--no-index \
--find-links https://pypi.nvidia.com/aisimulate/ \
--dest /opt/dynamo/dist \
--requirement /opt/dynamo/container/deps/requirements.aisimulate.txt
{% endif %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Use Python 3.11 through 3.13. Install AISimulate, which provides the retained `a
sizing command:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```

Collect:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Use Python 3.11 through 3.13.

```bash
# Install the distribution that provides the compatibility command
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"

# Optional: check whether the model/system/backend is covered
aiconfigurator cli support \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ path. They have no adapter migration.
AISimulate requires Python 3.11 through 3.13. Dynamo itself still supports Python 3.10, but the
`aisimulate` dependency and its CLI are not installed in a Python 3.10 environment.

The `dynamo-planner` image installs the published `aisimulate==0.1.0.dev2` wheel from its local
wheelhouse. Dynamo builds `aisimulate-core==0.1.0-dev.2` from crates.io instead of vendoring the
The `dynamo-planner` image installs the published `aisimulate==0.12.0` wheel from its local
wheelhouse. Dynamo builds `aisimulate-core==0.12.0` from crates.io instead of vendoring the
AISimulate source tree.

For Dynamo source development, install the published AISimulate wheel, Dynamo, and the Planner
dependencies from the Dynamo repository root:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
python3 -m pip install --no-deps -e .
python3 -m pip install -r container/deps/requirements.planner.txt
```
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ AISimulate requires Python 3.11 through 3.13. The `ai-dynamo` package remains in
3.10, but the AISimulate dependency and CLI are not installed there.

The supported prebuilt environment is the `dynamo-planner` image. The image stages and installs the
published `aisimulate==0.1.0.dev2` wheel alongside the Dynamo wheels. Dynamo's Rust workspace
resolves `aisimulate-core==0.1.0-dev.2` from crates.io.
published `aisimulate==0.12.0` wheel alongside the Dynamo wheels. Dynamo's Rust workspace
resolves `aisimulate-core==0.12.0` from crates.io.

For Dynamo source development, install the published AISimulate wheel and build the matching
Dynamo bindings:
Expand All @@ -30,7 +30,7 @@ cd lib/bindings/python
maturin develop --uv --release --features aic-forward-pass
cd ../../..
python3 -m pip install --no-deps -e .
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
python3 -m pip install -r container/deps/requirements.planner.txt
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ provide one config as a strict pin or several configs as a custom projection poo

## Configuration

Use this reference configuration:
The experiment used the following configuration shape. It documents the retired
`aisimulate==0.1.0.dev1` API and is not accepted by AISimulate 0.12.0:

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ subtitle: Experimental Sweeper search over Planner policies for MiniMax-M2.5 on
> treat them as production capacity guidance or a performance commitment. Sweeper's search behavior
> and output may change without a standard deprecation period.

Enable the planner (SLA mode) on the same deployment as the
[router experiment](router-end-to-end-latency-sweep.md), then use Sweeper's smart sweep to **find the
planner configuration that maximizes goodput-per-GPU** on the Mooncake toolagent trace, and
compare against the static deployment and the default planner.
This experiment enabled the planner (SLA mode) on the same deployment as the
[router experiment](router-end-to-end-latency-sweep.md), then used Sweeper's smart sweep to find the
planner configuration that maximized goodput-per-GPU on the Mooncake toolagent trace and compared
it against the static deployment and the default planner.

## Setup

Expand Down Expand Up @@ -50,8 +50,10 @@ scales the fleet to an average of ~13.5 GPUs instead of pinning 32.

## Sweep config

`Sweeper.run`, `goal.target = goodput_per_gpu`. Deployment + router + workload + SLA
fixed; only the planner knobs swept (full trace, open-loop):
The experiment used `Sweeper.run` with `goal.target = goodput_per_gpu`. It fixed the deployment,
router, workload, and SLA, and swept only the planner settings using the full open-loop trace. The
configuration below documents the retired `aisimulate==0.1.0.dev1` API and is not accepted by
AISimulate 0.12.0:

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ subtitle: Run a backend-neutral sweep with an injected replay runtime
Install AISimulate:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```

Sweeper requires a `RunnerFactory` supplied by the application that owns replay execution:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ subtitle: Experimental Sweeper search over KV router settings for MiniMax-M2.5 o
> treat them as production capacity guidance or a performance commitment. Sweeper's search behavior
> and output may change without a standard deprecation period.

Reproduce the DynoSim router experiment, then use Sweeper's smart sweep to **find the
KV-router configuration that minimizes mean end-to-end latency** on the Mooncake toolagent
trace, and validate the winner against the default-router baseline on the full trace.
This experiment reproduced the DynoSim router experiment, then used Sweeper's smart sweep to find
the KV-router configuration that minimized mean end-to-end latency on the Mooncake toolagent trace
and validated the winner against the default-router baseline on the full trace.

## Setup

Expand All @@ -32,9 +32,10 @@ concurrency) so any change in e2e latency is attributable to the **router** alon

## Sweep config

Searched with `Sweeper.run` (Vizier GP-bandit), `goal.target = e2e_latency`. Search
runs on a 2k-request subset (fast, representative — same long-context mix); the winner is
then validated on the full trace.
The experiment searched with `Sweeper.run` (Vizier GP-bandit) and
`goal.target = e2e_latency`. It ran on a 2k-request subset with the same long-context mix, then
validated the winner on the full trace. The configuration below documents the retired
`aisimulate==0.1.0.dev1` API and is not accepted by AISimulate 0.12.0.

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ forward-pass timing model.
retained `aiconfigurator` sizing command:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```
- Your checkpoint's HuggingFace ID, GPU system (for example `h200_sxm`), backend (`vllm`, `sglang`, or `trtllm`), the total GPUs you can allocate, and your SLA: Time To First Token (TTFT), Time Per Output Token (TPOT), and typical input/output sequence lengths (ISL/OSL).

Expand Down
4 changes: 2 additions & 2 deletions lib/bindings/kvbm/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions lib/bindings/python/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/bindings/python/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ request-trace-s3 = ["dynamo-llm/request-trace-s3"]
# former AIC crate surface. The `python` feature is required for
# AicEngineBuilder; this extension supplies the interpreter, so the dependency
# must not enable `extension-module`.
aisimulate-core = { version = "=0.12.0-dev.2", optional = true, features = ["python"] }
aisimulate-core = { version = "=0.12.0", optional = true, features = ["python"] }
dynamo-runtime = { path = "../../runtime" }
dynamo-mocker = { path = "../../mocker" }
dynamo-parsers = { version = "8.1.0" }
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ license-files = ["LICENSE"]
requires-python = ">=3.10"
dependencies = [
"ai-dynamo-runtime==1.5.0",
"aisimulate==0.12.0.dev2; python_version >= '3.11' and python_version < '3.14'",
"aisimulate==0.12.0; python_version >= '3.11' and python_version < '3.14'",
"aiohttp>=3.14.3,<4.0",
"transformers>=4.56.0",
"kubernetes>=32.0.1,<33.0.0",
Expand Down
2 changes: 1 addition & 1 deletion tests/dependencies/test_aiconfigurator_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def test_no_manifest_installs_retired_aic_distributions() -> None:
assert all(package["name"] != "aiconfigurator-core" for package in packages)
assert features["aic-forward-pass"] == ["dep:aisimulate-core"]
assert dependencies["aisimulate-core"] == {
"version": "=0.12.0-dev.2",
"version": "=0.12.0",
"optional": True,
"features": ["python"],
}
Expand Down
3 changes: 3 additions & 0 deletions tests/dependencies/test_aisimulate_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,10 @@ def test_container_stages_the_published_aisimulate_wheel() -> None:
"--requirement /opt/dynamo/container/deps/requirements.aisimulate.txt"
in wheel_builder
)
assert "--only-binary=:all:" in wheel_builder
assert "--no-deps" in wheel_builder
assert "--no-index" in wheel_builder
assert "--find-links https://pypi.nvidia.com/aisimulate/" in wheel_builder
assert "COPY aisimulate" not in wheel_builder
assert "/opt/dynamo/aisimulate" not in wheel_builder
assert not (ROOT / "aisimulate").exists()
Expand Down
Loading