Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/post-merge-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@ jobs:
platform: 'linux/amd64,linux/arm64'
builder_name: b-${{ github.run_id }}-${{ github.run_attempt }}
inline_compliance: true
build_timeout_minutes: 45
build_timeout_minutes: 90 # TEMPORARY CHANGE - TODO: INVESTIGATE AND SPEEDUP SLOW BUILDS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline note only narrates that this timeout increase is temporary and should be investigated; the changed value already records the operative configuration, while the comment preserves no lasting constraint or actionable context.

🤖 AI Fix

Remove the inline temporary/TODO comment and leave the timeout value unchanged.

# OSRB CSV diff baseline: post-merge context. main -> previous commit;
# release/* -> prior release tag (see resolve_diff_base.py).
diff_event_context: push
Expand Down
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ keywords = ["llm", "genai", "inference", "nvidia", "distributed"]

[workspace.dependencies]
# Published separately by https://github.com/ai-dynamo/aisimulate.
aisimulate-core = { version = "=0.12.0-dev.2" }
aisimulate-core = { version = "=0.12.0" }

# Local crates
dynamo-runtime = { path = "lib/runtime", version = "1.5.0" }
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ classifiers = [
]

dependencies = [
"aisimulate==0.12.0.dev2; python_version >= '3.11' and python_version < '3.14'",
"aisimulate==0.12.0; python_version >= '3.11' and python_version < '3.14'",
"aiperf==0.12.0",
"matplotlib",
"networkx",
Expand Down
2 changes: 1 addition & 1 deletion container/deps/requirements.aisimulate.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@
# SPDX-License-Identifier: Apache-2.0

# Published AISimulate wheel staged into planner and framework runtime wheelhouses.
aisimulate==0.12.0.dev2
aisimulate==0.12.0
11 changes: 7 additions & 4 deletions container/templates/wheel_builder.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -634,16 +634,19 @@ COPY deploy/inference-gateway/ext-proc/ /opt/dynamo/deploy/inference-gateway/ext
{% if target == "planner" or (target == "runtime" and framework in ("vllm", "sglang", "trtllm")) %}
COPY container/deps/requirements.aisimulate.txt /opt/dynamo/container/deps/requirements.aisimulate.txt

# AI Simulate is released separately as an abi3 wheel. Stage the exact published
# wheel consumed by ai-dynamo instead of rebuilding it from vendored source.
# Download only this distribution; runtime images own dependency installation
# through their requirements files and local wheels.
# AI Simulate is released separately as an abi3 wheel. Its public PyPI artifact
# is a small resolver sdist because the full wheel is too large for that registry;
# stage the actual wheel directly from NVIDIA's package index. Download only this
# distribution; runtime images own dependency installation through their
# requirements files and local wheels.
RUN --mount=type=cache,id=uv-root-{{ context.dynamo.uv_version }},target=/root/.cache/uv,sharing=shared \
export UV_CACHE_DIR=/root/.cache/uv && \
source ${VIRTUAL_ENV}/bin/activate && \
python -m pip download \
--only-binary=:all: \
--no-deps \
--no-index \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 0.12.0 wheel is fetched from NVIDIA's index only for planner and framework-runtime targets, but the frontend target also runs on Python 3.12 and installs the top-level ai-dynamo wheel, which requires aisimulate==0.12.0. The PR documents PyPI's artifact as only a resolver sdist, so the frontend image never receives the required binary wheel and its AIC smoke path cannot import aiconfigurator_core.

🤖 AI Fix

Stage the NVIDIA AISimulate wheel for the frontend target and install that local wheel before installing the top-level Dynamo wheel.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Premise does not hold. Verified by execution. The public PyPI artifact for aisimulate==0.12.0 is a wheel-stub redirector, not a dead end. A plain pip install aisimulate==0.12.0 against the default index builds through wheel_stub.buildapi, pulls the real 83 MB abi3 wheel from https://pypi.nvidia.com/, installs it and imports. The frontend target needs no staging change.

Measured: clean Python 3.12 venv, default index only, plus the sdist that proves the mechanism
$ python -m pip install --no-cache-dir "aisimulate==0.12.0"
Building wheels for collected packages: aisimulate
  Building wheel for aisimulate (pyproject.toml): finished with status 'done'
  Created wheel for aisimulate: filename=aisimulate-0.12.0-cp311-abi3-macosx_11_0_arm64.whl size=83827766
Successfully built aisimulate
$ python -c "import aisimulate; print(aisimulate.__file__)"
.../site-packages/aisimulate/__init__.py

The sdist contains only four files. Its pyproject.toml is:

[build-system]
requires = ["wheel-stub"]
build-backend = "wheel_stub.buildapi"

[tool.wheel_stub]
index_url = "https://pypi.nvidia.com/"

The staging step is different because it passes --only-binary=:all:, which refuses the stub. That is the exact reason the --no-index plus --find-links change in this hunk is needed there and only there. The staging command, run as written from this head, resolves a real wheel for both build architectures:

target wheel size
manylinux_2_28_x86_64 aisimulate-0.12.0-cp311-abi3-manylinux_2_28_x86_64.whl 83.6 MB
manylinux_2_28_aarch64 aisimulate-0.12.0-cp311-abi3-manylinux_2_28_aarch64.whl 83.3 MB

--find-links https://pypi.nvidia.com/aisimulate/ \
--dest /opt/dynamo/dist \
--requirement /opt/dynamo/container/deps/requirements.aisimulate.txt
{% endif %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Use Python 3.11 through 3.13. Install AISimulate, which provides the retained `a
sizing command:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```

Collect:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Use Python 3.11 through 3.13.

```bash
# Install the distribution that provides the compatibility command
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"

# Optional: check whether the model/system/backend is covered
aiconfigurator cli support \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,15 +42,15 @@ path. They have no adapter migration.
AISimulate requires Python 3.11 through 3.13. Dynamo itself still supports Python 3.10, but the
`aisimulate` dependency and its CLI are not installed in a Python 3.10 environment.

The `dynamo-planner` image installs the published `aisimulate==0.1.0.dev2` wheel from its local
wheelhouse. Dynamo builds `aisimulate-core==0.1.0-dev.2` from crates.io instead of vendoring the
The `dynamo-planner` image installs the published `aisimulate==0.12.0` wheel from its local
wheelhouse. Dynamo builds `aisimulate-core==0.12.0` from crates.io instead of vendoring the
AISimulate source tree.

For Dynamo source development, install the published AISimulate wheel, Dynamo, and the Planner
dependencies from the Dynamo repository root:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
python3 -m pip install --no-deps -e .
python3 -m pip install -r container/deps/requirements.planner.txt
```
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ AISimulate requires Python 3.11 through 3.13. The `ai-dynamo` package remains in
3.10, but the AISimulate dependency and CLI are not installed there.

The supported prebuilt environment is the `dynamo-planner` image. The image stages and installs the
published `aisimulate==0.1.0.dev2` wheel alongside the Dynamo wheels. Dynamo's Rust workspace
resolves `aisimulate-core==0.1.0-dev.2` from crates.io.
published `aisimulate==0.12.0` wheel alongside the Dynamo wheels. Dynamo's Rust workspace
resolves `aisimulate-core==0.12.0` from crates.io.

For Dynamo source development, install the published AISimulate wheel and build the matching
Dynamo bindings:
Expand All @@ -30,7 +30,7 @@ cd lib/bindings/python
maturin develop --uv --release --features aic-forward-pass
cd ../../..
python3 -m pip install --no-deps -e .
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
python3 -m pip install -r container/deps/requirements.planner.txt
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ provide one config as a strict pin or several configs as a custom projection poo

## Configuration

Use this reference configuration:
The experiment used the following configuration shape. It documents the retired
`aisimulate==0.1.0.dev1` API and is not accepted by AISimulate 0.12.0:

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ subtitle: Experimental Sweeper search over Planner policies for MiniMax-M2.5 on
> treat them as production capacity guidance or a performance commitment. Sweeper's search behavior
> and output may change without a standard deprecation period.

Enable the planner (SLA mode) on the same deployment as the
[router experiment](router-end-to-end-latency-sweep.md), then use Sweeper's smart sweep to **find the
planner configuration that maximizes goodput-per-GPU** on the Mooncake toolagent trace, and
compare against the static deployment and the default planner.
This experiment enabled the planner (SLA mode) on the same deployment as the
[router experiment](router-end-to-end-latency-sweep.md), then used Sweeper's smart sweep to find the
planner configuration that maximized goodput-per-GPU on the Mooncake toolagent trace and compared
it against the static deployment and the default planner.

## Setup

Expand Down Expand Up @@ -50,8 +50,10 @@ scales the fleet to an average of ~13.5 GPUs instead of pinning 32.

## Sweep config

`Sweeper.run`, `goal.target = goodput_per_gpu`. Deployment + router + workload + SLA
fixed; only the planner knobs swept (full trace, open-loop):
The experiment used `Sweeper.run` with `goal.target = goodput_per_gpu`. It fixed the deployment,
router, workload, and SLA, and swept only the planner settings using the full open-loop trace. The
configuration below documents the retired `aisimulate==0.1.0.dev1` API and is not accepted by
AISimulate 0.12.0:

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ subtitle: Run a backend-neutral sweep with an injected replay runtime
Install AISimulate:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```

Sweeper requires a `RunnerFactory` supplied by the application that owns replay execution:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ subtitle: Experimental Sweeper search over KV router settings for MiniMax-M2.5 o
> treat them as production capacity guidance or a performance commitment. Sweeper's search behavior
> and output may change without a standard deprecation period.

Reproduce the DynoSim router experiment, then use Sweeper's smart sweep to **find the
KV-router configuration that minimizes mean end-to-end latency** on the Mooncake toolagent
trace, and validate the winner against the default-router baseline on the full trace.
This experiment reproduced the DynoSim router experiment, then used Sweeper's smart sweep to find
the KV-router configuration that minimized mean end-to-end latency on the Mooncake toolagent trace
and validated the winner against the default-router baseline on the full trace.

## Setup

Expand All @@ -32,9 +32,10 @@ concurrency) so any change in e2e latency is attributable to the **router** alon

## Sweep config

Searched with `Sweeper.run` (Vizier GP-bandit), `goal.target = e2e_latency`. Search
runs on a 2k-request subset (fast, representative — same long-context mix); the winner is
then validated on the full trace.
The experiment searched with `Sweeper.run` (Vizier GP-bandit) and
`goal.target = e2e_latency`. It ran on a 2k-request subset with the same long-context mix, then
validated the winner on the full trace. The configuration below documents the retired
`aisimulate==0.1.0.dev1` API and is not accepted by AISimulate 0.12.0.

```yaml
search_space:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ forward-pass timing model.
retained `aiconfigurator` sizing command:

```bash
python3 -m pip install "aisimulate==0.1.0.dev2"
python3 -m pip install "aisimulate==0.12.0"
```
- Your checkpoint's HuggingFace ID, GPU system (for example `h200_sxm`), backend (`vllm`, `sglang`, or `trtllm`), the total GPUs you can allocate, and your SLA: Time To First Token (TTFT), Time Per Output Token (TPOT), and typical input/output sequence lengths (ISL/OSL).

Expand Down
4 changes: 2 additions & 2 deletions lib/bindings/kvbm/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions lib/bindings/python/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/bindings/python/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ request-trace-s3 = ["dynamo-llm/request-trace-s3"]
# former AIC crate surface. The `python` feature is required for
# AicEngineBuilder; this extension supplies the interpreter, so the dependency
# must not enable `extension-module`.
aisimulate-core = { version = "=0.12.0-dev.2", optional = true, features = ["python"] }
aisimulate-core = { version = "=0.12.0", optional = true, features = ["python"] }
dynamo-runtime = { path = "../../runtime" }
dynamo-mocker = { path = "../../mocker" }
dynamo-parsers = { version = "8.1.0" }
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ license-files = ["LICENSE"]
requires-python = ">=3.10"
dependencies = [
"ai-dynamo-runtime==1.5.0",
"aisimulate==0.12.0.dev2; python_version >= '3.11' and python_version < '3.14'",
"aisimulate==0.12.0; python_version >= '3.11' and python_version < '3.14'",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The frontend image installs this meta wheel with dependency resolution, but the new release's full AISimulate wheel is only staged from NVIDIA's index in wheel_builder for planner/runtime targets; target == "frontend" is excluded. The default PyPI resolver therefore sees only the documented resolver sdist (or no usable binary) when building the frontend for Python 3.11-3.13, causing the frontend image build to fail instead of installing this required dependency. Stage the NVIDIA wheel for the frontend target and make it an input to that image's uv pip install resolution.

🤖 AI Fix

Include the frontend target in AISimulate wheel staging and pass the staged local wheel or wheelhouse as a resolver source when frontend.Dockerfile installs ai-dynamo.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Premise does not hold. Verified by execution. See the reply on the wheel_builder.Dockerfile thread. The public PyPI artifact for aisimulate==0.12.0 is a wheel-stub redirector to https://pypi.nvidia.com/, so the frontend resolver does get the real wheel. A plain pip install aisimulate==0.12.0 against the default index installed the 83 MB abi3 wheel and imported it in a clean Python 3.12 environment. No frontend staging change is needed.

"aiohttp>=3.14.3,<4.0",
"transformers>=4.56.0",
"kubernetes>=32.0.1,<33.0.0",
Expand Down
2 changes: 1 addition & 1 deletion tests/dependencies/test_aiconfigurator_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def test_no_manifest_installs_retired_aic_distributions() -> None:
assert all(package["name"] != "aiconfigurator-core" for package in packages)
assert features["aic-forward-pass"] == ["dep:aisimulate-core"]
assert dependencies["aisimulate-core"] == {
"version": "=0.12.0-dev.2",
"version": "=0.12.0",
"optional": True,
"features": ["python"],
}
Expand Down
3 changes: 3 additions & 0 deletions tests/dependencies/test_aisimulate_consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,10 @@ def test_container_stages_the_published_aisimulate_wheel() -> None:
"--requirement /opt/dynamo/container/deps/requirements.aisimulate.txt"
in wheel_builder
)
assert "--only-binary=:all:" in wheel_builder
assert "--no-deps" in wheel_builder
assert "--no-index" in wheel_builder
assert "--find-links https://pypi.nvidia.com/aisimulate/" in wheel_builder
assert "COPY aisimulate" not in wheel_builder
assert "/opt/dynamo/aisimulate" not in wheel_builder
assert not (ROOT / "aisimulate").exists()
Expand Down
Loading