Skip to content
98 changes: 98 additions & 0 deletions .agents/skills/author-baseline-dgd/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
name: author-baseline-dgd
description: >-
Drafts a candidate baseline DynamoGraphDeployment from interview requirements when no catalog recipe matches the
user's model, hardware, and backend, presenting per-decision evidence for the user's confirmation. Use only from
user-interviewer at interview time, at rung 3 of the baseline-source ladder, and never to deploy or to replace a
baseline the user already provided.
license: Apache-2.0
metadata:
author: NVIDIA
tags:
- dynamo
- workload
- interview
- optimization
---

# Author Baseline DGD

Comment thread
BenHamm marked this conversation as resolved.
<!--
SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

Draft ONE candidate baseline DGD for a greenfield engagement and present it for the user's explicit confirmation.
Do not deploy, benchmark, apply, or record anything as the baseline: an unconfirmed draft is a proposal, and only
the user's confirmation makes it a user-provided baseline.

## Inputs

Require:

- the interview fact table from `synthesize-user-workload` (model source and revision, hardware type and count,
backend and precision preferences, workload shape, SLOs, Kubernetes context and namespace);
- the recipe catalog scan that established rung 3 (no exact or close recipe), including the nearest recipes
considered and why each was rejected as a base; and
- any user-stated constraints (`resources.pinned` candidates, budgets) already collected.

If model identity or hardware type and count is missing, return the question to `user-interviewer` instead of
guessing. Backend is different: when the user explicitly has no preference, CHOOSE it here with evidence - prefer
the backend whose nearest catalog recipe scaffolds this model family and hardware, per the knob guides' coverage -
and record the choice and its evidence in the decision table the user confirms. A confirmed draft's backend is a
confirmed decision, not an invented default; the contract's `preferences.framework` still records only what the
user themselves stated.

## Read The Applicable Knowledge

Always read:

- all three files under `agent-docs/guides/model-sizing/` (memory fit, `min_tp`, classification);
- `agent-docs/guides/knob-tuning/tuning-hierarchy.md`;
- the chosen backend's guide (`agent-docs/guides/knob-tuning/vllm.md`, `sglang.md`, or `tensorrt-llm.md`) -
when choosing the backend here, read the candidates' guides as needed to make the choice;
- `agent-docs/guides/knob-tuning/dynamo.md`; and
- the nearest catalog recipes' manifests, as structural scaffolding only.

Read `agent-docs/guides/rate-matching/matching.md` only when the draft is disaggregated (rare for a baseline;
prefer aggregated unless the user's SLOs demand otherwise).

## Author The Draft

1. **Size the model**: compute weight bytes, `min_tp`, and `headroom_ratio` per `memory.md`, showing the
arithmetic. Choose the serving TP per `parallelism.md` (prefer lower TP and more replicas for throughput
workloads; raise TP above `min_tp` only when headroom demands it, recording the replica cost).
2. **Choose topology conservatively**: an aggregated single-node layout unless the user's hardware or SLOs force
otherwise. The baseline's job is to run and measure, not to win; the optimization loop owns improvement.
3. **Scaffold from the nearest recipe**: copy its structure (components, probes, service wiring, image versions
for the chosen backend) and replace model, parallelism, resources, and any hardware-bound fields, naming every
replacement. Never carry a hardware-bound topology, transport, or checkpoint choice across without evidence it
fits the target.
4. **Set knobs to the backend guide's defaults**, deviating only where the sizing arithmetic requires it
(e.g. `gpu_memory_utilization`, `max_model_len` capped to the workload). Leave optimization headroom alone.
5. **Validate the draft**: parse as YAML, exactly one `DynamoGraphDeployment` document, no secret values, and
confirm it would pass `kubectl apply --dry-run=server` semantics (correct API version, resource names, required
fields) to the extent checkable offline.

## Present For Confirmation

Return to `user-interviewer`, for relay to the user:

- the complete draft manifest;
- a per-decision evidence table: each major choice (TP, replicas, memory settings, backend, image, topology) with
the guide citation or arithmetic that produced it;
- the nearest recipes considered and why each was rejected as a base; and
- the explicit statement that this draft is unvalidated on hardware and iteration 0 will characterize it.

Do not proceed on silence, enthusiasm, or a partial answer: confirmation is the user's explicit acceptance of THIS
manifest (or of it as amended by the user). The confirmed manifest goes to `synthesize-user-workload` for canonical
capture with `deployment.origin: agent-authored` and `deployment.origin_source: inputs/baseline-evidence.md`
(the interviewer writes the evidence table and confirmation there at capture time, per `run-artifacts.md`).

## Do Not

- Deploy, benchmark, or apply anything.
- Record an unconfirmed draft anywhere a downstream role could mistake it for the baseline.
- Author when a user DGD exists (that engagement has a baseline) or when rung 1 or 2 produced a viable base.
- Invent model, hardware, or SLO facts; missing facts return to the interview.
- Embed secret values or Kubernetes `Secret` resources.
3 changes: 3 additions & 0 deletions .agents/skills/perform-adversarial-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ Attack the proposal from these directions:
impact on the primary objective, reversibility, GPU cost, and risk at the target operating region?
- **Attribution**: Does the complete diff express one independently testable knob? For a coupled bundle, is every field
required for one mechanism or supported by prior interaction evidence, with an ablation where needed?
- **Provenance**: When `deployment.origin` is `recipe-confirmed` or `agent-authored`, reject any framing of the
baseline as a production reference; iteration 0 characterizes an unvalidated starting point, and topology
families inherited from it are open questions, not settled decisions.
- **Mechanism**: Does the proposed lever address a plausible reducible gap at the target operating region, or merely
move work that evidence suggests is already bounded? Are internal causes still labeled as hypotheses?
- **Evaluation**: Is the expected effect tied to the primary objective or failed SLO? Does the proposal state what the
Expand Down
23 changes: 16 additions & 7 deletions .agents/skills/synthesize-user-workload/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ not search for or select a recipe, deploy, benchmark, or propose tuning changes.
Require:

- the user's initial optimization request exactly as received;
- the user-provided DGD as an attachment, local file path, or pasted YAML;
- the user-provided DGD as an attachment, local file path, or pasted YAML, when one exists; otherwise the
ladder-confirmed baseline manifest handed over by `user-interviewer` with its confirmation record;
- any attached workload descriptions, traces, or other local file paths;
- an optional caller-supplied `EXP_ID` or `EXP_ROOT`; and
- an existing `user_workload.yaml` only when the user is refining the interview before downstream work begins.
Expand All @@ -40,7 +41,8 @@ final YAML.

Resolve these blocking fields:

- one concrete user-provided YAML document containing a `DynamoGraphDeployment`;
- one concrete baseline `DynamoGraphDeployment`: either user-provided, or produced by the baseline-source ladder
(`agents/user-interviewer/AGENTS.md`) and explicitly confirmed by the user;
- workload profile name, type, and a concrete description of the serving traffic;
- exact model source and revision when the user fixes one, plus the fallback policy when the requested revision
may be unsupported by the available engine or weights (fall back to a named alternative and mark the target
Expand Down Expand Up @@ -110,12 +112,18 @@ Create the canonical baseline input:

- When the user supplies a file, copy its bytes without editing the source or canonical copy.
- When the user pastes YAML, materialize that YAML without changing its configuration.
- Parse the canonical copy as YAML and require at least one mapping document whose `kind` is
`DynamoGraphDeployment`.
- When the baseline came from the ladder (rung 2 adapted recipe or rung 3 authored draft), materialize the exact
manifest the user confirmed - byte-for-byte as confirmed, including any user amendments - and record the
matching `origin` and `origin_source`.
- Parse the canonical copy as YAML and require exactly one mapping document whose `kind` is
`DynamoGraphDeployment`; reject a file containing zero or multiple such documents, since a multi-DGD file leaves
`recipe-deployer` without a deterministic baseline.
- Reject embedded secret values or Kubernetes `Secret` resources; references to pre-existing Secret names are allowed.
- Reject a recipe directory, catalog choice, generated substitute, or inferred default in place of the user's DGD.
A specific manifest the user explicitly presents as their baseline is a user-provided DGD, whatever its origin;
the rejection targets substitutes the USER did not supply.
A specific manifest the user explicitly presents OR confirms as their baseline is a user-provided baseline,
whatever its origin; the rejection targets substitutes the user never confirmed. Record the provenance in the
contract: `deployment.origin` (`user`, `recipe-confirmed`, or `agent-authored`) and `deployment.origin_source`
per the schema.
- Do not patch cluster compatibility or performance settings during capture.
- Compute the canonical copy's SHA256 before writing the workload contract.
- If the DGD contradicts an explicit workload constraint, return the contradiction as a blocking question; do not
Expand All @@ -136,7 +144,8 @@ Follow the schema and rules in `agent-docs/rules/execution/user-workload.md`.
Before finalizing:

1. Preserve all explicit user constraints without rounding or reinterpretation.
2. Record the exact canonical DGD path and SHA256 under `deployment`.
2. Record the exact canonical DGD path and SHA256 under `deployment`, plus `origin` and `origin_source`
(provenance of the confirmed baseline).
2a. Record the user's stated budgets (GPU-hours, wall clock, failed-deploy limit) under `budgets`, verbatim;
leave each `null` when the user declined to state one.
3. Represent permitted unknowns as `null`, `""`, or `[]` according to the schema.
Expand Down
9 changes: 6 additions & 3 deletions .codex/agents/user-interviewer.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,16 @@
# SPDX-License-Identifier: Apache-2.0

name = "user_interviewer"
description = "Act first, capture the user's DGD, and synthesize the canonical workload contract."
description = "Act first, establish the baseline DGD via the baseline-source ladder, and synthesize the canonical workload contract."

developer_instructions = """
Before acting, read agents/user-interviewer/AGENTS.md from the repository root and follow that role contract, including
the skill, documentation, rules, inputs, outputs, and stopping conditions it names.

Operate only as the user interviewer. Use the initial user request and minimal follow-up questions to produce the
canonical user_workload.yaml and immutable inputs/user_provided_dgd.yaml. Capture only the DGD the user supplies; do
not search for, select, generate, or edit a DGD, deploy, benchmark, or generate optimization hypotheses.
canonical user_workload.yaml and immutable inputs/user_provided_dgd.yaml. The baseline comes from the baseline-source
ladder in AGENTS.md: capture the DGD the user supplies; or propose an exact or close catalog recipe (a close match
with an explicit adaptation diff); or invoke author-baseline-dgd to draft one. Every rung requires the user's explicit
confirmation, and only the confirmed manifest is captured, with its origin recorded in the contract. Never deploy,
benchmark, or generate optimization hypotheses, and never record an unconfirmed manifest as the baseline.
"""
23 changes: 16 additions & 7 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@ to it — edit only the canonical copy. Reach for the right group first:

**For deploying and operating Dynamo:**

- `synthesize-user-workload` — interview the user, capture their DGD, and create the canonical workload contract
- `synthesize-user-workload` — interview the user, capture their confirmed baseline DGD, and create the canonical workload contract
- `author-baseline-dgd` — draft a baseline DGD from interview requirements when no recipe matches, for the user's confirmation
- `consult-perf-knowledge` — select one evidence-backed optimization proposal and write its reasoning record
- `create-optimization-hypothesis` — materialize a performance consultation as a challenger-ready DGD draft
- `perform-adversarial-review` — challenge a generated DGD candidate before it consumes GPU time
Expand Down Expand Up @@ -81,17 +82,25 @@ write and requires operator consent. Rules:

## Optimization Role Dispatch

When the first user message starts a new Dynamo recipe optimization run, dispatch `user_interviewer` before any other
specialized role. It must invoke `synthesize-user-workload` and produce a validated
`<EXP_ROOT>/user_workload.yaml` plus an immutable `<EXP_ROOT>/inputs/user_provided_dgd.yaml` copied from the DGD the
user supplied. Do not dispatch `recipe_deployer`, `perf_analyzer`, `hypothesis_generator`, or
When the first user message starts a new Dynamo recipe optimization run, FIRST read
`agent-docs/guides/optimization/optimize-loop.md` end to end - the individual SKILL.md files are auto-discoverable,
but the loop's sequencing, state machine, and stopping rules live only in that guide - then dispatch
`user_interviewer` before any other specialized role. It must invoke `synthesize-user-workload` and produce a validated
`<EXP_ROOT>/user_workload.yaml` plus an immutable `<EXP_ROOT>/inputs/user_provided_dgd.yaml` copied from the baseline the
user supplied or explicitly confirmed. Do not dispatch `recipe_deployer`, `perf_analyzer`, `hypothesis_generator`, or
`hypothesis_challenger` until both exact paths and SHA256 values are available. Pass both inputs directly to
`recipe_deployer`; pass the same immutable workload path and hash to every later role. Do not insert a recipe
exploration or selection step before the baseline deployment.
exploration or selection step after the interview: the baseline-source ladder
(`agents/user-interviewer/AGENTS.md`) is the only place selection or authoring happens, always with the user's
explicit confirmation, and always before the loop starts.

## Long-Running Runs And Harness Compatibility

An optimization loop is long-running, unattended work. An interactive harness ends its turn whenever the agent stops
An optimization loop is long-running, unattended work. Know which harness you are in: in a SINGLE-SHOT harness
(headless `-p`/print mode, one-turn API calls), background-job completion notifications can never reach you - the
session is gone when your turn ends. There, poll synchronously with bounded loops and never park the engagement on
a wake-up you cannot receive; parking is only valid where the harness can re-invoke you (interactive sessions, goal
mode). An interactive harness ends its turn whenever the agent stops
calling tools — a turn that ends on narrated intent ("now I'll test disagg") silently stalls the loop until a human
notices. Two rules:

Expand Down
25 changes: 15 additions & 10 deletions agent-docs/guides/optimization/optimize-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,11 @@ rules:

# Optimize Loop

Use this workflow for an end-to-end Dynamo configuration optimization job. The user supplies the baseline DGD;
`user-interviewer` captures it and hands it directly to `recipe-deployer`. There is no recipe-discovery or
recipe-selection step in this workflow.
Use this workflow for an end-to-end Dynamo configuration optimization job. The baseline DGD comes from the
interview's baseline-source ladder (`agents/user-interviewer/AGENTS.md`): supplied by the user, or a recipe or
authored draft the user explicitly confirmed. `user-interviewer` captures the confirmed baseline and hands it
directly to `recipe-deployer`. Selection and authoring happen only at interview time with user confirmation; the
LOOP itself has no recipe-discovery or recipe-selection step.

When using Codex multi-agent mode, dispatch registered roles through `.codex/config.toml`. Each launcher must read and
follow its corresponding `agents/<role>/AGENTS.md` contract.
Expand All @@ -58,7 +60,8 @@ answers; do not advance the workflow meanwhile.

## 2. Validate The Baseline Handoff

Require the exact `EXP_ROOT`, `user_workload.yaml` path and SHA256, `user_provided_dgd.yaml` path and SHA256, and
Require the exact `EXP_ROOT`, `user_workload.yaml` path and SHA256, `user_provided_dgd.yaml` path and SHA256,
`deployment.origin` (with `origin_source` for non-user origins), and
zero-based iteration `0`. Confirm that the user-provided DGD's model, framework, hardware, precision, and topology do
not contradict the user workload. Do not edit, replace, or select an alternative DGD.

Expand All @@ -69,11 +72,11 @@ Give the exact assigned DGD path and SHA256, `user_workload.yaml` path and SHA25
`user_provided_dgd.yaml`. No role selects or substitutes a baseline. When the user's DGD cannot run on the target as
provided — it targets different hardware, checkpoints, or fabric — the deployer records the blocking
incompatibilities in the deployment ledger and returns them; end the engagement with a report that states each
incompatibility and its evidence, and invite the user to start a new engagement with a target-compatible DGD (a
changed user DGD starts a new experiment, per `synthesize-user-workload`). Do not select a substitute, do not
rewrite the captured baseline, and do not park the run waiting for a new manifest. Greenfield engagements — no user
deployment at all — are likewise not supported by this workflow yet: say so at the interview, point the user at
`recipes/README.md` to pick a starting recipe, and invite them to return with it as their baseline.
incompatibility and its evidence, and invite the user to start a new engagement - with a target-compatible DGD of
their own, or through the baseline-source ladder (rungs 2-3), which may use the incompatibility report as input
evidence (a changed baseline starts a new experiment, per `synthesize-user-workload`). Do not select a substitute, do not
rewrite the captured baseline, and do not park the run waiting for a new manifest. (A greenfield user without any
DGD is handled at the interview by the baseline-source ladder, never here.)
Later iterations use the exact challenger-approved draft. The deployer creates:

```text
Expand All @@ -87,7 +90,9 @@ semantics to hide a deployment failure.
## 4. Configure, Run, And Analyze The Benchmark

Give the successful `DEPLOY_ROOT`, exact `user_workload.yaml` path and SHA256, and current performance question and
target operating region to `perf-analyzer`. For iteration 0, use a baseline-characterization question. For later
target operating region to `perf-analyzer`. For iteration 0, use a baseline-characterization question. When `deployment.origin` is not `user`, iteration 0
is pure characterization: the baseline has no production history, so no result may be framed as an improvement or
regression against it beyond the same-series comparisons the benchmark rules already govern. For later
iterations, use the question approved with the candidate.

- Select or create the benchmark series that best answers the question. Reuse a plan only when it remains fit; write
Expand Down
Loading
Loading