-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat(skills): baseline authoring and recipe confirmation when the user has no existing DGD #13625
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
0385ea2
feat(skills): baseline-source ladder - greenfield and recipe-confirme…
BenHamm 6e49e7b
docs(agents): single-shot harnesses must poll, never park on notifica…
BenHamm 42906bd
docs(agents): require reading optimize-loop.md before first dispatch
BenHamm 3d0ae92
docs(rules): fix self-contradictory request-count example in concurre…
BenHamm f0755b7
docs(rules): remove exception-reading ambiguity in request-count cap
BenHamm f9fa53a
fix(skills): address review findings on baseline capture and Codex in…
BenHamm c3a58dc
docs(agents): clarify that the baseline-only selection constraint doe…
BenHamm File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| --- | ||
| name: author-baseline-dgd | ||
| description: >- | ||
| Drafts a candidate baseline DynamoGraphDeployment from interview requirements when no catalog recipe matches the | ||
| user's model, hardware, and backend, presenting per-decision evidence for the user's confirmation. Use only from | ||
| user-interviewer at interview time, at rung 3 of the baseline-source ladder, and never to deploy or to replace a | ||
| baseline the user already provided. | ||
| license: Apache-2.0 | ||
| metadata: | ||
| author: NVIDIA | ||
| tags: | ||
| - dynamo | ||
| - workload | ||
| - interview | ||
| - optimization | ||
| --- | ||
|
|
||
| # Author Baseline DGD | ||
|
|
||
| <!-- | ||
| SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| SPDX-License-Identifier: Apache-2.0 | ||
| --> | ||
|
|
||
| Draft ONE candidate baseline DGD for a greenfield engagement and present it for the user's explicit confirmation. | ||
| Do not deploy, benchmark, apply, or record anything as the baseline: an unconfirmed draft is a proposal, and only | ||
| the user's confirmation makes it a user-provided baseline. | ||
|
|
||
| ## Inputs | ||
|
|
||
| Require: | ||
|
|
||
| - the interview fact table from `synthesize-user-workload` (model source and revision, hardware type and count, | ||
| backend and precision preferences, workload shape, SLOs, Kubernetes context and namespace); | ||
| - the recipe catalog scan that established rung 3 (no exact or close recipe), including the nearest recipes | ||
| considered and why each was rejected as a base; and | ||
| - any user-stated constraints (`resources.pinned` candidates, budgets) already collected. | ||
|
|
||
| If model identity or hardware type and count is missing, return the question to `user-interviewer` instead of | ||
| guessing. Backend is different: when the user explicitly has no preference, CHOOSE it here with evidence - prefer | ||
| the backend whose nearest catalog recipe scaffolds this model family and hardware, per the knob guides' coverage - | ||
| and record the choice and its evidence in the decision table the user confirms. A confirmed draft's backend is a | ||
| confirmed decision, not an invented default; the contract's `preferences.framework` still records only what the | ||
| user themselves stated. | ||
|
|
||
| ## Read The Applicable Knowledge | ||
|
|
||
| Always read: | ||
|
|
||
| - all three files under `agent-docs/guides/model-sizing/` (memory fit, `min_tp`, classification); | ||
| - `agent-docs/guides/knob-tuning/tuning-hierarchy.md`; | ||
| - the chosen backend's guide (`agent-docs/guides/knob-tuning/vllm.md`, `sglang.md`, or `tensorrt-llm.md`) - | ||
| when choosing the backend here, read the candidates' guides as needed to make the choice; | ||
| - `agent-docs/guides/knob-tuning/dynamo.md`; and | ||
| - the nearest catalog recipes' manifests, as structural scaffolding only. | ||
|
|
||
| Read `agent-docs/guides/rate-matching/matching.md` only when the draft is disaggregated (rare for a baseline; | ||
| prefer aggregated unless the user's SLOs demand otherwise). | ||
|
|
||
| ## Author The Draft | ||
|
|
||
| 1. **Size the model**: compute weight bytes, `min_tp`, and `headroom_ratio` per `memory.md`, showing the | ||
| arithmetic. Choose the serving TP per `parallelism.md` (prefer lower TP and more replicas for throughput | ||
| workloads; raise TP above `min_tp` only when headroom demands it, recording the replica cost). | ||
| 2. **Choose topology conservatively**: an aggregated single-node layout unless the user's hardware or SLOs force | ||
| otherwise. The baseline's job is to run and measure, not to win; the optimization loop owns improvement. | ||
| 3. **Scaffold from the nearest recipe**: copy its structure (components, probes, service wiring, image versions | ||
| for the chosen backend) and replace model, parallelism, resources, and any hardware-bound fields, naming every | ||
| replacement. Never carry a hardware-bound topology, transport, or checkpoint choice across without evidence it | ||
| fits the target. | ||
| 4. **Set knobs to the backend guide's defaults**, deviating only where the sizing arithmetic requires it | ||
| (e.g. `gpu_memory_utilization`, `max_model_len` capped to the workload). Leave optimization headroom alone. | ||
| 5. **Validate the draft**: parse as YAML, exactly one `DynamoGraphDeployment` document, no secret values, and | ||
| confirm it would pass `kubectl apply --dry-run=server` semantics (correct API version, resource names, required | ||
| fields) to the extent checkable offline. | ||
|
|
||
| ## Present For Confirmation | ||
|
|
||
| Return to `user-interviewer`, for relay to the user: | ||
|
|
||
| - the complete draft manifest; | ||
| - a per-decision evidence table: each major choice (TP, replicas, memory settings, backend, image, topology) with | ||
| the guide citation or arithmetic that produced it; | ||
| - the nearest recipes considered and why each was rejected as a base; and | ||
| - the explicit statement that this draft is unvalidated on hardware and iteration 0 will characterize it. | ||
|
|
||
| Do not proceed on silence, enthusiasm, or a partial answer: confirmation is the user's explicit acceptance of THIS | ||
| manifest (or of it as amended by the user). The confirmed manifest goes to `synthesize-user-workload` for canonical | ||
| capture with `deployment.origin: agent-authored` and `deployment.origin_source: inputs/baseline-evidence.md` | ||
| (the interviewer writes the evidence table and confirmation there at capture time, per `run-artifacts.md`). | ||
|
|
||
| ## Do Not | ||
|
|
||
| - Deploy, benchmark, or apply anything. | ||
| - Record an unconfirmed draft anywhere a downstream role could mistake it for the baseline. | ||
| - Author when a user DGD exists (that engagement has a baseline) or when rung 1 or 2 produced a viable base. | ||
| - Invent model, hardware, or SLO facts; missing facts return to the interview. | ||
| - Embed secret values or Kubernetes `Secret` resources. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.