feat: Add new apiVersion with redesigned DGDR by Jont828 · Pull Request #6095 · ai-dynamo/dynamo

Jont828 · 2026-02-09T23:57:37Z

Overview:

Work in progress following this proposal by @hhzhang16 .

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

closes GitHub issue: feat: Implement DGDR v1beta1 spec with field groups schema #6130

Signed-off-by: Jont828 <jt572@cornell.edu>

copy-pr-bot · 2026-02-09T23:57:41Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

github-actions · 2026-02-09T23:57:46Z

👋 Hi Jont828! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

…conditions, and print cols to DGDR status Signed-off-by: Jont828 <jt572@cornell.edu>

hhzhang16 · 2026-02-10T21:44:11Z

+	DGDRPhaseReady     DGDRPhase = "Ready"
+	DGDRPhaseDeploying DGDRPhase = "Deploying"
+	DGDRPhaseDeployed  DGDRPhase = "Deployed"


After looking at this, I think Ready isn't the best name for "successful profiling". Maybe we should change the phases:

Pending

Profiling

ProfilingComplete (this is where it stops with autoApply: false)

Deploying

Deployed (this is where it stops with autoApply: true)

Failed

I think "ProfilingSucceeded" or "ProfilingComplete" is a good name. Should there be a planner phase, or is Deploying/Deployed enough?

It's a DynamoDeployment, may or may not have Planner. So I don't think any phase should explicitly mention planner. Also DynamoDeploymentDeploying sounds kinda weird to me lol

Oh I see, I didn't realize the planner was a component instead of a step in DGDR controller. Should we have some kind of aggregate/overall terminal phase similar to the "succeeded" condition? I think that would make it more clear that it's done and we can check against that value if we're deploying or not.

hhzhang16 · 2026-02-10T21:44:57Z

+	// ConditionTypeSucceeded is the aggregate condition for the DGDR lifecycle.
+	// True = pipeline completed successfully; False = in progress or failed.
+	// Reason and Message reflect the current stage or error.
+	ConditionTypeSucceeded = "Succeeded"


For the records: this will happen if ProfilingComplete and autoApply: false or Deployed and autoApply: true

hhzhang16 · 2026-02-10T21:45:50Z

+	EventReasonValidationFailed     = "ValidationFailed"
+	EventReasonProfilingJobCreated  = "ProfilingJobCreated"
+	EventReasonProfilingJobFailed   = "ProfilingJobFailed"
+	EventReasonAIConfiguratorFailed = "AIConfiguratorFailed"


Not sure if this should be tied into/part of ProfilingJobFailed? Does the user need to know abt AIConfigurator explicitly

hhzhang16 · 2026-02-10T21:46:43Z

+	ProfilingPhaseDone ProfilingPhase = "Done"
+)
+
+// Profiling condition Reasons.


Should these be a bit more explanatory? @tedzhouhk

agreed, is it possible to show more details, like sweeping prefill 2/4

The 'Reason' needs to be a hard coded string. But each condition will have a message too, so I was thinking we could expose it in the message part, that way we don't need to say how many GPUs we're targeting for ex. Does that work?

That works for me

Signed-off-by: Jont828 <jt572@cornell.edu>

Move the +kubebuilder:storageversion marker from v1alpha1 to v1beta1 for all CRDs and regenerate manifests so v1beta1 is the version persisted in etcd going forward. Signed-off-by: Jont828 <jt572@cornell.edu>

julienmancuso

hi @Jont828 what justifies the switch to using a new CRD version ?
internally we are having discussions but we are leaning towards keeping v1alpha1 for now.
let's hold on this change

Implement the controller-runtime conversion interface for 5 CRDs: DynamoCheckpoint, DynamoComponentDeployment, DynamoGraphDeployment, DynamoGraphDeploymentScalingAdapter, and DynamoModel. v1beta1 is marked as the Hub and v1alpha1 implements ConvertTo/ConvertFrom with field-by-field mapping. DGDR is excluded because the two versions have fundamentally different schemas that cannot be losslessly converted. Signed-off-by: Jont828 <jt572@cornell.edu>

tedzhouhk · 2026-02-13T21:26:33Z

currently we have two different definitions on the DGDR fields. One version is defined in this PR, another version is defined for the shim python layer #6285. We need to manually sync between them now. However, we should automate this in the future.

hhzhang16 · 2026-02-14T00:52:31Z

+type OptimizationType string
+
+const (
+	OptimizationTypeHybrid     OptimizationType = "hybrid"


Let's get rid of this option and make "throughput" the default

Align the Go CRD types with the Python profiler's pydantic models so that JSON serialized between operator and profiler is structurally consistent: - Introduce ModelSpec (modelName + modelCache) replacing flat model string and top-level modelCache field - Introduce BackendSpec (backend + dynamoImage) replacing flat backend enum and top-level image field - Rename ModelCacheSpec.PVCPath to ModelPathInPVC (json: modelPathInPvc) - Add WorkloadSpec.Concurrency and RequestRate fields - Add SLASpec.E2ELatency field - Replace FeaturesSpec.Planner *bool with *PlannerSpec (enabled, plannerPreDeploymentSweeping, plannerArgsList) - Replace FeaturesSpec.Mocker *bool with *MockerSpec (enabled) - Add HardwareSpec with gpuSku, vramMb, totalGpus, numGpusPerNode - Add PlannerPreDeploymentSweepMode enum (none, rapid, thorough) - Remove OptimizationType "hybrid" value - Update printcolumn JSONPaths for new nesting - Update deepcopy methods for new and changed types Signed-off-by: Jonathan Zhou <hongkuanz@nvidia.com> Signed-off-by: Jont828 <jt572@cornell.edu>

Jont828 · 2026-02-14T01:58:39Z

@tedzhouhk I pushed some changes to sync it based on your Python code. Here's a summary of what I changed. Does this look right to you? There's some Go types that aren't in the Python for your PR so not sure if you want to add them or not.

  Go struct changes (dynamographdeploymentrequest_types.go)

  New struct: ModelSpec
  Wraps the model name and cache into a single nested object, replacing the old flat Model string + top-level ModelCache *ModelCacheSpec.
  ┌────────────┬────────────┬────────────────────────────┐
  │   Field    │  JSON tag  │            Type            │
  ├────────────┼────────────┼────────────────────────────┤
  │ ModelName  │ modelName  │ string (required)          │
  ├────────────┼────────────┼────────────────────────────┤
  │ ModelCache │ modelCache │ *ModelCacheSpec (optional) │
  └────────────┴────────────┴────────────────────────────┘
  → Python counterpart: class ModelSpec(BaseModel) — model_name: str, model_cache: Optional[ModelCacheSpec]

  ---
  New struct: BackendSpec
  Wraps the backend enum and image into a single nested object, replacing the old flat Backend BackendType + Image string.
  ┌─────────────┬─────────────┬─────────────────────────────┐
  │    Field    │  JSON tag   │            Type             │
  ├─────────────┼─────────────┼─────────────────────────────┤
  │ Backend     │ backend     │ BackendType (default: auto) │
  ├─────────────┼─────────────┼─────────────────────────────┤
  │ DynamoImage │ dynamoImage │ string (optional)           │
  └─────────────┴─────────────┴─────────────────────────────┘
  → Python counterpart: class BackendSpec(BaseModel) — backend: Backend, dynamo_image: str

  ---
  Changed struct: ModelCacheSpec
  Renamed one field:
  ┌────────────────┬───────────────────────┬──────────────────────────┐
  │     Before     │         After         │         JSON tag         │
  ├────────────────┼───────────────────────┼──────────────────────────┤
  │ PVCPath string │ ModelPathInPVC string │ pvcPath → modelPathInPvc │
  └────────────────┴───────────────────────┴──────────────────────────┘
  Other fields (PVCName, PVCMountPath) unchanged.

  → Python counterpart: class ModelCacheSpec(BaseModel) — pvc_name, model_path_in_pvc, pvc_mount_path

  ---
  Changed struct: DynamoGraphDeploymentRequestSpec
  ┌────────────────────────────────────────┬──────────────────────────────────────────────┐
  │                 Before                 │                    After                     │
  ├────────────────────────────────────────┼──────────────────────────────────────────────┤
  │ Model string (flat)                    │ Model ModelSpec (nested struct)              │
  ├────────────────────────────────────────┼──────────────────────────────────────────────┤
  │ Backend BackendType (flat enum)        │ Backend *BackendSpec (nested struct)         │
  ├────────────────────────────────────────┼──────────────────────────────────────────────┤
  │ Image string (flat)                    │ Removed — now inside BackendSpec.DynamoImage │
  ├────────────────────────────────────────┼──────────────────────────────────────────────┤
  │ ModelCache *ModelCacheSpec (top-level) │ Removed — now inside ModelSpec.ModelCache    │
  └────────────────────────────────────────┴──────────────────────────────────────────────┘
  All other fields unchanged: Hardware, Workload, SLA, Overrides, Features, SearchStrategy, AutoApply.

  → Python counterpart: class AugmentedDGDR(BaseModel) — model: ModelSpec, backend: BackendSpec, hardware, workload, sla, features, search_strategy, auto_apply

  ---
  Updated printcolumn JSONPaths:
  - .spec.model → .spec.model.modelName
  - .spec.backend → .spec.backend.backend

  ---
  Updated zz_generated.deepcopy.go:
  - DynamoGraphDeploymentRequestSpec.DeepCopyInto updated for new Model/Backend field shapes
  - Added ModelSpec.DeepCopyInto / DeepCopy
  - Added BackendSpec.DeepCopyInto / DeepCopy

  ---
  Unchanged structs (already in sync)
  ┌──────────────┬──────────────┬──────────────────────────────────────────────────────────────────────┐
  │  Go struct   │ Python class │                                Fields                                │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ HardwareSpec │ HardwareSpec │ gpuSku, vramMb, totalGpus, numGpusPerNode                            │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ WorkloadSpec │ WorkloadSpec │ isl, osl, concurrency, requestRate                                   │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ SLASpec      │ SLASpec      │ ttft, itl, e2eLatency (Go also has optimizationType — not in Python) │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ PlannerSpec  │ PlannerSpec  │ enabled, plannerPreDeploymentSweeping, plannerArgsList               │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ MockerSpec   │ MockerSpec   │ enabled                                                              │
  ├──────────────┼──────────────┼──────────────────────────────────────────────────────────────────────┤
  │ FeaturesSpec │ FeaturesSpec │ planner, mocker (Go also has kvRouter — not in Python)               │
  └──────────────┴──────────────┴──────────────────────────────────────────────────────────────────────┘
  Go-only types (no Python counterpart yet)

  - OverridesSpec (profilingJob, dgd)
  - SLASpec.OptimizationType (latency/throughput)
  - FeaturesSpec.KVRouter (*bool)

Add v1alpha1 ↔ v1beta1 conversion for DynamoGraphDeploymentRequest (DGDR) and split the monolithic conversion.go into per-type files matching the existing *_types.go naming convention. Hub marker: add DynamoGraphDeploymentRequest to v1beta1/hub.go. DGDR conversion handles three categories of field mappings: - Simple renames (Model, Backend, AutoApply, UseMocker, WorkersImage) - JSON blob ↔ structured fields (SLA ttft/itl, Workload isl/osl, ModelCache) with full-blob annotation for round-trip preservation - Annotation-based storage for v1alpha1 fields with no v1beta1 equivalent (ProfilerImage, ConfigMapRef, OutputPVC, etc.) State ↔ Phase mapping accounts for v1alpha1 "Ready" mapping to either Ready or Deployed depending on Deployment.Created context. Signed-off-by: Jont828 <jt572@cornell.edu>

hhzhang16 · 2026-02-20T17:18:10Z

Closing as covered in #6352

Jont828 added 2 commits February 9, 2026 18:48

Add initial DGDR type in v1beta1 apiVersion

fabaae5

Signed-off-by: Jont828 <jt572@cornell.edu>

Add default value types

1d7fed6

Signed-off-by: Jont828 <jt572@cornell.edu>

pull-request-size Bot added the size/L label Feb 9, 2026

github-actions Bot added the feat label Feb 9, 2026

github-actions Bot added external-contribution Pull request is from an external contributor deployment::k8s Relates to dynamo deployment in kubernetes labels Feb 9, 2026

Add 'Succeeded' condition type, profiling phase, and profiling phase …

26538c8

…conditions, and print cols to DGDR status Signed-off-by: Jont828 <jt572@cornell.edu>

pull-request-size Bot added size/XL and removed size/L labels Feb 10, 2026

hhzhang16 reviewed Feb 10, 2026

View reviewed changes

Copy over remaining CRDs from v1alpha1

50820f5

Signed-off-by: Jont828 <jt572@cornell.edu>

pull-request-size Bot added size/XXL and removed size/XL labels Feb 12, 2026

Set v1beta1 as storage version for all CRDs

fbfa59f

Move the +kubebuilder:storageversion marker from v1alpha1 to v1beta1 for all CRDs and regenerate manifests so v1beta1 is the version persisted in etcd going forward. Signed-off-by: Jont828 <jt572@cornell.edu>

julienmancuso reviewed Feb 12, 2026

View reviewed changes

hhzhang16 mentioned this pull request Feb 13, 2026

refactor: move core logics of DPP -> AIC and support static profiling #6285

Merged

hhzhang16 reviewed Feb 14, 2026

View reviewed changes

Jont828 force-pushed the v1beta1-apiversion branch from 846fefd to e824985 Compare February 14, 2026 02:26

Jont828 force-pushed the v1beta1-apiversion branch from e824985 to ae509a8 Compare February 14, 2026 02:29

hhzhang16 closed this Feb 20, 2026

Uh oh!

Conversation

Jont828 commented Feb 9, 2026 • edited by hhzhang16 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

Uh oh!

copy-pr-bot Bot commented Feb 9, 2026

Uh oh!

github-actions Bot commented Feb 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

julienmancuso left a comment

Choose a reason for hiding this comment

Uh oh!

tedzhouhk commented Feb 13, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jont828 commented Feb 14, 2026

Uh oh!

hhzhang16 commented Feb 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Jont828 commented Feb 9, 2026 •

edited by hhzhang16

Loading