Skip to content

refactor(operator): move Grove reconciliation into its workload program - #12261

Closed
julienmancuso wants to merge 1 commit into
jsm/12035-3from
jsm/12035-4
Closed

refactor(operator): move Grove reconciliation into its workload program#12261
julienmancuso wants to merge 1 commit into
jsm/12035-3from
jsm/12035-4

Conversation

@julienmancuso

@julienmancuso julienmancuso commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

This MR makes groveProgram directly own the complete Grove workload reconciliation sequence.

The Grove program now composes:

  1. Worker-hash migration and unsupported-path rollout handling.
  2. Shared resource and checkpoint input reconciliation.
  3. Restart-state resolution.
  4. Grove render preparation.
  5. PodCliqueSet generation and reconciliation.
  6. Grove scaling.
  7. Model-discovery and component Service reconciliation.
  8. Frontend Ingress and VirtualService reconciliation.
  9. Grove-specific readiness classification.
  10. Shared checkpoint-readiness and scaling-adapter result processing.

The temporary groveReconcileFunc adapter and DynamoGraphDeploymentReconciler.reconcileGroveResources entry point have been removed.

This MR is stacked on the preceding workload-program request/result contract MR.

Motivation

Although the outer controller already selected a complete groveProgram, the program still delegated its entire workload phase back to an injected DynamoGraphDeploymentReconciler.reconcileGroveResources method.

That left Grove control-flow ownership indirect:

DGD controller
    -> selects groveProgram
        -> invokes injected DGD reconciler method
            -> drives Grove reconciliation

After this change, the selected program visibly owns the complete sequence:

DGD controller
    -> selects groveProgram
        -> prepares Grove rendering
        -> reconciles the PodCliqueSet
        -> applies Grove scaling
        -> reconciles networking resources
        -> evaluates Grove readiness

This removes the temporary adapter without introducing provider lifecycle callbacks or a generic reconciliation framework.

Design

Direct GroveProgram ownership

groveProgram.Reconcile continues to own the graph-level Grove composition, including shared inputs, restart resolution, workload reconciliation, and result processing.

Its reconcileWorkloads method now contains the provider-specific Grove sequence directly rather than invoking an injected function.

Dedicated program file

The Grove program implementation now lives in dynamographdeployment_grove_program.go, keeping provider-specific orchestration and readiness logic separate from the common workload-program contract and the component program.

Temporary low-level dependencies

Low-level Grove rendering, scaling, Kubernetes persistence, and compatibility helpers remain on DynamoGraphDeploymentReconciler temporarily.

This MR moves control-flow ownership only. Narrowing those dependencies, extracting the Grove renderer, and localizing watches can happen as focused follow-up changes without combining package movement or renderer redesign with this ownership change.

Status ownership

Status behavior is unchanged:

  • The Grove program accumulates its proposed DGD status in the program result.
  • Grove-specific readiness classification is performed by the Grove program.
  • The outer DGD reconciler remains the only status-subresource writer.
  • The outer reconciler continues to own Ready, ObservedGeneration, common condition projection, and the final Status().Update().

Behavior and compatibility

This is intended to be a behavior-preserving refactor:

  • Grove pathway selection is unchanged.
  • Grove retains its existing unsupported managed-rollout behavior.
  • Shared resource ordering is unchanged.
  • Restart state is still resolved before Grove resource generation.
  • Render preparation still preserves legacy worker selectors.
  • PodCliqueSet reconciliation still precedes Grove scaling.
  • Model-discovery Services, component Services, Ingresses, and VirtualServices are reconciled in the same order.
  • Grove readiness classification and transient read-error propagation are unchanged.
  • Checkpoint startup readiness and scaling-adapter ordering are unchanged.
  • Existing scheduler, queue, topology, naming, and replica behavior are unchanged.
  • Finalizer, watch registration, resource ownership, and status persistence remain with their existing owners.

There are no CRD, rendered-resource, watch-registration, owner-reference, or persisted-state changes.


Open in Devin Review

@julienmancuso
julienmancuso requested a review from a team as a code owner July 28, 2026 13:48
@github-actions github-actions Bot added refactor deployment::k8s Relates to dynamo deployment in kubernetes and removed refactor labels Jul 28, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

Signed-off-by: Julien Mancuso <jmancuso@nvidia.com>
@datadog-official

datadog-official Bot commented Jul 31, 2026

Copy link
Copy Markdown

Tests

🔄 Datadog auto-retried 3 jobs - 2 passed on retry View in Datadog

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: bfff4f8 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deployment::k8s Relates to dynamo deployment in kubernetes refactor size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant