[sachalmalick]

This adds an opt-in single-pool batch scheduling configuration and a pure policy that returns an active-job replica floor, a safety-gated exact idle target, and a leased admission cap. It lets Planner reserve online capacity, reason about deadline pressure, and fail closed when required observations are missing, stale, or unsafe during startup.

CLOSES: LLM-184

How This Was Implemented

• Add strict nested configuration for Gateway, metrics, Redis actuation, pool capacity, margins, observation age, lease duration, and replica bounds.
• Keep Redis credentials secret-backed and excluded from serialized Planner configuration.
• Compute cumulative earliest-deadline-first demand bounds and convert them into a required replica floor.
• Limit batch admission to capacity remaining after online offered load while coupling the cap to a floor that can sustain it.
• Account separately for ready and verified-pending startup capacity, emitting zero-rate leases and diagnostics when safe capacity cannot be established.
• Treat pool.min_replicas as both the active-job floor and configurable post-batch warm target; emit an exact target only when Gateway, frontend, Async, and stable worker inventory are freshly idle, otherwise retain verified desired capacity as a floor.

Walkthrough

• config/planner_config.py validates deployment compatibility, URLs, labels, lease timing, secrets, and pool bounds before enabling the feature.
• config/batch_policy_config.py defines the policy-facing immutable configuration.
• core/batch_policy.py maps one TickInput snapshot to an active floor, safety-gated exact idle target, drain lease, deadline estimates, and diagnostics.
• tests/config/test_batch_scheduling_config.py covers disabled defaults, dependency requirements, secret handling, and invalid combinations.
• tests/unit/test_batch_policy.py covers EDF demand, headroom, startup capacity, floors, ceilings, stale data, pauses, and infeasibility reporting.

Validation

• Final stacked validation: 939 unique tests passed, including policy/config and zero/one warm-target regressions.
• uvx pre-commit run --from-ref origin/main --to-ref HEAD and git diff --check passed.

Summary by CodeRabbit

• New Features
  • Added configurable batch scheduling for Kubernetes aggregate deployments, including controls for replica limits, startup behavior, and admission rates.
  • Batch scheduling now adjusts capacity and admission based on workload demand, deadlines, and available capacity.
  • Scheduling pauses admission when required observations are missing, stale, or invalid, helping prevent unsafe workload dispatch.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: ai-dynamo/dynamo/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 1f9cfe12-b9ab-4026-937b-f6a1683fbc1e

📥 Commits

Reviewing files that changed from the base of the PR and between d19fa71 and 21b0a4b.

📒 Files selected for processing (6)

• components/src/dynamo/planner/config/batch_policy_config.py
• components/src/dynamo/planner/config/planner_config.py
• components/src/dynamo/planner/core/__init__.py
• components/src/dynamo/planner/core/batch_policy.py
• components/src/dynamo/planner/tests/config/test_batch_scheduling_config.py
• components/src/dynamo/planner/tests/unit/test_batch_policy.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The planner gains validated batch scheduling configuration and a policy that evaluates observations and job demand. The policy returns replica-floor and leased admission recommendations with diagnostics. New tests cover configuration validation and policy behavior.

Changes

Batch scheduling

Layer / File(s)	Summary
Batch scheduling configuration components/src/dynamo/planner/config/batch_policy_config.py, components/src/dynamo/planner/config/planner_config.py, components/src/dynamo/planner/tests/config/test_batch_scheduling_config.py	Adds validated policy and planner configuration for gateway observations, metrics, Redis actuation, and pool capacity. Planner validation checks mode, required settings, replica limits, and drain-lease duration. Configuration tests cover defaults, validation, and policy conversion.
Snapshot interpretation and safety decisions components/src/dynamo/planner/core/batch_policy.py, components/src/dynamo/planner/core/__init__.py, components/src/dynamo/planner/tests/unit/test_batch_policy.py	Adds snapshot evaluation for matching traffic, dispatcher feedback, active jobs, replica capacity, and observation freshness. Missing or invalid inputs can pause admission; an empty active-job set returns no replica opinion and zero admission. Exports the policy API and tests observation, safety, and bootstrap cases.
Deadline demand and admission planning components/src/dynamo/planner/core/batch_policy.py, components/src/dynamo/planner/tests/unit/test_batch_policy.py	Calculates cumulative deadline demand, derives a bounded replica floor, and limits drain admission to configured caps and planned capacity. Adds deadline, admission, feasibility, and dispatcher-feedback tests.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 21b0a

The batch policy reserves observed online capacity before admitting batch work. No concrete issue remains that requires a fix before merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 3.49% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 86 functions across 6 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely identifies the main change: adding a batch scheduling policy and configuration to the planner.
Description check	✅ Passed	The description explains the change, implementation details, reviewer starting points, and validation results. It also references issue LLM-184, so the required related-issue information is present.

• Fix all pre-merge checks with AI

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

✅ Dynamo PR CI passed — run 38018054487 (attempt 1) on 90442b881d

Gate checks: ✅ backend-status-check · ✅ deploy-status-check · ✅ dynamo-status-check

_{Posted automatically by Devin for run 38018054487. Updated on every full-CI run of this PR.}

[tedzhouhk]

Approved for the functional end-to-end scope of this batch scheduling stack. Reviewed the policy/configuration boundary. Automatic capacity estimation from the performance model remains the agreed follow-up.

[sachalmalick]

/ok to test 81a8c37

[sachalmalick]

/ok to test aa05456

[sachalmalick]

/ok to test 12030da

[sachalmalick]

/ok to test 90442b8