Skip to content

feat(deployments): scaffold plugin API and registry (AIRCORE-755) - #280

Merged
tylersbray merged 8 commits into
mainfrom
755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray
Jun 22, 2026
Merged

feat(deployments): scaffold plugin API and registry (AIRCORE-755)#280
tylersbray merged 8 commits into
mainfrom
755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray

Conversation

@tylersbray

@tylersbray tylersbray commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Scaffolds nemo-deployments-plugin as the substrate-agnostic contract layer for AIRCORE-755:

  • Entities: DeploymentConfig, Deployment, Volume plus RFC supporting types
  • API v1: CRUD for configs/deployments/volumes at /apis/deployments/v1/workspaces/{workspace}/...
  • Reconciler hooks: bulk status_in list filter; controller-only PUT .../status (service-principal gated via X-NMP-Principal-Id)
  • Backend contract: DeploymentBackend ABC + BackendStatusUpdate / VolumeStatusUpdate
  • Executor registry: named-instance pattern; plugin starts cleanly with zero backends registered
  • Validation: prerequisite cycle detection (paginated config graph)

No substrate backends or reconcile controller in this PR — those are AIRCORE-756/757/758.

Test plan

  • uv run pytest plugins/nemo-deployments/tests/unit -v (31 tests)
  • uv run ruff check plugins/nemo-deployments
  • uv run ty check plugins/nemo-deployments
  • Pre-commit hooks pass (including no-nmp-common-in-plugins)
  • Manual smoke: nemo services run → CRUD against /apis/deployments/v1/... (follow-on if desired)

Summary by CodeRabbit

Release Notes

  • New Features

    • Added the NeMo Deployments plugin for end-to-end deployment lifecycle management (configs, deployments, and volumes).
    • Introduced v2 REST API endpoints for full CRUD plus controller-only status updates.
    • Added support for multiple backend executors and backend-driven create/read/delete and logging.
    • Implemented prerequisite dependency validation (cycle detection) and referential integrity checks for safe deletes.
    • Added pagination, sorting, and filtering (including deployment status filtering).
  • Tests

    • Added unit tests covering API behavior, entity defaults/validation, reference resolution, and backend registry logic.
  • Documentation

    • Added plugin README with high-level lifecycle overview.
  • Chores

    • Enabled and wired the plugin into the workspace build and test discovery.

@tylersbray
tylersbray requested review from a team as code owners June 11, 2026 19:14
@github-actions github-actions Bot added the feat label Jun 11, 2026
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py Fixed
@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 21176/27762 76.3% 61.2%
Integration Tests 12215/26531 46.0% 19.5%

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds the nemo-deployments-plugin Python package under plugins/nemo-deployments. The plugin defines Pydantic entity schemas (DeploymentConfig, Deployment, Volume), FastAPI v2 CRUD routes, a DeploymentBackend ABC with an ExecutorRegistry, referential integrity helpers, prerequisite cycle detection, and a DeploymentsService entry point. Registers the plugin in the root workspace and pytest discovery.

Changes

NeMo Deployments Plugin

Layer / File(s) Summary
Constants, shared types, and plugin config
plugins/nemo-deployments/src/nemo_deployments_plugin/constants.py, plugins/nemo-deployments/src/nemo_deployments_plugin/types.py, plugins/nemo-deployments/src/nemo_deployments_plugin/config.py
Defines entity-type constants, Literal status/state type aliases, Endpoint model, and DeploymentsConfig with ExecutorConfigEntry.
Entity schemas, API schemas, and prerequisite validation
plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py, plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py, plugins/nemo-deployments/src/nemo_deployments_plugin/validation.py, plugins/nemo-deployments/tests/unit/test_entities.py, plugins/nemo-deployments/tests/unit/test_prerequisite_validation.py
Defines K8s-like container/volume/probe primitives, NemoEntity subclasses, CRUD request/filter schemas, DFS prerequisite cycle detection, and unit tests.
Backend interface and executor registry
plugins/nemo-deployments/src/nemo_deployments_plugin/backends/base.py, plugins/nemo-deployments/src/nemo_deployments_plugin/backends/registry.py, plugins/nemo-deployments/tests/unit/test_registry.py
Adds DeploymentBackend ABC with deployment/volume/log abstract methods, ExecutorRegistry with rollback on partial init, and registry unit tests.
Referential integrity helpers
plugins/nemo-deployments/src/nemo_deployments_plugin/references.py, plugins/nemo-deployments/tests/unit/test_references.py
Paginating helpers to find deployments referencing a config and configs referencing a volume, with unit tests for direct and container-level mounts.
Service-principal guard and status update API
plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/dependencies.py, plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/status.py, plugins/nemo-deployments/tests/unit/test_api_status.py
require_service_principal dependency enforcing service: header prefix, deployment/volume status PUT endpoints, and 403/200/404 unit tests.
DeploymentConfig CRUD API
plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/deployment_configs.py, plugins/nemo-deployments/tests/unit/test_api_deployment_configs.py
POST validates prerequisite cycles before creation; DELETE checks referencing deployments; full unit test coverage including cycle, conflict, and 404 cases.
Deployment and Volume CRUD APIs
plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/deployments.py, plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/volumes.py, plugins/nemo-deployments/tests/unit/helpers.py, plugins/nemo-deployments/tests/unit/test_api_deployments.py, plugins/nemo-deployments/tests/unit/test_api_volumes.py
Deployment routes with status_in filtering and DELETING transition on delete; volume routes with referential conflict guard; test helpers and endpoint unit tests.
Service wiring, packaging, and workspace enablement
plugins/nemo-deployments/src/nemo_deployments_plugin/service.py, plugins/nemo-deployments/pyproject.toml, pyproject.toml, pytest.ini, plugins/nemo-deployments/README.md, plugins/nemo-deployments/tests/unit/test_service_startup.py
DeploymentsService startup/shutdown wiring executor registry; plugin package metadata and nemo.services entry point; root workspace and pytest registration; README and route-mount tests.

Suggested reviewers

  • mckornfield
  • gabwow
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 9.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Title clearly describes the main change—scaffolding the deployments plugin with API and registry—and references the issue ticket. Concise and specific.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

🧹 Nitpick comments (3)
plugins/nemo-deployments/README.md (1)

3-4: 💤 Low value

Clarify "deployment lifecycle" with concrete scope.

Replace the vague term "deployment lifecycle" with specific capabilities: e.g., "Create, read, update, and delete deployment configurations and active deployments; manage volumes; track deployment status."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-deployments/README.md` around lines 3 - 4, Update the README
description to replace the vague phrase "deployment lifecycle" with a concrete
list of supported capabilities: state that the plugin provides
create/read/update/delete (CRUD) for deployment configurations and active
deployments, volume management, deployment status tracking, entity schemas, CRUD
APIs, a DeploymentBackend abstract base class, and an executor registry; ensure
the one-line summary in the README (top paragraph) explicitly enumerates these
capabilities so readers immediately see the plugin's scope.

Source: Coding guidelines

plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py (1)

243-247: ⚡ Quick win

No-op validator is misleading.

This @model_validator does not validate anything and always returns self, so it implies an invariant that is not enforced. Convert this to a plain code comment/doc note, or implement an actual check where both Deployment and its DeploymentConfig.restart_policy are available.

Suggested cleanup
-    `@model_validator`(mode="after")
-    def _document_status_restart_policy_consistency(self) -> Deployment:
-        """Document restart_policy vs terminal status expectations for the reconciler."""
-        # Reconciler enforces: Never → SUCCEEDED terminal; Always/OnFailure → READY while running.
-        return self
+    # Restart-policy/terminal-status consistency is enforced by the reconciler
+    # when DeploymentConfig is resolved for this Deployment.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py` around
lines 243 - 247, The _document_status_restart_policy_consistency method is a
no-op model_validator that misleads readers; either remove the `@model_validator`
and convert the docstring into a plain class/module comment, or implement an
actual validation that accesses self.status and
self.deployment_config.restart_policy (or DeploymentConfig.restart_policy) and
raises a ValueError/ValidationError when the pair is inconsistent (e.g.,
restart_policy == "Never" requires terminal SUCCEEDED state, restart_policy in
{"Always","OnFailure"} requires non-terminal/READY while running). Update or
remove the decorator accordingly and keep the descriptive text explaining the
reconciler expectations.
plugins/nemo-deployments/tests/unit/test_registry.py (1)

35-53: ⚡ Quick win

Use concrete return types in _StubBackend instead of Any.

These methods can return concrete LogResult/VolumeStatusUpdate types directly; keeping Any here weakens contract checking in tests.

As per coding guidelines, "Always prefer concrete type hints over string-based ones; do not import types under TYPE_CHECKING, instead prefer regular imports when possible."

Proposed diff
 from typing import Any
 
 import pytest
-from nemo_deployments_plugin.backends.abc import BackendStatusUpdate, DeploymentBackend
+from nemo_deployments_plugin.backends.abc import (
+    BackendStatusUpdate,
+    DeploymentBackend,
+    LogResult,
+    VolumeStatusUpdate,
+)
@@
-    async def get_logs(self, **kwargs: Any) -> Any:
-        from nemo_deployments_plugin.backends.abc import LogResult
-
+    async def get_logs(self, **kwargs: Any) -> LogResult:
         return LogResult(lines=[])
@@
-    async def create_volume(self, **kwargs: Any) -> Any:
-        from nemo_deployments_plugin.backends.abc import VolumeStatusUpdate
-
+    async def create_volume(self, **kwargs: Any) -> VolumeStatusUpdate:
         return VolumeStatusUpdate(status="PENDING")
@@
-    async def read_volume_status(self, **kwargs: Any) -> Any:
-        from nemo_deployments_plugin.backends.abc import VolumeStatusUpdate
-
+    async def read_volume_status(self, **kwargs: Any) -> VolumeStatusUpdate:
         return VolumeStatusUpdate(status="BOUND")
@@
-    async def delete_volume(self, workspace: str, name: str) -> Any:
-        from nemo_deployments_plugin.backends.abc import VolumeStatusUpdate
-
+    async def delete_volume(self, workspace: str, name: str) -> VolumeStatusUpdate:
         return VolumeStatusUpdate(status="RELEASED")
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-deployments/tests/unit/test_registry.py` around lines 35 - 53,
Update _StubBackend methods to use concrete return types instead of Any: change
signatures of get_logs to return nemo_deployments_plugin.backends.abc.LogResult
and create_volume/read_volume_status/delete_volume to return
nemo_deployments_plugin.backends.abc.VolumeStatusUpdate; add regular imports for
LogResult and VolumeStatusUpdate at top of the test file (do not use
TYPE_CHECKING or string types) and return the concrete instances as already
constructed in the method bodies so the type hints match the returned objects.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@plugins/nemo-deployments/pyproject.toml`:
- Around line 7-11: The dependencies list in pyproject.toml for the
nemo-deployments plugin is missing fastapi, causing runtime ModuleNotFoundError
when plugin API modules import it; update the [dependencies] block (the
dependencies = [...] list in pyproject.toml for the nemo-deployments package) to
include "fastapi" (optionally with a suitable version constraint, e.g. >=0.95)
so fastapi is installed at runtime alongside "nemo-platform",
"nemo-platform-plugin", and "pydantic>=2.10.6".

In `@plugins/nemo-deployments/README.md`:
- Around line 1-5: Add a top-level "Prerequisites" section before the existing
"# NeMo Deployments Plugin" content that lists required software and setup steps
(e.g., supported Python version, required package manager such as uv, NeMo
Platform runtime availability/configuration, any required environment variables
or credentials, and optional tooling like Docker/uv CLI if applicable); update
README.md to include this section title "Prerequisites" and concise bullet
points so readers know what must be installed or configured before continuing.
- Around line 15-18: Add a "Next Steps" section at the end of
plugins/nemo-deployments/README.md titled "Next Steps" that provides cross-links
to the plugin's API reference, the backend implementation guide (reference
AIRCORE-756, AIRCORE-757, AIRCORE-758), and any other related plugin
documentation; place it immediately after the setup/test commands shown (the uv
sync / pytest block), include one-line descriptions for each link (what the
reader will find there), and ensure links are relative or to the canonical docs
site consistent with other plugin READMEs.
- Around line 1-18: The README mixes reference, how-to, and explanatory content;
split it into a single-quadrant page or reorganize with progressive disclosure.
Choose either: (A) convert this README into a HOW-TO "Set up and test the
plugin" that keeps the uv sync/pytest commands and a brief 30s summary, then
move the API base path and architecture/Backend/DeploymentBackend ABC details
into separate REFERENCE and EXPLANATION pages; or (B) expand this README into
layered sections (Layer 1: 30s summary of purpose/value and intended audience;
Layer 2: 3–5min core concepts and mention of DeploymentBackend and executor
registry; Layer 3: 10min+ API endpoints with the
`/apis/deployments/v1/workspaces/{workspace}/...` example and cross-workspace
sentinel `-`; Layer 4: links to separate reference pages for full API spec and
schema). Update headings accordingly (e.g., "How to set up and test",
"Concepts", "API reference", "Further reading") and move details about the
DeploymentBackend ABC and executor registry into dedicated files or sections to
avoid mixing quadrants.

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/dependencies.py`:
- Around line 17-24: The code currently lets X-NMP-Principal-On-Behalf-Of
override X-NMP-Principal-Id via _effective_principal_id, allowing spoofing;
change require_service_principal to read the principal directly from
request.headers.get(_PRINCIPAL_ID_HEADER, "") (or alter _effective_principal_id
to not consult _ON_BEHALF_OF_HEADER) and perform the startswith("service:")
check against that trustworthy value (use the symbols _PRINCIPAL_ID_HEADER,
_ON_BEHALF_OF_HEADER, _effective_principal_id, and require_service_principal to
locate and update the logic).

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/deployments.py`:
- Around line 140-143: The update call to entity_client.update(deployment) can
raise NemoEntityConflictError during concurrent writes; modify the try/except
around that call to also catch NemoEntityConflictError (in addition to
NemoEntityNotFoundError) and return an HTTP 409 conflict response (matching the
status-update endpoints) with a concise message including the deployment name,
while keeping the existing logger.info for NemoEntityNotFoundError; locate the
entity_client.update(deployment) call and add the new except
NemoEntityConflictError branch to produce the 409 response.

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/backends/registry.py`:
- Around line 56-62: The registry construction loop in the classmethod that
builds executors is not failure-atomic: if one backend instantiation
(classes[spec.backend](sdk, spec.config)) raises, previously-created backend
instances remain running. Wrap the executor creation loop in a try/except, and
on any exception iterate over the already-created executors (executors.values())
and call their shutdown/close method (e.g., shutdown() or close(), whichever the
backend implements) inside its own try/except to swallow secondary errors, then
re-raise the original exception; keep the existing UnknownBackendTypeError check
and final return of cls(executors, default_executor=default_executor).

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/config.py`:
- Around line 32-33: Add a validation step to the config model that defines
port_range_start and port_range_end to enforce 1 <= port_range_start <= 65535, 1
<= port_range_end <= 65535 and port_range_start <= port_range_end at load time;
implement this as a Pydantic validator (e.g., a `@root_validator` or two
`@validator` methods) on the config class that contains the port_range_start and
port_range_end fields so it raises a clear ValueError when bounds or ordering
are invalid, preventing downstream runtime failures.

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py`:
- Line 77: The restart_policy field in the schema currently uses a loose type
(restart_policy: str | None = None); change it to use the RestartPolicy enum
type (e.g., restart_policy: RestartPolicy | None = None) so only valid policies
are accepted; import or reference the RestartPolicy enum (or class) used
elsewhere in the repo and update any related validation or type annotations in
the same module (schema.py) to use RestartPolicy instead of raw str.

In `@plugins/nemo-deployments/tests/unit/test_api_volumes.py`:
- Around line 32-39: The test should assert the object passed into
mock_entity_client.create to ensure the route sets Volume.status="PENDING"
before persisting; in test_create_volume_201 capture the awaited call to
mock_entity_client.create (e.g., inspect mock_entity_client.create.await_args or
use assert_awaited_once and read call args) and assert the passed volume object
has status == "PENDING" (and optionally name/size match the request) rather than
only asserting the mocked return value.

---

Nitpick comments:
In `@plugins/nemo-deployments/README.md`:
- Around line 3-4: Update the README description to replace the vague phrase
"deployment lifecycle" with a concrete list of supported capabilities: state
that the plugin provides create/read/update/delete (CRUD) for deployment
configurations and active deployments, volume management, deployment status
tracking, entity schemas, CRUD APIs, a DeploymentBackend abstract base class,
and an executor registry; ensure the one-line summary in the README (top
paragraph) explicitly enumerates these capabilities so readers immediately see
the plugin's scope.

In `@plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py`:
- Around line 243-247: The _document_status_restart_policy_consistency method is
a no-op model_validator that misleads readers; either remove the
`@model_validator` and convert the docstring into a plain class/module comment, or
implement an actual validation that accesses self.status and
self.deployment_config.restart_policy (or DeploymentConfig.restart_policy) and
raises a ValueError/ValidationError when the pair is inconsistent (e.g.,
restart_policy == "Never" requires terminal SUCCEEDED state, restart_policy in
{"Always","OnFailure"} requires non-terminal/READY while running). Update or
remove the decorator accordingly and keep the descriptive text explaining the
reconciler expectations.

In `@plugins/nemo-deployments/tests/unit/test_registry.py`:
- Around line 35-53: Update _StubBackend methods to use concrete return types
instead of Any: change signatures of get_logs to return
nemo_deployments_plugin.backends.abc.LogResult and
create_volume/read_volume_status/delete_volume to return
nemo_deployments_plugin.backends.abc.VolumeStatusUpdate; add regular imports for
LogResult and VolumeStatusUpdate at top of the test file (do not use
TYPE_CHECKING or string types) and return the concrete instances as already
constructed in the method bodies so the type hints match the returned objects.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 13375584-28cd-4917-9bf5-7df481d974cb

📥 Commits

Reviewing files that changed from the base of the PR and between c2dd51d and 212e31e.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (25)
  • plugins/nemo-deployments/README.md
  • plugins/nemo-deployments/pyproject.toml
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/dependencies.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/deployment_configs.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/deployments.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/status.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/volumes.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/backends/abc.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/backends/registry.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/config.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/constants.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/service.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/validation.py
  • plugins/nemo-deployments/tests/unit/helpers.py
  • plugins/nemo-deployments/tests/unit/test_api_deployment_configs.py
  • plugins/nemo-deployments/tests/unit/test_api_deployments.py
  • plugins/nemo-deployments/tests/unit/test_api_status.py
  • plugins/nemo-deployments/tests/unit/test_api_volumes.py
  • plugins/nemo-deployments/tests/unit/test_entities.py
  • plugins/nemo-deployments/tests/unit/test_prerequisite_validation.py
  • plugins/nemo-deployments/tests/unit/test_registry.py
  • plugins/nemo-deployments/tests/unit/test_service_startup.py
  • pyproject.toml

Comment thread plugins/nemo-deployments/pyproject.toml
Comment thread plugins/nemo-deployments/README.md Outdated
Comment thread plugins/nemo-deployments/README.md
Comment thread plugins/nemo-deployments/README.md
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/dependencies.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/api/v1/deployments.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/backends/registry.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/config.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py Outdated
Comment thread plugins/nemo-deployments/tests/unit/test_api_volumes.py
@tylersbray tylersbray changed the title feat(deployments): scaffold plugin entities, API, and executor registry (AIRCORE-755) feat(deployments): scaffold plugin API and registry (AIRCORE-755) Jun 11, 2026
Comment thread plugins/nemo-deployments/tests/unit/test_registry.py Fixed
@tylersbray
tylersbray requested a review from benmccown June 11, 2026 20:15
tylersbray added a commit that referenced this pull request Jun 12, 2026
Introduce DeploymentsController with deployment/volume reconcilers,
prerequisite gating, drift recovery, and orphan cleanup on top of the
755 plugin scaffold. Stacks on PR #280 (AIRCORE-755).

AIRCORE-758

Signed-off-by: Tyler Bray <tbray@nvidia.com>

@benmccown benmccown left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor stuff that an agent should be able to address without too much issue. Nice work so far.

Comment thread plugins/nemo-deployments/README.md Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/entities.py
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/service.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/types.py
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/config.py Outdated
Comment thread plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py
tylersbray added a commit that referenced this pull request Jun 15, 2026
Move API to v2, trim README, remove status_history from status PUT,
drop plugin-level port range, and add referential delete guards for
configs and volumes.

Signed-off-by: Tyler Bray <tbray@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
plugins/nemo-deployments/tests/unit/test_api_volumes.py (1)

33-44: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Assert persisted created.status in create-volume test.

This is still missing from the prior review point: the response can stay "PENDING" via mock return even if route stops setting status before persistence. Assert mock_entity_client.create payload status directly.

Patch
 def test_create_volume_201(client: TestClient, mock_entity_client: AsyncMock) -> None:
     mock_entity_client.create.return_value = make_volume()
     resp = client.post(
         "/apis/deployments/v2/workspaces/default/volumes",
         json={"name": "vol1", "size": "5Gi"},
     )
     assert resp.status_code == 201
     assert resp.json()["status"] == "PENDING"
+    mock_entity_client.create.assert_awaited_once()
     created = mock_entity_client.create.await_args.args[0]
+    assert created.status == "PENDING"
     assert created.name == "vol1"
     assert created.size == "5Gi"
     assert created.workspace == "default"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-deployments/tests/unit/test_api_volumes.py` around lines 33 -
44, The test_create_volume_201 function is missing an assertion to verify the
status field on the created volume object that gets persisted. Currently the
test asserts the response status is "PENDING" but does not verify that the
actual payload being passed to mock_entity_client.create() has the correct
status value set. Add an assertion after the existing assertions to verify that
created.status equals the expected status value (likely "PENDING"). This ensures
the route is properly setting the status on the volume object before persisting
it, not relying on the mock's return value to hide a missing status assignment.
🧹 Nitpick comments (1)
plugins/nemo-deployments/README.md (1)

1-11: ⚡ Quick win

README omits required prerequisites and next steps sections per coding guidelines.

Current file violates two mandatory guidelines:

  • "Always list prerequisites at the top of documentation pages before other content" — missing prerequisites section.
  • "Include 'Next Steps' section at the end with cross-links to related documentation content" — missing next steps section.

This reflects the maintainer's request to simplify during scaffolding, but guidelines require both sections. Resolve this tension with maintainer (benmccown) — either add minimal prerequisites (Python version, uv, NeMo Platform availability) and next steps (links to backend implementation PRs AIRCORE-756/757/758), or document in issue that these are deferred until backends are ready.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-deployments/README.md` around lines 1 - 11, The README.md file
for the NeMo Deployments Plugin is missing two mandatory sections required by
coding guidelines: a Prerequisites section at the top (before the current
content) and a Next Steps section at the end. Add a Prerequisites section
immediately after the main heading that lists Python version requirements, the
uv tool dependency, and NeMo Platform availability. Add a Next Steps section at
the end of the file with cross-links to related documentation and the backend
implementation PRs (AIRCORE-756, AIRCORE-757, AIRCORE-758) to guide users on
what to explore next. If there is uncertainty about the specific content,
coordinate with the maintainer (benmccown) to either confirm the exact
prerequisites and next steps to include, or document in an issue that these
sections are deferred pending backend readiness.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/deployment_configs.py`:
- Around line 132-147: The deployment config deletion is not atomic because the
check for referencing deployments (via deployment_names_using_config) and the
actual deletion (via entity_client.delete) happen in separate calls, creating a
race condition where a concurrent deployment creation could reference a config
after it passes the check but before deletion completes. Refactor the code to
use a single storage-level conditional delete operation that atomically verifies
no referencing deployments exist and deletes the DeploymentConfig in one call,
or implement an equivalent compare-and-swap contract to ensure the "no
referencing deployments" state and deletion occur together without allowing
concurrent operations to insert references in between.

---

Duplicate comments:
In `@plugins/nemo-deployments/tests/unit/test_api_volumes.py`:
- Around line 33-44: The test_create_volume_201 function is missing an assertion
to verify the status field on the created volume object that gets persisted.
Currently the test asserts the response status is "PENDING" but does not verify
that the actual payload being passed to mock_entity_client.create() has the
correct status value set. Add an assertion after the existing assertions to
verify that created.status equals the expected status value (likely "PENDING").
This ensures the route is properly setting the status on the volume object
before persisting it, not relying on the mock's return value to hide a missing
status assignment.

---

Nitpick comments:
In `@plugins/nemo-deployments/README.md`:
- Around line 1-11: The README.md file for the NeMo Deployments Plugin is
missing two mandatory sections required by coding guidelines: a Prerequisites
section at the top (before the current content) and a Next Steps section at the
end. Add a Prerequisites section immediately after the main heading that lists
Python version requirements, the uv tool dependency, and NeMo Platform
availability. Add a Next Steps section at the end of the file with cross-links
to related documentation and the backend implementation PRs (AIRCORE-756,
AIRCORE-757, AIRCORE-758) to guide users on what to explore next. If there is
uncertainty about the specific content, coordinate with the maintainer
(benmccown) to either confirm the exact prerequisites and next steps to include,
or document in an issue that these sections are deferred pending backend
readiness.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: edb6aae1-ca29-4691-8e33-3c31648352d3

📥 Commits

Reviewing files that changed from the base of the PR and between f6dbc92 and 32f18f5.

📒 Files selected for processing (16)
  • plugins/nemo-deployments/README.md
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/dependencies.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/deployment_configs.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/deployments.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/status.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/volumes.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/config.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/references.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/service.py
  • plugins/nemo-deployments/tests/unit/test_api_deployment_configs.py
  • plugins/nemo-deployments/tests/unit/test_api_deployments.py
  • plugins/nemo-deployments/tests/unit/test_api_status.py
  • plugins/nemo-deployments/tests/unit/test_api_volumes.py
  • plugins/nemo-deployments/tests/unit/test_references.py
  • plugins/nemo-deployments/tests/unit/test_service_startup.py
💤 Files with no reviewable changes (2)
  • plugins/nemo-deployments/src/nemo_deployments_plugin/api/v2/dependencies.py
  • plugins/nemo-deployments/src/nemo_deployments_plugin/schema.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • plugins/nemo-deployments/tests/unit/test_service_startup.py
  • plugins/nemo-deployments/tests/unit/test_api_deployment_configs.py
  • plugins/nemo-deployments/tests/unit/test_api_status.py

@benmccown

Copy link
Copy Markdown
Contributor

Changes look good so far. Thoughts on the 3 remaining?


Here's the final complete picture:

Fully addressed (5/9 actionable items):

  • ✅ v1 → v2 routes
  • ✅ README trimmed to just the Tests section — exactly what you asked for
  • ✅ Referential check on DeploymentConfig delete (409 with deployment names)
  • ✅ Referential check on Volume delete (409 listing blocking configs, wired into volumes.py)
  • port_range_start/end — need to verify, let me check config.py is updated

Not addressed (3/9 actionable items):

  • deployment_config_namedeployment_config on CreateDeploymentRequest — field unchanged
  • status_history shouldn't be on UpdateDeploymentStatusRequest — still there
  • endpoints shouldn't be on UpdateDeploymentStatusRequest — still there

tylersbray added a commit that referenced this pull request Jun 22, 2026
Move API to v2, trim README, remove status_history from status PUT,
drop plugin-level port range, and add referential delete guards for
configs and volumes.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
tylersbray added a commit that referenced this pull request Jun 22, 2026
Rename CreateDeploymentRequest and Deployment entity field to
deployment_config (with workspace/name ref parsing on create), and drop
endpoints from the controller status PUT body.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
@tylersbray
tylersbray force-pushed the 755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray branch from 16b2cb9 to 4f7b14a Compare June 22, 2026 00:14
@tylersbray

Copy link
Copy Markdown
Contributor Author

Thanks for the follow-up — quick status on the three:

status_history on UpdateDeploymentStatusRequest — already done in 32f18f59 (removed from the request schema and status handler). Reconciler (758) will append server-side.

endpoints on UpdateDeploymentStatusRequest — agreed; removed in latest push. Endpoints stay on the Deployment entity; the reconciler can set them via entity client when projecting BackendStatusUpdate, rather than through the status PUT body.

deployment_config_namedeployment_config — done on both CreateDeploymentRequest and the Deployment entity (plus DeploymentFilter). Accepts bare name (same workspace) or workspace/name on create.

Also rebased the branch onto current main (linear history, no merge commit).

tylersbray added a commit that referenced this pull request Jun 22, 2026
Introduce DeploymentsController with deployment/volume reconcilers,
prerequisite gating, drift recovery, and orphan cleanup on top of the
755 plugin scaffold. Stacks on PR #280 (AIRCORE-755).

AIRCORE-758

Signed-off-by: Tyler Bray <tbray@nvidia.com>
@benmccown
benmccown self-requested a review June 22, 2026 19:58
@tylersbray
tylersbray added this pull request to the merge queue Jun 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 22, 2026
Add nemo-deployments-plugin as the substrate-agnostic contract layer for
AIRCORE-755: DeploymentConfig/Deployment/Volume entities, v1 CRUD routes,
controller status endpoints, DeploymentBackend ABC, and named executor
registry. Plugin starts with zero backends; reconciler and docker/k8s
backends land in follow-on tickets.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Rename backends/abc.py to base.py and extract shared types to break the
stdlib abc shadowing cycle. Fix auth to use Principal-Id only, add registry
rollback and port-range validation, and wire pytest pythonpath for helpers.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Use import abc to avoid self-import false positive and fail backends via
init() hook so DeploymentBackend.__init__ is exercised in tests.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Move API to v2, trim README, remove status_history from status PUT,
drop plugin-level port range, and add referential delete guards for
configs and volumes.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Register get_authz_contribution on DeploymentsService so discovered
routes satisfy static authz validation. Document referential delete
TOCTOU limitation pending entity-store conditional delete.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Rename CreateDeploymentRequest and Deployment entity field to
deployment_config (with workspace/name ref parsing on create), and drop
endpoints from the controller status PUT body.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Rebase onto current main to pick up evaluator MetricFilter fix (#392)
that caused merge-queue lint-openapi/lint-web-sdk failures. Commit the
generated deployments plugin OpenAPI spec alongside other opted-in plugins.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
@tylersbray
tylersbray force-pushed the 755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray branch from 4f7b14a to da124d3 Compare June 22, 2026 22:07
Linux CI reuses ProcessPoolExecutor workers across plugins. Discovering
services for one plugin imports route modules for others, registering
query-param filter schemas at import time; the next plugin extraction
clears that registry without re-importing routes, leaving dangling Filter
refs (MetricFilter on evaluator). Use a fresh max_workers=1 pool per
plugin, matching the services sequential-isolation pattern.

Signed-off-by: Tyler Bray <tbray@nvidia.com>
@tylersbray
tylersbray added this pull request to the merge queue Jun 22, 2026
Merged via the queue into main with commit 7ef7c77 Jun 22, 2026
51 checks passed
@tylersbray
tylersbray deleted the 755-deployments-plugin-scaffold-the-plugin-entities-api-backend/tbray branch June 22, 2026 22:54
tylersbray added a commit that referenced this pull request Jun 22, 2026
Introduce DeploymentsController with deployment/volume reconcilers,
prerequisite gating, drift recovery, and orphan cleanup on top of the
755 plugin scaffold. Stacks on PR #280 (AIRCORE-755).

AIRCORE-758

Signed-off-by: Tyler Bray <tbray@nvidia.com>
tylersbray added a commit that referenced this pull request Jun 24, 2026
Introduce DeploymentsController with deployment/volume reconcilers,
prerequisite gating, drift recovery, and orphan cleanup on top of the
755 plugin scaffold. Stacks on PR #280 (AIRCORE-755).

AIRCORE-758

Signed-off-by: Tyler Bray <tbray@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants