feat(customizer): generate OpenAPI spec for customization plugin

openapi/README.md

@@ -2,35 +2,42 @@
 
 ## Generation
 
-To generate the `openapi.yaml` file that corresponds exactly to what is implemented in each microservice, we start each microservice and fetch the `/openapi.json` file generated by the FastAPI server.
+To generate the `openapi.yaml` files, the generator imports each FastAPI app in an isolated subprocess and calls its `.openapi()` method directly (no running server), then applies a series of schema fixes and validation passes.
 
 To generate the updated OpenAPI schema, run:
 
 ```bash
 uv run --frozen python -m script.generate_openapi_spec
 ```
 
-## OpenAPI Specs Overview
+## Output layout
 
-The following table lists all the OpenAPI specifications that are merged into the final `openapi.yaml` file, along with their sources:
+The generator no longer emits one spec per microservice and merges them. It now produces:
 
-| Service | Source | Output File |
-|---------|---------|-------------|
-| Entity Store | Generated from `entity_store.server:app` | `entity-store.openapi.yaml` |
-| Evaluator | Generated from `evaluator.server:app` | `evaluator.openapi.yaml` |
-| Guardrails | Generated from `guardrails.app:app` | `guardrails.openapi.yaml` |
-| Customization | Generated from `nemo-customizer-plugin` contributor routes | `customization.openapi.yaml` |
-| Deployment Management | Direct copy from `deployment.openapi.yaml` | `deployment-management.openapi.yaml` |
-| Jobs | Generated from `jobs.api.server:app` | `jobs.openapi.yaml` |
-| Data Designer | Generated from `data_designer.api.server:app` | `data-designer.openapi.yaml` |
-| Auditor | Generated from `auditor.server:app` | `auditor.openapi.yaml` |
-| Safe Synthesizer API | Generated from `safe_synthesizer_api.server:app` | `safe-synthesizer.openapi.yaml` |
-| Intake | Generated from `src.main:app` | `intake.openapi.yaml` |
-| Models | Generated from `models.api.server:app` | `models.openapi.yaml` |
-| Inference Gateway | Generated from `inference_gateway.api.server:create_app` | `inference-gateway.openapi.yaml` |
-| Platform Common | Generated from `nemoplatform.server:app` | `nmp-common.openapi.yaml` |
+**One aggregate platform spec**, built from the platform runner (`nmp.platform_runner.server:create_platform_openapi_app`) with plugin services deliberately excluded (`NEMO_PLUGIN_SERVICES_ALLOWLIST=""` — see `SERVICES` in `script/generate_openapi_spec.py`). This aggregate covers the core platform services (entities, jobs, models, inference gateway, secrets, files, platform-common, etc.) and lands in:
 
-All specs are merged with the `--keep-versions` flag to preserve version information in the final `openapi.yaml`.
+| File | Contents |
+|------|----------|
+| `openapi/openapi.yaml` | Final merged GA + EA platform spec |
+| `openapi/ga/openapi.yaml` | GA-only platform spec |
+| `openapi/ea/openapi.yaml` | EA-only platform spec |
+| `openapi/ga/individual/platform.openapi.yaml` | The platform spec before GA/EA merge |
+
+**One spec per opted-in plugin**, written next to each plugin — never merged into the platform spec. A plugin opts in by declaring a `[tool.nemo.openapi]` table in its own `pyproject.toml`; `discover_plugins()` (`script/openapi_helper/plugin_config.py`) enumerates those, builds each plugin's FastAPI app via the convention loader (or a `factory_override`), and emits:
+
+| Plugin | Output File |
+|--------|-------------|
+| Agents | `plugins/nemo-agents/openapi/openapi.yaml` |
+| Auditor | `plugins/nemo-auditor/openapi/openapi.yaml` |
+| Customization | `plugins/nemo-customizer/openapi/openapi.yaml` |
+| Data Designer | `plugins/nemo-data-designer/openapi/openapi.yaml` |
+| Deployments | `plugins/nemo-deployments/openapi/openapi.yaml` |
+| Evaluator | `plugins/nemo-evaluator/openapi/openapi.yaml` |
+| Safe Synthesizer | `plugins/nemo-safe-synthesizer/openapi/openapi.yaml` |
+
+The Customization spec is assembled at generation time from whichever customization contributors (`nemo.customization.contributors` entry points — e.g. `automodel`, `rl`, `unsloth`) are installed in the workspace, so its route surface depends on the synced environment. To add a new plugin to this list, add an (empty is fine) `[tool.nemo.openapi]` table to its `pyproject.toml`; if the plugin has more than one `nemo.services` entry point, set `service_name` in that table to disambiguate.
+
+The platform GA and EA specs are merged with the `--keep-versions` flag to preserve version information in the final `openapi.yaml`.
 
 ### Conflicts
 
@@ -54,4 +61,4 @@ Currently, the examples are manually generated from notebooks. A robust system t
 
 ## NOTE
 
-Ideally, the `nmp-common.openapi.yaml` file would be the platform OpenAPI spec directly, but for various reasons, it is not. We should get to that point soon. Until then, we take this merging approach to have a spec that corresponds 100% to what is implemented.
+The aggregate platform spec is now generated directly from the platform runner (`nmp.platform_runner.server`), so it corresponds to what is actually mounted at runtime. Only the GA and EA variants of that single platform spec are merged; individual core services are no longer emitted and merged separately.

packages/nemo_platform_plugin/src/nemo_platform_plugin/jobs/routes.py

@@ -209,9 +209,18 @@ def _derive_job_type(job_cls: type["NemoJob"]) -> str:
     """PascalCase form of ``job_cls.name`` for OpenAPI schema names.
 
     ``"generate"`` → ``"Generate"``; ``"metric-eval"`` → ``"MetricEval"``;
-    ``"raw_job"`` → ``"RawJob"``.
+    ``"raw_job"`` → ``"RawJob"``; ``"automodel.jobs"`` → ``"AutomodelJobs"``.
+
+    Splits on ``.`` as well as ``-``/``_`` so a dot never reaches the generated
+    schema class name (``{job_type}JobRequest``). Were a dot to survive, Pydantic
+    would encode it as a ``__`` separator in the OpenAPI ref name, and schema-name
+    normalization (which strips everything before the last ``__`` to drop module
+    namespaces) would then collapse per-backend names like ``automodel.jobs`` and
+    ``rl.jobs`` to a single ``jobsJobRequest``, making every backend's request
+    body alias the first one. Folding the dot into the PascalCase name keeps the
+    discriminator inside the final segment so each backend gets a distinct schema.
     """
-    parts = job_cls.name.replace("_", "-").split("-")
+    parts = job_cls.name.replace("_", "-").replace(".", "-").split("-")
     return "".join(part[:1].upper() + part[1:] for part in parts if part)
 
 

packages/nemo_platform_plugin/tests/test_jobs_routes.py

@@ -201,6 +201,73 @@ def run(self, config: dict) -> dict:
 
         assert _derive_job_type(J) == "RawJob"
 
+    def test_dotted_name_folds_into_pascalcase(self) -> None:
+        # A dot must not survive into the schema class name: Pydantic renders it
+        # as a ``__`` separator in the OpenAPI ref, which schema-name
+        # normalization then strips, collapsing per-backend names (e.g.
+        # "automodel.jobs" and "rl.jobs") to a single "jobsJobRequest".
+        class J(NemoJob):
+            name = "automodel.jobs"
+            spec_schema = _WidgetSpec
+
+            def run(self, config: dict) -> dict:
+                return config
+
+        job_type = _derive_job_type(J)
+        assert job_type == "AutomodelJobs"
+        assert "." not in job_type
+
+
+class TestPerJobTypeSchemaNaming:
+    """Guard the end-to-end path where dotted job names used to collapse.
+
+    Two backends whose ``name`` differs only before the dot (``alpha.jobs`` /
+    ``beta.jobs``) must produce distinct request schemas that survive
+    ``tweak_spec``'s schema-name normalization, each referencing its own spec
+    schema. Before the ``_derive_job_type`` dot fix, both collapsed to a single
+    ``jobsJobRequest`` and every backend's POST body aliased the first one.
+    """
+
+    def test_dotted_backends_keep_distinct_request_schemas(self) -> None:
+        from fastapi import FastAPI
+        from nmp.common.api.utils import tweak_spec
+
+        class AlphaSpec(BaseModel):
+            alpha_field: str
+
+        class BetaSpec(BaseModel):
+            beta_field: int
+
+        def _make_job(job_name: str, spec_cls: type[BaseModel]) -> type[NemoJob]:
+            class _J(NemoJob):
+                name = job_name
+                spec_schema = spec_cls
+
+                def run(self, config: dict) -> dict:
+                    return config
+
+                @classmethod
+                async def compile(cls, **kwargs):  # pragma: no cover - not invoked by openapi()
+                    raise NotImplementedError
+
+            return _J
+
+        app = FastAPI()
+        app.include_router(add_job_routes(_make_job("alpha.jobs", AlphaSpec)))
+        app.include_router(add_job_routes(_make_job("beta.jobs", BetaSpec)))
+
+        spec = tweak_spec(app.openapi())
+        schemas = spec["components"]["schemas"]
+
+        request_keys = {k for k in schemas if k.endswith("JobRequest")}
+        assert request_keys == {"AlphaJobsJobRequest", "BetaJobsJobRequest"}
+
+        def _spec_ref(request_key: str) -> str:
+            return schemas[request_key]["properties"]["spec"]["$ref"].split("/")[-1]
+
+        assert _spec_ref("AlphaJobsJobRequest") == "AlphaSpec"
+        assert _spec_ref("BetaJobsJobRequest") == "BetaSpec"
+
 
 # ---------------------------------------------------------------------------
 # _adapt_to_spec

packages/nmp_customization_common/src/nmp/customization_common/contributor/base.py

@@ -15,8 +15,7 @@
 from typing import Any, Callable, ClassVar
 
 import typer
-from fastapi import APIRouter
-from nemo_platform_plugin.authz import AuthzScope, CallerKind, path_rule
+from nemo_platform_plugin.authz import AuthzScope
 from nemo_platform_plugin.customization_contributor import CustomizationContributorSDKResources
 from nemo_platform_plugin.jobs.api_factory import JobRouteOption
 from nemo_platform_plugin.jobs.routes import add_job_routes
@@ -55,21 +54,18 @@ def _get_config(self) -> Any:
         raise NotImplementedError
 
     def get_routers(self) -> list[RouterSpec]:
-        """Health endpoint + ``add_job_routes`` for the backend job collection.
+        """``add_job_routes`` for the backend job collection.
 
-        HTTP authz is derived from the ``@path_rule``-decorated routes: the health
-        endpoint is authenticated-but-permissionless, and the job collection's
-        permissions (``customization.<name>.jobs.*``) are stamped onto the factory
-        routes via the ``customization`` :class:`AuthzScope` (scope ``customization``,
-        permission namespace deepened to ``customization.<name>.jobs``).
+        The job collection's permissions (``customization.<name>.jobs.*``) are
+        stamped onto the factory routes via the ``customization``
+        :class:`AuthzScope` (scope ``customization``, permission namespace
+        deepened to ``customization.<name>.jobs``).
+
+        Backend health is not exposed per contributor — the customization router
+        reports a single ``/apis/customization/v2/healthz`` that enumerates the
+        registered contributors.
         """
         config = self._get_config()
-        router = APIRouter()
-
-        @router.get("/healthz")
-        @path_rule(callers=[CallerKind.PRINCIPAL], permissions=[])
-        async def healthz() -> dict[str, str]:
-            return {"backend": self.name, "status": "ok"}
 
         jobs_router = add_job_routes(
             self.job_cls,
@@ -81,12 +77,6 @@ async def healthz() -> dict[str, str]:
         )
 
         return [
-            RouterSpec(
-                router=router,
-                prefix=f"/v2/workspaces/{{workspace}}/{self.name}",
-                tag=self._title,
-                description=f"{self._title} contributor health.",
-            ),
             RouterSpec(
                 router=jobs_router,
                 prefix="/v2/workspaces/{workspace}",

packages/nmp_customization_common/src/nmp/customization_common/sdk/client.py

@@ -168,32 +168,13 @@ def __init__(self, platform: PlatformClient) -> None:
         self._platform = platform
         self._http_client = platform._client
 
-    def _healthz_url(self) -> str:
-        return url(
-            self._platform,
-            f"v2/workspaces/{{workspace}}/{self.backend}/healthz",
-            self._platform.workspace,
-        )
-
     def _new_record(self, payload: Any) -> JobRecord:
         return self.record_schema.model_validate(payload)
 
 
 class JobsResource(_JobsResourceBase):
     """Sync SDK namespace at ``client.customization.<backend>.jobs``."""
 
-    def plugin_status(self) -> dict[str, object]:
-        """Return contributor health from the customization service."""
-        response = self._http_client.get(
-            self._healthz_url(),
-            headers=platform_default_headers(self._platform),
-        )
-        response.raise_for_status()
-        payload = response.json()
-        if not isinstance(payload, dict):
-            raise TypeError(f"{self.backend} health response must be a JSON object.")
-        return {str(key): value for key, value in payload.items()}
-
     def create(
         self,
         spec: BaseModel,
@@ -241,18 +222,6 @@ def get_job_resource(self, job_name: str, workspace: str | None = None) -> JobRe
 class AsyncJobsResource(_JobsResourceBase):
     """Async SDK namespace at ``client.customization.<backend>.jobs``."""
 
-    async def plugin_status(self) -> dict[str, object]:
-        """Return contributor health from the customization service."""
-        response = await self._http_client.get(
-            self._healthz_url(),
-            headers=platform_default_headers(self._platform),
-        )
-        response.raise_for_status()
-        payload = response.json()
-        if not isinstance(payload, dict):
-            raise TypeError(f"{self.backend} health response must be a JSON object.")
-        return {str(key): value for key, value in payload.items()}
-
     async def create(
         self,
         spec: BaseModel,

plugins/nemo-automodel/tests/test_api.py

@@ -31,19 +31,19 @@ def _make_automodel_app() -> FastAPI:
     return app
 
 
-def test_automodel_healthz_under_workspace() -> None:
-    client = TestClient(_make_automodel_app())
-    response = client.get("/v2/workspaces/test-ws/automodel/healthz")
-    assert response.status_code == 200
-    assert response.json() == {"backend": "automodel", "status": "ok"}
-
-
 def test_automodel_jobs_collection_path() -> None:
     paths = _route_paths(_make_automodel_app())
     assert "/v2/workspaces/{workspace}/automodel/jobs" in paths
 
 
-def test_customization_router_merges_automodel(monkeypatch: pytest.MonkeyPatch) -> None:
+def test_contributor_exposes_no_per_backend_healthz() -> None:
+    """Backend health is not exposed per contributor; the customization router
+    reports a single ``/v2/healthz`` that enumerates contributors instead."""
+    paths = _route_paths(_make_automodel_app())
+    assert not any(p.endswith("/healthz") for p in paths)
+
+
+def test_customization_router_healthz_lists_contributors(monkeypatch: pytest.MonkeyPatch) -> None:
     monkeypatch.setattr(
         "nemo_customizer.router.discover_customization_contributors",
         lambda: {"automodel": AutomodelContributor()},
@@ -55,13 +55,11 @@ def test_customization_router_merges_automodel(monkeypatch: pytest.MonkeyPatch)
         app.include_router(spec.router, prefix=prefix)
 
     client = TestClient(app)
-    assert client.get("/healthz").json()["contributors"] == ["automodel"]
-    assert client.get("/v2/workspaces/ws-a/automodel/healthz").status_code == 200
+    assert client.get("/v2/healthz").json()["contributors"] == ["automodel"]
 
 
 def test_workspace_isolation_list_uses_path_segment() -> None:
     """Job routes are under ``/v2/workspaces/{workspace}/automodel/jobs`` — distinct per workspace."""
     app = _make_automodel_app()
     paths = _route_paths(app)
     assert "/v2/workspaces/{workspace}/automodel/jobs" in paths
-    assert "/v2/workspaces/{workspace}/automodel/healthz" in paths

plugins/nemo-automodel/tests/test_contributor.py

@@ -28,7 +28,6 @@ def test_contributor_mounts_job_collection() -> None:
         app.include_router(spec.router, prefix=spec.prefix)
 
     paths = _route_paths(app)
-    assert "/v2/workspaces/{workspace}/automodel/healthz" in paths
     assert "/v2/workspaces/{workspace}/automodel/jobs" in paths