From c87eecc0d8b18855bddacd60e51c1ef2245f383e Mon Sep 17 00:00:00 2001 From: Matt Kornfield Date: Thu, 25 Jun 2026 03:22:17 +0000 Subject: [PATCH 1/3] fix: various docs updates, including hiding intake Signed-off-by: Matt Kornfield --- .../workflows/fern-docs-preview-comment.yaml | 1 + .gitignore | 3 + docs/AGENTS.md | 2 +- docs/agents/plugins.mdx | 4 +- docs/auth/authorization/managing-access.mdx | 2 +- .../authorization/permissions-reference.mdx | 16 -- docs/contributing/skills-spec.mdx | 1 - .../tutorials/format-training-dataset.mdx | 2 +- docs/evaluator/tutorials/index.mdx | 2 +- docs/fern/README.md | 5 +- docs/fern/generators.yml | 2 +- docs/fern/package.json | 13 +- docs/fern/scripts/docs-watch.mjs | 18 +- docs/fern/scripts/filter-public-openapi.mjs | 192 ++++++++++++++++++ docs/get-started/concepts/filtering.mdx | 22 +- docs/requirements.mdx | 4 +- docs/support-matrix.mdx | 8 +- services/core/auth/scripts/auth-tools.py | 6 + 18 files changed, 253 insertions(+), 50 deletions(-) create mode 100644 docs/fern/scripts/filter-public-openapi.mjs diff --git a/.github/workflows/fern-docs-preview-comment.yaml b/.github/workflows/fern-docs-preview-comment.yaml index 4477b20c2d..1a72358a96 100644 --- a/.github/workflows/fern-docs-preview-comment.yaml +++ b/.github/workflows/fern-docs-preview-comment.yaml @@ -62,6 +62,7 @@ jobs: HEAD_REF: ${{ steps.metadata.outputs.head_ref }} working-directory: ./docs/fern run: | + node scripts/filter-public-openapi.mjs OUTPUT=$(npx -y fern-api@latest generate --docs --preview --id "$HEAD_REF" 2>&1) echo "$OUTPUT" URL=$(echo "$OUTPUT" | grep -oP 'Published docs to \K.*(?= \()') diff --git a/.gitignore b/.gitignore index d90f2ac3dd..9e47336c0b 100644 --- a/.gitignore +++ b/.gitignore @@ -143,3 +143,6 @@ docker/logs/* # Linting artifact python-sdk-lint/ site/ + +# Generated Fern-only public API reference spec +docs/fern/openapi/openapi.public.yaml diff --git a/docs/AGENTS.md b/docs/AGENTS.md index f67976f8f7..ea3d83d37c 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -39,4 +39,4 @@ These are generated; edit the source and regenerate: ## API reference -The REST API reference is rendered natively by Fern from the OpenAPI spec — `docs/fern/generators.yml` points at `docs/fern/openapi/openapi.yaml` (a symlink to the generated repo-root `openapi/openapi.yaml`), surfaced by the `- api: API Reference` nav node. No `` embed. +The REST API reference is rendered natively by Fern from a docs-only filtered OpenAPI spec. `docs/fern/openapi/openapi.yaml` is a symlink to the generated repo-root `openapi/openapi.yaml`, and `npm run prepare:openapi` writes `docs/fern/openapi/openapi.public.yaml` for Fern. `docs/fern/generators.yml` must point at the public filtered spec, surfaced by the `- api: API Reference` nav node. No `` embed. diff --git a/docs/agents/plugins.mdx b/docs/agents/plugins.mdx index 1f43e1fd79..5fc8695d6b 100644 --- a/docs/agents/plugins.mdx +++ b/docs/agents/plugins.mdx @@ -4,7 +4,7 @@ description: "" --- -NeMo Platform 0.1.0 is extended through Python plugins and +NeMo Platform 0.2.0 is extended through Python plugins and coding-agent skills. Plugins add runtime capabilities to the local platform. Skills teach your coding agent how to operate those capabilities on your behalf. @@ -103,5 +103,5 @@ Plugin authors use the `nemo-platform-plugin` package. A minimal plugin declares points in `pyproject.toml`, implements the matching class, and restarts `nemo services run` so the local platform discovers it. -Keep plugins local-first for OSS 0.1.0: avoid requiring a cluster, +Keep plugins local-first for OSS 0.2.0: avoid requiring a cluster, container runtime, or external control plane for the documented launch path. diff --git a/docs/auth/authorization/managing-access.mdx b/docs/auth/authorization/managing-access.mdx index 1b97ffbf8e..0f82f98ef4 100644 --- a/docs/auth/authorization/managing-access.mdx +++ b/docs/auth/authorization/managing-access.mdx @@ -6,7 +6,7 @@ Add users to workspaces, assign roles, and control who can access your resources -The SDK examples on this page use `NeMoPlatform()` with no arguments so that the client reads your active CLI context (set by `nemo auth login`). That is the right pattern for authorization workflows: you act as your logged-in identity and pass the workspace explicitly in each API call. For the standard local initialization pattern, see [CLI and SDK initialization](/documentation/get-started#setup-init). +The SDK examples on this page use `NeMoPlatform()` with no arguments so that the client reads your active CLI context (set by `nemo auth login`). That is the right pattern for authorization workflows: you act as your logged-in identity and pass the workspace explicitly in each API call. For the standard local initialization pattern, see [CLI and SDK initialization](/documentation/get-started#nemo-setup). ## Creating Workspaces diff --git a/docs/auth/authorization/permissions-reference.mdx b/docs/auth/authorization/permissions-reference.mdx index 3c76966026..d3144f67cd 100644 --- a/docs/auth/authorization/permissions-reference.mdx +++ b/docs/auth/authorization/permissions-reference.mdx @@ -58,22 +58,6 @@ PlatformAdmin is omitted — it bypasses permission checks entirely at the polic | inference.virtual-models.(read | list) | Read, list inference virtual-models | ✓ | ✓ | ✓ | | inference.virtual-models.(create | update | delete) | Create, update, delete inference virtual-models | | ✓ | ✓ | -## Intake API - -| Permission | Description | Viewer | Editor | Admin | -|------------|-------------|:------:|:------:|:-----:| -| intake.annotations.(read | list) | Read, list intake annotations | ✓ | ✓ | ✓ | -| intake.annotations.(create | delete) | Create, delete intake annotations | | ✓ | ✓ | -| intake.evaluator-results.(read | list) | Read, list intake evaluator-results | ✓ | ✓ | ✓ | -| `intake.evaluator-results.create` | Create intake evaluator results | | ✓ | ✓ | -| `intake.experiment-groups.read` | Read intake experiment groups | ✓ | ✓ | ✓ | -| intake.experiment-groups.(create | update | delete) | Create, update, delete intake experiment-groups | | ✓ | ✓ | -| `intake.experiments.read` | Read intake experiments | ✓ | ✓ | ✓ | -| intake.experiments.(create | update | delete) | Create, update, delete intake experiments | | ✓ | ✓ | -| `intake.ingest.create` | Ingest traces into intake | | ✓ | ✓ | -| intake.spans.(read | list) | Read, list intake spans | ✓ | ✓ | ✓ | -| `intake.traces.read` | Read intake traces | ✓ | ✓ | ✓ | - ## Jobs API | Permission | Description | Viewer | Editor | Admin | diff --git a/docs/contributing/skills-spec.mdx b/docs/contributing/skills-spec.mdx index 3b135a050c..feebd3c4fc 100644 --- a/docs/contributing/skills-spec.mdx +++ b/docs/contributing/skills-spec.mdx @@ -2,7 +2,6 @@ title: "NeMo Platform Skills Spec" description: "" --- -**Status:** Draft. This document defines the conventions for skills shipped with NeMo Platform: how they're structured, what their frontmatter must contain, how they get tested, and where they live in the repo. diff --git a/docs/customizer/tutorials/format-training-dataset.mdx b/docs/customizer/tutorials/format-training-dataset.mdx index 4217a47763..ee575d7209 100644 --- a/docs/customizer/tutorials/format-training-dataset.mdx +++ b/docs/customizer/tutorials/format-training-dataset.mdx @@ -236,7 +236,7 @@ For illustrative purposes only, we show an example entry as multi-line JSON. ##### Shared Tools -When your dataset uses the same set of tools across all examples, include the shared tool definitions in **each training row** (Automodel jobs do not accept a separate `dataset_parameters` block). See the [Tool Calling tutorial](/documentation/example-applications/tool-calling) for a full workflow. +When your dataset uses the same set of tools across all examples, include the shared tool definitions in **each training row** (Automodel jobs do not accept a separate `dataset_parameters` block). The full tool-calling workflow is maintained in the gated example applications section and will be linked here when that section ships publicly. ```python import os diff --git a/docs/evaluator/tutorials/index.mdx b/docs/evaluator/tutorials/index.mdx index 1b2dbb5b12..1f5fbd69fc 100644 --- a/docs/evaluator/tutorials/index.mdx +++ b/docs/evaluator/tutorials/index.mdx @@ -32,4 +32,4 @@ Learn how to write a domain-specific Python metric, test it locally, and run it ## How It Works -For the conceptual overview of how Evaluator separates definition (library) from execution (platform), see [About Evaluating → How It Works](/documentation/evaluate-models#how-it-works-library-platform). For runnable SDK examples, see [SDK Resources](/documentation/evaluate-models/sdk-resources). +For the conceptual overview of how Evaluator separates definition (library) from execution (platform), see [About Evaluating → How It Works](/documentation/evaluate-models#how-it-works-library--platform). For runnable SDK examples, see [SDK Resources](/documentation/evaluate-models/sdk-resources). diff --git a/docs/fern/README.md b/docs/fern/README.md index 0633efe8d4..4acdcd5b5c 100644 --- a/docs/fern/README.md +++ b/docs/fern/README.md @@ -39,6 +39,7 @@ docs/ # page content (.mdx), one tree per product area ├── docs.yml # Site config, theme, css, redirects, versions ├── generators.yml # OpenAPI spec wiring for the API reference ├── openapi/openapi.yaml # symlink -> repo-root openapi/openapi.yaml (generated) + ├── openapi/openapi.public.yaml # generated Fern-only filtered spec (gitignored) ├── styles/ # CSS (notebook-viewer.css, button.css) ├── assets/ # logos, shared images ├── components/ # custom TSX MDX components @@ -81,10 +82,10 @@ Avoid source-path links such as `/get-started/concepts/workspaces`, `/latest/get The REST API reference is generated natively by Fern from the OpenAPI spec — no `` embed. Two pieces wire it up: -- `generators.yml` declares the spec: `api.specs[].openapi: ./openapi/openapi.yaml`. +- `generators.yml` declares the docs-only public spec: `api.specs[].openapi: ./openapi/openapi.public.yaml`. - `versions/latest.yml` surfaces it with an `- api: API Reference` navigation node (under the **Reference** section). -`openapi/openapi.yaml` is a symlink to the repo-root `openapi/openapi.yaml` (the generated source of truth), so the reference tracks the Platform API automatically. Regenerate the spec with `make refresh-openapi` from the repo root, then run `npm run check`. +`openapi/openapi.yaml` is a symlink to the repo-root `openapi/openapi.yaml` (the generated source of truth). Fern reads `openapi/openapi.public.yaml`, which is generated by `scripts/filter-public-openapi.mjs` and excludes services that should stay out of public docs while remaining in the root spec for SDK generation and API validation. Regenerate the root spec with `make refresh-openapi` from the repo root, then run `npm run check`. Fern groups endpoints by their OpenAPI tag in the sidebar (Customizer, Evaluator, Guardrails, …), which replaces the old per-service filter chips. Link to it from other pages with the nav URL `/documentation/reference/api-reference`. diff --git a/docs/fern/generators.yml b/docs/fern/generators.yml index a2c12e1747..4062263a04 100644 --- a/docs/fern/generators.yml +++ b/docs/fern/generators.yml @@ -1,3 +1,3 @@ api: specs: - - openapi: ./openapi/openapi.yaml + - openapi: ./openapi/openapi.public.yaml diff --git a/docs/fern/package.json b/docs/fern/package.json index 0b302da626..2d667613dd 100644 --- a/docs/fern/package.json +++ b/docs/fern/package.json @@ -1,16 +1,17 @@ { "private": true, "scripts": { - "check": "npx -y fern-api@latest check && npm run validate-mdx && npm run check:gated-links", - "check:fern": "npx -y fern-api@latest check", + "prepare:openapi": "node scripts/filter-public-openapi.mjs", + "check": "npm run prepare:openapi && npx -y fern-api@latest check && npm run validate-mdx && npm run check:gated-links", + "check:fern": "npm run prepare:openapi && npx -y fern-api@latest check", "validate-mdx": "node scripts/validate-mdx.mjs", "check:gated-links": "node scripts/delink-gated.mjs", "fix:gated-links": "node scripts/delink-gated.mjs --fix", - "broken-links": "npx -y fern-api@latest docs broken-links", - "dev": "npx -y fern-api@latest docs dev", + "broken-links": "npm run prepare:openapi && npx -y fern-api@latest docs broken-links", + "dev": "npm run prepare:openapi && npx -y fern-api@latest docs dev", "watch": "node scripts/docs-watch.mjs", - "generate": "npx -y fern-api@latest generate --docs", - "preview": "npx -y fern-api@latest generate --docs --preview" + "generate": "npm run prepare:openapi && npx -y fern-api@latest generate --docs", + "preview": "npm run prepare:openapi && npx -y fern-api@latest generate --docs --preview" }, "devDependencies": { "@mdx-js/mdx": "^3.1.0" diff --git a/docs/fern/scripts/docs-watch.mjs b/docs/fern/scripts/docs-watch.mjs index 0f5a814c37..1e764b5ef4 100644 --- a/docs/fern/scripts/docs-watch.mjs +++ b/docs/fern/scripts/docs-watch.mjs @@ -1,7 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -import { spawn } from "node:child_process"; +import { spawn, spawnSync } from "node:child_process"; import { watch } from "node:fs"; import { utimes } from "node:fs/promises"; import { constants } from "node:os"; @@ -52,6 +52,22 @@ function shouldIgnore(relativePath) { return normalized.startsWith(ignoredPrefix); } +function prepareOpenapi() { + const result = spawnSync("node", ["scripts/filter-public-openapi.mjs"], { + cwd: fernDir, + stdio: "inherit", + }); + + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + process.exit(result.status ?? 1); + } +} + +prepareOpenapi(); + const fern = spawn("npx", ["-y", "fern-api@latest", "docs", "dev"], { cwd: fernDir, stdio: "inherit", diff --git a/docs/fern/scripts/filter-public-openapi.mjs b/docs/fern/scripts/filter-public-openapi.mjs new file mode 100644 index 0000000000..a8c4f3b4f6 --- /dev/null +++ b/docs/fern/scripts/filter-public-openapi.mjs @@ -0,0 +1,192 @@ +#!/usr/bin/env node +/** + * SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + * SPDX-License-Identifier: Apache-2.0 + * + * Build the public Fern OpenAPI spec from the generated platform spec. + * + * The repo-root openapi/openapi.yaml remains the source of truth for SDK + * generation and API validation. Public docs exclude services that are not part + * of the shipped OSS docs surface yet. + */ + +import { mkdir, readFile, writeFile } from "node:fs/promises"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const scriptDir = dirname(fileURLToPath(import.meta.url)); +const fernDir = resolve(scriptDir, ".."); +const inputPath = resolve(fernDir, "openapi/openapi.yaml"); +const outputPath = resolve(fernDir, "openapi/openapi.public.yaml"); + +const gatedPathPrefixes = ["/apis/intake/v2/"]; +const httpMethods = new Set(["get", "put", "post", "delete", "options", "head", "patch", "trace"]); +const schemaRefPattern = /#\/components\/schemas\/([^'"\s\]}),]+)/g; + +const source = await readFile(inputPath, "utf8"); +const hadTrailingNewline = source.endsWith("\n"); +const lines = source.split("\n"); +if (hadTrailingNewline) { + lines.pop(); +} + +function isPathKey(line) { + return /^ \//.test(line) && /:\s*$/.test(line); +} + +function isTopLevelKey(line) { + return /^[A-Za-z][A-Za-z0-9_-]*:\s*$/.test(line); +} + +function pathFromKey(line) { + return line.trim().replace(/:\s*$/, ""); +} + +function isComponentSectionKey(line) { + return /^ [A-Za-z][A-Za-z0-9_-]*:\s*$/.test(line); +} + +function isSchemaKey(line) { + return /^ [^\s].*:\s*$/.test(line); +} + +function schemaNameFromKey(line) { + return line.trim().replace(/:\s*$/, ""); +} + +function collectSchemaRefs(linesToScan) { + const refs = new Set(); + const text = linesToScan.join("\n"); + for (const match of text.matchAll(schemaRefPattern)) { + refs.add(match[1]); + } + return refs; +} + +function splitSchemaBlocks(linesToSplit) { + const parts = []; + const schemas = new Map(); + let inSchemas = false; + + for (let i = 0; i < linesToSplit.length; ) { + const line = linesToSplit[i]; + + if (!inSchemas && line === " schemas:") { + inSchemas = true; + parts.push({ type: "line", line }); + i += 1; + continue; + } + + if (inSchemas) { + if (isSchemaKey(line)) { + const name = schemaNameFromKey(line); + const block = [line]; + i += 1; + while ( + i < linesToSplit.length && + !isSchemaKey(linesToSplit[i]) && + !isTopLevelKey(linesToSplit[i]) && + !isComponentSectionKey(linesToSplit[i]) + ) { + block.push(linesToSplit[i]); + i += 1; + } + schemas.set(name, block); + parts.push({ type: "schema", name }); + continue; + } + + if (isTopLevelKey(line) || isComponentSectionKey(line)) { + inSchemas = false; + } + } + + parts.push({ type: "line", line }); + i += 1; + } + + return { parts, schemas }; +} + +function expandSchemaRefs(seedRefs, schemas) { + const refs = new Set(seedRefs); + let changed = true; + + while (changed) { + changed = false; + for (const schema of Array.from(refs)) { + const block = schemas.get(schema); + if (!block) { + continue; + } + for (const ref of collectSchemaRefs(block)) { + if (!refs.has(ref)) { + refs.add(ref); + changed = true; + } + } + } + } + + return refs; +} + +const keptLines = []; +const removedLines = []; +let skippingPath = false; +let removedPaths = 0; +let removedOperations = 0; + +for (const line of lines) { + if (isPathKey(line)) { + const path = pathFromKey(line); + skippingPath = gatedPathPrefixes.some((prefix) => path.startsWith(prefix)); + if (skippingPath) { + removedPaths += 1; + removedLines.push(line); + continue; + } + } else if (skippingPath && isTopLevelKey(line)) { + skippingPath = false; + } + + if (skippingPath) { + const method = line.trim().replace(/:\s*$/, ""); + if (httpMethods.has(method)) { + removedOperations += 1; + } + removedLines.push(line); + continue; + } + + keptLines.push(line); +} + +const { parts, schemas } = splitSchemaBlocks(keptLines); +const baseLines = parts.flatMap((part) => (part.type === "line" ? [part.line] : [])); +const reachableFromPublicSpec = expandSchemaRefs(collectSchemaRefs(baseLines), schemas); +const reachableFromRemovedPaths = expandSchemaRefs(collectSchemaRefs(removedLines), schemas); + +let removedSchemas = 0; +const publicOutput = []; +for (const part of parts) { + if (part.type === "line") { + publicOutput.push(part.line); + continue; + } + + if (reachableFromRemovedPaths.has(part.name) && !reachableFromPublicSpec.has(part.name)) { + removedSchemas += 1; + continue; + } + + publicOutput.push(...schemas.get(part.name)); +} + +await mkdir(dirname(outputPath), { recursive: true }); +await writeFile(outputPath, `${publicOutput.join("\n")}${hadTrailingNewline ? "\n" : ""}`); + +console.log( + `filter-public-openapi: wrote openapi/openapi.public.yaml, removed ${removedOperations} operations across ${removedPaths} gated paths and ${removedSchemas} intake-only schemas` +); diff --git a/docs/get-started/concepts/filtering.mdx b/docs/get-started/concepts/filtering.mdx index 304b428b34..0ac4cd30da 100644 --- a/docs/get-started/concepts/filtering.mdx +++ b/docs/get-started/concepts/filtering.mdx @@ -19,14 +19,14 @@ Prefer the form that's most natural for the caller: Generated CLI list commands expose each simple filter field as its own option: ```bash -nemo evaluation benchmarks list --filter.name mmlu --filter.project my-project +nemo files filesets list --filter.purpose dataset --filter.storage-type local ``` Dot flags cover exact-match on scalar fields. For operators (`$gte`, `$like`, …) or boolean combinations, fall through to `--filter` with text or JSON: ```bash -nemo evaluation benchmarks list --filter 'name:"mmlu" AND created_at>="2025-01-01"' -nemo evaluation benchmarks list --filter '{"created_at":{"$gte":"2025-01-01"}}' +nemo files filesets list --filter 'purpose:"dataset" AND created_at>="2025-01-01"' +nemo files filesets list --filter '{"created_at":{"$gte":"2025-01-01"}}' ``` ### Bracket notation @@ -34,13 +34,13 @@ nemo evaluation benchmarks list --filter '{"created_at":{"$gte":"2025-01-01"}}' The preferred form for REST query strings. Each `?filter[field][$op]=value` pair is URL-encoding-friendly and composable: ``` -GET /v2/workspaces/{ws}/benchmarks?filter[name][$like]=mmlu&filter[created_at][$gte]=2025-01-01 +GET /apis/files/v2/workspaces/{workspace}/filesets?filter[purpose]=dataset&filter[created_at][$gte]=2025-01-01 ``` When no operator is specified, the default is `$eq`: ``` -?filter[name]=mmlu # equivalent to filter[name][$eq]=mmlu +?filter[purpose]=dataset # equivalent to filter[purpose][$eq]=dataset ``` @@ -53,7 +53,7 @@ On datetime fields, bare bracket values also default to `$eq` — `?filter[creat A compact, human-readable query string. Handy for CLI and SDK string filters; in URLs you need to URL-encode spaces and quotes, so bracket notation is usually easier there. ```bash -nemo evaluation benchmarks list --filter 'name:"mmlu" AND created_at>"2025-01-01"' +nemo files filesets list --filter 'purpose:"dataset" AND created_at>"2025-01-01"' ``` ```python @@ -90,20 +90,20 @@ Pass a dictionary to the `filter` parameter. Keys are field names; values are ei ```python # Exact match -results = client.evaluation.benchmarks.list(filter={"name": "mmlu"}) +results = client.files.filesets.list(filter={"purpose": "dataset"}) # Operator syntax -results = client.evaluation.benchmarks.list(filter={"name": {"$like": "mmlu"}}) +results = client.files.filesets.list(filter={"name": {"$like": "training"}}) # Multiple conditions -results = client.evaluation.benchmarks.list( +results = client.files.filesets.list( filter={ - "$and": [{"name": {"$like": "mmlu"}}, {"description": {"$like": "reasoning"}}] + "$and": [{"purpose": "dataset"}, {"description": {"$like": "training"}}] } ) # Date range -results = client.evaluation.benchmarks.list( +results = client.files.filesets.list( filter={ "created_at": {"$gte": "2025-01-01T00:00:00", "$lte": "2025-06-30T23:59:59"} } diff --git a/docs/requirements.mdx b/docs/requirements.mdx index 08f4519372..2b5998e5b6 100644 --- a/docs/requirements.mdx +++ b/docs/requirements.mdx @@ -2,9 +2,9 @@ title: "Hardware and Software Requirements for NeMo Platform" description: "" --- -This page lists the requirements for the OSS local-install path for NeMo Platform 0.1.0. For the full compatibility table, see the [Support Matrix](/documentation/reference/support-matrix). +This page lists the requirements for the OSS local-install path for NeMo Platform 0.2.0. For the full compatibility table, see the [Support Matrix](/documentation/reference/support-matrix). -The OSS 0.1.0 documentation is scoped to local setup with the Python package and `nemo setup`. Docker Compose, Helm, Kubernetes, and OpenShift deployment guides are not part of this release scope. +The OSS 0.2.0 documentation is scoped to local setup with the Python package and `nemo setup`. Docker Compose, Helm, Kubernetes, and OpenShift deployment guides are not part of this release scope. ## Local Setup Requirements diff --git a/docs/support-matrix.mdx b/docs/support-matrix.mdx index 08aabf005b..8dd50d1cb3 100644 --- a/docs/support-matrix.mdx +++ b/docs/support-matrix.mdx @@ -2,9 +2,9 @@ title: "Support Matrix" description: "" --- -This matrix defines the supported OSS local-install target for NeMo Platform 0.1.0. It applies to the Python package, CLI, SDK, and local services started by `nemo setup`. +This matrix defines the supported OSS local-install target for NeMo Platform 0.2.0. It applies to the Python package, CLI, SDK, and local services started by `nemo setup`. -Docker Compose, Helm, Kubernetes, and OpenShift deployment paths are not part of the OSS 0.1.0 documentation scope. +Docker Compose, Helm, Kubernetes, and OpenShift deployment paths are not part of the OSS 0.2.0 documentation scope. ## Host Platforms @@ -15,7 +15,7 @@ Docker Compose, Helm, Kubernetes, and OpenShift deployment paths are not part of | Linux | Ubuntu 22.04 LTS, Ubuntu 24.04 LTS, RHEL 9, Rocky Linux 9, Debian 12 | Supported for CLI, SDK, local services, and NVIDIA GPU workloads. | | macOS | [macOS Tahoe 26](https://support.apple.com/en-us/122868) and macOS Sequoia 15 | Supported for CLI, SDK, local services, and cloud/provider workflows. Local NVIDIA GPU workloads are not supported on macOS. | | Architecture | x86_64 Linux; Apple Silicon and Intel macOS | NVIDIA GPU workloads require x86_64 Linux. | -| Windows and WSL | Not in the OSS 0.1.0 support matrix | Use a supported Linux or macOS host for the local install path. | +| Windows and WSL | Not in the OSS 0.2.0 support matrix | Use a supported Linux or macOS host for the local install path. | ## Local Runtime @@ -55,7 +55,7 @@ Docker Compose, Helm, Kubernetes, and OpenShift deployment paths are not part of ## Out of Scope -The following are not part of the OSS 0.1.0 local support matrix: +The following are not part of the OSS 0.2.0 local support matrix: - Native Windows local install. - Docker Compose, Helm, Kubernetes, and OpenShift deployment guides. diff --git a/services/core/auth/scripts/auth-tools.py b/services/core/auth/scripts/auth-tools.py index e857efff76..16ed99e2f8 100755 --- a/services/core/auth/scripts/auth-tools.py +++ b/services/core/auth/scripts/auth-tools.py @@ -1091,6 +1091,10 @@ def stats( "workspaces": "Workspaces API", } +DOCS_EXCLUDED_PERMISSION_AREAS = { + "intake", +} + def _perm_role_signature(perm_name: str, role_perms_map: Dict[str, Set[str]], ordered_roles: List[str]) -> tuple: """Return a tuple of booleans indicating which roles have this permission.""" @@ -1197,6 +1201,8 @@ def _generate_permissions_reference(auth_config: Dict) -> str: area_groups: Dict[str, List[str]] = {} for perm_name in sorted(registry.keys()): area = perm_name.split(".")[0] + if area in DOCS_EXCLUDED_PERMISSION_AREAS: + continue area_groups.setdefault(area, []).append(perm_name) ordered_roles = ["Viewer", "Editor", "Admin"] From e9e74187e2d2ea12d79ceb5b86a3276cfab2d1da Mon Sep 17 00:00:00 2001 From: Matt Kornfield Date: Thu, 25 Jun 2026 14:35:36 +0000 Subject: [PATCH 2/3] chore: address cr Signed-off-by: Matt Kornfield --- docs/fern/scripts/docs-watch.mjs | 56 ++++++++++++++++++------ services/core/auth/scripts/auth-tools.py | 24 ++++++---- 2 files changed, 57 insertions(+), 23 deletions(-) diff --git a/docs/fern/scripts/docs-watch.mjs b/docs/fern/scripts/docs-watch.mjs index 1e764b5ef4..cb60e9358e 100644 --- a/docs/fern/scripts/docs-watch.mjs +++ b/docs/fern/scripts/docs-watch.mjs @@ -14,8 +14,12 @@ const docsRoot = path.resolve(fernDir, ".."); const reloadTrigger = path.join(fernDir, "docs.yml"); const reloadDebounceMs = 150; const ignoredPrefix = "fern/"; +const openapiInputPrefix = "fern/openapi/"; +const publicOpenapiPath = "fern/openapi/openapi.public.yaml"; +const filterOpenapiScriptPath = "fern/scripts/filter-public-openapi.mjs"; let debounceTimer = null; +let pendingReloadRequiresOpenapi = false; let shuttingDown = false; function log(message) { @@ -36,20 +40,40 @@ function clearPendingReload() { debounceTimer = null; } -function scheduleReload(relativePath) { +function scheduleReload(relativePath, requiresOpenapiPrepare = false) { clearPendingReload(); + pendingReloadRequiresOpenapi = pendingReloadRequiresOpenapi || requiresOpenapiPrepare; debounceTimer = setTimeout(() => { debounceTimer = null; - touchReloadTrigger(relativePath).catch((error) => { - log(`failed to trigger reload: ${error.message}`); - }); + const shouldRunOpenapiPrepare = pendingReloadRequiresOpenapi; + pendingReloadRequiresOpenapi = false; + Promise.resolve() + .then(() => { + if (shouldRunOpenapiPrepare) { + prepareOpenapi(); + } + }) + .then(() => touchReloadTrigger(relativePath)) + .catch((error) => { + log(`failed to trigger reload: ${error.message}`); + }); }, reloadDebounceMs); } -function shouldIgnore(relativePath) { - const normalized = path.posix.normalize(relativePath.split(path.sep).join("/")); - return normalized.startsWith(ignoredPrefix); +function normalizeWatchedPath(relativePath) { + return path.posix.normalize(relativePath.split(path.sep).join("/")); +} + +function shouldPrepareOpenapi(normalizedPath) { + return ( + normalizedPath === filterOpenapiScriptPath || + (normalizedPath.startsWith(openapiInputPrefix) && normalizedPath !== publicOpenapiPath) + ); +} + +function shouldIgnore(normalizedPath) { + return normalizedPath.startsWith(ignoredPrefix) && !shouldPrepareOpenapi(normalizedPath); } function prepareOpenapi() { @@ -66,22 +90,26 @@ function prepareOpenapi() { } } -prepareOpenapi(); +function spawnFernDev() { + prepareOpenapi(); + return spawn("npx", ["-y", "fern-api@latest", "docs", "dev"], { + cwd: fernDir, + stdio: "inherit", + }); +} -const fern = spawn("npx", ["-y", "fern-api@latest", "docs", "dev"], { - cwd: fernDir, - stdio: "inherit", -}); +const fern = spawnFernDev(); const watcher = watch( docsRoot, { recursive: true }, (_eventType, filename) => { const relativePath = filename ? filename.toString() : ""; - if (shouldIgnore(relativePath)) { + const normalizedPath = normalizeWatchedPath(relativePath); + if (shouldIgnore(normalizedPath)) { return; } - scheduleReload(relativePath); + scheduleReload(relativePath, shouldPrepareOpenapi(normalizedPath)); }, ); diff --git a/services/core/auth/scripts/auth-tools.py b/services/core/auth/scripts/auth-tools.py index 16ed99e2f8..f691e6853d 100755 --- a/services/core/auth/scripts/auth-tools.py +++ b/services/core/auth/scripts/auth-tools.py @@ -1126,6 +1126,17 @@ def _action_sort_key(action: str) -> tuple: return (_ACTION_ORDER.get(action, 100), action) +def _build_docs_area_groups(registry: Dict[str, Dict]) -> Dict[str, List[str]]: + """Group registered permissions by area, excluding areas hidden from public docs.""" + area_groups: Dict[str, List[str]] = {} + for perm_name in sorted(registry.keys()): + area = perm_name.split(".")[0] + if area in DOCS_EXCLUDED_PERMISSION_AREAS: + continue + area_groups.setdefault(area, []).append(perm_name) + return area_groups + + def _build_grouped_rows( perm_names: List[str], registry: Dict[str, Dict], @@ -1198,13 +1209,7 @@ def _generate_permissions_reference(auth_config: Dict) -> str: for role_name in roles_data: role_perms_map[role_name] = extract_role_permissions_recursive(roles_data, role_name) - area_groups: Dict[str, List[str]] = {} - for perm_name in sorted(registry.keys()): - area = perm_name.split(".")[0] - if area in DOCS_EXCLUDED_PERMISSION_AREAS: - continue - area_groups.setdefault(area, []).append(perm_name) - + area_groups = _build_docs_area_groups(registry) ordered_roles = ["Viewer", "Editor", "Admin"] lines: List[str] = [] @@ -1308,13 +1313,14 @@ def generate_docs( auth_config = load_yaml(auth_path) content = _generate_permissions_reference(auth_config) registry = extract_registered_permissions(auth_config) + area_groups = _build_docs_area_groups(registry) output_path.parent.mkdir(parents=True, exist_ok=True) output_path.write_text(content, encoding="utf-8") - area_count = len({name.split(".")[0] for name in registry}) + permission_count = sum(len(perm_names) for perm_names in area_groups.values()) console.print(f"[green]✓ Generated permissions reference at {output_path}[/green]") - console.print(f"[dim]{len(registry)} permissions across {area_count} areas[/dim]") + console.print(f"[dim]{permission_count} permissions across {len(area_groups)} areas[/dim]") def extract_role_permissions_recursive( From 47fc5e83e24ebf999ea918fc79d93b7a107cdc69 Mon Sep 17 00:00:00 2001 From: Matt Kornfield Date: Thu, 25 Jun 2026 09:25:11 -0700 Subject: [PATCH 3/3] chore: remove tool workflow ref, update auditor endpoint Signed-off-by: Matt Kornfield --- docs/auth/authorization/api-scopes.mdx | 2 +- docs/customizer/tutorials/format-training-dataset.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/auth/authorization/api-scopes.mdx b/docs/auth/authorization/api-scopes.mdx index d8dfd12272..fd65ae6d09 100644 --- a/docs/auth/authorization/api-scopes.mdx +++ b/docs/auth/authorization/api-scopes.mdx @@ -25,7 +25,7 @@ Each API has a read and write scope. A token with an API-specific scope can only | API | Read Scope | Write Scope | Endpoints | |------|-----------|-------------|------| -| Audit | `audit:read` | `audit:write` | `/apis/audit/` | +| Audit | `audit:read` | `audit:write` | `/apis/auditor/` | | Auth | `auth:read` | `auth:write` | `/apis/auth/` | | Data Designer | `data-designer:read` | `data-designer:write` | `/apis/data-designer/` | | Entities | `entities:read` | `entities:write` | `/apis/entities/` (internal — see note) | diff --git a/docs/customizer/tutorials/format-training-dataset.mdx b/docs/customizer/tutorials/format-training-dataset.mdx index ee575d7209..b08d157467 100644 --- a/docs/customizer/tutorials/format-training-dataset.mdx +++ b/docs/customizer/tutorials/format-training-dataset.mdx @@ -236,7 +236,7 @@ For illustrative purposes only, we show an example entry as multi-line JSON. ##### Shared Tools -When your dataset uses the same set of tools across all examples, include the shared tool definitions in **each training row** (Automodel jobs do not accept a separate `dataset_parameters` block). The full tool-calling workflow is maintained in the gated example applications section and will be linked here when that section ships publicly. +When your dataset uses the same set of tools across all examples, include the shared tool definitions in **each training row** (Automodel jobs do not accept a separate `dataset_parameters` block). ```python import os