docs: realign agent guides with the tab structure and add a docs lint job

[dagil-nvidia]

Summary

The tab restructure (#10855) moved docs content from docs/<section>/ to docs/fern/pages/<tab>/, but the agent-facing guides were never updated. They still described a two-tab site, told authors to create files at docs/<subdirectory>/<file>.md, and pointed at three paths that no longer exist. This realigns them and closes the enforcement gap that let the drift happen silently.

Two halves. Add a blocking Docs Lint job, and clean the docs tree to zero errors so that job gates all of docs/ rather than only changed files. The second half is what makes the first meaningful: a gate scoped to changed files would grandfather every existing violation, so the tree is cleared first and the job blocks from day one.

New: docs/fern/pages/AGENTS.md. A placement guide for the content tree: the nine tabs and the directory each is rooted at (the nav key and directory differ for the two guide tabs), the Kubernetes/CLI split, a tab-selection ladder, the nav grammar, and the URL model. Placement is the decision worth getting right, because fixing it later costs a move plus a redirect.

docs/fern/AGENTS.md becomes the mechanics guide. Every original rule is kept; it gains a pointer to the new file, a map of docs/fern/ separating content from site config, and the sync_site_css.py and docs-website gates.

The dynamo-docs skill is corrected throughout: content paths, nav grammar (tabs: map, path: relative to docs/fern/), the tab table and Kubernetes/CLI split, the URL join rule, the translations path and its ../ depth, the assets path, and the Key References table.

docs/fern/templates/ cleanup is no longer part of this PR. It originally deleted twelve stale flat template files describing a taxonomy the restructure removed. #13138 has since deleted the same files on main and replaced them with a tab-aligned directory structure, so that half is redundant and has dropped out of the diff. Scope is now 20 files.

The enforcement gap

Both docs/fern/AGENTS.md and the skill claimed a "Dynamo Docs Bot" enforced the structural rules pre-merge. No such workflow existed. Real enforcement was fern check, fern docs broken-links, and repo-wide lychee, which cover nav structure and link resolution but not SPDX headers, body # H1, or internal references. Authors were told CI would catch things it never looked at.

This adds docs/fern/scripts/docs_lint.py and wires it as the Docs Lint job in pre-merge.yml, which runs on every pull_request alongside its sibling Fern jobs, and registers it in pre-merge-status-check so it actually gates merge. That token is read-only, so the job reports through inline annotations on the offending line plus a job summary rather than a PR comment.

The linter's navigation check had been a silent no-op: it looked for docs/index.yml, which the restructure replaced, and returned early when absent. It now reads the real nav and resolves every path: against docs/fern/.

What the linter found

Running it surfaced two pages that were live on the docs site:

• pages/kubernetes/installation/model-storage/overview.md was 4 bytes containing the word TODO, shipping as "Overview" under Kubernetes Guide → Installation → Model Storage.
• pages/reference/general/examples.md was 0 bytes, empty, shipping as "Examples" under Reference → General.

Both are removed from the nav and deleted, along with the dangling <Card> that pointed at the first. The remaining baseline is cleaned to zero so the job gates the whole docs/ tree rather than only changed files: ten relative links that escaped docs/ converted to github.com URLs, one missing SPDX header, and one comment-only frontmatter fixed in deploy/operator/docs/header.md so regeneration keeps it.

Rebased on main

Merged main on 2026-08-22. Two conflicts, both resolved toward the newer intent: the
pre-merge.yml needs list takes main's, which dropped the snapshot job, and keeps
docs-lint; docs/fern/AGENTS.md keeps both sides, since main added the reviewer note on
curated Kubernetes pages while this branch added the regenerate-before-merge guidance and
the Validate section.

One adaptation was required. #13556 stopped committing the Python and Rust API references
and gitignored both trees, so they are generated at publish time and absent from a clean
checkout. The linter did not know that and reported 29 errors against nav entries and links
targeting them, which would have made this gate fail on its first run. docs_lint.py now
treats pages/reference/api/{python,rust} as satisfied for nav and link resolution, with a
keep-in-sync note against docs/fern/.gitignore.

That left two genuine errors, both in pages/recipes/model-recipes/qwen-3-8-2-4t-a95b-fp8.mdx:
a ../kubernetes/quickstart.mdx link that resolves nowhere, where every sibling recipe page
uses ../../kubernetes/getting-started/quickstart.mdx. Fixed. The linter catching real drift
on its first run against a tree it had not seen is the argument for the job.

Validation

python3 docs/fern/scripts/docs_lint.py --scan docs   # 0 errors, 450 files
cd docs/fern && fern check                 # 0 errors
python3 scripts/validate_skills.py .       # validated 17 skills: OK
pre-commit run --all-files                 # clean

The nav check was verified to actually fire by injecting a dangling path: into index.yml: it reported the correct file and line and exited 1. Both linter output paths were exercised against a scratch tree, confirming the annotation format, the job summary table, and the exit codes. The translations ../ depth rule stated in the skill was checked against all nine translated files (depth 2 uses 5, depth 4 uses 7).

Follow-up

examples/ and recipes/ carry a separate backlog of 32 missing SPDX headers and one broken link, so the job scans docs/ only for now. Widening --scan should land with those fixes, in a PR scoped to those owners.

🤖 Generated with Claude Code

---

Summary by CodeRabbit

• Documentation

  • Updated documentation guidance for improved organization, navigation, localization, and authoring workflows.
  • Removed outdated navigation entries, templates, placeholders, and obsolete page references.
  • Improved sidebar labeling and corrected links across the documentation.
  • Clarified recipe placement and publishing requirements.

• Quality Improvements

  • Added automated documentation validation for links, metadata, navigation, headings, licensing, and internal references.
  • Documentation checks now run automatically during pre-merge validation.

[datadog-official]

 

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR | DGDR Deploy Test / CPU / profiling     

See error

Deployment failed due to missing required inputs 'repository' and 'operator_tag' for the operator image during Helm upgrade.

PR | deploy-status-check     

ℹ️ Info

🔄 Datadog auto-retried 1 job - 0 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 48.27% (-5.90%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 91466e9 | Docs | Datadog PR Page | Give us feedback!}

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-01a05f09-8a50-7689-8f39-03ca1e68d324.docs.buildwithfern.com/dynamo/dev

[devin-ai-integration]

Devin Review found 6 potential issues.

[coderabbitai]

Walkthrough

The PR adds a deterministic documentation linter, runs it in pre-merge checks, updates Fern documentation guidance, removes obsolete templates and navigation entries, and revises affected links, metadata, and page-placement instructions.

Changes

Fern documentation tooling and structure

Layer / File(s)	Summary
Documentation linter scripts/docs_lint.py	Adds checks for SPDX headers, frontmatter, duplicate H1 headings, relative links, internal references, TODO markers, hosts, and Fern navigation. It supports local, JSON, and GitHub Actions output.
Pre-merge lint integration .github/workflows/pre-merge.yml	Adds a conditional docs-lint job and makes the pre-merge status check wait for it.
Fern authoring guidance .agents/skills/dynamo-docs/SKILL.md, AGENTS.md, docs/fern/AGENTS.md, docs/fern/pages/AGENTS.md, docs/fern/pages/community/contributing/documentation/documentation-style-guide.md	Updates repository and page-authoring rules for Fern tabs, navigation, assets, translations, versioned pages, redirects, and validation commands.
Navigation and page alignment docs/fern/index.yml, docs/fern/pages/kubernetes/getting-started/introduction.mdx, docs/fern/pages/recipes/_catalog/README.md, docs/fern/pages/reference/kubernetes-api/additional-resources/api-reference-k8s.md, deploy/operator/docs/header.md, docs/fern/templates/*	Removes obsolete navigation entries and templates, moves model recipe guidance under model-recipes/, removes the model storage card, and adds the Kubernetes API sidebar title.
Repository link and license cleanup docs/fern/pages/developer-guide/..., docs/fern/pages/recipes/feature-benchmarks/embedding-cache.md, docs/fern/pages/blog/_assets/flash-indexer/tools/README.md	Replaces repository-relative links with GitHub URLs and adds copyright and Apache-2.0 SPDX metadata to the tool README.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	⚠️ Warning	The description gives detailed scope and validation, but it omits the required Related Issues section and formal issue-link or no-issue confirmation.	Add the required Related Issues section and select either the applicable issue-closing statement or the confirmed no-related-issue checkbox.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly describes the primary documentation-guide updates and the addition of the docs lint job.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 8

🧹 Nitpick comments (3)

scripts/docs_lint.py (3)

334-336: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Default --scan value scans trees the PR does not clean up yet.

The PR objectives state the linter currently scans docs/ only, and CI invokes python3 scripts/docs_lint.py --scan docs --github. The default here is still "docs,examples,recipes", so a local contributor running the script with defaults (per the docstring's own usage example at line 16) will hit pre-existing findings in examples//recipes/ unrelated to their change, since those trees were not part of this PR's baseline cleanup.

Consider defaulting --scan to "docs" until examples//recipes/ are cleaned up, matching the CI invocation.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/docs_lint.py` around lines 334 - 336, Update the --scan argument
definition in the argument parser to default to "docs" instead of
"docs,examples,recipes", matching the documented and CI scan scope while
preserving explicit user-provided scan trees.

---

275-286: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Prune .git/node_modules from os.walk instead of skipping after descent.

for dirpath, _, names in os.walk(root): discards dirnames, so the continue at line 282 only skips file collection once already inside .git or node_modules — os.walk still recurses into every subdirectory of these trees before the skip takes effect. This wastes time scanning large .git object stores or node_modules trees (e.g., a local docs/fern/node_modules/ from Fern site tooling) on every run. Prune dirnames in place to stop the walk from entering these directories at all.

⚡ Proposed fix

-        for dirpath, _, names in os.walk(root):
-            if "/.git" in dirpath or "/node_modules" in dirpath:
-                continue
+        for dirpath, dirnames, names in os.walk(root):
+            dirnames[:] = [d for d in dirnames if d not in (".git", "node_modules")]
             for n in names:
                 if n.endswith(exts):
                     files.append(os.path.join(dirpath, n))

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/docs_lint.py` around lines 275 - 286, Update gather to retain the
mutable dirnames value from os.walk and remove .git and node_modules entries
from it before recursion. Keep collecting only the existing extensions and
preserve the current file filtering behavior while preventing traversal into
excluded directories.

---

1-413: 📐 Maintainability & Code Quality | 🔵 Trivial

Consider adding unit tests for the new linter.

scripts/docs_lint.py is a new, CI-blocking tool with non-trivial regex and traversal logic (frontmatter parsing, link resolution, fence blanking). No test file is included in this review. Per the applicable path instruction, tests for .py files should follow the repository's pytest guidelines.
[recommended_refactor]

As per path instructions, **/{pyproject.toml,*.py,*.pyi}: "Read .ai/pytest-guidelines.md and .ai/test-model-size-guardrails.md before writing tests, and follow their pytest and model-size constraints."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/docs_lint.py` around lines 1 - 413, Add pytest coverage for
scripts/docs_lint.py, following the repository’s pytest and model-size
guidelines in .ai/pytest-guidelines.md and .ai/test-model-size-guardrails.md.
Exercise the core helpers—frontmatter and blank_code parsing, SPDX/frontmatter
validation, link resolution, internal-content detection, navigation checks, and
the main scan flow—using temporary files and focused cases for valid and invalid
inputs.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/dynamo-docs/SKILL.md:
- Line 170: Update the deletion workflow in the skill instructions around the
git rm command to require explicit user confirmation immediately before
executing git rm, and only run the command after confirmation is received.
- Line 170: Update the page-removal instructions around the git rm command to
support both .md and .mdx files. In the removal workflow, use the actual
extension of the page being deleted and apply the same extension when removing
its navigation entry, rather than always targeting <filename>.md.
- Around line 154-160: Update the redirect guidance in
.agents/skills/dynamo-docs/SKILL.md lines 154-160 to require redirects only when
a tab, section, page label, or explicit slug changes the public URL; do not
require them for file-only moves. Apply the same condition in
docs/fern/pages/AGENTS.md lines 95-111 while retaining its prohibition on
unversioned and Latest redirects.

In `@docs/fern/pages/AGENTS.md`:
- Around line 38-45: Update the directory-tree code fence in the documentation
around the shown Kubernetes and CLI listings to include the text language tag on
its opening fence, resolving the MD040 markdownlint violation while leaving the
example content unchanged.
- Around line 124-133: Add python3 scripts/docs_lint.py --scan docs to the
Validate command list in the AGENTS.md checklist, and update the surrounding
statement so it accurately identifies Docs Lint, fern check, and broken-links as
pre-merge checks while keeping the catalog validator as a manual-only check.

In `@scripts/docs_lint.py`:
- Around line 230-254: Update check_internal to call blank_code once on the
complete text, then iterate raw lines alongside the corresponding blanked lines
and apply TODO_RE to the blanked line while preserving raw-line checks and
original line numbers for all other rules.
- Around line 1-20: Add scripts/docs_lint.py to the docs filter in
.github/filters.yaml, preferably by including scripts/** so this script is
covered. The .github/workflows/pre-merge.yml lines 86-101 require no direct
change; the filter update will ensure its fern-check, docs-lint, and
fern-broken-links jobs run for this change.
- Line 52: Update the JIRA_RE pattern used by check_internal to include the OPS
project prefix, so references such as OPS-1234 are recognized and produce the
expected INTERNAL finding while preserving existing project matches.

---

Nitpick comments:
In `@scripts/docs_lint.py`:
- Around line 334-336: Update the --scan argument definition in the argument
parser to default to "docs" instead of "docs,examples,recipes", matching the
documented and CI scan scope while preserving explicit user-provided scan trees.
- Around line 275-286: Update gather to retain the mutable dirnames value from
os.walk and remove .git and node_modules entries from it before recursion. Keep
collecting only the existing extensions and preserve the current file filtering
behavior while preventing traversal into excluded directories.
- Around line 1-413: Add pytest coverage for scripts/docs_lint.py, following the
repository’s pytest and model-size guidelines in .ai/pytest-guidelines.md and
.ai/test-model-size-guardrails.md. Exercise the core helpers—frontmatter and
blank_code parsing, SPDX/frontmatter validation, link resolution,
internal-content detection, navigation checks, and the main scan flow—using
temporary files and focused cases for valid and invalid inputs.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 65c76208-06a0-4dab-8f18-bc88470a943f

📥 Commits

Reviewing files that changed from the base of the PR and between d84042e and 28aa087.

📒 Files selected for processing (30)

• .agents/skills/dynamo-docs/SKILL.md
• .github/workflows/pre-merge.yml
• AGENTS.md
• deploy/operator/docs/header.md
• docs/fern/AGENTS.md
• docs/fern/index.yml
• docs/fern/pages/AGENTS.md
• docs/fern/pages/blog/_assets/flash-indexer/tools/README.md
• docs/fern/pages/community/contributing/documentation/documentation-style-guide.md
• docs/fern/pages/developer-guide/additional-resources/runtime-development-guide.md
• docs/fern/pages/developer-guide/knowledge-base/modular-components/router/router-testing.md
• docs/fern/pages/kubernetes/getting-started/introduction.mdx
• docs/fern/pages/kubernetes/installation/model-storage/overview.md
• docs/fern/pages/recipes/_catalog/README.md
• docs/fern/pages/recipes/feature-benchmarks/embedding-cache.md
• docs/fern/pages/reference/general/examples.md
• docs/fern/pages/reference/kubernetes-api/additional-resources/api-reference-k8s.md
• docs/fern/templates/README.md
• docs/fern/templates/backend-guide.md
• docs/fern/templates/backend-readme.md
• docs/fern/templates/component-design.md
• docs/fern/templates/component-examples.md
• docs/fern/templates/component-guide.md
• docs/fern/templates/component-readme.md
• docs/fern/templates/feature-backend.md
• docs/fern/templates/feature-readme.md
• docs/fern/templates/incode-readme.md
• docs/fern/templates/infrastructure-readme.md
• docs/fern/templates/integration-readme.md
• scripts/docs_lint.py

💤 Files with no reviewable changes (15)

• docs/fern/templates/incode-readme.md
• docs/fern/templates/infrastructure-readme.md
• docs/fern/templates/integration-readme.md
• docs/fern/templates/feature-readme.md
• docs/fern/pages/kubernetes/installation/model-storage/overview.md
• docs/fern/templates/feature-backend.md
• docs/fern/pages/kubernetes/getting-started/introduction.mdx
• docs/fern/templates/component-design.md
• docs/fern/templates/README.md
• docs/fern/index.yml
• docs/fern/templates/backend-readme.md
• docs/fern/templates/component-readme.md
• docs/fern/templates/component-guide.md
• docs/fern/templates/component-examples.md
• docs/fern/templates/backend-guide.md

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 28aa087002

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[dagil-nvidia]

/ok to test 1f43c8b

[dagil-nvidia]

/ok to test 583ec96

[dagil-nvidia]

/ok to test 728bf2c

[harryskim]

lgtm

[dagil-nvidia]

/ok to test 299f323

[dagil-nvidia]

/ok to test ba8b713

[dagil-nvidia]

/ok to test ee00518

[dagil-nvidia]

/ok to test 00fdbcc

[dmitry-tokarev-nv]

Thanks for addressing findings.
Approving to unblock.
Nothing here touches core, functional/perf tests, or any gate other than the docs jobs.
Any docs related issues can be fixed later.

[dagil-nvidia]

/ok to test 7af69c1

[dagil-nvidia]

/ok to test 99813a2

[dagil-nvidia]

/ok to test 6cf2786

[sttts]

For operator.

[dagil-nvidia]

/ok to test 5324137

[dagil-nvidia]

/ok to test 91466e9

[dagil-nvidia]

/ok to test ab0cc26

[dagil-nvidia]

/ok to test 2af874d

[dagil-nvidia]

/ok to test d99da1b

[dagil-nvidia]

Merging with --admin, past a red deploy-status-check. Recording why.

The three DGDR Deploy Test / CPU failures are an admission webhook rejection that has nothing to
do with this PR:

vdynamographdeploymentrequest.kb.io denied the request:
spec.runtimeVersionOverride: Required value: is required when
spec.image has no parseable semantic-version tag

CI tags its images ci-<run-id>-<hash>, which carries no semantic version, and
tests/deploy/test_dgdr.py never sets runtimeVersionOverride. Every DGDR create returns 422. It
is deterministic rather than flaky, so it does not clear on a re-run, and it is failing on #14117 as
well, which is unrelated to this work.

This PR changes documentation, agent guides, and adds the Docs Lint job. It touches no operator
code, no CRD, no deploy test, and no image tagging. Every other required context is green: DCO,
codeowners, copyright-checks, dynamo-status-check, and pre-merge-status-check.

The DGDR break needs the operator owners, not this branch. Either the DGDR tests set
runtimeVersionOverride for CI-tagged images, or the webhook tolerates a non-semver tag on a CI
build.

[dagil-nvidia]

Admin merge note for the audit trail: this PR was approved with every executed check green. The remaining required contexts are path-filtered for this change set and never report, so the merge gate cannot clear on its own - the override bypasses only the non-reporting contexts.