Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 126 additions & 0 deletions .github/workflows/regenerate-api-references.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Keep main's generated API references current.
#
# The per-PR freshness gate is deliberately scoped: api_freshness.py blocks a
# branch only when the branch itself touched something feeding a generator, so
# a PR is never failed for drift main introduced. That scoping is right -- the
# alternative fails whoever is unlucky at merge time. But it assumes the other
# half of the sentence in api_freshness.py's own docstring:
#
# "the next regeneration on main resolves it"
#
# Nothing regenerated main. Drift accumulated instead -- twice on 2026-08-11
# alone (#12985, #13035) -- and the cost is not a red check: the strict check
# runs BEFORE the sync/publish steps in fern-docs.yml, so stale output skips
# the publish and docs.nvidia.com stops updating.
#
# This job is that missing half. It pushes a BRANCH, never main, because two
# other routes are closed on this repo and both fail loudly if reintroduced:
#
# * pushing to main -> GH013 "Changes must be made through a pull
# request" (community-events-refresh.yml has
# failed this way on every run)
# * create-pull-request -> "GitHub Actions is not permitted to create or
# approve pull requests" (update-events.yml)
#
# So this mirrors auto-dep-upgrade-trigger.yml, the one automation here that
# actually lands: push a branch with OPS_BOT_PAT and let a human open the PR.

name: Regenerate API references

on:
push:
branches:
- main
workflow_dispatch:

permissions:
contents: read

# Serialize: two merges landing close together would otherwise race two
# regeneration branches.
concurrency:
group: regenerate-api-references
cancel-in-progress: false

jobs:
regenerate:
name: Regenerate API references
runs-on: ubuntu-latest
steps:
- name: Checkout main
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: main
token: ${{ secrets.OPS_BOT_PAT }}

- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.13'

- name: Install generator dependencies
run: python3 -m pip install 'griffe==2.1.0' pyyaml
Comment on lines +83 to +84

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Find existing dependency constraints and YAML consumers before selecting the pin.
fd -HI -t f . | rg '/(requirements[^/]*\.txt|constraints[^/]*\.txt|pyproject\.toml|uv\.lock|poetry\.lock|Pipfile\.lock)$|^(requirements[^/]*\.txt|constraints[^/]*\.txt|pyproject\.toml|uv\.lock|poetry\.lock|Pipfile\.lock)$'
rg -n -i 'pyyaml|^from yaml\b|^import yaml\b' .

Repository: ai-dynamo/dynamo

Length of output: 11263


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- workflow ---'
sed -n '45,80p' .github/workflows/regenerate-api-references.yml

printf '%s\n' '--- existing PyYAML constraints ---'
sed -n '145,165p' pyproject.toml
sed -n '25,42p' container/deps/requirements.common.txt
sed -n '48,60p' container/deps/requirements.test.txt
sed -n '10,22p' deploy/utils/requirements.txt

printf '%s\n' '--- generator dependency references ---'
rg -n -C 2 'griffe|pyyaml|generate|resolve_translation_links|check_reference' .github/workflows/regenerate-api-references.yml docs/fern/scripts

Repository: ai-dynamo/dynamo

Length of output: 50372


Pin the PyYAML release.

Install the tested version, such as PyYAML==6.0.3, or use a committed constraints file. The current unbounded dependency makes regeneration non-reproducible.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/regenerate-api-references.yml around lines 63 - 64, Update
the “Install generator dependencies” step in the regenerate workflow to pin
PyYAML to the tested version (for example, PyYAML==6.0.3) or install
dependencies through a committed constraints file, while preserving the existing
griffe pin.


- name: Regenerate
id: regen
run: |
set -euo pipefail
# gen_kubernetes_api.py re-renders a committed intermediate that
# `make generate-api-docs` produces from the operator Go types. That
# first hop needs Go and crd-ref-docs, so it stays a human's job;
# running the renderer here still catches renderer-side drift.
for g in gen_python_api gen_rust_api gen_kubernetes_api gen_llms_tables; do
python3 "docs/fern/scripts/${g}.py"
done
Comment on lines +94 to +96

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Generated pages that are new files may require navigation updates the bot does not make

gen_python_api.py creates and deletes per-module .mdx pages (docs/fern/scripts/gen_python_api.py:104-118 orphan handling), but no generator writes Fern navigation (docs.yml). If a regeneration adds or removes a module page, the pushed branch will carry the page change without the corresponding nav entry, and the Fern configuration check may fail on the resulting PR. This is a pre-existing generator limitation rather than something the workflow introduces, but the automation makes it more likely to surface unattended.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

if [ -z "$(git status --porcelain -- docs/fern)" ]; then
echo "All generated references current; nothing to do."
echo "drifted=false" >> "$GITHUB_OUTPUT"
else
echo "Drift detected:"
git status --porcelain -- docs/fern
echo "drifted=true" >> "$GITHUB_OUTPUT"
fi

- name: Compute branch name + idempotency check
if: steps.regen.outputs.drifted == 'true'
id: branch
run: |
# Keyed on the main SHA that drifted, so a re-run of the same merge
# reuses the branch instead of stacking near-identical ones.
BRANCH="docs/regen-api-references-${GITHUB_SHA::9}"
echo "name=$BRANCH" >> "$GITHUB_OUTPUT"
if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Derive branch metadata from the checked-out revision.

actions/checkout resolves main when the job runs. GITHUB_SHA remains the SHA from the triggering event. If another merge lands before checkout, this job regenerates newer main but creates a branch and commit message for the older SHA. This can create duplicate regeneration branches and break idempotency.

Read the SHA from HEAD and use it for both the branch name and commit metadata.

Proposed fix
-          BRANCH="docs/regen-api-references-${GITHUB_SHA::9}"
+          MAIN_SHA="$(git rev-parse --short=9 HEAD)"
+          echo "main_sha=$MAIN_SHA" >> "$GITHUB_OUTPUT"
+          BRANCH="docs/regen-api-references-${MAIN_SHA}"
           echo "name=$BRANCH" >> "$GITHUB_OUTPUT"
...
-            -m "Automated regeneration after ${GITHUB_SHA::9} moved a documented symbol." \
+            -m "Automated regeneration after ${{ steps.branch.outputs.main_sha }} moved a documented symbol." \

Also applies to: 111-114

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/regenerate-api-references.yml around lines 90 - 94, Derive
the regenerated API reference branch and commit metadata from the checked-out
revision instead of the triggering event’s GITHUB_SHA. In the branch-generation
step around BRANCH and the later commit metadata at the corresponding commit
block, read the SHA from HEAD once and reuse it for the branch name and commit
message, preserving the existing idempotency behavior.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reuse the outstanding regeneration branch across main pushes

When a repair for SHA A is waiting for a human to open or merge its PR, any subsequent main push SHA B still observes the same drift but computes a different branch name and pushes another regeneration commit. The concurrency group only serializes these runs, while this existence check deduplicates reruns of the same SHA, so a burst of merges accumulates near-identical branches; if B adds further API drift and a maintainer merges the older A branch, main remains stale and Fern publishing continues to be skipped. Reuse or update one outstanding regeneration branch, or detect an existing branch by the regeneration prefix.

Useful? React with 👍 / 👎.

echo "Branch $BRANCH already exists on origin; skipping."
echo "exists=true" >> "$GITHUB_OUTPUT"
else
echo "exists=false" >> "$GITHUB_OUTPUT"
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Every merge to main creates another near-identical docs branch while an earlier fix is still open

The regeneration branch is named from the commit that triggered the run (BRANCH="docs/regen-api-references-${GITHUB_SHA::9}" at .github/workflows/regenerate-api-references.yml:92) rather than from the content that changed, so as long as the outstanding fix has not been merged every later change to the main line pushes yet another branch holding the same regenerated files.
Impact: The repository fills up with duplicate documentation branches — potentially one per merge until someone merges the fix.

Why the existence check does not prevent accumulation

The workflow runs on every push to main (.github/workflows/regenerate-api-references.yml:33-37). Drift is measured against what is committed on main, so it persists until the human-opened PR from the pushed branch merges. Suppose merge A introduces drift: run A pushes docs/regen-api-references-<A>. Before that branch is merged, unrelated merge B lands (a README edit is enough) — the regeneration step still reports drifted=true, the idempotency check at .github/workflows/regenerate-api-references.yml:94 looks only for docs/regen-api-references-<B> which does not exist, and a second branch with essentially identical content is pushed. The same repeats for C, D, ... The header comment claims the SHA key avoids "stacking near-identical ones", but it only deduplicates re-runs of the same SHA.

A content-derived key (e.g. a hash of the regenerated diff) or a check for any existing docs/regen-api-references-* branch whose tree matches would make the dedup match the stated intent.

Prompt for agents
In .github/workflows/regenerate-api-references.yml the regeneration branch name is keyed on GITHUB_SHA. Because the workflow runs on every push to main and drift persists on main until the human-opened PR merges, each subsequent unrelated merge produces a new branch with essentially the same regenerated content, so branches accumulate rather than being reused. Consider keying the branch on the content of the regeneration (for example a short hash of `git diff -- docs/fern`) or checking whether any existing origin branch matching docs/regen-api-references-* already carries the same regenerated tree, and skipping the push in that case.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


- name: Commit + push regeneration branch
if: steps.regen.outputs.drifted == 'true' && steps.branch.outputs.exists == 'false'
run: |
set -euo pipefail
git config user.name "dynamo-ops"
git config user.email "170655669+dynamo-ops@users.noreply.github.com"
git checkout -b "${{ steps.branch.outputs.name }}"
# Only ever stage generated docs: a bare `git add -A` would sweep up
# anything else the toolchain left in the workspace.
git add docs/fern
git commit -s \
-m "docs(api): regenerate API references" \
-m "Automated regeneration after ${GITHUB_SHA::9} moved a documented symbol." \
-m "Generated by .github/workflows/regenerate-api-references.yml."
git push origin "${{ steps.branch.outputs.name }}"

- name: Report
if: steps.regen.outputs.drifted == 'true' && steps.branch.outputs.exists == 'false'
run: |
{
echo "### API references drifted"
echo
echo "Pushed \`${{ steps.branch.outputs.name }}\`. Open a PR from it to"
echo "restore \`main\`; until it merges, \`fern-docs.yml\` skips the publish"
echo "and docs.nvidia.com will not update."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Open the repair PR instead of only writing a summary

When drift is found, the workflow finishes successfully after writing this run summary, so no reviewer is actually handed the branch and production docs remain blocked until someone independently inspects the Actions run and opens a PR. The claimed precedent does not have this gap: .github/workflows/auto-dep-upgrade-complete.yml lines 52-87 uses the same OPS_BOT_PAT with gh pr create to open or reuse a draft PR, and lines 89-107 notify Slack. Use that working PAT-based handoff here (or add an equivalent explicit notification), rather than relying on a passive summary that leaves the regeneration branch undiscovered.

Useful? React with 👍 / 👎.

} >> "$GITHUB_STEP_SUMMARY"
Loading