ci(docs): regenerate API references on main after a merge - #13050
ci(docs): regenerate API references on main after a merge#13050dagil-nvidia wants to merge 4 commits into
Conversation
api_freshness.py scopes the per-PR gate so a branch is only failed for drift it caused. That is right -- the alternative fails whoever is unlucky at merge time -- but it rests on the second half of its own docstring: "the next regeneration on main resolves it". Nothing regenerates main. Drift accumulates until a human notices, twice on 2026-08-11 alone (#12985, #13035), and the cost is not a red check: the strict check runs before the sync/publish steps in fern-docs.yml, so stale output skips the publish and docs.nvidia.com stops updating. This job supplies the missing half. It pushes a branch and stops there. Two other routes are closed on this repo, and I confirmed both by reading failing runs rather than assuming: * push to main -- GH013 "Changes must be made through a pull request". community-events-refresh.yml does this and has failed on every run; its commit has never landed. * peter-evans/create-pull-request -- "GitHub Actions is not permitted to create or approve pull requests". update-events.yml does this and is likewise failing every run. So this mirrors auto-dep-upgrade-trigger.yml, the one automation here that lands: push a branch with OPS_BOT_PAT, idempotent on branch existence, and let a human open the PR. Both dead patterns are named in the header so the next author does not re-derive them. The branch name keys on the main SHA that drifted, so re-running the same merge reuses the branch instead of stacking near-identical ones. Only docs/fern is staged. gen_kubernetes_api runs, but its first hop (make generate-api-docs over the Go types) needs Go and crd-ref-docs and stays a human's job -- running the renderer still catches renderer-side drift. Validation: YAML parses; the only uses: entries are the pinned checkout and setup-python actions; the new file is covered by the ci filter, so the coverage gate passes; a dry run of all four generators on current main produces no diff, so the job correctly exits without pushing when main is clean. Signed-off-by: Dan Gil <dagil@nvidia.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| # Keyed on the main SHA that drifted, so a re-run of the same merge | ||
| # reuses the branch instead of stacking near-identical ones. | ||
| BRANCH="docs/regen-api-references-${GITHUB_SHA::9}" | ||
| echo "name=$BRANCH" >> "$GITHUB_OUTPUT" | ||
| if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then | ||
| echo "Branch $BRANCH already exists on origin; skipping." | ||
| echo "exists=true" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "exists=false" >> "$GITHUB_OUTPUT" | ||
| fi |
There was a problem hiding this comment.
🟡 Every merge to main creates another near-identical docs branch while an earlier fix is still open
The regeneration branch is named from the commit that triggered the run (BRANCH="docs/regen-api-references-${GITHUB_SHA::9}" at .github/workflows/regenerate-api-references.yml:92) rather than from the content that changed, so as long as the outstanding fix has not been merged every later change to the main line pushes yet another branch holding the same regenerated files.
Impact: The repository fills up with duplicate documentation branches — potentially one per merge until someone merges the fix.
Why the existence check does not prevent accumulation
The workflow runs on every push to main (.github/workflows/regenerate-api-references.yml:33-37). Drift is measured against what is committed on main, so it persists until the human-opened PR from the pushed branch merges. Suppose merge A introduces drift: run A pushes docs/regen-api-references-<A>. Before that branch is merged, unrelated merge B lands (a README edit is enough) — the regeneration step still reports drifted=true, the idempotency check at .github/workflows/regenerate-api-references.yml:94 looks only for docs/regen-api-references-<B> which does not exist, and a second branch with essentially identical content is pushed. The same repeats for C, D, ... The header comment claims the SHA key avoids "stacking near-identical ones", but it only deduplicates re-runs of the same SHA.
A content-derived key (e.g. a hash of the regenerated diff) or a check for any existing docs/regen-api-references-* branch whose tree matches would make the dedup match the stated intent.
Prompt for agents
In .github/workflows/regenerate-api-references.yml the regeneration branch name is keyed on GITHUB_SHA. Because the workflow runs on every push to main and drift persists on main until the human-opened PR merges, each subsequent unrelated merge produces a new branch with essentially the same regenerated content, so branches accumulate rather than being reused. Consider keying the branch on the content of the regeneration (for example a short hash of `git diff -- docs/fern`) or checking whether any existing origin branch matching docs/regen-api-references-* already carries the same regenerated tree, and skipping the push in that case.
Was this helpful? React with 👍 or 👎 to provide feedback.
| for g in gen_python_api gen_rust_api gen_kubernetes_api gen_llms_tables; do | ||
| python3 "docs/fern/scripts/${g}.py" | ||
| done |
There was a problem hiding this comment.
🔍 Generated pages that are new files may require navigation updates the bot does not make
gen_python_api.py creates and deletes per-module .mdx pages (docs/fern/scripts/gen_python_api.py:104-118 orphan handling), but no generator writes Fern navigation (docs.yml). If a regeneration adds or removes a module page, the pushed branch will carry the page change without the corresponding nav entry, and the Fern configuration check may fail on the resulting PR. This is a pre-existing generator limitation rather than something the workflow introduces, but the automation makes it more likely to surface unattended.
Was this helpful? React with 👍 or 👎 to provide feedback.
WalkthroughChangesAPI Reference Regeneration
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/regenerate-api-references.yml:
- Around line 90-94: Derive the regenerated API reference branch and commit
metadata from the checked-out revision instead of the triggering event’s
GITHUB_SHA. In the branch-generation step around BRANCH and the later commit
metadata at the corresponding commit block, read the SHA from HEAD once and
reuse it for the branch name and commit message, preserving the existing
idempotency behavior.
- Around line 63-64: Update the “Install generator dependencies” step in the
regenerate workflow to pin PyYAML to the tested version (for example,
PyYAML==6.0.3) or install dependencies through a committed constraints file,
while preserving the existing griffe pin.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 7c823952-dc4a-48f0-80b4-6d782f85c861
📒 Files selected for processing (1)
.github/workflows/regenerate-api-references.yml
| - name: Install generator dependencies | ||
| run: python3 -m pip install 'griffe==2.1.0' pyyaml |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find existing dependency constraints and YAML consumers before selecting the pin.
fd -HI -t f . | rg '/(requirements[^/]*\.txt|constraints[^/]*\.txt|pyproject\.toml|uv\.lock|poetry\.lock|Pipfile\.lock)$|^(requirements[^/]*\.txt|constraints[^/]*\.txt|pyproject\.toml|uv\.lock|poetry\.lock|Pipfile\.lock)$'
rg -n -i 'pyyaml|^from yaml\b|^import yaml\b' .Repository: ai-dynamo/dynamo
Length of output: 11263
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- workflow ---'
sed -n '45,80p' .github/workflows/regenerate-api-references.yml
printf '%s\n' '--- existing PyYAML constraints ---'
sed -n '145,165p' pyproject.toml
sed -n '25,42p' container/deps/requirements.common.txt
sed -n '48,60p' container/deps/requirements.test.txt
sed -n '10,22p' deploy/utils/requirements.txt
printf '%s\n' '--- generator dependency references ---'
rg -n -C 2 'griffe|pyyaml|generate|resolve_translation_links|check_reference' .github/workflows/regenerate-api-references.yml docs/fern/scriptsRepository: ai-dynamo/dynamo
Length of output: 50372
Pin the PyYAML release.
Install the tested version, such as PyYAML==6.0.3, or use a committed constraints file. The current unbounded dependency makes regeneration non-reproducible.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/regenerate-api-references.yml around lines 63 - 64, Update
the “Install generator dependencies” step in the regenerate workflow to pin
PyYAML to the tested version (for example, PyYAML==6.0.3) or install
dependencies through a committed constraints file, while preserving the existing
griffe pin.
| # Keyed on the main SHA that drifted, so a re-run of the same merge | ||
| # reuses the branch instead of stacking near-identical ones. | ||
| BRANCH="docs/regen-api-references-${GITHUB_SHA::9}" | ||
| echo "name=$BRANCH" >> "$GITHUB_OUTPUT" | ||
| if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Derive branch metadata from the checked-out revision.
actions/checkout resolves main when the job runs. GITHUB_SHA remains the SHA from the triggering event. If another merge lands before checkout, this job regenerates newer main but creates a branch and commit message for the older SHA. This can create duplicate regeneration branches and break idempotency.
Read the SHA from HEAD and use it for both the branch name and commit metadata.
Proposed fix
- BRANCH="docs/regen-api-references-${GITHUB_SHA::9}"
+ MAIN_SHA="$(git rev-parse --short=9 HEAD)"
+ echo "main_sha=$MAIN_SHA" >> "$GITHUB_OUTPUT"
+ BRANCH="docs/regen-api-references-${MAIN_SHA}"
echo "name=$BRANCH" >> "$GITHUB_OUTPUT"
...
- -m "Automated regeneration after ${GITHUB_SHA::9} moved a documented symbol." \
+ -m "Automated regeneration after ${{ steps.branch.outputs.main_sha }} moved a documented symbol." \Also applies to: 111-114
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/regenerate-api-references.yml around lines 90 - 94, Derive
the regenerated API reference branch and commit metadata from the checked-out
revision instead of the triggering event’s GITHUB_SHA. In the branch-generation
step around BRANCH and the later commit metadata at the corresponding commit
block, read the SHA from HEAD once and reuse it for the branch name and commit
message, preserving the existing idempotency behavior.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5d7295caa0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Keyed on the main SHA that drifted, so a re-run of the same merge | ||
| # reuses the branch instead of stacking near-identical ones. | ||
| BRANCH="docs/regen-api-references-${GITHUB_SHA::9}" | ||
| echo "name=$BRANCH" >> "$GITHUB_OUTPUT" | ||
| if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then |
There was a problem hiding this comment.
When a repair for SHA A is waiting for a human to open or merge its PR, any subsequent main push SHA B still observes the same drift but computes a different branch name and pushes another regeneration commit. The concurrency group only serializes these runs, while this existence check deduplicates reruns of the same SHA, so a burst of merges accumulates near-identical branches; if B adds further API drift and a maintainer merges the older A branch, main remains stale and Fern publishing continues to be skipped. Reuse or update one outstanding regeneration branch, or detect an existing branch by the regeneration prefix.
Useful? React with 👍 / 👎.
| echo "Pushed \`${{ steps.branch.outputs.name }}\`. Open a PR from it to" | ||
| echo "restore \`main\`; until it merges, \`fern-docs.yml\` skips the publish" | ||
| echo "and docs.nvidia.com will not update." |
There was a problem hiding this comment.
When drift is found, the workflow finishes successfully after writing this run summary, so no reviewer is actually handed the branch and production docs remain blocked until someone independently inspects the Actions run and opens a PR. The claimed precedent does not have this gap: .github/workflows/auto-dep-upgrade-complete.yml lines 52-87 uses the same OPS_BOT_PAT with gh pr create to open or reuse a draft PR, and lines 89-107 notify Slack. Use that working PAT-based handoff here (or add an equivalent explicit notification), rather than relying on a passive summary that leaves the regeneration branch undiscovered.
Useful? React with 👍 / 👎.
This comment has been minimized.
This comment has been minimized.
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
Changes the reconciler to push straight to main when it can, and to a branch when the rules reject it. A pure push would be dead on arrival. main is covered by two rulesets that each require a pull request, and GitHub enforces the union: @ai-dynamo/autobots already holds bypass on 4130136 ("Required PR Checks"), but 3932713 ("Default branches") grants bypass to nobody and overrules it, so every push dies on GH013. Granting autobots bypass on 3932713 turns this into a pure push with no further change here. Shipping it push-only before that grant would repeat a failure this repo already has twice over. community-events-refresh.yml pushes to main and has never once succeeded -- 5 runs, 0 successes, its commit has never landed, and the Community Events section has never refreshed. update-events.yml uses create-pull-request and has failed 100 runs since 2026-07-17. Both merged green, because a workflow file passes CI without anything executing it, and both sat broken unnoticed. So the fallback is not hedging. It is the difference between a reconciler that works today and one that silently does not. Retry semantics differ by cause, deliberately: a rules rejection breaks out immediately, since retrying cannot fix a permission, while a race with a concurrent merge regenerates against the new main and retries, because these files have no merge semantics and re-running the generators is always the right resolution. The step summary says which path ran, and when it falls back it names the exact ruleset to change. Validation: YAML parses; every run: block passes bash -n; the header no longer claims it never pushes to main. Signed-off-by: Dan Gil <dagil@nvidia.com>
Two fixes to the reconciler. The loop guard was dropped when this was rewritten to push to main. It matters here and not in community-events-refresh.yml, because this workflow triggers on push to main AND pushes to main with a PAT, and PAT pushes re-trigger workflows where GITHUB_TOKEN pushes do not. Each run that pushed would cause a second run; that run regenerates nothing and exits, so it terminates, but it doubles the run count and reads as if two merges drifted. The header claimed direct push is blocked because two rulesets each require a PR and GitHub enforces the union. That is wrong. community-events-refresh.yml pushed to main successfully at 23:57Z and landed 3d77ee3 while ruleset 3932713 still required a PR with zero bypass actors -- so bypass on 4130136 alone is sufficient, and the union theory is falsified. The fallback stays anyway, because the demonstration is narrower than it looks: one manual dispatch. The four scheduled runs before it failed on GH013, an identical dispatch the day before also failed, and no scheduled run has succeeded yet. Something changed that is not visible in the rulesets, and 'on: push' is untested by either path. Validation: YAML parses; the job carries the guard; no 'enforces the union' claim remains. Signed-off-by: Dan Gil <dagil@nvidia.com>
…elf-heal Two gaps found pressure-testing this, both of which matter more now that the publish gate is staying blocking rather than being relaxed. Blast radius. The happy path pushes to main unreviewed, so an unbounded diff is the one way this job can do real damage: a merge that moves a documented symbol touches a handful of pages, but a BROKEN generator rewrites all of them. Real regenerations for scale -- #12985 was 4 files, #13035 was 6 -- so the cap is 20 files or 2000 lines. Over that, the direct push is skipped entirely and the change goes to a branch for review. Silent failure. The job previously exited 0 after falling back to a branch. With the gate blocking, a fallback is not a nuisance, it is a publish outage: main carries stale references and docs.nvidia.com will not update until someone opens that PR. A green run hides exactly that, which is how update-events.yml failed 100 times without anyone noticing. The job now fails on the fallback path and routes through notify-slack.yml, the same notifier nightly and post-merge already use. Deliberate asymmetry: pushing to main is a success, falling back is a failure. Both leave the tree correct; only one leaves the publish blocked. Validation: YAML parses, both jobs resolve, every run: block passes bash -n. Signed-off-by: Dan Gil <dagil@nvidia.com>
|
Updated after pressure-testing this. Three changes, and one correction to the record. Correction. An earlier revision of this description claimed direct push to The scheduled run I expected to settle it exited early on "No calendar change" — my own earlier dispatch had already pushed the calendar, so there was nothing left to push. The experiment tested nothing. Guards added, both of which matter more now that #13056 is closed and the publish gate stays blocking:
🤖 Addressed by Claude Code |
1 participant
Summary
api_freshness.pyscopes the per-PR gate so a branch is only failed for drift it caused. That isright. But it rests on the second half of its own docstring — "the next regeneration on main
resolves it" — and nothing regenerates main.
Drift accumulates until a human notices. Twice on 2026-08-11 alone (#12985, #13035), by two
different people. The cost is not a red check: the strict check runs before the sync/publish
steps in
fern-docs.yml, so stale output skips the publish and docs.nvidia.com stops updating.This job is the missing half. It runs after every merge to
main, regenerates, and pushes.How drift actually arrives
Not from PRs that skip regeneration — the gate catches those. It arrives through merge skew,
which no per-PR check can see:
#13035's diff confirms it:
362 → 365public symbols,88 → 89in_core,"74 classes and 14 functions" → "15 functions". Real API surface landing between one PR's check and its merge.Push behaviour
Pushes straight to
mainwhen it can, falls back to a branch when the rules reject it, so it iscorrect under either permission state and needs no flag day.
Direct push is unproven unattended. One manual
workflow_dispatchsucceeded on 2026-08-11 at23:57Z (landing
3d77ee344forcommunity-events-refresh.yml, its first success after fivefailures) — but that was nine minutes after ruleset
3932713was modified, and the config todaystill shows a
pull_requestrule withbypass_actors: nullon~DEFAULT_BRANCH. The nextscheduled run exited early with "No calendar change", so it never exercised the push. The
fallback is therefore load-bearing, not insurance.
Guards
mainunreviewed, so an unbounded diff is the one way this can do real damage. A merge that moves a symbol touches a handful of pages; a broken generator rewrites all of them. Real regenerations for scale: #12985 was 4 files, #13035 was 6. Over the cap, the direct push is skipped and it goes to a branch.notify-slack.ymlmaincarries stale references until someone opens that PR. A green run hides it, which is howupdate-events.ymlfailed 100 times unnoticed.github.actor != 'dynamo-ops'mainand pushes tomainwith a PAT, and PAT pushes re-trigger workflows whereGITHUB_TOKENpushes do not.Deliberate asymmetry: pushing to
mainis a success, falling back is a failure. Both leave thetree correct; only one leaves the publish blocked.
Why not the alternatives
continue-on-error) — tried in fix(docs): stop generated-reference drift from blocking the publish #13056, closed. The gate is alsothe only forcing function; drift got fixed twice today precisely because the publish broke and
someone felt it. Nothing notifies on drift, so that change would have traded a loud signal for
silence.
peter-evans/create-pull-request—"GitHub Actions is not permitted to create or approve pull requests".update-events.ymldoes this and has failed every run since 2026-07-17.api_freshness.pywas builtto remove, and still misses skew, which is only visible after the merge.
Validation
run:block passesbash -n.checkoutandsetup-pythonactions are used.cifilter, so the changed-files coverage gate passes.mainproduces no diff — the job correctlyexits without pushing when main is clean.
Known limitation
gen_kubernetes_api.pyre-renders a committed intermediate thatmake generate-api-docsproducesfrom the operator Go types. That first hop needs Go and
crd-ref-docs, so it stays a human's job.Running the renderer here still catches renderer-side drift, but not Go-type drift.
🤖 Generated with Claude Code