Skip to content

docs: fix factual drift in README, PROMPT.md and tool headers - #16

Merged
eilandert merged 1 commit into
mainfrom
docs/factcheck-readme-prompt
Aug 3, 2026
Merged

eilandert merged 1 commit into
mainfrom
docs/factcheck-readme-prompt

Conversation

@eilandert

Copy link
Copy Markdown
Member

A proofread pass over README.md and ci/*.md, checking each claim against the machinery it describes rather than re-reading its prose. Docs and comment headers only; no workflow, script logic or C changes.

Eleven findings, one class: text that was true when written and drifted as the repo moved under it. Two of them would mislead someone acting on the file.

The two that bite

The quickstart built an unpinned nginx. README.md and rename-module.sh both printed ci-build.sh nginx 1.31.2. The pin is NGINX_MAINLINE=1.31.3, and .github/versions.env carries a sha256 per version string — 1.31.2 has no entry, so a copy-paste build fetches a tarball nothing verifies. That is the one place in the repo where following the docs opts you out of the digest check.

semgrep was documented as advisory; it blocks. The ## CI table listed semgrep (advisory). security-scanners.yml runs it twice: pass 1 full-severity into the artifact, never blocking, then pass 2 with --severity=WARNING --severity=ERROR --error, which gates. The step is called semgrep (gate on >=WARNING). ci/PROMPT.md:515 and ci/linter/README.md both had it right — the README was the only copy that was wrong, and it is the outward-facing one.

The rest

README.md:

  • "eight checkpoints", twice — ci/PROMPT.md is eleven (0–10) in five phases. The prose summary also enumerated nine things while silently omitting the decision seam (cp 1) and the depth pass (cp 10), which are the two newest checkpoints and the ones an adopter most needs to know exist.
  • "~350 lines of C" — src/ is 1073 (520 module + 413 scan + 140 header).
  • ci/fuzz/ described as one target; fuzz_scan and fuzz_body both build.
  • the layout tree omitted the four ci/tools/ bump/versions scripts, and all of .github/versions.env, .github/scripts/ and .github/actions/build-cache/ — so a reader learns the pin file exists only from the Caching section, three screens later.

ci/PROMPT.md:

  • checkpoint count off by one in two places (0–10 inclusive is eleven, not ten), plus "all nine checkpoints have merged" in §10 where the phase table says 0–9.
  • two dangling cross-references to a section 6b. There is no 6b; checkpoint 6 has no sub-parts. The semgrep --jobs=1 rationale they point at is 7b.

ci/linter/README.md: one run-on sentence that glued the LINT_ONLY-duplication warning to an unrelated fact about lint.yml's runner.

Tool headers — ci-build.sh, rename-module.sh, soak.sh, fuzz/build.sh, fuzz/regressions/README.md: usage lines still printed bare tools/… paths from before the ci/ move, so every documented invocation named a path that does not exist.

Testing

Docs-only, so the deterministic gate is the whole gate:

  • ci/linter/run-all.sh — == all linters clean == (includes lint-docs-drift: 10 workflows, all documented, both directions).
  • review-lint.sh --branch main — "no reviewable files in selection", as expected for a diff with no code.
  • The 1.31.2 and bare-tools/ sweeps that found the last six sites now return empty across all tracked *.md, *.sh and *.py.

The four new ci/tools/ layout-tree descriptions were written from each script's own header, not inferred from its name — two of my first drafts were wrong (bump-versions.sh is the umbrella that calls the other two; bump-tools.sh is PyPI linter pins specifically) and are corrected here.

Nothing in this PR is verified by a build: no claim in it depends on one.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@eilandert, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 46 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 28ec419d-99dc-4fbd-97b8-c112dcf49ffa

📥 Commits

Reviewing files that changed from the base of the PR and between 13460fd and b503854.

📒 Files selected for processing (8)
  • README.md
  • ci/PROMPT.md
  • ci/fuzz/build.sh
  • ci/fuzz/regressions/README.md
  • ci/linter/README.md
  • ci/tools/ci-build.sh
  • ci/tools/rename-module.sh
  • ci/tools/soak.sh

Walkthrough

The pull request updates project and CI documentation. It records eleven adoption checkpoints, expanded fuzzing coverage, blocking Semgrep warnings, nginx 1.31.3 examples, and the relocated ci/tools script paths.

Changes

Documentation and CI alignment

Layer / File(s) Summary
Adoption and policy documentation
README.md, ci/PROMPT.md
The documentation now describes eleven checkpoints, updated phase guidance, expanded fuzzing resources, version tooling, and blocking Semgrep WARNING findings.
CI tool reference updates
ci/fuzz/*, ci/linter/README.md, ci/tools/*
CI instructions now use ci/tools paths and nginx 1.31.3 examples. Linter documentation separately describes module-list differences and workflow wiring.

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description check ✅ Passed The description directly explains the documentation corrections, affected files, validation performed, and scope limits.
Title check ✅ Passed The title clearly identifies the documentation files and the factual-drift corrections that make up the main change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/factcheck-readme-prompt
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch docs/factcheck-readme-prompt

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ci/PROMPT.md (1)

19-28: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Keep the phase model consistent across both documents.

The phase table defines −1, 0, 1, 2, 3, and 4, but the summaries say five phases and omit close-out. Define which phases the count includes, then apply the same wording in both files.

  • ci/PROMPT.md#L19-L28: clarify the five-phase count or update it to include all six phase labels.
  • README.md#L296-L302: include Phase 4 close-out, or explicitly state that the summary excludes it.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e619c772-556c-4b6f-9843-d43720eb8d33

📥 Commits

Reviewing files that changed from the base of the PR and between 707fcbc and 13460fd.

📒 Files selected for processing (8)
  • README.md
  • ci/PROMPT.md
  • ci/fuzz/build.sh
  • ci/fuzz/regressions/README.md
  • ci/linter/README.md
  • ci/tools/ci-build.sh
  • ci/tools/rename-module.sh
  • ci/tools/soak.sh

Comment thread ci/fuzz/build.sh
#
# Needs clang with -fsanitize=fuzzer. The nginx source tree must already be at
# .build/nginx-<VER>/ (run tools/ci-build.sh first): the target links nginx's
# .build/nginx-<VER>/ (run ci/tools/ci-build.sh first): the target links nginx's

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Document the mode-specific build-tree path.

Line 11 still says .build/nginx-<VER>/, but this script resolves .build/nginx-<ver>-<mode> trees. Update the comment to include <MODE> or refer users to ci/tools/nginx-tree.sh.

@eilandert
eilandert force-pushed the docs/factcheck-readme-prompt branch from c13e9eb to 536bf76 Compare August 3, 2026 19:21
Proofread pass against the machinery each file describes.

README:
  - quickstart built nginx 1.31.2; the pinned mainline is 1.31.3 and
    1.31.2 has no sha256 in .github/versions.env, so a copy-paste build
    fetched a tarball nothing verifies
  - semgrep was listed as advisory in the CI table. security-scanners.yml
    runs a second pass with --severity=WARNING --error, which blocks
  - "eight checkpoints" x2: PROMPT.md has eleven (0-10), of which ten
    land a PR -- checkpoint 0 is read-only. The prose summary also
    omitted the decision seam and the depth pass entirely
  - "~350 lines of C": src/ is 1073
  - ci/fuzz/ described one target; fuzz_scan and fuzz_body both build
  - layout tree omitted the four ci/tools/ bump+versions scripts and all
    of .github/versions.env, scripts/ and actions/build-cache/

PROMPT.md:
  - checkpoint count off by one, and the PR count conflated with it
  - "five phases" against a table listing six (-1, 0, 1, 2, 3, 4).
    Phases -1 and 4 carry no checkpoint but are still work, so the count
    was wrong rather than the table
  - two dangling refs to a section 6b that does not exist; the semgrep
    --jobs=1 rationale they point at is 7b

ci/linter/README.md: split a run-on sentence that glued the LINT_ONLY
duplication warning to lint.yml's runner.

Tool headers: ci-build.sh, rename-module.sh, soak.sh, fuzz/build.sh and
fuzz/regressions/README.md all printed bare tools/ paths that have not
existed since the ci/ move, and the same stale 1.31.2.
@eilandert
eilandert force-pushed the docs/factcheck-readme-prompt branch from 536bf76 to b503854 Compare August 3, 2026 19:28
@eilandert

Copy link
Copy Markdown
Member Author

Confirmed, and it is the same drift class the PR is about — I propagated "five phases" into the README instead of counting the table.

The table is right and the count was wrong: −1, 0, 1, 2, 3, 4 is six. Phases −1 and 4 have an em dash in the Checkpoints column, which is what makes them easy to skip, but neither is empty — −1 is the scope jail and the stop-condition table, 4 is the close-out. Both are work, so they count.

Fixed in b503854: ci/PROMPT.md now says six and states explicitly that those two carry no checkpoint of their own, and the README summary says six and names the close-out phase it had omitted.

@eilandert
eilandert merged commit 2f2a371 into main Aug 3, 2026
14 checks passed
@eilandert
eilandert deleted the docs/factcheck-readme-prompt branch August 3, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant