test: lock README 'What is live in this repo' claims to code (promise audit, lane 2) - #60
Conversation
Promise-audit regression coverage: every claim in the README live section is now pinned to the constants, routes, quotas, and copy that must back it, so the section cannot silently drift from what ships. Covers the 1,000-page self-serve crawl cap, 50,000-URL sitemap inventory, 50K large-crawl targets with 1,000-page batches, 30-day report retention, five-competitor benchmark cap, 1-page 3/day Lite check quota, weekly monitor default, Worker route registration for the documented Cloudflare path, the D1 abuse-control buckets, and the public /demo /methodology /packages routing promise.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
nish3451 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
📝 WalkthroughWalkthroughThe pull request adds an integration test that compares README promises with Worker implementation details. It also runs this test from the aggregate ChangesPromise audit validation
Estimated code review effort: 3 (Moderate) | ~15–30 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@shared/promise-audit.test.mjs`:
- Around line 67-73: The test “README public page promise matches Worker routing
and copy” only verifies the packages price constant and not the documented copy
for each public page. Extend the assertions to inspect the rendered or Worker
response content for /demo, /methodology, and /packages, validating stable
documented text for each while preserving the existing routing checks.
- Around line 61-64: Strengthen the test around the schedule creation path to
verify that creating a monitor without an interval initializes its schedule to
seven days. Update the test using the relevant schedule creation or
initialization symbol, rather than relying only on the clampScheduleInterval
source assertion, while preserving the existing weekly monitor and helper
checks.
- Around line 75-120: Replace the string-presence assertions in the “README
Cloudflare path routes are actually registered in the Worker” test with
dispatch-level verification for every documented route, ensuring each request
reaches the Worker’s routing logic successfully. Alternatively, compare the
README route list against the route manifest consumed by dispatch, using the
same source of truth so undocumented or unregistered routes are detected.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 56698a18-454a-4e5d-9663-1e50ba669c57
📒 Files selected for processing (2)
package.jsonshared/promise-audit.test.mjs
| test("README weekly monitor promise matches the schedule interval", () => { | ||
| assert.match(liveSection, /weekly self-serve audit monitors/i); | ||
| assert.match(textSource, /clampScheduleInterval/, "schedule interval helper exists"); | ||
| assert.match(textSource, /if \(parsed <= 7\) return 7/, "default schedule interval is weekly"); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Assert the monitor default value.
Line 64 only proves that values at or below seven are clamped to seven. It does not prove that a monitor created without an interval defaults to seven days. A different default can still satisfy this assertion.
Exercise the schedule creation path with no interval, or assert the initialization default directly.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@shared/promise-audit.test.mjs` around lines 61 - 64, Strengthen the test
around the schedule creation path to verify that creating a monitor without an
interval initializes its schedule to seven days. Update the test using the
relevant schedule creation or initialization symbol, rather than relying only on
the clampScheduleInterval source assertion, while preserving the existing weekly
monitor and helper checks.
| test("README public page promise matches Worker routing and copy", () => { | ||
| assert.match(liveSection, /Public `\/demo`, `\/methodology`, and `\/packages` pages/i); | ||
| for (const path of ["/demo", "/methodology", "/packages"]) { | ||
| assert.ok(workerIndex.includes(`url.pathname === "${path}"`), `Worker must route ${path}`); | ||
| } | ||
| assert.match(pagesSource, /FIX_PACK_PUBLIC_PRICE/, "packages page price constant exists"); | ||
| }); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Assert the public page copy.
This test does not inspect copy returned for /demo or /methodology. It checks pagesSource only for FIX_PACK_PUBLIC_PRICE, so all documented page copy can change or be removed while this test passes.
Assert stable documented text for each page through its renderer or Worker response.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@shared/promise-audit.test.mjs` around lines 67 - 73, The test “README public
page promise matches Worker routing and copy” only verifies the packages price
constant and not the documented copy for each public page. Extend the assertions
to inspect the rendered or Worker response content for /demo, /methodology, and
/packages, validating stable documented text for each while preserving the
existing routing checks.
| test("README Cloudflare path routes are actually registered in the Worker", () => { | ||
| const claimedRoutes = [ | ||
| "/api/health", | ||
| "/api/deep-health", | ||
| "/api/waitlist", | ||
| "/api/access/request", | ||
| "/api/access/verify", | ||
| "/api/beta/login", | ||
| "/api/beta/session", | ||
| "/api/beta/logout", | ||
| "/api/account/summary", | ||
| "/api/audit", | ||
| "/api/audit/jobs/", | ||
| "/api/large-crawls", | ||
| "/api/audit/schedules", | ||
| "/api/developer", | ||
| "/api/developer/tokens", | ||
| "/api/developer/webhooks", | ||
| "/api/sites", | ||
| "/api/sites/claim", | ||
| "/api/sites/verify", | ||
| "/api/reports/", | ||
| "/api/webhooks/dodo", | ||
| "/api/billing/summary", | ||
| "/api/beta/fix-request", | ||
| "/api/beta/monitoring-checkout", | ||
| "/v1/audits", | ||
| "/v1/projects", | ||
| "/v1/large-crawls", | ||
| "/admin/summary", | ||
| "/admin/invites", | ||
| "/admin/leads.csv", | ||
| "/llms.txt", | ||
| "/sitemap.xml", | ||
| "/robots.txt", | ||
| "/privacy", | ||
| "/support", | ||
| "/terms", | ||
| "/demo", | ||
| "/methodology", | ||
| "/packages", | ||
| "/beta" | ||
| ]; | ||
| for (const route of claimedRoutes) { | ||
| assert.ok(workerIndex.includes(route), `Worker must register route ${route}`); | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Verify dispatch behavior instead of string presence.
workerIndex.includes(route) passes if the path remains in a link, constant, comment, or unreachable branch. It does not prove that the Worker dispatches the route. The hard-coded list also does not verify that it matches the README claims.
Test each public route through Worker dispatch, or compare a route manifest used by dispatch against the documented route list.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@shared/promise-audit.test.mjs` around lines 75 - 120, Replace the
string-presence assertions in the “README Cloudflare path routes are actually
registered in the Worker” test with dispatch-level verification for every
documented route, ensuring each request reaches the Worker’s routing logic
successfully. Alternatively, compare the README route list against the route
manifest consumed by dispatch, using the same source of truth so undocumented or
unregistered routes are detected.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6e1944471f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "/beta" | ||
| ]; | ||
| for (const route of claimedRoutes) { | ||
| assert.ok(workerIndex.includes(route), `Worker must register route ${route}`); |
There was a problem hiding this comment.
When a base route is removed while a descendant literal remains, this assertion still passes: for example, deleting the /beta page handler leaves /api/beta/login, and deleting /api/audit leaves /api/audit/schedules. Therefore the new CI gate does not actually detect several route-registration regressions it claims to lock; inspect the dispatch predicates or exercise the Worker instead of using unrestricted includes.
AGENTS.md reference: AGENTS.md:L7-L7
Useful? React with 👍 / 👎.
| test("README Cloudflare path routes are actually registered in the Worker", () => { | ||
| const claimedRoutes = [ |
There was a problem hiding this comment.
The route set omits documented Cloudflare endpoints such as /admin/beta-session, /api/branding, /api/report-domains, /api/team, and /api/team/members from README.md lines 87 and 103-109. If any of those registrations disappears while the README promise remains, test:promise-audit still passes, so this does not enforce the stated every-documented-route contract.
AGENTS.md reference: AGENTS.md:L7-L7
Useful? React with 👍 / 👎.
…code (#65) The lane-1 promise audit (PR #60-#64) pinned all 48 'What is live in this repo' bullets, but a fresh spot-check found eleven sub-claims inside pinned bullets with no regression pin of their own: - crawl inventory: 'from robots.txt and sitemaps' discovery was unpinned - large crawls: 'stored frontier/proof/retry state' tables/logic unpinned - competitor benchmarking: repair gaps added to reports and briefs unpinned - AI readiness: optional /llms.txt reachability checks unpinned - repair queue: acceptance checks, status, and action mode unpinned - repair board: 'no external publishing side effects' copy unpinned - implementation packs: approved change text requirement unpinned - repair proposals: execution modes, owner approval, delivery state unpinned - audit monitors: dashboard add/pause controls unpinned - developer API: safe repair_queue issue status unpinned - retention: cleanup for expired reports, sessions, and quota buckets unpinned Adds eleven pins to shared/promise-audit.test.mjs (55 tests total) so these claims cannot silently drift. Live spot-check of /demo, /methodology, and /packages against https://seofixkit.com still passes.
1 participant
What
Promise-audit lane 2: every claim in the README "What is live in this repo" section and the live
/demo,/methodology,/packagespages was spot-checked against the code, migrations, and the production deployment at seofixkit.com. All claims verified truthful — no copy or behavior needed changing.What changed
Added
shared/promise-audit.test.mjs(wired intonpm run checkastest:promise-audit) so the audited claims cannot silently drift from what ships:SELF_SERVE_MAX_CRAWL_PAGES)CRAWLRAVEN_PUBLIC_CRAWL_PAGES)LARGE_RENDERED_CRAWL_TARGET_PAGES/LARGE_RENDERED_CRAWL_BATCH_SIZE)REPORT_RETENTION_DAYS)parseAuditCompetitorUrls)audit:lite-day,limit: 3)clampScheduleInterval)/api/*,/v1/*,/admin/*,/beta,/demo,/methodology,/packages,/llms.txt,/sitemap.xml, etc.)/demo,/methodology,/packagesAudit trail (spot-checks performed)
curlagainst production:/,/demo,/methodology,/packages,/support,/terms,/privacy,/llms.txt,/sitemap.xml,/robots.txtall 200;/betaserves the SPA withnoindex, nofollow+no-store./api/healthreportsbrowserRun: true,waitlistDb: true,emailNotifications: true, version0.9.0./api/deep-healthreports bindings, D1 schema (22 checks), Dodo config (checkout ready, monitoring config-gated as documented), and truthful capability flags (paidProofMonitoring: false)./api/waitlistand/api/access/requestboth behave as documented (verified with a disposable probe email)./api/public-pricingreturns the$99.00price the packages page claims./demo,/methodology,/packagescopy matchesworker/routes/pages.jsand the README claims, including the "not a public anonymous audit", "no ranking guarantee", "early-access staged 50K crawls", and "no live AI-engine sampling" boundaries.Validation
npm run check(the single CI gate) passes: all test suites + production build.Summary by CodeRabbit