fix(public): declare preferred source pages for AI answers (dogfood 4473a99a9bc9) - #42
Merged
Merged
Conversation
…473a99a9bc9) The audit run 20260808T074205Z-msk2fl3n found the engines' preferred source pages unclear: q5/google cited tinystudio.io yet described the retired Agent Desk, and q7/google came back "Missing: pricing". Nothing on the site told an engine which page owns which fact. The machine-readable pair now declares, per controlled question, the preferred source page an engine should read first: llms.txt and offer.md carry an Answer Readiness section mapping every controlled question to exactly one served page, with price questions pointing at pricing.html, which owns the price. check-site.mjs and test-agent-ui.mjs fail loudly when a question is unmapped, mapped to two pages, mapped to an unserved page, mapped away from pricing.html for a price question, or mirrored inconsistently between the two files. The AI-search evidence fixture stays byte-identical; historical runs, states and captures are untouched, and no live engine run is claimed. Co-Authored-By: Claude <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
nish3451 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
📝 WalkthroughWalkthroughThe change adds preferred source-page mappings for seven controlled AI-search questions. It mirrors the mappings in ChangesAI answer-readiness mapping
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant ControlledQuestions
participant LlmsTxt
participant OfferMd
participant CheckSite
participant TestAgentUI
ControlledQuestions->>LlmsTxt: define preferred source mappings
LlmsTxt->>OfferMd: mirror mappings
CheckSite->>LlmsTxt: validate mapping section and URLs
CheckSite->>OfferMd: validate mirrored mappings
TestAgentUI->>LlmsTxt: validate controlled question coverage
TestAgentUI->>OfferMd: verify mapping parity
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c8ebacefb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+852
to
+854
| const offerLine = offerSection.split("\n").find((line) => line.includes(question.id)); | ||
| if (!offerLine || !offerLine.includes(preferred)) { | ||
| failures.push(`offer.md must mirror the preferred source page for ${question.id}: ${preferred}`); |
There was a problem hiding this comment.
When the offer.md mapping for q1 or q5 drifts from the homepage to any other TinyStudio page, this check still passes because every same-domain URL contains the preferred root string https://tinystudio.io/. The unit test repeats the same substring check, so CI would accept mappings such as https://tinystudio.io/audit.html while llms.txt still specifies the homepage, defeating the new mirror invariant; parse the offer-side URL and compare it for exact equality.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/evidence/ai-answer-readiness-2026-08-09.md`:
- Around line 81-85: Specify the shell language on the fenced command block by
changing its opening fence to use sh, while preserving the existing commands and
closing fence.
In `@scripts/check-site.mjs`:
- Around line 832-855: Update the validation loop around aiQuestions.questions
to collect and validate all llmsSection lines matching each question.id,
rejecting anything other than exactly one mapping. Also parse URLs from the
matching offer.md line and require exactly one URL equal to the preferred URL,
rejecting extra or conflicting mappings while preserving the served-page and
pricing checks.
In `@scripts/test-agent-ui.mjs`:
- Around line 612-624: Update the mapping checks around the llmsLine and
offerLine assertions to collect all lines containing each question.id and
require exactly one matching line in both files. Extract URLs from the offer.md
mapping line and assert it contains exactly one URL, while preserving the
existing preferred-URL equality and served-page validations.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: db0c7765-d5cb-4441-a201-0a7c5843c5ca
📒 Files selected for processing (6)
docs/evidence/ai-answer-readiness-2026-08-09.mdevidence-fixtures/ai-search/README.mdpublic/llms.txtpublic/offer.mdscripts/check-site.mjsscripts/test-agent-ui.mjs
Comment on lines
+81
to
+85
| ``` | ||
| npm run check | ||
| npm test | ||
| git diff --check | ||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Specify the fenced code language.
Add sh to the opening fence. This resolves markdownlint MD040.
Proposed fix
-```
+```sh
npm run check
npm test
git diff --check</details>
<!-- suggestion_start -->
<details>
<summary>📝 Committable suggestion</summary>
> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)
[warning] 81-81: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/evidence/ai-answer-readiness-2026-08-09.md` around lines 81 - 85,
Specify the shell language on the fenced command block by changing its opening
fence to use sh, while preserving the existing commands and closing fence.
Source: Linters/SAST tools
Comment on lines
+832
to
+855
| for (const question of aiQuestions.questions) { | ||
| const llmsLine = llmsSection.split("\n").find((line) => line.includes(question.id)); | ||
| if (!llmsLine) { | ||
| failures.push(`llms.txt must map the controlled question to a preferred source page: ${question.id}`); | ||
| continue; | ||
| } | ||
| const urls = [...llmsLine.matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]); | ||
| if (urls.length !== 1) { | ||
| failures.push(`Preferred source mapping must name exactly one page: ${question.id}`); | ||
| continue; | ||
| } | ||
| const preferred = urls[0]; | ||
| if (!servedPageUrls.has(preferred)) { | ||
| failures.push(`Preferred source page must be a served page: ${question.id} ${preferred}`); | ||
| } | ||
| const isPriceQuestion = | ||
| question.id === "q2-what-tinystudio-charges" || question.id === "q7-what-tinystudio-io-charges"; | ||
| if (isPriceQuestion && preferred !== "https://tinystudio.io/pricing.html") { | ||
| failures.push(`Price question ${question.id} must map to pricing.html (pricing.html owns the price).`); | ||
| } | ||
| const offerLine = offerSection.split("\n").find((line) => line.includes(question.id)); | ||
| if (!offerLine || !offerLine.includes(preferred)) { | ||
| failures.push(`offer.md must mirror the preferred source page for ${question.id}: ${preferred}`); | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reject duplicate mappings and extra offer.md URLs.
Line 833 selects only the first matching llms.txt line. A second mapping for the same question is ignored. Line 853 only checks that offer.md contains the selected URL, so it accepts an additional conflicting URL. This does not enforce the stated one-question-to-one-page contract.
Proposed fix
- const llmsLine = llmsSection.split("\n").find((line) => line.includes(question.id));
- if (!llmsLine) {
+ const mappingLines = (section) =>
+ section.split("\n").filter((line) => line.startsWith(`- ${question.id} `));
+ const llmsLines = mappingLines(llmsSection);
+ if (llmsLines.length !== 1) {
failures.push(`llms.txt must map the controlled question to a preferred source page: ${question.id}`);
continue;
}
+ const llmsLine = llmsLines[0];
const urls = [...llmsLine.matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]);
if (urls.length !== 1) {
failures.push(`Preferred source mapping must name exactly one page: ${question.id}`);
continue;
}
@@
- const offerLine = offerSection.split("\n").find((line) => line.includes(question.id));
- if (!offerLine || !offerLine.includes(preferred)) {
+ const offerLines = mappingLines(offerSection);
+ const offerUrls = offerLines.length === 1
+ ? [...offerLines[0].matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0])
+ : [];
+ if (offerUrls.length !== 1 || offerUrls[0] !== preferred) {
failures.push(`offer.md must mirror the preferred source page for ${question.id}: ${preferred}`);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| for (const question of aiQuestions.questions) { | |
| const llmsLine = llmsSection.split("\n").find((line) => line.includes(question.id)); | |
| if (!llmsLine) { | |
| failures.push(`llms.txt must map the controlled question to a preferred source page: ${question.id}`); | |
| continue; | |
| } | |
| const urls = [...llmsLine.matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]); | |
| if (urls.length !== 1) { | |
| failures.push(`Preferred source mapping must name exactly one page: ${question.id}`); | |
| continue; | |
| } | |
| const preferred = urls[0]; | |
| if (!servedPageUrls.has(preferred)) { | |
| failures.push(`Preferred source page must be a served page: ${question.id} ${preferred}`); | |
| } | |
| const isPriceQuestion = | |
| question.id === "q2-what-tinystudio-charges" || question.id === "q7-what-tinystudio-io-charges"; | |
| if (isPriceQuestion && preferred !== "https://tinystudio.io/pricing.html") { | |
| failures.push(`Price question ${question.id} must map to pricing.html (pricing.html owns the price).`); | |
| } | |
| const offerLine = offerSection.split("\n").find((line) => line.includes(question.id)); | |
| if (!offerLine || !offerLine.includes(preferred)) { | |
| failures.push(`offer.md must mirror the preferred source page for ${question.id}: ${preferred}`); | |
| } | |
| for (const question of aiQuestions.questions) { | |
| const mappingLines = (section) => | |
| section.split("\n").filter((line) => line.startsWith(`- ${question.id} `)); | |
| const llmsLines = mappingLines(llmsSection); | |
| if (llmsLines.length !== 1) { | |
| failures.push(`llms.txt must map the controlled question to a preferred source page: ${question.id}`); | |
| continue; | |
| } | |
| const llmsLine = llmsLines[0]; | |
| const urls = [...llmsLine.matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]); | |
| if (urls.length !== 1) { | |
| failures.push(`Preferred source mapping must name exactly one page: ${question.id}`); | |
| continue; | |
| } | |
| const preferred = urls[0]; | |
| if (!servedPageUrls.has(preferred)) { | |
| failures.push(`Preferred source page must be a served page: ${question.id} ${preferred}`); | |
| } | |
| const isPriceQuestion = | |
| question.id === "q2-what-tinystudio-charges" || question.id === "q7-what-tinystudio-io-charges"; | |
| if (isPriceQuestion && preferred !== "https://tinystudio.io/pricing.html") { | |
| failures.push(`Price question ${question.id} must map to pricing.html (pricing.html owns the price).`); | |
| } | |
| const offerLines = mappingLines(offerSection); | |
| const offerUrls = offerLines.length === 1 | |
| ? [...offerLines[0].matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]) | |
| : []; | |
| if (offerUrls.length !== 1 || offerUrls[0] !== preferred) { | |
| failures.push(`offer.md must mirror the preferred source page for ${question.id}: ${preferred}`); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/check-site.mjs` around lines 832 - 855, Update the validation loop
around aiQuestions.questions to collect and validate all llmsSection lines
matching each question.id, rejecting anything other than exactly one mapping.
Also parse URLs from the matching offer.md line and require exactly one URL
equal to the preferred URL, rejecting extra or conflicting mappings while
preserving the served-page and pricing checks.
Comment on lines
+612
to
+624
| const llmsLine = llmsSection.split("\n").find((line) => line.includes(question.id)); | ||
| assert.ok(llmsLine, `llms.txt must map the controlled question to a preferred source page: ${question.id}`); | ||
| const urls = [...(llmsLine ?? "").matchAll(/https:\/\/tinystudio\.io\/[^\s]*/g)].map((match) => match[0]); | ||
| assert.equal(urls.length, 1, `preferred source mapping must name exactly one page: ${question.id}`); | ||
| const preferred = urls[0]; | ||
| assert.ok(servedPages.includes(preferred), `preferred source page must be a served page: ${question.id}`); | ||
| const isPriceQuestion = | ||
| question.id === "q2-what-tinystudio-charges" || question.id === "q7-what-tinystudio-io-charges"; | ||
| if (isPriceQuestion) { | ||
| assert.equal(preferred, "https://tinystudio.io/pricing.html", `price question ${question.id} must map to pricing.html`); | ||
| } | ||
| const offerLine = offerSection.split("\n").find((line) => line.includes(question.id)); | ||
| assert.ok(offerLine?.includes(preferred), `offer.md must mirror the preferred source page for ${question.id}`); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Test the complete mirror contract.
Line 612 ignores duplicate mapping lines. Line 624 accepts an offer.md line that contains the preferred URL plus another URL. Add assertions that each file has exactly one mapping line and exactly one URL for each question.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@scripts/test-agent-ui.mjs` around lines 612 - 624, Update the mapping checks
around the llmsLine and offerLine assertions to collect all lines containing
each question.id and require exactly one matching line in both files. Extract
URLs from the offer.md mapping line and assert it contains exactly one URL,
while preserving the existing preferred-URL equality and served-page
validations.
nish3451
added a commit
that referenced
this pull request
Aug 12, 2026
…gainst current GitHub state (#146)
nish3451
added a commit
that referenced
this pull request
Aug 14, 2026
…ainst current main and live (2026-08-14) (#180) The preferred-source declaration (PR #42, extended by PR #102's q8) is unchanged since the 2026-08-12 receipt: npm run check passes with the "AI Answer Readiness (dogfood 4473a99a9bc9)" guard enforcing section presence, exactly-one-page coverage for all eight controlled questions, served-page membership, pricing.html ownership for q2/q7, and the offer.md mirror. The full test suite passes (117 tests, 0 failures). Fresh live fetches on 2026-08-14 show llms.txt and offer.md serve the identical eight-question mapping byte-for-byte, every mapped page is in the live sitemap, the homepage identity block still answers all eight questions with no retired Agent Desk framing, and the audit-page embed still matches the fixture. The finding stays closed with no code change. Co-authored-by: nish3451 <nish3451@users.noreply.github.com> Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
nish3451
pushed a commit
that referenced
this pull request
Aug 14, 2026
…ation closeout (2026-08-14)
nish3451
added a commit
that referenced
this pull request
Aug 14, 2026
…nt main (2026-08-14) (#189) * docs(evidence): re-verify PR #42 + #43 serial-merge closeout on current main (2026-08-14) Re-verifies the review item [unreviewed-by-opus] Merge PR #42 and #43 in serial order (or rebase the loser) against the current GitHub state on 2026-08-14. The 2026-08-12 closeout (PR #146, 4e2b94c) still holds: - PR #42 squash 95d2248 is in origin/main (f9214c1) - PR #43 merge ad9cee3 is in origin/main - git merge-base --is-ancestor 95d2248 ad9cee3 is true (serial order) - PR #42 declaration is on main; q8-conversion-audit (PR #102) layered on top - PR #43 branch tip ed62202 is an ancestor of main with zero unmerged content - No open PR #42 or #43 No code change is needed or proposed. * docs(lane-report): record lane 1 PR #42 + #43 serial-merge re-verification closeout (2026-08-14) --------- Co-authored-by: Nish <nish@tinystudio.io>
nish3451
added a commit
that referenced
this pull request
Aug 15, 2026
…ainst current main and live (2026-08-15) The preferred-source declaration (PR #42, extended by PR #102's q8) still holds on the current head: npm run check passes with the "AI Answer Readiness (dogfood 4473a99a9bc9)" guard enforcing section presence, exactly-one-page coverage for all eight controlled questions, served-page membership, pricing-page ownership for q2/q7, and the offer.md mirror, and the full test suite passes (121 tests, 0 failures). Two commits since the 2026-08-14 receipt touched llms.txt/offer.md — #193 extended the conversion-audit bridge statement, and #202 re-pointed the mappings from the 307-redirecting .html addresses to the clean extensionless addresses that serve 200 (/pricing, /audit) — neither regressed the declaration. Fresh live fetches on 2026-08-15 show llms.txt and offer.md serve the identical eight-question mapping byte-for-byte, every mapped page serves 200 and is in the live sitemap, the homepage identity block still answers all eight questions with no retired Agent Desk framing, and the audit-page embed still matches the fixture. The finding stays closed with no code change. Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
nish3451
added a commit
that referenced
this pull request
Aug 15, 2026
…ainst current main and live (2026-08-15) (#226) The preferred-source declaration (PR #42, extended by PR #102's q8) still holds on the current head: npm run check passes with the "AI Answer Readiness (dogfood 4473a99a9bc9)" guard enforcing section presence, exactly-one-page coverage for all eight controlled questions, served-page membership, pricing-page ownership for q2/q7, and the offer.md mirror, and the full test suite passes (121 tests, 0 failures). Two commits since the 2026-08-14 receipt touched llms.txt/offer.md — #193 extended the conversion-audit bridge statement, and #202 re-pointed the mappings from the 307-redirecting .html addresses to the clean extensionless addresses that serve 200 (/pricing, /audit) — neither regressed the declaration. Fresh live fetches on 2026-08-15 show llms.txt and offer.md serve the identical eight-question mapping byte-for-byte, every mapped page serves 200 and is in the live sitemap, the homepage identity block still answers all eight questions with no retired Agent Desk framing, and the audit-page embed still matches the fixture. The finding stays closed with no code change. Co-authored-by: nish3451 <nish3451@users.noreply.github.com> Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
nish3451
added a commit
that referenced
this pull request
Aug 17, 2026
…nt main (2026-08-17, lane 1) (#244) Co-authored-by: minimax-vps <minimax-vps@fleet.local>
nish3451
added a commit
that referenced
this pull request
Aug 21, 2026
…nt main (2026-08-21, lane 1) (#269) Co-authored-by: minimax-vps <minimax-vps@local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes dogfood finding 4473a99a9bc9 ("AI Answer Readiness: preferred source pages are unclear", audit run 20260808T074205Z-msk2fl3n) — the last open item from that audit run.
The controlled AI-search evidence shows the cost of the gap:
pricing.htmlstates the price. Nothing pointed the engine atpricing.html.What changed
public/llms.txtandpublic/offer.md(the machine-readable mirror pair) gain an## Answer Readiness: Preferred Source Pagessection: one bullet per controlled question, each naming exactly one preferred source page — the page that owns the fact. Price questions (q2, q7) map topricing.html, which owns the price; identity questions map to the homepage; where-based / who-with / client-work map to the audit page.scripts/check-site.mjsfails when: either file loses the section; a controlled question is unmapped, mapped to two pages, or mapped to a page the worker does not serve (checked against the sitemap surface, accepting the canonical.htmltwins); a price question maps away frompricing.html; or the mirror drifts between the two files.scripts/test-agent-ui.mjsasserts the same invariants as unit tests (suite now 16/16).evidence-fixtures/ai-search/README.mddocuments the tie.docs/evidence/ai-answer-readiness-2026-08-09.mdrecords the repository-side pass and its boundaries.The AI-search evidence fixture stays byte-identical; historical runs, states and captures are untouched, and no live engine run is claimed.
Verification
npm run checkpasses (verified the guard bites by temporarily mis-mapping q7 → guard fails; restored → passes).npm testpasses: headings 6/6, sitemap 7/7, worker 53/53, agent-UI 16/16.git diff --checkclean.Summary by CodeRabbit
Documentation
Quality Improvements