Repository navigation
fix(docs-site): write search-index.json one entry per line - #1794
Conversation
✅ Single Commit Policy - COMPLIANTStatus: Policy requirements met • 1 commit • Valid format • Ready for merge 📊 View validation details📝 Commit Details
✅ Validation Results
🤖 Automated validation by NeuroLink Single Commit Enforcement |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Important Review skippedWe couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting You can disable this status message by setting the Use the checkbox below for a quick retry:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: juspay/neurolink/.coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review. 📝 WalkthroughWalkthroughThe search index is now written as a multiline JSON array, with one document per line. The reproducibility script tests whether indexes generated from separate page edits merge to match an index generated with both edits. ChangesSearch Index Serialization and Reproducibility
Priority: ⬇️ Low Estimated code review effort: 2 (Simple) | ~10 minutes Change: Refactor Merge Risk: ⚪ Minimal · up to The index remains compatible with search readers, and the merge test exercises the intended independent page edits. No merge-blocking issue remains. Architecture SummaryArchitecture risk: 🔵 Low · up to The change affects 1 system. Changed systems: Architecture concerns Review detailsSystems and components
Before / after behavior
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Documentation Validation Results🚀 Documentation validation passed!
📦 Build artifact uploaded successfully. Ready for deployment preview. Commit: |
Tara-ag
left a comment
Tara-ag
left a comment
There was a problem hiding this comment.
One-entry-per-line search-index layout is correct and well-covered. One MINOR nit inline; not blocking.
| const merge = spawnSync( | ||
| "git", | ||
| ["merge-file", "-p", "first", "base", "second"], | ||
| { cwd: mergeDir, encoding: "utf8" }, | ||
| ); |
There was a problem hiding this comment.
MINOR — spawnSync("git", …) is not guarded for a missing/unusable git binary.
If git isn't on PATH (or the tmp dir isn't reachable as a work tree), spawnSync returns { status: null, error } and the very next assert.equal(merge.status, 0, …) fails on an unrelated, confusing message. Failing on the missing-tool condition directly makes the failure actionable. This test's premise is that git's three-way merge proves the merge-friendly layout, so surfacing the tool precondition is worth the 4 lines:
| const merge = spawnSync( | |
| "git", | |
| ["merge-file", "-p", "first", "base", "second"], | |
| { cwd: mergeDir, encoding: "utf8" }, | |
| ); | |
| const merge = spawnSync( | |
| "git", | |
| ["merge-file", "-p", "first", "base", "second"], | |
| { cwd: mergeDir, encoding: "utf8" }, | |
| ); | |
| assert.ok( | |
| merge.status !== null && merge.error === undefined, | |
| `git merge-file unavailable: ${merge.error?.message ?? "not found on PATH"}`, | |
| ); |
There was a problem hiding this comment.
CORRECTION to the suggestion above. The earlier suggestion block was misanchored and, worse, it dropped the assert.equal(merge.status, 0, …) merge-conflict assertion that is the whole point of the test. Please use this corrected version instead.
It anchors to the spawnSync("git", …) call and inserts the guard after it, keeping the existing merge-status check intact:
| const merge = spawnSync( | |
| "git", | |
| ["merge-file", "-p", "first", "base", "second"], | |
| { cwd: mergeDir, encoding: "utf8" }, | |
| ); | |
| const merge = spawnSync( | |
| "git", | |
| ["merge-file", "-p", "first", "base", "second"], | |
| { cwd: mergeDir, encoding: "utf8" }, | |
| ); | |
| assert.ok( | |
| merge.status !== null && merge.error === undefined, | |
| `git merge-file unavailable: ${merge.error?.message ?? "not found on PATH"}`, | |
| ); |
With this, the original assert.equal(merge.status, 0, "edits to two different pages must merge the index without a conflict") on the next line still runs, and the missing-git case fails with an actionable message instead of a confusing merge.status mismatch.
There was a problem hiding this comment.
Keeping this finding (recurring review, no author reply yet). The corrected suggestion above still stands as a MINOR, non-blocking improvement: when git is absent on PATH (or the tmp dir isn't usable as a work tree), spawnSync returns { status: null, error } and the following assert.equal(merge.status, 0, …) fails on an unrelated, confusing message. Adding the guard makes the tool-precondition failure actionable. No action required to merge — this is flagged for follow-up.
There was a problem hiding this comment.
Fixed in #1894: the search-index reproducibility test now asserts that git merge-file could run (no spawn error) before it checks the exit status.
|
Superseded — this summary was superseded by the recurring review. See the canonical review summary below (same verdict): comment 5841812208. Verdict: APPROVE. Keeping only one summary comment per the PR-review protocol (no duplicated findings or summaries). |
search-index.json is committed and every docs change regenerates it, but it was written as a single 8.5 MB line. Git merges line by line, so any two open PRs that touched docs conflicted on the whole file, however unrelated their pages were. On 2026-09-25, 14 of the 24 open PRs showed a conflict, and in most of them this file (with docs/api pages) was the only conflict. With the stable per-URL ids and sorted traversal from #1763, an unchanged page now produces an unchanged line, so writing one entry per line lets git merge two PRs that edit different pages. The data is unchanged: the new file parses to the same 11,042 entries as the old one. Checked with a real three-way merge of two regenerated indexes, one editing the DeepSeek page and one the HuggingFace page: one line: git merge-file reports 1 conflict entry per line: no conflict, and the merged file is byte-identical to a fresh build carrying both edits test-search-index-reproducibility.cjs now asserts exactly that, and fails on the one-line writer.
3249238 to
0eabcc5
Compare
|
Recurring review — overall: APPROVE ✅ I re-verified the change ( On the one existing finding ( Recommendation: the branch is ready to merge. The MINOR nit can be addressed as a follow-up. /approve |
Tara-ag
left a comment
Tara-ag
left a comment
There was a problem hiding this comment.
Approving this branch. The change is confined to the doc-search index writer (docs-site/tools/search-index/index.js): the JSON output changes from one pretty-printed object to one object per line, which is byte-stable across runs and merge-friendly (Git merge applies cleanly per line). The new test-search-index-reproducibility.cjs test validates this property and fails on the old writer. Output content is unchanged, so the Rust server and getFileText consumers are unaffected. One non-blocking minor (git presence guard in the test) is tracked in the existing thread.
|
🎉 This PR is included in version 12.24.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
…uides and plans Fixes the docs-accuracy review threads left open on merged PRs. Each claim was re-checked against the code on this checkout before editing. CLAUDE.md - CI-skip section: GitHub skips the push and pull_request runs when the head commit holds a directive, so the required check stays Pending and blocks the merge. `Reject CI-Skip Directives` is only a backstop and its regex does not cover a skip-checks trailer. (T3814059894-1, #1365) - Rule 15 allow list: the closed Grandfathered block is legacy debt without a per-file header and may shrink, never grow; same note beside the list in eslint.config.js. (T3818474525-allow-docs, #1378) - Audit snippet: the && chain moves into an `if`, so a failing audit cannot end a `set -e` caller's shell before the worktree cleanup. Proven with a bash `set -e` control. (T4051898811-1, #1676) - "Reading a CI result": incidents 1, 2 and 4 are the absence-of-signal mistake, 3 is its inverse. (T4042254379-intro-first-four, #1716) Provider and reference docs - openai.md and providers/index.md: gpt-5.4 context is 1.05M (mini and nano stay 400K), matching contextWindows.ts. (T4114160945 and T4114105048, #1824; one defect raised twice) - deepseek.md: close the unbalanced backtick that leaked into the search index. (T4112589028-b, #1800) - pareto-inference.md: no context window is published; 131,072 is a catalog fallback, not a floor or a vendor figure. (T4125607242, #1848) - docs/index.md: count MCP servers consistently. (T4072651139, #1776) - provider-selection.md: the Streaming row covers text-generation providers only; decision-only providers (four, not three) use decide(). (T4115057665, #1820) - README.md: drop the hand-kept tool-support counts and stop grouping LiteLLM with the zero-configuration local runtimes, since it needs a running proxy. (T4113418122-readme-count-stale-now, #1816; T4072651184, #1776) - openai-compat-catalog.md: every catalog provider except Groq maps TimeoutError to NetworkError. (T3806464799, #1353) - SAFETY-PRIMITIVES.md: only no-inline-secret-regex and provider-typed-errors still apply; SSRF, stream-span and isNeuroLink bypasses are review-only. (T3790049900-1, #1334) Plans - middleware plan: providers-mocked has no AI Studio section and is construction-only for Vertex and Bedrock; name the three real seams. (T3950529360#1, #1656) - dead-code-purge plan: record that the removal shipped in the major v11.0.0 and that there is no replacement for the removed types. (PF-T3790294047, #1335) - onboarding-playbook plan: repo-relative commands instead of machine-local paths, drop the uncommitted scratch spec links, "Every Tier 3+ provider" ends with a manifest (Tier 2 is declared in its catalog JSON), and the three misplaced closing fences are moved so the duplicate "Verification commands" H2s are gone. (T3790294048, T3790294049, T3790294054, #1335) Tooling - verify-provider-onboarding now requires addedInPR, filesTouched and manualTestStatus in a hand-written provider's manifest, as the manifests README already said. xor and perplexity-decider gain manualTestStatus "ci-mocked-only"; README lists "verified-live". New case in the provider-structure suite runs the real tool against a scratch manifests tree: red without the validator change, green with it. (T3790294060-a, #1335) - test-search-index-reproducibility asserts git merge-file could run, so a missing git reports ENOENT instead of a merge conflict. (T4108958700-git- guard, #1794) Regenerated: docs-site/static/search-index.json via the docs build; a second build leaves it byte-identical. Fixes from the review of this PR, found after it was opened: - openai.md: GPT-6 (September 2026) is newer than GPT-5.4 (March 2026), so the guide no longer calls GPT-5.4 the newest or the latest. - onboarding-playbook plan: the Tier 2 bullet described a hand-written catalog row and a descriptor row; a Tier 2 provider is one JSON file under src/lib/providers/catalog/, and the onboarding gate checks that file instead of a manifest. Skipped or deferred: - T3810290322+T3810299660 (a link from tiers/README.md back to its parent): not done. The first attempt added a bare README key to LINK_MAPPINGS in sync-docs.ts, which would have sent about 7,500 API-reference links to the provider-integration README instead of the API index. It was reverted; a fix needs a link rule scoped to provider-integration/tiers. - PF-T3790294047 is only partly fixed: the outcome note is in the plan, but docs/MIGRATION.md still has no v11.0.0 entry. perplexity-decider is marked ci-mocked-only, the conservative value; its owner may upgrade it if the live probe counts. The catalog description of pareto-inference still says "conservative floor"; that is catalog data, left alone to avoid a codegen change in a docs commit.
…uides and plans Fixes the docs-accuracy review threads left open on merged PRs. Each claim was re-checked against the code on this checkout before editing. CLAUDE.md - CI-skip section: GitHub skips the push and pull_request runs when the head commit holds a directive, so the required check stays Pending and blocks the merge. `Reject CI-Skip Directives` is only a backstop and its regex does not cover a skip-checks trailer. (T3814059894-1, #1365) - Rule 15 allow list: the closed Grandfathered block is legacy debt without a per-file header and may shrink, never grow; same note beside the list in eslint.config.js. (T3818474525-allow-docs, #1378) - Audit snippet: the && chain moves into an `if`, so a failing audit cannot end a `set -e` caller's shell before the worktree cleanup. Proven with a bash `set -e` control. (T4051898811-1, #1676) - "Reading a CI result": incidents 1, 2 and 4 are the absence-of-signal mistake, 3 is its inverse. (T4042254379-intro-first-four, #1716) Provider and reference docs - openai.md and providers/index.md: gpt-5.4 context is 1.05M (mini and nano stay 400K), matching contextWindows.ts. (T4114160945 and T4114105048, #1824; one defect raised twice) - deepseek.md: close the unbalanced backtick that leaked into the search index. (T4112589028-b, #1800) - pareto-inference.md: no context window is published; 131,072 is a catalog fallback, not a floor or a vendor figure. (T4125607242, #1848) - docs/index.md: count MCP servers consistently. (T4072651139, #1776) - provider-selection.md: the Streaming row covers text-generation providers only; decision-only providers (four, not three) use decide(). (T4115057665, #1820) - README.md: drop the hand-kept tool-support counts and stop grouping LiteLLM with the zero-configuration local runtimes, since it needs a running proxy. (T4113418122-readme-count-stale-now, #1816; T4072651184, #1776) - openai-compat-catalog.md: every catalog provider except Groq maps TimeoutError to NetworkError. (T3806464799, #1353) - SAFETY-PRIMITIVES.md: only no-inline-secret-regex and provider-typed-errors still apply; SSRF, stream-span and isNeuroLink bypasses are review-only. (T3790049900-1, #1334) Plans - middleware plan: providers-mocked has no AI Studio section and is construction-only for Vertex and Bedrock; name the three real seams. (T3950529360#1, #1656) - dead-code-purge plan: record that the removal shipped in the major v11.0.0 and that there is no replacement for the removed types. (PF-T3790294047, #1335) - onboarding-playbook plan: repo-relative commands instead of machine-local paths, drop the uncommitted scratch spec links, "Every Tier 3+ provider" ends with a manifest (Tier 2 is declared in its catalog JSON), and the three misplaced closing fences are moved so the duplicate "Verification commands" H2s are gone. (T3790294048, T3790294049, T3790294054, #1335) Tooling - verify-provider-onboarding now requires addedInPR, filesTouched and manualTestStatus in a hand-written provider's manifest, as the manifests README already said. xor and perplexity-decider gain manualTestStatus "ci-mocked-only"; README lists "verified-live". New case in the provider-structure suite runs the real tool against a scratch manifests tree: red without the validator change, green with it. (T3790294060-a, #1335) - test-search-index-reproducibility asserts git merge-file could run, so a missing git reports ENOENT instead of a merge conflict. (T4108958700-git- guard, #1794) Regenerated: docs-site/static/search-index.json via the docs build; a second build leaves it byte-identical. Fixes from the review of this PR, found after it was opened: - openai.md: GPT-6 (September 2026) is newer than GPT-5.4 (March 2026), so the guide no longer calls GPT-5.4 the newest or the latest. - CLAUDE.md: the CI-skip paragraph still blamed the %s-only format check for the bypass, which contradicted the sentence before it. GitHub skips the whole workflow before any step runs, so the paragraph now says the format check is not the cause. - onboarding-playbook plan: the Tier 2 bullet described a hand-written catalog row and a descriptor row; a Tier 2 provider is one JSON file under src/lib/providers/catalog/, and the onboarding gate checks that file instead of a manifest. Skipped or deferred: - T3810290322+T3810299660 (a link from tiers/README.md back to its parent): not done. The first attempt added a bare README key to LINK_MAPPINGS in sync-docs.ts, which would have sent about 7,500 API-reference links to the provider-integration README instead of the API index. It was reverted; a fix needs a link rule scoped to provider-integration/tiers. - PF-T3790294047 is only partly fixed: the outcome note is in the plan, but docs/MIGRATION.md still has no v11.0.0 entry. perplexity-decider is marked ci-mocked-only, the conservative value; its owner may upgrade it if the live probe counts. The catalog description of pareto-inference still says "conservative floor"; that is catalog data, left alone to avoid a codegen change in a docs commit.
…uides and plans Fixes the docs-accuracy review threads left open on merged PRs. Each claim was re-checked against the code on this checkout before editing. CLAUDE.md - CI-skip section: GitHub skips the push and pull_request runs when the head commit holds a directive, so the required check stays Pending and blocks the merge. `Reject CI-Skip Directives` is only a backstop and its regex does not cover a skip-checks trailer. (T3814059894-1, #1365) - Rule 15 allow list: the closed Grandfathered block is legacy debt without a per-file header and may shrink, never grow; same note beside the list in eslint.config.js. (T3818474525-allow-docs, #1378) - Audit snippet: the && chain moves into an `if`, so a failing audit cannot end a `set -e` caller's shell before the worktree cleanup. Proven with a bash `set -e` control. (T4051898811-1, #1676) - "Reading a CI result": incidents 1, 2 and 4 are the absence-of-signal mistake, 3 is its inverse. (T4042254379-intro-first-four, #1716) Provider and reference docs - openai.md and providers/index.md: gpt-5.4 context is 1.05M (mini and nano stay 400K), matching contextWindows.ts. (T4114160945 and T4114105048, #1824; one defect raised twice) - deepseek.md: close the unbalanced backtick that leaked into the search index. (T4112589028-b, #1800) - pareto-inference.md: no context window is published; 131,072 is a catalog fallback, not a floor or a vendor figure. (T4125607242, #1848) - docs/index.md: count MCP servers consistently. (T4072651139, #1776) - provider-selection.md: the Streaming row covers text-generation providers only; decision-only providers (four, not three) use decide(). (T4115057665, #1820) - README.md: drop the hand-kept tool-support counts and stop grouping LiteLLM with the zero-configuration local runtimes, since it needs a running proxy. (T4113418122-readme-count-stale-now, #1816; T4072651184, #1776) - openai-compat-catalog.md: every catalog provider except Groq maps TimeoutError to NetworkError. (T3806464799, #1353) - SAFETY-PRIMITIVES.md: only no-inline-secret-regex and provider-typed-errors still apply; SSRF, stream-span and isNeuroLink bypasses are review-only. (T3790049900-1, #1334) Plans - middleware plan: providers-mocked has no AI Studio section and is construction-only for Vertex and Bedrock; name the three real seams. (T3950529360#1, #1656) - dead-code-purge plan: record that the removal shipped in the major v11.0.0 and that there is no replacement for the removed types. (PF-T3790294047, #1335) - onboarding-playbook plan: repo-relative commands instead of machine-local paths, drop the uncommitted scratch spec links, "Every Tier 3+ provider" ends with a manifest (Tier 2 is declared in its catalog JSON), and the three misplaced closing fences are moved so the duplicate "Verification commands" H2s are gone. (T3790294048, T3790294049, T3790294054, #1335) Tooling - verify-provider-onboarding now requires addedInPR, filesTouched and manualTestStatus in a hand-written provider's manifest, as the manifests README already said. xor and perplexity-decider gain manualTestStatus "ci-mocked-only"; README lists "verified-live". New case in the provider-structure suite runs the real tool against a scratch manifests tree: red without the validator change, green with it. (T3790294060-a, #1335) - test-search-index-reproducibility asserts git merge-file could run, so a missing git reports ENOENT instead of a merge conflict. (T4108958700-git- guard, #1794) Regenerated: docs-site/static/search-index.json via the docs build; a second build leaves it byte-identical. Fixes from the review of this PR, found after it was opened: - openai.md: GPT-6 (September 2026) is newer than GPT-5.4 (March 2026), so the guide no longer calls GPT-5.4 the newest or the latest. - CLAUDE.md: the CI-skip paragraph still blamed the %s-only format check for the bypass, which contradicted the sentence before it. GitHub skips the whole workflow before any step runs, so the paragraph now says the format check is not the cause. - onboarding-playbook plan: the Tier 2 bullet described a hand-written catalog row and a descriptor row; a Tier 2 provider is one JSON file under src/lib/providers/catalog/, and the onboarding gate checks that file instead of a manifest. Skipped or deferred: - T3810290322+T3810299660 (a link from tiers/README.md back to its parent): not done. The first attempt added a bare README key to LINK_MAPPINGS in sync-docs.ts, which would have sent about 7,500 API-reference links to the provider-integration README instead of the API index. It was reverted; a fix needs a link rule scoped to provider-integration/tiers. - PF-T3790294047 is only partly fixed: the outcome note is in the plan, but docs/MIGRATION.md still has no v11.0.0 entry. perplexity-decider is marked ci-mocked-only, the conservative value; its owner may upgrade it if the live probe counts. The catalog description of pareto-inference still says "conservative floor"; that is catalog data, left alone to avoid a codegen change in a docs commit.
2 participants
What
Writes
docs-site/static/search-index.jsonone entry per line instead of as a single 8.5 MB line. The data doesn't change: the file parses to the same 11,042 entries asrelease.Why
The index is committed and every docs change regenerates it. As a single line, any two open PRs that touched docs conflicted on the whole file, whatever pages they edited. On 2026-09-25, 14 of the 24 open PRs showed a conflict, and for most of them this file (plus
docs/apipages) was the only one. That forced a rebase after every merge.#1763 made entry ids stable per URL and the traversal order fixed, so an unchanged page now produces an unchanged entry. Writing one entry per line turns that into unchanged lines, so git can merge two PRs that edit different pages.
Proof
Real three-way merge of two regenerated indexes, one editing the DeepSeek provider page and one the HuggingFace page:
git merge-filedocs-site/scripts/test-search-index-reproducibility.cjsnow asserts exactly that. It passes here, and fails with the one-line writer restored.test:docs-mcp3/3: the docs MCP server loads the new file.docs-sitebuilds give byte-identical output, so the "Generated artifacts are current" check stays deterministic.push-algolia-index.js, the docs MCP server) allJSON.parsethe file, so the layout is invisible to them.Summary by CodeRabbit