(MOT-4292) feat(openwiki): add openwiki worker - #651
Conversation
Source-grounded markdown wiki for any git repository, ported to the monorepo as a javascript bundle worker. - openwiki::* functions: generate / status / pages / page / search / refresh / set-schedule / lint / delete, scoped source readers for writer sub-agents (src::read / src::list / src::grep), write-page, cited ask, Mermaid diagram, AGENTS.md export, and the MCP trio (read-wiki-structure / read-wiki-contents / ask-question) - three generation tiers: harness lead agent spawning one writer sub-agent per page (line-cited), llm-router per-page completion, and a model-free heuristic fallback - per-wiki auto-refresh on cron triggers with a content-hash gate; incremental refresh regenerates only pages whose source changed - browser UI + JSON API served by the engine under /openwiki - typed request and response schemas on every registered function - release wiring: create-tag option, release.yml tag glob, modules table row, agent-permission denies for the internal trigger targets
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
skill-check — worker0 verified, 51 skipped (no docs/).
Four for four. Nicely done. |
|
Warning Review limit reached
Next review available in: 24 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
📝 WalkthroughWalkthroughOpenWiki adds a Node worker for source-grounded wiki generation, incremental refresh, cited pages, Q&A, diagrams, APIs, MCP tools, scheduling, and a browser UI. It also adds persistence, deployment integration, permissions, documentation, and automated tests. ChangesOpenWiki worker
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
🟡 Minor comments (12)
openwiki/src/index.mjs-1142-1147 (1)
1142-1147: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winThe live-progress channel has no failure contract. The server reports an unavailable stream with a success status, and the client discards stream errors. The result is a progress panel that stops updating with no fallback and no message.
openwiki/src/index.mjs#L1142-L1147: return501instead ofjsonResponse(200, { error: 'streaming unsupported' })whenw?.streamis absent, so the client can detect the failure.openwiki/src/lib/ui.mjs#L851-L892: ines.onerror, close theEventSourceand start the existing poll fallback when no event has arrived yet, instead of relying on automatic reconnect.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/index.mjs` around lines 1142 - 1147, The live-progress failure contract must be fixed in both sites: in openwiki/src/index.mjs:1142-1147, update the openwiki::http::events handler to return status 501 when w?.stream is unavailable; in openwiki/src/lib/ui.mjs:851-892, update es.onerror to close the EventSource and start the existing polling fallback when no event has arrived yet, rather than allowing automatic reconnect. Use the existing event-tracking and poll-fallback symbols in that handler.openwiki/src/lib/ask.mjs-124-151 (1)
124-151: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winFail fast when
harness::sendreturns nosession_id.
askDeeppassessession_idstraight toawaitTurn. If the trigger returns nosession_id,awaitTurnreceivesundefinedand the failure surfaces later as a timeout or an opaque error.runOrchestratorinopenwiki/src/lib/harness.mjs(Lines 349-350) already throws for this case. Match that behavior here.🐛 Proposed fix
}); + if (!session_id) throw new Error('harness::send returned no session_id'); const result = await awaitTurn(worker, session_id, { timeoutMs: 240_000 });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/ask.mjs` around lines 124 - 151, Update askDeep to validate the session_id returned by worker.trigger before calling awaitTurn. If it is missing, throw immediately using the same failure behavior and error semantics as runOrchestrator, while preserving the existing awaitTurn flow for valid session IDs.openwiki/src/lib/lint.mjs-30-44 (1)
30-44: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winTruncated files produce false
broken-citationissues.
readSourceFileinopenwiki/src/lib/inventory.mjs(Lines 173-185) caps the content atmaxBytesand appends\n...[truncated]. The line count at Line 38 then measures the truncated content only. For a source file larger than 200 KB, any citation past the cut is reported as "beyond N lines" even though the line exists.Read the full file for the line check, or skip the range check when the read is truncated.
🐛 Proposed fix
- let content; + let content; + let truncated = false; try { - ({ content } = await readSourceFile(dir, c.path, 200_000)); + ({ content, truncated } = await readSourceFile(dir, c.path, Number.MAX_SAFE_INTEGER)); } catch { issues.push({ slug, kind: 'broken-citation', detail: `missing file ${c.path}` }); continue; } - if (c.start_line || c.end_line) { + if (!truncated && (c.start_line || c.end_line)) {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/lint.mjs` around lines 30 - 44, Update the citation range validation around readSourceFile so truncated content does not produce false broken-citation issues: either read the complete source for line-count validation or detect the truncation marker and skip the start_line/end_line bounds check. Preserve missing-file handling and existing validation for complete content.openwiki/src/lib/search.mjs-19-41 (1)
19-41: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winDo not HTML-escape snippets in the JSON payload.
makeSnippetreturns escaped markup. The consumers treat the value as text:openwiki::http::searchserializes it to JSON, andrenderPageListinopenwiki/src/lib/ui.mjs(Line 1012) assigns it throughel('div', { class:'page-snippet', text: r.snippet }), which setstextContent. A snippet that contains<or&therefore renders as the literal<or&. Escaping belongs in the presentation layer, which already setstextContent.♻️ Proposed fix
-function escapeHtml(s) { - return s.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>'); -} - function makeSnippet(body, queryTokens) { const lower = body.toLowerCase(); let idx = -1; for (const t of queryTokens) { const i = lower.indexOf(t); if (i !== -1 && (idx === -1 || i < idx)) idx = i; } if (idx === -1) { const head = body.slice(0, 200); - return escapeHtml(head) + (body.length > 200 ? '…' : ''); + return head + (body.length > 200 ? '…' : ''); } const start = Math.max(0, idx - 80); const end = Math.min(body.length, idx + 120); let snip = body.slice(start, end); - snip = escapeHtml(snip); if (start > 0) snip = `…${snip}`; if (end < body.length) snip = `${snip}…`; return snip; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/search.mjs` around lines 19 - 41, Update makeSnippet to return raw snippet text instead of passing its output through escapeHtml, including both the matched and fallback branches. Preserve the existing slicing and ellipsis behavior; presentation already uses textContent, so remove only the payload-level HTML escaping.openwiki/src/lib/ui.mjs-736-741 (1)
736-741: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winConfirm the wiki deletion before the request.
The
×control callsdeleteWikion the first click.DELETE /openwiki/api/wikis/:idremoves the wiki and every page permanently, and no undo exists. The control is small and appears on hover next to the wiki name, so a mis-click destroys a full generation run.Add a confirmation step.
🛡️ Proposed fix
- onclick: (e) => { e.stopPropagation(); deleteWiki(w.id); }, + onclick: (e) => { + e.stopPropagation(); + if (confirm('Remove the wiki for ' + (w.repo_name || w.id) + '? This deletes all its pages.')) deleteWiki(w.id); + },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/ui.mjs` around lines 736 - 741, Update the `wiki-del` button handler in the wiki rendering block to request explicit user confirmation before calling `deleteWiki(w.id)`. Keep stopping event propagation, and invoke deletion only when confirmation is accepted; otherwise leave the wiki unchanged.openwiki/src/lib/configuration.mjs-7-11 (1)
7-11: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winSanitize
OPENWIKI_MAX_PARALLELandOPENWIKI_REFRESH_DEFAULT.If
OPENWIKI_MAX_PARALLELis not numeric,parseIntreturnsNaNandMath.max(1, NaN)returnsNaN.NaNthen reachesschema().properties.max_parallel.defaultandinitial_value, whereJSON.stringifyconverts it tonull. That value violates the declaredtype: 'integer'and can makeconfiguration::registerreject.OPENWIKI_REFRESH_DEFAULThas the same class of problem: an arbitrary string bypasses the declaredenum.🐛 Proposed fix
+const REFRESH_VALUES = ['off', '3h', '6h', '12h', 'daily', 'weekly']; +const parsedParallel = Number.parseInt(process.env.OPENWIKI_MAX_PARALLEL || '3', 10); +const envRefresh = process.env.OPENWIKI_REFRESH_DEFAULT || 'off'; + const DEFAULTS = { model: process.env.OPENWIKI_MODEL || 'claude-haiku-4-5-20251001', - max_parallel: Math.max(1, parseInt(process.env.OPENWIKI_MAX_PARALLEL || '3', 10)), - refresh_default: process.env.OPENWIKI_REFRESH_DEFAULT || 'off', + max_parallel: Number.isFinite(parsedParallel) ? Math.min(16, Math.max(1, parsedParallel)) : 3, + refresh_default: REFRESH_VALUES.includes(envRefresh) ? envRefresh : 'off', };The
Math.min(16, ...)clamp also keeps the default inside the declaredmaximum.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/configuration.mjs` around lines 7 - 11, Sanitize the DEFAULTS values for OPENWIKI_MAX_PARALLEL and OPENWIKI_REFRESH_DEFAULT in the configuration module: use a validated numeric fallback, clamp max_parallel to the declared 1–16 range, and accept refresh_default only when it matches the declared enum, otherwise use its valid default. Ensure these sanitized values are what schema().properties.max_parallel.default and initial_value receive.openwiki/README.md-50-51 (1)
50-51: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winUse a platform-neutral browser instruction.
The
opencommand is not available by default on Linux or Windows. Replace it with the URL itself or provide platform-specific alternatives. The documented quickstart otherwise fails for common deployments.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/README.md` around lines 50 - 51, Update the README quickstart browser instruction around the localhost URL to be platform-neutral: display the URL directly or provide appropriate alternatives for macOS, Linux, and Windows instead of relying on the `open` command.openwiki/src/lib/model.mjs-10-17 (1)
10-17: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winGuard against null catalog entries in the fallback branch.
Line 10 guards
m &&, andlistModelsfiltersm?.id. The fallback predicates on Lines 14-16 readx.supports_structured_outputwithout a guard, so a null entry in the router response throws a TypeError.🛡️ Proposed fix
- const list = Array.isArray(models) ? models : []; + const list = (Array.isArray(models) ? models : []).filter((m) => m?.id);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/model.mjs` around lines 10 - 17, Guard each fallback predicate in the model-selection logic around byId so null catalog entries are skipped before reading supports_structured_output or supports_tools. Preserve the existing preference order and list[0] fallback while ensuring null entries cannot cause a TypeError.openwiki/src/lib/schemas.mjs-16-45 (1)
16-45: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winAdd
refresh_schedule,last_refresh_at, andsteertoWIKI_META.
openwiki::wiki,openwiki::wikis, and the matching HTTP endpoints return the wiki record directly, but the schema requiresadditionalProperties: falseand does not define these persisted fields. Add them as properties, or project the record down to the current schema before returning it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/schemas.mjs` around lines 16 - 45, Update WIKI_META to define the persisted refresh_schedule, last_refresh_at, and steer fields in its properties while preserving additionalProperties: false. Use the existing schema type conventions and ensure direct wiki records and matching HTTP endpoint responses validate without dropping these fields.openwiki/src/lib/store.mjs-153-176 (1)
153-176: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winInvalidate the source cache when deleting a wiki.
src.mjskeepsinvCacheandreadStatsper wiki id, but both delete handlers call onlystore.deleteWiki(id). Move the cache invalidation to the delete path so a re-created wiki cannot reuse stale inventory while the module cache still holds the old keys.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/store.mjs` around lines 153 - 176, Update the wiki deletion flow in the delete handlers that call store.deleteWiki(id) to also remove the deleted wiki’s entries from the per-wiki invCache and readStats caches. Ensure both delete paths invalidate these caches after deletion so a recreated wiki cannot reuse stale inventory or statistics.openwiki/src/lib/store.mjs-222-228 (1)
222-228: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winSerialize
appendLoglike the page index.
appendLogperforms an unguarded read-modify-write onopenwiki:log. Concurrent writers (batch page writers and spawned sub-agents, the same concurrency the index lock was added for) overwrite each other's array, so log lines are lost. Reuse the per-wiki serialization.🔒 Proposed fix
export async function appendLog(wikiId, line) { const ts = new Date().toISOString().replace(/\.\d{3}Z$/, 'Z'); - const prev = (await sget(S_LOG, wikiId)) || []; - prev.push(`${ts} ${line}`); - if (prev.length > 500) prev.splice(0, prev.length - 500); - await sset(S_LOG, wikiId, prev); + await withIndexLock(wikiId, async () => { + const prev = (await sget(S_LOG, wikiId)) || []; + prev.push(`${ts} ${line}`); + if (prev.length > 500) prev.splice(0, prev.length - 500); + await sset(S_LOG, wikiId, prev); + }); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/store.mjs` around lines 222 - 228, Update appendLog to use the same per-wiki serialization mechanism already used by the page index, wrapping its read-modify-write of S_LOG so concurrent writers are serialized. Preserve the existing timestamping, 500-entry trimming, and persistence behavior.openwiki/src/lib/heuristic.mjs-167-206 (1)
167-206: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winGuard the inputs of the last-resort fallback.
generatePageinopenwiki/src/lib/generate.mjscalls this function from acatchblock. Ifcategories,sourceReads,allSlugs, oroutlineItem.source_pathsis nullish, this function throws aTypeError, the original LLM error is masked, and the page is lost.generatePageLLMalready defends the same inputs with(categories ?? [])and(sourceReads ?? []). Apply the same defaults here.🛡️ Proposed guards
export async function generatePageHeuristic({ outlineItem, - sourceReads, - allSlugs, - allTitles, - categories, + sourceReads = [], + allSlugs = [], + allTitles = [], + categories = [], repoName, repoUrl, }) { const title = outlineItem.title; - const category = categories.find((c) => c.id === outlineItem.category); + const category = categories.find((c) => c.id === outlineItem.category);lines.push('## Sources'); lines.push(''); - for (const p of outlineItem.source_paths) { + for (const p of outlineItem.source_paths || []) { lines.push(`- \`${p}\``); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/heuristic.mjs` around lines 167 - 206, Update generatePageHeuristic to safely handle nullish categories and sourceReads by using empty-array defaults before calling find, indexing, or iterating. Also guard any allSlugs and outlineItem.source_paths usage within this fallback with the same defaults, preserving the original LLM error instead of throwing a TypeError.
🧹 Nitpick comments (12)
openwiki/src/index.mjs (1)
716-720: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winApply
needIdto the remaining id-taking functions.The comment at Lines 66-70 explains that agents guess
wiki_idinstead ofid, andneedIdexists for that reason.openwiki::refresh,openwiki::status,openwiki::wiki,openwiki::lint,openwiki::gen-stats,openwiki::ask,openwiki::diagram,openwiki::read-wiki-structure, andopenwiki::read-wiki-contentsstill destructure{ id }directly. A wrong parameter name reaches the store asundefinedand produces the cryptic error the helper was added to prevent.♻️ Example for `openwiki::refresh`
-worker.registerFunction('openwiki::refresh', async ({ id }) => refreshWiki(id), { +worker.registerFunction('openwiki::refresh', async (a) => refreshWiki(needId(a, 'openwiki::refresh')), {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/index.mjs` around lines 716 - 720, Update the remaining id-taking worker registrations—refresh, status, wiki, lint, gen-stats, ask, diagram, read-wiki-structure, and read-wiki-contents—to use the existing needId helper instead of directly destructuring { id }. Preserve each handler’s current behavior while ensuring wiki_id is normalized to the id argument before reaching the store.openwiki/src/lib/ask.mjs (2)
186-202: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winLog the swallowed answer errors.
Both
catchblocks discard the error. When ask silently degrades to the heuristic stitch, no diagnostic remains for the operator. Capture the message in aconsole.warnso provider outages are visible.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/ask.mjs` around lines 186 - 202, Update both catch blocks in the answer-generation flow around askDeep and askFastLLM to capture the thrown error and emit its message with console.warn before setting answer to null, preserving the existing fallback behavior.
153-168: 🗄️ Data Integrity & Integration | 🔵 Trivial | 💤 Low value
fileAnsweroverwrites earlier answers for similar questions.
slugifystrips punctuation and truncates to 60 characters. "How does auth work?" and "How does auth work" produce the same slug, sostore.savePagereplaces the previous filed answer. Append a short hash or a counter to keep filed answers distinct.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/ask.mjs` around lines 153 - 168, Update fileAnswer so the slug derived from slugify(q) is made unique before passing it to store.savePage, appending a short deterministic hash or counter to distinguish questions that normalize to the same slug. Use the resulting unique slug consistently in the saved metadata and returned value.openwiki/src/lib/ui.mjs (1)
1046-1057: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe nav folder toggle is not keyboard-operable.
The
.fheadelement is adivwith a click handler only. It has notabindex, norole, and no key handler. The leaves at Lines 1039-1044 do settabindex: '0'. A keyboard user can therefore reach the pages of an expanded folder but cannot expand a collapsed one.♿ Proposed fix
folder.appendChild(el('div', { class: 'fhead', + tabindex: '0', + role: 'button', + 'aria-expanded': collapsed ? 'false' : 'true', + onkeydown: (e) => { if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); e.currentTarget.click(); } }, onclick: () => { if (state.collapsedNav.has(key)) state.collapsedNav.delete(key); else state.collapsedNav.add(key); renderPageList(); },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/ui.mjs` around lines 1046 - 1057, Make the `.fhead` navigation folder toggle keyboard-operable in `renderNavNode` by adding focusability and an appropriate button-like role, then handle keyboard activation for Enter and Space using the same collapse/expand logic as `onclick`. Preserve the existing mouse behavior and ensure Space does not trigger unwanted page scrolling.openwiki/src/lib/search.mjs (1)
43-85: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy liftSearch reads and tokenizes every page body on each query.
searchPagescallsgetPagefor each page and tokenizes the full body per query. The UI calls this on every debounced keystroke throughGET /openwiki/api/wikis/:id/search, so cost grows with page count and with query rate. For a wiki with many large pages, each search performs one state read per page.Consider caching a per-wiki inverted index or a token map, and invalidating it in
store.savePage.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/search.mjs` around lines 43 - 85, Optimize searchPages by reusing a per-wiki cached token map or inverted index instead of calling getPage and tokenizing every page body for each query. Build or refresh the cache from page data as needed, preserve the existing scoring, matched terms, snippets, sorting, and limit behavior, and invalidate the relevant wiki cache in store.savePage whenever a page is saved.openwiki/src/lib/schemas.mjs (1)
378-390: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider a model-facing citation schema without
url.
PAGE_HARNESS_OUTreusesCITATION, which exposesurl. The comment on Line 377 states that openwiki fills the host deep-link. A model can then supply an unverifiedurlthatmapResultoverwrites or, whencitationUrlreturns null, drops. A separate schema with onlypath,start_line,end_line, andnote(the shape already inlined inWRITE_PAGE_REQ) makes the contract explicit.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/schemas.mjs` around lines 378 - 390, Update PAGE_HARNESS_OUT to use a model-facing citation schema containing only path, start_line, end_line, and note, instead of reusing CITATION with url. Keep the existing citation array structure while ensuring model-supplied urls are not accepted.openwiki/src/lib/progress.mjs (1)
7-17: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low valueNamespace the event name.
pushProgressemits on the raw wiki id.EventEmitterreserveserror: an emit with that name and no listener throws. A prefix removes the collision class and keeps ids greppable.♻️ Proposed change
+const topic = (wikiId) => `wiki:${wikiId}`; + export function pushProgress(wikiId, evt) { - bus.emit(wikiId, evt); + bus.emit(topic(wikiId), evt); } export function onProgress(wikiId, cb) { - bus.on(wikiId, cb); - return () => bus.off(wikiId, cb); + bus.on(topic(wikiId), cb); + return () => bus.off(topic(wikiId), cb); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/progress.mjs` around lines 7 - 17, Update pushProgress and onProgress to use a shared prefixed event-name namespace derived from wikiId instead of emitting or subscribing to the raw wikiId. Ensure both functions apply the same prefix so progress delivery remains unchanged while preventing collisions with reserved EventEmitter names such as error.openwiki/src/lib/quality.mjs (1)
24-51: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winRun the structural checks on fence-stripped markdown.
countWordsstrips fenced code, but the heading,Relevant Source Files, andSources:checks read the raw markdown. A page that only contains those tokens inside a```block passes the gate. Strip fences once, then reuse the result for the structural checks.♻️ Proposed change
+const stripFences = (s) => s.replace(/```[\s\S]*?```/g, '\n'); + export function getPageQualityIssues(markdown, opts = {}) { const minWords = opts.minWords ?? 180; - const md = String(markdown || ''); + const md = stripFences(String(markdown || ''));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/quality.mjs` around lines 24 - 51, Update getPageQualityIssues to normalize markdown with stripFences(String(markdown || '')) before running heading, section, Sources, and word-count checks, reusing that single stripped result for all validations. Ensure stripFences is available through the existing quality module utility or define it alongside the function without changing other validation behavior.openwiki/src/lib/harness.mjs (2)
236-271: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
return bestis unreachable for validmaxAttempts.Every iteration either returns on Line 266 or continues, and the last iteration always satisfies
attempt === maxAttempts. So Line 271 only runs whenmaxAttempts < 1, and it then returnsnull, which callers do not expect. ClampmaxAttemptsto at least 1 and drop the trailing return, or keep the return and document the clamp.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/harness.mjs` around lines 236 - 271, Update the retry loop around runPageTurn to clamp maxAttempts to at least 1 before iteration, ensuring every valid execution reaches the existing return inside the loop; then remove the unreachable trailing return best, or retain it only if the clamp is explicitly documented and callers’ expected result is preserved.
16-27: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low valueEncode path segments in the deep link.
citationUrlinterpolatespathdirectly. A path with a space or#produces a broken GitHub URL. Encode each segment.♻️ Proposed change
- return `https://github.com/${owner}/${repo}/blob/${commit}/${path}${frag}`; + const safePath = String(path).split('/').map(encodeURIComponent).join('/'); + return `https://github.com/${owner}/${repo}/blob/${commit}/${safePath}${frag}`;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/harness.mjs` around lines 16 - 27, Update citationUrl to encode each slash-delimited path segment before interpolating path into the GitHub URL, preserving slash separators and line-fragment behavior. Use encodeURIComponent for individual segments so spaces, #, and other special characters produce valid deep links.openwiki/src/lib/inventory.mjs (1)
146-161: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winConsider hashing file metadata instead of file contents during the walk.
inventoryRepoopens and reads up tomaxBytesof every file only to derive a 12-character sha1 prefix. On a large repository this reads hundreds of megabytes on the clone path, and the inventory is rebuilt after each clone or refresh. If theshafield is only used for change detection,sizeplusmtimeMsfrom the existingfs.statresult gives the same signal without reading bodies. If content identity is required, read only the first few kilobytes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/inventory.mjs` around lines 146 - 161, Update inventoryRepo’s per-file hashing logic to avoid reading up to maxBytes during repository walks. Derive the 12-character sha1 value from the existing fs.stat metadata, such as size and mtimeMs, while preserving the sha field’s change-detection role; if content-based identity is required, cap reads to only a small fixed prefix instead of maxBytes.openwiki/src/lib/docs_oracle.mjs (1)
61-72: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winCap the total time spent probing documentation origins.
The loop probes up to four origins with two paths each. Every
webFetchcall allows 30 s, and the calls are sequential. A slow or blackholed documentation host therefore adds up to 240 s to the planning phase before thedocs/fallback runs. The caller inopenwiki/src/index.mjswraps the oracle intry/catch, not in a deadline.Add an overall budget, or lower the per-call
timeoutMs, and stop probing when the budget is exhausted.⏱️ Proposed budget
export async function fetchDocsIndex(worker, { repoUrl, readme, repoDir }) { // 1) llms.txt at README-referenced documentation origins. + const deadline = Date.now() + 45_000; for (const origin of candidateOrigins(repoUrl, readme)) { for (const p of ['/llms.txt', '/llms-full.txt']) { + if (Date.now() > deadline) break; const txt = await webFetch(worker, origin + p);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@openwiki/src/lib/docs_oracle.mjs` around lines 61 - 72, The fetchDocsIndex function contains nested loops that sequentially call webFetch with 30-second timeouts for each of up to eight calls (four origins and two paths each), potentially causing a 240-second hang before fallback. Implement an overall time budget for the entire fetchDocsIndex function that tracks elapsed time across all webFetch calls. Either reduce the per-call timeoutMs value passed to webFetch, or add logic to break out of both the inner path loop and outer candidateOrigins loop when the remaining budget is exhausted. Ensure the function returns or falls through to the docs/ fallback when the budget runs out.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@openwiki/src/index.mjs`:
- Around line 1072-1082: Update the HTTP DELETE handler registered as
openwiki::http::wiki-delete to call applyWikiSchedule(id, 'off') before
store.deleteWiki(id), matching the existing openwiki::delete behavior and
ensuring the per-wiki cron trigger is removed before deletion.
- Around line 551-620: Guard refreshWiki against concurrent runs by moving its
current implementation into a doRefresh helper and adding an inflight
check/entry at the outer refresh entry point before any gitPull or cloneRepo
work begins. Ensure the inflight marker remains active until the scheduled
runGeneration or runRefresh claims ownership, or re-add it inside the
setImmediate callback, while preserving cleanup and existing refresh responses.
In `@openwiki/src/lib/generate.mjs`:
- Around line 48-73: Update buildKeyDocs to accept the clone directory and read
each document using repoDir combined with e.relPath instead of the nonexistent
e.path. Thread repoDir through the caller chain from planWiki, preserving the
existing document filtering, truncation, and missing-file handling.
In `@openwiki/src/lib/git.mjs`:
- Around line 91-104: Update gitPull to fetch additional history instead of
using a depth-1 fetch, preserving previously stored commits for gitDiff
comparisons. In refreshWiki, handle gitDiff failures as unknown changes and
trigger the existing full rebuild path rather than leaving changed or affected
empty and returning up_to_date.
- Around line 75-82: Update cloneRepo to validate repoUrl and ref before
constructing git arguments: reject any value beginning with “-”, reject repoUrl
schemes outside https, http, git, and ssh-style URLs (including ext transport),
and terminate clone option parsing with “--” before repoUrl and destDir.
Preserve the existing ref handling and clone behavior for validated inputs.
In `@openwiki/src/lib/harness.mjs`:
- Around line 261-266: Update the schemas represented by PAGE_META and PAGE_RES
to declare the quality_issues frontmatter field with its numeric type,
preserving additionalProperties: false; keep the assignment in the quality-check
flow unchanged so returned persisted frontmatter validates against both schemas.
In `@openwiki/src/lib/inventory.mjs`:
- Around line 173-185: Update readSourceFile to read through a file handle with
a bounded buffer, requesting no more than maxBytes + 1 bytes instead of loading
the complete file via fs.readFile. Preserve the existing content and truncated
results: mark truncation when the extra byte is present, return only maxBytes
decoded bytes followed by the truncation marker, and close the handle reliably.
In `@openwiki/src/lib/model.mjs`:
- Around line 29-44: Update resolveModel so cache.set(key, out) runs only when
pickModel returns a resolved result; return unresolved results without caching
them, while preserving cached-result lookup and normal caching for successful
resolutions.
In `@openwiki/src/lib/src.mjs`:
- Around line 112-128: Update srcGrep to reject patterns exceeding a defined
maximum length before constructing the RegExp, returning the existing
empty-result response. Add a scan deadline and check it during the file/line
processing loop so the operation sets truncated and stops once the deadline is
reached, while preserving the existing max and MAX_GREP_FILES limits.
In `@openwiki/src/lib/ui.mjs`:
- Around line 1284-1332: Update openAbout and openAsk to use the shared
openModal helper with each modal box, declaring role="dialog" and
aria-modal="true". Use the returned close function for the Close buttons, and
ensure openModal handles Escape dismissal, traps Tab focus within the dialog,
and restores focus to the previously focused element when closed.
---
Minor comments:
In `@openwiki/README.md`:
- Around line 50-51: Update the README quickstart browser instruction around the
localhost URL to be platform-neutral: display the URL directly or provide
appropriate alternatives for macOS, Linux, and Windows instead of relying on the
`open` command.
In `@openwiki/src/index.mjs`:
- Around line 1142-1147: The live-progress failure contract must be fixed in
both sites: in openwiki/src/index.mjs:1142-1147, update the
openwiki::http::events handler to return status 501 when w?.stream is
unavailable; in openwiki/src/lib/ui.mjs:851-892, update es.onerror to close the
EventSource and start the existing polling fallback when no event has arrived
yet, rather than allowing automatic reconnect. Use the existing event-tracking
and poll-fallback symbols in that handler.
In `@openwiki/src/lib/ask.mjs`:
- Around line 124-151: Update askDeep to validate the session_id returned by
worker.trigger before calling awaitTurn. If it is missing, throw immediately
using the same failure behavior and error semantics as runOrchestrator, while
preserving the existing awaitTurn flow for valid session IDs.
In `@openwiki/src/lib/configuration.mjs`:
- Around line 7-11: Sanitize the DEFAULTS values for OPENWIKI_MAX_PARALLEL and
OPENWIKI_REFRESH_DEFAULT in the configuration module: use a validated numeric
fallback, clamp max_parallel to the declared 1–16 range, and accept
refresh_default only when it matches the declared enum, otherwise use its valid
default. Ensure these sanitized values are what
schema().properties.max_parallel.default and initial_value receive.
In `@openwiki/src/lib/heuristic.mjs`:
- Around line 167-206: Update generatePageHeuristic to safely handle nullish
categories and sourceReads by using empty-array defaults before calling find,
indexing, or iterating. Also guard any allSlugs and outlineItem.source_paths
usage within this fallback with the same defaults, preserving the original LLM
error instead of throwing a TypeError.
In `@openwiki/src/lib/lint.mjs`:
- Around line 30-44: Update the citation range validation around readSourceFile
so truncated content does not produce false broken-citation issues: either read
the complete source for line-count validation or detect the truncation marker
and skip the start_line/end_line bounds check. Preserve missing-file handling
and existing validation for complete content.
In `@openwiki/src/lib/model.mjs`:
- Around line 10-17: Guard each fallback predicate in the model-selection logic
around byId so null catalog entries are skipped before reading
supports_structured_output or supports_tools. Preserve the existing preference
order and list[0] fallback while ensuring null entries cannot cause a TypeError.
In `@openwiki/src/lib/schemas.mjs`:
- Around line 16-45: Update WIKI_META to define the persisted refresh_schedule,
last_refresh_at, and steer fields in its properties while preserving
additionalProperties: false. Use the existing schema type conventions and ensure
direct wiki records and matching HTTP endpoint responses validate without
dropping these fields.
In `@openwiki/src/lib/search.mjs`:
- Around line 19-41: Update makeSnippet to return raw snippet text instead of
passing its output through escapeHtml, including both the matched and fallback
branches. Preserve the existing slicing and ellipsis behavior; presentation
already uses textContent, so remove only the payload-level HTML escaping.
In `@openwiki/src/lib/store.mjs`:
- Around line 153-176: Update the wiki deletion flow in the delete handlers that
call store.deleteWiki(id) to also remove the deleted wiki’s entries from the
per-wiki invCache and readStats caches. Ensure both delete paths invalidate
these caches after deletion so a recreated wiki cannot reuse stale inventory or
statistics.
- Around line 222-228: Update appendLog to use the same per-wiki serialization
mechanism already used by the page index, wrapping its read-modify-write of
S_LOG so concurrent writers are serialized. Preserve the existing timestamping,
500-entry trimming, and persistence behavior.
In `@openwiki/src/lib/ui.mjs`:
- Around line 736-741: Update the `wiki-del` button handler in the wiki
rendering block to request explicit user confirmation before calling
`deleteWiki(w.id)`. Keep stopping event propagation, and invoke deletion only
when confirmation is accepted; otherwise leave the wiki unchanged.
---
Nitpick comments:
In `@openwiki/src/index.mjs`:
- Around line 716-720: Update the remaining id-taking worker
registrations—refresh, status, wiki, lint, gen-stats, ask, diagram,
read-wiki-structure, and read-wiki-contents—to use the existing needId helper
instead of directly destructuring { id }. Preserve each handler’s current
behavior while ensuring wiki_id is normalized to the id argument before reaching
the store.
In `@openwiki/src/lib/ask.mjs`:
- Around line 186-202: Update both catch blocks in the answer-generation flow
around askDeep and askFastLLM to capture the thrown error and emit its message
with console.warn before setting answer to null, preserving the existing
fallback behavior.
- Around line 153-168: Update fileAnswer so the slug derived from slugify(q) is
made unique before passing it to store.savePage, appending a short deterministic
hash or counter to distinguish questions that normalize to the same slug. Use
the resulting unique slug consistently in the saved metadata and returned value.
In `@openwiki/src/lib/docs_oracle.mjs`:
- Around line 61-72: The fetchDocsIndex function contains nested loops that
sequentially call webFetch with 30-second timeouts for each of up to eight calls
(four origins and two paths each), potentially causing a 240-second hang before
fallback. Implement an overall time budget for the entire fetchDocsIndex
function that tracks elapsed time across all webFetch calls. Either reduce the
per-call timeoutMs value passed to webFetch, or add logic to break out of both
the inner path loop and outer candidateOrigins loop when the remaining budget is
exhausted. Ensure the function returns or falls through to the docs/ fallback
when the budget runs out.
In `@openwiki/src/lib/harness.mjs`:
- Around line 236-271: Update the retry loop around runPageTurn to clamp
maxAttempts to at least 1 before iteration, ensuring every valid execution
reaches the existing return inside the loop; then remove the unreachable
trailing return best, or retain it only if the clamp is explicitly documented
and callers’ expected result is preserved.
- Around line 16-27: Update citationUrl to encode each slash-delimited path
segment before interpolating path into the GitHub URL, preserving slash
separators and line-fragment behavior. Use encodeURIComponent for individual
segments so spaces, #, and other special characters produce valid deep links.
In `@openwiki/src/lib/inventory.mjs`:
- Around line 146-161: Update inventoryRepo’s per-file hashing logic to avoid
reading up to maxBytes during repository walks. Derive the 12-character sha1
value from the existing fs.stat metadata, such as size and mtimeMs, while
preserving the sha field’s change-detection role; if content-based identity is
required, cap reads to only a small fixed prefix instead of maxBytes.
In `@openwiki/src/lib/progress.mjs`:
- Around line 7-17: Update pushProgress and onProgress to use a shared prefixed
event-name namespace derived from wikiId instead of emitting or subscribing to
the raw wikiId. Ensure both functions apply the same prefix so progress delivery
remains unchanged while preventing collisions with reserved EventEmitter names
such as error.
In `@openwiki/src/lib/quality.mjs`:
- Around line 24-51: Update getPageQualityIssues to normalize markdown with
stripFences(String(markdown || '')) before running heading, section, Sources,
and word-count checks, reusing that single stripped result for all validations.
Ensure stripFences is available through the existing quality module utility or
define it alongside the function without changing other validation behavior.
In `@openwiki/src/lib/schemas.mjs`:
- Around line 378-390: Update PAGE_HARNESS_OUT to use a model-facing citation
schema containing only path, start_line, end_line, and note, instead of reusing
CITATION with url. Keep the existing citation array structure while ensuring
model-supplied urls are not accepted.
In `@openwiki/src/lib/search.mjs`:
- Around line 43-85: Optimize searchPages by reusing a per-wiki cached token map
or inverted index instead of calling getPage and tokenizing every page body for
each query. Build or refresh the cache from page data as needed, preserve the
existing scoring, matched terms, snippets, sorting, and limit behavior, and
invalidate the relevant wiki cache in store.savePage whenever a page is saved.
In `@openwiki/src/lib/ui.mjs`:
- Around line 1046-1057: Make the `.fhead` navigation folder toggle
keyboard-operable in `renderNavNode` by adding focusability and an appropriate
button-like role, then handle keyboard activation for Enter and Space using the
same collapse/expand logic as `onclick`. Preserve the existing mouse behavior
and ensure Space does not trigger unwanted page scrolling.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 1d37fead-1b74-4d61-ab00-87cefe4e095f
⛔ Files ignored due to path filters (3)
openwiki/assets/openwiki-dark.pngis excluded by!**/*.pngopenwiki/assets/openwiki-light.pngis excluded by!**/*.pngopenwiki/pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (43)
.github/workflows/create-tag.yml.github/workflows/release.ymlREADME.mdiii-permissions.yamlopenwiki/.gitignoreopenwiki/README.mdopenwiki/biome.jsonopenwiki/iii.worker.yamlopenwiki/package.jsonopenwiki/scripts/build-bundle.mjsopenwiki/skills/SKILL.mdopenwiki/src/index.mjsopenwiki/src/lib/agents_md.mjsopenwiki/src/lib/ask.mjsopenwiki/src/lib/configuration.mjsopenwiki/src/lib/diagram.mjsopenwiki/src/lib/docs_oracle.mjsopenwiki/src/lib/generate.mjsopenwiki/src/lib/git.mjsopenwiki/src/lib/harness.mjsopenwiki/src/lib/heuristic.mjsopenwiki/src/lib/inventory.mjsopenwiki/src/lib/lint.mjsopenwiki/src/lib/model.mjsopenwiki/src/lib/nav.mjsopenwiki/src/lib/progress.mjsopenwiki/src/lib/quality.mjsopenwiki/src/lib/schemas.mjsopenwiki/src/lib/search.mjsopenwiki/src/lib/src.mjsopenwiki/src/lib/store.mjsopenwiki/src/lib/turnbus.mjsopenwiki/src/lib/ui.mjsopenwiki/tests/config.test.mjsopenwiki/tests/docs.test.mjsopenwiki/tests/git.test.mjsopenwiki/tests/harness.test.mjsopenwiki/tests/model.test.mjsopenwiki/tests/nav.test.mjsopenwiki/tests/quality.test.mjsopenwiki/tests/src.test.mjsopenwiki/tests/store.test.mjsopenwiki/tests/surfaces.test.mjs
…ded reads - refresh: per-wiki inflight guard (concurrent refresh returns in_progress), failed diffs rebuild instead of reporting up_to_date, and gitPull fetches with history so the recorded commit stays diffable - git: validate repo_url/ref before building argv (scheme allowlist, no leading dash) and terminate option parsing with -- - HTTP delete tears down the wiki's cron trigger and drops per-wiki inventory/read-stat caches, same as openwiki::delete - schemas: declare refresh_schedule/last_refresh_at/steer on wiki meta and quality_issues on page meta; model-facing citation schema omits url - planning: key-docs reader resolves inventory relPath against the clone dir (was silently reading nothing) - bounded reads: readSourceFile reads at most maxBytes+1 through a handle; src::grep caps pattern length and stops at a scan deadline - model resolution: never cache an unresolved router lookup; skip null catalog entries - config: clamp OPENWIKI_MAX_PARALLEL and validate OPENWIKI_REFRESH_DEFAULT against the declared schema - ask: fail fast on a missing harness session id, log fallback causes, hash-suffix filed-answer slugs; lint skips line checks on truncated reads; search snippets stay raw text (UI renders via textContent) - UI: shared modal helper (dialog role, Escape, focus trap and restore), delete confirmation, keyboard-operable nav folders, SSE errors before the first event fall back to polling (server returns 501 when unstreamable) - heuristic fallback tolerates partial options; docs probe runs under an overall time budget; quality checks ignore fenced code
1 participant
Ports openwiki from iii-experimental/openwiki into the monorepo as a javascript bundle worker, so it installs with
iii worker add openwiki.openwiki builds and maintains a source-grounded, interlinked markdown wiki for any git repository and serves a browser UI + JSON API under
/openwiki. A lead agent on the harness plans the index and spawns one writer sub-agent per page; each writer reads its files through openwiki's scoped readers and stores a cited page withopenwiki::write-page. Without the harness it falls back to onerouter::completeper page, and below that to a model-free heuristic tier, so a bare engine still produces a browsable wiki. Refresh is incremental (git diff to affected pages through the file-to-page index) and each wiki can run it on its own cron cadence.What changed for the monorepo
pnpm@10.18.2with committedpnpm-lock.yaml,scripts/build-bundle.mjs(esbuild single file, iii-sdk package.json inlined), manifestscripts.start: node ./index.mjs, noscripts.setup.^0.21.6; manifest dependencies per sibling convention:state ^0.21.2,cron ^0.21.0,llm-router ^1.0.0.tags:for registry discovery; per-workerbiome.json; source formatted with biome and two lint findings fixed (a restricted-nameescape()shadow renamed, an unused parameter dropped).worker-readme.md(Install / Quickstart / Configuration, absolute image links);skills/SKILL.mdperDOCUMENTATION_GUIDELINES.md.docs/sops/new-worker.mdsection 6:create-tag.ymloption,release.ymltag globopenwiki/v*, modules table row in the top-level README.iii-permissions.yaml: denies for the internal trigger targets (openwiki::cron::refresh-due,openwiki::on-turn-started,openwiki::on-turn-completed,openwiki::on-config-change); the read surface stays at the needs_approval default.Verification
biome ci openwikiclean (CI-exact invocation), 44/44 tests on Node 22,node --checkon entrypoint and bundle.build:bundleproduces a 764 KB self-containeddist/bundle/index.mjs./openwiki,openwiki::generateon a sample repository ran the harness agent tier toreadywith a cited page andopenwiki::lintreporting zero issues, thenopenwiki::deletecleaned up.Release notes
nextwith the experimental flag, per the new-worker SOP._bundle.ymlpnpm pin fix (fix(release): take the bundle pnpm version from packageManager #650) lands; the tag can be cut and the Release workflow re-dispatched afterwards.Fixes MOT-4292
Summary by CodeRabbit
New Features
Documentation