Repository navigation
feat: merge knowledge graph + conversation arc into memdir #1811
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 51 commits
Commits
Show all changes
53 commits
Select commit
Hold shift + click to select a range
92726f9
feat: merge knowledge graph + conversation arc into memdir
Gravirei 65b76a4
fix: type errors, add test suites, fix resetArc() disk-write bug
Gravirei 951a210
fix: remove generic type param from restore() call
Gravirei 123f824
fix: address CodeRabbit findings — YAML injection, secrets leak, cach…
Gravirei 91b235e
fix: redact URL credentials/query/hash in endpoint extraction; add pe…
Gravirei 4db3578
test: add regression for URL credential/query/hash redaction in fact …
Gravirei 1d13e61
feat: wire getOrchestratedMemory into query.ts prompt; remove dead pr…
Gravirei 3b52952
feat: enhance memory management by adding clearArcArtifacts function …
Gravirei 9049c3d
feat: enhance fact extraction by adding tests for absolute paths, bac…
Gravirei 09eed41
Review-fix: scoped IP tagging, scrubbedContent, arcMemoryDir null, ve…
Gravirei 7db4f33
Review-fix: isAutoMemoryEnabled gate, scrubbed paths, clearIndex in c…
Gravirei c77132e
Review-fix: URL-stripped path scan, digit-key/quoted-value env redact…
Gravirei 6ea502a
Review-fix: quoted multi-token env values fully redacted, regression …
Gravirei b973e5a
Review-fix: freshness check on each search, integration test, typeche…
Gravirei 58347fc
Review-fix: missing-index-file reinit, full-pipeline integration test
Gravirei 7ad10c5
Review-fix: gate arc/RAG on isAutoMemoryEnabled, add integration+stal…
Gravirei ffe0346
Review-fix: use >= for mtime staleness check to catch same-ms edits
Gravirei db8b551
Review-fix: move auto-memory gate to persistence layer
Gravirei 09fec56
Refactor memory handling: consolidate metadata retrieval and improve …
Gravirei f20f27f
Fix: update expectation to use toEqual for prompt comparison in conve…
Gravirei a41f198
fix: detect same-size content changes via content hash; handle text-b…
Gravirei d19cef8
fix: skip symlinked dirs in vector index walk; enable arc/multi-turn …
Gravirei 81f5e98
fix: follow symlinked dirs in vector index walk per review
Gravirei 7375ebc
fix: skip symlinked dirs in vector index walk; add symlink-boundary r…
Gravirei 3fb78fa
fix: skip indexing symlinked directories and ensure only files are pr…
Gravirei 241db19
fix: show cmd output on failure in verifier; add multi-turn coverage;…
Gravirei b04e14b
fix: clear memoized auto-mem path after teardown in knowledge + conve…
Gravirei eacfd78
fix: isolate vector index per memdir, filter secrets from backtick fa…
Gravirei 09d220a
fix: extend credential filter for AWS/GitLab tokens; reject all symli…
Gravirei 2cd56fa
fix: use repo's redactSecretSubstringsForDisplay; add npm/glpat/AKIA/…
Gravirei 14ff234
fix: P1 backtick credential safety + untrusted-data boundary; P2 lega…
Gravirei b6aac70
fix: B1-B5, M6, M8 — no message mutation, migration data loss, empty-…
Gravirei 1867090
fix: address 8 reviewer findings (R1-R8)
Gravirei 31f46a4
fix: close 9 memory findings — approval gate, non-fatal writes, secre…
Gravirei 72961b3
test: avoid global cwd pollution in knowledgeGraph tests
Gravirei dfcb5eb
fix: address 15 reviewer findings (R1-R15)
Gravirei 1263fb9
test: isolate governance mock by removing afterEach clear
Gravirei 1418ed7
fix: isolate governance mock per async context via executionAsyncId t…
Gravirei b82c632
Refactor knowledge graph legacy migration, secure secrets and IP octe…
Gravirei 9cc8743
Fix governancePolicy enablement in full test runs by checking for tes…
Gravirei ccb5fbb
fix: ensure auto memory is enabled in conversationArc, knowledgeGraph…
Gravirei 347fc26
fix: address P1/P2 review findings — redaction, bounds, legacy backup
Gravirei 92e7254
fix: address P1 findings — safe legacy retirement, multi-turn aggrega…
Gravirei bf33b8f
fix: address P1/P2 review findings — rule extraction gate, SQLite rea…
Gravirei c6d2b28
fix: tighten looksLikeOpaqueToken to avoid flagging compound model names
Gravirei 39dfb24
fix: address P1/P2 review findings — rebase, test isolation, SQLite r…
Gravirei 5d0f62a
fix: address P1/P2/P3 review findings — secret scrub on migrate, lean…
Gravirei 4121c52
fix: scrub secret entity names on migrate, exclude summary facts from…
Gravirei 5b69436
fix: address P1 review findings for legacy graph redaction, recovery-…
Gravirei cbf6b0e
Potential fix for pull request finding
Gravirei 2644050
fix: redact protocol-relative URL userinfo in legacy migration
Gravirei 4b1a1ac
Merge branch 'main' into codex/pr-1811
kevincodex1 dfcc402
fix(memory): harden memdir migration and retrieval
kevincodex1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)" | ||
| cd "$REPO_ROOT" | ||
|
|
||
| PASS=0 | ||
| FAIL=0 | ||
|
|
||
| pass() { echo " ✅ $1"; PASS=$((PASS+1)); } | ||
| fail() { echo " ❌ $1"; FAIL=$((FAIL+1)); } | ||
| check() { | ||
| local msg="$1"; shift | ||
| local tmpfile; tmpfile=$(mktemp) | ||
| if "$@" &>"$tmpfile"; then | ||
| rm -f "$tmpfile"; pass "$msg" | ||
| else | ||
| cat "$tmpfile"; rm -f "$tmpfile"; fail "$msg" | ||
| fi | ||
| } | ||
|
|
||
| echo "═══════════════════════════════════════════════════════════════" | ||
| echo " Verifying memdir-kg-merge refactor" | ||
| echo "═══════════════════════════════════════════════════════════════" | ||
| echo "" | ||
|
|
||
| # ── 1. File existence ────────────────────────────────────────────── | ||
| echo "── 1. File existence ──" | ||
|
|
||
| check "vectorIndex.ts exists" test -f src/memdir/vectorIndex.ts | ||
| check "autoExtractFacts.ts exists" test -f src/memdir/autoExtractFacts.ts | ||
| check "knowledgeGraph.ts exists" test -f src/utils/knowledgeGraph.ts | ||
| check "conversationArc.ts exists" test -f src/utils/conversationArc.ts | ||
|
|
||
| check "SQLiteProvider.ts is REMOVED" test ! -f src/utils/storage/SQLiteProvider.ts | ||
| check "JSONProvider.ts is REMOVED" test ! -f src/utils/storage/JSONProvider.ts | ||
| check "storage/ dir is REMOVED" test ! -d src/utils/storage | ||
|
|
||
| # ── 2. Build checks ──────────────────────────────────────────────── | ||
| echo "" | ||
| echo "── 2. Build checks ──" | ||
|
|
||
| check "knowledgeGraph.ts builds" bun build --no-bundle src/utils/knowledgeGraph.ts | ||
| check "conversationArc.ts builds" bun build --no-bundle src/utils/conversationArc.ts | ||
| check "vectorIndex.ts builds" bun build --no-bundle src/memdir/vectorIndex.ts | ||
| check "autoExtractFacts.ts builds" bun build --no-bundle src/memdir/autoExtractFacts.ts | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| # ── 3. Feature flags in build.ts ─────────────────────────────────── | ||
| echo "" | ||
| echo "── 3. Feature flags ──" | ||
|
|
||
| check "CONVERSATION_ARC=true in build.ts" grep -q "CONVERSATION_ARC: true" scripts/build.ts | ||
| check "MULTI_TURN_CONTEXT=true in build.ts" grep -q "MULTI_TURN_CONTEXT: true" scripts/build.ts | ||
|
|
||
| # ── 4. No dangling imports ───────────────────────────────────────── | ||
| echo "" | ||
| echo "── 4. No dangling imports ──" | ||
|
|
||
| # These symbols were removed — nothing should import them anymore | ||
| check "No addGlobalEntity imports" test "$(rg "import.*addGlobalEntity" src/ --type ts -c | wc -l)" -eq 0 | ||
| check "No addGlobalRelation imports" test "$(rg "import.*addGlobalRelation" src/ --type ts -c | wc -l)" -eq 0 | ||
| check "No addGlobalSummary imports" test "$(rg "import.*addGlobalSummary" src/ --type ts -c | wc -l)" -eq 0 | ||
| check "No addGlobalRule imports" test "$(rg "import.*addGlobalRule" src/ --type ts -c | wc -l)" -eq 0 | ||
| check "No SQLiteProvider imports" test "$(rg "SQLiteProvider" src/ --type ts -c | wc -l)" -eq 0 | ||
| check "No JSONProvider imports" test "$(rg "JSONProvider" src/ --type ts -c | wc -l)" -eq 0 | ||
|
|
||
| # ── 5. Export integrity ──────────────────────────────────────────── | ||
| echo "" | ||
| echo "── 5. Export integrity ──" | ||
|
|
||
| # knowledgeGraph.ts must export these (backward compat) | ||
| check "knowledgeGraph exports Entity" grep -q "export interface Entity" src/utils/knowledgeGraph.ts | ||
| check "knowledgeGraph exports extractKeywords" grep -q "export function extractKeywords" src/utils/knowledgeGraph.ts | ||
| check "knowledgeGraph exports getGlobalGraph" grep -q "export function getGlobalGraph" src/utils/knowledgeGraph.ts | ||
| check "knowledgeGraph exports getOrchestratedMemory" grep -q "export async function getOrchestratedMemory" src/utils/knowledgeGraph.ts | ||
| check "knowledgeGraph exports searchGlobalGraph" grep -q "export async function searchGlobalGraph" src/utils/knowledgeGraph.ts | ||
| check "knowledgeGraph exports resetGlobalGraph" grep -q "export function resetGlobalGraph" src/utils/knowledgeGraph.ts | ||
|
|
||
| # conversationArc.ts must export these | ||
| check "conversationArc exports Goal" grep -q "export interface Goal" src/utils/conversationArc.ts | ||
| check "conversationArc exports ConversationArc" grep -q "export interface ConversationArc" src/utils/conversationArc.ts | ||
| check "conversationArc exports initializeArc" grep -q "export function initializeArc" src/utils/conversationArc.ts | ||
| check "conversationArc exports updateArcPhase" grep -q "export async function updateArcPhase" src/utils/conversationArc.ts | ||
| check "conversationArc exports finalizeArcTurn" grep -q "export async function finalizeArcTurn" src/utils/conversationArc.ts | ||
| check "conversationArc exports getArcSummary" grep -q "export async function getArcSummary" src/utils/conversationArc.ts | ||
| check "conversationArc exports resetArc" grep -q "export function resetArc" src/utils/conversationArc.ts | ||
|
|
||
| # vectorIndex.ts must export these | ||
| check "vectorIndex exports initMemdirIndex" grep -q "export async function initMemdirIndex" src/memdir/vectorIndex.ts | ||
| check "vectorIndex exports searchMemdirIndex" grep -q "export async function searchMemdirIndex" src/memdir/vectorIndex.ts | ||
| check "vectorIndex exports rebuildIndex" grep -q "export async function rebuildIndex" src/memdir/vectorIndex.ts | ||
|
|
||
| # autoExtractFacts.ts must export extractFactsIntoMemdir | ||
| check "autoExtractFacts exports extractFactsIntoMemdir" grep -q "export async function extractFactsIntoMemdir" src/memdir/autoExtractFacts.ts | ||
|
|
||
| # ── 6. query.ts integration ──────────────────────────────────────── | ||
| echo "" | ||
| echo "── 6. query.ts integration ──" | ||
|
|
||
| # OpenClaude already has the user message hook for updateArcPhase | ||
| check "query.ts has updateArcPhase on user messages" grep -q "updateArcPhase" src/query.ts | ||
| check "query.ts has multiTurnContext startNewTurn" grep -q "startNewTurn" src/query.ts | ||
| check "query.ts has finalizeArcTurn" grep -q "finalizeArcTurn" src/query.ts | ||
|
|
||
| # ── 7. knowledge command ────────────────────────────────────────── | ||
| echo "" | ||
| echo "── 7. knowledge command ──" | ||
|
|
||
| check "knowledge.ts exists" test -f src/commands/knowledge/knowledge.ts | ||
| check "knowledge command registered" grep -q "knowledge" src/commands.ts | ||
|
|
||
| # ── 8. Config ────────────────────────────────────────────────────── | ||
| echo "" | ||
| echo "── 8. Config ──" | ||
|
|
||
| check "knowledgeGraphEnabled in config" grep -q "knowledgeGraphEnabled" src/utils/config.ts | ||
|
|
||
| # ── 9. No removed-symbol references in test files ─────────────────── | ||
| echo "" | ||
| echo "── 9. Test file hygiene ──" | ||
|
|
||
| check "No test imports addGlobalEntity" test "$(rg "addGlobalEntity" src/ --type ts -g '*.test.ts' -c | wc -l)" -eq 0 | ||
| check "No test imports loadProjectGraph" test "$(rg "loadProjectGraph" src/ --type ts -g '*.test.ts' -c | wc -l)" -eq 0 | ||
| check "No test imports getProjectGraphPath" test "$(rg "getProjectGraphPath" src/ --type ts -g '*.test.ts' -c | wc -l)" -eq 0 | ||
|
|
||
| # ── 10. full production build dry-run ────────────────────────────── | ||
| echo "" | ||
| echo "── 10. Production build check ──" | ||
|
|
||
| # Use bun build to verify the main entry points resolve | ||
| if bun build --no-bundle src/query.ts &>/dev/null; then | ||
| pass "query.ts builds (verifies full import chain)" | ||
| else | ||
| fail "query.ts builds (verifies full import chain)" | ||
| fi | ||
|
|
||
| # ── 11. Feature-flag entry path regression ────────────────────────── | ||
| echo "" | ||
| echo "── 11. Feature-flag entry path regression ──" | ||
|
|
||
| # Run the real production build flow using scripts/build.ts | ||
| if bun run scripts/build.ts &>/dev/null; then | ||
| # Verify that the build output does not contain un-transformed feature() calls | ||
| if ! grep -q "feature(\s*['\"]CONVERSATION_ARC['\"]" dist/cli.mjs && \ | ||
| ! grep -q "feature(\s*['\"]MULTI_TURN_CONTEXT['\"]" dist/cli.mjs; then | ||
| pass "production build resolved feature flags successfully" | ||
| else | ||
| fail "production build has unresolved feature() calls in bundle" | ||
| fi | ||
| else | ||
| fail "production build script (scripts/build.ts) failed" | ||
| fi | ||
|
|
||
| # ── 12. TypeScript type check ──────────────────────────────────────── | ||
| echo "" | ||
| echo "── 12. TypeScript type check ──" | ||
|
|
||
| check "bun run typecheck passes" bun run typecheck | ||
|
|
||
| # ── 13. Test suite ────────────────────────────────────────────────── | ||
| echo "" | ||
| echo "── 13. Test suite ──" | ||
|
|
||
| check "All focused tests pass" bun test src/memdir/vectorIndex.test.ts src/memdir/autoExtractFacts.test.ts src/utils/conversationArc.test.ts src/utils/knowledgeGraph.test.ts src/utils/multiTurnContext.test.ts src/commands/knowledge/knowledge.test.ts | ||
|
|
||
| echo "" | ||
| echo "═══════════════════════════════════════════════════════════════" | ||
| echo " Results: $PASS passed, $FAIL failed" | ||
| echo "═══════════════════════════════════════════════════════════════" | ||
|
|
||
| exit $FAIL | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Add focused regression coverage for the newly enabled runtime paths.
These flags turn on conversation-arc and multi-turn-context behavior in production builds. Please add or point to focused tests that exercise the exact enabled path, including
.arc.jsonpersistence/finalization and query hook integration. As per coding guidelines, “Add or update tests when the change affects behavior.” As per path instructions, “Block when risky runtime changes lack focused regression coverage.”🤖 Prompt for AI Agents
Sources: Coding guidelines, Path instructions