[group/sync] Fix ManifestExtractor never called — config.links always produced 0 cross-links#827
Conversation
ManifestExtractor was fully implemented in extractors/manifest-extractor.ts but never imported or called in sync.ts. As a result, any links declared in group.yaml were parsed and validated by config-parser.ts but silently dropped — config.links was always an empty dead-end as far as syncGroup was concerned. Changes: - Import ManifestExtractor in sync.ts - Call extractFromManifest(config.links, dbExecutors) inside the outer try block, after all repos are processed but before the finally closes the DB pools (symbol resolution via resolveSymbol requires open executors) - Collect the resulting contracts into autoContracts and the cross-links into a separate manifestCrossLinks array - Merge manifestCrossLinks into the final crossLinks alongside runExactMatch results Without this fix, users who declare explicit service dependencies in group.yaml links (the documented workaround for HTTP clients that use absolute URLs and are invisible to the auto-extractors) get 0 cross-links regardless of what they configure.
Add a unit test that asserts config.links entries produce contract pairs and a manifest cross-link (matchType: 'manifest') via syncGroup. Also refactors the manifest extraction call to sit outside the else/try block so it runs regardless of extractorOverride arity — makes the code testable without mocked DB pools and ensures links work when callers supply a zero-arity override (e.g. in tests or programmatic usage).
|
Someone is attempting to deploy a commit to the NexusCore Team on Vercel. A member of the Team first needs to authorize it. |
xkonjin
left a comment
There was a problem hiding this comment.
Code Review: Fix ManifestExtractor never called
This PR fixes a real functional gap: config.links were parsed but silently dropped because ManifestExtractor was imported but never instantiated in the sync pipeline.
What's good
- Clean placement after the DB try/finally block. Using synthetic UIDs as fallback when dbExecutors is unavailable is the right resilience pattern.
- Test covers the happy path: manifest links produce cross-links with matchType: 'manifest' and confidence 1.0.
Potential issues
-
Duplicate contracts when manifest links overlap with auto-extracted contracts. If an HTTP route extractor also produces a contract with the same ID, autoContracts will contain duplicates. runExactMatch and downstream consumers should be checked to see if they tolerate duplicate contract IDs.
-
Error handling in manifestEx.extractFromManifest is opaque. If a link references a repo not present in config.repos, what happens? The test uses repos that exist in the config. Consider adding a test for a dangling manifest link (e.g., from: 'app/missing').
-
Synthetic UID determinism. The comment says resolveSymbol falls back to synthetic UIDs when dbExecutors is undefined. If dbExecutors is defined but a specific repo pool is closed or missing, do all synthetic UIDs for the same symbol remain stable across runs? Non-deterministic UIDs would break cross-link stability.
Suggestions
- Add a test verifying behavior when dbExecutors is undefined to ensure synthetic fallback is exercised.
- Consider logging unmatched manifest links at warn level rather than silently skipping them.
Overall: solid bug fix, well-scoped. Just verify duplicate contract tolerance.
xkonjin
left a comment
There was a problem hiding this comment.
Code Review — PR #827
Overall this is a solid, well-documented fix for a real gap: was implemented but never wired into , so were silently dropped.
Bugs / edge cases
- The new variable is declared but not used beyond merging into . That usage is fine, but double-check that downstream consumers of (e.g., the registry writer or any exact-match dedup logic) won’t create duplicate entries if already produced a contract that overlaps with a manifest-declared one. Consider adding a cross-link dedup step or a unit test for overlap.
- is captured after the try/finally block. The comment says resolveSymbol falls back to synthetic UIDs when undefined, but verify that handles an empty Map correctly and never throws on .
Security
- No concerns. No new inputs are eval’d or executed. is already validated at the config layer.
Test coverage
- The new test covers the happy path well. I’d suggest one additional test: (e.g., pass that causes all pools to fail or mock a closed pool). That would validate the fallback behavior described in the code comment.
Nit
- Empty line added inside the finally block (after the block) is harmless but unrelated to this change. Consider removing to keep the diff minimal.
Nice catch shipping this — it closes a real functional hole in the pipeline.
xkonjin
left a comment
There was a problem hiding this comment.
Code Review — PR #827
Overall this is a solid, well-documented fix for a real gap: ManifestExtractor was implemented but never wired into syncGroup, so config.links were silently dropped.
Bugs / edge cases
- The new manifestCrossLinks variable is declared but not used beyond merging into crossLinks. That usage is fine, but double-check that downstream consumers of crossLinks (e.g., the registry writer or any exact-match dedup logic) won't create duplicate entries if runExactMatch already produced a contract that overlaps with a manifest-declared one. Consider adding a cross-link dedup step or a unit test for overlap.
- dbExecutors is captured after the try/finally block. The comment says resolveSymbol falls back to synthetic UIDs when undefined, but verify that ManifestExtractor.extractFromManifest handles an empty Map correctly and never throws on undefined.
Security
- No concerns. No new inputs are eval'd or executed. config.links is already validated at the config layer.
Test coverage
- The new test covers the happy path well. I'd suggest one additional test: "manifest links still produce cross-links when dbExecutors is unavailable" (e.g., pass opts that causes all pools to fail or mock a closed pool). That would validate the fallback behavior described in the code comment.
Nit
- Empty line added inside the finally block (after the missingRepos.push block) is harmless but unrelated to this change. Consider removing to keep the diff minimal.
Nice catch shipping this — it closes a real functional hole in the pipeline.
CI Report✅ All checks passed Pipeline Status
Test Results
✅ All 6239 tests passed 97 test(s) skipped — expand for details
Code CoverageTests
📋 View full run · Generated by CI |
|
@claude try to find bugs, performance issues, clean code violations. |
|
Claude finished @magyargergo's task in 2m 24s —— View job Code Review — PR #827
Bugs1. Duplicate cross-links for every manifest link (High)
autoContracts.push(...manifestResult.contracts); // ← manifest contracts injected here
// ...
const { matched, unmatched } = runExactMatch(autoContracts); // ← also sees them
const crossLinks: CrossLink[] = [...matched, ...manifestCrossLinks]; // ← both included
The new test at line 234 only checks Fix: Exclude manifest contracts from the // separate auto-detected from manifest contracts before matching
const { matched, unmatched } = runExactMatch(autoContracts.filter(c => c.meta?.source !== 'manifest'));Or keep track of the manifest contracts separately instead of merging them into 2. Stale executor references passed after pool close (Medium)
} finally {
for (const id of [...new Set(openPoolIds)]) {
await closeLbug(id).catch(() => {}); // ← all pools closed here
}
}
// ...
const manifestEx = new ManifestExtractor();
const manifestResult = await manifestEx.extractFromManifest(config.links, dbExecutors); // ← stale executors
The comment says "resolveSymbol falls back to synthetic UIDs when dbExecutors is undefined or a pool is closed" — the undefined path is clean, but the closed-pool path goes through exception handling. Passing // Option A: clear the map so resolveSymbol sees no executor
dbExecutors?.clear();
// or
// Option B: pass undefined
await manifestEx.extractFromManifest(config.links, undefined);Unless there is a concrete future plan to run Performance3. Sequential symbol resolution in
|
| Issue | Severity | |
|---|---|---|
| Bug | Manifest contracts fed into runExactMatch → duplicate cross-links per manifest link |
High |
| Bug | Stale pool executors passed post-close → spurious warn logs + failed DB calls | Medium |
| Perf | Sequential resolveSymbol per link — parallelizable with Promise.all |
Medium |
| Clean | allContracts alias is redundant |
Low |
| Test | Missing toHaveLength(1) on total crossLinks hides the duplicate bug |
Low |
The core fix (wiring ManifestExtractor) is correct and the placement decision is sound. Issue #1 is the most important to address before merging.
Also removes the stray empty line in the finally block (noted in review).
Addresses review feedback on PR #827: 1. Dedupe cross-links. Manifest contracts participate in runExactMatch, so a manifest-declared link also emitted a duplicate matchType:'exact' CrossLink for the same endpoint pair. Dedupe by (from, to, type, contractId) and prefer manifest (operator-declared intent). 2. Warn on dangling repos. When a manifest link references a repo not in config.repos, log a warning. Synthetic UIDs keep the cross-link deterministic, but the operator probably meant something else. 3. Tests: - Assert no duplicate 'exact' CrossLink is emitted alongside the manifest one. - Assert synthetic UID format when no DB executors are available. - New test: dangling manifest repo still produces a cross-link + logs a warning.
Previous implementation ran 2N sequential Cypher round-trips per manifest (one for provider side, one for consumer, awaited in-order per link). For manifests with tens of links this dominated syncGroup latency in groups with many declared cross-repo contracts. Changes: - Resolve provider + consumer in parallel per link (Promise.all). - Resolve all links in parallel (outer Promise.all over links.map). Each repo's executor pool is independent, so cross-repo fan-out scales with the number of distinct repos in the manifest. - Memoize by (repo, type, contract). Manifests frequently declare the same contract from both directions or across sibling groups, so duplicate triples now hit the DB once instead of 2× per link. Correctness: - resolveSymbol is a pure LIMIT 1 read, so caching + concurrent invocation is safe. - Iteration order over links is preserved in the final contracts / crossLinks arrays — result shape is identical. Test: - New test asserts that two links sharing (repo, type, contract) produce exactly one DB call per distinct repo-tuple.
|
@claude are we ready for production? review other comments if we delivered them |
|
Claude finished @magyargergo's task in 1m 39s —— View job Production Readiness Check
Issue Tracker — Prior Review FeedbackThe branch includes 5 commits since the initial fix. Here's the status of every raised issue:
Remaining Concerns1. CI Typecheck Failure (Blocker) 2. Stale 3. VerdictNot quite ready to merge — one CI blocker remains. The critical duplicate cross-link bug (#1) is fixed, performance is improved, and test coverage is solid. The core fix is production-quality. But the typecheck CI failure must be resolved (or confirmed pre-existing on To get this to green:
|
|
@claude Before we decide whether this PR is ready for production, we should first define what “production-ready” means in this context. I think the review criteria should be something along these lines: correctness and functional completeness Once those criteria are explicit, it will be much easier to assess the PR in a consistent and meaningful way. |
|
Claude finished @magyargergo's task in 2m 11s —— View job Production-Readiness Assessment — PR #827
Criteria DefinitionBefore scoring: what "production-ready" means for this PR across six dimensions.
1. Correctness & Functional Completeness — ✅ ReadyCore bug is fixed. Duplicate cross-link bug (raised in prior review) is fixed. Dangling repos are handled. A warn loop at One known correctness gap — stale executor map (Medium). After Undocumented behavior —
2. Code Clarity & Clean Code —
|
| Dimension | Status | Notes |
|---|---|---|
| Correctness | ✅ | Core fix correct; stale executor warn noise is safe but present |
| Code clarity | allContracts alias unresolved; otherwise clear |
|
| Test coverage | ✅ | Happy path, edge cases, and regression tests all present |
| Performance | ✅ | Promise.all + memoization in place |
| Operational risk | ✅ | Fully graceful degradation; safe rollback |
| Maintainability | ✅ | Clean structure; one undocumented custom type null case |
This PR is production-ready with one recommended change before merging:
- Remove
allContractsalias (low effort, removes reader confusion — flagged 3× across reviews)
Two optional improvements that don't block the merge but would reduce production noise:
dbExecutors?.clear()beforeextractFromManifest(one line, eliminates spurious warn logs from closed pools)- Add a comment in
resolveSymbolexplaining thecustomtype null return (one line)
The CI ❌ Tests flag with 0 actual failures should be confirmed as pre-existing before merge. If it reflects a platform where Swift tests can't run (not introduced by this PR), that's acceptable to document and proceed. If it's new, it needs investigation.
|
@jonasvanderhaegen-xve Thank you for your contribution! |
… RC, group sync - Take upstream splitRelCsvByLabelPair + tests (abhigyanpatwari#818/abhigyanpatwari#832); preserve fork closeLbugForPath and import evictPoolsForDbPath from pool-adapter. - Fix nightly-refresh evictPools import path to ../core/lbug/pool-adapter.js. - Includes abhigyanpatwari#818 drain fix, abhigyanpatwari#823 embeddings PK, abhigyanpatwari#825 RC workflow, abhigyanpatwari#827 manifest sync.
… produced 0 cross-links (abhigyanpatwari#827) * fix(group/sync): wire ManifestExtractor into syncGroup pipeline ManifestExtractor was fully implemented in extractors/manifest-extractor.ts but never imported or called in sync.ts. As a result, any links declared in group.yaml were parsed and validated by config-parser.ts but silently dropped — config.links was always an empty dead-end as far as syncGroup was concerned. Changes: - Import ManifestExtractor in sync.ts - Call extractFromManifest(config.links, dbExecutors) inside the outer try block, after all repos are processed but before the finally closes the DB pools (symbol resolution via resolveSymbol requires open executors) - Collect the resulting contracts into autoContracts and the cross-links into a separate manifestCrossLinks array - Merge manifestCrossLinks into the final crossLinks alongside runExactMatch results Without this fix, users who declare explicit service dependencies in group.yaml links (the documented workaround for HTTP clients that use absolute URLs and are invisible to the auto-extractors) get 0 cross-links regardless of what they configure. * test(group/sync): cover manifest links producing cross-links Add a unit test that asserts config.links entries produce contract pairs and a manifest cross-link (matchType: 'manifest') via syncGroup. Also refactors the manifest extraction call to sit outside the else/try block so it runs regardless of extractorOverride arity — makes the code testable without mocked DB pools and ensures links work when callers supply a zero-arity override (e.g. in tests or programmatic usage). * style: prettier format sync.ts and sync.test.ts Also removes the stray empty line in the finally block (noted in review). * fix(group/sync): dedupe cross-links and warn on dangling manifest repos Addresses review feedback on PR abhigyanpatwari#827: 1. Dedupe cross-links. Manifest contracts participate in runExactMatch, so a manifest-declared link also emitted a duplicate matchType:'exact' CrossLink for the same endpoint pair. Dedupe by (from, to, type, contractId) and prefer manifest (operator-declared intent). 2. Warn on dangling repos. When a manifest link references a repo not in config.repos, log a warning. Synthetic UIDs keep the cross-link deterministic, but the operator probably meant something else. 3. Tests: - Assert no duplicate 'exact' CrossLink is emitted alongside the manifest one. - Assert synthetic UID format when no DB executors are available. - New test: dangling manifest repo still produces a cross-link + logs a warning. * perf(group/manifest): parallelize and memoize symbol resolution Previous implementation ran 2N sequential Cypher round-trips per manifest (one for provider side, one for consumer, awaited in-order per link). For manifests with tens of links this dominated syncGroup latency in groups with many declared cross-repo contracts. Changes: - Resolve provider + consumer in parallel per link (Promise.all). - Resolve all links in parallel (outer Promise.all over links.map). Each repo's executor pool is independent, so cross-repo fan-out scales with the number of distinct repos in the manifest. - Memoize by (repo, type, contract). Manifests frequently declare the same contract from both directions or across sibling groups, so duplicate triples now hit the DB once instead of 2× per link. Correctness: - resolveSymbol is a pure LIMIT 1 read, so caching + concurrent invocation is safe. - Iteration order over links is preserved in the final contracts / crossLinks arrays — result shape is identical. Test: - New test asserts that two links sharing (repo, type, contract) produce exactly one DB call per distinct repo-tuple. --------- Co-authored-by: jonasvanderhaegen-xve <> Co-authored-by: Gergo Magyar <gergomagyar@icloud.com>
What changed
Wire
ManifestExtractorintosyncGroupsolinksdeclared ingroup.yamlactually produce cross-links.Files changed:
gitnexus/src/core/group/sync.ts— import + callManifestExtractorgitnexus/test/unit/group/sync.test.ts— new test covering manifest cross-link generationWhy
ManifestExtractor(extractors/manifest-extractor.ts) is fully implemented: it convertsGroupManifestLink[]into provider/consumerStoredContractpairs and directCrossLinkentries withmatchType: 'manifest'. Howeversync.tsnever imported or called it.config.linkswas parsed and validated byconfig-parser.tsbut then silently dropped —syncGroupnever accessed it.This bug most visibly affects repos where HTTP clients use absolute URLs (e.g. PHP
file_get_contents('https://api.example.com/...')). The HTTP extractor only emits providers (it normalises call URLs starting with/), so these consumer calls are invisible to auto-detection. Thelinksblock ingroup.yamlis the documented workaround — but it did nothing.How to verify
Or run the new unit test:
Design note
The manifest extraction is placed after the DB
try/finallyblock rather than inside it.ManifestExtractor.resolveSymbolgracefully falls back to synthetic UIDs (manifest::<repo>::<contractId>) whendbExecutorsis undefined or a pool is closed — cross-links are always generated regardless. This also makes the code testable without mocked DB pools.Related
Closes #826
Patch gist: https://gist.github.com/jonasvanderhaegen-xve/e2bf066d842f676aa16400e10ae5114f
Checklist
npx vitest run test/unit/group/sync.test.ts— 10/10)gitnexus-sharedworkspace errors unrelated to this PR)