chore(indexer): add Hyperindex migration foundation - #235
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughMigrates the ChangesHyperindex migration routing and tooling
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant IndexerRoute
participant MagicIndexer
participant Hyperindex
Client->>IndexerRoute: POST GraphQL request with operationName
IndexerRoute->>IndexerRoute: select backend via OPERATION_BACKENDS/HYPERINDEX_OPERATIONS
alt operation routed to Hyperindex
IndexerRoute->>Hyperindex: forward request
Hyperindex-->>IndexerRoute: response
else default routing
IndexerRoute->>MagicIndexer: forward request
MagicIndexer-->>IndexerRoute: response
end
IndexerRoute-->>Client: response with X-Indexer-Backend header
Related issues: None referenced in the provided changes. Related PRs: None referenced in the provided changes. Suggested labels: migration, indexer, documentation Suggested reviewers: None determinable from the provided changes. 🐇 A magic wand traded for hyperspeed, 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (5)
src/app/api/indexer/route.ts (1)
106-119: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAdd a parallel warning for misconfigured Hyperindex routing.
The production guard only checks that a Magic endpoint is configured; it doesn't warn if
HYPERINDEX_OPERATIONSis set in production withoutHYPERINDEX_URL, in which case requests silently fall back to the hardcodedDEFAULT_HYPERINDEX_URLwith no signal to operators.♻️ Suggested addition
if ( process.env.NODE_ENV === "production" && !process.env.MAGIC_INDEXER_URL && !process.env.INDEXER_URL && !process.env.NEXT_PUBLIC_INDEXER_URL ) { console.warn( "[indexer] no MAGIC_INDEXER_URL/INDEXER_URL set in production — using " + "the built-in fallback (magic-indexer-prod). Set MAGIC_INDEXER_URL " + "while unported operations still route to Magic.", ) } + +if ( + process.env.NODE_ENV === "production" && + HYPERINDEX_OPERATION_NAMES.size > 0 && + !process.env.HYPERINDEX_URL +) { + console.warn( + "[indexer] HYPERINDEX_OPERATIONS set in production without " + + "HYPERINDEX_URL — falling back to the built-in default endpoint.", + ) +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/app/api/indexer/route.ts` around lines 106 - 119, Add a production-time warning in the route module alongside the existing Magic endpoint guard, but for Hyperindex routing: if HYPERINDEX_OPERATIONS is enabled in production and HYPERINDEX_URL is missing, log a console.warn that the code will fall back to DEFAULT_HYPERINDEX_URL. Keep the check near the existing module-load warning in src/app/api/indexer/route.ts and reference the same routing constants/flags so operators get a clear signal when Hyperindex requests silently use the fallback.scripts/indexer-parity.mjs (2)
18-27: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueEndpoint resolution logic duplicated from
route.ts.The default URLs and env-var precedence (
MAGIC_INDEXER_URL→INDEXER_URL→NEXT_PUBLIC_INDEXER_URL) mirrorsrc/app/api/indexer/route.tsexactly. If the proxy's defaults or precedence change, this harness can silently drift and stop being a faithful parity check.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/indexer-parity.mjs` around lines 18 - 27, The endpoint resolution in indexer-parity.mjs is duplicated from the app proxy logic, so it can drift from the source of truth. Update the script to reuse the same URL resolution path as route.ts, ideally by extracting the default URLs and env-var precedence into a shared helper used by both scripts. Keep the magicUrl and hyperindexUrl setup in sync with the shared resolver so parity checks always reflect the proxy behavior.
49-68: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAdd a request timeout to avoid indefinite hangs.
fetch()calls here have noAbortSignal, so a slow/unresponsive Magic or Hyperindex endpoint will hangPromise.all(and the whole script) indefinitely with no useful output.🔧 Proposed fix
async function request(url) { const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ query, operationName: "CertifiedCountParity" }), + signal: AbortSignal.timeout(10_000), })Also applies to: 80-83
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/indexer-parity.mjs` around lines 49 - 68, Add a timeout to the request flow so slow Magic/Hyperindex endpoints cannot hang the script indefinitely. Update the request(url) helper to create and pass an AbortSignal to fetch(), and ensure the timeout is cleared after completion or failure. Keep the existing error handling in request() so any timeout surfaces as a useful failure for the Promise.all callers.docs/hyperindex-schema-capability-note-2026-07-07.md (1)
121-161: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider redacting or annotating the embedded live sample data.
The "Stage 1 smoke counts" snapshot embeds real base64 pagination cursors that decode to production
at://did:plc:.../collection/rkeyvalues. These are public atproto identifiers, not secrets, but baking live production samples into a docs file means this snapshot will drift from reality and could be mistaken for a stable contract. Consider either redacting the cursors or explicitly labeling this as a point-in-time, non-authoritative sample.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/hyperindex-schema-capability-note-2026-07-07.md` around lines 121 - 161, The Stage 1 smoke counts snapshot in the docs embeds live pagination cursors that look like stable production data, so update this sample to avoid being treated as a contract. In the markdown section containing the JSON snippet, either redact the cursor values or add explicit point-in-time / non-authoritative labeling, while keeping the surrounding “Stage 1 smoke counts” context intact. Refer to the embedded `activities`, `awards`, `orgs`, `profiles`, and `projects` sample blocks so the change is applied consistently across the whole snapshot.scripts/hyperindex-smoke.mjs (1)
28-47: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winSame missing-timeout risk as
indexer-parity.mjs.
fetch()here also has noAbortSignal; an unresponsive Hyperindex endpoint will hang this smoke test indefinitely instead of failing fast.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/hyperindex-smoke.mjs` around lines 28 - 47, The request helper in hyperindex-smoke.mjs has the same hanging risk as the parity script because fetch is called without any timeout or AbortSignal. Update request(query, variables) to create and pass an AbortSignal with a reasonable timeout to the existing fetch call, and make sure the timeout is cleared or the controller is handled cleanly after the request completes. Keep the behavior of the JSON parsing and GraphQL error handling in request unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docs/hyperindex-schema-capability-note-2026-07-07.md`:
- Around line 121-161: The Stage 1 smoke counts snapshot in the docs embeds live
pagination cursors that look like stable production data, so update this sample
to avoid being treated as a contract. In the markdown section containing the
JSON snippet, either redact the cursor values or add explicit point-in-time /
non-authoritative labeling, while keeping the surrounding “Stage 1 smoke counts”
context intact. Refer to the embedded `activities`, `awards`, `orgs`,
`profiles`, and `projects` sample blocks so the change is applied consistently
across the whole snapshot.
In `@scripts/hyperindex-smoke.mjs`:
- Around line 28-47: The request helper in hyperindex-smoke.mjs has the same
hanging risk as the parity script because fetch is called without any timeout or
AbortSignal. Update request(query, variables) to create and pass an AbortSignal
with a reasonable timeout to the existing fetch call, and make sure the timeout
is cleared or the controller is handled cleanly after the request completes.
Keep the behavior of the JSON parsing and GraphQL error handling in request
unchanged.
In `@scripts/indexer-parity.mjs`:
- Around line 18-27: The endpoint resolution in indexer-parity.mjs is duplicated
from the app proxy logic, so it can drift from the source of truth. Update the
script to reuse the same URL resolution path as route.ts, ideally by extracting
the default URLs and env-var precedence into a shared helper used by both
scripts. Keep the magicUrl and hyperindexUrl setup in sync with the shared
resolver so parity checks always reflect the proxy behavior.
- Around line 49-68: Add a timeout to the request flow so slow Magic/Hyperindex
endpoints cannot hang the script indefinitely. Update the request(url) helper to
create and pass an AbortSignal to fetch(), and ensure the timeout is cleared
after completion or failure. Keep the existing error handling in request() so
any timeout surfaces as a useful failure for the Promise.all callers.
In `@src/app/api/indexer/route.ts`:
- Around line 106-119: Add a production-time warning in the route module
alongside the existing Magic endpoint guard, but for Hyperindex routing: if
HYPERINDEX_OPERATIONS is enabled in production and HYPERINDEX_URL is missing,
log a console.warn that the code will fall back to DEFAULT_HYPERINDEX_URL. Keep
the check near the existing module-load warning in src/app/api/indexer/route.ts
and reference the same routing constants/flags so operators get a clear signal
when Hyperindex requests silently use the fallback.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 522fd255-fb47-4a7b-ab34-109f18e49d36
📒 Files selected for processing (9)
.env.local.exampledocs/hyperindex-schema-capability-note-2026-07-07.mddocs/magic-to-hyperindex-migration-plan.htmldocs/magic-to-hyperindex-switch-plan.mdpackage.jsonscripts/hyperindex-smoke.mjsscripts/indexer-parity.mjssrc/app/api/indexer/__tests__/route.test.tssrc/app/api/indexer/route.ts
1 participant
Summary
Stage 0 for the Magic Indexer → Hyperindex migration.
This PR does not migrate any production operation to Hyperindex yet. It keeps the app-facing
/api/indexercontract stable and preserves Magic as the default backend for every operation.Changes:
MAGIC_INDEXER_URLHYPERINDEX_URLHYPERINDEX_OPERATIONS/api/indexerso each operation can selectmagicorhyperindexinternally.X-Indexer-Backendresponse header.What did not change
Local checks run
npm run typechecknpm testnpm run lint— exits with existing warnings only; no errors.After removing the generic smoke/parity scripts per review feedback, I also reran:
npm test -- src/app/api/indexer/__tests__/route.test.tsnpm run typecheckManual test instructions
Run app with normal/default env.
Confirm existing Magic-backed surfaces still load:
/welcomestats/exploreIn dev/staging only, optionally set:
Then hit
/api/indexerthrough the app and confirm responses include:Leave
HYPERINDEX_OPERATIONSunset for normal behavior.Rollback
Revert this PR. Since no operation is flipped by default, rollback should not require data cleanup.