feat(sdk): add embed() and embedMany() support across providers and s… - #855
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
✅ Single Commit Policy - COMPLIANTStatus: Policy requirements met • 1 commit • Valid format • Ready for merge 📊 View validation details📝 Commit Details
✅ Validation Results
🤖 Automated validation by NeuroLink Single Commit Enforcement |
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis PR adds comprehensive embedding support by introducing Changes
Sequence Diagram(s)sequenceDiagram
actor Client
participant Server as Server<br/>(agentRoutes)
participant Provider as Provider<br/>(e.g., OpenAI)
participant AISDK as AI SDK
Client->>Server: POST /api/agent/embed<br/>(text, provider, model?)
Server->>Server: Validate EmbedRequest
Server->>Server: Resolve Provider instance
Server->>Provider: embed(text, modelName?)
Provider->>AISDK: Create embedding model<br/>& call embed
AISDK-->>Provider: embedding vector
Provider-->>Server: number[]
Server->>Server: Construct response<br/>(embedding, provider, model, dimension)
Server-->>Client: EmbedResponse (200)
sequenceDiagram
actor Client
participant Server as Server<br/>(agentRoutes)
participant Provider as Provider<br/>(e.g., OpenAI)
participant AISDK as AI SDK
Client->>Server: POST /api/agent/embed-many<br/>(texts[], provider?, model?)
Server->>Server: Validate EmbedManyRequest
Server->>Server: Resolve Provider instance
Server->>Provider: embedMany(texts[], modelName?)
Provider->>AISDK: Create embedding model<br/>& call embedMany
AISDK-->>Provider: embedding vectors[][]
Provider-->>Server: number[][]
Server->>Server: Construct response<br/>(embeddings, provider, model, count, dimension)
Server-->>Client: EmbedManyResponse (200)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
Documentation Validation Results🚀 Documentation validation passed!
📦 Build artifact uploaded successfully. Ready for deployment preview. Commit: |
e68f1ee to
c0425d8
Compare
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/advanced/api-reference.md`:
- Around line 25-80: The docs page mixes two API surfaces: the new /api/agent/*
endpoints (e.g., /api/agent/embed, /api/agent/embed-many) and older endpoints
(/api/generate, /api/stream, /api/status); fix by choosing one of two
approaches—either migrate the legacy endpoints to the /api/agent namespace
(update all examples, request/response fields, and references to use
/api/agent/generate, /api/agent/stream, /api/agent/status) or split the page
into clearly labeled sections "Current API (/api/agent/...)" and "Legacy API
(/api/...)" with a short deprecation note and consistent examples for each
section—ensure all endpoint paths and example bodies reference the chosen
namespace consistently.
In `@docs/sdk/api-reference.md`:
- Around line 487-495: The docs table adds GOOGLE_AI_EMBEDDING_MODEL and
VERTEX_EMBEDDING_MODEL but the Environment Configuration section doesn't list
them; update that main env section to document these two variables
(GOOGLE_AI_EMBEDDING_MODEL, VERTEX_EMBEDDING_MODEL), include their purpose as
embedding model overrides for Google AI Studio and Google Vertex respectively,
indicate default behavior when unset, and add them to any ENV key/value table or
example so readers can discover and use these overrides.
In `@src/lib/core/baseProvider.ts`:
- Around line 1096-1108: The embedMany method on BaseProvider lacks
timeout/abort propagation and throws a raw Error for unsupported batch
embedding; wrap the async work in the withTimeout (or withTimeoutAbortSignal)
utility so callers can enforce timeouts/abort signals and accept an optional
AbortSignal/timeout param, and replace the thrown Error with a typed error
created by ErrorFactory (e.g., ErrorFactory.create or the project-standard
factory) that indicates "Batch embedding not supported by providerName" and
includes providerName and supported provider list; update the embedMany
signature to accept and pass through timeout/abort controls and ensure the
logger call remains but the failure path uses the ErrorFactory-created error.
In `@src/lib/providers/amazonBedrock.ts`:
- Around line 2109-2130: The embedMany function currently fans out with
Promise.all causing unbounded concurrent Bedrock requests and duplicates the
default model literal; change embedMany to derive the model via
getDefaultEmbeddingModel() when modelName is absent, and replace
Promise.all(texts.map(...)) with a controlled concurrent mapper (e.g., use a
limited-concurrency iterator or p-map-style helper) that calls this.embed(text,
embeddingModelName) with a sensible concurrency limit to avoid request spikes
and timeouts; keep logging and return shape the same but ensure you await the
concurrency-limited mapping so embeddings are returned as number[][].
In `@src/lib/providers/openAI.ts`:
- Around line 741-742: embedMany currently hardcodes "text-embedding-3-small"
instead of using the configured default; update the function so that when
modelName is falsy it calls getDefaultEmbeddingModel() (same behavior as other
embedding helpers) and uses that result as embeddingModelName, ensuring batch
embeddings honor OPENAI_EMBEDDING_MODEL and produce the expected vector
dimension; modify embedMany to derive embeddingModelName = modelName ||
getDefaultEmbeddingModel().
In `@src/lib/server/routes/agentRoutes.ts`:
- Around line 230-240: Wrap the async calls that create and use the provider
with the withTimeout utility: when obtaining providerName and calling
ProviderFactory.createProvider(providerName, request.model) wrap that Promise
with withTimeout and likewise wrap provider.embedMany(request.texts,
request.model) with withTimeout so both operations enforce the same timeout
semantics; ensure you import/use the same timeout duration or error handling
pattern used by the /embed endpoint and propagate or handle timeout errors
consistently within the agentRoutes handler.
- Around line 181-188: Wrap the async calls to ProviderFactory.createProvider
and provider.embed with the withTimeout utility to enforce consistent timeouts
(use the same timeout value used elsewhere or pass a configurable timeout).
Update the code around ProviderFactory.createProvider(...) and embedding via
provider.embed(...) so both calls are invoked through withTimeout(...) and
properly await or catch timeout errors. Also address the default provider
mismatch by either removing the literal default "openai" so
ProviderFactory.createProvider decides the default, or add a comment documenting
the intentional override of the factory default; reference
ProviderFactory.createProvider, provider.embed, and withTimeout when making
these changes.
In `@src/lib/server/utils/validation.ts`:
- Around line 127-140: The current EmbedRequestSchema and EmbedManyRequestSchema
use z.string().min(1) which allows whitespace-only strings; update both schemas
to trim input before applying the length check so pure-whitespace texts are
rejected: for EmbedRequestSchema change the text field to trim the string (e.g.,
via z.string().transform(s => s.trim()) or use refine with s.trim()) and then
apply .min(1, "Text is required"); for EmbedManyRequestSchema update the array
item schema (the z.array element) to perform the same trim+min(1, ...)
transformation so each entry is validated after trimming while keeping the
array-level .min(1, "At least one text is required") and .max(2048, ...) intact.
In `@test/unit/server/routes/agentRoutes.test.ts`:
- Around line 89-90: Add unit tests exercising the two new embedding routes
(/agent/embed and /agent/embed-many) rather than only updating the route count:
add mocked happy-path tests that POST to "/agent/embed" and "/agent/embed-many"
asserting the handler response shape and that the default provider selection
logic runs, and add validation-error tests that send invalid payloads and assert
the handlers return the expected validation errors. In both tests mock external
API calls used by the embed and embed-many handlers (e.g., the embedding
provider client) so they return deterministic embeddings, and place assertions
on the handler functions (embed handler names or the route paths) to confirm
request validation, provider defaulting, and response shaping behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4ce55677-2f42-473e-b8cc-c8d3dae64994
📒 Files selected for processing (15)
CLAUDE.mddocs/advanced/api-reference.mddocs/api/type-aliases/AIProvider.mddocs/guides/server-adapters/index.mddocs/sdk/api-reference.mdsrc/lib/core/baseProvider.tssrc/lib/providers/amazonBedrock.tssrc/lib/providers/googleAiStudio.tssrc/lib/providers/googleVertex.tssrc/lib/providers/openAI.tssrc/lib/server/routes/agentRoutes.tssrc/lib/server/types.tssrc/lib/server/utils/validation.tssrc/lib/types/providers.tstest/unit/server/routes/agentRoutes.test.ts
| ## Embeddings | ||
|
|
||
| ### Generate Embedding | ||
|
|
||
| ```http | ||
| POST /api/agent/embed | ||
| ``` | ||
|
|
||
| **Request body:** | ||
|
|
||
| ```json | ||
| { | ||
| "text": "Hello world", | ||
| "provider": "googleAiStudio", | ||
| "model": "gemini-embedding-001" | ||
| } | ||
| ``` | ||
|
|
||
| **Response:** | ||
|
|
||
| ```json | ||
| { | ||
| "embedding": [0.123, -0.456, ...], | ||
| "provider": "googleAiStudio", | ||
| "model": "gemini-embedding-001", | ||
| "dimension": 768 | ||
| } | ||
| ``` | ||
|
|
||
| ### Generate Batch Embeddings | ||
|
|
||
| ```http | ||
| POST /api/agent/embed-many | ||
| ``` | ||
|
|
||
| **Request body:** | ||
|
|
||
| ```json | ||
| { | ||
| "texts": ["First document", "Second document", "Third document"], | ||
| "provider": "openai", | ||
| "model": "text-embedding-3-small" | ||
| } | ||
| ``` | ||
|
|
||
| **Response:** | ||
|
|
||
| ```json | ||
| { | ||
| "embeddings": [[0.123, -0.456, ...], [0.789, -0.012, ...], [0.345, -0.678, ...]], | ||
| "provider": "openai", | ||
| "model": "text-embedding-3-small", | ||
| "count": 3, | ||
| "dimension": 1536 | ||
| } | ||
| ``` |
There was a problem hiding this comment.
This page now mixes two different API surfaces.
The new section documents /api/agent/* endpoints, but the same page still lists /api/generate, /api/stream, and /api/status above. That makes the reference internally inconsistent and likely sends readers to routes that do not exist on the current server API. Please either migrate the older sections to the same namespace or split legacy/current APIs explicitly.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/advanced/api-reference.md` around lines 25 - 80, The docs page mixes two
API surfaces: the new /api/agent/* endpoints (e.g., /api/agent/embed,
/api/agent/embed-many) and older endpoints (/api/generate, /api/stream,
/api/status); fix by choosing one of two approaches—either migrate the legacy
endpoints to the /api/agent namespace (update all examples, request/response
fields, and references to use /api/agent/generate, /api/agent/stream,
/api/agent/status) or split the page into clearly labeled sections "Current API
(/api/agent/...)" and "Legacy API (/api/...)" with a short deprecation note and
consistent examples for each section—ensure all endpoint paths and example
bodies reference the chosen namespace consistently.
| **Supported providers and default models:** | ||
|
|
||
| | Provider | Default Embedding Model | Env Override | | ||
| | ---------------- | ------------------------------ | --------------------------- | | ||
| | OpenAI | `text-embedding-3-small` | — | | ||
| | Google AI Studio | `gemini-embedding-001` | `GOOGLE_AI_EMBEDDING_MODEL` | | ||
| | Google Vertex | `text-embedding-004` | `VERTEX_EMBEDDING_MODEL` | | ||
| | Amazon Bedrock | `amazon.titan-embed-text-v2:0` | — | | ||
|
|
There was a problem hiding this comment.
Document these embedding override vars in the main env section.
This table introduces GOOGLE_AI_EMBEDDING_MODEL and VERTEX_EMBEDDING_MODEL, but the later Environment Configuration section in this same file does not list them. Users looking there for the supported env vars will miss these overrides.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/sdk/api-reference.md` around lines 487 - 495, The docs table adds
GOOGLE_AI_EMBEDDING_MODEL and VERTEX_EMBEDDING_MODEL but the Environment
Configuration section doesn't list them; update that main env section to
document these two variables (GOOGLE_AI_EMBEDDING_MODEL,
VERTEX_EMBEDDING_MODEL), include their purpose as embedding model overrides for
Google AI Studio and Google Vertex respectively, indicate default behavior when
unset, and add them to any ENV key/value table or example so readers can
discover and use these overrides.
| async embedMany(texts: string[], _modelName?: string): Promise<number[][]> { | ||
| logger.warn( | ||
| `embedMany() called on ${this.providerName} which does not have a native implementation`, | ||
| { | ||
| count: texts.length, | ||
| }, | ||
| ); | ||
| throw new Error( | ||
| `Batch embedding generation is not supported by the ${this.providerName} provider. ` + | ||
| `Supported providers: openai, googleAiStudio, vertex/google, bedrock. ` + | ||
| `Use an embedding model like text-embedding-3-small (OpenAI), gemini-embedding-001 (Google AI), ` + | ||
| `text-embedding-004 (Vertex), or amazon.titan-embed-text-v2:0 (Bedrock).`, | ||
| ); |
There was a problem hiding this comment.
Harden embedMany() before exposing it via HTTP.
The new signature has no way to propagate timeout/abort control, and the fallback throws a raw Error when batching is unsupported. That leaves /api/agent/embed-many prone to hung requests on provider stalls and generic 500s for capability misses; please carry timeout/abort through this API and use ErrorFactory for the unsupported-provider path.
As per coding guidelines, "All async operations should be wrapped with withTimeout utility for consistent timeout handling" and "Use ErrorFactory for creating typed errors instead of throwing raw Error objects"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/core/baseProvider.ts` around lines 1096 - 1108, The embedMany method
on BaseProvider lacks timeout/abort propagation and throws a raw Error for
unsupported batch embedding; wrap the async work in the withTimeout (or
withTimeoutAbortSignal) utility so callers can enforce timeouts/abort signals
and accept an optional AbortSignal/timeout param, and replace the thrown Error
with a typed error created by ErrorFactory (e.g., ErrorFactory.create or the
project-standard factory) that indicates "Batch embedding not supported by
providerName" and includes providerName and supported provider list; update the
embedMany signature to accept and pass through timeout/abort controls and ensure
the logger call remains but the failure path uses the ErrorFactory-created
error.
| async embedMany(texts: string[], modelName?: string): Promise<number[][]> { | ||
| const embeddingModelName = modelName || "amazon.titan-embed-text-v2:0"; | ||
|
|
||
| logger.debug("Generating batch embeddings", { | ||
| provider: this.providerName, | ||
| model: embeddingModelName, | ||
| count: texts.length, | ||
| }); | ||
|
|
||
| try { | ||
| const embeddings = await Promise.all( | ||
| texts.map((text) => this.embed(text, embeddingModelName)), | ||
| ); | ||
|
|
||
| logger.debug("Batch embeddings generated successfully", { | ||
| provider: this.providerName, | ||
| model: embeddingModelName, | ||
| count: embeddings.length, | ||
| embeddingDimension: embeddings[0]?.length, | ||
| }); | ||
|
|
||
| return embeddings; |
There was a problem hiding this comment.
Bound batch fan-out before calling Bedrock.
Promise.all sends one Bedrock request per text with no concurrency cap. Large embed-many payloads will spike outbound calls, which makes throttling and timeout storms much more likely here. Also, this duplicates the default model literal instead of reusing getDefaultEmbeddingModel().
♻️ Suggested change
async embedMany(texts: string[], modelName?: string): Promise<number[][]> {
- const embeddingModelName = modelName || "amazon.titan-embed-text-v2:0";
+ const embeddingModelName = modelName || this.getDefaultEmbeddingModel();
@@
- const embeddings = await Promise.all(
- texts.map((text) => this.embed(text, embeddingModelName)),
- );
+ const embeddings: number[][] = [];
+ const batchSize = 5;
+
+ for (let i = 0; i < texts.length; i += batchSize) {
+ const batch = texts.slice(i, i + batchSize);
+ embeddings.push(
+ ...(await Promise.all(
+ batch.map((text) => this.embed(text, embeddingModelName)),
+ )),
+ );
+ }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/providers/amazonBedrock.ts` around lines 2109 - 2130, The embedMany
function currently fans out with Promise.all causing unbounded concurrent
Bedrock requests and duplicates the default model literal; change embedMany to
derive the model via getDefaultEmbeddingModel() when modelName is absent, and
replace Promise.all(texts.map(...)) with a controlled concurrent mapper (e.g.,
use a limited-concurrency iterator or p-map-style helper) that calls
this.embed(text, embeddingModelName) with a sensible concurrency limit to avoid
request spikes and timeouts; keep logging and return shape the same but ensure
you await the concurrency-limited mapping so embeddings are returned as
number[][].
| async embedMany(texts: string[], modelName?: string): Promise<number[][]> { | ||
| const embeddingModelName = modelName || "text-embedding-3-small"; |
There was a problem hiding this comment.
Honor the configured default embedding model in batch mode.
This new path hardcodes "text-embedding-3-small" instead of using getDefaultEmbeddingModel(). If a deployment sets OPENAI_EMBEDDING_MODEL to a different model, embedMany() will still emit 3-small vectors here, which can break downstream stores that expect the configured embedding dimension.
Suggested fix
- const embeddingModelName = modelName || "text-embedding-3-small";
+ const embeddingModelName =
+ modelName || this.getDefaultEmbeddingModel() || "text-embedding-3-small";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| async embedMany(texts: string[], modelName?: string): Promise<number[][]> { | |
| const embeddingModelName = modelName || "text-embedding-3-small"; | |
| async embedMany(texts: string[], modelName?: string): Promise<number[][]> { | |
| const embeddingModelName = | |
| modelName || this.getDefaultEmbeddingModel() || "text-embedding-3-small"; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/providers/openAI.ts` around lines 741 - 742, embedMany currently
hardcodes "text-embedding-3-small" instead of using the configured default;
update the function so that when modelName is falsy it calls
getDefaultEmbeddingModel() (same behavior as other embedding helpers) and uses
that result as embeddingModelName, ensuring batch embeddings honor
OPENAI_EMBEDDING_MODEL and produce the expected vector dimension; modify
embedMany to derive embeddingModelName = modelName ||
getDefaultEmbeddingModel().
| try { | ||
| const providerName = request.provider || "openai"; | ||
| const provider = await ProviderFactory.createProvider( | ||
| providerName, | ||
| request.model, | ||
| ); | ||
|
|
||
| const embedding = await provider.embed(request.text, request.model); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check how withTimeout is used elsewhere in the codebase for pattern consistency
rg -n "withTimeout" --type=ts -C2Repository: juspay/neurolink
Length of output: 27635
🏁 Script executed:
#!/bin/bash
# First, find agentRoutes.ts
fd agentRoutes.ts --type f
# Get the file size to determine how to read it
wc -l $(fd agentRoutes.ts --type f) 2>/dev/null || echo "File not found"Repository: juspay/neurolink
Length of output: 137
🏁 Script executed:
#!/bin/bash
# Check the actual code around lines 181-188 in agentRoutes.ts
# Read the relevant section with context
sed -n '160,210p' $(fd agentRoutes.ts --type f)Repository: juspay/neurolink
Length of output: 1567
🏁 Script executed:
#!/bin/bash
# Check ProviderFactory implementation for default provider behavior
fd ProviderFactory -t f | head -5
rg -n "createProvider" --type=ts -A 3 | grep -A 3 "default\|vertex\|openai" | head -30Repository: juspay/neurolink
Length of output: 650
🏁 Script executed:
#!/bin/bash
# Check how other async operations are handled in agentRoutes.ts
rg -n "await" $(fd agentRoutes.ts --type f) -C 2 | head -50Repository: juspay/neurolink
Length of output: 1457
🏁 Script executed:
#!/bin/bash
# Check ProviderFactory implementation and its default provider
fd -t f "ProviderFactory" | head -5
# Look for ProviderFactory class definition
rg -n "class ProviderFactory\|export.*ProviderFactory" --type=ts -A 10 | head -40Repository: juspay/neurolink
Length of output: 97
🏁 Script executed:
#!/bin/bash
# Check the createProvider method to see defaults
rg -n "createProvider" --type=ts -A 8 | grep -B 2 -A 8 "vertex\|openai" | head -50Repository: juspay/neurolink
Length of output: 1333
🏁 Script executed:
#!/bin/bash
# Check if other routes in agentRoutes.ts wrap async calls with withTimeout
grep -n "withTimeout" src/lib/server/routes/agentRoutes.ts
# Also check if they import timeout utilities
head -30 src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 762
🏁 Script executed:
#!/binbash
# Check ctx.neurolink and config structure
rg -n "ctx\.neurolink\.config\|neurolink\.config\|config\?" src/lib/server/routes/agentRoutes.ts -A 2 -B 2Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Find and inspect ProviderFactory implementation
find . -name "*providerFactory*" -type f | head -10Repository: juspay/neurolink
Length of output: 98
🏁 Script executed:
#!/bin/bash
# Look for ProviderFactory.createProvider implementation
rg -n "export.*class ProviderFactory\|createProvider.*{" --type=ts -A 15 src/lib/factories/ | head -60Repository: juspay/neurolink
Length of output: 219
🏁 Script executed:
#!/bin/bash
# Check what timeout utilities are available and how they're imported in routes
rg -n "import.*withTimeout\|import.*timeout" src/lib/server/routes/ --type=tsRepository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check other async routes to see if they wrap with withTimeout
sed -n '1,80p' src/lib/server/routes/agentRoutes.ts
sed -n '60,130p' src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 4638
🏁 Script executed:
#!/bin/bash
# Inspect ProviderFactory implementation
sed -n '1,100p' src/lib/factories/providerFactory.tsRepository: juspay/neurolink
Length of output: 3132
🏁 Script executed:
#!/bin/bash
# Look for createProvider method in ProviderFactory
rg -n "createProvider" src/lib/factories/providerFactory.ts -A 10 | head -50Repository: juspay/neurolink
Length of output: 898
🏁 Script executed:
#!/bin/bash
# Check if there's a default provider defined in ProviderFactory
rg -n "vertex\|openai\|default" src/lib/factories/providerFactory.ts | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check ServerContext type and config structure
rg -n "type ServerContext\|interface ServerContext" --type=ts -A 15 src/lib/server/types.ts | head -40Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check ServerContext type definition
rg -n "type ServerContext\|interface ServerContext" src/lib/server/types.ts -A 20 | head -50Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check if ctx.neurolink has config property
rg -n "neurolink.*config\|config.*timeout" src/lib/server/ --type=ts | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Verify that agentRoutes.ts doesn't use withTimeout anywhere
rg "withTimeout" src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check other similar route files to see the pattern
head -40 src/lib/server/routes/*.ts | grep -E "import.*withTimeout|withTimeout"Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check how other route handlers handle async operations and timeouts
cat src/lib/server/routes/agentRoutes.ts | grep -A 30 "path: \`\${basePath\}/agent/execute\`"Repository: juspay/neurolink
Length of output: 1015
🏁 Script executed:
#!/bin/bash
# Look for timeout configuration or defaults in server context or similar handlers
rg -n "timeout\|TIMEOUT" src/lib/server/ --type=ts | grep -i "config\|default" | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check other handlers to see if they use ctx.neurolink or similar patterns
rg -n "ctx.neurolink" src/lib/server/routes/ --type=ts -B 2 -A 2 | head -50Repository: juspay/neurolink
Length of output: 3001
🏁 Script executed:
#!/bin/bash
# Verify the full embed handler to understand the context better
sed -n '163,209p' src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 1460
🏁 Script executed:
#!/bin/bash
# Check for any embed-specific timeout constants or patterns
rg -n "embed.*timeout\|TIMEOUT.*EMBED\|embedding.*timeout" src/ --type=ts -i | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check what timeout values are commonly used for provider operations
rg -n "30000|30_000|60000|120000" src/lib/providers/ --type=ts | head -10Repository: juspay/neurolink
Length of output: 627
🏁 Script executed:
#!/bin/bash
# Check if there's a timeout constant or config in the server context
rg -n "type ServerContext\|interface ServerContext" src/lib/server/types.ts -A 50 | head -70Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look for ServerContext definition more thoroughly
find src/lib/server -name "*.ts" -exec grep -l "ServerContext" {} \;Repository: juspay/neurolink
Length of output: 769
🏁 Script executed:
#!/bin/bash
# Check the server context/types definition
cat src/lib/server/types.ts | head -100Repository: juspay/neurolink
Length of output: 2526
🏁 Script executed:
#!/bin/bash
# Look for how timeout is accessed in similar patterns in other handlers
rg -n "ctx\." src/lib/server/routes/ --type=ts -h | sort | uniq | head -30Repository: juspay/neurolink
Length of output: 2204
🏁 Script executed:
#!/bin/bash
# Check how NeuroLink instance is created and if it has config property
rg -n "ctx.neurolink\s*=" src/lib/server/ --type=ts -B 2 -A 2 | head -40Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check NeuroLink class definition for config property
rg -n "class NeuroLink\|config.*:" src/lib/neurolink.ts | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check if other route handlers show how to get timeout from context
rg -n "timeout" src/lib/server/routes/ --type=ts | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look at the full ServerContext type definition
sed -n '200,350p' src/lib/server/types.tsRepository: juspay/neurolink
Length of output: 3519
🏁 Script executed:
#!/bin/bash
# Check NeuroLink class to see what properties it exposes
sed -n '1,150p' src/lib/neurolink.tsRepository: juspay/neurolink
Length of output: 4692
🏁 Script executed:
#!/bin/bash
# Search for "config" in NeuroLink class definition
rg -n "this.config\|config\s*=" src/lib/neurolink.ts | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check if config is a public property or method
rg -n "public config\|private config\|config:" src/lib/neurolink.ts | head -10Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look at the complete embed handler including error handling
sed -n '163,210p' src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 1468
🏁 Script executed:
#!/bin/bash
# Search for public properties of NeuroLink class
rg -n "^\s*(public|private|protected).*config\|get config" src/lib/neurolink.tsRepository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look for all public properties of NeuroLink
rg -n "^\s*(public|readonly)" src/lib/neurolink.ts | head -30Repository: juspay/neurolink
Length of output: 93
🏁 Script executed:
#!/bin/bash
# Check if there's a way to pass timeout to embed operations
rg -n "embed.*timeout\|timeout.*embed" src/ --type=ts -B 2 -A 2 | head -40Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check embed method signature in BaseProvider
rg -n "async embed" src/lib/core/baseProvider.ts -A 5 | head -20Repository: juspay/neurolink
Length of output: 589
🏁 Script executed:
#!/bin/bash
# Check if ServerContext has any timeout-related property
rg -n "ServerContext\|ctx\." src/lib/server/types.ts -A 2 | grep -i "timeout\|config" | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check how the server adapter passes config to routes or context
rg -n "ServerContext.*config\|createServerContext\|new ServerContext" src/lib/server/ --type=ts -A 5 | head -40Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look for any existing timeout handling in agent routes or similar handlers
rg -n "timeout\|withTimeout\|TIMEOUT" src/lib/server/routes/ --type=tsRepository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check the entire embedMany handler to see if it also lacks timeout
sed -n '208,250p' src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 1292
🏁 Script executed:
#!/bin/bash
# Check if there's a standard timeout value used in providers or handlers
rg -n "30000\|30_000" src/lib/ --type=ts | grep -i "embed\|const.*timeout" | head -20Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Look for how other handlers get timeout values
rg -n "timeout" src/lib/server/abstract/ --type=ts -B 2 -A 2 | head -40Repository: juspay/neurolink
Length of output: 3051
🏁 Script executed:
#!/bin/bash
# Check if NeuroLink has any internal configuration
rg -n "constructor.*config\|this.config" src/lib/neurolink.ts | head -10Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Verify the exact lines of the embed handler one more time with full context
sed -n '160,195p' src/lib/server/routes/agentRoutes.tsRepository: juspay/neurolink
Length of output: 1146
Wrap async provider calls with withTimeout for consistent timeout handling.
Per coding guidelines, all async operations should be wrapped with the withTimeout utility. Both ProviderFactory.createProvider and provider.embed are async calls that could hang indefinitely without timeout protection.
Additionally, the default provider "openai" (line 182) differs from ProviderFactory.createProvider's internal default of "vertex". Consider either:
- Omitting the default here and letting the factory decide, or
- Documenting this intentional override in the API docs
Proposed fix with timeout wrapper
+import { withTimeout } from "../../utils/errorHandling.js";
+
// Inside handler:
try {
const providerName = request.provider || "openai";
- const provider = await ProviderFactory.createProvider(
- providerName,
- request.model,
+ const provider = await withTimeout(
+ ProviderFactory.createProvider(providerName, request.model),
+ 30000,
+ `Provider creation timed out`,
);
- const embedding = await provider.embed(request.text, request.model);
+ const embedding = await withTimeout(
+ provider.embed(request.text, request.model),
+ 30000,
+ `Embedding generation timed out`,
+ );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/server/routes/agentRoutes.ts` around lines 181 - 188, Wrap the async
calls to ProviderFactory.createProvider and provider.embed with the withTimeout
utility to enforce consistent timeouts (use the same timeout value used
elsewhere or pass a configurable timeout). Update the code around
ProviderFactory.createProvider(...) and embedding via provider.embed(...) so
both calls are invoked through withTimeout(...) and properly await or catch
timeout errors. Also address the default provider mismatch by either removing
the literal default "openai" so ProviderFactory.createProvider decides the
default, or add a comment documenting the intentional override of the factory
default; reference ProviderFactory.createProvider, provider.embed, and
withTimeout when making these changes.
| try { | ||
| const providerName = request.provider || "openai"; | ||
| const provider = await ProviderFactory.createProvider( | ||
| providerName, | ||
| request.model, | ||
| ); | ||
|
|
||
| const embeddings = await provider.embedMany( | ||
| request.texts, | ||
| request.model, | ||
| ); |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Same timeout wrapper needed here.
Similar to the /embed endpoint, both ProviderFactory.createProvider and provider.embedMany calls should be wrapped with withTimeout for consistent timeout handling across the codebase. As per coding guidelines: "All async operations should be wrapped with withTimeout utility."
Proposed fix with timeout wrapper
try {
const providerName = request.provider || "openai";
- const provider = await ProviderFactory.createProvider(
- providerName,
- request.model,
+ const provider = await withTimeout(
+ ProviderFactory.createProvider(providerName, request.model),
+ ctx.neurolink.config?.timeout ?? 30000,
+ `Provider creation timed out`,
);
- const embeddings = await provider.embedMany(
- request.texts,
- request.model,
+ const embeddings = await withTimeout(
+ provider.embedMany(request.texts, request.model),
+ ctx.neurolink.config?.timeout ?? 30000,
+ `Batch embedding generation timed out`,
);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| try { | |
| const providerName = request.provider || "openai"; | |
| const provider = await ProviderFactory.createProvider( | |
| providerName, | |
| request.model, | |
| ); | |
| const embeddings = await provider.embedMany( | |
| request.texts, | |
| request.model, | |
| ); | |
| try { | |
| const providerName = request.provider || "openai"; | |
| const provider = await withTimeout( | |
| ProviderFactory.createProvider(providerName, request.model), | |
| ctx.neurolink.config?.timeout ?? 30000, | |
| `Provider creation timed out`, | |
| ); | |
| const embeddings = await withTimeout( | |
| provider.embedMany(request.texts, request.model), | |
| ctx.neurolink.config?.timeout ?? 30000, | |
| `Batch embedding generation timed out`, | |
| ); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/server/routes/agentRoutes.ts` around lines 230 - 240, Wrap the async
calls that create and use the provider with the withTimeout utility: when
obtaining providerName and calling ProviderFactory.createProvider(providerName,
request.model) wrap that Promise with withTimeout and likewise wrap
provider.embedMany(request.texts, request.model) with withTimeout so both
operations enforce the same timeout semantics; ensure you import/use the same
timeout duration or error handling pattern used by the /embed endpoint and
propagate or handle timeout errors consistently within the agentRoutes handler.
| export const EmbedRequestSchema = z.object({ | ||
| text: z.string().min(1, "Text is required"), | ||
| provider: z.string().optional(), | ||
| model: z.string().optional(), | ||
| }); | ||
|
|
||
| /** | ||
| * Embed many request schema (batch texts) | ||
| */ | ||
| export const EmbedManyRequestSchema = z.object({ | ||
| texts: z | ||
| .array(z.string().min(1)) | ||
| .min(1, "At least one text is required") | ||
| .max(2048, "Maximum 2048 texts per batch"), |
There was a problem hiding this comment.
Reject whitespace-only embedding payloads.
z.string().min(1) still accepts " ", so both embedding endpoints can pass blank content through to the provider. Trim before the length check so empty/whitespace-only texts fail fast at validation time.
Suggested fix
export const EmbedRequestSchema = z.object({
- text: z.string().min(1, "Text is required"),
+ text: z.string().trim().min(1, "Text is required"),
provider: z.string().optional(),
model: z.string().optional(),
});
export const EmbedManyRequestSchema = z.object({
texts: z
- .array(z.string().min(1))
+ .array(z.string().trim().min(1, "Text is required"))
.min(1, "At least one text is required")
.max(2048, "Maximum 2048 texts per batch"),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/lib/server/utils/validation.ts` around lines 127 - 140, The current
EmbedRequestSchema and EmbedManyRequestSchema use z.string().min(1) which allows
whitespace-only strings; update both schemas to trim input before applying the
length check so pure-whitespace texts are rejected: for EmbedRequestSchema
change the text field to trim the string (e.g., via z.string().transform(s =>
s.trim()) or use refine with s.trim()) and then apply .min(1, "Text is
required"); for EmbedManyRequestSchema update the array item schema (the z.array
element) to perform the same trim+min(1, ...) transformation so each entry is
validated after trimming while keeping the array-level .min(1, "At least one
text is required") and .max(2048, ...) intact.
| it("should create five routes", () => { | ||
| expect(routes.routes.length).toBe(5); |
There was a problem hiding this comment.
Add handler coverage for the two new embedding routes.
Updating the route count to 5 alone will not catch regressions in POST /agent/embed and POST /agent/embed-many. The new handlers add request validation, default provider selection, and response shaping, but this file still only exercises execute, stream, and providers. Please add mocked happy-path and validation-error tests for both embedding endpoints.
As per coding guidelines, test/**/*.test.ts: Unit tests must mock external API calls; integration tests may use real API calls sparingly.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/unit/server/routes/agentRoutes.test.ts` around lines 89 - 90, Add unit
tests exercising the two new embedding routes (/agent/embed and
/agent/embed-many) rather than only updating the route count: add mocked
happy-path tests that POST to "/agent/embed" and "/agent/embed-many" asserting
the handler response shape and that the default provider selection logic runs,
and add validation-error tests that send invalid payloads and assert the
handlers return the expected validation errors. In both tests mock external API
calls used by the embed and embed-many handlers (e.g., the embedding provider
client) so they return deterministic embeddings, and place assertions on the
handler functions (embed handler names or the route paths) to confirm request
validation, provider defaulting, and response shaping behavior.
c0425d8 to
228d57c
Compare
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
…erver - Add embed() and embedMany() to Google AI Studio provider with gemini-embedding-001 default - Add embedMany() to Google Vertex, OpenAI, and Amazon Bedrock providers - Add embedMany() base stub in BaseProvider with descriptive error for unsupported providers - Add embed and embedMany to AIProvider type interface - Add POST /api/agent/embed and POST /api/agent/embed-many server routes - Add EmbedRequest, EmbedResponse, EmbedManyRequest, EmbedManyResponse server types - Add EmbedRequestSchema and EmbedManyRequestSchema Zod validation schemas - Replace dynamic imports of embed/embedMany with static imports from "ai" in all providers - Update SDK API reference, server adapters guide, advanced API reference, and AIProvider docs - Update agent routes test to account for new embedding endpoints
228d57c to
ecaf73e
Compare
🤖 AI Review & Build Compliance ✅Status: AI analysis complete • Build rules validated • Ready for review 📊 View detailed analysis results🛡️ Analysis Complete
📋 Ready for Merge When
🤖 AI analysis complete - check individual code comments for specific feedback |
|
🎉 This PR is included in version 9.18.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This branch was successfully deployed
1 participant
…erver
Pull Request
Description
What does this PR do?
A clear and concise description of the changes in this pull request.
Related Issues
Does this PR close any issues?
Fixes #(issue number)
Closes #(issue number)
Relates to #(issue number)
Type of Change
Please select the type of change:
Motivation and Context
Why is this change needed? What problem does it solve?
Provide context for reviewers:
Changes Made
What specific changes were made?
Provide a bullet-point list of the key changes:
Breaking Changes
Does this PR introduce breaking changes?
If yes, describe:
Testing
How has this been tested?
Please describe the tests you ran and their results:
Test Coverage
Manual Testing Steps
Provide steps for manual testing:
Code Quality
Have you followed code quality standards?
Documentation
Have you updated documentation?
Commit Message Format
Does your commit follow semantic commit conventions?
type(scope): descriptionExample:
feat(providers): add support for LiteLLM proxyDependencies
Does this PR add, update, or remove dependencies?
If yes, list dependencies and justification:
Performance Impact
Does this change affect performance?
If applicable, provide benchmark results:
Security Considerations
Are there any security implications?
If applicable, describe:
Deployment Notes
Special deployment instructions?
Screenshots / Videos
If applicable, add screenshots or videos to demonstrate changes:
[Add screenshots or videos here]
Reviewer Checklist
For reviewers:
Additional Notes
Any additional information for reviewers:
[Add any extra context, concerns, or questions here]
Pre-submission Checklist
Before submitting, ensure you have:
pnpm testpnpm buildpnpm run validate:alland all checks passThank you for contributing to NeuroLink!
Summary by CodeRabbit
New Features
/api/agent/embedfor single embeddings and/api/agent/embed-manyfor batch processing.Documentation