-
Notifications
You must be signed in to change notification settings - Fork 127
feat(sdk): add embed() and embedMany() support across providers and s… #855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -454,6 +454,47 @@ const result = await neurolink.gen({ | |
|
|
||
| --- | ||
|
|
||
| ### Embeddings | ||
|
|
||
| Generate embeddings directly via the provider's `embed()` and `embedMany()` methods. | ||
|
|
||
| #### `provider.embed(text, modelName?)` | ||
|
|
||
| Generate an embedding vector for a single text. | ||
|
|
||
| ```typescript | ||
| import { ProviderFactory } from "@juspay/neurolink"; | ||
|
|
||
| const provider = await ProviderFactory.createProvider("googleAiStudio"); | ||
| const embedding = await provider.embed("Hello world"); | ||
| // embedding: number[] (e.g., 768 dimensions) | ||
| ``` | ||
|
|
||
| #### `provider.embedMany(texts, modelName?)` | ||
|
|
||
| Generate embedding vectors for multiple texts in a single batch. The AI SDK automatically handles chunking for models with batch limits. | ||
|
|
||
| ```typescript | ||
| const provider = await ProviderFactory.createProvider("openai"); | ||
| const embeddings = await provider.embedMany([ | ||
| "First document", | ||
| "Second document", | ||
| "Third document", | ||
| ]); | ||
| // embeddings: number[][] (e.g., 3 × 1536 dimensions) | ||
| ``` | ||
|
|
||
| **Supported providers and default models:** | ||
|
|
||
| | Provider | Default Embedding Model | Env Override | | ||
| | ---------------- | ------------------------------ | --------------------------- | | ||
| | OpenAI | `text-embedding-3-small` | — | | ||
| | Google AI Studio | `gemini-embedding-001` | `GOOGLE_AI_EMBEDDING_MODEL` | | ||
| | Google Vertex | `text-embedding-004` | `VERTEX_EMBEDDING_MODEL` | | ||
| | Amazon Bedrock | `amazon.titan-embed-text-v2:0` | — | | ||
|
|
||
|
Comment on lines
+487
to
+495
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Document these embedding override vars in the main env section. This table introduces 🤖 Prompt for AI Agents |
||
| --- | ||
|
|
||
| ### RAG Integration | ||
|
|
||
| Pass `rag: { files: [...] }` to `generate()` or `stream()` for automatic RAG pipeline setup: | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -674,14 +674,14 @@ | |
| * IMPLEMENTATION NOTE: Uses streamText() under the hood and accumulates results | ||
| * for consistency and better performance | ||
| */ | ||
| async generate( | ||
|
Check warning on line 677 in src/lib/core/baseProvider.ts
|
||
| optionsOrPrompt: TextGenerationOptions | string, | ||
| _analysisSchema?: ValidationSchema, | ||
| ): Promise<EnhancedGenerateResult | null> { | ||
| return providerTracer.startActiveSpan( | ||
| "neurolink.provider.generate", | ||
| { kind: SpanKind.INTERNAL }, | ||
| async (span) => { | ||
|
Check warning on line 684 in src/lib/core/baseProvider.ts
|
||
| const options = this.normalizeTextOptions(optionsOrPrompt); | ||
| this.validateOptions(options); | ||
| const startTime = Date.now(); | ||
|
|
@@ -1081,6 +1081,33 @@ | |
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Generate embeddings for multiple texts in a single batch | ||
| * | ||
| * This is a default implementation that throws an error. | ||
| * Providers that support embeddings should override this method. | ||
| * The AI SDK's embedMany automatically handles chunking for models with batch limits. | ||
| * | ||
| * @param texts - The texts to embed | ||
| * @param _modelName - Optional embedding model name (provider-specific) | ||
| * @returns Promise resolving to an array of embedding vectors | ||
| * @throws Error if the provider does not support embeddings | ||
| */ | ||
| async embedMany(texts: string[], _modelName?: string): Promise<number[][]> { | ||
| logger.warn( | ||
| `embedMany() called on ${this.providerName} which does not have a native implementation`, | ||
| { | ||
| count: texts.length, | ||
| }, | ||
| ); | ||
| throw new Error( | ||
| `Batch embedding generation is not supported by the ${this.providerName} provider. ` + | ||
| `Supported providers: openai, googleAiStudio, vertex/google, bedrock. ` + | ||
| `Use an embedding model like text-embedding-3-small (OpenAI), gemini-embedding-001 (Google AI), ` + | ||
| `text-embedding-004 (Vertex), or amazon.titan-embed-text-v2:0 (Bedrock).`, | ||
| ); | ||
|
Comment on lines
+1096
to
+1108
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Harden The new signature has no way to propagate timeout/abort control, and the fallback throws a raw As per coding guidelines, "All async operations should be wrapped with withTimeout utility for consistent timeout handling" and "Use ErrorFactory for creating typed errors instead of throwing raw Error objects" 🤖 Prompt for AI Agents |
||
| } | ||
|
|
||
| /** | ||
| * Get the default embedding model for this provider | ||
| * | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2099,4 +2099,43 @@ export class AmazonBedrockProvider extends BaseProvider { | |
| throw this.handleProviderError(error); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Generate embeddings for multiple texts in a single batch | ||
| * @param texts - The texts to embed | ||
| * @param modelName - The embedding model to use (default: amazon.titan-embed-text-v2:0) | ||
| * @returns Promise resolving to an array of embedding vectors | ||
| */ | ||
| async embedMany(texts: string[], modelName?: string): Promise<number[][]> { | ||
| const embeddingModelName = modelName || "amazon.titan-embed-text-v2:0"; | ||
|
|
||
| logger.debug("Generating batch embeddings", { | ||
| provider: this.providerName, | ||
| model: embeddingModelName, | ||
| count: texts.length, | ||
| }); | ||
|
|
||
| try { | ||
| const embeddings = await Promise.all( | ||
| texts.map((text) => this.embed(text, embeddingModelName)), | ||
| ); | ||
|
|
||
| logger.debug("Batch embeddings generated successfully", { | ||
| provider: this.providerName, | ||
| model: embeddingModelName, | ||
| count: embeddings.length, | ||
| embeddingDimension: embeddings[0]?.length, | ||
| }); | ||
|
|
||
| return embeddings; | ||
|
Comment on lines
+2109
to
+2130
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bound batch fan-out before calling Bedrock.
♻️ Suggested change async embedMany(texts: string[], modelName?: string): Promise<number[][]> {
- const embeddingModelName = modelName || "amazon.titan-embed-text-v2:0";
+ const embeddingModelName = modelName || this.getDefaultEmbeddingModel();
@@
- const embeddings = await Promise.all(
- texts.map((text) => this.embed(text, embeddingModelName)),
- );
+ const embeddings: number[][] = [];
+ const batchSize = 5;
+
+ for (let i = 0; i < texts.length; i += batchSize) {
+ const batch = texts.slice(i, i + batchSize);
+ embeddings.push(
+ ...(await Promise.all(
+ batch.map((text) => this.embed(text, embeddingModelName)),
+ )),
+ );
+ }🤖 Prompt for AI Agents |
||
| } catch (error) { | ||
| logger.error("Batch embedding generation failed", { | ||
| error: error instanceof Error ? error.message : String(error), | ||
| model: embeddingModelName, | ||
| count: texts.length, | ||
| }); | ||
|
|
||
| throw this.handleProviderError(error); | ||
| } | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This page now mixes two different API surfaces.
The new section documents
/api/agent/*endpoints, but the same page still lists/api/generate,/api/stream, and/api/statusabove. That makes the reference internally inconsistent and likely sends readers to routes that do not exist on the current server API. Please either migrate the older sections to the same namespace or split legacy/current APIs explicitly.🤖 Prompt for AI Agents