From 60e8a8a2055d4f50fed17695162df4fe51d4f68f Mon Sep 17 00:00:00 2001 From: Abhinav Rai <122885696+AbhinavJD7@users.noreply.github.com> Date: Sat, 29 Aug 2026 17:03:52 +0530 Subject: [PATCH] perf(message-builder): parallelize file processing with a concurrency cap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the sequential await-in-loop in processUnifiedFilesArray and processExplicitCsvFiles with a capped concurrent detection pass, so a request carrying several files no longer pays the sum of their read and parse times. This reimplements MB-011 against code that still runs. The original branch applied the same technique to the CSV block inside buildMessagesArray, which 5c0db3d1 has since deleted as unreachable — both call sites handed that function empty arrays, and an instrumented run recorded zero hits across the bugfixes suite. The diagnosis and the approach are Abhinav's; only the location has moved, to the two functions that real generate() and stream() calls actually reach. What runs concurrently is deliberately narrow. FileDetector .detectAndProcess is the expensive half and the only half that is independent per file. Everything else stays in a sequential index-ordered pass, because it is order-dependent in ways not visible from its shape: appendDetectedFileResult appends to input.text, input.images and input.pdfFiles, so reordering it would reorder the prompt and the attachments relative to the files the caller supplied, and tryRegisterFileReference mutates the shared registry and draws reference ids from it. Ordering is therefore preserved by construction rather than by assertion — the loop that builds output is the same loop, in the same order, reading results computed earlier. Lazy-registration candidates are identified up front, from a synchronous predicate, and deliberately NOT detected in the concurrent pass: registration usually means the bytes are never processed at all, so pre-detecting them would perform exactly the work that path exists to avoid. The rare fall-through, where registration is attempted and declines, detects inline. Promise.allSettled rather than Promise.all, for two reasons. With `all` the rejection that surfaces is the one that happened first in time, so which file gets blamed for a batch failure would depend on disk scheduling; the ordered walk preserves the sequential loop's first-failure-by-index behaviour. And a second failure arriving after the first has already thrown cannot become an unhandled rejection. The cap is four, and it is a cap rather than an unbounded fan-out because this path admits files up to 100 MB — N in flight means N resident buffers plus whatever each processor allocates. An unbounded Promise.all would trade a latency problem for a memory one, which on a large batch is the worse of the two. Also adds csvFiles to the public TextGenerationOptions["input"], resolving the standing review thread. processExplicitCsvFiles has always read it and the internal GenerateOptions has always carried it; it was missing only from the public type, so callers reaching shipped behaviour had to widen the type themselves. No new test suite, and that is a deliberate call rather than an omission. File ordering is not observable end-to-end offline: the providers that reach this preprocessing (AI Studio, Vertex, Bedrock) either swallow file errors and continue or offer no endpoint override, so no local stand-in can observe the assembled prompt without a live key. Rather than assert on internals — which rule 15 exists to prevent — the ordering guarantee is kept structural, as described above. Verified against the existing suites that do exercise these paths: bugfixes passes, and context's one failure is a live-provider flake in a case that passes only input.text and so cannot reach either function. --- .../type-aliases/EnhancedGenerateResult.md | 2 +- .../type-aliases/GenerateOptionsNormalized.md | 2 +- docs/api/type-aliases/ModelAliasConfig.md | 4 +- .../type-aliases/NativeGenerateLoopArgs.md | 28 +- .../type-aliases/NativeGenerateLoopResult.md | 22 +- docs/api/type-aliases/SingleShotRequest.md | 12 +- docs/api/type-aliases/SingleShotResult.md | 8 +- docs/api/type-aliases/TTSMetadata.md | 10 +- .../api/type-aliases/TextGenerationOptions.md | 129 +++++---- docs/api/type-aliases/TextGenerationResult.md | 2 +- src/lib/types/generate.ts | 9 + src/lib/utils/messageBuilder.ts | 261 ++++++++++++------ 12 files changed, 306 insertions(+), 183 deletions(-) diff --git a/docs/api/type-aliases/EnhancedGenerateResult.md b/docs/api/type-aliases/EnhancedGenerateResult.md index 1f987a4da..b7dc2f52e 100644 --- a/docs/api/type-aliases/EnhancedGenerateResult.md +++ b/docs/api/type-aliases/EnhancedGenerateResult.md @@ -8,7 +8,7 @@ > **EnhancedGenerateResult** = [`GenerateResult`](GenerateResult.md) & `object` -Defined in: [types/generate.ts:1756](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1756) +Defined in: [types/generate.ts:1765](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1765) ## Type Declaration diff --git a/docs/api/type-aliases/GenerateOptionsNormalized.md b/docs/api/type-aliases/GenerateOptionsNormalized.md index c69ab00ca..64ce8307e 100644 --- a/docs/api/type-aliases/GenerateOptionsNormalized.md +++ b/docs/api/type-aliases/GenerateOptionsNormalized.md @@ -8,7 +8,7 @@ > **GenerateOptionsNormalized** = [`GenerateOptions`](GenerateOptions.md) & `object` -Defined in: [types/generate.ts:1783](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1783) +Defined in: [types/generate.ts:1792](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1792) Internal alias used by messageBuilder helpers after the entry-point (`buildMultimodalMessagesArray`) has guaranteed that `input` is non-null. diff --git a/docs/api/type-aliases/ModelAliasConfig.md b/docs/api/type-aliases/ModelAliasConfig.md index 96b59ec9a..1bead56d0 100644 --- a/docs/api/type-aliases/ModelAliasConfig.md +++ b/docs/api/type-aliases/ModelAliasConfig.md @@ -8,7 +8,7 @@ > **ModelAliasConfig** = `object` -Defined in: [types/generate.ts:1766](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1766) +Defined in: [types/generate.ts:1775](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1775) NL-004: Model alias/deprecation configuration. Allows mapping deprecated model names to their replacements. @@ -19,4 +19,4 @@ Allows mapping deprecated model names to their replacements. > **aliases**: `Record`\<`string`, \{ `target`: `string`; `action`: `"warn"` \| `"redirect"` \| `"block"`; `reason?`: `string`; \}\> -Defined in: [types/generate.ts:1767](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1767) +Defined in: [types/generate.ts:1776](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1776) diff --git a/docs/api/type-aliases/NativeGenerateLoopArgs.md b/docs/api/type-aliases/NativeGenerateLoopArgs.md index 8755dde53..2b00a1716 100644 --- a/docs/api/type-aliases/NativeGenerateLoopArgs.md +++ b/docs/api/type-aliases/NativeGenerateLoopArgs.md @@ -8,7 +8,7 @@ > **NativeGenerateLoopArgs** = `object` -Defined in: [types/generate.ts:1792](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1792) +Defined in: [types/generate.ts:1801](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1801) Inputs to the shared native generate loop (`core/nativeGenerateLoop.ts`). One loop serves every provider whose delegating model exposes a v3-shaped @@ -20,7 +20,7 @@ One loop serves every provider whose delegating model exposes a v3-shaped > **doGenerate**: (`options`) => `Promise`\<`Record`\<`string`, `unknown`\>\> -Defined in: [types/generate.ts:1793](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1793) +Defined in: [types/generate.ts:1802](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1802) #### Parameters @@ -38,7 +38,7 @@ Defined in: [types/generate.ts:1793](https://github.com/juspay/neurolink/blob/re > **conversation**: `Record`\<`string`, `unknown`\>[] -Defined in: [types/generate.ts:1797](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1797) +Defined in: [types/generate.ts:1806](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1806) Conversation in the message-builder shape each doGenerate converts itself. @@ -48,7 +48,7 @@ Conversation in the message-builder shape each doGenerate converts itself. > `optional` **tools?**: `Record`\<`string`, `unknown`\>[] -Defined in: [types/generate.ts:1799](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1799) +Defined in: [types/generate.ts:1808](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1808) Tool declarations in the v3 shape doGenerate already knows how to convert. @@ -58,7 +58,7 @@ Tool declarations in the v3 shape doGenerate already knows how to convert. > **toolsRecord**: `Record`\<`string`, `unknown`\> -Defined in: [types/generate.ts:1801](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1801) +Defined in: [types/generate.ts:1810](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1810) Registered tools, used to execute a call the model asks for. @@ -68,7 +68,7 @@ Registered tools, used to execute a call the model asks for. > `optional` **toolChoice?**: `unknown` -Defined in: [types/generate.ts:1802](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1802) +Defined in: [types/generate.ts:1811](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1811) --- @@ -76,7 +76,7 @@ Defined in: [types/generate.ts:1802](https://github.com/juspay/neurolink/blob/re > `optional` **responseFormat?**: `Record`\<`string`, `unknown`\> -Defined in: [types/generate.ts:1803](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1803) +Defined in: [types/generate.ts:1812](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1812) --- @@ -84,7 +84,7 @@ Defined in: [types/generate.ts:1803](https://github.com/juspay/neurolink/blob/re > `optional` **providerOptions?**: `Record`\<`string`, `Record`\<`string`, `unknown`\>\> -Defined in: [types/generate.ts:1804](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1804) +Defined in: [types/generate.ts:1813](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1813) --- @@ -92,7 +92,7 @@ Defined in: [types/generate.ts:1804](https://github.com/juspay/neurolink/blob/re > **maxSteps**: `number` -Defined in: [types/generate.ts:1805](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1805) +Defined in: [types/generate.ts:1814](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1814) --- @@ -100,7 +100,7 @@ Defined in: [types/generate.ts:1805](https://github.com/juspay/neurolink/blob/re > `optional` **maxOutputTokens?**: `number` -Defined in: [types/generate.ts:1806](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1806) +Defined in: [types/generate.ts:1815](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1815) --- @@ -108,7 +108,7 @@ Defined in: [types/generate.ts:1806](https://github.com/juspay/neurolink/blob/re > `optional` **temperature?**: `number` -Defined in: [types/generate.ts:1807](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1807) +Defined in: [types/generate.ts:1816](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1816) --- @@ -116,7 +116,7 @@ Defined in: [types/generate.ts:1807](https://github.com/juspay/neurolink/blob/re > `optional` **abortSignal?**: `AbortSignal` -Defined in: [types/generate.ts:1808](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1808) +Defined in: [types/generate.ts:1817](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1817) --- @@ -124,7 +124,7 @@ Defined in: [types/generate.ts:1808](https://github.com/juspay/neurolink/blob/re > `optional` **toolTimeoutMs?**: `number` \| `null` -Defined in: [types/generate.ts:1810](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1810) +Defined in: [types/generate.ts:1819](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1819) Per-tool-execution cap, forwarded into `guardToolExecutor`. `null` for no bound. @@ -134,7 +134,7 @@ Per-tool-execution cap, forwarded into `guardToolExecutor`. `null` for no bound. > **runStep**: (`call`) => `Promise`\<`Record`\<`string`, `unknown`\>\> -Defined in: [types/generate.ts:1812](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1812) +Defined in: [types/generate.ts:1821](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1821) Wraps one step: retry ladder plus provider error classification. diff --git a/docs/api/type-aliases/NativeGenerateLoopResult.md b/docs/api/type-aliases/NativeGenerateLoopResult.md index cca7a9ab6..0d51170fd 100644 --- a/docs/api/type-aliases/NativeGenerateLoopResult.md +++ b/docs/api/type-aliases/NativeGenerateLoopResult.md @@ -8,7 +8,7 @@ > **NativeGenerateLoopResult** = `object` -Defined in: [types/generate.ts:1817](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1817) +Defined in: [types/generate.ts:1826](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1826) ## Properties @@ -16,7 +16,7 @@ Defined in: [types/generate.ts:1817](https://github.com/juspay/neurolink/blob/re > **text**: `string` -Defined in: [types/generate.ts:1818](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1818) +Defined in: [types/generate.ts:1827](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1827) --- @@ -24,7 +24,7 @@ Defined in: [types/generate.ts:1818](https://github.com/juspay/neurolink/blob/re > `optional` **reasoning?**: `string` -Defined in: [types/generate.ts:1820](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1820) +Defined in: [types/generate.ts:1829](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1829) Joined reasoning content parts from the final step, when the vendor sent any. @@ -34,7 +34,7 @@ Joined reasoning content parts from the final step, when the vendor sent any. > **finishReason**: `string` -Defined in: [types/generate.ts:1821](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1821) +Defined in: [types/generate.ts:1830](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1830) --- @@ -42,7 +42,7 @@ Defined in: [types/generate.ts:1821](https://github.com/juspay/neurolink/blob/re > `optional` **rawFinishReason?**: `string` -Defined in: [types/generate.ts:1822](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1822) +Defined in: [types/generate.ts:1831](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1831) --- @@ -50,7 +50,7 @@ Defined in: [types/generate.ts:1822](https://github.com/juspay/neurolink/blob/re > **inputTokens**: `number` -Defined in: [types/generate.ts:1823](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1823) +Defined in: [types/generate.ts:1832](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1832) --- @@ -58,7 +58,7 @@ Defined in: [types/generate.ts:1823](https://github.com/juspay/neurolink/blob/re > **outputTokens**: `number` -Defined in: [types/generate.ts:1824](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1824) +Defined in: [types/generate.ts:1833](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1833) --- @@ -66,7 +66,7 @@ Defined in: [types/generate.ts:1824](https://github.com/juspay/neurolink/blob/re > **cacheReadTokens**: `number` -Defined in: [types/generate.ts:1825](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1825) +Defined in: [types/generate.ts:1834](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1834) --- @@ -74,7 +74,7 @@ Defined in: [types/generate.ts:1825](https://github.com/juspay/neurolink/blob/re > **cacheWriteTokens**: `number` -Defined in: [types/generate.ts:1826](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1826) +Defined in: [types/generate.ts:1835](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1835) --- @@ -82,7 +82,7 @@ Defined in: [types/generate.ts:1826](https://github.com/juspay/neurolink/blob/re > **toolsUsed**: `string`[] -Defined in: [types/generate.ts:1827](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1827) +Defined in: [types/generate.ts:1836](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1836) --- @@ -90,4 +90,4 @@ Defined in: [types/generate.ts:1827](https://github.com/juspay/neurolink/blob/re > **steps**: `number` -Defined in: [types/generate.ts:1828](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1828) +Defined in: [types/generate.ts:1837](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1837) diff --git a/docs/api/type-aliases/SingleShotRequest.md b/docs/api/type-aliases/SingleShotRequest.md index 464325c56..c008f5d75 100644 --- a/docs/api/type-aliases/SingleShotRequest.md +++ b/docs/api/type-aliases/SingleShotRequest.md @@ -8,7 +8,7 @@ > **SingleShotRequest** = `object` -Defined in: [types/generate.ts:1831](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1831) +Defined in: [types/generate.ts:1840](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1840) ## Properties @@ -16,7 +16,7 @@ Defined in: [types/generate.ts:1831](https://github.com/juspay/neurolink/blob/re > `optional` **system?**: `string` -Defined in: [types/generate.ts:1832](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1832) +Defined in: [types/generate.ts:1841](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1841) --- @@ -24,7 +24,7 @@ Defined in: [types/generate.ts:1832](https://github.com/juspay/neurolink/blob/re > **prompt**: `string` -Defined in: [types/generate.ts:1833](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1833) +Defined in: [types/generate.ts:1842](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1842) --- @@ -32,7 +32,7 @@ Defined in: [types/generate.ts:1833](https://github.com/juspay/neurolink/blob/re > `optional` **maxOutputTokens?**: `number` -Defined in: [types/generate.ts:1834](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1834) +Defined in: [types/generate.ts:1843](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1843) --- @@ -40,7 +40,7 @@ Defined in: [types/generate.ts:1834](https://github.com/juspay/neurolink/blob/re > `optional` **temperature?**: `number` -Defined in: [types/generate.ts:1835](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1835) +Defined in: [types/generate.ts:1844](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1844) --- @@ -48,4 +48,4 @@ Defined in: [types/generate.ts:1835](https://github.com/juspay/neurolink/blob/re > `optional` **abortSignal?**: `AbortSignal` -Defined in: [types/generate.ts:1836](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1836) +Defined in: [types/generate.ts:1845](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1845) diff --git a/docs/api/type-aliases/SingleShotResult.md b/docs/api/type-aliases/SingleShotResult.md index 0a3214f6c..563cbb5e0 100644 --- a/docs/api/type-aliases/SingleShotResult.md +++ b/docs/api/type-aliases/SingleShotResult.md @@ -8,7 +8,7 @@ > **SingleShotResult** = `object` -Defined in: [types/generate.ts:1839](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1839) +Defined in: [types/generate.ts:1848](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1848) ## Properties @@ -16,7 +16,7 @@ Defined in: [types/generate.ts:1839](https://github.com/juspay/neurolink/blob/re > **text**: `string` -Defined in: [types/generate.ts:1840](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1840) +Defined in: [types/generate.ts:1849](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1849) --- @@ -24,7 +24,7 @@ Defined in: [types/generate.ts:1840](https://github.com/juspay/neurolink/blob/re > `optional` **usage?**: `object` -Defined in: [types/generate.ts:1841](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1841) +Defined in: [types/generate.ts:1850](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1850) #### inputTokens? @@ -44,4 +44,4 @@ Defined in: [types/generate.ts:1841](https://github.com/juspay/neurolink/blob/re > `optional` **finishReason?**: `string` -Defined in: [types/generate.ts:1842](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1842) +Defined in: [types/generate.ts:1851](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1851) diff --git a/docs/api/type-aliases/TTSMetadata.md b/docs/api/type-aliases/TTSMetadata.md index c91f2316d..5caddde31 100644 --- a/docs/api/type-aliases/TTSMetadata.md +++ b/docs/api/type-aliases/TTSMetadata.md @@ -8,7 +8,7 @@ > **TTSMetadata** = `object` -Defined in: [types/generate.ts:1741](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1741) +Defined in: [types/generate.ts:1750](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1750) Enhanced result type with optional analytics/evaluation @@ -18,7 +18,7 @@ Enhanced result type with optional analytics/evaluation > **attempted**: `boolean` -Defined in: [types/generate.ts:1743](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1743) +Defined in: [types/generate.ts:1752](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1752) Whether TTS synthesis was invoked. False indicates TTS was skipped. @@ -28,7 +28,7 @@ Whether TTS synthesis was invoked. False indicates TTS was skipped. > **success**: `boolean` -Defined in: [types/generate.ts:1745](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1745) +Defined in: [types/generate.ts:1754](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1754) Whether TTS synthesis completed successfully. @@ -38,7 +38,7 @@ Whether TTS synthesis completed successfully. > `optional` **error?**: `object` -Defined in: [types/generate.ts:1747](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1747) +Defined in: [types/generate.ts:1756](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1756) Structured synthesis error details, present only when synthesis failed. @@ -60,6 +60,6 @@ Structured synthesis error details, present only when synthesis failed. > `optional` **latency?**: `number` -Defined in: [types/generate.ts:1753](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1753) +Defined in: [types/generate.ts:1762](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1762) TTS synthesis time in milliseconds. diff --git a/docs/api/type-aliases/TextGenerationOptions.md b/docs/api/type-aliases/TextGenerationOptions.md index 4fbab5a5a..fc9795f8c 100644 --- a/docs/api/type-aliases/TextGenerationOptions.md +++ b/docs/api/type-aliases/TextGenerationOptions.md @@ -54,6 +54,17 @@ For video generation, the first image is used as the source frame. > `optional` **pdfFiles?**: (`Buffer` \| `string`)[] +#### csvFiles? + +> `optional` **csvFiles?**: (`Buffer` \| `string`)[] + +CSV files to inline as tabular text, with tool instructions appended. + +Declared here because `processExplicitCsvFiles` has always read it and +the internal `GenerateOptions` has always carried it — it was missing +only from the public type, so callers reaching the shipped behaviour had +to widen the type themselves to do it. + #### files? > `optional` **files?**: (`Buffer` \| `string` \| [`FileWithMetadata`](FileWithMetadata.md))[] @@ -70,7 +81,7 @@ Director Mode segments (2-10). When provided, Director Mode is activated. > `optional` **provider?**: [`AIProviderName`](../enumerations/AIProviderName.md) -Defined in: [types/generate.ts:1267](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1267) +Defined in: [types/generate.ts:1276](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1276) --- @@ -78,7 +89,7 @@ Defined in: [types/generate.ts:1267](https://github.com/juspay/neurolink/blob/re > `optional` **model?**: `string` -Defined in: [types/generate.ts:1268](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1268) +Defined in: [types/generate.ts:1277](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1277) --- @@ -86,7 +97,7 @@ Defined in: [types/generate.ts:1268](https://github.com/juspay/neurolink/blob/re > `optional` **region?**: `string` -Defined in: [types/generate.ts:1269](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1269) +Defined in: [types/generate.ts:1278](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1278) --- @@ -94,7 +105,7 @@ Defined in: [types/generate.ts:1269](https://github.com/juspay/neurolink/blob/re > `optional` **temperature?**: `number` -Defined in: [types/generate.ts:1270](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1270) +Defined in: [types/generate.ts:1279](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1279) --- @@ -102,7 +113,7 @@ Defined in: [types/generate.ts:1270](https://github.com/juspay/neurolink/blob/re > `optional` **maxTokens?**: `number` -Defined in: [types/generate.ts:1271](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1271) +Defined in: [types/generate.ts:1280](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1280) --- @@ -110,7 +121,7 @@ Defined in: [types/generate.ts:1271](https://github.com/juspay/neurolink/blob/re > `optional` **topP?**: `number` -Defined in: [types/generate.ts:1273](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1273) +Defined in: [types/generate.ts:1282](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1282) Top-p (nucleus) sampling parameter. Controls diversity of generated tokens. @@ -120,7 +131,7 @@ Top-p (nucleus) sampling parameter. Controls diversity of generated tokens. > `optional` **topK?**: `number` -Defined in: [types/generate.ts:1275](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1275) +Defined in: [types/generate.ts:1284](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1284) Top-k sampling parameter. Limits the number of tokens considered. (Google/Gemini models only) @@ -130,7 +141,7 @@ Top-k sampling parameter. Limits the number of tokens considered. (Google/Gemini > `optional` **stopSequences?**: `string`[] -Defined in: [types/generate.ts:1277](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1277) +Defined in: [types/generate.ts:1286](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1286) Stop sequences that will halt generation when encountered. @@ -140,7 +151,7 @@ Stop sequences that will halt generation when encountered. > `optional` **systemPrompt?**: `string` -Defined in: [types/generate.ts:1278](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1278) +Defined in: [types/generate.ts:1287](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1287) --- @@ -148,7 +159,7 @@ Defined in: [types/generate.ts:1278](https://github.com/juspay/neurolink/blob/re > `optional` **schema?**: [`ZodUnknownSchema`](ZodUnknownSchema.md) \| [`Schema`](Schema.md)\<`unknown`\> -Defined in: [types/generate.ts:1279](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1279) +Defined in: [types/generate.ts:1288](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1288) --- @@ -156,7 +167,7 @@ Defined in: [types/generate.ts:1279](https://github.com/juspay/neurolink/blob/re > `optional` **output?**: `object` -Defined in: [types/generate.ts:1291](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1291) +Defined in: [types/generate.ts:1300](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1300) Output configuration options @@ -221,7 +232,7 @@ output: { > `optional` **tools?**: `Record`\<`string`, [`Tool`](Tool.md)\> -Defined in: [types/generate.ts:1323](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1323) +Defined in: [types/generate.ts:1332](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1332) --- @@ -229,7 +240,7 @@ Defined in: [types/generate.ts:1323](https://github.com/juspay/neurolink/blob/re > `optional` **enabledToolNames?**: `string`[] -Defined in: [types/generate.ts:1338](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1338) +Defined in: [types/generate.ts:1347](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1347) Filter available tools by name. Only tools with names in this array will be made available. @@ -251,7 +262,7 @@ await neurolink.generate({ > `optional` **timeout?**: `number` \| `string` -Defined in: [types/generate.ts:1339](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1339) +Defined in: [types/generate.ts:1348](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1348) --- @@ -259,7 +270,7 @@ Defined in: [types/generate.ts:1339](https://github.com/juspay/neurolink/blob/re > `optional` **turnTimeoutMs?**: `number` -Defined in: [types/generate.ts:1341](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1341) +Defined in: [types/generate.ts:1350](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1350) Wall-clock cap for the whole agentic turn (ms). See GenerateOptions.turnTimeoutMs. @@ -269,7 +280,7 @@ Wall-clock cap for the whole agentic turn (ms). See GenerateOptions.turnTimeoutM > `optional` **stallTimeoutMs?**: `number` -Defined in: [types/generate.ts:1343](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1343) +Defined in: [types/generate.ts:1352](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1352) Max time with no progress before the turn ends as "stalled" (ms). See GenerateOptions.stallTimeoutMs. @@ -279,7 +290,7 @@ Max time with no progress before the turn ends as "stalled" (ms). See GenerateOp > `optional` **wrapupTimeLeadMs?**: `number` -Defined in: [types/generate.ts:1345](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1345) +Defined in: [types/generate.ts:1354](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1354) Remaining-time threshold that triggers the wrap-up nudge (ms). See GenerateOptions.wrapupTimeLeadMs. @@ -289,7 +300,7 @@ Remaining-time threshold that triggers the wrap-up nudge (ms). See GenerateOptio > `optional` **toolTimeoutMs?**: `number` \| `null` -Defined in: [types/generate.ts:1347](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1347) +Defined in: [types/generate.ts:1356](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1356) Per-tool-execution timeout (ms, default 300_000; `null` for no bound). See GenerateOptions.toolTimeoutMs. @@ -299,7 +310,7 @@ Per-tool-execution timeout (ms, default 300_000; `null` for no bound). See Gener > `optional` **abortSignal?**: `AbortSignal` -Defined in: [types/generate.ts:1349](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1349) +Defined in: [types/generate.ts:1358](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1358) AbortSignal for external cancellation of the AI call @@ -309,7 +320,7 @@ AbortSignal for external cancellation of the AI call > `optional` **toolExecutionCapture?**: [`ToolExecutionCaptureOptions`](ToolExecutionCaptureOptions.md) -Defined in: [types/generate.ts:1351](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1351) +Defined in: [types/generate.ts:1360](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1360) Bounds for tool execution capture. See GenerateOptions.toolExecutionCapture. @@ -319,7 +330,7 @@ Bounds for tool execution capture. See GenerateOptions.toolExecutionCapture. > `optional` **disableTools?**: `boolean` -Defined in: [types/generate.ts:1359](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1359) +Defined in: [types/generate.ts:1368](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1368) --- @@ -327,7 +338,7 @@ Defined in: [types/generate.ts:1359](https://github.com/juspay/neurolink/blob/re > `optional` **disableToolCallRepair?**: `boolean` -Defined in: [types/generate.ts:1361](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1361) +Defined in: [types/generate.ts:1370](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1370) Disable the schema-driven tool call repair mechanism (BZ-665). Default: false (repair enabled). @@ -337,7 +348,7 @@ Disable the schema-driven tool call repair mechanism (BZ-665). Default: false (r > `optional` **maxSteps?**: `number` -Defined in: [types/generate.ts:1362](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1362) +Defined in: [types/generate.ts:1371](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1371) --- @@ -345,7 +356,7 @@ Defined in: [types/generate.ts:1362](https://github.com/juspay/neurolink/blob/re > `optional` **toolFilter?**: `string`[] -Defined in: [types/generate.ts:1365](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1365) +Defined in: [types/generate.ts:1374](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1374) Include only these tools by name (whitelist). If set, only matching tools are available. @@ -355,7 +366,7 @@ Include only these tools by name (whitelist). If set, only matching tools are av > `optional` **excludeTools?**: `string`[] -Defined in: [types/generate.ts:1368](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1368) +Defined in: [types/generate.ts:1377](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1377) Exclude these tools by name (blacklist). Applied after toolFilter. @@ -365,7 +376,7 @@ Exclude these tools by name (blacklist). Applied after toolFilter. > `optional` **disableToolCache?**: `boolean` -Defined in: [types/generate.ts:1371](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1371) +Defined in: [types/generate.ts:1380](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1380) Disable tool result caching for this request (overrides global mcp.cache.enabled) @@ -375,7 +386,7 @@ Disable tool result caching for this request (overrides global mcp.cache.enabled > `optional` **disableInternalFallback?**: `boolean` -Defined in: [types/generate.ts:1380](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1380) +Defined in: [types/generate.ts:1389](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1389) Caller owns fallback order. Read in two places: `directProviderGeneration` bounds its static provider-priority walk to one candidate, and @@ -389,7 +400,7 @@ invalid-model error surfaces as itself. Mapped from > `optional` **toolChoice?**: [`ToolChoice`](ToolChoice.md)\<`Record`\<`string`, [`Tool`](Tool.md)\>\> -Defined in: [types/generate.ts:1395](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1395) +Defined in: [types/generate.ts:1404](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1404) Tool choice configuration for the generation. Controls whether and which tools the model must call. @@ -409,7 +420,7 @@ will cause infinite tool calls until `maxSteps` is exhausted. > `optional` **prepareStep?**: (`options`) => `PromiseLike`\<\{ `model?`: [`LanguageModel`](LanguageModel.md); `toolChoice?`: [`ToolChoice`](ToolChoice.md)\<`Record`\<`string`, [`Tool`](Tool.md)\>\>; `experimental_activeTools?`: `string`[]; \} \| `undefined`\> -Defined in: [types/generate.ts:1420](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1420) +Defined in: [types/generate.ts:1429](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1429) Optional callback that runs before each step in a multi-step generation. Allows dynamically changing `toolChoice` and available tools per step. @@ -466,7 +477,7 @@ https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text#parameters > `optional` **tts?**: [`TTSOptions`](TTSOptions.md) -Defined in: [types/generate.ts:1463](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1463) +Defined in: [types/generate.ts:1472](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1472) Text-to-Speech (TTS) configuration @@ -503,7 +514,7 @@ const result = await neurolink.generate({ > `optional` **stt?**: [`STTOptions`](STTOptions.md) & `object` -Defined in: [types/generate.ts:1482](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1482) +Defined in: [types/generate.ts:1491](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1491) Speech-to-Text (STT) configuration @@ -543,7 +554,7 @@ const result = await neurolink.generate({ > `optional` **enableEvaluation?**: `boolean` -Defined in: [types/generate.ts:1485](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1485) +Defined in: [types/generate.ts:1494](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1494) --- @@ -551,7 +562,7 @@ Defined in: [types/generate.ts:1485](https://github.com/juspay/neurolink/blob/re > `optional` **enableAnalytics?**: `boolean` -Defined in: [types/generate.ts:1486](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1486) +Defined in: [types/generate.ts:1495](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1495) --- @@ -559,7 +570,7 @@ Defined in: [types/generate.ts:1486](https://github.com/juspay/neurolink/blob/re > `optional` **context?**: `Record`\<`string`, [`JsonValue`](JsonValue.md)\> -Defined in: [types/generate.ts:1487](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1487) +Defined in: [types/generate.ts:1496](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1496) --- @@ -567,7 +578,7 @@ Defined in: [types/generate.ts:1487](https://github.com/juspay/neurolink/blob/re > `optional` **evaluationDomain?**: `string` -Defined in: [types/generate.ts:1490](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1490) +Defined in: [types/generate.ts:1499](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1499) --- @@ -575,7 +586,7 @@ Defined in: [types/generate.ts:1490](https://github.com/juspay/neurolink/blob/re > `optional` **toolUsageContext?**: `string` -Defined in: [types/generate.ts:1491](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1491) +Defined in: [types/generate.ts:1500](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1500) --- @@ -583,7 +594,7 @@ Defined in: [types/generate.ts:1491](https://github.com/juspay/neurolink/blob/re > `optional` **conversationHistory?**: `object`[] -Defined in: [types/generate.ts:1492](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1492) +Defined in: [types/generate.ts:1501](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1501) #### role @@ -599,7 +610,7 @@ Defined in: [types/generate.ts:1492](https://github.com/juspay/neurolink/blob/re > `optional` **conversationMessages?**: [`ChatMessage`](ChatMessage.md)[] -Defined in: [types/generate.ts:1495](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1495) +Defined in: [types/generate.ts:1504](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1504) --- @@ -607,7 +618,7 @@ Defined in: [types/generate.ts:1495](https://github.com/juspay/neurolink/blob/re > `optional` **conversationMemoryConfig?**: `Partial`\<[`ConversationMemoryConfig`](ConversationMemoryConfig.md)\> -Defined in: [types/generate.ts:1498](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1498) +Defined in: [types/generate.ts:1507](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1507) --- @@ -615,7 +626,7 @@ Defined in: [types/generate.ts:1498](https://github.com/juspay/neurolink/blob/re > `optional` **originalPrompt?**: `string` -Defined in: [types/generate.ts:1499](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1499) +Defined in: [types/generate.ts:1508](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1508) --- @@ -623,7 +634,7 @@ Defined in: [types/generate.ts:1499](https://github.com/juspay/neurolink/blob/re > `optional` **middleware?**: [`MiddlewareFactoryOptions`](MiddlewareFactoryOptions.md) -Defined in: [types/generate.ts:1502](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1502) +Defined in: [types/generate.ts:1511](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1511) --- @@ -631,7 +642,7 @@ Defined in: [types/generate.ts:1502](https://github.com/juspay/neurolink/blob/re > `optional` **onFinish?**: [`OnFinishCallback`](OnFinishCallback.md) -Defined in: [types/generate.ts:1510](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1510) +Defined in: [types/generate.ts:1519](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1519) --- @@ -639,7 +650,7 @@ Defined in: [types/generate.ts:1510](https://github.com/juspay/neurolink/blob/re > `optional` **onError?**: [`OnErrorCallback`](OnErrorCallback.md) -Defined in: [types/generate.ts:1511](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1511) +Defined in: [types/generate.ts:1520](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1520) --- @@ -647,7 +658,7 @@ Defined in: [types/generate.ts:1511](https://github.com/juspay/neurolink/blob/re > `optional` **expectedOutcome?**: `string` -Defined in: [types/generate.ts:1514](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1514) +Defined in: [types/generate.ts:1523](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1523) --- @@ -655,7 +666,7 @@ Defined in: [types/generate.ts:1514](https://github.com/juspay/neurolink/blob/re > `optional` **evaluationCriteria?**: `string`[] -Defined in: [types/generate.ts:1515](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1515) +Defined in: [types/generate.ts:1524](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1524) --- @@ -663,7 +674,7 @@ Defined in: [types/generate.ts:1515](https://github.com/juspay/neurolink/blob/re > `optional` **csvOptions?**: [`CSVProcessorOptions`](CSVProcessorOptions.md) -Defined in: [types/generate.ts:1518](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1518) +Defined in: [types/generate.ts:1527](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1527) --- @@ -671,7 +682,7 @@ Defined in: [types/generate.ts:1518](https://github.com/juspay/neurolink/blob/re > `optional` **pdfOptions?**: `object` -Defined in: [types/generate.ts:1521](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1521) +Defined in: [types/generate.ts:1530](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1530) PDF processing options (#258). @@ -705,7 +716,7 @@ Max pages converted by the image fallback (#297); defaults to PDF_LIMITS.DEFAULT > `optional` **enableSummarization?**: `boolean` -Defined in: [types/generate.ts:1532](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1532) +Defined in: [types/generate.ts:1541](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1541) --- @@ -713,7 +724,7 @@ Defined in: [types/generate.ts:1532](https://github.com/juspay/neurolink/blob/re > `optional` **skipToolPromptInjection?**: `boolean` -Defined in: [types/generate.ts:1550](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1550) +Defined in: [types/generate.ts:1559](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1559) Skip injecting tool schemas into the system prompt. When true, tools are ONLY passed natively via the provider's `tools` parameter, @@ -726,7 +737,7 @@ Default: false (backward compatible — tool schemas are injected into system pr > `optional` **thinking?**: `boolean` -Defined in: [types/generate.ts:1616](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1616) +Defined in: [types/generate.ts:1625](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1625) Enable extended thinking capability (simplified option). Equivalent to `thinkingConfig.enabled = true`. @@ -738,7 +749,7 @@ Works with both Anthropic and Gemini 3 models. > `optional` **thinkingBudget?**: `number` -Defined in: [types/generate.ts:1623](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1623) +Defined in: [types/generate.ts:1632](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1632) Token budget for thinking (Anthropic models only). Equivalent to `thinkingConfig.budgetTokens`. @@ -750,7 +761,7 @@ Range: 5000-100000 tokens. Ignored for Gemini models. > `optional` **thinkingLevel?**: `"minimal"` \| `"low"` \| `"medium"` \| `"high"` -Defined in: [types/generate.ts:1634](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1634) +Defined in: [types/generate.ts:1643](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1643) Thinking level for Gemini 3 models only. Equivalent to `thinkingConfig.thinkingLevel`. @@ -767,7 +778,7 @@ Equivalent to `thinkingConfig.thinkingLevel`. > `optional` **thinkingConfig?**: `object` -Defined in: [types/generate.ts:1642](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1642) +Defined in: [types/generate.ts:1651](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1651) Full thinking/reasoning configuration (recommended for SDK usage). Takes precedence over simplified options (thinking, thinkingBudget, thinkingLevel). @@ -806,7 +817,7 @@ Above documentation for provider-specific behavior and option compatibility. > `optional` **credentials?**: [`NeurolinkCredentials`](NeurolinkCredentials.md) -Defined in: [types/generate.ts:1658](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1658) +Defined in: [types/generate.ts:1667](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1667) Per-provider credential overrides for this request. Overrides instance-level credentials set in `new NeuroLink({ credentials })`. @@ -818,7 +829,7 @@ Unset providers fall through to instance credentials, then environment variables > `optional` **requestId?**: `string` -Defined in: [types/generate.ts:1665](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1665) +Defined in: [types/generate.ts:1674](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1674) Optional request identifier for observability and log correlation. When provided, this ID is forwarded to spans, logs, and telemetry so @@ -830,7 +841,7 @@ callers can correlate generation traces back to their own request lifecycle. > `optional` **piiDetection?**: [`GenerateOptions`](GenerateOptions.md)\[`"piiDetection"`\] -Defined in: [types/generate.ts:1668](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1668) +Defined in: [types/generate.ts:1677](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1677) PII detection config — forwarded from GenerateOptions/StreamOptions. @@ -840,7 +851,7 @@ PII detection config — forwarded from GenerateOptions/StreamOptions. > `optional` **responseValidation?**: [`GenerateOptions`](GenerateOptions.md)\[`"responseValidation"`\] -Defined in: [types/generate.ts:1671](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1671) +Defined in: [types/generate.ts:1680](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1680) Response validation config — forwarded from GenerateOptions/StreamOptions. @@ -850,7 +861,7 @@ Response validation config — forwarded from GenerateOptions/StreamOptions. > `optional` **inputValidation?**: [`GenerateOptions`](GenerateOptions.md)\[`"inputValidation"`\] -Defined in: [types/generate.ts:1674](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1674) +Defined in: [types/generate.ts:1683](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1683) Input validation config — forwarded from GenerateOptions/StreamOptions. @@ -860,7 +871,7 @@ Input validation config — forwarded from GenerateOptions/StreamOptions. > `optional` **processors?**: [`ProcessorPipelineConfig`](ProcessorPipelineConfig.md) -Defined in: [types/generate.ts:1677](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1677) +Defined in: [types/generate.ts:1686](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1686) #### Deprecated diff --git a/docs/api/type-aliases/TextGenerationResult.md b/docs/api/type-aliases/TextGenerationResult.md index 6376cb020..c8a73f1b5 100644 --- a/docs/api/type-aliases/TextGenerationResult.md +++ b/docs/api/type-aliases/TextGenerationResult.md @@ -8,7 +8,7 @@ > **TextGenerationResult** = `object` & [`MediaGenerationOutputs`](MediaGenerationOutputs.md) -Defined in: [types/generate.ts:1683](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1683) +Defined in: [types/generate.ts:1692](https://github.com/juspay/neurolink/blob/release/src/lib/types/generate.ts#L1692) Text generation result (consolidated from core types) diff --git a/src/lib/types/generate.ts b/src/lib/types/generate.ts index 12ad9d6c4..52efc834c 100644 --- a/src/lib/types/generate.ts +++ b/src/lib/types/generate.ts @@ -1260,6 +1260,15 @@ export type TextGenerationOptions = { */ images?: Array; pdfFiles?: Array; // Support for PDF inputs (for image generation with Vertex AI) + /** + * CSV files to inline as tabular text, with tool instructions appended. + * + * Declared here because `processExplicitCsvFiles` has always read it and + * the internal `GenerateOptions` has always carried it — it was missing + * only from the public type, so callers reaching the shipped behaviour had + * to widen the type themselves to do it. + */ + csvFiles?: Array; files?: Array; // Auto-detect file types (including video for analysis) /** Director Mode segments (2-10). When provided, Director Mode is activated. */ segments?: DirectorSegment[]; diff --git a/src/lib/utils/messageBuilder.ts b/src/lib/utils/messageBuilder.ts index a16c36423..fa3567ef3 100644 --- a/src/lib/utils/messageBuilder.ts +++ b/src/lib/utils/messageBuilder.ts @@ -1,5 +1,6 @@ import { existsSync, readFileSync, statSync } from "fs"; import { readFile as readFileAsync, stat as statAsync } from "fs/promises"; +import pLimit from "p-limit"; import { request } from "undici"; import { redirectFollowingDispatcher } from "./redirectDispatcher.js"; import { @@ -766,6 +767,22 @@ const PREPROCESSED_FILES = new WeakMap>(); */ const FILE_READ_TIMEOUT_MS = 30_000; +/** + * How many files may be detected and processed at once. + * + * A cap rather than unbounded `Promise.all`, and the cap is the point: this + * path admits files up to 100 MB, so N files in flight means N decoded buffers + * resident at once plus whatever each processor allocates on top. Turning a + * sequential loop into an unbounded fan-out trades a latency problem for a + * memory one, and on a large batch that is the worse of the two. + * + * Four is chosen to keep the worst case bounded rather than to saturate a + * disk: the win here is overlapping I/O wait, which most of it is, and the + * marginal gain past a handful of concurrent reads is small next to the + * marginal cost in resident bytes. + */ +const FILE_PROCESSING_CONCURRENCY = 4; + /** * Read a file input's bytes, or null when they cannot be had. * @@ -1057,6 +1074,63 @@ function markFileProcessed(input: object, entry: unknown): void { PREPROCESSED_FILES.set(input, processed); } +/** + * Detect and process one entry of the unified `files` array. + * + * Extracted so the same call can be made from the concurrent pass and from the + * sequential fall-through, rather than duplicated between them. + */ +async function detectFileForUnifiedArray( + file: FileInput | FileWithMetadata, + options: GenerateOptions, + provider: string, + genericFileMaxSize: number, +) { + const rawFileInput = isFileWithMetadata(file) ? file.buffer : file; + // Forward the caller's mimetype hint (Slack/Curator-style + // extension-less buffers) so the eager path classifies correctly + // for tiny files — the lazy registry path has its own hint wiring. + const fileMimetypeHint = isFileWithMetadata(file) ? file.mimetype : undefined; + // The name has to travel the same way, and for the same reason: the + // line above unwraps the object to its buffer, so by the time + // detection resolves an extension there is no name left to read one + // from. Without this a `.tar` supplied as bytes-plus-name is + // unidentifiable — its "ustar" marker sits at byte 257, not at + // offset 0 — and reports "Could not extract content" for an archive + // that extracts perfectly when handed its filename. + const fileFilenameHint = isFileWithMetadata(file) ? file.filename : undefined; + return FileDetector.detectAndProcess(rawFileInput, { + maxSize: genericFileMaxSize, + allowedTypes: [ + "csv", + "image", + "pdf", + "svg", + "video", + "audio", + "archive", + "xlsx", + "docx", + "pptx", + "text", + "unknown", + ], + csvOptions: options.csvOptions, + // #478: videos arrive through this unified `files` path, so this is + // where the CLI's frame/quality/format request has to be handed on. + videoOptions: options.videoOptions + ? { + frames: options.videoOptions.frames, + quality: options.videoOptions.quality, + format: options.videoOptions.format, + } + : undefined, + provider: provider, + mimetypeHint: fileMimetypeHint, + filenameHint: fileFilenameHint, + }); +} + export async function processUnifiedFilesArray( options: GenerateOptions, maxSize: number, @@ -1126,17 +1200,61 @@ export async function processUnifiedFilesArray( | FileReferenceRegistry | undefined; + // Only detection runs concurrently, and only for files that will not + // take the lazy-registration branch. + // + // Everything else in this loop is order-dependent in a way that is not + // obvious from its shape. `appendDetectedFileResult` appends to + // `inp2.text`, `inp2.images` and `inp2.pdfFiles`, so running it out of + // order would reorder the prompt and the attachments against the files + // the caller supplied. `tryRegisterFileReference` mutates the shared + // registry and assigns reference ids from it. Both therefore stay in the + // sequential pass below; what moves off the critical path is + // `FileDetector.detectAndProcess`, which is where the read and the parse + // actually happen and which touches nothing shared. + // + // Lazy candidates are decided here — the predicate is synchronous — but + // are not detected up front, because registration usually means the + // bytes are never processed at all. Pre-detecting them would do exactly + // the work the lazy path exists to avoid. The rare fall-through, where + // registration is attempted and declines, detects inline below. + const isLazyCandidate = files.map( + (file) => + Boolean(fileRegistry) && + getFileSize(file) > SIZE_TIER_THRESHOLDS.TINY_MAX && + !isEagerMultimodalFile(file), + ); + + const genericFileMaxSize = Math.max(maxSize, 100 * 1024 * 1024); + const detectLimit = pLimit(FILE_PROCESSING_CONCURRENCY); + // allSettled, not all: with `all` the rejection that surfaces is the one + // that happened FIRST IN TIME, so which file is blamed for a batch + // failure would depend on disk scheduling. The sequential loop reported + // the first failure BY INDEX, and the ordered walk below preserves that. + // It also means a second failure cannot become an unhandled rejection + // after the first has already thrown. + const detected = await Promise.allSettled( + files.map((file, fileIdx) => + isLazyCandidate[fileIdx] + ? Promise.resolve(undefined) + : detectLimit(() => + detectFileForUnifiedArray( + file, + options, + provider, + genericFileMaxSize, + ), + ), + ), + ); + for (let fileIdx = 0; fileIdx < files.length; fileIdx++) { const file = files[fileIdx]; const filename = extractFilename(file, fileIdx); try { // ─── Lazy file registration path ────────────────────────────── const fileSize = fileRegistry ? getFileSize(file) : 0; - if ( - fileRegistry && - fileSize > SIZE_TIER_THRESHOLDS.TINY_MAX && - !isEagerMultimodalFile(file) - ) { + if (fileRegistry && isLazyCandidate[fileIdx]) { const registered = await tryRegisterFileReference( file, fileSize, @@ -1154,54 +1272,22 @@ export async function processUnifiedFilesArray( } // ─── Full processing path (current behavior) ────────────────── - const genericFileMaxSize = Math.max(maxSize, 100 * 1024 * 1024); - const rawFileInput = isFileWithMetadata(file) ? file.buffer : file; - // Forward the caller's mimetype hint (Slack/Curator-style - // extension-less buffers) so the eager path classifies correctly - // for tiny files — the lazy registry path has its own hint wiring. - const fileMimetypeHint = isFileWithMetadata(file) - ? file.mimetype - : undefined; - // The name has to travel the same way, and for the same reason: the - // line above unwraps the object to its buffer, so by the time - // detection resolves an extension there is no name left to read one - // from. Without this a `.tar` supplied as bytes-plus-name is - // unidentifiable — its "ustar" marker sits at byte 257, not at - // offset 0 — and reports "Could not extract content" for an archive - // that extracts perfectly when handed its filename. - const fileFilenameHint = isFileWithMetadata(file) - ? file.filename - : undefined; - const result = await FileDetector.detectAndProcess(rawFileInput, { - maxSize: genericFileMaxSize, - allowedTypes: [ - "csv", - "image", - "pdf", - "svg", - "video", - "audio", - "archive", - "xlsx", - "docx", - "pptx", - "text", - "unknown", - ], - csvOptions: options.csvOptions, - // #478: videos arrive through this unified `files` path, so this is - // where the CLI's frame/quality/format request has to be handed on. - videoOptions: options.videoOptions - ? { - frames: options.videoOptions.frames, - quality: options.videoOptions.quality, - format: options.videoOptions.format, - } - : undefined, - provider: provider, - mimetypeHint: fileMimetypeHint, - filenameHint: fileFilenameHint, - }); + // Normally already resolved by the concurrent pass above. The inline + // call is the fall-through: this file was a lazy candidate, so it was + // deliberately not detected up front, and registration then declined + // it. Detection has to happen somewhere, and here it is sequential. + const outcome = detected[fileIdx]; + if (outcome.status === "rejected") { + throw outcome.reason; + } + const result = + outcome.value ?? + (await detectFileForUnifiedArray( + file, + options, + provider, + genericFileMaxSize, + )); await appendDetectedFileResult(result, file, options); includedCount++; @@ -1305,34 +1391,31 @@ async function processExplicitCsvFiles( options.input.text = options.input.text || ""; - for (let i = 0; i < options.input.csvFiles.length; i++) { - const csvFile = options.input.csvFiles[i]; - - try { - const result = await FileDetector.detectAndProcess(csvFile, { - allowedTypes: ["csv"], - csvOptions: options.csvOptions, - }); - - const filename = extractFilename(csvFile, i); - const filePath = typeof csvFile === "string" ? csvFile : filename; - let csvSection = `\n\n## CSV Data from "${filename}":\n`; + const csvFiles = options.input.csvFiles; + + // Detection is the expensive half — a read plus a parse per file, and + // previously each one waited for the last. It is also the only half that is + // independent per file, so it is the only half that runs concurrently: the + // sections below are appended to a single `text` string, so they stay in a + // sequential index-ordered pass and the prompt reads identically. + const limit = pLimit(FILE_PROCESSING_CONCURRENCY); + const settled = await Promise.allSettled( + csvFiles.map((csvFile) => + limit(() => + FileDetector.detectAndProcess(csvFile, { + allowedTypes: ["csv"], + csvOptions: options.csvOptions, + }), + ), + ), + ); - if (result.metadata) { - const metadataText = formatCSVMetadata(result.metadata); - if (metadataText) { - csvSection += metadataText + `\n\n`; - } - } + for (let i = 0; i < csvFiles.length; i++) { + const csvFile = csvFiles[i]; + const outcome = settled[i]; - // Put the actual CSV content BEFORE the tool instructions — - // buildCSVToolInstructions references "the CSV data shown above" - // and the trailing position keeps that reference accurate. - csvSection += result.content; - csvSection += buildCSVToolInstructions(filePath); - options.input.text += csvSection; - logger.info(`[CSV] ✅ Processed: ${filename}`); - } catch (error) { + if (outcome.status === "rejected") { + const error: unknown = outcome.reason; const filename = extractFilename(csvFile, i); const errMsg = error instanceof Error ? error.message : String(error); // #273: fail loud instead of embedding the error into the prompt text @@ -1343,6 +1426,26 @@ async function processExplicitCsvFiles( error instanceof Error ? error : new Error(errMsg), ); } + + const result = outcome.value; + const filename = extractFilename(csvFile, i); + const filePath = typeof csvFile === "string" ? csvFile : filename; + let csvSection = `\n\n## CSV Data from "${filename}":\n`; + + if (result.metadata) { + const metadataText = formatCSVMetadata(result.metadata); + if (metadataText) { + csvSection += metadataText + `\n\n`; + } + } + + // Put the actual CSV content BEFORE the tool instructions — + // buildCSVToolInstructions references "the CSV data shown above" + // and the trailing position keeps that reference accurate. + csvSection += result.content; + csvSection += buildCSVToolInstructions(filePath); + options.input.text += csvSection; + logger.info(`[CSV] ✅ Processed: ${filename}`); } }