Repository navigation
feat(object): add support for object generation in generate() - #767
YaswanthKurapati24 wants to merge 1 commit into
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdds a schema-driven, three‑phase structured‑output pathway (extract → validate → repair → generateObject fallback) integrated into the core generation flow, new structured output utilities and message helpers, updated GenerationResult types, and tooling-context preservation across phases. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant BaseProvider
participant LanguageModel
participant JSONExtractor
participant SchemaValidator
participant RepairHandler
participant GenerateObject
Client->>BaseProvider: executeGeneration(model, messages, schema?)
BaseProvider->>LanguageModel: Phase 0: initial generation (generateText)
LanguageModel-->>BaseProvider: text response (GenerationResult)
alt shouldUseStructuredOutput && schema present
BaseProvider->>JSONExtractor: Phase 1: extractAndParseJSON(text)
JSONExtractor-->>BaseProvider: parsed JSON or null
BaseProvider->>SchemaValidator: validateAgainstSchema(parsed JSON, schema)
alt validation success
SchemaValidator-->>BaseProvider: valid
BaseProvider-->>Client: GenerationResult (structuredOutputAchieved: true)
else validation fails
SchemaValidator-->>BaseProvider: errors
BaseProvider->>RepairHandler: Phase 2: attemptRepair(model, response, schema, errors, messages)
RepairHandler->>LanguageModel: repair prompt
LanguageModel-->>RepairHandler: repaired text
RepairHandler-->>BaseProvider: repaired JSON (if valid)
alt repaired valid
BaseProvider->>GenerateObject: Phase 3: generateObject(...) to produce typed output
GenerateObject-->>BaseProvider: typed result
BaseProvider-->>Client: GenerationResult (structuredOutputAchieved: true)
else repair failed
BaseProvider-->>Client: GenerationResult (structuredOutputAchieved: false)
end
end
else no schema / not using structured output
BaseProvider-->>Client: GenerationResult (structuredOutputAchieved: false)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/lib/neurolink.ts (1)
2521-2527: EnsurestructuredOutputAchievedis preserved for direct provider fallback as wellIn the MCP path (
tryMCPGeneration), you correctly thread throughstructuredOutputAchieved:return { content: result.content || "", provider: providerName, usage: result.usage, structuredOutputAchieved: result.structuredOutputAchieved, ... };But in the direct provider fallback (
directProviderGeneration), the mapping omits this field entirely:return { content: result.content || "", provider: providerName, model: result.model, usage: result.usage, responseTime, toolsUsed: result.toolsUsed || [], enhancedWithTools: false, analytics: result.analytics, evaluation: result.evaluation, audio: result.audio, video: result.video, imageOutput: result.imageOutput, };If a provider’s
generate()already ran the 3‑phase structured-output flow and setresult.structuredOutputAchieved, that information is currently discarded, so:
TextGenerationResult.structuredOutputAchievedwill always beundefinedon the direct path.NeuroLink.generate()will then exposestructuredOutputAchievedasundefined/falseeven for successful structured-output runs when MCP is disabled or bypassed.Propagating the flag here keeps behavior consistent across MCP and non‑MCP flows.
Proposed fix: forward
structuredOutputAchievedfrom providersconst responseTime = Date.now() - startTime; logger.debug(`[${functionTag}] All providers failed`, { @@ logger.debug(`[${functionTag}] Provider ${providerName} succeeded`, { responseTime, contentLength: result.content?.length || 0, }); @@ - return { - content: result.content || "", - provider: providerName, - model: result.model, - usage: result.usage, - responseTime, - toolsUsed: result.toolsUsed || [], - enhancedWithTools: false, - analytics: result.analytics, - evaluation: result.evaluation, - audio: result.audio, - video: result.video, - // CRITICAL FIX: Include imageOutput for image generation models - imageOutput: result.imageOutput, - }; + return { + content: result.content || "", + provider: providerName, + model: result.model, + usage: result.usage, + responseTime, + toolsUsed: result.toolsUsed || [], + enhancedWithTools: false, + analytics: result.analytics, + evaluation: result.evaluation, + audio: result.audio, + video: result.video, + // CRITICAL FIX: Include imageOutput for image generation models + imageOutput: result.imageOutput, + // Preserve structured-output success info from BaseProvider.generate() + structuredOutputAchieved: result.structuredOutputAchieved, + };Also applies to: 2650-2664
🤖 Fix all issues with AI agents
In @src/lib/core/baseProvider.ts:
- Around line 547-590: The catch block for Phase 3 (around the call to
generationHandler.executeObjectGeneration in BaseProvider) currently returns
repairResult.json on failure, which may itself be invalid; instead, update the
catch to return the original initialGenerationResult (unmodified) with
structuredOutputAchieved set to false (and avoid injecting repairResult.json
into text), so that on generateObject failure you fall back to the safe Phase 1
output; reference initialGenerationResult, repairResult, and
generationHandler.executeObjectGeneration when making this change.
🧹 Nitpick comments (3)
src/lib/utils/structuredOutput.ts (1)
292-347: HardenattemptRepairaround timeouts and typingThe Phase 2 repair call is doing a raw
generateTextagainst theLanguageModelV1without any timeout or stronger typing:
- No timeout wrapper means a hung repair call can stall the whole structured-output flow, whereas other modules standardize on
withTimeoutfor external calls insrc/lib/**/*.ts.- The return type
z.infer<typeof schema>whereschema: ZodUnknownSchemaeffectively collapses tounknown; if you want type-safe structured results, this should be generic on the actual schema type.Consider:
- Wrapping the
generateTextcall with the sharedwithTimeoututility (taking the repair timeout fromoptions.timeoutor a sensible default).- Making
attemptRepairgeneric, e.g.<TSchema extends ZodUnknownSchema>(..., schema: TSchema, ...): Promise<{ success: boolean; json: z.infer<TSchema> | null }>to preserve the schema’s inferred type through the pipeline.src/lib/types/generateTypes.ts (1)
2-2: Align structured-output result typing and docsA few nits around the new structured-output types:
GenerateResult / TextGenerationResult field semantics (Lines 286-290, 655-658)
structuredOutputAchieved?: boolean;is documented onGenerateResultas “Parsed structured output (if schema provided)”, but the field is actually a status flag, not the parsed data.- Consider tightening the comment to something like: “Whether structured output passed schema validation for this call.”
GenerationResult doc vs type mismatch (Lines 694-705)
- The JSDoc mentions
structuredOutputPhasebut the type only addsstructuredOutputAchieved: boolean. Either addstructuredOutputPhaseto the type or remove it from the comment to avoid confusion.Type-level dependency on
ai.generateText(Lines 2, 703-704)
GenerationResult = Awaited<ReturnType<typeof generateText>> & { ... }pulls in a value import from"ai"in a types file.- If the AI SDK exposes a dedicated result type (e.g.
GenerateTextResultor similar), importing that as atypeand extending it would avoid an unnecessary runtime dependency fromsrc/lib/typesintoai.These are all type/doc polish items; behavior is fine as-is.
Also applies to: 286-290, 655-658, 694-705
src/lib/core/modules/GenerationHandler.ts (1)
150-208: New Phase 3 object generation method.The implementation is well-structured with:
- Clear documentation of purpose and parameters
- Tool context preservation via
addToolContextToMessages- Hardcoded
mode: "tool"andtemperature: 0for deterministic outputTwo considerations:
mode: "tool"- This forces tool-based extraction. Consider whether"auto"might be more flexible for providers that support native JSON mode.Comment on line 206 has a typo: "beforing" should be "before".
📝 Fix typo in comment
- //No need to pass the callback for handling storage, as tool calls and results are already set in-memory beforing calling this function during executeGeneration + // No need to pass the callback for handling storage, as tool calls and results are already set in-memory before calling this function during executeGeneration
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
src/lib/core/baseProvider.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/types/generateTypes.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tstest/unit/structured-output.test.ts
🧰 Additional context used
📓 Path-based instructions (4)
src/lib/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
src/lib/**/*.ts: Use MessageBuilder (src/lib/utils/messageBuilder.ts) as the central component for constructing all messages with multimodal support (text, images, PDFs, CSV)
Wrap async operations with withTimeout utility for timeout handling
Use ErrorFactory for creating typed errors instead of generic Error objects
thinkingLevel option supports "minimal", "low", "medium" (default), and "high" values for extended thinking on supported models (Anthropic Claude, Gemini 2.5+, Gemini 3)
Files:
src/lib/types/generateTypes.tssrc/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
src/lib/types/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
All type definitions must be organized by domain (providers, generation, streaming, MCP, etc.) to avoid circular dependencies
Files:
src/lib/types/generateTypes.ts
src/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Maintain strict TypeScript type safety across all modules with no any types unless justified
Files:
src/lib/types/generateTypes.tssrc/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
src/lib/utils/messageBuilder.ts
📄 CodeRabbit inference engine (CLAUDE.md)
src/lib/utils/messageBuilder.ts: Use FileDetector to automatically detect file types before processing in MessageBuilder
Consider token limits for multimodal content when processing files (images, PDFs, CSV)
Files:
src/lib/utils/messageBuilder.ts
🧠 Learnings (16)
📚 Learning: 2026-01-02T19:49:21.526Z
Learnt from: murdore
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2026-01-02T19:49:21.526Z
Learning: Project standard: Place reusable/shared types under src/lib/types/*.ts; test-only helper types under test/types/*.ts; avoid declaring local types inside source implementation files.
Applied to files:
src/lib/types/generateTypes.tstest/unit/structured-output.test.ts
📚 Learning: 2025-09-17T17:55:15.261Z
Learnt from: RajuSudhar
Repo: juspay/neurolink PR: 173
File: src/lib/index.ts:16-16
Timestamp: 2025-09-17T17:55:15.261Z
Learning: In src/lib/types/providers.ts, ProviderConfig was renamed to AIModelProviderConfig to deduplicate type names, as there was an existing ProviderConfig type that better suited the "ProviderConfig" name. This was an intentional breaking change for better type organization.
Applied to files:
src/lib/types/generateTypes.tssrc/lib/core/baseProvider.ts
📚 Learning: 2025-09-02T13:50:42.770Z
Learnt from: YasmeenOgo
Repo: juspay/neurolink PR: 145
File: src/lib/core/types.ts:0-0
Timestamp: 2025-09-02T13:50:42.770Z
Learning: The APIVersions enum in src/lib/core/types.ts now contains comprehensive API version constants for all major AI providers: Azure OpenAI (latest, stable, legacy), OpenAI (current, beta), Google AI (current, beta), and Anthropic (current). This centralization helps avoid API version drift across the codebase.
Applied to files:
src/lib/types/generateTypes.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/googleAiStudio.ts : Gemini models (AI Studio and Vertex) cannot use tools and JSON schema output simultaneously - design workflows to use either tools OR structured JSON output, not both
Applied to files:
src/lib/types/generateTypes.tstest/unit/structured-output.test.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/**/*.ts : Use MessageBuilder (src/lib/utils/messageBuilder.ts) as the central component for constructing all messages with multimodal support (text, images, PDFs, CSV)
Applied to files:
test/unit/structured-output.test.tssrc/lib/core/modules/Utilities.tssrc/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/transformationUtils.ts : Use transformation utilities (transformToolExecutions, transformAvailableTools, transformParamsForLogging) for data transformation across providers
Applied to files:
src/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
📚 Learning: 2026-01-02T19:49:21.526Z
Learnt from: murdore
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2026-01-02T19:49:21.526Z
Learning: Public API stability: Avoid tightening exported ToolContext; keep sessionId optional in public types unless a major version bump is intended.
Applied to files:
src/lib/core/modules/Utilities.ts
📚 Learning: 2025-12-20T07:04:14.181Z
Learnt from: BoraYaswanthReddy
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2025-12-20T07:04:14.181Z
Learning: In the neurolink repository, even though the ChatMessage `id` field was optional in the TypeScript interface, IDs were already being generated for every conversation message in practice. Making the field required enforces existing behavior rather than introducing a breaking change.
Applied to files:
src/lib/neurolink.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/messageBuilder.ts : Consider token limits for multimodal content when processing files (images, PDFs, CSV)
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/messageBuilder.ts : Use FileDetector to automatically detect file types before processing in MessageBuilder
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2025-09-01T22:58:39.149Z
Learnt from: sudharsan-juspay
Repo: juspay/neurolink PR: 140
File: src/lib/core/types.ts:198-203
Timestamp: 2025-09-01T22:58:39.149Z
Learning: In src/lib/core/types.ts, StreamOptions (imported from streamTypes.js) and StreamingOptions are intentionally different types with different use cases. StreamingOptions is for unified AI requests with multiple provider configurations, while StreamOptions is for individual streaming operations.
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2025-09-24T06:42:06.088Z
Learnt from: amreetkhuntia
Repo: juspay/neurolink PR: 185
File: src/lib/evaluation/contextBuilder.ts:79-85
Timestamp: 2025-09-24T06:42:06.088Z
Learning: In the NeuroLink codebase, using `(options.prompt || [])` pattern for handling potentially undefined prompt arrays is the preferred approach over extracting to a normalized variable when building conversation history in the ContextBuilder class.
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/**/*.ts : thinkingLevel option supports "minimal", "low", "medium" (default), and "high" values for extended thinking on supported models (Anthropic Claude, Gemini 2.5+, Gemini 3)
Applied to files:
src/lib/core/modules/GenerationHandler.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : All new providers must maintain consistency with existing provider interface and support same core operations
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : Extend base provider or implement the provider interface when creating new provider implementations in src/lib/providers/
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : All providers must use dynamic imports in the ProviderRegistry to avoid circular dependencies. Example: `const { GoogleAIStudioProvider } = await import("../providers/googleAiStudio.js");`
Applied to files:
src/lib/core/baseProvider.ts
🧬 Code graph analysis (5)
src/lib/types/generateTypes.ts (3)
src/lib/core/baseProvider.ts (1)
generateText(839-878)src/lib/neurolink.ts (1)
generateText(2091-2107)src/lib/index.ts (1)
generateText(430-436)
src/lib/utils/messageBuilder.ts (1)
src/lib/utils/structuredOutput.ts (1)
createUnifiedSchemaInstructions(131-176)
src/lib/utils/structuredOutput.ts (4)
src/lib/utils/logger.ts (2)
logger(358-401)error(239-241)src/lib/core/baseProvider.ts (2)
isZodSchema(1004-1006)generateText(839-878)src/lib/utils/schemaConversion.ts (1)
convertZodToJsonSchema(141-187)src/lib/types/generateTypes.ts (1)
TextGenerationOptions(439-650)
src/lib/core/modules/GenerationHandler.ts (2)
src/lib/types/generateTypes.ts (1)
GenerationResult(703-705)src/lib/core/modules/Utilities.ts (1)
addToolContextToMessages(457-483)
src/lib/core/baseProvider.ts (5)
src/lib/types/generateTypes.ts (2)
TextGenerationOptions(439-650)GenerationResult(703-705)src/lib/utils/messageBuilder.ts (1)
shouldUseStructuredOutput(327-336)src/lib/types/typeAliases.ts (1)
ZodUnknownSchema(19-19)src/lib/utils/structuredOutput.ts (3)
extractAndParseJSON(41-109)validateAgainstSchema(213-234)attemptRepair(292-348)src/lib/core/modules/Utilities.ts (1)
addToolContextToMessages(457-483)
🔇 Additional comments (18)
test/unit/structured-output.test.ts (1)
324-331: Good adaptation of tests to unified schema instructionsSwitching the multimodal tests to assert on key phrases (
"MANDATORY OUTPUT FORMAT","Your ENTIRE response MUST be a valid JSON object") instead of a full constant string makes the tests resilient to minor formatting tweaks while still tightly validating the new instruction contract.Also applies to: 351-358, 376-383
src/lib/neurolink.ts (1)
2001-2051: Propagation fromgenerateTextInternallooks correct, but relies on all paths setting the flagWiring
structuredOutputAchieved: textResult.structuredOutputAchievedinto the publicGenerateResultis the right shape; just be aware this assumes every internal path that constructs aTextGenerationResult(MCP and direct providers) actually setsstructuredOutputAchievedwhen structured output succeeds. See the direct provider path below, which currently drops the flag.src/lib/utils/messageBuilder.ts (4)
16-17: LGTM! Import reorganization for structured output support.The imports are correctly updated to bring in
createUnifiedSchemaInstructionsfrom the new structured output utilities, replacing the previous staticSTRUCTURED_OUTPUT_INSTRUCTIONS.
323-336: Good addition of JSDoc and export forshouldUseStructuredOutput.Exporting this function enables consistent structured output detection across the codebase (e.g., in
BaseProvider.executeGeneration). The logic correctly gates on both schema presence and output format.
362-364: LGTM! Dynamic schema instructions inbuildMessagesArray.The pattern correctly checks
shouldUseStructuredOutput(options)before appending schema instructions, and the redundantoptions.schemacheck ensures the schema exists before passing tocreateUnifiedSchemaInstructions.
699-701: LGTM! Consistent pattern inbuildMultimodalMessagesArray.The structured output handling mirrors
buildMessagesArray, ensuring multimodal requests also receive schema instructions when appropriate.src/lib/core/modules/Utilities.ts (2)
38-38: LGTM! Type imports for tool context preservation.Importing
CoreMessage,ToolCallPart, andToolResultPartdirectly from theaimodule ensures type compatibility with the Vercel AI SDK.
457-483: Well-structured tool context preservation function.The implementation correctly:
- Appends tool calls as an assistant message with content array
- Wraps each tool result in a separate tool message (matches the Vercel AI SDK's expected CoreMessage format)
- Returns a new array (immutable pattern)
src/lib/core/modules/GenerationHandler.ts (5)
16-24: LGTM! Updated imports for structured output support.The imports correctly bring in
generateObjectfor Phase 3 object generation and the necessary tool-related types for context preservation.
38-39: LGTM! Import of utility function and result type.
addToolContextToMessagesfrom Utilities andGenerationResulttype are correctly imported to support the new structured output flow.
67-78: Updated method documentation and tool gating.The JSDoc clearly explains that structured output orchestration is handled by
BaseProvider.executeGeneration(). TheshouldUseToolslogic correctly respects bothdisableToolsoption and provider capability.Also applies to: 87-87
213-224: LGTM! Updated logging for structured output.The
logGenerationCompletemethod now acceptsGenerationResultand logs thestructuredOutputAchievedflag, providing visibility into whether structured output was successfully obtained.
331-417: LGTM! Enhanced result formatting withstructuredOutputAchieved.The
formatEnhancedResultmethod correctly:
- Accepts
GenerationResulttype- Propagates
structuredOutputAchievedtoEnhancedGenerateResult- Maintains existing logic for content extraction and usage formatting
src/lib/core/baseProvider.ts (5)
28-32: LGTM! Imports for structured output orchestration.The imports correctly bring in all necessary utilities for the 3-phase structured output flow:
extractAndParseJSON,validateAgainstSchema,attemptRepairfrom structuredOutputaddToolContextToMessagesfor tool context preservationshouldUseStructuredOutputfor flow controlGenerationResulttype for return valuesAlso applies to: 39-39, 42-43
418-451: Well-designed routing logic inexecuteGeneration.The method cleanly separates standard generation from structured output orchestration:
- Executes initial generation first (serves as Phase 1 for structured output)
- Routes to
executeStructuredOutputGenerationonly when schema is valid- Returns appropriate
structuredOutputAchievedflag for both paths
506-540: Phase 2: Repair attempt with validation errors.Good implementation that:
- Preserves tool context from Phase 1 for the repair prompt
- Passes validation errors to
attemptRepairfor targeted fixes- Returns successfully repaired JSON with
structuredOutputAchieved: true
595-597: LGTM! Updated method signatures forGenerationResult.Both
logGenerationCompleteandformatEnhancedResultcorrectly acceptGenerationResulttype, propagating thestructuredOutputAchievedflag through the result formatting pipeline.Also applies to: 631-647
453-504: No actionable issue. The implementation correctly handlesnullinput. WhenextractAndParseJSONreturnsnull,validateAgainstSchema(null, schema)safely delegates to Zod'ssafeParse, which is designed to accept any input type. The function returns{ success: false, errors: [...] }rather than throwing an error, and Phase 1 properly checksvalidation.successbefore proceeding. The parameter typeunknownexplicitly allows null, confirming this design is intentional.Likely an incorrect or invalid review comment.
| // ===== PHASE 3: With generateObject ===== | ||
| logger.info( | ||
| `[BaseProvider] Phase 3: Using generateObject for guaranteed success`, | ||
| { | ||
| provider: this.providerName, | ||
| model: this.modelName, | ||
| }, | ||
| ); | ||
|
|
||
| // Use original messages to preserve full conversation context | ||
| // Tool calls and results from Phase 1 are passed separately and will be added by executeObjectGeneration | ||
| try { | ||
| const phase3Result = await this.generationHandler.executeObjectGeneration( | ||
| model, | ||
| messages, | ||
| schema, | ||
| options, | ||
| initialGenerationResult.toolCalls, | ||
| initialGenerationResult.toolResults, | ||
| ); | ||
|
|
||
| // Convert executeObjectGeneration result to generateText format | ||
| return { | ||
| ...initialGenerationResult, | ||
| text: JSON.stringify(phase3Result.object), | ||
| finishReason: phase3Result.finishReason, | ||
| usage: phase3Result.usage, | ||
| warnings: phase3Result.warnings, | ||
| structuredOutputAchieved: true, | ||
| }; | ||
| } catch (error) { | ||
| logger.error(`[BaseProvider] Phase 3: generateObject failed`, { error }); | ||
| return repairResult.json | ||
| ? { | ||
| ...initialGenerationResult, | ||
| text: JSON.stringify(repairResult.json), | ||
| structuredOutputAchieved: false, | ||
| } | ||
| : { | ||
| ...initialGenerationResult, | ||
| structuredOutputAchieved: false, | ||
| }; | ||
| } | ||
| } |
There was a problem hiding this comment.
Phase 3: Fallback to generateObject with robust error handling.
The fallback correctly:
- Uses
generateObjectfor guaranteed schema compliance - Preserves tool context from Phase 1
- Converts the result back to
GenerationResultformat
Error handling concern: When Phase 3 fails (lines 577-589), the fallback uses repairResult.json if available. However, this repairResult may have failed Phase 2 validation checks (including JSON format validation). Using potentially invalid JSON as a fallback introduces risk of malformed output. Return the original initialGenerationResult instead, which provides the unmodified Phase 1 text.
🤖 Prompt for AI Agents
In @src/lib/core/baseProvider.ts around lines 547 - 590, The catch block for
Phase 3 (around the call to generationHandler.executeObjectGeneration in
BaseProvider) currently returns repairResult.json on failure, which may itself
be invalid; instead, update the catch to return the original
initialGenerationResult (unmodified) with structuredOutputAchieved set to false
(and avoid injecting repairResult.json into text), so that on generateObject
failure you fall back to the safe Phase 1 output; reference
initialGenerationResult, repairResult, and
generationHandler.executeObjectGeneration when making this change.
There was a problem hiding this comment.
If the generation fails in all the three times, submitting to the user the repairedJSON (though malformed, better than the original), along with the key structuredOutputAchieved: false, indicating the end user that the expected object generation failed. So that the end user can handle it in his desired way.
There was a problem hiding this comment.
@YaswanthKurapati24, that makes sense! Your approach prioritizes providing the "best effort" output to the end user, even when all phases fail. Returning the repairResult.json (which at least attempted to address validation errors) along with structuredOutputAchieved: false gives the end user:
- The most refined attempt at structured output available
- A clear signal that validation didn't succeed
- Flexibility to decide how to handle the malformed output
This is a valid design choice that favors transparency and user control.
Optional suggestion: Consider adding a code comment in the Phase 3 catch block to document this intentional behavior for future maintainers:
} catch (error) {
logger.error(`[BaseProvider] Phase 3: generateObject failed`, { error });
// Intentional: Return repairResult.json (if available) even though it failed validation,
// as it represents the "best effort" at structured output. The structuredOutputAchieved: false
// flag signals to the end user that validation didn't succeed, allowing them to handle it appropriately.
return repairResult.json
? {
...initialGenerationResult,
text: JSON.stringify(repairResult.json),
structuredOutputAchieved: false,
}
: {
...initialGenerationResult,
structuredOutputAchieved: false,
};
}This would clarify the design intent for anyone reviewing the code later.
✏️ Learnings added
Learnt from: YaswanthKurapati24
Repo: juspay/neurolink PR: 767
File: src/lib/core/baseProvider.ts:547-590
Timestamp: 2026-01-09T07:07:04.204Z
Learning: In the BaseProvider structured output orchestration (src/lib/core/baseProvider.ts), when Phase 3 (generateObject) fails, the code intentionally returns repairResult.json (from Phase 2) even if malformed, with structuredOutputAchieved: false. This "best effort" approach gives end users the most refined attempt at structured output along with a failure signal, allowing them to decide how to handle it.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/googleAiStudio.ts : Gemini models (AI Studio and Vertex) cannot use tools and JSON schema output simultaneously - design workflows to use either tools OR structured JSON output, not both
fc9103f to
57a8d3a
Compare
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 11
🤖 Fix all issues with AI agents
In @src/lib/core/baseProvider.ts:
- Around line 468-527: The repair pass is reusing the original
TextGenerationOptions so tools may be invoked again; create a new options object
for the repair step that sets disableTools: true without mutating the original
(e.g., const repairOptions = { ...options, disableTools: true }) and pass
repairOptions into attemptRepair(...) instead of options to ensure the Phase 2
repair cannot trigger tools.
- Around line 548-589: Phase 3 currently calls
generationHandler.executeObjectGeneration without timeout protection; wrap that
call with the withTimeout utility using options.timeout (and mirror the same
change for Phase 1 where executeGeneration/generateText is called) so async
generation calls are bounded; update the call sites in BaseProvider where
executeObjectGeneration and executeGeneration (generateText) are invoked to use
withTimeout(options.timeout, () =>
this.generationHandler.executeObjectGeneration(...)) (or the project’s
withTimeout signature) and ensure errors from timeout are handled in the
existing catch blocks to preserve repairResult fallback behavior.
In @src/lib/types/generateTypes.ts:
- Around line 694-702: The JSDoc and comments claim a structuredOutputPhase but
the exported types lack it and structuredOutputAchieved is mis-described; update
the exported types (e.g., GenerateResult and GenerateStructuredOutputResult) to
include structuredOutputPhase?: 1 | 2 | 3 and change the comment for
structuredOutputAchieved to “Boolean indicating if schema validation succeeded”
(or, alternatively, remove the phase mention from JSDoc if you prefer not to
expose it) so the JSDoc, inline comments, and type declarations (symbols:
structuredOutputPhase, structuredOutputAchieved, GenerateResult) are consistent.
- Line 2: The file imports the runtime symbol generateText from "ai" just to
compute a return type; remove the value import and replace uses of
ReturnType<typeof generateText> with a type-query like ReturnType<typeof
import("ai").generateText> (or use an import type if preferred) so the module
has no runtime dependency; update all occurrences (including the ones around the
previously noted lines ~703-705) to use the type-query form or type-only import.
In @src/lib/utils/structuredOutput.ts:
- Around line 292-347: The comment "Dynamic import to avoid circular
dependencies" above the createRepairPrompt call in attemptRepair is incorrect—no
dynamic import occurs; either remove the comment or replace it with an accurate
description (e.g., "Prepare repair prompt from response, schema, and errors" or
"Create repair prompt for model") so the comment matches the actual synchronous
call to createRepairPrompt within the attemptRepair function.
- Around line 213-234: In validateAgainstSchema, normalize empty Zod issue paths
so root-level errors don’t produce an empty string; replace the path generation
for each issue with a conditional that uses issue.path.join(".") when
issue.path.length > 0 and a clear root placeholder (e.g., "root" or "(root)")
when issue.path is empty, keeping the message and type as issue.message and
issue.code respectively.
In @test/unit/structured-output.test.ts:
- Around line 351-358: Apply the same robustness checks to the PDF-focused test
as in structured-output.test.ts: find the PDF test that inspects the
assistant/system prompt (look for variables named messages and systemMsg within
that test) and add assertions verifying systemMsg?.content is defined and
contains the key phrases "MANDATORY OUTPUT FORMAT" and "Your ENTIRE response
MUST be a valid JSON object" using expect(...).toBeDefined() and
expect(systemMsg?.content).toContain(...); ensure you use the same
pattern/variable names (messages, systemMsg) as in the existing
structured-output assertions.
- Around line 376-383: Add the same robustness checks to the CSV
structured-output test: in test/unit/structured-output.test.ts locate the CSV
test that builds messages (uses the messages array and systemMsg variable) and
add assertions mirroring the JSON test—ensure systemMsg?.content is defined and
contains the unified-schema key phrases (e.g., "MANDATORY OUTPUT FORMAT" and
"Your ENTIRE response MUST be a valid JSON object") so the CSV test validates
the new unified instructions format.
- Around line 324-331: The assertions on systemMsg.content assume it's a string
and will throw if content becomes multimodal; update the test around systemMsg
(the variable computed via messages.find(...) in structured-output.test.ts) to
first assert the content is a string (e.g., expect(typeof
systemMsg?.content).toBe("string")) or normalize non-string content to a string
(e.g., coerce/join MessageContent[] into a string) before calling the existing
.toContain checks for "MANDATORY OUTPUT FORMAT" and "Your ENTIRE response MUST
be a valid JSON object".
🧹 Nitpick comments (5)
src/lib/utils/structuredOutput.ts (3)
41-109: JSON extraction is best-effort but brace-balancing will misbehave with{/}inside strings.This can incorrectly truncate/extract when the model returns JSON with brace characters inside quoted strings. If this is acceptable, consider at least documenting the limitation; otherwise you’ll want a string-aware scanner (or a small JSON tokenizer) instead of raw brace counting.
131-176: Avoid warning-level logs for non‑Zod schemas (likely common), and consider token-size safeguards.
createUnifiedSchemaInstructions()warns on non-Zod schema; butTextGenerationOptions.schemaalso supportsSchema<unknown>(AI SDK). This may become noisy in normal usage.JSON.stringify(jsonSchema, null, 2)can be huge for complex schemas; consider truncation or a size cap to avoid blowing system-prompt budgets.Proposed adjustment (log level + optional cap)
export function createUnifiedSchemaInstructions(schema: unknown): string { // Reuse existing validation from schemaConversion.ts if (!isZodSchema(schema)) { - logger.warn("[StructuredOutput] Non-Zod schema provided, using fallback"); + logger.debug("[StructuredOutput] Non-Zod schema provided, using fallback"); return FALLBACK_INSTRUCTIONS; } try { // Reuse existing conversion from schemaConversion.ts const jsonSchema = convertZodToJsonSchema(schema as ZodUnknownSchema); + // Optional: protect prompt budgets (tune as needed) + const schemaJson = JSON.stringify(jsonSchema, null, 2); + const schemaJsonCapped = + schemaJson.length > 20_000 ? schemaJson.slice(0, 20_000) + "\n..." : schemaJson; // Build emphatic instruction format let schemaInstruction = "\n\n" + "━".repeat(60) + "\n"; schemaInstruction += "MANDATORY OUTPUT FORMAT\n"; schemaInstruction += "━".repeat(60) + "\n\n"; @@ Your JSON must match this exact schema: -${JSON.stringify(jsonSchema, null, 2)} +${schemaJsonCapped}Also applies to: 178-204
245-280: Repair prompt may exceed token limits; consider truncatingPREVIOUS OUTPUTandREQUIRED SCHEMA.For large objects/schemas, the repair attempt can become counterproductive (model can’t fit the prompt). A simple cap (similar to schema instruction) is usually enough.
src/lib/utils/messageBuilder.ts (2)
327-336: Consider tightening types and clarifying the “schema but nooutput.format” behavior.
output?: { format?: string }could be narrowed toTextGenerationOptions["output"]/ the union ("text" | "structured" | "json") to avoid silently accepting typos.- Current behavior requires both
schemaandoutput.formatto opt in. If you want “schema implies structured output”, this helper is the gate to change.
362-364: Redundant&& options.schemacheck.
shouldUseStructuredOutput(options)already checks!!options.schema, so the extra&& options.schemacan be dropped (unless you’re intentionally narrowing for TS control flow).Also applies to: 699-701
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
src/lib/core/baseProvider.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/types/generateTypes.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tstest/unit/structured-output.test.ts
🧰 Additional context used
📓 Path-based instructions (4)
src/lib/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
src/lib/**/*.ts: Use MessageBuilder (src/lib/utils/messageBuilder.ts) as the central component for constructing all messages with multimodal support (text, images, PDFs, CSV)
Wrap async operations with withTimeout utility for timeout handling
Use ErrorFactory for creating typed errors instead of generic Error objects
thinkingLevel option supports "minimal", "low", "medium" (default), and "high" values for extended thinking on supported models (Anthropic Claude, Gemini 2.5+, Gemini 3)
Files:
src/lib/core/modules/Utilities.tssrc/lib/types/generateTypes.tssrc/lib/neurolink.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
src/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Maintain strict TypeScript type safety across all modules with no any types unless justified
Files:
src/lib/core/modules/Utilities.tssrc/lib/types/generateTypes.tssrc/lib/neurolink.tssrc/lib/utils/messageBuilder.tssrc/lib/utils/structuredOutput.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
src/lib/types/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
All type definitions must be organized by domain (providers, generation, streaming, MCP, etc.) to avoid circular dependencies
Files:
src/lib/types/generateTypes.ts
src/lib/utils/messageBuilder.ts
📄 CodeRabbit inference engine (CLAUDE.md)
src/lib/utils/messageBuilder.ts: Use FileDetector to automatically detect file types before processing in MessageBuilder
Consider token limits for multimodal content when processing files (images, PDFs, CSV)
Files:
src/lib/utils/messageBuilder.ts
🧠 Learnings (17)
📓 Common learnings
Learnt from: YaswanthKurapati24
Repo: juspay/neurolink PR: 767
File: src/lib/core/baseProvider.ts:547-590
Timestamp: 2026-01-09T07:07:04.204Z
Learning: In the BaseProvider structured output orchestration (src/lib/core/baseProvider.ts), when Phase 3 (generateObject) fails, the code intentionally returns repairResult.json (from Phase 2) even if malformed, with structuredOutputAchieved: false. This "best effort" approach gives end users the most refined attempt at structured output along with a failure signal, allowing them to decide how to handle it.
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/**/*.ts : Use MessageBuilder (src/lib/utils/messageBuilder.ts) as the central component for constructing all messages with multimodal support (text, images, PDFs, CSV)
Applied to files:
src/lib/core/modules/Utilities.tstest/unit/structured-output.test.tssrc/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/transformationUtils.ts : Use transformation utilities (transformToolExecutions, transformAvailableTools, transformParamsForLogging) for data transformation across providers
Applied to files:
src/lib/core/modules/Utilities.tssrc/lib/neurolink.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
📚 Learning: 2026-01-02T19:49:21.526Z
Learnt from: murdore
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2026-01-02T19:49:21.526Z
Learning: Public API stability: Avoid tightening exported ToolContext; keep sessionId optional in public types unless a major version bump is intended.
Applied to files:
src/lib/core/modules/Utilities.ts
📚 Learning: 2026-01-02T19:49:21.526Z
Learnt from: murdore
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2026-01-02T19:49:21.526Z
Learning: Project standard: Place reusable/shared types under src/lib/types/*.ts; test-only helper types under test/types/*.ts; avoid declaring local types inside source implementation files.
Applied to files:
src/lib/types/generateTypes.tstest/unit/structured-output.test.ts
📚 Learning: 2025-09-17T17:55:15.261Z
Learnt from: RajuSudhar
Repo: juspay/neurolink PR: 173
File: src/lib/index.ts:16-16
Timestamp: 2025-09-17T17:55:15.261Z
Learning: In src/lib/types/providers.ts, ProviderConfig was renamed to AIModelProviderConfig to deduplicate type names, as there was an existing ProviderConfig type that better suited the "ProviderConfig" name. This was an intentional breaking change for better type organization.
Applied to files:
src/lib/types/generateTypes.tssrc/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/googleAiStudio.ts : Gemini models (AI Studio and Vertex) cannot use tools and JSON schema output simultaneously - design workflows to use either tools OR structured JSON output, not both
Applied to files:
src/lib/types/generateTypes.tstest/unit/structured-output.test.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
📚 Learning: 2026-01-09T07:07:04.204Z
Learnt from: YaswanthKurapati24
Repo: juspay/neurolink PR: 767
File: src/lib/core/baseProvider.ts:547-590
Timestamp: 2026-01-09T07:07:04.204Z
Learning: In the BaseProvider structured output orchestration (src/lib/core/baseProvider.ts), when Phase 3 (generateObject) fails, the code intentionally returns repairResult.json (from Phase 2) even if malformed, with structuredOutputAchieved: false. This "best effort" approach gives end users the most refined attempt at structured output along with a failure signal, allowing them to decide how to handle it.
Applied to files:
src/lib/types/generateTypes.tssrc/lib/neurolink.tssrc/lib/utils/structuredOutput.tssrc/lib/core/modules/GenerationHandler.tssrc/lib/core/baseProvider.ts
📚 Learning: 2025-12-20T07:04:14.181Z
Learnt from: BoraYaswanthReddy
Repo: juspay/neurolink PR: 0
File: :0-0
Timestamp: 2025-12-20T07:04:14.181Z
Learning: In the neurolink repository, even though the ChatMessage `id` field was optional in the TypeScript interface, IDs were already being generated for every conversation message in practice. Making the field required enforces existing behavior rather than introducing a breaking change.
Applied to files:
src/lib/neurolink.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/messageBuilder.ts : Consider token limits for multimodal content when processing files (images, PDFs, CSV)
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/utils/messageBuilder.ts : Use FileDetector to automatically detect file types before processing in MessageBuilder
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2025-09-01T22:58:39.149Z
Learnt from: sudharsan-juspay
Repo: juspay/neurolink PR: 140
File: src/lib/core/types.ts:198-203
Timestamp: 2025-09-01T22:58:39.149Z
Learning: In src/lib/core/types.ts, StreamOptions (imported from streamTypes.js) and StreamingOptions are intentionally different types with different use cases. StreamingOptions is for unified AI requests with multiple provider configurations, while StreamOptions is for individual streaming operations.
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2025-09-24T06:42:06.088Z
Learnt from: amreetkhuntia
Repo: juspay/neurolink PR: 185
File: src/lib/evaluation/contextBuilder.ts:79-85
Timestamp: 2025-09-24T06:42:06.088Z
Learning: In the NeuroLink codebase, using `(options.prompt || [])` pattern for handling potentially undefined prompt arrays is the preferred approach over extracting to a normalized variable when building conversation history in the ContextBuilder class.
Applied to files:
src/lib/utils/messageBuilder.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/**/*.ts : thinkingLevel option supports "minimal", "low", "medium" (default), and "high" values for extended thinking on supported models (Anthropic Claude, Gemini 2.5+, Gemini 3)
Applied to files:
src/lib/core/modules/GenerationHandler.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : Extend base provider or implement the provider interface when creating new provider implementations in src/lib/providers/
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : All new providers must maintain consistency with existing provider interface and support same core operations
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2026-01-03T21:49:09.952Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-03T21:49:09.952Z
Learning: Applies to src/lib/providers/**/*.ts : All providers must use dynamic imports in the ProviderRegistry to avoid circular dependencies. Example: `const { GoogleAIStudioProvider } = await import("../providers/googleAiStudio.js");`
Applied to files:
src/lib/core/baseProvider.ts
🧬 Code graph analysis (5)
src/lib/types/generateTypes.ts (3)
src/lib/core/baseProvider.ts (1)
generateText(839-878)src/lib/neurolink.ts (1)
generateText(2091-2107)src/lib/index.ts (1)
generateText(430-436)
src/lib/utils/messageBuilder.ts (1)
src/lib/utils/structuredOutput.ts (1)
createUnifiedSchemaInstructions(131-176)
src/lib/utils/structuredOutput.ts (5)
src/lib/utils/logger.ts (2)
logger(358-401)error(239-241)src/lib/core/baseProvider.ts (2)
isZodSchema(1004-1006)generateText(839-878)src/lib/utils/schemaConversion.ts (1)
convertZodToJsonSchema(141-187)src/lib/types/generateTypes.ts (1)
TextGenerationOptions(439-650)src/lib/neurolink.ts (1)
generateText(2091-2107)
src/lib/core/modules/GenerationHandler.ts (2)
src/lib/types/generateTypes.ts (2)
TextGenerationOptions(439-650)GenerationResult(703-705)src/lib/core/modules/Utilities.ts (1)
addToolContextToMessages(457-483)
src/lib/core/baseProvider.ts (6)
src/lib/types/generateTypes.ts (2)
TextGenerationOptions(439-650)GenerationResult(703-705)src/lib/types/index.ts (2)
TextGenerationOptions(133-133)ZodUnknownSchema(13-13)src/lib/utils/messageBuilder.ts (1)
shouldUseStructuredOutput(327-336)src/lib/types/typeAliases.ts (1)
ZodUnknownSchema(19-19)src/lib/utils/structuredOutput.ts (3)
extractAndParseJSON(41-109)validateAgainstSchema(213-234)attemptRepair(292-348)src/lib/core/modules/Utilities.ts (1)
addToolContextToMessages(457-483)
🔇 Additional comments (5)
src/lib/neurolink.ts (2)
2001-2052: Propagation ofstructuredOutputAchievedlooks correct; confirm desired default semantics.You’re passing through
textResult.structuredOutputAchievedwithout coercion. If the public contract expects a boolean, consider normalizingundefinedtofalse; if it’s intentionally tri-state, current code is fine.
2521-2539: MCP path correctly forwardsstructuredOutputAchievedfrom provider result.Looks consistent with the non-MCP path.
src/lib/utils/messageBuilder.ts (1)
16-17: Good move centralizing schema instructions viacreateUnifiedSchemaInstructions.This reduces drift between tests and runtime prompt formatting.
src/lib/core/modules/Utilities.ts (1)
38-38: No issue found. TheaddToolContextToMessagesfunction correctly aligns with AI SDK 4.3.19 type definitions:
CoreAssistantMessageacceptscontent: Array<TextPart | ToolCallPart>, so assigningToolCallPart[]is valid.CoreToolMessageacceptscontent: Array<ToolResultPart>, so assigning[part]is correct.src/lib/core/modules/GenerationHandler.ts (1)
24-24: Use the project's sharedZodAnySchematype from src/lib/types/tools.ts instead of directly referencingz.ZodSchema.The
z.ZodSchematype is valid in Zod v3.22.0 and actively used in the codebase (e.g.,src/lib/types/tools.tsexportsZodAnySchema = z.ZodSchema<unknown>). However, per the project standard of placing reusable types undersrc/lib/types/, the schema parameter should use the sharedZodAnySchematype alias for consistency.Proposed diff
-import type { z } from "zod"; +import type { ZodAnySchema } from "../types/tools.js"; @@ async executeObjectGeneration( @@ - schema: z.ZodSchema, + schema: ZodAnySchema,Also applies to: 171-178
Likely an incorrect or invalid review comment.
| private async executeStructuredOutputGeneration( | ||
| model: LanguageModelV1, | ||
| messages: CoreMessage[], | ||
| initialGenerationResult: Awaited<ReturnType<typeof generateText>>, | ||
| schema: ZodUnknownSchema, | ||
| options: TextGenerationOptions, | ||
| ): Promise<GenerationResult> { | ||
| // ===== PHASE 1: Try extracting & validating JSON from initial generation ===== | ||
| logger.debug( | ||
| `[BaseProvider] Phase 1: Attempting JSON extraction from initial generation`, | ||
| { | ||
| provider: this.providerName, | ||
| model: this.modelName, | ||
| }, | ||
| ); | ||
|
|
||
| // Extract JSON (handles markdown fences, brace-balanced, etc.) | ||
| const parsed = extractAndParseJSON(initialGenerationResult.text); | ||
|
|
||
| // Validate against schema | ||
| const validation = validateAgainstSchema(parsed, schema); | ||
|
|
||
| if (validation.success) { | ||
| logger.info( | ||
| `[BaseProvider] Phase 1: Success - JSON extracted and validated`, | ||
| { | ||
| provider: this.providerName, | ||
| model: this.modelName, | ||
| }, | ||
| ); | ||
|
|
||
| return { | ||
| ...initialGenerationResult, | ||
| text: JSON.stringify(parsed), | ||
| structuredOutputAchieved: true, | ||
| }; | ||
| } | ||
|
|
||
| // ===== PHASE 2: Try repair if validation failed ===== | ||
| logger.debug(`[BaseProvider] Phase 2: Attempting repair`, { | ||
| provider: this.providerName, | ||
| model: this.modelName, | ||
| errorCount: validation.errors.length, | ||
| errors: validation.errors, | ||
| }); | ||
|
|
||
| const enhancedMessages = addToolContextToMessages( | ||
| messages, | ||
| initialGenerationResult.toolCalls || [], | ||
| initialGenerationResult.toolResults || [], | ||
| ); | ||
|
|
||
| const repairResult = await attemptRepair( | ||
| model, | ||
| parsed, | ||
| schema, | ||
| options, | ||
| validation.errors, | ||
| enhancedMessages, | ||
| ); |
There was a problem hiding this comment.
Phase 2 repair should force disableTools: true (avoid unintended tool side effects during “repair”).
attemptRepair(..., options, ...) gets the same options as the original generation. If tools are enabled, the “repair” pass can invoke tools again (non-deterministic, extra cost, possible side effects), which defeats the goal of a controlled schema-repair step.
Proposed diff
- const repairResult = await attemptRepair(
+ const repairResult = await attemptRepair(
model,
parsed,
schema,
- options,
+ { ...options, disableTools: true, temperature: options.temperature ?? 0 },
validation.errors,
enhancedMessages,
);🤖 Prompt for AI Agents
In @src/lib/core/baseProvider.ts around lines 468 - 527, The repair pass is
reusing the original TextGenerationOptions so tools may be invoked again; create
a new options object for the repair step that sets disableTools: true without
mutating the original (e.g., const repairOptions = { ...options, disableTools:
true }) and pass repairOptions into attemptRepair(...) instead of options to
ensure the Phase 2 repair cannot trigger tools.
| logger.info( | ||
| `[BaseProvider] Phase 3: Using generateObject for guaranteed success`, | ||
| { | ||
| provider: this.providerName, | ||
| model: this.modelName, | ||
| }, | ||
| ); | ||
|
|
||
| // Use original messages to preserve full conversation context | ||
| // Tool calls and results from Phase 1 are passed separately and will be added by executeObjectGeneration | ||
| try { | ||
| const phase3Result = await this.generationHandler.executeObjectGeneration( | ||
| model, | ||
| messages, | ||
| schema, | ||
| options, | ||
| initialGenerationResult.toolCalls, | ||
| initialGenerationResult.toolResults, | ||
| ); | ||
|
|
||
| // Convert executeObjectGeneration result to generateText format | ||
| return { | ||
| ...initialGenerationResult, | ||
| text: JSON.stringify(phase3Result.object), | ||
| finishReason: phase3Result.finishReason, | ||
| usage: phase3Result.usage, | ||
| warnings: phase3Result.warnings, | ||
| structuredOutputAchieved: true, | ||
| }; | ||
| } catch (error) { | ||
| logger.error(`[BaseProvider] Phase 3: generateObject failed`, { error }); | ||
| return repairResult.json | ||
| ? { | ||
| ...initialGenerationResult, | ||
| text: JSON.stringify(repairResult.json), | ||
| structuredOutputAchieved: false, | ||
| } | ||
| : { | ||
| ...initialGenerationResult, | ||
| structuredOutputAchieved: false, | ||
| }; | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
rg -n --type=ts "withTimeout\b|executeWithTimeout\b|AbortSignal|timeout:" src/lib/core src/lib/utilsRepository: juspay/neurolink
Length of output: 2917
🏁 Script executed:
# Check the Phase 1 and Phase 2 implementations in baseProvider.ts around the Phase 3 section
sed -n '480,590p' src/lib/core/baseProvider.ts | head -200Repository: juspay/neurolink
Length of output: 3240
🏁 Script executed:
# Find executeObjectGeneration implementation
rg -n "executeObjectGeneration" src/lib/core/baseProvider.ts -B 2 -A 15Repository: juspay/neurolink
Length of output: 1238
🏁 Script executed:
# Check how timeout/options are handled in generationHandler
rg -n "class.*GenerationHandler\|executeObjectGeneration\|executeTextGeneration" src/lib/core -A 5Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
# Search for GenerationHandler class and executeObjectGeneration method
fd -t f "\.ts$" src/lib | xargs rg -l "executeObjectGeneration" | head -5Repository: juspay/neurolink
Length of output: 130
🏁 Script executed:
# Find where generationHandler is instantiated or typed
rg -n "generationHandler" src/lib/core/baseProvider.ts | head -20Repository: juspay/neurolink
Length of output: 583
🏁 Script executed:
# Look for GenerationHandler class definition
rg -n "class GenerationHandler\|interface GenerationHandler" src/lib -A 10Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
# Read GenerationHandler.ts to find executeObjectGeneration implementation
cat -n src/lib/core/modules/GenerationHandler.ts | head -150Repository: juspay/neurolink
Length of output: 6277
🏁 Script executed:
# Search for executeObjectGeneration method in GenerationHandler
rg -n "executeObjectGeneration" src/lib/core/modules/GenerationHandler.ts -A 20Repository: juspay/neurolink
Length of output: 923
🏁 Script executed:
# Get the complete executeObjectGeneration method
sed -n '171,250p' src/lib/core/modules/GenerationHandler.tsRepository: juspay/neurolink
Length of output: 2637
Wrap Phase 3 (generateObject) with timeout protection as required by coding guidelines.
Phase 3's executeObjectGeneration calls generateObject directly without timeout wrapping, violating the requirement to "Wrap async operations with withTimeout utility for timeout handling" in src/lib/**/*.ts files. The options.timeout parameter is passed but never utilized. Additionally, Phase 1 (executeGeneration) has the same issue—generateText also lacks explicit timeout protection. Consider wrapping both Phase 1 and Phase 3 calls with withTimeout(options.timeout) to ensure consistent timeout enforcement across all generation phases.
🤖 Prompt for AI Agents
In @src/lib/core/baseProvider.ts around lines 548 - 589, Phase 3 currently calls
generationHandler.executeObjectGeneration without timeout protection; wrap that
call with the withTimeout utility using options.timeout (and mirror the same
change for Phase 1 where executeGeneration/generateText is called) so async
generation calls are bounded; update the call sites in BaseProvider where
executeObjectGeneration and executeGeneration (generateText) are invoked to use
withTimeout(options.timeout, () =>
this.generationHandler.executeObjectGeneration(...)) (or the project’s
withTimeout signature) and ensure errors from timeout are handled in the
existing catch blocks to preserve repairResult fallback behavior.
| return await generateObject({ | ||
| model, | ||
| messages: enhancedMessages, | ||
| schema, | ||
| mode: "tool", // 'auto', 'tool', or 'json' | ||
| temperature: options.temperature ?? 0, | ||
| maxTokens: options.maxTokens, | ||
| experimental_telemetry: this.getTelemetryConfigFn( | ||
| options, | ||
| "generate-object", | ||
| ), | ||
| //No need to pass the callback for handling storage, as tool calls and results are already set in-memory beforing calling this function during executeGeneration | ||
| }); |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Vercel AI SDK ai@4.3.19 generateObject mode="tool" provider support documentation
💡 Result:
Short answer
- ai@4.3.19: generateObject exists but it does not support a mode parameter (e.g. mode="tool" is ignored) and is deprecated in favor of using generateText with output/schema. [1][2][3]
Details / pointers
- Official reference: generateObject API (deprecated; use generateText + output) and parameter list (no mode parameter). [1]
- Known behavior/bugs: issues show generateObject relies on provider model capabilities (structuredOutputs) and that mode: "tool" / mode param is ignored — many users hit failures on providers that don’t natively support structured outputs; recommended workaround is generateText + tool definitions or use provider-specific features. See related GitHub issues. [2][4]
- Package/version: ai@4.3.19 npm page. [3]
References
[1] AI SDK docs — generateObject (API reference / deprecation).
[2] GitHub issue: generateObject has no mode param / mode:"tool" ignored.
[3] npm: ai v4.3.19 package page.
[4] GitHub issues reporting generateObject examples failing across providers.
🏁 Script executed:
# Locate and inspect the GenerationHandler.ts file
cat -n src/lib/core/modules/GenerationHandler.ts | head -220 | tail -40Repository: juspay/neurolink
Length of output: 1692
🏁 Script executed:
# Search for provider implementations in the codebase
fd -t f -e ts "provider" src/lib | head -20Repository: juspay/neurolink
Length of output: 563
🏁 Script executed:
# Check which providers are supported/used
rg "googleAiStudio|vertexAi|gemini" src/lib --type ts -lRepository: juspay/neurolink
Length of output: 1097
🏁 Script executed:
# Check what schema is being passed to generateObject in GenerationHandler
grep -B 10 -A 5 "generateObject" src/lib/core/modules/GenerationHandler.ts | head -30Repository: juspay/neurolink
Length of output: 760
🏁 Script executed:
# Inspect googleAiStudio.ts to understand Gemini's structured output handling
wc -l src/lib/providers/googleAiStudio.tsRepository: juspay/neurolink
Length of output: 100
🏁 Script executed:
# Search for any existing warnings or conflicts about tools + schema in Gemini
rg "tools.*schema|schema.*tools|structured.*output" src/lib/providers/googleAiStudio.ts --type ts -C 3Repository: juspay/neurolink
Length of output: 373
🏁 Script executed:
# Check if the schema variable in GenerationHandler is always defined
grep -B 20 "return await generateObject" src/lib/core/modules/GenerationHandler.ts | grep -E "schema|function|class"Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
# Check if there are guards/conditions around the generateObject call for Gemini
grep -B 30 "return await generateObject" src/lib/core/modules/GenerationHandler.ts | grep -E "if|Gemini|googleAi"Repository: juspay/neurolink
Length of output: 111
🏁 Script executed:
# Search for the full executeObjectGeneration method to understand the context
ast-grep --pattern 'executeObjectGeneration($$$) { $$$ }'Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
# Check the schema parameter - is it always defined?
grep -B 50 "return await generateObject" src/lib/core/modules/GenerationHandler.ts | grep -E "schema|function executeObjectGeneration"Repository: juspay/neurolink
Length of output: 287
🏁 Script executed:
# Look at the full method signature and beginning
sed -n '160,210p' src/lib/core/modules/GenerationHandler.tsRepository: juspay/neurolink
Length of output: 1988
Remove mode: "tool" parameter and add provider guards for schema + tool context.
The mode parameter is not recognized by ai@4.3.19's generateObject and is dead code. More critically, passing both schema and tool context (lines 188–192) to generateObject violates Gemini's current limitation: it cannot simultaneously use tools and JSON schema output. This will cause Phase 3 to fail when a Gemini model is used.
Options:
- Add a guard to route Gemini models away from this method during Phase 3, or
- Remove tool context preservation from this call and rely solely on schema-based output, or
- Wait for Gemini 3 Pro Preview (which will support combining tools + schemas) before using this pattern with Gemini.
Remove the unused mode parameter regardless of which option you choose.
| @@ -1,4 +1,5 @@ | |||
| import type { Tool, Schema } from "ai"; | |||
| import { generateText } from "ai"; | |||
There was a problem hiding this comment.
Avoid runtime-importing generateText in a types file; use a type-query instead.
This currently pulls in a value import (import { generateText } from "ai";) just to form ReturnType<typeof generateText>, which can introduce unwanted runtime deps in a types module.
Proposed diff
import type { Tool, Schema } from "ai";
-import { generateText } from "ai";
+type AIGenerateText = typeof import("ai").generateText;
@@
-export type GenerationResult = Awaited<ReturnType<typeof generateText>> & {
+export type GenerationResult = Awaited<ReturnType<AIGenerateText>> & {
structuredOutputAchieved: boolean;
};Also applies to: 703-705
🤖 Prompt for AI Agents
In @src/lib/types/generateTypes.ts at line 2, The file imports the runtime
symbol generateText from "ai" just to compute a return type; remove the value
import and replace uses of ReturnType<typeof generateText> with a type-query
like ReturnType<typeof import("ai").generateText> (or use an import type if
preferred) so the module has no runtime dependency; update all occurrences
(including the ones around the previously noted lines ~703-705) to use the
type-query form or type-only import.
| /** | ||
| * Structured output generation result type | ||
| * | ||
| * Extends the standard Vercel AI SDK generateText result with structured output tracking. | ||
| * Used internally by the 3-phase structured output orchestration system. | ||
| * | ||
| * @property structuredOutputAchieved - Boolean indicating if schema validation succeeded | ||
| * @property structuredOutputPhase - Which phase succeeded (1=extract, 2=repair, 3=generateObject) | ||
| */ |
There was a problem hiding this comment.
Fix JSDoc mismatch: structuredOutputPhase is documented but not present; boolean field comments are misleading.
- The JSDoc says
@property structuredOutputPhase(Line 701-702) but the type doesn’t expose it. structuredOutputAchieved?: booleanis described as “Parsed structured output” inGenerateResult(Line 289), but it’s only a success flag.
Either add structuredOutputPhase?: 1 | 2 | 3 to the exported types, or remove/update the JSDoc and field comments to match reality.
Also applies to: 703-705, 289-289, 657-657
🤖 Prompt for AI Agents
In @src/lib/types/generateTypes.ts around lines 694 - 702, The JSDoc and
comments claim a structuredOutputPhase but the exported types lack it and
structuredOutputAchieved is mis-described; update the exported types (e.g.,
GenerateResult and GenerateStructuredOutputResult) to include
structuredOutputPhase?: 1 | 2 | 3 and change the comment for
structuredOutputAchieved to “Boolean indicating if schema validation succeeded”
(or, alternatively, remove the phase mention from JSDoc if you prefer not to
expose it) so the JSDoc, inline comments, and type declarations (symbols:
structuredOutputPhase, structuredOutputAchieved, GenerateResult) are consistent.
| export function validateAgainstSchema( | ||
| data: unknown, | ||
| schema: ZodUnknownSchema, | ||
| ): { | ||
| success: boolean; | ||
| errors: Array<{ path: string; message: string; type: string }>; | ||
| } { | ||
| const result = schema.safeParse(data); | ||
|
|
||
| if (result.success) { | ||
| return { success: true, errors: [] }; | ||
| } | ||
|
|
||
| // Format Zod errors for repair with path/message/type | ||
| const errors = result.error.issues.map((issue) => ({ | ||
| path: issue.path.join("."), | ||
| message: issue.message, | ||
| type: issue.code, | ||
| })); | ||
|
|
||
| return { success: false, errors }; | ||
| } |
There was a problem hiding this comment.
Normalize Zod error paths for root-level errors.
issue.path can be empty ([]), which currently becomes "" and then shows up as Field "" in repair prompts.
Proposed fix
const errors = result.error.issues.map((issue) => ({
- path: issue.path.join("."),
+ path: issue.path.length ? issue.path.join(".") : "(root)",
message: issue.message,
type: issue.code,
}));🤖 Prompt for AI Agents
In @src/lib/utils/structuredOutput.ts around lines 213 - 234, In
validateAgainstSchema, normalize empty Zod issue paths so root-level errors
don’t produce an empty string; replace the path generation for each issue with a
conditional that uses issue.path.join(".") when issue.path.length > 0 and a
clear root placeholder (e.g., "root" or "(root)") when issue.path is empty,
keeping the message and type as issue.message and issue.code respectively.
| export async function attemptRepair( | ||
| model: LanguageModelV1, | ||
| response: unknown, | ||
| schema: ZodUnknownSchema, | ||
| options: TextGenerationOptions, | ||
| errors: Array<{ path: string; message: string; type: string }>, | ||
| messages: CoreMessage[], | ||
| ): Promise<{ success: boolean; json: z.infer<typeof schema> | null }> { | ||
| // Dynamic import to avoid circular dependencies | ||
| const repairPrompt = createRepairPrompt(response, schema, errors); | ||
|
|
||
| try { | ||
| logger.debug("[StructuredOutput] Attempting Phase 2 repair", { | ||
| errorCount: errors.length, | ||
| errorPaths: errors.map((e) => e.path), | ||
| }); | ||
|
|
||
| messages.push({ | ||
| role: "user", | ||
| content: repairPrompt.trim(), | ||
| }); | ||
|
|
||
| const result = await generateText({ | ||
| model: model, | ||
| maxTokens: options.maxTokens, | ||
| temperature: 0, | ||
| messages: messages, | ||
| }); | ||
|
|
||
| logger.debug("[StructuredOutput] Repair response received", { | ||
| responseLength: result.text.length, | ||
| }); | ||
|
|
||
| // Extract and validate repaired JSON | ||
| const parsed = extractAndParseJSON(result.text); | ||
|
|
||
| if (parsed !== null) { | ||
| const validation = validateAgainstSchema(parsed, schema); | ||
|
|
||
| if (validation.success) { | ||
| logger.info("[StructuredOutput] Phase 2 repair successful", { | ||
| errorCount: errors.length, | ||
| }); | ||
| return { success: true, json: parsed as z.infer<typeof schema> }; | ||
| } | ||
| } | ||
|
|
||
| logger.debug("[StructuredOutput] Repair extraction/validation failed"); | ||
| return { success: false, json: null }; | ||
| } catch (error) { | ||
| logger.warn("[StructuredOutput] Repair attempt failed", { | ||
| error: error instanceof Error ? error.message : String(error), | ||
| errorCount: errors.length, | ||
| }); | ||
| return { success: false, json: null }; | ||
| } |
There was a problem hiding this comment.
Comment is misleading: “Dynamic import to avoid circular dependencies” isn’t happening.
Line 300-301 says “Dynamic import…”, but it’s just calling createRepairPrompt. Either remove or update the comment.
🤖 Prompt for AI Agents
In @src/lib/utils/structuredOutput.ts around lines 292 - 347, The comment
"Dynamic import to avoid circular dependencies" above the createRepairPrompt
call in attemptRepair is incorrect—no dynamic import occurs; either remove the
comment or replace it with an accurate description (e.g., "Prepare repair prompt
from response, schema, and errors" or "Create repair prompt for model") so the
comment matches the actual synchronous call to createRepairPrompt within the
attemptRepair function.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# First, let's find and examine the structuredOutput.ts file
find . -name "structuredOutput.ts" -type fRepository: juspay/neurolink
Length of output: 95
🏁 Script executed:
#!/bin/bash
# Read the relevant section of the file
if [ -f "src/lib/utils/structuredOutput.ts" ]; then
wc -l src/lib/utils/structuredOutput.ts
fiRepository: juspay/neurolink
Length of output: 97
🏁 Script executed:
#!/bin/bash
# Check if withTimeout and ErrorFactory exist
fd -e ts -e js | head -20
rg "withTimeout|ErrorFactory" --max-count=5Repository: juspay/neurolink
Length of output: 3565
🌐 Web query:
Vercel AI SDK generateText timeout AbortSignal support ai@4.3.19
💡 Result:
Short answer: Yes — generateText in ai@4.3.19 supports both an AbortSignal (abortSignal) and a timeout option (number or { totalMs }) — you can pass an AbortController.signal and/or timeout to cancel/limit the call. [1][2]
Notes / caveats:
- The official API docs state abortSignal is supported and timeout can be a number or { totalMs?: number } and may be used alongside abortSignal. [1]
- There are reported bugs where stop/abort signals or timeouts don't behave as expected in some environments (e.g., Edge runtime / vercel dev and certain provider timeouts). See related GitHub issues. [2][3]
References:
[1] AI SDK generateText reference (abortSignal, timeout).
[2] vercel/ai GitHub issue: useChat stop() abortSignal doesn't stop in edge runtime.
[3] vercel/ai GitHub issue: Headers Timeout Error / generateText timeout.
🏁 Script executed:
#!/bin/bash
# Read the attemptRepair function
sed -n '292,347p' src/lib/utils/structuredOutput.tsRepository: juspay/neurolink
Length of output: 1794
🏁 Script executed:
#!/bin/bash
# Read imports at the top of the file to understand what's available
head -50 src/lib/utils/structuredOutput.tsRepository: juspay/neurolink
Length of output: 1837
🏁 Script executed:
#!/bin/bash
# Check if attemptRepair is used elsewhere and how messages are passed
rg "attemptRepair" src/lib --context=3Repository: juspay/neurolink
Length of output: 1237
🏁 Script executed:
#!/bin/bash
# Check ZodUnknownSchema definition
rg "ZodUnknownSchema" src/lib --context=2Repository: juspay/neurolink
Length of output: 15284
🏁 Script executed:
#!/bin/bash
# Find the exact call to attemptRepair and surrounding context
rg -A 10 -B 5 "const repairResult = await attemptRepair" src/lib/core/baseProvider.tsRepository: juspay/neurolink
Length of output: 462
🏁 Script executed:
#!/bin/bash
# Check if messages array is reused after attemptRepair call in baseProvider
sed -n '547,620p' src/lib/core/baseProvider.tsRepository: juspay/neurolink
Length of output: 2384
🏁 Script executed:
#!/bin/bash
# Check errorHandling.ts to verify withTimeout signature and ErrorFactory.toolTimeout
head -100 src/lib/utils/errorHandling.ts | tail -50Repository: juspay/neurolink
Length of output: 1482
🏁 Script executed:
#!/bin/bash
# Find ErrorFactory.toolTimeout method and withTimeout function signature
rg -A 5 "toolTimeout" src/lib/utils/errorHandling.tsRepository: juspay/neurolink
Length of output: 203
🏁 Script executed:
#!/antml:thinking
Need to see the full withTimeout function and ErrorFactory methods to understand the signature.Repository: juspay/neurolink
Length of output: 104
🏁 Script executed:
#!/bin/bash
# Check the withTimeout import and signature
rg -B 2 -A 8 "export async function withTimeout" src/lib/utils/errorHandling.tsRepository: juspay/neurolink
Length of output: 402
🏁 Script executed:
#!/bin/bash
# Check the full context of the orchestration in baseProvider.ts around the phase calls
sed -n '520,610p' src/lib/core/baseProvider.tsRepository: juspay/neurolink
Length of output: 2732
🏁 Script executed:
#!/bin/bash
# Verify ZodTypeAny type definition to confirm it's used as ZodUnknownSchema
rg "type ZodTypeAny|ZodTypeAny =" src/lib -B 1 -A 1Repository: juspay/neurolink
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Search for ZodTypeAny definition
rg "ZodTypeAny" src/lib/types -B 1 -A 1Repository: juspay/neurolink
Length of output: 541
🏁 Script executed:
#!/bin/bash
# Check the complete context around attemptRepair in baseProvider.ts - lines before the call
sed -n '485,540p' src/lib/core/baseProvider.tsRepository: juspay/neurolink
Length of output: 1561
🏁 Script executed:
#!/bin/bash
# Check addToolContextToMessages to confirm it creates a new array
rg -A 15 "function addToolContextToMessages" src/libRepository: juspay/neurolink
Length of output: 1079
Phase 2 repair: fix the messages mutation, add timeout wrapping, and improve type safety.
The current implementation has three issues:
-
Mutating caller state:
messages.push()mutates the passed array. While Phase 3 uses the originalmessages, mutating function parameters is error-prone and violates immutability conventions. -
Missing timeout protection: The
generateTextcall lacks thewithTimeoutwrapper required by repo guidelines forsrc/lib/**/*.ts. Although the AI SDK supports directtimeoutparameter, the codebase standardizes onwithTimeoutfor consistent error handling viaErrorFactory. -
Type safety: Using
z.infer<typeof schema>whereschemais a runtime value doesn't preserve type information; the generic approach fixes this by propagating the schema type through the function signature.
Proposed fix
-export async function attemptRepair(
+export async function attemptRepair<TSchema extends ZodUnknownSchema>(
model: LanguageModelV1,
response: unknown,
- schema: ZodUnknownSchema,
+ schema: TSchema,
options: TextGenerationOptions,
errors: Array<{ path: string; message: string; type: string }>,
messages: CoreMessage[],
-): Promise<{ success: boolean; json: z.infer<typeof schema> | null }> {
+): Promise<{ success: boolean; json: z.infer<TSchema> | null }> {
const repairPrompt = createRepairPrompt(response, schema, errors);
try {
- messages.push({
- role: "user",
- content: repairPrompt.trim(),
- });
+ const repairMessages: CoreMessage[] = [
+ ...messages,
+ { role: "user", content: repairPrompt.trim() },
+ ];
- const result = await generateText({
- model: model,
- maxTokens: options.maxTokens,
- temperature: 0,
- messages: messages,
- });
+ const { withTimeout, ErrorFactory } = await import("../utils/errorHandling.js");
+ const timeoutMs = typeof options.timeout === "number" ? options.timeout : 30_000;
+ const result = await withTimeout(
+ generateText({
+ model,
+ maxTokens: options.maxTokens,
+ temperature: 0,
+ messages: repairMessages,
+ }),
+ timeoutMs,
+ ErrorFactory.toolTimeout("structured-output-repair", timeoutMs),
+ );
- return { success: true, json: parsed as z.infer<typeof schema> };
+ return { success: true, json: parsed as z.infer<TSchema> };
}
}| const systemMsg = messages.find((m) => m.role === "system"); | ||
| expect(systemMsg?.content).toBeDefined(); | ||
| const { STRUCTURED_OUTPUT_INSTRUCTIONS } = await import( | ||
| "../../src/lib/config/conversationMemory.js" | ||
| // Check for key phrases from the new unified schema instructions format | ||
| expect(systemMsg?.content).toContain("MANDATORY OUTPUT FORMAT"); | ||
| expect(systemMsg?.content).toContain( | ||
| "Your ENTIRE response MUST be a valid JSON object", | ||
| ); | ||
| expect(systemMsg?.content).toContain(STRUCTURED_OUTPUT_INSTRUCTIONS.trim()); | ||
| }); |
There was a problem hiding this comment.
Harden the system-message assertions to avoid assuming content is always a string.
Right now toContain(...) implicitly assumes systemMsg?.content is a string; if the system message ever becomes multimodal/array (or MessageContent[]), these tests will fail in a confusing way.
Proposed tweak
const systemMsg = messages.find((m) => m.role === "system");
expect(systemMsg?.content).toBeDefined();
+ expect(typeof systemMsg?.content).toBe("string");
// Check for key phrases from the new unified schema instructions format
- expect(systemMsg?.content).toContain("MANDATORY OUTPUT FORMAT");
- expect(systemMsg?.content).toContain(
+ expect(systemMsg!.content).toContain("MANDATORY OUTPUT FORMAT");
+ expect(systemMsg!.content).toContain(
"Your ENTIRE response MUST be a valid JSON object",
);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const systemMsg = messages.find((m) => m.role === "system"); | |
| expect(systemMsg?.content).toBeDefined(); | |
| const { STRUCTURED_OUTPUT_INSTRUCTIONS } = await import( | |
| "../../src/lib/config/conversationMemory.js" | |
| // Check for key phrases from the new unified schema instructions format | |
| expect(systemMsg?.content).toContain("MANDATORY OUTPUT FORMAT"); | |
| expect(systemMsg?.content).toContain( | |
| "Your ENTIRE response MUST be a valid JSON object", | |
| ); | |
| expect(systemMsg?.content).toContain(STRUCTURED_OUTPUT_INSTRUCTIONS.trim()); | |
| }); | |
| const systemMsg = messages.find((m) => m.role === "system"); | |
| expect(systemMsg?.content).toBeDefined(); | |
| expect(typeof systemMsg?.content).toBe("string"); | |
| // Check for key phrases from the new unified schema instructions format | |
| expect(systemMsg!.content).toContain("MANDATORY OUTPUT FORMAT"); | |
| expect(systemMsg!.content).toContain( | |
| "Your ENTIRE response MUST be a valid JSON object", | |
| ); | |
| }); |
🤖 Prompt for AI Agents
In @test/unit/structured-output.test.ts around lines 324 - 331, The assertions
on systemMsg.content assume it's a string and will throw if content becomes
multimodal; update the test around systemMsg (the variable computed via
messages.find(...) in structured-output.test.ts) to first assert the content is
a string (e.g., expect(typeof systemMsg?.content).toBe("string")) or normalize
non-string content to a string (e.g., coerce/join MessageContent[] into a
string) before calling the existing .toContain checks for "MANDATORY OUTPUT
FORMAT" and "Your ENTIRE response MUST be a valid JSON object".
| const systemMsg = messages.find((m) => m.role === "system"); | ||
| expect(systemMsg?.content).toBeDefined(); | ||
| const { STRUCTURED_OUTPUT_INSTRUCTIONS } = await import( | ||
| "../../src/lib/config/conversationMemory.js" | ||
| // Check for key phrases from the new unified schema instructions format | ||
| expect(systemMsg?.content).toContain("MANDATORY OUTPUT FORMAT"); | ||
| expect(systemMsg?.content).toContain( | ||
| "Your ENTIRE response MUST be a valid JSON object", | ||
| ); | ||
| expect(systemMsg?.content).toContain(STRUCTURED_OUTPUT_INSTRUCTIONS.trim()); | ||
| }); |
There was a problem hiding this comment.
Same robustness improvement for PDF test. (See prior comment.)
🤖 Prompt for AI Agents
In @test/unit/structured-output.test.ts around lines 351 - 358, Apply the same
robustness checks to the PDF-focused test as in structured-output.test.ts: find
the PDF test that inspects the assistant/system prompt (look for variables named
messages and systemMsg within that test) and add assertions verifying
systemMsg?.content is defined and contains the key phrases "MANDATORY OUTPUT
FORMAT" and "Your ENTIRE response MUST be a valid JSON object" using
expect(...).toBeDefined() and expect(systemMsg?.content).toContain(...); ensure
you use the same pattern/variable names (messages, systemMsg) as in the existing
structured-output assertions.
| const systemMsg = messages.find((m) => m.role === "system"); | ||
| expect(systemMsg?.content).toBeDefined(); | ||
| const { STRUCTURED_OUTPUT_INSTRUCTIONS } = await import( | ||
| "../../src/lib/config/conversationMemory.js" | ||
| // Check for key phrases from the new unified schema instructions format | ||
| expect(systemMsg?.content).toContain("MANDATORY OUTPUT FORMAT"); | ||
| expect(systemMsg?.content).toContain( | ||
| "Your ENTIRE response MUST be a valid JSON object", | ||
| ); | ||
| expect(systemMsg?.content).toContain(STRUCTURED_OUTPUT_INSTRUCTIONS.trim()); | ||
| }); |
There was a problem hiding this comment.
Same robustness improvement for CSV test. (See prior comment.)
🤖 Prompt for AI Agents
In @test/unit/structured-output.test.ts around lines 376 - 383, Add the same
robustness checks to the CSV structured-output test: in
test/unit/structured-output.test.ts locate the CSV test that builds messages
(uses the messages array and systemMsg variable) and add assertions mirroring
the JSON test—ensure systemMsg?.content is defined and contains the
unified-schema key phrases (e.g., "MANDATORY OUTPUT FORMAT" and "Your ENTIRE
response MUST be a valid JSON object") so the CSV test validates the new unified
instructions format.
- Added phases 1, 2 and 3 for ensuring structured output. - Phase 1: Incorporating instructions in the System prompt and output schema expected. - Added a function to parse json from the response string generated by the model, handling the markdown fences. - Phase 2: Validating the response produced in Phase 1 with zod, and retrying the generation with the validation errors and the output generated. - Phase 3: If both the above phases fail, using the generateObject of vercel. - Passing a boolean variable structuredOutputAchieved to let the end user know about the status after handling it in multiple phases.
57a8d3a to
c1ac270
Compare
2 participants
Pull Request
Description
What does this PR do?
A clear and concise description of the changes in this pull request.
DevProof:
Related Issues
Does this PR close any issues?
Fixes #(issue number)
Closes #(issue number)
Relates to #(issue number)
Type of Change
Please select the type of change:
Motivation and Context
Why is this change needed? What problem does it solve?
Provide context for reviewers:
Changes Made
What specific changes were made?
Provide a bullet-point list of the key changes:
Breaking Changes
Does this PR introduce breaking changes?
If yes, describe:
Testing
How has this been tested?
Please describe the tests you ran and their results:
Test Coverage
Manual Testing Steps
Provide steps for manual testing:
Code Quality
Have you followed code quality standards?
Documentation
Have you updated documentation?
Commit Message Format
Does your commit follow semantic commit conventions?
type(scope): descriptionExample:
feat(providers): add support for LiteLLM proxyDependencies
Does this PR add, update, or remove dependencies?
If yes, list dependencies and justification:
Performance Impact
Does this change affect performance?
If applicable, provide benchmark results:
Security Considerations
Are there any security implications?
If applicable, describe:
Deployment Notes
Special deployment instructions?
Screenshots / Videos
If applicable, add screenshots or videos to demonstrate changes:
[Add screenshots or videos here]
Reviewer Checklist
For reviewers:
Additional Notes
Any additional information for reviewers:
[Add any extra context, concerns, or questions here]
Pre-submission Checklist
Before submitting, ensure you have:
pnpm testpnpm buildpnpm run validate:alland all checks passThank you for contributing to NeuroLink!
Summary by CodeRabbit
New Features
Refactor
Tests
✏️ Tip: You can customize this high-level summary in your review settings.