Repository navigation
feat(observability): Add support for custom metadata in Context - #722
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughMade the telemetry handler accessible to subclasses and centralized telemetry config generation in TelemetryHandler.getTelemetryConfig; Context gained an optional metadata field; providers now obtain telemetry config from telemetryHandler. Tests added to validate telemetry metadata behavior. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/lib/core/baseProvider.ts (1)
1144-1152: Reverse the metadata spread order to protect built-in telemetry fields.The current spread order allows user-provided
context.metadatato overwrite critical built-in fields likeprovider,model,operationType, etc. If a user passes metadata with conflicting keys (e.g.,{ metadata: { provider: "custom" } }), it will corrupt telemetry data and make observability unreliable.🔎 Proposed fix: Spread custom metadata first, then add built-in fields
- const metadata: Record<string, string | number | boolean> = { + const metadata: Record<string, string | number | boolean> = { + ...(context?.metadata || {}), provider: this.providerName, model: this.modelName, toolsEnabled: !options.disableTools, neurolink: true, operationType, originalProvider: this.providerName, - ...(context?.metadata || {}), };This ensures built-in fields always contain the correct values and cannot be accidentally or intentionally overwritten by user-provided metadata.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/lib/core/baseProvider.tssrc/lib/types/common.ts
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Maintain strict TypeScript type safety across all modules with comprehensive type definitions organized by domain to avoid circular dependencies
Use ErrorFactory for creating typed errors throughout the application
Wrap async operations with withTimeout utility for timeout handling
Files:
src/lib/core/baseProvider.tssrc/lib/types/common.ts
**/types/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Type definitions must be organized by domain (providers, generation, streaming, MCP, etc.) to avoid circular dependencies
Files:
src/lib/types/common.ts
🧠 Learnings (2)
📚 Learning: 2025-09-17T17:55:15.261Z
Learnt from: RajuSudhar
Repo: juspay/neurolink PR: 173
File: src/lib/index.ts:16-16
Timestamp: 2025-09-17T17:55:15.261Z
Learning: In src/lib/types/providers.ts, ProviderConfig was renamed to AIModelProviderConfig to deduplicate type names, as there was an existing ProviderConfig type that better suited the "ProviderConfig" name. This was an intentional breaking change for better type organization.
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2025-12-10T12:24:51.147Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-10T12:24:51.147Z
Learning: Applies to **/types/index.ts : Add new provider names to the AIProviderName enum in src/lib/types/index.ts when adding a new provider
Applied to files:
src/lib/core/baseProvider.ts
⏰ Context from checks skipped due to timeout of 900000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Agent
- GitHub Check: CodeQL analysis (javascript-typescript)
🔇 Additional comments (1)
src/lib/types/common.ts (1)
230-230: LGTM! Type definition is clean and appropriate for telemetry use.The optional metadata field with primitive value types (string | number | boolean) is well-suited for telemetry systems like Langfuse. The type constraint ensures serialization safety.
There was a problem hiding this comment.
Pull request overview
This pull request adds support for custom metadata in the Context type, enabling users to attach arbitrary key-value pairs to telemetry traces via the context parameter. The metadata is automatically merged into Langfuse telemetry traces across all AI providers.
Key Changes:
- Added optional
metadatafield to Context type supporting string, number, and boolean values - Integrated context metadata merging in BaseProvider's telemetry configuration
- Enables passing custom tracking data (e.g., department, feature flags, environment) through the context
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/lib/types/common.ts | Adds optional metadata field to Context type as Record<string, string | number | boolean> |
| src/lib/core/baseProvider.ts | Spreads context metadata into telemetry metadata object for Langfuse trace enrichment |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
f269ed8 to
321a5c4
Compare
|
@coderabbitai fullreview |
|
@adarshba, I'll perform a comprehensive review of all the changes in this PR. ✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/lib/core/modules/TelemetryHandler.ts (1)
186-194: Document or verify metadata override behavior.Custom
context.metadatais spread before the hardcoded system fields (provider,model,toolsEnabled, etc.), meaning system metadata will override any custom metadata with the same keys. This prevents users from overriding system fields, which is good for consistency and security, but should be documented to avoid confusion.Consider adding a JSDoc note clarifying which metadata keys are reserved and cannot be overridden by users.
📝 Suggested documentation addition
/** * Create telemetry configuration for Vercel AI SDK experimental_telemetry * This enables automatic OpenTelemetry tracing when telemetry is enabled + * + * Note: System metadata fields (provider, model, toolsEnabled, neurolink, operationType, originalProvider) + * are reserved and will override any custom metadata with the same keys. */ getTelemetryConfig(
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
src/lib/core/baseProvider.tssrc/lib/core/modules/TelemetryHandler.tssrc/lib/providers/anthropic.tssrc/lib/providers/azureOpenai.tssrc/lib/providers/googleAiStudio.tssrc/lib/providers/googleVertex.tssrc/lib/providers/openAI.tssrc/lib/types/common.tstest/unit/telemetry-config-metadata.test.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.{ts,tsx}: Maintain strict TypeScript type safety across all modules with comprehensive type definitions organized by domain to avoid circular dependencies
Use ErrorFactory for creating typed errors throughout the application
Wrap async operations with withTimeout utility for timeout handling
Files:
src/lib/types/common.tssrc/lib/providers/googleAiStudio.tssrc/lib/providers/openAI.tstest/unit/telemetry-config-metadata.test.tssrc/lib/providers/googleVertex.tssrc/lib/core/baseProvider.tssrc/lib/providers/azureOpenai.tssrc/lib/core/modules/TelemetryHandler.tssrc/lib/providers/anthropic.ts
**/types/**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Type definitions must be organized by domain (providers, generation, streaming, MCP, etc.) to avoid circular dependencies
Files:
src/lib/types/common.ts
**/providers/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
Providers must extend a base provider or implement the provider interface and register in ProviderRegistry.registerAllProviders() with provider name, factory function, default model, and aliases
Files:
src/lib/providers/googleAiStudio.tssrc/lib/providers/openAI.tssrc/lib/providers/googleVertex.tssrc/lib/providers/azureOpenai.tssrc/lib/providers/anthropic.ts
🧠 Learnings (6)
📚 Learning: 2025-09-17T17:55:15.261Z
Learnt from: RajuSudhar
Repo: juspay/neurolink PR: 173
File: src/lib/index.ts:16-16
Timestamp: 2025-09-17T17:55:15.261Z
Learning: In src/lib/types/providers.ts, ProviderConfig was renamed to AIModelProviderConfig to deduplicate type names, as there was an existing ProviderConfig type that better suited the "ProviderConfig" name. This was an intentional breaking change for better type organization.
Applied to files:
src/lib/providers/googleAiStudio.tssrc/lib/providers/openAI.tssrc/lib/providers/googleVertex.tssrc/lib/core/baseProvider.tssrc/lib/providers/azureOpenai.tssrc/lib/core/modules/TelemetryHandler.tssrc/lib/providers/anthropic.ts
📚 Learning: 2025-09-01T22:58:39.149Z
Learnt from: sudharsan-juspay
Repo: juspay/neurolink PR: 140
File: src/lib/core/types.ts:198-203
Timestamp: 2025-09-01T22:58:39.149Z
Learning: In src/lib/core/types.ts, StreamOptions (imported from streamTypes.js) and StreamingOptions are intentionally different types with different use cases. StreamingOptions is for unified AI requests with multiple provider configurations, while StreamOptions is for individual streaming operations.
Applied to files:
src/lib/providers/googleAiStudio.tssrc/lib/providers/openAI.tssrc/lib/providers/googleVertex.tssrc/lib/core/baseProvider.tssrc/lib/providers/azureOpenai.tssrc/lib/core/modules/TelemetryHandler.tssrc/lib/providers/anthropic.ts
📚 Learning: 2025-12-10T12:24:51.147Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-10T12:24:51.147Z
Learning: Applies to **/utils/transformationUtils.ts : Use transformToolExecutions(), transformAvailableTools(), and transformParamsForLogging() utilities from transformation utilities for consistent data transformations
Applied to files:
src/lib/providers/openAI.tssrc/lib/core/baseProvider.tssrc/lib/providers/azureOpenai.ts
📚 Learning: 2025-12-10T12:24:51.147Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-10T12:24:51.147Z
Learning: Applies to **/types/index.ts : Add new provider names to the AIProviderName enum in src/lib/types/index.ts when adding a new provider
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2025-12-10T12:24:51.147Z
Learnt from: CR
Repo: juspay/neurolink PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-10T12:24:51.147Z
Learning: Applies to **/utils/messageBuilder.ts : Message construction must be handled through MessageBuilder in src/lib/utils/messageBuilder.ts, which handles text, images, PDFs, and CSV files with provider-specific adapters
Applied to files:
src/lib/core/baseProvider.ts
📚 Learning: 2025-09-02T13:50:42.770Z
Learnt from: YasmeenOgo
Repo: juspay/neurolink PR: 145
File: src/lib/core/types.ts:0-0
Timestamp: 2025-09-02T13:50:42.770Z
Learning: The APIVersions enum in src/lib/core/types.ts now contains comprehensive API version constants for all major AI providers: Azure OpenAI (latest, stable, legacy), OpenAI (current, beta), Google AI (current, beta), and Anthropic (current). This centralization helps avoid API version drift across the codebase.
Applied to files:
src/lib/providers/azureOpenai.tssrc/lib/providers/anthropic.ts
🧬 Code graph analysis (3)
test/unit/telemetry-config-metadata.test.ts (1)
src/lib/core/modules/TelemetryHandler.ts (1)
TelemetryHandler(38-268)
src/lib/core/baseProvider.ts (1)
src/lib/core/modules/TelemetryHandler.ts (1)
TelemetryHandler(38-268)
src/lib/providers/azureOpenai.ts (1)
tools/automation/buildSystem.js (1)
options(432-437)
🪛 GitHub Check: test (20)
test/unit/telemetry-config-metadata.test.ts
[warning] 259-259:
Unexpected any. Specify a different type
[warning] 225-225:
Unexpected any. Specify a different type
[warning] 224-224:
Unexpected any. Specify a different type
[warning] 223-223:
Unexpected any. Specify a different type
[warning] 207-207:
Unexpected any. Specify a different type
[warning] 9-9:
Unexpected any. Specify a different type
🔇 Additional comments (11)
src/lib/types/common.ts (1)
226-231: LGTM! Well-designed metadata field addition.The optional
metadatafield with primitive type restrictions (string | number | boolean) is appropriate for telemetry use cases where values need to be serialized. This maintains backward compatibility while enabling custom trace metadata for Langfuse integration.src/lib/providers/googleAiStudio.ts (1)
190-191: LGTM! Telemetry config routing updated correctly.The change to use
this.telemetryHandler.getTelemetryConfig(options)centralizes telemetry configuration logic in the TelemetryHandler, improving maintainability and consistency across providers.src/lib/providers/openAI.ts (1)
378-379: LGTM! Consistent telemetry config implementation.Matches the pattern used across all providers. The centralized telemetry configuration ensures consistent metadata handling.
src/lib/providers/azureOpenai.ts (1)
155-156: LGTM! Telemetry configuration updated consistently.src/lib/providers/anthropic.ts (1)
172-173: LGTM! Consistent telemetry integration.src/lib/providers/googleVertex.ts (1)
917-918: LGTM! Telemetry configuration aligned with other providers.src/lib/core/baseProvider.ts (2)
65-65: LGTM! Access level change enables proper telemetry integration.Changing
telemetryHandlerfrom private to protected allows provider subclasses to access telemetry configuration directly, which is necessary for the centralized telemetry approach. This is a well-reasoned architectural improvement.
88-91: LGTM! Clean delegation to TelemetryHandler.The refactored callback properly delegates to
this.telemetryHandler.getTelemetryConfig, centralizing telemetry configuration logic. This eliminates duplication and ensures consistent metadata handling across all generation operations.test/unit/telemetry-config-metadata.test.ts (1)
1-304: Excellent test coverage for the metadata feature!The test suite comprehensively validates:
- Custom metadata merging into telemetry configuration
- Function ID determination logic (traceName → userId → "guest")
- Edge cases (empty, undefined, missing context)
- SessionId handling with type validation
- System field protection from user overrides
- Behavior when telemetry is disabled
- Cross-provider compatibility
This thorough coverage ensures the feature works correctly across all scenarios.
Note on static analysis warnings: The
anytypes flagged on lines 9, 207, 223-225, and 259 are in test mocks, which is acceptable for testing purposes. If desired, you could define a properMockNeuroLinktype, but it's not critical for test quality.src/lib/core/modules/TelemetryHandler.ts (2)
163-165: Documentation improvement looks good.The updated documentation clearly explains the integration with Vercel AI SDK's experimental_telemetry and the automatic OpenTelemetry tracing benefit.
166-175: No evidence of a method rename exists in the codebase. The methodgetStreamTelemetryConfigdoes not appear anywhere in the repository, including in version control history or search results. The current methodgetTelemetryConfigis consistently used across all provider implementations (Anthropic, Google AI Studio, OpenAI, Azure OpenAI, Google Vertex), base provider, and is thoroughly tested. Since the old method name has no references anywhere, this is not a breaking change, nor is it a rename—it appears to be the original method name used throughout.Likely an incorrect or invalid review comment.
- Added metadata field to Context type to allow users to pass custom metadata - Merged context metadata into telemetry metadata for Langfuse traces - Enables users to attach arbitrary key-value pairs to traces via context
321a5c4 to
995c4b9
Compare
|
🎉 This PR is included in version 8.25.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Pull Request
Description
Type of Change
Changes Made
AI Provider Impact
Component Impact
Testing
Test Environment
Performance Impact
Checklist
Additional Notes
Note: provider, model, toolsEnabled, neurolink cannot be overridden
Summary by CodeRabbit
New Features
Refactor
Tests
✏️ Tip: You can customize this high-level summary in your review settings.