fix(core): re-stamp MODEL_GENERATION span when fallback model takes over#15503
Merged
epinzur merged 2 commits intoApr 20, 2026
Merged
Conversation
Signed-off-by: Anurag Ojha <aojharaj2004@gmail.com>
🦋 Changeset detectedLatest commit: d94661a The changes in this PR will be included in the next version bump. This PR includes changesets to release 21 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
WalkthroughRe-stamps MODEL_GENERATION spans when a fallback model runs after a primary model failure so span attributes (model, provider, name) reflect the actual model that produced the response and downstream exporters receive correct generation attribution. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.changeset/slick-goats-cover.md:
- Line 5: Update the changeset (.changeset/slick-goats-cover.md) to be an
outcome-first, user-focused release note: remove internal implementation
references like "MODEL_GENERATION span" and technical jargon (e.g., "exporters",
"Langfuse"), shorten to one or two sentences that state the user-visible impact
— e.g., that fallback models are now correctly attributed in traces so usage and
cost reporting are accurate — and keep the entry brief and non-technical per the
repo guidelines.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 74653a43-29e2-4080-b854-7487b427cdbb
📒 Files selected for processing (3)
.changeset/slick-goats-cover.mdpackages/core/src/loop/workflows/agentic-execution/llm-execution-step.test.tspackages/core/src/loop/workflows/agentic-execution/llm-execution-step.ts
Signed-off-by: Anurag Ojha <aojharaj2004@gmail.com>
epinzur
approved these changes
Apr 20, 2026
intojhanurag
added a commit
to intojhanurag/mastra
that referenced
this pull request
Apr 21, 2026
…ver (mastra-ai#15503) ## Description Fixes fallback model attribution in observability traces. When an agent's primary model failed and the fallback model served the response, the `MODEL_GENERATION` span kept the primary model's `model` and `provider` attributes, so exporters like Langfuse attributed usage and cost to the wrong (failed) model or dropped them entirely. ## Before <img width="1512" height="982" alt="image" src="https://github.com/user-attachments/assets/28900c25-7403-4f35-ad09-34529247f85f" /> ## After <img width="1512" height="982" alt="image" src="https://github.com/user-attachments/assets/6244ef59-4b7d-4eef-912a-33060433444b" /> ## Related Issue(s) Fixes mastra-ai#13547 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Code refactoring - [ ] Performance improvement - [ ] Test update ## Checklist - [x] I have made corresponding changes to the documentation (if applicable) - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I have addressed all Coderabbit comments on this PR <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## ELI5 Imagine you ask a friend for help, but they can't and a backup friend does it. When someone asks who helped, you should say the backup friend. This PR ensures the system records the backup model when the main model fails, so usage and cost are attributed to the model that actually answered. ## Summary This PR fixes fallback model attribution in observability traces: when an agent's primary model fails and a fallback model serves the response, the MODEL_GENERATION span is re-stamped with the fallback model's modelId and provider so exporters (e.g., Langfuse) attribute usage and cost to the correct model. Key changes: - Implementation (packages/core/src/loop/workflows/agentic-execution/llm-execution-step.ts): inside executeStreamWithFallbackModels, call modelSpanTracker.updateGeneration() when a fallback model (activeFallbackModelIndex > 0) is selected to re-stamp the MODEL_GENERATION span with the active model's name, model, and provider attributes. - Test (packages/core/src/loop/workflows/agentic-execution/llm-execution-step.test.ts): added test "re-stamps MODEL_GENERATION span attributes when a fallback model takes over" which mocks modelSpanTracker, forces the primary model to throw, runs a successful secondary model, and asserts updateGeneration is called with the secondary model's attributes. - Release notes (.changeset/slick-goats-cover.md): added a changeset describing the patch release and observability fix. Design note: - The change reuses the existing update pattern used for processor-driven model swaps, implemented in Mastra core so downstream exporters (Langfuse, Datadog, Arize, otel-exporter) automatically receive corrected attribution. Related: fixes mastra-ai#13547 <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Anurag Ojha <aojharaj2004@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes fallback model attribution in observability traces. When an agent's primary model failed and the fallback model served the response, the
MODEL_GENERATIONspan kept the primarymodel's
modelandproviderattributes, so exporters like Langfuse attributed usage and cost to the wrong (failed) model or dropped them entirely.Before
After
Related Issue(s)
Fixes #13547
Type of Change
Checklist
ELI5
Imagine you ask a friend for help, but they can't and a backup friend does it. When someone asks who helped, you should say the backup friend. This PR ensures the system records the backup model when the main model fails, so usage and cost are attributed to the model that actually answered.
Summary
This PR fixes fallback model attribution in observability traces: when an agent's primary model fails and a fallback model serves the response, the MODEL_GENERATION span is re-stamped with the fallback model's modelId and provider so exporters (e.g., Langfuse) attribute usage and cost to the correct model.
Key changes:
Design note:
Related: fixes #13547