fix: make tracing span lookup nil-safe to prevent panic on streaming errors - #4896
Conversation
…eaming error panic A Bedrock streaming error finalizer could hit a nil span entry (or nil receiver) in the tracing path (completeDeferredSpan -> Trace.GetSpan), panicking the process instead of forwarding the stream error. Affected packages: - core/schemas/trace.go Fixes maximhq#3455 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughTrace and Span helpers now return early on nil receivers and invalid inputs, and new unit tests cover the nil-safe behavior for span lookup and mutators. ChangesNil-safety fixes for Trace and Span
Estimated code review effort: 1 (Trivial) | ~5 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Addresses greptile review on PR maximhq#4896. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* origin/dev: (79 commits) chore: add `helm-update` Claude skill for syncing Helm chart with `config.schema.json` (maximhq#5144) fix: web search options to google search mapping in gemini api (maximhq#5139) feat: add `postgresql.external.port` string support and `bifrost.mcp.toolGroups[*].id` to Helm chart (maximhq#5143) fix: parse `SecretVar` JSON with `ref`/`env_var` fields even when `value` is absent (maximhq#5146) Revert "fix: less strict unmarshalling for secret var (maximhq#4723)" (maximhq#5145) fix: max reasoning effort in openai (maximhq#5130) chore: replace manual `helm registry login` steps with `step-security/docker-login-action` (maximhq#5132) fix: support GA transcription-type sessions in POST /v1/realtime/client_secrets (maximhq#5092) community: add Xquik to MCP library (maximhq#5069) fix: warn callers not to truncate the #t= temp-token fragment on MCP inline-auth links (maximhq#5104) chore: build fix in core (maximhq#5129) fix: never persist masked provider key previews (maximhq#5106) Filter out provider-level keys from selector in prompt manager (maximhq#5018) fix: show user popover when `userInfo` exists and include `preferred_username` as display name fallback (maximhq#5098) fix: use `AutoMigrate` and add `runner_id`/`created_by_user_id` columns to sidekiq table migration (maximhq#5085) dds new harness skill and updates based on merged PRs (maximhq#5126) dds new harness skill and updates based on merged PRs (maximhq#5123) Add Trendshift badge to README (maximhq#5124) fix: make tracing span lookup nil-safe to prevent panic on streaming errors (maximhq#4896) Revert "fix: synthesize per-query rerank usage for Bedrock and Vertex (maximhq#4322)" (maximhq#5122) ...
…errors (#4896) * [fix]: core/schemas - make trace/span helpers nil-safe to prevent streaming error panic A Bedrock streaming error finalizer could hit a nil span entry (or nil receiver) in the tracing path (completeDeferredSpan -> Trace.GetSpan), panicking the process instead of forwarding the stream error. Affected packages: - core/schemas/trace.go Fixes #3455 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * [test]: core/schemas - cover empty span ID guard in GetSpan Addresses greptile review on PR #4896. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: nnNyx <64274427+nnNyx@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…errors (maximhq#4896) * [fix]: core/schemas - make trace/span helpers nil-safe to prevent streaming error panic A Bedrock streaming error finalizer could hit a nil span entry (or nil receiver) in the tracing path (completeDeferredSpan -> Trace.GetSpan), panicking the process instead of forwarding the stream error. Affected packages: - core/schemas/trace.go Fixes maximhq#3455 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * [test]: core/schemas - cover empty span ID guard in GetSpan Addresses greptile review on PR maximhq#4896. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: nnNyx <64274427+nnNyx@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…errors (maximhq#4896) * [fix]: core/schemas - make trace/span helpers nil-safe to prevent streaming error panic A Bedrock streaming error finalizer could hit a nil span entry (or nil receiver) in the tracing path (completeDeferredSpan -> Trace.GetSpan), panicking the process instead of forwarding the stream error. Affected packages: - core/schemas/trace.go Fixes maximhq#3455 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * [test]: core/schemas - cover empty span ID guard in GetSpan Addresses greptile review on PR maximhq#4896. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: nnNyx <64274427+nnNyx@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Problem
A Bedrock streaming error can panic the whole Bifrost process while finalizing tracing spans, instead of forwarding the stream error to the caller (SIGSEGV in
schemas.(*Trace).GetSpan, reported in production while streaming Anthropic models via Bedrock).Root cause
The streaming error path finalizes deferred tracing spans and sets span attributes through
Trace.GetSpan. The trace/span helpers assumed the trace receiver and every entry inTrace.Spansare non-nil. When trace cleanup/reuse leaves a nil span entry (or a nil span/trace reaches the finalizer, as visible in the reported stack wherecompleteDeferredSpanreceives a nil argument), the tracing code dereferences nil and crashes before the stream error is delivered.Fix
Make the trace/span helpers defensive no-ops for nil data, keeping tracing best-effort:
Trace.GetSpanreturns nil for nil traces, empty span IDs, and skips nil span entries.Trace.AddSpanignores nil traces/spans.Span.SetAttribute,Span.AddEvent, andSpan.Endignore nil receivers.Tracing edge cases can no longer crash provider streaming error handling; the stream error is forwarded gracefully.
Testing
GetSpan(nil trace, nil span entry, hit/miss) and nil-receiver mutators no-op instead of panicking — pass.CGO_ENABLED=0 go build ./...andgo vetincore/— pass.go test ./schemashas one pre-existing failure (TestResponsesMessageToolCallArguments/real_tool_search_call_frames_from_openai) that reproduces identically on a cleandevcheckout (6484317); unrelated to this change.Fixes #3455