Skip to content

fix(byoo): cap chunked log exporter batches - #478

Merged
kristinapathak merged 1 commit into
mainfrom
fix/byoo-logchunk-exporter-batch
Jul 27, 2026
Merged

fix(byoo): cap chunked log exporter batches#478
kristinapathak merged 1 commit into
mainfrom
fix/byoo-logchunk-exporter-batch

Conversation

@kristinapathak

@kristinapathak kristinapathak commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

TL;DR

  • Prevents chunked logs from being recombined into oversized export requests by capping exporterhelper batches at 900,000 bytes.
  • Releases the fix as BYOO OTel Collector 0.157.1.

Additional Details

  • Uses byte-sized sending_queue.batch with a 200 ms flush and 100 KB of envelope headroom below the 1 MB receiver limit.
  • Applies the default only when log chunking actively splits records; dry-run delivery remains unchanged.
  • Keeps batching scoped to the log exporter. Existing exporterhelper overrides still apply afterward.

For the Reviewer

  • Start with the batching helper and log-pipeline wiring in internal/otelconfig/render.go.
  • The Kratos regression test covers the failing exporter path and the dry-run test guards unchanged behavior.

For QA

  • go test ./...
  • make update-examples produced no diff.
  • make validate-otelconfig
  • Validated the exact Kratos sending_queue.batch shape with the pinned 0.157.0 collector.
  • BYOO version-gate tests passed.

No documentation changes are needed because this is an automatic safety default.

Issues

Relates to #273

Checklist

  • I am familiar with the Contributing Guidelines.
  • I have signed off my commits for Developer Certificate of Origin (DCO) compliance.
  • Tests cover the updated behavior.
  • The documentation is up to date.

Summary by CodeRabbit

  • New Features

    • Improved log delivery by batching data based on byte size, with configured flush timing and queue limits.
    • Log batching is enabled for active log exports while remaining disabled during dry runs.
  • Bug Fixes

    • Updated the BYO OpenTelemetry collector to version 0.157.1.
    • Improved default exporter configuration for Kratos log destinations.

Configure exporterhelper byte batching at 900,000 bytes when log chunking is active, leaving space for request envelope overhead. Preserve dry-run behavior and bump the BYOO collector release to 0.157.1.

Signed-off-by: Kristina Pathak <kpathak@nvidia.com>
@kristinapathak
kristinapathak requested a review from a team as a code owner July 27, 2026 17:56
@kristinapathak
kristinapathak requested a review from estroz July 27, 2026 17:56
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 82f16295-6ede-49e5-9e15-832451ee4915

📥 Commits

Reviewing files that changed from the base of the PR and between 5733325 and 933d8d7.

📒 Files selected for processing (3)
  • src/compute-plane-services/byoo-otel-collector/VERSION
  • src/compute-plane-services/byoo-otel-collector/internal/otelconfig/render.go
  • src/compute-plane-services/byoo-otel-collector/internal/otelconfig/render_test.go

📝 Walkthrough

Walkthrough

The BYO OpenTelemetry collector adds default byte-based batching for chunked log exporters outside dry-run mode, updates test coverage for Kratos log export configuration, and increments the collector version to 0.157.1.

Changes

Log exporter batching

Layer / File(s) Summary
Batching defaults and pipeline gating
src/compute-plane-services/byoo-otel-collector/internal/otelconfig/render.go
Adds default flush and byte-size limits, applies them to an exporter’s sending_queue, and skips the configuration during dry-run mode.
Exporter wiring tests and version update
src/compute-plane-services/byoo-otel-collector/internal/otelconfig/render_test.go, src/compute-plane-services/byoo-otel-collector/VERSION
Tests Kratos OTLP HTTP exporter batching defaults and updates the collector version to 0.157.1.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • NVIDIA/nvcf#194: Related OTEL log-chunking resolution and sending_queue batch configuration work.

Suggested reviewers: estroz

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title follows Conventional Commits and accurately describes the primary fix to cap chunked log exporter batches.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/byoo-logchunk-exporter-batch

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands.

@kristinapathak
kristinapathak added this pull request to the merge queue Jul 27, 2026
Merged via the queue into main with commit 799a329 Jul 27, 2026
20 checks passed
@kristinapathak
kristinapathak deleted the fix/byoo-logchunk-exporter-batch branch July 27, 2026 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants