Skip to content

feat: add kafka metrics exporter#3158

Merged
julianocosta89 merged 5 commits into
open-telemetry:mainfrom
Red-GV:add-kafka-exporter
Apr 3, 2026
Merged

feat: add kafka metrics exporter#3158
julianocosta89 merged 5 commits into
open-telemetry:mainfrom
Red-GV:add-kafka-exporter

Conversation

@Red-GV
Copy link
Copy Markdown
Contributor

@Red-GV Red-GV commented Mar 25, 2026

Changes

This PR adds the Kafka metrics exporter to bring in more Prometheus metrics from Kafka.

Merge Requirements

For new features contributions, please make sure you have completed the following
essential items:

  • CHANGELOG.md updated to document new feature additions
  • Appropriate documentation updates in the docs
  • Appropriate Helm chart updates in the helm-charts

Maintainers will not merge until the above have been completed. If you're unsure
which docs need to be changed ping the
@open-telemetry/demo-approvers.

@github-actions github-actions Bot added the helm-update-required Requires an update to the Helm chart when released label Mar 25, 2026
@Red-GV
Copy link
Copy Markdown
Contributor Author

Red-GV commented Mar 25, 2026

Original Request: #3139
Helm Chart PR: open-telemetry/opentelemetry-helm-charts#2125

@Red-GV Red-GV force-pushed the add-kafka-exporter branch from 53f1b93 to e251856 Compare March 25, 2026 19:48
@Red-GV Red-GV marked this pull request as ready for review March 25, 2026 19:53
@Red-GV Red-GV requested a review from a team as a code owner March 25, 2026 19:53
Copy link
Copy Markdown
Member

@cyrille-leclerc cyrille-leclerc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your proposal.
Please review question:

@Red-GV Red-GV force-pushed the add-kafka-exporter branch 3 times, most recently from e0a9153 to ac87f9a Compare March 30, 2026 16:09
@Red-GV
Copy link
Copy Markdown
Contributor Author

Red-GV commented Mar 30, 2026

@cyrille-leclerc After some testing, I've gone with the kafkametrics receiver that you pointed me too (thanks for that!). I am in the middle of updating the Helm charts as well to match.

@Red-GV Red-GV force-pushed the add-kafka-exporter branch from ac87f9a to 0b804d8 Compare March 30, 2026 16:11
Red-GV added 4 commits March 31, 2026 15:31
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
@Red-GV Red-GV force-pushed the add-kafka-exporter branch from 0b804d8 to 8815c72 Compare March 31, 2026 19:31
@cyrille-leclerc
Copy link
Copy Markdown
Member

Thanks! I'll test ASAP

@julianocosta89 julianocosta89 enabled auto-merge April 3, 2026 10:50
@julianocosta89 julianocosta89 dismissed cyrille-leclerc’s stale review April 3, 2026 10:51

PR was updated to use Collector kafkametrics receiver

@julianocosta89 julianocosta89 added this pull request to the merge queue Apr 3, 2026
Merged via the queue into open-telemetry:main with commit c72dbfe Apr 3, 2026
37 checks passed
@Red-GV Red-GV deleted the add-kafka-exporter branch April 3, 2026 13:02
@cyrille-leclerc
Copy link
Copy Markdown
Member

It doesn't work on my environment

image

OTelCol error message

2026-04-03T14:57:17.436416250Z 2026-04-03T14:57:17.435Z	error	scraperhelper@v0.146.1/obs_metrics.go:61	Error scraping metrics	{"resource": {"service.instance.id": "ad48bad7-0f36-4a00-a7d6-2519a640bbb7", "service.name": "otelcol-contrib", "service.version": "0.146.1"}, "otelcol.component.id": "kafkametrics", "otelcol.component.kind": "receiver", "otelcol.signal": "metrics", "scraper": "kafkametrics", "error": "franz-go: ListTopics failed: unable to dial: dial tcp :9092: connect: connection refused"}
2026-04-03T14:57:17.436437625Z go.opentelemetry.io/collector/scraper/scraperhelper.wrapObsMetrics.func1
2026-04-03T14:57:17.436440250Z 	go.opentelemetry.io/collector/scraper/scraperhelper@v0.146.1/obs_metrics.go:61
2026-04-03T14:57:17.436441458Z go.opentelemetry.io/collector/scraper.ScrapeMetricsFunc.ScrapeMetrics
2026-04-03T14:57:17.436442667Z 	go.opentelemetry.io/collector/scraper@v0.146.1/metrics.go:24
2026-04-03T14:57:17.436443625Z go.opentelemetry.io/collector/scraper/scraperhelper.scrapeMetrics
2026-04-03T14:57:17.436444583Z 	go.opentelemetry.io/collector/scraper/scraperhelper@v0.146.1/controller.go:167
2026-04-03T14:57:17.436445542Z go.opentelemetry.io/collector/scraper/scraperhelper.NewMetricsController.func1
2026-04-03T14:57:17.436446458Z 	go.opentelemetry.io/collector/scraper/scraperhelper@v0.146.1/controller.go:139
2026-04-03T14:57:17.436447375Z go.opentelemetry.io/collector/scraper/scraperhelper/internal/controller.(*Controller[...]).startScraping.func1
2026-04-03T14:57:17.436448417Z 	go.opentelemetry.io/collector/scraper/scraperhelper@v0.146.1/internal/controller/controller.go:117
2026-04-03T14:57:17.436449417Z sync.(*WaitGroup).Go.func1
2026-04-03T14:57:17.436450250Z 	sync/waitgroup.go:258
2026-04-03T14:57:17.436616167Z 

@cyrille-leclerc
Copy link
Copy Markdown
Member

I understand the bug, let me push a fix

fali007 pushed a commit to fali007/opentelemetry-demo that referenced this pull request May 31, 2026
* feat: add kafka metrics exporter

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* chore: update lint

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* chore: update CHANGELOG

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

* feat: switch to using kafkametricsreceiver

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>

---------

Signed-off-by: Gerard Vanloo <gerard.vanloo@ibm.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

helm-update-required Requires an update to the Helm chart when released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants