receiver/prometheus: roundtrip Prometheus->Pdata direct conversion without OpenCensus by odeke-em · Pull Request #3741 · open-telemetry/opentelemetry-collector

odeke-em · 2021-07-30T01:34:37Z

Wire up and use the direct Prometheus->Pdata conversion end to end.
With this change the receiver will no longer need OpenCensus.

This change will involve more follow-ups that just migrate over the tests,
because we don't want a super bloated/massive PR.

Fixes #3691
Depends on PR #3694
Depends on PR #3695

linux-foundation-easycla · 2021-07-31T00:20:19Z

The committers are authorized under a signed CLA.

✅ Emmanuel T Odeke (4b3c2be, c54089d, de99984, a847925, 5d85d10)

tigrannajaryan · 2021-08-03T08:57:58Z

Please rebase and fix conflicts.

bogdandrutu · 2021-08-05T20:07:49Z

ok  	go.opentelemetry.io/collector/receiver/otlpreceiver/internal/trace	0.058s
panic: interface conversion: v1.isMetric_Data is *v1.Metric_Sum, not *v1.Metric_Summary

goroutine 156 [running]:
go.opentelemetry.io/collector/model/pdata.Metric.Summary(...)
	D:/a/opentelemetry-collector/opentelemetry-collector/model/pdata/metrics.go:207
go.opentelemetry.io/collector/receiver/prometheusreceiver/internal.adjustStartTimestampPdata(0x40790ccccccccccd, 0xc000929708)
	D:/a/opentelemetry-collector/opentelemetry-collector/receiver/prometheusreceiver/internal/otlp_transaction.go:206 +0x1f0
go.opentelemetry.io/collector/receiver/prometheusreceiver/internal.(*transactionPdata).Commit(0xc0006328c0, 0x66fecc, 0xed89d97fb)
	D:/a/opentelemetry-collector/opentelemetry-collector/receiver/prometheusreceiver/internal/otlp_transaction.go:178 +0x1c5
github.com/prometheus/prometheus/scrape.(*scrapeLoop).scrapeAndReport.func1(0xc00001dce8, 0xc00001dcf8, 0xc00077f440)
	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1195 +0x50
github.com/prometheus/prometheus/scrape.(*scrapeLoop).scrapeAndReport(0xc00077f440, 0x3b9aca00, 0x3b9aca00, 0x0, 0x0, 0x0, 0x66fecc, 0xed89d97fb, 0x400bca0, 0x0, ...)
	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1262 +0xb58
github.com/prometheus/prometheus/scrape.(*scrapeLoop).run(0xc00077f440, 0x3b9aca00, 0x3b9aca00, 0x0)
	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1148 +0x37e
created by github.com/prometheus/prometheus/scrape.(*scrapePool).sync
	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:564 +0xa0e

odeke-em · 2021-08-05T20:09:51Z


ok  	go.opentelemetry.io/collector/receiver/otlpreceiver/internal/trace	0.058s

panic: interface conversion: v1.isMetric_Data is *v1.Metric_Sum, not *v1.Metric_Summary



goroutine 156 [running]:

go.opentelemetry.io/collector/model/pdata.Metric.Summary(...)

	D:/a/opentelemetry-collector/opentelemetry-collector/model/pdata/metrics.go:207

go.opentelemetry.io/collector/receiver/prometheusreceiver/internal.adjustStartTimestampPdata(0x40790ccccccccccd, 0xc000929708)

	D:/a/opentelemetry-collector/opentelemetry-collector/receiver/prometheusreceiver/internal/otlp_transaction.go:206 +0x1f0

go.opentelemetry.io/collector/receiver/prometheusreceiver/internal.(*transactionPdata).Commit(0xc0006328c0, 0x66fecc, 0xed89d97fb)

	D:/a/opentelemetry-collector/opentelemetry-collector/receiver/prometheusreceiver/internal/otlp_transaction.go:178 +0x1c5

github.com/prometheus/prometheus/scrape.(*scrapeLoop).scrapeAndReport.func1(0xc00001dce8, 0xc00001dcf8, 0xc00077f440)

	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1195 +0x50

github.com/prometheus/prometheus/scrape.(*scrapeLoop).scrapeAndReport(0xc00077f440, 0x3b9aca00, 0x3b9aca00, 0x0, 0x0, 0x0, 0x66fecc, 0xed89d97fb, 0x400bca0, 0x0, ...)

	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1262 +0xb58

github.com/prometheus/prometheus/scrape.(*scrapeLoop).run(0xc00077f440, 0x3b9aca00, 0x3b9aca00, 0x0)

	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:1148 +0x37e

created by github.com/prometheus/prometheus/scrape.(*scrapePool).sync

	C:/Users/runneradmin/go/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20210621150501-ff58416a0b02/scrape/scrape.go:564 +0xa0e

Thanks @bogdandrutu! Indeed, I should perhaps mark this PR as a draft because I have to translate the entire test suite, it's basically consumed most of my time for the past week. I'll actually mail small parts of it progressively.

This change wires together the logic to convert the pdata converters into a method ToMetricPdata that appends converted metrics. The next change or two, will then hook everything up directly and will allow us to delete all the prior code!! Updates #3137 Updates PR open-telemetry#3674 Requires PR open-telemetry#3694 Updates open-telemetry#3691

Wire up and use the direct Prometheus->Pdata conversion end to end. With this change the receiver will no longer need OpenCensus. This change will involve more follow-ups that just migrate over the tests, because we don't want a super bloated/massive PR. Fixes open-telemetry#3691 Depends on PR open-telemetry#3694 Depends on PR open-telemetry#3695

github-actions · 2021-08-25T03:15:55Z

This PR was marked stale due to lack of activity. It will be closed in 7 days.

bogdandrutu · 2021-08-25T03:59:39Z

@odeke-em unfortunately this needs to be moved to contrib. Sorry and thanks :)

odeke-em requested review from Aneurysm9 and alolita as code owners July 30, 2021 01:34

odeke-em requested a review from a team July 30, 2021 01:34

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch 3 times, most recently from 73b5f3c to 6cbd772 Compare July 30, 2021 08:41

Aneurysm9 mentioned this pull request Jul 30, 2021

receiver/prometheus: add ToMetricPdata method #3695

Merged

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch from 6cbd772 to d9343be Compare July 31, 2021 00:20

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch 4 times, most recently from 59fbe48 to e4163c9 Compare August 5, 2021 08:19

odeke-em marked this pull request as draft August 5, 2021 20:10

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch from e4163c9 to 8ac86ff Compare August 12, 2021 01:22

odeke-em added 11 commits August 16, 2021 17:18

Migrate some tests

e680c67

Migrate tests to use pdata

346b9e1

Migrate more tests to use pdata

ef0115b

Adjust and delete more old/legacy code

f2bbeab

Delete old metrics_adjuster and translate cumulative tests

e7be1ea

Translate tests for summary

6e8b3b1

Delete more stale old pdata code

641fcb4

Update adjustStartTimestampPdata

00fd557

Translate top level tests to use pdata

34e6396

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch from 0a0d196 to 34e6396 Compare August 17, 2021 00:18

Translate time in parity with old code

3cc2119

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch from f30e2ec to 3cc2119 Compare August 17, 2021 01:54

Port assertUp tests to support TestExternalLabels

9c7274b

odeke-em force-pushed the receiver-prometheus-wireUp-roundTrip branch from 621a931 to 9c7274b Compare August 17, 2021 04:28

odeke-em added 2 commits August 16, 2021 23:29

Port over more comparison functions

7ea53e6

Port over compare* test functions

651ced7

bogdandrutu mentioned this pull request Aug 19, 2021

Prometheus Receiver metric type fixes to match Prometheus functionality #3853

Closed

gracewehner mentioned this pull request Aug 24, 2021

Prometheus Receiver metric type fixes to match Prometheus functionality open-telemetry/opentelemetry-collector-contrib#4865

Merged

github-actions Bot added the Stale label Aug 25, 2021

bogdandrutu removed the Stale label Aug 25, 2021

bogdandrutu closed this Aug 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

receiver/prometheus: roundtrip Prometheus->Pdata direct conversion without OpenCensus#3741

receiver/prometheus: roundtrip Prometheus->Pdata direct conversion without OpenCensus#3741
odeke-em wants to merge 15 commits into
open-telemetry:mainfrom
orijtech:receiver-prometheus-wireUp-roundTrip

odeke-em commented Jul 30, 2021

Uh oh!

linux-foundation-easycla Bot commented Jul 31, 2021 •

edited

Loading

Uh oh!

tigrannajaryan commented Aug 3, 2021

Uh oh!

bogdandrutu commented Aug 5, 2021

Uh oh!

odeke-em commented Aug 5, 2021

Uh oh!

github-actions Bot commented Aug 25, 2021

Uh oh!

bogdandrutu commented Aug 25, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

odeke-em commented Jul 30, 2021

Uh oh!

linux-foundation-easycla Bot commented Jul 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tigrannajaryan commented Aug 3, 2021

Uh oh!

bogdandrutu commented Aug 5, 2021

Uh oh!

odeke-em commented Aug 5, 2021

Uh oh!

github-actions Bot commented Aug 25, 2021

Uh oh!

bogdandrutu commented Aug 25, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

linux-foundation-easycla Bot commented Jul 31, 2021 •

edited

Loading