Refactor componentattribute and move into service#13948
Conversation
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (86.12%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #13948 +/- ##
==========================================
- Coverage 91.61% 91.57% -0.05%
==========================================
Files 655 658 +3
Lines 42793 42824 +31
==========================================
+ Hits 39205 39216 +11
- Misses 2765 2784 +19
- Partials 823 824 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Regarding code coverage:
CodeCov doesn't take these uses into account as they are in different modules. |
|
@axw I can't tag you as reviewer but I would appreciate a review from you, notably on how well this fits into the "bring your own SDK" initiative. |
axw
left a comment
There was a problem hiding this comment.
Thanks @jade-guiton-dd, only minor concerns from me.
jmacd
left a comment
There was a problem hiding this comment.
Appreciate the PR desciption!
|
/easycla |
0f3b0c9
…or/otel to 141 + Remove deprecated components + Bump k8s min version to 1.21 (#4111) The clickhouse exporter supports TLS settings similar to otlp: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/945e5a71ef31793ff3280b28c1425086ea5332b6/exporter/clickhouseexporter/README.md#tls Some users need this to connect to clickhouse, adding them as options in the destination here This adds: * `insecure_skip_verify` * `ca_file` (using the k8sconfig interface to mount the secret as a file, similar to how the GCP exporter supports application default credentials) The direct string fields (such as CAPem, CertPem, KeyPem) aren't yet supported in the clickhouse exporter, so it has to be a mounted file. See open-telemetry/opentelemetry-collector-contrib#43911 (comment) --- To do this, it required bumping the collector/otel deps to 136 when TLS config support was added to clickhouse. This required the following changes: This actually needs collector v0.136.0 for these settings from open-telemetry/opentelemetry-collector-contrib#42581 (open-telemetry/opentelemetry-collector-contrib@d9769f7) Also needs to remove loki exporter (removed in 131) for 136 🙃 open-telemetry/opentelemetry-collector-contrib#41413, see https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.130.0/exporter/lokiexporter#deprecation-notice it's replaced with just otlp. The only destination that actually looks like it's using the loki exporter is OpsVerse As well as the opencensus exporter, removed in 133 upstream by open-telemetry/opentelemetry-collector-contrib#42239 Also routing processor open-telemetry/opentelemetry-collector-contrib#36616 See previous attempt #3669 (reverted in #3734) --- Then, it turns out that 136 was bugged and did not have full support for TLS settings like `insecure_skip_verify`. This was fixed in 141, which required the following extra changes: Actually needs collector v141 due to this bug in clickhouse not handling all tls settings: open-telemetry/opentelemetry-collector-contrib#43911 fixed in open-telemetry/opentelemetry-collector-contrib#44093 Remove deprecated carbon exporter support (unmaintained upstream) open-telemetry/opentelemetry-collector-contrib#44532 another upstream breaking change giving go mod trouble open-telemetry/opentelemetry-collector#13948 configgrpc update: open-telemetry/opentelemetry-collector#13996 and now metadata.yaml metrics require stablity levels open-telemetry/opentelemetry-collector#13756 ``` Error: failed loading /app/collector/receivers/odigosebpfreceiver/metadata.yaml: decoding failed due to the following error(s): 'telemetry.metrics[ebpf_memory_pressure_wait_time_total]' missing required field: `stability` 'telemetry.metrics[ebpf_total_bytes_read]' missing required field: `stability` 'telemetry.metrics[ebpf_lost_samples]' missing required field: `stability` Error: failed loading /app/collector/receivers/odigosebpfreceiver/metadata.yaml: decoding failed due to the following error(s): 'telemetry.metrics[ebpf_memory_pressure_wait_time_total]' missing required field: `stability` 'telemetry.metrics[ebpf_total_bytes_read]' missing required field: `stability` 'telemetry.metrics[ebpf_lost_samples]' missing required field: `stability` Error: metadata.yaml ordering check failed: [telemetry metrics] keys are not sorted: [odigos_log_data_size odigos_metric_data_size odigos_trace_data_size odigos_accepted_spans odigos_accepted_metric_points odigos_accepted_log_records] Error: metadata.yaml ordering check failed: [telemetry metrics] keys are not sorted: [odigos_log_data_size odigos_metric_data_size odigos_trace_data_size odigos_accepted_spans odigos_accepted_metric_points odigos_accepted_log_records] ``` This bump also required adding the `endpointslices` permission to the odiglet service account for the data-collection collector --- Finally, endpointslices was not GA in k8s 1.20. This PR bumps our minimum supported k8s version to 1.21. Enterprise update in odigos-io/odigos-enterprise#2117
Description
This PR splits the code handling component attribute injection, currently in
internal/telemetryandinternal/telemetry/componentattribute, into three parts:internal/telemetry;service/telemetry/otelconftelemetry, since this "copying" behavior is specific to the otelconf-based telemetry provider;service/internal/componentattribute.The main goals of this split are:
componentattributeinto the serviceI also rewrote a lot of the attribute injection code along the way:
Instead of adding an unexported field in
TelemetrySettingsto store the current set of injected attributes, this set is now stored inside the provider wrappers (which was already partly the case). This allowed me to move theTelemetrySettingsdefinition back intocomponent, removing the dependency it had oncomponentattribute.I completely changed the approach for injection in logs. Instead of a chain of custom core wrappers with a
withAttributeSetmethod, injected attributes are now set through the standardzapcore.Core.Withmethod. By introducing a customObjectMarshaler, we can make injected attributes act like regular Zap fields, while also allowing special handling in the otelzap wrapper core, which will set them as instrumentation scope attributes instead of log record attributes.If I'm not mistaken, this PR should have no externally-visible changes, so no changelog should be necessary. (Except maybe the
_ struct{}inTelemetrySettings, but there was no changelog when we added those elsewhere)Link to tracking issue
Updates #13842
Testing
The large amounts of splitting and refactoring led me to abandon the existing tests, as they would be too hard to adapt. I wrote new tests for the code in
service/telemetry/otelconftelemetryandservice/internal/componentattribute, which hopefully should fill that gap appropriately.