SPM is broken by OTEL's new version

[yurishkuro]

The PR #5968 is trying to upgrade to v0.109 and the SPM e2e test is consistently failing. It could be related to these breaking changes :

spanmetricsconnector:

• Improve consistency between metrics generated by spanmetricsconnector. Added traces.span.metrics as default namespace (#33227, #32818)
• Default namespace for the generated metrics is traces.span.metrics now. | The deprecated metrics are: calls, duration and events. | The feature flag connector.spanmetrics.legacyLatencyMetricNames was added to revert the behavior.

Tasks:

• understand what changes in OTEL and how it affects SPM in Jaeger
• make corresponding code changes to compensate
• provide backwards compatibility following Jaeger's policy https://github.com/jaegertracing/jaeger?tab=readme-ov-file#version-compatibility-guarantees

[FlamingSaint]

@yurishkuro I can take this up.

[yurishkuro]

@FlamingSaint sounds good

[shinigami-777]

hi @yurishkuro
There seems to be 2 solutions to this issue

1. To modify the SPM e2e test to check for metrics under the new namespace.
2. Use the Legacy metric names via the feature flag connector.spanmetrics.legacyLatencyMetricNames .
   The latter seems easy to implement but the first one seems like a more future-proof solution.

[yurishkuro]

We'd need to do both, for backwards compatibility for people running older versions of OTEL collector

[mahadzaryab1]

@yurishkuro i've tried out an approach to fix this - would you be able to review #6007?