feat: add retry layer for push metrics exporters

[rohan-b99]

Add RetryMetricExporter, which retries up to 3 times with exponential backoff to the apollo metrics and otlp named exporters.

---

Checklist

Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.

• PR description explains the motivation for the change and relevant context for reviewing
• PR description links appropriate GitHub/Jira tickets (creating when necessary)
• Changeset is included for user-facing changes
• Changes are compatible¹
• Documentation² completed
• Performance impact assessed and acceptable
• Metrics and logs are added³ and documented
• Tests added and passing⁴
  • Unit tests
  • Integration tests
  • Manual tests, as necessary

Notes

Footnotes

1. It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this. ↩

2. Configuration is an important part of many changes. Where applicable please try to document configuration examples. ↩

3. A lot of (if not most) features benefit from built-in observability and debug-level logs. Please read this guidance on metrics best-practices. ↩

4. Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions. ↩

[apollo-librarian]

✅ Docs preview has no changes

The preview was not built because there were no changes.

Build ID: 911ae5a7a132b5fa6fefbf00
Build Logs: View logs

---

✅ AI Style Review — No Changes Detected

No MDX files were changed in this pull request.

Review Log: View detailed log

This review is AI-generated. Please use common sense when accepting these suggestions, as they may not always be accurate or appropriate for your specific context.

[conwuegb]

It mostly looks good, I just have a few questions.

[conwuegb]

Out of curiosity, what does "exponential backoff" mean here? Is it referring to exponentially longer wait times in between attempts? And if so, why?

[rohan-b99]

That's right - its a backoff strategy designed to prevent overloading the service with excessive requests if its already at capacity. I checked the OTEL docs to see if it had any recommendations here (originally I didn't think to check) and interestingly it does recommend using exponential backoff with jitter https://opentelemetry.io/docs/specs/otel/protocol/exporter/#retry. I'll go ahead and add the jitter aspect for consistency

[carodewig]

I'm curious how this dovetails (or differs from) the retry mechanism already present in ApolloExporter::submit_report - does that already retry the export, so now there's double retries involved? Or are there now two main retry mechanisms - one for ApolloOtlpExporter and one for ApolloExporter?

This area of the codebases generally confuses me (ie the difference between ApolloExporter and ApolloOtlpExporter) so may not be relevant - but wanted to ask to help with my own understanding!

[rohan-b99]

I'm curious how this dovetails (or differs from) the retry mechanism already present in ApolloExporter::submit_report - does that already retry the export, so now there's double retries involved? Or are there now two main retry mechanisms - one for ApolloOtlpExporter and one for ApolloExporter?

This area of the codebases generally confuses me (ie the difference between ApolloExporter and ApolloOtlpExporter) so may not be relevant - but wanted to ask to help with my own understanding!

@carodewig I just double checked this as most of this is new to me as well - they do sent different data, ApolloExporter sends our proprietary protobuf-encoded Report struct, whereas ApolloOtlpExporter is for actual OTLP metrics. The backoff logic I see in ApolloExporter seems to deal with a case where studio actually responds to the router telling it to not send reports for a while, whereas the retry logic in this PR should deal with any scenarios where the request to an OTEL collector failed.

[conwuegb]

LGTM