Skip to content

[LiveMetrics] add LogProcessor, refactor DocumentHelper, update tests#43081

Merged
TimothyMothra merged 9 commits intomainfrom
tilee/livemetrics_refactor_document
Apr 17, 2024
Merged

[LiveMetrics] add LogProcessor, refactor DocumentHelper, update tests#43081
TimothyMothra merged 9 commits intomainfrom
tilee/livemetrics_refactor_document

Conversation

@TimothyMothra
Copy link
Copy Markdown

@TimothyMothra TimothyMothra commented Mar 29, 2024

This PR refactors how we translate models to Documents for LiveMetrics.
This includes a bug fix, for how ActivityEvents were converted to Exceptions.
This PR also includes some helper methods that will be used in my next PR to add full support for Logs.

Changes

  • add LiveMetricsLogProcessor
    No change to the public api, this is intentionally internal only.
  • refactor LiveMetricsActivityProcessor
    • moved some of the helper methods to the DocumentHelper.
  • refactor DocumentHelper
    • moved some of the helper methods out of LiveMetricsActivityProcessor. This is important because the LogProcessor will need to share some of these methods.
    • renamed methods to better reflect what these do.
    • added methods to create LogDocuments (aka AI Traces)
    • added try/catch to catch and log exceptions
    • added new tests to validate changes
  • Bug Fix: ActivityProcessor was incorrectly converting all ActivityEvents to Exception. The fix was to first inspect ActivityEvent.Name. This logic was moved to the DocumentHelper.

@TimothyMothra TimothyMothra changed the title [LiveMetrics] refactor DocumentHelper. update tests [LiveMetrics] add LogProcessor, refactor DocumentHelper, update tests Apr 17, 2024
var activitySourceName = $"activitySourceName{uniqueTestId}";
using var activitySource = new ActivitySource(activitySourceName);
// TODO: Replace this ActivityListener with an OpenTelemetry provider.
var listener = new ActivityListener
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TimothyMothra - Why not just use the SDK here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other tests in the live metrics follow the same pattern. As a follow up in later PRs this could be modified to use SDK to keep it simple.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean OTel/InMemoryExporter?

That introduces additional overhead. This test needs only an Activity to test the DocumentHelper conversion methods.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

he meant Sdk.CreateTracerProvider() instead of creating a listener.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This listener is listening to everything, that could be problematic. We should dispose it as well.

@TimothyMothra TimothyMothra merged commit 24ac977 into main Apr 17, 2024
@TimothyMothra TimothyMothra deleted the tilee/livemetrics_refactor_document branch April 17, 2024 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants