Skip to content

Add Console Activity Json Exporter #4548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

Add Console Activity Json Exporter #4548

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a27b181
Add Console Activity Json Exporter
brunomartinspro Jun 6, 2023
c7c6e25
Used System.Text.Json instead of Newtonsoft.Json
brunomartinspro Jun 6, 2023
3979688
Remove unused dependecy
brunomartinspro Jun 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions Directory.Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
</PropertyGroup>

<!-- prod packages -->
<ItemGroup>
<PackageVersion Include="Google.Protobuf" Version="[3.19.4,4.0)" />
Expand All @@ -26,11 +25,10 @@
<PackageVersion Include="StyleCop.Analyzers" Version="[1.2.0-beta.435,2.0)" />
<PackageVersion Include="System.Diagnostics.DiagnosticSource" Version="7.0.0" />
<PackageVersion Include="System.Reflection.Emit.Lightweight" Version="4.7.0" />
<PackageVersion Include="System.Text.Encodings.Web" Version="4.7.2" />
<PackageVersion Include="System.Text.Json" Version="4.7.2" />
<PackageVersion Include="System.Text.Encodings.Web" Version="7.0.0" />
Copy link
Member

@reyang reyang Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, indent

<PackageVersion Include="System.Text.Json" Version="7.0.2" />
Copy link
Member

@reyang reyang Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the impact of this version bump? How many breaking changes would this introduce?

E.g. https://blog.georgekosmidis.net/net-6-a-guide-for-the-high-impact-breaking-changes.html

<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.5.4" />
</ItemGroup>

<!-- non-prod packages -->
<!-- 'net7.0' is the default `TargetFramework`. Use `VersionOverride` in the project to override the package versions from a different `TargetFramework` -->
<ItemGroup>
Expand Down
6 changes: 6 additions & 0 deletions src/OpenTelemetry.Exporter.Console/.publicApi/netstandard2.0/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
OpenTelemetry.Exporter.ConsoleActivityJsonExporter
OpenTelemetry.Exporter.ConsoleActivityJsonExporter.ConsoleActivityJsonExporter(OpenTelemetry.Exporter.ConsoleExporterOptions options) -> void
override OpenTelemetry.Exporter.ConsoleActivityJsonExporter.Export(in OpenTelemetry.Batch<System.Diagnostics.Activity> batch) -> OpenTelemetry.ExportResult
static OpenTelemetry.Trace.ConsoleExporterHelperExtensions.AddConsoleJsonExporter(this OpenTelemetry.Trace.TracerProviderBuilder builder) -> OpenTelemetry.Trace.TracerProviderBuilder
static OpenTelemetry.Trace.ConsoleExporterHelperExtensions.AddConsoleJsonExporter(this OpenTelemetry.Trace.TracerProviderBuilder builder, string name, System.Action<OpenTelemetry.Exporter.ConsoleExporterOptions> configure) -> OpenTelemetry.Trace.TracerProviderBuilder
static OpenTelemetry.Trace.ConsoleExporterHelperExtensions.AddConsoleJsonExporter(this OpenTelemetry.Trace.TracerProviderBuilder builder, System.Action<OpenTelemetry.Exporter.ConsoleExporterOptions> configure) -> OpenTelemetry.Trace.TracerProviderBuilder
160 changes: 160 additions & 0 deletions src/OpenTelemetry.Exporter.Console/ConsoleActivityJsonExporter.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
using System.Diagnostics;
using System.Text.Json;
using System.Text.Json.Serialization;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

namespace OpenTelemetry.Exporter
{
public class ConsoleActivityJsonExporter : ConsoleExporter<Activity>
Copy link
Contributor

@utpilla utpilla Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brunomartinspro What's the motivation behind adding this new exporter to the repo? ConsoleExporter is a simple lightweight exporter meant for quick and naive testing/ troubleshooting for the developers. It was designed to be simple.

Copy link
Member

@cijothomas cijothomas Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we just need an option to existing exporter to write json output

Copy link
Contributor

@utpilla utpilla Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cijothomas Why do we want to add this to the existing exporter though? Users are allowed to create their own custom exporters and use them.

Copy link
Member

@cijothomas cijothomas Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its good to support a more standard/predictable output in the consoleexporter, without forcing everyone to author their own exporter.
The predictable/standard output could be the https://github.com/open-telemetry/opentelemetry-proto/blob/main/docs/specification.md#json-protobuf-encoding OR could be the Prometheus exposition format for Metrics (there were issues asking for these earlier (this capability existed long back, but got removed), can't find it with my quick search)

Copy link
Contributor

@utpilla utpilla Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its good to support a more standard/predictable output in the consoleexporter, without forcing everyone to author their own exporter.

I definitely see value in standardizing the ConsoleExporter output. If we decide to go forward with this though, I think the implementation should meet certain requirements:

  • We want to keep ConsoleExporter simple, so I don't think we should make this an optional feature. Instead, we should simply update the existing behavior to output data in the standardized format
  • ConsoleExporter should not take any additional dependencies or bump the version of existing dependencies that might cause the user to run into version conflict issues with their application code

Copy link
Member

@cijothomas cijothomas Jun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with this.
It would also be good to add disclaimer that the output format could change (it has changed many times in the past) and people should not take dependency on it. (i.e use it for learning purposes, but not for production purposes to have telemetry output to stdout, and have some agent parse it to store telemetry)

{
public ConsoleActivityJsonExporter(ConsoleExporterOptions options)
: base(options)
{
}

public override ExportResult Export(in Batch<Activity> batch)
{
foreach (var activity in batch)
{
dynamic activityBuilder = new System.Dynamic.ExpandoObject();


activityBuilder.TraceId = activity.TraceId.ToString();
activityBuilder.SpanId = activity.SpanId.ToString();
activityBuilder.ActivityTraceFlags = activity.ActivityTraceFlags;
activityBuilder.ActivitySourceName = activity.Source.Name;
activityBuilder.DisplayName = activity.DisplayName;
activityBuilder.Kind = activity.Kind;
activityBuilder.StartTime = activity.StartTimeUtc.ToString("yyyy-MM-ddTHH:mm:ss.fffffffZ");
activityBuilder.Duration = activity.Duration;

if (!string.IsNullOrEmpty(activity.TraceStateString))
{
activityBuilder.TraceState = activity.TraceStateString;
}

if (activity.ParentSpanId != default)
{
activityBuilder.ParentSpanId = activity.ParentSpanId.ToString();
}

var statusCode = string.Empty;
var statusDesc = string.Empty;

if (activity.TagObjects.Any())
{
foreach (ref readonly var tag in activity.EnumerateTagObjects())
{
if (tag.Key == SpanAttributeConstants.StatusCodeKey)
{
statusCode = tag.Value as string;
continue;
}

if (tag.Key == SpanAttributeConstants.StatusDescriptionKey)
{
statusDesc = tag.Value as string;
continue;
}

if (ConsoleTagTransformer.Instance.TryTransformTag(tag, out var result))
{
activityBuilder.Tags = result;
}
}
}

if (activity.Status != ActivityStatusCode.Unset)
{
activityBuilder.StatusCode = activity.Status;

if (!string.IsNullOrEmpty(activity.StatusDescription))
{
activityBuilder.StatusDescription = activity.StatusDescription;
}
}
else if (!string.IsNullOrEmpty(statusCode))
{
activityBuilder.StatusCode = statusCode;
if (!string.IsNullOrEmpty(statusDesc))
{
activityBuilder.StatusDescription = statusDesc;
}
}

if (activity.Events.Any())
{
activityBuilder.Events = new List<dynamic>();

foreach (ref readonly var activityEvent in activity.EnumerateEvents())
{
dynamic activityEventBuilder = new System.Dynamic.ExpandoObject();

activityEventBuilder.Name = activityEvent.Name;
activityEventBuilder.Timestamp = activityEvent.Timestamp;

foreach (ref readonly var attribute in activityEvent.EnumerateTagObjects())
{
if (ConsoleTagTransformer.Instance.TryTransformTag(attribute, out var result))
{
activityEventBuilder.Tags = result;
}
}

activityBuilder.Events.add(activityEventBuilder);
}
}

if (activity.Links.Any())
{
activityBuilder.Links = new List<dynamic>();

foreach (ref readonly var activityLink in activity.EnumerateLinks())
{
dynamic activityLinkBuilder = new System.Dynamic.ExpandoObject();

activityLinkBuilder.TraceId = activityLink.Context.TraceId.ToString();
activityLinkBuilder.SpanId = activityLink.Context.SpanId.ToString();

foreach (ref readonly var attribute in activityLink.EnumerateTagObjects())
{
if (ConsoleTagTransformer.Instance.TryTransformTag(attribute, out var result))
{
activityLinkBuilder.Tags = result;
}
}

activityBuilder.Links.add(activityLinkBuilder);
}
}

var resource = this.ParentProvider.GetResource();
if (resource != Resource.Empty)
{

foreach (var resourceAttribute in resource.Attributes)
{
if (ConsoleTagTransformer.Instance.TryTransformTag(resourceAttribute, out var result))
{
activityBuilder.Attributes = result;
}
}
}

var options = new JsonSerializerOptions
{
WriteIndented = false,
ReferenceHandler = ReferenceHandler.IgnoreCycles,
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
};

string json = JsonSerializer.Serialize(activityBuilder, options);

this.WriteLine(json);
}

return ExportResult.Success;
}
}
}
47 changes: 47 additions & 0 deletions src/OpenTelemetry.Exporter.Console/ConsoleExporterHelperExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,5 +68,52 @@ public static TracerProviderBuilder AddConsoleExporter(
return new SimpleActivityExportProcessor(new ConsoleActivityExporter(options));
});
}

/// <summary>
/// Adds Console JSON exporter to the TracerProvider.
/// </summary>
/// <param name="builder"><see cref="TracerProviderBuilder"/> builder to use.</param>
/// <returns>The instance of <see cref="TracerProviderBuilder"/> to chain the calls.</returns>
public static TracerProviderBuilder AddConsoleJsonExporter(this TracerProviderBuilder builder)
=> AddConsoleJsonExporter(builder, name: null, configure: null);

/// <summary>
/// Adds Console JSON exporter to the TracerProvider.
/// </summary>
/// <param name="builder"><see cref="TracerProviderBuilder"/> builder to use.</param>
/// <param name="configure">Callback action for configuring <see cref="ConsoleExporterOptions"/>.</param>
/// <returns>The instance of <see cref="TracerProviderBuilder"/> to chain the calls.</returns>
public static TracerProviderBuilder AddConsoleJsonExporter(this TracerProviderBuilder builder, Action<ConsoleExporterOptions> configure)
=> AddConsoleJsonExporter(builder, name: null, configure);


/// <summary>
/// Adds Console JSON exporter to the TracerProvider.
/// </summary>
/// <param name="builder"><see cref="TracerProviderBuilder"/> builder to use.</param>
/// <param name="name">Name which is used when retrieving options.</param>
/// <param name="configure">Callback action for configuring <see cref="ConsoleExporterOptions"/>.</param>
/// <returns>The instance of <see cref="TracerProviderBuilder"/> to chain the calls.</returns>
public static TracerProviderBuilder AddConsoleJsonExporter(
this TracerProviderBuilder builder,
string name,
Action<ConsoleExporterOptions> configure)
{
Guard.ThrowIfNull(builder);

name ??= Options.DefaultName;

if (configure != null)
{
builder.ConfigureServices(services => services.Configure(name, configure));
}

return builder.AddProcessor(sp =>
{
var options = sp.GetRequiredService<IOptionsMonitor<ConsoleExporterOptions>>().Get(name);

return new SimpleActivityExportProcessor(new ConsoleActivityJsonExporter(options));
});
}
}
}