Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aws sdk api phase1 #3401

Closed
wants to merge 3 commits into from

Conversation

birojnayak
Copy link
Contributor

Why

Enabling AWS SDK Calls

What

Re-using OpenTelemetry.Instrumentation.AWS and bringing SDK api calls to auto instrumentation

Tests

Added integration test

Checklist

  • CHANGELOG.md is updated.
  • Documentation is updated.
  • New features are covered by tests.

private readonly PluginManager _pluginManager;

public AWSInitializer(PluginManager pluginManager)
: base("OpenTelemetry.Instrumentation.AWS")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@birojnayak, I think that this line is wrong.

You are listening on "OpenTelemetry.Instrumentation.AWS" be loaded by application. It should never happen.

I suppose that you need put here AWSSDK.DynamoDBv2 or any other library you want to instrument (instead of instrumentation library).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the same but still don't see any aws span.. here is the gist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kielek made it work.. the culprit was the initialization.

@@ -60,6 +61,9 @@

<ItemGroup Label="Transient dependencies auto-generated by GenerateNetFxTransientDependencies" Condition=" '$(TargetFramework)' == 'net462' AND $(_IsPacking) != true ">
<!-- These dependencies are also not included in the NuGet package since the conflicts can be left to NuGet version resolution. -->
<PackageReference Include="AWSSDK.Core" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these OpenTelemetry Instrumentation libraries for AWS?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajkumar-rangaraj I believe these are dependent packages as here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vendor library references should not be a part of this project, as this is against the project's principles. These references could be included in the tests, but not in the main project.

Copy link
Contributor Author

@birojnayak birojnayak May 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajkumar-rangaraj TBH, I didn't understand fully what's vendor library, I assumed for each service you can add a package to get trace. But, I fully understand there are 4 libraries we are adding and bringing those dependencies, so I was asking in the SIG (last 2 meetings) and checking with everyone if it's ok.

The other option is to enable via Plugin(I have to validate that too), but in that case it would be in it's own repo with many other attach(exporter, agent etc), which won't benefit broader community (AFAIK) . Let's discuss what would be ideal in SIG.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the main issue here is that the OpenTelemetry.Instrumentation.AWS NuGet package references other AWS libraries. For our purposes, this is not acceptable because we should not be bringing in other libraries. Compare this to the OpenTelemetry.Instrumentation.Quartz package which only references the OpenTelemetry.Api

I think this prevents us from taking the change, until the AWS instrumentation package can be rewritten in a way that doesn't require referencing the AWS packages.

Copy link
Contributor Author

@birojnayak birojnayak May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand guys and in line with this.... at the same time let's discuss about any alternative...

@birojnayak birojnayak marked this pull request as ready for review May 9, 2024 07:04
@birojnayak birojnayak requested a review from a team May 9, 2024 07:04
@birojnayak
Copy link
Contributor Author

@Kielek , @rajkumar-rangaraj tests those are failing here doesn't look related to my change, let me know if I am missing something.

  17:47:41 [DBG] [xUnit.net 00:04:20.05] IntegrationTests: === RUN   IntegrationTests.BuildTests.DistributionStructure()
  Error: [xUnit.net 00:04:20.07]     IntegrationTests.BuildTests.DistributionStructure [FAIL]
  17:47:41 [ERR] [xUnit.net 00:04:20.07]     IntegrationTests.BuildTests.DistributionStructure [FAIL]
  17:47:41 [DBG] [xUnit.net 00:04:20.07] IntegrationTests: --- FAIL: IntegrationTests.BuildTests.DistributionStructure() (0.01s)

@Kielek
Copy link
Contributor

Kielek commented May 13, 2024

@Kielek , @rajkumar-rangaraj tests those are failing here doesn't look related to my change, let me know if I am missing something.

  17:47:41 [DBG] [xUnit.net 00:04:20.05] IntegrationTests: === RUN   IntegrationTests.BuildTests.DistributionStructure()
  Error: [xUnit.net 00:04:20.07]     IntegrationTests.BuildTests.DistributionStructure [FAIL]
  17:47:41 [ERR] [xUnit.net 00:04:20.07]     IntegrationTests.BuildTests.DistributionStructure [FAIL]
  17:47:41 [DBG] [xUnit.net 00:04:20.07] IntegrationTests: --- FAIL: IntegrationTests.BuildTests.DistributionStructure() (0.01s)

It is fully related.

These changes brings at least 5 dependencies to the project not included into *.verified.txt files

  • AWSSDK.Core
  • AWSSDK.SimpleNotificationService
  • AWSSDK.SQS
  • OpenTelemetry.Extensions.AWS
  • OpenTelemetry.Instrumentation.AWS

This set of tests protect us from including unwanted dependencies (things mentioned by Raj here.

{
var instrumentationType = Type.GetType("OpenTelemetry.Instrumentation.AWS.Implementation.AWSClientsInstrumentation, OpenTelemetry.Instrumentation.AWS")!;

var awsClientOptions = new AWSClientInstrumentationOptions();
Copy link
Contributor

@zacharycmontoya zacharycmontoya May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajkumar-rangaraj @birojnayak I think we could try experimenting with the following:

  1. Creating this instrumentation options type dynamically
  2. Remove the OpenTelemetry.Instrumentation.AWS nuget package reference (see my below edit)

What do you think?

Copy link
Contributor

@zacharycmontoya zacharycmontoya May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I do believe we'll need the OpenTelemetry.Instrumentation.AWS package as we should not be expecting the user to provide this. As a result, we would need to do the same dependency trimming as MongoDb so we only bring in the OpenTelemetry.Instrumentation.AWS DLL and not its dependencies. However if we create the instrumentation options dynamically then we shouldn't get the other TypeReferences and AssemblyReferences.

Does this approach sound reasonable to you @Kielek?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is ugly (exactly as for MongoDB) and put additional cost for militance, but I agree that it is the best solution we can find.

While implementing/updating package we need to be extremely carefully about exact version to remove from our project.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so what's the verdict guys ? I am open to what team decides.. (sorry couldn't attend the SIG Today).

@birojnayak
Copy link
Contributor Author

We are implementing internally in the plugins, so closing this. We do have plan to have 1 package for AWS SDK related changes, once that is out, I will re-submit another PR.
Closing for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants