Worker instrumentation #3195

RohitRanjanMS · 2025-10-13T08:00:15Z

Updated worker instrumentation to improve telemetry handling.
Added support for OpenTelemetry schema version backward compatibility, allowing users to adopt newer schema versions at their own pace.
Removed the previous “internal” activity that was created solely for context propagation but could not be tracked.
Introduced a new internal activity that is now published and provides proper context propagation.

Issue describing the changes in this PR

resolves #3198
resolves #3199

Pull request checklist

My changes do not require documentation changes
- Otherwise: Documentation issue linked to PR
My changes should not be added to the release notes for the next release
- Otherwise: I've added my notes to release_notes.md
My changes do not need to be backported to a previous version
- Otherwise: Backport tracked by issue/PR #issue_or_pr
I have added all required tests (Unit tests, E2E tests)

Copilot

Pull Request Overview

This PR updates worker instrumentation to improve telemetry handling by adding support for OpenTelemetry schema version backward compatibility and introducing a new internal activity system for proper context propagation.

Added support for multiple OpenTelemetry schema versions (1.17.0 and 1.37.0) with backward compatibility
Replaced the old activity system with a new telemetry provider architecture that creates properly tracked activities
Updated trace context to include attributes and modified various telemetry-related components

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/DotNetWorker.Core/FunctionsApplication.cs	Replaced old activity factory with new telemetry provider system
src/DotNetWorker.Core/Diagnostics/	Added new telemetry provider architecture with schema version support
src/DotNetWorker.Core/Context/	Enhanced trace context to include attributes
src/DotNetWorker.OpenTelemetry/	Updated OpenTelemetry package versions and configuration
test/	Updated tests to work with new telemetry system and validate schema versions

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

test/DotNetWorker.OpenTelemetry.Tests/EndToEndTests.cs

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProviderV1_37_0.cs

src/DotNetWorker.Core/Context/TraceContext.cs

jviau

This PR has a lot of changes that impact the telemetry from DotNetWorker.Core irrespective of customer using OTel or not. These seem like breaking changes for those customers.

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProviderBase.cs

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProviderV1_17_0.cs

src/DotNetWorker.Core/Diagnostics/TraceConstants.cs

src/DotNetWorker.Core/Hosting/ServiceCollectionExtensions.cs

jviau · 2025-10-13T22:30:12Z

src/DotNetWorker.Core/FunctionsApplication.cs

-
-            using var logScope = _logger.BeginScope(scope);
-            using Activity? invokeActivity = _functionActivitySourceFactory.StartInvoke(context);
+            using var logScope = _logger.BeginScope(_functionTelemetryProvider.GetTelemetryAttributes(context).ToList());


This is a significant change to log scope. Also what happens if OTel is not enabled here?

I’ve kept all existing attributes intact and added new ones according to v17.0 without any breaking changes.
If OpenTelemetry/AppInsights isn’t enabled, the activity will be null, but you’ll always have the scope.
The default specification version is v37.0

Won't this new scope approach be adding OTel specific attributes even when customer is using legacy app insights SDK?

I also wonder if we need all of those in the log scope specifically? Or can they be part of the OTel resource?

jviau · 2025-10-13T22:32:17Z

src/DotNetWorker.OpenTelemetry/OpenTelemetryConstants.cs

        internal const string RegionNameEnvVar = "REGION_NAME";
        internal const string ResourceGroupEnvVar = "WEBSITE_RESOURCE_GROUP";
        internal const string OwnerNameEnvVar = "WEBSITE_OWNER_NAME";
+        internal const string WorkerSchemaVersion = "1.37.0";


This directly ties to OTel schema version to the package. This will prevent us from servicing old schema versions in new releases. Is that acceptable?

I’ll add support for new versions as they’re released. Version 1.37 will remain our baseline, and we’ll bump it to a newer base in the next major release.

jviau · 2025-10-14T22:52:17Z

src/DotNetWorker.Core/Context/TraceContext.cs

        /// Gets the attributes associated with the trace.
        /// </summary>
-        public abstract IReadOnlyDictionary<string, string> Attributes { get; }
+        public virtual IReadOnlyDictionary<string, string> Attributes => new Dictionary<string, string>();


consider using a common empty dictionary to avoid any allocations when this is not overridden.

Could even use ImmutableDictionary<string, string>.Empty

jviau · 2025-10-14T22:53:40Z

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProvider.cs

+        {
+            OpenTelemetrySchemaVersion.V1_17_0 => new TelemetryProviderV1_17_0(),
+            OpenTelemetrySchemaVersion.V1_37_0 => new TelemetryProviderV1_37_0(),
+            _ => throw new InvalidOperationException($"Unsupported OpenTelemetry schema version: {version}")


Consider ArgumentOutOfRangeException or ArgumentException

jviau · 2025-10-17T21:56:33Z

src/DotNetWorker.Core/Hosting/ServiceCollectionExtensions.cs

                Activator.CreateInstance(startupCodeExecutorInfoAttr.StartupCodeExecutorType) as WorkerExtensionStartup;
            startupCodeExecutorInstance!.Configure(builder);
-        }
+        }        


nit remove extra whitespace

Suggested change

}

}

jviau · 2025-10-20T19:48:43Z

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProviderV1_37_0.cs

+        => OpenTelemetrySchemaVersion.V1_37_0;
+
+    protected override ActivityKind Kind
+        => ActivityKind.Internal;


Are we assuming the func host emits the Server span?

jviau · 2025-10-20T19:51:56Z

src/DotNetWorker.Core/FunctionsApplication.cs

-
-            using var logScope = _logger.BeginScope(scope);
-            using Activity? invokeActivity = _functionActivitySourceFactory.StartInvoke(context);
+            using var logScope = _logger.BeginScope(_functionTelemetryProvider.GetTelemetryAttributes(context).ToList());


Won't this new scope approach be adding OTel specific attributes even when customer is using legacy app insights SDK?

I also wonder if we need all of those in the log scope specifically? Or can they be part of the OTel resource?

jviau · 2025-10-20T19:53:57Z

src/DotNetWorker.OpenTelemetry/ResourceSemanticConventions.cs

        internal const string CloudPlatform = "cloud.platform";
        internal const string CloudRegion = "cloud.region";
-        internal const string CloudResourceId = "cloud.resource.id";
+        internal const string CloudResourceId = "cloud.resource_id";


Is this not a schema breaking change?

Good question. Wasn't always the cloud resource id defined as cloud.resource_id? Tried to see if there was a resource change from cloud.resource.id to cloud.resource_id but didn't find that in the schema specs.

RohitRanjanMS added 3 commits October 13, 2025 00:31

Updating worker instrumentation.

4fee6df

Remove empty lines.

e6922ed

Adding a test for empty version.

e8f5a11

RohitRanjanMS requested a review from Copilot October 13, 2025 08:00

Copilot AI reviewed Oct 13, 2025

View reviewed changes

test/DotNetWorker.OpenTelemetry.Tests/EndToEndTests.cs Outdated Show resolved Hide resolved

src/DotNetWorker.Core/Diagnostics/Telemetry/TelemetryProviderV1_37_0.cs Outdated Show resolved Hide resolved

RohitRanjanMS requested a review from jviau October 13, 2025 09:32

jviau reviewed Oct 13, 2025

View reviewed changes

src/DotNetWorker.Core/Context/TraceContext.cs Outdated Show resolved Hide resolved

jviau requested changes Oct 13, 2025

View reviewed changes

microsoft-github-policy-service bot added the Needs: Author Feedback label Oct 13, 2025

Addressing review comments.

422b21e

microsoft-github-policy-service bot removed the Needs: Author Feedback label Oct 14, 2025

RohitRanjanMS mentioned this pull request Oct 15, 2025

Versioning for Span and Metrics specifications #3199

Open

Updating default OpenTelemetrySchemaVersion to 1.37

f5002ad

RohitRanjanMS requested a review from jviau October 17, 2025 21:48

Setting default schemaversion to 1.17

0f40ec6

jviau reviewed Oct 20, 2025

View reviewed changes

Worker instrumentation #3195

Are you sure you want to change the base?

Worker instrumentation #3195

Uh oh!

Conversation

RohitRanjanMS commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue describing the changes in this PR

Pull request checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jviau left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RohitRanjanMS commented Oct 13, 2025 •

edited

Loading