-
Notifications
You must be signed in to change notification settings - Fork 10k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NRE in Kestrel Telemetry EventSources #44817
Comments
cc: @davmason |
Thanks for contacting us. We're moving this issue to the |
This looks like the same underlying cause as the issue fixed in dotnet/runtime#76965. EventSource has always called DoCommand in the base EventSouce constructor (before any derived class constructors are run), we've just happened to not run in to it so far because most of our internal EventSources have a static Log field and were being touched before they were enabled. If you enable an EventSource via the pause on startup feature, then it is enabled before any managed code runs and will always hit this issue. We solved it in MetricsEventSource by moving initialization outside of the constructor. |
Mirror of dotnet/runtime#77434
EventSource.IsEnabled()
may start returningtrue
before theOnEventCommand
completes.We initialize shared
EventCounter
instances inOnEventCommand
that we then use from instrumented code paths.If the
EventSource
is initialized before being enabled, and the instance is accessed from multiple threads, a thread could seeIsEnabled() == true
and then read anull
EventCounter
instance.This race condition only appears once per process.
I believe this is the cause of an exception YARP hit in CI:
We should fix these cases.
cc: @Tratcher aspnetcore has 2 such cases as well:
aspnetcore/src/Middleware/ConcurrencyLimiter/src/ConcurrencyLimiterEventSource.cs
Line 47 in 1e8fe7e
aspnetcore/src/SignalR/common/Http.Connections/src/Internal/HttpConnectionsEventSource.cs
Line 46 in 1e8fe7e
An internal customer hit something similar:
The text was updated successfully, but these errors were encountered: