Add the concept of "notification profilers" to the runtime

[davmason]

This allows multiple profilers to register if they only request a subset of profiler behavior that is multiple profiler safe - ReJIT, IL modification, ELT stubs, etc are forbidden for these profilers.

The basic approach is relatively straightforward, instead of calling one profiler back we call back a list of them.

Items left:
Testing:

• Run runtime and profiler tests for regressions
• Stress test profilers attaching and detaching
• Run all runtime tests with notification profiler attached
• Test multiple profilers each with an eventpipe session

Documentation

• Write description of feature
• Write and publish sample to dotnet/samples

[ghost]

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This allows multiple profilers to register if they only request a subset of profiler behavior that is multiple profiler safe - ReJIT, IL modification, ELT stubs, etc are forbidden for these profilers.

The basic approach is relatively straightforward, instead of calling one profiler back we call back a list of them.

Items left:
Testing:

• Run runtime and profiler tests for regressions
• Stress test profilers attaching and detaching
• Run all runtime tests with notification profiler attached
• Test multiple profilers each with an eventpipe session

Documentation

• Write description of feature
• Write and publish sample to dotnet/samples

Author:	davmason
Assignees:	davmason
Labels:	area-Diagnostics-coreclr
Milestone:	6.0.0

[josalem]

I didn't look too closely at the thread safety around loading/unloading, but the general gist of the work looks good to me! Excited to see where this goes 🚀

[noahfalk]

This looks good to me! Comments inline on some small stuff I noticed.

[noahfalk]

Aren't some of these side-effecting? Since the notification profilers are intended to be SxS perhaps we should scope it back to the options that are allowable on attach?

[davmason]

It's a good conversation to have. I was taking the approach to not ban any side effects, but rather ban side effects that would be non trivial to implement (ELT hooks) or are hard to provide in a way that is consistent and reproducible (IL rewriting).

I am worried about being too limiting in what scenarios we support. It is unfortunate to look back in a few releases and think "we could have done it except we arbitrarily decided to block this thing that is actually safe"

[noahfalk]

I don't mind taking the risk as long as you think you've had suitable time to consider it and test the additional APIs : )

[noahfalk]

Although one other thought, if we are going to allow side-effects then 'NotificationOnly' feels like a misleading name. What about calling them "SxS", "Cooperative", or another name similar to that?