Lack of PGO probes in instrumented builds of libcoreclr

[hoyosjs]

When building coreclr with --pgoinstrument, libcoreclr.so ends up with essentially no PGO instrumentation probes. The add_pgo(coreclr) call in src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt uses target_compile_options(PRIVATE), which only applies -flto -fprofile-instr-generate to the source files compiled directly into the coreclr shared library target - just mscoree.cpp and exports.cpp. The resulting library has very few probes (20-100 over ~30 functions). The vast majority of the runtime code lives in static libraries (cee_wks, utilcode, coreclrpal, etc.) that are linked in already compiled, without instrumentation flags. This doesn't affect clrjit because the JIT compiles all its source files directly into the shared library target. Windows doesn't have this issue since this is a link time option.

[hoyosjs]

Possible fix 1 (and I'd go a step further of cleanup and just move it to configure compiler at that point) is to define the compilation option globally

Possible fix 2 - walk the graph of deps when an add_pgo target is specified.

Both are in https://github.com/hoyosjs/runtime/commits/juhoyosa/fix-pgo

Fix 2 preserves behavior, but i'm not sure this complexity is buying us much. These builds mostly just get used for training and picking partial sets is not something I think gives us any benefit. After change 1, I see coreclr having ~5.1k probes. With the second, I see it have around 4.7k.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[steveisok]

Fix 2 preserves behavior, but i'm not sure this complexity is buying us much. These builds mostly just get used for training and picking partial sets is not something I think gives us any benefit. After change 1, I see coreclr having ~5.1k probes. With the second, I see it have around 4.7k.

Push what you think is best and have us react there.

[jkotas]

Possible fix 1 (and I'd go a step further of cleanup and just move it to configure compiler at that point) is to define the compilation option globally

I think this is the right option.

Ideally, we would get the PGO data for all native binaries that we ship, including e.g. System.Compression.Native.so.

[EgorBo]

Some time ago I tried to do that via #67820

[AndyAyersMS]

@davidwrighton I wonder if this contributes to the TP issues you see in #123830, maybe the newly split off files no longer get handled via -flto? It seems odd in your PR the TP is so much worse on Linux than Windows.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.