[WIP] Add jl_start_tracking_dispatches / jl_finish_tracking_dispatches

[NHDaly]

Allow users to track all dynamic dispatches that occur when tracking is enabled.

This can be useful for debugging, such as to understand where your code has type-instabilities that lead to a dispatch.

It will also be used by StagedFunctions.jl (and ideally base Generated functions) to cheaply record the set of MethodInstances that need to have backedges set to the generated function. Merging this into #32774 should be enough to fix all the tests! :)

There is an example of this working, in the StagedFunctions package, here:
NHDaly/StagedFunctions.jl#9

[Keno]

I'm a little worried about adding extra instructions in a path that's this hot, even with the unlikely annotation. If it's just for debugging I would have suggested a patch point, but sounds like you're imagining a case where generated functions would turn this on automatically?

[JeffBezanson]

I have to agree it does not seem acceptable to add code to this path.

[NHDaly]

Yeah, i had the same concern. :( Thanks for the quick feedback, i was going to ask that same question myself. ❤️

Are there any good alternatives that jump to your mind(s)?

[NHDaly]

For completeness, I was trying to measure how much of an impact this would have, but I haven't found a benchmark that actually spends a lot of cpu in jl_invoke. I thought something like this might, but it looks like it's only spending 2.5% of time in jl_invoke according to the Profile:

r(n) = n == 0 ? 0 : Core._apply(r, n-1)

using PProf, Profile

@profile r(1) # warmup r() and profile

Profile.clear()
@profile for _ in 1:10_000; r(1000); end

pprof(from_c=true)

This is showing 2.3% cpu actually in the invoke frame (after this change):

Can you recommend a better micro-benchmark that would make invoke hotter?

And/or, is there a different spot i could put this check that would catch "all dynamic dispatches"?

EDIT: Actually, it seems to vary between like 2.3% - 3.5%.

[NHDaly]

(I bumped it up to more like 3% - 5% when i added more methods to r, to avoid it somehow inlining and avoiding the dispatch.)

[vtjnash]

Usually just something like Base.inferencebarrier(==)(1, 1)

On my machine, this takes about 11.4 nanoseconds, which is probably about 36 cycles or 650 flops (2.7GHz base frequency, BLAS.set_num_threads(1); time * peakflops())

[vtjnash]

replaced by --trace-dispatch