Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PGO: Instrument cold blocks with profile validators #53840

Closed
wants to merge 5 commits into from

Conversation

EgorBo
Copy link
Member

@EgorBo EgorBo commented Jun 7, 2021

Profile Data can't be 100% reliable and there are cases where a hot or a semi-hot code ends up in a block with zero weight (aka "cold block"):

  • Static PGO that we ship is based on TE benchmarks and can be less relevant for other workloads.
  • We don't support context-sensitive profiling yet. It's not a big problem for static PGO where we don't promote methods to tier1 during profile collection, but it is a problem with the dynamic one where we can bake some weights during first seconds of the app session (it's enough to call a method 30 times) and use them for all other callsites. (see example below)
  • Mistakes in the logic where we scale/propagate/mix weights in the JIT.
  • Workflow can change over time and the blocks initially recognized as cold can become hot. The only way to fix this is the ability to deoptimize code and re-collect profile.
  • Sampling-based profiling is even less accurate if we switch to that.

In order to get a sense of the big picture, I'm introducing a sort of a late instrumentation where I insert helper calls into every cold block with a profile data at some late JIT Phase. I decided to use calls instead of counters to be able to quickly compose a CSV report on app shutdown. It allows me to quickly get a statistics which methods have problematic weights.

Example:

static void Main()
{
    // Promote DoWork to tier1 and bake weights
    for (int i = 0; i < 100; i++)
    {
        DoWork(i);
        Thread.Sleep(16);
    }

    // Weights in DoWork are baked at this point.
    // Call it with unusual (from profile's point of view) weight.
    DoWork(-10);
    DoWork(-20);
}

[MethodImpl(MethodImplOptions.NoInlining)]
static int DoWork(int i)
{
    if (i >= 0)
    {
        return 42; // always taken in the first loop
    }
    return -42;
}

And run with:

DOTNET_TieredPGO=1
DOTNET_ProfileValidationPath="C:\prj\report.csv"

It will save a list of problematic methods where cold blocks were hit (in a CSV format) on app exit:
image

I tried to run a desktop app AvaloniaILSpy with default parameters and here is what it printed (I closed the form by hands after 10 seconds and random clicking):

image
(38 methods)

If I run it with DOTNET_TieredPGO=1 and DOTNET_TC_QuickJitForLoops=1 it lists 422 methods

PS: It doesn't tell which blocks specifically have invalid weights - it's just for overall sense.
PS2: I guess I should use full method names

/cc @AndyAyersMS @davidwrighton @dotnet/jit-contrib

@AndyAyersMS
Copy link
Member

Can you give some examples where this tech helped you sort out a problem?

I like the idea, but if we're going to go down this road I think we should look at something that has more long-lasting diagnostic capabilities, like doing a late instrumentation pass on all blocks, or relying on something like PIN.

@EgorBo
Copy link
Member Author

EgorBo commented Jun 8, 2021

Can you give some examples where this tech helped you sort out a problem?

I was just testing how much we can trust profile data (for inlining). Static PGO looks good so far in different benchmarks, there were some issues, like PowerShell benchmarks had in the hot path Path::HasExtension with different expectations.
Same for String::Equals, CastHelpers::ChkCast_Helper.

A small benchmark that just deserializes a complicated JSON file also had issues:

image

I can try to rewrite it to counters if you give me some pointers where to look at. But I guess it's not a priority for net6.0 so I'll just leave this as a prototype.

@EgorBo EgorBo closed this Jun 8, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jul 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants