Hints for optimisation level or visibility into tiering decisions

[tangobravo]

We compile computer vision code to WASM that runs live on getUserMedia camera frames for AR use cases on smartphone platforms.

We show a spinner during loading/compilation but ideally would want to continue to show a simple loading UI whilst the browser further optimises our code. For AR use cases the quality of tracking is related to the frame rate at which the camera can be processed (faster frame rate = less movement between frames = easier tracking problem). Users also expect a smooth experience from the start.

I believe dynamic tiering is now the default in both Chromium and WebKit, which enables fast startup with a "baseline" compiler, and waits until hot functions are identified before choosing to spend more effort further optimising them - good for power use as it avoids working hard optimising code that is never actually called. The reasoning for dynamic tiering seems sound and I expect implementations will generally keep using that technique.

However the downside is less predictable behaviour at the start - which unfortunately is also the time users will be forming their crucial "first impressions" of the quality of the experience. The combination of poorly optimised baseline code along with CPU contention with the optimising compiler busily trying to tier up the hot functions gives a less than ideal user experience.

I'm open to various options for improving the situation. Here's a couple of potential options:

1. A hinting mechanism for certain functions / whole modules to be more heavily optimised right from the start
2. Visibility into the tiering decisions

It would be possible to be running the code without showing the experience to the user to "pre-warm" things and allow dynamic tiering to kick in for the hot functions, but would be great to get some "everything that will be tiered has been tiered" kind of signal rather than needing to just measure the call timings to try to reason about whether optimal performance has been reached or not.

[alexp-sssup]

V8 supports an unofficial / never specified compilation hints custom section.

https://chromium.googlesource.com/v8/v8/+/dfdc8f68793c420113d53e1e877d535ed9f31612/src/wasm/wasm-feature-flags.h#19

Our use case https://webvm.io (x86 virtualization via WASM JIT compilation) would also benefit from better insights into the decision process of the underlying engine.

[penzn]

V8 supports an unofficial / never specified compilation hints custom section.

How to emit that section and what is its format?

[ajklein]

@backes who's working on this experimental feature in V8

[backes]

V8 supports an unofficial / never specified compilation hints custom section.

How to emit that section and what is its format?

The "compilation hints" were in intern project in 2019, which still exists in roughly unchanged form since then. The format is pretty simple; quoting from an internal document created back then:

Compilation hints are encoded in an optional custom section named "compilationHints". The section must appear after the function section and before the code section to allow for its validation and use. Utilizing essentially one byte per function, the section is defined as:

Field	Type	Description
count	varuint32	Number of compilation hints (Should match number of functions).
hints	bytes	Compilation hints: one byte per function.

Each byte encodes one compilation hint and consists itself of three values:

Spare bits (2 bit)	Top tier (2 bit)	Baseline tier (2 bit)	Compilation strategy (2 bit)
	0x00 Default	0x00 Default	0x00 Default
	0x01 Baseline	0x01 Baseline	0x01 Lazy compilation
	0x02 Optimised	0x02 Optimised	0x02 Eager compilation
			0x03 Lazy baseline compilation, eager top tier compilation

There are some basic tests for this feature but it is still in experimental state. It's well possible that some cases do not work as expected any more, in particular since we enabled both dynamic tiering and lazy compilation since then.

For reference, this is where we decode the custom section in V8: https://source.chromium.org/chromium/chromium/src/+/main:v8/src/wasm/module-decoder-impl.h;l=1378;drc=19f3c214cd4f78e0fe47b2ccafaca406aaacd42f
And this is where the hints are applied during compilation: https://source.chromium.org/chromium/chromium/src/+/main:v8/src/wasm/module-compiler.cc;l=3016;drc=19f3c214cd4f78e0fe47b2ccafaca406aaacd42f

Looking at the latter, the "eager compilation" strategy seems to be broken when lazy compilation is enabled globally.
For local tests, you should thus disable lazy compilation and enable compilation hints. In d8, that's --no-wasm-lazy-compilation --experimental-wasm-compilation-hints. In Chrome, you need to wrap those flags further: --js-flags="--no-wasm-lazy-compilation --experimental-wasm-compilation-hints".

Please let us know if this is helpful for your use case. We are currently prototyping a similar mechanism meant to be used for PGO, but it would also replace the compilation hints.