vtune: smoke test profiling in CI #9185
Conversation
abrown
force-pushed
the
vtune-testing
branch
5 times, most recently
from
f210c88
to
c3d2e41
Compare
tests/all/cli_tests.rs
Outdated
| assert!(stdout.contains("CPU Time")); | ||
| println!("> stderr:\n{stderr}"); | ||
| assert!(!stderr.contains("Error")); | ||
| assert!(!stderr.contains("Warning")); |
There was a problem hiding this comment.
The reason for searching for this kind of string is due to error dumps like this one.
abrown
force-pushed
the
vtune-testing
branch
2 times, most recently
from
f9d27d2
to
d054145
Compare
abrown
force-pushed
the
vtune-testing
branch
2 times, most recently
from
450d531
to
a85a717
Compare
|
One thing to note here: I've tried to only install VTune where needed (when we check wasmtime-cli) because it increases the build time by at least a minute. I know we've worked in the past to reduce build times so there's a trade-off to think about: faster builds or test more features? |
| &std::env::temp_dir().to_string_lossy(), | ||
| // ...then run Wasmtime with profiling enabled: | ||
| &get_wasmtime_path()?.to_string_lossy(), | ||
| "--profile=vtune", |
There was a problem hiding this comment.
To confirm, does this test fail if this flag is left out?
There was a problem hiding this comment.
No, one can always run vanilla Wasmtime in VTune and that should be valid whether this flag is present or not. What this flag does is inform VTune about the JIT-compiled code so it can display it appropriately in the UI. We've had problems in the past with incorrectly telling VTune about this code which did result in errors and that is what this test is checking: in effect, this is a regression test to check that we correctly tell VTune about the JIT code.
|
The integration path here seems like a reasonable starting point to me, thanks! I think it's fine to start here and see how it goes |
I worked on the |
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
|
Not sure what is going on with CI: with the latest |
This adds a smoke test for checking that the VTune functionality still works. - if `vtune` is not on the system path, skip the test - if it is, run Wasmtime with VTune profiling enabled - check that it captured the CPU time and didn't fail. This test doesn't rigorously check all the ins and outs of the VTune functionality but is better than before.
In order to actually run the VTune CLI test, we need `vtune` on the system. The `install-vtune-action` I created does this ([script]). [script]: https://github.com/abrown/install-vtune-action/blob/main/install.sh
Running the VTune installation and CLI test on any `ubuntu` runner meant that even test runs for `riscv64` (emulated on QEMU) would attempt this. Instead, restrict this test only to `linux-x64` runs. prtest:full
abrown
force-pushed
the
vtune-testing
branch
from
9d19479
to
81e1d40
Compare
|
Ah I think the issue is that this needs to be skipped in qemu. Otherwise it's running a non-x64 binary in vtune which isn't going to work. We don't have a great "am I in qemu" test other than |
This checks that Wasmtime's
--profile=vtunefunctionality continues to work. See commit messages for more details.