Profile-Guided Optimization (PGO) benchmark results

[zamazan4ik]

Hi!

Today I came up with an idea to try to optimize the library performance with PGO (as I already did for many other applications - all the results are available here). I performed some tests and want to share the results. Hopefully, for varpro users, the results will be interesting to read. Please do not treat this as an actual issue - it's just a performance report.

Test environment

• Fedora 39
• Linux kernel 6.7.3
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.76
• varpro version: the latest for now from the main branch on commit f2adeec4a100714d7915fb7014ce09d91246d299
• Disabled Turbo boost (for more stable results across benchmark runs)

Benchmark

Built-in benchmarks are invoked with cargo bench. PGO instrumentation phase on benchmarks is done with cargo pgo bench . PGO optimization phase is done with cargo pgo optimize bench.

All PGO optimization steps are done with cargo-pgo tool.

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/1dc05d0f78e10ebe567988031e5a0cb2
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/c20cd7d4a32d3d8202c3a1ec7890cd09
• (just for reference) PGO instrumented compared to Release: https://gist.github.com/zamazan4ik/1ba5190ceac795e048cb7fa09e3bcb0c

At least in the provided by the project benchmarks, I see measurable performance improvements. I don't know how these benchmarks are helpful for real-life performance evaluation - I just believe the project maintainers in this case.

Possible further steps

I can suggest the following things to consider:

• Perform more PGO benchmarks in other scenarios. If it shows improvements - add a note to the documentation about possible improvements in the tracing library performance with PGO (I guess somewhere in the README file will be enough).
• Evaluate where and why PGO brings performance improvements. Maybe some helpful performance optimization insights can be gathered from it e.g. better code organization for better inlining decisions.

I will be happy to answer all your questions about PGO.

[geo-ant]

Wow that's amazing, thank you! I will definitely look into this but it might be a while before I get to it.