Upgrade tt-metal(ium) level tests such as pgm dispatch benchmarks to use new benchmark data infra

[tt-rkim]

What is the context?

There is some existing benchmarking infrastructure that can record performance results or benchmarks and upload them to a central Superset dashboard for later presentation and consumption.

The tools themselves were written by @skhorasganiTT and are housed in models/perf/benchmarking_utils.py. Infra team manages the infrastructure that transforms and submits the data to data teams' pipeline.

An example of use is here: falcon7b_common/demo/demo.py

Though it was written with models in mind, it should be easily applicable to runtime-level benchmarks.

What is there to do?

The current infrastructure is only supported in Python and was written with models in mind.

So, the following will likely need to be done by runtime team:

• Get a feeling for what kind of insights Superset can offer
• Decide what kind of benchmarks the team would like to see
• Start making C++ versions of these benchmarking tools
• Ensure data format compatibility with the current infrastructure (usually would mean Pydantic in Python, but need to come up with alternatives for C++ workflows w/ infra team)
• Instrument current u-benchmarks to use these utilities

[jbaumanTT]

I'm thinking it might make sense to export the data using the csv or json format supported by https://github.com/google/benchmark, which should be flexible enough for our purposes.

[tt-rkim]

Do we use that? We should.

We can definitely edit the final schema however we choose to fit the requirements from data team.