[vedithal-amd]

Analysis data dump

• Add --db option to analyze mode which creates database file
  containiing analysis result

• Add documentation on rocpd output format and analysis database file

• Create sqlite3 database using object relation mapping (ORM) provided
  by sqlalchemy library

• Fix metrics config to remove metrics marked as null, fix Unit header, add
  missing title

• Add test cases to ensure analysis data dump works

• Update CHANGELOG

[feizheng10]

2025

[feizheng10]

It seems to me duplicated somewhere else, is it?

[feizheng10]

It is debatable for me, "db" is a end-user level analysis mode. It more likes on the same level of "--save-dfs". Thought?

[vedithal-amd]

I would like to skip analysis report generation and only focus on creating database. That is why I am not using CLI mode with --save-dfs option. Remember, its not only about saving metrics dfs. We also need to think about roofline, pc sampling information storage in database.

For the end user, they can still do both analyze --save-dfs and analyze --db <db_name>, so we are keeping it flexible.

[feizheng10]

2025

[feizheng10]

Some general comments:

• Before removing the Grafana solution with Mongo DB. To reduce the confusion for the end users, we might want to explicitly claim there is no relationship for those 2 in the doc.
• As a general design, or user view, db output and df csv output are just alternatives to save the output as user's choice, they are just data storage, not analysis mode.

[feizheng10]

I'd apologize here: we should have the doc for "--save-dfs" early.
Since you adding "data-dump" section here, I'd say we might want to include description for both db and csv format. And even more, the option could be in more unified fashion: --output [txt/csv/db]

[vedithal-amd]

Agreed we should replace -o, --save-dfs, --db analysis report options with ``--output txt/csv/db`. I will make this change and also update documentation.

[vedithal-amd]

As a general design, or user view, db output and df csv output are just alternatives to save the output as user's choice, they are just data storage, not analysis mode.

I would like to have the analysis database results calculation in a separate file like analysis_db.py instead of depending on analysis_cli.py.

analysis_cli.py only creates dataframe for metric values, it does not create dataframe for pc sampling and top kernels (instead it writes csv files, which is not necessary in analysis database mode). It also does not handle roofline data points calculation. That is why I am using different file analysis_db.py. This will require me to use a new analysis mode called analysis db mode. However, analysis mode is transparent to the end user, they don't specify analysis mode and should not care about it, all they have to do is pass --db option to analysis command.

[vedithal-amd]

I will move roofline data point calculation logic inside another function in roofline_calc.py and use that in analysis_db.py

[vedithal-amd]

Before removing the Grafana solution with Mongo DB. To reduce the confusion for the end users, we might want to explicitly claim there is no relationship for those 2 in the doc.

That is precisely why i am using analysis db or analysis data dump everywhere

[vedithal-amd]

This PR will need dev efforts and re-testing due to #325. Calculation of roofline metrics should use the analysis config expressions, instead of hardcoded expressions.

[cfallows-amd]

Does this need to be exactly this version? Can we do >= or <=? It is already not latest release

[vedithal-amd]

Good idea, will covert to >=, we don't want to be stuck using an outdated version

[vedithal-amd]

Closing against #748