NXP backend: Switch to the default ExecuTorch graph visualization.

[MartinPavella]

Summary

This PR updates the existing ExecuTorch graph visualization tool to suit NXP use-cases, and removes the old custom visualizer.

Test plan

N/A

cc @robert-kalmar @roman-janik-nxp @StrycekSimon @jirioc

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14297

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 118 Pending

As of commit 1c79d1b with merge base 82a2324 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[MartinPavella]

@pytorchbot label "module: nxp" "release notes: nxp"

[MartinPavella]

The failing linting checks are related to imports from model_explorer. As far as I can tell, it is caused by missing stub files.
I checked all the 4 commits which modified this module before, and they all had MYPY linting errors. I don't know if I can fix it.
@Gasoonjia do you know of any way to fix this?

[Erik-Lundell]

Chiming in with some comments, hope that's ok.

[MartinPavella]

Chiming in with some comments, hope that's ok.

Thank you for the review @Erik-Lundell .
I have renamed the function and added the callbacks as you suggested. The visualization is no longer just for NXP, and it can be used by any backend.
The model_explorer_cluster_style.json can be loaded in the Model Explorer GUI (by clicking the palette in the top right corner), to add colors to the QDQ clusters and partitions for better readability. (I have added a note to the commit message as you suggested)

Do you have any information regarding the MYPY import warnings (the failing lintrunner)?

I am attaching an example of the visualization.

[Erik-Lundell]

Thanks for your adjustments! I was probably not too clear what I meant with callback, I meant that the function takes a callback(Node) -> namespace, that makes it possible to use for any type of clustering. You can then include a new function that does the particular clustering you are interested in. But my main concerns are fixed, so I'll leave the rest of the review to someone who can actually approve.

Regarding mypy, I don't know, sorry. Maybe just add ignore markers?

[MartinPavella]

The failing linting checks are related to imports from model_explorer. As far as I can tell, it is caused by missing stub files. I checked all the 4 commits which modified this module before, and they all had MYPY linting errors. I don't know if I can fix it. @Gasoonjia do you know of any way to fix this?

@Gasoonjia pinging you in case you missed the comment.

[robert-kalmar]

@MartinPavella, Implementation and concept wise looks good to me.

What I recommend:

• Integrate it as example usage into nxp's aot_neutron_example where on the CLI argument e.g. --visualize=show|store the example would either show the visualization of the model or store it to a json file based on the input model name. This will give everyone an example use and context for the reviewers, especially @Gasoonjia as code owner.
• In the example store/visualize the exported quantized graph (shows the QDQ cluster visualization) and final exported program with NeutronBackend node. Optionally you can also visualize/store the output of the partitioner, although it is a bit more difficult as the CLI argument must be propagated from example to the NeutronPartitioner (e.g. via neutron_compile_spec, although bit hacky). Or ExecuTorch's to_edge_transform_and_lower and/or to_edge must be extended to support:
  • either propagation of visualize argument, then the visualization can be done on the level of ExecuTorch for all the backends. Or
  • the aforementioned routines would return the partitioned graph .
• Document the usage in a new md in docs\sources\ showing how to use the visualize_with_cluster, opening a model explorer, using the predefined color scheme.
• There is no documentation available how to use the visualization at all. The docs/source/devtools-overview.md mentions the "visualization - Comming soon" is outdated, as the visualization is already here for some time :-). You spent time in investigation, so feel free to add short guideline how to use it, this will give you a prerequisity for the documentation of visualize_with_cluster and fill the gap with missing visualization documentation.
• Move the color coding style definition (model_explorer_cluster_style.json) to a subfolder. This way, there will be a foundation for other backends to add their style definition to visualize their features.

@digantdesai , for the context, at our early development days we mentioned our visualization tool, based on .dot files, and discussed using the ExecuTorch visualizer instead. Martin worked out a proposal. Feel free to comment on the PR or my suggestions.

[roman-janik-nxp]

Besides small adjustments and documentation it looks fine.

[MartinPavella]

@MartinPavella, Implementation and concept wise looks good to me.

What I recommend:

* Integrate it as example usage into nxp's `aot_neutron_example` where on the CLI argument e.g. `--visualize=show|store` the example would either show the visualization of the model or store it to a json file based on the input model name. This will give everyone an example use and context for the reviewers, especially @Gasoonjia as code owner.

* In the example store/visualize the exported quantized graph (shows the QDQ cluster visualization) and final exported program with NeutronBackend node. Optionally you can also visualize/store the output of the partitioner, although it is a bit more difficult as the CLI argument must be propagated from example to the NeutronPartitioner (e.g. via `neutron_compile_spec`, although bit hacky). Or ExecuTorch's `to_edge_transform_and_lower` and/or `to_edge` must be extended to support:
  
  * either propagation of visualize argument, then the visualization can be done on the level of ExecuTorch for all the backends. Or
  * the aforementioned routines would return the partitioned graph .

* Document the usage in a new `md` in `docs\sources\` showing how to use the `visualize_with_cluster`, opening a model explorer, using the predefined color scheme.

* There is no documentation available how to use the visualization at all. The `docs/source/devtools-overview.md` mentions the "visualization - Comming soon" is outdated, as the visualization is already here for some time :-). You spent time in investigation, so feel free to add short guideline how to use it, this will give you a prerequisity for the documentation of `visualize_with_cluster` and fill the gap with missing visualization documentation.

* Move the color coding style definition (`model_explorer_cluster_style.json`) to a subfolder. This way, there will be a foundation for other backends to add their style definition to visualize their features.

@digantdesai , for the context, at our early development days we mentioned our visualization tool, based on .dot files, and discussed using the ExecuTorch visualizer instead. Martin worked out a proposal. Feel free to comment on the PR or my suggestions.

Thank you for the suggestions. I have implemented the changes except for the visualization of the partitioning. I didn't want to make such a large change to the entire pipeline within this PR.

[MartinPavella]

@Erik-Lundell, @mergennachin the PR has been updated recently to better document the Model Explorer visualization. Please feel free to review the changes.

[StrycekSimon]

Have no comments here. Nice work!

[mergennachin]

this page looks great, thank you

https://docs-preview.pytorch.org/pytorch/executorch/14297/visualize.html

[MartinPavella]

The failing checks appear unrelated. Merging the PR.

[digantdesai]

Nit: this doesn't point to the code but becomes a download link.