Skip to content

[Unity][Contrib] Introduce several features of cutlass profiler #14275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Mar 18, 2023
Merged

[Unity][Contrib] Introduce several features of cutlass profiler #14275

merged 6 commits into from
Mar 18, 2023

Conversation

@spectrometerHBH
Copy link
Contributor

@spectrometerHBH spectrometerHBH commented Mar 12, 2023 •
edited
Loading

@tvm-bot
Copy link
Collaborator

tvm-bot commented Mar 12, 2023 •
edited
Loading

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

Generated by tvm-bot

@spectrometerHBH spectrometerHBH marked this pull request as ready for review March 12, 2023 03:34
junrushao
junrushao approved these changes Mar 12, 2023
View reviewed changes
Copy link
Member

@junrushao junrushao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Could you fix the linter?

@Hzfengsy
Copy link
Member

Hzfengsy commented Mar 13, 2023

Could you please send the PR directly to main? since it's not unity related.

@junrushao junrushao merged commit 0ffd24c into apache:unity Mar 18, 2023
tqchen pushed a commit that referenced this pull request Mar 20, 2023
@spectrometerHBH @tqchen
[Unity][Contrib] Introduce several features of cutlass profiler (#14275)
e2b1d93 
- allow Conv2d using different alignment factors for input and epilogue, which can influence performance 
- store the profiler cache on disk, reducing CUTLASS profiler overhead across different runs
- use the same set of default tile configurations as CUTLASS for sm80 https://github.com/NVIDIA/cutlass/blob/master/tools/library/scripts/generator.py#L1881
tqchen pushed a commit that referenced this pull request Apr 1, 2023
@spectrometerHBH @tqchen
[Unity][Contrib] Introduce several features of cutlass profiler (#14275)
3e66b20 
- allow Conv2d using different alignment factors for input and epilogue, which can influence performance 
- store the profiler cache on disk, reducing CUTLASS profiler overhead across different runs
- use the same set of default tile configurations as CUTLASS for sm80 https://github.com/NVIDIA/cutlass/blob/master/tools/library/scripts/generator.py#L1881
tqchen pushed a commit that referenced this pull request Apr 1, 2023
@spectrometerHBH @tqchen
[Unity][Contrib] Introduce several features of cutlass profiler (#14275)
e47cd36 
- allow Conv2d using different alignment factors for input and epilogue, which can influence performance 
- store the profiler cache on disk, reducing CUTLASS profiler overhead across different runs
- use the same set of default tile configurations as CUTLASS for sm80 https://github.com/NVIDIA/cutlass/blob/master/tools/library/scripts/generator.py#L1881
tqchen pushed a commit that referenced this pull request Apr 1, 2023
@spectrometerHBH @tqchen
[Unity][Contrib] Introduce several features of cutlass profiler (#14275)
b10e2a2 
- allow Conv2d using different alignment factors for input and epilogue, which can influence performance 
- store the profiler cache on disk, reducing CUTLASS profiler overhead across different runs
- use the same set of default tile configurations as CUTLASS for sm80 https://github.com/NVIDIA/cutlass/blob/master/tools/library/scripts/generator.py#L1881
@masahi
Copy link
Member

masahi commented Aug 10, 2023 •
edited
Loading

Just noticed that this PR was sent to unity, but it should have been sent to main. @spectrometerHBH please follow up.

The search for the output alignments added in this PR does nothing to our CUTLASS BYOC (since the C alignment doesn't seem to be used in codegen), but increases the conv2d profiling time by 4x. I was going to fix that on main and found that this code only exists on unity.

UPDATE: Sorry I did find that the C alignment is actually used here

tvm/python/tvm/contrib/cutlass/conv2d_operation.py

Line 248 in 907b29e

min(operation.C.alignment * DataTypeSize[operation.C.element], 128)
. But we shouldn't not search for all candidate alignments when profile_all_alignments is False, otherwise conv2d profiling becomes super slow.

@junrushao
Copy link
Member

junrushao commented Aug 10, 2023

Yeah we should probably send this commit to main as well. The co-existence of the branches does cause some trouble in maintenance

This was referenced Aug 16, 2023
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@junrushao junrushao junrushao approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@spectrometerHBH @tvm-bot @Hzfengsy @masahi @junrushao