add cuctc tutorial, change blank skip threshould into prob #3297
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3297
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 3 Unrelated FailuresAs of commit e8a870c: BROKEN TRUNK - The following jobs failed but were present on the merge base 84b1230:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hello @yuekaizhang, Many thanks for your CUDA Implementation. When it will be released in the pip install branch I will definitely make sure that we support it in SpeechBrain. In the meantime, can I ask you if you have any documentation that explain how you did your implementation ? This is something that could be relevant to the community to have a PyTorch version which does not involve custom CUDA kernels. Indeed, it is easier for people to directly modify PyTorch code than CUDA code when one wants to modify/implement research papers. Hence, I'm asking you if you could elaborate a little bit more on how we could re-implement what you did but with PyTorch directly. Many thanks! Adel |
|
Thanks. @mthrok Do you know when it will be available in pip install ? We would have a detailed documentation about how the cuda kernels were implemented in one or two months later. Unfortunately, it can't be reimplemented with pure pytorch. (Or you would get worse or similar performace with current cpu version decoder in torchaudio with pure pytorch.) Maybe you could try openai triton for python programmers, which I thought will be slower comparing with cuda. |
|
Ok good! Tbh I don't think Triton will be a good alternative to your implementation. The CPU kernel overheads hits really hard with Triton especially in this case where we have a for loop on the time steps. Btw did you tried CUDA Graph with your implementation? I guess it could improve the results by a lot since there is a for loop. And do you know if you are planning to integrate support for kenLM ? Thanks again. |
We have not tried cuda graph yet. We have implemented skip frames based on blank frame probs, which converts the frame sync decoding into label sync decoding. |
|
Thanks for the answers! Do you have a timeline in mind of when your code will be available in the pip install branch please? |
|
Hi folks Sorry for the belated response. The next PyTorch release is scheduled at Oct 2023. https://dev-discuss.pytorch.org/t/pytorch-release-2-1-0/1271 The CUCTC code is accessible through nightly channel, so if you would like to try out, please refer to https://pytorch.org and install nightly build. @yuekaizhang Sorry for not responding these days, I will review this soon. |
|
@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Hey @mthrok. Some guidance:Use 'module: ops' for operations under 'torchaudio/{transforms, functional}', and ML-related components under 'torchaudio/csrc' (e.g. RNN-T loss). Things in "examples" directory:
Regarding examples in code documentations, please also use 'module: docs'. Please use 'other' tag only when you’re sure the changes are not much relevant to users, or when all other tags are not applicable. Try not to use it often, in order to minimize efforts required when we prepare release notes. When preparing release notes, please make sure 'documentation' and 'tutorials' occur as the last sub-categories under each primary category like 'new feature', 'improvements' or 'prototype'. Things related to build are by default excluded from the release note, except when it impacts users. For example: |
Add a separate tutorial for cuctc.
Reslove #3096