Delay the initialization of CUDA tensor converter #3419
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/audio/3419
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 3 PendingAs of commit f6cc0ac: NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@mthrok has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: StreamReader decoding process is composed of the three steps; 1. Decode the incoming AVPacket into AVFrame 2. Pass AVFrame through AVFilter to perform post process 3. Convert the resulgint AVFrame The internal of StreamReader was refactored in pytorch#3188 so that the above pipeline is initialized at the time output stream is defined and output stream shape can be retrieved. For CPU decoder, this works fine because resizing happens in step 2, and the resulting shape can be retrievable. However, this is problematic for GPU decoder, as resizing is currently done using GPU decoder option (step 1) and there seems to be no interface to retrieve the output shape. This refactor introduced regression, which is described in pytorch#3405 AVFilter internally is adoptive to the change of input frame size. This commit changes the conversion process to be similar, so that it will wait until the first frame comes in to finalize the frame shape. Fix pytorch#3405 Pull Request resolved: pytorch#3419 Differential Revision: D46557505 Pulled By: mthrok fbshipit-source-id: 4cfe83a5b02fc372a5bc9fd5930711360dad0e93
|
This pull request was exported from Phabricator. Differential Revision: D46557505 |
|
Hey @mthrok. Some guidance:Use 'module: ops' for operations under 'torchaudio/{transforms, functional}', and ML-related components under 'torchaudio/csrc' (e.g. RNN-T loss). Things in "examples" directory:
Regarding examples in code documentations, please also use 'module: docs'. Please use 'other' tag only when you’re sure the changes are not much relevant to users, or when all other tags are not applicable. Try not to use it often, in order to minimize efforts required when we prepare release notes. When preparing release notes, please make sure 'documentation' and 'tutorials' occur as the last sub-categories under each primary category like 'new feature', 'improvements' or 'prototype'. Things related to build are by default excluded from the release note, except when it impacts users. For example: |
StreamReader decoding process is composed of the three steps;
The internal of StreamReader was refactored in #3188 so that the above pipeline is initialized at the time output stream is defined and output stream shape can be retrieved.
For CPU decoder, this works fine because resizing happens in step 2, and the resulting shape can be retrievable.
However, this is problematic for GPU decoder, as resizing is currently done using GPU decoder option (step 1) and there seems to be no interface to retrieve the output shape. This refactor introduced regression, which is described in #3405
AVFilter internally is adoptive to the change of input frame size. This commit changes the conversion process to be similar, so that it will wait until the first frame comes in to finalize the frame shape.
Fix #3405