You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Narrow is a very cheap operation in forwards, because it doesn't have to do much, but it is expensive in backwards, since you must allocate a full size tensor and zero pad it out. This is a good opportunity for the optimizer.
Narrow is a very cheap operation in forwards, because it doesn't have to do much, but it is expensive in backwards, since you must allocate a full size tensor and zero pad it out. This is a good opportunity for the optimizer.
Discovered this in the conversation at pytorch/pytorch#3313 (comment)
The text was updated successfully, but these errors were encountered: