Improve color accuracy of BT709 frames on CUDA #372
Conversation
facebook-github-bot
added
the
CLA Signed
| @@ -44,7 +44,7 @@ def assert_tensor_equal(*args, **kwargs): | |||
|
|
|||
| # Asserts that at least `percentage`% of the values are within the absolute tolerance. | |||
| def assert_tensor_close_on_at_least( | |||
| actual_tensor, ref_tensor, percentage=90, abs_tolerance=20 | |||
| actual_tensor, ref_tensor, percentage=90, abs_tolerance=19 | |||
There was a problem hiding this comment.
This tolerance is still a bit high I think, so might be good to see if it is due to limited color range vs full color range.
I had some good results with limited color range more closely matching the output of some videos.
The code I used was in https://github.com/fairinternal/amaia/blob/fmassa/video_reader/amaia_video/nv12_to_rgb.py
There was a problem hiding this comment.
In the torchvision decoder, this function was used. According to the NPP Documentation, it is noted that HDTV conversion assumes a full color range of 0 - 255; use the CSC version for limited range color.
Maybe using this function seems to significantly improve issues with color discrepancies.
There was a problem hiding this comment.
When I directly replaced the function with the CSC function, the difference in values between the CPU and CUDA results was only 2 across all values. While further validation is necessary, it seems much more accurate compared to a difference of around 20. I will provide more details in the issue.
libnpp actually has a specialized function to handle images with BT709 colorspace. So we use that instead of the generic one when appropriate. This improves the color accuracy of the image slightly.
Hat tip to @fmassa
Drive-by: fixed a test that doesn't make sense on CUDA -- we can only expect exactly equal frames on CPU.
Visual inspection. Before on the left and after on the right: