Decoding full-range yuv444p to RGB differs from ffmpeg

[tom-bola]

Overview

When decoding a full-range yuv444p video stream and converting to RGB like so

import av
from PIL import Image
with open(video_path, 'rb') as file:
    with av.open(file) as container:
        for pack in container.demux(0):
            for frame in pack.decode():
                rgb = frame.to_ndarray(format='rgb24')
                Image.fromarray(rgb, mode='RGB').show()

I found that the resulting images are different than what I get when extracting images with ffmpeg:

ffmpeg -I video.mov video%06d.png

Expected behavior

The expected behaviour is that the resulting RGB is very close to what is generated by running the above ffmpeg command. For other formats (e.g. limited range yuv420p) this is the case.

Actual behavior

The actual behaviour is that the images differ significantly. It seems like the RGB created with PyAV is scaled incorrectly, using more range than the reference:

import matplotlib.pyplot as plt
plt.figure()
plt.scatter(rgb_ffmpeg[::10, ::10, :].ravel(), rgb_pyav[::10, ::10, :].ravel(), s=1, marker='.')
plt.xlabel('RGB from ffmpeg')
plt.ylabel('RGB from pyav')

Investigation

I found that specifying the src_color_range (or dst_color_range until av 12.0.0) fixes the scaling:

from av.video.reformatter import ColorRange
rgb = frame.to_ndarray(format='rgb24')                                    # does not work in either 12.0.0 or 12.1.0
rgb = frame.to_ndarray(format='rgb24', src_color_range=ColorRange.MPEG)   # works in 12.0.0 and 12.1.0
rgb = frame.to_ndarray(format='rgb24', dst_color_range=ColorRange.MPEG)   # works in 12.0.0 but not in 12.1.0

Reproduction

To create a video with yuv444p full-range I used the following command

fmpeg -i yuv420p.mov -vf scale=in_range=limited:out_range=full -color_range 2 -pix_fmt yuv444p -c:v hevc -tag:v hvc1 yuv444p_full_range.mov

This is the video that I tested with:

yuv444p_full_range.mov

Versions

• OS: MacOS 14.4
• Pyton 3.11.5 (macOS-14.4-arm64-arm-64bit)
• PyAV runtime:

PyAV v12.1.0
library configuration: --disable-static --enable-shared --libdir=/tmp/vendor/lib --prefix=/tmp/vendor --disable-alsa --disable-doc --disable-libtheora --disable-mediafoundation --disable-videotoolbox --enable-fontconfig --enable-gmp --disable-gnutls --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwebp --disable-libxcb --enable-libxml2 --enable-lzma --enable-zlib --enable-version3 --extra-ldflags='-Wl,-ld_classic' --enable-libx264 --disable-libopenh264 --enable-libx265 --enable-libxvid --enable-gpl
library license: GPL version 3 or later
libavcodec     60. 31.102
libavdevice    60.  3.100
libavfilter     9. 12.100
libavformat    60. 16.100
libavutil      58. 29.100
libswresample   4. 12.100
libswscale      7.  5.100

Research

I have done the following:

• Checked the PyAV documentation
• Searched on Google
• Searched on Stack Overflow
• Looked through old GitHub issues

Additional context

Related Issue#1378

[WyattBlue]

I don't think this is possible to fix in the general case because nobody has actually bothered to specify what frame.to_ndarray should actually do and which properties are most important.

[tom-bola]

Even if the behaviour isn't formally specified: the minimal assumption that PyAV users have is that the default behaviour is matching that of ffmpeg. The src_color_range and dst_color_range parameters of frame.to_ndarray behave in wrong/unexpected ways. IMHO this is an obvious bug and the reference behaviour is well defined (which is: to match ffmpeg). I propose to leave this bug open.