Regressed deinterlacing support as compared to i965-va-driver

[mamarley]

I have tried out media-driver 19.3.x on a few systems with Broadwell and Skylake CPUs (Specifically, HD Graphics 5500 and Iris 540) (reproduced with Linux 5.3, 5.4, and 5.5-rc) and have found that the hardware deinterlacing support doesn't work as well compared to the i965-va-driver. Here's what I have found:

1. If ENABLE_NONFREE_KERNELS=OFF is set, double-frame-rate deinterlacing doesn't work at all, resulting in 1080i30 content created from a 60FPS source playing jerkily at 30FPS instead of 60FPS as it did with the i965-va-driver. (The previous driver supported double-frame-rate even when the non-free features were disabled.) This is important for many news and sports broadcasts on 1080i ATSC channels.
2. If ENABLE_NONFREE_KERNELS=ON is set, double-frame-rate deinterlacing works only sometimes and with reduced quality compared to i965-va-driver. In VLC, the frame rate is always doubled, but the video seems subjectively fuzzier than with i965-va-driver and horizontal lines seem to "shimmer" up and down. In Kodi, it seems to alternate between no frame rate doubling (resulting in 30FPS) and the degraded frame rate doubling as I saw in VLC every few seconds at random intervals.

I apologize for not including any more information with this report, but there doesn't seem to be anything relevant in the logging output from either application and I'm not sure what else would be helpful. I will certainly provide any requested information though. Thanks!

[XinfengZhang]

thanks reporting, one question: are you using Bob DI or some advance DI?

[mamarley]

VLC doesn't seem to give me an option, but with Kodi I have tried both Bob and Motion-Adaptive and I see the same results with both.

[XinfengZhang]

@FurongZhang , @Jister could you help to check it

[yasij]

I noticed the same issue. I just tested 19.4.0r on Coffee Lake with ENABLE_NONFREE_KERNELS=ON . Motion Adaptive deinterlace looks exactly the same as Bob in Kodi. I notice the flickering line at the top of the screen. Motion-Compensated deinterlacing is not available. i965-va-driver presents Bob, Motion-Adaptive and Motion-Compensated. Motion-Adaptive and Motion-Compensated look much better, and don't have he characteristic Bob artifacts that intel-media-driver does.

[Jister]

Hi, @mamarley @yasij , could you please provide your test clip and reproducing steps so that we can reproduce the issue

[mamarley]

I have an example file, but it is too large to upload here. Is there any place you would prefer me to put it?

[yasij]

Here's a clip from a recording that shows the issue. I just took the first 6 MiB of the TS file:
sampledeint.ts.gz

Reproduction steps:
gunzip sampledeint.ts.gz
env LIBVA_DRIVER_NAME=iHD mpv --fs --hwdec=vaapi --vf=vavpp:deint=motion-adaptive sampledeint.ts

Look at the text. Notice that it bobs up and down like bob deinterlace
Now play it with the i965 driver:
mpv --fs --hwdec=vaapi --vf=vavpp:deint=motion-adaptive sampledeint.ts
The text doesn't not bob with i965.

i965 with the VAAPI bob deinterlacer
mpv --fs --hwdec=vaapi --vf=vavpp:deint=bob sampledeint.ts
The text bobs like iHD with the motion-adaptive filter.

The hardware is Intel Core i7-8700K UHD Graphics 630.

[yasij]

This might be the same issue as #640 . However, that one mentions that the reference frames aren't passed in VAProcPipelineParameterBuffer. I checked mpv, kodi and gstreamer. All of them are passing reference frames in VAProcPipelineParameterBuffer.

[yasij]

media_libva.h#L82

The driver defines DDI_CODEC_NUM_BK_REF and DDI_CODEC_NUM_FWD_REF as 0. That's why the reference frames aren't passed. The driver reports that max number of FWD and BK references as 0 in DdiMedia_QueryVideoProcPipelineCaps.

[Jister]

Hi, @yasij , I add the correct reference number in #820 , could you help to try the patch? Thanks

[mamarley]

This doesn't seem to have any effect for me. I still get 60fps with shimmering in VLC and alternating between 30fps and 60fps with no option for motion-compensated deinterlacing in Kodi.

[Jister]

Hi, @mamarley , I update #820 , could you help to have a try again? Thanks.

[mamarley]

With that patch both VLC and Kodi crash with a segmentation fault as soon as the video starts playing. That is probably because I am using 19.4 and not git master, but I'm not in a position to be able to try git master easily right now. Sorry. Perhaps @yasij can?

[yasij]

It'll be a few days before I can find time to test it, but I'll try it.

[yasij]

I just tried #820 on top of git master. motion-adaptive still has bad bob artifacts on the sample I submitted, and motion-compensated is not presented as an available option in kodi with iHD (but is with intel-vaapi-driver).

[yasij]

I'll check this again tonight. I might have forgotten to git commit after git apply of that patch before generating the archive to build the package.

[yasij]

I just tried it with #820 actually applied this time. It crashes at media_driver/linux/common/vp/ddi/media_libva_vp.c:2212 because to pSrc->pDeinterlaceParams is NULL. Did you test this before sending it out?

[yasij]

I fixed the crash by moving the allocate for pSrc->pDeinterlaceParams before the switch:
0001-Allocate-pDeinterlaceParams-before-using-it.patch.txt

Now both MADI and MCDI are available in Kodi. The text bobbing is gone, and some content looks better. However, the sample I posted above plays back extremely choppy in mpv and kodi with iHD. I will note this particular clip is a PAFF video recorded from TV. Can the iHD deinterlacer handle PAFF properly?

[Jister]

@yasij Thanks for your debugging. For PAFF video, can i965 driver playback normally? What parameters will app pass to driver for such video?

[yasij]

Yes, the i965 driver plays it back normally. There was a discussion about PAFF in Kodi a while ago when it was not playing correctly. See: xbmc/xbmc#15817 and the code change: xbmc/xbmc#15830
Before that change, kodi was toggling the deinterlace method flag to none and back everytime it encountered a progressive frame in PAFF. This caused stuttering with the i965 driver. From that bug report: "it's a PAFF video. mpv by the way copes with this by deinterlacing every frame no matter whether it is marked as interlaced (in default settings) :-)" They keep the post processing filter on with the deinterlace setting, and send the progressive frames or fields as they come from the PAFF video. Kodi now also does this (and previously did this in v17). This works with the i965 driver.

[yasij]

Is there an update on the status of this? #820 does allow MADI and MCDI to work, so I think it should be merged. A follow up issue can be created to fix the problems with PAFF.

[XinfengZhang]

#820 will be merged after some test.

[mamarley]

Any progress on getting this merged?

[Jister]

WIP, under internal test, will merge once test done.

[smp79]

I tried #820 with https://github.com/intel/media-driver/files/4075058/0001-Allocate-pDeinterlaceParams-before-using-it.patch.txt but MADI and MCDI still don't appear in Kodi. Tested with Kodi 19 current master.

[chcore]

@XinfengZhang @FurongZhang @LhGu any progress on this? It has been nearly 4.5 years since this was reported. If you won't merge #1162 (which works), why not devise a more suitable fix?

[teeedubb]

I am experiencing the second issue described by the OP on a alder lake GPU with kodi.
Happy to provide more info, but I am not seeing any info in any log files relating to the issues - only visual judder of deinterlaced content.
Posting here because the issue has been flagged as low impact, which is not the case for myself.

[LhGu]

It is in known issue list. https://github.com/intel/media-driver/wiki/Existing-Known-Issue-List#video-processing-forward-and-backward-reference

[teeedubb]

Are you able to provide any updates on fixes for this issue? This issue is almost 5 years old

[Simon566]

The devs are able to fix it , The patch is available. But they see it as a "minor" issue, even so all satellite tv programms in HD+ transmit in interlaced ^^ . No idea whats "minor" about it.

LibreElec is patching it with the existing , non merged patch and im doing it since over a year as well.

Maybe we should fork the repo in order to get it corrected

[smp79]

We should try to contact someone higher up at Intel regarding this. This is not a minor issue. Someone is obviously not doing their job.

[jahutchi]

We should try to contact someone higher up at Intel regarding this. This is not a minor issue. Someone is obviously not doing their job.

I whole-heartedly agree. I recently moved from LibreELEC to a pure Arch Linux installation and immediately noticed poor results in interlaced content... Took quite some time and research to stumble upon this thread after spotting that LE were patching their Intel-media-driver package. I then spent (even more) time figuring out how to patch and build my own version of the intel media driver on Arch, which did (of course) resolve the problem. How this issue has been around so long without the tried-and-tested known working patch being merged is quite beyond me. Shame on you intel!!!

[chcore]

I used to compile this project with #1162 for use with Kodi, but have since moved to the flatpak build. Anyone know how to apply the fix to flatpak apps? Compile a custom org.freedesktop.Platform.VAAPI.Intel? How do I even do that?

@LhGu what's the reason for not merging #1162?

[LhGu]

I used to compile this project with #1162 for use with Kodi, but have since moved to the flatpak build. Anyone know how to apply the fix to flatpak apps? Compile a custom org.freedesktop.Platform.VAAPI.Intel? How do I even do that?

@LhGu what's the reason for not merging #1162?

Actually, I am not the owner of this issue and fix merging does not depend on me. As I know, this fix is not enough and will cause other issues in some tests.

It is documented in intel known issue list. https://github.com/intel/media-driver/wiki/Existing-Known-Issue-List#video-processing-forward-and-backward-reference

[chcore]

I used to compile this project with #1162 for use with Kodi, but have since moved to the flatpak build. Anyone know how to apply the fix to flatpak apps? Compile a custom org.freedesktop.Platform.VAAPI.Intel? How do I even do that?
@LhGu what's the reason for not merging #1162?

Actually, I am not the owner for this issue and fix merging is not blocked by me. As I know, this fix is not enough and will cause other issues in some tests.

OK, but it is approaching 5 years without a fix. The comments in this issue and related pull request prove that there is a real impact.

What is the blocker exactly? Are there plans for a fix at all? Can this be escalated internally? Am I eligible for a refund on my intel boxes if there are no serious plans for a fix?

[Simon566]

there are always excuses and fingerpointing somewhere else, but nothing happens.

fact is that LibreElec users dont face any issue and im running Kodi on Ubuntu with latest intel-media-driver releases patched also not seeing any issues

but i do bet on that nothing will be happening again because:

• they see "some" issues in "some" tests that no other real user is seeing
• deinterlacing is very unimportant :-) :-) because we have millions of Astra 19.2E and other satellite users watching HD :-) :-)

if i wouldnt be affected , i would fall of my seat because im laughing so hard

[teeedubb]

Does anyone have any pointers on how to compile the media-driver with patch applied on ubuntu? I can compile the master branch of the media-driver on ubuntu 22.04 successfully by following the instructions in the readme, but when I apply the patches to master compilation fails. I have also tried pulling Jister's fork and ADI branch (which contains an older ersion of media-driver) and compiling, but I get the same error.

The majority of my use of an intel igpu is watching interlaced content, so a 12th gen cpu has been a downgrade compared to a gemini lake cpu.

Any help is greatly appreciated!

[FurongZhang]

I saw our maintainer @LhGu would not take care of this issue again. Let me continue on that. We had PR #1162 for this issue and had discussion in the PR thread.

[FurongZhang]

@teeedubb , sure, we will rebase Jason's PR based on the latest code.

I have requested my colleague to rebase Jason's PR. It will need 1-2 days.

[teeedubb]

@teeedubb , sure, we will rebase Jason's PR based on the latest code.

I have requested my colleague to rebase Jason's PR. It will need 1-2 days.

Are there any updates on the rebase?

[smp79]

I'm not sure if #1162 needs the rebase. No compile errors in LibreELEC build system.

[teeedubb]

I get a compile error with the patch applied, without the patch media-driver compiles successfully. I have tried on ubuntu 22.04 and 24.04 on multiple machines and get the same. I use master branch and compile libva and gmmlib as per the included instructions. Steps I follow for applying the patch + compiling and error I get are below.

cd ~/Downloads/software_source/intel-media-driver
git clone https://github.com/intel/media-driver.git
mkdir build_media
cd media-driver
git fetch origin pull/1162/head:vd
git switch vd
cd ../build_media/
cmake ../media-driver
make -j"$(nproc)"

[ 12%] Building CXX object media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp.o
/home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp: In member function 'void CodecHalEncodeSfcBase::GetCscMatrix(MHW_CSPACE, MHW_CSPACE, float*, float*, float*)':
/home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp:371:25: error: 'cscMatrix' may be used uninitialized [-Werror=maybe-uninitialized]
371 | MOS_SecureMemcpy(
| ~~~~~~~~~~~~~~~~^
372 | &cscCoeff[i3],
| ~~~~~~~~~~~~~~~
373 | sizeof(float) * 3,
| ~~~~~~~~~~~~~~~~~~
374 | &cscMatrix[i4],
| ~~~~~~~~~~~~~~~~
375 | sizeof(float) * 3);
| ~~~~~~~~~~~~~~~~~~
In file included from /home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/os/mos_os.h:32,
from /home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal.h:29,
from /home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal_encode_sfc_base.h:31,
from /home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp:28:
/home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/os/mos_utilities.h:1200:12: note: by argument 3 of type 'const void*' to 'MOS_STATUS MOS_SecureMemcpy(void*, size_t, const void*, size_t)' declared here
1200 | MOS_STATUS MOS_SecureMemcpy(
| ^~~~~~~~~~~~~~~~
/home/blaster/Downloads/software_source/intel-media-driver/media-driver/media_driver/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp:360:13: note: 'cscMatrix' declared here
360 | float cscMatrix[12];
| ^~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/build.make:1280: media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/agnostic/common/codec/hal/codechal_encode_sfc_base.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:286: media_driver/CMakeFiles/iHD_drv_video_OBJ.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

GCC version info from ubuntu 22.04:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-1ubuntu122.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-serialization=2
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu122.04)

I did search the cscMatrix error and found this, tried what it mentions, but it did not work.

This is all above my head unfortunately. If anyone has any pointers on how to compile I would really appreciate it.

[mamarley]

The patch for this bug seems to have been applied in 8017d9c!

[FurongZhang]

Yes. This issue is resolved by the patch 8017d9c.