Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mpv + yt-dlp + sponsorblock + seek = no new video frames #13033

Closed
Obegg opened this issue Dec 3, 2023 · 30 comments
Closed

mpv + yt-dlp + sponsorblock + seek = no new video frames #13033

Obegg opened this issue Dec 3, 2023 · 30 comments

Comments

@Obegg
Copy link

Obegg commented Dec 3, 2023

Important Information

Provide following Information:

  • mpv version:
mpv v0.37.0-56-g67aa5684 Copyright © 2000-2023 mpv/MPlayer/mplayer2 projects
 built on Dec  2 2023 12:38:53
libplacebo version: v6.338.0-60-g795600a-dirty
FFmpeg version: N-112889-g5230257ea
FFmpeg library versions:
   libavutil       58.32.100
   libavcodec      60.35.100
   libavformat     60.18.100
   libswscale      7.6.100
   libavfilter     9.14.100
   libswresample   4.13.100
  • Windows Version:
Edition	Windows 10 Enterprise LTSC
Version	21H2
Installed on	‎21/‎10/‎2023
OS build	19044.3693
Experience	Windows Feature Experience Pack 1000.19053.1000.0

Reproduction steps

While using mpv (zhongfly) + yt-dlp (master) + mpv_sponsorblock (PR 71 which skips poi_highlight) it seems like when seeking on this YouTube video as example (doesn't matter if it's auto seek or a manual seek) then yt-dlp will not show me any new frames from the video and I can only hear the audio track, or in some cases the stream itself will not load.

For testing purposes I've used mpv portable_config directory with the following commands on mpv.conf:

no-config
log-file="~~/yt-dlp-mpv-seek-issue.txt"

And added sponsorblock script with the following options on sponsorblock.conf:

categories=sponsor,selfpromo,interaction,intro,outro,preview,music_offtopic,filler,poi_highlight
skip_categories=sponsor,selfpromo,interaction,intro,outro,preview,music_offtopic,filler,poi_highlight

Note 1: Some information on the log file I've changed to GitHub.
Note 2: This issue has been bothering me for quite some time so it's not a new bug.
Note 3: The bug can be reproduced even without sponsorblock script, just a manual seek at the start of the stream to the middle of the stream.
Note 4: Created the same issue over at yt-dlp issue tracker.

Expected behavior

Show new video frames.

Actual behavior

No new video frames, only audio.

Log file

yt-dlp-mpv-seek-issue.txt

Sample files

portable_config.zip
https://www.youtube.com/watch?v=2QI88aLyaOs

@Obegg
Copy link
Author

Obegg commented Dec 7, 2023

Seems like this issue doesn't get any attention, so allow me to clarify.

When playing a video file (not a local file) using yt-dlp and then seeking forward (could be auto-skip using scripts and could be a manual skip) the result will be one out of two:

  1. The video will seek to the desired point, but no new video frames will draw, only audio will be heard.
  2. The video will seek to the desired point, but will not load either video and audio.

This doesn't seem to be an issue with yt-dlp, because I did open the same issue there and they believe it's mpv issue.
The logs show "EOF reached" while encountering the first outcome I mentioned:

[  11.665][v][lavf] EOF reached.
[  11.665][v][timeline] switch to segment 0
[  11.665][v][mkv] queuing seek to 540.790000
[  11.665][v][mkv] execute seek (to 540.790000 flags 32)
[  11.665][v][ffmpeg] stream level seek from 98304 to 9184314
[  11.665][d][ffmpeg] tcp: Starting connection attempt to IPv6 port 443
[  11.669][d][ffmpeg] tcp: Successfully connected to IPv6 port 443
[  11.692][v][mkv] seek done
[  11.692][v][timeline] seek done
[  11.692][d][timeline] stream 0: resize index to 128
[  11.740][d][timeline] stream 0: resize index to 256
[  11.770][d][timeline] stream 0: resize index to 512
[  11.809][v][mkv] EOF reached.
[  11.809][v][timeline] EOF reached.
[  11.809][v][vf] filter input EOF
[  11.809][v][vf] filter output EOF
[  11.809][v][cplayer] video EOF reached
[  11.809][d][cplayer] video EOF (status=4)
[  11.812][d][cplayer] video EOF (status=4)
[  11.812][v][cplayer] audio ready
[  11.812][d][cplayer] video EOF (status=4)
[  11.812][v][cplayer] starting audio playback
[  11.812][d][ao/wasapi] Thread Resume
[  11.812][d][ao/wasapi] Thread Reset
[  11.812][v][cplayer] playback restart complete @ 540.790000, audio=playing, video=eof
[  11.812][d][cplayer] video EOF (status=4)
[  11.813][d][cplayer] video EOF (status=4)
[  11.813][d][cplayer] video EOF (status=4)
[  11.823][d][cplayer] video EOF (status=4)
[  11.823][d][cplayer] video EOF (status=4)
[  11.838][d][cplayer] video EOF (status=4)
[  11.838][d][cplayer] video EOF (status=4)
[  11.843][d][cplayer] video EOF (status=4)
[  11.843][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)
[  11.858][d][cplayer] video EOF (status=4)

@mooreye
Copy link

mooreye commented Dec 21, 2023

I just rechecked and it seems that even without that sponsorblock LUA script, if you just click on a part of video on OSD that is not yet cached, only audio will be heard but no new frames will be displayed. So this seems to be an issue with mpv specifically.

I use yt-dlp version 2023.11.16 and mpv version 0.36.0.

Even reloading the file with the following script doesn't fix it:

function reload_file()
    local path = mp.get_property("path")
    if path ~= nil then
        local time = mp.get_property_number("time-pos")
        mp.commandv("loadfile", path, "replace", "start=" .. time)
    end
end

mp.add_key_binding("ctrl+r", "reload", reload_file)

@TheHardew
Copy link

For me the problem was ytdl-format set to prioritize the vp09 codec.

617     mp4   1920x1080   60    │ ~754.85MiB 5259k m3u8  │ vp09.00.41.08 5259k video only
303     webm  1920x1080   60    │  325.23MiB 2267k https │ vp9           2267k video only          1080p60, webm_dash

It might seem that format 617 is much better, but after downloading them, it turns out they are exactly the same.

If that doesn't help, you can debug by running mpv --no-config to test whether you can seek. If it works, you can then run just mpv and try to enable user scripts and config options inside of mpv.conf until it breaks.

@guidocella
Copy link
Contributor

--cache=no fixes seeking youtube videos for some reason. If you want to keep the cache you can use one of methods described in #8655 (comment)

@geextahslex
Copy link

@guidocella cache=no doesn't fix it in my case

@guidocella
Copy link
Contributor

Using av1 instead of vp9 in https://www.youtube.com/watch?v=PgHZ4itvgVs fixes seeking rather than being fixed by --cachie=no.

@geextahslex
Copy link

geextahslex commented Jun 30, 2024

But most other videos run fine with VP9, there must be a some difference that triggers this issue

@guidocella
Copy link
Contributor

I can reproduce being able to seek the AV1 stream and not the VP9 stream in ffplay so it's an ffmpeg issue.

@geextahslex
Copy link

As @Obegg is mentioning "frames" the only thing I can see is that the video that doesn't work has a weird FPS 29.97003.
Videos with VP9 60/30/25 FPS work well
fps

@Obegg
Copy link
Author

Obegg commented Jun 30, 2024

To be honest I am quite happy I am not the only one experiencing this, this gives us more information about this issue since more users are experiencing it and thereby reaching to the solution quicker.
I'm quite sure I already tested cache=no and it did not fix the issue (if it did I would have used it the second I found out it fixes the issue)
I have not tested other formats, only the default, I would love to test new formats, if someone is willing to share the command of how to do so in mpv.conf that would be nice.
the screenshot of the reply above me looks about right, the video will stop rendering new frames and I would only hear the audio (this is only one out of the two scenario which could happen as I mentioned 2nd reply)
I have no idea if it's a ffmpeg issue or not, I also not sure how I can determine it by myself since I probably lack the knowledge of doing so.

@guidocella
Copy link
Contributor

ytdl-format=(bestvideo[vcodec^=av01]/bestvideo)+bestaudio/best circumvents the issue.

@geextahslex
Copy link

geextahslex commented Jun 30, 2024

To be honest I am quite happy I am not the only one experiencing this

@Obegg I felt the same as I wasn't sure if to make the issue in the first place.

ytdl-format=(bestvideo[vcodec^=av01]/bestvideo)+bestaudio/best circumvents the issue

@guidocella A workaround is always nice but VP9 and VP9 2 is youtubes workhorse. Going forward it would be nice to have this issue solved

Would it be possible to only switch to AV01 if the faulty framerate is detected?

@WPMGPRoSToTeMa
Copy link

@guidocella A workaround is always nice but VP9 and VP9 2 is youtubes workhorse. Going forward it would be nice to have this issue solved

Would it be possible to only switch to AV01 if the faulty framerate is detected?

As far as I can see it is broken only for vp09.AB.CD.EF (e.g. vp09.00.40.08) and this codec is not always preferred by yt-dlp, e.g. I also see vp9 is preferred in some cases. I use the following selector to filter out the buggy codec:
ytdl-format=bv[vcodec!^=vp09.]+ba/b

@Obegg
Copy link
Author

Obegg commented Jun 30, 2024

I have a more fundamental question here, which might sound off-topic:
Isn't AV1 better than VP9?
I remember reading that AV1 is the new codec which is better quality and lower size, so why it isn't the default for everyone?
I remember seeing some commits on FFMPEG regarding AV1, haven't tested it out yet.
But if it is the case and AV1 is the best option - why isn't it the default on yt-dlp or mpv for that matter?
Am I missing something here?

@guidocella
Copy link
Contributor

yt-dlp/yt-dlp#5629

@geextahslex
Copy link

geextahslex commented Jun 30, 2024

I'm not sure but for me I have a better 4k performance with VP9 than with AV01
AV01
av01
VP9
vp9

@SnnUntz
Copy link

SnnUntz commented Jul 1, 2024

These can be used to solve the issue.
ytdl-raw-options = format-sort="proto"
or
ytdl-raw-options = extractor-args="youtube:skip=hls"
or
ytdl-format = "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

@geextahslex
Copy link

ytdl-raw-options = format-sort="proto"

what does it mean/do? ^^'

@SnnUntz
Copy link

SnnUntz commented Jul 1, 2024

ytdl-raw-options = format-sort="proto"

what does it mean/do? ^^'

# Download best video available via direct link over HTTP/HTTPS protocol,
# or the best video available via any protocol if there is no such video
$ yt-dlp -f "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

# Download best video available via the best protocol
# (https/ftps > http/ftp > m3u8_native > m3u8 > http_dash_segments ...)
$ yt-dlp -S "proto"

@geextahslex
Copy link

@SnnUntz

These can be used to solve the issue.
ytdl-raw-options = format-sort="proto"
or
ytdl-raw-options = extractor-args="youtube:skip=hls"
or
ytdl-format = "(bv*+ba/b)[protocol^=http][protocol!*=dash] / (bv*+ba/b)"

Thats amazing, all of them work. But what does it change? Why is it working like this? ^^'

@SnnUntz
Copy link

SnnUntz commented Jul 1, 2024

Thats amazing, all of them work. But what does it change? Why is it working like this? ^^'

The bug occurs when using the m3u8 protocol.

@WPMGPRoSToTeMa
Copy link

WPMGPRoSToTeMa commented Jul 1, 2024

The bug occurs when using the m3u8 protocol.

It's important to note that only vp9 via m3u8 is bugged, e.g. h264 via m3u8 works fine.

@SnnUntz
Copy link

SnnUntz commented Jul 2, 2024

The bug occurs when using the m3u8 protocol.

It's important to note that only vp9 via m3u8 is bugged, e.g. h264 via m3u8 works fine.

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

@geextahslex
Copy link

@SnnUntz

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

@SnnUntz
Copy link

SnnUntz commented Jul 2, 2024

After some research I found that this is an ffmpeg issue.
https://trac.ffmpeg.org/ticket/7359
https://trac.ffmpeg.org/ticket/11006

@WPMGPRoSToTeMa
Copy link

Then this should work too: ytdl-format = bv*[vcodec!*=vp09]+ba/b

Yeah, I mentioned a similar fix previously #13033 (comment).

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

If you use uosc like me, then it can be fixed in their .lua code. But yeah, this is exactly the reason I currently use ytdl-raw-options = extractor-args="youtube:skip=hls".

@SnnUntz
Copy link

SnnUntz commented Jul 2, 2024

Yes this works too, but you loose the ability to pick your resolution with something like ytdl-format=bestvideo[height<=?1440]+bestaudio/best

If you use uosc like me, then it can be fixed in their .lua code. But yeah, this is exactly the reason I currently use ytdl-raw-options = extractor-args="youtube:skip=hls".

You can also use it this way: bestvideo*[vcodec!*=vp09][height<=?1440]+bestaudio/best

@veganomy
Copy link

veganomy commented Dec 8, 2024

This is more of a manifest.googlevideo.com issue than m3u8/hls/ffmpeg/yt-dlp. HLS playback seems fine on many websites to me.
It seems to happen only when the server doesn't request the specific HLS fragment when you seek on your favorite player. Some servers support third party players. Some don't. Instead, they force sequential fragment unlock only. YouTube seems to be the one which does that.

That's why @guidocella's piping method seems to work, because you're restricting yourself to load fragments in sequential order and avoiding seeking issues.

@ValeZAA
Copy link

ValeZAA commented Dec 21, 2024

After some research I found that this is an ffmpeg issue. https://trac.ffmpeg.org/ticket/7359 https://trac.ffmpeg.org/ticket/11006

That is fixed btw

@Dudemanguy
Copy link
Member

Allegedly fixed upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants