Twitch VOD download breaks at fixed points

[A13XIS]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2021.02.04.1
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

User Log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', '720p60', '--all-subs', 'https://www.twitch.tv/videos/66775413']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.04.1
[debug] Python version 3.8.5 (CPython) - Linux-5.8.0-41-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Proxy map: {}
[twitch:vod] 66775413: Downloading stream metadata GraphQL
[twitch:vod] 66775413: Downloading video access token GraphQL
[twitch:vod] 66775413: Downloading m3u8 information
[info] Writing video subtitles to: Dekay hat mal wieder in der Mülltonne rum gewühlt - Wildes Tier-Rennen-v66775413.rechat.json
[debug] Invoking downloader on 'https://dqrpb9wgowsf5.cloudfront.net/aafc46d0ca_herrdekay_20408790320_423767420/chunked/highlight-66775413-muted-PK4XCBTOS1.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1291
[download] Destination: Dekay hat mal wieder in der Mülltonne rum gewühlt - Wildes Tier-Rennen-v66775413.mp4
[download]  45.5% of ~1.13GiB at 19.56MiB/s ETA 00:50

ERROR: Did not get any data blocks
  File "/usr/local/bin/youtube-dl", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/__init__.py", line 475, in main
    _real_main(argv)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/__init__.py", line 465, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 2055, in download
    res = self.extract_info(
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 799, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 806, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 838, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 872, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 1679, in process_video_result
    self.process_info(new_info)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 1963, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 1902, in dl
    return fd.download(name, info)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/hls.py", line 149, in real_download
    success, frag_content = self._download_fragment(
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/fragment.py", line 104, in _download_fragment
    success = ctx['dl'].download(fragment_filename, fragment_info_dict)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/http.py", line 352, in real_download
    return download()
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/http.py", line 322, in download
    self.report_error('Did not get any data blocks')
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/downloader/common.py", line 165, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 628, in report_error
    self.trouble(error_message, tb)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 590, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

Description

Twitch VOD download throws "ERROR: did not receive any data blocks" in specific cases. This happens always at the same percentage (I suppose the same fragment). Trying to resume the download causes it to fail with "ERROR: did not receive any data blocks" immediately.

[A13XIS]

Other examples of broken VODs

• https://www.twitch.tv/videos/66775604 (At 28.6%)
• https://www.twitch.tv/videos/15981529 (At 36.0%)

The error trace is identical

[pukkandan]

Try with --skip-unavailable-fragment

[A13XIS]

Unfortunately this didn't change anything. Still getting same error with exact same trace.

[october262]

i was able to download & playback the video using the m3u8 file.

[A13XIS]

Can confirm. Downloading the m3u8 file and using ffmpeg to get the actual video works.

[mk-pmb]

I tried to reproduce with current master (be008e6) but got 403:

[twitch:vod] 66775413: Downloading stream metadata GraphQL
[twitch:vod] 66775413: Downloading video access token GraphQL
[twitch:vod] 66775413: Downloading m3u8 information
ERROR: Failed to download m3u8 information: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); […]

(Show trace)

[…] please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/mnt/████/youtube_dl/extractor/common.py", line 665, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/mnt/████/youtube_dl/YoutubeDL.py", line 2461, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

[dirkf]

Please open a new issue for this according to #30839 if appropriate, having regard to existing issues such as #22677, #30945, #31683.

I'll close this issue for now: let's see if it comes back with the next version of the extractor.