KeyError crash when viewing video (may be due to content warning / age restriction?)

[robertknight]

Attempting to view https://www.youtube.com/watch?v=oIWqTIFRlYU in Via (https://via.hypothes.is/https://www.youtube.com/watch?v=oIWqTIFRlYU) results in a 5xx error from our backend: .

Error details from the response:

{
    "errors": [
        {
            "status": 500,
            "code": "KeyError",
            "title": "Something went wrong",
            "detail": "'captions'"
        }
    ]
}

This video does have an auto-generated transcript available on YouTube itself. The video is preceded by a content warning (it is about the Israel-Palestine conflict), which might be related.

[robertknight]

Stack trace from local Via instance:

Traceback (most recent call last):
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/tweens.py", line 41, in excview_tween
    response = handler(request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid_sanity/tweens.py", line 34, in tween
    return check(request, handler, registry)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid_sanity/tweens.py", line 109, in ascii_safe_redirects_tween_factory
    response = handler(request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/router.py", line 143, in handle_request
    response = _call_view(
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/sentry_sdk/integrations/pyramid.py", line 91, in sentry_patched_call_view
    return old_call_view(registry, request, *args, **kwargs)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/view.py", line 674, in _call_view
    response = view_callable(context, request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/config/views.py", line 170, in attr_view
    return view(context, request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/config/views.py", line 196, in predicate_wrapper
    return view(context, request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/viewderivers.py", line 319, in secured_view
    return view(context, request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/viewderivers.py", line 427, in rendered_view
    result = view(context, request)
  File "/Users/robert/hypothesis/via/.tox/dev/lib/python3.8/site-packages/pyramid/viewderivers.py", line 141, in _requestonly_view
    response = view(request)
  File "/Users/robert/hypothesis/via/via/views/api/youtube.py", line 21, in get_transcript
    transcript = request.find_service(YouTubeService).get_transcript(video_id)
  File "/Users/robert/hypothesis/via/via/services/youtube.py", line 110, in get_transcript
    transcript_infos = self._transcript_svc.get_transcript_infos(video_id)
  File "/Users/robert/hypothesis/via/via/services/youtube_transcript.py", line 95, in get_transcript_infos
    dicts = json["captions"]["playerCaptionsTracklistRenderer"]["captionTracks"]
KeyError: 'captions'

The JSON response which was parsed into the json variable here looks like https://gist.github.com/robertknight/75521c28cb7f087cdedea9e5574c222c.

[robertknight]

If you try to view this video on docdrop.org, which uses the same unofficial APIs to get the transcript, you get an error about the transcript not being available: https://docdrop.org/video/oIWqTIFRlYU/. I get the same result using the youtube-transcript-api Python library directly - https://github.com/jdepoix/youtube-transcript-api.

On that basis, I think a minimal acceptable fix here would be to return a nicer error message. If the problem is confirmed to be due to a content warning, then it would be helpful if the error message mentioned that.

[robertknight]

Likely related upstream issue: jdepoix/youtube-transcript-api#111

The text displayed to the user on YouTube mentions a content warning rather than an age restriction, but I note that the JSON API response linked above indicates that YouTube treats this kind of "upsetting content warning" like age-restricted content.

[seanh]

Via doesn't use https://github.com/jdepoix/youtube-transcript-api/ anymore so this will be a bug in our own code