🗼See problems before debugging
- Experienced player developers spend a lot of time answering the question: is this bug caused by bad media, or our player? To answer the question they analyze the instrinsics (timing, etc.) of media to see if it's well-formed - we use ffprobe to do this, and it can be fairly time consuming to extract basic info. We also analyze the playlist itself to see if it corroborates the media.
- Inexperienced player developers aren't skilled in analyzing streams. ffprobe is a complex tool, and the commands to extract the data you need are arcane. And furthermore, they may not know the significance of timing, and how it can cause unsolvable playback errors.
- There are certain "must-haves" in media - for example, if you have a discontinuity during a live stream, you must use the DISCONTINUITY-SEQUENCE tag. A bot could take the extracted manifest/media information and run unit tests against them.
- Debugging media is pretty difficult, even for an experienced developer. Neat visualizations would help expose problems that may be hard to see when scrolling through text.
The solution to these above problems is being called "Media Lighthouse". The project was proposed at FOMS 2018 with a breakout session of player developers, including Shaka, Hls.js, JW, VideoJS, and Kaltura.
- Expose media information (both manifest and media intrinsics) via an API. Live streams require additonal work, and need to specify a capture window.
- Program a git bot to use this API and to automatically run it against new issues, leaving a comment with the media information.
- Write a unit testing framework with this API as the center, so that developers may write unit tests against media
- Create visualizations using this API, and wire it into the git bot.
- Wire this information into our player harness, so that we can correlate media playback with the underlying media ([~rob])
- Thumbcoil (https://github.com/videojs/thumbcoil) sounds like a great starting point for this effort.
- There is significant interest from other video developers around this project. Check out the #media-lighthouse chat in the video-dev slack to get involved. The goal of this project is to create an open-source tool for use by the video-dev community.