Standards and specifications

Moonfire NVR implements or uses various standards. They can be hard to track down. Here are a few links.

General notes on standards organizations

• ISO/IEC controls many of the relevant standards. Unfortunately you often need to jump through hoops to get copies of their standards.
  • It's unreasonable for an individual to pay for ISO/IEC standards. Not only are they often CHF 198 apiece, but to do anything you need a long list of interrelated ones. We're talking new car money here.
  • Older revisions of ISO/IEC standards are more likely to be available for free than current revisions and almost as good. ISO/IEC rarely breaks compatibility; more often revisions make minor clarifications and add new parts that are less widely implemented.
  • ISO/IEC sometimes removes standards from their publicly available standards page, but archive.org remembers.
  • Sometimes there's an equivalent standard from another organization that is free. For example, ITU T.REC H.264 is word-for-word identical to ISO/IEC 14496-10. Even within the list of ISO/IEC standards, sometimes there are near-equivalent standards. I believe some of ISO/IEC 14496-* is based on ISO/IEC 13818-*.
  • Sometimes you can find committee drafts of ISO/IEC standards which I'm told are effectively the same as the published standard, and possibly are redistributable (verify this first). Please edit this page if you have a reliable way of finding them.
  • Your local library may be able to help you get free access. Eg John Poole wrote that while his local library doesn't have access, they found for him the closest library which does.
• Many ITU standards are freely available, at least older versions.

Multimedia container formats

• .mp4 / ISO Base Media File Format (ISO-BMFF): ISO/IEC 14496-12:2015 is available as a zip file that was once linked from ISO's publicly available standards page. (See this archive.org copy of that page.) server/src/mp4.rs likely won't make any sense without at least skimming this standard.
  • Common Media Application Format (CMAF). Specified in ISO 23000-19. A draft is freely available.
• MPEG2 (including MPEG-TS): specified by ISO 13181-1 or ITU T-REC-H.222.0.
  • The 08/18 revision of T-REC-H.222.0 is available for free.
  • H.222.0 2000 amendment 3 specifies conveyance of H.264 over MPEG2.

Video codecs

Moonfire NVR mostly avoids knowing too much about video codecs. It doesn't encode video. Decoding uses ffmpeg's libavcodec and is only necessary for (in-development) on-NVR video analytics. However, it needs some understanding of these standards to properly place the encoded video frames into the container format and to describe features of the video such as its dimensions.

• H.264
  • ISO/IEC 14496-10:2014 is available as a zip file on ISO's publicly available standards page page. Parts of this help understand server/src/h264.rs.
  • ISO/IEC 14496-15 describes how H.264 is encoded into .mp4 files. I can't find a free, legal copy of the published standard.
    • There's a summary of the most important parts and some other relevant links on this blog post.
    • Working draft of amendment 1: Enhanced carriage of HEVC and support of MVC with depth information, including an extended definition of AVCDecoderConfigurationRecord.
    • Working draft of amendment 2: Carriage of high efficiency video coding (HEVC)
    • Recent meeting notes
• H.265: free download from T-REC-H.265 on ITU's website. Not yet implemented; see #33.

Audio codecs

TODO

ONVIF camera standards

ONVIF is the main standards body for IP security cameras. The full text of their standards and supporting WSDL/XSD files are linked from their Network Interface Specifications page.

These specifications have information that would be helpful in implementing many features:

• camera discovery, as in finding all cameras on a network segment.
• automatically learning cameras' RTSP stream URIs.
• consuming on-camera video analytics and audio analytics. (The vnd.onvif.metadata RTSP stream appears to be the most well-supported way to do this.)
• configuring the cameras from the Moonfire NVR UI, including video/audio configuration and analytics.

Streaming protocols

• RTSP
  • RFC 2326 defines RTSP/1.0, the version used by ONVIF IP cameras. It's defined in terms of other protocols, including SDP and RTP.
  • RFC 3550 defines RTP.
  • RFC 8866 defines SDP.
  • RFC 6184 defines the payload format for H.264 over RTP.
  • RFC 3640 defines the payload format for AAC over RTP. (It also defines the payload format for the older mp4 video format. This is different from H.264, and no one uses it anymore.)
• RTMP
  • https://github.com/veovera/enhanced-rtmp updates RTMP to include support for H.265/HEVC, VP9, and AV1 codecs. This support is implemented in OBS Studio.
• HTTP Live Streaming (HLS)
  • Specified in RFC 8216. A newer draft includes extensions such as LL-HLS (low-latency HLS).
  • Apple has a developer-focused website.
  • Akamai has a blog entry describing approaches for taking advantage of the LL-HLS extensions.
• MoQ (Media over QUIC) – a promising but still-evolving standard. (github org, official IETF working group info)
  • QUIC is defined in RFC 9000; skim this, particularly section 1.3 Notational Conventions, to make sense of the MoQ transport layer spec.
  • MoQ transport layer (github repo, latest editor's draft, official IETF status)
  • MoQ common catalog format (github, editor's drafts, official IETF status)
  • MoQ packaging format for CMAF (github, latest editor's draft, official IETF status)
  • MoQ packaging format for LOC — "low-overhead container" — a simple container format that is meant to make it easy to stuff media data into WebCodecs browser-side (github, official IETF status)