H.265 support

[scottlamb]

I think this is fairly simple to add (very similar to src/h264.rs) but there are a couple caveats:

• I haven't found a freely available spec. Old versions of most of the ISO/IEC 14496 standards are freely available but I haven't seen this. We could probably puzzle together what to do based on ffmpeg source code or something but that's not ideal.
• cameras might always do B-frames along with H.265, which would require reworking the video_index format.

It'd be nice to have though. My Dahua camera supports H.265, and I understand it's a significant improvement in video quality per byte.

[scottlamb]

Ahh, there's a draft of ISO/IEC 14496-15 here.

[jlpoolen]

Some points regarding High Efficiency Video Coding (HEVC), or spcifically H.264 vs. H.265:

• patented technology: moonfire would be decoding, don't know if license is needed to decode H.265 content.
• h.265 is subsequent to h.264 and HEVC promises a 50% storage reduction as its algorithm uses efficient coding by encoding video at the lowest possible bit rate while maintaining a high image quality level.
• HEVC/H.265 comes with the trade-off requiring almost 10x more computing power. [Boldface added.]

Source: https://medium.com/advanced-computer-vision/h-264-vs-h-265-a-technical-comparison-when-will-h-265-dominate-the-market-26659303171a

See also: http://trac.ffmpeg.org/wiki/Encode/H.265

[scottlamb]

Yeah, those might be good considerations to add to a future "choosing your codec" section. I definitely don't want to drop H.264 support so folks would be able to choose (stream-by-stream) which they use. They could for example do video analytics on H.264 sub streams and have H.265 main streams that Moonfire NVR never decodes, only passes along to the client.

IIRC the Raspberry Pi 4 has hardware decoding support and a matching license for both H.264 and H.265. So I think neither the patent nor the decoding speed are necessarily a problem in some common scenarios. I'm not a lawyer and haven't sought legal advice on this, though, so take that with a grain of salt.

Also, on-NVR video analytics will always be optional. They're not even fully implemented yet (#30) (though I really want to fix that). And we can support on-camera video analytics (#29) as a CPU-friendlier alternative.

[scottlamb]

I'm demoting this from the 1.0? milestone to wishlist. Neither Chrome nor Firefox supports H.265 (https://caniuse.com/hevc). That means Moonfire NVR couldn't just store H.265 from the cameras and stream it unmodified to those browsers as it does for H.264. When using H.265, we'd have to do one of the following:

• Users would have to stick with Safari or use a (presumably quite slow) pure software Javascript/WebAssembly H.265 decoder.
• Or we'd have transcode on playback (and have a slower "download .mp4 video for export" process).

[ghost]

1. Mozilla (and possibly google) will not support HEVC while it is encumbered by patents (link)

2. Chips&Media Launches AV1 Video Encoder Hardware (link)

So it is possible that in a couple of years cheaper (since no royalties) cameras with AV1 support will appear :)

[jlpoolen]

Scott indicated that Reolink has upgraded their version of LIVE555 used as the rtsp servers in the Reolink cameras. The previous version was from 2013 and recent inquiry and discussion apparently caused Reolink to upgrade the server contained therein.

(I'll note I tried to reach out to the owner of the LIVE555 project about Reolink and my inquiries went unanswered and the maintainer refuses to allow posting from my gmail account despite the fact that I'm a developer with a major software company for 22 years. I'm too tired of trying to maintain my own mail server, it's not worth the time.)

Reolink currently is offering an 8 megapixel camera which send h.265 as its main stream. I posed a question about h.264 support and on October 8th a responder identified by Amazon as a "ReolinkDirect Seller " responded with:

  Hello, for Reolink 8MP cameras, the compression type of main-stream is H.265 while the sub-stream is H.264. 

Given Scott's rapport with Reolink, I'd like to suggest he confirm with Reolink this design decision: h.265 for main and h.264 for sub streams. I have older Reolink cameras which have the LIVE555 2013 version and streams are really full of interruptions, in part caused by a possible network interface problem with my Raspberry Pi. Any rate, just to make the problem go away, I'd like to replace the problem cameras with the current offerings. Having cameras where I have them has proven invaluable: captures 2nd class felony theft from a contractor's truck, squatters, car thieves, and homeless people creating small warming fires and indecently exposing themselves. Just the presence of the cameras and the police advising trespassers that we have it all captured on video makes them wary of returning. Moonfire-nvr has come through capturing full streams where Reolink's capture-per-motion-detection-settings have missed a lot. All this is to say h.265 would be welcomed and it appears it's the direction camera manufacturers are heading. But, and that's a big one, h.265 is patented technology and I cannot opine about the legality of decoding. Approaching the licensing manager for h.265 might be something to consider to see what their position is about a project such as moonfire-nvr decoding streams that have been legitimately encoded --- maybe they only want to collect license fees from the encoding side?

[scottlamb]

I actually tried out one of Reolink's 8MP cameras for a bit. I wrote a bit about it on the wiki (just added a bit more). tl;dr: I recommend the 5MP models over the 8MP models. The 8MP has the H.265 problem, and it just doesn't have a large enough sensor for that many pixels anyway. I expect it to have worse images than the 5MP models at night.

All the other manufacturers I've seen have let the user choose between H.265 and H.264 in the camera's config UI. I hope Reolink starts doing the same.

But, and that's a big one, h.265 is patented technology and I cannot opine about the legality of decoding. Approaching the licensing manager for h.265 might be something to consider to see what their position is about a project such as moonfire-nvr decoding streams that have been legitimately encoded --- maybe they only want to collect license fees from the encoding side?

Even if they said "sure, open source projects can do whatever", there'd still be the technical problem. Other than Safari, browsers don't support H.265. Presumably this is also due to the licensing but it doesn't really matter why; they just don't. So we're stuck with the options I mentioned above.

[MikePfaff]

According to Reolink, their 8MP cameras will automatically switch from H.265 to H.264, if the stream resolution is lower than 8MP:

Yes, for the new version 8MP camera, if the resolution is higher than 8MP, the video format will switch to H.265. If it is less than 8MP, it will be H.264.

One user has confirmed it on his RLC-820A and Reolink posted it on a RLC-842A thread, which leads me to believe that this undocumented "lower stream resolution to get H.264" trick might also work on some (or even all) other Reolink 8MP (and higher MP?) camera models?

[jlpoolen]

The Reolink Duo is really two cameras in one unit where the feeds are from two sensors positioned such that they cover 180 degrees with no gaps. Their streams are stitched together into one stream which is 8MP and encoded using H.265. The really neat thing about this camera is that you just place it some place where you want 180 degree coverage, like near a straight sidewalk, and its done, you do not have to install two cameras (and most cameras do you support 90 degree viewing angles).

So the nature of the 8MP beast has changed with this shift of stitching together two 4 MP streams to create a panorama view. I have an extensive video surveillance system and one problem I have faced is what happens a bit further down the street when a person of interest goes beyond my camera's field of view. The 180 degree coverage is an important advancement.

[jlpoolen]

Here's an example of the day and night versions of the sub stream:

[scottlamb]

Yeah, that stitching is nice for the situation. In terms of Moonfire support, there is another user interested in implementing H.265 in Retina, and from there it'd be pretty easy to support Moonfire recording H.265. But actually playing it back in a web UI is another matter. Browser support is still bad: https://caniuse.com/hevc

In theory we could also add support for realtime transcoding in Moonfire, e.g. decoding the H.265 and encoding as H.264, given adequate hardware. The Rockchip RK3588 should be able to do this for example, or some desktop-class GPUs, or add-in accelerator cards such as NETINTs. (Coincidentally, I was talking with them in person today. Their older "Logan" generation T432 cards are down to $300 iirc.)

[jlpoolen]

Several weeks ago I spent some time with Perl using the API and if I recall, I was able to easily pull segments from Moonfire. I'm probably the out-in-left-field use case, but having the H.265 files may be useful even though they cannot be played back on most web browsers at this time. I still have to compare the quality of the "main" feed at H.265 vs. "sub" feed at H.264 to see if there is an appreciable difference. With the 180 degree camera and the ability to see way beyond, having better resolution could be a real plus. I'll update here with my findings and examples supporting.

[jlpoolen]

Here's a study documenting the difference between the 4MP feed encoded with H.264 and the 8MP feed encoded with H.265: https://salemdata.net/jlp/reolink/h265/reolink_duo_h265_h264_comparison.html

I spoke with a police officer on an unrelated matter and he mentioned that many people share videos with them, but the lack of resolution practically makes the videos useless. So from law enforcement's perspective, the higher the resolution and ability to discern numbers and features, the better. It would be nice to be able to use Moonfire-nvr to preserve the H.265 feeds even though they cannot be displayed in the project's web interface. I'd use the existing H.264 content to review knowing that if I have a really important segment, I can retrieve a noticeably better quality stream to hand to law enforcement. But I live in a pretty high crime area... that's the price I pay for cheaper real estate.

[scottlamb]

The h265 branch has a version of this. Feedback welcome, if anyone cares to give it a spin. There's no schema change necessary; the db format was already codec-agnostic.

[Suxsem]

hi @scottlamb , any news on this? can you please sync the h265 branch with master and publish a docker package so I can test it? thank you!

[scottlamb]

@nalonsor1 I opened a new issue #331 for interoperability with your camera. From those logs, the problem is unrelated to #33; the camera is using H.264 rather than H.265.