CCTV and Motion Detection

⚠️ You are viewing the v0.7 Alpha documentation. If you are not cloning the latest code from this repository or using dev packages from MyGet, then you may wish to look at the v0.6 examples instead. This section is still being written for v0.7

Revisions

v0.7 Alpha
v0.6 (Current)

Motion Detection Basics

Frame differencing

Frame differencing is a common motion-detection technique whereby a test frame (sometimes called the "background frame") is compared against new frames (or "current frame") for changes exceeding various thresholds. The MMALSharp library has new APIs and classes that let you configure motion detection behavior, including callbacks to run custom code when motion is detected.

There are different strategies to detect differences between frames. The provided implementation combines two techniques which help reject sensor noise and small localized motion (such as an insect, or even a small pet).

At the most basic level, the algorithm compares individual pixels. This is called "RGB summing" because the red, green, and blue values are added together for each pixel in both images. If the difference between the test frame and the new frame exceeds a threshold, the pixel is considered changed. The image is subdivided into a grid of smaller rectangles called cells. The size of each cell and the number of pixels in the cell depends on the image resolution. There is a second threshold which defines the percentage of pixels in the cell which must change for the entire cell to be considered changed. This is how sensor noise and other minor changes are discarded. Finally, there is a third threshold, which is the number of cells across the entire image that must register changes in order to signal that motion detection has occurred. This is how real but small and unimportant motion is ignored (insects, pets, and distant background movement, for example). All of these thresholds are configurable.

Typically motion detection doesn't require or benefit from high resolution. 640 x 480 should be adequate, although you should always feed raw RGB24, RGB32, or RGBA images into the system. Image artifacts from lossy compression algorithms like h.264 will be mistaken for motion and the RGB summing algorithm is not compatible with the YUV pixel format. At 640 x 480 x RGB24, a Raspberry Pi 4B can easily process full-motion video using the provided algorithms (an improvement over v0.6 which could only process about 5 frames per second on the same hardware).

The new FrameBufferCaptureHandler class provides management and control of motion detection. The following example demonstrates the most basic possible motion detection. This does nothing but write messages to the console when motion is detected.

public async Task SimpleMotionDetection(int totalSeconds)
{
    // Assumes the camera has been configured.
    var cam = MMALCamera.Instance;

    using (var motionCaptureHandler = new FrameBufferCaptureHandler())
    using (var resizer = new MMALIspComponent())
    {
        // The ISP resizer is used to output a small (640x480) image to ensure high performance. As described in the
        // wiki, frame difference motion detection only works reliably on uncompressed, unencoded raw RGB data. The
        // resizer outputs this raw frame data directly into the motion detection handler.
        resizer.ConfigureInputPort(new MMALPortConfig(MMALEncoding.OPAQUE, MMALEncoding.I420), cam.Camera.VideoPort, null);
        resizer.ConfigureOutputPort<VideoPort>(0, new MMALPortConfig(MMALEncoding.RGB24, MMALEncoding.RGB24, width: 640, height: 480), motionCaptureHandler);

        cam.Camera.VideoPort.ConnectTo(resizer);

        // Camera warm-up.
        await Task.Delay(2000);

        // We'll use the default settings for this example.
        var motionConfig = new MotionConfig(algorithm: new MotionAlgorithmRGBDiff());

        // Duration of the motion-detection operation.
        var stoppingToken = new CancellationTokenSource(TimeSpan.FromSeconds(totalSeconds));
        Console.WriteLine($"Detecting motion for {totalSeconds} seconds.");

        await cam.WithMotionDetection(
            motionCaptureHandler,
            motionConfig,
            // This callback will be invoked when motion has been detected.
            async () =>
            {
                // When motion is detected, temporarily disable notifications
                motionCaptureHandler.DisableMotionDetection();
                        
                // Wait 2 seconds
                Console.WriteLine($"\n     {DateTime.Now:hh\\:mm\\:ss} Motion detected, disabling detection for 2 seconds.");
                await Task.Delay(2000, stoppingToken.Token);

                // Re-enable motion detection
                if(!stoppingToken.IsCancellationRequested)
                {
                    Console.WriteLine($"     {DateTime.Now:hh\\:mm\\:ss} ...motion detection re-enabled.");
                    motionCaptureHandler.EnableMotionDetection();
                }
            })
            .ProcessAsync(cam.Camera.VideoPort, stoppingToken.Token);
    }

    cam.Cleanup();
}