Proposed fix to crashing of the CameraWebServer example

[Funky118]

Hardware:

Board: AI Thinker ESP32-CAM
Core Installation version: 1.0.6
IDE name: Arduino IDE, Platform.io
Flash Frequency: 80Mhz
PSRAM enabled: yes
Upload Speed: 115200
Computer OS: Windows 10

Description:

I've encountered this old bug when borrowing some of the code from the CameraWebServer example in Arduino IDE.

So I'd gone through the code and found this section under app_httpd.cpp:

`

box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);

if (net_boxes){
    detected = true;
    if(recognition_enabled){
        face_id = run_face_recognition(image_matrix, net_boxes);
    }
    draw_face_boxes(image_matrix, net_boxes, face_id);
    free(net_boxes->score);
    free(net_boxes->box);
    free(net_boxes->landmark);
    free(net_boxes);
}

`

Since the person who wrote it was trying to free a pointer from, presumably, the stack, I suspect this is what's causing occasional crashes for some users. And indeed, when removed, the code runs smoothly.

Could someone please look into it? This is my first issue ever and I don't currently have the time to google how to submit a bug fix.
Thanks.

[Funky118]

Oh and I've only needed to copy pasted the still taking part. This bug also occurs in the streaming part.

[yukiman76]

I can also confirm that this fixes the Heap Crashes on CameraWebServer example, Project

[easytarget]

You are 'fixing' it by disabling face detection entirely. Possibly a valid approach, depending on whether you want face detection ;-)

I was hoping that free(net_boxes->category); added to the list the free list above would work, box_array_t is defined in ~.arduino15/packages/esp32/hardware/esp32/1.0.6/tools/sdk/include/esp-face/image_util.h, and I was assuming that this was failure to free part of the boxes structure.

But no joy, I think the corruption lies deeper in the ESP libs.

I also added debug to my example so it dumped the heap and free space after every frame, and I do not see any signs of memory leaks.The error happens out of the blue.

This is repeatable, it happens every time face detection is on and a face is detected in frame.
It is independent of whether face recognition is also enabled.

[Funky118]

You are 'fixing' it by disabling face detection entirely. Possibly a valid approach, depending on whether you want face detection ;-)

I was hoping that free(net_boxes->category); added to the list the free list above would work, box_array_t is defined in ~.arduino15/packages/esp32/hardware/esp32/1.0.6/tools/sdk/include/esp-face/image_util.h, and I was assuming that this was failure to free part of the boxes structure.

But no joy, I think the corruption lies deeper in the ESP libs.

I also added debug to my example so it dumped the heap and free space after every frame, and I do not see any signs of memory leaks.The error happens out of the blue.

This is repeatable, it happens every time face detection is on and a face is detected in frame.
It is independent of whether face recognition is also enabled.

I'm afraid I didn't explain myself with clarity in mind. I've only removed the calls to free(), rectangles would still be drawn around faces when I've tested it. Could you please check if just deleting the free()'s results in memory leaks? I suspect there's nothing being allocated that should need freeing, but I'm not certain.

[easytarget]

Light dawns; I'll try that properly again. I did briefly try commenting the free()s out earlier today, but was being a bit chaotic so I'll do it more systematically this time.

Edit: Yes, Looking good so far. Boxes drawn and no errors. I realise I only commented the free()s out of the image capture block, not the stream. Which is why I assumed this wasn't working earlier. Mea Culpa.

[andrewfhart]

@Funky118 @easytarget -- I was just suffering from this as well, and did some research. This blog post [1] uses dl_lib_free [2] instead of free to release memory in a similar use case. Replacing the relevant free calls with dl_lib_free as follows eliminated the corrupt heap issue for me. Be sure to apply it in both the 'still' and 'streaming' portions of the code, as applicable.

box_array_t *net_boxes = face_detect(image_matrix, &mtmn_config);

if (net_boxes){
    detected = true;
    if(recognition_enabled){
        face_id = run_face_recognition(image_matrix, net_boxes);
    }
    draw_face_boxes(image_matrix, net_boxes, face_id);
    dl_lib_free(net_boxes->score);
    dl_lib_free(net_boxes->box);
    dl_lib_free(net_boxes->landmark);
    dl_lib_free(net_boxes);
}

[1] https://techtutorialsx.com/2020/06/13/esp32-camera-face-detection/
[2] https://github.com/espressif/esp-face/blob/b4a2ef17043b5d62564cdb71d190d1f36f38f171/lib/include/dl_lib_matrix3d.h#L115

[maddisondesigns]

I've been having issues with getting Face Detection working as well. I just tried @andrewfhart's suggestion of replacing the call to free() with dl_lib_free() instead. I did this within app_httpd.cpp in both the capture_handler() and stream_handler() functions and now Face Detection and Face Recognition both work.

So to clarify, in both the above mentioned functions the code was:

draw_face_boxes(image_matrix, net_boxes, face_id);
free(net_boxes->score);
free(net_boxes->box);
free(net_boxes->landmark);
free(net_boxes);

and this has now been changed to the following to get it working.

draw_face_boxes(image_matrix, net_boxes, face_id);
dl_lib_free(net_boxes->score);
dl_lib_free(net_boxes->box);
dl_lib_free(net_boxes->landmark);
dl_lib_free(net_boxes);