Unable to restart camera with aravissrc gstreamer

[sean-kiwi]

Describe the bug
I need the ability to remove and recreate the arravissrc element of the gstreamer pipeline, so that if my camera has any issues I can restart it without having to restart my entire pipeline.

To Reproduce

• Create aravissrc element
• Attempt to destroy and create new aravissrc element
• Attempt to start new element

Expected behavior
Camera able to restart without crashing gstreamer pipeline

Camera description:
Lucid Pheonix
GigE

Platform description:

• Aravis version 0.8
• OS: Linux (NVIDIA Jetpack)
• Hardware aarch64

Additional context
My main query is about safely destroying the camera object, and if/how to use g_object_unref or gst_object_unref.

// Simple script which will start an aravis pipeline, then restart ONLY the camera
#include <chrono>
#include <condition_variable>
#include <cstdlib>
#include <iostream>
#include <mutex>
#include <string>
#include <thread>

#include <gst/gst.h>


int main(int argc, char *argv[])
{
    gst_init (&argc, &argv);

    GstElement *pipeline = gst_pipeline_new ("aravis_pipeline");
    GstElement *source = gst_element_factory_make ("aravissrc", "source");
    if (!source) {
        std::cerr << "Failed to create the source element." << std::endl;
        return false;
    }

    #define cfg_OffsetY ((1200-1184)/2)
    const char *cam1 = "10.42.0.154";

    g_object_set(G_OBJECT(source),
            "camera-name", cam1,
            "num-arv-buffers", 64,
            "packet-resend", false,
            "packet-size", 9000,
            "auto-packet-size", false,
            NULL);
    
    std::this_thread::sleep_for(std::chrono::seconds(5));


    GstElement *sink = gst_element_factory_make ("fakesink", "fake_sink");

    if (!pipeline || !source || !sink) {
        g_printerr ("Not all elements could be created.\n");
        return -1;
    }

    gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);

    if (!gst_element_link (source, sink)) {
        g_printerr ("Elements could not be linked.\n");
        gst_object_unref (pipeline);
        return -1;
    }

    GMainLoop *main_loop = g_main_loop_new(NULL, FALSE);
    // CustomData data = {pipeline, main_loop};
    std::cout << "Starting the pipeline." << std::endl;
    gst_element_set_state(pipeline, GST_STATE_PLAYING);

    std::this_thread::sleep_for(std::chrono::seconds(10));

    std::cout << "Restarting the camera source." << std::endl;
    // gst_element_set_state(pipeline, GST_STATE_PAUSED);
    // Restart the camera source

    // Stop camera
    gst_element_set_state(source, GST_STATE_NULL);

    // Wait for data to stop flowing
    std::this_thread::sleep_for(std::chrono::seconds(10));

    std::cout << "Camera source stopped." << std::endl;

    // Remove the source from the pipeline
    gst_element_unlink(source, sink);

    // Remove the source from the pipeline
    gst_bin_remove(GST_BIN(pipeline), source);
    
    // UNREF?
    g_object_unref(source);

    std::this_thread::sleep_for(std::chrono::seconds(10));

    std::cout << "Create a new source." << std::endl;

    // Create a new source
    GstElement *new_source = gst_element_factory_make ("aravissrc", "new_source");
    if (!new_source) {
        std::cerr << "Failed to create the source element." << std::endl;
        return false;
    }

    g_object_set(G_OBJECT(new_source),
        "camera-name", cam1,
        "num-arv-buffers", 64,
        "packet-resend", false,
        "packet-size", 9000,
        "auto-packet-size", false,
        NULL);
    
    std::this_thread::sleep_for(std::chrono::seconds(5));
    std::cout << "Add to pipeline." << std::endl;


    gst_bin_add(GST_BIN(pipeline), new_source);

    std::this_thread::sleep_for(std::chrono::seconds(5));
    std::cout << "Link to pipeline." << std::endl; 
    gst_element_link(new_source, sink);

    std::this_thread::sleep_for(std::chrono::seconds(5));
    std::cout << "sync to pipeline." << std::endl; 

    gst_element_sync_state_with_parent(new_source);

    std::this_thread::sleep_for(std::chrono::seconds(10));


    std::cout << "Starting the pipeline." << std::endl;

    gst_element_set_state(new_source, GST_STATE_PLAYING);


    g_main_loop_run (main_loop);

    /* Free resources */
    g_main_loop_unref (main_loop);
    // gst_object_unref (bus);
    gst_element_set_state (pipeline, GST_STATE_NULL);
    gst_object_unref (pipeline);
}

[sean-kiwi]

Any ideas for this?

[EmmanuelP]

The issue comes from the fact you are setting the state on the source element, not the pipeline.

Replace gst_element_set_state(source, GST_STATE_NULL); by gst_element_set_state(pipeline), GST_STATE_NULL); and gst_element_set_state(new_source, GST_STATE_PLAYING); by gst_element_set_state(pipeline, GST_STATE_PLAYING);, and remove g_object_unref(source); (the bin is the owner of the element, removing the element automatically unrefs it), and it should work.

[EmmanuelP]

I'm not an expert in gstreamer. You have better chance to an get explanation on why elements are only destroyed when the pipeline is stopped on the gstreamer discourse instance: https://discourse.gstreamer.org/

[Summengardin]

I struggled with the same thing. My application is based on the Deepstream runtime_source_add_delete. I dont know if this is the correct solution, but my testing revealed that the gst_aravis_finalize method is never called. However, as stated in the Gstreamer docs - GstBaseSrcClass, Subclasses should use the stop-method to close resources. I added the content from the gst_aravis_finalize method into the gst_aravis_stop-method and now it works perfectly.

[EmmanuelP]

Thanks @Summengardin

Could you please test #914 ?

[Summengardin]

Thank you! Yes, #914 fixes this

@sean-kiwi, fyi

[sean-kiwi]

Thanks, I'll have a look. I made a fair bit of progress on this too, will post my findings when I get some time. My main concern with modifying the stop function, is if this will always kill the camera source? I.e it is no longer possible to just pause and play the same source without destroying it?

[EmmanuelP]

My main concern with modifying the stop function, is if this will always kill the camera source? I.e it is no longer possible to just pause and play the same source without destroying it?

Yes, you're right. May be it was not a good idea after all to destroy the camera in the stop function.

The thing is, in the sample code you posted, the source element is destroyed only after the pipeline is also destroyed. Which I do not understand. The gstreamer documentation says the element is unrefed when gst_bin_remove() is called, and as there is no more reference to it, it should be destroyed.

I guess the first step would be to fix the sample code to be in accordance with the pipeline manipulation documentation. I have created a branch here #918 that builds it as one of the aravis sample codes.

[Summengardin]

I.e it is no longer possible to just pause and play the same source without destroying it?

Pausing the source, wouldn't that normally be done using gst_element_set_state() with GST_STATE_PLAYING and GST_STATE_PAUSED?

Using the stop method is described to free resources. Here
: stop() Stop processing. Subclasses should use this to close resources.

I will look at #918 to stay in touch on this

[sean-kiwi]

From memory the paused state has no effect on a live source, but I will double check. I did manage to get the camera to consistently be destroyed without setting pipeline to null. The main issue was clearing the message bus which holds on to references.

After that I still had a few issues in reconnecting the new source, I think related to flushing old data, but I'll confirm that and share when I get a chance. Perhaps your other additions in this branch related to caps will help