RealSense2FrameGrabber

This wiki page explains on how to use the RealSense2FrameGrabber which adds basic support for the Intel RealSense2 device family. The wrapper itself is based on the C-API of the librealsense2 library.

Basic Usage

This is the most basic example which creates a new RealSense2FrameGrabber, adds the streams and grabs a single frame. To select a specific camera, use the deviceNumber parameter of the constructor.

RealSense2FrameGrabber rs2 = new RealSense2FrameGrabber();

rs2.enableColorStream(640, 480, 30);
rs2.start();

Frame frame = rs2.grab();

rs2.stop();
rs2.release();

Here is an extended example on how to use depth stream and color stream together: samples/RealSense2DepthMeasuring.java

Multiple Streams

To use multiple streams on one device enable them before start and read the different streams with the corresponding method.

rs2.enableColorStream(640, 480, 30);
rs2.enableDepthStream(640, 480, 30);

Frame colorFrame = rs2.grabColor()
Frame depthFrame = rs2.grabDepth()

IR Streams

If there are multiple streams of the same type enabled (only Infrared), use the stream index parameter to define which stream should be grabbed. The index follows the order on how the streams have been enabled.

// enable both IR streams 
// (last parameter is the sensor index, not the stream index!)
rs2.enableIRStream(640, 480, 90, 1);
rs2.enableIRStream(640, 480, 90, 2);

// read by using stream index
Frame firstIRStream = rs2.grabIR(0)
Frame secondIRStream = rs2.grabIR(1)

Custom Stream Settings

To enable a stream with custom settings use the enableStream method. This example show how to read the RGB IR stream of an Intel RealSense2 D415.

rs2.enableStream(new RealSense2FrameGrabber.RealSenseStream(
        RS2_STREAM_INFRARED,
        0,
        new Size(1280, 720),
        30,
        RS2_FORMAT_BGR8
));

Frame frame = rs2.grab(RS2_STREAM_INFRARED, 0, IPL_DEPTH_8U, 3);

Multi Device

Create an instance for each device to use multiple devices together.

RealSense2FrameGrabber cam1 = new RealSense2FrameGrabber();
RealSense2FrameGrabber cam2 = new RealSense2FrameGrabber();

cam1.enableColorStream(640, 480, 30);
cam2.enableColorStream(640, 480, 30);

cam1.start();
cam2.start();

Important: The device index should always be 0, because the method rs2_create_device does not count already created devices.

Sensor Options

At the moment sensor option are very limited supported. It is possible to set a sensor option before the device has been started. After that it is currently not possible to change an option. Because of that, the device first have to be opened, the option set and then started.

// open the device
rs2.open();

// setting some example sensor options for the D415
rs2.setSensorOption(RealSense2FrameGrabber.Rs2SensorType.StereoModule, RS2_OPTION_ENABLE_AUTO_EXPOSURE, false);
rs2.setSensorOption(RealSense2FrameGrabber.Rs2SensorType.StereoModule, RS2_OPTION_ENABLE_AUTO_WHITE_BALANCE, true);
rs2.setSensorOption(RealSense2FrameGrabber.Rs2SensorType.StereoModule, RS2_OPTION_EXPOSURE, 3000f);
rs2.setSensorOption(RealSense2FrameGrabber.Rs2SensorType.StereoModule, RS2_OPTION_EMITTER_ENABLED, false);

// start pipeline
rs2.start();

Trigger Mode

To use frames from different stream but the same frameset, there is the trigger mode. If the trigger mode is enabled, the RealSense2FrameGrabber will not load any new frameset until trigger() has been called. To enable the trigger mode call trigger() or via rs2.setTriggerMode(true).

// loads new frameset
rs2.trigger();

// both frames are from the same frameset (synchronised)
Frame colorFrame = rs2.grabColor()
Frame depthFrame = rs2.grabDepth()

List Devices

This method displays all the devices which are connected.

RealSense2FrameGrabber rs2 = new RealSense2FrameGrabber();

// list all cameras
for (RealSense2FrameGrabber.RealSense2DeviceInfo info : rs2.getDeviceInfos()) {
    System.out.printf("Device: %s %s %s Locked: %b\n",
            info.getName(),
            info.getFirmware(),
            info.getSerialNumber(),
            info.isLocked());
}

Limitations

The RealSense2FrameGrabber class does only support frame grabbing and very basic control over the camera. There is no post-processing and there are no filters available.

Sometimes it happens that the camera is in a bricked mode. In this mode, the device is either not responding or the frames are not displayed correctly. This usually happens if the camera was not shutdown correctly. Always try to stop the camera. To fix the bricked mode, just re-plug the camera into the USB port.