Large glass-to-glass latency with Blackfly S USB3

[nicshackle]

Describe the bug
I have a Blackfly S BFS-US-16S2M connected over USB3 to a powerful computer running Ubuntu 22.04. If I point the camera to a blinking pattern on the screen, and then open the aravis monitor next to the blinking pattern, I notice a significant latency (not measured, but probably in the 100-200ms range). Is this normal?
Attached is a screen recording of the blinking dot, alongside the viewer. The camera is pointed at the red blinking dot.
flashing_dotshort.webm

To Reproduce

• Build aravis from source (release 0.8.31)
  • with meson configured to enable introspection, viewer, usb.
• Blackfly S BFS-US-16S2M connected over USB 3
• Point the camera at a blinking pattern on the computer monitor
• Open aravis viewer
  • Pixel format: Mono8, Binning:1x1, Region size: 1440x1080 (also tried with smaller region sizes; same result)
  • Nothing set to auto; exposure set to 12uS, gain set to 47
  • fps at 199Hz

Expected behavior
Less latency

Camera description:

• Flir
• Blackfly S BFS-US-16S2M
• USB3

Platform description:

• Aravis version 0.8.13
• OS: Ubuntu 22.04
• x86_64, i9-14900K, GTX4080

Additional context
Trying with Spinnaker SDK (& spinview), I see the similar latency. Also tried setting the buffer to "newest only" (see https://www.flir.com/support-center/iis/machine-vision/application-note/understanding-buffer-handling/)

[EmmanuelP]

Hi,

Arv-viewer is mostly a debugging tool. The latency may increase if the gstreamer based video output can not keep up with the incoming frames from aravis.

What is the CPU use ? Did you try to lower the frame rate ?

[nicshackle]

Thanks for your reply!

CPU usage (the flatter left-hand area of the chart is before opening the viewer):

If I reduce the fps (e.g. to 50fps), 50fps is maintained but the latency is still equally present.

[EmmanuelP]

I'm short of idea on how to improve things. This sort of latency is something I have already seen with some cameras. As the viewer is pushing buffers to the gstreamer pipeline as soon as they are completed, the issue comes from how gstreamer handle the latency in its pipeline. May be an issue with the buffer timestamps, or a difference between the frame rate we give to gstreamer and the actual one.