Fix MJPEG camera decoding on Linux#113359
Conversation
| if (p_strict) { | ||
| print_verbose(vformat("libjpeg-turbo decompress error: %s", tj3GetErrorStr(tj_instance))); | ||
| } |
There was a problem hiding this comment.
From my testing that when decoding partially corrupted MJPEG frames, tj3Decompress8 still returns -1 indicating a warning is encountered. An issue from libjpeg-turbo also mentioned such behaviour but for tjDecompress2.
We can use tjGetErrorCode instead to determine if the error is fatal (TJERR_FATAL) or not (TJERR_WARNING).
| if (!p_strict) { | ||
| // Allow partial/corrupt JPEG decoding for streaming sources like cameras. | ||
| tj3Set(tj_instance, TJPARAM_STOPONWARNING, 0); | ||
| } | ||
|
|
There was a problem hiding this comment.
TJPARAM_STOPONWARNING is already 0 by default since libjpeg-turbo 2.1.90, maybe the additional p_strict parameter is not needed?
fc1fd80 to
3142dcd
Compare
|
It seems like the libjpeg-turbo upgrade in #113945 may have resolved the noise and frame drop issues. This PR might not be necessary anymore. |
3142dcd to
c36f7d4
Compare
|
When Motion JPEG is selected, commit 94971e1 on master repeatedly throws the following error under high load (high resolution or frame rate). This PR fixes the error but introduces more block noise, as shown in the attached video. The block noise is likely caused by processing load and remains unresolved. linux-camera-mjpeg.mp4 |
|
For reference, playing the camera with the same format using ffmpeg also produces block noise and errors. ffmpeg-linux-mjpeg.mp4 |
j20001970
left a comment
There was a problem hiding this comment.
c36f7d4f13b6d2ff502138113738e91241180f6b works as expected with Logitech C270 webcam, no visual glitch is observed from decoded MJPEG frames.
|
I tested with Steam Deck + Logitech C920 and there was no noise. The noise was occurring in a WSL2 + usbipd USB passthrough environment, so it appears the overhead was causing latency. steamdeck-camera-mjpeg.mp4 |
41c06b0 to
567fe2e
Compare
567fe2e to
b3e3825
Compare
Decode MJPEG frames to YUV planes using libjpeg-turbo's tj3DecompressToYUVPlanes8 with lenient mode and actual frame size, instead of converting to RGB on CPU. This offloads YUV to RGB conversion to the GPU shader, reducing CPU overhead and memory bandwidth for camera streaming.
b3e3825 to
9747ffa
Compare
fixed #113357
MJPEG streams from cameras have two issues:
→ Solution:
v4l2_buffer.bytesusedto get the actual frame size.