Skip to content

Fix MJPEG camera decoding on Linux#113359

Open
shiena wants to merge 1 commit intogodotengine:masterfrom
shiena:fix/linux_camera_mjpeg_decode
Open

Fix MJPEG camera decoding on Linux#113359
shiena wants to merge 1 commit intogodotengine:masterfrom
shiena:fix/linux_camera_mjpeg_decode

Conversation

@shiena
Copy link
Copy Markdown
Contributor

@shiena shiena commented Nov 30, 2025

fixed #113357
MJPEG streams from cameras have two issues:

  1. V4L2 buffer size is allocated for the maximum frame size, which is larger than the actual JPEG data size. Passing the entire buffer to the decoder causes it to fail due to invalid trailing data.
  2. Real-time streams may have partially corrupted frames. The standard JPEG decoder strictly rejects these as errors.

Solution:

  • Use v4l2_buffer.bytesused to get the actual frame size.
  • Add an internal function that decodes using libjpeg-turbo's lenient mode (tolerates partial corruption).

Comment on lines +79 to +81
if (p_strict) {
print_verbose(vformat("libjpeg-turbo decompress error: %s", tj3GetErrorStr(tj_instance)));
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing that when decoding partially corrupted MJPEG frames, tj3Decompress8 still returns -1 indicating a warning is encountered. An issue from libjpeg-turbo also mentioned such behaviour but for tjDecompress2.

We can use tjGetErrorCode instead to determine if the error is fatal (TJERR_FATAL) or not (TJERR_WARNING).

Comment on lines +41 to +45
if (!p_strict) {
// Allow partial/corrupt JPEG decoding for streaming sources like cameras.
tj3Set(tj_instance, TJPARAM_STOPONWARNING, 0);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TJPARAM_STOPONWARNING is already 0 by default since libjpeg-turbo 2.1.90, maybe the additional p_strict parameter is not needed?

@j20001970 j20001970 mentioned this pull request Jan 9, 2026
5 tasks
@shiena shiena force-pushed the fix/linux_camera_mjpeg_decode branch from fc1fd80 to 3142dcd Compare January 9, 2026 19:26
@shiena
Copy link
Copy Markdown
Contributor Author

shiena commented Jan 9, 2026

It seems like the libjpeg-turbo upgrade in #113945 may have resolved the noise and frame drop issues. This PR might not be necessary anymore.

@shiena shiena force-pushed the fix/linux_camera_mjpeg_decode branch from 3142dcd to c36f7d4 Compare January 10, 2026 15:18
@shiena
Copy link
Copy Markdown
Contributor Author

shiena commented Jan 10, 2026

When Motion JPEG is selected, commit 94971e1 on master repeatedly throws the following error under high load (high resolution or frame rate). This PR fixes the error but introduces more block noise, as shown in the attached video. The block noise is likely caused by processing load and remains unresolved.

ERROR: Condition "err" is true. Returning: Ref<Image>()
   at: _jpeg_turbo_mem_loader_func (modules/jpg/image_loader_libjpeg_turbo.cpp:104)
ERROR: Condition "image.is_null()" is true. Returning: ERR_PARSE_ERROR
   at: _load_from_buffer (core/io/image.cpp:4542)
linux-camera-mjpeg.mp4

@shiena
Copy link
Copy Markdown
Contributor Author

shiena commented Jan 10, 2026

For reference, playing the camera with the same format using ffmpeg also produces block noise and errors.

$ ffplay -f v4l2 -input_format mjpeg -video_size 640x480 -i /dev/video0
[mjpeg @ 0x726320002440] mjpeg_decode_dc: bad vlc: 0:0 (0x726320004048)
[mjpeg @ 0x726320002440] error dc
[mjpeg @ 0x726320002440] error y=47 x=19
[mjpeg @ 0x726320002440] EOI missing, emulating
ffmpeg-linux-mjpeg.mp4

Copy link
Copy Markdown
Contributor

@j20001970 j20001970 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c36f7d4f13b6d2ff502138113738e91241180f6b works as expected with Logitech C270 webcam, no visual glitch is observed from decoded MJPEG frames.

@shiena
Copy link
Copy Markdown
Contributor Author

shiena commented Jan 11, 2026

I tested with Steam Deck + Logitech C920 and there was no noise. The noise was occurring in a WSL2 + usbipd USB passthrough environment, so it appears the overhead was causing latency.

steamdeck-camera-mjpeg.mp4

@Repiteo Repiteo modified the milestones: 4.6, 4.x Jan 26, 2026
@shiena shiena force-pushed the fix/linux_camera_mjpeg_decode branch 2 times, most recently from 41c06b0 to 567fe2e Compare January 29, 2026 14:10
@shiena shiena force-pushed the fix/linux_camera_mjpeg_decode branch from 567fe2e to b3e3825 Compare March 22, 2026 18:44
Decode MJPEG frames to YUV planes using libjpeg-turbo's
tj3DecompressToYUVPlanes8 with lenient mode and actual frame size,
instead of converting to RGB on CPU. This offloads YUV to RGB
conversion to the GPU shader, reducing CPU overhead and memory
bandwidth for camera streaming.
@shiena shiena force-pushed the fix/linux_camera_mjpeg_decode branch from b3e3825 to 9747ffa Compare March 22, 2026 19:12
@shiena shiena requested a review from a team as a code owner March 22, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux camera feed fails to decode MJPEG frames

4 participants