Warning: XShm get_pixels_ptr XImage is NULL

[totaam]

Issue migrated from trac ticket # 1054

component: server | priority: blocker | resolution: fixed

2015-12-16 01:28:38: afarr created the issue

---

During the testing for #849 (osx client 0.16.0 r11304 against fedora 21 0.16.0 r11357 server) ... after about 4 hours of an overnight test, I got the following warning/traceback on the server (also saw with 0.16.0 11392 windows client against 0.16.0 r11366 fedora 21 server after about 4 hours of playing sound/video).

2015-12-14 22:19:58,994 Warning: XShm get_pixels_ptr XImage is NULL
2015-12-14 22:19:58,996 error processing encode queue: failed to get pixels from XShmImageWrapper(BGRX: 85, 137, 480, 360)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1335, in encode_from_queue
    self.make_data_packet_cb(*item)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1364, in make_data_packet_cb
    packet = self.make_data_packet(damage_time, process_damage_time, wid, image, coding, sequence, options, flush)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1707, in make_data_packet
    ret = encoder(coding, image, options)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_video_source.py", line 1310, in video_encode
    ret = ve.compress_image(csc_image, quality, speed, options)
  File "encoder.pyx", line 593, in xpra.codecs.enc_x264.encoder.Encoder.compress_image (xpra/codecs/enc_x264/encoder.c:5709)
AssertionError: failed to get pixels from XShmImageWrapper(BGRX: 85, 137, 480, 360)
2015-12-14 22:28:59,061 Warning: XShm get_pixels_ptr XImage is NULL
2015-12-14 22:28:59,071 error processing encode queue: failed to get pixels from XShmImageWrapper(BGRX: 85, 137, 480, 360)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1335, in encode_from_queue
    self.make_data_packet_cb(*item)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1364, in make_data_packet_cb
    packet = self.make_data_packet(damage_time, process_damage_time, wid, image, coding, sequence, options, flush)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_source.py", line 1707, in make_data_packet
    ret = encoder(coding, image, options)
  File "/usr/lib64/python2.7/site-packages/xpra/server/window/window_video_source.py", line 1310, in video_encode
    ret = ve.compress_image(csc_image, quality, speed, options)
  File "encoder.pyx", line 593, in xpra.codecs.enc_x264.encoder.Encoder.compress_image (xpra/codecs/enc_x264/encoder.c:5709)
AssertionError: failed to get pixels from XShmImageWrapper(BGRX: 85, 137, 480, 360)

Wasn't actually present during the tests, but in the case of the osx client test, didn't see a sound source stopping message for another 2 hours after the message - so I don't know if it produced any observable effects.

[totaam]

2015-12-16 02:13:21: antoine changed priority from major to critical

[totaam]

2015-12-16 02:13:21: antoine changed status from new to assigned

[totaam]

2015-12-16 02:13:21: antoine commented

---

I have seen this same trace twice, and I had recorded it - just not in a ticket.
This can only happen if we free() the xshm image before we use it, this should never happen - and I don't see how it can happen...
The fact that it is so hard to reproduce makes it harder to debug.

[totaam]

2015-12-22 13:20:54: antoine changed priority from critical to blocker

[totaam]

2015-12-22 13:20:54: antoine commented

---

We create the [/browser/xpra/tags/v0.16.x/src/xpra/x11/bindings/ximage.pyx#L580 XShmImageWrapper] in [/browser/xpra/tags/v0.16.x/src/xpra/server/window/window_source.py#L1256 process_damage_region], which runs in the UI thread.

The XImage is freed and set to NULL by calling the Cython function [/browser/xpra/tags/v0.16.x/src/xpra/x11/bindings/ximage.pyx#L367 free_image] on the image wrapper, this can only be called from [/browser/xpra/tags/v0.16.x/src/xpra/x11/bindings/ximage.pyx#L362 XImageWrapper.free()].

This is only called from 2 places:

• immediately if the operation has been cancelled - unlikely to be a problem
• via [/browser/xpra/tags/v0.16.x/src/xpra/server/window/window_source.py#L1245 free_image_wrapper], which is called from:

• [/browser/xpra/tags/v0.16.x/src/xpra/server/window/window_source.py#L747 cancel_damage] for av-sync delayed images
• [/browser/xpra/tags/v0.16.x/src/xpra/server/window/window_source.py#L1308 encode_from_queue] when av-sync is enabled
• [/browser/xpra/tags/v0.16.x/src/xpra/server/window/window_source.py#L1359 make_data_packet_cb] after the compression (via a UI thread callback since this runs in the encode thread)

There is also some similar code in the server's root overlay paint code (#988), but this is not enabled by default and looks straightforward as it does it all in the UI thread.

The av-sync stuff looks suspicious (#835).

[totaam]

2015-12-22 13:32:33: antoine changed status from assigned to closed

[totaam]

2015-12-22 13:32:33: antoine set resolution to fixed

[totaam]

2015-12-22 13:32:33: antoine commented

---

Huge bug found in av-sync: r11462 fixes this (will backport), r11463 tries to ensure this does not break again.

I am closing this because it must be quite hard to hit this problem.
We can re-open it if it happens again.

[totaam]

2015-12-22 13:47:30: antoine changed title from XShm warning and traceback on fedora 21 server to Warning: XShm get_pixels_ptr XImage is NULL

[totaam]

2015-12-22 13:47:30: antoine commented

---

(more descriptive bug title)