shmdata output failure

[spezi]

i currently trying to get ossia score with splashmapper (https://splashmapper.xyz/en/) to work and would like to connect them via shmdata.
Send video from splashmapper output shmdata to ossia score works. But shmdata output from score to splashmapper fails.
also consume score shmdata output with gst-launch fails. Next i tried to consume shmdata output of one score instance with another as shmdata input.

i used ossia.score-3.1.13-linux-amd64.AppImage on linux

It seems to be a failure in ossia score shmdata output. it does not crash but throws debug messages at input side:

debug: received server info, shm_size 3686400, type video/x-raw,format=RGBA,width=1280,height=720,framerate=30/1
Debug: Shmdata Input format: QList("video/x-raw", "format=RGBA", "width=1280", "height=720", "framerate=30/1") (:0)
debug: client connected
debug: reader initialization done
debug: received server info, shm_size 1048576, type video/x-raw,format=(string)RGBA,width=(int)512,height=(int)512,framerate=(fraction)30/1,pixel-aspect-ratio=(fraction)1/1
Debug: Shmdata Input format: QList("video/x-raw", "format=(string)RGBA", "width=(int)512", "height=(int)512", "framerate=(fraction)30/1", "pixel-aspect-ratio=(fraction)1/1") (:0)
Debug: ShmdataInput: supported format "RGBA" (:0)
debug: client connected
debug: reader initialization done
Debug: Unhandled pixel format:   (:0)

with working stream from splashmapper i got this debug messages:

debug: received server info, shm_size 3686400, type video/x-raw,format=RGBA,width=1280,height=720,framerate=30/1
Debug: Shmdata Input format: QList("video/x-raw", "format=RGBA", "width=1280", "height=720", "framerate=30/1") (:0)
debug: client connected
debug: reader initialization done
debug: received server info, shm_size 1048576, type video/x-raw,format=(string)RGBA,width=(int)512,height=(int)512,framerate=(fraction)30/1,pixel-aspect-ratio=(fraction)1/1
Debug: Shmdata Input format: QList("video/x-raw", "format=(string)RGBA", "width=(int)512", "height=(int)512", "framerate=(fraction)30/1", "pixel-aspect-ratio=(fraction)1/1") (:0)
Debug: ShmdataInput: supported format "RGBA" (:0)

But shmdata2ndi (https://gitlab.com/sat-mtl/tools/ndi2shmdata) is able to convert the shmdata output of score to ndi and it is a workaround for that. splashmapper is also able to consume ndi stream.

[spezi]

a nice workaround works great!
gst-launch-1.0 --gst-plugin-path=/usr/lib/gstreamer-1.0/ shmdatasrc socket-path=/tmp/score_shm_video ! videoconvert ! shmdatasink socket-path=/tmp/score_converted_shmdata

[paperManu]

Hi @spezi , Splash maintainer speaking. I found this issue while looking for NDI information in Score, so that's an interesting coincidence ;)

From what I can tell, the shmdata input in Splash from the Score output should have worked, the caps seems alright. I'm not very familiar with Score, would you mind sharing a configuration file so that I can test it on my side?

[jcelerier]

heya @paperManu :) if you can tell me if it's some format conversion thing I'm missing i'll gladly add it, I haven't had much time to look into it...
For NDI all the code is contained in this plug-in : https://github.com/ossia/score-addon-ndi feel free to ping me if you have any questions !

[paperManu]

Hey @jcelerier :) Yep I found the addon! I'll first install it, and then maybe come back with questions.

[jcelerier]

if you grab the official release it's already built-in ! :) otherwise if you build from source, you need to clone it in src/addons and then run cmake on the root folder, it will pick it up

[jcelerier]

to get NDI to show up on linux you need to set the NDI env var (for instance NDI_RUNTIME_DIR_V5=/opt/ndi/lib/x86_64-linux-gnu)

[paperManu]

I had a quick look, and the issue is that the caps from Score looks like:

video/x-raw,format=RGBA,width=1280,height=720,framerate=60/1

where usually the types are specified, like this:

video/x-raw, format=(string)RGBA, width=(int)1280, height=(int)720, framerate=(fraction)60/1

I always dealt with caps generated by me, or by gstreamer, so this has never been an issue. I'll fix the issue in Splash to allow for the type of each field to be omitted, but there might be other software which are picky in this regard so it could be a good thing to add the types on Score side.

Anyway, this should be fixed in Splash next release :)

[paperManu]

@spezi Just so you know, a new release of Splash has been... well, released today, with what should be a fix to your issue!

[jcelerier]

cool, added the types to caps in a commit for safety, thanks for the debugging help @paperManu !!!

[paperManu]

My pleasure!

[spezi]

Hi @spezi , Splash maintainer speaking. I found this issue while looking for NDI information in Score, so that's an interesting coincidence ;)

From what I can tell, the shmdata input in Splash from the Score output should have worked, the caps seems alright. I'm not very familiar with Score, would you mind sharing a configuration file so that I can test it on my side?

@spezi Just so you know, a new release of Splash has been... well, released today, with what should be a fix to your issue!

sorry for delay, i was on vacation. I will check it in the next days. thanks a lot for fixing this issue!