single quotes in docker image CMD are not escaped in singularity runscript

[DavyCats]

Version of Singularity:

What version of Singularity are you using? Run:

$ singularity --version
singularity version 3.8.0

Expected behavior

When running a docker container through singularity, the command specified in the Dockerfile (which includes single quotes ') should be run as specified.

Actual behavior

Single quotes disappear from the command. This is the result of single quotes being used to encapsulate the command in the runscript, while single quotes which are part of the command are not being escaped.

Steps to reproduce this behavior

The following command can be used try and run an image which uses single quotes in the command:

$ singularity run --containall docker://quay.io/davycats/test_docker_runscript:latest
INFO:    Using cached SIF image
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'hello' is not defined

The Dockerfile for this image look like this:

FROM python

CMD python -c "print('hello')"

You can see the problem when you inspect the first few lines of the runscript inside the singularity container:

$ singularity exec --containall docker://quay.io/davycats/test_docker_runscript:latest head -n 3 /.singularity.d/runscri
pt
INFO:    Using cached SIF image
#!/bin/sh
OCI_ENTRYPOINT=''
OCI_CMD='"/bin/sh" "-c" "python -c \"print('hello')\""'

This last line should look like this instead:

OCI_CMD='"/bin/sh" "-c" "python -c \"print('"'"'hello'"'"')\""'

What OS/distro are you running

Debian GNU/Linux 10 (buster)

How did you install Singularity

From source, using the instructions here: https://singularity.hpcng.org/user-docs/3.8/quick_start.html

[DrDaveD]

Looks like Sylabs already fixed this in their pr #252. I'll mark this for 3.8.2.

[gmkurtzer]

The Sylabs fix looks good to me, but it is unfortunate that they are monitoring our upstream issues and fixing them on their own commercially sponsored and controlled fork without collaborating with the upstream community.

This is not a sustainable collaboration -- note issue #6113.

[dtrudg]

This particular issue was also reported directly to Sylabs at sylabs/singularity#233 and we note that in the hpcng/singularity repository more generally there have been suggestions made by hpcng/singularity project members to report problems to sylabs/singularity also.

We specifically linked the hpcng issue from the sylabs issue (as in other cases) so that our work is visible to the hpcng/singularity community on the GitHub interface:

The fix is available to hpcng/singularity and others under a non-commercial open-source license. SingularityCE is a fork in the older sense of the term, with independent development occurring and therefore it is not an upstream nor a downstream of hpcng/singularity. However, work ongoing in SingularityCE is available to other projects to incorporate.

[gmkurtzer]

My point was not visibility, it was collaboration. When an issue comes into the HPCng issue tracker, but you silently develop and incorporates a fix without involving the UPSTREAM community and other developers, it puts us and the community, into a bad situation.

This is because now we either have to accept your fix wholesale, or we fix it ourselves differently where the latter will accelerate the misalignment between projects.

SingularityCE is a fork in the older sense of the term, with independent development occurring and therefore it is not an upstream nor a downstream of hpcng/singularity. However, work ongoing in SingularityCE is available to other projects to incorporate.

The usage of the word "fork" in this case is indeed the older form (which is not a good thing). In open source history, it is generally a detrimental action to "fork" a project so there must be very good and clear reasons why any project would be forked (which has not been clearly articulated by Sylabs). In my 20'ish years of working in open source, I can think of no cases where a community has benefited from a commercially controlled fork.

So to be clear, SingularityCE is absolutely a downstream "fork" of Singularity as both the Git commit logs and the derivative name clearly demonstrates. Where there was one, there are now two, both running in parallel, neither being truly independent (as your fixing issues submitted to HPCng clearly demonstrates). This is highly un-optimal for the project(s) and Sylabs knows this well.

We appreciate Sylabs releasing fixes under the projects original open source license, but your statements and actions are disingenuous.