Failed to find loop device: could not attach image file to loop device: no loop devices available

[yasu-sh]

@felixleopoldo Thanks for your works, benchpress. It works fine on the machines.
I was using benchpress and issues on WSL2 at win11 with docker.
So now I moved to native Ubuntu machine with docker-image as the instruction manual at benchpress.

One symptom is worth telling you on some cases.

• countermeasure-1 reduce cores - worked.
  • snakemake --cores 4 --use-singularity --configfile config/config.json
• countermeasure-2 install as Linux(Ubuntu) on WSL - worked!
  • Straightforwardly used miniforge(mambaforge) instead of miniconda since both conda make corruption of base environment at conda.
  • Apptainer version 1.2.3 installed with non-setuid installation

I guess the root cause has not been solved yet. But it in some cases updates makes a solutions.
I hope my report be a help for benchpress users.
sylabs/singularity#67 <- the same symptoms.

When I run the code as below:

(snakemake) root@:/mnt# snakemake --cores all --use-singularity --configfile config/config.json

Then I faced the error as below. Both have the same. loop device was not enough.

• Win11 on Docke Desktop with WSL2, volume mounted at Windows file system
  PS > docker run -it -w /mnt --privileged -v F:/benchpress:/mnt bpimages/snakemake:v7.32.3

• Win11 on Docke Desktop with WSL2, volume mounted at WSL2 file system
  docker run -it -w /mnt --privileged --name bntab -v /home/path/benchpress:/mnt bpimages/snakemake:v7.32.3

• Symptom:

(omit)
[Fri Sep 29 05:17:07 2023]
Finished job 112.
1 of 346 steps (0.3%) done
FATAL:   container creation failed: mount /proc/self/fd/3->/opt/conda/envs/snakemake/var/singularity/mnt/session/rootfs error: while mounting image /proc/self/fd/3: failed to find loop device: could not attach image file to loop device: no loop devices available

(snakemake) root@d6f240d00620:/mnt# lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       48 bits physical, 48 bits virtual
CPU(s):              12
On-line CPU(s) list: 0-11
Thread(s) per core:  2
Core(s) per socket:  6
Model name:          AMD Ryzen 5 5500
CPU MHz:             3593.164

Memory 64GB / GPU nvidia 8GB

The loop device increased from 8 to 255.

(snakemake) root@:/mnt# ls /dev/loop*
/dev/loop-control  /dev/loop119  /dev/loop140  /dev/loop162  /dev/loop184  /dev/loop205  /dev/loop227  /dev/loop249  /dev/loop40  /dev/loop62  /dev/loop84
/dev/loop0         /dev/loop12   /dev/loop141  /dev/loop163  /dev/loop185  /dev/loop206  /dev/loop228  /dev/loop25   /dev/loop41  /dev/loop63  /dev/loop85
(omit)
/dev/loop118       /dev/loop14   /dev/loop161  /dev/loop183  /dev/loop204  /dev/loop226  /dev/loop248  /dev/loop4    /dev/loop61  /dev/loop83

[felixleopoldo]

Thanks for this report. I also experienced a loop device error before, both when running through Docker on Linux and natively on Linux.
I didn't find out exactly what the solution is but it seemed to work when I created a new conda environment or when updating Apptainer. I was not able to reproduce the error once it worked.

[felixleopoldo]

If anyone experience a similar problem, please make a comment here and we can try to figure out how to solve it.

[gomathi-l]

Running it with a reduced number of cores (1 to 4) instead of using all available cores resolves the issue for me.
snakemake --cores 4 --use-singularity --configfile config/config.json