Emscripten tries to write to /nix/store

[busti]

Describe the bug

I am trying to compile a project which (among other things) uses emscripten.
I have created a custom shell.nix file to create a development environment for the project.
Executing the makefile fails when it gets to compiling the emscripten part of the project with the following error message:
OSError: [Errno 30] Read-only file system: '/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/cache/ports'

Steps To Reproduce

The project makes use of the open source project imgui. One of it's examples can be used to reproduce the error message.

1. git clone https://github.com/ocornut/imgui.git
2. cd examples/example_emscripten_opengl3
3. create a file named shell.nix with the following contents:

   {pkgs ? import <nixpkgs> {}}: pkgs.mkShell {
     nativeBuildInputs = with pkgs; [
       gnumake gcc
     ];
   
     buildInputs = with pkgs; [
       emscripten
     ];
   }

4. run nix-shell
5. run make

Expected behavior

The command succeeds resulting in a compiled executable of the example project.

Real behavior

The execution of make fails with the following error:

$ make
em++ -DIMGUI_DISABLE_FILE_FUNCTIONS -I../.. -I../../backends -Wall -Wformat -Os -s USE_SDL=2 -s DISABLE_EXCEPTION_CATCHING=1  -c -o main.o main.cpp
Traceback (most recent call last):
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/em++.py", line 14, in <module>
    sys.exit(emcc.run(sys.argv))
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/emcc.py", line 1074, in run
    linker_inputs = phase_compile_inputs(options, state, newargs, input_files)
  File "/nix/store/dqxic3j7csd4ywn94n4smmnz55p039g3-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/emcc.py", line 2436, in phase_compile_inputs
    compile_source_file(i, input_file)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/emcc.py", line 2416, in compile_source_file
    cmd = get_clang_command(input_file)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/emcc.py", line 2357, in get_clang_command
    return get_compiler(use_cxx(src_file)) + get_cflags(options, state.orig_args) + compile_args + [src_file]
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/emcc.py", line 879, in get_cflags
    system_libs.add_ports_cflags(cflags, settings)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 1969, in add_ports_cflags
    port.get(Ports, settings, shared)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/tools/ports/sdl2.py", line 23, in get
    ports.fetch_project('sdl2', 'https://github.com/emscripten-ports/SDL2/archive/' + TAG + '.zip', SUBDIR, sha512hash=HASH)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 1765, in fetch_project
    fullname = os.path.join(Ports.get_dir(), name)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 1746, in get_dir
    shared.safe_ensure_dirs(dirname)
  File "/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/tools/utils.py", line 27, in safe_ensure_dirs
    os.makedirs(dirname, exist_ok=True)
  File "/nix/store/dqxic3j7csd4ywn94n4smmnz55p039g3-python3-3.9.6/lib/python3.9/os.py", line 225, in makedirs
    mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/cache/ports'
make: *** [Makefile:70: main.o] Error 1

During building of the example project make tries to execute the command:
em++ -DIMGUI_DISABLE_FILE_FUNCTIONS -I../.. -I../../backends -Wall -Wformat -Os -s USE_SDL=2 -s DISABLE_EXCEPTION_CATCHING=1 -c -o main.o main.cpp
which fails with the error:
OSError: [Errno 30] Read-only file system: '/nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/cache/ports'

Notify maintainers

@qknight
@matthewbauer

Metadata

• system: "x86_64-linux"
• host os: Linux 5.13.4, NixOS, 21.11 (Porcupine)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.15
• channels(root): "nixos-21.11pre318500.51bcdc4cdaa"
• channels(mbust): ""
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[gebner]

Emscripten's compilation model clashes a bit with nix. For obvious reasons, emscripten requires wasm-compiled libc and libc++ libraries. Vanilla emscripten compiles these libraries on demand when they are first needed (this is called caching, and the compiled library is stored in the cache directory).

In nix, this approach is very unsatisfactory because it means that emscripten would recompile libc again from scratch in every derivation that uses emscripten (because using an outside cache would of course make the derivation nonreproducible). Therefore we are creating a read-only cache during the emscripten build that contains "all possible" pre-compiled versions of libc and libc++ (there are different builds depending on whether you exceptions enabled or not, etc.).

Sometimes we are missing one such combination, maybe this is the case here.

[busti]

Thanks to @yu-re-ka for providing me with a quick-fix for this:

cp -r /nix/store/ajv94air67dbyjramrjlv0y6j969hsyx-emscripten-2.0.27/share/emscripten/cache ~/.emscripten_cache
chmod u+rwX -R ~/.emscripten_cache
export EM_CACHE=~/.emscripten_cache

@gebner I see how this approach is necessary to provide an experience that is satisfactory for nixos.
However, doesn't emscripten also cache all libraries that the compiled sources link against? And if that is the case, doesn't that mean that a mutable cache location is still needed, although most common libraries are already provided by the readonly cache?

Whatever the case, I am personally not trying to create a nixos package, but instead want to use emscripten on the command line.
The above commands provide a way to make emscripten work on the command line by changing the cache to a mutable location.

Could this behavior perhaps be implemented into the package so that using emscripten in private projects does not fail with cryptic error messages?

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/emscripten-tries-to-write-to-nix/15263/2

[gebner]

However, doesn't emscripten also cache all libraries that the compiled sources link against?

To the best of my knowledge, no, it doesn't. It only caches libc and libc++. You are responsible to provide other libraries (like you would with libfoo.so for a native target).

Could this behavior perhaps be implemented into the package so that using emscripten in private projects does not fail with cryptic error messages?

This is an interesting idea. Maybe we should just add a line here to check if there is a precompiled file available. (That is, first try the emscripten package and then fall back to the emscripten cache in the home directory.)

[busti]

Good suggestion.
I must admit that I am looking at this from a pure user perspective. I just expect the commands to work as the would on any other distribution. I know almost nothing about what's going on behind the scenes.
But from what you are saying it sounds like the problem isn't with nixos but with emscripten itself.
Almost all other Unix tools that have config or cache files somewhere usually have a system wide and a user specific config, so why shouldn't emscripten.

[stale]

I marked this as stale due to inactivity. → More info

[willcohen]

Note that #172207 attempts to resolve this by just creating a local dot directory for each package. I'm not sure how else this could be addressed with a more global system-wide cache given the restrictions on previous derivations being writable.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/improving-an-emscripten-yarn-dev-shell-flake/33045/1

[bergkvist]

This worked for me:

{ pkgs ? import (builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/refs/tags/23.05.tar.gz";
    sha256 = "10wn0l08j9lgqcw8177nh2ljrnxdrpri7bp0g7nvrsn9rkawvlbf";
  }) {}
}:
pkgs.mkShell {
  EM_CONFIG = pkgs.writeText ".emscripten" ''
    EMSCRIPTEN_ROOT = '${pkgs.emscripten}/share/emscripten'
    LLVM_ROOT = '${pkgs.emscripten.llvmEnv}/bin'
    BINARYEN_ROOT = '${pkgs.binaryen}'
    NODE_JS = '${pkgs.nodejs-18_x}/bin/node'
    CACHE = '${toString ./.cache}'
  '';
  buildInputs = [
    pkgs.emscripten
  ];
}

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/emscripten-tries-to-write-to-nix/15263/3