HCR not working on OSX

[sacredbirdman]

Compilation of a hot code reloading sample with:

$ nim c --hotcodereloading:on mymain.nim

went fine but running it doesn't work:

$ ./mymain 
could not load: libnimhcr.dylib

Looks like libnimhcr.dylib is nowhere to be found.

$ nim -v
Nim Compiler Version 1.3.1 [MacOSX: amd64]
Compiled at 2020-04-16
Copyright (c) 2006-2020 by Andreas Rumpf

git hash: 6914de0d8dfb14e4ebe0448c558466c9bb80d833

[alaviss]

See #13924

[timotheecour]

very related: #13388 (comment)

the existing https://nim-lang.github.io/Nim/hcr.html is doing a better job at that than a platform specific code that's not even being used anymore (guaranteed to go out of sync; and doesn't support OSX which uses DYLD_LIBRARY_PATH, not LD_LIBRARY_PATH)
plus, IMO nim is good enough not to need any bash script except for bootstrap

that PR got reverted (which is fine) but the non-portable bash scripts (tests/dll/test_nimhcr_integration.sh, tests/dll/test_nimhcr_integration.bat) should be turned into a single nim test that would work cross platform and be part of CI, instead of remaining as platform specific dead code that doesnt' work on OSX

IMO we should keep this issue opened until it works on OSX and the sh+bat script is converted to a nim test tested in CI

---

here's what I tried:

nim c lib/nimrtl.nim
nim c lib/nimhcr.nim
# copy example from $nim/doc/hcr.rst to main.nim
nim c --hotcodereloading:on -d:nimDebugDlOpen main
DYLD_LIBRARY_PATH=lib main
./main
# edit main.nim
nim c --hotcodereloading:on -d:nimDebugDlOpen main
# press F9 (or fn F9 on osx depending on your setup)
# => nothing happens
# press F9 again
# => crash

Traceback (most recent call last)
/Users/timothee/git_clone/nim/timn/tests/nim/all/t10655.nim(23) t10655
/Users/timothee/git_clone/nim/timn/tests/nim/all/t10655.nim(20) main
/Users/timothee/git_clone/nim/timn/tests/nim/all/t10655b.nim(37) update
SIGBUS: Illegal storage access. (Attempt to read from nil?)

TODO:

• we should have CT error or warning when nim versions for (nimrtl, nimhcr, mainproj) don't match (otherwise you may get a crash, or worse, wrong results)
• we should have an option to automatically (re)build nimrtl,nimhcr if they don't exist or are out of date
• we shouldn't have to use DYLD_LIBRARY_PATH ; maybe instead inserting appropriate {.passl.} flags (eg rpath on OSX or similar)
• that sdl example should work on OSX and be moved to a nim file (maybe $nim/tests/dll/test_nimhcr_dll.nim, and linked from the docs but be tested to work in CI, otherwise it will eventually stop working

[timotheecour]

@zah since you've introduced HCR, help would be welcome to make HCR work on OSX, as it also affects PR's seemingly unrelated to HCR, such as #16223; thanks!

[timotheecour]

some progress: retried #13999 (comment) with latest devel and this works!

[timotheecour]

• another weird bug on OSX, which could be related: commandLineParams doesn't seem to work with --hotCodeReloading:on but it's hard to reproduce

# main.nim:
import os
echo commandLineParams()

nim c lib/nimhcr.nim
nim c --forceBuild --hotCodeReloading:on -o:/tmp/z08 main.nim
DYLD_LIBRARY_PATH=lib /tmp/z08 a1 a2
@[] # bug

nim c --forceBuild --hotCodeReloading:on -o:/tmp/z08 --passl:-Wl,-rpath,lib
/tmp/z08 a1 a2
@[]

using 1.5.1 edce589

however if I use --nimcache, this bug goes away, which is weird because --forceBuild should ignore what's in nimcache

nim c --forceBuild --hotCodeReloading:on -o:/tmp/z08 --passl:-Wl,-rpath,lib --nimcache:/tmp/d07 main.nim
/tmp/z08 a1 a2
@["a1", "a2"]

EDIT: filed #16265 for this

[timotheecour]

• and yet another bug:

nim c --forceBuild --hotCodeReloading:on -o:/tmp/z09 -d:case2 --passl:-Wl,-rpath,lib --listcmd --hint:cc --hint:link --skipusercfg --skipparentcfg --nimcache:/Users/timothee/git_clone/nim/timn/build/nimcache/ $timn_D/tests/nim/all/t11450.nim

shows link commands like look like cc commands:

CC: stdlib_os.nim: clang -c  -w -ferror-limit=3  -fPIC  -I/Users/timothee/git_clone/nim/Nim_devel/lib -I/Users/timothee/git_clone/nim/timn/tests/nim/all -o /Users/timothee/git_clone/nim/timn/build/nimcache/stdlib_os.nim.c.o /Users/timothee/git_clone/nim/timn/build/nimcache/stdlib_os.nim.c
CC: t11450.nim: clang -c  -w -ferror-limit=3   -I/Users/timothee/git_clone/nim/Nim_devel/lib -I/Users/timothee/git_clone/nim/timn/tests/nim/all -o /Users/timothee/git_clone/nim/timn/build/nimcache/@mt11450.nim.c.o /Users/timothee/git_clone/nim/timn/build/nimcache/@mt11450.nim.c
Link: stdlib_since.nim: clang -c  -w -ferror-limit=3  -fPIC  -I/Users/timothee/git_clone/nim/Nim_devel/lib -I/Users/timothee/git_clone/nim/timn/tests/nim/all -o /Users/timothee/git_clone/nim/timn/build/nimcache/stdlib_since.nim.c.o /Users/timothee/git_clone/nim/timn/build/nimcache/stdlib_since.nim.c