Error C stack usage too close to the limit (when precompiling w Julia 1.2)

[tlienart]

Hello, I added RCall today and:

using RCall
[ Info: Precompiling RCall [6f49c342-dc21-5d91-9882-a32aef131414]
Error: C stack usage  140727995553552 is too close to the limit
Error: C stack usage  140727995553600 is too close to the limit
Error: C stack usage  140727995553520 is too close to the limit
Fatal error: unable to initialize the JIT

(I tried this twice, it failed both time with a similar error)

version info

Julia Version 1.2.0-DEV.17
Commit 01fe7c2698 (2018-12-14 21:29 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

Edit:

I tried this with a couple of versions:

• 1.0.2 works
• 1.1 fails (same error)

[randy3k]

I have seen this error before in other context when the R engine was running in a second thread. Not sure if it’s related here.

[tlienart]

if you're suggesting that it may be because I was running R at the same time, this does not seem to be the case (I just retried now making sure that there was zero active R processes).

[randy3k]

No. It wasn't what I meant. It seems that the new version of Julia is compiling RCall in a second thread and R doesn't like to be running in a second thread.

[mkborregaard]

I have this in 1.1

[mkborregaard]

But it works on 1.0

[randy3k]

I could also reproduce it with macOS + julia 1.2-dev.

Rhome = readchomp(`R RHOME`)
ENV["R_HOME"] = Rhome
ENV["R_DOC_DIR"] = joinpath(Rhome,"doc")
ENV["R_INCLUDE_DIR"] = joinpath(Rhome,"include")
ENV["R_SHARE_DIR"] = joinpath(Rhome,"share")
const libR = Rhome * "/lib/libR.dylib"    # mac
# const libR = Rhome * "/lib/libR.so"    # linux
argv = ["REmbeddedJulia","--silent","--no-save"]
ccall((:Rf_initEmbeddedR,libR),Cint,(Cint,Ptr{Ptr{Cchar}}),length(argv),argv)

Definitely caused by a change in upstream code.
@simonbyrne Do you have any idea?

[Non-Contradiction]

Everything works fine if we use JuliaCall to start an embedded Julia in R, and then use RCall inside the embedded Julia, since we don't have to start embedded R in this way. And I also have some interesting observation:

> library(JuliaCall)
> julia_setup(JULIA_HOME = "/Applications/Julia-1.1.app/Contents/Resources/julia/bin/")
Julia version 1.1.0-rc1.0 at location /Applications/Julia-1.1.app/Contents/Resources/julia/bin will be used.
Loading setup script for JuliaCall...
Finish loading setup script for JuliaCall.
> julia_console()
Preparing julia REPL, press Ctrl + D to quit julia REPL.
......
julia> using RCall

julia> R"1+1"
Error: C stack usage  140727517926320 is too close to the limit
ERROR: LoadError: REvalError: 
in expression starting at REPL[2]:1

julia> 
> julia_command('R"1+1"')
[1] 2
RObject{RealSxp}

So a stack error appears again when we try to use RCall after we start a Julia REPL, but is totally fine without a Julia REPL. I don't know if the two errors are related.

[Non-Contradiction]

Inspired by the last example, I also tried something like this:

/Applications/Julia-1.1.app/Contents/Resources/julia/bin/julia -e 'using RCall; println(R"1+1")'
RObject{RealSxp}
[1] 2

So using RCall non-interactively is also fine. I guess the problem is related to REPL, event loop or something like that.

[randy3k]

Great discovery. It must be because now the events are being processed asynchronously.
Disabling https://github.com/JuliaInterop/RCall.jl/blob/master/src/setup.jl#L175 should avoid the error.

[randy3k]

But it doesn’t explain why my code example above fails.

[mkborregaard]

Given that 1.1 is going to be out any day this is likely to get a lot of attention soon.

[randy3k]

Unfortunately, I don't really have a clue on how to fix the issue.

[randy3k]

Can anyone confirm that the same issue occurs on Windows?

[randy3k]

I have started a discussion at julia forum: https://discourse.julialang.org/t/initialization-of-r-fails-on-julia-1-1/19735

Hope we will be able to get some help for julia dev.

[simonbyrne]

The issue is that Julia no longer loads libraries in the main thread, which interacts badly with R's design.

It should be possible to work around this according to these instructions:
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Threading-issues

[randy3k]

But it doesn’t explain why it works in the non-interactive environment.

[simonbyrne]

But it doesn’t explain why it works in the non-interactive environment.

It probably has something to do with R_running_as_main_program: it's not really documented, but it has something to do with how the stack checking works.

[randy3k]

Is there any Windows user to verify whether this bug also affects them?

[helske]

I have similar, probably related issue with Windows 10, Julia 1.1.0 and R-devel (2019-01-24 r76013) (also happened with older R version, 3.5.1 I think), and latest Atom and related packages. This does not happen all the time though. Basically I am running some MCMC simulations, and afterwards I have

using RCall
@rput output
"R
save(output, file="test.rds")
"

and with fresh session this works, but if I try to use @rput second time (even with same object) I get

REvalError: Error: C stack usage  388116496 is too close to the limit
in top-level scope at base\none
in setindex! at RCall\RPlFw\src\methods.jl:545
in setindex! at RCall\RPlFw\src\methods.jl:540
in setindex! at RCall\RPlFw\src\methods.jl:531
in  at base\none
in #rcall_p#36 at RCall\RPlFw\src\language.jl:30
in reval_p at RCall\RPlFw\src\eval.jl:95 
in reval_p at RCall\RPlFw\src\eval.jl:99
in #handle_eval_stderr at base\none 
in #handle_eval_stderr#41 at RCall\RPlFw\src\io.jl:112

edit: But now after closing Rstudio and opening it again, I don't see the error anymore...

[Morpheu5]

Hey, just adding my 2c here, I have the same issue running Julia 1.1 and RCall on macOS 10.14.2 (18C54). In my case, I can't run using RCall without the REPL choking and spitting out

Error: C stack usage  140727749152224 is too close to the limit
Error: C stack usage  140727749152272 is too close to the limit
Error: C stack usage  140727749152176 is too close to the limit
Fatal error: unable to initialize the JIT

I am running Julia inside a Jupyter notebook, so I guess my only option is using it interactively. I also tried overwriting isinteractive to make it return false

import Base.isinteractive

function isinteractive()
    return false
end

to no avail.

EDIT: I can confirm that this does not seem to be a problem with Julia 1.0.

[randy3k]

@helske and @Morpheu5
There is a potential fix in place #293, would you be able to verify that the fix works for you?

[mkborregaard]

I can confirm the fix works for me on Julia 1.1 on macOS 10.14.2

[Morpheu5]

@randy3k it seems to work fine for me, thanks 👍

[bjce]

same issue here with Julia 1.1 on macOS 10.14. What would be the step by step fix for a non programmer like me...

[mkborregaard]

for now ]add RCall#sb/stack

[randy3k]

and run ]free RCall after a new version of RCall is released.

[bjce]

Great guys!! thank you very much! As I don't quite understand what those two commands mean, and in general have trouble to understand the github/version/commit system but I'm very eager to learn as much as I can about Julia, and github, could you advise me a tutorial link as a good starting point to learn about those topics (I know it is not related to RCall anymore and the question is quite broad but if you have any link in mind I would really appreciate!). Many thanks @mkborregaard and @randy3k

[mkborregaard]

So, essentially Simon Byrne made a fix and put it in a "branch" of the repository, which just means a version of the repository that is not yet official. The name of that branch is "sb/stack". The guys behind RCall wanted to make sure that it actually did solve everybodys problems before putting it in the default branch (called "master") and releasing it out to all the RCall users out there.

Pkg in Julia allows users to use other branches than the default one. ]add RCall#sb/stack simply means "I want to use the unofficial version of RCall called sb/stack" (read more on Pkg here https://docs.julialang.org/en/v1/stdlib/Pkg/ ).

The other command ]free RCall to call once the fix becomes official simply means "I don't want to follow the special branch anymore". If you didn't do that you would stop getting updates from RCall because they only happen to master.

[bjce]

Many thanks @mkborregaard. That really helped a lot!