WIP: static compile part 4 (user-interface)

[vtjnash]

(continuation of #8656 – as I make progress towards incremental module compilation, I'll push the code to this branch for comments and API discussion)

since Julia already has a strong tradition that file != module, I think the primary implementation of this needs to embrace that and use a *.jlc file independent of the filesystem. this unifies the behavior using and everything in Base across restarts. Therefore, a session could look something like the following:

julia> using A
note: cache file ~/.julia/v0.4/.lib/A.jlc cannot be used with this Julia image, ignoring file
note: regenerating cache file for module A with code from /home/user/.julia/v0.4/A/src/A.jl

julia> using B
note: cache file ~/.julia/v0.4/.lib/B.jlc cannot be used with this Julia image, ignoring file
note: could not find `B.jl` in PATH to satisfy requested module dependency
ERROR: B not found
 in require at ./loading.jl:49
 in require at /Volumes/Lion/Users/jameson/Documents/julia/usr/lib/julia/sys.dylib

julia> using C
note: regenerating cache file for module C with code from /home/user/.julia/v0.4/C/src/C.jl

julia> using D
note: could not find `D.jl` in PATH to satisfy requested module dependency
ERROR: B not found
 in require at ./loading.jl:49
 in require at /Volumes/Lion/Users/jameson/Documents/julia/usr/lib/julia/sys.dylib

julia> using D # this one is already in the session

julia> using E # this one is already in the cache (`~/.julia/v0.4/E.jlc`)

julia>

And invoking the cache file engine would look something like the following:

$ ./julia -f --build $HOME/.julia/v$(cut -d . -f 1,2 VERSION)\
    -J sys.ji -L LOAD_PATH.jl
    ~/.julia/v0.4/FixedPointNumbers/src/FixedPointNumbers.jl

or via stdin:

$ ./julia -f --build $HOME/.julia/v$(cut -d . -f 1,2 VERSION)\
    -J sys.ji -
empty!(LOAD_PATH)
push!(LOAD_PATH, "path1")
push!(LOAD_PATH, "path2")
include("~/.julia/v0.4/FixedPointNumbers/src/FixedPointNumbers.jl")
^D

any logic to handle dependencies differently would be handled externally, e.g. https://github.com/malmaud/Autoreload.jl

[vtjnash]

note, it will perhaps be necessary to protect against semi-malicious users when trying to derive file paths:

julia> Expr(:module, false, symbol("/tmp/bad"), Expr(:block))
:(baremodule /tmp/bad
    end)

julia> eval(Expr(:module, false, symbol("/tmp/bad"), Expr(:block)))

julia> getfield(Main,symbol("/tmp/bad"))|>names
1-element Array{Symbol,1}:
 symbol("/tmp/bad")

[JeffBezanson]

In this scheme, what are the chances of the caching being fully automatic? Package authors selectively enabling the functionality might be ok, but anything more than that I'm not sure.

[vtjnash]

some things that are very likely to break this are quite easy to detect:

1. getting warnings about redefining something (a constant, type, or module)
2. calling eval(::OtherModule, :(some_code))

but consider the following code:

module A
  mycount = 0
  register() = global mycount += 1
end
module B
  A.register()
  const counterwas = A.mycount
end

unless the author explicitly marks B as "cache-able", I don't know how you would know whether to re-run A.register() or not. Fortunately, the answer is obviously to just move the call to A.register() inside B.__init__. But unfortunately, I cannot think of anyway to detect the user's intentions with any degree of accuracy, and don't think you possibly could.

[samoconnor]

A couple of months back I posted my experience with using Julia in a web services context:
https://groups.google.com/d/msg/julia-dev/E3LjK65jH6Y/EYBaZ1fYMSgJ
Very short summary: Start-up time is 10 seconds when a bunch of support libraries are required, that's too slow.

At this point I'm checking back to see what progress has been towards fixing this.

Question 1: Is there a single "master issue" in github for the startup time problem?

As an outsider I've found it frustrating to try to find where the dev's are discussing a particular issue. It seems that the core devs don't use the julia-dev list to discuss day to day progress. I can see why it's better in a lot of ways for this discussion to happen in pull-request comments and issue comments. However, for an outsider it makes it really hard to find "what's been going on with startup time recently".

Question 2: Does the #8745 scheme address dynamic startup code in libraries?

Words below snipped from julia-dev post: https://groups.google.com/d/msg/julia-dev/E3LjK65jH6Y/XwiD4RoFLiQJ

... Julia has nice dynamic language features, so it is tempting to do things dynamically at startup. e.g.

• Nettle.jl dynamically enumerates the hash algorithms provided by libnettle.so and dynamically generates Julia functions every time it starts up.
  https://github.com/staticfloat/Nettle.jl/blob/master/src/hash.jl#L31
• CRC.jl does not provide a CRC32 function, instead you do crc32 = crc(CRC_32) which generates a CRC32 lookup table and a generates a function that knows how to use the table to compute CRC32.
  https://github.com/andrewcooke/CRC.jl

It seems to me there will need to be some mechanism whereby the compiler can be sure that a particular piece of start-up code is statically deterministic and can therefore safely be pre-executed and serialised. Maybe this can be inferred by the compiler, or maybe there needs to be a keyword. If Nettle.jl startup depends on the version or config of the installed libnettle.so, then it presumably can't be statically pre executed and serialised for fast startup.

Perhaps it would make sense to have declared dependancies for startup code. e.g. The cached Nettle.jl binary can be used unless the hash of libnettle.so has changed...

[timholy]

@samoconnor, click on the "Watch" button above, and be prepared for a lot of email.

[stevengj]

@samoconnor, there is already a mechanism for runtime initialization: you define an __init__ function that executes anything that needs to happen at runtime. This was introduced in Julia 0.3, and is documented by #9240.

@vtjnash, unless we can statically determine whether a module is precompilable (which seems hard), it might be a good idea to precompile only modules that contain an __init__ function. The presence of an __init__ (which may be a no-op, of course) should be a reasonable heuristic to decide whether the programmer has gone through the code and made sure it is precompile-ready.

[timholy]

@stevengj, to me that sounds like a good solution---it makes it opt-in. Of course there's the chance that a change might make the system suddenly not precompilable, but presumably that would show up in tests.

[vtjnash]

that sounds like a great heuristic. the trouble is that we want to take some extra precautions when generating a cache version (sandboxing the run), which means that the ability to static compile needs to be something that can be determined ahead-of-time. the user interface for this thing is really the hardest part of this change to design. otherwise, the flexibility of the julia parser (with macros especially) makes it impossible to know what is able to be cached. sometimes I wonder if it might be worth forcing all modules to accept pre-compilation – they will just fail really fast if they weren't coded to expect it (explicit pointers are converted to NULL when serialized)

[stevengj]

Couldn't a very simplified version of the parser detect whether an __init__ is present? (Because this is a heuristic performance optimization, it's reasonable to ignore the unlikely possibility that __init__ is generated by a macro...you can even make that a feature: this is how to use __init__ without enabling precompilation.)

I'm concerned that it will make Julia much more challenging for newbies if we force this degree of discipline on them as soon as they make a module. A lot of my students have no formal training in computer science, have never dealt with a compiled language, and don't know what a pointer is.

[vtjnash]

Yes, that should work. Another way to hide the __init__ function then would just to be to put it in a separate file and include it.

[ihnorton]

We could call it __init__.jl...
:ducks:

[stevengj]

I think it would be fine to require that __init__ be in the top-level module file if you want precompilation, at least for now (although detecting include would not be too hard to add).

[vtjnash]

Detecting the function call is somewhat easy, but determining what file is going to be found is potentially brittle

[vtjnash]

i didn't mean to be silent on this PR for so long. Here's a (functional) sample demonstrating my latest progress:

$ ./julia -J usr/lib/julia/Base.ji --build usr/lib/julia2 <<EOF
using Compat
EOF
$ ./julia -J usr/lib/julia/Base.ji --build usr/lib/julia2 <<EOF
using FixedPointNumbers
@time C = ccall(:jl_restore_new_module, Any, (Ptr{Uint8},), "usr/lib/julia2/Compat.ji")
using Color
using Cairo
using Gtk
EOF
$ ./julia -q
julia> @time begin
           FP = ccall(:jl_restore_new_module, Any,
        (Ptr{Uint8},), "usr/lib/julia2/FixedPointNumbers.ji")
           C0 = ccall(:jl_restore_new_module, Any,
       (Ptr{Uint8},), "usr/lib/julia2/Compat.ji")
           C = ccall(:jl_restore_new_module, Any, 
       (Ptr{Uint8},), "usr/lib/julia2/Color.ji")
           C2 = ccall(:jl_restore_new_module, Any, 
       (Ptr{Uint8},), "usr/lib/julia2/Cairo.ji")
           nothing
       end
elapsed time: 0.614841921 seconds (11179028 bytes allocated)

julia> @time FP = ccall(:jl_restore_new_module, Any, 
       (Ptr{Uint8},), "usr/lib/julia2/Gtk.ji")
elapsed time: 1.424939038 seconds (25944208 bytes allocated)

julia> Gtk.GLib.__init__()

julia> Gtk.__init__()

julia> evalfile(Pkg.dir("Gtk/test/runtests.jl")); echo("SUCCESS")
SUCCESS

julia>

(edit: for comparison, the timings for loading these modules was 8.5s and 13.5s, respectively)

[timholy]

If more of us reading this had sharp teeth and claws, you could lose an arm by dangling such treats in front of us. How soon can we get this? Tomorrow?

[tknopp]

From the timing this could be a nice christmas present for the Julia community :-)

[samoconnor]

@vtjnash, loading modules in 10% of the original time is great.
But, 2000ms to load 5 libraries still seems about 100 x too long.

I don't know the details (and I assume that the devil is in the details), but I'm imagining that all the complexity and magic is in the compilation and serialisation code; and that the module restore basically just does: mmap(module.so); call module_init_fn(). Is all the startup time in the module init function?
If the module was written to be completely statically compilable (no init function) would the startup time just be the mmap() time?

I'm really interested to know what the 1400ms GTK load time is comprised of.
Would it be difficult for you to post a timing breakdown that shows how long each step takes?

1400ms is about 3 billion instruction cycles. It seems to me that there's got to be something badly inefficient somewhere in the stack for it to take 3 billion instructions to load a GUI toolkit language binding.

[vtjnash]

Jeffs call_overload branch hurt the timing of this branch. For Gtk it caused a massive regression in the operation of the serializer. It was a known issue and will be fixed before this is merged, but it wasnt originally a problem.

When this is closer to being complete, I'll start to work on profiling and optimization. For now, I'm focused on just getting it working.

[timholy]

I admire the thoughtful analysis about what should be possible, @samoconnor. I don't think that is done often enough.

That said, I'll take a "mere" 10x improvement without any complaints!

[JeffBezanson]

This is not the kind of thing that will run at a CPU's peak instruction rate.

@vtjnash Any clues about why it caused a regression? The dump.c serializer, or serialize.jl? I seem to recall Gtk uses both.

[ihnorton]

Well, only the inferred AST for now since it does not store native code yet.

Potentially relevant to the latter: Pyston has a new approach where (IIUC) they still do lowering to LLVM but then hash the IR and check that against a cache before emitting machine code.

http://blog.pyston.org/2015/07/14/caching-object-code/

[StefanKarpinski]

The Pyston approach is pretty clever. It seems like it works at a rather different level than package compilation, but could help the case where you do end up having to recompile things.

[JeffBezanson]

A very minor nit to be sure, but I wonder if compile should accept a string instead, since this is how we refer to package names in Pkg (and used to with require as well).

[malmaud]

That has bitten me at least 20 times in the last two days.

[stevengj]

Especially since the documentation says compile takes a string...

[stevengj]

Ultimately we are expecting this to be more automated, yes? i.e. if you make a @cacheable module Foo then it will automatically compile Foo whenever it has changed? I guess that is a separate PR?

[timholy]

I think @vtjnash was using a Symbol to try to convey "cached if available" rather than "source file"---the idea being, I suppose, that a Symbol is that much closer to a Module. See #12040. I do agree this is a bit of a subtle distinction, though.

[JeffBezanson]

That's understandable, but I think it would be better to be consistent with the other related functions.

[StefanKarpinski]

Agree, regarding consistency. Arguably all of these functions could take strings or symbols, but currently they take strings, so let's stick with that.

[vtjnash]

these functions used to take filepaths. they don't anymore.

(documentation is fixed now)

[tkelman]

Should we open a new issue for brainstorming on how to implement the automatic recompilation make part of this?

edit: moved to #12259

[stevengj]

Yes, please, @tkelman.

[sbromberger]

Hi all,

I just wanted to extend my thanks for this work. Here's another datapoint in case anyone cares:

julia> tic(); using LightGraphs; toc()
elapsed time: 2.959553017 seconds
2.959553017

julia> Base.compile(:LightGraphs)
"/Users/seth/.julia/lib/v0.4/LightGraphs.ji"

julia> tic(); using LightGraphs; toc()
elapsed time: 0.167611647 seconds
0.167611647

[StefanKarpinski]

17x improvement ain't bad.

[sbromberger]

~~So, silly question: I know that modifying the module source will require cache invalidation, but what about building a new julia master? Better safe than sorry and just wipe out ~/.julia/lib ?~~

The answer, of course, is yes, since Base gets rebuilt.

[JeffreySarnoff]

this is more fun than than ice cream in summer .. excellent elegant effort

[coveralls]

Changes Unknown when pulling e2d842a on jn/static_compile_4 into ** on master**.