performance: mark REPL and Doc code as non-specializeable #28065

vtjnash · 2018-07-11T20:05:40Z

Here, we introduce a new meaning to the @nospecialize macro and allow it to be used at global scope and then apply to an entire module (and mostly any type signature).

When used without arguments, it applies to all arguments of the parent.
In local scope, this means the containing function.
In global scope, this means all methods subsequently defined.

stevengj · 2018-07-11T20:44:56Z

base/essentials.jl

 When used in a function body, the macro must occur in statement position and
 before any code.

+When used without arguments, it applies to all arguments of the parent scope.
+In local scope, this means all arguments of the containing function.
+In global (top-level) scope, this means all methods subsequently defined in the current module.


It seems more natural to me to use a syntax like

@nospecialize foo(...) = ... # applies to all arguments @nospecialize begin ... end # applies to all methods defined in this block @nospecialize module Foo ... end # applies to all methods in module Foo

rather than acting like an imperative side-effect, and to have a @nospecialize false begin ... end antonym.

Sure, all of those can be added later to the macro. They'd just be syntax transforms to one of these forms, since this form – as an imperative side-effect – represents where that information needs to be present in the processing order (since method declaration is itself also imperative).

FWIW, @nospecialize false is double negation.

It does seem clearer for the low-level control to be @specialize! true|false defaulting to true to avoid the double negative. Then @nospecialize block forms like @stevengj outlined can be built on top of that.

Yes, for the module-level form @specialize is better, but I'd keep @nospecialize as well since it's much more convenient when annotating a single argument. Then we could potentially use those instead of the true/false argument.

How about:

@nospecialize [begin] define() = functions @nospecialize [end]

I can’t tell if you’re trolling

KristofferC · 2018-07-13T11:29:14Z

Any numbers to show the impact of this change?

KristofferC · 2018-07-13T11:30:12Z

base/compiler/ssair/show.jl

@@ -26,7 +28,7 @@ end
 print_ssa(io::IO, @nospecialize(val), argnames) = Base.show(io, val)


-function print_node(io::IO, idx::Int, @nospecialize(stmt), used, argnames, maxsize; color = true, print_typ=true)
+function print_node(io::IO, idx::Int, @nospecialize(stmt), used, argnames, maxsize; color::Bool=true, print_typ::Bool=true)


Can the @nospecialize be removed here since it is in a @nospecialize true block?

It could, but I like the reminders that this code will run very slowly if you aren't careful

Implicit return is bad for compiler performance (and sometimes runtime performance) and can adversely affect code readability, so every function which does _not_ return a value should end in a `return` statement. Here, we also introduce a new meaning to the `@nospecialize` macro, and a new macro `@specialize` to reverse its effect. When used without arguments, it applies to all arguments of the parent. In local scope, this means the containing function. In global scope, this means all methods subsequently defined.

JeffBezanson · 2018-07-20T05:22:47Z

So far I don't see any improvement from this in the sysimg build or using CSV or using PyPlot. It seems to make corecompiler.ji take ~10 seconds longer to build. Delays for actions in the REPL seem to improve a bit but that's harder to measure.

JeffBezanson · 2018-07-20T05:50:19Z

base/compiler/tfuncs.jl

@@ -466,7 +468,7 @@ add_tfunc(<:, 2, 2,
              return Bool
          end, 0)

-function const_datatype_getfield_tfunc(sv, fld)
+function const_datatype_getfield_tfunc(@nospecialize(sv), @nospecialize(fld))


fld should be an Int.

JeffBezanson · 2018-07-20T05:54:03Z

Ok, the 10 seconds is not reliable. Maybe closer to 5 seconds, and seems to possibly be caused by the fld argument to const_datatype_getfield_tfunc.

JeffBezanson · 2018-07-25T14:55:54Z

This increased the time to get to the prompt (timed by modifying the repl to exit as soon as it gets to the prompt) from 0.7 seconds to 1.18 seconds. It's probably preventing us from recursively finding more code during precompilation. I assume it will be fixed by #28118.

KristofferC · 2018-07-25T14:59:47Z

Could you share the timing code you use and I can compare.

JeffBezanson · 2018-07-25T15:16:20Z

I added exit() to the beginning of write_prompt(terminal, p::Prompt) in LineEdit.jl, and ran time ./julia -q.

KristofferC · 2018-07-25T15:51:54Z

Master: 1.05s, #28118: 0.18s.

StefanKarpinski · 2018-07-25T20:24:55Z

Why was this merged? There was a request for some performance numbers which was completely ignored. Jeff has timed it and it turns out it had a significant detrimental impact on startup time.

vtjnash · 2018-07-26T03:55:58Z

I think KristofferC just posted that this’ll bring startup time down to 0.25s. This causes too many transient effects (precompile changes) to give a useful demo. But it’s just another API to trigger an existing optimization, so timing isn’t important. Just don’t use it if it doesn’t help you.

StefanKarpinski · 2018-07-26T04:26:09Z

This increased the time to get to the prompt (timed by modifying the repl to exit as soon as it gets to the prompt) from 0.7 seconds to 1.18 seconds.

?? 0.25s is not mentioned anywhere that I can see.

KristofferC · 2018-07-26T05:34:21Z

But it’s just another API to trigger an existing optimization, so timing isn’t important. Just don’t use it if it doesn’t help you.

I don't understand this comment. The discussions here are not about the new API itself but rather where this was used and the effects on timing this had. If the REPL is slower to start now than before, the what was the point?

martinholters · 2018-07-26T06:40:56Z

Assuming we get #28118 into 0.7 (fingers crossed), the relevant question seems to be what the timing of #28118 vs. #28118 with this reverted is.

vtjnash · 2018-07-27T16:34:02Z

@KristofferC The point is to make compilation more effective, since this guarantees that we can precompile the entirety of this module. We can always regenerate the precompile statements for whatever specific cases are not running fast enough, and alter the precompilation heuristics to make more effective use of this information later for specific use cases, so performance numbers of specific examples aren't that meaningful.

StefanKarpinski · 2018-07-27T16:41:23Z

Much of the confusion on this issue could have been headed off by simply saying that.

JeffBezanson · 2018-07-27T16:44:29Z

Yes, on the one hand it's true that the purpose of this is to introduce a new tool that can be used to address latency, but it's important to have an example use case that demonstrates the utility.

As for compilation itself, I think what's happening here is that the REPL code calls other code (e.g. in Base) that does need to be specialized, but since the REPL code isn't specialized we don't find those signatures until run time.

stevengj reviewed Jul 11, 2018

View reviewed changes

vtjnash mentioned this pull request Jul 12, 2018

Make julia + repl start twice as fast [do not merge] #28075

Closed

JeffBezanson added the compiler:latency Compiler latency label Jul 12, 2018

KristofferC reviewed Jul 13, 2018

View reviewed changes

vtjnash force-pushed the jn/nospecialize-module branch 3 times, most recently from b3538a5 to 483a668 Compare July 18, 2018 19:11

vtjnash force-pushed the jn/nospecialize-module branch from 483a668 to 03dfc52 Compare July 19, 2018 16:30

vtjnash merged commit 9ed6287 into master Jul 20, 2018

vtjnash deleted the jn/nospecialize-module branch July 20, 2018 04:01

JeffBezanson reviewed Jul 20, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

performance: mark REPL and Doc code as non-specializeable #28065

performance: mark REPL and Doc code as non-specializeable #28065

vtjnash commented Jul 11, 2018

stevengj Jul 11, 2018 •

edited

Loading

vtjnash Jul 12, 2018

alyst Jul 12, 2018

StefanKarpinski Jul 13, 2018

JeffBezanson Jul 13, 2018

vtjnash Jul 13, 2018

StefanKarpinski Jul 13, 2018

KristofferC commented Jul 13, 2018 •

edited

Loading

KristofferC Jul 13, 2018

vtjnash Jul 19, 2018

JeffBezanson commented Jul 20, 2018

JeffBezanson Jul 20, 2018

JeffBezanson commented Jul 20, 2018

JeffBezanson commented Jul 25, 2018

KristofferC commented Jul 25, 2018

JeffBezanson commented Jul 25, 2018

KristofferC commented Jul 25, 2018 •

edited

Loading

StefanKarpinski commented Jul 25, 2018

vtjnash commented Jul 26, 2018

StefanKarpinski commented Jul 26, 2018

KristofferC commented Jul 26, 2018

martinholters commented Jul 26, 2018

vtjnash commented Jul 27, 2018

StefanKarpinski commented Jul 27, 2018

JeffBezanson commented Jul 27, 2018

performance: mark REPL and Doc code as non-specializeable #28065

performance: mark REPL and Doc code as non-specializeable #28065

Conversation

vtjnash commented Jul 11, 2018

stevengj Jul 11, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KristofferC commented Jul 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JeffBezanson commented Jul 20, 2018

Choose a reason for hiding this comment

JeffBezanson commented Jul 20, 2018

JeffBezanson commented Jul 25, 2018

KristofferC commented Jul 25, 2018

JeffBezanson commented Jul 25, 2018

KristofferC commented Jul 25, 2018 • edited Loading

StefanKarpinski commented Jul 25, 2018

vtjnash commented Jul 26, 2018

StefanKarpinski commented Jul 26, 2018

KristofferC commented Jul 26, 2018

martinholters commented Jul 26, 2018

vtjnash commented Jul 27, 2018

StefanKarpinski commented Jul 27, 2018

JeffBezanson commented Jul 27, 2018

stevengj Jul 11, 2018 •

edited

Loading

KristofferC commented Jul 13, 2018 •

edited

Loading

KristofferC commented Jul 25, 2018 •

edited

Loading