-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide alternative to warming up multiple identical Contexts #67
Comments
Thank you for this excellent question.
It currently depends on the language how much of the code is reused for multiple contexts per engine. Currently for Graal.JS the ASTs and code cache is only shared if the previous context was closed. This is currently necessary as we are not fully confident to share the ASTs with multiple threads. To clarify, this one shares the code:
And, this one doesn't atm:
I agree the current situation is not ideal. This is what we will do to improve the situation:
Would 1) help you already? Or do you need your contexts on multiple threads? |
Hi @chumer, thanks a lot for that info!
Ok great, I gave something similar to the below a try and it didn't perform the way I expected. I expected that the eval in the second Engine engine = Engine.create();
Source source = ...;
try(Context context = Context.newBuilder().engine(engine).build()) {
for (i=0; i<10000; i++) context.eval(source);
context.eval(source); // <-- at this point, eval takes ~5 ms
}
try(Context context = Context.newBuilder().engine(engine).build()) {
context.eval(source); // <-- at this point, eval takes 300+ ms
}
I don't think that would help. My goal is to have a number of different contexts, each bound to a specific thread. In other words, I don't want to share contexts between threads, I'd just like to create one context, warm it up, create warm copies, then allocate it and the copies to a thread each. In simple sample code I'm envisioning something like this: Engine engine = Engine.create()
Context context1 = Context.newBuilder().engine(engine).build();
warmup(context1);
Context context2 = context1.copy();
Context context3 = context1.copy();
Context[] contextsPerThread = new Array(context1, context2, context3); Where as at the moment I need to do something like this: Engine engine = Engine.create()
Context context1 = Context.newBuilder().engine(engine).build();
warmup(context1);
Context context2 = Context.newBuilder().engine(engine).build();
warmup(context2);
Context context3 = Context.newBuilder().engine(engine).build();
warmup(context3);
Context[] contextsPerThread = new Array(context1, context2, context3); (Oh and btw, I absolutely ❤️love ❤️ the work that you all have done on Graal! It's amazing! Thank you so much for all you have done and will continue to do, it's much appreciated!) |
We will have a look at your example. Just to clarify which GraalVM version are you using? |
I'm using rc8. Thanks
…On Thu., 1 Nov. 2018, 11:27 pm Christian Humer ***@***.*** wrote:
We will have a look at your example. Just to clarify which version are you
using?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#67 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAMYtyoSKdo_rVfAv2n_6skfvaVjDe1Lks5uquiugaJpZM4YDpmi>
.
|
(on Linux)
…On Fri., 2 Nov. 2018, 9:26 am David Barri ***@***.*** wrote:
I'm using rc8. Thanks
On Thu., 1 Nov. 2018, 11:27 pm Christian Humer ***@***.***
wrote:
> We will have a look at your example. Just to clarify which version are
> you using?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#67 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAMYtyoSKdo_rVfAv2n_6skfvaVjDe1Lks5uquiugaJpZM4YDpmi>
> .
>
|
Looks like the warmup overhead disappears after a few contexts..
After the 6th context, see a sudden speedup. And this is on windows, openjdk11 with the rc8 graal/truffle/graaljs jars.. |
Hi @hanzr What do you mean with "overall execution is a bit slower"? With But what I also see is that with
The reason for that seems to be that in the first set, we speculate on having only one Context, while in the second set, this assumption does not hold any longer. With
@chumer can go into more detail, but I think this drop is somewhat expected. |
Hi Christian, I'm getting these numbers for
For context 6, the timings (after the first one) are slower than those of the earlier contexts. Then for context 7, there seems to be no more slow first timing (no more warmup?), and times remain pretty constant after that. |
Hi. I've created a reproduction repo that you can run that demonstrates the kind of results I'm seeing. To reproduce, install SBT (Scala Build Tool), check out https://github.com/japgolly/misc/tree/graaljs-67 and type What it does is,
Results are:
and if I don't call
|
We are also looking to migrate from Nashorn to Graal and this is one of our biggest blockers. In nashorn we could compile just once and use the same object across multiple threads. You can argue that its not thread safe, but most of our functions have no side effects and this does not pose much problems for us. But in Graal we have to initialize the context per thread and pay the warmup costs. |
Hi @karthickpdy, did you follow the description in http://www.graalvm.org/docs/graalvm-as-a-platform/embed/#enable-source-caching ? As described above, multi-threading might have an impact on warmup and the first few iterations (as our compiler is competing for CPU resources with other threads), but the -- Christian |
I don't think there is anything open here. Documentation on Source Code caching is at https://www.graalvm.org/reference-manual/embed-languages/#code-caching-across-multiple-contexts Please reopen this ticket with a clarifying question or open a new ticket if anything is unclear there. |
I'm planning to create a pool of Contexts that will all do the same thing, that I warm up for ~30 sec first.
The problem is even though I'm sharing the same Engine between the Contexts and evaluating the same Sources in each in the same order, context B doesn't benefit from context A's warmup. I was thinking the Engine is where all of the JIT memory would be and warming up one context would warm up the engine thus having the effect of all contexts being warmed up.
Is this as designed?
If so, what options are there for sharing optimisation between identical contexts? Being used to FP and everything being immutable, an idea I had was to create and warm up one Context, then copy it to create the other Contexts (from which point they could diverge) but I didn't see such an option. Is there something similar available?
Even better would be to warmup a Context then be able to serialise it! That would allow us to create one at build-time, include it in our applications docker builds, then on startup simply deserialise it mulltiple time to not even need warmup on application startup.
What do you think?
The text was updated successfully, but these errors were encountered: