Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[js] rework enum implementation #5109

Closed
nadako opened this issue Apr 12, 2016 · 17 comments
Closed

[js] rework enum implementation #5109

nadako opened this issue Apr 12, 2016 · 17 comments
Labels
discussion platform-javascript Everything related to JS / JavaScript
Milestone

Comments

@nadako
Copy link
Member

nadako commented Apr 12, 2016

Enums are one of the greatest features in Haxe, but its implementation often performs slower than other solutions, so I think we should maximize its performance.

I decided to start with JS target and did some benchmarking for the different possible implementations of enum: https://github.com/nadako/haxe-js-enum-benchmark

I ran the benchmark on Node 5.10.1 (V8 4.6.85.31) which I think is pretty indicative, since V8 nowadays is the most used js runtime (chrome, node, electron, etc). It's safe to assume that other popular engines are not very different from it.

Unless I did the benchmark wrong (please look at the (generated) source), it shows that using simple anon objects for enums is the fastest option for both creation and matching, so we might consider using those instead of arrays for enums.

I think this is related to how V8 optimizes objects by using hidden classes, and how it fails to optimize arrays which elements are of different type (like our current enums).

also cc @back2dos, @fponticelli

@nadako nadako added discussion platform-javascript Everything related to JS / JavaScript labels Apr 12, 2016
@nadako nadako added this to the 3.4 milestone Apr 12, 2016
@fponticelli
Copy link
Contributor

The benchmarks seem to confirm my gut feelings. Objects are generally
faster than mixed arrays. Another side effect might be that the debugging
in the browser console might be slightly easier to read.

PS I really need to find better characters for the table generation that
play nicely with supposedly monospace fonts.

On Tue, Apr 12, 2016 at 9:30 AM, Dan Korostelev [email protected]
wrote:

Enums are one of the greatest features in Haxe, but its implementation
often performs slower than other solutions, so I think we should maximize
its performance.

I decided to start with JS target and did some benchmarking for the
different possible implementations of enum:
https://github.com/nadako/haxe-js-enum-benchmark

I ran the benchmark on Node 5.10.1 (V8 4.6.85.31) which I think is pretty
indicative, since V8 nowadays is the most used js runtime (chrome, node,
electron, etc). It's safe to assume that other popular engines are not very
different from it.

Unless I did the benchmark wrong (please look at the (generated) source),
it shows that using simple anon objects for enums is the fastest option for
both creation and matching, so we might consider using those instead of
arrays for enums.

I think this is related to how V8 optimizes objects by using hidden
classes, and how it fails to optimize arrays which elements are of
different type (like our current enums).

also cc @back2dos https://github.com/back2dos, @fponticelli
https://github.com/fponticelli


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#5109

@nadako
Copy link
Member Author

nadako commented Apr 12, 2016

Yeah, it's not only better for debugging, but for general interop too.

In fact, I was working on something similar for the C#/Java targets: https://gist.github.com/nadako/5dcc9ba86343ed3795d3

nadako added a commit to nadako/haxe that referenced this issue Apr 12, 2016
@nadako
Copy link
Member Author

nadako commented Apr 12, 2016

I hacked in a simple anon-object based implementation (no reflection/toString for now) and measured like this: https://gist.github.com/nadako/793960bff7c496786174589499fd3d8d

Creation is a lot faster, matching is on par, it seems.

@back2dos
Copy link
Member

You may want to measure it with multiple constructors, with different numbers of arguments.

Another plus is that this representation also maps much better to TypeScript (at least if we have the constructor name?): microsoft/TypeScript#1003

You're probably going to need inheritance anyway, to know the enum. Unless of course you wish to store it in a field instead. Unless the latter is critically faster, I think the former is preferable, since it's maps to native semantics a bit better.

But I agree that this the way forward ;)

@Simn
Copy link
Member

Simn commented Apr 12, 2016

I don't see much of a point in using inheritance for this. Sure it looks nicer if you imagine the code being in Haxe, but what sense does it make in Javascript? Having an extra field that "points" to all the reflection details (which are the same for all instances of a given constructor anyway) looks like the better approach to me.

@nadako
Copy link
Member Author

nadako commented Apr 12, 2016

I agree with Simon on the inheritance, in the original benchmark I actually used inheritance (see object enum constuction/matching) and it also turned out to be slower (I assume that's because of extra ctor/super calls). So simple anon object with an __enum__ (or $enum) field looks better.

As for the constructor name - I don't see the point in putting enum string tag in every instance and I think it hurts performance without real gain. "normal" haxe code works with index integer tags,and string ones are only needed for Type.enumConstructor (and toString which can use the former), so IMO it should be retrieved via e.__enum__.__constructs__[e.index].

@back2dos
Copy link
Member

Ok, after giving it some though, let me argue that we should use names instead of indices: using the string instead of an index yields no performance degradation on JS: http://try.haxe.org/#BB98D.

It does however make things more

  1. readable: particularly the JS output for switch over enum will become at least somewhat reminiscent of the original code.
  2. reliable: code will not rely on the order of constructors, which from a Haxe perspective does not actually matter, so it should not matter at runtime either.

And as mentioned before, it will give Haxe enums a straight forward representation in type script, which is a nice plus.

For compatibility reasons a name->index mapping can be used to determine the index (if needed).

As for inheritance:

Given that we can put the name in the EnumValue it's not so important. Still, if we do use inheritance, we get some stuff for free:

  1. We (and JavaScript devs) can just use instanceof directly.
  2. We can call getIndex() and getEnum() on the thing directly.
  3. No "garbage" in the object itself.

Only reflection is invoked through the inheritance chain, which tends to be slow anyway.

For other access, you can see the difference is negligible here: http://try.haxe.org/#3241e

This is not particularly important, but I don't see a reason to not get it right.

The most important question though: How will we represent arguments? Consider this:

enum Example {
   Foo(s:String, d:Double);
   Bar(s:Something, d:Different);
}

What is the representation of Foo("foo", 3.14) and what is that of Bar(new Something(), new Different())? Are there fields called s and d with a mixed types, that risk making optimization harder for the JS runtime? Or will there be different fields, most of which are not set most of the time? My worry in this case is that the inferred classes may wind up being rather huge (assuming that all constructors get to share the same class?).

The difference needs to be measured, but we should also take readability and robustness of each approach into account. For example, if you rename a constructor argument, it should not break anything. So maybe Foo1, Foo2, Bar1, Bar2 would be good candidates. If it's not slower than the alternatives, I think it's probably the best fit. Assuming that the EnumValue is an instanceof the Enum, then users can still add native properties that map argument names to the correct field depending on name. Actually, in es5 mode the compiler might even emit that code for enums with @:expose, but let's not get ahead of ourselves here ^^

@Simn
Copy link
Member

Simn commented Apr 13, 2016

You can already use name-matching by adding @:matchDebug to the function. I added it while rewriting the pattern matcher because debugging-by-index was quite cumbersome. It will spam the stdout but should be good enough to do some real performance comparisons

Your try haxe does indicate a difference for me:

12:06:46:353   Test.hx:7: 0.21799993515014648s
12:06:46:542   Test.hx:14: 0.18799996376037598s

Also I don't think this is a valid benchmark for switches. An int-switch is not necessarily a sequential equality check, it can be implemented more efficiently (though I don't know what V8 does exactly).

@back2dos
Copy link
Member

Right you are. So here's a benchmark that sort of does the real thing: http://try.haxe.org/#932A8

The output (ints, strings, good old enums) on chrome is sad:

13:10:40:496   Test.hx:9: 0.12400007247924805s
13:10:40:497   Test.hx:19: true
13:10:40:669   Test.hx:21: 0.17199993133544922s
13:10:40:870   Test.hx:34: 0.19999980926513672s

Conceptually, there is absolutely no reason why comparing ints should be faster than comparing strings that stem from equal literals, as the references should point to the same thing (somewhere in the constant pool), which should lead to a decent fast path. If I run the same benchmark on Firefox, the output coincides with my claim, for whatever that's worth:

13:09:29:118   Test.hx:9: 0.12899994850158691s
13:09:29:119   Test.hx:19: true
13:09:29:195   Test.hx:21: 0.07599997520446777s
13:09:29:293   Test.hx:34: 0.09800004959106445s

Yep, that's right, on Firefox the optimized version with ints actually is slower than enums ... oO

I mostly use nodejs, so the V8 is what I care about most. Still, I think it's wiser to make this "right" and wait for the V8 to catch up. The alternative would be to change the implementation again once the numbers look more like they do on Firefox (TBH there's no sane reason why strings should be faster than ints, but I hope you know what I'm getting at), which seems a little silly, considering how many things can break with such a change.

With such discrepancies between JS runtimes, maybe it's simply not a good time to optimize performance here. And maybe better interoperability with TypeScript and JavaScript alone are not worth the hassle either, although believe the contrary and I'm sorry for hijacking a thread motivated by performance to discuss a completely different aspect, but changes to the representation affect both of them alike. To me this kind of seems like a big deal, and making the decisions based on some numbers that may look exactly the opposite way a few months from now, seems rather short-sighted.

Before letting any numbers govern optimization decisions, we must first decide what cases we want to optimize for and then optimize for them. Do we want small enums like Option to be fast? Do we want big enums like Expr to be fast? One cannot measure some aspect in isolation and assume the same behavior for significantly more complex data. Right now, I feel like we're kind of measuring spherical chicken in a vacuum, so I wouldn't read too much into the numbers we get anyway. By the way, memory usage should be measured too.

I for one would be very interested to see how this affects something real, such as haxeparser.

@kevinresol
Copy link
Contributor

This sounds more friendly to document-based databases too.

@stroncium
Copy link
Contributor

I've done some benchmarks on this too a couple of years ago, though can't find them. I started with objects using different keys and found out there was not much to tingle with further. However the inheritance is pretty interesting topic.

Pointer to enum is quite fast, however it increases memory usage and construction time a bit.
Fastest construction was achieved with constructor of index and inlined setup of arguments, next performant version was with a function calling constructor with index and then setting up arguments.
However there was an interesting case when you have parents per enum version. It is a bit slower than mentioned versions, however it allowed to further reduce memory usage due to index being stored in prototype and I was able to construct it in a way prototype lookup didn't reduce usage performance somehow(by comparing not index, but prototype with prototype list I guess).

Some things may have changed in VM since then, but I'd be happy if you consider this last option, may be quite interesting.

It was going like this:

enum E{
    A;
    B(b:Int);
    C(c1:String, c2:String);
}

function test(v){
    switch(v){
        case A: trace('A');
        case B(b): trace('B', b);
        case C(c1,c2): trace('C', c1, c2);
    }
}

var a = A;
var b = B(1);
var c = C('z', 'x');

test(a);
test(b);
test(c);

trace(name(a));
trace(name(b));
trace(name(c));

translates to something like

var E = {};

var E_A = function(){
};
E_A.prototype.idx = 0;
E_A.prototype.name = "A";
E_A.prototype.enum = E;

var E_A_inst = new E_A();

var E_B = function(b){
    this.b = b;
}
E_B.prototype.idx = 1;
E_B.prototype.name = "B";
E_B.prototype.enum = E;

var E_C = function(c1, c2){
    this.c1 = c1;
    this.c2 = c2;
}
E_C.prototype.idx = 2;
E_C.prototype.name = "C";
E_C.prototype.enum = E;

function test(v){
    switch(v.__proto__){
        case E_A.prototype:
            console.log("A");
            break;
        case E_B.prototype:
            var b = v.b;
            console.log("B", b);
            break;
        case E_C.prototype:
            var c1 = v.c1;
            var c2 = v.c2;
            console.log("C", c1, c2);
            break;
        default: console.log('fail', v);
    }
}

function name(v){
    return v.name;
}

var a = E_A_inst;
var b = new E_B(1);
var c = new E_C('z', 'x');

test(a);
test(b);
test(c);

console.log(name(a));
console.log(name(b));
console.log(name(c));

The only problem I see here is the ability to get argument by index, but it since we'd want to keep argument naming in objects if possible, we'd probably want to go about it somewhat like this:

E_A.prototype.args = [];
E_B.prototype.args = ['b'];
E_C.prototype.args = ['c1', 'c2'];


function argument_by_index(val, idx){
    return val[val.args[idx]]
}

Not tat quite a lot of access methods can actually be inlined.

Note that something may be moved to some additional object in prototype(however it's faster to keep everything in prototypes) to "hide" from object, but I don't believe it's needed, because you can't get to it by unless using various magics.

Also it's possible to compress all this in one factory function to generate all enums by declarations. It breaks debugger naming, however we don't care about it that much anyhow.

@stroncium
Copy link
Contributor

Oh, And if we do

E = function(){}
E_A.prototype = new E();
E_B.prototype = new E();
E_B.prototype = new E();

in corresponding places, we get new E_B(1) instanceof E_B and new E_B(1) instanceof E both true, which isn't quite OOP clean, but quite nice for native js interoperation.

@vantreeseba
Copy link

I did something similar for just js

package libnoise;
//Dirty hack to expose enum in js                                                  
#if js                                                                             

enum QualityMode {                                                                 
  LOW;                                                                             
  MEDIUM;                                                                          
  HIGH;                                                                            
}                                                                                  

@:expose("libnoise.QualityMode")                                                   
class E_QualityMode {                                                              
    public static var LOW   : QualityMode = QualityMode.LOW;                       
    public static var MEDIUM: QualityMode = QualityMode.MEDIUM;                    
    public static var HIGH  : QualityMode = QualityMode.HIGH;                      
}                                                                                  

#else                                                                              

enum QualityMode {                                                                 
  LOW;                                                                             
  MEDIUM;                                                                          
  HIGH;                                                                            
}                                                                                  

#end                                           

@Simn Simn modified the milestones: 3.4, 4.0 Jan 9, 2017
@Pauan
Copy link

Pauan commented Jul 26, 2017

I have run some fairly thorough micro-benchmarks on enums, and here are the results:

create: classes (inline)  135018131  100.00000%
create: objects (inline)  133091013  98.57270%
create: tag (inline)      131139837  97.12758%
create: tag               129727822  96.08178%
create: classes           127032524  94.08553%
create: objects           125234733  92.75401%
create: array (inline)    120009041  88.88365%
create: array             112223883  83.11764%
metadata: array    152397197  100.00000%
metadata: classes  140868885  92.43535%
metadata: objects  140493647  92.18913%
metadata: tag      139357332  91.44350%
access: classes (method)         101521448  100.00000%
access: array (switch)           100435904  98.93072%
access: classes (switch)          99881350  98.38448%
access: classes (if)              99111576  97.62624%
access: array (if)                98438736  96.96349%
access: tag (switch)              98144580  96.67374%
access: tag (if)                  96078489  94.63861%
access: tag (vtable)              94289691  92.87662%
access: objects (method inline)   94056627  92.64705%
access: objects (method)          93122980  91.72740%
access: array (vtable)            92610179  91.22228%
access: classes (instanceof)      71630275  70.55679%
toString: tag      50361915  100.00000%
toString: objects  48203005  95.71321%
toString: classes  46077982  91.49371%
toString: array    40705556  80.82607%

All benchmarks were done on Node version 8.1.3, v8 version 5.8.283.41

The number is the amount of times it called the function within 10 seconds, therefore bigger number = faster

The results are sorted so that the fastest is at the top, and the slowest is at the bottom

These are the four things I tested:

  1. create is how fast it creates the enum (inline means that it creates the enum directly and doesn't use a function call)

  2. metadata is how fast it can access the metadata object (which would contain __constructs__, etc.)

    It's not really important, but I included it for completeness

  3. access is how fast it can access the data within the enum

  4. toString is how fast it can convert the enum to a string (e.g. when printing it to the console)

And these are the various implementations I tested:

  • tag is using anonymous objects with an __enum__ (metadata) and __tag__ (tag index) fields

  • array is using an array where the first element is the metadata and the second element is the tag index

  • classes is using ES5 classes which inherit from the metadata, and they have a __tag__ field for the tag index

  • objects is using anonymous objects which contain methods (no tag index)

    This implementation isn't relevant for Haxe, but I included it for completeness

Here is my summary of the data:

  • instanceof is significantly slower than the alternatives, so I recommend not using it

  • When creating the enum, all of the implementations perform about the same

    array performs slightly slower because it has to mutate the array to add the toString property

  • When accessing data in the enum, all of the implementations perform exactly the same

  • array is the smallest in terms of code size (437 characters), tag is second smallest (500 characters), and classes is the biggest (516 characters)

  • array seems the best suited for a generic toString implementation which works on all enums

  • classes can inherit from the metadata (which also means it can inherit a single toString implementation for all the constructors)

  • Regardless of what implementation is chosen, it might be a good idea to inline the enum constructors (right now they are not inlined)

@Pauan
Copy link

Pauan commented Jul 26, 2017

As for strings vs integers, I vote for integer tags:

  • TypeScript handles integer tags in ADTs just fine (in addition, enum in TypeScript defaults to integers)

  • I don't buy the argument that debugging will be worse: enums have a toString implementation which pretty-prints it with the proper constructor name, and source maps solve the issue of mapping back to Haxe source code

  • Having the runtime be independent of the order of the enums is nice, but I think a better solution is to have a way to expose enums to the target

  • Using integers is guaranteed to be fast, strings being fast depends on the specific details of the VM

    V8 in particular has many optimizations specifically for 32-bit integers, including unboxing (I am unaware of any optimizations it has for strings)

  • Using integers should be smaller memory usage than strings (4 bytes for an integer, way more bytes for a string)

  • Using integers is smaller for code size, which can sometimes matter just as much as runtime performance

@Pauan
Copy link

Pauan commented Jul 31, 2017

After reading @stroncium's post, I decided to redo the benchmarks.

I added in an isPrototypeOf, __proto__, Object.getPrototypeOf, and constructor implementations:

access:
  classes (method)                      7943.27444/ms  8187.05835/ms  100.00000%
  tag (switch)                          7810.64984/ms  8163.48159/ms   99.71202%
  array (switch)                        7589.51454/ms  8095.04807/ms   98.87615%
  classes (if)                          8021.00487/ms  8072.37040/ms   98.59916%
  tag (if)                              6771.99844/ms  7971.90672/ms   97.37205%
  classes (constructor)                 7930.93921/ms  7944.17104/ms   97.03328%
  classes (switch)                      7855.73262/ms  7909.32891/ms   96.60770%
  array (if)                            7751.46225/ms  7885.21516/ms   96.31317%
  tag (vtable)                          7407.74607/ms  7579.91186/ms   92.58407%
  array (vtable)                        7569.68553/ms  7538.23995/ms   92.07507%
  objects (method inline)               7068.74915/ms  7405.29629/ms   90.45125%
  objects (method)                      7121.39811/ms  7285.93048/ms   88.99326%
  classes (Object.getPrototypeOf)       6937.64265/ms  7217.50308/ms   88.15746%
  classes (Object.getPrototypeOf fast)  7161.91341/ms  7201.83692/ms   87.96611%
  classes (__proto__ fast)              7056.13285/ms  7065.27804/ms   86.29813%
  classes (__proto__)                   6621.06470/ms  6948.29342/ms   84.86923%
  classes (instanceof)                  6160.97749/ms  6035.10166/ms   73.71514%
  classes (isPrototypeOf fast)          5753.90350/ms  5933.15429/ms   72.46992%
  classes (isPrototypeOf)               5729.77106/ms  5878.15554/ms   71.79814%
create:
  tag (inline)      10449.82705/ms  10597.23887/ms  100.00000%
  classes (inline)   8157.48298/ms  10566.02444/ms   99.70545%
  objects (inline)  10204.85592/ms  10521.65256/ms   99.28674%
  classes           10024.07218/ms  10177.87290/ms   96.04269%
  tag                9101.76581/ms  10058.64493/ms   94.91760%
  objects            9856.21783/ms   9946.45550/ms   93.85893%
  array (inline)     7838.42483/ms   9414.10544/ms   88.83546%
  array              8926.07098/ms   8914.03975/ms   84.11663%
metadata:
  array    11387.72089/ms  11727.42418/ms  100.00000%
  classes  10221.01969/ms  10827.65594/ms   92.32766%
  tag      10308.39227/ms  10749.57983/ms   91.66190%
  objects  10745.15226/ms  10736.76278/ms   91.55261%
toString:
  tag      4529.31633/ms  4546.27216/ms  100.00000%
  objects  3839.68956/ms  4502.92767/ms   99.04659%
  classes  4165.03658/ms  4159.09801/ms   91.48370%
  array    3672.77482/ms  3766.57644/ms   82.84978%

The first column is the speed when running the benchmark for 100 milliseconds, the second column is the speed when running the benchmark for 10,000 milliseconds. Bigger = faster.

The 100 millisecond column is supposed to represent a program which runs quickly (e.g. a shell script), whereas the 10,000 millisecond column is supposed to represent a program which runs for a long time (e.g. a server or web app).

Checking constructor is quite fast, but __proto__ and Object.getPrototypeOf are quite slow.

That's a shame, I was hoping they would be fast, because the JS VMs should be able to just grab the [[Prototype]] field directly, without needing to do any lookups.

The microbenchmarks are nice and all, but I want to take a look at IR Hydra to see what the actual compiled code looks like. That should help settle the debate.

@Simn Simn modified the milestones: Release 4.0, Design Apr 17, 2018
@Simn
Copy link
Member

Simn commented May 5, 2018

eb1d00f

@Simn Simn closed this as completed May 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion platform-javascript Everything related to JS / JavaScript
Projects
None yet
Development

No branches or pull requests

8 participants