Libmpeff first benchmarks

[codrutiftode]

I implemented the one-shot benchmarks in libmpeff as well, using clang-14 for ease for comparison to libseff:

• countdown
• fibonacci recursive
• product early
• iterator
• generator
• parsing dollars
• resume nontail
• handler sieve

Out of these, I could only get handler sieve working by making the prime() operation in the handler definition SCOPED_ONCE, and then resuming with mpe_resume_tail in the handler clause. It takes ~60 seconds when given 60,000 as input, which is substantially longer than any of the other languages. @teofr @dhil maybe you know why? It might have to do with the flags, but if I used TAIL_NOOP instead of SCOPED_ONCE, the nested handlers would fail to perform the same effect from child to parent.

[teofr]

I'm not sure what it could be, I'll try to take a look later, have you tried any other opkind? (MPE_OP_TAIL, MPE_OP_ONCE, MPE_OP_MULTI)

libhandler has some kind of bug with scoped handlers (koka-lang/libhandler#13), so maybe the same is happening here?

[codrutiftode]

I tried the different opkinds, but it did not help much. In terms of time, I got t(MPE_OP_SCOPED_ONCE) < t(MPE_OP_ONCE) < t(MPE_OP_MULTI), which makes sense.

But I thought since I am always resuming in tail position, that I could maybe use MPE_OP_TAIL (for the inner handlers) and MPE_OP_TAIL_NOOP (for the top-level handler). However, this caused the program to break, seemingly entering an infinite loop where a handler in a stack of nested handlers performed an effect, but then it handled it itself instead of yielding to its parent handler. I'm thinking maybe because OP_TAIL performs some optimization which does not work with the algorithm of handler sieve? Let me know if you can make sense of it.

[jasigal]

To add to this, the reason why handler sieve is so slow is that it's quadratic w.r.t. the input. This is because we're essentially traversing a linearly increasing handler stack.

Using hyperfine, I get

Summary
  ./main 1000 ran
    4.11 ± 0.97 times faster than ./main 2000
    9.32 ± 2.00 times faster than ./main 3000
   18.19 ± 3.66 times faster than ./main 4000
   28.10 ± 4.82 times faster than ./main 5000
   38.81 ± 6.27 times faster than ./main 6000
   51.92 ± 8.25 times faster than ./main 7000
   71.31 ± 13.28 times faster than ./main 8000
   90.76 ± 16.54 times faster than ./main 9000
  111.68 ± 20.80 times faster than ./main 10000

which is clearly quadratic. Of coures the same is true of other systems, such as libseff. Using hyperfine, I get

Summary
  ./main 1000 ran
    2.98 ± 0.79 times faster than ./main 2000
    5.96 ± 1.45 times faster than ./main 3000
   10.39 ± 2.58 times faster than ./main 4000
   16.33 ± 3.82 times faster than ./main 5000
   24.38 ± 5.52 times faster than ./main 6000
   34.42 ± 7.80 times faster than ./main 7000
   48.98 ± 11.19 times faster than ./main 8000
   60.54 ± 13.23 times faster than ./main 9000
   87.45 ± 22.18 times faster than ./main 10000

which also quadratic.

However, libmpeff (with SCOPED_ONCE) seems to be about 3 times slower than libseff, so it's unsuprising that it takes 60 seconds for 60,000.

[jasigal]

Given that libmpeff seems to not be updated any longer, I just added a note about the weirdness and will just merge as is.