Benchmark confirmed 8% diff to Python 3.8!

[vprelovac]

Hi!

I am the maintainer of python-speed benchmark (https://github.com/vprelovac/python-speed) and I could confirm the increased speed in Pyston. It is about 8% overall (not exactly 20% that is claimed) mostly due to much better stack handling (which shows about 40% speed increase). Overall fantastic job and looking forward to next release!

Python 3
python-speed v1.2 using python v3.8.5
string/mem: 2144.74 ms
pi calc/math: 2939.3 ms
regex: 2950.72 ms
fibonnaci/stack: 1845.43 ms

total: 9880.2 ms (lower is better)

Pyston 3
python-speed v1.2 using python v3.8.2
string/mem: 2152.05 ms
pi calc/math: 2971.2 ms
regex: 2707.27 ms
fibonnaci/stack: 1197.36 ms

total: 9027.89 ms (lower is better)

[undingen]

Hi thanks for trying out pyston and sharing your benchmark and results :)
I took a quick look at your benchmark, first thing I want to note is that we don't try to optimize pyston for small synthetic benchmarks but are mainly interested in the performance of bigger web framework workloads. Nevertheless we hope to do also well on those but I'm not surprised that we are not much faster because e.g. the regex benchmark spends likely most of time in the regex implementation and we use the same implementation as cpython (written in C) so we can't get a 20% speedup.

One thing I noticed is that a lot of the benchmarks functions are only called once and contain large loops inside.
The released versions of pyston (v2.1) will only enter JITed code at the start of the function. Which means the whole loop will be executed by the interpreter. We will soon release pyston v2.2 which contains many improvements and one of it is that it can enter JITed code at the loop backedge so is able to switch from interpreting a hot loop to continue executing it via machine code even if the function is only called once.