Set of Benchmarks for myc.
Running on Ryzen 3800x and ubuntu 24.04. Add myc-llvm, myc-qbe, myc-c, mycc to Path.
git clone https://github.com/kostya/myc-benchmarks.git
cd myc-benchmarks
LangArena is a benchmark suite of 50 tests and 9,000 lines of non-trivial C code (json, base64, multithreaded matmul, neural net, compression, maze A*, bf interpreter, and others) with heavy macros like uthash. The ./c directory contains 29 .c files (230KB total), which we'll use for comparison. This is not just a random benchmark of micro-optimizations or synthetic loops - it's a set of tasks close to production use. Each of the 50 tests validates its output using checksums. A compiler can't "cheat" by deleting or skipping work.
cd LangArena/c; make MODE=prod target/deps/prod/yyjson.o target/deps/prod/libbase64.o; cd -
Compile LangArena C benchmark, from single IR file (to remove parsing overhead). Both Myc LangArena/langarena-single-myc/langarena.myc - 2.07Mb and LL LangArena/langarena-single-ll/langarena.ll - 2.9Mb, represent the same program and both generated from the same 29 C files (in ./LangArena/c folder) in O0 mode without any processing (by scripts LangArena/gen_myc.rb and LangArena/gen_ll.rb, files in the repo was generated for linux64, for macOS need to regenerate with this scripts). This benchmark shows raw optimization and code generation skills for both engines.
Ubuntu clang version 20.1.2 (0ubuntu1~24.04.3) vs myc 0.10.0-dev-4e16e50 LLVM 20.1.2
cd LangArena; ruby run_single_ir_benchmark.rb; cd -
| Compiler | Compile time | Runtime |
|---|---|---|
| clang(-O3, ll) | 1603ms | 52.1s |
| clang(-O2, ll) | 1572ms | 52.6s |
| clang(-O1, ll) | 1286ms | 55.4s |
| clang(-O0, ll) | 193ms | 139.0s |
| myc-llvm(default) | 889ms | 59.6s |
| myc-llvm(final) | 1880ms | 52.8s |
| myc-qbe(default) | 262ms | 68.3s |
| myc-c(default, clang) | 1589ms | 60.6s |
| myc-c(final, clang) | 3115ms | 53.3s |
-
myc-qbe(default) - compiles only 35% slower than Clang -O0 (262ms vs 193ms), and 6x faster than Clang -O3 (262ms vs 1603ms). Despite near-instant compilation, runtime is only 31% slower than Clang -O3 (68.3s vs 52.1s). myc-qbe delivers a tradeoff that even Go would envy.
-
myc-llvm(default) - 1.8x faster compilation than Clang -O3 (889ms vs 1603ms), with runtime only 14% slower (59.6s vs 52.1s). A solid balance: fast compiles, decent performance.
-
myc-llvm(final) - runtime is nearly identical to Clang -O3 (52.8s vs 52.1s, just 1.3% slower), but compilation takes 17% longer (1880ms vs 1603ms). LLVM spends ~280ms more optimizing Myc-generated IR - needs investigation.
-
myc-c - adds overhead by generating C code and compiling it through the full Clang stack, so it can't compete with the other two backends on compile time. It's primarily a fallback backend for portability. Runtime is respectable: 60.6s default, 53.3s final - within 2% of Clang -O3 when using clang as the final compiler.
This scatter plot shows the tradeoff between compile time and runtime. The ideal is the bottom-left corner: fast compiles, fast runtime. This is the Pareto frontier - you can't improve one without sacrificing the other. Myc-qbe(default) and myc-llvm(default) occupy a spot that even Go would respect. Measured on pure IR files - no C parsing overhead, just optimization and code generation for both MycIR and LLVM-LL.
This benchmark compiles the C files in LangArena/c dir (excluding two precompiled dependencies: yyjson.o and libbase64.o). All files are compiled at once, not one by one (clang LangArena/c/src/*.c), to remove the overhead of multiple command invocations.
mycc 0.10.0-dev-4e16e50 c99-subset compiler (backend: unknown) (https://github.com/kostya/myc), gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, Ubuntu clang version 20.1.2 (0ubuntu1~24.04.3), cproc master
cd LangArena; ruby run_c_benchmark.rb; cd -
| Compiler | Compile time | Runtime |
|---|---|---|
| mycc(default, llvm) | 2713ms | 60.4s |
| mycc(final, llvm) | 3669ms | 53.6s |
| mycc(default, qbe) | 2442ms | 68.5s |
| mycc(default, c, clang) | 4471ms | 59.5s |
| mycc(final, c, clang) | 4920ms | 52.0s |
| clang(-O3, c) | 3042ms | 51.8s |
| clang(-O2, c) | 3001ms | 52.0s |
| clang(-O1, c) | 2739ms | 54.2s |
| clang(-O0, c) | 1605ms | 141.1s |
| gcc(-O3, c) | 3470ms | 52.4s |
| gcc(-O2, c) | 2986ms | 54.7s |
| gcc(-O1, c) | 2143ms | 58.0s |
| gcc(-O0, c) | 1304ms | 134.9s |
| cproc | 726ms | 72.8s |
-
Parsing is the real bottleneck for clang. Compare Clang -O0 on C (1605ms) vs Clang -O0 on LLVM IR (193ms). 88% of Clang's compile time is spent on parsing C and generating IR, not on optimization or codegen. Shocking O_o.
-
cproc proves fast C parsing is possible. cproc compiles the same C code in 726ms - 2.2x faster than Clang -O0. Runtime is only 40% slower than Clang -O3 (72.8s vs 51.8s). As we saw earlier from myc-qbe results, QBE (cproc's backend) spends ~262ms on codegen, meaning cproc's parsing itself is roughly ~464ms.
-
mycc doesn't shine in this benchmark. mycc relies on libclang for parsing, and has very rough frontend implementation (3-week POC), not a production C frontend.
-
GCC has better O0->O3 scaling.
This benchmark compiles the amalgamated C file LangArena/c-amalgamation/single.c. It can now be directly compared with Benchmark 1 (single IR), since the unexpected per-file overhead (from Benchmark 2) has been eliminated.
cd LangArena; ruby run_c_amalgamation_benchmark.rb; cd -
| Compiler | Compile time | Runtime |
|---|---|---|
| mycc(default, llvm) | 1285ms | 59.3s |
| mycc(final, llvm) | 2306ms | 52.6s |
| mycc(default, qbe) | 716ms | 67.8s |
| mycc(default, c, clang) | 2114ms | 56.6s |
| mycc(final, c, clang) | 2647ms | 51.9s |
| clang(-O3, c) | 1803ms | 51.9s |
| clang(-O2, c) | 1767ms | 51.6s |
| clang(-O1, c) | 1463ms | 54.9s |
| clang(-O0, c) | 355ms | 141.7s |
| gcc(-O3, c) | 2712ms | 52.3s |
| gcc(-O2, c) | 2154ms | 54.4s |
| gcc(-O1, c) | 1219ms | 57.9s |
| gcc(-O0, c) | 543ms | 134.4s |
| cproc | 166ms | 72.6s |
-
mycc consistently adds ~450ms over single IR - this is the cost of libclang overhead.
-
cproc is the absolute leader in compilation speed.
-
Clang adds roughly ~200ms for all optimization levels compared to single IR.
This benchmark compares different IRs (LLVM, QBE, C, Myc) against each other. The same Brainfuck program generates each IR, then compiles and runs. This shows whether Myc adds overhead compared to using each backend directly.
cd brainfuck-compiler && ruby run.rb
| IR | Compiler | IR size, Kb | Compile time | Run time |
|---|---|---|---|---|
| llvm-ll | clang(-O3) | 1529 | 1298ms | 624ms |
| myc | myc-llvm(default) | 486 | 213ms | 714ms |
| myc | myc-llvm(final) | 486 | 1073ms | 616ms |
| qbe-ssa | qbe + clang(as+linker) | 345 | 122ms + 140ms | 780ms |
| myc | myc-qbe(default) | 486 | 479ms | 772ms |
| c | clang(-O3) | 128 | 1166ms | 615ms |
| myc | myc-c(default) | 486 | 1336ms | 623ms |
| myc | myc-c(final) | 486 | 1544ms | 641ms |
This benchmark compare myc backens on simple micro benchmarks:
cd examples && ruby run.rb
| Benchmark | Backend | Compile | Run |
|---|---|---|---|
| mandel.myc | myc-llvm | 320ms | 721ms |
| myc-qbe | 499ms | 773ms | |
| myc-c | 1320ms | 614ms | |
| myc-llvm-final | 1111ms | 615ms | |
| myc-qbe-final | 483ms | 775ms | |
| myc-c-final | 1577ms | 637ms | |
| bf.myc | myc-llvm | 91ms | 1350ms |
| myc-qbe | 105ms | 2677ms | |
| myc-c | 117ms | 1601ms | |
| myc-llvm-final | 102ms | 1292ms | |
| myc-qbe-final | 104ms | 2697ms | |
| myc-c-final | 115ms | 1429ms | |
| loop.myc | myc-llvm | 85ms | 411ms |
| myc-qbe | 100ms | 369ms | |
| myc-c | 101ms | 234ms | |
| myc-llvm-final | 89ms | 327ms | |
| myc-qbe-final | 98ms | 368ms | |
| myc-c-final | 100ms | 247ms | |
| loop.c | myc-llvm | 231ms | 420ms |
| myc-qbe | 137ms | 368ms | |
| myc-c | 146ms | 224ms | |
| myc-llvm-final | 122ms | 403ms | |
| myc-qbe-final | 132ms | 421ms | |
| myc-c-final | 127ms | 423ms | |
| sieve.c | myc-llvm | 141ms | 229ms |
| myc-qbe | 144ms | 285ms | |
| myc-c | 148ms | 237ms | |
| myc-llvm-final | 134ms | 234ms | |
| myc-qbe-final | 132ms | 235ms | |
| myc-c-final | 132ms | 233ms |