Skip to content

Refresh benchmark chart with finalized L0+L1+L2 stack#12

Merged
intech merged 1 commit intomainfrom
chore/refresh-benchmark-chart
Apr 19, 2026
Merged

Refresh benchmark chart with finalized L0+L1+L2 stack#12
intech merged 1 commit intomainfrom
chore/refresh-benchmark-chart

Conversation

@intech
Copy link
Copy Markdown

@intech intech commented Apr 19, 2026

Summary

Regenerates `benchmarks/chart.svg` and `benchmarks/README.md` table after merge of the full optimization stack (#6, #8, #10, #11). Previous chart measured pre-L0 prototype state where `toBinaryFast` referenced H2 JSON-roundtrip; now it measures the real L0+L1+L2 pipeline on `main`.

Results (Node 25.8, log-scale chart)

Fixture Baseline toBinaryFast (L0+L1+L2) pbjs Delta vs baseline
OTel 100 spans 525 2,501 3,110 +376% (0.80x pbjs)
OTel Metrics 50 series 891 4,773 - +435%
OTel Logs 100 records 880 3,772 - +329%
K8sPodList 20 pods 712 3,510 - +393%
Stress depth=8 width=200 2,568 14,378 - +460%
SimpleMessage 1.39M 1.81M - +30%
RpcRequest 111K 374K - +236%
GraphQLResponse 145K 558K - +284%

Highlights

  • OTel deep-nested workload: achieves 80% of pbjs throughput without generated code (reflective path alone)
  • Memory profile: ~82% reduction on nested workloads (L0 contiguous buffer eliminates fork/join allocations)
  • No regression on any fixture (verified against previous baseline)

Test plan

  • `npm run bench:report` runs cleanly on main
  • Chart.svg XML-valid, opens in browser
  • README table renders between markers
  • All 2,800+ existing tests + 54 correctness tests pass

🤖 Generated with Claude Code

Regenerated after merge of #6 (benchmark matrix), #8 (L0 contiguous
writer), #10 (L1+L2 schema plans + specialized writers), #11
(correctness tests).

Key results (Node 25.8, log-scale chart):
- OTel 100 spans:    525 -> 2,501 ops/s (+376%), 0.80x pbjs (3,110)
- OTel Metrics 50:   891 -> 4,773 ops/s (+435%)
- OTel Logs 100:     880 -> 3,772 ops/s (+329%)
- K8sPodList 20:     712 -> 3,510 ops/s (+393%)
- Stress d=8 w=200:  2,568 -> 14,378 ops/s (+460%)
- SimpleMessage:     1.39M -> 1.81M ops/s (+30%)

Memory allocations per encode reduced proportionally via L0 contiguous
buffer + L1 schema-plan opcode interpreter + L2 specialized field writers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@intech intech self-assigned this Apr 19, 2026
@github-actions
Copy link
Copy Markdown

Benchmark: 20 regression(s)

Thresholds: throughput regression >5%, memory regression >10%. Current run on linux/x64, Node v22.22.2, captured 2026-04-19T22:27:15.250Z.
Baseline captured 2026-04-19T22:25:26.442Z on linux/x64, Node v22.22.2.

Summary: 20 regressed, 0 improved, 0 new, 0 unchanged.

Fixture Baseline ops/s PR ops/s Δ ops Baseline B/op PR B/op Δ mem Status
SimpleMessage :: toBinary (pre-built, 19 B) 938,596 759,585 -19.1% REGRESSION
ExportTraceRequest (100 spans) :: toBinary (pre-built, 32926 B) 1,575 1,189 -24.5% REGRESSION
ExportMetricsRequest (50 series) :: toBinary (pre-built, 17696 B) 2,756 2,125 -22.9% REGRESSION
ExportLogsRequest (100 records) :: toBinary (pre-built, 21319 B) 2,760 2,086 -24.4% REGRESSION
K8sPodList (20 pods) :: toBinary (pre-built, 28900 B) 3,045 2,244 -26.3% REGRESSION
GraphQLRequest :: toBinary (pre-built, 624 B) 221,540 172,310 -22.2% REGRESSION
GraphQLResponse :: toBinary (pre-built, 1366 B) 287,305 230,118 -19.9% REGRESSION
RpcRequest :: toBinary (pre-built, 501 B) 370,710 287,232 -22.5% REGRESSION
RpcResponse :: toBinary (pre-built, 602 B) 532,200 434,849 -18.3% REGRESSION
StressMessage (depth=8, width=200) :: toBinary (pre-built, 12868 B) 10,305 7,656 -25.7% REGRESSION
SimpleMessage :: fromBinary (19 B) 1,368,997 1,092,932 -20.2% REGRESSION
ExportTraceRequest (100 spans) :: fromBinary (32926 B) 800.6 663.3 -17.2% REGRESSION
ExportMetricsRequest (50 series) :: fromBinary (17696 B) 1,526 1,271 -16.7% REGRESSION
ExportLogsRequest (100 records) :: fromBinary (21319 B) 1,403 1,203 -14.2% REGRESSION
K8sPodList (20 pods) :: fromBinary (28900 B) 1,887 1,405 -25.6% REGRESSION
GraphQLRequest :: fromBinary (624 B) 415,652 301,686 -27.4% REGRESSION
GraphQLResponse :: fromBinary (1366 B) 355,329 292,554 -17.7% REGRESSION
RpcRequest :: fromBinary (501 B) 350,950 269,288 -23.3% REGRESSION
RpcResponse :: fromBinary (602 B) 477,458 382,903 -19.8% REGRESSION
StressMessage (depth=8, width=200) :: fromBinary (12868 B) 5,431 4,051 -25.4% REGRESSION

Produced by benchmarks/scripts/compare-results.ts. Artifacts: bench-results-<pr> (current), bench-baseline-main (baseline).

@intech intech merged commit 0c0e383 into main Apr 19, 2026
27 of 29 checks passed
@intech intech deleted the chore/refresh-benchmark-chart branch April 19, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant