Actualize benchmarks README and fix chart-delta layout#19
Merged
Conversation
Per design spec (analysis/p1-t6-l3-design-spec.md). Observes message
shapes across first 10 encodes per schema; graduates frequent shapes
to specialized plan variants that skip the generic `isFieldSet`
presence gate for known-present fields and known-absent slots. 4-
variant cap with seal-on-breach prevents cache explosion.
Two execution modes:
- Mode A (CSP-safe, default): variant = pre-computed `VariantStep[]`
list of known-present slots; executor is a statically-imported
interpreter that delegates to the L1+L2 `estimate*/write*`
helpers. Safe under strict CSP.
- Mode B (CSP-unsafe, opt-in): per-variant `new Function()` executor
with unrolled call sites for per-variant IC isolation. Enabled
via `globalThis[Symbol.for('@bufbuild/protobuf.adaptive-codegen')] = true`.
Spec adaptations for current main:
- The L1+L2 reference implementation on main is the direct estimate/
write function set in `to-binary-fast.ts` rather than the opcode-
based `schema-plan.ts` assumed by the spec. The variant plan shape
therefore drops the opcode trim/filter step and instead carries a
compact `VariantStep[]` — same monomorphization effect, fewer
moving parts for this code base.
- `buildVariantExecutor` is replaced by the two closures (Mode A
static / Mode B codegen) in `compileVariantPlan` with identical
semantics.
Gates (5-run median on pinned CPU):
- Byte-parity: preserved across 11 new L3 tests + 16 pre-existing
toBinaryFast tests + correctness-matrix.
- SimpleMessage multi-shape: +55.5% (spec target: >= +10%)
- SimpleMessage single-shape: +40.8% (spec target: regression <= 3%)
- Span multi-shape: +19.0% (spec target: >= +10%)
- Span single-shape: +12.2% (spec target: regression <= 3%)
- Memory overhead: bounded by D3 + D7 (shared side tables).
Opt-in: `toBinaryFast(schema, msg, { adaptive: true })` or
`PROTOBUF_ES_L3=1`. Default behaviour unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…chains - schema-plan-adaptive.ts: remove suppressions for a rule biome isn't enforcing in this project config - schema-plan-adaptive.ts + to-binary-fast.ts: collapse defensive globalThis process env lookups into optional chain form - biome.json: ignore gen/gen-protobufjs/.tmp from root scope so turbo lint doesn't catch pbjs-generated files and scratch dirs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
README: - Drop the pre-L0 "Current results" section (stale hand-curated numbers that contradicted the auto-generated table a few paragraphs above) - Add a "Reading the results" section that explains the L0+L1+L2 stack and the ~0.80x-protobufjs floor in plain language - Replace "Future work: CI integration" (already shipped) with the next-real-work items (ts-proto comparison, multi-shape in CI matrix, fromBinaryFast) chart-delta.svg (generator in report-helpers.ts): - Top margin 80 -> 120 px so axis tick labels do not collide with legend, and legend does not collide with subtitle - Legend now anchored 100 px left of the chart body (was flush with chart-left) — matches the fixture-label column visually - Axis tick labels pushed 5 px higher so they clear the top of the chart grid - Per-row bar positions shifted 3 px further from the center split so numeric labels on the two half-bars never overlap each other Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Benchmark: no regressionsThresholds: throughput regression Summary:
Produced by |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmarks/README.md
The "Current results" section carried pre-L0 hand-curated numbers that contradicted the auto-generated table a few paragraphs above. Replaced with a "Reading the results" section that explains the L0+L1+L2 stack and the measured ~0.80x-protobufjs floor. "Future work" loses the "CI integration" item (shipped in #9 / #15) and gains realistic next steps (ts-proto comparison, multi-shape in CI matrix, decoder fast path).
chart-delta.svg layout fix
Addresses reader complaints about overlapping text and tight top margin:
Chart regenerated via `npm run bench:report` on main; committed alongside the helper change so the committed SVG matches the updated generator.