Skip to content

perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes)#2

Closed
gsoldevila wants to merge 1 commit intomainfrom
zod-memory-optimization-v2
Closed

perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes)#2
gsoldevila wants to merge 1 commit intomainfrom
zod-memory-optimization-v2

Conversation

@gsoldevila
Copy link
Copy Markdown
Owner

Summary

Fixes schema instances being pushed past V8's fast-property threshold, causing ~24x slower property access (see colinhacks#5760).

Own-property count before/after this PR:

Schema Before After
z.string() ~91 ~18
z.string().min(1).max(10).email() ~91 ~18
z.number() ~91 ~17
z.object({}) ~91 ~18

Problem

Two sources contributed to the ~91 own properties per schema instance:

  1. The copy-loop in $constructor copies every method from _.prototype to each new instance, adding ~60 own properties.
  2. Classic ZodType.init assigns ~30 builder methods as per-instance own properties.

V8 switches from fast-property (inline-cache) to dictionary mode at ~20–30 own properties, degrading all property access by up to 24×.

Solution: shadow-proto

Introduce a hidden internal prototype layer between the user-visible _.prototype and the parent prototype. Library methods are placed on internalProto; the copy-loop continues to target the empty _.prototype.

inst
 └── _.prototype           ← user-visible (empty by default; copy-loop targets this)
      └── internalProto    ← library space (_initProto writes here)
           └── Parent / Object.prototype

core.ts

  • Export $internalProto (a unique Symbol) so schemas.ts can locate each constructor's internal prototype.
  • Inside $constructor, create internalProto = Object.create(Parent.prototype), wire _.prototype → internalProto via Object.setPrototypeOf, and expose it as _[$internalProto].
  • The copy-loop is unchangedObject.keys(_.prototype) now returns [] by default, making it a no-op unless a user explicitly adds methods to _.prototype.

classic/schemas.ts

  • Add _protoInitMap (WeakMap) and _initProto helper that targets inst._zod.constr[$internalProto].
  • All 133 per-instance builder method assignments replaced with _initProto calls.
  • Parse-family closures (parse, safeParse, parseAsync, safeParseAsync, encode/decode variants) remain per-instance — they capture inst and must work when detached.

mini/schemas.ts

  • Same _initProto infrastructure added.
  • The 6 builder methods in ZodMiniType (check, with, clone, brand, register, apply) moved to the internal prototype.

Key advantage: no breaking changes

The original prototype-extension contract is fully preserved:

// Still works exactly as before
z.ZodType.prototype.myHelper = function() { return "custom"; };
z.string().myHelper(); // ✓  "custom"

Because _.prototype is now empty by default, the copy-loop activates only when users add methods there — propagating them to new instances exactly as the original code did.

Both prototypes.test.ts suites (classic and mini) pass without modification.

New test: fast-properties.test.ts

Asserts for common schemas that:

  • Own-property count is < 25
  • Builder methods (optional, nullable, etc.) are NOT own properties
  • Parse methods ARE own properties (detached usage must work: const { parse } = schema; parse("x"))

Comparison with alternative approach

This PR sits alongside perf/shared-method-references (PR #1 in this fork), which takes a simpler approach: it removes the copy-loop and places methods directly on _.prototype. That version is slightly simpler but is a breaking change for code that augments z.ZodType.prototype. This PR achieves the same performance gain with zero breaking changes.

PR #1 (perf/shared-method-references) This PR (zod-memory-optimization-v2)
Breaking change Yes (ZodType.prototype augmentation must target concrete type) None
z.ZodType.prototype.x = fn; z.string().x() ❌ breaks ✅ works
Own-property reduction ~91 → ~18 ~91 → ~18
Prototype hops for built-in method 0 extra 1 extra (negligible)
Mini optimized No Yes
Regression tests No Yes (fast-properties.test.ts)

Test results

Test Files  325 passed (325)
Tests       3589 passed (3589)
Type Errors no errors

Related: colinhacks#5760

Made with Cursor

…aking changes)

## Problem

Zod v4 schema instances accumulate too many own properties, pushing them past
V8's fast-property threshold (~20–30) and into dictionary mode. Dictionary mode
degrades property access by up to 24× (see issue colinhacks#5760).

Two sources contribute:
1. The `// support prototype modifications` copy-loop in `$constructor` copies
   every method from `_.prototype` to each new instance — adding ~60 own props.
2. Classic `ZodType` init assigns ~30 builder methods as per-instance own props.

## Solution: shadow-proto design

Introduce a hidden *internal* prototype layer between the user-visible `_.prototype`
and the parent prototype chain. Library methods are placed there; the copy-loop
continues to target only the user-visible `_.prototype`.

```
inst
 └── _.prototype           ← user space (empty by default; copy-loop targets this)
      └── internalProto    ← library space (_initProto writes here)
           └── Parent / Object.prototype
```

### core.ts

- Export `$internalProto` (a unique symbol) to identify each constructor's
  internal prototype from `schemas.ts`.
- In `$constructor`, create `internalProto` for each constructor, wire
  `_.prototype → internalProto`, and expose it as `_[$internalProto]`.
- The copy-loop is **unchanged** — but since `_.prototype` is now empty by
  default, it is a no-op unless a user explicitly adds methods there.

### classic/schemas.ts

- Add `_protoInitMap` (WeakMap) and `_initProto` helper.
- `_initProto` targets `inst._zod.constr[$internalProto]` (the internal proto),
  not `Object.getPrototypeOf(inst)` (the user-visible proto).
- All 133 per-instance builder method assignments replaced with `_initProto`
  calls; parse-family closures stay per-instance for detached-usage safety.

### mini/schemas.ts

- Same `_initProto` infrastructure added.
- The 6 builder methods in `ZodMiniType` (`check`, `with`, `clone`, `brand`,
  `register`, `apply`) moved to the internal prototype.

## Key advantage over a simpler approach

Unlike removing the copy-loop entirely (which breaks prototype augmentation),
this design preserves the original contract:

```js
// Still works — copy-loop propagates user extensions from _.prototype to instances
z.ZodType.prototype.myHelper = function() { ... };
z.string().myHelper(); // ✓
```

The copy-loop is only a no-op for *library* methods (they live on internalProto,
which `Object.keys(_.prototype)` never enumerates).

## Results

- `z.string()` own-property count: ~91 → ~18 (well below V8's threshold)
- No breaking changes; all 3,589 tests pass including both `prototypes.test.ts`
  suites without modification
- `z.ZodType.prototype` augmentation works exactly as before

## New test

`fast-properties.test.ts` asserts:
- Own-property count < 25 for common schemas
- Builder methods are NOT own properties
- Parse methods ARE own properties (detached usage must work)

Closes: colinhacks#5760
Made-with: Cursor
@gsoldevila
Copy link
Copy Markdown
Owner Author

This PR was a showcase of the changes.
Official PR has been submitted upstream colinhacks#5870

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant