perf(v4): eliminate dictionary-mode overhead via shadow-proto (no breaking changes)

[gsoldevila]

Summary

Fixes #5760

Own-property count before/after this PR:

Schema	Before	After
z.string()	~91	~18
z.string().min(1).max(10).email()	~91	~18
z.number()	~91	~17
z.object({})	~91	~18

Problem

Two sources contributed to the ~91 own properties per schema instance:

1. The copy-loop in $constructor copies every method from _.prototype to each new instance, adding ~60 own properties.
2. Classic ZodType.init assigns ~30 builder methods as per-instance own properties.

V8 switches from fast-property (inline-cache) to dictionary mode at ~20–30 own properties, degrading all property access by up to 24×.

Solution: shadow-proto

Introduce a hidden internal prototype layer between the user-visible _.prototype and the parent prototype. Library methods are placed on internalProto; the copy-loop continues to target the empty _.prototype.

inst
 └── _.prototype           ← user-visible (empty by default; copy-loop targets this)
      └── internalProto    ← library space (_initProto writes here)
           └── Parent / Object.prototype

core.ts

• Export $internalProto (a unique Symbol) so schemas.ts can locate each constructor's internal prototype.
• Inside $constructor, create internalProto = Object.create(Parent.prototype), wire _.prototype → internalProto via Object.setPrototypeOf, and expose it as _[$internalProto].
• The copy-loop is unchanged — Object.keys(_.prototype) now returns [] by default, making it a no-op unless a user explicitly adds methods to _.prototype.

classic/schemas.ts

• Add _protoInitMap (WeakMap) and _initProto helper that targets inst._zod.constr[$internalProto].
• All 133 per-instance builder method assignments replaced with _initProto calls.
• Parse-family closures (parse, safeParse, parseAsync, safeParseAsync, encode/decode variants) remain per-instance — they capture inst and must work when detached.

mini/schemas.ts

• Same _initProto infrastructure added.
• The 6 builder methods in ZodMiniType (check, with, clone, brand, register, apply) moved to the internal prototype.

Key advantage: no breaking changes

The original prototype-extension contract is fully preserved:

// Still works exactly as before
z.ZodType.prototype.myHelper = function() { return "custom"; };
z.string().myHelper(); // ✓  "custom"

Because _.prototype is now empty by default, the copy-loop activates only when users add methods there — propagating them to new instances exactly as the original code did.

Both prototypes.test.ts suites (classic and mini) pass without modification.

New test: fast-properties.test.ts

Asserts for common schemas that:

• Own-property count is < 25
• Builder methods (optional, nullable, etc.) are NOT own properties
• Parse methods ARE own properties (detached usage must work: const { parse } = schema; parse("x"))

All tests passed ✅

[pullfrog]

TL;DR — Reduces per-schema own-property count from ~91 to ~17–18 by introducing a hidden internal prototype layer ("shadow-proto") between instances and the user-visible _.prototype. This keeps V8 in fast-property / inline-cache mode instead of falling into dictionary mode, which was causing up to ~24× slower property access. No breaking changes — the existing prototype-extension contract is fully preserved.

Key changes

• Add $internalProto layer in $constructor — A hidden prototype is inserted between _.prototype (user space) and the parent prototype; library methods are installed there so Object.keys(_.prototype) stays empty by default and the copy-loop becomes a no-op.
• Replace ~133 per-instance closures with shared _initProto calls in classic schemas.ts — Builder methods (check, optional, nullable, describe, string/number/bigint/date/array/object/enum/file validators, wrapper unwrap, etc.) are now this-based functions installed once per concrete type on the internal prototype, eliminating ~50 function object allocations per schema.
• Apply same _initProto pattern to mini schemas.ts — The 6 ZodMiniType builder methods (check, with, clone, brand, register, apply) are moved to the internal prototype.
• Add fast-properties.test.ts regression suite — Asserts own-property counts stay below 25, builder methods are not own properties, and parse-family methods are own properties (so detached usage like const { parse } = schema keeps working).

_{Summary ｜ 4 files ｜ 1 commit ｜ base: main ← zod-memory-optimization-v2}

Shadow-proto architecture in core.ts

Before: $constructor created a flat prototype chain (inst → _.prototype → Parent.prototype); all library methods were copied to every instance as own properties, pushing V8 into dictionary mode at ~91 own props.
After: A new internalProto layer sits between _.prototype and the parent. Library methods live on internalProto; _.prototype stays empty by default, so the copy-loop is a no-op for stock schemas.

The prototype chain is now: inst → _.prototype (user space, empty) → internalProto (library methods) → Parent.prototype. The $internalProto symbol is exported so schemas.ts can locate the internal proto for each constructor.

User prototype extensions continue to work — adding methods to z.ZodType.prototype causes the existing copy-loop to activate, propagating them to new instances exactly as before.

core.ts

Shared this-based methods in classic schemas.ts

Before: Every schema instance received ~30+ builder methods as per-instance arrow closures capturing inst, plus ~60 methods copied from the prototype — totaling ~91 own properties.
After: Builder methods are defined once as named this-based functions and installed on the internal prototype via _initProto. Own-property count drops to ~17–18 per instance.

The _initProto helper uses a WeakMap<object, Set<string>> to track which method groups have already been installed on each internal prototype, ensuring one-time setup per concrete type per key. Parse-family methods (parse, safeParse, parseAsync, etc.) intentionally remain per-instance closures — they capture inst for detached usage patterns like arr.map(schema.parse).

Why keep parse methods as own properties?

Parse methods are commonly used in a detached manner (e.g. const { parse } = schema; parse(value) or arr.map(schema.parse)). Prototype-based this binding would break in these cases. The ~8 parse/encode/decode closures are the only per-instance functions that remain.

schemas.ts · fast-properties.test.ts

Mini schemas get the same treatment

Before: ZodMiniType assigned 6 builder methods (check, with, clone, brand, register, apply) as per-instance closures.
After: These are shared this-based functions installed once via _initProto.

The mini _initProto is a slightly simpler copy of the classic version (no defineProps parameter needed).

mini/schemas.ts

  ^{｜ View workflow run ｜ via Pullfrog ｜ Using Claude Opus ｜ 𝕏}

[colinhacks]

Thanks, Gerard. Unfortunately, despite the efforts here to avoid breaking changes, there are still breaking changes for any users who pull out references to schema methods. Today this works:

const opt = schema.optional;
opt(); // ZodOptional<typeof schema>

After this PR it silently returns a corrupt schema that fails the next time anything tries to parse with it. Parse-family methods are kept as per-instance closures so const { parse } = schema still works; nothing else does.

Zod is now widely used enough that there's a Murphy's law effect with breaking changes like this. Somebody, probably hundreds of somebodies, rely on the ability to pull out methods like this and use them as standalone functions. In fact, I specifically like the fact that there are no gotchas around this-binding with Zod's current approach.

The memory overhead is unfortunate. I'm receptive to anything that could be done to reduce it. I'd even be open to introducing some minor breakage if it meant a significant runtime parsing performance boost. But at this point I'm not convinced that this is a good trade-off.

Closing for now because I'm going through a backlog here. Perhaps DM me if you'd like to float other approaches or continue this discussion.

[gsoldevila]

Thanks for the thorough review, @colinhacks. I totally missed the extracted references scenario.

For us in Kibana, upgrading to 4.x has a huge memory impact (> 100 Mb), so we're exploring non-breaking alternatives.

There is a backward-compatible variant worth considering: auto-bind getters on the prototype. Instead of placing a plain shared function on internalProto, each builder method could be a getter that returns _sharedFn.bind(this):

Object.defineProperty(internalProto, 'optional', {
  get() { return _sharedOptional.bind(this); },
  configurable: true,
});

I'm doing some tests locally, but I believe it should preserve the behaviour when doing:

const opt = schema.optional;
opt(); // ✓ works — `this` is permanently bound to schema at access time
schema.optional(); // ✓ works as before

Happy to submit a new PR if you think the trade-off is acceptable.

[colinhacks]

Thanks, Gerard. Seems like you posted this before seeing #5897, which is substantially identical to what you suggested. (Good idea!)

[gsoldevila]

Hi! yes, I saw your PR after posting.
Both are similar, but your approach is even better as you only pay the allocation overhead once.

FYI we cherry-picked your commit and applied it to 4.3.6.
We did some memory tests and there are substantial memory savings, so I confirm #5760 is correctly addressed.

Thanks a lot for the fix!