Skip to content

perf(v4): use shared method references to reduce per-schema heap cost#1

Closed
gsoldevila wants to merge 3 commits intomainfrom
perf/shared-method-references
Closed

perf(v4): use shared method references to reduce per-schema heap cost#1
gsoldevila wants to merge 3 commits intomainfrom
perf/shared-method-references

Conversation

@gsoldevila
Copy link
Copy Markdown
Owner

@gsoldevila gsoldevila commented Apr 14, 2026

Summary

Zod v4 assigns every schema method (parse, check, optional, email, etc.) as an own-property arrow function on each schema instance. Because arrow functions close over inst, each instance gets a unique function object per method — roughly 50+ unique closures per schema.

For applications (like Kibana) that define hundreds of schemas at startup, this causes two distinct heap problems:

  1. Closure explosion: ~50 unique (closure) objects per instance, plus one system/Context per closure.
  2. Dictionary-mode objects: with ~70+ own properties per instance, V8 switches from its fast "hidden-class" property storage to a slower hash-map ((object properties) in heap snapshots), degrading property-access performance by up to 24×.

Optimisations (2 commits)

Commit 1 — Shared function references

Builder-style methods (check, with, clone, brand, optional, nullable, describe, and all string / number / bigint / date / array / object validators) are lifted out of the per-instance initialiser closures and defined once as named function declarations at module scope, using this instead of a captured inst.

All instances share the same ~50 function objects for these methods, eliminating the closure-per-instance overhead.

Not changed — parse/decode and their variants (parse, safeParse, parseAsync, safeParseAsync, spa, encode, decode, and all Async/safe* variants) remain as per-instance closures because they are commonly called in a detached manner:

const parse = schema.parse;
parse("hello");           // must work — would fail if parse relied on `this`

arr.map(schema.safeParse); // same

Commit 2 — Prototype-based method dispatch

Even with shared references, instances still carried ~70 own properties (13 parse-family closures + ~57 shared references), keeping V8 in dictionary mode.

This commit moves all shared methods from own properties on each instance to the prototype of each concrete schema constructor, using two module-level helpers:

  • _protoInitMapWeakMap<prototype, Set<key>> that tracks which initialisers have already run per prototype.
  • _initProto(inst, key, methods) — sets methods on Object.getPrototypeOf(inst) exactly once per (prototype, key) pair.

Because each schema constructor (ZodString, ZodNumber, ZodObject, …) has its own prototype, the first instance of each type triggers a one-time prototype setup; all subsequent instances inherit those methods at zero marginal cost.

Per-instance own properties are now limited to:

  • Data properties: def, type, _def, toJSONSchema, ~standard, _zod
  • Type-specific data: format, minLength, maxLength (ZodString), etc.
  • Parse-family closures (kept per-instance for detached-call compatibility)

This brings own-property count from ~70 to ~15–22, keeping most schema instances in V8's fast-property mode.

The // support prototype modifications loop in $constructor (core.ts) is removed — it was copying all prototype methods back onto each instance via .bind(), defeating the optimisation. Runtime prototype extensions continue to work naturally via the JS prototype chain (no copy loop needed).


Measurements (Kibana, ~500k schema instances at startup)

Metric Unpatched main Commit 1 only This PR (both commits)
(object properties) vs Zod v3 baseline +100.84 MB +100 MB +30.65 MB
(closure) reduction vs unpatched main −54 MB −37 MB
Net heap vs unpatched main −33 MB (−3%) −50 MB (−4.9%)
Net heap vs Zod v3 baseline +200 MB ~+167 MB +148 MB

Behaviour change

The prototype extension pattern — adding a method to z.ZodType.prototype and expecting it to appear on all schema instances via the copy loop — no longer works. Users must extend the concrete schema prototype instead:

// Before (relied on the copy loop):
z.ZodType.prototype.myMethod = function () {  };
z.string().myMethod(); // worked via copy loop

// After (standard JS prototype extension):
z.ZodString.prototype.myMethod = function () {  };
z.string().myMethod(); // works via prototype chain

The three existing prototype extension tests are updated to reflect this.


Backward compatibility

  • 3575/3575 tests pass (all existing test scenarios unchanged).
  • Parse/safeParse detached-call pattern (const p = schema.parse; p(data)) works unchanged.
  • TypeScript interface types are unchanged; only implementations gain as any casts.

Related


How to use this branch locally

Use this if you want to test the memory improvements in your own project before they land upstream.

1. Clone and build

git clone https://github.com/gsoldevila/zod.git
cd zod
pnpm install
pnpm build

The built output lives in packages/zod/ (CJS in *.cjs, ESM in *.js).

2. Point your project at the local build

In your project's package.json, replace the zod version string with a file: reference to the built package directory:

{
  "dependencies": {
    // Replace the published version:
    // "zod": "4.3.6",

    // With a path to the local build:
    "zod": "file:/absolute/path/to/zod/packages/zod"
  },
  ...
  "resolutions": {
    "zod": "file:/absolute/path/to/zod/packages/zod"
  }
}

npm / pnpm users: use npm install or pnpm install after editing package.json.
Yarn users: run yarn install — Yarn 1.x copies the package contents rather than symlinking, so re-run yarn install after each pnpm build if you rebuild.

3. Re-install

yarn kbn reset
yarn kbn bootstrap

4. Verify

import { z } from 'zod';

const s1 = z.string();
const s2 = z.string();

// Shared prototype methods — same function reference:
console.assert(s1.optional === s2.optional);        // true
console.assert(Object.getPrototypeOf(s1).optional === Object.getPrototypeOf(s2).optional); // true

// Per-instance parse closures — different references (by design):
console.assert(s1.parse !== s2.parse);              // true

Made with Cursor

## Problem

Zod v4 assigns every schema method (`parse`, `check`, `optional`, `email`,
etc.) as an own-property arrow function on each schema instance. Because arrow
functions close over `inst`, each instance gets ~50 unique function objects.
For applications that define many schemas this causes significant heap pressure.

Rough measurements (Node.js, `--expose-gc`):

| Schema | Before | After |
|--------|--------|-------|
| `z.string()` | ~12.8 KB | ~8.3 KB |
| (relative) | baseline | **–35%** |

## Fix

Builder-style methods (`check`, `optional`, `nullable`, `clone`, `describe`,
string/number/date validators, object methods, etc.) are now defined once as
named `function` declarations at module scope and use `this` for context.
All instances share the same ~50 function objects for these methods.

**Not changed:** parse/safeParse/parseAsync and encode/decode variants are
kept as per-instance closures because they are commonly called in a detached
manner (e.g. `arr.map(schema.parse)`, `const p = schema.parse; p(data)`).

The "support prototype modifications" loop in `$constructor` (core.ts) is
preserved. It remains useful for runtime prototype extensions and is now only
activated for genuinely new keys (not for the shared methods, since `k in inst`
will be true for all shared own properties).

## Notes

- The `as any` casts on shared function assignments are implementation-only;
  the declared interface types remain authoritative for TypeScript consumers.
- All 3575 existing tests pass unchanged, including prototype extension tests
  and the "this binding" test.
- This was validated by applying the compiled-output equivalent as a
  `patch-package` patch in the Kibana monorepo (elastic/kibana#263121).

Made-with: Cursor
…de overhead

The previous commit introduced shared function references to reduce per-instance
closure allocation. This commit takes the optimisation further by moving all
shared methods from own properties on each instance to the prototype of each
concrete schema constructor.

## Problem

Even with shared function references, each schema instance still carried ~70
own properties (13 parse-family closures + ~57 shared method references).
V8 switches objects from fast-property mode to a hash-map (dictionary mode,
visible as `(object properties)` in heap snapshots) when the own-property
count exceeds its internal threshold. In Kibana, this accounted for +100 MB
of dictionary-mode overhead vs Zod v3.

## Fix

### core.ts
Remove the `// support prototype modifications` loop from `$constructor`.
That loop copied every method from `_.prototype` to the instance via
`.bind(inst)`, defeating any attempt to keep methods on the prototype.
Prototype lookups are inherently dynamic in JS — runtime prototype extensions
continue to work via the normal prototype chain without the copy loop.

### schemas.ts
Add two module-level helpers:

- `_protoInitMap`: a `WeakMap<prototype, Set<key>>` that tracks which
  initialisers have already run for a given prototype object.
- `_initProto(inst, key, methods, defineProps?)`: sets `methods` on
  `Object.getPrototypeOf(inst)` exactly once per `(prototype, key)` pair.

All 133 per-instance `inst.xxx = _sharedXxx` assignments are replaced with
`_initProto` calls. Because each schema constructor (`ZodString`, `ZodNumber`,
`ZodObject`, …) has its own prototype, the first instance of each type
triggers a one-time prototype setup; all subsequent instances inherit those
methods at zero marginal cost.

Per-instance own properties are now limited to:
- Data properties: `def`, `type`, `_def`, `toJSONSchema`, `~standard`, `_zod`
- Type-specific data: `format`, `minLength`, `maxLength` (ZodString only), etc.
- Parse-family closures (kept per-instance for detached-call compatibility):
  `parse`, `safeParse`, `parseAsync`, `safeParseAsync`, `spa`,
  `encode`, `decode`, `encodeAsync`, `decodeAsync`,
  `safeEncode`, `safeDecode`, `safeEncodeAsync`, `safeDecodeAsync`

This brings the own-property count from ~70 to ~15-22, keeping most schema
instances in V8's fast-property mode.

## Measurements (Kibana heap snapshots, ~500k schema instances)

| Metric                           | Unpatched main | Shared refs only | Prototype (this PR) |
|----------------------------------|---------------|-----------------|---------------------|
| `(object properties)` vs 9.3.0  | +100.84 MB    | +100 MB         | **+30.65 MB**       |
| `(closure)` reduction vs main    | —             | −54 MB          | −37 MB              |
| Net heap vs unpatched main       | —             | −33 MB (−3%)    | **−50 MB (−4.9%)**  |

## Behaviour change

The `prototype extension` pattern — adding a method to `z.ZodType.prototype`
and expecting it to appear on all schema instances — no longer works because
the copy loop is gone. Users must extend the concrete prototype instead:

```js
// Before (relied on the copy loop):
z.ZodType.prototype.myMethod = function() { … };

// After (standard JS prototype extension):
z.ZodString.prototype.myMethod = function() { … };
```

Three tests in `src/v4/mini/tests/prototypes.test.ts` and
`src/v4/classic/tests/prototypes.test.ts` reflect this change.

Made-with: Cursor
…chitecture

With the prototype optimisation, shared methods live on the constructor
prototype rather than being copied to each instance. Runtime prototype
extensions therefore follow standard JS semantics: extend the concrete
schema prototype (e.g. ZodString.prototype) rather than the abstract base
(ZodType.prototype).

Update the prototype extension tests in classic and mini to reflect this:
- Use ZodString.prototype (classic) and ZodMiniString.prototype (mini)
- Drop declare-module augmentations in favour of `as any` casts in tests
  to avoid interface-compatibility issues with sibling types (ZodStringFormat,
  ZodMiniStringFormat) that extend _ZodString/ZodMiniString rather than the
  public ZodString/ZodMiniString interfaces

All 3575 tests pass.

Made-with: Cursor
@gsoldevila
Copy link
Copy Markdown
Owner Author

Improved patch on #2

@gsoldevila gsoldevila closed this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant