perf(v4): lazy-bind builder methods to shared internal prototype#5897
perf(v4): lazy-bind builder methods to shared internal prototype#5897colinhacks merged 5 commits intomainfrom
Conversation
Builder methods (.optional, .nullable, .array, .pick, etc.) were allocated as fresh per-instance closures during construction. On a plain z.string() that's ~50 closures up front, dominated by JSFunction + Context bytes, even when the user never calls them. Move builders onto a hidden prototype layer ($internalProto) inserted between _.prototype (user-extension space) and the parent. On first access from any instance, defineLazy allocates fn.bind(this) and caches it as an own enumerable property; subsequent accesses skip the getter. Detached usage (const m = schema.optional; m()) keeps working because m is a bound function with this already resolved. Parse-family methods (parse, safeParse, encode, etc.) intentionally stay as eager per-instance closures — they're the hot path AND the most-detached methods (arr.map(schema.parse)), so paying the allocation up front is worth the monomorphic call site. Adds a regression test guarding detached method behavior.
|
Reviewed PR #5897 — approved with no actionable issues. The lazy-bind prototype mechanism is correct: Task list (6/6 completed)
|
|
TL;DR — Builder methods on Zod v4 schemas ( Key changes
Summary | 2 files | 5 commits | base: Lazy-bind prototype getters without layer indirection
The implementation is minimal: a single Parse-family methods (
Detached-method regression tests
A broad sweep test iterates over a table of method names and probe arguments, detaching each one and verifying that the call neither throws nor produces a corrupt schema.
|
There was a problem hiding this comment.
Reviewed — no issues found. The prototype chain wiring, lazy-bind caching via defineLazy, and the _installLazyMethods memoization are all correct. Detached usage is preserved by binding on first access and caching as an own property. The new test file covers the critical regression vector from #5870.
Two minor observations (non-blocking):
_sharedUnwrapand_sharedLazyUnwrap(schemas.ts:275-281) are defined but unused, suppressed with@ts-ignore. Fine if wrapper types are planned for a follow-up; worth removing otherwise.- ~72 per-instance builder closures remain on other schema types (ZodBigInt, ZodDate, ZodSet, ZodMap, ZodFile, ZodEnum, 11 wrapper
unwrapmethods, ZodString format methods). Presumably intentional scope cut for this PR — the shared_sharedUnwrap/_sharedLazyUnwrapplaceholders suggest follow-up work.
Task list (6/6 completed)
- Read the full diff and understand the changes
- Examine core.ts changes (defineLazy, $internalProto, $constructor)
- Examine schemas.ts changes (_installLazyMethods, builder method refactor)
- Examine detached-methods test file
- Read surrounding context in source files for correctness verification
- Submit review
Claude Opus | 𝕏
There was a problem hiding this comment.
Reviewed — no issues found.
Task list (6/6 completed)
- Read the full diff and understand the changes
- Examine core.ts changes (defineLazy, $internalProto, $constructor)
- Examine schemas.ts changes (_installLazyMethods, builder method refactor)
- Examine detached-methods test file
- Read surrounding context in source files for correctness verification
- Submit review
Claude Opus | 𝕏
Two simplifications on top of the previous commit, no behavior change:
1. Remove all changes from `core/core.ts`. The `$internalProto` symbol
and `defineLazy` helper are gone. Classic now creates the
library-owned prototype layer lazily on first install per
constructor, via `Object.setPrototypeOf(constr.prototype, layer)`
cached in a `WeakMap`. Net diff to `core/core.ts` is zero.
2. Drop all ~50 `_sharedXxx` named functions in favor of inline
method-shorthand bodies inside each `_installLazyMethods({...})`
call site. The `_shared` prefix is gone; the only underscore-
prefixed helper left in classic is `_installLazyMethods` itself.
Bundle size impact (esbuild --bundle --minify --format=esm, gzipped):
Classic minimal (z.string().parse): 15,677 -> 15,889 (+212 B, +1.4%)
Classic rich (object schema): 19,100 -> 19,355 (+255 B, +1.3%)
Mini minimal (z.string()): 2,707 -> 2,705 (-2 B, identical)
Mini rich (object schema): 5,367 -> 5,365 (-2 B, identical)
Mini bundles are now byte-for-byte identical to main; classic overhead
is roughly halved versus the previous design (~430 B -> ~212 B gz).
All 3,743 tests still pass; heap/construction numbers unchanged.
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Removed
$internalProto,defineLazy, and the intermediate prototype layer wiring fromcore.ts— the lazy-bind infrastructure is now entirely self-contained inschemas.ts _installLazyMethodsnow creates the prototype layer lazily per constructor via_layerCache+_installedGroupsWeakMaps, inserting betweenconstr.prototypeand its parent on first call- Inlined all ~50
_shared*functions (_sharedCheck,_sharedOptional,_sharedObjPick, etc.) as method definitions in the object literals passed to_installLazyMethods— same behavior, one less indirection layer
Prior review feedback:
-
_sharedUnwrapand_sharedLazyUnwrapunused with@ts-ignore— removed
Claude Opus | 𝕏
Install lazy method getters directly on each concrete schema's `_.prototype` instead of inserting a library-owned layer between `_.prototype` and its parent. The layer was guarding `Object.keys(proto)` from seeing our methods, but the existing copy-loop in `core.$constructor` already uses `Object.keys` (enumerable own only), and our lazy getters are non-enumerable. So the layer wasn't doing real work — just adding prototype-chain depth and one `Object.setPrototypeOf` call per constructor (the only V8-deopt-prone op in the helper). Drops `_layerCache: WeakMap`, drops `Object.setPrototypeOf`, helper shrinks from 45 to 30 lines. Behavior: `'optional' in z.ZodString.prototype` is now `true` (was `false`) and `Object.getOwnPropertyNames(z.ZodString.prototype)` now lists the lazy method names — both arguably more intuitive since the methods conceptually do live on the prototype. Numbers (vs main, post-refactor): classic minimal bundle: +860 B raw / +175 B gz (was +212 gz) classic rich bundle: +1,135 B raw / +221 B gz (was +255 gz) mini bundles: byte-identical to main (unchanged) heap per instance: same -40% as before construction: same -25% as before All 3,743 tests pass. User prototype extensions still work (z.ZodType.prototype.x = ... copies to instances via the existing copy-loop, and explicitly setting an enumerable value on the prototype shadows the lazy getter as before).
There was a problem hiding this comment.
No new issues. Reviewed the following changes:
- Removed the intermediate prototype layer (
_layerCache,Object.create,Object.setPrototypeOf) — lazy-bind getters are now installed directly on each constructor's ownprototypeviaObject.getPrototypeOf(inst) - Eliminated
_layerCacheWeakMap;_installedGroupsalone is sufficient for memoization - Updated comments to reflect the simpler approach
The simplification is sound: user extensions are enumerable own properties on the prototype (copied by $constructor's init loop via Object.keys), while lazy-bind getters are non-enumerable — no interference. Removing the extra prototype hop and Object.setPrototypeOf is a net win for both simplicity and performance.
Claude Opus | 𝕏
The previous `_installLazyMethods` typed `methods` as
`Record<string, (this: any, ...args: any[]) => any>`, which gave
zero type safety on the inline method bodies — `this` was `any` and
arguments were unchecked, so any signature drift between the body
and the declared interface method silently slipped through.
Replace with a generic helper that derives the expected method
shapes from the instance's interface:
type _LazyMethodsOf<T> = Partial<{
[K in keyof T]: T[K] extends (...args: infer A) => infer R
? (this: T, ...args: A) => R
: never;
}>;
function _installLazyMethods<T extends object>(
inst: T,
group: string,
methods: _LazyMethodsOf<T>
): void
Each call site now infers `T` from `inst` (e.g. `ZodType` /
`_ZodString` / `ZodNumber` / `ZodArray` / `ZodObject`), and the
inline method bodies are checked against the interface declarations.
Adding/removing/renaming an arg in a body without updating the
interface (or vice versa) is now a compile error pointing at both
sites.
The only body that needed adjustment is `meta()`, which is
overloaded — the mapped type picks up the last overload, so we
return `any` from the body to satisfy both the no-arg and 1-arg
cases at runtime.
All 3,743 tests pass; same 13 pre-existing tsc errors as `main`.
With the previous commit's strict typing on `_installLazyMethods`,
parameters in the inline method bodies receive their types via
contextual inference from the schema interfaces — explicit `: any`
annotations on every arg were leftover noise from before. Strip them.
Two methods need to keep variadic `any[]` because they intentionally
diverge from the picked overload's static signature at runtime:
- `meta(...args: any[]): any` — runtime branches on `args.length`
to handle both `meta()` and `meta(data)` overloads, but the
picked overload is `meta(data: GlobalMeta) => this`, which would
type-narrow `args.length === 0` to "never true".
- `catchall(catchall)` — needs `catchall as any` on the spread
because `SomeType` isn't structurally compatible with the strict
`$ZodType` shape `clone` expects.
Everything else (~50 method bodies) loses its annotations and reads
as `optional() { ... }`, `min(value, params) { ... }`, etc. with full
contextual type-checking from the interface declarations.
…d#5897) (#266343) ## Summary Adopts the memory optimization implemented by Colin McDonnell in [colinhacks/zod#5897](colinhacks/zod#5897) as a `patch-package` patch, ahead of the next official zod release. Also fixes [colinhacks/zod#5760](colinhacks/zod#5760). ### Background The previous patch (#263121) introduced a **shadow-proto** architecture that shared builder methods on a hidden prototype layer. While it reduced per-schema heap cost by ~80%, it introduced a breaking change: **detached method calls** stopped working. ```ts const opt = schema.optional; // extracts method reference opt(); // ❌ previously threw — `this` was undefined ``` The root cause was that `_initProto` used `Object.assign` to copy methods as plain value properties onto the prototype. Detaching such a property loses the `this` context. ### This PR Replaces `_initProto` with Colin's `_installLazyMethods` approach from [colinhacks/zod#5897](colinhacks/zod#5897): ```js function _installLazyMethods(inst, group, methods) { // ... one-time setup per (proto, group) ... for (const key in methods) { const fn = methods[key]; Object.defineProperty(proto, key, { configurable: true, enumerable: false, get() { const bound = fn.bind(this); // Cache on the instance — subsequent accesses skip the getter Object.defineProperty(this, key, { configurable: true, writable: true, enumerable: true, value: bound, }); return bound; }, set(v) { Object.defineProperty(this, key, { configurable: true, writable: true, enumerable: true, value: v, }); }, }); } } ``` **How it works:** - Builder methods live as non-enumerable getters on each concrete schema's prototype - On first access per instance, the getter allocates `fn.bind(this)` and caches it as an own property — subsequent accesses skip the getter entirely - Detached calls (`const m = schema.optional; m()`) work because `m` is a bound function with `this` already resolved - Parse-family methods (`parse`, `safeParse`, etc.) stay as eager per-instance closures — they're the hot path and most-detached methods **Memory characteristics are unchanged:** builder methods are not own properties until first accessed, so V8 keeps schema instances in fast-property mode. Per Colin's numbers: -40% heap per instance, -25% construction time vs Zod 4.3.6 baseline. ## Why `patch-package` Upstream zod CI has been broken for months and there is no scheduled release. `patch-package` allows us to ship the fix now. Once a new zod version including [colinhacks/zod#5897](colinhacks/zod#5897) is released, this patch can be dropped. ## Changes - `patches/zod+4.3.6.patch` — replaces the previous shadow-proto patch with Colin's `_installLazyMethods` approach (only touches `v4/classic/schemas.cjs` and `v4/classic/schemas.js`) - `src/platform/packages/shared/kbn-zod/v4/detached_methods.test.ts` — behavioural unit tests proving detached calls work and that the lazy-cache correctly returns the same bound function on repeated access ## Test plan - [x] `node scripts/jest src/platform/packages/shared/kbn-zod/v4/detached_methods.test.ts` — 5/5 pass - [ ] Full CI green --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…d#5897) (elastic#266343) ## Summary Adopts the memory optimization implemented by Colin McDonnell in [colinhacks/zod#5897](colinhacks/zod#5897) as a `patch-package` patch, ahead of the next official zod release. Also fixes [colinhacks/zod#5760](colinhacks/zod#5760). ### Background The previous patch (elastic#263121) introduced a **shadow-proto** architecture that shared builder methods on a hidden prototype layer. While it reduced per-schema heap cost by ~80%, it introduced a breaking change: **detached method calls** stopped working. ```ts const opt = schema.optional; // extracts method reference opt(); // ❌ previously threw — `this` was undefined ``` The root cause was that `_initProto` used `Object.assign` to copy methods as plain value properties onto the prototype. Detaching such a property loses the `this` context. ### This PR Replaces `_initProto` with Colin's `_installLazyMethods` approach from [colinhacks/zod#5897](colinhacks/zod#5897): ```js function _installLazyMethods(inst, group, methods) { // ... one-time setup per (proto, group) ... for (const key in methods) { const fn = methods[key]; Object.defineProperty(proto, key, { configurable: true, enumerable: false, get() { const bound = fn.bind(this); // Cache on the instance — subsequent accesses skip the getter Object.defineProperty(this, key, { configurable: true, writable: true, enumerable: true, value: bound, }); return bound; }, set(v) { Object.defineProperty(this, key, { configurable: true, writable: true, enumerable: true, value: v, }); }, }); } } ``` **How it works:** - Builder methods live as non-enumerable getters on each concrete schema's prototype - On first access per instance, the getter allocates `fn.bind(this)` and caches it as an own property — subsequent accesses skip the getter entirely - Detached calls (`const m = schema.optional; m()`) work because `m` is a bound function with `this` already resolved - Parse-family methods (`parse`, `safeParse`, etc.) stay as eager per-instance closures — they're the hot path and most-detached methods **Memory characteristics are unchanged:** builder methods are not own properties until first accessed, so V8 keeps schema instances in fast-property mode. Per Colin's numbers: -40% heap per instance, -25% construction time vs Zod 4.3.6 baseline. ## Why `patch-package` Upstream zod CI has been broken for months and there is no scheduled release. `patch-package` allows us to ship the fix now. Once a new zod version including [colinhacks/zod#5897](colinhacks/zod#5897) is released, this patch can be dropped. ## Changes - `patches/zod+4.3.6.patch` — replaces the previous shadow-proto patch with Colin's `_installLazyMethods` approach (only touches `v4/classic/schemas.cjs` and `v4/classic/schemas.js`) - `src/platform/packages/shared/kbn-zod/v4/detached_methods.test.ts` — behavioural unit tests proving detached calls work and that the lazy-cache correctly returns the same bound function on repeated access ## Test plan - [x] `node scripts/jest src/platform/packages/shared/kbn-zod/v4/detached_methods.test.ts` — 5/5 pass - [ ] Full CI green --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit a9ecabf)
…cks/zod#5897) (#266343) (#266401) # Backport This will backport the following commits from `main` to `9.4`: - [fix(zod): adopt upstream lazy-bind memory optimization (colinhacks/zod#5897) (#266343)](#266343) <!--- Backport version: manual --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) Made with [Cursor](https://cursor.com) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
|
Landed in Zod 4.4 |
…types (#267326) ### Summary Cleanup pass on `@kbn/evals-common` and the `evals` plugin, mirroring what was done for `@kbn/inbox-common` in #265634. No functional changes for end users, but the OAS docs generated from these routes are now correct, and the in-memory cost of the Zod schemas drops via the upstream lazy-bind optimization. ### Changes - **Switch to canonical `buildRouteValidationWithZod` from `@kbn/zod-helpers/v4`** - Deleted the local copy in `kbn-evals-common/impl/schemas/common.ts` and its re-export from `kbn-evals-common/index.ts`. - Updated all 18 evals route files to import the helper from `@kbn/zod-helpers/v4`. - Pruned `@kbn/zod-helpers` and `@kbn/core` from `kbn-evals-common`'s `moon.yml` / `tsconfig.json`; added `@kbn/zod-helpers` to the `evals` plugin's instead. - The canonical helper attaches `_sourceSchema` to the returned validator (added in #263354), which `kbn-router-to-openapispec` unwraps so route `params` / `query` / `body` actually appear in generated OAS docs. The local copy did not, so any OAS doc generation for these routes was silently dropping schema info. - **Regenerated OAS types** (`yarn openapi:generate` in `kbn-evals-common`) - Picks up the `lazySchema(() => …)` wrappers introduced by #264125 and tuned in #266343 (upstream `colinhacks/zod#5897` lazy-bind memory optimization). 19 `.gen.ts` files updated; pattern matches the prior inbox regen exactly. - **README cleanup** — pointed at the new helper location and dropped the obsolete "after regenerating, you may need to fix unused imports" workaround (no longer needed thanks to the OAS-generator `fix_eslint.ts` fix from #265634, which forces the non-editor branch when invoked from agent/IDE terminals). _PR developed with Cursor + Claude Opus 4.7 Super Duper xHigh Thinking++_
…types (elastic#267326) ### Summary Cleanup pass on `@kbn/evals-common` and the `evals` plugin, mirroring what was done for `@kbn/inbox-common` in elastic#265634. No functional changes for end users, but the OAS docs generated from these routes are now correct, and the in-memory cost of the Zod schemas drops via the upstream lazy-bind optimization. ### Changes - **Switch to canonical `buildRouteValidationWithZod` from `@kbn/zod-helpers/v4`** - Deleted the local copy in `kbn-evals-common/impl/schemas/common.ts` and its re-export from `kbn-evals-common/index.ts`. - Updated all 18 evals route files to import the helper from `@kbn/zod-helpers/v4`. - Pruned `@kbn/zod-helpers` and `@kbn/core` from `kbn-evals-common`'s `moon.yml` / `tsconfig.json`; added `@kbn/zod-helpers` to the `evals` plugin's instead. - The canonical helper attaches `_sourceSchema` to the returned validator (added in elastic#263354), which `kbn-router-to-openapispec` unwraps so route `params` / `query` / `body` actually appear in generated OAS docs. The local copy did not, so any OAS doc generation for these routes was silently dropping schema info. - **Regenerated OAS types** (`yarn openapi:generate` in `kbn-evals-common`) - Picks up the `lazySchema(() => …)` wrappers introduced by elastic#264125 and tuned in elastic#266343 (upstream `colinhacks/zod#5897` lazy-bind memory optimization). 19 `.gen.ts` files updated; pattern matches the prior inbox regen exactly. - **README cleanup** — pointed at the new helper location and dropped the obsolete "after regenerating, you may need to fix unused imports" workaround (no longer needed thanks to the OAS-generator `fix_eslint.ts` fix from elastic#265634, which forces the non-editor branch when invoked from agent/IDE terminals). _PR developed with Cursor + Claude Opus 4.7 Super Duper xHigh Thinking++_

Summary
Builder methods on Zod schemas (
.optional,.nullable,.array,.pick,.partial,.regex,.min, etc.) are currently allocated as fresh per-instance closures duringinit. On a plainz.string()that's ~50 closures up front — dominated by V8JSFunction+Contextbytes — even when the user never calls them.This PR moves them onto a hidden prototype layer inserted between
_.prototype(user-extension space) and the parent. On first access from any instance the lazy getter allocatesfn.bind(this)and caches it as an own enumerable property; subsequent accesses skip the getter entirely.Detached usage keeps working — that was the dealbreaker for #5870, and it's why this PR adds a regression test guarding it:
The cached value is a bound function with
thisalready resolved on first access, so it's safe to detach.Parse-family methods (
parse,safeParse,parseAsync,encode,decode, etc.) intentionally stay as eager per-instance closures — they're the hot path AND the most-detached methods (arr.map(schema.parse),const { parse } = schema), so paying the allocation up front is worth the monomorphic call site and predictable behavior.Benchmarks
Run with
node --expose-gc+tsx, 10k instances each,process.memoryUsage().heapUseddelta after twoglobal.gc()calls. Numbers are stable across runs (variance < 5% on heap, ~10% on parse times).Heap per instance (lower is better)
z.string()z.string().min(1).max(10).email()z.number()z.boolean()z.array(z.string())z.object({ a, b })z.string().optional()The savings come from not allocating
JSFunction+Contextpairs for the ~50 builder methods up front. Methods are still bound and cached on the instance after first access, so a schema you actually.optional()ends up paying close to today's cost on that one method, but nothing for the other ~49 you didn't touch.Construction time (lower is better)
z.string()z.string().min(1).max(10).email()z.number()z.array(z.string())z.object({ a, b })Construction is faster because the per-instance loop that used to assign ~50 closures now installs them on the prototype once per constructor and skips the work on every subsequent instance.
Parse hot path (lower is better, 100k iterations, 3-run avg)
z.string().min(1).max(50).email()× 100kz.object({ name, age, tags }).safeParse× 100kParse-family methods are unchanged (eager per-instance closures), so any movement here is JIT/GC noise. Run-to-run variance was higher than the delta in both directions; we treat this as effectively unchanged.
Property access throughput
schema._zodschema.parseschema.optionalAfter first access the lazy methods are cached as own properties, so steady-state access cost is identical to before (the getter only runs once per instance per method).
Bundle size
The new infrastructure is small fixed overhead and lives entirely in classic — mini bundles are bytes-for-bytes identical to
main. Bundled withesbuild --bundle --minify --format=esm, gzipped:Classic —
import * as z from "zod"; z.string().parse("")Classic — rich object schema (
z.object({ email, age, tags, role }).strict()with checks)Mini —
import * as z from "zod/mini"; z.parse(z.string(), "")Mini — rich object schema
Notes:
main. The lazy-bind layer lives entirely inclassic/schemas.ts; nothing was added tocore/core.ts.Implementation
core/core.ts— no changes. The lazy-bind layer is entirely classic-side.classic/schemas.ts:_installLazyMethods(inst, group, methods)helper. Installs each method as a non-enumerable getter directly on the instance's prototype (idempotent per (prototype, group) viaWeakMap). The getter, on first access from an instance, allocatesfn.bind(this)and stores it as an own property on the instance — subsequent accesses skip the getter entirely.ZodType,_ZodString,ZodNumber,ZodArray,ZodObjectare passed in as inline method-shorthand bodies — no top-level_sharedXxxnamed functions, no_sharedprefix.Compatibility
ZodType/ZodObject/etc. are unchanged.detached-methods.test.tscovers ~30 such patterns).z.ZodType.prototype.x = ...user extensions still work. The lazy methods are installed as non-enumerable getters on_.prototype, so the existing copy-loop incore.$constructor(which usesObject.keys, enumerable own only) doesn't see them, and any enumerable user extension on the same prototype shadows the lazy method as before.Object.keys(z.ZodString.prototype)is unchanged (still empty by default) andObject.keys(schema)enumerates the same keys as before once methods have been touched.Object.getOwnPropertyNames(z.ZodString.prototype)and'optional' in z.ZodString.prototypenow reflect the lazy methods (was previously empty /false). This is arguably more intuitive — the methods conceptually do live on the prototype.Test plan
pnpm vitest run— 3,743 passedpnpm tsc --noEmit -p packages/zod/tsconfig.json— same 13 pre-existing unused-variable errors asmain, no new errorsObject.keys