[Jest] Lazy core mocks by dgieselaar · Pull Request #235003 · elastic/kibana

dgieselaar · 2025-09-13T07:09:09Z

Lazily instantiate core mocks, to relieve memory pressure. This has the benefit of tests using less memory usage, and thus us being able to run them in parallel without upsizing our CI workers, and CPU time spent on garbage collection goes down. Here's how it affects one of the alerting plugin test files, significant improvements across the board:

Benchmark diff: cwd -> f69f92
cwd: [Jest] Lazily instantiate mocks
f69f92: [Agent Builder] Setup doclinks structure (#235804)
alerting-plugin-test
-
                         cwd            f69f92                    Δ                   CI 
 Duration         6.3s ±1.7%        7.0s ±1.8%       711ms (+11.3%)    95%, +7.9%–+14.8% 
 CPU Usage        6.5s ±2.0%        7.4s ±2.1%       852ms (+13.1%)    95%, +9.1%–+17.2% 
 Max RSS     857.90 MB ±1.3%   977.20 MB ±2.4%   119.30 MB (+13.9%)   95%, +10.3%–+17.6% 
 GC time         190ms ±4.4%       252ms ±3.4%        62ms (+32.6%)   95%, +23.8%–+42.0%

How it works

Core mocks are instantiated using lazyObject. This is a helper utility that will be rewritten using a Babel plugin to create annotated getters that will only instantiate the mock when it is accessed.

Why it helps

Core mocks are eagerly created, and this means that many objects will be allocated. Specifically, jest will keep mocks in a global registry to support jest.clearAllMocks(). This means only when a test file completes these functions will be garbage collected. Increased memory consumption has two consequences that are relevant in this context:

they increase memory pressure, which means garbage collection has to run more frequently to prevent the process from running out of memory
our CI workers (that run Jest tests) have 16gb of memory. running jest configs in parallel means that there currently is a risk that either the processes themselves run out of memory, or the agent will. if we reduce memory usage, we can run them in parallel.

jloleysens

Just did an initial pass and thought I'd share my thoughts.

This is pretty impressive. It seems like we are patching jest, which is OK, but would be nice if they could make mock instantiation more perf on their side.

In the meantime, I'm wondering whether we can find a solution that does not rely on meta programming. If I have it right, the primary culprit leading to GC thrashing is calls to jest.fn() adding a lot of mock objects to the heap/creating work for jest. Is it possible to create a more targeted solution to handle this?

Something like this to be run at test setup time (using jest's setupFiles, did not test, not sure if possible)

const originalJestFn = jest.fn;
const lazyJestFn: typeof jest.fn = (impl?: (...args: any[]) => any): jest.Mock => {
  /* logic to create jest mock instance lazily */
  return fn as any;
};
jest.fn = lazyJestFn;

This seems like it could get the same thing done and touch a lot fewer files. If it's not possible that's OK too, overall this seems like a really nice improvement.

src/platform/packages/shared/kbn-lazy-object/src/create_lazy_object_from_annotations.ts

jloleysens · 2025-09-15T07:49:52Z

src/platform/packages/shared/kbn-lazy-object/src/create_lazy_object_from_annotations.ts

+        set(v) {
+          Object.defineProperty(target, key, {


If a lazy prop gets set before it got get will the getter logic be clobbered (effectively nullified)? It seems that's the intent based on your tests, just wanted to confirm.

yes, it's treated as an override

jloleysens · 2025-09-15T08:11:23Z

src/platform/packages/shared/kbn-lazy-object/src/create_lazy_object_from_factories.ts

+
+const DISABLED = !!process.env.DISABLE_LAZY_OBJECT;
+
+export function createLazyObjectFromFactories<TFactories extends Record<string, () => any>>(


nit: not quite clear to me where this fn is being used.

I'll add some clarifying comments - the idea is that you can use createLazyObjectFromFactories directly (without the implicit rewrite that lazyObject does).

dgieselaar · 2025-09-15T11:01:43Z

This is pretty impressive. It seems like we are patching jest, which is OK, but would be nice if they could make mock instantiation more perf on their side.

In the meantime, I'm wondering whether we can find a solution that does not rely on meta programming. If I have it right, the primary culprit leading to GC thrashing is calls to jest.fn() adding a lot of mock objects to the heap/creating work for jest. Is it possible to create a more targeted solution to handle this?

Something like this to be run at test setup time (using jest's setupFiles, did not test, not sure if possible)

const originalJestFn = jest.fn;
const lazyJestFn: typeof jest.fn = (impl?: (...args: any[]) => any): jest.Mock => {
/* logic to create jest mock instance lazily */
return fn as any;
};
jest.fn = lazyJestFn;

This seems like it could get the same thing done and touch a lot fewer files. If it's not possible that's OK too, overall this seems like a really nice improvement.

I think this would help with jest's internals, but it still allocates many objects even when they are not being used, and we'd have to reimplement the Jest Spy API surface, and I'm not sure if doing things like .mockResolvedValue() would immediately side-step the lazy invalidation. I don't have the data to quantify one vs the other, but my guess is that it's more effective to "lazify" mocks higher up so we don't allocate at all.

cesco-f

Changes in x-pack/solutions/observability/plugins/infra/server/lib/alerting/metric_threshold/metric_threshold_executor.test.ts LGTM.

janmonschke

kibana-cases changes lgtm!

dmlemeshko

src/platform/packages/shared/kbn-test/jest-preset.js changes LGTM

jloleysens

Nice work @dgieselaar , given the trade-offs this approach makes sense to me. Thanks for driving this to a conclusion.

Do you think it's worth raising the impact of 1000s of repeat calls to jest.fn() to the Jest team in an issue? Perhaps they can delay mock instantiation internally -- I'm happy to do so at some point.

jloleysens · 2025-09-22T12:34:12Z

src/platform/packages/shared/kbn-lazy-object/src/plugin/lazy_babel_plugin.js

+    visitor: {
+      CallExpression(path) {
+        const { node } = path;
+        if (!isLazyObjectCallee(node.callee)) return;


Couldn't quite tell whether it is already handled, but we should prevent this transformation from replacing a definition of lazyObject that is not defined by @kbn/lazy-object.

thanks for the nudge, implemented an import check and added a test for it

kc13greiner

Security changes LGTM!

Ikuni17

The new package @kbn/lazy-object needs to be added to Codeowners. The package itself doesn't have any tests either.

dgieselaar · 2025-09-22T19:59:44Z

@Ikuni17 it does have tests actually. What do you mean that it needs to be added to CODEOWNERS? A codeowner is defined in kibana.jsonc, thats usually enough?

elasticmachine · 2025-09-23T07:27:45Z

⏳ Build in-progress, with failures

Buildkite Build
Commit: 3881e3c

Failed CI Steps

Check Types

History

💚 Build #342020 succeeded a721815
💔 Build #341915 failed 9fe74cd
💔 Build #341796 failed 1a2073f
💛 Build #341727 was flaky 1a2073f
💔 Build #341723 failed 656ecab
💔 Build #339049 failed 2cb2d3c

Closes elastic#235079 Lazily instantiate core mocks, to relieve memory pressure. This has the benefit of tests using less memory usage, and thus us being able to run them in parallel without upsizing our CI workers, and CPU time spent on garbage collection goes down. Here's how it affects one of the alerting plugin test files, significant improvements across the board: ```stdout Benchmark diff: cwd -> f69f92 cwd: [Jest] Lazily instantiate mocks f69f92: [Agent Builder] Setup doclinks structure (elastic#235804) alerting-plugin-test - cwd f69f92 Δ CI Duration 6.3s ±1.7% 7.0s ±1.8% 711ms (+11.3%) 95%, +7.9%–+14.8% CPU Usage 6.5s ±2.0% 7.4s ±2.1% 852ms (+13.1%) 95%, +9.1%–+17.2% Max RSS 857.90 MB ±1.3% 977.20 MB ±2.4% 119.30 MB (+13.9%) 95%, +10.3%–+17.6% GC time 190ms ±4.4% 252ms ±3.4% 62ms (+32.6%) 95%, +23.8%–+42.0% ``` ### How it works Core mocks are instantiated using `lazyObject`. This is a helper utility that will be rewritten using a Babel plugin to create annotated getters that will only instantiate the mock when it is accessed. ### Why it helps Core mocks are eagerly created, and this means that many objects will be allocated. Specifically, jest will keep mocks in a global registry to support `jest.clearAllMocks()`. This means only when a test file completes these functions will be garbage collected. Increased memory consumption has two consequences that are relevant in this context: - they increase memory pressure, which means garbage collection has to run more frequently to prevent the process from running out of memory - our CI workers (that run Jest tests) have 16gb of memory. running jest configs in parallel means that there currently is a risk that either the processes themselves run out of memory, or the agent will. if we reduce memory usage, we can run them in parallel. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Closes #235079 Lazily instantiate core mocks, to relieve memory pressure. This has the benefit of tests using less memory usage, and thus us being able to run them in parallel without upsizing our CI workers, and CPU time spent on garbage collection goes down. Here's how it affects one of the alerting plugin test files, significant improvements across the board: ```stdout Benchmark diff: cwd -> f69f92 cwd: [Jest] Lazily instantiate mocks f69f92: [Agent Builder] Setup doclinks structure (#235804) alerting-plugin-test - cwd f69f92 Δ CI Duration 6.3s ±1.7% 7.0s ±1.8% 711ms (+11.3%) 95%, +7.9%–+14.8% CPU Usage 6.5s ±2.0% 7.4s ±2.1% 852ms (+13.1%) 95%, +9.1%–+17.2% Max RSS 857.90 MB ±1.3% 977.20 MB ±2.4% 119.30 MB (+13.9%) 95%, +10.3%–+17.6% GC time 190ms ±4.4% 252ms ±3.4% 62ms (+32.6%) 95%, +23.8%–+42.0% ``` ### How it works Core mocks are instantiated using `lazyObject`. This is a helper utility that will be rewritten using a Babel plugin to create annotated getters that will only instantiate the mock when it is accessed. ### Why it helps Core mocks are eagerly created, and this means that many objects will be allocated. Specifically, jest will keep mocks in a global registry to support `jest.clearAllMocks()`. This means only when a test file completes these functions will be garbage collected. Increased memory consumption has two consequences that are relevant in this context: - they increase memory pressure, which means garbage collection has to run more frequently to prevent the process from running out of memory - our CI workers (that run Jest tests) have 16gb of memory. running jest configs in parallel means that there currently is a risk that either the processes themselves run out of memory, or the agent will. if we reduce memory usage, we can run them in parallel. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Closes elastic#235079 Lazily instantiate core mocks, to relieve memory pressure. This has the benefit of tests using less memory usage, and thus us being able to run them in parallel without upsizing our CI workers, and CPU time spent on garbage collection goes down. Here's how it affects one of the alerting plugin test files, significant improvements across the board: ```stdout Benchmark diff: cwd -> f69f92 cwd: [Jest] Lazily instantiate mocks f69f92: [Agent Builder] Setup doclinks structure (elastic#235804) alerting-plugin-test - cwd f69f92 Δ CI Duration 6.3s ±1.7% 7.0s ±1.8% 711ms (+11.3%) 95%, +7.9%–+14.8% CPU Usage 6.5s ±2.0% 7.4s ±2.1% 852ms (+13.1%) 95%, +9.1%–+17.2% Max RSS 857.90 MB ±1.3% 977.20 MB ±2.4% 119.30 MB (+13.9%) 95%, +10.3%–+17.6% GC time 190ms ±4.4% 252ms ±3.4% 62ms (+32.6%) 95%, +23.8%–+42.0% ``` ### How it works Core mocks are instantiated using `lazyObject`. This is a helper utility that will be rewritten using a Babel plugin to create annotated getters that will only instantiate the mock when it is accessed. ### Why it helps Core mocks are eagerly created, and this means that many objects will be allocated. Specifically, jest will keep mocks in a global registry to support `jest.clearAllMocks()`. This means only when a test file completes these functions will be garbage collected. Increased memory consumption has two consequences that are relevant in this context: - they increase memory pressure, which means garbage collection has to run more frequently to prevent the process from running out of memory - our CI workers (that run Jest tests) have 16gb of memory. running jest configs in parallel means that there currently is a risk that either the processes themselves run out of memory, or the agent will. if we reduce memory usage, we can run them in parallel. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

dgieselaar changed the title ~~Create Babel transform config with custom cachekey config~~ [Jest] Lazy core mocks Sep 13, 2025

dgieselaar force-pushed the lazy-mocks branch 4 times, most recently from a4f901c to 2cb2d3c Compare September 13, 2025 10:47

jloleysens reviewed Sep 15, 2025

View reviewed changes

dgieselaar force-pushed the lazy-mocks branch 5 times, most recently from 426ea4a to d1355c9 Compare September 20, 2025 15:50

[Jest] Lazily instantiate mocks

1a2073f

dgieselaar force-pushed the lazy-mocks branch from d1355c9 to 1a2073f Compare September 20, 2025 16:08

dgieselaar added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting backport:version Backport to applied version labels v9.2.0 labels Sep 20, 2025

dgieselaar marked this pull request as ready for review September 22, 2025 07:03

dgieselaar requested review from a team as code owners September 22, 2025 07:03

adcoelho approved these changes Sep 22, 2025

View reviewed changes

cesco-f approved these changes Sep 22, 2025

View reviewed changes

janmonschke approved these changes Sep 22, 2025

View reviewed changes

dmlemeshko approved these changes Sep 22, 2025

View reviewed changes

kowalczyk-krzysztof approved these changes Sep 22, 2025

View reviewed changes

dgieselaar added 2 commits September 22, 2025 11:52

Merge branch 'main' into lazy-mocks

9fe74cd

Merge branch 'main' into lazy-mocks

a721815

jloleysens approved these changes Sep 22, 2025

View reviewed changes

kc13greiner self-requested a review September 22, 2025 13:44

kc13greiner approved these changes Sep 22, 2025

View reviewed changes

Ikuni17 requested changes Sep 22, 2025

View reviewed changes

Ikuni17 approved these changes Sep 22, 2025

View reviewed changes

dgieselaar added 3 commits September 23, 2025 08:09

Merge branch 'main' of github.com:elastic/kibana into lazy-mocks

24a717f

Only rewrite lazyObject calls from @kbn/lazy-object helper

1bef5f2

Shared ownership

0ee2c4d

dgieselaar enabled auto-merge (squash) September 23, 2025 07:26

Avoid circular reference

b6f8c81

dgieselaar force-pushed the lazy-mocks branch from 3881e3c to b6f8c81 Compare September 23, 2025 07:36

dgieselaar merged commit 1c83150 into elastic:main Sep 23, 2025
15 checks passed

dgieselaar deleted the lazy-mocks branch September 23, 2025 10:13

maryam-saeidi mentioned this pull request Sep 25, 2025

[DevEx] Investigate elasticsearch mock performance in jest tests #234366

Closed


		const DISABLED = !!process.env.DISABLE_LAZY_OBJECT;

		export function createLazyObjectFromFactories<TFactories extends Record<string, () => any>>(

Conversation

dgieselaar commented Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How it works

Why it helps

Uh oh!

jloleysens left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jloleysens Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

dgieselaar Sep 20, 2025

Choose a reason for hiding this comment

Uh oh!

jloleysens Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

dgieselaar Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

dgieselaar commented Sep 15, 2025

Uh oh!

cesco-f left a comment

Choose a reason for hiding this comment

Uh oh!

janmonschke left a comment

Choose a reason for hiding this comment

Uh oh!

dmlemeshko left a comment

Choose a reason for hiding this comment

Uh oh!

jloleysens left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dgieselaar Sep 23, 2025

Choose a reason for hiding this comment

Uh oh!

kc13greiner left a comment

Choose a reason for hiding this comment

Uh oh!

Ikuni17 left a comment

Choose a reason for hiding this comment

Uh oh!

dgieselaar commented Sep 22, 2025

Uh oh!

elasticmachine commented Sep 23, 2025

⏳ Build in-progress, with failures

Failed CI Steps

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

dgieselaar commented Sep 13, 2025 •

edited

Loading

jloleysens left a comment •

edited

Loading

jloleysens left a comment •

edited

Loading

jloleysens Sep 22, 2025 •

edited

Loading