Fix ES health check poller retries by jloleysens · Pull Request #248496 · elastic/kibana

jloleysens · 2026-01-09T16:47:50Z

Summary

Ensure that our retry logic calls the network request again the configured number of retries.

Description

After introducing the retries to pollEsNodesVersion a failed API request to the nodes info API would not actually retry the network request. Each "retry" would just be a no-op delayed by the healthCheckInterval. So e.g. if a request times out we would wait healthCheckInterval*healthCheckRetry ms and emit the failure. The nodes info API call would only be called again on the next healthCheckInterval.

This PR changes the behavior to actually call the API during retries. Assuming the API calls continuously fails, this would create healthCheckRetry API requests and only if all of these fail it would emit the failure.

elasticmachine · 2026-01-09T16:47:54Z

Pinging @elastic/kibana-core (Team:Core)

jloleysens · 2026-01-09T16:48:19Z

src/core/packages/elasticsearch/server-internal/src/version_check/ensure_es_version.test.ts

Added expected info API call counts

elasticmachine · 2026-01-09T19:15:52Z

💚 Build Succeeded

Buildkite Build
Commit: 2929c67

Metrics [docs]

✅ unchanged

jloleysens · 2026-01-10T10:57:58Z

src/core/packages/elasticsearch/server-internal/src/version_check/ensure_es_version.test.ts

        },
-        complete: done,
+        complete: () => {
+          expect(internalClient.nodes.info).toHaveBeenCalledTimes(1);


Nit: just thinking maybe we can increase health retries here to something like 1000 to really make it clear that we only expect 1 network request for successful calls

jloleysens · 2026-01-10T10:58:38Z

src/core/packages/elasticsearch/server-internal/src/version_check/ensure_es_version.test.ts

+          expect(result.isCompatible).toBeDefined();
+        },
+        complete: () => {
+          done();


Can we add an expect against internalClient.info.nodes here for the expected number of calls?

jloleysens · 2026-01-10T10:59:27Z

src/core/packages/elasticsearch/server-internal/src/version_check/ensure_es_version.test.ts

    nodeInfosSuccessOnce(createNodes('5.1.1', '5.1.2', '5.1.3')); // emit
    nodeInfosSuccessOnce(createNodes('5.1.1', '5.1.2', '5.1.3')); // ignore
    nodeInfosSuccessOnce(createNodes('5.0.0', '5.1.0', '5.2.0')); // emit, different from previous version
+    nodeInfosSuccessOnce(createNodes('5.1.0', '5.1.0', '5.1.0')); // emit, no warning nodes, used to detect end of test


jloleysens · 2026-01-10T10:59:42Z

src/core/packages/elasticsearch/server-internal/src/version_check/ensure_es_version.ts

    .map((key) => nodesInfoResponse.nodes[key])
-    .map((node) => Object.assign({}, node, { name: getHumanizedNodeName(node) }));
+    .map((node) => Object.assign({}, node, { name: getHumanizedNodeName(node) }))
+    .sort(sortNodes); // Sorting ensures stable ordering for comparison


kibanamachine · 2026-01-10T12:25:12Z

Starting backport for target branches: 8.19, 9.1, 9.2, 9.3

https://github.com/elastic/kibana/actions/runs/20878265645

## Summary Ensure that our `retry` logic calls the network request again the configured number of retries. (cherry picked from commit 33fa57c)

kibanamachine · 2026-01-10T12:33:16Z

💔 Some backports could not be created

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts
❌	9.1	Backport failed because of merge conflicts
❌	9.2	Backport failed because of merge conflicts
✅	9.3

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 248496

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `9.3`: - [Fix ES health check poller (#248496)](#248496)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Jean-Louis Leysens <jeanlouis.leysens@elastic.co>

## Summary Ensure that our `retry` logic calls the network request again the configured number of retries.

* commit 'c4304e27736c62f17af20d145770b2ae9d3fae30': (418 commits) skip failing suite (elastic#89079) [ES|QL] Update grammars (elastic#248600) skip failing test suite (elastic#248579) [ES|QL] Update function metadata (elastic#248601) skip failing test suite (elastic#248554) Fix flaky test runner serverless flag for Search solution (elastic#248559) [Security Solution][Attacks/Alerts][Attacks page][Table section] Remember last selected attack details tab (Summary or Alerts) (elastic#247519) (elastic#247988) Fix ES health check poller (elastic#248496) Fix collector schema ownership (elastic#241292) [api-docs] 2026-01-10 Daily api_docs build (elastic#248574) Update dependency cssstyle to v5.3.5 (main) (elastic#237637) Update dependency @octokit/rest to v22.0.1 (main) (elastic#243102) skip failing test suite (elastic#248504) skip failing test suite (elastic#247685) Remove broken ecommerce_dashboard journeys (elastic#248162) [Obs AI] Hide AI Insight component when there are no connectors (elastic#248542) skip failing suite (elastic#248433) [Security Solution][Attacks/Alerts][Attacks page][Table section] Hide tabs for generic attack groups (elastic#248444) [Agent Builder] [AI Infra] Adds product documentation tool and task evals (elastic#248370) [Controls Anywhere] Keep controls focused when creating + editing other panels (elastic#248021) ...

kibanamachine · 2026-01-12T10:49:23Z

Starting backport for target branches: 8.19, 9.1, 9.2, 9.3

https://github.com/elastic/kibana/actions/runs/20916590759

kibanamachine · 2026-01-12T10:58:00Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts
❌	9.1	Backport failed because of merge conflicts
❌	9.2	Backport failed because of merge conflicts
❌	9.3	Cherrypick failed because the selected commit (`33fa57c`) is empty. It looks like the commit was already backported in #248577

Manual backport

To create the backport manually run:

node scripts/backport --pr 248496

Questions ?

Please refer to the Backport tool documentation

defer and update test assertions

5cd1c80

jloleysens requested a review from a team as a code owner January 9, 2026 16:47

jloleysens added Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// release_note:skip Skip the PR/issue when compiling release notes backport:all-open Backport to all branches that could still receive a release labels Jan 9, 2026

jloleysens commented Jan 9, 2026

View reviewed changes

clear mocks

2929c67

jloleysens enabled auto-merge (squash) January 9, 2026 16:51

Unskip test

3448993

rudolf approved these changes Jan 10, 2026

View reviewed changes

jloleysens changed the title ~~Fix ES health check poller~~ Fix ES health check poller retries Jan 10, 2026

rudolf added 2 commits January 10, 2026 11:36

Fix to ignore same versions in different ordering

2b84b21

Fix unrelated test after sorting by version

8d1ee1e

jloleysens commented Jan 10, 2026

View reviewed changes

jloleysens merged commit 33fa57c into elastic:main Jan 10, 2026
13 checks passed

kibanamachine added the v9.4.0 label Jan 10, 2026

kibanamachine mentioned this pull request Jan 10, 2026

[9.3] Fix ES health check poller (#248496) #248577

Merged

kibanamachine added the v9.3.0 label Jan 10, 2026

devamanv pushed a commit to devamanv/kibana that referenced this pull request Jan 12, 2026

Fix ES health check poller (elastic#248496)

96d31b6

## Summary Ensure that our `retry` logic calls the network request again the configured number of retries.

jloleysens added backport:version Backport to applied version labels and removed backport:all-open Backport to all branches that could still receive a release labels Jan 12, 2026

jloleysens deleted the fix/es-health-poller branch January 12, 2026 10:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix ES health check poller retries#248496

Fix ES health check poller retries#248496
jloleysens merged 5 commits intoelastic:mainfrom
jloleysens:fix/es-health-poller

jloleysens commented Jan 9, 2026 •

edited by kibanamachine

Loading

Uh oh!

elasticmachine commented Jan 9, 2026

Uh oh!

jloleysens Jan 9, 2026 •

edited

Loading

Uh oh!

elasticmachine commented Jan 9, 2026

Uh oh!

jloleysens Jan 10, 2026

Uh oh!

jloleysens Jan 10, 2026 •

edited

Loading

Uh oh!

jloleysens Jan 10, 2026

Uh oh!

jloleysens Jan 10, 2026

Uh oh!

Uh oh!

kibanamachine commented Jan 10, 2026

Uh oh!

kibanamachine commented Jan 10, 2026

Uh oh!

kibanamachine commented Jan 12, 2026

Uh oh!

kibanamachine commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jloleysens commented Jan 9, 2026 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Description

Uh oh!

elasticmachine commented Jan 9, 2026

Uh oh!

jloleysens Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jan 9, 2026

💚 Build Succeeded

Metrics [docs]

Uh oh!

jloleysens Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

jloleysens Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

jloleysens Jan 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Jan 10, 2026

Uh oh!

kibanamachine commented Jan 10, 2026

💔 Some backports could not be created

Manual backport

Questions ?

Uh oh!

kibanamachine commented Jan 12, 2026

Uh oh!

kibanamachine commented Jan 12, 2026

💔 All backports failed

Manual backport

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jloleysens commented Jan 9, 2026 •

edited by kibanamachine

Loading

jloleysens Jan 9, 2026 •

edited

Loading

jloleysens Jan 10, 2026 •

edited

Loading