Skip to content

Upgrade ES client to 9.0.0-alpha.4#213375

Merged
afharo merged 24 commits into
elastic:mainfrom
afharo:es-9.0.0-alpha.4
Apr 3, 2025
Merged

Upgrade ES client to 9.0.0-alpha.4#213375
afharo merged 24 commits into
elastic:mainfrom
afharo:es-9.0.0-alpha.4

Conversation

@afharo
Copy link
Copy Markdown
Member

@afharo afharo commented Mar 6, 2025

Summary

Upgrading the ES client to v9.0.0-alpha.4 to test the changes mentioned in elastic/elasticsearch-js#2584

This new version introduces some type changes, most notably, the FieldValue is string | number | boolean | null | undefined instead of any, leading to some new type checks to be implemented (like on aggregation results bucket.key, search_after, and sort options).

On top of that, it adds the new behavior where unknown properties are placed in the body (when the request has a body). If they must be in as a query parameter, they should be placed under the querystring option.

cc @JoshMock

TODO:

  • Stabilize the type errors
  • Address all the query parameters that are now placed in the body (by wrapping them inside the option querystring: {})

I will address // @ts-expect-error elasticsearch@9.0.0 https://github.com/elastic/elasticsearch-js/issues/2584 in a separate PR to reduce noise.

Related #208776

@afharo afharo self-assigned this Mar 6, 2025
@afharo afharo added Feature:elasticsearch Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting elasticsearch-js-9 Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// Team:Fleet Team label for Observability Data Collection Fleet team Team:Detections and Resp Security Detection Response Team Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:SharedUX Platform AppEx-SharedUX (formerly Global Experience) t// Team:obs-onboarding Observability Onboarding Team Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// Team:Detection Rule Management Security Detection Rule Management Team Team:Cloud Security Cloud Security team related Team:Detection Engine Security Solution Detection Engine Area Team:Obs AI Assistant Observability AI Assistant Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. Team:Security Generative AI Security Generative AI Team:AI Infra Platform AppEx AI Infrastructure Team t// Feature:Streams This is the label for the Streams Project labels Mar 6, 2025
@afharo afharo force-pushed the es-9.0.0-alpha.4 branch from c2a6ef3 to 6cadeaa Compare March 6, 2025 22:02
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoshMock, here's a missing query parameter in the spec.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I opened elastic/elasticsearch-specification#3954 to add it (plus a bunch of others that were also missing).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

require_alias is accepted by the IndexRequest but not the CreateRequest, so I went with the querystring passthrough for both here.

}`
);
const result = (await originalEqlSearch.call(opts.esClient, params, {
const result = (await originalEqlSearch.call(opts.esClient.eql, params, {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems weird that this would have worked before!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes... I think that the internals changed to address the body vs. query parameters... but maybe @JoshMock can provide a more accurate reason.

Copy link
Copy Markdown
Member

@JoshMock JoshMock Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one's a bit tricky. 🤔 It may be because, in 8.x, That is effectively the same thing as Pick<Client, "transport">, whereas in 9.0 That also holds the acceptable parameters so it is no longer a subset of properties from Client.

I wasn't thinking about that side effect when making this change, but now I'm wondering if there's a better pattern to avoid this being a breaking change.

}) ?? {};
const byTaskType = ((aggregations.byTaskType.buckets as OverdueTaskAggBucket[]) ?? []).reduce(
(acc: Record<string, number>, bucket: OverdueTaskAggBucket) => {
// @ts-expect-error there's no way that buckets (array) matches `number`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the acc var typed wrong here? Seems like it should be an Record<string, something[]>? So seems like this may require a more involved fix, not clear what it would be though ...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ I think that the fact that FieldValue used to be any didn't highlight this wrongly typed reduce.

I didn't know what's the goal of this reduce, so I thought I'd flag it and leave it to the owners to fix it 😇

@afharo afharo requested a review from a team as a code owner April 2, 2025 14:39
@afharo afharo added Team:Search Team:Threat Hunting Security Solution Threat Hunting Team labels Apr 2, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

Copy link
Copy Markdown
Contributor

@baileycash-elastic baileycash-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on behalf of obs-ux-mgmt:
investigate_app
slo
observability-server (?)

changes make sense to me

query?: estypes.QueryDslQueryContainer;
sort?: estypes.SortOptions[];
_source?: string[] | false;
search_after?: Array<string | number>;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, because this way we reduce the actual definition of search_after (which includes boolean and null 🤷 )

Copy link
Copy Markdown
Contributor

@kdelemme kdelemme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 3, 2025

💚 Build Succeeded

  • Buildkite Build
  • Commit: 2d83fc2
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-213375-2d83fc2d16b9

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
security 515.6KB 515.6KB +5.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
data 55 56 +1

Total ESLint disabled count

id before after diff
data 56 57 +1

History

cc @afharo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting ci:build-serverless-image elasticsearch-js-9 Feature:elasticsearch Feature:Streams This is the label for the Streams Project release_note:skip Skip the PR/issue when compiling release notes Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. Team:AI Infra Platform AppEx AI Infrastructure Team t// Team:Cloud Security Cloud Security team related Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:Detection Engine Security Solution Detection Engine Area Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team:Fleet Team label for Observability Data Collection Fleet team Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// Team:Obs AI Assistant Observability AI Assistant Team:obs-onboarding Observability Onboarding Team Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// Team:Search Team:Security Generative AI Security Generative AI Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// Team:SharedUX Platform AppEx-SharedUX (formerly Global Experience) t// Team:Threat Hunting Security Solution Threat Hunting Team v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.