Skip to content

[8.19] [Streams] Refactor schemas (#218308)#219830

Merged
dgieselaar merged 6 commits intoelastic:8.19from
dgieselaar:backport/8.19/pr-218308
May 1, 2025
Merged

[8.19] [Streams] Refactor schemas (#218308)#219830
dgieselaar merged 6 commits intoelastic:8.19from
dgieselaar:backport/8.19/pr-218308

Conversation

@dgieselaar
Copy link
Contributor

Backport

This will backport the following commits from main to 8.19:

Questions ?

Please refer to the Backport tool documentation

Refactors Streams models to use namespaces. `Streams` contains both the
TypeScript interfaces and the validation schemas. In general, there are
four variations of each Stream type:

- `Definition` - the definition that we largely work with in the Streams
client and in the browser.
- `Source` - how the definition is stored in ES. This is a pass-through
at the minute, but separating this from the definition allows us to do
migrations on read and write.
- `GetResponse` - what is returned from `GET /api/streams/{name}`. This
includes the definition, but also other assets that are not part of the
definition, but rather a function of the definition, like Elasticsearch
index templates etc, and attached Kibana assets like Dashboards and
Queries.
- `UpsertRequest` - what is required in `PUT /api/streams/{name}`. This
includes the stream definition, but, importantly, without a name - the
name is not allowed in the request endpoint. Similarly, this allows the
user to update assets like Dashboards and Queries.

Schemas are defined as Validations, which are left/right schemas - that
is, the type on the right is always a refinement of the type on the
left. This allows for more safe transitions than going from any object
to the specific type. Additionally, it includes helpers such as `is`,
`as`, `parse` and `asserts`, which help with both runtime validation and
type refinement.

## Additional changes

- Added special formatting for `AggregateError` in MessageConversion
- Simplified DeepStrict

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit a1f2691)

# Conflicts:
#	oas_docs/bundle.json
#	oas_docs/output/kibana.yaml
#	x-pack/test/api_integration/deployment_agnostic/apis/observability/streams/helpers/requests.ts
@dgieselaar dgieselaar requested a review from kibanamachine as a code owner May 1, 2025 10:17
@dgieselaar dgieselaar added the backport This PR is a backport of another PR label May 1, 2025
@dgieselaar dgieselaar enabled auto-merge (squash) May 1, 2025 10:17
@botelastic botelastic bot added the Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. label May 1, 2025
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@prodsecmachine
Copy link
Collaborator

prodsecmachine commented May 1, 2025

🎉 Snyk checks have passed. No issues have been found so far.

security/snyk check is complete. No issues have been found. (View Details)

license/snyk check is complete. No issues have been found. (View Details)

kibanamachine and others added 5 commits May 1, 2025 10:42
…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/dashboards --include-path /api/alerting/maintenance_window --update --no-serverless'
@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #10 / useDeleteAction deletes the selected cases

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
integrationAssistant 505 506 +1
inventory 235 236 +1
securitySolution 7256 7257 +1
total +3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/streams-schema 377 212 -165
@kbn/zod-helpers 20 22 +2
total -163

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id before after diff
@kbn/streams-schema 13 0 -13

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
streamsApp 428.3KB 431.3KB +3.0KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/streams-schema 1 17 +16
streams 26 28 +2
total +18

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
core 433.4KB 433.8KB +398.0B
Unknown metric groups

API count

id before after diff
@kbn/streams-schema 392 221 -171
@kbn/zod-helpers 29 31 +2
total -169

ESLint disabled in files

id before after diff
@kbn/streams-schema 0 7 +7

Total ESLint disabled count

id before after diff
@kbn/streams-schema 0 7 +7

History

@dgieselaar dgieselaar merged commit 62c0397 into elastic:8.19 May 1, 2025
8 checks passed
@dgieselaar dgieselaar deleted the backport/8.19/pr-218308 branch May 1, 2025 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants