[Streams 🌊] Prevent routing simulation timeout and improve cancellations by tonyghiani · Pull Request #226374 · elastic/kibana

tonyghiani · 2025-07-03T10:16:27Z

📓 Summary

Closes #219494

Refactor the samples fetching logic and address the following issues:

Add a timeout (10 seconds) to avoid requests on incomplete or unmatching conditions to keep running with no feedback.
Update cancellation logic, previously was not catching correctly the request abortions.
Sample data fetching performed on the creation form only.

tonyghiani · 2025-07-03T10:23:51Z

...etail_routing/state_management/stream_routing_state_machine/routing_samples_state_machine.ts

+    conditionUpdateDebounceTime: 500,
+  },
+}).createMachine({
+  /** @xstate-layout N4IgpgJg5mDOIC5QCcD2BXALgSwHZQGUBDAWwAcAbOAYjSz0NMrgDpkwAzd2ACwG0ADAF1EoMqljYcqXKJAAPRADYAjCwDsAJiUCBSgJwqAHAGYVAViXqlAGhABPRABZLLc9aUmr6rSpMBffzs6HHxicipYWgxQxgjWdDIIIkwwAGEZCClsGUERJBBxSWlZAsUEL3MWIxUVJ2N9ARV9cxVbB2cTExYdXU16vyNzIydA4JiGcOZYFggwACMMXABjBgzcLJLqeVhMFLAWIg5U5AAKZczsmQBVJP2AEQWl5bAAFWwSMABKaPowpkisye6BWa0uJTyciKV1KoHKJnUVTq6hMTicAjM6IEljsjgQnm6mi0NRM+nq5j0+jGIBCkwBrA4YEwyx4DFmqGW6E+uEwMwoqCIWXw1AgMgOeAAbqgANYHC4UKjLTD3DlcsA82CQgrQkpycrmExGFhOKxGAyIymafS4xD6O0sO0Y7HqCwYlTU2n-eIzRnM1n4dmc7m8lj8wUMahgZBoZAsSgpDioZAkFjyxXK1XBzXCKESGF6xAG-QsPzuQxE-RKE02hBGdRG8zmJx2pxGTReTzqD0TL3TFi+lls0VB9WYDIgkNhoVQEViliSmVy1AKsBKlUjjXjnlasR53VlQtoh3qASNJ31EbWjoIPwqdRuUwmPpafp+bt-OJ9gf+qCBtU8rdJwFadI2jJM4woBMkxTNNVwzDdeUAndCj3HJYQUQ8qk8Iw22MLo-HUK88RUTQBE0Y1zErAR1CcDxdHMQIghAXBUDmeACk9T9IlzYo0ILBAsMNUxhhwusTRUGtNCMAQS10AR0TvTRtBcd9YimQE5kWEFVnwdZNj47VUJkfj0WLZsVGomjaKaPQa0sGS2jLEZsUbRtVLpb1+yZQd8B4-MDwE7psOE0SxNUGsT26RtzOaC9K00dze0Bb8h0zUd2N3XjjICizyJRFp9FMQihn0JSIsMFhsWbA1DSJQiqSYzj1IZbyfz-LNQ2Ahg-P3OFEFdDRSUoorCsosrryMfR70aVtRpafCGMansuJav1UoQmZRVwMAeoMvrayaB1SMRA1NAsk0TBrSajXrCxrFPWp9BMBKlo-ZqfVa9b-zHJZeV27L9ostRgpMEScJo8Lr1qDFjS8JSDWk9xnsSlaPrWgNh2+wC+S63zDKy9DyjqYsQbBsKa0emSntUIlaZcFoUferz0d-THg2x9ltv+wnEGk7oyIsO8alIswJKhs7yOpto2wNJGXsCIA */


elasticmachine · 2025-07-03T12:34:58Z

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

flash1293

Didn't test thoroughly yet

But it seems like there is an issue if no results are returned - it's not handled properly but shown as an error:

Investigated and this is happening in case of intermediate results of the async search, which can be an empty object.

tonyghiani · 2025-07-07T09:54:47Z

@flash1293 I could replicate the issue and simulate longer and more complex requests to handle this scenario. The type lies when it comes to partial results, but we can specify to always get partial results to get it right, and then handle the search correctly.
Here is a quick video with an increasingly more specific condition over a ~800mb dataset, it times out after 10s (we can tune this value) when it doesn't get any partial result, and it constantly gives feedback on the ongoing data fetching:

Screen.Recording.2025-07-07.at.11.49.33.mov

Kerry350

Conversion to the state machine etc looks good to me barring the ongoing discussion about partial results.

The type lies when it comes to partial results, but we can specify to always get partial results to get it right, and then handle the search correctly.

This is interesting. So I can't remember the specifics as I did the original code ages ago, but there were caveats around aggregations and using partial results. If you're always able to specify this now, that sounds good 👌

...app/public/components/data_management/stream_detail_enrichment/processor_outcome_preview.tsx

...shared/streams_app/public/components/data_management/stream_detail_routing/preview_panel.tsx

tonyghiani · 2025-07-07T13:26:14Z

@flash1293 @Kerry350 thanks for your first reviews, this should be ready for another look

flash1293 · 2025-07-08T10:02:46Z

Seems like there is still a bug here - if I change a condition that does match something to one that doesn't, the old result keeps showing instead of switching to "no results found":

tonyghiani · 2025-07-08T10:55:59Z

Seems like there is still a bug here - if I change a condition that does match something to one that doesn't, the old result keeps showing instead of switching to "no results found":

Yeah good catch, I missed a check in the response filtering, fixed!

flash1293

Tested and works fine for me now! I'll leave a detailed review of the fetching logic to Kerry, but it looks fine to me.

One note: During testing I noticed that we reset the data grid state when refreshing which is quite annoying:

But I guess we are touching this table anyway soon as part of the routing improvements, we can probably fix there

Kerry350

Thanks for the changes, LGTM 👌

kibanamachine · 2025-07-09T10:32:09Z

Starting backport for target branches: 9.1

https://github.com/elastic/kibana/actions/runs/16166971911

elasticmachine · 2025-07-09T10:32:19Z

💚 Build Succeeded

Buildkite Build
Commit: 666f7a1

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`streamsApp`	585.0KB	589.9KB	+4.9KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`streamsApp`	8	7	-1

Total ESLint disabled count

id	before	after	diff
`streamsApp`	12	11	-1

History

💚 Build #316686 succeeded cfb9eb8
💚 Build #316635 succeeded 1fc47a2
💔 Build #316612 failed d966fb1
💚 Build #316271 succeeded 833900a
💔 Build #316186 failed 799345f
💚 Build #316090 succeeded 688ef97

kibanamachine · 2025-07-09T10:38:23Z

💔 All backports failed

Status	Branch	Result
❌	9.1	Backport failed because of merge conflicts You might need to backport the following PRs to 9.1: - [Streams] Significant events view (#220197)

Manual backport

To create the backport manually run:

node scripts/backport --pr 226374

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2025-07-09T12:41:12Z

Starting backport for target branches: 9.1

https://github.com/elastic/kibana/actions/runs/16169591225

kibanamachine · 2025-07-09T12:47:28Z

💔 All backports failed

Status	Branch	Result
❌	9.1	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 226374

Questions ?

Please refer to the Backport tool documentation

…ons (elastic#226374) ## 📓 Summary Closes elastic#219494 Refactor the samples fetching logic and address the following issues: - Add a timeout (10 seconds) to avoid requests on incomplete or unmatching conditions to keep running with no feedback. - Update cancellation logic, previously was not catching correctly the request abortions. - Sample data fetching performed on the creation form only. (cherry picked from commit 00333f2) # Conflicts: # x-pack/platform/plugins/shared/streams_app/public/components/data_management/stream_detail_enrichment/processor_outcome_preview.tsx

…ellations (#226374) (#227246) # Backport This will backport the following commits from `main` to `9.1`: - [[Streams 🌊] Prevent routing simulation timeout and improve cancellations (#226374)](#226374)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

…ons (elastic#226374) ## 📓 Summary Closes elastic#219494 Refactor the samples fetching logic and address the following issues: - Add a timeout (10 seconds) to avoid requests on incomplete or unmatching conditions to keep running with no feedback. - Update cancellation logic, previously was not catching correctly the request abortions. - Sample data fetching performed on the creation form only.

tonyghiani added 7 commits June 30, 2025 16:36

refactor(streams): udpate routing data fetching

dd2cf8c

wip(streams): wire samples machine

8c8fe42

wip(streams): update samples machine

5807f04

wip(streams): update samples machine

8839654

wip(streams): update samples machine for time handling

b4379b8

feat(streams): refactor and fix data fetching

861003c

refactor(streams): simplify machine

a62fe3d

tonyghiani changed the title ~~219494 prevent routing timeout~~ [Streams 🌊] Prevent routing simulation timeout and improve cancellations Jul 3, 2025

tonyghiani commented Jul 3, 2025

View reviewed changes

tonyghiani added 3 commits July 3, 2025 12:29

refactor(streams): move code around

80591d1

refactor(streams): fix ts issue

14ef1fe

Merge branch 'main' into 219494-prevent-routing-timeout

81bed30

tonyghiani marked this pull request as ready for review July 3, 2025 12:33

tonyghiani requested review from a team as code owners July 3, 2025 12:33

tonyghiani added release_note:skip Skip the PR/issue when compiling release notes Team:obs-onboarding Observability Onboarding Team backport:version Backport to applied version labels Feature:Streams This is the label for the Streams Project v9.1.0 labels Jul 3, 2025

Kerry350 self-requested a review July 3, 2025 12:40

Merge branch 'main' into 219494-prevent-routing-timeout

4a5bb9f

flash1293 reviewed Jul 4, 2025

View reviewed changes

tonyghiani added 2 commits July 7, 2025 09:37

Merge branch 'main' into 219494-prevent-routing-timeout

688ef97

refactor(streams): udpate async search

799345f

Kerry350 reviewed Jul 7, 2025

View reviewed changes

...app/public/components/data_management/stream_detail_enrichment/processor_outcome_preview.tsx Show resolved Hide resolved

...shared/streams_app/public/components/data_management/stream_detail_routing/preview_panel.tsx Outdated Show resolved Hide resolved

tonyghiani added 2 commits July 7, 2025 12:27

refactor(streams): address request changes

3a19d8b

Merge branch 'main' into 219494-prevent-routing-timeout

833900a

tonyghiani added 3 commits July 8, 2025 09:56

Merge branch 'main' into 219494-prevent-routing-timeout

d966fb1

Update processor_outcome_preview.tsx

a85ca1b

Merge branch 'main' into 219494-prevent-routing-timeout

1fc47a2

refactor(streams): update list when not matching data

cfb9eb8

flash1293 approved these changes Jul 8, 2025

View reviewed changes

Kerry350 approved these changes Jul 9, 2025

View reviewed changes

Merge branch 'main' into 219494-prevent-routing-timeout

666f7a1

tonyghiani enabled auto-merge (squash) July 9, 2025 08:41

tonyghiani merged commit 00333f2 into elastic:main Jul 9, 2025
12 checks passed

kibanamachine added the v9.2.0 label Jul 9, 2025

tonyghiani added backport:version Backport to applied version labels and removed backport:version Backport to applied version labels labels Jul 9, 2025

tonyghiani deleted the 219494-prevent-routing-timeout branch July 9, 2025 12:41

tonyghiani mentioned this pull request Jul 9, 2025

[9.1] [Streams 🌊] Prevent routing simulation timeout and improve cancellations (#226374) #227246

Merged

Conversation

tonyghiani commented Jul 3, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📓 Summary

Uh oh!

tonyghiani Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jul 3, 2025

Uh oh!

flash1293 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tonyghiani commented Jul 7, 2025

Uh oh!

Kerry350 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tonyghiani commented Jul 7, 2025

Uh oh!

flash1293 commented Jul 8, 2025

Uh oh!

tonyghiani commented Jul 8, 2025

Uh oh!

flash1293 left a comment

Choose a reason for hiding this comment

Uh oh!

Kerry350 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Jul 9, 2025

Uh oh!

elasticmachine commented Jul 9, 2025

💚 Build Succeeded

Metrics [docs]

Async chunks

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

kibanamachine commented Jul 9, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

kibanamachine commented Jul 9, 2025

Uh oh!

kibanamachine commented Jul 9, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

tonyghiani commented Jul 3, 2025 •

edited by kibanamachine

Loading

flash1293 left a comment •

edited

Loading