🌊 LLM-powered parsing suggestions

[flash1293]

Depends on #209985

Add suggestions for grok processing:

The logic for generating suggestions works like this:

• Take the current sample
• Split it into patterns based on a simple regex-based grouping replacing runs of numbers with a placeholder, runs of regular numbers with a placeholder, etc.
• For the top 5 found groups, pass a couple messages to the LLM in parallel to come up with a grok pattern
• Check the grok patterns whether they actually match something and don't break
• Report the patterns that have a positive match rate

For the Generate patterns button to show in the UI, make sure a connector is configured and the license level is above basic (trial license is easiest to test with).

I did some light refactoring on the processing routes, moving the simulation bits into a separate file - no changes in this area though.

[flash1293]

@tonyghiani I think I addressed all points raised except for the advertisement of the AI feature if it's not enabled. I'm going to add that in a follow-up PR

This is how the screen looks when no suggestions could be found:

[flash1293]

You probably want to wait with another round of review until #209985 is merged (I pulled it in here and the diffs are mixed now)

[flash1293]

@tonyghiani should be rebased with main - there are two things missing that I will address in a follow-up:

• Unit tests for the suggestions handler
• CTA if no LLM connector is available, but the user has permissions to configure it

[tonyghiani]

Looks good, there are some client side parts that will probably change with the state management refactor, but I'll handle that once I rebase this work into my WIP changes.

Agree on having some API test for the suggestions, there is a lot of logic going on there and having a test safety guard seems very necessary.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6b3ce93
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-208777-6b3ce93ddc78

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
streamsApp	308	310	+2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/streams-schema	266	268	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
streamsApp	287.4KB	293.1KB	+5.7KB

Unknown metric groups

API count

id	before	after	diff
@kbn/streams-schema	269	271	+2

History

• 💔 Build #277730 failed 44cba3c
• 💔 Build #277364 failed 288dc3f
• 💔 Build #277327 failed 1740676
• 💛 Build #275066 was flaky 6659dfc
• 💔 Build #275042 failed 996573e

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/13430306734

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [streams] lifecycle - ingestion and total docs metadata (#210301)

Manual backport

To create the backport manually run:

node scripts/backport --pr 208777

Questions ?

Please refer to the Backport tool documentation