[Lens] Suggestion performance

[flash1293]

Based on #43688 and #43865

Improves the performance of suggestions by not re-running the expression if it didn't change. Also debounces the preview instead of the suggestion to show the user other options as fast as possible.

We could add a fallback to the icon as long as the expression loading hasn't finished here.

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[flash1293]

@chrisdavies @wylieconlon I do not intend to merge this PR as is (note the draft status), but this PR shown an approach to keep the suggestions from re-rendering until something actually changed.

The actual preview renderer is memoized, making it necessary for the expression string to change for re-rendering.

To show new suggestions as early as possible, only the preview component is debounced, not the suggestion list itself.

To keep the expression string stable across multiple state changes, generateId is made deterministic by using a local counter. By resetting the counter before creating the suggestions, the exact same suggestion is produced as long as nothing relevant for the suggestions changed. This comes with some drawbacks, because generateId is not controlled by the frame but the extension itself. Each plugin has to take care of resetting the generator appropriately and has to pass in currently used column ids to avoid duplicates. By moving the id generator to the frame and passing it down into the suggestion functions, the logic about resetting and duplicate-avoidance can be solved there. @chrisdavies would this be possible with your current refactoring work about state handling?

Maybe you have a better idea how to tackle this.

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[flash1293]

After syncing with @chrisdavies most of these problems will disappear if we expose an addColumn function in the datasource public api the visualization can call. Then the generation of unique ids in a given scope is an implementation detail of the datasource and the visualization never has to do this.

[elasticmachine]

Pinging @elastic/kibana-app

[flash1293]

@chrisdavies Do you plan to refactor the usage of state setting before feature freeze to make this possible or should we find an interim solution?

[flash1293]

Closing this as it will get addressed later