[Lens] [ES|QL] Refresh ES|QL results grid on data load

[mariairiartef]

Summary

Fix #259082

How it was fixed

When the visualization finishes reloading (signaled by dataLoading$ → false), the editor re-runs getSuggestions for the last submitted ES|QL query with current time/filters so dataGridAttrs (the results section) stays in sync with what the chart just used. Additionally, suppressNextChartLoadGridRefreshRef avoids a duplicate grid fetch right after submit.

Before

Screen.Recording.2026-04-17.at.12.35.26.mov

Changing the time range or applying filters updated the visualization but did not refresh the “ES|QL query results” section.

After

Screen.Recording.2026-04-17.at.12.57.34.mov

Changing the time range or applying filters updates the visualization and refreshes the “ES|QL query results” section.

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• Flaky Test Runner was used on any tests changed
• The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
• Review the backport guidelines and apply applicable backport:* labels.

[mariairiartef]

/ci

[mariairiartef]

/ci

[elasticmachine]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 9df9a8e

Failed CI Steps

• FTR Configs #60
• FTR Configs #147
• affected Scout: [ observability / synthetics ] plugin

Test Failures

• [job] [logs] affected Scout: [ observability / synthetics ] plugin / local-stateful-classic - FilterMonitors - filters monitors by tags with AND/OR logic

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lens	2.0MB	2.0MB	+339.0B

History

• 💚 Build #431844 succeeded 1509065
• 💛 Build #431332 was flaky 95a7125
• 💔 Build #431072 failed eabcb51
• 💛 Build #430610 was flaky 051c913
• 💛 Build #430262 was flaky a14584a

cc @mariairiartef

[walterra]

Do we need the useEffect here? Maybe we can do the same like with the refs further up like:

const lensAdaptersRef = useRef(lensAdapters);
lensAdaptersRef.current = lensAdapters;

[walterra]

Maybe we can be more specific with that triggers an update? Then we don't have to track duplicates but can separate what's triggering an update. As far as I could see searchSessionId is in redux what tracks external context change (time range, filters, auto-refresh, manual refresh), so it would cover triggers besides the editor submit. So we could do something like the following:

const searchSessionId = useLensSelector(selectSearchSessionId);
  const isInitialRenderRef = useRef(true);

  useEffect(() => {
    // Skip the initial render — the grid is populated by useInitializeChart → runQuery
    if (isInitialRenderRef.current) {
      isInitialRenderRef.current = false;
      return;
    }
    const lastSubmittedQuery = submittedQueryRef.current;
    if (isOfAggregateQueryType(lastSubmittedQuery)) {
      getSuggestions(
        lastSubmittedQuery,
        data,
        http,
        uiSettings,
        datasourceMap,
        visualizationMap,
        adHocDataViews,
        undefined,
        undefined,
        setDataGridAttrs,
        esqlVariables,
        false,
        currentAttributesRef.current
      ).catch(() => {
        // Grid refresh is best-effort; errors here (e.g. expired time range) are non-fatal.
        // The chart itself will surface query errors via its own error handling path.
      });
    }
  }, [
    searchSessionId,
    esqlVariables,
    data,
    http,
    uiSettings,
    datasourceMap,
    visualizationMap,
    adHocDataViews,
  ]);

We could then use useObservable() instead of managing the subscription ourselves. Like:

// Update activeData and column limit indicator when chart data finishes loading
  const isDataLoading = useObservable(dataLoading$ ?? EMPTY);

  useEffect(() => {
    if (isDataLoading !== false) {
      return;
    }

    const activeData = getActiveDataFromDatatable(layerId, lensAdaptersRef.current?.tables?.tables);

    const table = activeData?.[layerId];

    if (table) {
      // there are cases where a query can return a big amount of columns
      // at this case we don't suggest all columns in a table but the first `MAX_NUM_OF_COLUMNS`
      setSuggestsLimitedColumns(table.columns.length >= MAX_NUM_OF_COLUMNS);
    }

    if (Object.keys(activeData).length > 0) {
      dispatch(onActiveDataChange({ activeData }));
    }
  }, [isDataLoading, dispatch, layerId]);

We won't get rid of refs completely but it's less nested/complex than what suppressNextChartLoadGridRefreshRef tries to do maybe. What do you think?

[mariairiartef]

I like you idea! It makes sense to make it less complicated. Applied your suggestion here

[mariairiartef]

Nit

I've noticed that when you run the query, the "ES|QL Query Results" section is collapsed:

Screen.Recording.2026-05-05.at.16.39.23.mov

@teresaalvarezsoler, do you think we should do the same when the ES|QL query results section updates on date / filters change?

[walterra]

Latest changes LTGM!

[kibanamachine]

💚 Build Succeeded

• Buildkite Build
• Commit: df23d72

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
lens	1749	1751	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lens	2.0MB	2.0MB	+758.0B

History

• 💔 Build #438247 failed 46b9cf4
• 💛 Build #437895 was flaky 96fa9b2

cc @mariairiartef