[App Search] Refactor empty engine polling to EngineLogic by cee-chen · Pull Request #103041 · elastic/kibana

cee-chen · 2021-06-23T02:54:01Z

Summary

This PR accomplishes several things:

It addresses [App Search] Convert Engine subnav and Engine Overview pages to new page template #102679 (comment) and removes a full-page loading return. Now on the engine overview page, the page title for either Engine setup or Engine overview loads in immediately for a snappier feel.
The Engine Overview should still update/change when empty and new document(s) are indexed for the first time, but should no longer poll for engines that are not empty (OK'd by Casey)
The Documents view should now also automatically refresh/update when empty and new document(s) are indexed for the first time (https://github.com/elastic/app-search-team/issues/1345). This came basically for free, which was neat.
Non-empty engines do not poll for engine updates.

I strongly recommend following along by commit and checking out the commit messages.

Screencaps

Engine Overview:

Documents:

Search UI:

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios

+ update selector tests with mock engines to just an `engine` var (I saw Jason do this down below for `searchKey` and liked it, so I copied it) + update Documents & Search UI pages to new selectors

- Per Casey, polling is primarily only needed to dynamically update from empty state to non-empty state, which we're going to handle at the engine level in the next commit

- Now that both Engines Overview, Documents, and Search UI views have empty states that are hooked up to EngineLogic, this poll will update all of the above pages automatically when a new document comes in!

- to match other create page header buttons

cee-chen · 2021-06-23T03:02:38Z

...plugins/enterprise_search/public/applications/app_search/components/engine/engine_router.tsx

+    pollEmptyEngine();
+
+    return () => {
+      stopPolling();
+      clearEngine();
+    };


Here's a screencap of me QAing that polling correctly stops/starts when switching between engines:

Doh, the gif cut off so here's a 2nd screencap of me QAing that polling completely stops when navigating away from an empty engine:

cee-chen · 2021-06-23T03:14:47Z

...k/plugins/enterprise_search/public/applications/app_search/components/engine/engine_logic.ts

+          flashErrorToast(POLLING_ERROR_TITLE, {
+            text: POLLING_ERROR_TEXT,
+            toastLifeTimeMs: POLLING_DURATION * 0.75,
+          });


Here's a screencap of this toast behavior:

Quick UX note: This toast won't appear when Enterprise Search is down (because then the app 502s and the generic Error Connecting view gets shown). Instead, this toast occurs when Kibana is down and the user has a stale/cached frontend (hence the screencap briefly showing my terminal, where I stop Kibana). It's fairly unlikely and 404/401 redirects are going to be the most common case, but I thought I should add in something as a generic 5xx catch.

cee-chen · 2021-06-23T03:17:02Z

@elasticmachine merge upstream

JasonStoltz · 2021-06-23T18:07:56Z

...plugins/enterprise_search/public/applications/app_search/components/engine/engine_router.tsx

    setEngineName(engineNameFromUrl);
    initializeEngine();
-    return clearEngine;
+    pollEmptyEngine();


Did you consider making pollEmptyEngine a side effect of calling initializeEngine?

Currently, the UI needs to call initializeEngine, call pollEmptyEngine, and also call stopPolling. It'd be great if the UI could just call initializeEngine and the polling automatically started and stopped when appropriate.

initializeEngine:

Fetch Engine data

If the Engine is empty, and not already polling, start polling

If the engine is not empty, stop polling

You could use an unmount hook in Kea to ensure that polling is cancelled whenever the logic unmounts: https://kea.js.org/docs/guide/additional/#events

Or better yet

Maybe we don't even need to consider the polling state...

initializeEngine:

Fetch Engine data

If the Engine is empty, wait x second and call initializeEngine again

Just thinking out loud.

I always personally waffle between making things more explicit and easier for devs to follow in code, or making things "magical" (side effects) and less code but potentially harder to follow when reading/debugging.

You could use an unmount hook in Kea to ensure that polling is cancelled whenever the logic unmounts: https://kea.js.org/docs/guide/additional/#events

FYI, we can't do this. I tried this initially, and what's happening is EngineLogic is not fully unmounting between engine changes (primarily meta engine->source engine navigation), it's only clearing state, so the unmount hook does not get fired. You do need to very explicitly call a specific action in useEffect's return. If you skip this call you end up with intervals not properly getting cleared when switching engines.

if the Engine is empty, wait x second and call initializeEngine again

Going to strongly push back against using timeouts over setting an interval. We've done this in the past and it has previously lead to duplicate API calls - we had a discussion in Slack a while back about it, and most of the frontend devs who replied were in favor of moving away from setTimeout for polling to setInterval. In additional, it's a significantly clearer from an API signal standpoint that a poll is happening when you use setInterval over setTimeout.

Ah, yeah. We ran into that issue before. Marius's recommendation was to key the logic and use BindLogic so that it gets reset when the engine changes.

Anyway, this still looks like it could be simplified to me, but you're in this deeper than me though, I'm not familiar with all of the guts and history, so if you think this is the best path then I'm fine with it.

Ahh that would definitely solve that issue & I'd super be down to key EngineLogic sometime! Probably going to punt on it for 7.14 FF for the sake of time if that's cool (currently focusing on UI/UX polish over strictly dev/tech debt only items). I've made a backlog item though: https://github.com/elastic/app-search-team/issues/1853

I think keying actually lets us refactor out the "is engine stale" check also which is super cool. Definitely wanna do this for 7.15 😎

Totally! Also, wanted to say I really like the way you implemented it. Putting it in the EngineLogic is dope, we can take advantage of that in like every view without actually doing anything. Super elegant.

JasonStoltz

This LGTM, but I feel like there could be potential to simplify a bit. Left a suggestion, let me know what you think.

...gins/enterprise_search/public/applications/app_search/components/engine/engine_logic.test.ts

Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com>

cee-chen · 2021-06-23T20:21:29Z

@elasticmachine merge upstream

cee-chen · 2021-06-23T22:36:58Z

@elasticmachine merge upstream

cee-chen · 2021-06-24T02:58:29Z

@elasticmachine merge upstream

kibanamachine · 2021-06-24T05:04:06Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`enterpriseSearch`	1455	1456	+1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`enterpriseSearch`	2.1MB	2.1MB	+1.1KB

History

💔 Build #133735 failed 308a880
💚 Build #133416 succeeded 96c9319
💔 Build #133414 failed fd62872

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @JasonStoltz

…3041) * Set up isEngineEmpty and isEngineSchemaEmpty selectors + update selector tests with mock engines to just an `engine` var (I saw Jason do this down below for `searchKey` and liked it, so I copied it) + update Documents & Search UI pages to new selectors * Update EngineOverview to use isEngineEmpty + remove polling - Per Casey, polling is primarily only needed to dynamically update from empty state to non-empty state, which we're going to handle at the engine level in the next commit * Add empty engine polling behavior - Now that both Engines Overview, Documents, and Search UI views have empty states that are hooked up to EngineLogic, this poll will update all of the above pages automatically when a new document comes in! * [Misc UI polish] Add (+) icon to Index documents button - to match other create page header buttons * [PR feedback] Test improvements Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com>

kibanamachine · 2021-06-24T05:06:30Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

…103205) * Set up isEngineEmpty and isEngineSchemaEmpty selectors + update selector tests with mock engines to just an `engine` var (I saw Jason do this down below for `searchKey` and liked it, so I copied it) + update Documents & Search UI pages to new selectors * Update EngineOverview to use isEngineEmpty + remove polling - Per Casey, polling is primarily only needed to dynamically update from empty state to non-empty state, which we're going to handle at the engine level in the next commit * Add empty engine polling behavior - Now that both Engines Overview, Documents, and Search UI views have empty states that are hooked up to EngineLogic, this poll will update all of the above pages automatically when a new document comes in! * [Misc UI polish] Add (+) icon to Index documents button - to match other create page header buttons * [PR feedback] Test improvements Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com> Co-authored-by: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com>

cee-chen added 4 commits June 22, 2021 18:49

Set up isEngineEmpty and isEngineSchemaEmpty selectors

a423a04

+ update selector tests with mock engines to just an `engine` var (I saw Jason do this down below for `searchKey` and liked it, so I copied it) + update Documents & Search UI pages to new selectors

Update EngineOverview to use isEngineEmpty + remove polling

c6da0d8

- Per Casey, polling is primarily only needed to dynamically update from empty state to non-empty state, which we're going to handle at the engine level in the next commit

Add empty engine polling behavior

fba82ef

- Now that both Engines Overview, Documents, and Search UI views have empty states that are hooked up to EngineLogic, this poll will update all of the above pages automatically when a new document comes in!

[Misc UI polish] Add (+) icon to Index documents button

fd62872

- to match other create page header buttons

cee-chen added release_note:skip Skip the PR/issue when compiling release notes v7.14.0 auto-backport Deprecated - use backport:version if exact versions are needed labels Jun 23, 2021

cee-chen requested a review from a team June 23, 2021 02:54

cee-chen commented Jun 23, 2021

View reviewed changes

Merge branch 'master' into empty-engine-polling

96c9319

JasonStoltz self-assigned this Jun 23, 2021

JasonStoltz reviewed Jun 23, 2021

View reviewed changes

...gins/enterprise_search/public/applications/app_search/components/engine/engine_logic.test.ts Outdated Show resolved Hide resolved

...gins/enterprise_search/public/applications/app_search/components/engine/engine_logic.test.ts Show resolved Hide resolved

cee-chen commented Jun 23, 2021

View reviewed changes

...gins/enterprise_search/public/applications/app_search/components/engine/engine_logic.test.ts Show resolved Hide resolved

[PR feedback] Test improvements

308a880

Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com>

JasonStoltz approved these changes Jun 23, 2021

View reviewed changes

cee-chen enabled auto-merge (squash) June 23, 2021 19:12

Merge branch 'master' into empty-engine-polling

36565b0

Merge branch 'master' into empty-engine-polling

e0dc548

Merge branch 'master' into empty-engine-polling

346f0ad

cee-chen merged commit 023b163 into elastic:master Jun 24, 2021

cee-chen deleted the empty-engine-polling branch June 24, 2021 05:05

kibanamachine mentioned this pull request Jun 24, 2021

[7.x] [App Search] Refactor empty engine polling to EngineLogic (#103041) #103205

Merged

Conversation

cee-chen commented Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Screencaps

Engine Overview:

Documents:

Search UI:

Checklist

Uh oh!

cee-chen Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cee-chen Jun 23, 2021

Choose a reason for hiding this comment

Uh oh!

cee-chen Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cee-chen commented Jun 23, 2021

Uh oh!

JasonStoltz Jun 23, 2021

Choose a reason for hiding this comment

Uh oh!

cee-chen Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JasonStoltz Jun 23, 2021

Choose a reason for hiding this comment

Uh oh!

cee-chen Jun 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JasonStoltz Jun 23, 2021

Choose a reason for hiding this comment

Uh oh!

JasonStoltz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cee-chen commented Jun 23, 2021

Uh oh!

cee-chen commented Jun 23, 2021

Uh oh!

cee-chen commented Jun 24, 2021

Uh oh!

kibanamachine commented Jun 24, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

History

Uh oh!

kibanamachine commented Jun 24, 2021

💚 Backport successful

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cee-chen commented Jun 23, 2021 •

edited

Loading

cee-chen Jun 23, 2021 •

edited

Loading

cee-chen Jun 23, 2021 •

edited

Loading

cee-chen Jun 23, 2021 •

edited

Loading

cee-chen Jun 23, 2021 •

edited

Loading