[Uptime] Update monitor saved object mappings by shahzad31 · Pull Request #130433 · elastic/kibana

shahzad31 · 2022-04-17T09:02:43Z

Summary

It updates the default field name, urls, and type mapping from keyword to text, while adding .keyword fields to support sorting by each field.

This also makes importableAndExportable false, since these saved objects contains secrets and import/export without decryption doesn't makes sense.

Testing

Add the following keys to Kibana.yml https://p.elstc.co/paste/-uRAA0Lw#b4xgrKoWb6yEPQgW+JitoB4WN5eTLV8hLEEaavaV4qA
Navigate to Monitor Management. Click Enable Monitor Management
Add at least 4 monitors, one of each type, with mock data
Navigate back to Monitor Management. Ensure that no Kibana errors are present
On the Monitor Management List page, click the Type column heading. Ensure that monitors are sorted by type
Click the URL column heading. Ensure the monitors are sorted by url. Note: There is a known bug where sorting does not work correctly for tcp and icmp monitors. Issue is pending.
Click the Name column heading. Ensure the monitors are sorted by name.
Navigate to Saved Object Management. Ensure that synthetics monitors are not exportable
Search within Saved Objects. Ensure that no Kibana errors are present.

shahzad31 · 2022-04-17T09:08:44Z

x-pack/plugins/uptime/server/lib/saved_objects/synthetics_monitor.ts

      name: {
-        type: 'keyword',
+        type: 'text',
+        fielddata: true,


without settings this, it would raise this error

[2022-04-17T11:08:06.011+02:00][ERROR][http] ResponseError: all shards failed: search_phase_execution_exception: [illegal_argument_exception] Reason: Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [synthetics-monitor.name] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

Why do we not need fielddata for other properties, do you know?

@jportner any idea about this?

I'm actually reviewing this now. I can see the error is related to the sorting feature in the monitor management list, and text fields not being optimized for sorting. This error is actually present when attempting to sort on the other fields we enable sorting for, urls and type, but does not apepar until you attempt to sort by those fields.

I am currently resolving this issue using multi fields and ensuring that sorting always uses the .keyword property. I will push directly to the branch.

dominiqueclarke · 2022-04-18T02:09:21Z

x-pack/plugins/uptime/server/lib/saved_objects/synthetics_monitor.ts

-        type: 'keyword',
+        type: 'text',
      },
      secrets: {


This is my fault, but can we remove this? There is no reason to have secrets indexed.

dominiqueclarke · 2022-04-18T02:10:01Z

x-pack/plugins/uptime/server/lib/saved_objects/synthetics_monitor.ts

      name: {
-        type: 'keyword',
+        type: 'text',
+        fielddata: true,


Why do we not need fielddata for other properties, do you know?

dominiqueclarke · 2022-04-18T02:11:16Z

x-pack/plugins/uptime/server/lib/saved_objects/synthetics_monitor.ts

      tags: {
-        type: 'keyword',
+        type: 'text',
      },


We might want to review upcoming feature requests for searching, and ensure that anything we want to search in the future has mappings.

One thing that comes to mind, for example, is location name.

That being said, we may end up simply converting Uptime saved objects to Synthetics saved objects in the future, and Uptime may never get full search and filter features

We might want to review upcoming feature requests for searching, and ensure that anything we want to search in the future has mappings.

You can update the mappings after the fact.
For this release, I'd recommend omitting all fields that don't need to be searched in this release.

See https://docs.elastic.dev/kibana-dev-docs/tutorials/saved-objects#mappings for more details.

dominiqueclarke · 2022-04-18T02:11:56Z

@elasticmachine merge upstream

elasticmachine · 2022-04-18T10:17:31Z

Pinging @elastic/uptime (Team:uptime)

dominiqueclarke · 2022-04-18T12:10:10Z

@elasticmachine merge upstream

…pping

justinkambic · 2022-04-18T20:53:25Z

I noted this offline to @dominiqueclarke but aside from the sort not working between individual http/browser monitors, everything seems to fall within the parameters expected by her instructions.

Here's a GIF of me "sorting" the monitors:

When I added another http monitor, the URLs did sort based on my input.

kibana-ci · 2022-04-19T00:42:25Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`uptime`	777.2KB	777.2KB	+83.0B

History

💔 Build #38837 failed add58f0
💔 Build #38818 failed b183f87
💚 Build #38707 succeeded 336cd72
💔 Build #38646 failed 3a4b61f
💔 Build #38613 failed 6b53ad4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

shahzad31 · 2022-04-19T07:59:50Z

Everything seems to be working as expected including sorting.

lucasfcosta

I have reviewed the code as per @shahzad31's request (after reading this super helpful explanation) and it LGTM.

I haven't E2E this PR as per Shahzad's request considering this PR has already been tested.

kibanamachine · 2022-04-19T08:24:46Z

💔 All backports failed

Status	Branch	Result
❌	8.2	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 130433

Questions ?

Please refer to the Backport tool documentation

* update mappings * PR feedback * update * update mappings * update mappings * port and hosts * adjust jest test * adjust mappings for sorting by url, name, and type * remove unused import * synthetics - do not make synthetics monitors exportable * adjust tests Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co> (cherry picked from commit a843db3)

…disable-server-side * 'main' of github.com:elastic/kibana: (103 commits) [Osquery] Update eslint config (elastic#129637) [Uptime] Update monitor saved object mappings (elastic#130433) Add links to metricbeat module docs (elastic#130519) Add link to troubleshooting guide in confirm data copy (elastic#130420) [Step 3] Cleanup charts plugin (elastic#130132) [Visualize] Adds a deprecation warning to the pie app (elastic#130447) [Maps] fix vector tile load errors not displayed in legend (elastic#130395) [CI] Split alerting-api-integration tests into separate cigroups (elastic#130414) [CI] Use spot instances for default cigroups in PR CI (elastic#130476) [functional-tests] TimePicker optimizations (elastic#130200) [kbn/pm] use stable module ids in dist (elastic#130497) [8.2.1][Security Solution][Session view] fix full screen session view margin (elastic#130496) Fix wrong config in comments (elastic#130378) Add deprecated telemetry (elastic#130458) Add eslint rule to support breaking up packages (elastic#130483) [Security Solution][Endpoint] Fix test stability and un-skip flaky tests (elastic#130176) Update object types for SharePoint Online external connector (elastic#130478) [Workplace Search] Fix broken feedback link (elastic#130475) Rename the term "execution" in config to "run" (elastic#130172) [Cloud Posture] use index with keyword mapping (elastic#130456) ... # Conflicts: # docs/user/reporting/index.asciidoc # x-pack/plugins/reporting/public/types.ts # x-pack/plugins/screenshotting/server/screenshots/index.test.ts # x-pack/plugins/screenshotting/server/screenshots/index.ts

* update mappings * PR feedback * update * update mappings * update mappings * port and hosts * adjust jest test * adjust mappings for sorting by url, name, and type * remove unused import * synthetics - do not make synthetics monitors exportable * adjust tests Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co> (cherry picked from commit a843db3)

* update mappings * PR feedback * update * update mappings * update mappings * port and hosts * adjust jest test * adjust mappings for sorting by url, name, and type * remove unused import * synthetics - do not make synthetics monitors exportable * adjust tests Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co>

update mappings

8ae276d

shahzad31 added v8.3.0 v8.2.0 release_note:skip Skip the PR/issue when compiling release notes labels Apr 17, 2022

shahzad31 mentioned this pull request Apr 17, 2022

8.2.0 BC1: "Bad Request" error when searching saved objects #129424

Closed

shahzad31 commented Apr 17, 2022

View reviewed changes

dominiqueclarke reviewed Apr 18, 2022

View reviewed changes

kibanamachine and others added 3 commits April 17, 2022 21:11

Merge branch 'main' into fix-mapping

f851fb8

PR feedback

9e95489

Merge branch 'main' of github.com:elastic/kibana into fix-mapping

e3ca5e0

shahzad31 marked this pull request as ready for review April 18, 2022 10:12

shahzad31 requested a review from a team as a code owner April 18, 2022 10:12

update

6b53ad4

botelastic bot added the Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability label Apr 18, 2022

kibanamachine and others added 9 commits April 18, 2022 07:10

Merge branch 'main' into fix-mapping

a49d93b

update mappings

ba2d07f

update mappings

a7a3460

Merge branch 'fix-mapping' of github.com:shahzad31/kibana into fix-ma…

0450d5f

…pping

port and hosts

3a4b61f

adjust jest test

336cd72

adjust mappings for sorting by url, name, and type

b183f87

remove unused import

d9fcc8f

synthetics - do not make synthetics monitors exportable

add58f0

dominiqueclarke added the bug Fixes for quality problems that affect the customer experience label Apr 18, 2022

adjust tests

6a0dce9

lucasfcosta approved these changes Apr 19, 2022

View reviewed changes

shahzad31 merged commit a843db3 into elastic:main Apr 19, 2022

shahzad31 deleted the fix-mapping branch April 19, 2022 08:19

shahzad31 added the auto-backport Deprecated - use backport:version if exact versions are needed label Apr 19, 2022

shahzad31 mentioned this pull request Apr 19, 2022

[8.2] [Uptime] Update monitor saved object mappings (#130433) #130536

Merged

dominiqueclarke mentioned this pull request Apr 19, 2022

[Uptime] monitor management - adjust sort field map for inline monitor errors #130606

Merged

Conversation

shahzad31 commented Apr 17, 2022 • edited by dominiqueclarke Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

shahzad31 Apr 17, 2022

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke Apr 18, 2022

Choose a reason for hiding this comment

Uh oh!

shahzad31 Apr 18, 2022

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke Apr 18, 2022

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke Apr 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke Apr 18, 2022

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke Apr 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jportner Apr 18, 2022

Choose a reason for hiding this comment

Uh oh!

dominiqueclarke commented Apr 18, 2022

Uh oh!

elasticmachine commented Apr 18, 2022

Uh oh!

dominiqueclarke commented Apr 18, 2022

Uh oh!

justinkambic commented Apr 18, 2022

Uh oh!

kibana-ci commented Apr 19, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

History

Uh oh!

shahzad31 commented Apr 19, 2022

Uh oh!

lucasfcosta left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Apr 19, 2022

💔 All backports failed

Manual backport

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

shahzad31 commented Apr 17, 2022 •

edited by dominiqueclarke

Loading

dominiqueclarke Apr 18, 2022 •

edited

Loading

dominiqueclarke Apr 18, 2022 •

edited

Loading