Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch) by mikecote · Pull Request #130819 · elastic/kibana

mikecote · 2022-04-21T18:27:13Z

In this PR, I'm making event.duration a string in the TypeScript definition while keeping it a long within Elasticsearch. This forces Kibana to work with these values as strings / BigInt while letting Elasticsearch properly parse it and store it as a number. The aggregations on event.duration still return number values after this change.

The event log HTTP APIs will return a string for event.duration but these APIs are only used by the functional test and have been fixed accordingly. The event log findEventsBySavedObjectIds API has type safety and has been fixed accordingly.

…icsearch

elasticmachine · 2022-04-26T20:10:21Z

Pinging @elastic/response-ops (Team:ResponseOps)

x-pack/plugins/event_log/generated/schemas.ts

mikecote · 2022-04-29T12:26:45Z

@elasticmachine merge upstream

x-pack/plugins/alerting/server/task_runner/task_runner.ts

x-pack/plugins/event_log/common/lib/millis_to_nanos.ts

x-pack/plugins/event_log/server/event_logger.ts

ymao1

LGTM!

ersin-erdal

LGTM

mikecote · 2022-05-03T17:38:08Z

@elasticmachine merge upstream

kibana-ci · 2022-05-03T18:54:11Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`eventLog`	91	99	+8

Unknown metric groups

API count

id	before	after	diff
`eventLog`	91	99	+8

History

💚 Build #41532 succeeded ca8902c
💚 Build #41508 succeeded 082120b
💚 Build #41441 succeeded 4a0dbdb
💔 Build #41433 failed 91fd4f8
💔 Build #40695 failed d433f75
💚 Build #39917 succeeded e6ff58a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @mikecote

* Add severity field to create API and migration * Adds integration test for severity field migration * remove exclusive test * Change severity levels * Update integration tests for post case * Add more integration tests * Fix all cases list test * Fix some server test * Fix util server test * Fix client util test * Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch) (#130819) * Convert event.duration to string in TypeScript, keep as long in Elasticsearch * Fix jest test * Fix functional tests * Add ecsStringOrNumber to event log schema * Fix jest test * Add utility functions to event log plugin * Use new event log utility functions * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * filter o11y rule aggregations (#131301) * [Cloud Posture] Display and save rules per benchmark (#131412) * Adding aria-label for discover data grid select document checkbox (#131277) * Update API docs (#130999) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * [CI] Use GCS buckets for bazel remote caching (#131345) * [Actionable Observability] Add license modal to rules table (#131232) * Add fix license link * fix localization * fix CI error * fix more translation issues Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * [RAM] Add shareable rule status filter (#130705) * rule state filter * turn off experiment * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Status filter API call * Fix tests * rename state to status, added tests * Address comments and fix tests * Revert experiment flag * Remove unused translations * Addressed comments Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> * [storybook] Watch for changes in packages (#131467) * [storybook] Watch for changes in packages * Update default_config.ts * Improve saved objects migrations failure errors and logs (#131359) * [Unified observability] Add tour step to guided setup (#131149) * [Lens] Improved interval input (#131372) * [Vega] Adjust vega doc for usage of ems files (#130948) * adjust vega doc * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Excess intersections * Create severity user action * Add severity to create_case user action * Fix and add integration tests * Minor improvements Co-authored-by: Mike Côté <mikecote@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: mgiota <panagiota.mitsopoulou@elastic.co> Co-authored-by: Jordan <51442161+JordanSh@users.noreply.github.com> Co-authored-by: Bhavya RM <bhavya@elastic.co> Co-authored-by: Thomas Neirynck <thomas@elastic.co> Co-authored-by: Brian Seeders <brian.seeders@elastic.co> Co-authored-by: Jiawei Wu <74562234+JiaweiWu@users.noreply.github.com> Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co> Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co> Co-authored-by: Alejandro Fernández Gómez <alejandro.fernandez@elastic.co> Co-authored-by: Joe Reuter <johannes.reuter@elastic.co> Co-authored-by: Nick Peihl <nickpeihl@gmail.com> Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>

… "long" in Elasticsearch) (elastic#130819) * Convert event.duration to string in TypeScript, keep as long in Elasticsearch * Fix jest test * Fix functional tests * Add ecsStringOrNumber to event log schema * Fix jest test * Add utility functions to event log plugin * Use new event log utility functions * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* Add severity field to create API and migration * Adds integration test for severity field migration * remove exclusive test * Change severity levels * Update integration tests for post case * Add more integration tests * Fix all cases list test * Fix some server test * Fix util server test * Fix client util test * Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch) (elastic#130819) * Convert event.duration to string in TypeScript, keep as long in Elasticsearch * Fix jest test * Fix functional tests * Add ecsStringOrNumber to event log schema * Fix jest test * Add utility functions to event log plugin * Use new event log utility functions * PR fixes Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * filter o11y rule aggregations (elastic#131301) * [Cloud Posture] Display and save rules per benchmark (elastic#131412) * Adding aria-label for discover data grid select document checkbox (elastic#131277) * Update API docs (elastic#130999) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * [CI] Use GCS buckets for bazel remote caching (elastic#131345) * [Actionable Observability] Add license modal to rules table (elastic#131232) * Add fix license link * fix localization * fix CI error * fix more translation issues Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * [RAM] Add shareable rule status filter (elastic#130705) * rule state filter * turn off experiment * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * Status filter API call * Fix tests * rename state to status, added tests * Address comments and fix tests * Revert experiment flag * Remove unused translations * Addressed comments Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> * [storybook] Watch for changes in packages (elastic#131467) * [storybook] Watch for changes in packages * Update default_config.ts * Improve saved objects migrations failure errors and logs (elastic#131359) * [Unified observability] Add tour step to guided setup (elastic#131149) * [Lens] Improved interval input (elastic#131372) * [Vega] Adjust vega doc for usage of ems files (elastic#130948) * adjust vega doc * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Update docs/user/dashboard/vega-reference.asciidoc Co-authored-by: Nick Peihl <nickpeihl@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Nick Peihl <nickpeihl@gmail.com> * Excess intersections * Create severity user action * Add severity to create_case user action * Fix and add integration tests * Minor improvements Co-authored-by: Mike Côté <mikecote@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: mgiota <panagiota.mitsopoulou@elastic.co> Co-authored-by: Jordan <51442161+JordanSh@users.noreply.github.com> Co-authored-by: Bhavya RM <bhavya@elastic.co> Co-authored-by: Thomas Neirynck <thomas@elastic.co> Co-authored-by: Brian Seeders <brian.seeders@elastic.co> Co-authored-by: Jiawei Wu <74562234+JiaweiWu@users.noreply.github.com> Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co> Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co> Co-authored-by: Alejandro Fernández Gómez <alejandro.fernandez@elastic.co> Co-authored-by: Joe Reuter <johannes.reuter@elastic.co> Co-authored-by: Nick Peihl <nickpeihl@gmail.com> Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>

…eep as "long" in Elasticsearch) (#144837) Fixes #143562. Backport of #130819 to `7.x`. ## Steps to verify 1. Fake alert event log durations to be 1 year by changing the following code >https://github.com/elastic/kibana/pull/144837/files#diff-27ff694dc534e9e509d43b00c563eabada722db2371c9bcf016ff3c8b929c5fbR706 >const duration = millisToNanos(31556926000); 2. Create a rule that creates an alert consistently (ex: index threshold) 3. Let the rule run a few times 4. See event log now capturing long durations successfully >curl -XGET 'https://elastic:changeme@localhost:9200/.kibana-event-log*/_search?pretty' -k Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

mikecote added 3 commits April 21, 2022 13:55

Convert event.duration to string in TypeScript, keep as long in Elast…

b21c4db

…icsearch

Fix jest test

907c5c8

Fix functional tests

e6ff58a

mikecote added release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// Feature:EventLog Feature:Alerting/RulesFramework Issues related to the Alerting Rules Framework v8.3.0 labels Apr 21, 2022

mikecote self-assigned this Apr 21, 2022

Merge branch 'main' of github.com:elastic/kibana into fix-127802

d433f75

mikecote marked this pull request as ready for review April 26, 2022 20:10

mikecote requested a review from a team as a code owner April 26, 2022 20:10

ymao1 reviewed Apr 27, 2022

View reviewed changes

x-pack/plugins/event_log/generated/schemas.ts Outdated Show resolved Hide resolved

mikecote added 2 commits April 28, 2022 15:28

Merge branch 'main' of github.com:elastic/kibana into fix-127802

6420060

Add ecsStringOrNumber to event log schema

0b61084

Merge branch 'main' into fix-127802

91fd4f8

mikecote requested a review from ymao1 April 29, 2022 12:29

mikecote added 2 commits April 29, 2022 08:57

Fix jest test

0b49582

Merge branch 'fix-127802' of github.com:mikecote/kibana into fix-127802

4a0dbdb

ersin-erdal reviewed Apr 29, 2022

View reviewed changes

x-pack/plugins/alerting/server/task_runner/task_runner.ts Outdated Show resolved Hide resolved

mikecote added 2 commits April 29, 2022 10:46

Add utility functions to event log plugin

ea053d6

Use new event log utility functions

082120b

mikecote requested a review from ersin-erdal April 29, 2022 15:12

ymao1 reviewed Apr 29, 2022

View reviewed changes

x-pack/plugins/event_log/common/lib/millis_to_nanos.ts Outdated Show resolved Hide resolved

ymao1 reviewed Apr 29, 2022

View reviewed changes

x-pack/plugins/event_log/server/event_logger.ts Outdated Show resolved Hide resolved

PR fixes

ca8902c

mikecote requested a review from ymao1 April 29, 2022 16:46

ymao1 approved these changes Apr 29, 2022

View reviewed changes

ersin-erdal approved these changes May 2, 2022

View reviewed changes

Merge branch 'main' into fix-127802

f422eea

mikecote enabled auto-merge (squash) May 3, 2022 17:38

mikecote merged commit d3ac9aa into elastic:main May 3, 2022

kibanamachine added the backport:skip This PR does not require backporting label May 3, 2022

pmuellr mentioned this pull request Oct 18, 2022

[ResponseOps] backport to 7.17 fix for: "event.duration" must be a safe number #143562

Closed

mikecote mentioned this pull request Nov 8, 2022

[7.x] Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch) #144837

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch)#130819

Convert event log's duration from number to string in Kibana (keep as "long" in Elasticsearch)#130819
mikecote merged 13 commits intoelastic:mainfrom
mikecote:fix-127802

mikecote commented Apr 21, 2022 •

edited

Loading

Uh oh!

elasticmachine commented Apr 26, 2022

Uh oh!

Uh oh!

mikecote commented Apr 29, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ymao1 left a comment

Uh oh!

ersin-erdal left a comment

Uh oh!

mikecote commented May 3, 2022

Uh oh!

kibana-ci commented May 3, 2022

API count

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

mikecote commented Apr 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Apr 26, 2022

Uh oh!

Uh oh!

mikecote commented Apr 29, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

ersin-erdal left a comment

Choose a reason for hiding this comment

Uh oh!

mikecote commented May 3, 2022

Uh oh!

kibana-ci commented May 3, 2022

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mikecote commented Apr 21, 2022 •

edited

Loading