Skip to content

[ML] Aggregate anomalies table data using configured Kibana timezone#26192

Merged
peteharverson merged 2 commits intoelastic:masterfrom
peteharverson:ml-anomalies-table-timezone
Nov 27, 2018
Merged

[ML] Aggregate anomalies table data using configured Kibana timezone#26192
peteharverson merged 2 commits intoelastic:masterfrom
peteharverson:ml-anomalies-table-timezone

Conversation

@peteharverson
Copy link
Contributor

@peteharverson peteharverson commented Nov 26, 2018

Summary

Uses the configured timezone, as stored in the dateFormat:tz config setting and configured under the Kibana Advanced Settings tab, when aggregating anomaly data by day or by hour for display in the anomalies table in the Single Metric Viewer and Anomaly Explorer.

Previously there could be a discrepancy between the time shown in the time cell of the table and the chart and the expanded row of the table, if the configured timezone was different to the timezone on which the Kibana server was running. The configured timezone set in the dateFormat.tz property is now passed to the server side endpoint which aggregates the data by day or by hour.

e.g. Before - Aggregating by day, server timezone Europe/London, Kibana timezone Europe/Athens (2 hours ahead of Europe/London), date in table cell incorrectly shows Feb 8th:

table_timezone_before

After - time in table cell now correctly displays Feb 9th:
table_timezone_after

Checklist

For maintainers

Fixes #21787

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be in $scope (is it used somewhere else in a template or passed on to child directives?) - otherwise could be just const dateFormatTz = ...?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that if this doesn't need to be shared and is only being passed around as a param then we shouldn't keep in in $scope. Other than that looks good. 😄

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question like in the other file if it's necessary to put this in $scope.

Copy link
Contributor

@walterra walterra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted a question about assignment of dateFormatTz to $scope and if that's necessary.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Member

@jgowdyelastic jgowdyelastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@walterra walterra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with latest changes!

@elasticmachine
Copy link
Contributor

💔 Build Failed

@peteharverson
Copy link
Contributor Author

retest

@elasticmachine
Copy link
Contributor

💔 Build Failed

@peteharverson peteharverson force-pushed the ml-anomalies-table-timezone branch from 27ec767 to 9d598e4 Compare November 27, 2018 11:00
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@peteharverson peteharverson merged commit d97609e into elastic:master Nov 27, 2018
@peteharverson peteharverson deleted the ml-anomalies-table-timezone branch November 27, 2018 11:49
peteharverson added a commit to peteharverson/kibana that referenced this pull request Nov 27, 2018
…lastic#26192)

* [ML] Aggregate anomalies table data using configured Kibana timezone

* [ML] Move dataFormatTz prop out of controller scope
peteharverson added a commit that referenced this pull request Nov 27, 2018
…26192) (#26252)

* [ML] Aggregate anomalies table data using configured Kibana timezone

* [ML] Move dataFormatTz prop out of controller scope
Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🙌

peteharverson added a commit to peteharverson/kibana that referenced this pull request Nov 27, 2018
…lastic#26192)

* [ML] Aggregate anomalies table data using configured Kibana timezone

* [ML] Move dataFormatTz prop out of controller scope
peteharverson added a commit that referenced this pull request Nov 27, 2018
…26192) (#26257)

* [ML] Aggregate anomalies table data using configured Kibana timezone

* [ML] Move dataFormatTz prop out of controller scope
sebelga added a commit that referenced this pull request Nov 28, 2018
* [APM] Fix horizontal scrollbar being visible in windows 8.1 (#25988)

* [APM] Changed 'Response Time' to 'Duration' in transactions screens (#25990)

* translate InfraOps visualization component (Part 3) (#25213)

* translate InfraOps visualization component (Part 3 - part of folder components)

* update translation of Infra Ops vizualization component (Part 3)

* update translation of Infra Ops vizualization component (Part 3)

* change some ids and add pluralization

* update Infra Ops Part 3 - change some ids, change some intl.formatMessage() to <FormattedMessage> and directly wrap some classes by injectI18n()

* update Infra-III - add static to displayName

* [i18n] Translate Agg_types(part_3) (#26118)

* Translate agg_types - metrics

* Fix issues

* [ML] Aggregate anomalies table data using configured Kibana timezone (#26192)

* [ML] Aggregate anomalies table data using configured Kibana timezone

* [ML] Move dataFormatTz prop out of controller scope

* [ML] Fix alignment of filter icons in anomalies table (#26253)

* [ML] Fix alignment of filter icons in anomalies table

* [ML] Teak y position of icons in expanded row of table

* translate sample data (#26069)

translate sample data

* [ML] Wrap controller initialization in assertions. (#26265)

- The controller tests introduced in #25382 had a flaw: If a controller initialization would fail and throw an error, that test suite wouldn't be able to clean up any stubs. So tests using the same stubs would report and error because the stubs couldn't be wrapped again.
- This PR wraps every controller initialization inside an assertion and catches those errors properly as part of the test.

* [APM] fixes #20145 by displaying span.context.http.url in the span details flyout (#26238)

* Fix spaces license check (#26270)

## Summary

Allows the public spaces API to work with a gold license

Resolves #26271

* Job Info button in Reporting Listing (#25421)

* Job Info button in Reporting Listing

* use lodash directly

* start of flyout use

* description list in flyout

* capitalize

* undefined guard

* expire info on close

* add jest test

* better at error handling + messaging

* Add description for vis types (#26243)

* [migrations] Throw error if reindex task fails (#26062)

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* [Reporting] Better logging for waitForSelector failure (#25762)

* [Reporting] Better logging for waitForSelector failure

* break waitForSelector

* experimental changes

* cleanup/consistency

* fix some test report title strings

* test disable chromium

* roll back test code

* take out non-working phantom changes

* roll back disable chromium test

* allow logger to use curried tags

* temporarily re-do report failure-causing change for test

* replace newline with escaped for single log line

* undo test change

* remove obsolete test

* [kbn/pm] allow packages to define extra paths to clean (#26132)

I noticed some discussion about how kbn clean should probably clear out the `.eslintcache` file, since it doesn't handle changes in related modules (for things like the import plugin) and it would be nice if `yarn kbn clean` took care of the issue. I figured it's not a bad idea, but adding `.eslintcache` directly to `@kbn/pm` felt wrong, so instead I've added another config options that can go in the package.json file, `clean.extraPatterns`. This array of patterns is passed into `del()` so that it can include things like negation.

As the name suggests this doesn't override the initial paths, just adds some extras that will be checked and cleared when `yarn kbn clean` is run.

* [config] fix logging.useUTC deprecation unset (#26053)

* Extend precommit hook script to support git GUI apps (#25883)

* feat(NA): extend support from precommit hook to git GUI apps.

* docs(NA): more descriptive error message.

* [DOCS] Clarify monitoring dependencies (#26229)

* apm: add ECS fields to index pattern (#26214)

* support standard license (#26294)

* [kbn-pm] update build

* [eslint] use disallow license header rule (#26309)

Fixes #26295

There are several places where we have accidentally added new license headers with linters but failed to remove old license headers manually. This prevents that by applying the an inverted version of the license headers rule that removed invalid license headers when files are moved.

* Bump node to 8.14.0 (#26313)

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Watch optimizer cache invalidation  (#24172)

* chore(NA): cherry pick work from spencer on impleting the cache invalidation system and merging it with current master.

* feat(NA): add support for dlls bundle into the cache state invalidation system.

* chore(NA): merge with master.

* feat(NA): first working version for the watch cache.

* feat(NA): added logger, correct cache delete and removed last todos.

* feat(NA): remove some useless features for the time being.

* refact(NA): just pass kibanaHapiServer.log function directly instead of an anonimous function that calls the kibanaHapiServer.log one.

* refact(NA): move everything to async.

* refact(NA): remove dll mentions.

* chore(NA): removed types/mkdirp as we dont use mkdirp into typescript.
@sophiec20 sophiec20 added the Feature:Anomaly Detection ML anomaly detection label Jun 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ML] Inconsistent dates in anomalies table when aggregation interval set to 1 day

6 participants