Check if Kibana files are up-to-date #1168

simitt · 2018-07-23T10:01:14Z

APM dashboards and index pattern are stored in Kibana. Add test to
ensure that those files are kept up-to-date.

implements #1042

Current Differences found by this script:

---- Comparing Dashboards
================================
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.aggs.id=1.params.customLabel'
"Avg. Resp. Time"
Value in APM Server: Avg. Trans. Time
Value in Kibana: Avg. Resp. Time
---
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.aggs.id=3.params.customLabel'
"Resp. Time"
Value in APM Server: Trans. Time
Value in Kibana: Resp. Time
---
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Top Services by Response Time [APM]"
Value in APM Server: Top Services by Transaction Time [APM]
Value in Kibana: Top Services by Response Time [APM]
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Top Services by Response Time [APM]"
Value in APM Server: Top Services by Transaction Time [APM]
Value in Kibana: Top Services by Response Time [APM]
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.value_template'
"{{value}} rpm"
Value in APM Server: {{value}} tpm
Value in Kibana: {{value}} rpm
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Top Apps by Request Per Minute [APM]"
Value in APM Server: Top Apps by Transaction Per Minute [APM]
Value in Kibana: Top Apps by Request Per Minute [APM]
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Top Services by Request Per Minute [APM]"
Value in APM Server: Top Services by Transaction Per Minute [APM]
Value in Kibana: Top Services by Request Per Minute [APM]
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'c618e4e0-7c69-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'ceefd050-7c6a-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.filter.query={u'match': {u'processor.event': {u'query': u'error', u'type': u'phrase'}}}.meta.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'ceefd050-7c6a-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '22518e70-7c69-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'a2e199b0-7820-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Response Times [APM]"
Value in APM Server: Transaction Times [APM]
Value in Kibana: Response Times [APM]
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Response Times [APM]"
Value in APM Server: Transaction Times [APM]
Value in Kibana: Response Times [APM]
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.value_template'
"{{value}} rpm"
Value in APM Server: {{value}} tpm
Value in Kibana: {{value}} rpm
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.metrics.id=61ca57f2-469d-11e7-af02-69e470af7417.type'
"count"
Value in APM Server: cardinality
Value in Kibana: count
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.split_mode'
"terms"
Value in APM Server: everything
Value in Kibana: terms
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.show_legend'
1
Value in APM Server: 0
Value in Kibana: 1
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Request Per Minute [APM]"
Value in APM Server: Transaction Per Minute [APM]
Value in Kibana: Request Per Minute [APM]
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Request Per Minute [APM]"
Value in APM Server: Transaction Per Minute [APM]
Value in Kibana: Request Per Minute [APM]
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'd7735b90-7ddf-11e7-b115-df9c90da2df1' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.filter.query={u'match': {u'processor.event': {u'query': u'span', u'type': u'phrase'}}}.meta.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'd7735b90-7ddf-11e7-b115-df9c90da2df1' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
---- Comparing Index Pattern
================================
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_id.searchable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_id.aggregatable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_index.searchable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_index.aggregatable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'id'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
make: *** [are-kibana-objects-updated] Error 2

APM dashboards and index pattern are stored in Kibana. Add test to ensure that those files are kept up-to-date. implements elastic#1042

jalvz · 2018-07-23T11:04:57Z

make are-kibana-objects-updated master
gives me

---- Comparing Dashboards
================================
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.aggs.id=1.params.customLabel'
"Avg. Resp. Time"
Value in APM Server: Avg. Trans. Time
Value in Kibana: Avg. Resp. Time
---
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.aggs.id=3.params.customLabel'
"Resp. Time"
Value in APM Server: Trans. Time
Value in Kibana: Resp. Time
---
Difference for id '1ffc5e20-7827-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Top Services by Response Time [APM]"
Value in APM Server: Top Services by Transaction Time [APM]
Value in Kibana: Top Services by Response Time [APM]
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Top Services by Response Time [APM]"
Value in APM Server: Top Services by Transaction Time [APM]
Value in Kibana: Top Services by Response Time [APM]
---
Difference for id '1bdca740-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.value_template'
"{{value}} rpm"
Value in APM Server: {{value}} tpm
Value in Kibana: {{value}} rpm
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Top Apps by Request Per Minute [APM]"
Value in APM Server: Top Apps by Transaction Per Minute [APM]
Value in Kibana: Top Apps by Request Per Minute [APM]
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Top Services by Request Per Minute [APM]"
Value in APM Server: Top Services by Transaction Per Minute [APM]
Value in Kibana: Top Services by Request Per Minute [APM]
---
Difference for id '804ffc40-7828-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'c618e4e0-7c69-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'ceefd050-7c6a-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.filter.query={u'match': {u'processor.event': {u'query': u'error', u'type': u'phrase'}}}.meta.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'ceefd050-7c6a-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '22518e70-7c69-11e7-aa55-3b0d52c71c60' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'a2e199b0-7820-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Response Times [APM]"
Value in APM Server: Transaction Times [APM]
Value in Kibana: Response Times [APM]
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Response Times [APM]"
Value in APM Server: Transaction Times [APM]
Value in Kibana: Response Times [APM]
---
Difference for id '09bcf890-7822-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.value_template'
"{{value}} rpm"
Value in APM Server: {{value}} tpm
Value in Kibana: {{value}} rpm
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.metrics.id=61ca57f2-469d-11e7-af02-69e470af7417.type'
"count"
Value in APM Server: cardinality
Value in Kibana: count
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.series.id=61ca57f1-469d-11e7-af02-69e470af7417.split_mode'
"terms"
Value in APM Server: everything
Value in Kibana: terms
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.params.show_legend'
1
Value in APM Server: 0
Value in Kibana: 1
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.visState.title'
"Request Per Minute [APM]"
Value in APM Server: Transaction Per Minute [APM]
Value in Kibana: Request Per Minute [APM]
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.title'
"Request Per Minute [APM]"
Value in APM Server: Transaction Per Minute [APM]
Value in Kibana: Request Per Minute [APM]
---
Difference for id '55606a60-7823-11e7-8c47-65b845b5cfb3' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'd7735b90-7ddf-11e7-b115-df9c90da2df1' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.filter.query={u'match': {u'processor.event': {u'query': u'span', u'type': u'phrase'}}}.meta.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
Difference for id 'd7735b90-7ddf-11e7-b115-df9c90da2df1' for key 'attributes.kibanaSavedObjectMeta.searchSourceJSON.index'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
---- Comparing Index Pattern
================================
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_id.searchable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_id.aggregatable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_index.searchable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'attributes.fields.name=_index.aggregatable'
true
Value in APM Server: False
Value in Kibana: True
---
Difference for id '12e52550-6354-11e8-9d01-ed6a4badd083' for key 'id'
"12e52550-6354-11e8-9d01-ed6a4badd083"
Value in APM Server: apm-*
Value in Kibana: 12e52550-6354-11e8-9d01-ed6a4badd083
---
make: *** [are-kibana-objects-updated] Error 2

Is this expected?

simitt · 2018-07-23T11:27:28Z

Yes, I posted the outcome above, but probably too late for you to see it.

The test actually does uncover differences between the files.

jalvz

very nice!

should this step be added to check-full?

simitt · 2018-07-23T12:37:35Z

I am planning on opening a PR for Infra that runs this test on every PR, once we have figured out how to make this test successful (excluding parts of the test or aligning the files).

simitt · 2018-07-23T13:18:10Z

cc @sqren and @makwarth

sorenlouv · 2018-07-24T10:12:21Z

Awesome! Thanks for doing this @simitt !

makwarth · 2018-07-24T11:08:26Z

++ good job @simitt !

* Check if Kibana files are up-to-date APM dashboards and index pattern are stored in Kibana. Add test to ensure that those files are kept up-to-date. implements elastic#1042

* Check if Kibana files are up-to-date APM dashboards and index pattern are stored in Kibana. Add test to ensure that those files are kept up-to-date. implements #1042

Check if Kibana files are up-to-date

5135ec8

APM dashboards and index pattern are stored in Kibana. Add test to ensure that those files are kept up-to-date. implements elastic#1042

simitt added this to the 6.4 milestone Jul 23, 2018

jalvz approved these changes Jul 23, 2018

View reviewed changes

alvarolobato assigned simitt Jul 23, 2018

alvarolobato added [zube]: Inbox and removed [zube]: Inbox labels Jul 23, 2018

slightly adapt output.

57df397

simitt merged commit 494e5d8 into elastic:master Jul 24, 2018

zube bot added [zube]: Done and removed [zube]: In Review labels Jul 24, 2018

This was referenced Jul 24, 2018

[6.x] Check if Kibana files are up-to-date (#1168) #1179

Merged

Add jsondiff to test requirements. elastic/beats#7694

Merged

simitt deleted the 1042-add-kibana-integration-test branch August 3, 2018 06:11

graphaelli mentioned this pull request Nov 19, 2018

Update beats framework to 1304682 #1567

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Check if Kibana files are up-to-date #1168

Check if Kibana files are up-to-date #1168

Uh oh!

simitt commented Jul 23, 2018 •

edited by zube bot

Loading

Uh oh!

jalvz commented Jul 23, 2018

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

jalvz left a comment

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

sorenlouv commented Jul 24, 2018

Uh oh!

makwarth commented Jul 24, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Check if Kibana files are up-to-date #1168

Check if Kibana files are up-to-date #1168

Uh oh!

Conversation

simitt commented Jul 23, 2018 • edited by zube bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jalvz commented Jul 23, 2018

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

jalvz left a comment

Choose a reason for hiding this comment

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

simitt commented Jul 23, 2018

Uh oh!

sorenlouv commented Jul 24, 2018

Uh oh!

makwarth commented Jul 24, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

simitt commented Jul 23, 2018 •

edited by zube bot

Loading