Skip to content

[Uptime][Synthetics] fleet managed monitor telemetry#97502

Merged
dominiqueclarke merged 10 commits intoelastic:masterfrom
dominiqueclarke:feature/91062-fleet-monitor-telemetry
May 4, 2021
Merged

[Uptime][Synthetics] fleet managed monitor telemetry#97502
dominiqueclarke merged 10 commits intoelastic:masterfrom
dominiqueclarke:feature/91062-fleet-monitor-telemetry

Conversation

@dominiqueclarke
Copy link
Copy Markdown
Contributor

@dominiqueclarke dominiqueclarke commented Apr 19, 2021

Fixes #97301

Summary

Adds fleet managed monitor logic to the Uptime's telemetry adapter to count unique number of fleet managed monitors. This is accomplished by filtering for the monitor.fleet_enabled: true term.

Step 1: Run Kibana Locally

  • Clone this repo and pull down this PR
  • Update kibana.dev.yaml to include the following fields:
xpack.fleet.enabled: true
xpack.fleet.registryUrl: "https://epr-staging.elastic.co"
xpack.fleet.agents.enabled: true
xpack.fleet.agents.fleet_server.hosts: []
xpack.fleet.agents.tlsCheckDisabled: true
xpack.fleet.agents.fleetServerEnabled: true
xpack.encryptedSavedObjects.encryptionKey: "12345678901234567890123456789012"
  • run es with command yarn es snapshot -E xpack.security.authc.api_key.enabled=true --license trial
  • run kibana with command yarn kbn bootstrap && yarn start --no-base-path
  • Log into Kibana UI
  • Navigate to fleet
  • Click on the agents tab
  • You should see the Fleet Server UI. Click add agent where it should complain about not having a fleet server url.
  • Click fleet settings and add the fleet server url as localhost:8220
  • Follow the instructions for adding fleet server provided by the fleet server UI. Including downloading agent (the latest artifacts can be found at https://artifacts-api.elastic.co/v1/search/8.0-SNAPSHOT/elastic-agent, simply search for your operating type), extracting agent, generating token and running the listed command within the extracted agent folder

Step 2: Create integration

  • Click the integrations tab
  • Search for Synthetics
  • Click add integration on the synthetics page
  • Make sure to change the policy from Default Agent Policy to Default Fleet Server Policy

Step 3: Check telemetry

  • Navigate to Uptime and verify that your fleet managed monitors appear
  • Navigate to Stack Management, then to Advanced Settings. Scroll down to the bottom where it says "Usage Data". Click "cluster data" for a preview of the telemetry we collect. Search for "fleet_no_of_unique_monitors" to ensure that the count is correct, and verify the reset of the fleet prepended telemetry.

@dominiqueclarke dominiqueclarke added enhancement New value added to drive a business result v8.0.0 Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability release_note:skip Skip the PR/issue when compiling release notes v7.13.0 auto-backport Deprecated - use backport:version if exact versions are needed labels Apr 19, 2021
@dominiqueclarke dominiqueclarke requested a review from a team as a code owner April 19, 2021 17:43
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/uptime (Team:uptime)

@dominiqueclarke dominiqueclarke requested review from a team as code owners April 19, 2021 18:20
Copy link
Copy Markdown
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dominiqueclarke would you mind documenting the new fields. We are trying to enforce that, at least, new fields are documented. Thank you!

Comment on lines +69 to +75
fleet_monitor_name_stats: {
avg_length: { type: 'float' },
max_length: { type: 'long' },
min_length: { type: 'long' },
},
fleet_monitor_frequency: { type: 'array', items: { type: 'long' } },
fleet_no_of_unique_monitors: { type: 'long' },
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dominiqueclarke do you mind documenting the fields by adding the _meta.description like:

{ type: 'float', _meta: { description: 'This field represents...' } }

@dominiqueclarke dominiqueclarke force-pushed the feature/91062-fleet-monitor-telemetry branch 2 times, most recently from 34ba07c to aa01c58 Compare April 21, 2021 13:41
@dominiqueclarke dominiqueclarke requested a review from afharo April 21, 2021 13:42
@dominiqueclarke dominiqueclarke force-pushed the feature/91062-fleet-monitor-telemetry branch from aa01c58 to b3808ec Compare April 21, 2021 15:09
Copy link
Copy Markdown
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @dominiqueclarke! Telemetry changes LGTM!

"type": "float"
"type": "float",
"_meta": {
"description": "This field represents the average length of monitor names"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for adding theses additional descriptions

@dominiqueclarke
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

@dominiqueclarke
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

Copy link
Copy Markdown
Contributor

@shahzad31 shahzad31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, telemetry works !!

WFG !!

@dominiqueclarke
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

@dominiqueclarke
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dominiqueclarke dominiqueclarke merged commit 16b4f71 into elastic:master May 4, 2021
@dominiqueclarke dominiqueclarke deleted the feature/91062-fleet-monitor-telemetry branch May 4, 2021 11:48
@kibanamachine
Copy link
Copy Markdown
Contributor

💔 Backport failed

Status Branch Result
7.13 Commit could not be cherrypicked due to conflicts
7.x Commit could not be cherrypicked due to conflicts

To backport manually run:
node scripts/backport --pr 97502

dominiqueclarke added a commit to dominiqueclarke/kibana that referenced this pull request May 5, 2021
* add fleet monitor telemetry

* add changes to telemetry json file

* add functional tests

* document telemetry fields

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
#	x-pack/test/api_integration/apis/uptime/rest/helper/make_ping.ts
dominiqueclarke added a commit to dominiqueclarke/kibana that referenced this pull request May 5, 2021
* add fleet monitor telemetry

* add changes to telemetry json file

* add functional tests

* document telemetry fields

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
#	x-pack/test/api_integration/apis/uptime/rest/helper/make_ping.ts
dominiqueclarke added a commit that referenced this pull request May 5, 2021
* add fleet monitor telemetry

* add changes to telemetry json file

* add functional tests

* document telemetry fields

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
#	x-pack/test/api_integration/apis/uptime/rest/helper/make_ping.ts
dominiqueclarke added a commit that referenced this pull request May 5, 2021
* add fleet monitor telemetry

* add changes to telemetry json file

* add functional tests

* document telemetry fields

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
#	x-pack/test/api_integration/apis/uptime/rest/helper/make_ping.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Deprecated - use backport:version if exact versions are needed enhancement New value added to drive a business result release_note:skip Skip the PR/issue when compiling release notes Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v7.13.0 v7.14.0 v8.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Telemetry to count fleet managed monitors

6 participants