Skip to content

[Metricbeat] Fix getting compute instance metadata with partial zone/region config #18757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
May 28, 2020
Merged

[Metricbeat] Fix getting compute instance metadata with partial zone/region config #18757

merged 8 commits into from
May 28, 2020

Conversation

@kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented May 26, 2020 •
edited
Loading

What does this PR do?

This PR fixes getting compute instance metadata when zone/region in googlecloud.yml is partial. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-"
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m

This bug is introduced from #17913

Why is it important?

Instances.Get API does not accept partial zone name and Regions.Get API does not accept partial region name. So instead of using the region/zone information from the config, use the zone label from time series metric.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@kaiyan-sheng kaiyan-sheng self-assigned this May 26, 2020
@kaiyan-sheng kaiyan-sheng removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@kaiyan-sheng kaiyan-sheng added bug needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels May 26, 2020
@elasticmachine
Copy link
Contributor

elasticmachine commented May 26, 2020

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@elasticmachine
Copy link
Contributor

elasticmachine commented May 26, 2020 •
edited
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18757 updated]

  • Start Time: 2020-05-28T21:12:50.403+0000

  • Duration: 56 min 55 sec

Test stats 🧪

Test Results
Failed 0
Passed 1224
Skipped 199
Total 1423

Steps errors

Expand to view the steps failures

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Duration: 2 min 22 sec

    • Start Time: 2020-05-28T22:05:24.365+0000

    • log

@kaiyan-sheng kaiyan-sheng requested a review from sayden May 27, 2020 20:14
@kaiyan-sheng
Copy link
Contributor Author

kaiyan-sheng commented May 28, 2020

More unit tests need to be added with mocking the gcp APIs in a separate PR.

sayden
sayden approved these changes May 28, 2020
View reviewed changes
Copy link
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine! I left a comment

@kaiyan-sheng kaiyan-sheng merged commit f4c65f3 into elastic:master May 28, 2020
@kaiyan-sheng kaiyan-sheng deleted the metadata_compute branch May 28, 2020 22:20
@kaiyan-sheng kaiyan-sheng mentioned this pull request May 28, 2020
Merged
6 tasks
@kaiyan-sheng kaiyan-sheng added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 28, 2020
@kaiyan-sheng kaiyan-sheng mentioned this pull request May 28, 2020
Merged
6 tasks
kaiyan-sheng added a commit that referenced this pull request May 29, 2020
@kaiyan-sheng
Cherry-pick #18757 to 7.8: [Metricbeat] Fix getting compute instance …
9ef64e3 
…metadata with partial zone/region config (#18840)

* [Metricbeat] Fix getting compute instance metadata with partial zone/region config (#18757)

* use zone for getting instance metadata

(cherry picked from commit f4c65f3)

* update changelog
kaiyan-sheng added a commit that referenced this pull request May 29, 2020
@kaiyan-sheng
Cherry-pick #18757 to 7.x: [Metricbeat] Fix getting compute instance …
8928a7d 
…metadata with partial zone/region config (#18839)

* [Metricbeat] Fix getting compute instance metadata with partial zone/region config (#18757)

* use zone for getting instance metadata

(cherry picked from commit f4c65f3)

* update changelog
v1v added a commit to v1v/beats that referenced this pull request Jun 2, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/module-at…
7509af0 
…-stage-level

* upstream/master: (30 commits)
  Add a GRPC listener service for Agent (elastic#18827)
  Disable host.* fields by default for iptables module (elastic#18756)
  [WIP] Clarify capabilities of the Filebeat auditd module (elastic#17068)
  fix: rename file and remove extra separator (elastic#18881)
  ci: enable JJBB (elastic#18812)
  Disable host.* fields by default for Checkpoint module (elastic#18754)
  Disable host.* fields by default for Cisco module (elastic#18753)
  Update latest.yml testing env to 7.7.0 (elastic#18535)
  Upgrade k8s.io/client-go and k8s keystore tests (elastic#18817)
  Add missing Jenkins stages for Auditbeat (elastic#18835)
  [Elastic Log Driver] Create a config shim between libbeat and the user (elastic#18605)
  Use indexers and matchers in config when defaults are enabled (elastic#18818)
  Fix panic on `metricbeat test modules` (elastic#18797)
  [CI] Fix permissions in MacOSX agents (elastic#18847)
  [Ingest Manager] When not port are specified and the https is used fallback to 443 (elastic#18844)
  [Ingest Manager] Fix install service script for windows (elastic#18814)
  [Metricbeat] Fix getting compute instance metadata with partial zone/region config (elastic#18757)
  Improve error messages in s3 input (elastic#18824)
  Add memory metrics into compute googlecloud (elastic#18802)
  include bucket name when logging error (elastic#18679)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@sayden sayden sayden approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@kaiyan-sheng kaiyan-sheng

Labels

bug Team:Platforms Label for the Integrations - Platforms team v7.8.0 v7.9.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kaiyan-sheng @elasticmachine @sayden @andresrc