[GCP] Add redis data stream by gpop63 · Pull Request #4325 · elastic/integrations

gpop63 · 2022-09-27T15:32:47Z

What does this PR do?

Adds redis data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes [GCP] Collect metrics for Redis #4293

Screenshots

elasticmachine · 2022-09-27T15:53:52Z

🚀 Benchmarks report

Package `gcp` 👍(4) 💚(0) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`dns`	1908.4	1605.14	-303.26 (-15.89%)	💔

To see the full report comment with /test benchmark fullreport

elasticmachine · 2022-09-27T15:54:04Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-10-05T12:10:47.074+0000
Duration: 20 min 57 sec

Test stats 🧪

Test	Results
Failed	0
Passed	84
Skipped	0
Total	84

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2022-09-27T15:54:15Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`5/5`)	💚
Files	100.0% (`5/5`)	💚 2.53
Classes	100.0% (`5/5`)	💚 2.53
Methods	90.099% (`91/101`)	👎 -0.135
Lines	95.752% (`1375/1436`)	👍 4.15
Conditionals	100.0% (`0/0`)	💚

endorama · 2022-09-28T15:05:01Z

packages/gcp/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "2.12.6"


Suggested change

- version: "2.12.6"

- version: "2.12.0"

endorama · 2022-09-28T15:07:16Z

packages/gcp/data_stream/redis/elasticsearch/ingest_pipeline/default.yml

@@ -0,0 +1,131 @@
+---
+description: Pipeline for parsing GCP Redis metrics.
+processors:


All the fields here should have proper suffixes, like mentioned in https://www.elastic.co/guide/en/beats/devguide/current/event-conventions.html

endorama · 2022-09-28T15:09:15Z

packages/gcp/data_stream/redis/fields/fields.yml

+- name: gcp.redis
+  description: Google Cloud Redis metrics
+  type: group
+  fields:


Please add unit, metric_type and dimension where appropriate (see #4099 for more details)

endorama · 2022-09-28T15:12:24Z

packages/gcp/manifest.yml

 name: gcp
 title: Google Cloud Platform
-version: "2.11.6"
+version: "2.12.6"


Suggested change

version: "2.12.6"

version: "2.12.0"

endorama · 2022-09-29T09:23:49Z

@gpop63 please fix the CI (needs a build)

endorama

Apart from the red CI looks good to me!

endorama · 2022-10-05T08:36:22Z

/test

gpop63 · 2022-10-05T09:38:58Z

Tested data stream with new field units etc and works as expected.

efd6 · 2022-10-06T00:20:45Z

packages/gcp/data_stream/redis/fields/agent.yml

+  group: 2
+  description: Fields related to the cloud or infrastructure the events are coming from.
+  footnote: 'Examples: If Metricbeat is running on an GCP Compute VM and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.'
+  type: group


Some of these fields are duplicates of fields that are defined in ecs.yml. Fixed in #4397. You can check for duplicates in a package if you are running elastic-package v0.65.0 with format version 2.0.0 (do this temporarily before pushing).

add redis data stream

867777a

gpop63 added 3 commits September 28, 2022 16:49

fix field types

3c20d6f

add ingest pipeline

58ea070

update docs

6f10450

gpop63 marked this pull request as ready for review September 28, 2022 14:36

gpop63 requested review from a team as code owners September 28, 2022 14:36

endorama mentioned this pull request Sep 28, 2022

[GCP] support multiple regions when gathering metrics #4197

Merged

11 tasks

endorama requested changes Sep 28, 2022

View reviewed changes

gpop63 added 2 commits September 28, 2022 22:08

fix manifest and changelog version

f1cd1fc

fix sample event metricset name

b58f789

gpop63 force-pushed the add_gcp-redis branch from ad39ca8 to 2cd85c9 Compare September 28, 2022 19:10

add suffixes units and metric types

161f7dd

gpop63 force-pushed the add_gcp-redis branch from 2cd85c9 to 161f7dd Compare September 28, 2022 19:19

Merge remote-tracking branch 'upstream/main' into add_gcp-redis

60944a4

gpop63 requested a review from endorama September 28, 2022 19:24

endorama added the Integration:gcp Google Cloud Platform label Sep 29, 2022

endorama mentioned this pull request Sep 29, 2022

carbonblack_edr,checkpoint,cisco_umbrella,gcp,zeek: remove duplicate fields and fix types #4339

Merged

4 tasks

endorama approved these changes Sep 29, 2022

View reviewed changes

gpop63 and others added 2 commits September 29, 2022 16:06

update docs

9177979

Merge branch 'main' into add_gcp-redis

9b12bab

gpop63 and others added 3 commits October 5, 2022 13:24

Merge remote-tracking branch 'upstream/main' into add_gcp-redis

5f7fc98

fix sample event field name

a06acd1

Merge branch 'main' into add_gcp-redis

19e4ee4

endorama merged commit 2891ec1 into elastic:main Oct 5, 2022

efd6 reviewed Oct 6, 2022

View reviewed changes

andrewkroh mentioned this pull request Oct 20, 2022

[production] Promote many packages from snapshot elastic/package-storage#5929

Merged

		@@ -1,4 +1,9 @@
		# newer versions go on top
		- version: "2.12.6"

Conversation

gpop63 commented Sep 27, 2022

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Package gcp 👍(4) 💚(0) 💔(1)

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama Sep 28, 2022

Choose a reason for hiding this comment

Uh oh!

endorama commented Sep 29, 2022

Uh oh!

endorama left a comment

Choose a reason for hiding this comment

Uh oh!

endorama commented Oct 5, 2022

Uh oh!

gpop63 commented Oct 5, 2022

Uh oh!

efd6 Oct 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

elasticmachine commented Sep 27, 2022 •

edited

Loading

Package `gcp` 👍(4) 💚(0) 💔(1)

elasticmachine commented Sep 27, 2022 •

edited

Loading

elasticmachine commented Sep 27, 2022 •

edited

Loading

efd6 Oct 6, 2022 •

edited

Loading