Skip to content

[GCP] support multiple regions when gathering metrics #4197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
endorama merged 14 commits into from
Nov 22, 2022
Merged

[GCP] support multiple regions when gathering metrics #4197

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
70856d6
compute: support regions
endorama Sep 12, 2022
5e43e06
firestore: support regions
endorama Sep 12, 2022
06b0ffd
gke: support regions
endorama Sep 12, 2022
087f648
loadbalancing_metrics: support regions
endorama Sep 12, 2022
a5f5240
storage: support regions
endorama Sep 12, 2022
dc72063
update changelog
endorama Sep 12, 2022
1fcbb97
update kibana min version
endorama Sep 12, 2022
f85620a
remove support for 7.17.x
endorama Sep 15, 2022
78003d4
remove region setting
endorama Sep 21, 2022
7520b5c
redis: support multiple regions
endorama Oct 14, 2022
be52d90
add regions to redis
endorama Oct 20, 2022
91e52df
Revert "remove region setting"
endorama Oct 20, 2022
3f9905b
deprecate region setting
endorama Oct 20, 2022
0ec221f
dataproc: add region
endorama Oct 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions packages/gcp/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
# newer versions go on top
- version: "2.15.0"
changes:
- description: Remove support for Kibana 7.17.x
type: enhancement
link: https://github.com/elastic/integrations/pull/4197
- description: Support multiple regions for metrics data streams
type: enhancement
link: https://github.com/elastic/integrations/pull/4197
- version: "2.14.0"
changes:
- description: Update package to ECS 8.5.0.
Expand Down
6 changes: 6 additions & 0 deletions packages/gcp/data_stream/compute/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: compute
Expand Down
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/compute/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
6 changes: 6 additions & 0 deletions packages/gcp/data_stream/dataproc/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ credentials_json: '{{credentials_json}}'
{{#if region}}
region: {{region}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: dataproc
Expand Down
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/dataproc/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
6 changes: 6 additions & 0 deletions packages/gcp/data_stream/firestore/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: firestore
Expand Down
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/firestore/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
8 changes: 7 additions & 1 deletion packages/gcp/data_stream/gke/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: gke
Expand Down Expand Up @@ -57,4 +63,4 @@ metrics:
- "pod/network/sent_bytes_count"
- "pod/volume/total_bytes"
- "pod/volume/used_bytes"
- "pod/volume/utilization"
- "pod/volume/utilization"
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/gke/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
8 changes: 7 additions & 1 deletion packages/gcp/data_stream/loadbalancing_metrics/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: loadbalancing
Expand Down Expand Up @@ -45,4 +51,4 @@ metrics:
- "https/total_latencies"
- "l3/external/rtt_latencies"
- "l3/internal/rtt_latencies"
- "tcp_ssl_proxy/frontend_tcp_rtt"
- "tcp_ssl_proxy/frontend_tcp_rtt"
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/loadbalancing_metrics/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
6 changes: 6 additions & 0 deletions packages/gcp/data_stream/redis/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: redis
Expand Down
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/redis/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
6 changes: 6 additions & 0 deletions packages/gcp/data_stream/storage/agent/stream/stream.yml.hbs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ region: {{region}}
{{#if zone}}
zone: {{zone}}
{{/if}}
{{#if regions}}
regions:
{{#each regions as |region i|}}
- {{region}}
{{/each}}
{{/if}}
exclude_labels: {{exclude_labels}}
metrics:
- service: storage
Expand Down
8 changes: 8 additions & 0 deletions packages/gcp/data_stream/storage/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,17 @@ streams:
- name: region
type: text
title: GCP Region
description: deprecated, use regions instead
multi: false
required: false
show_user: true
- name: regions
type: text
title: GCP Regions
description: A list of GCP regions to pull data from
multi: true
required: false
show_user: true
- name: period
type: text
title: Period
Expand Down
4 changes: 2 additions & 2 deletions packages/gcp/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: gcp
title: Google Cloud Platform
version: "2.14.0"
version: "2.15.0"
release: ga
description: Collect logs from Google Cloud Platform with Elastic Agent.
type: integration
Expand All @@ -15,7 +15,7 @@ categories:
- google_cloud
- cloud
conditions:
kibana.version: ^8.3.0
kibana.version: ^8.5.0
screenshots:
- src: /img/filebeat-gcp-audit.png
title: filebeat gcp audit
Expand Down