Skip to content

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Oct 15, 2024

Proposed commit message

This is a follow-up to #41188 where I am adding support to source linked accounts when using log_group_name_prefix to derive log groups.

PR introduce include_linked_accounts_for_prefix_mode boolean property, which is disabled by default. If enabled (include_linked_accounts_for_prefix_mode : true), then we set includeLinkedAccounts property of the DescribeLogGroups API [1] to obtain log groups matching prefix and included in linked accounts of the monitoring account.

ex:-

- type: aws-cloudwatch
  ...
  log_group_name_prefix : /development/AppA/
  include_linked_accounts_for_prefix_mode: true
  ... 

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

This require a linked cloudwatch account. If already has one, then,

  • Push logs to a newly created log group OR use an already existing log group in a source account
    • Note - you may use data-gen Go program to generate and push logs to your log group (using output CLOUDWATCH_LOG) [2]
  • Configure filebeat cloudwatch input with log_group_name_prefix with desired prefix & set include_linked_accounts_for_prefix_mode to value true (enabled)
  • Run filebeat and observe logs in Kibana discover which include logs from log groups (that match provided prefix)

Related issues

[1] - https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_DescribeLogGroups.html
[2] - https://github.com/Kavindu-Dodan/data-gen


This is an automatic backport of pull request #41206 done by Mergify.

…counts when using log_group_name_prefix (#41206)

* configuration parsing to support arn & linked accounts

Signed-off-by: Kavindu Dodanduwa <[email protected]>

# Conflicts:
#	x-pack/filebeat/input/awscloudwatch/input.go

* code review change - fix typo

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* add support to linked accounts when using prefix mode

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* add changelog entry

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* review suggestion

Signed-off-by: Kavindu Dodanduwa <[email protected]>

* use non-pointer struct property

Signed-off-by: Kavindu Dodanduwa <[email protected]>

---------

Signed-off-by: Kavindu Dodanduwa <[email protected]>
(cherry picked from commit 7e1b528)
@mergify mergify bot requested review from a team as code owners October 15, 2024 20:11
@mergify mergify bot added the backport label Oct 15, 2024
@mergify mergify bot requested review from faec and mauri870 and removed request for a team October 15, 2024 20:11
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 15, 2024
@Kavindu-Dodan Kavindu-Dodan added the Team:obs-ds-hosted-services Label for the Observability Hosted Services team label Oct 15, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 15, 2024
@pierrehilbert pierrehilbert enabled auto-merge (squash) October 15, 2024 21:00
@pierrehilbert pierrehilbert merged commit ac439c8 into 8.x Oct 15, 2024
@pierrehilbert pierrehilbert deleted the mergify/bp/8.x/pr-41206 branch October 15, 2024 21:52
@khushijain21 khushijain21 mentioned this pull request Jun 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Team:obs-ds-hosted-services Label for the Observability Hosted Services team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants