Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions packages/aws/_dev/build/docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,20 @@ Use this integration if you only need to collect data from the AWS Config servic
5. Add all the required integration configuration parameters, including the aws_region to enable data collection.
6. Select "Save and continue" to save the integration.

::::{important}
The AWS Config data stream uses the CEL input with manual AWS SigV4 request signing
rather than the AWS SDK. This means it **only** supports static credentials
(`Access Key ID` + `Secret Access Key`, and optionally `Session Token` for temporary
credentials).

**Role ARN, IAM instance profiles, shared credential files, and environment-based
credentials are not supported.** If you attempt to use this data stream without
providing static credentials, you will see an error indicating that `access_key_id`
and `secret_access_key` are required.
::::

**Note**
1. For the current integration package, it is compulsory to add Secret Access Key and Access Key ID.
2. The AWS Config integration performs a full ingestion of all findings during each interval.
1. The AWS Config integration performs a full ingestion of all findings during each interval.

## Troubleshooting

Expand Down
7 changes: 7 additions & 0 deletions packages/aws/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
# newer versions go on top
- version: "6.5.1"
changes:
- description: |
Add credential validation to AWS Config CEL program to emit a clear error when access_key_id or secret_access_key are missing.
Document that the AWS Config data stream requires static credentials and does not support Role ARN or other SDK-based credential methods.
type: bugfix
link: https://github.com/elastic/integrations/pull/18333
- version: "6.5.0"
changes:
- description: Add timestamp range filter and storage tier exclusion to latest transform source queries to reduce scan scope and improve performance.
Expand Down
10 changes: 10 additions & 0 deletions packages/aws/data_stream/config/_dev/test/scripts/env.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[!exec:echo] skip 'Skipping test requiring absent echo command'

exec echo ${PACKAGE_NAME}
stdout '^aws$'

exec echo ${DATA_STREAM}
stdout '^config$'

exec echo ${CURRENT_VERSION}
stdout '^[0-9]+\.[0-9]+\.[0-9]+$'
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Test that the AWS Config data stream emits a clear error event when
# access_key_id and secret_access_key are not provided.

[!external_stack] skip 'Skipping external stack test.'
[!exec:jq] skip 'Skipping test requiring absent jq command'

use_stack -profile ${CONFIG_PROFILES}/${PROFILE}
install_agent -profile ${CONFIG_PROFILES}/${PROFILE} -network_name NETWORK_NAME

add_package -profile ${CONFIG_PROFILES}/${PROFILE}
add_package_policy -profile ${CONFIG_PROFILES}/${PROFILE} test_config.yaml DATA_STREAM_NAME

# The credential check fires before any HTTP request, so exactly one error
# event should be indexed per evaluation cycle. Wait for at least 1.
get_docs -profile ${CONFIG_PROFILES}/${PROFILE} -want 1 -timeout 5m ${DATA_STREAM_NAME}
cp stdout got_docs.json

# Verify the error message names the missing credentials.
exec jq -r '[.hits.hits[]._source.error.message // empty] | flatten | .[]' got_docs.json
stdout 'access_key_id and secret_access_key'

# Verify no data events were produced — only error events.
exec jq '[.hits.hits[]._source | select(.aws.config != null)] | length' got_docs.json
stdout '^0$'

# Clean up.
remove_package_policy -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_NAME}
uninstall_agent -profile ${CONFIG_PROFILES}/${PROFILE} -timeout 1m

-- test_config.yaml --
input: cel
vars:
access_key_id: ""
secret_access_key: ""
data_stream:
vars:
aws_region: us-east-1
interval: 10s
batch_size: 2
preserve_original_event: true
15 changes: 15 additions & 0 deletions packages/aws/data_stream/config/agent/stream/cel.yml.hbs
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,21 @@ redact:
- session_token
program: |
(
// Credential validation: access_key_id and secret_access_key are required
// for hand-rolled SigV4 signing. Role ARN, IAM instance profiles, and
// other SDK-based credential methods are not supported.
state.?secret_key.orValue("") == "" || state.?access_key.orValue("") == "" ?
{
"events": {
"error": {
"code": "configuration_error",
"id": "missing_credentials",
"message": "AWS Config requires access_key_id and secret_access_key. Role ARN, IAM instance profiles, and other credential methods are not supported for this data stream.",
},
},
"want_more": false,
}
:
// Stage 1: Check if we have existing config rules to process
// If we have worklist with ConfigRules, we already fetched rules and now need to get compliance details
has(state.?worklist.ConfigRules) && size(state.worklist.ConfigRules) > 0 ?
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/data_stream/config/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ilm_policy: logs-aws.config-default_policy
streams:
- input: cel
title: Collect AWS Config Findings from AWS
description: Collect AWS Config Findings from AWS via API.
description: Collect AWS Config Findings from AWS via API. Requires Access Key ID and Secret Access Key — Role ARN and other credential methods are not supported.
template_path: cel.yml.hbs
enabled: false
vars:
Expand Down
15 changes: 13 additions & 2 deletions packages/aws/docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,20 @@ Use this integration if you only need to collect data from the AWS Config servic
5. Add all the required integration configuration parameters, including the aws_region to enable data collection.
6. Select "Save and continue" to save the integration.

::::{important}
The AWS Config data stream uses the CEL input with manual AWS SigV4 request signing
rather than the AWS SDK. This means it **only** supports static credentials
(`Access Key ID` + `Secret Access Key`, and optionally `Session Token` for temporary
credentials).

**Role ARN, IAM instance profiles, shared credential files, and environment-based
credentials are not supported.** If you attempt to use this data stream without
providing static credentials, you will see an error indicating that `access_key_id`
and `secret_access_key` are required.
::::

**Note**
1. For the current integration package, it is compulsory to add Secret Access Key and Access Key ID.
2. The AWS Config integration performs a full ingestion of all findings during each interval.
1. The AWS Config integration performs a full ingestion of all findings during each interval.

## Troubleshooting

Expand Down
2 changes: 1 addition & 1 deletion packages/aws/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.4.0
name: aws
title: AWS
version: 6.5.0
version: 6.5.1
description: Collect logs and metrics from Amazon Web Services (AWS) with Elastic Agent.
type: integration
categories:
Expand Down
Loading