Skip to content

Add routing rules for WAF logs based on log format #7836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kaiyan-sheng merged 5 commits into from
Sep 21, 2023
Merged

Add routing rules for WAF logs based on log format #7836

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5d8789d
Add routing rules for WAF logs based on log format and remove namespace
kaiyan-sheng Sep 14, 2023
2f43741
add changelog
kaiyan-sheng Sep 14, 2023
7475312
simplify routing rule when checking for waf logs
kaiyan-sheng Sep 18, 2023
87308eb
update kibana version
kaiyan-sheng Sep 20, 2023
0247088
put namespace back
kaiyan-sheng Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/awsfirehose/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: 0.2.1
changes:
- description: Route WAF logs based on log format
type: enhancement
link: https://github.com/elastic/integrations/pull/7836
- version: 0.2.0
changes:
- description: Add support for routing api gateway logs
Expand Down
12 changes: 11 additions & 1 deletion packages/awsfirehose/data_stream/logs/routing_rules.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,18 @@
- "{{data_stream.namespace}}"
- default
- target_dataset: aws.waf
# Kinesis Data Firehose stream name begins with `aws-waf-logs-`
# CloudWatch log group name begins with `aws-waf-logs-`
# Log fields:
# timestamp formatVersion webaclld terminatingRuleId terminatingRuleType action
# terminatingRuleMatchDetails httpSourceName httpSourceId ruleGroupList rateBasedRuleList
# nonTerminatingMatchingRules httpRequest labels
if: >-
(ctx['aws.kinesis.name'] != null && ctx['aws.kinesis.name'].contains('aws-waf-logs-')) || (ctx['aws.cloudwatch.log_group'] != null && ctx['aws.cloudwatch.log_group'].contains('aws-waf-logs-'))
(ctx['aws.kinesis.name'] != null && ctx['aws.kinesis.name'].contains('aws-waf-logs-'))
|| (ctx['aws.cloudwatch.log_group'] != null && ctx['aws.cloudwatch.log_group'].contains('aws-waf-logs-'))
|| (ctx.message != null && ctx.message.contains('webaclld') && ctx.message.contains('terminatingRule')
&& ctx.message.contains('httpSource') && ctx.message.contains('ruleGroupList') && ctx.message.contains('rateBasedRuleList')
&& ctx.message.contains('nonTerminatingMatchingRules') && ctx.message.contains('httpRequest') && ctx.message.contains('labels'))
namespace:
- "{{data_stream.namespace}}"
- default
Expand Down
4 changes: 2 additions & 2 deletions packages/awsfirehose/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
format_version: 2.9.0
name: awsfirehose
title: Amazon Kinesis Data Firehose
version: 0.2.0
version: 0.2.1
description: Stream logs from Amazon Kinesis Data Firehose into Elastic Cloud.
type: integration
categories:
- observability
- aws
conditions:
kibana.version: "^8.10.0"
kibana.version: "^8.10.1"
owner:
github: elastic/obs-cloud-monitoring
icons:
Expand Down