Skip to content

feat(aws/logs): MetricFilter resource — log patterns as alertable metrics - #879

Closed
zeyuri wants to merge 1 commit into
alchemy-run:mainfrom
zeyuri:feat/logs-metric-filter
Closed

feat(aws/logs): MetricFilter resource — log patterns as alertable metrics#879
zeyuri wants to merge 1 commit into
alchemy-run:mainfrom
zeyuri:feat/logs-metric-filter

Conversation

@zeyuri

@zeyuri zeyuri commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Adds AWS.Logs.MetricFilter — the CloudWatch Logs primitive that turns matching log lines into a custom metric. The Logs family currently has only LogGroup, so today a stack can create the group but not the filter that makes its contents alertable.

The use case — alerting on a crash-loop that emits no metric

Our production agent cluster (a long-running service on ECS-on-EC2) has one failure mode that produces no metric at all: the process crashes during boot, logs the failure, and the orchestrator restarts it — CPU/memory look healthy, request metrics just flatline. The only signal is a log line. The standard AWS answer is a metric filter:

const logs = yield* Logs.LogGroup("ClusterLogs", { retentionInDays: 30 });

// Log pattern → custom metric. This is the piece this PR adds.
yield* Logs.MetricFilter("BootFailures", {
  logGroupName: logs.logGroupName,
  filterPattern: '"FATAL: cluster boot failed"',
  metricTransformations: [{
    metricName: "ClusterBootFailures",
    metricNamespace: "MyApp",
    metricValue: "1",
  }],
});

// Downstream, the normal alarm path takes over:
// CloudWatch.Alarm on MyApp/ClusterBootFailures → SNS topic → email/pager.

The full chain (log line → metric → alarm → notification) has been how we catch cluster crash-loops in production since June; the filter was the one link that couldn't be declared upstream.

Behavior

  • putMetricFilter is create-or-update keyed by (logGroupName, filterName); the diff models that — name or log-group change forces replacement, pattern/transformation changes update in place.
  • Delete via deleteMetricFilter, tolerating not-found.
  • Real list() via describeMetricFilters, following LogGroup's conventions.

Ported from a vendored resource running this exact alerting chain in production.

@zeyuri

zeyuri commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Closing — superseded by #797, which landed a generated AWS.Logs.MetricFilter covering this. Great to see the full-coverage generation land; we'll consume the generated resource. (For context, our use case was log-pattern → metric → alarm crash-loop alerting on an ECS service.)

@zeyuri zeyuri closed this Jul 18, 2026
@sam-goodwin

sam-goodwin commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

@zeyuri I followed up the 100% generation PR with focused improvements to ECS and EKS. See #867

Please LMK if there's anything i missed or doesn't feel right

@zeyuri

zeyuri commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@sam-goodwin

@zeyuri I followed up the 100% generation PR with focused improvements to ECS and EKS. See #867

Please LMK if there's anything i missed or doesn't feel right

No that PR 100% covers every use case, thats why I closed this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants