Skip to content

chore: Use shared source structures for aws firehose #5739

Merged
kalleep merged 3 commits into
mainfrom
kalleep/fix-aws-firehose
Mar 11, 2026
Merged

chore: Use shared source structures for aws firehose #5739
kalleep merged 3 commits into
mainfrom
kalleep/fix-aws-firehose

Conversation

@kalleep
Copy link
Copy Markdown
Contributor

@kalleep kalleep commented Mar 10, 2026

Pull Request Details

  1. Use loki.Fanout and source.Consume
  2. Remove grpc block, dskit will always start a grpc server along side http but there is no way for aws to call this server.
  3. Don't use component id in metric names, dskit server exposes some metrics that we currently don't document but these should not be prefixed with component id but with component name like we do in loki.source.api. This is fine because all metircs have component_id as label and these are not documented.

There is other fixes I want to do too this component but will do that in a follow up.

Issue(s) fixed by this Pull Request

Notes to the Reviewer

PR Checklist

  • Documentation added
  • Tests updated
  • Config converters updated

BREAKING-CHANGE: Renamed undocumented metrics that was previously prefixed with <component_id><metric_name> to loki_source_awsfirehose<metric_name>

@kalleep kalleep requested review from a team and clayton-cornell as code owners March 10, 2026 13:34
@kalleep kalleep changed the title chore: use shared source structures for aws firehose chore: Use shared source structures for aws firehose Mar 10, 2026
@github-actions
Copy link
Copy Markdown
Contributor

💻 Deploy preview available (chore: use shared source structures for aws firehose ):

Comment thread internal/component/loki/source/aws_firehose/component.go
Copy link
Copy Markdown
Contributor

@clayton-cornell clayton-cornell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense form a docs standpoint. Data Firehose only sends HTTP requests.

@kalleep
Copy link
Copy Markdown
Contributor Author

kalleep commented Mar 11, 2026

@ptodev Created a issue about documenting these metrics #5751

@kalleep
Copy link
Copy Markdown
Contributor Author

kalleep commented Mar 11, 2026

It's a bit weird that we cannot prevent the grpc server from starting. Currently these components do not pass a port for it so a port will be selected at random. But this also means a random port will be occupied and while it's unlikely it could collide with another configured port.

@ptodev ptodev self-assigned this Mar 11, 2026
@kalleep kalleep merged commit aef19dc into main Mar 11, 2026
50 of 51 checks passed
@kalleep kalleep deleted the kalleep/fix-aws-firehose branch March 11, 2026 12:18
@github-actions
Copy link
Copy Markdown
Contributor

💻 Deploy preview deleted (chore: Use shared source structures for aws firehose ).

@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Mar 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants