[receiver/awslambda] Adopt streaming for AWS Lambda receiver by Kavindu-Dodan · Pull Request #46608 · open-telemetry/opentelemetry-collector-contrib

Kavindu-Dodan · 2026-03-03T19:14:21Z

Description

This PR adopts streaming support introduced by AWS encoding extensions (refer #46214 )

This PR adopt streaming by,

For S3 triggers, use S3 stream reader so we won't load entire object into memory
For CloudWatch triggers, we load entire message into memory however, streaming support is there for batching needs.
Utilize xstreamencoding helpers to adopt streaming support for non-streaming extensions
To provide consistency and help implementation, default handling behaviour (when no extension is set) is extracted to built-in default decoders.

Above approach allows to use the receiver with a streaming or non-streaming encoding extension.

Testing

Unit tests to validate API contracts changed/added by this PR

Documentation

Updated documents to highlight streaming encoding support

MichaelKatsoulis

Thank you @Kavindu-Dodan

Overall your approach looks very good. I just left some suggestions!

One minor concern I have is what if an encoding extension at the top level advertises LogsDecoderExtension but some of the formats (example of awslogs encoding extension) do not support streaming. In that case we will get an error that streaming not supported for format and not a backward compatibility with usage of legacy unmarshaler.
Any way maybe that could be handled best in the encoding extension itself and not in the receiver.

Kavindu-Dodan · 2026-03-04T21:32:28Z

Thank you @Kavindu-Dodan

Overall your approach looks very good. I just left some suggestions!

One minor concern I have is what if an encoding extension at the top level advertises LogsDecoderExtension but some of the formats (example of awslogs encoding extension) do not support streaming. In that case we will get an error that streaming not supported for format and not a backward compatibility with usage of legacy unmarshaler. Any way maybe that could be handled best in the encoding extension itself and not in the receiver.

Thank you for the review 🤝

I think we cannot do much about sub-formats as they are internal 🤔 (If they expose the Decoder API contract but internally return error as implementation is missing, then those sub-formats will fail to work with the receiver)

However, the adopter layer allows any extension that does not implement the Decoder API to work with receiver.

MichaelKatsoulis · 2026-03-05T13:26:47Z

@Kavindu-Dodan does it need to be DRAFT ?

MichaelKatsoulis

Looks way better

Kavindu-Dodan · 2026-03-05T19:53:38Z

@Kavindu-Dodan does it need to be DRAFT ?

No we do not have to make this a draft. We can merge this if we get approvals

MichaelKatsoulis

Looks very good to me! I approve it but lets wait for @axw or @pjanotti reviews as well!

#### Description Introduce adapters for plog.LogsUnmarshaler and pmetric.MetricsUnmarshaler which, on first call, read the entire stream into memory and invoke the unmarshaler. Subsequent calls return EOF. The offset is always the entire stream size in bytes, and if an offset is provided then that many bytes will discarded first. We also extract LogsDecoderFactory and MetricsDecoderFactory interfaces from LogsDecoderExtension and MetricsDecoderExtension in the encoding package. This PR was built with the assistance of Cursor. #### Link to tracking issue Relates to #46608 #### Testing Tests updated #### Documentation N/A

#### Description Introduce adapters for plog.LogsUnmarshaler and pmetric.MetricsUnmarshaler which, on first call, read the entire stream into memory and invoke the unmarshaler. Subsequent calls return EOF. The offset is always the entire stream size in bytes, and if an offset is provided then that many bytes will discarded first. We also extract LogsDecoderFactory and MetricsDecoderFactory interfaces from LogsDecoderExtension and MetricsDecoderExtension in the encoding package. This PR was built with the assistance of Cursor. #### Link to tracking issue Relates to open-telemetry#46608 #### Testing Tests updated #### Documentation N/A

axw

Thanks @Kavindu-Dodan, just one last thing and I think it's good to go.

atoulme · 2026-03-12T06:45:50Z

Please resolve conflicts

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co> # Conflicts: # receiver/awslambdareceiver/go.mod # receiver/awslambdareceiver/go.sum # Conflicts: # receiver/awslambdareceiver/go.mod # receiver/awslambdareceiver/go.sum # Conflicts: # receiver/awslambdareceiver/go.sum # Conflicts: # receiver/awslambdareceiver/go.mod # receiver/awslambdareceiver/go.sum # Conflicts: # receiver/awslambdareceiver/go.mod # receiver/awslambdareceiver/go.sum # Conflicts: # receiver/awslambdareceiver/go.mod # receiver/awslambdareceiver/go.sum

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

Kavindu-Dodan · 2026-03-12T14:54:44Z

Please resolve conflicts

@atoulme thank you, all validations are green with the latest rebase 🙏

Could you please go ahead and merge this PR ?

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch 5 times, most recently from 75b0c51 to fc28403 Compare March 3, 2026 22:34

MichaelKatsoulis reviewed Mar 4, 2026

View reviewed changes

Comment thread receiver/awslambdareceiver/internal/decoder_wrapper.go Outdated

Comment thread receiver/awslambdareceiver/handler.go Outdated

Kavindu-Dodan changed the title ~~WIP : Adopt streaming for AWS Lambda receiver~~ [receiver/awslambda] Adopt streaming for AWS Lambda receiver Mar 4, 2026

Kavindu-Dodan marked this pull request as ready for review March 4, 2026 21:23

Kavindu-Dodan requested review from a team, axw and pjanotti as code owners March 4, 2026 21:23

github-actions Bot assigned bogdandrutu Mar 4, 2026

github-actions Bot added the receiver/awslambda label Mar 4, 2026

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from ef26253 to 66c6900 Compare March 4, 2026 21:25

MichaelKatsoulis reviewed Mar 5, 2026

View reviewed changes

Comment thread receiver/awslambdareceiver/receiver.go Outdated

MichaelKatsoulis reviewed Mar 5, 2026

View reviewed changes

Comment thread receiver/awslambdareceiver/receiver.go

MichaelKatsoulis reviewed Mar 5, 2026

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch 2 times, most recently from a28f271 to 918a544 Compare March 5, 2026 19:49

Kavindu-Dodan requested a review from MichaelKatsoulis March 5, 2026 19:53

Kavindu-Dodan mentioned this pull request Mar 5, 2026

Multi-Format Log Support in awslambdareceiver #46458

Open

MichaelKatsoulis approved these changes Mar 6, 2026

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from 918a544 to a744ae5 Compare March 6, 2026 16:42

axw reviewed Mar 9, 2026

View reviewed changes

axw mentioned this pull request Mar 9, 2026

[pkg/xstreamencoding] add unmarshaler adapters #46754

Merged

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from a744ae5 to d761fa1 Compare March 9, 2026 18:04

Kavindu-Dodan requested a review from axw March 9, 2026 18:09

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from d761fa1 to 790664e Compare March 9, 2026 18:47

axw reviewed Mar 10, 2026

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from 38734f7 to 6d8c26c Compare March 10, 2026 18:39

Kavindu-Dodan requested a review from axw March 10, 2026 18:52

axw reviewed Mar 11, 2026

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from 6d8c26c to 1987919 Compare March 11, 2026 17:26

Kavindu-Dodan requested a review from axw March 11, 2026 17:28

axw approved these changes Mar 12, 2026

View reviewed changes

axw added the ready to merge Code review completed; ready to merge by maintainers label Mar 12, 2026

atoulme removed the ready to merge Code review completed; ready to merge by maintainers label Mar 12, 2026

Kavindu-Dodan added 6 commits March 12, 2026 06:57

review changes: improve interface contracts & add unit tests

3b6b9e1

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

review change: unify logs extension loading

7b30c0f

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

review: adopt xstreaming, handler updates

52a37ef

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

review: reflection usage, default handler, test fixes

15b461e

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

review: migrate default CW decoder to a wrapper

e4fb78c

Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co>

Kavindu-Dodan force-pushed the feat/aws-lambda-receiver-stream-support branch from 0aa1a17 to e4fb78c Compare March 12, 2026 13:59

atoulme merged commit 3314aed into open-telemetry:main Mar 12, 2026
191 checks passed

dloucasfx mentioned this pull request Mar 24, 2026

[chore] Changelog for v0.148.0 signalfx/splunk-otel-collector#7357

Merged

Conversation

Kavindu-Dodan commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Documentation

Uh oh!

MichaelKatsoulis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Kavindu-Dodan commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MichaelKatsoulis commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MichaelKatsoulis left a comment

Choose a reason for hiding this comment

Uh oh!

Kavindu-Dodan commented Mar 5, 2026

Uh oh!

MichaelKatsoulis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

axw left a comment

Choose a reason for hiding this comment

Uh oh!

atoulme commented Mar 12, 2026

Uh oh!

Kavindu-Dodan commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kavindu-Dodan commented Mar 3, 2026 •

edited

Loading

Kavindu-Dodan commented Mar 4, 2026 •

edited

Loading

MichaelKatsoulis commented Mar 5, 2026 •

edited

Loading

Kavindu-Dodan commented Mar 12, 2026 •

edited

Loading