Skip to content

[OTel] Support Serializing nested data#45008

Merged
pickypg merged 3 commits intomainfrom
otel/handle-nested-object-arrays-and-slices
Jun 25, 2025
Merged

[OTel] Support Serializing nested data#45008
pickypg merged 3 commits intomainfrom
otel/handle-nested-object-arrays-and-slices

Conversation

@pickypg
Copy link
Copy Markdown
Member

@pickypg pickypg commented Jun 24, 2025

This adds support for map[string]any and also arrays in general (as opposed to just slices) to the converted map.

Like mapstr.M and []mapstr.M, this will dive into the map[string]any and []map[string]any objects to ensure that they are properly converted for the pdata code.

Proposed commit message

Properly handles nested map[string]any and array types (not just slices) for OTel conversion to avoid Invalid type warnings from OTel.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

This should make the code more flexible -- not less.

Author's Checklist

  • Ensure nested types are serialized.

How to test this PR locally

The easiest way is to inspect the new test and try removing the updated code to see what changes.

Related issues

Use cases

Metricsets can now ship nested map[string]any that contain arrays and slices, including of []map[string]any.

This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.
@pickypg pickypg self-assigned this Jun 24, 2025
@pickypg pickypg requested a review from a team as a code owner June 24, 2025 18:09
@pickypg pickypg requested review from faec and mauri870 June 24, 2025 18:09
@botelastic botelastic Bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 24, 2025
@github-actions

This comment was marked as off-topic.

@mergify

This comment was marked as resolved.

@pickypg pickypg added the backport-active-all Automated backport with mergify to all the active branches label Jun 24, 2025
@mauri870 mauri870 requested a review from VihasMakwana June 24, 2025 19:50
@mauri870
Copy link
Copy Markdown
Member

Added Vihas as a reviewer because of his recent work on otelmap.

@cmacknz cmacknz added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jun 24, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic Bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 24, 2025
@pickypg pickypg enabled auto-merge (squash) June 24, 2025 20:27
Copy link
Copy Markdown
Member

@mauri870 mauri870 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this. I don’t see anything obviously out of place. The recursion in certain use cases definitely makes it harder to wrap your head around all the possible scenarios.

Please wait for at least another review.

Comment thread libbeat/otelbeat/otelmap/otelmap.go
@pickypg
Copy link
Copy Markdown
Member Author

pickypg commented Jun 24, 2025

Please wait for at least another review.

No problem waiting. I am marking as auto-merge since after your review it's not going to do it on its own so that it can end up in the nightly build on the next pass.

@pickypg pickypg enabled auto-merge (squash) June 24, 2025 22:43
@pickypg pickypg merged commit 7bfa78e into main Jun 25, 2025
203 checks passed
@pickypg pickypg deleted the otel/handle-nested-object-arrays-and-slices branch June 25, 2025 11:26
@github-actions
Copy link
Copy Markdown
Contributor

@Mergifyio backport 8.17 8.18 8.19 9.0

@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Jun 25, 2025

backport 8.17 8.18 8.19 9.0

✅ Backports have been created

Details

mergify Bot pushed a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)

# Conflicts:
#	libbeat/otelbeat/otelmap/otelmap.go
#	libbeat/otelbeat/otelmap/otelmap_test.go
mergify Bot pushed a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)
mergify Bot pushed a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)

# Conflicts:
#	libbeat/otelbeat/otelmap/otelmap.go
#	libbeat/otelbeat/otelmap/otelmap_test.go
mergify Bot pushed a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)
pickypg added a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)

Co-authored-by: Chris Earle <pickypg@users.noreply.github.com>
pickypg added a commit that referenced this pull request Jun 25, 2025
This adds support for `map[string]any` and also arrays in general
(as opposed to just slices) to the converted map.

Like `mapstr.M` and `[]mapstr.M`, this will dive into the
`map[string]any` and `[]map[string]any` objects to ensure that
they are properly converted for the pdata code.

(cherry picked from commit 7bfa78e)

Co-authored-by: Chris Earle <pickypg@users.noreply.github.com>
@pickypg pickypg added the release-note:skip The PR should be ignored when processing the changelog label Jun 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-active-all Automated backport with mergify to all the active branches release-note:skip The PR should be ignored when processing the changelog Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[OTel Mapper] Fails to convert nested arrays and slices to []any

5 participants