Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow implementations to export log records with duplicate keys in a map as opt-in #3987

Merged
merged 25 commits into from
Apr 26, 2024

Conversation

pellared
Copy link
Member

@pellared pellared commented Apr 5, 2024

Fixes #3931

Per agreement: #3931 (comment)

The SDKs should handle the key-value deduplication by default. It is acceptable to add an option to disable deduplication.

Previous PR: #3938

I think it is fine to do the deduplication anywhere you want as long as externally observable data complies with this document.

The main purpose of this PR is to have an agreement for following questions (and update the specification to to make it more clear):

  1. Is the deduplication required for all log exporters or only OTLP log exporters? Answer: It is required for all exporters.
  2. Can the key-value deduplication for log records be opt-in? Answer: Yes, it is OK as long as it is documented that it can cause problems in case maps duplicated keys are exported.

Related to:

@pellared pellared closed this Apr 5, 2024
@pellared pellared reopened this Apr 5, 2024
@pellared pellared marked this pull request as ready for review April 6, 2024 05:54
@pellared pellared requested review from a team April 6, 2024 05:54
@pellared
Copy link
Member Author

pellared commented Apr 6, 2024

CC @open-telemetry/dotnet-maintainers @open-telemetry/rust-maintainers @open-telemetry/cpp-maintainers @open-telemetry/go-maintainers

@pellared pellared requested a review from joaopgrassi April 8, 2024 13:34
@pellared
Copy link
Member Author

pellared commented Apr 8, 2024

@open-telemetry/specs-logs-approvers PTAL

@pellared pellared changed the title Clarify map representation in Logs Data Model [WIP] Clarify map representation in Logs Data Model Apr 9, 2024
@pellared
Copy link
Member Author

pellared commented Apr 9, 2024

Changing to draft until we have some agreement on #3931 (comment)

@pellared pellared marked this pull request as draft April 9, 2024 07:34
@pellared pellared changed the title [WIP] Clarify map representation in Logs Data Model Logs: allow SDK to have duplicated keys in a map as opt-in Apr 10, 2024
@pellared pellared changed the title Logs: allow SDK to have duplicated keys in a map as opt-in Allow SDK to export log records with duplicated keys in a map as opt-in Apr 10, 2024
specification/logs/data-model.md Outdated Show resolved Hide resolved
@pellared pellared changed the title Allow SDK to export log records with duplicated keys in a map as opt-in Allow SDK to export log records with duplicate keys in a map as opt-in Apr 16, 2024
specification/logs/data-model.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
specification/logs/data-model.md Outdated Show resolved Hide resolved
specification/logs/data-model.md Outdated Show resolved Hide resolved
@pellared pellared changed the title Allow SDK to export log records with duplicate keys in a map as opt-in Allow implementations to export log records with duplicate keys in a map as opt-in Apr 18, 2024
@pellared pellared requested a review from MrAlias April 18, 2024 15:49
Copy link
Member

@cijothomas cijothomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pellared for working passionately to drive clarity and strive for performance!

@tigrannajaryan
Copy link
Member

@open-telemetry/specs-logs-approvers please review.

@tigrannajaryan
Copy link
Member

Merging. Feel free to suggest further refinements.

@tigrannajaryan tigrannajaryan merged commit 95fc80a into open-telemetry:main Apr 26, 2024
7 checks passed
@pellared pellared deleted the patch-14 branch April 26, 2024 17:13
carlosalberto pushed a commit to carlosalberto/opentelemetry-specification that referenced this pull request Oct 31, 2024
…map as opt-in (open-telemetry#3987)

Fixes open-telemetry#3931

Per agreement: open-telemetry#3931 (comment)

> The SDKs should handle the key-value deduplication by default. It is acceptable to add an option to disable deduplication.

Previous PR: open-telemetry#3938


> I think it is fine to do the deduplication anywhere you want as long as externally observable data complies with this document.

The main purpose of this PR is to have an agreement for following questions (and update the specification to to make it more clear):
1. Is the deduplication required for all log exporters or only OTLP log exporters? Answer: It is required for all exporters.
2. Can the key-value deduplication for log records be opt-in? Answer: Yes, it is OK as long as it is documented that it can cause problems in case maps duplicated keys are exported.

Related to:
- open-telemetry/opentelemetry-go#5086
- open-telemetry/opentelemetry-dotnet#4324
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

logs: Allow duplicate keys