Skip to content

[pkg/ottl] Add Coalesce converter for first-non-nil value selection#46862

Merged
atoulme merged 1 commit into
open-telemetry:mainfrom
57Ajay:ottl-add-coalesce-converter
Apr 2, 2026
Merged

[pkg/ottl] Add Coalesce converter for first-non-nil value selection#46862
atoulme merged 1 commit into
open-telemetry:mainfrom
57Ajay:ottl-add-coalesce-converter

Conversation

@57Ajay

@57Ajay 57Ajay commented Mar 12, 2026

Copy link
Copy Markdown
Contributor

fixes #46847

Description

Adds a Coalesce converter to OTTL that returns the first non-nil value from a list of arguments.

OTTL currently has no way to express "use the first non-nil value from a set of candidates" in a single statement. Users who need to resolve a canonical attribute from multiple possible sources must write multiple chained set statements with increasingly complex where clauses. This is verbose, ordering-dependent, and scales poorly.

Coalesce solves this:

# Before: 3 statements, fragile ordering
- set(attributes["user"], attributes["user.id"]) where attributes["user.id"] != nil
- set(attributes["user"], attributes["enduser.id"]) where attributes["user"] == nil and attributes["enduser.id"] != nil
- set(attributes["user"], "unknown") where attributes["user"] == nil

# After: 1 statement
- set(attributes["user"], Coalesce([attributes["user.id"], attributes["enduser.id"], "unknown"]))

Implementation

  • Registered as a standard OTTL Converter with []ottl.Getter[K] parameter (same pattern as Format).
  • Iterates getters left to right, returns the first non-nil result.
  • Returns nil if all values are nil.
  • No side effects, pure function, bounded execution — consistent with OTTL function design principles.
  • No grammar or parser changes.

Testing

Unit tests (func_coalesce_test.go): 13 test cases covering first-value-wins, nil-skip-to-second, nil-skip-to-third, all-nil, single-arg, int/bool/float type preservation, short-circuit evaluation (does not call getters past the first non-nil), getter error propagation, and factory validation (empty args, wrong type).

E2e tests (e2e/e2e_test.go): 3 test cases exercising Coalesce through the real OTTL parser — first-arg-wins, skip-nil-pick-second, all-nil-pick-literal.

Live collector test: Built otelcontribcol, ran the transform processor with 5 Coalesce statements against a real OTLP/HTTP log request. All 5 produced correct output:

-> test1: Str(alice)       # first arg non-nil
-> test2: Str(bob-legacy)  # first nil, second non-nil
-> test3: Str(fallback)    # all paths nil, literal wins
-> test4: Str(alice)       # single arg
-> test5: Int(42)          # first nil, int value wins

@57Ajay

57Ajay commented Mar 12, 2026

Copy link
Copy Markdown
Contributor Author

/rerun

@57Ajay 57Ajay force-pushed the ottl-add-coalesce-converter branch 2 times, most recently from f5265bb to 05aab32 Compare March 14, 2026 09:24

@57Ajay 57Ajay left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how file .chloggen/fix_gcp_snake_case.yaml started to give error, as it was added when i fixed issue #46588 where component name googlecloudlogentry_encoding was correct, now i guess it's name has been changed to google_cloud_logentry_encoding.
#46588 changes: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/46588/changes

@MohamedElDegwi MohamedElDegwi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @57Ajay, thanks for working on this. I've few questions I'd appreciate if you could clarify.

Isn't having multiple statements for different semconvs a migration problem not an ottl one? If understand this correctly, the growth of ottl statements is intended as alert of migration needed, so If I have 3 statements to cover canonical.user this is an indication that I need to sync/update my setup?

Do you have any use-case that require this function? From what I see, set + where is explicit, readable, and capable enough. See PR#46443 as an example.

Sorry If I've been bit skeptical haha :)

Comment thread .chloggen/fix_gcp_snake_case.yaml Outdated
@57Ajay

57Ajay commented Mar 15, 2026

Copy link
Copy Markdown
Contributor Author

Hi @57Ajay, thanks for working on this. I've few questions I'd appreciate if you could clarify.

Isn't having multiple statements for different semconvs a migration problem not an ottl one? If understand this correctly, the growth of ottl statements is intended as alert of migration needed, so If I have 3 statements to cover canonical.user this is an indication that I need to sync/update my setup?

Do you have any use-case that require this function? From what I see, set + where is explicit, readable, and capable enough. See PR#46443 as an example.

Sorry If I've been bit skeptical haha :)

Thanks @MohamedElDegwi, fair question.
The semconv migration case was just one example -- you're right that it could signal a migration need. But the more common use case is permanent multi-source fallback, where you receive data from heterogeneous sources you don't control (different SDKs, different systems) and need one canonical attribute for downstream querying. There's nothing to migrate in that case -- the diversity is the long-term reality.
On set + where -- it works for few candidates such as 1 or 2, but with more like 3+ the where clauses grow quadratically (each must check all prior candidates were nil), and reordering them silently changes behavior. Coalesce makes the intent explicit in one statement. It's the same reason SQL has COALESCE and JavaScript has ?? -- the pattern is common enough to warrant a primitive.
also Will remove the stale chloggen file in the next push.

@57Ajay 57Ajay force-pushed the ottl-add-coalesce-converter branch 2 times, most recently from 512123f to fd72454 Compare March 15, 2026 11:40
Comment thread .chloggen/fix_gcp_snake_case.yaml Outdated
@github-actions

Copy link
Copy Markdown
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions Bot added the Stale label Mar 30, 2026
@edmocosta edmocosta removed the Stale label Mar 30, 2026

@edmocosta edmocosta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for working on this, @57Ajay. LGTM!

Comment thread .chloggen/ottl-add-coalesce-converter.yaml Outdated
Comment thread pkg/ottl/ottlfuncs/README.md Outdated
@57Ajay 57Ajay force-pushed the ottl-add-coalesce-converter branch from e6143a9 to 832a1d3 Compare March 30, 2026 16:26

@57Ajay 57Ajay left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed with @edmocosta suggestions

@edmocosta edmocosta added the ready to merge Code review completed; ready to merge by maintainers label Apr 2, 2026
@atoulme atoulme merged commit dca15e4 into open-telemetry:main Apr 2, 2026
207 checks passed
@otelbot

otelbot Bot commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution @57Ajay! 🎉 We would like to hear from you about your experience contributing to OpenTelemetry by taking a few minutes to fill out this survey. If you are getting started contributing, you can also join the CNCF Slack channel #opentelemetry-new-contributors to ask for guidance and get help.

araiu pushed a commit to araiu/opentelemetry-collector-contrib that referenced this pull request Apr 6, 2026
…pen-telemetry#46862)

fixes open-telemetry#46847

### Description

Adds a `Coalesce` converter to OTTL that returns the first non-nil value
from a list of arguments.

OTTL currently has no way to express "use the first non-nil value from a
set of candidates" in a single statement. Users who need to resolve a
canonical attribute from multiple possible sources must write multiple
chained `set` statements with increasingly complex `where` clauses. This
is verbose, ordering-dependent, and scales poorly.

`Coalesce` solves this:
```yaml
# Before: 3 statements, fragile ordering
- set(attributes["user"], attributes["user.id"]) where attributes["user.id"] != nil
- set(attributes["user"], attributes["enduser.id"]) where attributes["user"] == nil and attributes["enduser.id"] != nil
- set(attributes["user"], "unknown") where attributes["user"] == nil

# After: 1 statement
- set(attributes["user"], Coalesce([attributes["user.id"], attributes["enduser.id"], "unknown"]))
```

### Implementation

- Registered as a standard OTTL Converter with `[]ottl.Getter[K]`
parameter (same pattern as `Format`).
- Iterates getters left to right, returns the first non-nil result.
- Returns nil if all values are nil.
- No side effects, pure function, bounded execution — consistent with
OTTL function design principles.
- No grammar or parser changes.

### Testing

**Unit tests** (`func_coalesce_test.go`): 13 test cases covering
first-value-wins, nil-skip-to-second, nil-skip-to-third, all-nil,
single-arg, int/bool/float type preservation, short-circuit evaluation
(does not call getters past the first non-nil), getter error
propagation, and factory validation (empty args, wrong type).

**E2e tests** (`e2e/e2e_test.go`): 3 test cases exercising `Coalesce`
through the real OTTL parser — first-arg-wins, skip-nil-pick-second,
all-nil-pick-literal.

**Live collector test**: Built `otelcontribcol`, ran the transform
processor with 5 Coalesce statements against a real OTLP/HTTP log
request. All 5 produced correct output:
```
-> test1: Str(alice)       # first arg non-nil
-> test2: Str(bob-legacy)  # first nil, second non-nil
-> test3: Str(fallback)    # all paths nil, literal wins
-> test4: Str(alice)       # single arg
-> test5: Int(42)          # first nil, int value wins
```

Signed-off-by: 57Ajay <57ajay.u@gmail.com>
AndrewCharlesHay pushed a commit to AndrewCharlesHay/opentelemetry-collector-contrib that referenced this pull request Apr 23, 2026
…pen-telemetry#46862)

fixes open-telemetry#46847

### Description

Adds a `Coalesce` converter to OTTL that returns the first non-nil value
from a list of arguments.

OTTL currently has no way to express "use the first non-nil value from a
set of candidates" in a single statement. Users who need to resolve a
canonical attribute from multiple possible sources must write multiple
chained `set` statements with increasingly complex `where` clauses. This
is verbose, ordering-dependent, and scales poorly.

`Coalesce` solves this:
```yaml
# Before: 3 statements, fragile ordering
- set(attributes["user"], attributes["user.id"]) where attributes["user.id"] != nil
- set(attributes["user"], attributes["enduser.id"]) where attributes["user"] == nil and attributes["enduser.id"] != nil
- set(attributes["user"], "unknown") where attributes["user"] == nil

# After: 1 statement
- set(attributes["user"], Coalesce([attributes["user.id"], attributes["enduser.id"], "unknown"]))
```

### Implementation

- Registered as a standard OTTL Converter with `[]ottl.Getter[K]`
parameter (same pattern as `Format`).
- Iterates getters left to right, returns the first non-nil result.
- Returns nil if all values are nil.
- No side effects, pure function, bounded execution — consistent with
OTTL function design principles.
- No grammar or parser changes.

### Testing

**Unit tests** (`func_coalesce_test.go`): 13 test cases covering
first-value-wins, nil-skip-to-second, nil-skip-to-third, all-nil,
single-arg, int/bool/float type preservation, short-circuit evaluation
(does not call getters past the first non-nil), getter error
propagation, and factory validation (empty args, wrong type).

**E2e tests** (`e2e/e2e_test.go`): 3 test cases exercising `Coalesce`
through the real OTTL parser — first-arg-wins, skip-nil-pick-second,
all-nil-pick-literal.

**Live collector test**: Built `otelcontribcol`, ran the transform
processor with 5 Coalesce statements against a real OTLP/HTTP log
request. All 5 produced correct output:
```
-> test1: Str(alice)       # first arg non-nil
-> test2: Str(bob-legacy)  # first nil, second non-nil
-> test3: Str(fallback)    # all paths nil, literal wins
-> test4: Str(alice)       # single arg
-> test5: Int(42)          # first nil, int value wins
```

Signed-off-by: 57Ajay <57ajay.u@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg/ottl ready to merge Code review completed; ready to merge by maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[pkg/ottl] Add Coalesce() converter for first-non-nil value selection

5 participants