Stabilize Logger.Enabled #4208

pellared · 2024-09-10T17:17:09Z

Stabilize Logger.Enabled API

Blockers:

pellared · 2024-09-13T05:52:52Z

Question to @open-telemetry/technical-committee: Do we want to stabilize the Logger.Enabled API sooner than we stabilize the spec defining how SDK implements it? Or do we want to stabilize Enabled for API and SDK at the same time?

cijothomas · 2024-09-20T01:09:30Z

Question to @open-telemetry/technical-committee: Do we want to stabilize the Logger.Enabled API sooner than we stabilize the spec defining how SDK implements it? Or do we want to stabilize Enabled for API and SDK at the same time?

Same for Metrics too: https://github.com/open-telemetry/opentelemetry-specification/pull/4219/files#r1767789558

pellared · 2024-10-02T07:30:23Z

The lack of stabilization of Logger.Enabled API blocks stabilization of OTel Go Logs.
Logger.Enabled API is required for bridging most popular Go logging libraries (including slog from the Go standard library).

From OTel Go perspective, the SDK support can be experimental. See: https://pkg.go.dev/go.opentelemetry.io/otel/sdk/log/internal/x.

This is currently the only known blocker for stabilizing the OTel Go Logs.

pellared · 2024-10-03T10:54:33Z

@open-telemetry/technical-committee, are you able to revalidate if the issues listed as blockers are still seen as blockers or if they can be addressed after stabilization of Logger.Enabled in Logs Bridge API?

Personally, I think the main blocker is to have at least 3 prototypes of the API in different languages.

tigrannajaryan · 2024-10-16T15:47:40Z

To clarify the process: we expect 3 prototypes in 3 different languages that can be used by the end users, so that they can try the feature, provide feedback, submit bugs and issues about it. This is a necessary process before the spec section is marked "Stable".

From this perspective a PR does not counts as a prototype since it is not easily usable by the end users. A PR is fine for proposing new experimental features and demonstrating how they would work, but it is not enough for stabilizing the spec.

The lack of stabilization of Logger.Enabled API blocks stabilization of OTel Go Logs.
Logger.Enabled API is required for bridging most popular Go logging libraries (including slog from the Go standard library).

@pellared you either need to find a way to have unstable APIs in Go or wait until other languages implement the prototypes. Either way the ability to have unstable APIs is very valuable and this is likely to come up again as Otel evolves and we keep adding new experimental APIs to existing signals.

--

As a side note: I encourage using maturity levels between "Development" and "Stable" to signal increasing level of confidence in the capability (both in the spec and the SDK). For example if we have 1-2 prototypes then we can move the maturity level of the feature from "Development" to "Alpha" or "Beta" to signal it is moving closer to the "Stable" state.

pellared · 2024-10-16T17:40:45Z

OK. See we need 3 different languages to have it released as experimental API.

Here is how the experimental Logger.Enabled API is currently defined in a 3 languages:

I will do my best to work on this with others to move this forward (as we have inconsistencies).

@pellared you either need to find a way to have unstable APIs in Go

All major log bridges need it so it does not even make sense to stabilize the rest as the Logs API would not be usable in Go ecosystem. From #3917:

Multiple logging libraries in Go provide this optimization¹²³⁴. If the Go SIG is going to be able to support these critical logging systems we need this functionality in the Logs Bridge API.

or wait until other languages implement the prototypes.

We then need to wait for other languages to add it.

As a side note: I encourage using maturity levels between "Development" and "Stable" to signal increasing level of confidence in the capability (both in the spec and the SDK)

I am not sure if we can use Alpha and Beta in the spec documents as these are not defined in https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/document-status.md#lifecycle-status.

tigrannajaryan · 2024-10-16T18:25:32Z

I am not sure if we can use Alpha and Beta in the spec documents as these are not defined in https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/document-status.md#lifecycle-status.

We can bring as many levels from OTEP 0232 to the spec as we will believe is useful. I started with 3 but we can bring more if we feel there is value. I personally think it can be valuable to have more granularity between Development (the most immature) and Stable (the most mature). It is an important signal and having just a binary value for it I think is not nuanced enough. Stabilization is a process, often a long one at Otel. As you move along that process it is important to indicate the progress by updating the level labels.

MrAlias · 2024-10-17T21:45:13Z

From this perspective a PR does not counts as a prototype since it is not easily usable by the end users. A PR is fine for proposing new experimental features and demonstrating how they would work, but it is not enough for stabilizing the spec.

FWIW this is a change in policy. Many features have been stabilized that relied on "Go implementations" which were just PRs.

I'm not sure it is fair to make this change in policy in such an ad hoc manner.

tigrannajaryan · 2024-10-17T23:08:45Z

@MrAlias my post is a result of a discussion by a few TCs while triaging this issue, so it is not an official policy change yet.

I tried finding the current policy but couldn't. This document does not seem to have an opinion about what criteria must be met before spec stabilization.

If anyone is aware of where do we state how many prototypes are needed please post the link.

If the policy does not exist in written form or we need to modify it I will create an issue so that we can discuss and formalize it.

Let's keep this issue open for now so that we can apply consistent rules after we clarify what the rules are.

tigrannajaryan · 2024-10-17T23:10:36Z

I am gonna move this back to TC inbox.

pellared · 2025-01-22T08:08:05Z

@jack-berg, I have question on what is required regarding the SDK implementation specification/design to unblock the stabilization of Logger.Enabled API.

Is an OTEP like this good enough: OTEP: Logger.Enabled #4290?
Or maybe addressing is good enough: Add min_severity to LoggerConfig #4364 would be good enough?
Or both of them are required (OTEP + some experimental feature in specification)?
Something else?

From our OTel Go experience, it is better to stabilize the API first and then gather feedback for months before stabilizing anything in the SDK. In OTel Go all our logging bridges use Logger.Enabled and we have an experimental support for adding hooks on the SDK side (as described in the OTEP). Nobody reported any issue or negative feedback since a few months.

The stabilization of Logger.Enabled on the API side is one of our main blockers for releasing a stable OTel Go Logs API and SDK.

pellared · 2025-01-22T08:14:10Z

I also want to call out that we already have 4 working prototypes of Logger.Enabled:

They have slight differences in the accepted parameters and I think this needs to be sorted out.

@open-telemetry/cpp-maintainers, @open-telemetry/php-maintainers, @open-telemetry/rust-maintainers, I need your help here.

jack-berg · 2025-01-22T19:52:08Z

@jack-berg, I have question on what is required regarding the SDK implementation specification/design to unblock the stabilization of Logger.Enabled API.

Honestly, I've been struggling to keep up with / track all the open issues / PRs related to this.

The current state of the operation is that enabled accepts context and severity number parameters. My original critique was that we shouldn't let API drift away, or stabilize, without corresponding features in the SDK. We could of overlook this requirement, given that the usefulness of the parameters seems pretty obvious, but I think doing so would set a bad precedent so I would vote against it.

Your OTEP #4290 tries to establish this corresponding SDK behavior. I agree with some of the content in there, and in particular, think that adding LoggerConfig#min_severity and LoggerConfig#trace_based properties are no brainers. I wonder if anyone would disagree? If not, then adding those two properties (along with extending #4381 to reference them) seems like the easiest way forward to me.

I also want to call out that we already have 4 working prototypes of Logger.Enabled:

Five - I originally proposed this experimental method with an accompanying java prototype 🙂

trask · 2025-01-22T20:09:36Z

adding LoggerConfig#min_severity and LoggerConfig#trace_based properties are no brainers. I wonder if anyone would disagree? If not, then adding those two properties (along with extending #4381 to reference them) seems like the easiest way forward to me.

👍

brettmc · 2025-01-22T23:47:16Z

I also want to call out that we already have 4 working prototypes of Logger.Enabled

PHP's implementation of enabled hasn't caught up to more recent spec changes, so is based on https://github.com/open-telemetry/opentelemetry-specification/blob/v1.34.0/specification/logs/bridge-api.md#enabled

When we do catch up, I'll follow the latest spec (although I might wait for this effort to complete first)

pellared · 2025-01-23T19:33:45Z

in particular, think that adding LoggerConfig#min_severity and LoggerConfig#trace_based properties are no brainers. I wonder if anyone would disagree?

I do not see them as no brainers. I am personally not convinced that these are good proposals.

Regarding LoggerConfig#min_severity: #4290 (comment)

Regarding LoggerConfig#trace_based: #4290 (comment)

I find adding opt-in Enabled to LogRecordProcessor more flexible, cohesive, composable. I think this is the route I will try to pursue even though I am aware that it will be harder for me, I think this is simply a better design and the way to go.

jack-berg · 2025-01-30T00:07:23Z

I think this is simply a better design and the way to go.

I've said something to this effect in other comments, but extending LogRecordProcessor with enabled and the work to keep LogRecordProcessor mutations isolated from each other doesn't have precedence in the tracing or metrics signals. Generally, its trying to bring a collector-style pipeline paradigm to the SDKs.

We could do this. Some users will benefit from it, preferring to do pipeline style work in the SDK vs. the collector. But most users will have a single batch log record processor paired with the OTLP exporter. And these users will want / expect easy ways to configure which logs make it into their pipeline. Filtering logs by severity is table stakes for any log system. For a system like OpenTelemetry that prioritizes correlation across signals, filtering logs based on whether the active span is sampled is obvious low hanging fruit.

So if we need some mechanism for filtering logs by severity and trace context, the next question is where does that configuration mechanism live. Options:

At the logger provider level, analog to how exemplars are configurable at the meter provider level. This is too broad a brush stroke for severity. Maybe trace context could get away with a global config like this - not sure.
At the logger level (i.e. LoggerConfig). This gives users plenty of granularity, while also having the ability to paint broad brush strokes since scope config allows you to do pattern matching of the form "set min severity to info for all loggers matching 'foo*'". Downside is that logger config would apply to all processors: in the event that SDKs go in a pipeline direction, the logs that a pipeline would get would be the intersection of the records that pass LoggerConfig criteria and the LogProcessor criteria. Not ideal to have competing config concepts.
At the processor level. This is your proposal - bringing pipelines to the SDK and giving each processor to decide which log records they are interested in. Processor A wants all logs from the "foo" logger at severity "info" or higher; Processor B wants all logs from the "foo" logger at severity "debug" or higher.

I'm opposed to bringing proper pipeline support to SDKs (its currently possible but you have to jump through hoops) because its a large implementation burden that has to be paid 11 times (once for each language implementation), and it duplicates the capabilities of the general purpose and extremely powerful collector. From a prioritization standpoint, asking resource-constrained language maintainers to implement better pipeline tooling doesn't seem like a good use of time right now, given all the other project objectives - especially semconv and stable instrumentation.

Not bringing proper pipeline support to SDKs means the users who want to do collector-style things in SDKs have to jump through more hoops with worse ergonomics. This isn't ideal, but tradeoffs.

If the community decides to that bringing proper pipeline support to SDKs is important, I do think its important to solve it holistically, and look at how the concepts apply to traces and metrics as well as logs.

trask · 2025-01-30T00:39:41Z

If the community decides to that bringing proper pipeline support to SDKs is important, I do think its important to solve it holistically, and look at how the concepts apply to traces and metrics as well as logs.

👍

Fixes #4363 Towards #4208 (uses Severity Level passed via Logger.Enabled) Towards stabilization of OpenTelemetry Go Logs API and SDK. ## Use cases Below are some use cases where the new functionality can be used: 1. Bridge features like `LogLevelEnabled` in log bridge/appender implementations. This is needed for **all** (but one) currently supported log bridges in OTel Go Contrib. 2. Configure a minimum log severity level for a certain log processor. 3. Filter out log and event records when they are inside a span that has been sampled out (span is valid and has sampled flag of `false`). 4. **Efficiently** support high-performance logging destination like [Linux user_events](https://docs.kernel.org/trace/user_events.html) and [ETW (Event Tracing for Windows)](https://learn.microsoft.com/windows/win32/etw/about-event-tracing). 5. Bridge Logs API to a language-specific logging library (the other way than usual). ## Changes Add `Enabled` opt-in operation to the `LogRecordProcessor`. I created an OTEP first which was a great for having a lot of discussions and evaluations of different proposals: - #4290 Most importantly from #4290 (comment): > Among Go SIG we were evaluating a few times an alternative to provide some new "filter" abstraction which is decoupled from the "processor". However, we faced more issues than benefits going this route (some if this is described here, but there were more issues: open-telemetry/opentelemetry-go#5825 (comment) . With the current opt-in `Processor.Enabled` we faced less issues so far. > We also do not want to replicate all features from the logging libraries. If someone prefer the log4j (or other) filter design then someone can always use a bridge and use log4j for filtering. `Enabled` callback hook is the simplest design (yet very flexible) which makes it easy to implement in the SDKs. This design is inspired from the design of the two most popular Go structured logging libraries: https://pkg.go.dev/log/slog (standard library) and https://pkg.go.dev/go.uber.org/zap. > > It is worth to adding that Rust design is similar and it also has an `Enabled` hook. See #4363 (comment). Basically we want to add something like https://docs.rs/log/latest/log/trait.Log.html#tymethod.enabled to the `LogRecordProcessor` and allow users to implement `Enabled` in the way that it will meet their requirements. > > I also want to call out form https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/0265-event-vision.md#open-questions: > > > How to support routing logs from the Logs API to a language-specific logging library > > To support this we would need a log record processor which bridges the Logs API calls to given logging library. For such case we would need an `Enabled` hook in `Processor` to efficiently bridge `Logger.Enabled` calls. A filterer design would not satisfy such use case. I decided to name the new operation `Enabled` as: 1. this name is already used in logging libraries in many languages: #4439 (comment) 2. it matches the name of the API call (for all trace, metrics and logs APIs). I was also considering `OnEnabled` to have the same pattern as for `Emit` and `OnEmit`. However, we already have `ForceFlush` and `Shutdown` which does not follow this pattern so I preferred to keep the simple `Enabled` name. For `OnEmit` I could also imagine `OnEmitted` (or `OnEmitting`) which does something after (or just before like we have `OnEnding` in `SpanProcessor`) `OnEmit` on all registered processors were called. Yet, I do not imagine something similar for `Enabled` as calling `Enabled` should not have any side-effects. Therefore, I decided to name it `Enabled`. I want to highlight that a processor cannot assume `Enabled` was called before `OnEmit`, because of the following reasons: 1. **Backward compatibility** – Existing processors may already perform filtering without relying on `Enabled`. For example: [Add Advanced Processing to Logs Supplementary Guidelines #4407](#4407). 2. **Self-sufficiency of `OnEmit`** – Since `Enabled` is optional, `OnEmit` should be able to handle filtering independently. A processor filtering events should do so in `OnEmit`, not just in `Enabled`. 3. **Greater flexibility** – Some processors, such as the ETW processor, don’t benefit from redundant filtering. ETW already filters out events internally, making an additional check unnecessary. 4. **Performance considerations** – Calling `Enabled` from `OnEmit` introduces overhead, as it requires converting `OnEmit` parameters to match `Enabled`'s expected input. 5. **Avoiding fragile assumptions** – Enforcing constraints that the compiler cannot validate increases the risk of introducing bugs. This feature is already implemented in OpenTelemetry Go: - open-telemetry/opentelemetry-go#6317 We have one processor in Contrib which takes advantage of this functionality: - https://pkg.go.dev/go.opentelemetry.io/contrib/processors/minsev This feautre (however with some differences) is also avaiable in OTel Rust; #4363 (comment): > OTel Rust also has this capability. Here's an example where it is leveraged to improve performance by dropping unwanted log early. https://github.com/open-telemetry/opentelemetry-rust/blob/88cae2cf7d0ff54a042d281a0df20f096d18bf82/opentelemetry-appender-tracing/benches/logs.rs#L78-L85 --------- Co-authored-by: Tyler Yahn <[email protected]> Co-authored-by: Sam Xie <[email protected]>

pellared · 2025-03-26T15:36:39Z

According to spec compliance matrix it is implemented in 3 languages:

opentelemetry-specification/spec-compliance-matrix.md

Line 202 in d203575

    
           | Logger.Enabled                               | X        | +   |      |     |        |      |        |     | +    | +   |      |       |

I saw that Logger.Enabled is also added to PHP (which gives 4 languages):

https://github.com/open-telemetry/opentelemetry-php/blob/015da800aa005d69f1b54a54c28af9173e236c7f/src/API/Logs/LoggerInterface.php#L15-L20

@open-telemetry/technical-committee, are there any required steps to mark this as stable?

jack-berg · 2025-03-26T15:53:43Z

Did a quick peak at the go, rust, cpp, php implementations:

Go - 👍
Rust - has an event_enabled method which appears to accept severity and event name arguments. Not sure if this is supposed to represented Logger.Enabled
cpp - 👍 also accepts an EventId which I don't quite understand - maybe a cpp specific construct?
php - doesnt accept any parameters
java - Java also has a enabled method, but I didn't fill out the compliance table because its still experimental and that's a grey area. It doesn't accept any parameters yet, but will be straight forward to add.

Personally, I'm comfortable stabilizing Logger#Enabled even if the prototype implementations don't currently all perfectly represent the spec. The parameters are intuitive and should be easy to add to implementations where they are missing.

cijothomas · 2025-03-26T16:03:33Z

Rust - has an event_enabled method which appears to accept severity and event name arguments. Not sure if this is supposed to represented Logger.Enabled

Otel Rust added more parameters than the spec has. The extra parameters are compile time generated/static in most logging libraries in Rust and is readily available. So it won't cost extra, but allows more advanced filtering.

pellared · 2025-03-26T16:05:31Z

@jack-berg, thanks. I think it would be beneficial to have approval from at least one maintainer per language listed above. If nobody objects I can create a PR tomorrow.

cijothomas · 2025-03-26T16:05:44Z

cpp - 👍 also accepts an EventId which I don't quite understand - maybe a cpp specific construct?

Yes. Otel C++'s logging API was meant for end-user usage too, even before Spec made that relaxation recently.
(From Rust side, we wish to have an EventId too, but none of the existing libraries in Rust has that concept)

pellared · 2025-03-26T16:07:28Z

According to #4208 (comment) PHP should also be fine. @brettmc, am I correct?

brettmc · 2025-03-26T22:18:54Z

According to #4208 (comment) PHP should also be fine. @brettmc, am I correct?

@pellared correct, that won't be an issue for PHP.

pellared added the spec:logs Related to the specification/logs directory label Sep 10, 2024

pellared mentioned this issue Sep 10, 2024

Define Enabled parameters for Logger #4203

Merged

pellared added this to Go: Logs (GA) Sep 10, 2024

github-project-automation bot moved this to Todo in Go: Logs (GA) Sep 10, 2024

pellared moved this from Todo to Blocked in Go: Logs (GA) Sep 10, 2024

pellared mentioned this issue Sep 13, 2024

[Spike] sdk/log: Add Logger.Enabled and EnabledParameters open-telemetry/opentelemetry-go#5816

Closed

trask added the triage:deciding:community-feedback Open to community discussion. If the community can provide sufficient reasoning, it may be accepted label Sep 17, 2024

pellared changed the title ~~Stabilize Logs Enabled~~ Stabilize Logger.Enabled Oct 1, 2024

github-actions bot added the triage:followup Needs follow up during triage label Oct 2, 2024

pellared added this to Logs SIG Oct 2, 2024

This was referenced Oct 2, 2024

Make severity required to allow filtering by severity as early as possible #4159

Open

Specify how Logs SDK implements Enabled #4207

Closed

pellared self-assigned this Oct 15, 2024

tigrannajaryan self-assigned this Oct 16, 2024

tigrannajaryan removed the triage:deciding:tc-inbox Needs attention from the TC in order to move forward label Oct 16, 2024

tigrannajaryan added the triage:deciding:tc-inbox Needs attention from the TC in order to move forward label Oct 17, 2024

jack-berg removed the triage:deciding:tc-inbox Needs attention from the TC in order to move forward label Oct 23, 2024

svrnm added the triage:accepted:ready-with-sponsor Ready to be implemented and has a specification sponsor assigned label Nov 25, 2024

github-project-automation bot added this to Specification + OTEPs Nov 25, 2024

pellared added the maintainer-request Escalated by SIG maintainers label Jan 21, 2025

pellared mentioned this issue Jan 22, 2025

Define how SDK implements Logger.Enabled #4381

Merged

pellared mentioned this issue Mar 4, 2025

Add opt-in LogRecordProcessor.Enabled #4439

Merged

pellared moved this from Blocked to Todo in Go: Logs (GA) Mar 25, 2025

pellared moved this from Todo to In Progress in Go: Logs (GA) Mar 26, 2025

pellared moved this from Todo to In progress in Logs SIG Mar 26, 2025

pellared mentioned this issue Mar 27, 2025

Stabilize Logger.Enabled #4463

Merged

reyang closed this as completed in #4463 Apr 2, 2025

github-project-automation bot moved this from In Progress to Done in Go: Logs (GA) Apr 2, 2025

github-project-automation bot moved this to Spec - Closed in Specification + OTEPs Apr 2, 2025

github-project-automation bot moved this from In progress to Done in Logs SIG Apr 2, 2025

brettmc mentioned this issue Apr 14, 2025

stabilize logger.enabled open-telemetry/opentelemetry-php#1563

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stabilize Logger.Enabled #4208

Stabilize Logger.Enabled #4208

pellared commented Sep 10, 2024 •

edited

Loading

pellared commented Sep 13, 2024

cijothomas commented Sep 20, 2024

pellared commented Oct 2, 2024 •

edited

Loading

pellared commented Oct 3, 2024 •

edited

Loading

tigrannajaryan commented Oct 16, 2024

pellared commented Oct 16, 2024 •

edited

Loading

tigrannajaryan commented Oct 16, 2024

MrAlias commented Oct 17, 2024 •

edited

Loading

tigrannajaryan commented Oct 17, 2024

tigrannajaryan commented Oct 17, 2024

pellared commented Jan 22, 2025 •

edited

Loading

pellared commented Jan 22, 2025 •

edited

Loading

jack-berg commented Jan 22, 2025

trask commented Jan 22, 2025

brettmc commented Jan 22, 2025

pellared commented Jan 23, 2025

jack-berg commented Jan 30, 2025

trask commented Jan 30, 2025

pellared commented Mar 26, 2025

jack-berg commented Mar 26, 2025 •

edited

Loading

cijothomas commented Mar 26, 2025

pellared commented Mar 26, 2025

cijothomas commented Mar 26, 2025

pellared commented Mar 26, 2025

brettmc commented Mar 26, 2025

Stabilize Logger.Enabled #4208

Stabilize Logger.Enabled #4208

Comments

pellared commented Sep 10, 2024 • edited Loading

pellared commented Sep 13, 2024

cijothomas commented Sep 20, 2024

pellared commented Oct 2, 2024 • edited Loading

pellared commented Oct 3, 2024 • edited Loading

tigrannajaryan commented Oct 16, 2024

pellared commented Oct 16, 2024 • edited Loading

Footnotes

tigrannajaryan commented Oct 16, 2024

MrAlias commented Oct 17, 2024 • edited Loading

tigrannajaryan commented Oct 17, 2024

tigrannajaryan commented Oct 17, 2024

pellared commented Jan 22, 2025 • edited Loading

pellared commented Jan 22, 2025 • edited Loading

jack-berg commented Jan 22, 2025

trask commented Jan 22, 2025

brettmc commented Jan 22, 2025

pellared commented Jan 23, 2025

jack-berg commented Jan 30, 2025

trask commented Jan 30, 2025

pellared commented Mar 26, 2025

jack-berg commented Mar 26, 2025 • edited Loading

cijothomas commented Mar 26, 2025

pellared commented Mar 26, 2025

cijothomas commented Mar 26, 2025

pellared commented Mar 26, 2025

brettmc commented Mar 26, 2025

pellared commented Sep 10, 2024 •

edited

Loading

pellared commented Oct 2, 2024 •

edited

Loading

pellared commented Oct 3, 2024 •

edited

Loading

pellared commented Oct 16, 2024 •

edited

Loading

MrAlias commented Oct 17, 2024 •

edited

Loading

pellared commented Jan 22, 2025 •

edited

Loading

pellared commented Jan 22, 2025 •

edited

Loading

jack-berg commented Mar 26, 2025 •

edited

Loading