Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
a9d8b20
envcar: ignore non-normalized env names on Carrier.Get and Carrier.Keys
pellared Jun 9, 2026
0d68660
add PR number
pellared Jun 9, 2026
a55d64e
Potential fix for pull request finding
pellared Jun 9, 2026
4886024
skip some tests on windows
pellared Jun 9, 2026
da59bd7
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 9, 2026
c570540
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 9, 2026
d74740b
examples to restore the original value when it existed
pellared Jun 9, 2026
651c756
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 11, 2026
f2dd0db
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 11, 2026
6cb0baa
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 14, 2026
bec22d3
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 18, 2026
ca0cacf
avoid environment scan in Carrier.Get
pellared Jun 18, 2026
5b1606f
add environment carrier operational guidance
pellared Jun 18, 2026
0fedf62
handle empty env key normalization
pellared Jun 18, 2026
f57490e
isolate non-normalized env tests
pellared Jun 18, 2026
de3aca6
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 19, 2026
fd398ac
optimize caching
pellared Jun 20, 2026
c08dd30
git pushMerge branch 'env-var-get-keys-dont-normalize' of github.com:…
pellared Jun 20, 2026
8c69898
rename cache
pellared Jun 20, 2026
39c204e
update GoDoc
pellared Jun 20, 2026
c484525
refactor tests
pellared Jun 20, 2026
591c0c3
fix TestCarrierKeys
pellared Jun 20, 2026
060e0d3
optimize fetch
pellared Jun 20, 2026
6a2ef55
add a note regarding Windows
pellared Jun 20, 2026
aeaccb0
Keys to read directly from environ
pellared Jun 20, 2026
ef3ca09
note that applications should extract context during startup
pellared Jun 20, 2026
8da7802
improve GoDoc
pellared Jun 20, 2026
19c35a0
refactor examples
pellared Jun 20, 2026
9dcb3ef
improve example comments
pellared Jun 20, 2026
d7db01a
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 22, 2026
9845fa3
Merge branch 'main' into env-var-get-keys-dont-normalize
pellared Jun 25, 2026
f15acc3
do not use SDK in ExampleCarrier_childProcess
pellared Jun 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

### Changed

- Use direct normalized-key lookups in `Carrier.Get` and `Carrier.Keys` in `go.opentelemetry.io/contrib/propagators/envcar`. (#9112)
- Update log bridge conversions to use attribute key-values instead of the removed log key-values in `go.opentelemetry.io/contrib/bridges/otellogr`, `go.opentelemetry.io/contrib/bridges/otellogrus`, `go.opentelemetry.io/contrib/bridges/otelslog`, and `go.opentelemetry.io/contrib/bridges/otelzap`. (#9180)
- The `Version()` function in `go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux` has been replaced by `const Version`. (#9076)
- Set `error.type` attribute instead of adding `exception` span events in `go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin`. (#8977)
Expand Down
92 changes: 41 additions & 51 deletions propagators/envcar/carrier.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,61 +6,52 @@ package envcar // import "go.opentelemetry.io/contrib/propagators/envcar"
import (
"os"
"strings"
"sync"

"go.opentelemetry.io/otel/propagation"
)

// Carrier is a TextMapCarrier that uses the environment variables as a
// storage medium for propagated key-value pairs. The keys are normalized
// before being used to access the environment variables.
// This is useful for propagating values that are set in the environment
// and need to be accessed by different processes or services.
// The keys are uppercased to avoid case sensitivity issues across different
// operating systems and environments.
// Carrier is a [propagation.TextMapCarrier] for environment variables.
//
// If you do not set SetEnvFunc, [Carrier.Set] will do nothing.
// Using [os.Setenv] here is discouraged as the environment should
// be immutable:
// https://opentelemetry.io/docs/specs/otel/context/env-carriers/#environment-variable-immutability
// [Carrier.Get] and [Carrier.Keys] read from the current process environment.
// [Carrier.Get] normalizes the key before lookup. [Carrier.Keys] only lists
// environment variable names that are already normalized.
//
// [Carrier.Set] writes through [Carrier.SetEnvFunc] with a normalized key. If
// SetEnvFunc is nil, [Carrier.Set] does nothing. This lets injection target the
// environment of a child process without mutating the current process
// environment. Using [os.Setenv] as SetEnvFunc is discouraged because
// applications should not modify their own context-related environment
// variables.
//
// Key name normalization is defined by the [OpenTelemetry specification].
//
// [OpenTelemetry specification]: https://opentelemetry.io/docs/specs/otel/context/env-carriers/#key-name-normalization
type Carrier struct {
// SetEnvFunc is the function that sets the environment variable.
// Usually, you want to set the environment variables for processes
// that are spawned by the current process.
// SetEnvFunc sets an environment variable for injected context.
// [Carrier.Set] calls SetEnvFunc with a normalized key.
//
// Set this to update the environment that will be passed to a child
// process. Leave it nil for an extract-only carrier.
SetEnvFunc func(key, value string)
values map[string]string
once sync.Once
}

// Compile time check that Carrier implements the TextMapCarrier.
var _ propagation.TextMapCarrier = (*Carrier)(nil)

// fetch runs once on first access, and stores the environment in the
// carrier.
func (c *Carrier) fetch() {
c.once.Do(func() {
environ := os.Environ()
c.values = make(map[string]string, len(environ))
for _, kv := range environ {
kvPair := strings.SplitN(kv, "=", 2)
key := normalize(kvPair[0])
c.values[key] = kvPair[1]
}
})
}

// Get returns the value associated with the normalized passed key.
// The first call to [Carrier.Get] or [Carrier.Keys] for a
// given Carrier will read and store the values from the
// environment and all future reads will be from that store.
func (c *Carrier) Get(key string) string {
c.fetch()
return c.values[normalize(key)]
// Get normalizes key and returns the corresponding value from the current
// process environment. It returns an empty string if the normalized
// environment variable is unset or set to an empty value.
//
// On platforms with case-insensitive environment lookup, such as Windows, the
// lookup may match an environment variable whose name differs from the
// normalized key only by case.
func (*Carrier) Get(key string) string {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the change back from store on first call? The Carrier was not supposed to (potentially) return different values for the same key, see the original PR for discussions on this.

@pellared pellared Jun 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check this discussion #9112 (comment)

This is also a related PR open-telemetry/opentelemetry-specification#5166

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long story short the appropriate usage is to use this carrier once near application startup.

Caching and enumeration causes unnecessary overhead that we want to avoid.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I unresolved the mentioned thread so that you can navigate to it

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean that Keys and Get can respond differently (if used contra to that startup expectation). Shouldn't we drop the store behaviour for Keys too? If the expectation is basically only one call to Keys ever then the store is pretty pointless.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get wanting to avoid enumerating all environment variables for efficiency, however Get now reads directly from the env vars.

The mutability of env vars is a concern of mine and I worry that reading directly from env vars without a cache could lead to unpredictable runtime bugs (eg. when an instrumented process emits several spans using the env var context as parent context and the vars were changed in between). Even instantiating the carrier once during startup would not protect against this.

The more I think about it, the more I want to add normative language to the spec like: Get MUST NOT return different values when called with the same argument implying the use of a cache.

@pellared pellared Jun 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Joibel, @kamphaus, thanks a lot for your questions/concerns. These are exactly the discussions that we need to have before going RC.

Does that mean that Keys and Get can respond differently (if used contra to that startup expectation). Shouldn't we drop the store behaviour for Keys too? If the expectation is basically only one call to Keys ever then the store is pretty pointless.

@Joibel, I agree that Keys and Get should have the same observation model. If the expected usage is extracting once near startup, then Keys will typically be called at most once for that extraction path (e.g. when using https://pkg.go.dev/go.opentelemetry.io/contrib/propagators/ot which is the only known propagator that uses Keys method).

Removing the Keys cache also seems consistent with the current intended usage, and I think it would avoid the confusing case where Get is live but Keys is a snapshot. It also looks fine for your vendored carrier usage here: https://github.com/argoproj/argo-workflows/blob/4cb7eb4e89299595ba8fddd3b7f9d95872c3b39d/workflow/executor/tracing/tracing.go#L26.

Addressed aeaccb0

Just to double-check. Would you see any benefit from caching Get and Keys, or would it just be unnecessary state/overhead?

Even instantiating the carrier once during startup would not protect against this.

@kamphaus, agreed that construction alone would not protect against later environment mutation. What I meant is that applications should extract from the environment once during startup and then use the extracted context, rather than repeatedly using the process environment as a live parent-context source.

Addressed ef3ca09

The more I think about it, the more I want to add normative language to the spec like: Get MUST NOT return different values when called with the same argument implying the use of a cache.

@kamphaus, I do not think we have evidence yet that this is something which needs to be normative.

For OTel Go, C++ and Rust, we usually care very much about any overhead, I think it is fine if we can expect users to not call Get more than once for the same argument.

At the same time, I think that for some languages it may be fine to cache. But maybe even for these languages it should be opt-in as it cause overhead that users may not want.

At this point of time, I lean towards the non-normative proposal from open-telemetry/opentelemetry-specification#5166

I do not take this lightly and I plan to further discuss this with maintainers of other language implementations.

@Joibel, WDYT?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not take this lightly and I plan to also discuss this with other language maintainers.

I agree that this needs to be discussed with other language maintainers and thank you for taking this seriously.

Might I suggest in the meantime to document in the godoc that applications using envcar/Carrier should cache the context based on it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might I suggest in the meantime to document in the godoc that applications using envcar/Carrier should cache the context based on it?

I did this before I read your comment 😉

PTAL ef3ca09

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL ef3ca09

And I reviewed it before seeing this.

return os.Getenv(normalize(key))
}

// Set stores the key-value pair in the environment variable.
// The key is normalized before being used to set the
// environment variable.
// Set stores the key-value pair by calling [Carrier.SetEnvFunc].
// The key is normalized before SetEnvFunc is called.
//
// If SetEnvFunc is not set, this method does nothing.
func (c *Carrier) Set(key, value string) {
if c.SetEnvFunc == nil {
Expand All @@ -70,16 +61,15 @@ func (c *Carrier) Set(key, value string) {
c.SetEnvFunc(k, value)
}

// Keys lists the keys stored in this carrier.
// This returns all the keys in the environment variables.
// The first call to [Carrier.Get] or [Carrier.Keys] for a
// given Carrier will read and store the values from the
// environment and all future reads will be from that store.
// The keys are returned in their normalized form.
func (c *Carrier) Keys() []string {
c.fetch()
keys := make([]string, 0, len(c.values))
for key := range c.values {
// Keys returns normalized environment variable names from the current process.
func (*Carrier) Keys() []string {
environ := os.Environ()
keys := make([]string, 0, len(environ))
for _, kv := range environ {
key, _, _ := strings.Cut(kv, "=")
if !normalized(key) {
continue
}
keys = append(keys, key)
}
return keys
Expand Down
27 changes: 19 additions & 8 deletions propagators/envcar/carrier_example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,20 @@ import (
"go.opentelemetry.io/contrib/propagators/envcar"
)

// This example is a go program where the environment variables are carrying the
// trace information, and we're going to pick them up into our context.
// An example where the environment variables are carrying the trace
// information, and we're going to pick them up into our context.
func ExampleCarrier_extractFromParent() {
// Simulate environment variables set by a parent process.
// In practice, these would already be set when this process starts.
// Simulate an environment variable set by a parent process. In practice,
// this would already be set when this process starts, and the application
// would extract context during initialization.
orig, ok := os.LookupEnv("TRACEPARENT")
defer func() {
if ok {
_ = os.Setenv("TRACEPARENT", orig)
return
}
_ = os.Unsetenv("TRACEPARENT")
}()
_ = os.Setenv("TRACEPARENT", "00-0102030405060708090a0b0c0d0e0f10-0102030405060708-01")

// Create a carrier to read trace context from environment variables.
Expand All @@ -29,7 +38,8 @@ func ExampleCarrier_extractFromParent() {
prop := propagation.TraceContext{}
ctx := prop.Extract(context.Background(), &carrier)

// The context now contains the span context from the parent.
// The context now contains the span context from the parent. Keep and pass
// this context instead of repeatedly extracting from the environment.
spanCtx := trace.SpanContextFromContext(ctx)
fmt.Printf("Trace ID: %s\n", spanCtx.TraceID())
fmt.Printf("Span ID: %s\n", spanCtx.SpanID())
Expand All @@ -40,8 +50,8 @@ func ExampleCarrier_extractFromParent() {
// Sampled: true
}

// This example is a go program where we have a trace and we'd like to inject it
// into a command we're going to run.
// An example where we have a trace and we'd like to inject it into a command
// we're going to run.
func ExampleCarrier_childProcess() {
// Create a span context with a known trace ID.
traceID := trace.TraceID{0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10}
Expand All @@ -54,6 +64,7 @@ func ExampleCarrier_childProcess() {
ctx := trace.ContextWithSpanContext(context.Background(), spanCtx)

// Prepare a command that prints the TRACEPARENT environment variable.
// Each child process gets its own copy of the environment.
cmd := exec.CommandContext(context.Background(), "printenv", "TRACEPARENT")
cmd.Env = os.Environ()

Expand All @@ -65,7 +76,7 @@ func ExampleCarrier_childProcess() {
},
}

// Inject trace context into the child's environment.
// Inject trace context into the child's environment copy.
prop := propagation.TraceContext{}
prop.Inject(ctx, &carrier)

Expand Down
95 changes: 66 additions & 29 deletions propagators/envcar/carrier_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ package envcar_test
import (
"os"
"os/exec"
"slices"
"runtime"
"strings"
"sync"
"testing"
Expand Down Expand Up @@ -61,19 +61,6 @@ func TestExtractValidTraceContextEnvCarrier(t *testing.T) {
Remote: true,
}),
},
{
name: "lowercase env names",
envs: map[string]string{
"traceparent": "00-000000000000007b00000000000001c8-000000000000007b-00",
"tracestate": stateStr,
},
want: trace.NewSpanContext(trace.SpanContextConfig{
TraceID: traceID,
SpanID: spanID,
TraceState: state,
Remote: true,
}),
},
}

for _, tc := range tests {
Expand All @@ -88,6 +75,21 @@ func TestExtractValidTraceContextEnvCarrier(t *testing.T) {
}
}

func TestExtractIgnoresNonNormalizedEnvNames(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Windows environment variables are case-insensitive, so this test is not applicable.")
}

unsetenv(t, "TRACEPARENT")
unsetenv(t, "TRACESTATE")
t.Setenv("traceparent", "00-000000000000007b00000000000001c8-000000000000007b-00")
t.Setenv("tracestate", "key1=value1,key2=value2")

ctx := prop.Extract(t.Context(), &envcar.Carrier{})

assert.False(t, trace.SpanContextFromContext(ctx).IsValid())
}

func TestInjectTraceContextEnvCarrier(t *testing.T) {
stateStr := "key1=value1,key2=value2"
state, err := trace.ParseTraceState(stateStr)
Expand Down Expand Up @@ -147,13 +149,16 @@ func TestInjectTraceContextEnvCarrier(t *testing.T) {
}

func TestCarrierKeys(t *testing.T) {
t.Setenv("traceparent", "value")
t.Setenv("TRACEPARENT", "value")
unsetenv(t, "ENVCAR_NON_NORMALIZED_KEY")
t.Setenv("envcar-non-normalized-key", "ignored")

Comment thread
pellared marked this conversation as resolved.
Comment thread
pellared marked this conversation as resolved.
Comment thread
pellared marked this conversation as resolved.
c := envcar.Carrier{}
keys := c.Keys()

assert.Contains(t, keys, "TRACEPARENT")
assert.NotContains(t, keys, "traceparent")
assert.NotContains(t, keys, "envcar-non-normalized-key")
assert.NotContains(t, keys, "ENVCAR_NON_NORMALIZED_KEY")
}
Comment thread
pellared marked this conversation as resolved.

func TestCarrierSetNilFunc(_ *testing.T) {
Expand All @@ -162,13 +167,22 @@ func TestCarrierSetNilFunc(_ *testing.T) {
}

func TestCarrierGetNormalizesKey(t *testing.T) {
t.Setenv("traceparent", "myvalue")
t.Setenv("TRACEPARENT", "myvalue")

c := envcar.Carrier{}
assert.Equal(t, "myvalue", c.Get("traceparent"))
assert.Equal(t, "myvalue", c.Get("TRACEPARENT"))
}

func TestCarrierGetIgnoresNonNormalizedEnvNames(t *testing.T) {
unsetenv(t, "ENVCAR_GET_NON_NORMALIZED_KEY")
t.Setenv("envcar-get-non-normalized-key", "ignored")

c := envcar.Carrier{}
assert.Empty(t, c.Get("envcar_get_non_normalized_key"))
assert.Empty(t, c.Get("ENVCAR_GET_NON_NORMALIZED_KEY"))
}

func TestCarrierSetUppercasesUnderscoresKey(t *testing.T) {
var gotKey string
var gotValue string
Expand All @@ -192,6 +206,29 @@ func TestCarrierSetUppercasesUnderscoresKey(t *testing.T) {
assert.Equal(t, "🧳", gotValue)
}

func TestCarrierGetReadsDirectly(t *testing.T) {
t.Setenv("TRACEPARENT", "myvalue")
c := envcar.Carrier{}
require.Equal(t, "myvalue", c.Get("TRACEPARENT"))

t.Setenv("TRACEPARENT", "bad")
t.Setenv("ENVCAR_GET_READS_DIRECTLY_NEW", "value")

assert.Equal(t, "bad", c.Get("TRACEPARENT"))
assert.Contains(t, c.Keys(), "ENVCAR_GET_READS_DIRECTLY_NEW")
}

func TestCarrierKeysReadsDirectly(t *testing.T) {
t.Setenv("TRACEPARENT", "myvalue")
c := envcar.Carrier{}
require.Contains(t, c.Keys(), "TRACEPARENT")

t.Setenv("ENVCAR_KEYS_FETCH_ONCE_NEW", "value")

assert.Contains(t, c.Keys(), "TRACEPARENT")
assert.Contains(t, c.Keys(), "ENVCAR_KEYS_FETCH_ONCE_NEW")
}

func TestConcurrentChildProcesses(t *testing.T) {
// Test that concurrent goroutines can each spawn child processes
// with their own unique trace context.
Expand Down Expand Up @@ -261,16 +298,16 @@ func TestConcurrentChildProcesses(t *testing.T) {
}
}

// Ensure Get and Keys to fetch from cache.
func TestCarrierGetFetchOnce(t *testing.T) {
t.Setenv("TRACEPARENT", "myvalue")
c := envcar.Carrier{}
require.Equal(t, "myvalue", c.Get("TRACEPARENT"))
require.NotEqual(t, -1, slices.Index(c.Keys(), "TRACEPARENT"))
t.Setenv("TRACEPARENT", "bad")
t.Setenv("SOMETHINGNEW", "bad")
// Assert a carrier instance reads the env vars only once:
assert.Equal(t, "myvalue", c.Get("TRACEPARENT"))
assert.NotEqual(t, -1, slices.Index(c.Keys(), "TRACEPARENT"))
assert.Equal(t, -1, slices.Index(c.Keys(), "SOMETHINGNEW"))
func unsetenv(t *testing.T, key string) {
t.Helper()

value, ok := os.LookupEnv(key)
require.NoError(t, os.Unsetenv(key))
t.Cleanup(func() {
if ok {
require.NoError(t, os.Setenv(key, value)) //nolint:usetesting // t.Setenv cannot restore after an explicit unset.
return
}
require.NoError(t, os.Unsetenv(key))
})
}
20 changes: 18 additions & 2 deletions propagators/envcar/doc.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

// Package envcar implements the Environment Carrier specification as documented
// here https://opentelemetry.io/docs/specs/otel/context/env-carriers/
// Package envcar implements the
// [Environment Variables as Context Propagation Carriers specification].
//
// Environment variable propagation is intended for process boundaries where
// network protocols are not available, such as batch jobs, CI/CD, and command
// line tools. Treat context-related environment variables as process-startup
// input.
//
// Applications should extract context from the environment once during startup
// and then use the extracted context. Repeated extraction uses the process
// environment as a live parent-context source and may observe later environment
// variable changes.
//
// Note that environment variables can be visible to code in the same process
// and, on many systems, to other users or processes with sufficient
// permissions. Do not use this carrier for sensitive context.
//
// [Environment Variables as Context Propagation Carriers specification]: https://opentelemetry.io/docs/specs/otel/context/env-carriers/
package envcar // import "go.opentelemetry.io/contrib/propagators/envcar"
Loading
Loading