Skip to content

doc: add detail to confmap watcher #12150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 32 additions & 4 deletions confmap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ that can be used by code that is oblivious to the usage of `Providers` and `Conv
or an individual value (partial configuration) when the `configURI` is embedded into the `Conf` as a values using
the syntax `${configURI}`.

**Limitation:**
**Limitation:**
- When embedding a `${configURI}` the uri cannot contain dollar sign ("$") character unless it embeds another uri.
- The number of URIs is limited to 100.

Expand Down Expand Up @@ -87,7 +87,26 @@ The `Resolve` method proceeds in the following steps:
After the configuration was processed, the `Resolver` can be used as a single point to watch for updates in the
configuration retrieved via the `Provider` used to retrieve the “initial” configuration and to generate the “effective” one.

```terminal
```terminal
Resolver Provider
│ │
Watch │ │
───────────►│ │
│ │
. .
. .
. .
│ onChange │
│◄────────────────────┤
◄───────────┤ │

```

The `Resolver` does that by passing an `onChange` func to each `Provider.Retrieve` call and capturing all watch events.

Calling the `onChange` func from a provider triggers the collector to re-resolve new configuration:

```terminal
Resolver Provider
│ │
Watch │ │
Expand All @@ -98,17 +117,26 @@ configuration retrieved via the `Provider` used to retrieve the “initial” co
. .
│ onChange │
│◄────────────────────┤
◄───────────┤ │
| |
Resolve │ │
───────────►│ │
│ │
│ Retrieve │
├────────────────────►│
│ Conf │
│◄────────────────────┤
◄───────────┤ │
```

The `Resolver` does that by passing an `onChange` func to each `Provider.Retrieve` call and capturing all watch events.
An example of a `Provider` with an `onChange` func that periodically gets notified can be found in provider_test.go as UpdatingProvider

## Troubleshooting

### Null Maps

Due to how our underlying merge library, [koanf](https://github.com/knadh/koanf), behaves, configuration resolution
will treat configuration such as
will treat configuration such as

```yaml
processors:
Expand Down
2 changes: 1 addition & 1 deletion confmap/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ type Provider interface {
// - For testing, all implementation MUST check that confmaptest.ValidateProviderScheme returns no error.
//
// `watcher` callback is called when the config changes. watcher may be called from
// a different go routine. After watcher is called Retrieved.Get should be called
// a different go routine. After watcher is called, Provider.Retrieve should be called
// to get the new config. See description of Retrieved for more details.
// watcher may be nil, which indicates that the caller is not interested in
// knowing about the changes.
Expand Down
73 changes: 73 additions & 0 deletions confmap/provider_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,86 @@ package confmap
import (
"context"
"errors"
"fmt"
"testing"
"time"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

// This is an example of a provider that calls a provided WatcherFunc to update configuration dynamically every second.
// The example is useful for implementing Providers of configuration that changes over time.
type UpdatingProvider struct{}

func (p UpdatingProvider) getCurrentConfig(_ string) any {
return "hello"
}

func (p UpdatingProvider) Retrieve(ctx context.Context, uri string, watcher WatcherFunc) (*Retrieved, error) {
ticker := time.NewTicker(1 * time.Second)
stop := make(chan bool, 1)

retrieved, err := NewRetrieved(p.getCurrentConfig(uri), WithRetrievedClose(func(_ context.Context) error {
// the retriever should call this function when it no longer wants config updates
ticker.Stop()
stop <- true
return nil
}))
if err != nil {
return nil, err
}
// it's necessary to start a go function that can notify the caller of changes asynchronously
go func() {
for {
select {
case <-ctx.Done():
// if the context is closed, then we should stop sending updates
ticker.Stop()
return
case <-stop:
// closeFunc was called, so stop updating the watcher
return
case <-ticker.C:
// the configuration has "changed". Notify the watcher that a new config is available
// the watcher is expected to call Provider.Retrieve again to get the update
// note that the collector calls closeFunc before calling Retrieve for the second time,
// so these go functions don't accumulate indefinitely. (see otelcol/collector.go, Collector.reloadConfiguration)
watcher(&ChangeEvent{})
}
}
}()

return retrieved, nil
}

func ExampleProvider() {
provider := UpdatingProvider{}

receivedNotification := make(chan bool)

watcherFunc := func(_ *ChangeEvent) {
fmt.Println("received notification of new config")
receivedNotification <- true
}

retrieved, err := provider.Retrieve(context.Background(), "example", watcherFunc)
if err != nil {
fmt.Println("received an error")
} else {
fmt.Printf("received: %s\n", retrieved.rawConf)
}

// after one second, we should receive a notification that config has changed
<-receivedNotification
// signal that we no longer want updates
retrieved.Close(context.Background())

// Output:
// received: hello
// received notification of new config
}

func TestNewRetrieved(t *testing.T) {
ret, err := NewRetrieved(nil)
require.NoError(t, err)
Expand Down
Loading