Add option to programmatically customization declarative config components, add supplementary guidance

specification/configuration/README.md

@@ -55,6 +55,8 @@ Declarative configuration consists of the following main components:
   operations to parse configuration files and interpret the configuration data
   model.
 
+See also [supplementary guidelines](./supplementary-guidelines.md).
+
 ### Other Mechanisms
 
 Additional configuration mechanisms SHOULD be provided in whatever

specification/configuration/sdk.md

@@ -284,6 +284,29 @@ This SHOULD return an error if it encounters an error in `configuration` (i.e.
 fail fast) in accordance with
 initialization [error handling principles](../error-handling.md#basic-error-handling-principles).
 
+SDK implementations MAY provide callbacks to allow programmatic customization of
+the components initialized by `Create`. This allows configuration of concepts which
+are not yet or may never be representable in the configuration model. For example,
+java OTLP exporters allow configuration of the [ExecutorService](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ExecutorService.html),
+a niche but important option for applications which need strict control of thread pools.
+This programmatic customization might take the form of passing an optional callback to
+`Create`, invoked with each SDK subcomponent (or a subset of SDK component types) as
+they are initialized. For example, consider the following snippet:
+
+```yaml
+file_format: 1.0
+tracer_provider:
+  processors:
+    - batch:
+        exporter:
+          otlp_http:
+```
+
+The callback would be invoked with the SDK representation of an OTLP HTTP exporter, a
+Batch SpanProcessor, and a Tracer Provider. This pattern provides the opportunity
+to programmatically configure lower-level without needing to walk to a particular
+component from the resolved top level SDK components.
+
 TODO: define behavior if some portion of configuration model is not supported
 
 #### Register ComponentProvider

specification/configuration/supplementary-guidelines.md

@@ -0,0 +1,65 @@
+# Supplementary Guidelines
+
+Note: this document is NOT a spec, it is provided to support the declarative config
+[API](./api.md) and [SDK](./sdk.md) specifications, it does NOT add any extra
+requirements to the existing specifications.
+
+<details>
+<summary>Table of Contents</summary>
+
+<!-- toc -->
+
+- [Configuration interface prioritization and `create`](#configuration-interface-prioritization-and-create)
+- [Programmatic customization and `create`](#programmatic-customization-and-create)
+
+<!-- tocstop -->
+
+</details>
+
+## Configuration interface prioritization and `create`
+
+With the [environment variable](./sdk-environment-variables.md) configuration
+interface, the spec failed to answer the question of whether programmatic or
+environment variable configuration took precedence. This led to differences in
+implementations that were ultimately stabilized and difficult to resolve after
+the fact.
+
+With declarative config, we don't have ambiguity around configuration interface
+precedence:
+
+* [`parse`](./sdk.md#parse) is responsible for parsing config file contents and
+  returning the corresponding in-memory data model. Along the way, it
+  performs [environment variable substitution](./data-model.md#environment-variable-substitution).
+* [`create`](./sdk.md#create) is responsible for interpreting an in-memory
+  config data model and creating SDK components.
+
+There is no precedence ambiguity with the environemnt variable configuration
+interface: The language of `parse` and `create` is explicit about
+responsibilities and makes no mention of merging environment variables outside
+of environment variable substitution.
+Furthermore, [OTEL_EXPERIMENTAL_CONFIG_FILE](./sdk-environment-variables.md#declarative-configuration)
+explicitly states that the environment variable configuration scheme is ignored.
+
+There is no precedence ambiguity with
+the [programmatic configuration interface](./README.md#programmatic): `create`
+consumes an in-memory config data model and creates SDK components. According to
+the trace, metric, and log specs, SDKs MAY support updating the config, but
+there is no conflict with declarative config which doesn't already exist.
+However the SDK handles programmatic config updates to SDK components which
+originally programmatically configured applies here as well. If an SDK supports
+it, all programmatic config updates are applied after `create` initializes SDK
+components and therefore take precedence. The semantics of what programmatic
+config updates are allowed and how they merge with existing SDK components are
+out of scope for declarative config.
+
+## Programmatic customization and `create`
+
+While `create` does provide an optional mechanism for programmatic
+customization, its use should be considered a code smell, to be addressed by
+improving the declarative config data model.
+
+For example, the fact that configuration of dynamic authentication for OTLP
+exporters is not possible to express with declarative config should not
+encourage the OpenTelemetry community to have better programmatic customization.
+Instead, we should pursue adding authentication as an SDK extension plugin
+interface and modeling this new plugin in declarative config.