[Enhancement] Traces And Metrics

[mc-dorzo]

Motivation

We want to implement and refine tracing and metrics using OpenTelemetry to ensure accurate analysis of key performance indicators.

These metrics will help us monitor and better understand system performance, cost, latency, and identify bottlenecks, while guiding ongoing improvements.

Solution Proposal

Step One – OpenTelemetry Implementation

Currently, we use the Logger and ContextualCorrelator classes internally (in-memory) to manage engine scope and propagate correlation IDs to generated events and logs.

We propose the following changes:

a. Extract ContextualCorrelator into an abstract base class (ABC).
b. Create BasicContextualCorrelator, which replicates today’s functionality.
c. Create a new class: class OpenTelemetry(Logger, ContextualCorrelator, Meter) (i.e., implementing three interfaces).
d. In this class, implement scope() using OTel spans, and have all logger calls (info(), debug(), etc.) emit OTel events.
e. Generate tracing_id ourselves, making it equal to the correlation_id for easier searching.
f. Since most logs in Parlant are essentially events—and because OTel logs are less widely supported—it makes sense to implement all logs as events.
g. For capturing metrics, introduce a Meter class as follows:

AttributeValue: TypeAlias = Union[
    str,
    bool,
    int,
    float,
    Sequence[str],
    Sequence[bool],
    Sequence[int],
    Sequence[float],
]

class Meter(ABC):
    @abstractmethod
    async def record_counter(
        self,
        name: str,
        value: int = 1,
        attributes: Mapping[str, AttributeValue] | None = None,
    ) -> None: ...

    @abstractmethod
    async def record_histogram(
        self,
        name: str,
        value: float,
        attributes: Mapping[str, AttributeValue] | None = None,
    ) -> None: ...

    @asynccontextmanager
    async def measure_duration(
        self,
        name: str,
        attributes: Mapping[str, AttributeValue] | None = None,
    ) -> AsyncGenerator[None, None]:
        """
        Measure the duration of a block of code.
        Usage:
            async with meter.measure_duration("my_duration"):
                # Code to measure
        """
        start_time = asyncio.get_event_loop().time()
        try:
            yield
        finally:
            duration = asyncio.get_event_loop().time() - start_time
            await self.record_histogram(name, duration, attributes)

h. Implement NoOpMeter to preserve the current behavior when metrics are disabled.
i. In bin/server.py, add a new flag --open-telemetry. When enabled, the following environment variables will be used:
1. OTEL_SERVICE_NAME (optional, defaults to "parlant")
2. OTEL_TRACES_EXPORTER (required)
3. OTEL_METRICS_EXPORTER (required)
4. OTEL_EXPORTER_OTLP_INSECURE (optional, defaults to False)

Step Two - Add metrics

TBD

Discussion

@kichanyurd What your thoughts?

[kichanyurd]

Questions:

1. How will the OTel correlator work? Will it replace the basic one, or work alongside it - and if so, how exactly? Please elaborate.
2. What will each of the env vars control? Can you elaborate on the purpose and use cases of each of them?
3. Not sure we need the server param. Pretty much no one is supposed to manually start the server from the CLI anymore. I think we can just control it using env vars, but we need to design them properly.

[0x-Professor]

Thanks for the feedback, @kichanyurd. and also @mc-dorzo for raising this issue. These are great questions. Here are my thoughts:

1. How will the OTel correlator work?

The OpenTelemetry class, which will implement the ContextualCorrelator interface, is intended to replace the BasicContextualCorrelator when OpenTelemetry features are enabled. It will not work alongside it.

The decision of which correlator to use will be made at application startup, based on the presence of the --open-telemetry flag or corresponding environment variables.

• If OpenTelemetry is enabled: The OpenTelemetry class will be used. It will leverage OTel spans for context propagation. The scope() method will create and manage these spans, and all logging calls will generate OTel events within the current span. This will allow us to have distributed tracing across our services.
• If OpenTelemetry is disabled: The BasicContextualCorrelator will be used, preserving the existing in-memory correlation mechanism.

This approach ensures that we can introduce OpenTelemetry without affecting existing setups and provides a clear path for enabling it.

2. What will each of the environment variables control?

These environment variables are standard for configuring OpenTelemetry and control how telemetry data (traces and metrics) is collected and exported.

• OTEL_SERVICE_NAME: This variable sets a logical name for our service (e.g., "parlant"). This name will appear in the telemetry data, allowing us to filter and query for traces and metrics from this specific service in our observability backend. If not set, it will default to "parlant" as proposed.

• OTEL_TRACES_EXPORTER and OTEL_METRICS_EXPORTER: These variables determine the destination for our traces and metrics. An "exporter" is a component that sends telemetry data to a specific backend. Common values include:

  • otlp: To send data to an OTLP-compatible endpoint (like Jaeger, Prometheus, or a vendor-specific one).
  • console: To print traces and metrics to the console for debugging purposes.
  • Other options like jaeger or zipkin are also available for traces.
    By setting these, we can direct our telemetry data to the desired analysis tools without changing the code.

• OTEL_EXPORTER_OTLP_INSECURE: This variable is specific to the OTLP exporter.

  • If set to true, the OTLP exporter will send data over an insecure connection (HTTP). This is useful for local development or in environments where traffic is secured by other means (e.g., within a service mesh).
  • If false (the default), the exporter will use a secure connection (HTTPS).

3. Server parameter vs. Environment Variables

You've made a good point. Relying on environment variables for configuration is a more modern and container-friendly approach. We can remove the --open-telemetry command-line flag and use an environment variable to enable or disable OpenTelemetry.

A good convention is to use a specific environment variable to enable the feature, for example OTEL_ENABLED=true. If this variable is set to true, the application will initialize and use the OpenTelemetry correlator and meter. If it's not set or is false, it will fall back to the BasicContextualCorrelator and NoOpMeter. This makes the configuration more consistent with the other OTEL_* variables.