Log exporter (or unbuffered output) for OTEL instrumentation

[maxmcd]

related to: #26852

The new OTEL instrumentation (which is awesome btw), provides an opportunity to get log/request/metric information from the Deno process. There are some instances where I'd want to use this to get a live stream of log data from a running deno program. The only issue with the current setup is that it buffers up otel events every 1s.

At @val-town we patch the console object to capture granular log information in the Deno process and this has been quite messy. If there was a way to hook into the unbuffered stream of otel information we'd have a robust way to capture logs and server traces. We're sensitive to latency (people are waiting for logs to stream back), so the 1s delay means we can't migrate atm. Any thoughts on how we might get access to an unbuffered stream?

I believe adding the option to set the LogExporter for each telemetry type would print the unbuffered output to stdout.

Thanks!

[lucacasonato]

You can configure buffering with OTEL_BLRP_SCHEDULE_DELAY, OTEL_BLRP_EXPORT_TIMEOUT, OTEL_BLRP_MAX_QUEUE_SIZE, and OTEL_BLRP_MAX_EXPORT_BATCH_SIZE. Probably setting OTEL_BLRP_SCHEDULE_DELAY=1 or OTEL_BLRP_MAX_EXPORT_BATCH_SIZE=1 should work?

Do note that this will degrade performance very significantly

[maxmcd]

Ah nice, thank you.

edit:

I ended up configuring:

OTEL_BLRP_SCHEDULE_DELAY: "100",
OTEL_BSP_SCHEDULE_DELAY: "100",