Update log and trace column names to match ClickHouse opentelemetry defaults #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The logs and trace tables use the order: ServiceName, TimestampTime, Timestamp Currently the defaults when `use_otel` is enabled sets the timestamp column to `Timestamp`. As a result ClickHouse has to search much more data, making queries against very large tables quite slow. The correct timestamp column to use would be `TimestampTime`. Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/clickhouseexporter/internal/sqltemplates/logs_table.sql
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In ClickHouse the logs and trace tables use the order:
ServiceName, TimestampTime, Timestamp
This is set by the clickhouse exporter for opentelemetry-collector-contrib.
Currently in this plugin the defaults when
use_otelis enabled sets the timestamp column toTimestamp. As a result ClickHouse has to search much more data, making queries against very large tables quite slow.On my 1.5TB logs table,
Timestampqueries take 94.5s, whereas the sameTimestampTimequeries take 10.1s.The correct timestamp column to use would be
TimestampTime.Ref: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/clickhouseexporter/internal/sqltemplates/logs_table.sql