docs: Update labels, add cardinality topic #15399
Open
+324
−108
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JStickler
added
the
type/docs
|
|
||
| - deployment_environment = development | ||
| - host = server-1 | ||
| - pod_name = myapp-abcdef |
There was a problem hiding this comment.
pod_name is a tough one because I have seen users with unique hashes added to their podnames. In most cases if the name is static its fine. But there are cases like this:
NAME READY STATUS RESTARTS AGE
nginx-554b9c67f9-c5cv4 1/1 Running 0 74s
nginx-554b9c67f9-hjkjq 1/1 Running 0 74s
nginx-554b9c67f9-wbwdm 1/1 Running 0 2m7s
There was a problem hiding this comment.
I see pod name frequently called out as a poor label choice in Slack, which is why I put it on the list.
| For Grafana Cloud Logs, see the [current OpenTelemetry guidance](https://grafana.com/docs/grafana-cloud/send-data/otlp/otlp-format-considerations/#logs). | ||
| {{< /admonition >}} | ||
|
|
||
| The default list of Resource Attributes to store as labels can be configured using `default_resource_attributes_as_index_labels` under the [distributor's otlp_config](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/#distributor). If you are using Grafana Cloud, contact support to configure this setting. |
There was a problem hiding this comment.
I am not sure if this has changed but you can also promote and exclude labels otel via the limits_config
limits_config:
otlp_config:
| - Unsupported characters in the label should be converted to an underscore. For example, the label `app.kubernetes.io/name` should be written as `app_kubernetes_io_name`. | ||
| - However, do not begin and end your label names with double underscores, as this naming convention is used for internal labels, for example, \__stream_shard__, that are hidden by default in the label browser, query builder, and autocomplete to avoid creating confusion for users. | ||
|
|
||
| In Loki, you do not need to add labels based on the content of the log message. |
There was a problem hiding this comment.
Slight repeat of line 140 but its good for emphasis
|
|
||
| High cardinality causes Loki to create many streams, especially when labels have many unique values, and when those values are short-lived (for example, active for seconds or minutes). This causes Loki to build a huge index, and to flush thousands of tiny chunks to the object store. | ||
|
|
||
| Loki was not designed or built to support high cardinality label values. In fact, it was built for exactly the opposite. It was built for very long-lived streams and very low cardinality in the labels. In Loki, the fewer labels you use, the better. |
There was a problem hiding this comment.
This feels slightly a repeat of line 22 but
|
Really great write up on labels and cardinality. These are just a few points I noticed but they are only suggestions :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Reorganizes and revises the Labels topic to take into account:
Adds a new topic for cardinality.