You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
We collect our logs from our instances by using an OtterCol agent to retrieve all files contained in /var/log/**/*.
We then receive this stream on our OTLP endpoint where we would like to index the log_file_path contained in the logs context of events.
However, it seems to be limited to resource_attributes.
Our problem is that the definition of a resource_attribute at the OtterCol agent level requires a transformation step that is not compatible with the retrieval of multiple files :(
To Reproduce
Steps to reproduce the behavior:
loki CONFIG :
level=error ts=2024-11-26T16:03:53.256584508Z caller=loki.go:560 msg="module failed" module=distributor error="failed to start distributor, because it depends on module runtime-config, which has failed: invalid service state: Failed, expected: Running, failure: starting module runtime-config: invalid service state: Failed, expected: Running, failure: failed to load runtime config: load file: invalid override for tenant default: index_label action is only supported for resource_attributes"
Environment:
Infrastructure: EC2
Deployment tool: RPM
Screenshots, Promtail config, or terminal output
OtelCOL config :
receivers:
filelog/access:
# Get Nginx logsinclude_file_name: trueinclude_file_path: trueinclude:
- /var/log/nginx/access.log
- /var/log/nginx/error.logretry_on_failure:
enabled: truestart_at: endmax_batches: 1processors:
# Add Compagnie attribute for identify serviceattributes:
actions:
- action: upsertkey: applicationvalue: "${env:ENV_ST}"
- action: upsertkey: deployment.environment.namevalue: "${env:ENV_NAME}"
- action: upsertkey: service.namevalue: "${env:COMPOSANT_NAME}"
- action: upsertkey: host.namevalue: "${env:HOSTNAME}"transform:
error_mode: ignorelog_statements:
- context: log# Convert Attribute to Ressource Attribute for Loki Indexation (Attribute can't be index on loki backend).statements:
- set(resource.attributes["application"], attributes["application"]) where attributes["application"] != nil
- set(resource.attributes["deployment.environment.name"], attributes["deployment.environment.name"]) where attributes["deployment.environment.name"] != nil
- set(resource.attributes["service.name"], attributes["service.name"]) where attributes["service.name"] != nil
- set(resource.attributes["host.name"], attributes["host.name"]) where attributes["host.name"] != nilexporters:
# output to loki backendotlphttp/logs:
endpoint: "https://loki.prod.local.fr:8443/otlp"headers:
"X-Scope-OrgID": "trash"service:
pipelines:
logs/nginx:
receivers:
- filelog/accessprocessors:
- attributes
- transformexporters:
- otlphttp/logs
The text was updated successfully, but these errors were encountered:
Describe the bug
We collect our logs from our instances by using an OtterCol agent to retrieve all files contained in /var/log/**/*.
We then receive this stream on our OTLP endpoint where we would like to index the log_file_path contained in the logs context of events.
In the documentation, we saw that it was possible to index the attribute: https://grafana.com/docs/loki/latest/send-data/otel/
However, it seems to be limited to resource_attributes.
Our problem is that the definition of a resource_attribute at the OtterCol agent level requires a transformation step that is not compatible with the retrieval of multiple files :(
To Reproduce
Steps to reproduce the behavior:
loki CONFIG :
Expected behavior
Environment:
Screenshots, Promtail config, or terminal output
The text was updated successfully, but these errors were encountered: