Skip to content
This repository was archived by the owner on Jul 28, 2026. It is now read-only.

Support autologging span attributes as loki labels - #703

Merged
mapno merged 15 commits into
grafana-cold-storage:mainfrom
mapno:autologging-labels
Jul 15, 2021
Merged

Support autologging span attributes as loki labels#703
mapno merged 15 commits into
grafana-cold-storage:mainfrom
mapno:autologging-labels

Conversation

@mapno

@mapno mapno commented Jul 2, 2021

Copy link
Copy Markdown
Contributor

PR Description

Add support autologging span attributes as log labels. This skips the need to format the log line and allows to directly search via labels. But as a downside, it increases cardinality.

Which issue(s) this PR fixes

Fixes #661

Notes to the Reviewer

PR Checklist

  • CHANGELOG updated
  • Documentation added
  • Tests updated

@mapno
mapno requested a review from joe-elliott as a code owner July 2, 2021 11:39
@mapno

mapno commented Jul 2, 2021

Copy link
Copy Markdown
Contributor Author

I'm not completely sold on this approach. It depends on the attribute/tag to be logged in the log line, duplicating data. It also does not support to add default values.

I wanted to make it as simple as possible to configure, but maybe we can make it a bigger feature.

@mapno mapno changed the title Support autologging span attributes as log labels WIP: Support autologging span attributes as log labels Jul 2, 2021
cfg.Overrides.DurationKey = override(cfg.Overrides.DurationKey, defaultDurationKey)
cfg.Overrides.TraceIDKey = override(cfg.Overrides.TraceIDKey, defaultTraceIDKey)

labels := make(map[string]bool, len(cfg.Labels))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map[string]struct{} is i think the "right" way to do a flag map like this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map[string]struct{} uses less memory since an empty struct is 0 bytes, but it's very similar in performance to map[string]bool (see https://gist.github.com/davecheney/3be245c92b61e5045f75). I think map[string]bool makes for a bit clearer code.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I try to use map[string]struct{} whenever I can, because there's always a chance that a bug could creep in that sets an entry to false in the map which would be treated as if it didn't exist.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, changed in 7efae7e

Comment thread pkg/tempo/automaticloggingprocessor/automaticloggingprocessor.go Outdated
Comment thread docs/configuration-reference.md Outdated
k := keyValues[i]
v := keyValues[i+1]
if p.labels[k.(string)] {
ls[model.LabelName(k.(string))] = model.LabelValue(v.(string))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if k.(string) is guaranteed to not panic can we just take a []string instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the logic a bit to make it cleaner and safer.

Comment thread pkg/tempo/automaticloggingprocessor/automaticloggingprocessor.go Outdated
@mapno
mapno force-pushed the autologging-labels branch from 2675a83 to d403022 Compare July 7, 2021 13:37
@mapno mapno changed the title WIP: Support autologging span attributes as log labels Support autologging span attributes as log labels Jul 7, 2021
@mapno mapno changed the title Support autologging span attributes as log labels Support autologging span attributes as loki labels Jul 7, 2021
@mapno
mapno force-pushed the autologging-labels branch from 7efae7e to f71f02c Compare July 14, 2021 15:14
@mapno

mapno commented Jul 14, 2021

Copy link
Copy Markdown
Contributor Author

@rfratto @joe-elliott could you please take a second look?

@joe-elliott joe-elliott left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one nit. nice work.

this will be nice in our infra. everytime i query this log stream i wish there were more filters.

Comment thread docs/configuration/tempo-config.md Outdated
Comment thread pkg/tempo/automaticloggingprocessor/automaticloggingprocessor.go
@mapno
mapno merged commit a647486 into grafana-cold-storage:main Jul 15, 2021
@mapno
mapno deleted the autologging-labels branch July 15, 2021 09:10
mattdurham pushed a commit that referenced this pull request Jul 22, 2021
* Support autologging span attributes as log labels

* Update test

* Fix test

* apply suggestions

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Faster stringify

* Support no keyvalues

* Correctly size map

* Fix config

* Skip STATUS_CODE_UNSET to be less spammy

* Fix tests

* Use struct map

* Fix rebase

* Improve docs

* Apply suggestions

* yeah, don't change that
mattdurham added a commit that referenced this pull request Jul 22, 2021
* Initialize biggest ref to existing ref when reading a segment (#676)

* initialize biggest ref to existing ref when reading a segment

* changelog

* allow new test to be less hard-coded

* explain error if test fails

* clarify changelog issue

* prepare for v0.16.1 release (#679)

* adding docker-compose example for local testing

(cherry picked from commit 992e3cf)

* Cherry picking kafka importer

* update Operator FAQ to describe custom scrape jobs (#658)

* Merge

* merge for race condition

* Wall cherry pick

* Import Kafka

* Bump version number to v0.17.0

* Add TLS config options for tempo remote_write (#666)

* Add TLS config options for tempo s

* cleanup

* Apply suggestions

* Rebase

* Update operator to use v0.16.1 by default (#681)

* update operator to use v0.16.1 by default

* changelog

* improve changelog phrasing?

* Add OTLP HTPP exporter support (#683)

* Tanka Jsonnet Integrations Fixes (#698)

* Hack the integrationsmixin to be always applied

* Remove hack which always applies integrations mixin. Fix copy/pasta

* Conditionally mutate agent daemonset or deployment when node_exporter integration is enabled

* document horizontal scaling (#696)

* document horizontal scaling

* clean up format of pros/cons

* Upgrade otel to v0.29.0 (#700)

* Upgrade otel to v0.29.0

Using grafana's fork which removes internal folders.

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Changelog

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Explain fork

* renamed guide and updated headings (#704)

* Docs: Getting started (#705)

* moved docs

* Update docs/getting-started/_index.md

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* Docs: automatic logging is at the top level of instance config (#711)

See the actual config struct
https://github.com/grafana/agent/blob/50efa00ac3edeec52353549d4e786126cbd35370/pkg/tempo/config.go#L107-L137

* Docs: Configuration (#712)

* moved file

* added server-config.md

* added prometheus-config.md

* Create loki-config.md

* Update configuration-reference.md

* Create tempo-config.md

* added integrations-config

* updated links

* Update grafana-agent config regex replacement (#707)

This PR fixes an issue that users of config.expand-env face.

* moved and renamed files (#706)

* trigger reload configuration using SIGHUP (#710)

* trigger reload configuration using SIGHUP

* fix review changes - stop and close signal on defer

* add changelog for reloading config using SIGHUP

* add post method for config reload api (#709)

* move maintainers guide out of docs (#714)

move each to cmd/<project>/DEVELOPERS.md

* Add github action job to check if links in docs are correct (#708)

* Add debug log in prom-sd processor to indicate missing ip tag (#718)

Signed-off-by: Annanay <annanay.agarwal@grafana.com>

* Update dashboards setup in production mixin. (#685)

This changes the following:
- fix `Scrape failures` queries to use the proper variables like the
  rest of queries
- add `grafana-agent-mixin` tag to all dashboards to allow easy link
  and discovery

* fix race condition in initializing cluster (#690)

* Docs: More cleanup (#720)

* added metadata and renamed

* Update scraping-service.md

* Update operation-guide.md

* edits

* Update _index.md

* Update loki-config.md

* Update _index.md

* Update prometheus-config.md

* Create create-config-file.md

* Update _index.md

* Update architecture.md

* Update getting-started.md

* Update README.md

* Update README.md

* Update _index.md

* Operator: create CRDs for logging (#721)

* operator: create CRDs for logging

* fix lint nits

* Docs: Integrations (#723)

* content edits

* Create consul-exporter-config.md

* Create statsd-exporter-config.md

* Create postgres-exporter-config.md

* Create memcached-exporter-config.md

* Create elasticsearch-exporter-config.md

* Create dnsmasq-exporter-config.md

* Create redis-exporter-config.md

* Create mysqld-exporter-config.md

* Create process-exporter-config.md

* Create node-exporter-config.md

* Update _index.md

* Update api.md

* Update operation-guide.md

* Update operation-guide.md

* remove duplicated config blobs (#719)

* Remove duplicated blobs from Prometheus and link to Prometheus
* Move tls config entry to server-config where it belongs
* Use consistent strategy for linking to external documentation

* Docs: change operator custom scrape jobs example to use integrations job_name (#726)

* update operator custom scrape jobs example to use integrations-specific job_name

* add known issue

* add rollout command corresponding to getting started guide

* Set DNSPolicy correctly when host network is used in daemonset (#728)

* Rename "loki" to "logs" (#727)

* Embedding Kafka Lag Exporter (#674)

* Embedding the Kafka Lag Exporter

* update default config

* changelog and docs updated

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* Embedding the Kafka Lag Exporter

* update default config

* changelog merged

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* some other renames and identations

* fixing misspelling

* fixing field names and docs

* Remove trailing empty line

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* adding docker-compose example for local testing

* Merge from main and reformat config

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Matt Durham <mattdurham@ppog.org>

* Support autologging span attributes as loki labels (#703)

* Support autologging span attributes as log labels

* Update test

* Fix test

* apply suggestions

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Faster stringify

* Support no keyvalues

* Correctly size map

* Fix config

* Skip STATUS_CODE_UNSET to be less spammy

* Fix tests

* Use struct map

* Fix rebase

* Improve docs

* Apply suggestions

* yeah, don't change that

* Grafana Agent Operator: generate logs config (#740)

* wip generate logging config

* generate log client

* generate config for pipelines

* generate pod logs

* fix lint error

* buckets precision warning

* crds: denote required fields, fix typos

* buffer notifier channel (#744)

* Clean up K8s deployment methods (#749)

* Cleans up some deployment methods:

- Updates README directing users to Graf Cloud docs for full instructions
- Removes generation of `agent.yaml` and `agent-sigv4.yaml` from build script
- Removes generated `agent.yaml` and `agent-sigv4.yaml`
- Updates logs Tanka to remove ConfigMap (moved to Graf Cloud docs)
- Removes install scripts

* Update production/README.md

* Add link

* Update links

* Fix broken links and update doc

* Add line to direct users to quickstart guides

* Add install-bare.sh warning (#752)

* add install-bare.sh warning

* use emoji to make warning more warning-er

* Ignore regex capture groups when expanding environment vars (#754)

* ignore regex capture groups when expanding environment vars

* document behavior of regex capture group coercion

* remove weird formatting in changelog

* Implement Drone Build system, moving away from release via Github Actions (#732)

* Initial drone support

* Fix drone file

* Drone changes

* Added support for drone building

* Echo from makefile

* Add logging info

* Add logging info

* Add logging info

* Add debugging info

* Add make debugging info

* Add seego debugging

* add ls to see why files arent getting copied

* More debugging

* simplify

* Drone changes

* Drone v2


Former-commit-id: 914b4a3

* Remove invalid files


Former-commit-id: 7426669

* Fix freebsd build


Former-commit-id: 8f1fc64

* Setting up tokens


Former-commit-id: 246be58

* Removing windows build output


Former-commit-id: 241d9bb

* Fix for docker builds


Former-commit-id: 3babea6

* Remove containerize no docker


Former-commit-id: 90660b5

* Make buildx single threaded


Former-commit-id: e1fa585

* Building against personal docker account


Former-commit-id: 83bb1cc

* Merge changes


Former-commit-id: f6b8df9

* Remove exe


Former-commit-id: 79ce552

* Removed V6 temporarily to see if fixed


Former-commit-id: e6c5680

* Removed extra steps


Former-commit-id: cb89f55

* Signed yaml


Former-commit-id: e2665dd

* Fix makefile


Former-commit-id: 6bfcc9d

* Fix tag


Former-commit-id: 9ac2297

* Using drone tag


Former-commit-id: 475d8cd

* Fix drone tag


Former-commit-id: 4d7029e

* Fix drone issues


Former-commit-id: bbb95ba

* Fix docker not being available


Former-commit-id: 4387301

* Set build in container flag


Former-commit-id: 7ed76a9

* fix


Former-commit-id: 2623c6e

* fix


Former-commit-id: c27a8ff

* fix


Former-commit-id: 06d8e36

* remove dead code


Former-commit-id: b074c9e

* comments


Former-commit-id: eee8e0c

* sign


Former-commit-id: c29ec03

* comments


Former-commit-id: 5c320aa

* comments


Former-commit-id: acacde9

* comments


Former-commit-id: 8acfe15

* comments


Former-commit-id: 3a40480

* comments


Former-commit-id: d7e3103

* comments


Former-commit-id: 84303ac

* comments


Former-commit-id: 2c56ddc

* comments


Former-commit-id: 978dc72

* comments


Former-commit-id: 61a5cfb

* Update changelog


Former-commit-id: 2348bf7

* comments


Former-commit-id: 4ce65c6

* test build


Former-commit-id: fbf9ecc

* test build


Former-commit-id: 1a845d8

* test build


Former-commit-id: 57c0350

* test build


Former-commit-id: 75305d7

* test build


Former-commit-id: 76b79a4

* test build


Former-commit-id: 9517eae

* test build


Former-commit-id: 9f57b5d

* test build


Former-commit-id: 6c37355

* test build


Former-commit-id: 738b53b

* test build


Former-commit-id: d3a389e

* Cleanup


Former-commit-id: c44965e

* Remove release.key


Former-commit-id: 1f743bf

* Fix for windows installer build


Former-commit-id: b8221d5

* Allow windows installer builds


Former-commit-id: a977ecb

* Simplify makefile

* Remove unused files

* Add test back to github

* Fix normal makes

* Reorder changelog

* Fix arm64

* Arm fix

* Remove comment lines

* rename governance

* use seego

* Simplify seego usage

* include fix for test

* Fix for dout

* Only build docker container if needed

* Add depends on

* Fix naming

* Unsilent any remaining output

* unsilent

* Remove check-seego and switch to using docker by default

* Change triggers

* rename master to main

* Fix dependency order now that Dist only triggers on tags

* Sign drone

* mount local go mod cache to seego if one exists

* Clean up triggers

* Add trigger to dist

* testing change for synchronized

* testing change for synchronized

* Undo changeS

* sign drone

* Prevent double build

* remove extra check

* deduplicate build

* deduplicate build

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Logs:  create positionsdir per config (#757)

* create positions directory per instance

* update docs

* operator: add cluster label to all logs (#758)

* Merge changelog into main

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Gabriel <g.amaral.antunes@gmail.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
Co-authored-by: Ryan Geyer <me@ryangeyer.com>
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>
Co-authored-by: Justin Walz <justin.walz10@gmail.com>
Co-authored-by: Tharun Rajendran <rajendrantharun@live.com>
Co-authored-by: Alexander <kazakov.a.s.90@gmail.com>
Co-authored-by: Annanay Agarwal <annanay25@users.noreply.github.com>
Co-authored-by: Cristian Greco <cristian@regolo.cc>
Co-authored-by: hanif <hjet@users.noreply.github.com>
Co-authored-by: Robert Fratto <robertfratto@gmail.com>
rgeyer added a commit that referenced this pull request Jul 22, 2021
* Initialize biggest ref to existing ref when reading a segment (#676)

* initialize biggest ref to existing ref when reading a segment

* changelog

* allow new test to be less hard-coded

* explain error if test fails

* clarify changelog issue

* prepare for v0.16.1 release (#679)

* adding docker-compose example for local testing

(cherry picked from commit 992e3cf)

* Cherry picking kafka importer

* update Operator FAQ to describe custom scrape jobs (#658)

* Merge

* merge for race condition

* Wall cherry pick

* Import Kafka

* Bump version number to v0.17.0

* Add TLS config options for tempo remote_write (#666)

* Add TLS config options for tempo s

* cleanup

* Apply suggestions

* Rebase

* Update operator to use v0.16.1 by default (#681)

* update operator to use v0.16.1 by default

* changelog

* improve changelog phrasing?

* Add OTLP HTPP exporter support (#683)

* Tanka Jsonnet Integrations Fixes (#698)

* Hack the integrationsmixin to be always applied

* Remove hack which always applies integrations mixin. Fix copy/pasta

* Conditionally mutate agent daemonset or deployment when node_exporter integration is enabled

* document horizontal scaling (#696)

* document horizontal scaling

* clean up format of pros/cons

* Upgrade otel to v0.29.0 (#700)

* Upgrade otel to v0.29.0

Using grafana's fork which removes internal folders.

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Changelog

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Explain fork

* renamed guide and updated headings (#704)

* Docs: Getting started (#705)

* moved docs

* Update docs/getting-started/_index.md

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* Docs: automatic logging is at the top level of instance config (#711)

See the actual config struct
https://github.com/grafana/agent/blob/50efa00ac3edeec52353549d4e786126cbd35370/pkg/tempo/config.go#L107-L137

* Docs: Configuration (#712)

* moved file

* added server-config.md

* added prometheus-config.md

* Create loki-config.md

* Update configuration-reference.md

* Create tempo-config.md

* added integrations-config

* updated links

* Update grafana-agent config regex replacement (#707)

This PR fixes an issue that users of config.expand-env face.

* moved and renamed files (#706)

* trigger reload configuration using SIGHUP (#710)

* trigger reload configuration using SIGHUP

* fix review changes - stop and close signal on defer

* add changelog for reloading config using SIGHUP

* add post method for config reload api (#709)

* move maintainers guide out of docs (#714)

move each to cmd/<project>/DEVELOPERS.md

* Add github action job to check if links in docs are correct (#708)

* Add debug log in prom-sd processor to indicate missing ip tag (#718)

Signed-off-by: Annanay <annanay.agarwal@grafana.com>

* Update dashboards setup in production mixin. (#685)

This changes the following:
- fix `Scrape failures` queries to use the proper variables like the
  rest of queries
- add `grafana-agent-mixin` tag to all dashboards to allow easy link
  and discovery

* fix race condition in initializing cluster (#690)

* Docs: More cleanup (#720)

* added metadata and renamed

* Update scraping-service.md

* Update operation-guide.md

* edits

* Update _index.md

* Update loki-config.md

* Update _index.md

* Update prometheus-config.md

* Create create-config-file.md

* Update _index.md

* Update architecture.md

* Update getting-started.md

* Update README.md

* Update README.md

* Update _index.md

* Operator: create CRDs for logging (#721)

* operator: create CRDs for logging

* fix lint nits

* Docs: Integrations (#723)

* content edits

* Create consul-exporter-config.md

* Create statsd-exporter-config.md

* Create postgres-exporter-config.md

* Create memcached-exporter-config.md

* Create elasticsearch-exporter-config.md

* Create dnsmasq-exporter-config.md

* Create redis-exporter-config.md

* Create mysqld-exporter-config.md

* Create process-exporter-config.md

* Create node-exporter-config.md

* Update _index.md

* Update api.md

* Update operation-guide.md

* Update operation-guide.md

* remove duplicated config blobs (#719)

* Remove duplicated blobs from Prometheus and link to Prometheus
* Move tls config entry to server-config where it belongs
* Use consistent strategy for linking to external documentation

* Docs: change operator custom scrape jobs example to use integrations job_name (#726)

* update operator custom scrape jobs example to use integrations-specific job_name

* add known issue

* add rollout command corresponding to getting started guide

* Set DNSPolicy correctly when host network is used in daemonset (#728)

* Rename "loki" to "logs" (#727)

* Embedding Kafka Lag Exporter (#674)

* Embedding the Kafka Lag Exporter

* update default config

* changelog and docs updated

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* Embedding the Kafka Lag Exporter

* update default config

* changelog merged

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* some other renames and identations

* fixing misspelling

* fixing field names and docs

* Remove trailing empty line

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* adding docker-compose example for local testing

* Merge from main and reformat config

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Matt Durham <mattdurham@ppog.org>

* Support autologging span attributes as loki labels (#703)

* Support autologging span attributes as log labels

* Update test

* Fix test

* apply suggestions

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Faster stringify

* Support no keyvalues

* Correctly size map

* Fix config

* Skip STATUS_CODE_UNSET to be less spammy

* Fix tests

* Use struct map

* Fix rebase

* Improve docs

* Apply suggestions

* yeah, don't change that

* Grafana Agent Operator: generate logs config (#740)

* wip generate logging config

* generate log client

* generate config for pipelines

* generate pod logs

* fix lint error

* buckets precision warning

* crds: denote required fields, fix typos

* buffer notifier channel (#744)

* Clean up K8s deployment methods (#749)

* Cleans up some deployment methods:

- Updates README directing users to Graf Cloud docs for full instructions
- Removes generation of `agent.yaml` and `agent-sigv4.yaml` from build script
- Removes generated `agent.yaml` and `agent-sigv4.yaml`
- Updates logs Tanka to remove ConfigMap (moved to Graf Cloud docs)
- Removes install scripts

* Update production/README.md

* Add link

* Update links

* Fix broken links and update doc

* Add line to direct users to quickstart guides

* Add install-bare.sh warning (#752)

* add install-bare.sh warning

* use emoji to make warning more warning-er

* Ignore regex capture groups when expanding environment vars (#754)

* ignore regex capture groups when expanding environment vars

* document behavior of regex capture group coercion

* remove weird formatting in changelog

* Implement Drone Build system, moving away from release via Github Actions (#732)

* Initial drone support

* Fix drone file

* Drone changes

* Added support for drone building

* Echo from makefile

* Add logging info

* Add logging info

* Add logging info

* Add debugging info

* Add make debugging info

* Add seego debugging

* add ls to see why files arent getting copied

* More debugging

* simplify

* Drone changes

* Drone v2

Former-commit-id: 914b4a3

* Remove invalid files

Former-commit-id: 7426669

* Fix freebsd build

Former-commit-id: 8f1fc64

* Setting up tokens

Former-commit-id: 246be58

* Removing windows build output

Former-commit-id: 241d9bb

* Fix for docker builds

Former-commit-id: 3babea6

* Remove containerize no docker

Former-commit-id: 90660b5

* Make buildx single threaded

Former-commit-id: e1fa585

* Building against personal docker account

Former-commit-id: 83bb1cc

* Merge changes

Former-commit-id: f6b8df9

* Remove exe

Former-commit-id: 79ce552

* Removed V6 temporarily to see if fixed

Former-commit-id: e6c5680

* Removed extra steps

Former-commit-id: cb89f55

* Signed yaml

Former-commit-id: e2665dd

* Fix makefile

Former-commit-id: 6bfcc9d

* Fix tag

Former-commit-id: 9ac2297

* Using drone tag

Former-commit-id: 475d8cd

* Fix drone tag

Former-commit-id: 4d7029e

* Fix drone issues

Former-commit-id: bbb95ba

* Fix docker not being available

Former-commit-id: 4387301

* Set build in container flag

Former-commit-id: 7ed76a9

* fix

Former-commit-id: 2623c6e

* fix

Former-commit-id: c27a8ff

* fix

Former-commit-id: 06d8e36

* remove dead code

Former-commit-id: b074c9e

* comments

Former-commit-id: eee8e0c

* sign

Former-commit-id: c29ec03

* comments

Former-commit-id: 5c320aa

* comments

Former-commit-id: acacde9

* comments

Former-commit-id: 8acfe15

* comments

Former-commit-id: 3a40480

* comments

Former-commit-id: d7e3103

* comments

Former-commit-id: 84303ac

* comments

Former-commit-id: 2c56ddc

* comments

Former-commit-id: 978dc72

* comments

Former-commit-id: 61a5cfb

* Update changelog

Former-commit-id: 2348bf7

* comments

Former-commit-id: 4ce65c6

* test build

Former-commit-id: fbf9ecc

* test build

Former-commit-id: 1a845d8

* test build

Former-commit-id: 57c0350

* test build

Former-commit-id: 75305d7

* test build

Former-commit-id: 76b79a4

* test build

Former-commit-id: 9517eae

* test build

Former-commit-id: 9f57b5d

* test build

Former-commit-id: 6c37355

* test build

Former-commit-id: 738b53b

* test build

Former-commit-id: d3a389e

* Cleanup

Former-commit-id: c44965e

* Remove release.key

Former-commit-id: 1f743bf

* Fix for windows installer build

Former-commit-id: b8221d5

* Allow windows installer builds

Former-commit-id: a977ecb

* Simplify makefile

* Remove unused files

* Add test back to github

* Fix normal makes

* Reorder changelog

* Fix arm64

* Arm fix

* Remove comment lines

* rename governance

* use seego

* Simplify seego usage

* include fix for test

* Fix for dout

* Only build docker container if needed

* Add depends on

* Fix naming

* Unsilent any remaining output

* unsilent

* Remove check-seego and switch to using docker by default

* Change triggers

* rename master to main

* Fix dependency order now that Dist only triggers on tags

* Sign drone

* mount local go mod cache to seego if one exists

* Clean up triggers

* Add trigger to dist

* testing change for synchronized

* testing change for synchronized

* Undo changeS

* sign drone

* Prevent double build

* remove extra check

* deduplicate build

* deduplicate build

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Logs:  create positionsdir per config (#757)

* create positions directory per instance

* update docs

* operator: add cluster label to all logs (#758)

* Merge changelog into main

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Gabriel <g.amaral.antunes@gmail.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
Co-authored-by: Ryan Geyer <me@ryangeyer.com>
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>
Co-authored-by: Justin Walz <justin.walz10@gmail.com>
Co-authored-by: Tharun Rajendran <rajendrantharun@live.com>
Co-authored-by: Alexander <kazakov.a.s.90@gmail.com>
Co-authored-by: Annanay Agarwal <annanay25@users.noreply.github.com>
Co-authored-by: Cristian Greco <cristian@regolo.cc>
Co-authored-by: hanif <hjet@users.noreply.github.com>
Co-authored-by: Robert Fratto <robertfratto@gmail.com>
@mattdurham mattdurham mentioned this pull request Sep 7, 2021
3 tasks
mattdurham pushed a commit that referenced this pull request Nov 11, 2021
* Support autologging span attributes as log labels

* Update test

* Fix test

* apply suggestions

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Faster stringify

* Support no keyvalues

* Correctly size map

* Fix config

* Skip STATUS_CODE_UNSET to be less spammy

* Fix tests

* Use struct map

* Fix rebase

* Improve docs

* Apply suggestions

* yeah, don't change that
mattdurham added a commit that referenced this pull request Nov 11, 2021
* Initialize biggest ref to existing ref when reading a segment (#676)

* initialize biggest ref to existing ref when reading a segment

* changelog

* allow new test to be less hard-coded

* explain error if test fails

* clarify changelog issue

* prepare for v0.16.1 release (#679)

* adding docker-compose example for local testing

(cherry picked from commit 992e3cf)

* Cherry picking kafka importer

* update Operator FAQ to describe custom scrape jobs (#658)

* Merge

* merge for race condition

* Wall cherry pick

* Import Kafka

* Bump version number to v0.17.0

* Add TLS config options for tempo remote_write (#666)

* Add TLS config options for tempo s

* cleanup

* Apply suggestions

* Rebase

* Update operator to use v0.16.1 by default (#681)

* update operator to use v0.16.1 by default

* changelog

* improve changelog phrasing?

* Add OTLP HTPP exporter support (#683)

* Tanka Jsonnet Integrations Fixes (#698)

* Hack the integrationsmixin to be always applied

* Remove hack which always applies integrations mixin. Fix copy/pasta

* Conditionally mutate agent daemonset or deployment when node_exporter integration is enabled

* document horizontal scaling (#696)

* document horizontal scaling

* clean up format of pros/cons

* Upgrade otel to v0.29.0 (#700)

* Upgrade otel to v0.29.0

Using grafana's fork which removes internal folders.

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Changelog

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Explain fork

* renamed guide and updated headings (#704)

* Docs: Getting started (#705)

* moved docs

* Update docs/getting-started/_index.md

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* Docs: automatic logging is at the top level of instance config (#711)

See the actual config struct
https://github.com/grafana/agent/blob/50efa00ac3edeec52353549d4e786126cbd35370/pkg/tempo/config.go#L107-L137

* Docs: Configuration (#712)

* moved file

* added server-config.md

* added prometheus-config.md

* Create loki-config.md

* Update configuration-reference.md

* Create tempo-config.md

* added integrations-config

* updated links

* Update grafana-agent config regex replacement (#707)

This PR fixes an issue that users of config.expand-env face.

* moved and renamed files (#706)

* trigger reload configuration using SIGHUP (#710)

* trigger reload configuration using SIGHUP

* fix review changes - stop and close signal on defer

* add changelog for reloading config using SIGHUP

* add post method for config reload api (#709)

* move maintainers guide out of docs (#714)

move each to cmd/<project>/DEVELOPERS.md

* Add github action job to check if links in docs are correct (#708)

* Add debug log in prom-sd processor to indicate missing ip tag (#718)

Signed-off-by: Annanay <annanay.agarwal@grafana.com>

* Update dashboards setup in production mixin. (#685)

This changes the following:
- fix `Scrape failures` queries to use the proper variables like the
  rest of queries
- add `grafana-agent-mixin` tag to all dashboards to allow easy link
  and discovery

* fix race condition in initializing cluster (#690)

* Docs: More cleanup (#720)

* added metadata and renamed

* Update scraping-service.md

* Update operation-guide.md

* edits

* Update _index.md

* Update loki-config.md

* Update _index.md

* Update prometheus-config.md

* Create create-config-file.md

* Update _index.md

* Update architecture.md

* Update getting-started.md

* Update README.md

* Update README.md

* Update _index.md

* Operator: create CRDs for logging (#721)

* operator: create CRDs for logging

* fix lint nits

* Docs: Integrations (#723)

* content edits

* Create consul-exporter-config.md

* Create statsd-exporter-config.md

* Create postgres-exporter-config.md

* Create memcached-exporter-config.md

* Create elasticsearch-exporter-config.md

* Create dnsmasq-exporter-config.md

* Create redis-exporter-config.md

* Create mysqld-exporter-config.md

* Create process-exporter-config.md

* Create node-exporter-config.md

* Update _index.md

* Update api.md

* Update operation-guide.md

* Update operation-guide.md

* remove duplicated config blobs (#719)

* Remove duplicated blobs from Prometheus and link to Prometheus
* Move tls config entry to server-config where it belongs
* Use consistent strategy for linking to external documentation

* Docs: change operator custom scrape jobs example to use integrations job_name (#726)

* update operator custom scrape jobs example to use integrations-specific job_name

* add known issue

* add rollout command corresponding to getting started guide

* Set DNSPolicy correctly when host network is used in daemonset (#728)

* Rename "loki" to "logs" (#727)

* Embedding Kafka Lag Exporter (#674)

* Embedding the Kafka Lag Exporter

* update default config

* changelog and docs updated

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* Embedding the Kafka Lag Exporter

* update default config

* changelog merged

* removing tabs from doc file

* removing tabs from doc file

* solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags

* some other renames and identations

* fixing misspelling

* fixing field names and docs

* Remove trailing empty line

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>

* adding docker-compose example for local testing

* Merge from main and reformat config

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Matt Durham <mattdurham@ppog.org>

* Support autologging span attributes as loki labels (#703)

* Support autologging span attributes as log labels

* Update test

* Fix test

* apply suggestions

Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com>

* Faster stringify

* Support no keyvalues

* Correctly size map

* Fix config

* Skip STATUS_CODE_UNSET to be less spammy

* Fix tests

* Use struct map

* Fix rebase

* Improve docs

* Apply suggestions

* yeah, don't change that

* Grafana Agent Operator: generate logs config (#740)

* wip generate logging config

* generate log client

* generate config for pipelines

* generate pod logs

* fix lint error

* buckets precision warning

* crds: denote required fields, fix typos

* buffer notifier channel (#744)

* Clean up K8s deployment methods (#749)

* Cleans up some deployment methods:

- Updates README directing users to Graf Cloud docs for full instructions
- Removes generation of `agent.yaml` and `agent-sigv4.yaml` from build script
- Removes generated `agent.yaml` and `agent-sigv4.yaml`
- Updates logs Tanka to remove ConfigMap (moved to Graf Cloud docs)
- Removes install scripts

* Update production/README.md

* Add link

* Update links

* Fix broken links and update doc

* Add line to direct users to quickstart guides

* Add install-bare.sh warning (#752)

* add install-bare.sh warning

* use emoji to make warning more warning-er

* Ignore regex capture groups when expanding environment vars (#754)

* ignore regex capture groups when expanding environment vars

* document behavior of regex capture group coercion

* remove weird formatting in changelog

* Implement Drone Build system, moving away from release via Github Actions (#732)

* Initial drone support

* Fix drone file

* Drone changes

* Added support for drone building

* Echo from makefile

* Add logging info

* Add logging info

* Add logging info

* Add debugging info

* Add make debugging info

* Add seego debugging

* add ls to see why files arent getting copied

* More debugging

* simplify

* Drone changes

* Drone v2


Former-commit-id: 914b4a3

* Remove invalid files


Former-commit-id: 7426669

* Fix freebsd build


Former-commit-id: 8f1fc64

* Setting up tokens


Former-commit-id: 246be58

* Removing windows build output


Former-commit-id: 241d9bb

* Fix for docker builds


Former-commit-id: 3babea6

* Remove containerize no docker


Former-commit-id: 90660b5

* Make buildx single threaded


Former-commit-id: e1fa585

* Building against personal docker account


Former-commit-id: 83bb1cc

* Merge changes


Former-commit-id: f6b8df9

* Remove exe


Former-commit-id: 79ce552

* Removed V6 temporarily to see if fixed


Former-commit-id: e6c5680

* Removed extra steps


Former-commit-id: cb89f55

* Signed yaml


Former-commit-id: e2665dd

* Fix makefile


Former-commit-id: 6bfcc9d

* Fix tag


Former-commit-id: 9ac2297

* Using drone tag


Former-commit-id: 475d8cd

* Fix drone tag


Former-commit-id: 4d7029e

* Fix drone issues


Former-commit-id: bbb95ba

* Fix docker not being available


Former-commit-id: 4387301

* Set build in container flag


Former-commit-id: 7ed76a9

* fix


Former-commit-id: 2623c6e

* fix


Former-commit-id: c27a8ff

* fix


Former-commit-id: 06d8e36

* remove dead code


Former-commit-id: b074c9e

* comments


Former-commit-id: eee8e0c

* sign


Former-commit-id: c29ec03

* comments


Former-commit-id: 5c320aa

* comments


Former-commit-id: acacde9

* comments


Former-commit-id: 8acfe15

* comments


Former-commit-id: 3a40480

* comments


Former-commit-id: d7e3103

* comments


Former-commit-id: 84303ac

* comments


Former-commit-id: 2c56ddc

* comments


Former-commit-id: 978dc72

* comments


Former-commit-id: 61a5cfb

* Update changelog


Former-commit-id: 2348bf7

* comments


Former-commit-id: 4ce65c6

* test build


Former-commit-id: fbf9ecc

* test build


Former-commit-id: 1a845d8

* test build


Former-commit-id: 57c0350

* test build


Former-commit-id: 75305d7

* test build


Former-commit-id: 76b79a4

* test build


Former-commit-id: 9517eae

* test build


Former-commit-id: 9f57b5d

* test build


Former-commit-id: 6c37355

* test build


Former-commit-id: 738b53b

* test build


Former-commit-id: d3a389e

* Cleanup


Former-commit-id: c44965e

* Remove release.key


Former-commit-id: 1f743bf

* Fix for windows installer build


Former-commit-id: b8221d5

* Allow windows installer builds


Former-commit-id: a977ecb

* Simplify makefile

* Remove unused files

* Add test back to github

* Fix normal makes

* Reorder changelog

* Fix arm64

* Arm fix

* Remove comment lines

* rename governance

* use seego

* Simplify seego usage

* include fix for test

* Fix for dout

* Only build docker container if needed

* Add depends on

* Fix naming

* Unsilent any remaining output

* unsilent

* Remove check-seego and switch to using docker by default

* Change triggers

* rename master to main

* Fix dependency order now that Dist only triggers on tags

* Sign drone

* mount local go mod cache to seego if one exists

* Clean up triggers

* Add trigger to dist

* testing change for synchronized

* testing change for synchronized

* Undo changeS

* sign drone

* Prevent double build

* remove extra check

* deduplicate build

* deduplicate build

Co-authored-by: Robert Fratto <robertfratto@gmail.com>

* Logs:  create positionsdir per config (#757)

* create positions directory per instance

* update docs

* operator: add cluster label to all logs (#758)

* Merge changelog into main

Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Co-authored-by: Gabriel <g.amaral.antunes@gmail.com>
Co-authored-by: Mario <mariorvinas@gmail.com>
Co-authored-by: Ryan Geyer <me@ryangeyer.com>
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net>
Co-authored-by: Justin Walz <justin.walz10@gmail.com>
Co-authored-by: Tharun Rajendran <rajendrantharun@live.com>
Co-authored-by: Alexander <kazakov.a.s.90@gmail.com>
Co-authored-by: Annanay Agarwal <annanay25@users.noreply.github.com>
Co-authored-by: Cristian Greco <cristian@regolo.cc>
Co-authored-by: hanif <hjet@users.noreply.github.com>
Co-authored-by: Robert Fratto <robertfratto@gmail.com>
@github-actions github-actions Bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Apr 12, 2024
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Apr 12, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autologging: span attributes as log labels

3 participants