This repository was archived by the owner on Jul 28, 2026. It is now read-only.
Add option for Operator to pass arguments to GrafanaAgent #1227 - #1248
Merged
Conversation
…storage#689) * Enabled flag should drive collection * Satisfy linter * Fix mock integration so it sets enabled = true by default * Make enabled the equivalent of not being in the config, removing it at the config reloader so that it propagates from the source downstream * Fix issue where removing the config/disabling did NOT stop the scraper * Remove whitespace * remove unneeded comment * Remove duplicated delete config * Update comments and add unit tests. Also had to fix mock integration so that it generated yaml. * Simplify unit test * change case because of refactor * filter integrations to enabled set * Move check up to the first statement * fix merge * Fix for double loop Co-authored-by: Robert Fratto <robertfratto@gmail.com>
…smaps (grafana-cold-storage#760) * structwalk * fix lint errors
* Initialize biggest ref to existing ref when reading a segment (grafana-cold-storage#676) * initialize biggest ref to existing ref when reading a segment * changelog * allow new test to be less hard-coded * explain error if test fails * clarify changelog issue * prepare for v0.16.1 release (grafana-cold-storage#679) * adding docker-compose example for local testing (cherry picked from commit 992e3cf) * Cherry picking kafka importer * update Operator FAQ to describe custom scrape jobs (grafana-cold-storage#658) * Merge * merge for race condition * Wall cherry pick * Import Kafka * Bump version number to v0.17.0 * Add TLS config options for tempo remote_write (grafana-cold-storage#666) * Add TLS config options for tempo s * cleanup * Apply suggestions * Rebase * Update operator to use v0.16.1 by default (grafana-cold-storage#681) * update operator to use v0.16.1 by default * changelog * improve changelog phrasing? * Add OTLP HTPP exporter support (grafana-cold-storage#683) * Tanka Jsonnet Integrations Fixes (grafana-cold-storage#698) * Hack the integrationsmixin to be always applied * Remove hack which always applies integrations mixin. Fix copy/pasta * Conditionally mutate agent daemonset or deployment when node_exporter integration is enabled * document horizontal scaling (grafana-cold-storage#696) * document horizontal scaling * clean up format of pros/cons * Upgrade otel to v0.29.0 (grafana-cold-storage#700) * Upgrade otel to v0.29.0 Using grafana's fork which removes internal folders. Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com> * Changelog Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com> * Explain fork * renamed guide and updated headings (grafana-cold-storage#704) * Docs: Getting started (grafana-cold-storage#705) * moved docs * Update docs/getting-started/_index.md Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Docs: automatic logging is at the top level of instance config (grafana-cold-storage#711) See the actual config struct https://github.com/grafana/agent/blob/50efa00ac3edeec52353549d4e786126cbd35370/pkg/tempo/config.go#L107-L137 * Docs: Configuration (grafana-cold-storage#712) * moved file * added server-config.md * added prometheus-config.md * Create loki-config.md * Update configuration-reference.md * Create tempo-config.md * added integrations-config * updated links * Update grafana-agent config regex replacement (grafana-cold-storage#707) This PR fixes an issue that users of config.expand-env face. * moved and renamed files (grafana-cold-storage#706) * trigger reload configuration using SIGHUP (grafana-cold-storage#710) * trigger reload configuration using SIGHUP * fix review changes - stop and close signal on defer * add changelog for reloading config using SIGHUP * add post method for config reload api (grafana-cold-storage#709) * move maintainers guide out of docs (grafana-cold-storage#714) move each to cmd/<project>/DEVELOPERS.md * Add github action job to check if links in docs are correct (grafana-cold-storage#708) * Add debug log in prom-sd processor to indicate missing ip tag (grafana-cold-storage#718) Signed-off-by: Annanay <annanay.agarwal@grafana.com> * Update dashboards setup in production mixin. (grafana-cold-storage#685) This changes the following: - fix `Scrape failures` queries to use the proper variables like the rest of queries - add `grafana-agent-mixin` tag to all dashboards to allow easy link and discovery * fix race condition in initializing cluster (grafana-cold-storage#690) * Docs: More cleanup (grafana-cold-storage#720) * added metadata and renamed * Update scraping-service.md * Update operation-guide.md * edits * Update _index.md * Update loki-config.md * Update _index.md * Update prometheus-config.md * Create create-config-file.md * Update _index.md * Update architecture.md * Update getting-started.md * Update README.md * Update README.md * Update _index.md * Operator: create CRDs for logging (grafana-cold-storage#721) * operator: create CRDs for logging * fix lint nits * Docs: Integrations (grafana-cold-storage#723) * content edits * Create consul-exporter-config.md * Create statsd-exporter-config.md * Create postgres-exporter-config.md * Create memcached-exporter-config.md * Create elasticsearch-exporter-config.md * Create dnsmasq-exporter-config.md * Create redis-exporter-config.md * Create mysqld-exporter-config.md * Create process-exporter-config.md * Create node-exporter-config.md * Update _index.md * Update api.md * Update operation-guide.md * Update operation-guide.md * remove duplicated config blobs (grafana-cold-storage#719) * Remove duplicated blobs from Prometheus and link to Prometheus * Move tls config entry to server-config where it belongs * Use consistent strategy for linking to external documentation * Docs: change operator custom scrape jobs example to use integrations job_name (grafana-cold-storage#726) * update operator custom scrape jobs example to use integrations-specific job_name * add known issue * add rollout command corresponding to getting started guide * Set DNSPolicy correctly when host network is used in daemonset (grafana-cold-storage#728) * Rename "loki" to "logs" (grafana-cold-storage#727) * Embedding Kafka Lag Exporter (grafana-cold-storage#674) * Embedding the Kafka Lag Exporter * update default config * changelog and docs updated * removing tabs from doc file * removing tabs from doc file * solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags * Embedding the Kafka Lag Exporter * update default config * changelog merged * removing tabs from doc file * removing tabs from doc file * solving naming standards and identiation issues, adding defaults according to the documentation and checks for mandatory config flags * some other renames and identations * fixing misspelling * fixing field names and docs * Remove trailing empty line Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * adding docker-compose example for local testing * Merge from main and reformat config Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> * Support autologging span attributes as loki labels (grafana-cold-storage#703) * Support autologging span attributes as log labels * Update test * Fix test * apply suggestions Signed-off-by: Mario Rodriguez <mariorvinas@gmail.com> * Faster stringify * Support no keyvalues * Correctly size map * Fix config * Skip STATUS_CODE_UNSET to be less spammy * Fix tests * Use struct map * Fix rebase * Improve docs * Apply suggestions * yeah, don't change that * Grafana Agent Operator: generate logs config (grafana-cold-storage#740) * wip generate logging config * generate log client * generate config for pipelines * generate pod logs * fix lint error * buckets precision warning * crds: denote required fields, fix typos * buffer notifier channel (grafana-cold-storage#744) * Clean up K8s deployment methods (grafana-cold-storage#749) * Cleans up some deployment methods: - Updates README directing users to Graf Cloud docs for full instructions - Removes generation of `agent.yaml` and `agent-sigv4.yaml` from build script - Removes generated `agent.yaml` and `agent-sigv4.yaml` - Updates logs Tanka to remove ConfigMap (moved to Graf Cloud docs) - Removes install scripts * Update production/README.md * Add link * Update links * Fix broken links and update doc * Add line to direct users to quickstart guides * Add install-bare.sh warning (grafana-cold-storage#752) * add install-bare.sh warning * use emoji to make warning more warning-er * Ignore regex capture groups when expanding environment vars (grafana-cold-storage#754) * ignore regex capture groups when expanding environment vars * document behavior of regex capture group coercion * remove weird formatting in changelog * Implement Drone Build system, moving away from release via Github Actions (grafana-cold-storage#732) * Initial drone support * Fix drone file * Drone changes * Added support for drone building * Echo from makefile * Add logging info * Add logging info * Add logging info * Add debugging info * Add make debugging info * Add seego debugging * add ls to see why files arent getting copied * More debugging * simplify * Drone changes * Drone v2 Former-commit-id: 914b4a3 * Remove invalid files Former-commit-id: 7426669 * Fix freebsd build Former-commit-id: 8f1fc64 * Setting up tokens Former-commit-id: 246be58 * Removing windows build output Former-commit-id: 241d9bb * Fix for docker builds Former-commit-id: 3babea6 * Remove containerize no docker Former-commit-id: 90660b5 * Make buildx single threaded Former-commit-id: e1fa585 * Building against personal docker account Former-commit-id: 83bb1cc * Merge changes Former-commit-id: f6b8df9 * Remove exe Former-commit-id: 79ce552 * Removed V6 temporarily to see if fixed Former-commit-id: e6c5680 * Removed extra steps Former-commit-id: cb89f55 * Signed yaml Former-commit-id: e2665dd * Fix makefile Former-commit-id: 6bfcc9d * Fix tag Former-commit-id: 9ac2297 * Using drone tag Former-commit-id: 475d8cd * Fix drone tag Former-commit-id: 4d7029e * Fix drone issues Former-commit-id: bbb95ba * Fix docker not being available Former-commit-id: 4387301 * Set build in container flag Former-commit-id: 7ed76a9 * fix Former-commit-id: 2623c6e * fix Former-commit-id: c27a8ff * fix Former-commit-id: 06d8e36 * remove dead code Former-commit-id: b074c9e * comments Former-commit-id: eee8e0c * sign Former-commit-id: c29ec03 * comments Former-commit-id: 5c320aa * comments Former-commit-id: acacde9 * comments Former-commit-id: 8acfe15 * comments Former-commit-id: 3a40480 * comments Former-commit-id: d7e3103 * comments Former-commit-id: 84303ac * comments Former-commit-id: 2c56ddc * comments Former-commit-id: 978dc72 * comments Former-commit-id: 61a5cfb * Update changelog Former-commit-id: 2348bf7 * comments Former-commit-id: 4ce65c6 * test build Former-commit-id: fbf9ecc * test build Former-commit-id: 1a845d8 * test build Former-commit-id: 57c0350 * test build Former-commit-id: 75305d7 * test build Former-commit-id: 76b79a4 * test build Former-commit-id: 9517eae * test build Former-commit-id: 9f57b5d * test build Former-commit-id: 6c37355 * test build Former-commit-id: 738b53b * test build Former-commit-id: d3a389e * Cleanup Former-commit-id: c44965e * Remove release.key Former-commit-id: 1f743bf * Fix for windows installer build Former-commit-id: b8221d5 * Allow windows installer builds Former-commit-id: a977ecb * Simplify makefile * Remove unused files * Add test back to github * Fix normal makes * Reorder changelog * Fix arm64 * Arm fix * Remove comment lines * rename governance * use seego * Simplify seego usage * include fix for test * Fix for dout * Only build docker container if needed * Add depends on * Fix naming * Unsilent any remaining output * unsilent * Remove check-seego and switch to using docker by default * Change triggers * rename master to main * Fix dependency order now that Dist only triggers on tags * Sign drone * mount local go mod cache to seego if one exists * Clean up triggers * Add trigger to dist * testing change for synchronized * testing change for synchronized * Undo changeS * sign drone * Prevent double build * remove extra check * deduplicate build * deduplicate build Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Logs: create positionsdir per config (grafana-cold-storage#757) * create positions directory per instance * update docs * operator: add cluster label to all logs (grafana-cold-storage#758) * Merge changelog into main Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Gabriel <g.amaral.antunes@gmail.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Koenraad Verheyden <koenraad.verheyden@posteo.net> Co-authored-by: Justin Walz <justin.walz10@gmail.com> Co-authored-by: Tharun Rajendran <rajendrantharun@live.com> Co-authored-by: Alexander <kazakov.a.s.90@gmail.com> Co-authored-by: Annanay Agarwal <annanay25@users.noreply.github.com> Co-authored-by: Cristian Greco <cristian@regolo.cc> Co-authored-by: hanif <hjet@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com>
…ups (grafana-cold-storage#771) * deb package: add grafana-agent user to adm and systemd-journal groups Closes grafana-cold-storage#737 * changelog * fix comment * mirror change to rpm * update changelog * lint packaging scripts * fix lint errors * invert condition for group check
…rafana-cold-storage#769) * Update dependencies * Add image tag and operator image * Add signed drone * Remove extra depends on * Update Makefile Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
* Split load balancing * Point to load balancing from tail sampling * Old configs are not supported * Add warning log when no load balancing * Improve some sentences
* Update postgres exporter version * Update changelog with postgres exporter version bump * Preferred ordering in Changelog. Updated postgres integration docs to reflect new configuration options * Set the default metric prefix
…rafana-cold-storage#776) * TestEnqueueRequestForSelector: only run test if -online flag is set This test does download and invoke kubebuilder-tools, to make sure there is no extra steps needed just to run this one test. (grafana-cold-storage#731 (comment)). However, distros usually build and run tests in a sandbox with network access disabled, causing this test to fail. Introduce a `online` flag, defaulting to false, and skip this test as long as `-online` is not passed. CI, which invokes `make test` deliberately sets this to true, to have coverage for that test provided by CI. Fixes grafana-cold-storage#731. * fix flag passing Co-authored-by: Robert Fratto <robertfratto@gmail.com>
* Use Deployment for traces quickstart * Add link to traces quickstart * Set HOSTNAME env var regardless of application * format, move container env to agent.libsonnet Co-authored-by: Robert Fratto <robertfratto@gmail.com>
The original [documentation](../operator/_index.md) makes the link broken; removing the _index.md part should fix it based on similar links in this and other pages.
* Initial commit of github exporter integration * Add github exporter config docs. Use a fork of github-exporter with small bug fixes and labeling improvements. gofmt * Linting fixes for github-exporter * Okay... More linting then * Maybe running the linter locally instead of waiting for CI to bark at you is the wiser course. * Minor github-exporter documentation change. Update github-exporter dependency to fork which cleans up the config steps * Update docs/configuration/integrations/github-exporter-config.md Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update docs/configuration/integrations/github-exporter-config.md Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Check for errors from github exporter config * Actually import the packages you're using * Rebase to main and resolve go.sum conflict Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
* Initialize biggest ref to existing ref when reading a segment (grafana-cold-storage#676) * initialize biggest ref to existing ref when reading a segment * changelog * allow new test to be less hard-coded * explain error if test fails * clarify changelog issue * prepare for v0.16.1 release (grafana-cold-storage#679) * adding docker-compose example for local testing (cherry picked from commit 992e3cf) * Cherry picking kafka importer * update Operator FAQ to describe custom scrape jobs (grafana-cold-storage#658) * Merge * merge for race condition * Wall cherry pick * Import Kafka * Bump version number to v0.17.0 * Initial release of v0.18.0 includes github_exporter, enabled flag issue, OTLP HTPP, and updated postgres exporter * Tempo/traces docs fixes for v0.18.0 (grafana-cold-storage#784) * Needs to be loki for the naming change * Update date * No change, just trying to trigger drone * Fix for drone naming * Changes that needed to me imported for drone to work. * Update drone issues * sign drone * Drone changes * Update operator * standardize on seego * Test tag * fix tag * updated makefile * release tag fixing * remove extra tag * Readd missing upgrade info * Restructure based on new doc format * Small tweaks * Merge issues and naming * Change some minor formatting * Change some minor formatting Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Gabriel <g.amaral.antunes@gmail.com> Co-authored-by: Mario <mariorvinas@gmail.com>
* Initialize biggest ref to existing ref when reading a segment (grafana-cold-storage#676) * initialize biggest ref to existing ref when reading a segment * changelog * allow new test to be less hard-coded * explain error if test fails * clarify changelog issue * prepare for v0.16.1 release (grafana-cold-storage#679) * adding docker-compose example for local testing (cherry picked from commit 992e3cf) * Cherry picking kafka importer * update Operator FAQ to describe custom scrape jobs (grafana-cold-storage#658) * Merge * merge for race condition * Wall cherry pick * Import Kafka * Bump version number to v0.17.0 * Initial release of v0.18.0 includes github_exporter, enabled flag issue, OTLP HTPP, and updated postgres exporter * Tempo/traces docs fixes for v0.18.0 (grafana-cold-storage#784) * Needs to be loki for the naming change * Update date * No change, just trying to trigger drone * Fix for drone naming * Changes that needed to me imported for drone to work. * Update drone issues * sign drone * Drone changes * Update operator * standardize on seego * Test tag * fix tag * updated makefile * release tag fixing * remove extra tag * consul specific list support added * Fix type * Add comment * Fix for etcd issue that I introduced * Update to v0.18.1 * Fix for test error * Fix small naming/style issues * Fix small naming/style issues * Update go.mod * Update configuration changes to v0.18.1 Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Gabriel <g.amaral.antunes@gmail.com> Co-authored-by: Mario <mariorvinas@gmail.com>
…fana-cold-storage#790) * operator: regenerate CRDs with controller-gen controller-gen@latest now uses kubebuilder v0.6.2, which does linebreaks. Reformat to keep the diff smaller for future changes. * operator: crds: fix typo in EnforcedNamespaceLabel * controller-gen: explicitly pin versions Right now, we only asked to install `@latest`. controller-gen@v0.6.2 changed the word wrapping. Developers installing this according to `cmd/agent-operator/DEVELOPERS.md` will get CRDs formated differently than what the container image will create, and the container image will produce new outputs whenever the container layer gets recreated. Explicitly provide a version, and add cross-references on both sides to ensure this stays in sync. * Bump build-image version Co-authored-by: Robert Fratto <robertfratto@gmail.com>
…storage#808) * Ensure drone tag always has a value, default to v0.0.0 * Sign drone and remove info messages * This is a temporary fix to test the containerize functionality. * Rollback check for containerization on branch
* add HOSTNAME env variable to service file * changelog * changelog sort Co-authored-by: Robert Fratto <robertfratto@gmail.com>
…orage#819) * Fix darwin filesystem collector defaults * Changelog update
* Initialize biggest ref to existing ref when reading a segment (grafana-cold-storage#676) * initialize biggest ref to existing ref when reading a segment * changelog * allow new test to be less hard-coded * explain error if test fails * clarify changelog issue * prepare for v0.16.1 release (grafana-cold-storage#679) * adding docker-compose example for local testing (cherry picked from commit 992e3cf) * Cherry picking kafka importer * update Operator FAQ to describe custom scrape jobs (grafana-cold-storage#658) * Merge * merge for race condition * Wall cherry pick * Import Kafka * Bump version number to v0.17.0 * Initial release of v0.18.0 includes github_exporter, enabled flag issue, OTLP HTPP, and updated postgres exporter * Tempo/traces docs fixes for v0.18.0 (grafana-cold-storage#784) * Needs to be loki for the naming change * Update date * No change, just trying to trigger drone * Fix for drone naming * Changes that needed to me imported for drone to work. * Update drone issues * sign drone * Drone changes * Update operator * standardize on seego * Test tag * fix tag * updated makefile * release tag fixing * remove extra tag * consul specific list support added * Fix type * Add comment * Fix for etcd issue that I introduced * Update to v0.18.1 * Fix for test error * Fix small naming/style issues * Update version number to v0.18.2 and fix issues with prefixes. * Update missed merges * Add operate allowed version Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Gabriel <g.amaral.antunes@gmail.com> Co-authored-by: Mario <mariorvinas@gmail.com>
* basic working example * increase default sample backoff to 1.25s * increase check frequency * tests for ready * review feedback * finish initial implementation * sign drone * fix lint issues * Update pkg/crow/samples.go
…old-storage#831) * Fix issue where windows logger info messages only log "\n" * Update changelog
* adding mongodb exporter - draft * commiting for local tests * adding general collector * updating mongodb exporter version * removing direct connect option - always use direct connection * mongodb exporter draft * adding mongodb to example/docker-compose * restoring agent-local default config * restoring docker-compose agent default config * adjusting identation * Adjusting logging and solving lint problems * nolint on logruskit * yet some other lint problems * replacing elastic to mongodb in comment
* update docs from website changes * fix broken links Co-authored-by: Robert Fratto <robertfratto@gmail.com>
…torage#791) Co-authored-by: Robert Fratto <robert.fratto@grafana.com>
Contributor
Author
Looks like I have to rewrite history to change my username etc. Is that ok or would you rather me open a new PR? Also ok if you would like to do it. Edit - Tried rewiritng history - looks like it worked. Sorry I somehow seem to have duplicated a bunch of writes. |
shamsalmon
requested review from
gaantunes,
joe-elliott,
mapno and
rgeyer
as code owners
February 16, 2022 21:32
rfratto
enabled auto-merge (squash)
February 16, 2022 22:03
Contributor
rewriting history is fine 👍 I set this to automerge once the CI goes green. Thanks again for the contribution! |
mattdurham
added a commit
that referenced
this pull request
Feb 25, 2022
* Update node_exporter dependency to v1.3.1 (#1228) * Add node_exporter to depcheck * update weaveworks/common dependency * map current release flags and changed defaults * documentation * revert accidental checkin * print out flags when node_exporter test fails to assist debugging * oops, i introduced some flags from master by mistake * Introduce experimental integrations revamp (#1198) * [dev.multiple-integrations] Enable present integrations by default, deprecate enabled field (#1062) * integrations: default to enabled by default * document deprecation of enabled * pkg/integrations: support *_configs field for integrations (#1130) Creates the basic code to unmarshal integrations from a YAML field called <integration name>_configs, which is a slice of that integration. Note that this is NOT wired up to the integrations manager yet, and trying to run the agent with more than one integration of the same type will likely cause problems. * [dev.multiple-integrations] Prototype new integrations subsystem (#1142) * wip: prototype new integrations subsystem * implement Controller with basic logic for Integration and UpdateIntegration * Implement HTTPIntegration for Controller * decouple controller and subsystem * don't have controller implement integration slightly less smelly now * multiplexer integration * rely on boilerplate for multiplexing for now generics would be nice here * remove multiplex_integration.go Also a little code smelly. Instead of having integrations that run other integrations, I'm going to fall back to having only one controller. * introduce Subsystem, unexport Controller start wiring up things to Subsystem * introduce v2 agent integration to use for testing * start wiring metrics integrations * rename Options to Globals call a spade a spade * add subsystem options to globals * remove dead code * metricsutils: calculate self-scraping based on globals * complete HTTP target API * working example with agent integration * appease the linter * don't return an error when context to cancel an integration is closed * once again i am asking the linter to forgive my typos * fix bug where labels from individual targets were getting dropped at the API endpoint * pkg/config: fix broken test * finish unit tests for integrations v2 controller * metricsutil/metricshandler_integration: make job name unique Before this change, the job name would have collided when using multiple instances of the same integration. * ensure that global subsystem labels are injected into targets * integrations/v2: Infer target hostname from SD API host (#1175) * [dev.multiple-integrations] integrations/v2: allow shimming between v1 and v2 integrations. (#1179) * integrations/v2: allow shimming between v1 and v2 integrations. Shimming is done by changing how the integration registration works; a new RegisterDynamic was added that allows for creating Configs at runtime. Here be dragons; this should be removed whenever we no longer have a need for it. * fix lint * pkg/integrations/v2: use "RegisterLegacy" instead of a generic mechanism * fine, I won't add the deprecation notice if it will make the linter sad * pkg/integrations: re-align (#1181) This commit reverts 69ba2dd in favor of allowing the new subsystem to handle multiple instances of integrations. This commit also removes the wal_truncate_frequency field from integrations as it is the only field from old integrations that does not have a current counterpart. * [dev.multiple-integrations] Hide integrations/v2 behind a feature flag (#1185) * feature flag wip * dynamically switch between integrations v1 and v2 default to v1. * pkg/integrations/versionselector to file in pkg/config * pkg/config: fix defaults for Integrations * pkg/config: use more generic way to unmarshal differently based on flag * add missing godoc comment * more comments * switch to deferred unmarshaling * remove unused Config field * simplify completeUnmarshal * do not perform lazy deferred unmarshaling * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter * Merge main into dev.multiple-integrations (#1184) * Fix typo (#1141) * Traces: Improved pod association in PromSD processor (#1137) * Improve k8s pod association * Add tests * Changelog * typo * Add prom_sd_pod_association * Extend tests for pod associations * Docs for pod association config * Lint fixes * Move to unreleased * Add instrumentation recommendations * Remove uncessary constants * Improve tests * remote config with http(s) provider (#1143) * sample remote config code with http provider * use t.TempDir() in unit test * no need to clean up after T.TempDir() * use NewClientFromConfig and make caller responsible for calling SetDirectory * handle nil HTTPClientConfig * remove blank identifier assignment * pass basic auth command line flags for remote config * address pr nits * add expiremental flag * set loader inline * update changelog * add remote config section in docs * pr comment updates * announce patch releases for cve-2021-41090 (#1152) * Merge patch release to main (#1153) * Add secret type to sensitive values * Break out config tests to their own implementation. Also remove username has a sensitive value. * Update changelog * Fix failing test * Scrub secrets when marshaling instance configs * update for v0.21 * Updated changes from the merge. * Remove changelog * Scrub out receivers has ***receivers_scrubber***:null * obscure etcd/consul credentials * Update pkg/traces/config_test.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update pkg/config/config.go * go fmt * Change to using custom object and return <secret> * Fix bad merge * [v0.21.2] toggle config endpoint (#19) * disable /-/config endpoint by default * disable scraping api get endpoint as well * fix new test * add test and rename flag Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Update version to v0.21.2 * Update defaults.go * fix /-/config endpoint * also fix non-pointer config bug * temporarily disable linting for release * fix lint errors Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter (#1162) * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter A recent change broke the usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME for the postgres_exporter. As the incorrect variable was checked in the if clause, it always raises an error. * changelog: keep feature -> enhancement -> bugfix order * postgres_exporter: add regression test Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix syntax error in Jsonnet logs helper method (#1174) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * cAdvisor Integration (#1081) * Add cadvisor module * Begin creating common config for cadvisor * Don't export internal state * Finish config options for cadvisor * Set config options, and implement cAdvisor collectors * Linting * Buildflags for cadvisor only in linux * I R LEArN Build Tags * Don't zero value the zero value * Offload sketchy global var manipulation to the integrations Run func * Remove unused collectors * Lint * Create generic stub integration and use it for cadvisor * Lint * Final refactor of cAdvisor config for unsupported platforms. Pared down stub integrations. * Lint * Docs for cadvisor config * Update changelog * Update pkg/integrations/stub_integration.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Reorder changelog * Instance key clarity * Inclusive naming * Finish name changes Keep default disable metric list in sync with upstream Idiomatic golang * Hardcode disabled metrics for cadvisor Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Remove log-level flag from systemd unit file (#1177) * Upgrade to OTel v0.40.0 (#1176) * Upgrade to OTel v0.40.0 * Changelog * Add factories check * go mod tidy * config/features: create package to standardize experimental features (#1170) * config/features: create package to standardize experiemental features This commit introduces a new package, pkg/config/features, which allows defining a set of features and validating whether flags associated with those features are allowed to be set. Closes #1163 * update documentation (also s/enabled-features/enable-features) * Fix typo * Update pkg/config/features/features.go Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Revert "Merge main into dev.multiple-integrations (#1184)" (#1189) This reverts commit ad76ec5. * [dev.multiple-integrations] Revert breaking changes to existing integrations (#1191) * revert breaking changes to integrations v1 This commit reverts #1062 in favor of making breaking changes directly in integrations-next instead. The part of #1181 to remove `wal_truncate_frequency` has also been reverted. As part of this change, the enabled field is removed from the v2 common metrics configs, and v2 integrations can no longer be disabled. v2 integrations can only be disabled by removing them from the YAML. * integrations/v2: remove stale reference to ErrDisabled (fix typo too) * integrations/v2: bring in common config decoupling * [dev.multiple-integrations] Introduce autoscraper (#1195) * pkg/integrations/v2: introduce self-scraping * linting * [dev.multiple-integrations] Multiple instances of integrations (#1196) * multiple instances of integrations opt in relevant v1 integrations into supporting multiple instances * shims should check for instance key override * Document integrations-next (#1197) * document integrations-next * remove json tags since they make markdown unhappy * changelog * s/Run/RunIntegration * remove stale comment about integrations.controller purpose * create dedicated run method for instanceScraper * s/expoter/exporter/g * Document why an autoscrape.Scraper manages a set of per-instance scrapers * spell out prerequisite instead of pre-req * use go.uber.org/atomic to make the code a little easier to follow * remove started callback for running integration * use smaller interface for autoscrape Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Fix panic when using 'stdout' in automatic logging (#1233) * integrations-next: fix bug where v2 integrations were not being strictly unmarshaled (#1235) * Remove jsonnet vendor folders (#1222) * remove jsonnet vendor This adds all vendor folders into .gitignore and removes cached vendor files from the repository. Closes #1221 * Update scripts and instructions for jsonnet vendor removal * `make example-dashboards` will now also run `jb install` * k3d environment instructions now include `jb install` * smoke-test.bash will now run `jb install` prior to `tk apply` * Fix link to k3d example in DEVELOPERS.md (#1242) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix node_exporter upgrade docs (#1239) * Fix panic in automatic logging with stdout backend (#1243) * pkg/util: support custom yaml.Unmarshaler implementations for util.UnmarshalYAMLMerged (#1244) It's common for config types to have implement yaml.Unmarshaler for: * Applying defaults * Applying extra logic post-unmarshal If these config types were unmarshaled through util.UnmarshalYAMLMerged, the yaml.Unmarshaler implementation would never complete successfully, preventing the post-unmarshal logic from running. This issue was introduced in #1192, but went unnoticed until #1228 implemented yaml.Unmarshaler to perform field migrations. #1240 reported the issue. This commit fixes the bug by performing a second non-strict unmarshal to ensure that all input values unmarshal successfully, with the exception of unmarshal errors unrelated to unrecognized field names. This is hacky, but it's worthwhile noting that util.UnmarshalYAMLMerged is a temporary workaround needed for the integrations-next migration, and will eventually be removed. * Update k3d example grafana/grafonnet-lib version (#1246) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Create an e2e framework with support for running tests against k8s (#1234) * e2e: create an e2e framework with support for running tests against a k3d cluster * add new E2E drone job * E2E tests should pass when doing a release * sign drone.yml again * move e2e lint to different step that has golangci-lint installed * upgrade golangci-lint and go for e2e test * e2e: add gcc * E2E: install build-essential to get a working full gcc env * :( * e2e: support running from inside of docker * fix lint error * address review feedback * Operator: fix bug where /-/ready and /-/healthy always returned 404 (#1252) * operator: fix bug where /-/ready and /-/healthy always returned 404 controller-runtime must have at least one ready/healthy check for the endpoints to exist * fix lint error, use healthz.Ping * Make scraping-svc use the new `metrics:` key (#1259) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * update prometheus dependency (#1260) * corrected typo (#1265) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags (#1264) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use :main tag for images in smoke test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Set IMAGE_BRANCH_TAG env var in drone and actions pipelines Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove quotes from Makefile variable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove force_release action Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * prepare for v0.22.0 release (#1266) * prepare for v0.22.0 release * remove E2E pipeline * Add basic testing framework for operator (#1268) * remove dedicated go.mod for e2e/ * move e2e/k8s to pkg/util/k8s * Migrate operator tests to pkg/util/k8s * remove dedicated e2e tests * allow skipping TestCluster in pkg/util/k8s * remove e2e/ * fix bad merge * fix order of make env args for windows * actually declare referenced docker volume * introduce pkg/util/subset for asserting subset of objects * refactor operator so it's testable * define basic integration test for operator * fix lint errors * fix invalid address in operator test config * Update release-note.md (#1267) * Set scrape User-Agent header during init (#1274) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Upgrade to Go 1.17 (#1278) * Upgrade to 1.17.6 in go.mod and Dockerfiles * Update CHANGELOG.md to mention the update * Update Go version in drone/actions pipelines * Update go.mod, go.sum files via * Re-sign drone.yml * Remove leading newline causing drone build to fail * Bump golangci-lint image to a version using Go 1.17 * Re-attempt to solve linter issue with new golangci-lint image * Remove suffix of exclude rules * Clean previous Go version before unpacking Go 1.17 * Also clean up previous Go versions in other steps * fix typo (#1284) * Use custom Go version in agent-operator Dockerfile (#1286) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * pkg/operator: refactor resource hierarchy discovery (#1271) * pkg/operator: refactor resource hierarchy discovery This commit moves common logic related to discovering the resource hierarchy to pkg/operator/hierarchy. This new package requires less boilerplate, which the reconciler is updated to take advantage of. * remove unused code * test construction of resource hierarchy * add missing build constraints * small extra cleanup to use pointer package * review feedback * update agent-build-image for go 1.17 (#1287) (also use a consistent base image tag instead of latest) * Skip non-ready entries when listing instances (#1289) * Skip non-ready instances in LoadInstances() Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix panic in prom_sd_processor when address is empty (#1279) * Fix panic in prom_sd_processor when address is empty * Fix panic in prom_sd_processor when address is empty * Fix docs * Add test case * Lint * Move to unreleased * Operator: generate proxy_url for remote_write (#1298) * operator: generate proxy_url for remote_write * fix weird indentation in test * Use log format in traces subsystem (#1272) * Use log format in traces subsystem * Changelog * Undo unwanted change * Fix changelog entry * integrations-next: Add extra_labels to inject extra labels for an integration (#1312) * integrations-next: Add extra_labels to inject extra labels for an integration. * separate tests * Fix anchor link on operator docs (#1302) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * updated config URL (#1304) The existing URL returns a 404: https://grafana.com/docs/agent/latest/getting-started/configuration/_index.md Updated to https://grafana.com/docs/agent/latest/configuration/ * Fix typo in node_exporter (#1325) * Allow remote_write URL credentials (#1329) * Bypass Prometheus password redaction Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add inline secret in existing test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry * Add to scrubbed testcase as well Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Stop appending duplicate exemplars (#1316) * Add memExemplar in stripeSeries as first iteration Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add test for skipped duplicate exemplars; Simplify conditional Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry; discard test errors * Move changelog entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add Benchmark for AppendExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Discard error on added benchmark Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use original exemplar struct instead of custom memExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Surround benchmark loop with start/stop timers and close test storage Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add comment about prepopulating exemplars on WAL startup Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in the totalAppendedExemplars metric Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make comment more discoverable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make sure we're recording exemplars for non-nil series ref only Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * integrations-next: wait for integrations to exit after stopping them (#1318) * integrations-next: wait for integrations to exit after stopping them * fix lint errors * minor refactor * integrations-next: stop holding config mutex for entire reload * make controller.run authoritative over running integrations * fix log line * move running integrations into a dedicated worker pool * operator/hierarchy: stop using field selector when listing Secrets & ConfigMaps (#1340) The initial implementation of hierarchy.KeySelector injected a FieldSelector when listing Secrets and ConfigMaps to immediately return the single object being queried for. This causes a problem with the client generated by the controller-runtime framework, where the client is wrapped in a cache and field indexer (where only the namespace is indexed by default). This commit avoids using the field selector and the index lookup. The resulting behavior aligns more closely with discovering other resources in the hierarchy (i.e., ServiceMonitors), where the List call is also insufficient and needs post-processing via Matches to find the final list of resources. Given the controller-runtime client uses an informer for reads, all relevant Secrets and ConfigMaps are already in-memory anyway, and using the index for a faster List is a bit of an over-optimization at the moment. * Add dependabot to update go modules and github actions. (#1217) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * smoke framework refactor (#1326) * Agent smoke test (#1291) * convert smoke script to go program * update build for agent-smoke * fix pr comments * use existing log helper package * refactor context cancel * update exit codes * use ticker * prefer oklog/run instead of errgroup * use nop logger * refactor task interface * remove functional options * log.With for task loggers * move smoke to tools * build smoke image, push to internal registry * move crow to tools * add gcr_admin secret * fix link to crow * add smoke libsonnet and use in local k3d smoke test * add deletePodBySelectorTask * scale smoke-test replica down after local test * refactor smoke Options to Config * update duration usage message * add some basic unit tests * newlines * pass mutation frequency and chaos frequency from smoke script * pull crow image from gcr * update smoke script * move monitoring to smoke libsonnet * move additional smoke resources needed in deployment tools * reference libsonnet files from grafana-agent dep * make drone * fix images in smoke script * get rid of extVars * update k3d example environment to reference etcd from new location * update smoke docker builds to use go1.17 * use pointer.Int64 * refactor smoke jsonnet (#1296) * add policy rule for list and delete pods (#1319) * refactor smoke.new function to take config object (#1327) * Apply suggestions from code review * Update production/tanka/grafana-agent/smoke/crow/main.libsonnet * Update production/tanka/grafana-agent/smoke/main.libsonnet * Update example/k3d/scripts/smoke-test.bash Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * readme update (#1338) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Correct link to the configuration (#1036) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Add stale check Github Action (#1345) * Add a stale check GH action to run every 24 hours * remove old stale.yml file * add permissions to action * update the stale message to clarify when the stale label will get removed * Update .github/workflows/stale.yml * stale action: fix missing indent (#1346) * Fix mssql issue (#1351) * Add K8s Events integration (#1330) * Add K8s eventhandler integration (#1310) * Add docs and sample manifests to eventhandler integration (#1328) * Wait for cache to flush before returning * Clarify eventhandler docs (#1334) * Clarify docs * Update CHANGELOG.md * Review changes (#1349) * stale action: fix typo in label exemptions (#1347) * update withVolumesMixin for agent jsonnet (#1358) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Configure cluster label using logs client external_labels param (#1357) * Configure cluster label using logs client external_labels param * Update CHANGELOG.md * add password file and basic auth round tripper in crow (#1361) * add password file and basic auth round tripper in crow * add ca-certificates in crow image * add orgID flag * update help text * default send_exemplars to true in remote_write (#1352) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update eventhandler labels (#1368) * Update eventhandler integration labels * Update CHANGELOG * Remove unnecessary kind label * update changelog (#1374) Remove BUGFIX entries that fix a bug introduced by main (i.e., bugs which were never part of a release) * Prepare for release of v0.23.0 (#1377) * Update version references * Fix fat-fingered delete; Remove mention of upgrade Go * RFC: Design in the open (#1055) * rfc: first draft of RFC0001 * add placeholder for PR * update PR link * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify "designing in the open" is best-effort * update 0001 * fix dead link in production/README.md * add recommended sections for RFC proposals * describe the process for approving a proposal * ignore RFC template in link checker * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * do my nitty 80-char line length limit change * indent pros/cons to a single section * document process for superseding RFCs * remove RFC mutability requirement * add extra flavor around not recommending google docs * require Google Doc -> RFC conversion * move new files Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Add Grafana Labs SECURITY.md (#1356) Signed-off-by: Richard Hartmann <richih@richih.org> * Add readiness check to metrics component (#1369) * PR Base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix autoscrape's mockInstance Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in atomic readiness check Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Add CHANGELOG.md entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Reference page to download windows installer (#1372) Fixes #1366 * fix typo in node_exporter_config (#1389) which should be `privileged` instead of `priviliged` * Add option for Operator to pass arguments to GrafanaAgent #1227 (#1248) * 1250 oauth2 tracing (#1386) * Add oauth support for trace Otel trace exporter via opentelemetry-collector-contrib oauth2clientauthextension * start extensions on collector instance startup fix decoding to otelconfig build extensions add oauth extension to service map * Update traces config documentation * lint fixes * fix godoc comments * pass exporter index directly to exporter name generator * PR feedback; Update Changelog * sort extensions when sorting pipelines for testing determinism * README: Fix link to agent logo (#1396) * update MAINTAINERS.md (#1402) * add smoke alerts to mixin; move local alerts into examples dir (#1397) * add smoke alerts to mixin; move local alerts into examples dir * add podPrefix for smoke test * podPrefix in libsonnet config * [RFC] Integrations in Grafana Agent Operator (#1224) * rfc: integrations in grafana agent operator Supersedes #883 * add missing links * Apply suggestions from code review Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify how many daemonsets/deployments/service/secrets are created * add example of defining secrets * try defining integrations * s/IntegrationsMonitor/IntegrationMonitor/g * simplify proposal * add alternatives * remove old reference to `hasMetrics` field * document example generated agent configuration file * assign ID RFC-0002 * add missing PR link Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * add fake rw endpoint to smoke program (#1405) * fix alerts typo (#1407) * continuous delivery for smoke images (#1408) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * fix continuous delivery job errors (#1409) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * [operator] - Use _file variants for basic auth credentials. (#1411) * use password_file alternatives in operator config * update tests * reduce smoke alert noise (#1412) * reduce smoke alert noise Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update production/grafana-agent-mixin/alerts.libsonnet Co-authored-by: Robert Fratto <robertfratto@gmail.com> * update cpu check comment Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * add minimum load threshold to cpu alert Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Clarify usage of instanceNamespaceSelector (#1413) * RFC-0001: Add status to RFC (#1391) * rfc-0001: add rules for when RFC PRs should be merged * use status field instead of merge to indicating state * Parametrize logs DaemonSet K8s manifests (#1420) * Parametrize logs daemonset K8s manifests * Update CHANGELOG.md * Extend linting configuration file (#1421) * Add depguard linter to reject packages we tend to avoid * Replace golint with revive, since golint is deprecated * Remove interfacer, which is deprecated with no replacement * Add makezero linter to detect misuse of make with append * Add tenv to prefer t.Setenv over os.Setenv in tests * Add whitespace to report unnecessary blank lines * Ignore test files for errcheck In addition to the above, the following changes were made: * Remove settings that just re-set default values, instead pointing to the website to retrieve defaults. * Simplify the errcheck rule to only include functions we actually need to ignore. * Main merge changes Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: Patrick Koenig <pkoenig10@gmail.com> Co-authored-by: DataPoints <langer.markus@gmail.com> Co-authored-by: Alex <52292902+alexrudd2@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: melGL <81323402+melgl@users.noreply.github.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Joseph Woodward <josephwoodward@xeuse.com> Co-authored-by: hanif <hjet@users.noreply.github.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: laiwei <laiwei.ustc@gmail.com> Co-authored-by: Sam <shamsalmon@users.noreply.github.com> Co-authored-by: Chris Knutson <christopher.knutson@gmail.com> Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Craig Peterson <192540+captncraig@users.noreply.github.com>
mattdurham
added a commit
that referenced
this pull request
Feb 25, 2022
* Update node_exporter dependency to v1.3.1 (#1228) * Add node_exporter to depcheck * update weaveworks/common dependency * map current release flags and changed defaults * documentation * revert accidental checkin * print out flags when node_exporter test fails to assist debugging * oops, i introduced some flags from master by mistake * Introduce experimental integrations revamp (#1198) * [dev.multiple-integrations] Enable present integrations by default, deprecate enabled field (#1062) * integrations: default to enabled by default * document deprecation of enabled * pkg/integrations: support *_configs field for integrations (#1130) Creates the basic code to unmarshal integrations from a YAML field called <integration name>_configs, which is a slice of that integration. Note that this is NOT wired up to the integrations manager yet, and trying to run the agent with more than one integration of the same type will likely cause problems. * [dev.multiple-integrations] Prototype new integrations subsystem (#1142) * wip: prototype new integrations subsystem * implement Controller with basic logic for Integration and UpdateIntegration * Implement HTTPIntegration for Controller * decouple controller and subsystem * don't have controller implement integration slightly less smelly now * multiplexer integration * rely on boilerplate for multiplexing for now generics would be nice here * remove multiplex_integration.go Also a little code smelly. Instead of having integrations that run other integrations, I'm going to fall back to having only one controller. * introduce Subsystem, unexport Controller start wiring up things to Subsystem * introduce v2 agent integration to use for testing * start wiring metrics integrations * rename Options to Globals call a spade a spade * add subsystem options to globals * remove dead code * metricsutils: calculate self-scraping based on globals * complete HTTP target API * working example with agent integration * appease the linter * don't return an error when context to cancel an integration is closed * once again i am asking the linter to forgive my typos * fix bug where labels from individual targets were getting dropped at the API endpoint * pkg/config: fix broken test * finish unit tests for integrations v2 controller * metricsutil/metricshandler_integration: make job name unique Before this change, the job name would have collided when using multiple instances of the same integration. * ensure that global subsystem labels are injected into targets * integrations/v2: Infer target hostname from SD API host (#1175) * [dev.multiple-integrations] integrations/v2: allow shimming between v1 and v2 integrations. (#1179) * integrations/v2: allow shimming between v1 and v2 integrations. Shimming is done by changing how the integration registration works; a new RegisterDynamic was added that allows for creating Configs at runtime. Here be dragons; this should be removed whenever we no longer have a need for it. * fix lint * pkg/integrations/v2: use "RegisterLegacy" instead of a generic mechanism * fine, I won't add the deprecation notice if it will make the linter sad * pkg/integrations: re-align (#1181) This commit reverts 69ba2dd in favor of allowing the new subsystem to handle multiple instances of integrations. This commit also removes the wal_truncate_frequency field from integrations as it is the only field from old integrations that does not have a current counterpart. * [dev.multiple-integrations] Hide integrations/v2 behind a feature flag (#1185) * feature flag wip * dynamically switch between integrations v1 and v2 default to v1. * pkg/integrations/versionselector to file in pkg/config * pkg/config: fix defaults for Integrations * pkg/config: use more generic way to unmarshal differently based on flag * add missing godoc comment * more comments * switch to deferred unmarshaling * remove unused Config field * simplify completeUnmarshal * do not perform lazy deferred unmarshaling * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter * Merge main into dev.multiple-integrations (#1184) * Fix typo (#1141) * Traces: Improved pod association in PromSD processor (#1137) * Improve k8s pod association * Add tests * Changelog * typo * Add prom_sd_pod_association * Extend tests for pod associations * Docs for pod association config * Lint fixes * Move to unreleased * Add instrumentation recommendations * Remove uncessary constants * Improve tests * remote config with http(s) provider (#1143) * sample remote config code with http provider * use t.TempDir() in unit test * no need to clean up after T.TempDir() * use NewClientFromConfig and make caller responsible for calling SetDirectory * handle nil HTTPClientConfig * remove blank identifier assignment * pass basic auth command line flags for remote config * address pr nits * add expiremental flag * set loader inline * update changelog * add remote config section in docs * pr comment updates * announce patch releases for cve-2021-41090 (#1152) * Merge patch release to main (#1153) * Add secret type to sensitive values * Break out config tests to their own implementation. Also remove username has a sensitive value. * Update changelog * Fix failing test * Scrub secrets when marshaling instance configs * update for v0.21 * Updated changes from the merge. * Remove changelog * Scrub out receivers has ***receivers_scrubber***:null * obscure etcd/consul credentials * Update pkg/traces/config_test.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update pkg/config/config.go * go fmt * Change to using custom object and return <secret> * Fix bad merge * [v0.21.2] toggle config endpoint (#19) * disable /-/config endpoint by default * disable scraping api get endpoint as well * fix new test * add test and rename flag Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Update version to v0.21.2 * Update defaults.go * fix /-/config endpoint * also fix non-pointer config bug * temporarily disable linting for release * fix lint errors Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter (#1162) * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter A recent change broke the usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME for the postgres_exporter. As the incorrect variable was checked in the if clause, it always raises an error. * changelog: keep feature -> enhancement -> bugfix order * postgres_exporter: add regression test Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix syntax error in Jsonnet logs helper method (#1174) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * cAdvisor Integration (#1081) * Add cadvisor module * Begin creating common config for cadvisor * Don't export internal state * Finish config options for cadvisor * Set config options, and implement cAdvisor collectors * Linting * Buildflags for cadvisor only in linux * I R LEArN Build Tags * Don't zero value the zero value * Offload sketchy global var manipulation to the integrations Run func * Remove unused collectors * Lint * Create generic stub integration and use it for cadvisor * Lint * Final refactor of cAdvisor config for unsupported platforms. Pared down stub integrations. * Lint * Docs for cadvisor config * Update changelog * Update pkg/integrations/stub_integration.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Reorder changelog * Instance key clarity * Inclusive naming * Finish name changes Keep default disable metric list in sync with upstream Idiomatic golang * Hardcode disabled metrics for cadvisor Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Remove log-level flag from systemd unit file (#1177) * Upgrade to OTel v0.40.0 (#1176) * Upgrade to OTel v0.40.0 * Changelog * Add factories check * go mod tidy * config/features: create package to standardize experimental features (#1170) * config/features: create package to standardize experiemental features This commit introduces a new package, pkg/config/features, which allows defining a set of features and validating whether flags associated with those features are allowed to be set. Closes #1163 * update documentation (also s/enabled-features/enable-features) * Fix typo * Update pkg/config/features/features.go Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Revert "Merge main into dev.multiple-integrations (#1184)" (#1189) This reverts commit ad76ec5. * [dev.multiple-integrations] Revert breaking changes to existing integrations (#1191) * revert breaking changes to integrations v1 This commit reverts #1062 in favor of making breaking changes directly in integrations-next instead. The part of #1181 to remove `wal_truncate_frequency` has also been reverted. As part of this change, the enabled field is removed from the v2 common metrics configs, and v2 integrations can no longer be disabled. v2 integrations can only be disabled by removing them from the YAML. * integrations/v2: remove stale reference to ErrDisabled (fix typo too) * integrations/v2: bring in common config decoupling * [dev.multiple-integrations] Introduce autoscraper (#1195) * pkg/integrations/v2: introduce self-scraping * linting * [dev.multiple-integrations] Multiple instances of integrations (#1196) * multiple instances of integrations opt in relevant v1 integrations into supporting multiple instances * shims should check for instance key override * Document integrations-next (#1197) * document integrations-next * remove json tags since they make markdown unhappy * changelog * s/Run/RunIntegration * remove stale comment about integrations.controller purpose * create dedicated run method for instanceScraper * s/expoter/exporter/g * Document why an autoscrape.Scraper manages a set of per-instance scrapers * spell out prerequisite instead of pre-req * use go.uber.org/atomic to make the code a little easier to follow * remove started callback for running integration * use smaller interface for autoscrape Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Fix panic when using 'stdout' in automatic logging (#1233) * integrations-next: fix bug where v2 integrations were not being strictly unmarshaled (#1235) * Remove jsonnet vendor folders (#1222) * remove jsonnet vendor This adds all vendor folders into .gitignore and removes cached vendor files from the repository. Closes #1221 * Update scripts and instructions for jsonnet vendor removal * `make example-dashboards` will now also run `jb install` * k3d environment instructions now include `jb install` * smoke-test.bash will now run `jb install` prior to `tk apply` * Fix link to k3d example in DEVELOPERS.md (#1242) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix node_exporter upgrade docs (#1239) * Fix panic in automatic logging with stdout backend (#1243) * pkg/util: support custom yaml.Unmarshaler implementations for util.UnmarshalYAMLMerged (#1244) It's common for config types to have implement yaml.Unmarshaler for: * Applying defaults * Applying extra logic post-unmarshal If these config types were unmarshaled through util.UnmarshalYAMLMerged, the yaml.Unmarshaler implementation would never complete successfully, preventing the post-unmarshal logic from running. This issue was introduced in #1192, but went unnoticed until #1228 implemented yaml.Unmarshaler to perform field migrations. #1240 reported the issue. This commit fixes the bug by performing a second non-strict unmarshal to ensure that all input values unmarshal successfully, with the exception of unmarshal errors unrelated to unrecognized field names. This is hacky, but it's worthwhile noting that util.UnmarshalYAMLMerged is a temporary workaround needed for the integrations-next migration, and will eventually be removed. * Update k3d example grafana/grafonnet-lib version (#1246) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Create an e2e framework with support for running tests against k8s (#1234) * e2e: create an e2e framework with support for running tests against a k3d cluster * add new E2E drone job * E2E tests should pass when doing a release * sign drone.yml again * move e2e lint to different step that has golangci-lint installed * upgrade golangci-lint and go for e2e test * e2e: add gcc * E2E: install build-essential to get a working full gcc env * :( * e2e: support running from inside of docker * fix lint error * address review feedback * Operator: fix bug where /-/ready and /-/healthy always returned 404 (#1252) * operator: fix bug where /-/ready and /-/healthy always returned 404 controller-runtime must have at least one ready/healthy check for the endpoints to exist * fix lint error, use healthz.Ping * Make scraping-svc use the new `metrics:` key (#1259) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * update prometheus dependency (#1260) * corrected typo (#1265) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags (#1264) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use :main tag for images in smoke test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Set IMAGE_BRANCH_TAG env var in drone and actions pipelines Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove quotes from Makefile variable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove force_release action Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * prepare for v0.22.0 release (#1266) * prepare for v0.22.0 release * remove E2E pipeline * Add basic testing framework for operator (#1268) * remove dedicated go.mod for e2e/ * move e2e/k8s to pkg/util/k8s * Migrate operator tests to pkg/util/k8s * remove dedicated e2e tests * allow skipping TestCluster in pkg/util/k8s * remove e2e/ * fix bad merge * fix order of make env args for windows * actually declare referenced docker volume * introduce pkg/util/subset for asserting subset of objects * refactor operator so it's testable * define basic integration test for operator * fix lint errors * fix invalid address in operator test config * Update release-note.md (#1267) * Set scrape User-Agent header during init (#1274) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Upgrade to Go 1.17 (#1278) * Upgrade to 1.17.6 in go.mod and Dockerfiles * Update CHANGELOG.md to mention the update * Update Go version in drone/actions pipelines * Update go.mod, go.sum files via * Re-sign drone.yml * Remove leading newline causing drone build to fail * Bump golangci-lint image to a version using Go 1.17 * Re-attempt to solve linter issue with new golangci-lint image * Remove suffix of exclude rules * Clean previous Go version before unpacking Go 1.17 * Also clean up previous Go versions in other steps * fix typo (#1284) * Use custom Go version in agent-operator Dockerfile (#1286) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * pkg/operator: refactor resource hierarchy discovery (#1271) * pkg/operator: refactor resource hierarchy discovery This commit moves common logic related to discovering the resource hierarchy to pkg/operator/hierarchy. This new package requires less boilerplate, which the reconciler is updated to take advantage of. * remove unused code * test construction of resource hierarchy * add missing build constraints * small extra cleanup to use pointer package * review feedback * update agent-build-image for go 1.17 (#1287) (also use a consistent base image tag instead of latest) * Skip non-ready entries when listing instances (#1289) * Skip non-ready instances in LoadInstances() Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix panic in prom_sd_processor when address is empty (#1279) * Fix panic in prom_sd_processor when address is empty * Fix panic in prom_sd_processor when address is empty * Fix docs * Add test case * Lint * Move to unreleased * Operator: generate proxy_url for remote_write (#1298) * operator: generate proxy_url for remote_write * fix weird indentation in test * Use log format in traces subsystem (#1272) * Use log format in traces subsystem * Changelog * Undo unwanted change * Fix changelog entry * integrations-next: Add extra_labels to inject extra labels for an integration (#1312) * integrations-next: Add extra_labels to inject extra labels for an integration. * separate tests * Fix anchor link on operator docs (#1302) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * updated config URL (#1304) The existing URL returns a 404: https://grafana.com/docs/agent/latest/getting-started/configuration/_index.md Updated to https://grafana.com/docs/agent/latest/configuration/ * Fix typo in node_exporter (#1325) * Allow remote_write URL credentials (#1329) * Bypass Prometheus password redaction Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add inline secret in existing test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry * Add to scrubbed testcase as well Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Stop appending duplicate exemplars (#1316) * Add memExemplar in stripeSeries as first iteration Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add test for skipped duplicate exemplars; Simplify conditional Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry; discard test errors * Move changelog entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add Benchmark for AppendExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Discard error on added benchmark Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use original exemplar struct instead of custom memExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Surround benchmark loop with start/stop timers and close test storage Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add comment about prepopulating exemplars on WAL startup Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in the totalAppendedExemplars metric Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make comment more discoverable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make sure we're recording exemplars for non-nil series ref only Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * integrations-next: wait for integrations to exit after stopping them (#1318) * integrations-next: wait for integrations to exit after stopping them * fix lint errors * minor refactor * integrations-next: stop holding config mutex for entire reload * make controller.run authoritative over running integrations * fix log line * move running integrations into a dedicated worker pool * operator/hierarchy: stop using field selector when listing Secrets & ConfigMaps (#1340) The initial implementation of hierarchy.KeySelector injected a FieldSelector when listing Secrets and ConfigMaps to immediately return the single object being queried for. This causes a problem with the client generated by the controller-runtime framework, where the client is wrapped in a cache and field indexer (where only the namespace is indexed by default). This commit avoids using the field selector and the index lookup. The resulting behavior aligns more closely with discovering other resources in the hierarchy (i.e., ServiceMonitors), where the List call is also insufficient and needs post-processing via Matches to find the final list of resources. Given the controller-runtime client uses an informer for reads, all relevant Secrets and ConfigMaps are already in-memory anyway, and using the index for a faster List is a bit of an over-optimization at the moment. * Add dependabot to update go modules and github actions. (#1217) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * smoke framework refactor (#1326) * Agent smoke test (#1291) * convert smoke script to go program * update build for agent-smoke * fix pr comments * use existing log helper package * refactor context cancel * update exit codes * use ticker * prefer oklog/run instead of errgroup * use nop logger * refactor task interface * remove functional options * log.With for task loggers * move smoke to tools * build smoke image, push to internal registry * move crow to tools * add gcr_admin secret * fix link to crow * add smoke libsonnet and use in local k3d smoke test * add deletePodBySelectorTask * scale smoke-test replica down after local test * refactor smoke Options to Config * update duration usage message * add some basic unit tests * newlines * pass mutation frequency and chaos frequency from smoke script * pull crow image from gcr * update smoke script * move monitoring to smoke libsonnet * move additional smoke resources needed in deployment tools * reference libsonnet files from grafana-agent dep * make drone * fix images in smoke script * get rid of extVars * update k3d example environment to reference etcd from new location * update smoke docker builds to use go1.17 * use pointer.Int64 * refactor smoke jsonnet (#1296) * add policy rule for list and delete pods (#1319) * refactor smoke.new function to take config object (#1327) * Apply suggestions from code review * Update production/tanka/grafana-agent/smoke/crow/main.libsonnet * Update production/tanka/grafana-agent/smoke/main.libsonnet * Update example/k3d/scripts/smoke-test.bash Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * readme update (#1338) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Correct link to the configuration (#1036) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Add stale check Github Action (#1345) * Add a stale check GH action to run every 24 hours * remove old stale.yml file * add permissions to action * update the stale message to clarify when the stale label will get removed * Update .github/workflows/stale.yml * stale action: fix missing indent (#1346) * Fix mssql issue (#1351) * Add K8s Events integration (#1330) * Add K8s eventhandler integration (#1310) * Add docs and sample manifests to eventhandler integration (#1328) * Wait for cache to flush before returning * Clarify eventhandler docs (#1334) * Clarify docs * Update CHANGELOG.md * Review changes (#1349) * stale action: fix typo in label exemptions (#1347) * update withVolumesMixin for agent jsonnet (#1358) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Configure cluster label using logs client external_labels param (#1357) * Configure cluster label using logs client external_labels param * Update CHANGELOG.md * add password file and basic auth round tripper in crow (#1361) * add password file and basic auth round tripper in crow * add ca-certificates in crow image * add orgID flag * update help text * default send_exemplars to true in remote_write (#1352) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update eventhandler labels (#1368) * Update eventhandler integration labels * Update CHANGELOG * Remove unnecessary kind label * update changelog (#1374) Remove BUGFIX entries that fix a bug introduced by main (i.e., bugs which were never part of a release) * Prepare for release of v0.23.0 (#1377) * Update version references * Fix fat-fingered delete; Remove mention of upgrade Go * RFC: Design in the open (#1055) * rfc: first draft of RFC0001 * add placeholder for PR * update PR link * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify "designing in the open" is best-effort * update 0001 * fix dead link in production/README.md * add recommended sections for RFC proposals * describe the process for approving a proposal * ignore RFC template in link checker * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * do my nitty 80-char line length limit change * indent pros/cons to a single section * document process for superseding RFCs * remove RFC mutability requirement * add extra flavor around not recommending google docs * require Google Doc -> RFC conversion * move new files Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Add Grafana Labs SECURITY.md (#1356) Signed-off-by: Richard Hartmann <richih@richih.org> * Add readiness check to metrics component (#1369) * PR Base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix autoscrape's mockInstance Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in atomic readiness check Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Add CHANGELOG.md entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Reference page to download windows installer (#1372) Fixes #1366 * fix typo in node_exporter_config (#1389) which should be `privileged` instead of `priviliged` * Add option for Operator to pass arguments to GrafanaAgent #1227 (#1248) * 1250 oauth2 tracing (#1386) * Add oauth support for trace Otel trace exporter via opentelemetry-collector-contrib oauth2clientauthextension * start extensions on collector instance startup fix decoding to otelconfig build extensions add oauth extension to service map * Update traces config documentation * lint fixes * fix godoc comments * pass exporter index directly to exporter name generator * PR feedback; Update Changelog * sort extensions when sorting pipelines for testing determinism * README: Fix link to agent logo (#1396) * update MAINTAINERS.md (#1402) * add smoke alerts to mixin; move local alerts into examples dir (#1397) * add smoke alerts to mixin; move local alerts into examples dir * add podPrefix for smoke test * podPrefix in libsonnet config * [RFC] Integrations in Grafana Agent Operator (#1224) * rfc: integrations in grafana agent operator Supersedes #883 * add missing links * Apply suggestions from code review Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify how many daemonsets/deployments/service/secrets are created * add example of defining secrets * try defining integrations * s/IntegrationsMonitor/IntegrationMonitor/g * simplify proposal * add alternatives * remove old reference to `hasMetrics` field * document example generated agent configuration file * assign ID RFC-0002 * add missing PR link Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * add fake rw endpoint to smoke program (#1405) * fix alerts typo (#1407) * continuous delivery for smoke images (#1408) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * fix continuous delivery job errors (#1409) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * [operator] - Use _file variants for basic auth credentials. (#1411) * use password_file alternatives in operator config * update tests * reduce smoke alert noise (#1412) * reduce smoke alert noise Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update production/grafana-agent-mixin/alerts.libsonnet Co-authored-by: Robert Fratto <robertfratto@gmail.com> * update cpu check comment Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * add minimum load threshold to cpu alert Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Clarify usage of instanceNamespaceSelector (#1413) * RFC-0001: Add status to RFC (#1391) * rfc-0001: add rules for when RFC PRs should be merged * use status field instead of merge to indicating state * Parametrize logs DaemonSet K8s manifests (#1420) * Parametrize logs daemonset K8s manifests * Update CHANGELOG.md * Extend linting configuration file (#1421) * Add depguard linter to reject packages we tend to avoid * Replace golint with revive, since golint is deprecated * Remove interfacer, which is deprecated with no replacement * Add makezero linter to detect misuse of make with append * Add tenv to prefer t.Setenv over os.Setenv in tests * Add whitespace to report unnecessary blank lines * Ignore test files for errcheck In addition to the above, the following changes were made: * Remove settings that just re-set default values, instead pointing to the website to retrieve defaults. * Simplify the errcheck rule to only include functions we actually need to ignore. * Merging again! Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: Patrick Koenig <pkoenig10@gmail.com> Co-authored-by: DataPoints <langer.markus@gmail.com> Co-authored-by: Alex <52292902+alexrudd2@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: melGL <81323402+melgl@users.noreply.github.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Joseph Woodward <josephwoodward@xeuse.com> Co-authored-by: hanif <hjet@users.noreply.github.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: laiwei <laiwei.ustc@gmail.com> Co-authored-by: Sam <shamsalmon@users.noreply.github.com> Co-authored-by: Chris Knutson <christopher.knutson@gmail.com> Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Craig Peterson <192540+captncraig@users.noreply.github.com>
mattdurham
added a commit
that referenced
this pull request
Feb 25, 2022
* Update node_exporter dependency to v1.3.1 (#1228) * Add node_exporter to depcheck * update weaveworks/common dependency * map current release flags and changed defaults * documentation * revert accidental checkin * print out flags when node_exporter test fails to assist debugging * oops, i introduced some flags from master by mistake * Introduce experimental integrations revamp (#1198) * [dev.multiple-integrations] Enable present integrations by default, deprecate enabled field (#1062) * integrations: default to enabled by default * document deprecation of enabled * pkg/integrations: support *_configs field for integrations (#1130) Creates the basic code to unmarshal integrations from a YAML field called <integration name>_configs, which is a slice of that integration. Note that this is NOT wired up to the integrations manager yet, and trying to run the agent with more than one integration of the same type will likely cause problems. * [dev.multiple-integrations] Prototype new integrations subsystem (#1142) * wip: prototype new integrations subsystem * implement Controller with basic logic for Integration and UpdateIntegration * Implement HTTPIntegration for Controller * decouple controller and subsystem * don't have controller implement integration slightly less smelly now * multiplexer integration * rely on boilerplate for multiplexing for now generics would be nice here * remove multiplex_integration.go Also a little code smelly. Instead of having integrations that run other integrations, I'm going to fall back to having only one controller. * introduce Subsystem, unexport Controller start wiring up things to Subsystem * introduce v2 agent integration to use for testing * start wiring metrics integrations * rename Options to Globals call a spade a spade * add subsystem options to globals * remove dead code * metricsutils: calculate self-scraping based on globals * complete HTTP target API * working example with agent integration * appease the linter * don't return an error when context to cancel an integration is closed * once again i am asking the linter to forgive my typos * fix bug where labels from individual targets were getting dropped at the API endpoint * pkg/config: fix broken test * finish unit tests for integrations v2 controller * metricsutil/metricshandler_integration: make job name unique Before this change, the job name would have collided when using multiple instances of the same integration. * ensure that global subsystem labels are injected into targets * integrations/v2: Infer target hostname from SD API host (#1175) * [dev.multiple-integrations] integrations/v2: allow shimming between v1 and v2 integrations. (#1179) * integrations/v2: allow shimming between v1 and v2 integrations. Shimming is done by changing how the integration registration works; a new RegisterDynamic was added that allows for creating Configs at runtime. Here be dragons; this should be removed whenever we no longer have a need for it. * fix lint * pkg/integrations/v2: use "RegisterLegacy" instead of a generic mechanism * fine, I won't add the deprecation notice if it will make the linter sad * pkg/integrations: re-align (#1181) This commit reverts 69ba2dd in favor of allowing the new subsystem to handle multiple instances of integrations. This commit also removes the wal_truncate_frequency field from integrations as it is the only field from old integrations that does not have a current counterpart. * [dev.multiple-integrations] Hide integrations/v2 behind a feature flag (#1185) * feature flag wip * dynamically switch between integrations v1 and v2 default to v1. * pkg/integrations/versionselector to file in pkg/config * pkg/config: fix defaults for Integrations * pkg/config: use more generic way to unmarshal differently based on flag * add missing godoc comment * more comments * switch to deferred unmarshaling * remove unused Config field * simplify completeUnmarshal * do not perform lazy deferred unmarshaling * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter * Merge main into dev.multiple-integrations (#1184) * Fix typo (#1141) * Traces: Improved pod association in PromSD processor (#1137) * Improve k8s pod association * Add tests * Changelog * typo * Add prom_sd_pod_association * Extend tests for pod associations * Docs for pod association config * Lint fixes * Move to unreleased * Add instrumentation recommendations * Remove uncessary constants * Improve tests * remote config with http(s) provider (#1143) * sample remote config code with http provider * use t.TempDir() in unit test * no need to clean up after T.TempDir() * use NewClientFromConfig and make caller responsible for calling SetDirectory * handle nil HTTPClientConfig * remove blank identifier assignment * pass basic auth command line flags for remote config * address pr nits * add expiremental flag * set loader inline * update changelog * add remote config section in docs * pr comment updates * announce patch releases for cve-2021-41090 (#1152) * Merge patch release to main (#1153) * Add secret type to sensitive values * Break out config tests to their own implementation. Also remove username has a sensitive value. * Update changelog * Fix failing test * Scrub secrets when marshaling instance configs * update for v0.21 * Updated changes from the merge. * Remove changelog * Scrub out receivers has ***receivers_scrubber***:null * obscure etcd/consul credentials * Update pkg/traces/config_test.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update pkg/config/config.go * go fmt * Change to using custom object and return <secret> * Fix bad merge * [v0.21.2] toggle config endpoint (#19) * disable /-/config endpoint by default * disable scraping api get endpoint as well * fix new test * add test and rename flag Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Update version to v0.21.2 * Update defaults.go * fix /-/config endpoint * also fix non-pointer config bug * temporarily disable linting for release * fix lint errors Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter (#1162) * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter A recent change broke the usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME for the postgres_exporter. As the incorrect variable was checked in the if clause, it always raises an error. * changelog: keep feature -> enhancement -> bugfix order * postgres_exporter: add regression test Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix syntax error in Jsonnet logs helper method (#1174) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * cAdvisor Integration (#1081) * Add cadvisor module * Begin creating common config for cadvisor * Don't export internal state * Finish config options for cadvisor * Set config options, and implement cAdvisor collectors * Linting * Buildflags for cadvisor only in linux * I R LEArN Build Tags * Don't zero value the zero value * Offload sketchy global var manipulation to the integrations Run func * Remove unused collectors * Lint * Create generic stub integration and use it for cadvisor * Lint * Final refactor of cAdvisor config for unsupported platforms. Pared down stub integrations. * Lint * Docs for cadvisor config * Update changelog * Update pkg/integrations/stub_integration.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Reorder changelog * Instance key clarity * Inclusive naming * Finish name changes Keep default disable metric list in sync with upstream Idiomatic golang * Hardcode disabled metrics for cadvisor Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Remove log-level flag from systemd unit file (#1177) * Upgrade to OTel v0.40.0 (#1176) * Upgrade to OTel v0.40.0 * Changelog * Add factories check * go mod tidy * config/features: create package to standardize experimental features (#1170) * config/features: create package to standardize experiemental features This commit introduces a new package, pkg/config/features, which allows defining a set of features and validating whether flags associated with those features are allowed to be set. Closes #1163 * update documentation (also s/enabled-features/enable-features) * Fix typo * Update pkg/config/features/features.go Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Revert "Merge main into dev.multiple-integrations (#1184)" (#1189) This reverts commit ad76ec5. * [dev.multiple-integrations] Revert breaking changes to existing integrations (#1191) * revert breaking changes to integrations v1 This commit reverts #1062 in favor of making breaking changes directly in integrations-next instead. The part of #1181 to remove `wal_truncate_frequency` has also been reverted. As part of this change, the enabled field is removed from the v2 common metrics configs, and v2 integrations can no longer be disabled. v2 integrations can only be disabled by removing them from the YAML. * integrations/v2: remove stale reference to ErrDisabled (fix typo too) * integrations/v2: bring in common config decoupling * [dev.multiple-integrations] Introduce autoscraper (#1195) * pkg/integrations/v2: introduce self-scraping * linting * [dev.multiple-integrations] Multiple instances of integrations (#1196) * multiple instances of integrations opt in relevant v1 integrations into supporting multiple instances * shims should check for instance key override * Document integrations-next (#1197) * document integrations-next * remove json tags since they make markdown unhappy * changelog * s/Run/RunIntegration * remove stale comment about integrations.controller purpose * create dedicated run method for instanceScraper * s/expoter/exporter/g * Document why an autoscrape.Scraper manages a set of per-instance scrapers * spell out prerequisite instead of pre-req * use go.uber.org/atomic to make the code a little easier to follow * remove started callback for running integration * use smaller interface for autoscrape Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Fix panic when using 'stdout' in automatic logging (#1233) * integrations-next: fix bug where v2 integrations were not being strictly unmarshaled (#1235) * Remove jsonnet vendor folders (#1222) * remove jsonnet vendor This adds all vendor folders into .gitignore and removes cached vendor files from the repository. Closes #1221 * Update scripts and instructions for jsonnet vendor removal * `make example-dashboards` will now also run `jb install` * k3d environment instructions now include `jb install` * smoke-test.bash will now run `jb install` prior to `tk apply` * Fix link to k3d example in DEVELOPERS.md (#1242) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix node_exporter upgrade docs (#1239) * Fix panic in automatic logging with stdout backend (#1243) * pkg/util: support custom yaml.Unmarshaler implementations for util.UnmarshalYAMLMerged (#1244) It's common for config types to have implement yaml.Unmarshaler for: * Applying defaults * Applying extra logic post-unmarshal If these config types were unmarshaled through util.UnmarshalYAMLMerged, the yaml.Unmarshaler implementation would never complete successfully, preventing the post-unmarshal logic from running. This issue was introduced in #1192, but went unnoticed until #1228 implemented yaml.Unmarshaler to perform field migrations. #1240 reported the issue. This commit fixes the bug by performing a second non-strict unmarshal to ensure that all input values unmarshal successfully, with the exception of unmarshal errors unrelated to unrecognized field names. This is hacky, but it's worthwhile noting that util.UnmarshalYAMLMerged is a temporary workaround needed for the integrations-next migration, and will eventually be removed. * Update k3d example grafana/grafonnet-lib version (#1246) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Create an e2e framework with support for running tests against k8s (#1234) * e2e: create an e2e framework with support for running tests against a k3d cluster * add new E2E drone job * E2E tests should pass when doing a release * sign drone.yml again * move e2e lint to different step that has golangci-lint installed * upgrade golangci-lint and go for e2e test * e2e: add gcc * E2E: install build-essential to get a working full gcc env * :( * e2e: support running from inside of docker * fix lint error * address review feedback * Operator: fix bug where /-/ready and /-/healthy always returned 404 (#1252) * operator: fix bug where /-/ready and /-/healthy always returned 404 controller-runtime must have at least one ready/healthy check for the endpoints to exist * fix lint error, use healthz.Ping * Make scraping-svc use the new `metrics:` key (#1259) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * update prometheus dependency (#1260) * corrected typo (#1265) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags (#1264) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use :main tag for images in smoke test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Set IMAGE_BRANCH_TAG env var in drone and actions pipelines Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove quotes from Makefile variable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove force_release action Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * prepare for v0.22.0 release (#1266) * prepare for v0.22.0 release * remove E2E pipeline * Add basic testing framework for operator (#1268) * remove dedicated go.mod for e2e/ * move e2e/k8s to pkg/util/k8s * Migrate operator tests to pkg/util/k8s * remove dedicated e2e tests * allow skipping TestCluster in pkg/util/k8s * remove e2e/ * fix bad merge * fix order of make env args for windows * actually declare referenced docker volume * introduce pkg/util/subset for asserting subset of objects * refactor operator so it's testable * define basic integration test for operator * fix lint errors * fix invalid address in operator test config * Update release-note.md (#1267) * Set scrape User-Agent header during init (#1274) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Upgrade to Go 1.17 (#1278) * Upgrade to 1.17.6 in go.mod and Dockerfiles * Update CHANGELOG.md to mention the update * Update Go version in drone/actions pipelines * Update go.mod, go.sum files via * Re-sign drone.yml * Remove leading newline causing drone build to fail * Bump golangci-lint image to a version using Go 1.17 * Re-attempt to solve linter issue with new golangci-lint image * Remove suffix of exclude rules * Clean previous Go version before unpacking Go 1.17 * Also clean up previous Go versions in other steps * fix typo (#1284) * Use custom Go version in agent-operator Dockerfile (#1286) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * pkg/operator: refactor resource hierarchy discovery (#1271) * pkg/operator: refactor resource hierarchy discovery This commit moves common logic related to discovering the resource hierarchy to pkg/operator/hierarchy. This new package requires less boilerplate, which the reconciler is updated to take advantage of. * remove unused code * test construction of resource hierarchy * add missing build constraints * small extra cleanup to use pointer package * review feedback * update agent-build-image for go 1.17 (#1287) (also use a consistent base image tag instead of latest) * Skip non-ready entries when listing instances (#1289) * Skip non-ready instances in LoadInstances() Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix panic in prom_sd_processor when address is empty (#1279) * Fix panic in prom_sd_processor when address is empty * Fix panic in prom_sd_processor when address is empty * Fix docs * Add test case * Lint * Move to unreleased * Operator: generate proxy_url for remote_write (#1298) * operator: generate proxy_url for remote_write * fix weird indentation in test * Use log format in traces subsystem (#1272) * Use log format in traces subsystem * Changelog * Undo unwanted change * Fix changelog entry * integrations-next: Add extra_labels to inject extra labels for an integration (#1312) * integrations-next: Add extra_labels to inject extra labels for an integration. * separate tests * Fix anchor link on operator docs (#1302) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * updated config URL (#1304) The existing URL returns a 404: https://grafana.com/docs/agent/latest/getting-started/configuration/_index.md Updated to https://grafana.com/docs/agent/latest/configuration/ * Fix typo in node_exporter (#1325) * Allow remote_write URL credentials (#1329) * Bypass Prometheus password redaction Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add inline secret in existing test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry * Add to scrubbed testcase as well Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Stop appending duplicate exemplars (#1316) * Add memExemplar in stripeSeries as first iteration Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add test for skipped duplicate exemplars; Simplify conditional Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry; discard test errors * Move changelog entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add Benchmark for AppendExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Discard error on added benchmark Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use original exemplar struct instead of custom memExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Surround benchmark loop with start/stop timers and close test storage Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add comment about prepopulating exemplars on WAL startup Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in the totalAppendedExemplars metric Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make comment more discoverable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make sure we're recording exemplars for non-nil series ref only Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * integrations-next: wait for integrations to exit after stopping them (#1318) * integrations-next: wait for integrations to exit after stopping them * fix lint errors * minor refactor * integrations-next: stop holding config mutex for entire reload * make controller.run authoritative over running integrations * fix log line * move running integrations into a dedicated worker pool * operator/hierarchy: stop using field selector when listing Secrets & ConfigMaps (#1340) The initial implementation of hierarchy.KeySelector injected a FieldSelector when listing Secrets and ConfigMaps to immediately return the single object being queried for. This causes a problem with the client generated by the controller-runtime framework, where the client is wrapped in a cache and field indexer (where only the namespace is indexed by default). This commit avoids using the field selector and the index lookup. The resulting behavior aligns more closely with discovering other resources in the hierarchy (i.e., ServiceMonitors), where the List call is also insufficient and needs post-processing via Matches to find the final list of resources. Given the controller-runtime client uses an informer for reads, all relevant Secrets and ConfigMaps are already in-memory anyway, and using the index for a faster List is a bit of an over-optimization at the moment. * Add dependabot to update go modules and github actions. (#1217) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * smoke framework refactor (#1326) * Agent smoke test (#1291) * convert smoke script to go program * update build for agent-smoke * fix pr comments * use existing log helper package * refactor context cancel * update exit codes * use ticker * prefer oklog/run instead of errgroup * use nop logger * refactor task interface * remove functional options * log.With for task loggers * move smoke to tools * build smoke image, push to internal registry * move crow to tools * add gcr_admin secret * fix link to crow * add smoke libsonnet and use in local k3d smoke test * add deletePodBySelectorTask * scale smoke-test replica down after local test * refactor smoke Options to Config * update duration usage message * add some basic unit tests * newlines * pass mutation frequency and chaos frequency from smoke script * pull crow image from gcr * update smoke script * move monitoring to smoke libsonnet * move additional smoke resources needed in deployment tools * reference libsonnet files from grafana-agent dep * make drone * fix images in smoke script * get rid of extVars * update k3d example environment to reference etcd from new location * update smoke docker builds to use go1.17 * use pointer.Int64 * refactor smoke jsonnet (#1296) * add policy rule for list and delete pods (#1319) * refactor smoke.new function to take config object (#1327) * Apply suggestions from code review * Update production/tanka/grafana-agent/smoke/crow/main.libsonnet * Update production/tanka/grafana-agent/smoke/main.libsonnet * Update example/k3d/scripts/smoke-test.bash Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * readme update (#1338) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Correct link to the configuration (#1036) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Add stale check Github Action (#1345) * Add a stale check GH action to run every 24 hours * remove old stale.yml file * add permissions to action * update the stale message to clarify when the stale label will get removed * Update .github/workflows/stale.yml * stale action: fix missing indent (#1346) * Fix mssql issue (#1351) * Add K8s Events integration (#1330) * Add K8s eventhandler integration (#1310) * Add docs and sample manifests to eventhandler integration (#1328) * Wait for cache to flush before returning * Clarify eventhandler docs (#1334) * Clarify docs * Update CHANGELOG.md * Review changes (#1349) * stale action: fix typo in label exemptions (#1347) * update withVolumesMixin for agent jsonnet (#1358) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Configure cluster label using logs client external_labels param (#1357) * Configure cluster label using logs client external_labels param * Update CHANGELOG.md * add password file and basic auth round tripper in crow (#1361) * add password file and basic auth round tripper in crow * add ca-certificates in crow image * add orgID flag * update help text * default send_exemplars to true in remote_write (#1352) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update eventhandler labels (#1368) * Update eventhandler integration labels * Update CHANGELOG * Remove unnecessary kind label * update changelog (#1374) Remove BUGFIX entries that fix a bug introduced by main (i.e., bugs which were never part of a release) * Prepare for release of v0.23.0 (#1377) * Update version references * Fix fat-fingered delete; Remove mention of upgrade Go * RFC: Design in the open (#1055) * rfc: first draft of RFC0001 * add placeholder for PR * update PR link * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify "designing in the open" is best-effort * update 0001 * fix dead link in production/README.md * add recommended sections for RFC proposals * describe the process for approving a proposal * ignore RFC template in link checker * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * do my nitty 80-char line length limit change * indent pros/cons to a single section * document process for superseding RFCs * remove RFC mutability requirement * add extra flavor around not recommending google docs * require Google Doc -> RFC conversion * move new files Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Add Grafana Labs SECURITY.md (#1356) Signed-off-by: Richard Hartmann <richih@richih.org> * Add readiness check to metrics component (#1369) * PR Base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix autoscrape's mockInstance Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in atomic readiness check Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Add CHANGELOG.md entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Reference page to download windows installer (#1372) Fixes #1366 * fix typo in node_exporter_config (#1389) which should be `privileged` instead of `priviliged` * Add option for Operator to pass arguments to GrafanaAgent #1227 (#1248) * 1250 oauth2 tracing (#1386) * Add oauth support for trace Otel trace exporter via opentelemetry-collector-contrib oauth2clientauthextension * start extensions on collector instance startup fix decoding to otelconfig build extensions add oauth extension to service map * Update traces config documentation * lint fixes * fix godoc comments * pass exporter index directly to exporter name generator * PR feedback; Update Changelog * sort extensions when sorting pipelines for testing determinism * README: Fix link to agent logo (#1396) * update MAINTAINERS.md (#1402) * add smoke alerts to mixin; move local alerts into examples dir (#1397) * add smoke alerts to mixin; move local alerts into examples dir * add podPrefix for smoke test * podPrefix in libsonnet config * [RFC] Integrations in Grafana Agent Operator (#1224) * rfc: integrations in grafana agent operator Supersedes #883 * add missing links * Apply suggestions from code review Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify how many daemonsets/deployments/service/secrets are created * add example of defining secrets * try defining integrations * s/IntegrationsMonitor/IntegrationMonitor/g * simplify proposal * add alternatives * remove old reference to `hasMetrics` field * document example generated agent configuration file * assign ID RFC-0002 * add missing PR link Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * add fake rw endpoint to smoke program (#1405) * fix alerts typo (#1407) * continuous delivery for smoke images (#1408) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * fix continuous delivery job errors (#1409) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * [operator] - Use _file variants for basic auth credentials. (#1411) * use password_file alternatives in operator config * update tests * reduce smoke alert noise (#1412) * reduce smoke alert noise Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update production/grafana-agent-mixin/alerts.libsonnet Co-authored-by: Robert Fratto <robertfratto@gmail.com> * update cpu check comment Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * add minimum load threshold to cpu alert Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Clarify usage of instanceNamespaceSelector (#1413) * RFC-0001: Add status to RFC (#1391) * rfc-0001: add rules for when RFC PRs should be merged * use status field instead of merge to indicating state * Parametrize logs DaemonSet K8s manifests (#1420) * Parametrize logs daemonset K8s manifests * Update CHANGELOG.md * Extend linting configuration file (#1421) * Add depguard linter to reject packages we tend to avoid * Replace golint with revive, since golint is deprecated * Remove interfacer, which is deprecated with no replacement * Add makezero linter to detect misuse of make with append * Add tenv to prefer t.Setenv over os.Setenv in tests * Add whitespace to report unnecessary blank lines * Ignore test files for errcheck In addition to the above, the following changes were made: * Remove settings that just re-set default values, instead pointing to the website to retrieve defaults. * Simplify the errcheck rule to only include functions we actually need to ignore. * Merging again! Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: Patrick Koenig <pkoenig10@gmail.com> Co-authored-by: DataPoints <langer.markus@gmail.com> Co-authored-by: Alex <52292902+alexrudd2@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: melGL <81323402+melgl@users.noreply.github.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Joseph Woodward <josephwoodward@xeuse.com> Co-authored-by: hanif <hjet@users.noreply.github.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: laiwei <laiwei.ustc@gmail.com> Co-authored-by: Sam <shamsalmon@users.noreply.github.com> Co-authored-by: Chris Knutson <christopher.knutson@gmail.com> Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Craig Peterson <192540+captncraig@users.noreply.github.com>
mattdurham
added a commit
that referenced
this pull request
Feb 25, 2022
* Update node_exporter dependency to v1.3.1 (#1228) * Add node_exporter to depcheck * update weaveworks/common dependency * map current release flags and changed defaults * documentation * revert accidental checkin * print out flags when node_exporter test fails to assist debugging * oops, i introduced some flags from master by mistake * Introduce experimental integrations revamp (#1198) * [dev.multiple-integrations] Enable present integrations by default, deprecate enabled field (#1062) * integrations: default to enabled by default * document deprecation of enabled * pkg/integrations: support *_configs field for integrations (#1130) Creates the basic code to unmarshal integrations from a YAML field called <integration name>_configs, which is a slice of that integration. Note that this is NOT wired up to the integrations manager yet, and trying to run the agent with more than one integration of the same type will likely cause problems. * [dev.multiple-integrations] Prototype new integrations subsystem (#1142) * wip: prototype new integrations subsystem * implement Controller with basic logic for Integration and UpdateIntegration * Implement HTTPIntegration for Controller * decouple controller and subsystem * don't have controller implement integration slightly less smelly now * multiplexer integration * rely on boilerplate for multiplexing for now generics would be nice here * remove multiplex_integration.go Also a little code smelly. Instead of having integrations that run other integrations, I'm going to fall back to having only one controller. * introduce Subsystem, unexport Controller start wiring up things to Subsystem * introduce v2 agent integration to use for testing * start wiring metrics integrations * rename Options to Globals call a spade a spade * add subsystem options to globals * remove dead code * metricsutils: calculate self-scraping based on globals * complete HTTP target API * working example with agent integration * appease the linter * don't return an error when context to cancel an integration is closed * once again i am asking the linter to forgive my typos * fix bug where labels from individual targets were getting dropped at the API endpoint * pkg/config: fix broken test * finish unit tests for integrations v2 controller * metricsutil/metricshandler_integration: make job name unique Before this change, the job name would have collided when using multiple instances of the same integration. * ensure that global subsystem labels are injected into targets * integrations/v2: Infer target hostname from SD API host (#1175) * [dev.multiple-integrations] integrations/v2: allow shimming between v1 and v2 integrations. (#1179) * integrations/v2: allow shimming between v1 and v2 integrations. Shimming is done by changing how the integration registration works; a new RegisterDynamic was added that allows for creating Configs at runtime. Here be dragons; this should be removed whenever we no longer have a need for it. * fix lint * pkg/integrations/v2: use "RegisterLegacy" instead of a generic mechanism * fine, I won't add the deprecation notice if it will make the linter sad * pkg/integrations: re-align (#1181) This commit reverts 69ba2dd in favor of allowing the new subsystem to handle multiple instances of integrations. This commit also removes the wal_truncate_frequency field from integrations as it is the only field from old integrations that does not have a current counterpart. * [dev.multiple-integrations] Hide integrations/v2 behind a feature flag (#1185) * feature flag wip * dynamically switch between integrations v1 and v2 default to v1. * pkg/integrations/versionselector to file in pkg/config * pkg/config: fix defaults for Integrations * pkg/config: use more generic way to unmarshal differently based on flag * add missing godoc comment * more comments * switch to deferred unmarshaling * remove unused Config field * simplify completeUnmarshal * do not perform lazy deferred unmarshaling * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter * Merge main into dev.multiple-integrations (#1184) * Fix typo (#1141) * Traces: Improved pod association in PromSD processor (#1137) * Improve k8s pod association * Add tests * Changelog * typo * Add prom_sd_pod_association * Extend tests for pod associations * Docs for pod association config * Lint fixes * Move to unreleased * Add instrumentation recommendations * Remove uncessary constants * Improve tests * remote config with http(s) provider (#1143) * sample remote config code with http provider * use t.TempDir() in unit test * no need to clean up after T.TempDir() * use NewClientFromConfig and make caller responsible for calling SetDirectory * handle nil HTTPClientConfig * remove blank identifier assignment * pass basic auth command line flags for remote config * address pr nits * add expiremental flag * set loader inline * update changelog * add remote config section in docs * pr comment updates * announce patch releases for cve-2021-41090 (#1152) * Merge patch release to main (#1153) * Add secret type to sensitive values * Break out config tests to their own implementation. Also remove username has a sensitive value. * Update changelog * Fix failing test * Scrub secrets when marshaling instance configs * update for v0.21 * Updated changes from the merge. * Remove changelog * Scrub out receivers has ***receivers_scrubber***:null * obscure etcd/consul credentials * Update pkg/traces/config_test.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Update pkg/config/config.go * go fmt * Change to using custom object and return <secret> * Fix bad merge * [v0.21.2] toggle config endpoint (#19) * disable /-/config endpoint by default * disable scraping api get endpoint as well * fix new test * add test and rename flag Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Update version to v0.21.2 * Update defaults.go * fix /-/config endpoint * also fix non-pointer config bug * temporarily disable linting for release * fix lint errors Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter (#1162) * Fix POSTGRES_EXPORTER_DATA_SOURCE_NAME usage for postgres_exporter A recent change broke the usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME for the postgres_exporter. As the incorrect variable was checked in the if clause, it always raises an error. * changelog: keep feature -> enhancement -> bugfix order * postgres_exporter: add regression test Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix syntax error in Jsonnet logs helper method (#1174) Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * cAdvisor Integration (#1081) * Add cadvisor module * Begin creating common config for cadvisor * Don't export internal state * Finish config options for cadvisor * Set config options, and implement cAdvisor collectors * Linting * Buildflags for cadvisor only in linux * I R LEArN Build Tags * Don't zero value the zero value * Offload sketchy global var manipulation to the integrations Run func * Remove unused collectors * Lint * Create generic stub integration and use it for cadvisor * Lint * Final refactor of cAdvisor config for unsupported platforms. Pared down stub integrations. * Lint * Docs for cadvisor config * Update changelog * Update pkg/integrations/stub_integration.go Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Reorder changelog * Instance key clarity * Inclusive naming * Finish name changes Keep default disable metric list in sync with upstream Idiomatic golang * Hardcode disabled metrics for cadvisor Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Remove log-level flag from systemd unit file (#1177) * Upgrade to OTel v0.40.0 (#1176) * Upgrade to OTel v0.40.0 * Changelog * Add factories check * go mod tidy * config/features: create package to standardize experimental features (#1170) * config/features: create package to standardize experiemental features This commit introduces a new package, pkg/config/features, which allows defining a set of features and validating whether flags associated with those features are allowed to be set. Closes #1163 * update documentation (also s/enabled-features/enable-features) * Fix typo * Update pkg/config/features/features.go Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * enable cadvisor by default * switch to using real feature flag * fix postgres_exporter Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Revert "Merge main into dev.multiple-integrations (#1184)" (#1189) This reverts commit ad76ec5. * [dev.multiple-integrations] Revert breaking changes to existing integrations (#1191) * revert breaking changes to integrations v1 This commit reverts #1062 in favor of making breaking changes directly in integrations-next instead. The part of #1181 to remove `wal_truncate_frequency` has also been reverted. As part of this change, the enabled field is removed from the v2 common metrics configs, and v2 integrations can no longer be disabled. v2 integrations can only be disabled by removing them from the YAML. * integrations/v2: remove stale reference to ErrDisabled (fix typo too) * integrations/v2: bring in common config decoupling * [dev.multiple-integrations] Introduce autoscraper (#1195) * pkg/integrations/v2: introduce self-scraping * linting * [dev.multiple-integrations] Multiple instances of integrations (#1196) * multiple instances of integrations opt in relevant v1 integrations into supporting multiple instances * shims should check for instance key override * Document integrations-next (#1197) * document integrations-next * remove json tags since they make markdown unhappy * changelog * s/Run/RunIntegration * remove stale comment about integrations.controller purpose * create dedicated run method for instanceScraper * s/expoter/exporter/g * Document why an autoscrape.Scraper manages a set of per-instance scrapers * spell out prerequisite instead of pre-req * use go.uber.org/atomic to make the code a little easier to follow * remove started callback for running integration * use smaller interface for autoscrape Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: Matt Durham <mattdurham@ppog.org> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Fix panic when using 'stdout' in automatic logging (#1233) * integrations-next: fix bug where v2 integrations were not being strictly unmarshaled (#1235) * Remove jsonnet vendor folders (#1222) * remove jsonnet vendor This adds all vendor folders into .gitignore and removes cached vendor files from the repository. Closes #1221 * Update scripts and instructions for jsonnet vendor removal * `make example-dashboards` will now also run `jb install` * k3d environment instructions now include `jb install` * smoke-test.bash will now run `jb install` prior to `tk apply` * Fix link to k3d example in DEVELOPERS.md (#1242) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix node_exporter upgrade docs (#1239) * Fix panic in automatic logging with stdout backend (#1243) * pkg/util: support custom yaml.Unmarshaler implementations for util.UnmarshalYAMLMerged (#1244) It's common for config types to have implement yaml.Unmarshaler for: * Applying defaults * Applying extra logic post-unmarshal If these config types were unmarshaled through util.UnmarshalYAMLMerged, the yaml.Unmarshaler implementation would never complete successfully, preventing the post-unmarshal logic from running. This issue was introduced in #1192, but went unnoticed until #1228 implemented yaml.Unmarshaler to perform field migrations. #1240 reported the issue. This commit fixes the bug by performing a second non-strict unmarshal to ensure that all input values unmarshal successfully, with the exception of unmarshal errors unrelated to unrecognized field names. This is hacky, but it's worthwhile noting that util.UnmarshalYAMLMerged is a temporary workaround needed for the integrations-next migration, and will eventually be removed. * Update k3d example grafana/grafonnet-lib version (#1246) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Create an e2e framework with support for running tests against k8s (#1234) * e2e: create an e2e framework with support for running tests against a k3d cluster * add new E2E drone job * E2E tests should pass when doing a release * sign drone.yml again * move e2e lint to different step that has golangci-lint installed * upgrade golangci-lint and go for e2e test * e2e: add gcc * E2E: install build-essential to get a working full gcc env * :( * e2e: support running from inside of docker * fix lint error * address review feedback * Operator: fix bug where /-/ready and /-/healthy always returned 404 (#1252) * operator: fix bug where /-/ready and /-/healthy always returned 404 controller-runtime must have at least one ready/healthy check for the endpoints to exist * fix lint error, use healthz.Ping * Make scraping-svc use the new `metrics:` key (#1259) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * update prometheus dependency (#1260) * corrected typo (#1265) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags (#1264) * Use RELEASE_TAG to choose between `:main` and `:latest` docker tags Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use :main tag for images in smoke test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Set IMAGE_BRANCH_TAG env var in drone and actions pipelines Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove quotes from Makefile variable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Remove force_release action Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * prepare for v0.22.0 release (#1266) * prepare for v0.22.0 release * remove E2E pipeline * Add basic testing framework for operator (#1268) * remove dedicated go.mod for e2e/ * move e2e/k8s to pkg/util/k8s * Migrate operator tests to pkg/util/k8s * remove dedicated e2e tests * allow skipping TestCluster in pkg/util/k8s * remove e2e/ * fix bad merge * fix order of make env args for windows * actually declare referenced docker volume * introduce pkg/util/subset for asserting subset of objects * refactor operator so it's testable * define basic integration test for operator * fix lint errors * fix invalid address in operator test config * Update release-note.md (#1267) * Set scrape User-Agent header during init (#1274) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Upgrade to Go 1.17 (#1278) * Upgrade to 1.17.6 in go.mod and Dockerfiles * Update CHANGELOG.md to mention the update * Update Go version in drone/actions pipelines * Update go.mod, go.sum files via * Re-sign drone.yml * Remove leading newline causing drone build to fail * Bump golangci-lint image to a version using Go 1.17 * Re-attempt to solve linter issue with new golangci-lint image * Remove suffix of exclude rules * Clean previous Go version before unpacking Go 1.17 * Also clean up previous Go versions in other steps * fix typo (#1284) * Use custom Go version in agent-operator Dockerfile (#1286) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * pkg/operator: refactor resource hierarchy discovery (#1271) * pkg/operator: refactor resource hierarchy discovery This commit moves common logic related to discovering the resource hierarchy to pkg/operator/hierarchy. This new package requires less boilerplate, which the reconciler is updated to take advantage of. * remove unused code * test construction of resource hierarchy * add missing build constraints * small extra cleanup to use pointer package * review feedback * update agent-build-image for go 1.17 (#1287) (also use a consistent base image tag instead of latest) * Skip non-ready entries when listing instances (#1289) * Skip non-ready instances in LoadInstances() Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Fix panic in prom_sd_processor when address is empty (#1279) * Fix panic in prom_sd_processor when address is empty * Fix panic in prom_sd_processor when address is empty * Fix docs * Add test case * Lint * Move to unreleased * Operator: generate proxy_url for remote_write (#1298) * operator: generate proxy_url for remote_write * fix weird indentation in test * Use log format in traces subsystem (#1272) * Use log format in traces subsystem * Changelog * Undo unwanted change * Fix changelog entry * integrations-next: Add extra_labels to inject extra labels for an integration (#1312) * integrations-next: Add extra_labels to inject extra labels for an integration. * separate tests * Fix anchor link on operator docs (#1302) Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * updated config URL (#1304) The existing URL returns a 404: https://grafana.com/docs/agent/latest/getting-started/configuration/_index.md Updated to https://grafana.com/docs/agent/latest/configuration/ * Fix typo in node_exporter (#1325) * Allow remote_write URL credentials (#1329) * Bypass Prometheus password redaction Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add inline secret in existing test Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry * Add to scrubbed testcase as well Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Stop appending duplicate exemplars (#1316) * Add memExemplar in stripeSeries as first iteration Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add test for skipped duplicate exemplars; Simplify conditional Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add changelog entry; discard test errors * Move changelog entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add Benchmark for AppendExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Discard error on added benchmark Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Use original exemplar struct instead of custom memExemplar Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Surround benchmark loop with start/stop timers and close test storage Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Add comment about prepopulating exemplars on WAL startup Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in the totalAppendedExemplars metric Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make comment more discoverable Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Make sure we're recording exemplars for non-nil series ref only Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * integrations-next: wait for integrations to exit after stopping them (#1318) * integrations-next: wait for integrations to exit after stopping them * fix lint errors * minor refactor * integrations-next: stop holding config mutex for entire reload * make controller.run authoritative over running integrations * fix log line * move running integrations into a dedicated worker pool * operator/hierarchy: stop using field selector when listing Secrets & ConfigMaps (#1340) The initial implementation of hierarchy.KeySelector injected a FieldSelector when listing Secrets and ConfigMaps to immediately return the single object being queried for. This causes a problem with the client generated by the controller-runtime framework, where the client is wrapped in a cache and field indexer (where only the namespace is indexed by default). This commit avoids using the field selector and the index lookup. The resulting behavior aligns more closely with discovering other resources in the hierarchy (i.e., ServiceMonitors), where the List call is also insufficient and needs post-processing via Matches to find the final list of resources. Given the controller-runtime client uses an informer for reads, all relevant Secrets and ConfigMaps are already in-memory anyway, and using the index for a faster List is a bit of an over-optimization at the moment. * Add dependabot to update go modules and github actions. (#1217) Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com> * smoke framework refactor (#1326) * Agent smoke test (#1291) * convert smoke script to go program * update build for agent-smoke * fix pr comments * use existing log helper package * refactor context cancel * update exit codes * use ticker * prefer oklog/run instead of errgroup * use nop logger * refactor task interface * remove functional options * log.With for task loggers * move smoke to tools * build smoke image, push to internal registry * move crow to tools * add gcr_admin secret * fix link to crow * add smoke libsonnet and use in local k3d smoke test * add deletePodBySelectorTask * scale smoke-test replica down after local test * refactor smoke Options to Config * update duration usage message * add some basic unit tests * newlines * pass mutation frequency and chaos frequency from smoke script * pull crow image from gcr * update smoke script * move monitoring to smoke libsonnet * move additional smoke resources needed in deployment tools * reference libsonnet files from grafana-agent dep * make drone * fix images in smoke script * get rid of extVars * update k3d example environment to reference etcd from new location * update smoke docker builds to use go1.17 * use pointer.Int64 * refactor smoke jsonnet (#1296) * add policy rule for list and delete pods (#1319) * refactor smoke.new function to take config object (#1327) * Apply suggestions from code review * Update production/tanka/grafana-agent/smoke/crow/main.libsonnet * Update production/tanka/grafana-agent/smoke/main.libsonnet * Update example/k3d/scripts/smoke-test.bash Co-authored-by: Robert Lankford <robert.lankford@grafana.com> * readme update (#1338) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Correct link to the configuration (#1036) Co-authored-by: Robert Fratto <robert.fratto@grafana.com> * Add stale check Github Action (#1345) * Add a stale check GH action to run every 24 hours * remove old stale.yml file * add permissions to action * update the stale message to clarify when the stale label will get removed * Update .github/workflows/stale.yml * stale action: fix missing indent (#1346) * Fix mssql issue (#1351) * Add K8s Events integration (#1330) * Add K8s eventhandler integration (#1310) * Add docs and sample manifests to eventhandler integration (#1328) * Wait for cache to flush before returning * Clarify eventhandler docs (#1334) * Clarify docs * Update CHANGELOG.md * Review changes (#1349) * stale action: fix typo in label exemptions (#1347) * update withVolumesMixin for agent jsonnet (#1358) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Configure cluster label using logs client external_labels param (#1357) * Configure cluster label using logs client external_labels param * Update CHANGELOG.md * add password file and basic auth round tripper in crow (#1361) * add password file and basic auth round tripper in crow * add ca-certificates in crow image * add orgID flag * update help text * default send_exemplars to true in remote_write (#1352) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update eventhandler labels (#1368) * Update eventhandler integration labels * Update CHANGELOG * Remove unnecessary kind label * update changelog (#1374) Remove BUGFIX entries that fix a bug introduced by main (i.e., bugs which were never part of a release) * Prepare for release of v0.23.0 (#1377) * Update version references * Fix fat-fingered delete; Remove mention of upgrade Go * RFC: Design in the open (#1055) * rfc: first draft of RFC0001 * add placeholder for PR * update PR link * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify "designing in the open" is best-effort * update 0001 * fix dead link in production/README.md * add recommended sections for RFC proposals * describe the process for approving a proposal * ignore RFC template in link checker * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Update docs/rfcs/0001-designing-in-the-open.md Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * do my nitty 80-char line length limit change * indent pros/cons to a single section * document process for superseding RFCs * remove RFC mutability requirement * add extra flavor around not recommending google docs * require Google Doc -> RFC conversion * move new files Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> * Add Grafana Labs SECURITY.md (#1356) Signed-off-by: Richard Hartmann <richih@richih.org> * Add readiness check to metrics component (#1369) * PR Base Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Fix autoscrape's mockInstance Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> * Wire in atomic readiness check Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Add CHANGELOG.md entry Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Reference page to download windows installer (#1372) Fixes #1366 * fix typo in node_exporter_config (#1389) which should be `privileged` instead of `priviliged` * Add option for Operator to pass arguments to GrafanaAgent #1227 (#1248) * 1250 oauth2 tracing (#1386) * Add oauth support for trace Otel trace exporter via opentelemetry-collector-contrib oauth2clientauthextension * start extensions on collector instance startup fix decoding to otelconfig build extensions add oauth extension to service map * Update traces config documentation * lint fixes * fix godoc comments * pass exporter index directly to exporter name generator * PR feedback; Update Changelog * sort extensions when sorting pipelines for testing determinism * README: Fix link to agent logo (#1396) * update MAINTAINERS.md (#1402) * add smoke alerts to mixin; move local alerts into examples dir (#1397) * add smoke alerts to mixin; move local alerts into examples dir * add podPrefix for smoke test * podPrefix in libsonnet config * [RFC] Integrations in Grafana Agent Operator (#1224) * rfc: integrations in grafana agent operator Supersedes #883 * add missing links * Apply suggestions from code review Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * clarify how many daemonsets/deployments/service/secrets are created * add example of defining secrets * try defining integrations * s/IntegrationsMonitor/IntegrationMonitor/g * simplify proposal * add alternatives * remove old reference to `hasMetrics` field * document example generated agent configuration file * assign ID RFC-0002 * add missing PR link Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> * add fake rw endpoint to smoke program (#1405) * fix alerts typo (#1407) * continuous delivery for smoke images (#1408) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * fix continuous delivery job errors (#1409) Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * [operator] - Use _file variants for basic auth credentials. (#1411) * use password_file alternatives in operator config * update tests * reduce smoke alert noise (#1412) * reduce smoke alert noise Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * Update production/grafana-agent-mixin/alerts.libsonnet Co-authored-by: Robert Fratto <robertfratto@gmail.com> * update cpu check comment Signed-off-by: Robbie Lankford <robert.lankford@grafana.com> * add minimum load threshold to cpu alert Co-authored-by: Robert Fratto <robertfratto@gmail.com> * Clarify usage of instanceNamespaceSelector (#1413) * RFC-0001: Add status to RFC (#1391) * rfc-0001: add rules for when RFC PRs should be merged * use status field instead of merge to indicating state * Parametrize logs DaemonSet K8s manifests (#1420) * Parametrize logs daemonset K8s manifests * Update CHANGELOG.md * Extend linting configuration file (#1421) * Add depguard linter to reject packages we tend to avoid * Replace golint with revive, since golint is deprecated * Remove interfacer, which is deprecated with no replacement * Add makezero linter to detect misuse of make with append * Add tenv to prefer t.Setenv over os.Setenv in tests * Add whitespace to report unnecessary blank lines * Ignore test files for errcheck In addition to the above, the following changes were made: * Remove settings that just re-set default values, instead pointing to the website to retrieve defaults. * Simplify the errcheck rule to only include functions we actually need to ignore. Co-authored-by: Robert Fratto <robert.fratto@grafana.com> Co-authored-by: Ursula Kallio <73951760+osg-grafana@users.noreply.github.com> Co-authored-by: Mario <mariorvinas@gmail.com> Co-authored-by: Robert Lankford <robert.lankford@grafana.com> Co-authored-by: f11r <fiete.gruenter@rwth-aachen.de> Co-authored-by: f11r <f11r@users.noreply.github.com> Co-authored-by: Nick Pillitteri <56quarters@users.noreply.github.com> Co-authored-by: Ryan Geyer <me@ryangeyer.com> Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de> Co-authored-by: Robert Lankford <rlankfo@gmail.com> Co-authored-by: Paschalis Tsilias <tpaschalis@users.noreply.github.com> Co-authored-by: Patrick Koenig <pkoenig10@gmail.com> Co-authored-by: DataPoints <langer.markus@gmail.com> Co-authored-by: Alex <52292902+alexrudd2@users.noreply.github.com> Co-authored-by: Robert Fratto <robertfratto@gmail.com> Co-authored-by: melGL <81323402+melgl@users.noreply.github.com> Co-authored-by: Tom Wilkie <tomwilkie@users.noreply.github.com> Co-authored-by: Joseph Woodward <josephwoodward@xeuse.com> Co-authored-by: hanif <hjet@users.noreply.github.com> Co-authored-by: Richard Hartmann <RichiH@users.noreply.github.com> Co-authored-by: laiwei <laiwei.ustc@gmail.com> Co-authored-by: Sam <shamsalmon@users.noreply.github.com> Co-authored-by: Chris Knutson <christopher.knutson@gmail.com> Co-authored-by: Florian Klink <flokli@flokli.de> Co-authored-by: Craig Peterson <192540+captncraig@users.noreply.github.com>
github-actions
Bot
added
the
frozen-due-to-age
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
Adds Args CRD to agent. Mostly for new config read endpoint options (disabled by default).
Which issue(s) this PR fixes
#1227
Notes to the Reviewer
Can be enabled like so:
PR Checklist