Skip to content
This repository was archived by the owner on Jul 28, 2026. It is now read-only.

Dynamic integration registration system - #278

Merged
rfratto merged 5 commits into
grafana-cold-storage:masterfrom
rfratto:dynamic-integrations
Jan 5, 2021
Merged

Dynamic integration registration system#278
rfratto merged 5 commits into
grafana-cold-storage:masterfrom
rfratto:dynamic-integrations

Conversation

@rfratto

@rfratto rfratto commented Dec 8, 2020

Copy link
Copy Markdown
Contributor

PR Description

This PR adds a dynamic integration registration system similar to (and mostly copied from) Prometheus' SD registration mechanism.

Which issue(s) this PR fixes

Notes to the Reviewer

Packages have been re-arranged since integrations can now depend on the base integrations package without a cyclic dependency.

PR Checklist

  • CHANGELOG updated (internal change, not needed)
  • Documentation added (internal change, not needed)
  • Tests updated

Comment thread pkg/integrations/register.go Outdated
// Prometheus:
//
// https://github.com/prometheus/prometheus/blob/511511324adfc4f4178f064cc104c2deac3335de/discovery/registry.go#L111
func unmarshalIntegrationsWithList(integrations []IntegrationConfig, out interface{}, unmarshal func(interface{}) error) error {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function makes me sad but there's really no other way to handle this. We need strict unmarshaling to still work, so we have to do what prometheus does: create a merged struct of the target + all of the possible integration fields, copy defaults to the merged struct, unmarshal to the merged struct, copy original fields back to the output, and then fill in the integrations slice.

I'll try to document this more because my original look at Prometheus' code had me really confused about what was going on and why.

@@ -1,23 +0,0 @@
package build

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A go mod vendor got rid of this.

This commit adds a dynamic integration registration system similar to
(and mostly copied from) Prometheus' SD registration mechanism.
1. Integration interface moved to integrations
2. Integration Config interface now responsible for providing
   name/config
@rfratto
rfratto force-pushed the dynamic-integrations branch from a72c83f to 69fcba5 Compare December 11, 2020 17:18
@rfratto
rfratto marked this pull request as ready for review December 11, 2020 17:19
@rfratto
rfratto requested a review from a team December 11, 2020 17:20
@rfratto

rfratto commented Dec 14, 2020

Copy link
Copy Markdown
Contributor Author

This needs a test for marshaling; pretty sure that doesn't work with this yet.

@rfratto
rfratto marked this pull request as draft December 14, 2020 20:24
@dafydd-t

Copy link
Copy Markdown
Contributor

I haven't looked at the PR in detail, but I really like this approach 👍 that manager is getting bloated with all the integration inits.

@rfratto
rfratto marked this pull request as ready for review December 15, 2020 22:08
@rfratto

rfratto commented Dec 15, 2020

Copy link
Copy Markdown
Contributor Author

Ready for review again! Implemented the marshaling.

Comment thread pkg/integrations/register.go
Comment thread pkg/integrations/register.go
@rfratto
rfratto merged commit 9e37baf into grafana-cold-storage:master Jan 5, 2021
@rfratto
rfratto deleted the dynamic-integrations branch January 5, 2021 19:21
@mattdurham mattdurham mentioned this pull request Sep 7, 2021
3 tasks
mattdurham pushed a commit that referenced this pull request Nov 11, 2021
* add dynamic integration registration system

This commit adds a dynamic integration registration system similar to
(and mostly copied from) Prometheus' SD registration mechanism.

* fix lint error

* fix runtime errors

* refactor integrations

1. Integration interface moved to integrations
2. Integration Config interface now responsible for providing
   name/config

* marshal manager config with embedded integrations
@github-actions github-actions Bot added the frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed. label Apr 22, 2024
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Apr 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

frozen-due-to-age Locked due to a period of inactivity. Please open new issues or PRs if more discussion is needed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants