[chore] RFC for scraper controller extension#14469
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #14469 +/- ##
==========================================
- Coverage 91.83% 91.23% -0.60%
==========================================
Files 680 698 +18
Lines 43043 44708 +1665
==========================================
+ Hits 39528 40790 +1262
- Misses 2445 2774 +329
- Partials 1070 1144 +74 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
|
||
| ## Open Questions | ||
|
|
||
| 1. Should there be a way to pass dynamic configuration from the extension to the scraper, e.g. `targets` for httpcheck? |
There was a problem hiding this comment.
This is probably better handled with something like receivercreator (e.g. scrapercreator), which dynamically creates one-shot scrapers in response to some event(s). The event payload could then be referenced with variables in the scraper's configuration.
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
@open-telemetry/collector-approvers can I please get a review on this? 🙏 |
|
Could we mesh this proposal and its use cases with telemetry policies? open-telemetry/opentelemetry-specification#4738 |
|
@atoulme did you have something in particular in mind? I read through that proposal, and I think I get it (composeable, declarative policies to apply on telemetry data) -- but I don't see very clearly how my proposal would overlap with it. From the description of that PR:
From my read, the generation of telemetry is outside the scope of policies; they're policies to apply on telemetry. I guess where the line becomes blurry is with things like metrics interval, which effectively controls generation (like scraper controllers). I suppose you could extend that to generating telemetry in exactly one instance of a fleet of collectors, on an interval, and how that happens is an implementation detail. This RFC is really about making those implementation detail possible. I think we can extend telemetry policies to take advantage of them later. |
|
Can you add to the PR description how you have fulfilled the requirements regarding announcing this RFC/RFC Amendment? |
|
@mx-psi done |
Yes, I had this link in mind. Agree the relationship is tenuous, just want to make sure we try to converge when at all sensible. |
codeboten
left a comment
There was a problem hiding this comment.
This proposal looks good to me. I think the details of the one-shot execution can sorted out later, but i like the flexibility of the extension proposed here.
|
I have marked this as "final comment period". I will merge this on Friday EU morning unless there are blocking comments by then. |
Description
This RFC proposes introducing an extension interface for scraper controllers to enable event-driven scraping patterns.
The current time-based scraper controller will be maintained with an option to disable the timer, and new configuration will allow specifying one or more scraper controller extensions that can trigger scrapes based on external events.
This is a proposed solution to #12449
RFC announced