refactor: extract shared gate config - #295
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors gate configuration handling by introducing a shared pipeline.GateConfig (with typed map[string]any params) and updating gate creation/parsing across flows and tests to avoid coercing typed JSON values into strings.
Changes:
- Introduces
pipeline.GateConfigand updates thepipeline.GateFactoryinterface to accept it. - Refactors
flowcontrol.GateFactoryto parse typed params via helper extractors (paramFloat,paramInt, etc.), including structured composite/wait-on-refuse configs. - Updates Redis SortedSet / PubSub flows and integration/unit tests to pass typed gate params.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/integration/prometheus_gate_factory_test.go | Updates integration tests to use pipeline.GateConfig with typed params. |
| test/integration/gate_factory_redis_test.go | Updates Redis quota gate integration tests to use typed params via GateConfig. |
| test/integration/endpoint_scrape_gate_factory_test.go | Updates endpoint-scrape integration tests to use typed params via GateConfig. |
| pkg/server/runner.go | Refactors runner setup and updates pool-gate creation to use GateConfig. |
| pkg/redis/sortedset_impl.go | Switches sortedset queue config gate params to typed map[string]any and passes GateConfig to factory. |
| pkg/redis/sortedset_impl_test.go | Updates tests to assert numeric gate params are preserved as native types. |
| pkg/pubsub/pubsubimpl.go | Embeds pipeline.GateConfig in topic config and passes it to gate factory. |
| pkg/async/inference/flowcontrol/promql_metric_source_factory.go | Updates saturation PromQL source config parsing to accept typed params. |
| pkg/async/inference/flowcontrol/metric_source_test.go | Updates metric source tests for typed params. |
| pkg/async/inference/flowcontrol/gate_factory.go | Refactors gate creation around GateConfig and adds typed param extractors. |
| pkg/async/inference/flowcontrol/gate_factory_test.go | Updates gate factory tests to use GateConfig and typed params. |
| pkg/async/inference/flowcontrol/gate_factory_composite_test.go | Updates composite gate tests to use GateConfig inputs. |
| pipeline/gate.go | Adds GateConfig and updates GateFactory interface signature. |
Comments suppressed due to low confidence (1)
pkg/server/runner.go:143
- Gate params coming from worker-pool config files are still decoded via pipeline.StringMap (which coerces numbers/bools to strings). Even after converting to map[string]any here, those values remain strings, so typed gate params from pool configs are not preserved as described in the PR. Consider switching WorkerPoolConfig to use pipeline.GateConfig / map[string]any to keep types end-to-end.
poolGates := make(map[string]pipeline.Gate)
for poolID, pool := range poolsMap {
if pool.GateType != "" {
gate, err := gateFactory.CreateGate(pipeline.GateConfig{GateType: pool.GateType, GateParams: toAnyMap(pool.GateParams)})
if err != nil {
setupLog.Error(err, "Failed to create pool gate", "poolID", poolID, "gateType", pool.GateType)
os.Exit(1)
}
poolGates[poolID] = gate
setupLog.Info("Created pool gate", "poolID", poolID, "gateType", pool.GateType, "gateParams", pool.GateParams)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@evacchi is this ready for review? The other PR has been merged. |
|
ah yes I missed that merge! |
|
Two non blocking comments:
Minor: paramString silently %v-coerces a wrong-typed value instead of erroring (inconsistent with the numeric helpers), and dropping StringMap means non-scalar gate_params are now silently accepted rather than rejected — worth a note in the description. |
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
77f0134 to
1615bd2
Compare
Signed-off-by: Edoardo Vacchi <evacchi@users.noreply.github.com>
1615bd2 to
c3ca7ae
Compare
What does this PR do?
Refactors common parsing code for Gates to a single place (under
pipeline/gate.go), exports a commonGateConfigstruct, and uses it everywhere; the unified parser accepts a typed JSON, and additional gate-specific config is now passed asmap[string]any(instead of map[string]string`, which caused all typed values to be coerced)Notice this is stacked on #287, so ignore the corresponding commit b99a773 (PR is hence draft for now)Why is this change needed?
Simplify Gate handling, currently there was quite a bit of duplication. It is also a step further towards improving the config parsing described in #284
How was this tested?
Checklist
git commit -s) per DCOmake test)make lint)Related Issues
Stacked on top of #287, related to #284