fix(metrics): make pool_name mean one thing across every gate series - #372
Merged
Conversation
The five series an operator needs to answer "why is this queue throttled?"
carried three different pool_name values, so none of them joined:
- async_gate_metric_value/_threshold labeled pool_name with the gate's
'pool' param, i.e. the InferencePool being queried, and carried no
queue labels at all;
- async_dispatch_budget and async_gate_decisions_total read pool_name
from configMap, which stored the raw queue config — so a queue that
omitted worker_pool_id got pool_name="" ...
- ... while the request channel, async_broker_backlog and
async_pool_worker_limit all said "default" for that same queue.
Give them one source. GateConfig gains an Owner (queue id, queue name,
worker pool) that the Redis, Pub/Sub and pool-gate construction paths
stamp before calling CreateGate and the factory propagates through its
recursive gate types (composite, wait-on-refuse, tier-priority-admission).
The gate gauges are labeled from it, so they join with the rest of the
queue's series on the queue triple; a pool-level gate leaves the queue
labels empty and is keyed by pool_name alone. The InferencePool a gate
queries is not lost — it moves to its own inference_pool label, which is
what the 'pool' param always meant. Two worker pools gating on the same
InferencePool are now distinguishable, which they were not before.
Separately, normalize WorkerPoolID to "default" before storing the queue
config, so configMap and the request channel cannot disagree.
Documents both gauges in the README metrics table (they were missing),
what each label means, the 'pool' param of prometheus-query, and the
PromQL for a "why is the gate closed?" panel.
Signed-off-by: Shimi Bandiel <shimib@google.com>
shimib
requested review from
RishabhSaini,
ahg-g,
evacchi and
jtechapps
as code owners
July 29, 2026 18:51
Signed-off-by: Shimi Bandiel <shimib@google.com>
jtechapps
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #369.
Problem
Five series answer "why is this queue throttled?". They carried three different
pool_namevalues, so none of them joined:pool_namewasasync_gate_metric_value/_thresholdpoolparam — the InferencePoolasync_dispatch_budget,async_gate_decisions_totalconfigMap[queueID].WorkerPoolID—""when the queue omittedworker_pool_idqueue_id,queue_nameasync_broker_backlog,async_pool_worker_limit"default"queue_id,queue_nameTwo separate defects:
A plain bug.
NewRedisSortedSetFlowdefaultsworker_pool_idto"default"for the request channel, then stores the raw config inconfigMap:So one queue emits
pool_name=""on two of its series and"default"on the rest. Pub/Sub was never affected — it reads the pool off the already-defaulted request channel.A label-semantics collision.
pool_nameon the gate gauges is the InferencePool the gate measures; everywhere else it is the async worker pool the series belongs to. One label name, two concepts — and the gate gauges carried noqueue_nameto fall back on, so there was nothing to join on either. In the multitenant guide these happen to be equal strings, which is why it went unnoticed.Change
One source for the label.
pipeline.GateConfiggains anOwner(queue_id,queue_name,worker_pool_id),json:"-"— stamped by the caller, never deserialized from user config. The three construction paths set it (pkg/redis/sortedset_impl.go,pkg/pubsub/pubsubimpl.go,pkg/server/runner.go), andGateFactory.CreateGatepropagates it through all three recursive gate types (composite,wait-on-refuse,tier-priority-admission) — the inner configs round-trip through JSON, so this has to be explicit.async_dispatch_budgetand everything else onon(queue_id, queue_name, pool_name). A pool-level gate has no single queue, so it leaves the queue labels empty and is keyed bypool_namealone.inference_poollabel, which is what thepoolparam always meant. Two worker pools gating on the same InferencePool are now distinguishable; before, they overwrote each other's gauge.WorkerPoolIDbefore storing it, soconfigMapand the request channel cannot disagree. This is the issue's own suggested minimum, and it is still worth doing on its own — several other reads go throughconfigMap.Chose
GateConfig.Ownerover changing theGateFactoryinterface:CreateGate(cfg GateConfig) (Gate, error)is public API in a separately tagged module, and a new field is additive.Docs
Both gauges were missing from the README metrics table entirely. Added them, spelled out that
pool_namealways names the async worker pool andinference_poolthe queried InferencePool, documented the (previously undocumented)poolparam ofprometheus-query, and added the PromQL for a "why is the gate closed?" panel:Tests
pkg/async/inference/flowcontrol: the owner reaches a metric gate directly, and survives all three recursion sites — composite → wait-on-refuse → metric gate, and tier-priority-admission's inner saturation gate. Updated the [Feature]: Metrics #217 gauge test for the new label set.pkg/redis: a queue config with noworker_pool_idlands inconfigMapas"default", and agrees with its request channel.Compatibility
Breaking for dashboards on
async_gate_metric_value/async_gate_metric_threshold. They gain three labels, andpool_namechanges meaning from the InferencePool to the worker pool. A query that selected{pool_name="my-inference-pool"}should now select{inference_pool="my-inference-pool"}; one that only aggregated is unaffected. The two gauges shipped recently (#339, v0.8.0) and are not referenced by any chart or dashboard in this repo.Queues that omitted
worker_pool_idwill seeasync_dispatch_budgetandasync_gate_decisions_totalmove frompool_name=""topool_name="default"— the value their sibling series already reported.Known limitation, unchanged by this PR: two metric gates inside one composite still share a series unless their
poolparams differ.Note on verification
There is no Go toolchain on this machine, so I could not run
make testlocally — and this repo's CI does not run unit tests either (.github/workflows/pre-commit.ymlrunsmake test-integration, helm lint and helm unittest;make testis never invoked). The new and updated tests are type-checked bygo vetandgolangci-lintin pre-commit, but have not been executed. Flagging it rather than implying otherwise. Adding amake teststep to that workflow looks worth doing separately.