docs: power_of_two can compare candidates by in-flight requests: the --power-of-two-load-metric setting

src/lib/content/concepts/routing/load-balancing.md

@@ -34,7 +34,7 @@ Routes to the worker with the lowest expected wait: queued token work over throu
 
 ### :material-scale-balance: Power of Two
 
-Samples two workers and routes to the one with the lower expected wait. Near-least-load placement for two load reads per request.
+Samples two workers and routes to the one with the lower expected wait; a flag can switch the comparison to in-flight request counts. Near-least-load placement for two load reads per request.
 
 </div>
 
@@ -81,15 +81,15 @@ Forwards every request to the single available worker without reading load or ca
 | `random` | :material-close: | :material-close: | :material-close: | O(1) | Testing |
 | `passthrough` | :material-close: | :material-close: | :material-close: | O(1) | Single worker |
 
-Complexity is the selection cost per request over `n` available workers. [Sticky sessions](sticky-sessions.md) (`--routing-key-override`) add session affinity on top of any policy.
+Complexity is the selection cost per request over `n` available workers; `power_of_two` becomes O(n) with [`--power-of-two-load-metric least_requests`](#comparison-metric), which scans every available worker. [Sticky sessions](sticky-sessions.md) (`--routing-key-override`) add session affinity on top of any policy.
 
 ---
 
 ## Load Signals
 
 Load-aware policies read one or both of two signals:
 
-- **In-flight requests.** The gateway counts the requests it has sent to each worker that have not finished yet; a streamed response counts until its body ends. The count is kept for every request under every policy, on the HTTP, gRPC and PD paths alike, and is exported as `smg_worker_requests_active`. Each gateway replica counts only its own traffic. `prefix_hash` and the `cache_aware` spill gate compare these counts, and `least_load` and `power_of_two` fall back to them for workers without a load report.
+- **In-flight requests.** The gateway counts the requests it has sent to each worker that have not finished yet; a streamed response counts until its body ends. The count is kept for every request under every policy, on the HTTP, gRPC and PD paths alike, and is exported as `smg_worker_requests_active`. Each gateway replica counts only its own traffic. `prefix_hash` and the `cache_aware` spill gate compare these counts, `least_load` and `power_of_two` fall back to them for workers without a load report, and [`--power-of-two-load-metric`](#comparison-metric) can make them `power_of_two`'s primary comparison.
 - **Load reports.** The gateway polls each worker's load every `--load-monitor-interval` seconds (default `10`). HTTP workers are asked for `/v1/loads` whatever the engine, falling back to the engine's Prometheus `/metrics` (SGLang, vLLM) when that route does not answer; gRPC and ZMQ workers report over their engine connection (`GetLoads`). A report carries the worker's waiting requests and KV-cache usage, plus queued tokens and generation throughput where the engine exposes them. `least_load`, `power_of_two` and `cache_aware` read these reports. See [Load Monitoring](../../reference/configuration.md#load-monitoring-configuration) for the polling options.
 
 In `least_load`, `power_of_two` and `cache_aware`, scores that tie exactly are broken uniformly at random, so equally idle workers share traffic instead of the first worker taking every tie.
@@ -186,6 +186,8 @@ Before it uses a request's bucket, the policy checks balance. The load it compar
 
 Samples two distinct available workers at random and routes to the one with the lower expected wait, using the same formula as [`least_load`](#least-load): queued token work over throughput plus the KV-pressure penalty, with credit for requests sent since the last load report. If neither sampled worker has a load report, it compares their in-flight requests. It approaches least-load placement while reading only two workers' load per request.
 
+The expected wait is the default comparison; [`--power-of-two-load-metric`](#comparison-metric) can switch it to the gateway's own in-flight request counts.
+
 ```bash
 smg --policy power_of_two --worker-urls http://w1:8000 http://w2:8000
 ```
@@ -198,7 +200,7 @@ smg --policy power_of_two --worker-urls http://w1:8000 http://w2:8000
 
 - Load-aware at two load reads per request
 - Counts queue depth, KV pressure and in-flight work
-- No full-fleet scan per request
+- No full-fleet scan per request (except the `least_requests` metric)
 
 </div>
 
@@ -214,6 +216,26 @@ smg --policy power_of_two --worker-urls http://w1:8000 http://w2:8000
 
 </div>
 
+### Comparison Metric
+
+`--power-of-two-load-metric` sets what the policy compares its candidates by:
+
+| Value | Candidates | Comparison |
+|-------|------------|------------|
+| `expected_wait` | Two sampled at random | Lower expected wait wins (the default, described above) |
+| `requests` | Two sampled at random | Fewer in-flight requests wins; the expected wait breaks a tie |
+| `least_requests` | Every available worker | Keeps the workers with the fewest in-flight requests; the expected wait breaks a tie |
+
+```bash
+smg --policy power_of_two --power-of-two-load-metric requests --worker-urls http://w1:8000 http://w2:8000
+```
+
+Both in-flight metrics compare the gateway's own per-worker request counts (see [Load Signals](#load-signals)), read without any load report, so they balance on a live signal even where load reports are coarse or absent; only a tie goes on to the expected-wait score.
+
+`least_requests` gives up the policy's O(1) selection: it samples no pair and scans every available worker's count. Whatever the metric, a single available worker is selected directly.
+
+`--power-of-two-load-metric` is newer than v1.12.0. In v1.12.0 and earlier the comparison is always the expected wait.
+
 **Use when:** Large or heterogeneous fleets where cache locality doesn't matter and each pick should read only two workers' load.
 
 ---

src/lib/content/getting-started/load-balancing.md

@@ -71,6 +71,12 @@ Samples two random workers and routes to the one with the lower expected wait, u
 smg --policy power_of_two --worker-urls http://w1:8000 http://w2:8000
 ```
 
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `--power-of-two-load-metric` | `expected_wait` | What candidates are compared by: `expected_wait` (lower expected wait of the sampled pair), `requests` (the pair's in-flight request counts, the expected wait breaking ties), or `least_requests` (no pair — the fewest in-flight requests across every available worker, then the expected wait) |
+
+The in-flight metrics compare the gateway's own request counts without reading load reports; see [Comparison Metric](../concepts/routing/load-balancing.md#comparison-metric) for how each value selects, and for the versions that support the flag.
+
 Best for large or heterogeneous fleets where cache locality doesn't matter.
 
 ---