admin: Track reverse_proxy in-flight requests#7517
Conversation
2805368 to
25da121
Compare
|
@huang041 Brilliant catches! Thank you for the detailed review.You are totally right about the global lock contention under high RPS. I've updated the implementation to use sync.Map with atomic.Int64, which makes the hot path lock-free.Spot on regarding the |
2fbd5d1 to
30c4e21
Compare
…server#7277) This refactors the initial approach in PR caddyserver#7281, replacing the UsagePool with a dedicated package-level sync.Map and atomic.Int64 to track in-flight requests without global lock contention. It also introduces a lookup map in the admin API to fix a potential O(n^2) iteration over upstreams, ensuring that draining upstreams are correctly exposed across config reloads without leaking memory. Co-authored-by: Y.Horie <u5.horie@gmail.com> reverseproxy: optimize in-flight tracking and admin API - Replaced sync.RWMutex with sync.Map and atomic.Int64 to avoid lock contention under high RPS. - Introduced a lookup map in the admin API to fix a potential O(n^2) iteration over upstreams.
30c4e21 to
a19eb89
Compare
|
I'll review this soon... thanks! |
|
While reviewing this it made me think about dynamic upstreams a bit more, and I came up with this #7539, which should pair nicely with the improved in-flight count tracking. |
|
Amazing work, @francislavoie! Breaking out the dynamic hosts into their own registry makes total sense. The background cleanup goroutine is a very elegant approach. It acts like a TTL cache, preventing memory leaks while ensuring the health state persists long enough for the passive health checks to actually trip and do their job. Really glad my refactor could help kick off this improvement! I'll be following this implementation closely to learn from it. |
mholt
left a comment
There was a problem hiding this comment.
Neato, thank you for the revisions!
Assistance Disclosure
I consulted an AI assistant (Gemini) to help refactor the UsagePool logic into the
sync.RWMutexmap approach suggested by @mholt. I fully reviewed, authored the final structure, and verified the code's correctness.This PR supersedes #7281 and implements the feature requested in #7277. Huge thanks to @u5surf for laying the groundwork!
As discussed in the previous PR and suggested by @mholt, this replaces the
UsagePoolimplementation with a lightweight, package-levelmap[string]uintprotected by async.RWMutex.Changes:
inflightHostsUsagePool logic.inFlightRequestsmap to track requests during the proxy loop securely.admin.goto expose these draining hosts alongside the static hosts in/reverse_proxy/upstreams.