Skip to content

feat(supervisor): host resource gate throttles autonomous dispatch on CPU pressure - #459

Merged
getappz merged 5 commits into
masterfrom
task/435
Aug 12, 2026
Merged

feat(supervisor): host resource gate throttles autonomous dispatch on CPU pressure#459
getappz merged 5 commits into
masterfrom
task/435

Conversation

@getappz

@getappz getappz commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Host resource gate for autonomous dispatch (item #435)

Adds agentflare-resource-gate, a new crate providing a process-wide,
continuously-sampled host-pressure tier that gates autonomous dispatch
independently of the existing per-agent auth_db cooldown check.

What it does:

  • signals.rs — samples global CPU usage via a fresh sysinfo::System per
    call (regression guard against a Sentry-reported panic when a long-lived
    System outlives a runtime core-count change), plus a lightweight
    server/container/headless heuristic (AGENTFLARE_DEPLOYMENT env override,
    k8s, /.dockerenv, no-display-on-Linux).
  • policy.rs — pure decide(signals, cfg) -> Policy with tiers
    Aggressive (server/always-on) → NormalThrottledPaused, in a
    documented evaluation order (user override → server mode → CPU severity →
    CPU busy threshold), thresholds clamped so a malformed config can't
    silently disable or force-throttle dispatch.
  • gate.rs — a OnceLock-backed singleton: init_global() starts a 30s
    background sampler (idempotent, safe to call more than once), and
    current_policy() is a cheap cached read defaulting to Normal if never
    initialized.
  • pool_size.rs — the pre-existing dashboard::concurrency worker-pool
    sizing logic, relocated here as it's the same "how much can this host
    handle" concern (work_max_concurrency's CPU+memory-aware sizing).

Integration: supervisor::run_discovery_tick and self_repair_or_gate
now take a host_policy: Policy parameter and skip dispatch (counting the
item as waiting, not vanishing it) when host_policy.blocks_dispatch()
independent of, and alongside, the existing is_cooling_down agent check.
dashboard/server.rs::run calls init_global() once at daemon startup and
threads the live policy into both dispatch call sites.

Scope decisions:

  • Battery/AC probing (present in the openhuman reference design) is
    deliberately deferred — agentflare's autonomous work runs mostly on
    always-on dev boxes/servers, and pulling in starship-battery adds a real
    macOS FFI dependency chain for a signal unlikely to matter. Documented in
    signals.rs as a Policy/Signals field addable later with no API break.
  • The per-worker concurrency semaphore described in the item's "proposed
    shape" (hard-cap concurrent dispatch under Throttled/Paused) is not
    implemented — out of scope for the "Done when" bar, which only requires
    current_policy() existing and consulted by both dispatch paths plus a
    CPU-pressure test proving it blocks dispatch. Left as a natural follow-up.
  • src/supervisor.rs's 3 new host_gate_tests were split into
    src/supervisor_host_gate_tests.rs (via #[path]) purely to stay under
    the repo's 1500-line LOC gate — no logic change.
  • crates/agentflare-jobs/src/supervisor.rs had picked up an unrelated
    task/440 dead-code fix borrowed for local clippy testing (its own diff
    said "will be reverted") — reverted back to master's version; that fix
    belongs to task/440, not this item.

Verification: cargo test -p agentflare --bin agentflare supervisor::
22/22 passed including the 3 new host-gate tests. cargo test -p agentflare-resource-gate — 23/23 passed. cargo clippy -p agentflare -p agentflare-resource-gate --all-targets -- -A unsafe_code -A clippy::pedantic — zero warnings. A full --workspace -D warnings clippy
run is blocked by the pre-existing (master, Windows-only) agentflare-jobs
dead-code/unused-import issue that task/440 already fixes separately —
intentionally left untouched rather than folding an unrelated fix into this
PR.

This branch also recovers uncommitted work-in-progress that was sitting in
this item's worktree from an earlier, incomplete session (never committed,
discovered during an unrelated branch/worktree cleanup pass) — the crate
and integration above are that work, finished and verified.

Summary by CodeRabbit

  • New Features

    • Added host-resource awareness to dynamically manage workload dispatch.
    • Added automatic, always-on, and disabled resource-gating modes.
    • Added CPU pressure detection with throttling and pause behavior.
    • Added server-environment detection and adaptive worker pool sizing.
    • Dispatches now remain queued when host resources are constrained, and resume when conditions improve.
  • Bug Fixes

    • Improved cross-platform handling for resource monitoring.

…n (uncommitted work rescued from task/435 worktree)

Agentflare-Agent: claude-code_2-1-228_harness
Agentflare-Branch: wip/resource-gate
…supervisor.rs tests to stay under LOC gate

Agentflare-Agent: claude-code_2-1-228_harness
Agentflare-Branch: task/435
Agentflare-Item: 435
Adds `agentflare-resource-gate`, a new crate providing a process-wide,
continuously-sampled host-pressure tier that gates autonomous dispatch
independently of the existing per-agent `auth_db` cooldown check.

**What it does:**
- `signals.rs` — samples global CPU usage via a *fresh* `sysinfo::System` per
  call (regression guard against a Sentry-reported panic when a long-lived
  `System` outlives a runtime core-count change), plus a lightweight
  server/container/headless heuristic (`AGENTFLARE_DEPLOYMENT` env override,
  k8s, `/.dockerenv`, no-display-on-Linux).
- `policy.rs` — pure `decide(signals, cfg) -> Policy` with tiers
  `Aggressive` (server/always-on) → `Normal` → `Throttled` → `Paused`, in a
  documented evaluation order (user override → server mode → CPU severity →
  CPU busy threshold), thresholds clamped so a malformed config can't
  silently disable or force-throttle dispatch.
- `gate.rs` — a `OnceLock`-backed singleton: `init_global()` starts a 30s
  background sampler (idempotent, safe to call more than once), and
  `current_policy()` is a cheap cached read defaulting to `Normal` if never
  initialized.
- `pool_size.rs` — the pre-existing `dashboard::concurrency` worker-pool
  sizing logic, relocated here as it's the same "how much can this host
  handle" concern (`work_max_concurrency`'s CPU+memory-aware sizing).

**Integration:** `supervisor::run_discovery_tick` and `self_repair_or_gate`
now take a `host_policy: Policy` parameter and skip dispatch (counting the
item as `waiting`, not vanishing it) when `host_policy.blocks_dispatch()` —
independent of, and alongside, the existing `is_cooling_down` agent check.
`dashboard/server.rs::run` calls `init_global()` once at daemon startup and
threads the live policy into both dispatch call sites.

**Scope decisions:**
- Battery/AC probing (present in the openhuman reference design) is
  deliberately deferred — agentflare's autonomous `work` runs mostly on
  always-on dev boxes/servers, and pulling in `starship-battery` adds a real
  macOS FFI dependency chain for a signal unlikely to matter. Documented in
  `signals.rs` as a `Policy`/`Signals` field addable later with no API break.
- The per-worker concurrency semaphore described in the item's "proposed
  shape" (hard-cap concurrent dispatch under `Throttled`/`Paused`) is not
  implemented — out of scope for the "Done when" bar, which only requires
  `current_policy()` existing and consulted by both dispatch paths plus a
  CPU-pressure test proving it blocks dispatch. Left as a natural follow-up.
- `src/supervisor.rs`'s 3 new `host_gate_tests` were split into
  `src/supervisor_host_gate_tests.rs` (via `#[path]`) purely to stay under
  the repo's 1500-line LOC gate — no logic change.
- `crates/agentflare-jobs/src/supervisor.rs` had picked up an unrelated
  `task/440` dead-code fix borrowed for local clippy testing (its own diff
  said "will be reverted") — reverted back to master's version; that fix
  belongs to `task/440`, not this item.

**Verification:** `cargo test -p agentflare --bin agentflare supervisor::` —
22/22 passed including the 3 new host-gate tests. `cargo test -p
agentflare-resource-gate` — 23/23 passed. `cargo clippy -p agentflare -p
agentflare-resource-gate --all-targets -- -A unsafe_code -A
clippy::pedantic` — zero warnings. A full `--workspace -D warnings` clippy
run is blocked by the pre-existing (master, Windows-only) `agentflare-jobs`
dead-code/unused-import issue that `task/440` already fixes separately —
intentionally left untouched rather than folding an unrelated fix into this
PR.

This branch also recovers uncommitted work-in-progress that was sitting in
this item's worktree from an earlier, incomplete session (never committed,
discovered during an unrelated branch/worktree cleanup pass) — the crate
and integration above are that work, finished and verified.

Agentflare-Agent: claude-code_2-1-228_harness
Agentflare-Branch: task/435
Agentflare-Item: 435
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 8 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 99929449-ab90-4dca-b56c-e0a27cbe1530

📥 Commits

Reviewing files that changed from the base of the PR and between b5fc226 and 8ff6ff3.

📒 Files selected for processing (5)
  • crates/agentflare-resource-gate/src/config.rs
  • crates/agentflare-resource-gate/src/policy.rs
  • src/dashboard/server.rs
  • src/supervisor.rs
  • src/supervisor/tests/host_gate_tests.rs
📝 Walkthrough

Walkthrough

The PR adds the agentflare-resource-gate crate. It samples host CPU and deployment signals, derives cached dispatch policies, sizes worker pools, and applies host gating to discovery and self-repair dispatch.

Changes

Host Resource Gate

Layer / File(s) Summary
Gate configuration and policy contract
Cargo.toml, crates/agentflare-resource-gate/...
Adds the workspace crate, environment-based configuration, host signal sampling, policy decisions, public exports, and unit tests.
Cached policy sampling
crates/agentflare-resource-gate/src/gate.rs
Adds process-wide policy initialization, synchronous initial sampling, and 30-second background refreshes.
Shared pool sizing portability
crates/agentflare-resource-gate/src/pool_size.rs, src/dashboard/mod.rs, src/dashboard/server.rs
Uses shared pool-size helpers and limits cgroup-specific code and tests to Linux builds.
Resource-gated supervisor dispatch
src/dashboard/server.rs, src/supervisor.rs, src/supervisor_host_gate_tests.rs
Passes host policies into supervisor operations, preserves waiting items when dispatch is blocked, initializes the gate during startup, and tests throttled, paused, and aggressive policies.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DashboardServer
  participant ResourceGate
  participant Supervisor
  participant WorkQueue
  DashboardServer->>ResourceGate: current_policy()
  ResourceGate-->>DashboardServer: Policy
  DashboardServer->>Supervisor: run_discovery_tick(..., Policy)
  Supervisor->>WorkQueue: enqueue work when policy allows
  Supervisor-->>DashboardServer: report dispatched or waiting items
Loading

Possibly related PRs

  • getappz/agentflare#376: Extends the supervisor auto-dispatch loop with host resource-gate handling.
  • getappz/agentflare#417: Both changes modify supervisor dispatch control in the dashboard server and supervisor.
  • getappz/agentflare#436: Replaces dashboard-local concurrency helpers with shared resource-gate pool sizing and memory-budget helpers.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: host resource gating for supervisor dispatch under CPU pressure.
Description check ✅ Passed The description explains the change, integration, scope decisions, risks, and verification results, although it does not use every template heading.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch task/435

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/agentflare-resource-gate/src/config.rs`:
- Around line 25-35: Validate the threshold ordering in GateConfig::from_env by
restoring the default threshold pair when cpu_busy_threshold_pct >=
cpu_severe_pct, rather than relying only on individual range clamping. Add the
same guard in policy::decide because GateConfig fields are public, preserving
the intended Normal, Throttled, and Paused tiers. Add tests covering inverted
and equal thresholds.

In `@src/supervisor.rs`:
- Around line 534-538: Update the self-repair review flow around the dispatch
result and run_review_sweep so cooldown and host-policy deferrals return a
distinct deferred outcome rather than false/skipped. Add
ReviewSweepResult.waiting, increment it for retryable deferrals including
host_policy.blocks_dispatch(), and update the dashboard log to report waiting;
add throttled and paused self-repair tests covering these cases.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 415efb6f-56e8-4408-bc61-2c6825d1fa86

📥 Commits

Reviewing files that changed from the base of the PR and between a3fd25b and b5fc226.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (12)
  • Cargo.toml
  • crates/agentflare-resource-gate/Cargo.toml
  • crates/agentflare-resource-gate/src/config.rs
  • crates/agentflare-resource-gate/src/gate.rs
  • crates/agentflare-resource-gate/src/lib.rs
  • crates/agentflare-resource-gate/src/policy.rs
  • crates/agentflare-resource-gate/src/pool_size.rs
  • crates/agentflare-resource-gate/src/signals.rs
  • src/dashboard/mod.rs
  • src/dashboard/server.rs
  • src/supervisor.rs
  • src/supervisor_host_gate_tests.rs
💤 Files with no reviewable changes (1)
  • src/dashboard/mod.rs

Comment thread crates/agentflare-resource-gate/src/config.rs
Comment thread src/supervisor.rs
@getappz getappz changed the title Host resource gate for autonomous dispatch — throttle work/supervisor on CPU/battery pressure feat(supervisor): host resource gate throttles autonomous dispatch on CPU pressure Aug 12, 2026
…ed CPU thresholds, defer instead of skip

Three fixes on top of the resource-gate work:

- The host-gate test module used a #[path] override with a '..' traversal
  (src/supervisor/tests/../../). Windows resolved it lexically, but Linux
  walks the real path and those intermediate directories don't exist, so
  fmt/clippy/build all failed in CI. Moved the file to the location Rust's
  default module resolution already looks for and dropped #[path] entirely.

- policy::decide clamped cpu_busy_threshold_pct and cpu_severe_pct to 0..100
  independently, which still admits an inverted pair (busy >= severe). That
  collapses the tier ladder -- every reading above severe pauses before it
  can be judged merely busy, making Throttled unreachable. GateConfig's
  fields are public, so the ordering guard has to live in decide() rather
  than only in from_env(). Falls back to the shared default pair.

- self_repair_or_gate returned a bool, so a cooldown or host-pressure block
  was counted as 'skipped' -- which reads to an operator as a decision that
  won't be revisited, when in fact the next sweep retries it. Replaced with
  a SelfRepairOutcome enum and added ReviewSweepResult.waiting, mirroring
  DiscoveryTickResult.waiting from item #82.

Agentflare-Agent: claude-code_2-1-228_agent
Agentflare-Branch: task/435
Agentflare-Item: 435
…, not just in decide()

CodeRabbit's config.rs finding asked for the ordering guard in
GateConfig::from_env as well, which the previous commit only added to
policy::decide. That was the weaker half: env vars are the actual
user-facing input path, so from_env could hand back a config whose
threshold pair disagreed with what the gate would really enforce --
misleading to anything else reading those fields (a log line, a future
dashboard tile), even though decide() recovered internally.

Extracted the clamp-and-order logic into GateConfig::normalized() so there
is one implementation, applied at the env boundary and re-applied
(idempotently) in decide() -- still needed there because the fields are
public and a hand-built GateConfig never passes through from_env.

Agentflare-Agent: claude-code_2-1-228_agent
Agentflare-Branch: task/435
Agentflare-Item: 435
@getappz
getappz merged commit 0d552e0 into master Aug 12, 2026
17 checks passed
@getappz
getappz deleted the task/435 branch August 12, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant