Skip to content

fix(agent): optimise headroom and litellm caching - #387

Merged
yxtay merged 4 commits into
mainfrom
fix/headroom
Jul 30, 2026
Merged

fix(agent): optimise headroom and litellm caching#387
yxtay merged 4 commits into
mainfrom
fix/headroom

Conversation

@yxtay

@yxtay yxtay commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Summary

  • Switch headroom savings profile from general to coding — uses cache-mode delta compression, protects file read outputs from lossy compression, enables tool schema deduplication and cross-turn dedup; better fit for coding agent workloads
  • Add headroom healthcheck (GET /health) so litellm's depends_on: condition: service_healthy actually gates on readiness
  • Enable local in-memory response cache in litellm with 1h TTL to avoid redundant upstream calls for identical prompts
  • Set user_api_key_cache_ttl: 300 to reduce Postgres auth lookup pressure

Test plan

  • docker compose up — litellm waits for headroom healthy before starting
  • Repeat identical prompt twice — second response returns immediately from cache (check litellm logs for cache_hit)
  • Hermes coding tasks work correctly with coding savings profile

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added local caching for requests and responses to improve repeat-request performance.
    • Added short-term caching for API key data.
  • Improvements

    • Updated coding-focused configuration and cache settings for more efficient operation.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@yxtay, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 03789e0c-f0c5-46ff-9d1f-56faf09dff6a

📥 Commits

Reviewing files that changed from the base of the PR and between d4cdec8 and 0ea5e01.

📒 Files selected for processing (1)
  • agent/compose.yaml
📝 Walkthrough

Walkthrough

The agent stack updates Headroom environment variables and enables LiteLLM user-key and local request/response caching with configured TTLs.

Changes

Agent runtime configuration

Layer / File(s) Summary
Headroom environment settings
agent/compose.yaml
The Headroom service uses the coding savings profile and XDG_CACHE_HOME, while HEADROOM_LOSSLESS and HOME are removed.
LiteLLM cache configuration
agent/litellm/config.yaml
Adds a 300-second user API key cache and enables a local request/response cache with a 3600-second TTL.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Poem

I’m a bunny tuning caches bright,
Headroom hops in coding light.
Keys rest briefly, requests stay,
Local caches help them on their way.
Config carrots crunch—hooray! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes to headroom and LiteLLM caching.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/headroom

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ BASH shfmt 11 0 0 0 0.02s
✅ JSON prettier 1 0 0 0 0.23s
✅ MARKDOWN markdownlint 5 0 0 0 0.46s
✅ MARKDOWN markdown-table-formatter 5 0 0 0 0.13s
✅ TERRAFORM terraform-fmt 7 0 0 0 0.3s
✅ YAML prettier 49 0 0 0 0.68s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters BASH_SHFMT,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,TERRAFORM_TERRAFORM_FMT,YAML_PRETTIER

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@github-actions

Copy link
Copy Markdown

MegaLinter analysis: Success

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ BASH bash-exec 11 0 0 0.04s
✅ BASH shellcheck 11 0 0 0.21s
✅ REPOSITORY betterleaks yes no no 1.63s
✅ REPOSITORY checkov yes no no 26.04s
✅ REPOSITORY dustilock yes no no 0.03s
✅ REPOSITORY gitleaks yes no no 1.02s
✅ REPOSITORY grype yes no no 64.26s
✅ REPOSITORY kingfisher yes no no 13.65s
✅ REPOSITORY osv-scanner yes no no 0.42s
✅ REPOSITORY secretlint yes no no 1.67s
✅ REPOSITORY syft yes no no 2.47s
✅ REPOSITORY trivy yes no no 12.26s
✅ REPOSITORY trivy-sbom yes no no 0.12s
✅ REPOSITORY trufflehog yes no no 3.57s
✅ TERRAFORM tflint yes no no 7.55s

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters BASH_EXEC,BASH_SHELLCHECK,REPOSITORY_CHECKOV,REPOSITORY_DUSTILOCK,REPOSITORY_GITLEAKS,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,REPOSITORY_KINGFISHER,TERRAFORM_TFLINT

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@github-actions

Copy link
Copy Markdown

⚠️MegaLinter analysis: Success with warnings

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ ACTION actionlint 4 0 0 0.23s
✅ ACTION zizmor 4 0 0 0 0.29s
✅ BASH bash-exec 11 0 0 0.03s
✅ BASH shellcheck 11 0 0 0.22s
✅ BASH shfmt 11 0 0 0 0.01s
⚠️ COPYPASTE jscpd yes 1 no 0.1s
✅ EDITORCONFIG editorconfig-checker 107 0 0 0.08s
✅ JSON prettier 1 0 0 0 0.28s
✅ JSON v8r 1 0 0 1.39s
✅ MARKDOWN markdownlint 5 0 0 0 0.82s
✅ MARKDOWN markdown-table-formatter 5 0 0 0 0.18s
✅ REPOSITORY betterleaks yes no no 1.42s
✅ REPOSITORY checkov yes no no 26.47s
✅ REPOSITORY gitleaks yes no no 0.52s
✅ REPOSITORY git_diff yes no no 0.01s
✅ REPOSITORY grype yes no no 66.14s
✅ REPOSITORY osv-scanner yes no no 0.21s
✅ REPOSITORY secretlint yes no no 1.46s
✅ REPOSITORY syft yes no no 1.7s
✅ REPOSITORY trivy yes no no 10.72s
✅ REPOSITORY trivy-sbom yes no no 0.24s
✅ REPOSITORY trufflehog yes no no 6.25s
⚠️ SPELL lychee 56 44 0 4.76s
✅ YAML prettier 49 0 0 0 1.12s
✅ YAML v8r 49 0 0 17.03s
✅ YAML yamllint 49 0 0 1.02s

Detailed Issues

⚠️ COPYPASTE / jscpd - 1 error
Using config from /action/lib/.automation/.jscpd.json
Clone found (bash)
 - bin/oci-rm-stack-create.sh [32:1 - 40:5] (9 lines, 51 tokens)
   bin/oci-rm-stack-update.sh [12:1 - 20:5]
┌────────┬────────────────┬─────────────┬──────────────┬──────────────┬──────────────────┬───────────────────┐
│ Format │ Files analyzed │ Total lines │ Total tokens │ Clones found │ Duplicated lines │ Duplicated tokens │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ bash   │ 7              │ 214         │ 1120         │ 1            │ 8 (3.74%)        │ 51 (4.55%)        │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ hcl    │ 4              │ 382         │ 1407         │ 0            │ 0 (0.00%)        │ 0 (0.00%)         │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ txt    │ 1              │ 29          │ 379          │ 0            │ 0 (0.00%)        │ 0 (0.00%)         │
├────────┼────────────────┼─────────────┼──────────────┼──────────────┼──────────────────┼───────────────────┤
│ Total: │ 12             │ 625         │ 2906         │ 1            │ 8 (1.28%)        │ 51 (1.75%)        │
└────────┴────────────────┴─────────────┴──────────────┴──────────────┴──────────────────┴───────────────────┘
Found 1 clones.
HTML report saved to megalinter-reports/copy-paste/jscpd-report.html
ERROR: jscpd found too many duplicates (1.3%) over threshold (0.0%)
time: 21.820ms
⚠️ SPELL / lychee - 44 errors
📝 Summary
---------------------
🔍 Total...........71
🔗 Unique..........58
✅ Successful......20
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........0
❓ Unknown..........0
🚫 Errors..........44
⛔ Unsupported.....44

Errors in agent/compose.yaml
[ERROR] http://localhost:2375/_ping (at 89:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:4000/health/liveliness (at 131:70) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9119/api/status (at 41:33) | Connection refused - server may be down or port blocked
[ERROR] https://hermes/ (at 12:67) | Connection failed. Check network connectivity and firewall settings

Errors in agent/litellm/config.yaml
[ERROR] http://headroom:8787/ (at 100:17) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://headroom:8787/ (at 108:17) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://headroom:8787/ (at 118:17) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://headroom:8787/ (at 92:17) | Connection failed. Check network connectivity and firewall settings
[404] https://inference-api.nousresearch.com/v1 (at 94:30) | Rejected status code: 404 Not Found
[404] https://opencode.ai/zen/go/v1 (at 110:30) | Rejected status code: 404 Not Found
[404] https://opencode.ai/zen/v1 (at 102:30) | Rejected status code: 404 Not Found

Errors in arcane/compose.yaml
[ERROR] http://localhost:3552/ (at 6:27) | Connection refused - server may be down or port blocked

Errors in homeassistant/compose.yaml
[ERROR] http://localhost:10000/health (at 146:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:6052/version (at 105:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8095/ (at 66:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8123/ (at 27:33) | Connection refused - server may be down or port blocked

Errors in immich/compose.yaml
[ERROR] http://localhost:8080/ (at 160:32) | Connection refused - server may be down or port blocked

Errors in infra/compose.yaml
[ERROR] http://localhost:2375/_ping (at 60:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9002/healthz (at 95:36) | Connection refused - server may be down or port blocked

Errors in monitoring/compose.yaml
[ERROR] http://localhost:8090/ (at 73:44) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8090/ (at 94:27) | Connection refused - server may be down or port blocked

Errors in pangolin/compose.yaml
[ERROR] http://gerbil:3004/ (at 63:23) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://localhost/ping (at 121:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3001/api/v1/ (at 51:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3004/healthz (at 87:32) | Connection refused - server may be down or port blocked
[ERROR] http://pangolin:3001/api/v1/ (at 65:24) | Connection failed. Check network connectivity and firewall settings

Errors in pangolin/traefik/dynamic/config.yml
[ERROR] http://pangolin:3000/ (at 80:18) | Connection failed. Check network connectivity and firewall settings
[ERROR] http://pangolin:3002/ (at 85:18) | Connection failed. Check network connectivity and firewall settings

Errors in pangolin/traefik/traefik.template.yml
[ERROR] http://pangolin:3001/api/v1/traefik-config (at 7:15) | Connection failed. Check network connectivity and firewall settings

Errors in proxy/compose.yaml
[ERROR] http://localhost/healthz (at 36:32) | Connection refused - server may be down or port blocked
[ERROR] https://tinyauth/ (at 50:24) | Connection failed. Check network connectivity and firewall settings

Errors in security/compose.yaml
[ERROR] http://localhost:8080/ (at 52:16) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8080/health (at 34:32) | Connection refused - server may be down or port blocked

Errors in torrent/compose.yaml
[ERROR] http://localhost:6868/ (at 220:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:7878/ping (at 140:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8191/health (at 16:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:8989/ping (at 182:33) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:9696/ping (at 45:33) | Connection refused - server may be down or port blocked
[ERROR] https://profilarr/ (at 205:15) | Connection failed. Check network connectivity and firewall settings

Errors in usenet/compose.yaml
[ERROR] http://127.0.0.1:7000/ (at 83:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:1337/ (at 288:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:3000/health (at 26:32) | Connection refused - server may be down or port blocked
[ERROR] http://localhost:7000/health (at 134:32) | Connection refused - server may be down or port blocked
[ERROR] https://aiostreams/ (at 232:18) | Connection failed. Check network connectivity and firewall settings

Hint: You can configure accepted/rejected response codes with `-a` or `--accept`

Notices

📣 MegaLinter 9.5.0 is out! Discover the new features and security recommendations in the release announcement. (Skip this info by defining SECURITY_SUGGESTIONS: false)

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

  • Documentation: Custom Flavors
  • Command: npx mega-linter-runner@9.6.0 --custom-flavor-setup --custom-flavor-linters ACTION_ACTIONLINT,ACTION_ZIZMOR,BASH_EXEC,BASH_SHELLCHECK,BASH_SHFMT,COPYPASTE_JSCPD,EDITORCONFIG_EDITORCONFIG_CHECKER,JSON_V8R,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_GITLEAKS,REPOSITORY_BETTERLEAKS,REPOSITORY_GRYPE,REPOSITORY_OSV_SCANNER,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,SPELL_LYCHEE,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R

MegaLinter is graciously provided by OX Security
Show us your support by starring ⭐ the repository

@yxtay
yxtay merged commit 9f4aa38 into main Jul 30, 2026
12 of 14 checks passed
@yxtay
yxtay deleted the fix/headroom branch July 30, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant