Skip to content

fix(skills): add 429 rate limit retry to GitHub Contents API fallback - #3033

Closed
0xbyt4 wants to merge 1 commit into
NousResearch:mainfrom
0xbyt4:fix/skills-hub-silent-data-loss
Closed

fix(skills): add 429 rate limit retry to GitHub Contents API fallback#3033
0xbyt4 wants to merge 1 commit into
NousResearch:mainfrom
0xbyt4:fix/skills-hub-silent-data-loss

Conversation

@0xbyt4

@0xbyt4 0xbyt4 commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Summary

When the Git Trees API falls back to the recursive Contents API (truncated tree or API unavailable), HTTP 429 rate-limited responses caused silent subdirectory loss:

  • Files were skipped with only a DEBUG log
  • The skill was recorded as fully installed
  • Users ended up with broken/incomplete skills without any warning

This adds retry with Retry-After header support to both _download_directory_recursive and _fetch_file_content:

  • 3 attempts with configurable backoff (from Retry-After header, capped at 15s)
  • Guards against non-numeric Retry-After values (HTTP-date format)
  • Matches the existing retry pattern in ClawHub._download_zip (line ~1729)

Log levels follow codebase convention:

  • debug: individual retry attempts, non-200 responses, network errors
  • info: tree truncation (normal for large repos)
  • warning: only when all 3 retries exhausted (actual data loss risk)

Test plan

  • tests/tools/test_skills_hub.py — 73/73 passed
  • Verified retry loop for/else pattern is correct
  • Verified int(Retry-After) guarded with try/except for non-numeric headers
  • Verified empty subdirectories don't trigger false "failed" warnings

When Git Trees API falls back to the recursive Contents API (truncated
tree or API unavailable), rate-limited responses (HTTP 429) caused
silent subdirectory loss — files were skipped with only a DEBUG log,
and the skill was recorded as fully installed.

Added retry with Retry-After header support (3 attempts, 15s cap) to
both _download_directory_recursive and _fetch_file_content. Matches
the existing retry pattern in ClawHub source.

Also: guard against non-numeric Retry-After headers (HTTP-date format),
promote tree truncation log to INFO level.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists tool/skills Skills system (list, view, manage) labels May 2, 2026
teknium1 added a commit that referenced this pull request Jul 5, 2026
…-retry

fix(skills): retry rate-limited Contents API directory listings (salvage #3033)
liuhao1024 pushed a commit to liuhao1024/hermes-agent that referenced this pull request Jul 5, 2026
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).

Salvage of PR NousResearch#3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.

Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).

Salvage of PR NousResearch#3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.

Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…contents-api-retry

fix(skills): retry rate-limited Contents API directory listings (salvage NousResearch#3033)
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for identifying the silent Contents API fallback failure mode. This is an automated hermes-sweeper review; the implementation has already landed on main via a salvage that preserves the intent and contributor credit.

This shipped in v2026.7.7.

@teknium1 teknium1 closed this Jul 12, 2026
@teknium1 teknium1 added sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).

Salvage of PR NousResearch#3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.

Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…contents-api-retry

fix(skills): retry rate-limited Contents API directory listings (salvage NousResearch#3033)
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).

Salvage of PR NousResearch#3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.

Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…contents-api-retry

fix(skills): retry rate-limited Contents API directory listings (salvage NousResearch#3033)
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
The Contents-API fallback's directory listing used a raw httpx.get with
no retry, so a 429/403 rate limit aborted the whole skill download even
though file fetches already retry via _github_get. Route the directory
listing through the same helper (429/reset-aware backoff, 5xx retry,
rate-limit flagging).

Salvage of PR NousResearch#3033's intent — rerouted through the _github_get helper
that landed after the PR was opened, instead of the PR's ad-hoc retry
loops.

Co-authored-by: 0xbyt4 <35742124+0xbyt4@users.noreply.github.com>
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…contents-api-retry

fix(skills): retry rate-limited Contents API directory listings (salvage NousResearch#3033)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:implemented-on-main Sweeper: behavior already present on current main tool/skills Skills system (list, view, manage) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants