fix(skills-index): give the offline builder a walk budget the full ClawHub catalog fits in - #44494
Closed
AIalliAI wants to merge 1 commit into
Closed
fix(skills-index): give the offline builder a walk budget the full ClawHub catalog fits in#44494AIalliAI wants to merge 1 commit into
AIalliAI wants to merge 1 commit into
Conversation
…awHub catalog fits in The interactive 12s CATALOG_WALK_BUDGET_SECONDS (added to stop browse/ search hanging on a slow catalog walk) also bounds the offline index builder, truncating the ~50k-skill ClawHub walk at ~16 pages. Every skills-index/deploy-site run since 2026-06-10 06:22 UTC fails the EXPECTED_FLOORS health check (clawhub 1797-3195 < 20000), freezing the docs site and skills index. Let ClawHubSource take an explicit per-instance walk budget and pass 600s from scripts/build_skills_index.py (full walk needs ~260s at the observed ~1 page/s). Interactive callers keep the 12s default. Fixes NousResearch#38240
Collaborator
AIalliAI
added a commit
to AIalliAI/Hermes
that referenced
this pull request
Jun 14, 2026
…ousResearch#44500) Upstream implementation by @austinpickett, merged to main as 2ee69d0. Replaces our reverted local variant (2621a96, PR NousResearch#44494 closed as superseded): instead of an explicit 600s builder budget, the wall-clock budget now applies only to bounded browse walks (max_items > 0) and the index builder uses the unbounded path via SOURCE_LIMITS clawhub=0. (patch applied from PR NousResearch#44500 diff; shallow clone prevented a direct cherry-pick of 2ee69d0)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Unblocks the skills index / docs-site deploys, which have failed on every run since 2026-06-10 06:22 UTC.
Root cause: the interactive 12-second
CATALOG_WALK_BUDGET_SECONDSadded in 105625d ("honour overall_timeout and bound ClawHub catalog walk", the #38459 hang fix) also applies to the offline index builder. The builder's walk-to-exhaustion over the ~50k-skill ClawHub catalog needs ~250 sequential pages (~260s at the observed ~1 page/s), so the 12s budget truncates it at ~16 pages. TheEXPECTED_FLOORShealth check from #42347 then correctly refuses to ship the degenerate index (clawhub: 1797 < expected floor 20000…3195 < 20000), failingskills-index.ymlanddeploy-site.ymlever since — the live docs site is frozen at the June 9 state and the watchdog keeps bumping #38240.Evidence
clawhub: 1797 skills (12.9s)— the crawl stops at exactly the 12s budget; later runs report 3195, i.e. the count varies with network speed, which is a truncation signature, not a catalog shrink.Fix:
ClawHubSourcenow accepts an explicit per-instancecatalog_walk_budget_seconds;scripts/build_skills_index.pypasses 600s (≈2.3× the measured full-walk time). Interactive callers (browse/search cold start) keep the tight 12s default, so the #38459 hang fix is untouched.Related Issue
Fixes #38240
Type of Change
Changes Made
tools/skills_hub.py—ClawHubSource.__init__(catalog_walk_budget_seconds=None)instance override of the class defaultscripts/build_skills_index.py— builder constructsClawHubSource(catalog_walk_budget_seconds=600)tests/tools/test_skills_hub_clawhub.py— regression test: instance budget wins over an expired class-level deadline, walk completes and cachestests/scripts/test_build_skills_index_health.py— fake-source stub accepts the new kwargHow to Test
pytest tests/tools/test_skills_hub_clawhub.py tests/scripts/test_build_skills_index_health.py -q→ 18 passedpytest tests/tools -k skills_hub -q→ 166 passed, 1 skippedClawHubSource(catalog_walk_budget_seconds=30)._load_catalog_index()gathers ~5.8k skills vs ~3.2k at the 12s default; with 600s the walk reaches cursor exhaustion (~50k) and the health check floor passesgh workflow run skills-index.ymlshould go green and the watchdog probe on [skills-index-watchdog] Skills index is stale or degraded (degraded) #38240 should recoverChecklist
Code
Documentation & Housekeeping
cli-config.yaml.exampleif I added/changed config keys — N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/AScreenshots / Logs
First failing run (skills-index #27261057718):
Latest deploy-site failure (#27366487914):
clawhub: 3195 < expected floor 20000