feat(scheduler): per-job max_iterations override from jobs.json - #33323
feat(scheduler): per-job max_iterations override from jobs.json#33323inquistiff wants to merge 2 commits into
Conversation
Add job.get(max_iterations) as first priority before config.yaml fallback. Allows P5 Eval Judge to run at 120 iterations while global default stays at 90. Line 1156: was _cfg.get(agent).get(max_turns) or ... or 90 Now: job.get(max_iterations) or _cfg.get(agent).get(max_turns) or ...
- Extract inline one-liner into validated block with logger.warning on invalid type (string, float, negative, zero) — matches spec from PR description; closes gap that got NousResearch#2168 closed - Add tests/cron/test_scheduler_max_iterations.py: 13 tests covering fallback chain, type validation, boundary values, and scheduler import - Fallback chain: job.max_iterations (positive int) > agent.max_turns > max_turns > 90 hard default Production-validated: patch running in live cron fleet 6+ weeks.
|
Thanks for the context on #2168 — appreciated. To address the gaps flagged there: Scope: This PR covers the scheduler read path ( Fallback chain (composes with #33305): This PR handles the per-job leaf; #33305 handles the global floor. The two are complementary — if both land, operators get a global cap + per-job escape hatch. Validation (commit Tests (commit Production: This patch has been running in a 50-job Happy to rebase on top of #33305 once that lands if you'd prefer the integration to be explicit in a single PR. |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the focused scheduler change. The underlying gap is present on current main: cron/scheduler.py:2899 resolves only the global budget, then passes it to AIAgent at cron/scheduler.py:3054.
Problems
cron/scheduler.py:1541acceptsTruebecause Python booleans are integers. A JSONtruewould silently impose a one-iteration cap despite the stated positive-integer validation contract.tests/cron/test_scheduler_max_iterations.py:21duplicates production logic in_resolveinstead of exercising_run_job_impl/run_job; the tests can pass even if the scheduler does not use the field. Existing scheduler tests inspect constructor kwargs attests/cron/test_scheduler.py:995.- The field is not exposed by the normal job-writing surfaces (
cron/jobs.py:1033,tools/cronjob_tools.py:659,hermes_cli/subcommands/cron.py:27), so it is limited to manualjobs.jsonedits.
Suggested changes
- Reject booleans with
type(value) is int, and cover both boolean values. - Assert the real
AIAgent(..., max_iterations=...)constructor argument throughrun_job, including fallback and warning cases. - If this is intended as a supported per-job configuration, integrate it through the job store and public cron interfaces, preferably using the existing
max_turnsvocabulary.
Automated hermes-sweeper review.
| # Max iterations: per-job override > agent.max_turns > max_turns > 90 | ||
| _global_max_iter = _cfg.get("agent", {}).get("max_turns") or _cfg.get("max_turns") or 90 | ||
| _job_max_iter = job.get("max_iterations") | ||
| if _job_max_iter is None: |
There was a problem hiding this comment.
bool is a subclass of int, so a hand-edited JSON true passes this condition and silently sets a one-iteration cap. Use type(_job_max_iter) is int and add boolean cases to the validation coverage.
| if job_val is None: | ||
| return global_max, warnings | ||
| if isinstance(job_val, int) and job_val > 0: | ||
| return job_val, warnings |
There was a problem hiding this comment.
This duplicates the resolver instead of exercising the scheduler, so it can pass while the production run_job path ignores the field. Mock AIAgent through run_job and assert its max_iterations constructor kwarg, as existing scheduler tests inspect constructor kwargs.
What does this PR do?
Hermes scheduler has a single global
max_iterationscap applied to every cron job. In multi-tenant cron fleets, this is too coarse — some jobs (deep-research / multi-step debugging) need 40+ iterations; others (single-shot watchdogs / digest builders) should cap at 3 to fail-fast. A global cap forces a worst-case ceiling that wastes tokens on jobs that should never need that many turns.This patch reads
max_iterationsfrom the per-jobjobs.jsonentry if present, falling back to the global cap if not specified.{ "id": "deep-research-cron", "prompt": "...", "enabled_toolsets": ["agent", "browser"], "max_iterations": 40 }Type of Change
Changes Made
cron/scheduler.pyline 1156 (approx): per-jobmax_iterationskey checked before global config fallback. One-line change, fully backward-compatible.How to Test
"max_iterations": 5to a job entry injobs.json.Checklist