Skip to content

fix(context): use safe 8192 default instead of 128k - #139

Closed
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/safe-context-default
Closed

fix(context): use safe 8192 default instead of 128k#139
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/safe-context-default

Conversation

@Bartok9

@Bartok9 Bartok9 commented Feb 27, 2026

Copy link
Copy Markdown
Contributor

Fixes #132

Problem

The 128k default context length was too optimistic and caused failures on models with smaller contexts:

  • Mistral: 65536 tokens
  • Qwen 2.5: 32768 tokens
  • Various local models: often 4096-16384

This led to errors like:

Error code: 400 - request (67374 tokens) exceeds the available context size (65536 tokens)

Solution

  • Changed default from 128000 to 8192 (safe for all models)
  • Added SAFE_DEFAULT_CONTEXT_LENGTH constant for clarity
  • Log a warning when using the fallback so users know to configure their model
  • Updated docstrings to explain the fallback chain

Why 8192?

  • Conservative but won't crash
  • Works with even small local models
  • Users can easily configure larger values for known models

The right fix long-term is to require context length in config for unknown models, but this is a safe improvement that prevents crashes today.

Bartok Moltbot Bartok Moltbot
fix(context): use safe 8192 default instead of 128k
7b416bd 
Fixes NousResearch#132

The previous 128k default was too optimistic and caused failures on models
with smaller contexts (e.g., 65k Mistral, 32k Qwen, various local models).

Changes:
- Added SAFE_DEFAULT_CONTEXT_LENGTH constant (8192)
- Log warning when using fallback so users know to configure their model
- Updated docstring to explain the fallback chain

8192 is conservative but won't crash - users can configure larger values
for known models via DEFAULT_CONTEXT_LENGTHS or use OpenRouter for auto-detection.
@teknium1

teknium1 commented Feb 28, 2026

Copy link
Copy Markdown
Contributor

I think instead we should implement a halving retry - if 128K is too large, try 64k, then 32k, then 16k, etc, and lock that config value in for that model configuration. wdyt?

@teknium1 teknium1 closed this Feb 28, 2026
teddyjfpender added a commit to teddyjfpender/superforecasting-agent that referenced this pull request Jun 24, 2026
@teddyjfpender @claude
feat(forecast): dedicated thesis dashboard surface (NousResearch#139)…
1b6b2b0 
… — CLI + forecast.theses RPC + tool

Theses already rendered inside the forecasts workspace (a lens + ThesisDeskRead), but
there was no STANDALONE thesis dashboard — every client had to load the whole forecast
workspace to see them. This makes the thesis master list a first-class surface across
CLI / gateway / tool, reusing the existing build_thesis_summary payload (health /
score / Δ / coverage / n_eff / members / status), so no new aggregation math.

- `forecast thesis dashboard` (+ --json): the master list as a table — the dedicated
  thesis view from the terminal.
- gateway `forecast.theses` RPC: returns {theses, factors} standalone (no full
  forecast workspace), the data contract a dedicated TUI/web thesis overlay consumes.
- `thesis_dashboard` tool action: same payload for the agent.

Verified on the live ledger (2 theses + 1 factor render). Tests: build_thesis_summary
lists active theses, the tool action, the CLI json + table, and the gateway RPC.
Full tests/forecasting + tests/tools + gateway regression green (6829 passed).

Follow-on (UI polish, not blocking): a dedicated Ink thesis OVERLAY consuming
forecast.theses (master-list table + drill-down reusing ThesisDeskRead) — left out of
this commit as it's UX-heavy and not verifiable headlessly; the data surface + CLI are
usable now and theses still render in the existing TUI lens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
teddyjfpender added a commit to teddyjfpender/superforecasting-agent that referenced this pull request Jun 24, 2026
@teddyjfpender @claude
fix(forecast): address adversarial-review findings on the 4 TODOs (1 …
0ab5090 
…high + 10 med + lows)

A multi-agent review of NousResearch#150/NousResearch#138/NousResearch#137/NousResearch#139 confirmed 21 findings. Fixed:

HIGH (NousResearch#139): forecast.theses ran inline on the gateway dispatch thread while doing
per-thesis N+1 work — it can stall interrupt/approval RPCs on a large book. Added it to
_LONG_HANDLERS (thread pool) + a routing regression test. Also: one shared ForecastLedger
per call (was two), error code 5008 (was a colliding 5021), _num excludes bool, and the
CLI dashboard now renders factors too (parity with the RPC/tool; matches its docstring).

NousResearch#150 (vote-share intervals): intervals silently dropped on the fraction-scale path
(the PMF branch never attached them) — now attached on both branches. Interval lookup
now tolerates case/whitespace divergence from the scorer/hook key normalization. And the
intervals (always percentage-points) are rescaled to the payload's scale, so a
fraction-scale share renders `0.70 [0.50-0.85]` not `0.70 [50-85]`.

NousResearch#138 (readiness benchmarks): evaluating with the default --last window or a --dataset
filter could hide the freshly-run suite (a closed gap looked reopened / zero closed) —
the improve path now evaluates over ALL runs, unfiltered. Dropped `naive` (can't beat
baselines, so it can't close the edge gap). The tool mirror wraps the run in try/except.

NousResearch#137 (cycle --agent): selection now filters on alert.scope_type == "question" (a
domain/topic/portfolio alert's scope_ref is not a question; this also keeps a
question-scoped domain_error_profile_applies trigger). --max-questions now counts
processed (expensive) LLM runs, not just commits, so it actually caps. --max-iterations
0 is honored (was coerced to 12).

Deferred (noted): a per-candidate central-within-interval COMMIT gate (a new backend
invariant mirroring central_within) — warrants its own pass. New tests cover every fix
above. Full tests/forecasting + tests/tools + gateway regression green (6834 passed);
TUI type-check + 99 chart/workspace tests green; bundle rebuilt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
teknium1 pushed a commit that referenced this pull request Jun 28, 2026
@kshitijk4poor @teknium1
feat(logging): opt-in HTTP/WS body capture to an isolated, share-excl…
ca52dd8 
…uded gui_bodies.log

Stacked on #49003. That PR added always-on metadata (method/path/status/
latency + WS lifecycle) to the gui surface. This adds the heavy diagnostic
tier — actual HTTP request bodies and PTY/WebSocket frames — for the hard
dashboard/TUI bugs where metadata alone isn't enough.

Body content can carry conversation data, so this is opt-in and built to be
structurally incapable of leaking into a shared debug report (see #22016):

- New config logging.capture_bodies (default false), surfaced in the dashboard
  / hermes tools config UI via _SCHEMA_OVERRIDES with a warning description.
- When enabled, bodies go to a SEPARATE gui_bodies.log written by a dedicated
  logger (hermes_body_capture, propagate=False) that is deliberately NOT a
  member of any COMPONENT_PREFIXES. Four structural guarantees, all tested:
    1. not under any component prefix  -> never lands in gui.log / agent.log
    2. not in hermes_cli/logs.py LOG_FILES -> not tailable via --- ~/.hermes/logs/agent.log (last 50) ---
2026-06-19 18:48:30,316 INFO [20260619_173001_f45949] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=288583 out=529 total=289112 latency=10.6s cache=284912/288583 (99%)
2026-06-19 18:48:30,318 INFO [20260619_173001_f45949] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=4/16 budget=4/16 tool_turns=110 last_msg_role=assistant response_len=1474 session=20260619_173001_f45949
2026-06-19 18:48:30,325 INFO [20260619_173001_f45949] run_agent: OpenAI client closed (agent_close, shared=True, tcp_force_closed=0) thread=bg-review:6349795328 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:30,652 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-747 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:30,653 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #139: model=anthropic/claude-opus-4.8 provider=openrouter in=243960 out=991 total=244951 latency=11.5s cache=242196/243960 (99%)
2026-06-19 18:48:31,348 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.69s, 161 chars)
2026-06-19 18:48:31,384 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:51,510 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-749 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:51,511 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #140: model=anthropic/claude-opus-4.8 provider=openrouter in=245038 out=1783 total=246821 latency=20.1s cache=243477/245038 (99%)
2026-06-19 18:48:52,215 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (0.70s, 153 chars)
2026-06-19 18:48:52,245 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:59,489 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-751 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:48:59,490 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #141: model=anthropic/claude-opus-4.8 provider=openrouter in=246873 out=493 total=247366 latency=7.3s cache=244127/246873 (99%)
2026-06-19 18:49:13,666 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (14.17s, 979 chars)
2026-06-19 18:49:13,692 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:22,930 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-753 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:22,932 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #142: model=anthropic/claude-opus-4.8 provider=openrouter in=247686 out=548 total=248234 latency=9.3s cache=245109/247686 (99%)
2026-06-19 18:49:23,254 INFO [20260619_153431_51fd01] agent.tool_executor: tool patch completed (0.10s, 1394 chars)
2026-06-19 18:49:23,287 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:26,661 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-762 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:26,662 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #143: model=anthropic/claude-opus-4.8 provider=openrouter in=248814 out=104 total=248918 latency=3.4s cache=246934/248814 (99%)
2026-06-19 18:49:27,958 INFO [20260619_153431_51fd01] agent.tool_executor: tool terminal completed (1.29s, 14487 chars)
2026-06-19 18:49:27,984 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:43,991 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-764 (_call):6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:43,992 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #144: model=anthropic/claude-opus-4.8 provider=openrouter in=255375 out=938 total=256313 latency=16.0s cache=247771/255375 (97%)
2026-06-19 18:49:44,087 INFO [20260619_153431_51fd01] agent.conversation_loop: Turn ended: reason=text_response(finish_reason=stop) model=anthropic/claude-opus-4.8 api_calls=36/90 budget=31/90 tool_turns=129 last_msg_role=assistant response_len=2300 session=20260619_153431_51fd01
2026-06-19 18:49:44,112 INFO run_agent: OpenAI client created (agent_init, shared=True) thread=bg-review:6421311488 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:44,454 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='Review the conversation above and update the skill library. Be ACTIVE — most ses...'
2026-06-19 18:49:44,573 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:54,258 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-765 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:49:54,259 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #1: model=anthropic/claude-opus-4.8 provider=openrouter in=258322 out=423 total=258745 latency=9.8s cache=248822/258322 (96%)
2026-06-19 18:49:54,360 INFO [20260619_153431_51fd01] agent.tool_executor: tool skills_list completed (0.10s, 21152 chars)
2026-06-19 18:49:54,383 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:02,705 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-766 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:02,706 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #2: model=anthropic/claude-opus-4.8 provider=openrouter in=266258 out=313 total=266571 latency=8.3s cache=258320/266258 (97%)
2026-06-19 18:50:02,814 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.11s, 111769 chars)
2026-06-19 18:50:02,836 INFO [20260619_153431_51fd01] tools.tool_result_storage: Persisted large tool result: skill_view (toolu_01G7Zvw8ttjsUkomENppFu5T, 111769 chars -> /var/folders/p5/nqn3gs293rv3wtvf01pl9_vr0000gn/T/hermes-results/toolu_01G7Zvw8ttjsUkomENppFu5T.txt)
2026-06-19 18:50:02,861 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:12,687 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-769 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:12,688 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #3: model=anthropic/claude-opus-4.8 provider=openrouter in=267367 out=386 total=267753 latency=9.8s cache=258694/267367 (97%)
2026-06-19 18:50:12,749 INFO [20260619_153431_51fd01] agent.tool_executor: tool skill_view completed (0.06s, 22856 chars)
2026-06-19 18:50:12,776 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:18,989 INFO [20260619_153431_51fd01] agent.turn_context: conversation turn: session=20260619_153431_51fd01 model=anthropic/claude-opus-4.8 provider=openrouter platform=cli history=310 msg='yes'
2026-06-19 18:50:19,032 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:21,530 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-770 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:21,531 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #4: model=anthropic/claude-opus-4.8 provider=openrouter in=276591 out=415 total=277006 latency=8.8s cache=266515/276591 (96%)
2026-06-19 18:50:21,585 WARNING [20260619_153431_51fd01] agent.tool_executor: Tool skill_view returned error (0.05s): {"success": false, "error": "File 'references/stacked-feature-prs.md' not found in skill 'incremental-architecture-refactor'.", "available_files": {}, "hint": "Use one of the available file paths list
2026-06-19 18:50:21,613 INFO run_agent: OpenAI client created (chat_completion_stream_request, shared=False) thread=Thread-773 (_call):6150942720 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:36,019 INFO run_agent: OpenAI client closed (stream_request_complete, shared=False, tcp_force_closed=0) thread=Thread-772 (_call):12901707776 provider=openrouter base_url=https://openrouter.ai/api/v1 model=anthropic/claude-opus-4.8
2026-06-19 18:50:36,020 INFO [20260619_153431_51fd01] agent.conversation_loop: API call #145: model=anthropic/claude-opus-4.8 provider=openrouter in=256317 out=997 total=257314 latency=17.0s cache=256311/256317 (100%)
    3. not in debug.py _capture_default_log_snapshots() -> NEVER uploaded by
       ⚠️  This will upload the following to a public paste service:
  • System info (OS, Python version, Hermes version, provider, which API keys
    are configured — NOT the actual keys)
  • Recent log lines (agent.log, errors.log, gateway.log, desktop.log — may
    contain conversation fragments and file paths)
  • Full agent.log, gateway.log, and desktop.log (up to 512 KB each — likely
    contains conversation content, tool outputs, and file paths)

Pastes auto-delete after 6 hours.

Collecting debug report...
Uploading...

Debug report uploaded:
  Report  https://paste.rs/nnfZj

  (failed to upload: agent.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, gateway.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>, desktop.log: Failed to upload to any paste service:
  paste.rs: HTTP Error 500: Internal Server Error
  dpaste.com: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1016)>)

⏱  Pastes will auto-delete in 6 hours.
To delete now:  hermes debug delete <url>

Share these links with the Hermes team for support.
    4. still redacted via RedactingFormatter as defence-in-depth
- Disabled state attaches a NullHandler and sets the level above CRITICAL, so
  _capture_body() is a cheap no-op (single isEnabledFor check) on the hot path.
  Captured bodies are truncated to 4096 bytes. request.body() is Starlette-
  cached, so reading it in the access middleware does not consume the stream
  for downstream handlers.

Capture sites: HTTP request body (access middleware), PTY in/out frames.

Tests (tests/test_hermes_logging.py::TestBodyCaptureOptIn): disabled-by-default
creates no file and captures nothing; enabled writes to gui_bodies.log and the
payload is ABSENT from gui.log; large bodies truncate; the body logger is
isolated from every component; and the body file is excluded from both
LOG_FILES and the debug-share snapshot set.
@queeph queeph mentioned this pull request Jul 27, 2026
Closed
7 tasks
Meraniya pushed a commit to Meraniya/hermes-agent that referenced this pull request Aug 6, 2026
@dizhaky @claude
docs(website): register four orphaned guides in the sidebar (NousRese…
74bc41f 
…arch#139)

website/sidebars.ts lists the Guides category by hand. Four pages had never
been added to it: google-gemini, local-ollama-setup, minimax-oauth, and
pipe-script-output. Each one builds and resolves by direct URL but appears
nowhere in navigation. onBrokenLinks: 'warn' means this never failed a build,
which is how four of them accumulated without anyone noticing.

Placed by subject rather than appended to the end: local-ollama-setup beside
local-llm-on-mac, pipe-script-output beside cron-script-only, and the two
provider guides inside the existing aws-bedrock / azure-foundry /
xai-grok-oauth block.

website/docs/guides/ holds 29 .md files and the sidebar now lists 29 unique
guide ids, so the two sets match exactly — no orphans, no duplicates.

Follow-up to NousResearch#138, which fixed the same failure mode for one new guide.


Claude-Session: https://claude.ai/code/session_01Jy1tjok1XKgpTUK69b8dKT

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Unsafe context length assumption

2 participants

@Bartok9 @teknium1