fix(discord): distinguish 30032 cap error from generic sync failures - #5
Open
itsXactlY wants to merge 41 commits into
Open
fix(discord): distinguish 30032 cap error from generic sync failures#5itsXactlY wants to merge 41 commits into
itsXactlY wants to merge 41 commits into
Conversation
added 30 commits
April 10, 2026 17:35
Local semantic memory with knowledge graph, spreading activation, and auto-connections. Follows MemoryProvider ABC pattern from upstream. Features: - Semantic search via vector embeddings (hash/tfidf/sentence-transformers) - Knowledge graph with automatic connection discovery - Spreading activation for exploring connected ideas - Fully offline — no API keys required - Auto-detects best embedding backend (CUDA-accelerated if available) Tools: neural_remember, neural_recall, neural_think, neural_graph Config: memory.provider=neural in config.yaml feat(neural): add save_config and provider tests - save_config() writes to config.yaml under memory.neural - 37 tests covering config, schemas, handlers, prefetch, sync_turn, system prompt, lifecycle hooks, and availability - All 127 memory plugin tests pass feat(neural): add on_pre_compress to save context before compression Before /compress or /new discards old messages, on_pre_compress now: - Scans user/assistant pairs about to be compressed - Extracts meaningful facts (skips garbage/meta) - Stores as 'pre-compress' memories in the knowledge graph - Returns summary for the compressor to preserve +3 tests (saves exchanges, skips garbage, handles empty). feat(neural): add initial context prefetch, cpp bridge fix, mssql_store - _load_initial_context() queries summaries, recent memories, graph hubs - system_prompt_block() includes historical context from turn 0 - prefetch() returns initial context on first call (no more empty first turn) - cpp_bridge.py: fixed library search paths - mssql_store.py: added for optional MSSQL backend - +2 tests for initial context + prefetch - 42 tests total, all pass
- dream_engine.py: NREM (replay), REM (bridge discovery), Insight (communities) - dream_mssql_store.py: MSSQL backend with .env credentials (no hardcoded passwords) - dream_worker.py: Standalone full-stack worker (MSSQL + sentence-transformers) - __init__.py: neural_dream + neural_dream_stats tools, auto-start on init - test_dream_engine.py: 23 tests covering all phases + SQLite backend Credentials: env vars > .env (~/.hermes/.env) > config.yaml > defaults Model cache: ~/.neural_memory/models/ (shared with embed_provider)
…nto neural-memory-clean
_load_initial_context was too restrictive: only matched 'session-summary' labels or 'Session topics' text, missing all turn-N and recovered-session entries. Now directly queries last 8 non-memory entries from SQLite, filters garbage/short content, deduplicates. Also added logging.
- _initial_context no longer cleared by prefetch() — stays available - Context injected in BOTH system_prompt_block() AND prefetch() user message - Reordered: dream engine starts before _load_initial_context - Added debug logging for initial context loading
…nto neural-memory-clean
- __init__.py: NeuralMemory(use_cpp=True) for SIMD-accelerated recall - System prompt shows stack info (C++ SIMD, Cython) - fast_ops.so: 66x cosine_similarity acceleration - cpp_bridge: fixed CSearchResult struct (segfault fix)
Prevents CUDA OOM when llama-server (13.7GB) occupies the GPU. Uses torch.cuda.mem_get_info() to check free VRAM.
Results now only contain: id, label, content (max 500/300 chars), similarity. Removes ~70K chars of embedding vectors and connection ID lists from prompt. Saves significant context window space on every tool call.
Before compression, neural memory now: 1. Extracts dominant topics from messages (keyword frequency + recency) 2. Weights topic-relevant exchanges as 'compress-focus' (salience 1.5x) 3. Weights off-topic exchanges as 'compress-bg' (salience 0.5x) 4. Summary shows ★ focused vs · background entries Creates perceptual bias — focused topics survive compression better. Works with /compress <topic> or auto-detected topic clusters.
1. CSearchResult 1024→4096 2. fast_ops cdef set fix 3. _initial_context thread-safe via store.get_all()
…6 tools - _load_config: env vars now take precedence over config.yaml (NEURAL_EMBEDDING_BACKEND was silently overridden by config.yaml) - test_neural_provider: updated schema count 4→6, fixed assertion text to match actual on_pre_compress output - All 65 tests green (42 provider + 23 dream engine)
Problem: NREM phase iterated 24K+ connections one-by-one, each calling individual SQL UPDATE + INSERT (log_connection_change). That's ~49K round-trips to MSSQL → hangs indefinitely. Fix: - batch_strengthen_connections(): executemany for activated edges - batch_weaken_connections(): single bulk UPDATE for all connections above threshold (no per-row loop) - Removed per-connection log_connection_change in NREM (too expensive) - Both SQLite and MSSQL backends implement batch methods Result: Full 3-phase dream cycle on MSSQL: NREM: 0.4s (24683 weakened, 153 pruned) REM: 0.1s Insights: 0.2s Total: 0.7s All 65 tests green.
Problem: Memory recall returned irrelevant results (raw conversation dumps, banner text, pre-compress logs). Initial context flooded with noise. Fix: - _extract_facts: structured format 'Topic: X\nResult: Y' instead of raw 'User: X\nAssistant: Y'. Skips boilerplate responses. - _is_garbage: extended patterns (banners, loading messages, UI box chars) - _NOISE_LABELS: filter 'pre-compress' from recall results - _load_initial_context: skip raw 'User: X\nAssistant: Y' dumps, count quality memories instead of blindly taking last 8 - _handle_recall: filter noise labels, min similarity 0.1, skip garbage - queue_prefetch: fetch 2x for filtering, min similarity 0.25, filter noise All 155 memory tests green.
757/978 memories were auto-generated cron reports (msg:hermes:hermes-agent:*)
flooding the DB. All recall results were health reports/backup logs instead of
actual useful memories.
- Added _NOISE_LABEL_PREFIXES = ('msg:',) to filter ALL auto-generated
gateway/cron messages from recall and initial context
- Added _is_noise_label() helper for consistent noise detection
- All 4 filter sites use the helper (_load_initial_context, queue_prefetch,
_handle_recall, on_pre_compress)
Result: initial context now shows structured memories (architecture notes,
benchmarks, config). Recall returns relevant results, not health reports.
When assistant says 'I don't know', 'no specific memory', 'can you remind me' etc., store as 'Question: X' instead of 'Topic: X\nResult: I don't know'. The TOPIC (what was asked) is still valuable for future search, but the non-answer won't pollute results. Also adds 'Question:' prefix for topic-only memories (vs 'Topic:' for substantive exchanges) for cleaner semantic distinction.
Problem: ctrl+x x5 kills process before sync_turn/pre_compress hooks run.
All conversation data lost.
Solution: Sponge Mode — absorb every message IMMEDIATELY via background queue.
Architecture:
User msg arrives → absorb_message('user', text) → Queue → Worker → SQLite
Asst response → absorb_message('assistant', text) → Queue → Worker → SQLite
Components:
- _sponge_queue: thread-safe queue.Queue (maxsize=100)
- _sponge_worker: daemon thread draining queue continuously
- _do_absorb(role, content): stores 'Q: ...' for user, 'A: ...' for assistant
- absorb_message(): non-blocking put_nowait, drops on full queue
- MemoryManager.absorb_message(): forwards to all providers
- run_agent.py: hooks at message arrival (line 7508) and response complete (line 9983)
Safety:
- Garbage filtering (_is_garbage)
- Non-answer detection (don't store 'I don't know')
- Content-aware deduplication (exact match, not just embedding similarity)
- Min length filter (skip < 10 chars)
- Non-blocking: put_nowait, drops on full queue
Crash resilience: even if process dies mid-response, user message is already
queued for absorption. Next session finds it in the DB.
155 memory tests green.
…nto neural-memory-clean
…nto neural-memory-clean
…bounced, self-versioning New files: - tools/snapshot_engine.py: Content-addressed snapshot engine - SHA-256 deduplication (same file stored once across N snapshots) - SQLite safe copy via backup API (handles WAL mode) - Debounced auto-snapshots (30s default, 15s for memory writes) - safe_run() transaction wrapper with rollback - Diff between snapshots - Smart pruning (keep last N + hourly + daily) - history.db index for fast queries - tests/test_snapshot_engine.py: 23 tests (all passing) - Roundtrip, dedup, SQLite safe copy, diff, prune, debounce, concurrency CLI integration: - /snapshot [list|create|rewind <id>|diff <a> <b>|prune|head] - /snap alias - CommandDef in commands.py Runtime hooks in run_agent.py (surgical, debounced): - After memory write (on_memory_write bridge) - After sequential tool execution batch - After concurrent tool execution batch All hooks are try/except wrapped — zero risk to existing behavior.
WAL (Write-Ahead Log): - wal_append() / wal_append_file() — append state changes to WAL - wal_flush() — mark entries as belonging to a snapshot - wal_replay() — crash recovery: restore unflushed entries - wal_unflushed() — list pending WAL entries - wal_prune() — auto-cleanup of old flushed entries - Zero data loss between debounced snapshots Branching: - create_branch(name, from_snapshot=) - switch_branch(name) — snapshots current state, restores branch HEAD - delete_branch(name) — protected: cannot delete main or active branch - list_branches() — shows all branches with HEAD snapshot - get_branch() — current branch name - Branch-protected pruning: NEVER deletes non-main branch snapshots Auto-Prune: - Default retention changed to 3 days (was 30) - Prune also cleans WAL entries >72h old CLI expanded: /snapshot branch — list branches /snapshot branch <name> — create branch /snapshot branch switch <name> /snapshot branch delete <name> /snapshot wal — show unflushed WAL entries /snapshot wal replay — crash recovery /snapshot list <branch> — filter by branch 41 tests (all green): WAL append/flush/replay/dedup/prune, branch CRUD/switch/protection/track, integration lifecycle.
Full explanation covering: - Why not Git (performance, locking, designed for code not state) - What gets snapshotted (state files, not dev code) - Architecture: content-addressed storage, WAL, branching - Debouncing strategy and auto-pruning (3-day default) - Complete CLI reference with all subcommands - Programmatic API reference - 5 detailed use cases (updates, config, crash recovery, debugging, risky ops) - Storage overhead analysis - Dateistruktur detail - SQLite safe copy internals - Performance benchmarks
Complete English documentation for the Runtime Snapshot Engine: - Why not Git (performance, locking, code vs state) - Architecture: content-addressed storage, WAL, branching - CLI reference with all subcommands - Programmatic API - 5 use cases (updates, config, crash recovery, debugging, risky ops) - Storage overhead, file structure, SQLite safe copy internals - Performance benchmarks, test coverage summary
…nto feat/snapshot-engine
…nto neural-memory-clean
… neural-memory-clean
… feat/snapshot-engine
… a fast path for cosine similarity search over pre-computed embeddings, while C++ Hopfield network serves as a fallback. This allows for sub-millisecond recall times on large memory stores, while maintaining compatibility with existing embedding formats and workflows. The GPU recall engine loads embeddings onto the GPU and performs cosine similarity search using PyTorch, while the C++ engine uses a custom Hopfield network implementation for recall. The NeuralMemory class integrates both engines and chooses the best available option for recall based on the system's capabilities and the state of the memory store.
Discord's hard 100-global-application-commands cap (HTTP 400, code 30032) shows up as a 'Slash command sync failed' log line — same generic message as every other sync error, with a full stack trace. On loaded installs this hides the actual cause and buries the one useful hint (the cap) under noise that looks like a real bug. Detect the cap error explicitly in _run_post_connect_initialization and emit a distinct, actionable warning that tells the operator what hit the cap and how to fix it (drop plugins / trim COMMAND_REGISTRY). Detection: prefer exc.code == 30032, fall back to status == 400 plus 'Maximum number of application commands reached' in exc.text so the check works against older discord.py forks / mocked transports. The broad except Exception that prevented the gateway from dying is preserved — this is a logging-clarity change, not a behavior change for the success path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On loaded installs, the Discord adapter occasionally hits the 100-global-application-commands cap and returns HTTP 400 / code 30032. Today this surfaces in the gateway log as:
…followed by a full stack trace — identical to every other sync failure (timeouts, network errors, random HTTP 500s). The actual cause is buried under noise that looks like a real bug, and the operator has no hint that the cap was hit or what to do about it.
Fix
Detect 30032 explicitly in
_run_post_connect_initializationand emit a distinct, actionable warning. ~30 lines total.Detection prefers
exc.code == 30032(set by discord.py's HTTPException), with a fallback tostatus == 400+"Maximum number of application commands reached"inexc.textso the check works against older discord.py forks / mocked transports.The broad
except Exceptionthat prevented the gateway from dying is preserved — this is a logging-clarity change, not a behavior change for the success path. Cap errors are still caught; they just produce a different log line now.Diff
113 lines added, 0 deleted, 1 commit. No churn.
Tests
test_post_connect_initialization_logs_cap_error_with_distinct_message"cap reached"log line and does NOT fall through to the generic"sync failed"branchtest_is_discord_command_cap_error_detects_30032What this does NOT do
_DISCORD_CMD_LIMIT = 100at line 1742) — that already exists onmain.except Exceptionthat prevents the cap error from killing the gateway — that's still there.It's purely a "make the log line say what's actually wrong" change.
Companion PR
A parallel PR exists against upstream
NousResearch/hermes-agent(PR NousResearch#48087) targeting the olderplugins/platforms/discord/adapter.pycode path that still exists in their tree.