Skip to content

fix(daemoncraft): wire embodiment toolset + daemoncraft platform into resolver chain - #7

Open
Pablomonte wants to merge 244 commits into
nicoechaniz:mainfrom
Pablomonte:fix/daemoncraft-toolset-wiring
Open

fix(daemoncraft): wire embodiment toolset + daemoncraft platform into resolver chain#7
Pablomonte wants to merge 244 commits into
nicoechaniz:mainfrom
Pablomonte:fix/daemoncraft-toolset-wiring

Conversation

@Pablomonte

Copy link
Copy Markdown

Summary

The DaemonCraft canonical install (Path B, the embodied_plan single-body-tool architecture introduced 2026-05-09) is broken on a fresh origin/main install in a non-obvious way. Three companion hooks are missing from the toolset-resolver chain, so:

  • The model receives tools=[] on every API call to Kimi-K2.6 (or any provider)
  • It emits the tool invocation as raw text in the Minecraft chat:
    <AsciiProbe> ok. reviso. embodied_plan:1 {"intent":"Check my inventory…","autonomy_level":2,"deadline_seconds":30}
    
  • The gateway log shows tool_turns=0 every turn, the embodied service at :7790 is never hit, and nothing physical happens.

This PR adds the three missing wiring entries. They are pure additions to the respective registries — no behavior change for any other platform or toolset.

The chain that's broken

Chat → daemoncraft platform                                        ← OK (gateway/config.py:112, 78a7c23e6)
     → ¿qué toolsets enable para esta platform?
       → PLATFORMS.get('daemoncraft')                              ← MISSING (patch 1)
       → platform_toolsets.daemoncraft = [embodiment, …]
     → ¿'embodiment' es un toolset configurable válido?
       → CONFIGURABLE_TOOLSETS                                     ← MISSING (patch 2) ← load-bearing
     → expand toolset to concrete tools
       → TOOLSETS['embodiment'] → ['embodied_plan']                ← MISSING (patch 3)
     → tools=[embodied_plan] → send to model
     → model emits structured tool_call → dispatch to /intent

Without any of the three, the chain breaks at its link. The user sees JSON in the chat, no action happens, and there's no error message — the failure mode is silent text leak.

The three patches (one-line each, except #3)

1. hermes_cli/platforms.py — register daemoncraft in the CLI platform catalog

("daemoncraft", PlatformInfo(label="⛏️  DaemonCraft", default_toolset="embodiment")),

Platform.DAEMONCRAFT is in gateway/config.py (since 78a7c23) but the CLI-side hermes_cli/platforms.py:PLATFORMS is a separate OrderedDict consumed by _get_platform_tools() and the TUI menus. Without the entry, PLATFORMS.get('daemoncraft') returns None, the resolver synthesizes a non-existent hermes-daemoncraft toolset name, and zero toolsets resolve.

2. hermes_cli/tools_config.py — list embodiment as a configurable toolset (load-bearing)

(\"embodiment\", \"🤖 DaemonCraft Embodied\", \"embodied_plan — single body tool dispatched to Gemma-Andy\"),

This is the patch that flips the gate. _get_platform_tools filters explicit profile toolsets by configurable_keys:

configurable_keys = {ts_key for ts_key, _, _ in CONFIGURABLE_TOOLSETS}
has_explicit_config = any(ts in configurable_keys for ts in toolset_names)
if has_explicit_config:
    enabled_toolsets = {ts for ts in toolset_names
                        if ts in configurable_keys
                        and _toolset_allowed_for_platform(ts, platform)}

The canonical DaemonCraft profile lists [embodiment, clarify, messaging]. clarify and messaging are in configurable_keys so has_explicit_config=True, but embodiment itself is dropped because it isn't. Result: the agent runs with {clarify, messaging} only, without embodied_plan — the entire body interface is invisible to the model.

3. toolsets.py — add the static embodiment toolset entry

"embodiment": {
    "description": "DaemonCraft body orchestration — exposes the single canonical tool `embodied_plan` …",
    "tools": ["embodied_plan"],
    "includes": []
},

tools/embodied_plan_tool.py:445 registers the toolset dynamically (registry.register(toolset=\"embodiment\")), and get_toolset() has a fallback to the dynamic registry — but the fallback labels it \"Plugin toolset\" / \"MCP server\" and code paths that read TOOLSETS.get(\"embodiment\") directly (without falling back) treat it as absent. The explicit entry makes it a first-class built-in, consistent with minecraft legacy and every other toolset.

Smoke test

Profile config (canonical DaemonCraft):
```yaml
model:
default: kimi-k2.6
provider: kimi-coding
toolsets: [embodiment, messaging]
platform_toolsets:
daemoncraft: [embodiment, clarify, messaging]
platforms:
daemoncraft:
enabled: true
extra:
bot_api_url: http://localhost:3001
bot_username: AsciiProbe
embodied_service_url: http://localhost:7790
```

Then:
```bash
HERMES_HOME=~/.hermes/profiles/daemoncraft-base hermes gateway run -v
```

Send @AsciiProbe contame qué ves alrededor in the Minecraft chat:

Before (origin/main) After (this PR)
Gateway log tool_turns 0 ≥1
Embodied service /intent request not hit hit, 8.4s Ollama round-trip
Bot chat response raw JSON: `embodied_plan:1 {"intent":...}` natural prose: `"vi carbón cerca. ¿quieres que mine un poco, o preferís que te siga?"`
tools=[] in dumped request count: 0 count: 1 (`embodied_plan`)

Reproduced both sides on a fresh `origin/main` clone with the DaemonCraft profile from `agents/embodied-service/profile-templates/` (mirror in the daemoncraft repo).

No-op for other platforms

All three patches are pure additions to their respective dicts:

  • New `PlatformInfo` row in `PLATFORMS` OrderedDict
  • New 3-tuple entry in `CONFIGURABLE_TOOLSETS`
  • New key in `TOOLSETS` dict

No existing entries modified, no logic branches changed, no dependencies added. Telegram, Discord, CLI etc. resolve identically before and after.

Why this was missed

The commits that introduced the canonical DaemonCraft architecture landed in pieces:

  • Platform enum: `78a7c23e6 fix(gateway): register daemoncraft platform + handle kimi tool_choice with thinking`
  • Tool registration: `tools/embodied_plan_tool.py:445` does `registry.register(toolset="embodiment")`

But the integration through the CLI-side toolset/platform resolver (which is what's actually consulted at gateway runtime) was never wired. The model's "tool emitted as raw text" symptom is silent — no error log surfaces in either the gateway or the embodied service — so it slips past manual testing.

Test plan

  • Reproduce `tool_turns=0` + JSON text leak on a fresh `origin/main` install
  • After patches: confirm `tool_turns≥1`, embodied service hit, natural chat response
  • Confirm `hermes tools list` now shows `embodiment` in built-in toolsets (previously absent)
  • Confirm `hermes profile use daemoncraft-base + hermes gateway run` end-to-end with canonical profile

When tui.history_nav_requires_empty_input is true, Up/Down arrows
no longer cycle history/queue if the composer input has text.
This makes multiline editing less surprising: arrow-up on the first
line of a non-empty buffer stays in the buffer instead of jumping
to the previous history item.

Also adds ConfigTuiConfig to gatewayTypes so the TUI can read the
setting from config.get full responses.
…anscript corruption

Adds _sanitize_unanswered_tool_calls() to backfill synthetic role=tool
results for any assistant tool_calls that weren't answered before an
interrupt or error exits the loop. This prevents the next API call from
failing with a missing tool response error.

Also removes the duplicated inline logic from the outer-loop error
handler and calls the helper from _persist_session() so every exit
path guarantees an API-valid transcript.
When a user interrupts with an image paste/attachment, the payload is a
tuple (text, images). The post-interrupt re-queue code did
'\n'.join(all_parts) which crashed with:

  TypeError: sequence item 0: expected str instance, tuple found

Split text and images from each part, combine text with join, and
preserve image attachments so process_loop can unpack them normally.
Recovers input_max_lines, collapse_large_pastes, history_nav_requires_empty_input,
and show_full_input that were accidentally dropped during Kimi cleanup (bbb91391).
…ture pinning

- Reads Kimi CLI tokens from ~/.kimi/credentials/kimi-code.json
- resolve_kimi_coding_runtime_credentials(): OAuth first, refresh token support, fallback to KIMI_API_KEY
- kimi_coding_default_headers(): proper User-Agent and X-Msh-* headers for coding endpoint
- kimi_coding_required_temperature(): pins temperature to 0.6 for kimi-k2.6 on coding endpoint
- 401 retry with token refresh before aborting
- Integrates into run_agent.py and auxiliary_client.py
The upstream _fixed_temperature_for_model() already omits temperature for
Kimi models, letting the server choose the correct value. Our manual
0.6 pinning was unnecessary and could conflict with server-side mode
selection (thinking vs non-thinking). Verified working without it.
The auxiliary client's _refresh_provider_credentials() handled auth
refresh for Codex, Nous, and Anthropic, but not for Kimi. When the
Kimi OAuth token expired, auxiliary calls (memory flush, compression,
session search) failed with HTTP 401 while the main client recovered
automatically.

Add a kimi-coding / kimi-coding-cn case that calls
resolve_kimi_coding_runtime_credentials(force_refresh=True) and evicts
the cached auxiliary client, mirroring the existing provider refresh
paths.

Fixes auxiliary memory flush failures when using Kimi OAuth.
# Conflicts:
#	cli.py
#	hermes_cli/config.py
# Conflicts:
#	ui-tui/src/app/interfaces.ts
#	ui-tui/src/app/uiStore.ts
#	ui-tui/src/app/useConfigSync.ts
- Add protect_first_n to DEFAULT_CONFIG['compression'] (default 3, allows 0)
- Add compression.prompt {preamble, template} for custom summary prompts
- Bump config version 22 -> 23
- Extract hardcoded prompt constants in ContextCompressor to module-level defaults
- Pass custom preamble/template through ContextCompressor constructor
- Read protect_first_n and prompt config in run_agent.py, pass to compressor
- Update status display to show protect_first_n
- Add tests for protect_first_n=0 and custom prompts
- Always preserve system prompt as literal even when protect_first_n=0
- Fix missing import resolve_kimi_coding_runtime_credentials in runtime_provider.py
- Update wiki and website docs
# Conflicts:
#	hermes_cli/config.py
#	tests/agent/test_auxiliary_client.py
#	tests/hermes_cli/test_runtime_provider_resolution.py
#	ui-tui/src/app/uiStore.ts
#	ui-tui/src/app/useConfigSync.ts
# Conflicts:
#	hermes_cli/config.py
…_messages transport

PR NousResearch#12846 enabled Anthropic prompt caching for third-party gateways,
but gated it on is_claude, which excluded providers like MiniMax
that serve their own model families (MiniMax-M2.7, etc.) through the
native Anthropic protocol.

MiniMax documents full cache_control support on its /anthropic
endpoints (global and China). This patch adds MiniMax detection to
_anthropic_prompt_cache_policy() using:

- Built-in provider id (minimax, minimax-cn), or
- Known Anthropic-compatible hostname (api.minimax.io,
  api.minimaxi.com)

Both paths receive the native cache_control layout.

Refs: NousResearch#8294 (related, but only covered Claude-named models on
third-party gateways).
Closes NousResearch#17332
AIAgent.__init__ now detects provider=minimax/minimax-cn and defaults to:
- api_mode='anthropic_messages' (was 'chat_completions')
- base_url='https://api.minimax.io/anthropic' or 'https://api.minimaxi.com/anthropic'

This ensures prompt caching (and all other Anthropic-protocol features)
work out of the box for AIAgent users, not just CLI users.

Previously, AIAgent(provider='minimax') fell through to chat_completions
because base_url was empty and there was no provider-name detection for
MiniMax in the api_mode resolution logic. The CLI already resolved this
correctly via runtime_provider.py; this change mirrors that behaviour in
the low-level agent constructor.

Tests added:
- 5 new tests in test_minimax_provider.py covering defaults, cn variant,
  explicit base_url preservation, explicit api_mode override, and
  prompt caching enabled by default.
- 2 new tests in test_anthropic_prompt_cache_policy.py covering empty
  base_url with provider=minimax/minimax-cn.
…base_url

PR NousResearch#17425 (merged) enabled prompt caching for MiniMax models on the
anthropic_messages transport, but users still had to manually configure
both api_mode and base_url to actually benefit from it.

This patch makes the defaults ergonomic:

- AIAgent.__init__ now auto-detects provider=minimax / minimax-cn and
  defaults to api_mode=anthropic_messages + the correct /anthropic base_url
  (global or China endpoint respectively).
- .env.example suggests the /anthropic endpoints instead of /v1.
- Explicit base_url or api_mode are preserved when the user sets them.

Tests: 5 new cases covering both providers, explicit overrides, and
prompt-caching flags.

Refs: NousResearch#17332, NousResearch#17333, NousResearch#17425
Documents the current working prototype where Grok web generates
shell snippets (CMD: prefix) that the proxy executes locally and
pipes back. Includes open questions and next steps.
Copied STRATEGY_MAP and NEEDS_SETUP from DaemonCraft gemma_policy.py.
embodied_plan_tool.py now includes strategy and needs_setup in
mitigation output so callers can auto-select execution path.

Navigation → embodied_plan with narrow tools.
Building → embodied_plan with needs_setup flag.
Fallback → mc_direct when Andy fails.
# Conflicts:
#	CHANGELOG.md
#	agent/auxiliary_client.py
#	gateway/platforms/daemoncraft.py
#	tests/tools/test_embodied_plan_tool.py
#	tools/embodied_plan_tool.py
Gateway writes chat events to {bot}-events.jsonl so the agent loop
can inject them as context. The gateway continues to handle chat
normally — this is a parallel mirror, not a replacement.

Part of singleton session architecture.
Gateway writes chat events to {bot}-events.jsonl so the agent loop
can inject them as context. The gateway continues to handle chat
normally — this is a parallel mirror, not a replacement.

Part of singleton session architecture.
nicoechaniz and others added 2 commits May 17, 2026 10:36
mc_bit's description said "Use this INSTEAD of mc_perceive(type='nearby')"
and started with "Perceive a 3D chunk..." — this caused tool selection loops
where the LLM tried to call mc_perceive but the system routed to mc_bit.

Changes:
- Description: "Perceive" → "Scan a 3D volume as raw text"
- Removed "Use this INSTEAD of mc_perceive"
- Added clear separation: RAW block scanner vs high-level perception
- Error message: removed mc_perceive reference
- Short description synced
… resolver chain

The DaemonCraft canonical install (Path B, 2026-05-09) registered
`embodied_plan` as a tool with `toolset="embodiment"` in
`tools/embodied_plan_tool.py`, and the gateway/config.py enum entry for
the `daemoncraft` platform was added in 78a7c23. But three companion
hooks were missing, leaving the toolset-resolver chain broken end-to-end
for any clean install: Kimi-K2.6 received `tools=[]` in every API call,
the model emitted the tool invocation as raw text in the Minecraft chat
(e.g. `embodied_plan:1 {"intent":"..."}`), and nothing dispatched.

The three missing pieces (independent files, additive only):

1. `hermes_cli/platforms.py` — add `daemoncraft` to the PLATFORMS
   OrderedDict so `_get_platform_tools` finds a default toolset
   (`embodiment`) when a profile doesn't specify `platform_toolsets.daemoncraft`
   explicitly.  Without this, the resolver synthesizes `hermes-daemoncraft`
   which doesn't exist, and zero toolsets resolve.

2. `hermes_cli/tools_config.py` — add `("embodiment", ...)` to
   CONFIGURABLE_TOOLSETS.  `_get_platform_tools` builds
   `configurable_keys` from this list and uses it as a filter in the
   `has_explicit_config` branch:
       enabled_toolsets = {ts for ts in toolset_names
                           if ts in configurable_keys ...}
   When a daemoncraft profile lists `[embodiment, clarify, messaging]`,
   `clarify` and `messaging` are in `configurable_keys` so
   `has_explicit_config=True`, but `embodiment` is dropped because it
   isn't.  Result: the agent runs with `{clarify, messaging}` only, and
   `embodied_plan` is invisible to the model.  This was the load-bearing
   gap.

3. `toolsets.py` — add the `"embodiment"` entry to the static TOOLSETS
   dict.  `tools/embodied_plan_tool.py:445` does register the toolset
   dynamically at import via `registry.register(toolset="embodiment")`,
   and `toolsets.py:get_toolset()` has a registry fallback, but the
   fallback labels it `"Plugin toolset"` / `"MCP server"` and code paths
   that read `TOOLSETS.get("embodiment")` directly (without falling back
   to the dynamic registry) treat it as absent. The explicit entry makes
   it a first-class built-in toolset, consistent with how every other
   tool is wired.

Result: with all three, a fresh install of `nicoechaniz/hermes-agent`
`origin/main`, with the canonical DaemonCraft profile (`toolsets:
[embodiment, messaging]` + `platform_toolsets: { daemoncraft:
[embodiment, clarify, messaging] }`), produces real structured
`tool_calls` (`tool_turns≥1`) instead of `text_response` leaks, and
`embodied_plan` reaches the embodied service at :7790 as designed.

Smoke test
- Send `@<bot> contame qué ves alrededor` in MC chat.
- Before: bot responds with `embodied_plan:N {"intent":"..."}` literal
  text; gateway log shows `tool_turns=0`.
- After: bot responds in natural Spanish/English with the scan result;
  gateway log shows `tool_turns=1`, `tool embodied_plan completed`, and
  a `POST http://localhost:7790/intent` round-trip.

No new dependencies, no behavior change for other platforms / toolsets
(all three are pure additions in their respective dicts).
@Pablomonte
Pablomonte force-pushed the fix/daemoncraft-toolset-wiring branch from 29b1700 to 31281c5 Compare May 17, 2026 14:20
Pablomonte added a commit to Pablomonte/DaemonCraft that referenced this pull request May 17, 2026
… box

The `place` action in agents/bot/server.js (the canonical
embodied-side dispatcher target for `place_block` tool calls) had a
load-bearing failure mode: when the requested (x, y, z) was inside the
bot's own bounding box, the server silently rejected the placement and
the materialize-verification at the bottom surfaced an opaque
"did not materialize. Retry or choose different coordinates." error.

The captain LLM then had no useful information — Gemma-Andy retried
the same coordinates, hit the same silent reject, and after a few
loops told the player "no puedo construir, el sistema sigue roto".

Root cause: the action accepted (x, y, z) as the literal target and
walked the placement neighbour search without first checking whether
the bot itself occupied that cell. Servers reject `_genericPlace`
calls that would clip the player's hitbox, but the rejection comes
back as a no-op (no exception), so the only signal was the
post-place blockAt() check failing.

Fix: before the existing occupied / neighbour-search logic, check
whether the floor(target) cell equals the bot's feet cell (floor(pos))
or head cell (feet+1). If yes, pick the first candidate from
[(+x), (-x), (+z), (-z), feet+2 above-head, feet-1 below-feet] that
is:
  - not in the bot's own occupied cells
  - air / cave_air
  - has at least one solid neighbour to place against
…and rewrite (x, y, z) to that. Log the shift so operators can see it
in dispatch traces.

If no candidate fits, raise a clear error explaining the placement
cannot proceed without first moving the bot — not the opaque
"did not materialize" the captain previously had to guess at.

End-to-end repro before the fix:

  Captain (Kimi-K2.6) → embodied_plan(intent=
    "Place a crafting_table near my position, then craft a stone_shovel.")
  → Gemma-Andy plan: [place_block(crafting_table, x=bot.x, y=bot.y, z=bot.z),
                      craft_item(stone_shovel)]
  → bot place action: target == bot feet cell → silent server reject
  → materialize check fails → throws "did not materialize"
  → craft_item fails: "needs crafting table nearby"
  → captain to player: "no puedo construir"

After: place action detects the overlap, shifts target to the cell
immediately east of the bot (or first viable adjacent), places the
crafting_table, and the follow-up craft_item finds it within the
4-block radius.

This is one of three concurrent layers being hardened for "build with
confidence" alongside the canonical-loop policy patches
(nicoechaniz#16 — craft category + narrow inventory_query)
and the canonical-loop toolset wiring
(nicoechaniz/hermes-agent#7 + nicoechaniz#8).

No new dependencies, no API change. Pure additive guard inside the
existing place action body.
@nicoechaniz
nicoechaniz force-pushed the main branch 2 times, most recently from 26d4c51 to b54dc71 Compare May 24, 2026 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants