Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
c8cad0d
feat(cuga-lite): native function-calling execution mode
segevshlomovIBM Sep 15, 2026
cafcca1
feat(cuga-lite): step discipline — one tool call per step, both modes
segevshlomovIBM Sep 15, 2026
dd3edea
fix(cuga-lite): function-calling review fixes — valid replay across t…
segevshlomovIBM Sep 15, 2026
948350b
fix(cuga-lite): app catalogue in the FC prompt, real-path fail-closed…
segevshlomovIBM Sep 15, 2026
a66aca4
fix(cuga-lite): fail the function-calling approval guard closed on ve…
segevshlomovIBM Sep 15, 2026
da9a584
fix(cuga-lite): address haroldship's review — native transcript acros…
segevshlomovIBM Sep 27, 2026
b1d13d0
fix(cuga-lite): function-calling correction cap is per turn; fences o…
segevshlomovIBM Sep 28, 2026
b91abea
fix(cuga-lite): read WatsonX's reasoning key on the function-calling …
segevshlomovIBM Oct 1, 2026
a9ea3ec
fix(cuga-lite): provider-safe tool names for native tool binding
segevshlomovIBM Oct 1, 2026
e7a25c6
fix(cuga-lite): map tool aliases back at the decode boundary
segevshlomovIBM Oct 5, 2026
e653ea1
refactor(cuga-lite): trim provider-safe tool names to the two seams
segevshlomovIBM Oct 7, 2026
2d86dc8
Merge remote-tracking branch 'origin/main' into fix/segevshlomov-prov…
segevshlomovIBM Oct 7, 2026
9cfae1a
test(cuga-lite): record the feature-off CodeAct snapshot
segevshlomovIBM Oct 9, 2026
1e9678b
feat(cuga-lite): run function-calling tool calls through the sandbox
segevshlomovIBM Oct 9, 2026
fd27513
test(cuga-lite): translator, sandbox-reply and equivalence tests for …
segevshlomovIBM Oct 9, 2026
dad9b38
docs(cuga-lite): function calling through the sandbox
segevshlomovIBM Oct 9, 2026
eb375ac
Merge remote-tracking branch 'origin/fix/segevshlomov-provider-safe-t…
segevshlomovIBM Oct 9, 2026
5a285f1
feat(cuga-lite): aliases, find_tools and provider-error recovery in F…
segevshlomovIBM Oct 9, 2026
5d102ee
fix(cuga-lite): record function-calling outcomes in a node-owned channel
segevshlomovIBM Oct 11, 2026
43c97d8
fix(cuga-lite): CodeRabbit round on the sandbox path
segevshlomovIBM Oct 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1173,6 +1173,45 @@ See `docs/design/pluggable-shortlister.md` for the full design.

</details>

<details>
<summary>🔁 Execution mode: CodeAct or native function calling</summary>

## What it is

By default CUGA Lite works in **CodeAct**: the model writes a Python block and the sandbox runs it. `cuga_lite_execution_mode = "function_calling"` switches to native tool calls: the model emits `tool_calls`, CUGA translates them into a short internal block that the same sandbox runs (same approval check, VERIFY gate, budgets, tracker, timeout and variables), and replies with one `ToolMessage` per call id, so the model reads the results before it calls again or answers. The block is never shown to the model or persisted; its transcript stays native. Tools are advertised through `bind_tools`; a short dedicated prompt replaces the CodeAct prompt.

**Step discipline** (`cuga_lite_step_discipline = "one_tool_per_step"`) works in both modes: only the first tool call of a step is attempted — success or error, that is the result the model reads before it decides the next call. CodeAct keeps the block's variables (including that first result); function-calling answers the extra calls with a "deferred" reply.

## Configuration

| Key | Default | Values |
|---|---|---|
| `cuga_lite_execution_mode` | `"codeact"` | `codeact` \| `function_calling` (alias `fc`) |
| `cuga_lite_step_discipline` | `"off"` | `off` \| `one_tool_per_step` |
| `cuga_lite_fc_prompt_fragments` | `[]` | Opt-in prompt lines for function-calling mode. `evidence_first`: call a tool before answering, never answer from prior knowledge |

Precedence, highest first: raw keys in `configurable` → per-invoke `invoke(..., execution_mode=..., step_discipline=...)` → constructor `CugaAgent(execution_mode=..., step_discipline=...)` → per-model runtime profile → `[advanced_features]` in `settings.toml` (also where `DYNACONF_ADVANCED_FEATURES__CUGA_LITE_EXECUTION_MODE` lands). An unknown value falls back to the default with a warning.

```python
agent = CugaAgent(tools=[...], execution_mode="function_calling", step_discipline="one_tool_per_step")
result = await agent.invoke("...") # native tool calls
result = await agent.invoke("...", execution_mode="codeact") # per-call override
```

## Gotchas

- **Tool-approval policies** apply exactly as in CodeAct: the approval check runs on the translated block, so a policy on a tool name pauses the run before the call and the approved call resumes into the sandbox.
- Bind mode `none` (the default) is upgraded in function-calling mode to advertise exactly the tools the sandbox could call, plus `find_tools` whenever the prompt advertises it (set `configurable["cuga_lite_bind_tools_include_find_tools"] = False` to opt out). Past `cuga_lite_bind_tools_max_count` (128) the bind-cap shortlister runs every turn; `[shortlister.bind_cap] strategy = "embedding"` avoids the extra LLM call.
- Tool names providers reject (over 64 characters, or with characters outside `[A-Za-z0-9_-]`) are bound under a provider-safe alias; the model calls the alias, the real tool runs, and the reply keeps the alias.
- A provider that rejects a malformed native call but returns the attempt (Groq `tool_use_failed`) gets it back as a real call; a retryable rejection is retried once, then the turn ends with a clear error.
- The bundled CodeAct few-shot demos are not sent in function-calling mode; pass your own through `configurable["mcp_few_shot_examples"]`.
- Tools run in-process and sequentially, on the local executor even when `e2b_sandbox` is on. One turn's calls are one block, bounded by `sandbox_execution_timeout` as a whole; results computed before a timeout are kept. The pre-execute VERIFY gate applies in both modes; reflection is CodeAct-only for now.
- Each result is kept as a variable (`tool_result_<call id>`): a large result is truncated in the reply to `execution_output_max_length` but survives whole for later turns, and a later CodeAct turn on the same thread can use it by name.
- Function-calling needs tools bound natively: if `bind_tools` fails or the model does not support it, or the agent has no executable tools, the run stops with a clear error (fail closed) instead of silently answering without tools. A static prompt is ignored in this mode.
- A thread can switch modes between turns: results from function-calling turns are shown to a later CodeAct turn as text and as variables.

</details>

<details>
<summary>📝 Special Instructions Configuration</summary>

Expand Down
62 changes: 62 additions & 0 deletions docs/design/fc-through-sandbox.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Function calling through the sandbox

Status: implemented on the `feature/native-function-calling-mode` branch (PR #777), 2026-10-09.

## Decision

`cuga_lite_execution_mode = "function_calling"` keeps native function calling on the
model side (tools via `bind_tools`, `tool_calls` in, one `ToolMessage` per call id out)
and executes the calls through the existing CodeAct sandbox instead of a separate
`tool_exec` runner. A native turn is translated into a short internal Python block,
the sandbox runs it, and the block's variables are read back into the replies.

Why: every CUGA guarantee already lives at the sandbox step — tool approval, the
pre-execute VERIFY gate, variables, output trimming, the block timeout, budgets, the
tracker, recovery. A second executor had to re-earn each one; after three review
rounds of #777 five gaps remained (approval refused to start, no recovery, no
variables, no observed shapes, no todos). One pipeline closes them at once.

## Shape

- `adapter/fc_actions.py` — `plan_tool_calls` (calls → block + plan) and
`replies_from_execution` (variables → `ToolMessage`s). Rules that keep the tool side
identical to a direct call: identifier names are called as themselves (approval and
VERIFY see the real name), other names go through an injected `_fc_tools` lookup,
arguments are `**{...}` of JSON values only, one `try` per call, results kept through
`_fc_keep` so nothing is dropped, non-executable calls answered at planning time.
- `graph_adapter.execute_call_model_fc` — routes to the sandbox with the block as
`script` and the plan under `cuga_lite_metadata["fc_pending"]`; runs the CodeAct
approval check on the block first.
- `sandbox_node` — when a plan is pending: local executor, listing variables kept,
replies instead of the "Execution output:" message at every exit (VERIFY revise,
normal, failure), plan cleared on every exit, reflection skipped.
- Feature off: no branch above is reachable (gated on a pending plan a CodeAct turn
never writes); every new parameter defaults to the old behavior. Golden:
`tests/snapshots/codeact_feature_off.json`.

## Deliberate differences from the direct runner

- Timeout is per block (whole batch), as for CodeAct; finished results are kept.
- Results persist as variables (`tool_result_<call id>`), visible to later turns and
in `InvokeResult.variables`.
- Approval follows CodeAct's check (text match on the block), including its
fail-open on infrastructure errors; the former refuse-to-start guard is gone.

## Also in this round (Vakra readiness)

- Provider-safe aliases (#882, merged into this branch): the translator resolves an
alias to the real name before planning, so the block, approval and VERIFY see the
real tool while the reply keeps the alias the provider knows. 6 of 10 Vakra test
domains have a tool name over 64 characters.
- `find_tools` is bound in function-calling mode whenever prepare enabled it, since the
FC prompt advertises it; its listing is replied, never kept as a variable.
- The FC model call has the recovery CodeAct has, in native terms: a Groq
`failed_generation` becomes a real call, a retryable rejection is retried once, a
second failure ends the turn with a clear error, anything else propagates.

## Follow-ups

1. A `run_python` tool backed by the same sandbox for data-heavy steps.
2. Per-thread dynamic binding: remember the shortlist and discovered tools, query on
the current task rather than the first message.
3. Function-calling wording for VERIFY, reflection and the empty-reply nudge.
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,35 @@ async def ainvoke_model(self, bound: Any, messages: list, invoke_config: dict) -
"""
return await bound.ainvoke(messages, config=invoke_config)

async def execute_call_model_fc(
self,
*,
state: Any,
config: Any,
configurable: dict,
active_model: Any,
bound: Any,
invoke_config: dict,
system_content: str,
modified_messages: list,
budget_exhausted: bool,
playbook_fired: bool,
variables_addendum: str = "",
) -> Optional[Command]:
"""Native function-calling seam.

Called by the shared ``call_model`` after the bound model is resolved and
before it is invoked. Returning ``None`` means "not my turn": the CodeAct
path below runs exactly as before. Returning a ``Command`` short-circuits
``call_model`` entirely.

The base returns ``None``, so the Supervisor graph and every CodeAct run
never enter function-calling code. CugaLite overrides this to run the
native ``tool_calls`` -> ``ToolMessage`` loop when
``cuga_lite_execution_mode`` resolves to ``function_calling``.
"""
return None

# ── Post-invocation hooks ──────────────────────────────────────────────

def normalize_response(self, response: Any) -> Tuple[str, Optional[str]]:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

START → prepare --Command--> call_model ↔ execute (loop) → END

CugaLite's function-calling mode uses the same three nodes: a native tool-call
turn is translated into a block the execute node runs.

Both CugaLite and CugaSupervisor share this structure. The nodes themselves
are provided by the caller (produced by adapter factories), so the graph
builder stays graph-agnostic.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,22 @@ async def call_model(state: Any, config: RunnableConfig = None) -> Command:
ai_content = msg.content if hasattr(msg, "content") else (msg.get("content") or "")
messages_for_model.append({"role": "assistant", "content": ai_content})

elif msg_role == "tool":
# A native tool result left by a function-calling turn on this thread.
# Matched on ``type``: persisted history crosses the SDK boundary as bare
# BaseMessage shells. Rendered like an execution output so a CodeAct
# turn on the same thread still sees what the tool returned.
modified_messages.append(msg)
tool_name = (
getattr(msg, "name", None)
or (msg.get("name") if isinstance(msg, dict) else None)
or "tool"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
)
tool_content = msg.content if hasattr(msg, "content") else (msg.get("content") or "")
messages_for_model.append(
{"role": "user", "content": f"Tool result ({tool_name}):\n{tool_content}"}
)

else:
modified_messages.append(msg)
logger.warning("call_model: skipping message {} with unknown role: {}", i, msg_role)
Expand Down Expand Up @@ -226,6 +242,28 @@ async def call_model(state: Any, config: RunnableConfig = None) -> Command:
# Pass the full node config so LangChain keeps parent_run_id linkage for
# Langfuse. Passing only {"callbacks": [...]} starts orphan root traces.
invoke_config = config if config is not None else {}

# ── Native function-calling seam ───────────────────────────────────
# The base adapter returns None, so this is a no-op for the Supervisor
# graph and for every CodeAct run. CugaLite returns a Command here when
# cuga_lite_execution_mode resolves to function_calling, and the CodeAct
# path below is skipped entirely — never blended.
fc_command = await adapter.execute_call_model_fc(
state=state,
config=config,
configurable=configurable,
active_model=active_model,
bound=bound,
invoke_config=invoke_config,
system_content=system_content,
modified_messages=modified_messages,
Comment thread
coderabbitai[bot] marked this conversation as resolved.
budget_exhausted=budget_exhausted,
playbook_fired=playbook_fired,
variables_addendum=variables_addendum,
)
if fc_command is not None:
return fc_command

response = await adapter.ainvoke_model(bound, messages_for_model, invoke_config)

# ── Normalise response ─────────────────────────────────────────────
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,15 @@ async def check_and_create_approval_interrupt(
code: str,
content: str,
config: dict = None,
*,
ai_message: Optional[AIMessage] = None,
) -> Optional[Command]:
"""Check if code requires approval and create an interrupt if needed."""
"""Check if code requires approval and create an interrupt if needed.

``ai_message`` is the assistant turn to persist with the interrupt; the
default is a text message carrying ``content``. A function-calling turn
passes its own ``AIMessage`` so the ``tool_calls`` survive the pause.
"""
from cuga.backend.cuga_graph.policy.configurable import PolicyConfigurable

try:
Expand Down Expand Up @@ -173,7 +180,7 @@ async def check_and_create_approval_interrupt(
adapter.set_metadata(state, approval_metadata)

return ToolApprovalHandler._create_approval_interrupt(
adapter, state, code, content, preview_lines
adapter, state, code, content, preview_lines, ai_message=ai_message
)

except Exception as e:
Expand All @@ -187,6 +194,8 @@ def _create_approval_interrupt(
code: str,
content: str,
preview_lines: List[str],
*,
ai_message: Optional[AIMessage] = None,
) -> Command:
"""Create an interrupt Command for tool approval."""
from cuga.backend.cuga_graph.nodes.human_in_the_loop.followup_model import create_tool_approval_action
Expand Down Expand Up @@ -223,7 +232,7 @@ def _create_approval_interrupt(
)

updated_messages, error_message = _core_append_with_step_limit(
adapter, state, [AIMessage(content=content)]
adapter, state, [ai_message if ai_message is not None else AIMessage(content=content)]
)
if error_message:
return _core_create_error_command(adapter, updated_messages, error_message, state.step_count)
Expand Down
Loading
Loading