-
Notifications
You must be signed in to change notification settings - Fork 1
fix: strengthen immutability for BaseTool schema and ToolInvoker boundaries #117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
2b2f36b
1ae87e6
db36d19
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -6,6 +6,7 @@ | |||||||||||
|
|
||||||||||||
| import copy | ||||||||||||
| from abc import ABC, abstractmethod | ||||||||||||
| from types import MappingProxyType | ||||||||||||
| from typing import Any | ||||||||||||
|
|
||||||||||||
| from pydantic import BaseModel, ConfigDict, Field | ||||||||||||
|
|
@@ -26,9 +27,12 @@ class ToolExecutionResult(BaseModel): | |||||||||||
| to the LLM and is available only for programmatic consumers. | ||||||||||||
|
|
||||||||||||
| Note: | ||||||||||||
| The ``metadata`` dict is shallowly immutable under the frozen | ||||||||||||
| model — reassignment is prevented but contents can still be | ||||||||||||
| mutated. Callers should treat it as read-only. | ||||||||||||
| The ``metadata`` dict is shallowly frozen by Pydantic's | ||||||||||||
| ``frozen=True``. Tool implementations construct and return | ||||||||||||
| this model, but the invoker converts it into a provider-facing | ||||||||||||
| ``ToolResult`` — ``metadata`` is not forwarded to LLM providers | ||||||||||||
| or other external boundaries, so no additional boundary copy | ||||||||||||
| is needed at this layer. | ||||||||||||
|
|
||||||||||||
| Attributes: | ||||||||||||
| content: Tool output as a string. | ||||||||||||
|
|
@@ -57,7 +61,8 @@ class BaseTool(ABC): | |||||||||||
| name: Non-blank tool name. | ||||||||||||
| description: Human-readable description of the tool. | ||||||||||||
| parameters_schema: JSON Schema dict describing expected arguments, | ||||||||||||
| or ``None`` if the tool accepts any arguments. | ||||||||||||
| or ``None`` if no parameter schema is defined (the invoker | ||||||||||||
| skips validation). | ||||||||||||
| """ | ||||||||||||
|
|
||||||||||||
| def __init__( | ||||||||||||
|
|
@@ -83,8 +88,10 @@ def __init__( | |||||||||||
| raise ValueError(msg) | ||||||||||||
| self._name = name | ||||||||||||
| self._description = description | ||||||||||||
| self._parameters_schema: dict[str, Any] | None = ( | ||||||||||||
| copy.deepcopy(parameters_schema) if parameters_schema is not None else None | ||||||||||||
| self._parameters_schema: MappingProxyType[str, Any] | None = ( | ||||||||||||
| MappingProxyType(copy.deepcopy(parameters_schema)) | ||||||||||||
| if parameters_schema is not None | ||||||||||||
| else None | ||||||||||||
| ) | ||||||||||||
|
|
||||||||||||
| @property | ||||||||||||
|
|
@@ -101,9 +108,12 @@ def description(self) -> str: | |||||||||||
| def parameters_schema(self) -> dict[str, Any] | None: | ||||||||||||
| """JSON Schema for tool parameters, or None if unspecified. | ||||||||||||
|
|
||||||||||||
| Returns a deep copy to prevent mutation of the internal schema. | ||||||||||||
| Returns a deep copy to prevent mutation of internal state. | ||||||||||||
| """ | ||||||||||||
| return copy.deepcopy(self._parameters_schema) | ||||||||||||
| if self._parameters_schema is None: | ||||||||||||
| return None | ||||||||||||
| # dict() needed: MappingProxyType cannot be deep-copied directly | ||||||||||||
| return copy.deepcopy(dict(self._parameters_schema)) | ||||||||||||
|
Comment on lines
+115
to
+116
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Misleading comment — The inline comment
Suggested change
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/ai_company/tools/base.py
Line: 115-116
Comment:
**Misleading comment — `MappingProxyType` *can* be deep-copied**
The inline comment `# dict() needed: MappingProxyType cannot be deep-copied directly` is inaccurate. In Python 3, `copy.deepcopy(MappingProxyType(…))` is well-defined and returns a new `MappingProxyType` wrapping a deep-copied dict. The real reason `dict()` is needed here is to satisfy the `dict[str, Any]` return-type annotation — a `deepcopy` of a `MappingProxyType` would return another `MappingProxyType`, which mismatches the declared return type.
```suggestion
# Convert to dict first: deepcopy of MappingProxyType returns a
# MappingProxyType, but the public API declares dict[str, Any].
return copy.deepcopy(dict(self._parameters_schema))
```
How can I resolve this? If you propose a fix, please make it concise. |
||||||||||||
|
|
||||||||||||
| def to_definition(self) -> ToolDefinition: | ||||||||||||
| """Convert this tool to a ``ToolDefinition`` for LLM providers. | ||||||||||||
|
|
||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,13 @@ | ||
| """Tool invoker — validates and executes tool calls. | ||
|
|
||
| Bridges LLM ``ToolCall`` objects with concrete ``BaseTool.execute`` | ||
| methods. Never propagates exceptions — always returns a ``ToolResult``. | ||
|
|
||
| Note: | ||
| ``BaseException`` subclasses (``KeyboardInterrupt``, ``SystemExit``, | ||
| ``asyncio.CancelledError``) are NOT caught and will propagate | ||
| normally. Non-recoverable errors (``MemoryError``, | ||
| ``RecursionError``) are re-raised after logging. | ||
| methods. Recoverable errors are returned as ``ToolResult(is_error=True)``; | ||
| non-recoverable errors (``MemoryError``, ``RecursionError``) and | ||
| ``BaseException`` subclasses (``KeyboardInterrupt``, ``SystemExit``, | ||
| ``asyncio.CancelledError``) propagate after logging. | ||
| """ | ||
|
|
||
| import copy | ||
| from typing import TYPE_CHECKING | ||
|
|
||
| import jsonschema | ||
|
|
@@ -19,6 +17,7 @@ | |
|
|
||
| from ai_company.observability import get_logger | ||
| from ai_company.observability.events import ( | ||
| TOOL_INVOKE_DEEPCOPY_ERROR, | ||
| TOOL_INVOKE_EXECUTION_ERROR, | ||
| TOOL_INVOKE_NON_RECOVERABLE, | ||
| TOOL_INVOKE_NOT_FOUND, | ||
|
|
@@ -234,9 +233,40 @@ async def _execute_tool( | |
| tool: BaseTool, | ||
| tool_call: ToolCall, | ||
| ) -> ToolExecutionResult | ToolResult: | ||
| """Execute the tool, catching errors as ``ToolResult``.""" | ||
| """Deep-copy arguments for isolation, then execute the tool. | ||
|
|
||
| Copy failures and execution errors are caught and returned as | ||
| ``ToolResult(is_error=True)``. Non-recoverable errors | ||
| (``MemoryError``, ``RecursionError``) propagate after logging. | ||
| """ | ||
| try: | ||
| safe_args = copy.deepcopy(tool_call.arguments) | ||
| except (MemoryError, RecursionError) as exc: | ||
| logger.exception( | ||
| TOOL_INVOKE_NON_RECOVERABLE, | ||
| tool_call_id=tool_call.id, | ||
| tool_name=tool_call.name, | ||
| error=f"{type(exc).__name__}: {exc}", | ||
| ) | ||
| raise | ||
| except Exception as exc: | ||
| error_msg = str(exc) or f"{type(exc).__name__} (no message)" | ||
| logger.exception( | ||
| TOOL_INVOKE_DEEPCOPY_ERROR, | ||
| tool_call_id=tool_call.id, | ||
| tool_name=tool_call.name, | ||
| error=f"Failed to deep-copy arguments: {error_msg}", | ||
| ) | ||
|
greptile-apps[bot] marked this conversation as resolved.
Comment on lines
+254
to
+259
|
||
| return ToolResult( | ||
| tool_call_id=tool_call.id, | ||
| content=( | ||
| f"Tool {tool_call.name!r} arguments could not be " | ||
| f"safely copied: {error_msg}" | ||
| ), | ||
| is_error=True, | ||
| ) | ||
| try: | ||
| return await tool.execute(arguments=dict(tool_call.arguments)) | ||
| return await tool.execute(arguments=safe_args) | ||
| except (MemoryError, RecursionError) as exc: | ||
| logger.exception( | ||
| TOOL_INVOKE_NON_RECOVERABLE, | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -98,13 +98,29 @@ def test_default_schema_none(self) -> None: | |
| tool = _ConcreteTool(name="t") | ||
| assert tool.parameters_schema is None | ||
|
|
||
| def test_schema_deep_copied_on_construction(self) -> None: | ||
| def test_schema_isolated_on_construction(self) -> None: | ||
| props: dict[str, Any] = {"x": {"type": "string"}} | ||
| schema: dict[str, Any] = {"type": "object", "properties": props} | ||
| tool = _ConcreteTool(name="t", parameters_schema=schema) | ||
| props["y"] = {"type": "integer"} | ||
| schema["injected"] = True | ||
| assert tool.parameters_schema is not None | ||
| assert "y" not in tool.parameters_schema["properties"] | ||
| assert "injected" not in tool.parameters_schema | ||
|
|
||
| def test_schema_nested_isolated_on_construction(self) -> None: | ||
| schema: dict[str, Any] = { | ||
| "type": "object", | ||
| "properties": {"x": {"type": "string"}}, | ||
| } | ||
| tool = _ConcreteTool(name="t", parameters_schema=schema) | ||
| schema["properties"]["x"]["type"] = "integer" | ||
| assert tool.parameters_schema is not None | ||
| assert tool.parameters_schema["properties"]["x"]["type"] == "string" | ||
|
|
||
| def test_schema_internal_is_read_only(self) -> None: | ||
| schema = {"type": "object", "properties": {"x": {"type": "string"}}} | ||
| tool = _ConcreteTool(name="t", parameters_schema=schema) | ||
| with pytest.raises(TypeError): | ||
| tool._parameters_schema["injected"] = True # type: ignore[index] | ||
|
Comment on lines
+119
to
+123
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The test correctly asserts that tool._parameters_schema["properties"]["x"]["type"] = "integer" # No TypeError raised!Given the PR's stated goal of strengthening immutability and the explicit def test_schema_internal_is_read_only(self) -> None:
schema = {"type": "object", "properties": {"x": {"type": "string"}}}
tool = _ConcreteTool(name="t", parameters_schema=schema)
with pytest.raises(TypeError):
tool._parameters_schema["injected"] = True # type: ignore[index]
# Note: MappingProxyType provides only top-level protection. Nested dicts
# in _parameters_schema remain mutable, but direct access is discouraged
# since it's a private attribute. Public isolation is guaranteed by the
# parameters_schema property, which returns a deep copy.Prompt To Fix With AIThis is a comment left during a code review.
Path: tests/unit/tools/test_base.py
Line: 119-123
Comment:
**`test_schema_internal_is_read_only` only covers top-level mutation**
The test correctly asserts that `tool._parameters_schema["injected"] = True` raises `TypeError` (top-level `MappingProxyType` enforcement). However, it does not cover the nested mutability gap: since `_parameters_schema["properties"]` is a plain `dict`, the following silently succeeds and corrupts internal state:
```python
tool._parameters_schema["properties"]["x"]["type"] = "integer" # No TypeError raised!
```
Given the PR's stated goal of strengthening immutability and the explicit `test_schema_nested_isolated_on_construction` / `test_schema_property_nested_mutation_isolated` additions, adding a clarifying comment would make the documented guarantees clear and prevent future confusion:
```python
def test_schema_internal_is_read_only(self) -> None:
schema = {"type": "object", "properties": {"x": {"type": "string"}}}
tool = _ConcreteTool(name="t", parameters_schema=schema)
with pytest.raises(TypeError):
tool._parameters_schema["injected"] = True # type: ignore[index]
# Note: MappingProxyType provides only top-level protection. Nested dicts
# in _parameters_schema remain mutable, but direct access is discouraged
# since it's a private attribute. Public isolation is guaranteed by the
# parameters_schema property, which returns a deep copy.
```
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
|
|
||
| def test_schema_property_returns_copy(self) -> None: | ||
| schema: dict[str, Any] = { | ||
|
|
@@ -118,6 +134,19 @@ def test_schema_property_returns_copy(self) -> None: | |
| assert tool.parameters_schema is not None | ||
| assert "injected" not in tool.parameters_schema | ||
|
|
||
| def test_schema_property_nested_mutation_isolated(self) -> None: | ||
| schema: dict[str, Any] = { | ||
| "type": "object", | ||
| "properties": {"x": {"type": "string"}}, | ||
| } | ||
| tool = _ConcreteTool(name="t", parameters_schema=schema) | ||
| returned = tool.parameters_schema | ||
| assert returned is not None | ||
| returned["properties"]["x"]["type"] = "integer" | ||
| fresh = tool.parameters_schema | ||
| assert fresh is not None | ||
| assert fresh["properties"]["x"]["type"] == "string" | ||
|
|
||
| def test_to_definition(self) -> None: | ||
| schema = {"type": "object", "properties": {"x": {"type": "string"}}} | ||
| tool = _ConcreteTool( | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This implementation can be simplified.
copy.deepcopy()correctly handles bothMappingProxyTypeandNonevalues, so the explicit check forNoneand the cast todict()are unnecessary. A single line would be more concise and achieve the same result.