Skip to content

fix: 修复executor的任务取消后无法续聊的问题 - #453

Closed
Micro66 wants to merge 5 commits into
mainfrom
human/chameleon-20260205-015939
Closed

Micro66 wants to merge 5 commits into
mainfrom
human/chameleon-20260205-015939

Conversation

@Micro66

@Micro66 Micro66 commented Feb 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Bug Fixes

    • Improved cancellation flow to allow resumed work, ensure cleanup on interrupt timeouts, and avoid hangs during shutdown
    • Hardened error handling with retry behavior for recoverable failures and clearer terminal statuses
    • More robust session/client termination to guarantee resource cleanup
  • Logging / Security

    • Masked sensitive connection data in logs
  • Tests

    • Added tests for cancellation, timeout, error-handling, and cleanup scenarios

- Treat cancel as soft interrupt; reset CANCELLED -> RUNNING on next execute

- Add interrupt timeout and force process cleanup on timeout/error

- Fail fast on error_during_execution to avoid stuck RUNNING

- Add regression tests for cancellation/resume
Avoid anyio cancel-scope mismatch when disconnecting from a different task by falling back to process termination and cache cleanup; also fix AgentService ClaudeCode name checks.
When a task is cancelled and the Claude Code subprocess is terminated (exit 143), avoid reporting FAILED from response processing; mark workbench completed and return COMPLETED.
@coderabbitai

coderabbitai Bot commented Feb 5, 2026 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds timeout-based shutdown and forceful termination to ClaudeCode client lifecycle, implements soft-cancel recovery and retry-aware response processing, adjusts service-level agent type checks, improves sensitive-data logging, and adds tests covering cancellation, error handling, and timeout-driven cleanup.

Changes

Cohort / File(s) Summary
Logging & MCP client
chat_shell/chat_shell/tools/mcp/client.py, executor/agents/claude_code/config_manager.py
Changed debug logging to avoid dumping full task_data and added masking of sensitive MCP server data in claude options logging.
Agent Core & Cancellation
executor/agents/claude_code/claude_code_agent.py, executor/agents/claude_code/session_manager.py
Added INTERRUPT_TIMEOUT_SECONDS and DISCONNECT_TIMEOUT_SECONDS; implemented soft-cancel reset, timeout-protected interrupt, forced process termination helpers, multi-stage close/cleanup, and improved error handling around client shutdown.
Response Processing & Retry Logic
executor/agents/claude_code/response_processor.py
process_response now propagates terminal statuses and is cancellation-aware; _handle_system_message returns Optional[TaskStatus]; _process_result_message gains explicit return types, retry detection/flow (returns "RETRY" or TaskStatus), and enhanced error/cancellation handling.
Service dispatch adjustment
executor/services/agent_service.py
Replaced type checks referencing ClaudeCodeAgent with the identifier string "ClaudeCode" in agent session closure paths and error reporting keys.
Tests
executor/tests/agents/test_claude_code_cancellation_resume.py
New async test module validating cancelled-state reset and resume, system-message failure handling, exit-143 cancellation detection, and interrupt-timeout forcing cleanup.

Sequence Diagram(s)

sequenceDiagram
    participant Agent as ClaudeCodeAgent
    participant Timer as Interrupt Timer
    participant Proc as Client Process
    participant Cleanup as cleanup_task_clients

    Agent->>Timer: Start interrupt timeout (INTERRUPT_TIMEOUT_SECONDS)
    Agent->>Proc: Send interrupt / soft-cancel
    alt Interrupt completes before timeout
        Proc-->>Agent: Acknowledge interrupt
        Agent->>Agent: Continue or recreate client as needed
    else Interrupt times out
        rect rgba(255,100,100,0.5)
            Timer-->>Agent: Interrupt timeout
        end
        Agent->>Cleanup: Invoke cleanup_task_clients (force termination)
        Cleanup-->>Agent: Clients removed / cleared
        Agent->>Proc: Force terminate process if still present
    end
Loading
sequenceDiagram
    participant Processor as process_response
    participant Handler as _handle_system_message
    participant Result as _process_result_message
    participant Client as SDK Client

    Processor->>Handler: Inspect system message
    alt Handler returns TaskStatus (e.g., FAILED)
        Handler-->>Processor: TaskStatus (terminal)
        Processor-->>Processor: Propagate terminal status and stop
    else None
        Processor->>Result: Handle result message
        alt Result detects retryable API error
            Result->>Client: Attempt retry via client
            Client-->>Result: Retry outcome
            Result-->>Processor: Return "RETRY" (requeue)
        else Success
            Result-->>Processor: Return TaskStatus.COMPLETED
        end
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50 minutes

Possibly related PRs

Suggested reviewers

  • qdaxb
  • feifei325

Poem

🐰 I nudge the timer, give a hop and stare,
If interrupts stall I’ll tidy with care.
A soft-step back, then onward we run—
Retries and restarts till the job is done. ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing the inability to continue conversations after task cancellation in the executor. This is the core objective addressed across multiple files.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch human/chameleon-20260205-015939

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
executor/services/agent_service.py (1)

437-451: ⚠️ Potential issue | 🟡 Minor

Inconsistent naming for Agno error detail key.

The ClaudeCode agent uses consistent naming ("ClaudeCode" on lines 437 and 443), but the Agno agent uses "Agno" for the type check (line 445) but "AgnoAgent" for the error_detail key (line 451). This should be consistent.

🔧 Proposed fix for consistency
             if status == TaskStatus.SUCCESS:
                 results.append("Agno")
             else:
                 errors.append("Agno")
-                error_detail["AgnoAgent"] = msg or "Unknown error"
+                error_detail["Agno"] = msg or "Unknown error"
🤖 Fix all issues with AI agents
In `@executor/agents/claude_code/claude_code_agent.py`:
- Around line 1423-1467: The close_client method currently returns
TaskStatus.SUCCESS even when an exception is caught; update the exception
handler in claude_code_agent. close_client to return TaskStatus.FAILED instead
of TaskStatus.SUCCESS after performing the existing cleanup (cls._clients.pop
and cls._session_id_map removals); keep the logger.exception call as-is and
ensure cleanup still runs (the finally block and the except cleanup should
remain). This change affects the exception return only (use TaskStatus.FAILED)
so callers checking the returned TaskStatus (e.g.,
executor.services.agent_service) will see failures correctly.
🧹 Nitpick comments (2)
executor/tests/agents/test_claude_code_cancellation_resume.py (1)

116-124: Unreachable code after exception raise.

Lines 122-123 (if False: yield None) are technically unreachable. While this is a common pattern to make an async generator that only raises, consider using a cleaner approach.

♻️ Cleaner async generator pattern
     class FakeClient:
         async def receive_response(self):
             raise Exception(
                 "Command failed with exit code 143 (exit code: 143)\n"
                 "Error output: Check stderr output for details"
             )
-            if False:
-                yield None
+            # Make this an async generator by yielding before raise would also work,
+            # but the current pattern is acceptable for test purposes
+            return
+            yield  # type: ignore[misc]  # Makes this an async generator

Alternatively, the existing pattern is acceptable for test code since it achieves the intended behavior.

executor/agents/claude_code/claude_code_agent.py (1)

1394-1421: Consider logging on successful termination path.

The _force_close_client_process method handles process termination well, but the broad except Exception (flagged by static analysis BLE001) could silently swallow unexpected errors. The current implementation is acceptable for cleanup code, but consider logging the specific exception type for debugging.

♻️ Optional: More specific exception handling
         except Exception as e:
             logger.warning(
-                f"Failed to force-terminate process for session_id={session_id}, PID={pid}: {e}"
+                f"Failed to force-terminate process for session_id={session_id}, PID={pid}: {type(e).__name__}: {e}"
             )

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
executor/agents/claude_code/session_manager.py (1)

376-399: ⚠️ Potential issue | 🟠 Major

del cls._clients[session_id] can raise KeyError due to async interleaving.

Between the if session_id in cls._clients check (Line 377) and the del (Line 380), the await on Line 379 yields to the event loop. A concurrent close_client call could .pop() the same key during this window, causing del to raise KeyError. The same pattern appears at Lines 396-398.

Use .pop(session_id, None) for safe removal:

🐛 Proposed fix
         for internal_key, session_id in internal_keys_to_cleanup:
             if session_id in cls._clients:
                 client = cls._clients[session_id]
                 await cls._terminate_client_process(client, session_id)
-                del cls._clients[session_id]
+                cls._clients.pop(session_id, None)
                 cleaned_count += 1
             else:
             if session_id.startswith(task_id_prefix) or session_id == task_id_str:
                 client = cls._clients[session_id]
                 await cls._terminate_client_process(client, session_id)
-                del cls._clients[session_id]
+                cls._clients.pop(session_id, None)
                 cleaned_count += 1

Note: Line 396 also has client = cls._clients[session_id] after an iteration check, which could similarly fail. Consider:

-            if session_id.startswith(task_id_prefix) or session_id == task_id_str:
-                client = cls._clients[session_id]
+            if session_id.startswith(task_id_prefix) or session_id == task_id_str:
+                client = cls._clients.get(session_id)
+                if not client:
+                    continue
🤖 Fix all issues with AI agents
In `@executor/agents/claude_code/claude_code_agent.py`:
- Around line 66-67: Replace the duplicated timeout constants by removing
INTERRUPT_TIMEOUT_SECONDS and DISCONNECT_TIMEOUT_SECONDS definitions from
claude_code_agent.py and import the single source of truth from session_manager
(use the existing DISCONNECT_TIMEOUT_SECONDS/INTERRUPT_TIMEOUT_SECONDS
definitions there); update claude_code_agent.py to import
INTERRUPT_TIMEOUT_SECONDS from session_manager and delete the unused local
DISCONNECT_TIMEOUT_SECONDS constant so each constant is defined only in
session_manager and used via import in claude_code_agent.py.
🧹 Nitpick comments (5)
executor/agents/claude_code/response_processor.py (2)

569-579: Consider replacing the "RETRY" string sentinel with a dedicated type.

The Union[TaskStatus, str, None] return type uses a magic string "RETRY" as a signal. This is fragile and could collide with future TaskStatus string values. A small enum or a @dataclass wrapper would make the contract explicit.

♻️ Suggested approach
# At module level:
class _ResultAction(str, enum.Enum):
    RETRY = "RETRY"

# Return type becomes: Union[TaskStatus, _ResultAction, None]

Also applies to: 756-756


786-792: Inconsistent null-guarding of state_manager.

In _handle_system_message (Line 248), state_manager is guarded with if state_manager: before use. Here and in the success path (Lines 684, 711), it is used without a guard. While callers currently always pass a valid state_manager, the inconsistency could cause an AttributeError if invoked differently in the future.

executor/agents/claude_code/claude_code_agent.py (1)

1022-1044: Timeout-guarded interrupt with forced cleanup fallback looks solid.

The three-level approach (interrupt → timeout → force-kill) provides a robust cancellation path. The return at Line 1031 correctly prevents the cleanup fallback from running on success.

Nit from static analysis (Ruff TRY401): logger.exception at Line 1038 already attaches the traceback, so interpolating {interrupt_error} in the message is redundant.

♻️ Minor cleanup
                except Exception as interrupt_error:
                    logger.exception(
-                        f"Interrupt failed for task_id={self.task_id}, session_id={self.session_id}: {interrupt_error}. "
+                        f"Interrupt failed for task_id={self.task_id}, session_id={self.session_id}. "
                         "Forcing process cleanup"
                    )
executor/agents/claude_code/session_manager.py (2)

232-274: Multi-level shutdown in close_client is well-structured.

The graceful disconnect → timeout → force-terminate cascade with finally-based cleanup is a solid pattern. The RuntimeError catch for cancel-scope mismatches (Line 249) addresses a known real-world edge case with async task boundaries.

Nit (Ruff RUF010/TRY401): at Line 268, str(e) is redundant inside logger.exception which already appends the exception info.

♻️ Minor cleanup
         except Exception as e:
             logger.exception(
-                f"Error closing client for session_id {session_id}: {str(e)}"
+                f"Error closing client for session_id {session_id}"
             )

321-327: Extract hardcoded timeouts to named constants.

The 2.0 and 1.0 second waits at Lines 321 and 327 are magic numbers. Extracting them to module-level constants (e.g., TERMINATE_WAIT_SECONDS and KILL_WAIT_SECONDS) improves readability and makes them easier to tune.

Comment on lines +66 to +67
INTERRUPT_TIMEOUT_SECONDS = 5.0
DISCONNECT_TIMEOUT_SECONDS = 2.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Duplicated timeout constants — define them in one place.

INTERRUPT_TIMEOUT_SECONDS and DISCONNECT_TIMEOUT_SECONDS are declared identically in both this file and session_manager.py (Lines 24-25). Moreover, each file only uses one of the two constants it defines:

  • This file uses INTERRUPT_TIMEOUT_SECONDS (Line 1024) but never DISCONNECT_TIMEOUT_SECONDS.
  • session_manager.py uses DISCONNECT_TIMEOUT_SECONDS (Line 239) but never INTERRUPT_TIMEOUT_SECONDS.

Define them once in session_manager.py and import here, to avoid drift.

♻️ Suggested fix

In this file, replace the local constants with an import:

-INTERRUPT_TIMEOUT_SECONDS = 5.0
-DISCONNECT_TIMEOUT_SECONDS = 2.0
+from executor.agents.claude_code.session_manager import INTERRUPT_TIMEOUT_SECONDS
🤖 Prompt for AI Agents
In `@executor/agents/claude_code/claude_code_agent.py` around lines 66 - 67,
Replace the duplicated timeout constants by removing INTERRUPT_TIMEOUT_SECONDS
and DISCONNECT_TIMEOUT_SECONDS definitions from claude_code_agent.py and import
the single source of truth from session_manager (use the existing
DISCONNECT_TIMEOUT_SECONDS/INTERRUPT_TIMEOUT_SECONDS definitions there); update
claude_code_agent.py to import INTERRUPT_TIMEOUT_SECONDS from session_manager
and delete the unused local DISCONNECT_TIMEOUT_SECONDS constant so each constant
is defined only in session_manager and used via import in claude_code_agent.py.

@Micro66 Micro66 closed this Feb 6, 2026
@qdaxb
qdaxb deleted the human/chameleon-20260205-015939 branch September 18, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant