feat(telemetry): Track tool execution outcomes

docs/design/2026-07-31-tool-execution-status.md

@@ -0,0 +1,128 @@
+# Tool Execution Status
+
+## Motivation
+
+The terminal tool-call status describes whether the overall call succeeded,
+failed, or was cancelled. It does not say whether the dispatcher actually
+entered `invocation.execute()`. Validation failures, permission rejection,
+execution failures, and post-execution failures therefore need a separate
+execution outcome before they can be measured accurately.
+
+## Contract
+
+`ToolCallResponseInfo` carries an optional `executionStatus` for source and
+recording compatibility:
+
+```ts
+type ToolExecutionStatus = 'not_started' | 'success' | 'error' | 'cancelled';
+```
+
+The Core scheduler (`CoreToolScheduler`) and ACP `Session.runTool` always set
+the field. Missing values from older recordings, third-party producers, and
+subagent result projections (the non-interactive `buildResponse` path, which
+replays another agent's reported outcome) become `unknown` only at the
+telemetry boundary and are never inferred from the terminal call status.
+
+The terminal and execution axes are intentionally independent:
+
+| Terminal status | Execution status | Example                                                                              |
+| --------------- | ---------------- | ------------------------------------------------------------------------------------ |
+| `success`       | `success`        | Normal tool completion                                                               |
+| `success`       | `not_started`    | Protocol-level synthetic sibling response                                            |
+| `error`         | any value        | Pre-execution denial, execution error, post-processing error, or batch-hook override |
+| `cancelled`     | any value        | Cancellation before, during, or after execution                                      |
+
+Reading each row as a (terminal, execution) pair, the only invalid combinations are `success/error` and `success/cancelled`: a call that terminates `success` can only carry execution status `success` or `not_started`.
+Execution status freezes when `invocation.execute()` settles; hooks, result
+bridging, persistence, and batch processing cannot overwrite it.
+PostToolBatch enablement and its parent tool span are snapshotted when a
+scheduler batch starts, so runtime hook reconfiguration affects the next
+batch rather than changing completion behavior for an in-flight batch.
+
+## Telemetry
+
+The normalized `tool_call` event adds `call_id` and `execution_status`.
+Normalization occurs once before all sinks:
+
+- empty tool names become `unknown_tool`;
+- `success` is recomputed from terminal `status`;
+- terminal errors without an error type use `unknown`;
+- success and cancellation omit call-level error fields;
+- missing execution status becomes `unknown`.
+
+The terminal `status` dimension on `qwen-code.tool.call.count`, established by
+the terminal telemetry contract, is unchanged by this design. A new
+`qwen-code.tool.execution.count` counter uses only `execution_status` and
+`tool_type` event-specific dimensions. Globally configured common metric
+attributes, such as the opt-in `session.id`, may also be present. The execution
+failure rate is:
+
+```text
+execution_status = error
+────────────────────────────────────────
+execution_status in {success, error}
+```
+
+Cancellation, `not_started`, and `unknown` are excluded. Error type, function
+name, call ID, messages, and MCP server names remain in logs or spans rather
+than metric labels. The counter deliberately omits `function_name`, so an
+execution-failure rate cannot be attributed to a specific tool from the metric
+alone; drill down through the `tool_call` logs, which carry both `call_id` and
+`function_name`.
+
+An execution span exists only after the dispatcher attempts `execute()`.
+It records the tool identity, frozen execution status, and execution error
+type. Parent tool spans continue to represent the terminal call status, and
+cancelled spans remain unset rather than error. Core opens the parent span
+after tool resolution and invocation validation; earlier terminal paths are
+covered by the normalized event and execution counter and do not synthesize a
+span from an unresolved request name.
+
+QwenLogger receives the normalized terminal status, execution status, call ID,
+and tool type, but not MCP server names or function arguments. MCP server names
+remain outside QwenLogger and are available to configured telemetry log and
+span exporters.
+
+## Compatibility and Scope
+
+The public response and event fields stay optional. Built-in producers use an
+internal required shape, while old JSONL recordings are not migrated or
+backfilled. New JSONL recordings include `executionStatus` on recorded tool
+results; the field is additive, so replay readers that ignore unknown fields
+are unaffected. Manual recording projections in Core, ACP, TUI, and
+non-interactive modes copy the new scalar without exposing it in user-facing
+JSON output. A call cancelled before tool resolution can omit `tool` and
+`invocation` from the public `CancelledToolCall` variant, so consumers of that
+variant must guard those fields before use.
+When such a pre-resolution cancel is emitted through telemetry, `tool_type`
+defaults to `"native"` because the tool identity is not yet resolved; this
+is a known skew in the `tool_type` dimension for pre-validation cancels.
+
+Per-call execution errors no longer reject `CoreToolScheduler.schedule()`;
+the outcome is delivered through the existing update and completion callbacks
+as a terminal `error` call, so one tool's failure does not abort its siblings.
+The method still returns `Promise<void>` and can reject for scheduler-level
+setup or queue failures. `handleConfirmationResponse()` terminalizes
+confirmation-flow errors before rethrowing them, preserving its existing
+failure signal without leaving a call in `awaiting_approval`. Embedders should
+read terminal `status` and `executionStatus` from callback-delivered calls,
+not expect either public entry point to return completed calls.
+
+The first release covers `CoreToolScheduler` and ACP `Session.runTool`.
+Speculation, direct `/fork` execution, MCP-internal retries, provisional
+subagent result reconciliation, shell exit metadata, retryability, ownership,
+and generic failure phases remain out of scope.
+
+Core and ACP must ship together. Dashboards should cut over by deployment time
+or `service.version`, monitor `unknown` separately, and never use the legacy
+`success` metric as the execution-failure SLI.
+
+## Known Maintenance Hazards
+
+The pre-execution cancellation invariant ("every `await` in the pre-execution
+path is followed by an abort check") is enforced by hand-placed checks at each
+call site in `CoreToolScheduler` and `Session.runTool` rather than by a
+structural mechanism. Adding a new `await` to either path without a following
+check silently reintroduces the stale-execution bug this design fixes. A
+future refactor should wrap the awaits in a guarded helper; until then,
+reviewers of those paths should verify the invariant manually.

docs/developers/development/telemetry.md

@@ -576,8 +576,8 @@ The following events are logged:
 
 #### Tool Events
 
-- `qwen-code.tool_call`: Each function/tool call. Terminal events are normalized so `status` is authoritative: success and cancelled events omit error fields, while error events always have a non-empty `error_type` (`unknown` when the producer did not classify the error). Blank tool names are emitted as `unknown_tool`.
-  - **Attributes**: `function_name` (string), `function_args` (object), `duration_ms` (int), `status` (string: "success", "error", or "cancelled"), `success` (boolean), `decision` (string: "accept", "reject", "auto_accept", or "modify", optional), `error` (string, optional), `error_type` (string, present for error events), `prompt_id` (string), `response_id` (string, optional), `content_length` (int, optional), `tool_type` (string: "native" or "mcp"), `mcp_server_name` (string, optional), `metadata` (object, optional — for file-writing tools contains `model_added_lines`, `model_removed_lines`, `user_added_lines`, `user_removed_lines`, `model_added_chars`, `model_removed_chars`, `user_added_chars`, `user_removed_chars`)
+- `qwen-code.tool_call`: Each function/tool call. Terminal events are normalized so `status` is authoritative: success and cancelled events omit error fields, while error events always have a non-empty `error_type` (`unknown` when the producer did not classify the error). Blank tool names are emitted as `unknown_tool`. A missing `execution_status` is normalized to `unknown` and is never inferred from the terminal `status`.
+  - **Attributes**: `function_name` (string), `function_args` (object), `call_id` (string, optional), `duration_ms` (int), `status` (string: "success", "error", or "cancelled"), `execution_status` (string: "not_started", "success", "error", "cancelled", or "unknown"), `success` (boolean), `decision` (string: "accept", "reject", "auto_accept", or "modify", optional), `error` (string, optional), `error_type` (string, present for error events), `prompt_id` (string), `response_id` (string, optional), `content_length` (int, optional), `tool_type` (string: "native" or "mcp"), `mcp_server_name` (string, optional), `metadata` (object, optional — for file-writing tools contains `model_added_lines`, `model_removed_lines`, `user_added_lines`, `user_removed_lines`, `model_added_chars`, `model_removed_chars`, `user_added_chars`, `user_removed_chars`)
 
 - `qwen-code.file_operation`: Each file operation.
   - **Attributes**: `tool_name` (string), `operation` (string: "create", "read", "update"), `lines` (int, optional), `mimetype` (string, optional), `extension` (string, optional), `programming_language` (string, optional)
@@ -723,6 +723,9 @@ Metrics are numerical measurements of behavior over time. Metric names use the `
 - `qwen-code.tool.call.count` (Counter, Int): Counts tool calls.
   - **Attributes**: `function_name`, `status` ("success"/"error"/"cancelled"), `success` (boolean, retained for compatibility), `decision` ("accept"/"reject"/"auto_accept"/"modify", optional), `tool_type` ("mcp"/"native", optional)
 
+- `qwen-code.tool.execution.count` (Counter, Int): Counts tool execution outcomes. Deliberately carries no `function_name` dimension to stay low-cardinality, so an execution-failure rate cannot be attributed to a specific tool without dropping to the `qwen-code.tool_call` logs; exclude `unknown`, `not_started`, and `cancelled` when computing execution-failure ratios (denominator is `success` + `error`).
+  - **Attributes**: `execution_status` ("not_started"/"success"/"error"/"cancelled"/"unknown"), `tool_type` ("mcp"/"native"), plus globally configured common metric attributes such as the opt-in `session.id`
+
 - `qwen-code.tool.call.latency` (Histogram, ms): Measures tool call latency.
   - **Attributes**: `function_name` (string)
 
@@ -866,10 +869,10 @@ Distributed tracing spans form a tree rooted at `qwen-code.interaction`. Each in
   - Streaming requests emit `gen_ai.request.stream=true`. `gen_ai.response.time_to_first_chunk` measures seconds from the provider call to the first normalized response yielded by the provider adapter, which may differ from the first raw network frame. Non-streaming requests omit both standard streaming attributes because an absent `gen_ai.request.stream` means non-streaming in the semantic convention.
 
 - `qwen-code.tool`: Wraps the full tool lifecycle (approval wait + execution).
-  - **Attributes**: `session.id`, optional ARMS extension `gen_ai.user.id`, `gen_ai.operation.name` (`execute_tool`), `gen_ai.tool.name`, `gen_ai.tool.type` (`function`), `gen_ai.tool.call.id`, `tool.call_id`, `duration_ms`, `success`, `error`
+  - **Attributes**: `session.id`, optional ARMS extension `gen_ai.user.id`, `gen_ai.operation.name` (`execute_tool`), `gen_ai.tool.name`, `gen_ai.tool.type` (`function`), `gen_ai.tool.call.id`, `tool.call_id`, `duration_ms`, `success`, `error`, `tool.failure_kind` (string, optional — the specific failure reason, e.g. "cancelled", "tool_error", "tool_exception", "timeout", "permission_denied", "pre_hook_blocked")
 
-- `qwen-code.tool.execution`: Wraps the tool execution phase (after approval).
-  - **Attributes**: `session.id`, `duration_ms`, `success`, `error`
+- `qwen-code.tool.execution`: Wraps the tool execution phase (after approval). Emitted only for attempted executions.
+  - **Attributes**: `session.id`, `gen_ai.tool.name` (optional), `tool.call_id` (optional), `duration_ms`, `success`, `error`, `execution_status` ("success"/"error"/"cancelled"), `error_type`, `error.type`
 
 - `qwen-code.tool.blocked_on_user`: Time a tool spends waiting on user approval.
   - **Attributes**: `session.id`, `tool.name`, `tool.call_id`, `duration_ms`, `decision` ("proceed_once"/"proceed_always"/"cancel"/"aborted"/"auto_approved"/"error"), `source` ("cli"/"ide"/"hook"/"auto"/"system")