diff --git a/docs/developers/development/telemetry.md b/docs/developers/development/telemetry.md index cee947a0857..f145738131e 100644 --- a/docs/developers/development/telemetry.md +++ b/docs/developers/development/telemetry.md @@ -76,7 +76,7 @@ These settings can be overridden by environment variables or CLI flags. | `otlpLogsEndpoint` | `QWEN_TELEMETRY_OTLP_LOGS_ENDPOINT` | - | Per-signal endpoint override for logs (HTTP only) | URL string | - | | `otlpMetricsEndpoint` | `QWEN_TELEMETRY_OTLP_METRICS_ENDPOINT` | - | Per-signal endpoint override for metrics (HTTP only) | URL string | - | | `outfile` | `QWEN_TELEMETRY_OUTFILE` | `--telemetry-outfile ` | Save telemetry to file (overrides OTLP export) | file path | - | -| `logPrompts` | `QWEN_TELEMETRY_LOG_PROMPTS` | `--telemetry-log-prompts` / `--no-telemetry-log-prompts` | Include prompts in telemetry logs | `true`/`false` | `true` | +| `logPrompts` | `QWEN_TELEMETRY_LOG_PROMPTS` | `--telemetry-log-prompts` / `--no-telemetry-log-prompts` | Include user prompt content and API request/response text in telemetry logs | `true`/`false` | `true` | | `userId` | `QWEN_TELEMETRY_USER_ID` | - | Stable end-user identifier written to GenAI spans as the ARMS extension `gen_ai.user.id`; prefer a pseudonymous value | string | - | | `includeSensitiveSpanAttributes` | `QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES` | - | Include standard GenAI messages, instructions, tool definitions, tool arguments, and successful tool results as native span attributes | `true`/`false` | `false` | | `sensitiveSpanAttributeMaxLength` | `QWEN_TELEMETRY_SENSITIVE_SPAN_ATTRIBUTE_MAX_LENGTH` | - | Maximum compact JSON string length for each sensitive native span attribute. Set lower if your backend rejects large attributes. | `1..104857600` | `1048576` | @@ -128,8 +128,9 @@ two things happen: therefore occupy more bytes after OTLP export. 2. **Log-to-span bridge spans** (used when HTTP traces are exported without a - logs endpoint) keep their existing `prompt`, `function_args`, and - `response_text` fields, instead of being dropped. + logs endpoint) retain `function_args`, `error`, `error.message`, and + `error_message`, plus `prompt`, `request_text`, and `response_text` when + `logPrompts` is also enabled, instead of dropping these attributes. ⚠️ **Security warning:** enabling this flag streams full conversation history, file contents read by `read_file`, shell commands and their output (including @@ -642,10 +643,10 @@ The following events are logged: #### API Events - `qwen-code.api_request`: Outgoing request to the LLM API. - - **Attributes**: `model` (string), `prompt_id` (string), `request_text` (string, optional), `subagent_name` (string, optional) + - **Attributes**: `model` (string), `prompt_id` (string), `request_text` (string, optional — contains request content only when `log_prompts_enabled` is true; log-to-span bridge spans additionally require `includeSensitiveSpanAttributes`; opaque `thoughtSignature` provider payload is included, with its policy decision tracked in #11682), `subagent_name` (string, optional) - `qwen-code.api_response`: Response received from LLM API. - - **Attributes**: `response_id` (string), `model` (string), `status_code` (int/string, optional), `duration_ms` (int), `input_token_count` (int), `output_token_count` (int), `cached_content_token_count` (int), `thoughts_token_count` (int), `total_token_count` (int), `prompt_id` (string), `auth_type` (string, optional), `response_text` (string, optional), `subagent_name` (string, optional) + - **Attributes**: `response_id` (string), `model` (string), `status_code` (int/string, optional), `duration_ms` (int), `input_token_count` (int), `output_token_count` (int), `cached_content_token_count` (int), `thoughts_token_count` (int), `total_token_count` (int), `prompt_id` (string), `auth_type` (string, optional), `response_text` (string, optional — contains visible response content only when `log_prompts_enabled` is true; log-to-span bridge spans additionally require `includeSensitiveSpanAttributes`; it carries no content for internal prompt ids or responses with no visible text), `subagent_name` (string, optional) - `qwen-code.api_error`: API request failed. - **Attributes**: `model` (string), `prompt_id` (string), `duration_ms` (int), `error_message` (string), `response_id` (string, optional), `auth_type` (string, optional), `error_type` (string, optional), `status_code` (int/string, optional), `subagent_name` (string, optional) diff --git a/docs/users/configuration/settings.md b/docs/users/configuration/settings.md index 790569fe2cb..fb0c5e271ad 100644 --- a/docs/users/configuration/settings.md +++ b/docs/users/configuration/settings.md @@ -661,7 +661,7 @@ Configures logging and metrics collection for Qwen Code. For more information, s | `telemetry.target` | string | Informational label for the telemetry destination (`local` or `gcp`). Does not control exporter routing; set `telemetry.otlpEndpoint` or `telemetry.outfile` to configure where data is sent. | | | `telemetry.otlpEndpoint` | string | The endpoint for the OTLP Exporter. | | | `telemetry.otlpProtocol` | string | The protocol for the OTLP Exporter (`grpc` or `http`). | | -| `telemetry.logPrompts` | boolean | Whether or not to include the content of user prompts in the logs. | | +| `telemetry.logPrompts` | boolean | Whether or not to include user prompt content and API request/response text in the logs. | | | `telemetry.userId` | string | Stable end-user identifier written to GenAI spans as the ARMS extension `gen_ai.user.id`. Prefer a pseudonymous value. Do not set a process-wide value for a shared multi-user daemon or channel instance. | | | `telemetry.includeSensitiveSpanAttributes` | boolean | When enabled, attaches verbatim user prompts, system prompts, tool inputs/outputs, and model responses to native OTel span attributes (in addition to log-to-span bridge spans). ⚠️ Streams sensitive data — file contents, shell commands, conversation history — to your OTLP backend. | `false` | | `telemetry.sensitiveSpanAttributeMaxLength` | number | Maximum JavaScript string length for each sensitive native OTel span attribute content payload. Must be between `1` and `104857600` (100 MiB). Set lower if your collector or backend rejects large attributes. | `1048576` | @@ -804,7 +804,7 @@ For authentication-related variables (like `OPENAI_*`) and the recommended `.qwe | `QWEN_TELEMETRY_OTLP_PROTOCOL` | Sets the OTLP protocol (`grpc` or `http`). | Overrides the `telemetry.otlpProtocol` setting. | | `QWEN_TELEMETRY_LOG_PROMPTS` | Set to `true` or `1` to enable or disable logging of user prompts. Any other value is treated as disabling it. | Overrides the `telemetry.logPrompts` setting. | | `QWEN_TELEMETRY_USER_ID` | Sets a stable end-user identifier on interaction, LLM, Tool, and Agent spans as `gen_ai.user.id`. Prefer a pseudonymous value. | Overrides `telemetry.userId` after trimming. A blank value falls back to settings. This is process-wide and must not be used as per-request identity in a shared multi-user process. | -| `QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES` | Set to `true` or `1` to attach verbatim user prompts, system prompts, tool I/O, and model responses to native OTel span attributes (and keep `prompt` / `function_args` / `response_text` on log-to-span bridge spans). Any other value disables it. | Overrides the `telemetry.includeSensitiveSpanAttributes` setting. ⚠️ Streams sensitive data to your OTLP backend. | +| `QWEN_TELEMETRY_INCLUDE_SENSITIVE_SPAN_ATTRIBUTES` | Set to `true` or `1` to attach verbatim user prompts, system prompts, tool I/O, and model responses to native OTel span attributes (and keep `function_args`, `error`, `error.message`, and `error_message` on log-to-span bridge spans, plus `prompt` / `request_text` / `response_text` when `telemetry.logPrompts` is also enabled). Any other value disables it. | Overrides the `telemetry.includeSensitiveSpanAttributes` setting. ⚠️ Streams sensitive data to your OTLP backend. | | `QWEN_TELEMETRY_SENSITIVE_SPAN_ATTRIBUTE_MAX_LENGTH` | Sets the maximum JavaScript string length for each sensitive native OTel span attribute content payload. Must be a positive integer no greater than `104857600` (100 MiB). | Overrides the `telemetry.sensitiveSpanAttributeMaxLength` setting. Default is `1048576` (1 MiB); lower it if your collector or backend rejects large span attributes. | | `QWEN_TELEMETRY_OUTFILE` | Sets the file path to write telemetry to. When set, overrides OTLP export. | Overrides the `telemetry.outfile` setting. | | `QWEN_SANDBOX` | Alternative to the `sandbox` setting in `settings.json`. | Accepts `true`, `false`, `docker`, `podman`, or a custom command string. | diff --git a/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts b/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts index e21a20f65d8..d619a3d1063 100644 --- a/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts +++ b/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.test.ts @@ -327,6 +327,8 @@ const createConfig = (overrides: Record = {}): Config => { getWorkingDir: () => process.cwd(), getTelemetryIncludeSensitiveSpanAttributes: () => Boolean(configContent['includeSensitiveSpanAttributes']), + getTelemetryLogPromptsEnabled: () => + Boolean(configContent['logPrompts'] ?? true), getTelemetrySensitiveSpanAttributeMaxLength: () => (configContent['sensitiveSpanAttributeMaxLength'] as number) ?? 1024 * 1024, @@ -844,6 +846,156 @@ describe('LoggingContentGenerator', () => { expect(openaiError).toBeUndefined(); }); + it('omits request_text and response_text from API telemetry when logPrompts is false', async () => { + const wrapped = createWrappedGenerator( + vi + .fn() + .mockResolvedValue( + createResponse('resp-noprompts', 'test-model', [ + { text: 'SENSITIVE_RESPONSE_MARKER' }, + ]), + ), + vi.fn(), + ); + const generator = new LoggingContentGenerator( + wrapped, + createConfig({ logPrompts: false }), + { + model: 'test-model', + authType: AuthType.USE_OPENAI, + }, + ); + + await generator.generateContent( + { + model: 'test-model', + contents: [ + { role: 'user', parts: [{ text: 'SENSITIVE_REQUEST_MARKER' }] }, + ], + } as unknown as GenerateContentParameters, + 'prompt-noprompts', + ); + + expect(logApiRequest).toHaveBeenCalledTimes(1); + const [, requestEvent] = vi.mocked(logApiRequest).mock.calls[0]; + expect(requestEvent.request_text).toBeUndefined(); + + expect(logApiResponse).toHaveBeenCalledTimes(1); + const [, responseEvent] = vi.mocked(logApiResponse).mock.calls[0]; + expect(responseEvent.response_text).toBeUndefined(); + }); + + it('keeps request_text and response_text in API telemetry when logPrompts is true', async () => { + const wrapped = createWrappedGenerator( + vi + .fn() + .mockResolvedValue( + createResponse('resp-prompts', 'test-model', [ + { text: 'KEEP_RESPONSE_MARKER' }, + ]), + ), + vi.fn(), + ); + const generator = new LoggingContentGenerator( + wrapped, + createConfig({ logPrompts: true }), + { + model: 'test-model', + authType: AuthType.USE_OPENAI, + }, + ); + + await generator.generateContent( + { + model: 'test-model', + contents: [{ role: 'user', parts: [{ text: 'KEEP_REQUEST_MARKER' }] }], + } as unknown as GenerateContentParameters, + 'prompt-prompts', + ); + + const [, requestEvent] = vi.mocked(logApiRequest).mock.calls[0]; + expect(requestEvent.request_text).toContain('KEEP_REQUEST_MARKER'); + + const [, responseEvent] = vi.mocked(logApiResponse).mock.calls[0]; + expect(responseEvent.response_text).toBe('KEEP_RESPONSE_MARKER'); + }); + + it('omits request_text and response_text from API telemetry for streaming when logPrompts is false', async () => { + const streamFn = vi.fn().mockResolvedValue( + (async function* () { + yield createResponse('resp-stream-noprompts', 'test-model', [ + { text: 'SENSITIVE_RESPONSE_MARKER' }, + ]); + })(), + ); + const wrapped = createWrappedGenerator(vi.fn(), streamFn); + const generator = new LoggingContentGenerator( + wrapped, + createConfig({ logPrompts: false }), + { + model: 'test-model', + authType: AuthType.USE_OPENAI, + }, + ); + + const stream = await generator.generateContentStream( + { + model: 'test-model', + contents: [ + { role: 'user', parts: [{ text: 'SENSITIVE_REQUEST_MARKER' }] }, + ], + } as unknown as GenerateContentParameters, + 'prompt-stream-noprompts', + ); + for await (const _ of stream) { + // Drain the stream so response logging finalizes. + } + + expect(logApiRequest).toHaveBeenCalledTimes(1); + const [, requestEvent] = vi.mocked(logApiRequest).mock.calls[0]; + expect(requestEvent.request_text).toBeUndefined(); + + expect(logApiResponse).toHaveBeenCalledTimes(1); + const [, responseEvent] = vi.mocked(logApiResponse).mock.calls[0]; + expect(responseEvent.response_text).toBeUndefined(); + }); + + it('keeps request_text and response_text in API telemetry for streaming when logPrompts is true', async () => { + const streamFn = vi.fn().mockResolvedValue( + (async function* () { + yield createResponse('resp-stream-prompts', 'test-model', [ + { text: 'KEEP_RESPONSE_MARKER' }, + ]); + })(), + ); + const wrapped = createWrappedGenerator(vi.fn(), streamFn); + const generator = new LoggingContentGenerator( + wrapped, + createConfig({ logPrompts: true }), + { + model: 'test-model', + authType: AuthType.USE_OPENAI, + }, + ); + + const stream = await generator.generateContentStream( + { + model: 'test-model', + contents: [{ role: 'user', parts: [{ text: 'KEEP_REQUEST_MARKER' }] }], + } as unknown as GenerateContentParameters, + 'prompt-stream-prompts', + ); + for await (const _ of stream) { + // Drain the stream so response logging finalizes. + } + + const [, requestEvent] = vi.mocked(logApiRequest).mock.calls[0]; + expect(requestEvent.request_text).toContain('KEEP_REQUEST_MARKER'); + + const [, responseEvent] = vi.mocked(logApiResponse).mock.calls[0]; + expect(responseEvent.response_text).toContain('KEEP_RESPONSE_MARKER'); + }); + it('creates and closes the non-stream API span on success', async () => { const wrapped = createWrappedGenerator( vi diff --git a/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts b/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts index f2366a51e2c..8870b143f9f 100644 --- a/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts +++ b/packages/core/src/core/loggingContentGenerator/loggingContentGenerator.ts @@ -235,7 +235,9 @@ export class LoggingContentGenerator implements ContentGenerator { promptId: string, sessionId: string, ): void { - const requestText = JSON.stringify(contents); + const requestText = this.config.getTelemetryLogPromptsEnabled() + ? JSON.stringify(contents) + : undefined; logApiRequest( this.config, new ApiRequestEvent( @@ -268,7 +270,7 @@ export class LoggingContentGenerator implements ContentGenerator { prompt_id, this.generatorAuthType, usageMetadata, - responseText, + this.config.getTelemetryLogPromptsEnabled() ? responseText : undefined, subagentNameContext.getStore(), ttftMs, ), diff --git a/packages/core/src/telemetry/file-exporters.test.ts b/packages/core/src/telemetry/file-exporters.test.ts index 9cca4175fd3..712f29494f0 100644 --- a/packages/core/src/telemetry/file-exporters.test.ts +++ b/packages/core/src/telemetry/file-exporters.test.ts @@ -9,7 +9,8 @@ import * as os from 'node:os'; import * as path from 'node:path'; import { afterEach, beforeEach, describe, expect, it } from 'vitest'; import { ExportResultCode } from '@opentelemetry/core'; -import { FileSpanExporter } from './file-exporters.js'; +import type { ReadableLogRecord } from '@opentelemetry/sdk-logs'; +import { FileLogExporter, FileSpanExporter } from './file-exporters.js'; type SerializeAccess = { serialize: (data: unknown) => string }; @@ -55,6 +56,35 @@ describe('FileExporter.serialize', () => { expect(out).toContain('"[Circular]"'); expect(out.endsWith('\n')).toBe(true); }); + + // Pins the outfile sink's key-absent contract: when logPrompts is off the + // producer emits request_text/response_text as undefined, and safeJsonStringify + // (JSON.stringify) drops undefined-valued keys instead of writing `null` or an + // empty string. FileLogExporter inherits FileExporter.serialize unchanged. + it('FileLogExporter omits undefined-valued attributes (logPrompts off)', async () => { + const outfile = path.join(tmpDir, 'logs.jsonl'); + const logExporter = new FileLogExporter(outfile); + const logRecord = { + body: 'api response', + hrTime: [1000, 0] as [number, number], + attributes: { + request_text: undefined, + response_text: 'visible', + }, + } as unknown as ReadableLogRecord; + + await new Promise((resolve) => { + logExporter.export([logRecord], (result) => { + expect(result.code).toBe(ExportResultCode.SUCCESS); + resolve(); + }); + }); + await logExporter.shutdown(); + + const out = fs.readFileSync(outfile, 'utf8'); + expect(out).not.toContain('request_text'); + expect(out).toContain('"response_text": "visible"'); + }); }); // Telemetry is best-effort and must never crash the agent. The outfile is often diff --git a/packages/core/src/telemetry/log-to-span-processor.test.ts b/packages/core/src/telemetry/log-to-span-processor.test.ts index ea78629fa8b..7ee82be9d1e 100644 --- a/packages/core/src/telemetry/log-to-span-processor.test.ts +++ b/packages/core/src/telemetry/log-to-span-processor.test.ts @@ -173,6 +173,7 @@ describe('LogToSpanProcessor', () => { error_message: 'secret upstream error', prompt: 'secret prompt', function_args: '{"token":"secret"}', + request_text: 'secret request', response_text: 'secret response', error_type: 'RateLimitError', safe: 'visible', @@ -188,6 +189,7 @@ describe('LogToSpanProcessor', () => { expect(attrs).not.toHaveProperty('error_message'); expect(attrs).not.toHaveProperty('prompt'); expect(attrs).not.toHaveProperty('function_args'); + expect(attrs).not.toHaveProperty('request_text'); expect(attrs).not.toHaveProperty('response_text'); expect(attrs['error_type']).toBe('RateLimitError'); expect(attrs['safe']).toBe('visible'); @@ -210,6 +212,7 @@ describe('LogToSpanProcessor', () => { error_message: 'secret upstream error', prompt: 'secret prompt', function_args: '{"token":"secret"}', + request_text: 'secret request', response_text: 'secret response', safe: 'visible', }, @@ -224,6 +227,7 @@ describe('LogToSpanProcessor', () => { expect(attrs['error_message']).toBe('secret upstream error'); expect(attrs['prompt']).toBe('secret prompt'); expect(attrs['function_args']).toBe('{"token":"secret"}'); + expect(attrs['request_text']).toBe('secret request'); expect(attrs['response_text']).toBe('secret response'); expect(attrs['safe']).toBe('visible'); expect(attrs['log.bridge']).toBe(true); diff --git a/packages/core/src/telemetry/log-to-span-processor.ts b/packages/core/src/telemetry/log-to-span-processor.ts index f263e69402b..490e8afc034 100644 --- a/packages/core/src/telemetry/log-to-span-processor.ts +++ b/packages/core/src/telemetry/log-to-span-processor.ts @@ -61,6 +61,7 @@ const SENSITIVE_ATTRIBUTE_KEYS = new Set([ 'error_message', 'prompt', 'function_args', + 'request_text', 'response_text', ]);