Feat: Add reasoning summary and tool details display for AI responses #14414
Conversation
…d queries and migrations - Added optional reasoningSummary field to AgentChatMessage in GraphQL schema. - Updated GetAgentChatMessagesQuery to include reasoningSummary. - Modified GraphQL query for fetching agent chat messages to retrieve reasoningSummary. - Created migration to add reasoningSummary column to agentChatMessage table in the database. - Updated AgentChatService and DTO to handle reasoningSummary in message creation and data transfer.
- Introduced `reasoningSummary` field in `agentStreamingMessageState` for tracking reasoning messages. - Updated `useAgentChat` hook to handle `onReasoningSummary` callback, appending messages to the reasoning summary. - Modified `AgentStreamingEvent` type to include a new event type for reasoning messages. - Enhanced `parseAgentStreamingChunk` to process reasoning events accordingly.
- Introduced a new `ReasoningSummaryDisplay` component to show reasoning summaries in the chat interface. - Updated `AIChatMessage` to include a `reasoningSummary` field in the `agentStreamingMessage` type. - Implemented logic to conditionally render the `ReasoningSummaryDisplay` based on the message state. - Enhanced the UI to handle reasoning states and provide a toggle for expanded view of reasoning summaries.
- Added support for Anthropic model provider in the AgentExecutionService. - Introduced conditional providerOptions for the Anthropic model, enabling reasoning budget configuration. - Cleaned up imports by removing duplicate AIBillingService import.
- Introduced a new variable `reasoningSummary` to accumulate reasoning messages during streaming. - Enhanced the streaming logic to process and send reasoning messages as part of the response. - Updated the final message sent to include the accumulated `reasoningSummary` for better context in agent interactions.
- Updated HttpToolParametersZodSchema, SendEmailToolParametersZodSchema, and AGENT_HANDOFF_SCHEMA to replace toolDescription with loadingMessage. - Added completionMessage to each schema for improved user feedback during tool execution.
- Introduced a new migration to add a jsonb column "streamData" to the "agentChatMessage" table. - Updated the AgentChatMessageEntity to include the new streamData field. - Modified the AgentChatService to accept streamData in message creation. - Enhanced the AgentChatMessageDTO to include streamData for data transfer.
…iles - Changed the type of the streamData field from jsonb to text in the agentChatMessage table migration. - Updated the AgentChatMessageEntity to reflect the new text type for streamData. - Modified the AgentChatService to accept streamData as a string during message creation. - Adjusted the AgentChatMessageDTO to define streamData as a string for data transfer.
- Updated the AgentChatMessage type in both generated GraphQL files to include the new streamData field. - Adjusted the GetAgentChatMessagesQuery type to reflect the addition of streamData, ensuring compatibility with the updated schema.
…ualization - Introduced a new LoadingExpandableDisplay component to manage loading states with expandable content. - Implemented styled components for loading text, toggle button, and content display. - Added animation for loading text to improve user experience during data fetching.
- Updated the GET_AGENT_CHAT_MESSAGES query to include the new streamData field, enhancing the data returned for agent chat messages. - This change aligns with recent updates to the AgentChatMessage type and ensures consistency across the GraphQL schema.
…treamed chat messages - Introduced AIChatMessageStreamRenderer component to render chat messages from a stream, supporting tool calls, reasoning, and text steps. - Added groupSteps utility to organize streamed events into structured steps, enhancing the processing of chat message data. - Utilized styled components for consistent UI presentation during loading and message display.
…e handling - Removed unnecessary complexity in the AIChatMessage component by simplifying the agentStreamingMessage structure to a string. - Integrated AIChatMessageStreamRenderer for rendering streamed messages, enhancing the display of tool calls and reasoning summaries. - Cleaned up styled components and removed unused animations to improve code maintainability and performance.
- Integrated a new parseStream utility to simplify the processing of streamed chat messages, replacing the previous event handling logic. - Updated AIChatMessageStreamRenderer to utilize the new parsing method, improving the structure and readability of the code. - Introduced ReasoningSummaryDisplay for better visualization of reasoning steps, enhancing user experience during AI interactions.
- Replaced inline rendering logic with dedicated components for tool, reasoning, and text steps, improving code organization and readability. - Introduced ReasoningStepRenderer, TextStepRenderer, and ToolStepRenderer for better separation of concerns in rendering different message types. - Updated parseStream utility to align with new step structure, enhancing the overall processing of streamed chat messages.
- Replaced ReasoningStepRenderer with ReasoningSummaryDisplay for improved rendering of reasoning steps. - Removed the now redundant ReasoningStepRenderer component to streamline the codebase. - Updated styling and props in ReasoningSummaryDisplay for better handling of reasoning content and thinking states.
…splay components - Introduced a new ShimmerEffect component to enhance loading state visuals. - Updated LoadingExpandableDisplay and ReasoningSummaryDisplay to utilize the Shimmer effect for loading text, improving user experience during data fetching. - Removed inline shimmer animation styles from both components for cleaner code and better maintainability.
…lity into ToolStepRenderer - Deleted the LoadingExpandableDisplay component to streamline the codebase. - Refactored ToolStepRenderer to include loading state handling and expandable content display. - Enhanced user experience by maintaining the loading visuals and toggle functionality within ToolStepRenderer.
…aw stream data logging - Simplified the sendStreamEvent method to accept a generic event object, reducing redundancy in event handling. - Introduced rawStreamString to accumulate and log the raw stream data for better debugging and analysis. - Enhanced error handling and logging for improved reliability during streaming operations.
…sage - Updated ToolStepRenderer to display loadingMessage instead of completionMessage for better user feedback during tool execution. - Removed completionMessage from ToolCallEvent and related schemas in HTTP and email tools, streamlining the data structure. - Enhanced clarity in tool status messaging by focusing on loading states.
…xibility in ToolStepRenderer
- Updated ToolService methods to consistently return success status, messages, and results in a unified format. - Enhanced error handling across various tools, ensuring error messages are included in the response. - Modified HttpTool and SendEmailTool to align with the new response structure, improving clarity and consistency in API responses. - Adjusted ToolOutput type to enforce required success and message fields, enhancing type safety.
…ages - Updated ToolStepRenderer to support standardized response formats, improving error and result handling. - Introduced displayMessage for clearer user feedback based on tool execution results. - Enhanced conditional rendering for success and error states, ensuring better visibility of tool outcomes. - Added StyledPre for improved formatting of JSON results in the expandable content area.
- Introduced a new function to determine and render appropriate icons based on tool names, improving user interface clarity. - Added styled container for icon and text alignment, enhancing the overall layout of tool step displays. - Updated imports to include necessary icon components from the UI library.
- Adjusted background size and position properties to enhance the shimmer effect animation. - Simplified keyframe definitions for smoother transitions, improving visual feedback during loading states.
- Condensed error message rendering by removing unnecessary markup, improving readability. - Enhanced clarity of the component structure while maintaining functionality.
…and filtering - Updated mapMessagesToCoreMessages method to use TypeScript's 'as const' for role assignments, enhancing type safety. - Added a filter to exclude messages with empty content, improving the quality of mapped messages.
… in AgentExecutionService to streamline code and improve clarity.
…dering - Introduced AIChatAssistantMessageRenderer to handle various message types including tool, reasoning, text, and error. - Updated AIChatMessage to utilize the new renderer for assistant messages, improving structure and clarity. - Renamed AIChatMessageStreamRendererProps to AIChatAssistantMessageRendererProps for consistency with the new component.
- Moved error message extraction logic to a new utility function for better separation of concerns and reusability. - Updated ErrorStepRenderer to utilize the new extractErrorMessage function, improving clarity and maintainability. - Adjusted styling properties for consistency with theme usage.
- Simplified prop definition by directly destructuring in the function parameters, enhancing readability. - Updated the condition for rendering the finished state to use hasContent and isThinking, improving clarity in the component's logic.
- Integrated IconBrain into the ReasoningSummaryDisplay component to visually indicate thinking and finished states. - Updated StyledToggleButton to a div for better styling flexibility and adjusted font size for improved readability. - Introduced StyledIconContainer for consistent icon and text alignment, enhancing overall component aesthetics.
- Added a StyledStepsContainer to the AIChatAssistantMessageRenderer for improved layout of message steps. - Updated the rendering logic to wrap steps in the new container, enhancing visual organization. - Removed unnecessary margin from ToolStepRenderer for cleaner styling.
- Consolidated the rendering of the thinking state by wrapping the Shimmer and reasoning text in a fragment for improved structure. - Enhanced readability and maintainability of the component by reducing redundant conditional checks.
…ith LazyMarkdownRenderer - Removed the TextStepRenderer component and integrated LazyMarkdownRenderer for rendering text steps. - Updated import statements accordingly to streamline the component structure and improve maintainability.
- Modified the result property to include success status, optional result, error message, and a message string for improved clarity and usability. - This change aims to provide a more comprehensive representation of tool results in the AI module.
- Removed unused StyledLoadingText component to streamline the code. - Simplified the props definition by directly destructuring in the function parameters. - Updated loading message rendering to use StyledDisplayMessage for better clarity and consistency in styling.
- Removed the AIChatAssistantMessageRendererProps type and directly defined the props in the component function parameters for improved clarity. - Cleaned up the streamTypes file by deleting the unused props type, streamlining the codebase.
- Simplified the logic for extracting error messages by consolidating conditions and reducing redundancy. - Enhanced clarity by returning a default message for unexpected errors, ensuring consistent error reporting.
- Changed the fallback icon from IconDatabase to IconTool to ensure a more relevant representation in the tool icon mapping.
- Deleted the groupSteps utility from the AI module as it was no longer needed, streamlining the codebase and reducing complexity.
…t type - Updated the constructAssistantMessageContentFromStream function to eliminate the FilePart type from its return type, simplifying the output structure to only include TextPart and ReasoningPart. This change enhances clarity and focuses the function's purpose.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Greptile Summary
This PR implements a comprehensive AI response enhancement system that adds reasoning summary display and tool execution details to the Twenty CRM's AI chat interface. The changes introduce three main features: collapsible AI reasoning summaries with real-time streaming, detailed tool execution rendering with expandable results and loading states, and improved error handling for stream events.
The implementation spans both frontend and backend components. On the backend, a new streamData field has been added to the AgentChatMessage entity to store structured streaming information, with corresponding database migrations and GraphQL schema updates. The AI execution service has been enhanced to support reasoning capabilities for Anthropic models with a 12,000 token budget, and all tools (HTTP, SendEmail, Workflow, etc.) have been refactored to return standardized response formats with success/error indicators and user-friendly messages.
On the frontend, the changes introduce several new React components: ReasoningSummaryDisplay for collapsible AI thinking process visualization, ToolStepRenderer for detailed tool execution display with loading states, ErrorStepRenderer for consistent error formatting, and AIChatAssistantMessageRenderer as the main orchestrator. A new parseStream utility function processes the complex streaming data into structured ParsedStep objects, replacing the simpler callback-based approach. The streaming architecture has been simplified at the hook level, moving parsing responsibility to dedicated presentation components.
The refactoring standardizes tool output formats across the system with a consistent ToolOutput interface that includes success, message, result, and optional error fields. This enables the frontend to reliably parse and display tool execution states, loading indicators, and error conditions. The system now supports provider-specific AI model capabilities, distinguishing models that support reasoning (currently Anthropic Claude models) from those that don't.
Confidence score: 3/5
- This PR introduces significant architectural changes to AI response handling but contains several implementation issues that could cause runtime problems
- Score lowered due to critical bugs including typos in type definitions, potential data loss in streaming logic, and missing error handling in key utility functions
- Pay close attention to
packages/twenty-front/src/modules/ai/types/streamTypes.ts,packages/twenty-server/src/engine/metadata-modules/agent/agent-streaming.service.ts, andpackages/twenty-server/src/engine/metadata-modules/agent/utils/constructAssistantMessageContentFromStream.ts
40 files reviewed, 16 comments
| return { result: response.data }; | ||
| return { | ||
| success: true, | ||
| message: `HTTP ${method} request to ${url} completed successfully`, |
There was a problem hiding this comment.
style: Consider making message more descriptive by including response status code: HTTP ${method} request to ${url} completed successfully (${response.status})
| }: { | ||
| content: string; | ||
| isThinking?: boolean; | ||
| }) => { |
There was a problem hiding this comment.
style: Consider using TypeScript type instead of inline object for component props per project convention
| }: { | |
| content: string; | |
| isThinking?: boolean; | |
| }) => { | |
| type ReasoningSummaryDisplayProps = { | |
| content: string; | |
| isThinking?: boolean; | |
| }; | |
| export const ReasoningSummaryDisplay = ({ | |
| content, | |
| isThinking = false, | |
| }: ReasoningSummaryDisplayProps) => { |
Context Used: Context - Use 'type' instead of 'interface' for type definitions in the project. (link)
packages/twenty-front/src/modules/ai/components/ReasoningSummaryDisplay.tsx
Outdated
Show resolved
Hide resolved
packages/twenty-front/src/modules/ai/components/ReasoningSummaryDisplay.tsx
Outdated
Show resolved
Hide resolved
| const agentStreamingMessage = useRecoilValue(agentStreamingMessageState); | ||
| const isStreaming = Boolean(agentStreamingMessage); | ||
|
|
||
| const steps = parseStream(streamData); |
There was a problem hiding this comment.
style: Consider memoizing parseStream result with useMemo to avoid reparsing stream data on every render, especially for large streams.
packages/twenty-server/src/engine/metadata-modules/agent/agent-execution.service.ts
Outdated
Show resolved
Hide resolved
| } | ||
| `; | ||
|
|
||
| export const ToolStepRenderer = ({ events }: { events: ToolEvent[] }) => { |
There was a problem hiding this comment.
style: Consider defining a TypeScript interface for component props to enhance maintainability
| export const ToolStepRenderer = ({ events }: { events: ToolEvent[] }) => { | |
| type ToolStepRendererProps = { | |
| events: ToolEvent[]; | |
| }; | |
| export const ToolStepRenderer = ({ events }: ToolStepRendererProps) => { |
Context Used: Context - Use TypeScript interface definitions for component props to enhance maintainability. (link)
| @@ -95,8 +102,7 @@ export class AgentHandoffExecutorService { | |||
|
|
|||
| return { | |||
| success: false, | |||
| newAgentId: handoffRequest.toAgentId, | |||
| newAgentName: 'Unknown', | |||
| message: `Failed to execute handoff to agent ${handoffRequest.toAgentId}`, | |||
There was a problem hiding this comment.
style: Using handoffRequest.toAgentId instead of targetAgent.name in error message creates inconsistency. If targetAgent was found earlier, use targetAgent.name for consistency with success case.
| default: | ||
| if (textContent) { | ||
| output.push({ | ||
| type: 'text', | ||
| text: textContent, | ||
| }); | ||
| textContent = ''; | ||
| } | ||
| break; | ||
| } |
There was a problem hiding this comment.
logic: The default case flushes text content for any unrecognized event type. This could cause premature text flushing and interfere with proper event processing. Consider removing this logic or being more specific about when to flush.
There was a problem hiding this comment.
Greptile Summary
This PR implements a comprehensive AI reasoning summary and tool details display system that significantly enhances the user experience in AI chat interactions. The changes introduce three major features:
Reasoning Summary Display: The system now captures and displays AI thinking processes in real-time. For AI models that support reasoning (currently Anthropic/Claude models), the implementation adds a doesSupportThinking flag to the model registry and configures reasoning budget tokens (12,000). Stream data containing reasoning steps is parsed and rendered in collapsible UI components that show the AI's thought process as it arrives.
Tool Execution Visualization: Tool execution has been completely restructured with standardized output formats. All tools (HTTP, email, workflow, database operations) now return consistent ToolOutput objects with success, message, and result fields. This enables the new ToolStepRenderer component to display tool calls with loading states (using shimmer effects), expandable results, and clear success/error indicators. The system handles tool call/result pairing through toolCallId matching.
Enhanced Error Handling: Error events from the AI stream are now captured and displayed through a dedicated ErrorStepRenderer component that provides user-friendly error messages with proper formatting and visual styling.
The architecture separates streaming data collection from presentation logic. Raw stream data is persisted in a new streamData database column, allowing reconstruction of conversation history with full context. Stream parsing has been centralized in utility functions that handle different event types (text-delta, tool-call, tool-result, reasoning, error), while specialized React components render each type appropriately. The system maintains backward compatibility while enabling rich AI interaction visualization throughout the application.
Confidence score: 3/5
- This PR introduces significant architectural changes that affect multiple layers of the application, from database schema to frontend rendering
- Score reflects the complexity of streaming data handling, type assertion usage, and removal of comprehensive test coverage for streaming functionality
- Pay close attention to the streaming service refactoring and test file deletions that may impact error handling reliability
Context used:
Rule - Use lingui for all user-facing strings to make them translatable, including field type labels and UI text. (link)
Context - Use TypeScript interface definitions for component props to enhance maintainability. (link)
Context - Use 'type' instead of 'interface' for type definitions in the project. (link)
Context - Ensure that type assertions are safe and consider using type guards instead of direct assertions. (link)
39 files reviewed, 3 comments
| return error.data.error.message; | ||
| } | ||
|
|
||
| return 'An unexpected error occurred'; |
There was a problem hiding this comment.
style: Consider using lingui for the fallback error message to make it translatable
| return 'An unexpected error occurred'; | |
| return t`An unexpected error occurred`; |
Context Used: Rule - Use lingui for all user-facing strings to make them translatable, including field type labels and UI text. (link)
| }; | ||
| } catch (error) { | ||
| return { | ||
| success: false, | ||
| error: error.message, | ||
| message: `Failed to create workflow "${parameters.name}": ${error.message}`, |
There was a problem hiding this comment.
style: Both message and error fields contain the same error information. Consider if both are needed or if one should contain different details (e.g., user-friendly message vs technical error).
…for improved clarity and accuracy in type definition.
…ay component - Added localization support by using the `t` function from `@lingui/core/macro` for the "Thinking..." and "Finished thinking" messages, enhancing internationalization capabilities.
…downRenderer for improved rendering consistency
…ing a default empty string, ensuring consistent message content construction.
- Modified the condition for finding tool entries to check if any event within the item has a type of 'tool-call' and matches the toolCallId, enhancing the accuracy of event parsing.
- Adjusted the expected results in the parseStream test cases to ensure accurate validation of tool and reasoning events. - Added a new expectation for the 'Finished!' text event to enhance test coverage.
Closes #1405
Implemented Features
Demo
Screen.Recording.2025-09-12.at.12.44.00.AM.mov