feat: add ContentBlock streaming visualization #4370
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…erties' in message processing
…t` field in `ContentBlock` model
…te_message function
…data' for chat attributes
- Introduced BaseContent interface and various content types: ErrorContent, TextContent, MediaContent, JSONContent, CodeContent, ToolStartContent, ToolEndContent, and ToolErrorContent. - Created a union type ContentType for all content types. - Updated ContentBlock interface to include new content structure and additional fields.
- Introduced new properties: `background_color`, `chat_icon`, and `text_color` to chat components across various starter projects. - These properties allow for customization of the chat message appearance, including icon and text styling. - Updated the `ChatInput` and `ChatOutput` components to handle these new properties and include them in the message response.
…handling in chat IO
- Modified event handler functions to return a tuple of `agent_message` and `start_time`. - Updated `_calculate_duration` to handle `start_time` as an integer. - Ensured `start_time` is reset using `perf_counter()` after each event handling. - Adjusted tool and chain event handler calls to accommodate the new return type.
- Updated test cases for `handle_on_chain_start`, `handle_on_chain_end`, `handle_on_tool_start`, `handle_on_tool_end`, `handle_on_tool_error`, and `handle_on_chain_stream` to include `start_time` as a return value. - Added assertions to verify that `start_time` is a float and `duration` is an integer where applicable.
… event notification
…handling - Introduced new language model options: Anthropic, Groq, and NVIDIA. - Refactored `AgentComponent` to utilize `MODEL_PROVIDERS_DICT` for dynamic provider handling. - Simplified input management by removing hardcoded provider inputs and using a dictionary-based approach. - Enhanced `update_build_config` to support flexible provider configurations and custom options.
joaoguilhermeS
pushed a commit
that referenced
this pull request
Nov 7, 2024
* Refactor cardComponent and storeCardComponent to comment out playground buttons * Refactor message model to replace 'meta_data' with 'properties' and update related schema and components * Add new schema for content types with Pydantic models * Add ContentBlock schema with support for multiple content types * Refactor: Import ContentBlock from langflow.schema.content_block instead of defining locally * Add SendMessageFunctionType protocol to log schema for message handling * Refactor chat message model to replace 'meta_data' with 'properties' and update related components * Add ContentBlock import and enhance send_message method in component.py - Import ContentBlock from langflow.schema.content_block. - Replace 'meta_data' with 'properties' in message construction. - Update send_message method to handle optional Message and text parameters. * Add async on_chain_start method to callback for logging chain start events * Refactor add_messagetables function to replace 'meta_data' with 'properties' in message processing * Fix typo in 'traceback' property name in ErrorDetails interface * Add condition to check chat history length before rendering chat lock * Refactor error content structure and enhance message event handling * Add `update_messages` function to handle message updates in memory module * Add check for 'id' attribute in stored_message for message validation * Add new content types and TypeAlias for schema in content_types.py * Add custom serialization for UUID and BaseModel types in data schema * Refactor `ContentType` to use `ContentTypes` union and update `content` field in `ContentBlock` model * Update model to use specific types for properties and content_blocks fields * Change return type of 'serialize_params' method to string * Add sender fields and properties handling to PlaygroundEvent and create_message function * Refactor chat message component to use 'properties' instead of 'meta_data' for chat attributes * Refactor message handling to improve error categorization and session management * Enhance ContentBlock initialization to update model fields with defaults * Add new content types and update ContentBlock interface in chat types - Introduced BaseContent interface and various content types: ErrorContent, TextContent, MediaContent, JSONContent, CodeContent, ToolStartContent, ToolEndContent, and ToolErrorContent. - Created a union type ContentType for all content types. - Updated ContentBlock interface to include new content structure and additional fields. * Refactor error handling in chat messages to use `ErrorContent` type * Add ErrorMessage class for handling error-specific messages in schema * Add type annotations for cache and logs in custom_component.py * Refactor error handling in custom component to use `send_error` method * Add customizable properties for chat components - Introduced new properties: `background_color`, `chat_icon`, and `text_color` to chat components across various starter projects. - These properties allow for customization of the chat message appearance, including icon and text styling. - Updated the `ChatInput` and `ChatOutput` components to handle these new properties and include them in the message response. * Add 'category' field to MessageEvent and update create_message function * Remove unused message handling methods from ChatComponent * Refactor message event handling in chat module * Add 'category' field to Message creation and pass 'id_' in _send_message_event * Add 'category' field with Literal type to Message schema and default handling in chat IO * Refactor message event handling in chat module * Refactor buttonSendWrapper component to add data-testid attribute * Refactor ChatComponent to replace 'store_message' with 'send_message' method * refactor: streamline message handling in Component Update message processing to utilize new methods for improved clarity and efficiency. Enhance event data management to ensure accuracy and consistency in message updates. * feat: Update input handling and output storage for chat components Enhance chat input and output components to improve message handling, including updates to storage and display properties. This facilitates better user experience and message management in various agent projects. * Ensure 'id' field is set in message data dictionary if not present * refactor: Simplify property retrieval from source component in ChatComponent * refactor: Replace property retrieval logic with a dedicated method in ChatOutput * Refactor `TokenEvent` class and add `timestamp` field with default factory * refactor: Simplify message processing by removing unused parameters in async handling * Refactor message text update logic in messagesStore * Refactor border styling in ChatInput component * refactor: Consolidate properties handling in PlaygroundEvent and create_message function * refactor: Update timestamp formatting to include timezone information across multiple models * refactor: Add StreamingError exception for better error handling in streaming components * refactor: Add source_display_name attribute to Properties model for enhanced data representation * refactor: Enhance ChatComponent to return source display name and update properties handling in ChatOutput * refactor: Improve error handling in Component class and enhance create_error function with timestamp support * feat: enhance ChatOutput component for improved message handling Update the message_response method to include source display name and improve message storage functionality across various project templates. This enhances clarity and ensures consistent behavior of chat components. * feat: add delete_message function to remove messages by ID from the monitor service * Add error handling for message streaming and implement message deletion on failure * feat: Enhance message streaming and event handling Improve the streaming process to send message events only for non-streaming messages and ensure the initial message is sent correctly when processing chunks, enhancing the overall message handling flow. * improve UI * Refactor buttonSendWrapper component to add data-testid attribute and update tests * fix edited tag * add tooltip to add button * fix bug after merge * update examples * updateinput without chat * fix some frontend tests * fix test * update test * Update fileUploadComponent.spec.ts to fix test and update input without chat * refactor event creation functions and enhance ErrorEvent structure * refactor ErrorMessage to use Source for sender and component details * refactor Properties to use Source model for source details and add validation * refactor ChatMessage component for improved readability and structure * refactor ChatMessageType to use PropertiesType for improved structure and clarity * Add Source ID to message properties and update exception handling - Updated `message_response` method to include `source_id` in the `Properties` object for better traceability. - Modified `StreamingError` to use `Source` object instead of `component_name` for more detailed error context. - Adjusted `get_properties_from_source_component` to return `source_id` along with other properties. * Add 'Source' property to error handling in custom_component module - Introduced 'Source' property to enhance error message details. - Updated error handling to include 'source' instead of 'display_name'. - Modified exception raising to use 'source' for better traceability. * Enhance event ID generation with event type prefix in `send_event` method * Refactor ChatOutput class to use Properties and Source schemas for message properties * update style in tailwind and add fallback for nullable values * fix playground test * Update message handling in Component class to send modified message copy on first chunk * Add flow_id and session_id to ErrorEvent creation function * Update content_blocks initialization and add flow_id parameter to ErrorMessage * feat: include vertex ID in class instantiation Pass the vertex ID to the custom component during instantiation to facilitate event tracking or management capabilities. * fix: update message streaming logic to handle None messages and refactor _stream_message parameters * feat: include flow_id in ChatOutput initialization Add flow_id to the ChatOutput class to improve the tracking of conversational flows, enhancing the context management within the chat system. * fix: update chat message source display to use 'source' instead of 'display_name' * feat: add flow_id to message initialization in ChatOutput class * fix: handle JSON parsing with type checks for message properties Refine JSON loading to ensure proper type handling for message properties and content blocks, enhancing stability and preventing potential errors with non-string inputs. * refactor: update logging structure and message handling Improve the output logs to support logging multiple entries and ensure robust message streaming with type checking. Adjust error handling parameters for consistency. * feat: restrict event types in registration and sending Enhance event management by explicitly defining allowed event types, improving code clarity and reducing potential errors. This ensures only valid event types are registered and sent, leading to more robust event handling. * Update `logs` attribute to store lists of `Log` objects in `Vertex` class * feat: introduce TypedDicts for ContentBlock and Properties, update default values in Message model * fix: restrict event types in send_event method to improve data validation * Set default values for 'id', 'display_name', and 'source' fields in Source schema * [autofix.ci] apply automated fixes * fix: update query to use equality check for error messages * make lint * update simple agent test * test: enhance EventManager tests for event_id validation * feat: add background, chat icon, and text color properties to component toolkit * fix: update LogComponent name to display_name for consistency * remove playground from new cards on main page * refactor: Update Properties handling in MessageTable and remove unused PropertiesDict * fix: Set default value for category in MessageBase model * fix: Update properties default factory in MessageTable model to use model_dump * Add _build_source method to ChatOutput and update test inputs to use Message objects * Fix incorrect parameter names in _build_source method across multiple JSON files * ✅ (freeze.spec.ts, playground.spec.ts, stop-building.spec.ts, linkComponent.spec.ts, sliderComponent.spec.ts, tableInputComponent.spec.ts, stop-button-playground.spec.ts, generalBugs-shard-7.spec.ts): enable tests that were previously skipped to ensure proper functionality and coverage. Remove outdated comments and update test descriptions for clarity. * Refactor agent event processing to support message streaming and improve content handling * Refactor ChatOutput to enhance message handling and streamline property assignment * Fix session ID assignment in send_message method for better message handling * Add OnTokenFunctionType protocol for enhanced token handling * Add BorderTrail component for animated border effects * Add ContentBlockDisplay component for enhanced content visualization and loading state * Refactor TextShimmer to use motion.create for improved component animation * Add ContentBlockDisplay to render chat content blocks in newChatMessage component * Refactor `ChatOutput` class to use `MessageInput` instead of `MessageTextInput` for `input_value` across starter projects. * Update edge class name from 'runned' to 'ran' in flow components and store * Add 'on_build_start' and 'on_build_end' events to EventManager * Add build status updates for 'build_start' and 'build_end' events in buildUtils.ts * Integrate event management for output function in ComponentToolkit to track build start and end events * Refactor event handling in ComponentToolkit to improve build event tracking * Refactor messagesStore to update existing message if it already exists * update properties to have state attribute * format * Refactor chatMessage component to display loading state for partial chat messages * Refactor reactflowUtils to handle broken edges with missing display names * fix agent text output * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix agent component initialization * Enhance error message formatting to include detailed exception information * Refactor agent component for improved performance and stability * Refactor `LCAgentComponent` to streamline agent execution logic and remove redundant methods * Add type casting for send_message function in agent.py * Change return type of function to 'Message' in log schema * Add `embedding_service` and `client` fields to JSON configurations and refactor ChatOutput class - Introduced `embedding_service` field in `Vector Store RAG.json` and `Travel Planning Agents.json` to support Astra Vectorize and embedding model selection. - Added `client` field to `OpenAIEmbeddingsComponent` in JSON configurations for enhanced client configuration. - Refactored `ChatOutput` class across multiple JSON files to include a new `_build_source` method for constructing `Source` objects, improving code modularity and readability. - Updated import paths for `OpenAIEmbeddings` to reflect changes in the module structure. * Update attribute handling in Component class to merge inputs with existing attributes * Update ChatInput and ChatOutput initialization to include session_id * Refactor test_load_flow_from_json_object to be asynchronous * Add asynchronous tests for agent event handling in test_agent_events.py * Add event handling functions for agent tool and chain events - Implement functions to handle various agent events such as `on_chain_start`, `on_chain_end`, `on_tool_start`, `on_tool_end`, and `on_tool_error`. - Introduce helper functions to build content blocks for agent messages. - Create mappings for chain and tool event handlers. - Refactor `process_agent_events` to utilize the new event handling functions. - Remove the old `process_agent_events` implementation from `agent.py` and import the new one from `events.py`. * feat: update content structure and validation logic Enhance content block to accept a list of contents instead of a single item, and implement validation to ensure proper format. Refactor tool-related content types for improved clarity and functionality. * Handle ValidationError in error message generation and update import paths * feat: enhance error event creation to include structured error content * move validators and remove utils.py * feat: add event handler tests for agent tool and chain events * refactor: streamline tool content structure Combine multiple tool-related types into a single, more flexible interface. This simplifies the content handling by allowing optional properties for output, error, and timestamp, enhancing overall code maintainability. Adjust the ContentBlock interface to accommodate an array of contents for improved flexibility. * feat: refactor error handling in ChatMessage component to support dynamic content blocks * feat: enhance ContentBlockDisplay to support Markdown rendering and dynamic content handling * Refactor agent event handling to simplify content block management * Enhance `ContentBlockDisplay` with separators and improved content rendering logic * add inline block to prevent text overflow in reason * Refactor code to enhance Markdown rendering and dynamic content handling in ChatMessage component * [autofix.ci] apply automated fixes * Add timestamp field to BaseContent class and remove from ToolContent class * Update timestamp validation to include fractional seconds in BaseContent class * Add duration calculation for agent and tool events in events.py * Refactor ContentBlock to use Pydantic's Discriminator and Tag for content types * Add field validators for content_blocks and properties in Message class * Add field serializer for source in Properties class * Add type hint for serialize_properties method in MessageTable class * create duration component and refactor contents * [autofix.ci] apply automated fixes * Refactor duration calculation to use perf_counter for improved accuracy and update ContentBlockDict to use dict for contents * Update properties field validator to use 'before' mode for improved validation * Refactor MessageTable to use ContentBlock instead of ContentBlockDict and enhance validation and serialization for properties and content_blocks * Refactor AgentComponent to streamline agent initialization by using the set method directly * Refactor event handling functions to use synchronous definitions and improve performance with asyncio.to_thread * Refactor PythonREPLToolComponent to raise ToolException on error instead of returning error message * Refactor CalculatorToolComponent to enhance error handling by raising ToolException and improving expression evaluation feedback * Fix TextShimmer width in ContentBlockDisplay for improved layout consistency * Enhance ContentBlockDisplay to support external links and math rendering in Markdown * Enhance error handling in Calculator and Python REPL tools by introducing `ToolException` and improving expression evaluation logic. * Refactor agent message properties to remove unnecessary icons and update content block titles for clarity * feat: enhance agent and tool content messaging Add headers to agent and tool content messages for better clarity and user experience. * Handle nested schema structure in ContentBlock initialization * fix: update agent input text content return type and enhance error message formatting * Add placeholder and default value for model selection in AstraVectorStoreComponent * Refactor event handling functions to remove `send_message_method` parameter and streamline message processing logic. * Add model configuration to ToolContent for alias population * Refactor agent event tests to improve message handling and content block initialization * test: Update test durations for performance metrics Adjust the recorded execution times for various test cases to ensure accurate performance tracking and facilitate optimization efforts. * Refactor test_load_flow_from_json_object to improve project loading logic * Make `HeaderDict` fields optional and update `header` field in `BaseContent` * Enhance `Message` and `ChatOutput` with additional attributes for testing * Update test duration metrics in .test_durations file * Refactor `ContentDisplay` component to support multiple content types and improve rendering logic * feat: Enhance duration display with human-readable format Integrate the pretty-ms library to convert duration display metrics into a more user-friendly format, improving the overall user experience. Update dependencies to ensure compatibility with the latest version and optimize for newer Node.js versions. * feat: Add optional duration and header properties to BaseContent interface * fix: Change duration type from float to int in BaseContent class * Remove default value for 'start_time' in duration calculations * Enhance `ContentBlockDisplay` with dynamic header and expand/collapse animation - Added `state` prop to `ContentBlockDisplay` to dynamically set header icon and title based on content state. - Implemented `renderContent` function to encapsulate content rendering logic with animation. - Integrated `ForwardedIconComponent` for displaying header icons. - Updated `newChatMessage.tsx` to pass `state` prop to `ContentBlockDisplay`. * Add animation to header title using AnimatePresence and motion components * Add support for processing zip files with multithreading in FileComponent - Enhanced FileComponent to handle individual and zipped text files. - Introduced multithreading option for parallel processing of files within zip archives. - Updated supported file types to include zip files. - Improved error handling and logging for file processing. * Expand `Message` conversion to support `AsyncIterator` and `Iterator` types. * Remove unused function _find_or_create_tool_content from events.py * Add header information to tool content and streamline message sending in event processing * Add send_message_method parameter to event handlers for message dispatching - Updated event handler functions to include a new parameter, `send_message_method`, allowing for message dispatching within the handlers. - Modified the logic in each handler to utilize the `send_message_method` for sending messages. - Adjusted the main event processing logic to pass the `send_message_method` to the appropriate handlers. * Refactor ContentBlockDisplay component to improve rendering performance and user experience * Refactor `ContentBlockDisplay.tsx` to improve readability and update header title logic * [autofix.ci] apply automated fixes * Add 'start_time' to event data and update test functions to include send_message mock * Refactor ContentBlockDisplay component to include total duration and improve user experience * Refactor ContentBlockDisplay component to handle isLoading state for total duration calculation * Refactor animatedNumbers component and add AnimatedNumberBasic component - Refactor the animatedNumbers component to improve readability and update the header title logic. - Add a new component called AnimatedNumberBasic that displays an animated number with a specific value. * Refactor agent message creation to improve event processing efficiency * Refactor session ID assignment in run_graph_internal for improved clarity * [autofix.ci] apply automated fixes * fix: update agent message properties handling for improved compatibility * fix: enhance message property validation to support dictionary input * Add session_id to agent message initialization in tests - Updated test cases in `test_agent_events.py` to include `session_id` in the `Message` initialization. - Adjusted calls to `process_agent_events` to pass the `agent_message` with `session_id`. * feat: Enhance event handling with duration tracking Add start_time parameter to event handling functions to ensure consistent duration calculations across different stages of agent processing. This improves accuracy in measuring event durations and enhances overall event management. * Add optional humanized value to AnimatedNumber component and update DurationDisplay usage * Refactor `ContentDisplay` and add separator to `ContentBlockDisplay` component * Add header to text content in agent message events * Add separator between each content block in ContentBlockDisplay component * Refactor layout in ContentDisplay component for improved styling and positioning * Refactor event handlers to return updated start_time and agent_message - Modified event handler functions to return a tuple of `agent_message` and `start_time`. - Updated `_calculate_duration` to handle `start_time` as an integer. - Ensured `start_time` is reset using `perf_counter()` after each event handling. - Adjusted tool and chain event handler calls to accommodate the new return type. * Add start_time and duration checks to agent event handlers in tests - Updated test cases for `handle_on_chain_start`, `handle_on_chain_end`, `handle_on_tool_start`, `handle_on_tool_end`, `handle_on_tool_error`, and `handle_on_chain_stream` to include `start_time` as a return value. - Added assertions to verify that `start_time` is a float and `duration` is an integer where applicable. * [autofix.ci] apply automated fixes * update colors and spacing of time * [autofix.ci] apply automated fixes * Refactor DurationDisplay component to display duration in seconds * Rename case from "message" to "add_message" in buildUtils.ts switch statement * Add event registration for message removal in EventManager * Add category handling for message events in Component class * Add custom exception handling for agent message processing * Handle exceptions in agent event processing with message deletion and event notification * Add new LLM options and refactor AgentComponent for dynamic provider handling - Introduced new language model options: Anthropic, Groq, and NVIDIA. - Refactored `AgentComponent` to utilize `MODEL_PROVIDERS_DICT` for dynamic provider handling. - Simplified input management by removing hardcoded provider inputs and using a dictionary-based approach. - Enhanced `update_build_config` to support flexible provider configurations and custom options. * Refactor buildUtils.ts and add message removal handling * Update Source model to allow None values for id, display_name, and source fields * Refactor event handler functions to return tuple of Message and float --------- Co-authored-by: anovazzi1 <[email protected]> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: cristhianzl <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request includes several changes to improve the handling and processing of messages within the agent and chat components. The most significant changes include refactoring the
message_responseandrun_agentmethods, updating theprocess_agent_eventsfunction, and modifying theChatOutputcomponent to handleMessageobjects more effectively.Refactoring and Improvements to Message Handling:
src/backend/base/langflow/base/agents/agent.py: Refactored themessage_responseandrun_agentmethods to handleMessageobjects instead of plain text, improving the consistency and functionality of message processing. [1] [2] [3]src/backend/base/langflow/base/agents/agent.py: Updated theprocess_agent_eventsfunction to use a singleMessageobject that gets updated throughout the agent's execution, enhancing the clarity and structure of the message flow.Changes to Chat Components:
src/backend/base/langflow/components/outputs/chat.py: Modified theChatOutputcomponent to handleMessageobjects directly, ensuring that all message properties are correctly set and stored. [1] [2] [3]Additional Updates:
src/backend/base/langflow/custom/custom_component/component.py: Adjusted thesend_messagemethod to check forNonesession IDs more effectively.src/backend/base/langflow/initial_setup/starter_projects/Agent Flow.json: Updated the starter project JSON to reflect the changes in theChatOutputcomponent, ensuring consistency with the new message handling logic.