Skip to content

Commit

Permalink
feat: Add Astra DB Tool components for use with Tool Agents (langflow…
Browse files Browse the repository at this point in the history
…-ai#3911)

* Adding Astra Tools

* Format

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* Lint and Renaming to AstraDB

* Adding Astra Tools

* Format

* [autofix.ci] apply automated fixes

* ignoring .dspy cache

* [autofix.ci] apply automated fixes

* docs: Improve the search UX in Langflow docs (langflow-ai#4089)

* Add Mendable search bar component

* Align Mendable anon_key retrieval

* Update url and tagline in docusaurus config

* Move sitemap config to preset options

* Add Mendable anon key to docusaurus config

* docs: remove old examples (langflow-ai#4102)

Removed old examples

* fix: network error handling and build errors (langflow-ai#4088)

* Fixed API not throwing network errors

* Fix onBuildError assigning wrong status for the components

* Fix Network Error handling, making it call onBuildError

* Fixed build stop not triggering the alert

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jordan Frazier <[email protected]>

* fix: store api key pydantic error (langflow-ai#4103)

Fixed User pydantic error

Co-authored-by: Jordan Frazier <[email protected]>

* version: upgrade to 1.0.19 and 0.0.97 (langflow-ai#4104)

* upgrade to 1.0.19 and 0.0.97

* build: add readme to dockerfile (langflow-ai#4105)

Add readme to dockerfile

* feat: Add traceback to fastapi exception handler logs (langflow-ai#4099)

Add traceback to fastapi exception handler logs

* fix: prevent possible race condition on upload flows/folders (langflow-ai#4114)

* ✨ (create-file-upload.ts): improve file upload functionality by adding cleanup logic and handling edge cases for resolving file selection

* 🐛 (create-file-upload.ts): fix removing input element from the DOM by checking if it is contained in the document body before removal
💡 (create-file-upload.ts): add a comment to clarify the purpose of the setTimeout function for a fallback timeout of 1 minute

* ✨ (create-file-upload.ts): change createFileUpload function to be asynchronous to support Promise return type for better handling of file upload operations

* 📝 (create-file-upload.ts): improve error handling when removing input element from the DOM
📝 (create-file-upload.ts): remove unnecessary comment about timeout value in the code

* fix: truncate "params" column on vertex_build table to prevent memory heap on database (langflow-ai#4118)

* ✨ (model.py): add new method `serialize_params` to serialize parameters data in VertexBuildBase class

* 🐛 (util_strings.py): fix truncation logic to correctly handle long strings by truncating them with ellipsis if they exceed the maximum length

* ✅ (test_truncate_long_strings.py): add unit tests for the truncate_long_strings function to ensure it works correctly with various input scenarios

* ✅ (test_truncate_long_strings_on_objects.py): update test assertion message to reflect the expected behavior of the function

* refactor: add pagination on folders (langflow-ai#4020)

* ⬆️ (uv.lock): Update authlib version from 1.3.2 to 1.3.1
⬆️ (uv.lock): Update httpx version from 0.27.2 to 0.27.0
⬆️ (uv.lock): Add new package grpcio-health-checking version 1.62.3

⬆️ (uv.lock): upgrade weaviate-client package from version 3.26.7 to 4.8.1 and add new dependencies grpcio, grpcio-health-checking, grpcio-tools, httpx, and pydantic. Update package source and distribution URLs accordingly.

* ✨ (flows.py): Add pagination support for retrieving a list of flows to improve performance and user experience
📝 (flows.py): Update documentation to reflect changes made for pagination and additional query parameters
📝 (folders.py): Add a new endpoint to retrieve a folder with paginated flows for better organization and readability

* ✨ (model.py): add PaginatedFlowResponse class to handle paginated flow responses in API calls

* ✨ (test_database.py): add support for fetching all flows by adding "get_all" parameter to the request
🐛 (test_database.py): fix endpoint for fetching read-only starter projects to use correct URL path

* packages changes

* packages changes

* 📝 (paginatorComponent/index.tsx): refactor PaginatorComponent to use constants for default values and improve code readability
🐛 (paginatorComponent/index.tsx): fix issue with setting maxPageIndex when pages prop is provided to PaginatorComponent

* 🐛 (storeCardComponent): fix height of store card component to prevent overflow and improve layout aesthetics

* ✨ (constants.ts): introduce new constants for search tabs, pagination settings, and store pagination settings to enhance user experience and improve functionality

* ✨ (use-post-login-user.ts): add queryClient to UseRequestProcessor to enable refetching queries on mutation settled state
📝 (use-post-login-user.ts): update useLoginUser function to include onSettled callback in options to refetch queries after mutation settles

* ✨ (use-get-basic-examples.ts): introduce a new query function to fetch basic examples of flows from the API and update the state with the fetched data.

* ✨ (use-get-refresh-flows.ts): introduce new functionality to fetch and process flows with query parameters for components_only, get_all, folder_id, remove_example_flows, page, and size. This allows for more flexible and specific retrieval of flow data.

* ✨ (use-get-folder.ts): Add support for pagination and filtering options in the useGetFolderQuery function to enhance flexibility and customization for fetching folder data.

* 🔧 (use-get-folders.ts): Remove unused code related to refreshFlows and setStarterProjectId to improve code readability and maintainability
✨ (use-get-folders.ts): Update useGetFoldersQuery to setFolders with the fetched data instead of filtering out starter projects to simplify the logic and improve performance

* ✨ (use-upload-flow.ts): update refreshFlows function call to include an object with get_all property set to true to fetch all flows when refreshing

* 🔧 (codeAreaModal/index.tsx): remove unused import postCustomComponent to clean up code and improve readability

* 📝 (AdminPage/index.tsx): Update initial page size and index values to use constants for better maintainability
📝 (AdminPage/index.tsx): Update resetFilter function to set page size and index using constants for consistency
📝 (AdminPage/index.tsx): Update logic to handle loading state and empty user list based on isIdle state for better user experience
📝 (AdminPage/index.tsx): Update PaginatorComponent props to use constant for rows count and simplify paginate function assignment

* ✨ (AppInitPage/index.tsx): introduce new queries to fetch basic examples and folders data for improved functionality and user experience

* ✨ (FlowPage/index.tsx): update refreshFlows function call to include a parameter to get all flows at once for better performance.

* ✨ (frontend): introduce flowsPagination and setFlowsPagination functions to manage pagination for flows in the UtilityStoreType

* ✨ (frontend): introduce new 'folders' and 'setFolders' properties to FoldersStoreType to manage folder data efficiently

* ✨ (types.ts): introduce Pagination type to define structure for pagination data in frontend application

* ✨ (frontend): introduce PaginatedFlowsType to represent paginated flow data structure in the application.

* ✨ (frontend): add optional 'pages' property to PaginatorComponentType to allow customization of the number of pages displayed in the paginator

* ✨ (utilityStore.ts): introduce flowsPagination object with default values for page and size to manage pagination in the utility store

* ✨ (foldersStore.tsx): introduce new state 'folders' and setter function 'setFolders' to manage folder data in the store

* ✨ (ViewPage/index.tsx): update refreshFlows function call to include a parameter to get all flows at once, improving efficiency and reducing unnecessary calls to the backend.

* 📝 (StorePage/index.tsx): update constants import to include new pagination constants for better organization
♻️ (StorePage/index.tsx): refactor pagination logic to use new pagination constants for clarity and consistency throughout the file

* ✨ (componentsComponent/index.tsx): Add support for pagination feature in ComponentsComponent to improve user experience and performance.

* ✨ (myCollectionComponent/index.tsx): introduce pagination and search functionality to improve user experience and data handling in the MyCollectionComponent component

* ✨ (Playground/index.tsx): Update refreshFlows function call to include a parameter to get all flows at once for better performance.

* ✨ (entities/index.tsx): introduce PaginatedFolderType to represent a folder with pagination information for better organization and handling of paginated data.

* ✨ (headerTabsSearchComponent/index.tsx): add support for changing tabs and searching functionality in headerTabsSearchComponent
📝 (headerTabsSearchComponent/index.tsx): update tabsOptions to use constant SEARCH_TABS for consistency and reusability

* 📝 (folders.py): Remove redundant import and unused code related to FolderWithPaginatedFlows class
🔧 (folders.py): Update query conditions to use explicit boolean values instead of implicit truthiness for better clarity and readability

* 📝 (folders.py): reorganize imports to improve readability and maintain consistency

* 📝 (flows.py): import FlowSummary model to support new functionality in the API
📝 (flows.py): add header_flows parameter to read_flows function to return a simplified list of flows if set to true

* 📝 (folders.py): remove unnecessary empty line to improve code readability

* ✨ (model.py): introduce new FlowSummary class to represent a summary of flow data for better organization and readability

* ✨ (pagination_model.py): introduce a new model 'FolderWithPaginatedFlows' to represent a folder with paginated flows for better organization and readability.

* 📝 (cardComponent/index.tsx): add missing semicolon to improve code consistency
♻️ (cardComponent/index.tsx): refactor logic to use data parameter directly instead of fetching flow by id to simplify code and improve readability

* ✨ (use-get-refresh-flows.ts): introduce new query parameter 'header_flows' to support fetching header flows in API requests.

* ✨ (use-get-folders.ts): add functionality to refresh flows when fetching folders to ensure data consistency and up-to-date information.

* ✨ (use-upload-flow.ts): add support for fetching header flows along with all flows when refreshing flows in useUploadFlow hook

* ✨ (use-redirect-flow-card-click.tsx): introduce a new custom hook 'useFlowCardClick' to handle flow card click events in the newFlowModal component. This hook utilizes react-router-dom for navigation, custom analytics tracking, and various utility functions to manage flow data and update the UI.

* ✨ (NewFlowCardComponent/index.tsx): refactor onClick event handler into a separate function handleClick for better readability and maintainability

* 📝 (undrawCards/index.tsx): Remove unused imports and variables to clean up the code
♻️ (undrawCards/index.tsx): Refactor onClick event handler to use a separate function for handling flow card click events

* ✨ (flowsManager/index.ts): introduce new state and setter for flowToCanvas to manage the flow displayed on canvas

* ✨ (flowsManagerStore.ts): introduce new feature to set and update the flow to be displayed on the canvas asynchronously

* ✨ (ViewPage/index.tsx): add support for fetching header flows along with all flows when refreshing data to improve data retrieval efficiency

* ✨ (collectionCard/index.tsx): introduce useFlowsManagerStore to manage flows in the application
📝 (collectionCard/index.tsx): update handleClick function to set flow to canvas before navigating to editFlowLink

* ✨ (Playground/index.tsx): add support for fetching header flows along with all flows when refreshing data in the Playground page.

* 🐛 (FlowPage/index.tsx): Fix setCurrentFlow logic to correctly set the current flow based on flowToCanvas value. Add flowToCanvas dependency to useEffect to ensure proper rendering.

* ✨ (use-get-flow.ts): introduce a new file to handle API queries for fetching flow data in the frontend application. This file defines a custom hook 'useGetFlow' that makes a GET request to the API endpoint to retrieve flow data based on the provided flow ID.

* 📝 (flows.py): update import statement to use FlowHeader instead of FlowSummary for better clarity
📝 (flows.py): update response_model in read_flows endpoint to use FlowHeader for consistency and clarity

* ✨ (model.py): rename FlowSummary class to FlowHeader for better clarity and consistency in naming conventions
📝 (model.py): add is_component field to FlowHeader class to indicate if the flow is a component or not

* ✨ (use-get-folder.ts): introduce processFlows function to process flows data
📝 (use-get-folder.ts): add cloneDeep function to safely clone data before processing

* 🔧 (use-get-refresh-flows.ts): refactor useGetRefreshFlows function to simplify processing of dbDataFlows and update state accordingly

* ✨ (FlowPage/index.tsx): Add useGetFlow hook to fetch flow data and update current flow on canvas. Add getFlowToAddToCanvas function to handle fetching and setting flow data on canvas.

* ✨ (nodeToolbarComponent/index.tsx): improve user experience by automatically closing the override modal after successful flow override

* ✨ (use-post-login-user.ts): add queryClient.refetchQueries for "useGetTags" after successful login to update tags data in the frontend.

* ✨ (use-get-flow.ts): add processFlows function to process flow data before returning it to improve code readability and maintainability

* 🐛 (use-get-refresh-flows.ts): fix asynchronous flow to correctly handle data retrieval and processing before setting state and returning flows

* ✨ (use-get-tags.ts): add functionality to set tags in utility store after fetching them from the API

* 📝 (shareModal/index.tsx): Update import statement for useUtilityStore to improve code organization and readability
🔧 (shareModal/index.tsx): Replace useGetTagsQuery with useUtilityStore to manage tags state and remove unnecessary API call for tags data
🔧 (shareModal/index.tsx): Replace references to data with tags variable to ensure consistency and avoid potential bugs
🔧 (shareModal/index.tsx): Update TagsSelector component to use tags variable instead of data for better data management

* ✨ (AppInitPage/index.tsx): introduce useGetTagsQuery to fetch tags data from the store API for AppInitPage.

* ✨ (StorePage/index.tsx): refactor to use useUtilityStore for tags state management instead of useGetTagsQuery for better separation of concerns and code organization. Update TagsSelector component to use tags from useUtilityStore hook.

* ✨ (frontend): introduce Tag type to UtilityStoreType and add tags and setTags functions to manage tags in the store.

* ✨ (types.ts): introduce new Tag type to represent a tag with id and name properties

* ✨ (utilityStore.ts): introduce new 'tags' array and 'setTags' function to manage tags in the utility store

* 📝 (App.css): add a blank line for better readability and consistency in the CSS file

* ✨ (test_database.py): add pagination support for reading flows and folders to improve data retrieval efficiency and user experience

* ✨ (tabsComponent/index.tsx): refactor changeLocation function to use useCallback hook for better performance and stability
✨ (tabsComponent/index.tsx): update onClick event handler to directly call changeLocation function for cleaner code and improved readability

* ✨ (headerTabsSearchComponent/index.tsx): refactor handleChangeTab, handleSearch, handleInputChange, and handleKeyDown functions to use useCallback for better performance and memoization
📝 (headerTabsSearchComponent/index.tsx): update tabActive prop to use the activeTab prop passed from parent component for consistency and clarity

* ✨ (myCollectionComponent/index.tsx): refactor filter state initialization to dynamically set based on current location pathname
♻️ (myCollectionComponent/index.tsx): refactor onSearch and onChangeTab functions to use useCallback for better performance and memoization

* ✨ (create-query-param-string.ts): introduce a new utility function to build query parameter strings for URLs in the frontend controllers.

* ✨ (use-get-folder.ts): introduce buildQueryStringUrl function to create query parameter strings for API requests
📝 (use-get-folder.ts): add comments to explain the purpose of the code and improve code readability
🔧 (use-get-folder.ts): update useGetFolderQuery function to include additional configuration options for the query, such as refetchOnWindowFocus: false

* ✨ (use-delete-folders.ts): add functionality to update local store after deleting a folder to keep it in sync with the server data

* 📝 (use-post-add-flow.ts): import useFolderStore to access myCollectionId state for refetching queries with the correct folder_id
🐛 (use-post-add-flow.ts): fix queryClient.refetchQueries to include the correct queryKey with folder_id or myCollectionId if response.folder_id is null

* ✨ (use-get-refresh-flows.ts): refactor addQueryParams function to use buildQueryStringUrl utility function for better code readability and maintainability

* ♻️ (flows.py): remove unnecessary commented out code and add pagination functionality to the router for better code organization and readability

* 🔧 (NodeDescription/index.tsx): remove console.log statement for better code cleanliness and readability

* ✨ (index.tsx): Add DialogClose component from @radix-ui/react-dialog to handle cancel action in ConfirmationModal. Refactor handleCancel function to improve code readability and maintainability.

* ♻️ (use-redirect-flow-card-click.tsx): remove unused setFlowToCanvas function to clean up code and improve maintainability

* 📝 (use-patch-update-flow.ts): update onSettled callback to refetch useGetFolders query with the updated folder_id after patching a flow

* 🔧 (use-delete-folders.ts): update onSettled function to correctly refetch queries with the specific folder id when deleting folders

* ✨ (use-get-folder.ts): update useGetFolderQuery to include additional query parameters for pagination and filtering options

* 🔧 (use-post-upload-to-folder.ts): update onSettled callback to refetch useGetFolders query with the correct queryKey and folder_id parameter

* ✨ (use-add-flow.ts): add functionality to refresh flows after adding a new flow to ensure the UI is up to date with the latest data.

* ✨ (AppInitPage/index.tsx): enable fetching basic examples and tags only when isFetched is true to improve performance and reduce unnecessary API calls

* ✨ (myCollectionComponent/index.tsx): refactor onPaginate function to handlePageChange callback for better code organization and readability. Update key prop in ComponentsComponent to include filter and search variables for proper re-rendering.

* ✨ (use-get-folder.ts): add placeholderData option to useGetFolderQuery to provide initial data while fetching folder information

* [autofix.ci] apply automated fixes

* 🐛 (use-post-upload-to-folder.ts): fix queryKey values to correctly refetch queries after uploading a file to a folder

* ⬆️ (folders.spec.ts): increase timeout for waiting in test to improve reliability and prevent flakiness

* ✨ (folders.spec.ts): update selectors to target specific elements by using the first() method to improve test reliability

* ✅ (auto-save-off.spec.ts): update test to match changes in UI for auto-save feature and improve test coverage for hover functionality.

* 📝 (model.py): update model fields to set default values for folder_id, is_component, endpoint_name, and description to None for consistency and clarity

* ♻️ (index.tsx): refactor isUpdatingFolder logic to include isFetchingFolder variable for better accuracy and readability

* 🐛 (index.tsx): fix variable name typo in isLoadingFolder assignment to correctly reference isFetchingFolder

* 🐛 (use-patch-update-flow.ts): fix issue with missing closing parenthesis in queryClient.refetchQueries() method
📝 (use-patch-update-flow.ts): update queryKey in queryClient.refetchQueries() to ["useGetFolder"] to correctly refetch the folder data after updating a flow

* ✨ (use-save-flow.ts): add support for fetching flow data before saving if not already present to ensure data consistency and accuracy

* ✅ (folders.spec.ts): update selectors to target specific elements correctly for testing purposes

* ✨ (use-on-file-drop.tsx): add support for checking flow names in a specific collection to prevent duplicates
♻️ (use-add-flow.ts): refactor to use the same logic for checking flow names in a specific collection to prevent duplicates

* ✨ (index.tsx): Add useIsMutating hook to check if a folder is being deleted to update UI accordingly
♻️ (index.tsx): Refactor logic to determine if a folder is being updated to include check for folder deletion
🔧 (index.tsx): Refactor Select component to use a separate function for handling value change
🔧 (index.tsx): Refactor Button components to disable based on separate variables for first and last page
🔧 (index.ts): Remove unused import and refactor code to use useRef hook for storing the latest folder id in useGetFolderQuery

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* Fix: Add UTC timezone info to message.timestamp. (langflow-ai#4129)

* fix: enhance recursive_serialize_or_str to handle BaseModelV1 and improve exception logging (langflow-ai#4132)

Enhance `recursive_serialize_or_str` to handle `BaseModelV1` and improve exception logging

* fix: name/description bug in "Flow as Tool" (langflow-ai#4097)

Fix name/description bug in "Flow as Tool".

* chore: add opensearch-py dependency (langflow-ai#4134)

Add opensearch-py dependency to pyproject.toml

* refactor: starter projects templates and components (langflow-ai#4121)

* Update starter flows

* Update button with new menu variants

* Updated AstraDB icon

* Made header be optional on baseModal and added classname to content

* Removed old newFlowModal

* added a Nav component to show the sidebar categories

* Created new templates modal

* Updated components and modals to use new Templates modal

* Added used icons

* added template card

* changed templates modal to use modular components

* Add template content

* Add get started content

* added other size for templates

* Added size in base modal

* Changed menu ring-0 to be important

* Added all of the images of get started

* Fix hover effect on spirals

* Implement clicking get started templates

* Fix shiny design

* update package lock

* update examples

* updated card design

* Added grid of examples to templates

* Implemented card click

* Changed hover effect

* Added arrows

* delete unused

* implemented fuse search

* Added create blank project

* Made tags be read

* Added tags to basic prompting

* Added types

* Added tags to the tabs

* remove important from tailwind config

* updated setup and model to include icon

* added random-gradient npm package

* updated colors to remove white

* added icons

* inserted metadata of icons and etc into starter flows

* Removed integrations and added blank project creation

* Added gradient to cards

* Reset query when changing tab, reset scroll when typing

* added mix blend overlay to text

* Added id and gradient to templatecard type

* made icons for components still work

* added important on stroke

* formatting

* Fixed infinite render

* added test id to create blank project

* added data test id for templates title

* ✨ (navComponent/index.tsx): add data-testid attribute with converted test name for side navigation options to improve testability and accessibility

* ✨ (starter-projects.spec.ts): add test to ensure user can interact with starter projects in the frontend application

* ✨ (TemplateCardComponent/index.tsx): add data-testid attribute with converted test name for better testability and accessibility

* ✨ (TemplateCategoryComponent/index.tsx): import convertTestName function to convert test names for data-testid attributes in TemplateCategoryComponent

* fixed currentTab not changing results

* Fixed various tests

* ✨ (similarity.spec.ts): improve user experience by adding functionality to fit view and scroll using mouse wheel in the test suite

* Fix other tests relying on multiple

* fix two edges test

* Fix hover on test 3 of generalBugs

---------

Co-authored-by: cristhianzl <[email protected]>

* ref: Add ruff rules TRY3xx (langflow-ai#4098)

Add ruff rules TRY3xx

* ref: Some ruff rule fixes from preview mode (langflow-ai#4131)

Some ruff rule fixes from preview mode

* ref: Add ruff rules for pydocstyle (D) (langflow-ai#4120)

Add ruff rules for pydocstyle (D)

* ref: Add ruff rules for arguments (ARG) (langflow-ai#4123)

Add ruff rules for arguments (ARG)

* ref: Add ruff rules for boolean trap (FBT) (langflow-ai#4126)

Add ruff rules for boolean trap (FBT)

* fix: escape directory to prevent \n on Windows directory name to fail on Pathlib + Tests (langflow-ai#4101)

* 📝 (utils.py): add format_directory_path function to properly escape and format directory paths for consistency and validity

* ✨ (test_format_directory_path.py): add unit tests for the format_directory_path function to ensure correct formatting of directory paths
📝 (test_format_directory_path.py): add documentation and examples for different types of directory paths in the unit tests to improve code readability and maintainability

* 🐛 (utils.py): fix the incorrect replacement of backslashes with newline characters in the format_directory_path function
📝 (test_rewrite_file_path.py): update test cases and function names to reflect the changes made in the format_directory_path function in utils.py

* 🐛 (test_format_directory_path.py): update parameter name from 'path' to 'input_path' for clarity and consistency
📝 (test_format_directory_path.py): improve test case descriptions and handle newline characters in paths correctly

* feat: update theme (langflow-ai#4084)

* update theme

* update component styles

* change output node background

* update background for component folderrs

* update astra icon coloring

* fix: border and editor display issues (langflow-ai#4142)

* Fixed border on chat input on playground

* Fixed ace editor not taking up entire screen

* fix: Union type on components  (langflow-ai#4137)

* 🐛 (type_extraction.py): fix condition to correctly handle UnionType objects in type extraction process

* ✨ (test_schema.py): add support for additional data types and nested structures in post_process_type function to improve type handling and flexibility

* ✅ (test_schema.py): add additional test cases for post_process_type function to cover various Union types and combinations for better test coverage and accuracy

* Adding Astra Tools

* [autofix.ci] apply automated fixes

* Lint and Renaming to AstraDB

* Adding Astra Tools

* Cleanup old files and re-format

* More formatting

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <[email protected]>
Co-authored-by: Eric Schneider <[email protected]>
Co-authored-by: Lucas Oliveira <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jordan Frazier <[email protected]>
Co-authored-by: Christophe Bornet <[email protected]>
Co-authored-by: Cristhian Zanforlin Lousa <[email protected]>
Co-authored-by: dhlidongming <[email protected]>
Co-authored-by: Mike Fortman <[email protected]>
  • Loading branch information
11 people authored and diogocabral committed Nov 26, 2024
1 parent e4aaa23 commit 46ab673
Show file tree
Hide file tree
Showing 18 changed files with 3,002 additions and 900 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ yarn-error.log*
lerna-debug.log*
qdrant_storage

.dspy_cache
# Mac
.DS_Store

Expand Down
47 changes: 46 additions & 1 deletion docs/docs/Components/components-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -296,4 +296,49 @@ This component does not have any input parameters.

| Name | Type | Description |
|------|------|----------------------------------------------|
| tool | Tool | Yahoo Finance News tool for use in LangChain |
| tool | Tool | Yahoo Finance News tool for use in LangChain |


## Astra DB Tool

The `Astra DB Tool` allows agents to connect to and query data from Astra DB Collections.

### Parameters

#### Inputs

| Name | Type | Description |
|-------------------|--------|----------------------------------------------------------------------------------------------------------------------------------|
| Tool Name | String | The name used to reference the tool in the agent's prompt. |
| Tool Description | String | A brief description of the tool. This helps the model decide when to use it. |
| Collection Name | String | The name of the Astra DB collection to query. |
| Token | SecretString | The authentication token for accessing Astra DB. |
| API Endpoint | String | The Astra DB API endpoint. |
| Projection Fields | String | The attributes to return, separated by commas. Default: "*". |
| Tool Parameters | Dict | Parameters the model needs to fill to execute the tool. For required parameters, use an exclamation mark (e.g., "!customer_id"). |
| Static Filters | Dict | Attribute-value pairs used to filter query results. |
| Limit | String | The number of documents to return. |



## Astra DB CQL Tool

The `Astra DB CQL Tool` allows agents to query data from CQL Tables in Astra DB.

### Parameters

#### Inputs

| Name | Type | Description |
|-------------------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------|
| Tool Name | String | The name used to reference the tool in the agent's prompt. |
| Tool Description | String | A brief description of the tool to guide the model in using it. |
| Keyspace | String | The name of the keyspace. |
| Table Name | String | The name of the Astra DB CQL table to query. |
| Token | SecretString | The authentication token for Astra DB. |
| API Endpoint | String | The Astra DB API endpoint. |
| Projection Fields | String | The attributes to return, separated by commas. Default: "*". |
| Partition Keys | Dict | Required parameters that the model must fill to query the tool. |
| Clustering Keys | Dict | Optional parameters the model can fill to refine the query. Required parameters should be marked with an exclamation mark (e.g., "!customer_id"). |
| Static Filters | Dict | Attribute-value pairs used to filter query results. |
| Limit | String | The number of records to return. |
Empty file modified scripts/setup/check_env.sh
100644 → 100755
Empty file.
4 changes: 3 additions & 1 deletion src/backend/base/langflow/base/agents/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,9 @@ async def run_agent(self, agent: AgentExecutor) -> Text:
class LCToolsAgentComponent(LCAgentComponent):
_base_inputs = [
*LCAgentComponent._base_inputs,
HandleInput(name="tools", display_name="Tools", input_types=["Tool", "BaseTool"], is_list=True),
HandleInput(
name="tools", display_name="Tools", input_types=["Tool", "BaseTool", "StructuredTool"], is_list=True
),
]

def build_agent(self) -> AgentExecutor:
Expand Down
180 changes: 180 additions & 0 deletions src/backend/base/langflow/components/tools/AstraDBCQLTool.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
import urllib
from http import HTTPStatus
from typing import Any

import requests
from langchain.pydantic_v1 import BaseModel, Field, create_model
from langchain_core.tools import StructuredTool

from langflow.base.langchain_utilities.model import LCToolComponent
from langflow.io import DictInput, IntInput, SecretStrInput, StrInput
from langflow.schema import Data


class AstraDBCQLToolComponent(LCToolComponent):
display_name: str = "Astra DB CQL Tool"
description: str = "Create a tool to get data from DataStax Astra DB CQL Table"
documentation: str = "https://astra.datastax.com"
icon: str = "AstraDB"

inputs = [
StrInput(name="tool_name", display_name="Tool Name", info="The name of the tool.", required=True),
StrInput(
name="tool_description",
display_name="Tool Description",
info="The tool description to be passed to the model.",
required=True,
),
StrInput(
name="keyspace",
display_name="Keyspace",
value="default_keyspace",
info="The keyspace name within Astra DB where the data is stored.",
required=True,
),
StrInput(
name="table_name",
display_name="Table Name",
info="The name of the table within Astra DB where the data is stored.",
required=True,
),
SecretStrInput(
name="token",
display_name="Astra DB Application Token",
info="Authentication token for accessing Astra DB.",
value="ASTRA_DB_APPLICATION_TOKEN",
required=True,
),
StrInput(
name="api_endpoint",
display_name="API Endpoint",
info="API endpoint URL for the Astra DB service.",
value="ASTRA_DB_API_ENDPOINT",
required=True,
),
StrInput(
name="projection_fields",
display_name="Projection fields",
info="Attributes to return separated by comma.",
required=True,
value="*",
),
DictInput(
name="partition_keys",
display_name="Partition Keys",
is_list=True,
info="Field name and description to the model",
required=True,
),
DictInput(
name="clustering_keys",
display_name="Clustering Keys",
is_list=True,
info="Field name and description to the model",
),
DictInput(
name="static_filters",
display_name="Static Filters",
is_list=True,
info="Field name and value. When filled, it will not be generated by the LLM.",
),
IntInput(
name="number_of_results",
display_name="Number of Results",
info="Number of results to return.",
advanced=True,
value=5,
),
]

def astra_rest(self, args):
headers = {"Accept": "application/json", "X-Cassandra-Token": f"{self.token}"}
ASTRA_URL = f"{self.api_endpoint}/api/rest/v2/keyspaces/{self.keyspace}/{self.table_name}/"
key = []
# Partition keys are mandatory
for k in self.partition_keys:
if k in args:
key.append(args[k])
elif self.static_filters[k] is not None:
key.append(self.static_filters[k])
else:
# TO-DO: Raise error - Missing information
key.append("none")

# Clustering keys are optional
for k in self.clustering_keys:
if k in args:
key.append(args[k])
elif self.static_filters[k] is not None:
key.append(self.static_filters[k])

url = f'{ASTRA_URL}{"/".join(key)}?page-size={self.number_of_results}'

if self.projection_fields != "*":
url += f'&fields={urllib.parse.quote(self.projection_fields.replace(" ","")) }'

res = requests.request("GET", url=url, headers=headers)

if int(res.status_code) >= HTTPStatus.BAD_REQUEST:
return res.text

try:
res_data = res.json()
return res_data["data"]
except ValueError:
return res.status_code

def create_args_schema(self) -> dict[str, BaseModel]:
args: dict[str, tuple[Any, Field]] = {}

for key in self.partition_keys:
# Partition keys are mandatory is it doesn't have a static filter
if key not in self.static_filters:
args[key] = (str, Field(description=self.partition_keys[key]))

for key in self.clustering_keys:
# Partition keys are mandatory if has the exclamation mark and doesn't have a static filter
if key not in self.static_filters:
if key.startswith("!"): # Mandatory
args[key[1:]] = (str, Field(description=self.clustering_keys[key]))
else: # Optional
args[key] = (str | None, Field(description=self.clustering_keys[key], default=None))

model = create_model("ToolInput", **args, __base__=BaseModel)
return {"ToolInput": model}

def build_tool(self) -> StructuredTool:
"""Builds a Astra DB CQL Table tool.
Args:
name (str, optional): The name of the tool.
Returns:
Tool: The built AstraDB tool.
"""
schema_dict = self.create_args_schema()
return StructuredTool.from_function(
name=self.tool_name,
args_schema=schema_dict["ToolInput"],
description=self.tool_description,
func=self.run_model,
return_direct=False,
)

def projection_args(self, input_str: str) -> dict:
elements = input_str.split(",")
result = {}

for element in elements:
if element.startswith("!"):
result[element[1:]] = False
else:
result[element] = True

return result

def run_model(self, **args) -> Data | list[Data]:
results = self.astra_rest(args)
data: list[Data] = [Data(data=doc) for doc in results]
self.status = data
return results
Loading

0 comments on commit 46ab673

Please sign in to comment.