v0.9 - Agent Views

Agent selection optimization

This release focuses on UX optimization for selecting agents. Both the chat and editor now use a modal popup to display agents. This optimizes the path to open agents in the editor. It also clears the way for agent references to be dropped into an agent chain.

EditorAgents.mp4

Agent Editor

• Editor now properly loads the agent chain when switching from one agent to another, for most cases.

v0.8

This release features a number of bugfixes and new components.

IX Client Updates

Startup fixes #254

IX client should now start the app as expected.

pip install agent-ix
ix up

Client Config #254 #255 #256

The published IX images now include the config needed to start the app with the IX client.

Components

Agent Memory #146

OpenAI Function Agents now support memory

Document Loaders #183

Several new document loaders are supported:

• HTML Loader
• CSV Loader
• JSON Loader
• PDF Loader
• Unstructured HTML Loader
• Unstructured Markdown Loader

Improved Chroma Support #243

Chroma is now started in a separate container for better response times.

Editor

• Dropping nodes while a node is selected works even if selected node does not have connectors #253

Component Config

Fixture Simplification #244

Component config is simplified by fixtures now using class_path as a natural key. This enabled the deprecation of the node_types.json fixture. Now components only need to be defined in the python based fixtures use by import_langchain.

All components may now be imported to the database with ./manage.py import_langchain from the make bash shell.

Misc

• Now supports LangChain 0.0.299
• Adding Hashicorp Vault container to support secrets. (secrets to come in a later release) #250
• Adding backend for datasources #251

v0.7 - UX optimizations

This release focuses on a significant usability revamp, and other updates including:

• simplified agent creation flow
• chain editor usability enhancements
• agent-ix quick launch
• published docker images
• new agents: @readme and @metaphor
• new components: MultiQueryRetriever, Metaphor Search Tools
• experimental local models: ollama, llama-cpp

Agent-ix client #232

The new agent-ix client a lightweight python library capable of bootstrapping the cluster in a local environment. It uses docker-compose to launch a complete IX cluster in a local environment. It's intended for local users who want to test or demo the app without configuring a full development environment.

pip install agent-ix
ix up

or start a specific tagged image

ix up --version dev

Chain Editor

Simplified Agent Creation #199

Agents are now created and configured through the chain editor. Agents are just outward facing chains exposed through chat interfaces and the API. Merging agent properties into the same user interface where the agent's core chain is edited makes for a much simpler and streamlined flow to create a new agent.

For now agents are the only type of chain. Later releases will expand on this concept and add additional chain types such as reusable chains, retriever workflows.

MetaphorCreateShort_V2.mp4

Connector help #213

Shift+clicking on node connectors now opens a help popup. The default popup indicates the type of accepted nodes and basic help instruction. Later PRs may expand this to add more in depth help instructions specific to the connector and component type.

ConnectorHelp.mp4

Connector searches #206

Clicking on node connectors now opens the component search with types pre-filled to quickly find matching components. This streamlines the chain creation flow and helps discovery of components.

Connector.Search.mp4

Auto-connecting nodes #191

New nodes now auto-connect if a connector or node is selected. If a node is selected the best matching node will be selected.

AutoConnect.mp4

Integrated Chat #195 #200 #201

The editor now has a chat instance built in. This allows for quick testing of agents and components without leaving the
editor. This can be very helpful during development while fine-tuning and debugging the settings and behaviors of
components.

The chat interface is the same component as the full chat interface. It has the same smart auto-complete options for
references within chat.

SideBarChat.mp4

Revamped layout

The layout of the editor was redesigned to provide easier to use forms and more workspace for the graph. The left
sidebar is now smaller and a right sidebar was added for forms and other tools. The right sidebar allows for more space
for rich interactive forms. It's adjustable and can be collapsed to open full screen to the graph.

Sidebar #211

Right Sidebar #194 #196 #204 #205 #209 #234 #235

Chat

Chat was also reworked with the same features.

New Agents

@readme #237

Example of a RAG agent that answers questions about the README.md file in the master branch of the IX github repository.
It uses Chroma and MultiQueryRetriever to find the best answer. It's a default agent available in all chats.

https://github.com/kreneskyp/ix/assets/68635/6bbc93f1-0564-49c8-8203-596b84e3bf65

@metaphor #239

Example using Metaphor web search.

New Components

MultiQueryRetriever #177

MultiQueryRetriever implements an advanced retrieval method that makes additional retrieval queries to resolve better context. See @readme for an example.

Metaphor Search #144 #218

Metaphor search engine tools are now available. Check out @metaphor agent for an example

Ollama (experimental) #192

Ollama added but I couldn't get Ollama working on my PC. It might work, let me know if you get it working!

Llama-cpp #186

LLama-cpp available in embedded mode. It isn't GPU accelerated so it's a bit limited for now.

To use:

1. Add a model to <ix>/models
2. Drag a Llama-cpp node onto the chain graph
3. configure model path to /var/app/models/<model_name>

Misc

• Markdown codeblocks in chat messages are now rendered as code blocks. #237
• Markdown links in chat messages are now rendered as links. #190
• Added dump_agent command to easily export agents to a django fixture. All built-in agents are now exported this way. #233 #236
• Docker build size optimized #227
• Adds image publishing flow: #216 #220 #225 #226 #230
• README updates: #223
• added start of chat theming #195
• nodes now snap to the grid #203
• agent address book usability improvements #212
• local environment startup improvements + working scaling command #217
• SyncToAsyncRun tool mixin compatability fix #189
• Node connectors now support lists #182

v0.6 Retrieval Components

Retrieval Components

IX now supports loading retrieval components. Retrieval components allow the ingestion and querying of data for use in LLM contexts. This concept covers everything from reading source code to importing data from web services.

This release focuses on the core support for these types of components. Later releases will expand the set of support components as well as improved ingestion workflows.

Supported Components

• Parsers: LanguageParser
• Loaders: GenericLoader (files), WebBaseLoader (urls)
• Text Splitters: `RecursiveTextSplitter
• Vector store: Choma DB, Redis
• Embeddings: OpenAI and more.
• Chains: ConversationalRetrievalChain

Example chain: File Loader

Initial support is for a combined ingestion + querying in a single chain.

Loader (documents) -> Text Splitter (chunks) -> VectorStore (as retriever) -> Retrieval Chain 

Example chain: Web loader

Agent collaboration

This release introduces DelegateToAgentChain. A simple form of delegation between agents.

FastAPI

#155 #156 #157 #158 #159 #160 #162 #163 #164 #165 #167

FastAPI has replaced all GraphQL endpoints for REST based access. All UX components updated to use FastAPI along with updated React hooks for easy integration.

Streaming events (chat messages, message streams, artifact updates) still use a graphql subscription over websockets + django-channels. This remains the easiest solution for multi-user chat even if it means keeping graphql around.

Misc

• #169 #174 - NodeTypeFields.get_fields can now import from Pydantic models, ABCs, and methods. This covers all the ways LangChain defines components and initializers

v0.5 FastAPI + Bot Smith

Bot Smith prototype (#115, #108)

A new agent named the Bot Smith (@smithy) has been added. The Bot Smith can access IX's chain editor API to retrieve and edit chains. This initial version is limited in capability, but will be improved over time.

The bot uses the OpenAPI w/ Functions Chain to access the API. There's still a lot that needs to be done to teach it to make complex edits. This bot will be a testing ground for creating a bot that can interact with OpenAPI.

Agents

• Add @wikipedia to demonstrate OpenAIFunctionAgent #110
• Add @klarna to demonstrate OpenAPIWithFunctionChain #114

Pydantic + FastAPI (#95, #122)

This PR introduces pydantic types and a FastAPI based API for the chain editor. This API will eventually replace the GraphQL API entirely. This was the eventual path but was accelerated to enable the Bot Smith.

Initially the frontend will continue to use the GraphQL API. The FastAPI is only used for the Bot Smith agent via OpenAPI functions chain

Async Tools #100

All available tools have been updated to asyncio. This is a workaround using sync_to_async so it's not a perfect solution but it runs.

Components

Chain Tool #121

Chain tool was added to enable any chain to be converted into a tool.

(The backend also supports connecting agents, but the frontend requires updated validation logic that supports multiple source types)

Misc

• LLMReply now works when called by an agent or as a tool #105
• missing memory connectors added #102
• SequentialChain now has an input_variables field. #104

Agent Editor

• a number of fixes made so agents can be edited and saved #119
• Added LLMSymbolicMathChain #120
• Agent auto-reply is now configurable. #113

Chain Editor

• Implicit SequentialChain now work correctly from the root node #111
• Implicit SequentialChain now works for agents #112
• list fields can now be edited with an input field #

UI Cleanup

A number of changes were made to simplify the UI

• Chain editor ConfigNode connectors are now grouped to left/right side depending on whether they are properties that contribute to a node (left) or data flows to them such as an agent, chain, or tool (right) #123, #124
• Chain editor ConfigNode connectors are now hidden except when hovering near the node #126
• Light mode now has a dimmed background and lighter foreground to be easier on the eyes. #128
• margin added to top of message stream #117

LangChain

• updated to version 0.0.231 #116
• added build flag for installing a local copy of LangChain for integrated development. #118

v0.4 Agents & Tools

Agents & Tools (#86, #90)

LangChain Agents and Tools may be added via the UX and used in chains.

Supported Agents

• OPENAI_FUNCTIONS_AGENT
• OPENAI_MULTIFUNCTION_AGENT
• ZERO_SHOT_REACT_DESCRIPTION_AGENT
• REACT_DOCSTORE_AGENT
• SELF_ASK_WITH_SEARCH_AGENT
• CONVERSATIONAL_REACT_DESCRIPTION_AGENT
• CHAT_ZERO_SHOT_REACT_DESCRIPTION_AGENT
• CHAT_CONVERSATIONAL_REACT_DESCRIPTION_AGENT
• STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION_AGENT

Supported Tools

• ARXIV_SEARCH
• BING_SEARCH
• DUCK_DUCK_GO_SEARCH
• GOOGLE_SEARCH
• GOOGLE_SERPER
• GRAPHQL_TOOL
• LAMBDA_API
• PUB_MED
• WIKIPEDIA
• WOLFRAM

Pydantic Types #87

Added pydantic types and field import tool for NodeType.

This is the start of conversion to using Pydantic types where possible. Future revisions will expand on this and include a migration to FastAPI for OpenAPI compliance. (See work in progress: #95)

Langchain importing #87

NodeType.get_fields was added to simplify importing fields from Pydantic objects. The utility is structured to support the continued use of wrapper functions and non-pydantic objects that load components.

(See example usages in #90, #97, #86)

Chain Editor

• ConfigNode now displays NodeType.name if not named. #84
• ConfigNode connectors now have a larger hitbox so they may be grabbed easier #85
• PromptNode and FunctionSchema can now be resized from both left & right edges. #88
• PromptNode and FunctionSchema resize handles are easier to grab now. #88
• ChatPromptTemplate created in UX can now be saved #89
• Moving ConfigNode now sends just position, to avoid saving stale config #92
• TypeAutoFields now applies NodeTypeField.style for stlying fields #93

Agent runner

• Agent runner now runs in async loop #79
• Agent inputs are set automatically when not set #91

Chat

• Chat messages now send to the correct chat after clicking new chat #80
• SideBarArtifactList overflow now scrolls #83

Misc

• now supports LangChain 0.0.222

v0.3.1

This is a bugfix release for a few issues introduced in v0.3

Bugfixes

• make dev_setup no longer raises an error with a fresh install #82 #81
• dev_setup now tested by github flow to prevent regressions #82
• Chat input now works correctly after clicking the new chat button #80

Features

• Agent workers now uses asyncio #79
  - Agent process updated to asyncio
  - Custom chains and components updated for asyncio

v0.3 Chain Editor, Graph model update, OpenAI Functions

Chain graph model and extended Langchain support

Django models for the chain graph were refined to support the broader set of LangChain components. v0.2 introduced support for memory components but the model was not flexible enough to easily support the graph of components. The refactored models enable a configurable graph of components and can more easily be extended to support the entire LangChain component set.

LangChain components may now be imported into NodeType model instances. The definitions include properties required to load the LangChain component and to support the ChainEditor UX.

This release adds support for these LangChain components:

• LLMS: PaLM, Anthropic (experimental)
• Vector store: Redis
• Embeddings: OpenAI, HuggingFace, PaLM, VertexAI, MosaicML, LlamaCpp

Chain Editor UX

A ReactFlow based graphical chain editor is now available. Components may be added to
the chain graph and connected to form the logic for an agent. Chains may then be assigned
to an Agent and used within the chat interface.

The UI supports a dynamically defined configuration of NodeType components. The editor
provides dynamically generated forms from NodeType definitions and allows for custom
display components.

OpenAI Functions

OpenAI functions are now supported by LLMChain and subclasses. tool and FunctionSchema may be connected to the functions property. Ix will automatically load both object types for use as OpenAI functions.

The @ix moderator and @code agents are updated to use functions.

PG Vector

PGVector VectorFields may now be added to any django model. This will eventually power
similarity searches for components, chains, agents, and chat artifacts.

Misc

Chains:
- from_config methods are deprecated in favor of new loaders.
- SaveArtifact and MapSubchain have improved error messages.
- SaveArtifact storage_id/identifier logic improved
- ArtifactMemory now searches pk for matching artifact_keys

UX:
- markdown urls and inline code references are now highlighted in chat messages
- UI better handles overflow and scrolling
- Chain editor NodeSelector now expands for long names
- Fixed double border around ChatMessageAvatar in light mode

Build:
- dev_setup now imports NodeTypes with import_langchain command
- node_types_fixture target added to generate node_types.json for tests

Memory Support

Version 0.2 focuses on adding memory to agents and chains.

Langchain Memory Support #57

• Langchain memory classes and backends may now be loaded via config.
• memory classes have configurable session support:
  • shared and localized memories via scopes
  • chat, agent, task, user scopes set from Ix runtime
  • memories may be grouped within scope

Artifact Memory #57

• ArtifactMemory now available to load artifacts from {artifact} references in the prompt.
• @code agent now uses ArtifactMemory

Improved ChatInput #58

ChatInput component has been rewritten:

• supports multi-line input
• auto-completions and highlighting for @mentions and {artifacts}
• no longer submits if empty
• auto expands for new lines

Misc

• Chat messages now respect spacing and linebreaks #60
• added @pirate agent with both conversational and artifact memory for testing. #62
• 3rd party security updates #64

v0.1.1

Fixing a few bugs from v0.1

• update to langchain 0.177 #45
• MacOS setup should now fallback to md5 when md5sum is not available. #46
• The sidebar artifact list now updates as they are created #47
• Chat input bar should now be aligned with chat messages #48
• The sidebar artifact list now updates when switching between chats #50
• Artifacts write to disk in the correct format #52