Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphrag integration #4612

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e3e8f45
add initial global search draft
lpinheiroms Dec 7, 2024
8242378
add graphrag dep
lpinheiroms Dec 9, 2024
fb2fb19
Merge branch 'main' into lpinheiro/feat/add-graphrag-tools
lpinheiroms Dec 10, 2024
a13c18b
fix local search embedding
lpinheiroms Dec 17, 2024
8f3c484
linting
lpinheiroms Dec 17, 2024
0c05047
add from config constructor
lpinheiroms Dec 17, 2024
0e53f91
Merge branch 'main' into lpinheiro/feat/add-graphrag-tools
lspinheiro Dec 17, 2024
c1e7ea2
remove draft notebook
lpinheiroms Dec 17, 2024
a8b38ad
Merge branch 'main' into lpinheiro/feat/add-graphrag-tools
lspinheiro Dec 19, 2024
6d61c8e
update config factory and add docstrings
lpinheiroms Dec 20, 2024
1c4ed3d
add graphrag sample
lpinheiroms Dec 20, 2024
95f329c
add sample prompts
lpinheiroms Dec 20, 2024
3bc104b
update readme
lpinheiroms Dec 20, 2024
2ae6812
Merge branch 'main' into lpinheiro/feat/add-graphrag-tools
lspinheiro Dec 20, 2024
33523df
update deps
lpinheiroms Dec 20, 2024
8080ddb
Add API docs
ekzhu Dec 30, 2024
603c1c9
Update python/samples/agentchat_graphrag/requirements.txt
ekzhu Dec 30, 2024
934230b
Update python/samples/agentchat_graphrag/requirements.txt
ekzhu Dec 30, 2024
1c5fcd3
merge main, fix conflicts
lpinheiroms Dec 30, 2024
4f0c71f
update docstrings with snippet and doc ref
lpinheiroms Dec 30, 2024
e3dc1f9
lint
lpinheiroms Dec 30, 2024
f24fb6c
improve set up instructions in docstring
lpinheiroms Jan 3, 2025
4a5d611
lint
lpinheiroms Jan 3, 2025
74a2a23
Merge branch 'main' into lpinheiro/feat/add-graphrag-tools
lpinheiroms Jan 3, 2025
cac2aef
update lock
lpinheiroms Jan 3, 2025
e42f027
Update python/packages/autogen-ext/src/autogen_ext/tools/graphrag/_gl…
lspinheiro Jan 4, 2025
e60a9aa
Update python/packages/autogen-ext/src/autogen_ext/tools/graphrag/_lo…
lspinheiro Jan 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@
import tiktoken
from autogen_core import CancellationToken
from autogen_core.tools import BaseTool
from pydantic import BaseModel, Field

from graphrag.config.config_file_loader import load_config_from_file
from graphrag.query.indexer_adapters import (
read_indexer_communities,
Expand All @@ -17,6 +15,7 @@
from graphrag.query.llm.get_client import get_llm
from graphrag.query.structured_search.global_search.community_context import GlobalCommunityContext
from graphrag.query.structured_search.global_search.search import GlobalSearch
from pydantic import BaseModel, Field

from ._config import GlobalContextConfig as ContextConfig
from ._config import GlobalDataConfig as DataConfig
Expand All @@ -39,12 +38,54 @@ class GlobalSearchTool(BaseTool[GlobalSearchToolArgs, GlobalSearchToolReturn]):

This tool allows you to perform semantic search over a corpus of documents using the GraphRAG framework.
The search combines graph-based document relationships with semantic embeddings to find relevant information.
Example usage with AssistantAgent:

.. code-block:: python

import asyncio
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient
from autogen_ext.tools.graphrag import GlobalSearchTool
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from autogen_agentchat.agents import AssistantAgent


async def main():
# Initialize the OpenAI client
openai_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key="<api-key>",
)

# Set up global search tool
global_tool = GlobalSearchTool.from_settings(settings_path="./settings.yaml")
ekzhu marked this conversation as resolved.
Show resolved Hide resolved

# Create assistant agent with the global search tool
assistant_agent = AssistantAgent(
name="search_assistant",
tools=[global_tool],
model_client=openai_client,
system_message=(
"You are a tool selector AI assistant using the GraphRAG framework. "
"Your primary task is to determine the appropriate search tool to call based on the user's query. "
"For broader, abstract questions requiring a comprehensive understanding of the dataset, call the 'global_search' function."
),
)

# Run a sample query
query = "What is the overall sentiment of the community reports?"
response_stream = assistant_agent.run_stream(task=query)
async for msg in response_stream:
if hasattr(msg, "content"):
print(f"\nAgent response: {msg.content}")


if __name__ == "__main__":
asyncio.run(main())

.. note::

This tool requires the :code:`graphrag` extra for the :code:`autogen-ext` package.

This tool requires indexed data created by the GraphRAG indexing process. See the GraphRAG documentation
This tool requires indexed data created by the GraphRAG indexing process. See the [GraphRAG documentation](https://microsoft.github.io/graphrag/)
for details on how to prepare the required data files.


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@
import tiktoken
from autogen_core import CancellationToken
from autogen_core.tools import BaseTool
from pydantic import BaseModel, Field

from graphrag.config.config_file_loader import load_config_from_file
from graphrag.query.indexer_adapters import (
read_indexer_entities,
Expand All @@ -19,6 +17,7 @@
from graphrag.query.structured_search.local_search.mixed_context import LocalSearchMixedContext
from graphrag.query.structured_search.local_search.search import LocalSearch
from graphrag.vector_stores.lancedb import LanceDBVectorStore
from pydantic import BaseModel, Field

from ._config import LocalContextConfig, SearchConfig
from ._config import LocalDataConfig as DataConfig
Expand All @@ -40,12 +39,54 @@ class LocalSearchTool(BaseTool[LocalSearchToolArgs, LocalSearchToolReturn]):

This tool allows you to perform semantic search over a corpus of documents using the GraphRAG framework.
The search combines local document context with semantic embeddings to find relevant information.
Example usage with AssistantAgent:

.. code-block:: python

import asyncio
from autogen_ext.models.openai import AzureOpenAIChatCompletionClient
from autogen_ext.tools.graphrag import LocalSearchTool
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from autogen_agentchat.agents import AssistantAgent


async def main():
# Initialize the OpenAI client
openai_client = OpenAIChatCompletionClient(
model="gpt-4o-mini",
api_key="<api-key>",
)

# Set up local search tool
local_tool = LocalSearchTool.from_settings(settings_path="./settings.yaml")

# Create assistant agent with the local search tool
assistant_agent = AssistantAgent(
name="search_assistant",
tools=[local_tool],
model_client=openai_client,
system_message=(
"You are a tool selector AI assistant using the GraphRAG framework. "
"Your primary task is to determine the appropriate search tool to call based on the user's query. "
"For specific, detailed information about particular entities or relationships, call the 'local_search' function."
),
)

# Run a sample query
query = "What does the station-master say about Dr. Becher?"
response_stream = assistant_agent.run_stream(task=query)
async for msg in response_stream:
if hasattr(msg, "content"):
print(f"\nAgent response: {msg.content}")


if __name__ == "__main__":
asyncio.run(main())

.. note::

This tool requires the :code:`graphrag` extra for the :code:`autogen-ext` package.

This tool requires indexed data created by the GraphRAG indexing process. See the GraphRAG documentation
This tool requires indexed data created by the GraphRAG indexing process. See the [GraphRAG documentation](https://microsoft.github.io/graphrag/)
for details on how to prepare the required data files.

Args:
Expand Down
3 changes: 2 additions & 1 deletion python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@ dev = [

override-dependencies = [
"tenacity==9.0.0",
"pydantic==2.9.2",
]
[tool.uv.workspace]
members = ["packages/*"]

[tool.uv.sources]
agbench = { workspace = true }
Expand Down
Loading
Loading