Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
b1d56a8
sdk
Feb 4, 2026
a3a0e0a
fix: update license to AGPL-3.0 and format client.py imports
Feb 4, 2026
fb223f7
Merge branch 'staging' into dvirdu_python_api
DvirDukhan Feb 4, 2026
c045f05
fix: address PR review comments - unused variable and empty except block
Feb 4, 2026
e4936fa
test: improve SDK tests with content validation
Feb 4, 2026
ba871ed
ci: disable acceptable pylint warnings in CI
Feb 4, 2026
28818b8
refactor: fix pylint issues properly without disabling warnings
Feb 5, 2026
291a74d
testing + readme
DvirDukhan Feb 5, 2026
e9a8371
Merge branch 'staging' into dvirdu_python_api
DvirDukhan Feb 16, 2026
834706c
text2sql_common - common file
DvirDukhan Feb 16, 2026
f052cc3
PR comments
DvirDukhan Feb 16, 2026
80386f2
Merge branch 'staging' into dvirdu_python_api
gkorland Feb 21, 2026
4a6648c
Merge staging into dvirdu_python_api: resolve conflicts
gkorland Mar 8, 2026
5a779df
fix: resolve CI failures across all workflows
gkorland Mar 8, 2026
9e27a62
update lock files
gkorland Mar 8, 2026
731484c
merge: resolve conflicts with staging
gkorland Mar 12, 2026
d729e85
fix: add SDK to spellcheck wordlist
gkorland Mar 12, 2026
04385b6
refactor(sdk): make queryweaver_sdk pip-installable with shared core
galshubeli Apr 20, 2026
fdcc3d7
fix(sdk): address review bugs in refresh path and stale docstring
galshubeli May 3, 2026
46c3b2c
docs(examples): add SDK end-to-end example with 12-table postgres schema
galshubeli May 3, 2026
b9372ab
merge: resolve conflicts with staging
galshubeli May 3, 2026
ca1e9ed
fix(sdk): address critical bot-review bugs
galshubeli May 3, 2026
eacaf39
fix(sdk): address remaining major + minor review issues
galshubeli May 3, 2026
61fb5fe
chore(sdk): docs/tests/infra hygiene from review
galshubeli May 3, 2026
a0cbb05
fix(sdk): final round of review issues
galshubeli May 3, 2026
2c32f5b
refactor(core): rename text2sql_common → pipeline
galshubeli May 3, 2026
33b8b1b
refactor(core): add unified run_query/run_confirmed orchestration
galshubeli May 3, 2026
dd26c74
refactor(core): switch routes and SDK to unified pipeline; delete syn…
galshubeli May 3, 2026
8d42bf4
fix(ci): unblock build/test/sdk-tests + harden stream error path
galshubeli May 3, 2026
4601362
fix(sdk): address Copilot follow-up review
galshubeli May 4, 2026
0050733
fix(docker): install server extra so uvicorn is available
galshubeli May 4, 2026
27eea6d
refactor(sdk): rename queryweaver_sdk → queryweaver
galshubeli May 4, 2026
e4e2fa4
refactor(core): dedupe pipeline helpers across streaming and SDK paths
galshubeli May 4, 2026
1dcf746
fix(routes): suppress py/stack-trace-exposure on _stream_pipeline sink
galshubeli May 4, 2026
5b120cf
fix(routes,sdk): eager-validate, gate console script, tighten URL det…
galshubeli May 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -120,4 +120,5 @@ pylint
pytest
Radix
Zod
SDK
Dependabot
2 changes: 1 addition & 1 deletion .github/workflows/playwright.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ jobs:

# Install Python dependencies
- name: Install Python dependencies
run: uv sync --locked
run: uv sync --locked --all-extras

# Install Node dependencies (root - for Playwright)
- name: Install root dependencies
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:

- name: Install dependencies
run: |
uv sync
uv sync --locked --all-extras

- name: Run pylint
run: |
Expand Down
77 changes: 75 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:

- name: Install dependencies
run: |
uv sync --locked
uv sync --locked --all-extras

- name: Install frontend dependencies
run: |
Expand All @@ -63,8 +63,81 @@ jobs:

- name: Run unit tests
run: |
uv run python -m pytest tests/ -k "not e2e" --verbose
uv run python -m pytest tests/ -k "not e2e and not test_sdk" --verbose

- name: Run lint
run: |
make lint

sdk-tests:
runs-on: ubuntu-latest

services:
falkordb:
image: falkordb/falkordb:latest
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

postgres:
image: postgres:15
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: testdb
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5

mysql:
image: mysql:8
env:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: testdb
ports:
- 3306:3306
options: >-
--health-cmd "mysqladmin ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5

steps:
- uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.12'

- name: Install uv
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
with:
version: "latest"

- name: Install dependencies
run: |
uv sync --locked --all-extras

- name: Create test environment file
run: |
cp .env.example .env
echo "FASTAPI_SECRET_KEY=test-secret-key" >> .env
echo "FALKORDB_URL=redis://localhost:6379" >> .env

- name: Run SDK tests
env:
FALKORDB_URL: redis://localhost:6379
TEST_POSTGRES_URL: postgresql://postgres:postgres@localhost:5432/testdb
TEST_MYSQL_URL: mysql://root:root@localhost:3306/testdb
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
uv run python -m pytest tests/test_sdk/ -v
Comment thread
galshubeli marked this conversation as resolved.
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ ENV UV_SYSTEM_PYTHON=1
ENV PATH="/app/.venv/bin:$PATH"

# Install Python dependencies only (project itself installed after COPY)
RUN uv sync --frozen --no-dev --no-install-project
RUN uv sync --frozen --no-dev --extra server --no-install-project

# Install Node.js (Node 22) so we can build the frontend inside the image.
# Use NodeSource setup script to get a recent Node version on Debian-based images.
Expand Down Expand Up @@ -78,7 +78,7 @@ RUN npm --prefix ./app run build
COPY . .

# Install the project package now that source code is available
RUN uv sync --frozen --no-dev
RUN uv sync --frozen --no-dev --extra server

# Copy and make start.sh executable
COPY start.sh /start.sh
Expand Down
31 changes: 27 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
.PHONY: help install test test-unit test-e2e test-e2e-headed lint format clean setup-dev build lint-frontend
.PHONY: help install test test-unit test-e2e test-e2e-headed lint format clean setup-dev build lint-frontend test-sdk docker-test-services docker-test-stop build-package

help: ## Show this help message
@echo 'Usage: make [target]'
@echo ''
@echo 'Targets:'
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf " %-15s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf " %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)

install: ## Install dependencies
uv sync
Expand All @@ -23,10 +23,14 @@ build-dev:
build-prod:
npm --prefix ./app run build

build-package: ## Build distributable package (wheel + sdist)
uv build
@echo "Built packages in dist/"

test: build-dev test-unit test-e2e ## Run all tests

test-unit: ## Run unit tests only
uv run python -m pytest tests/ -k "not e2e" --verbose
test-unit: ## Run unit tests only (excludes SDK and E2E tests)
uv run python -m pytest tests/ -k "not e2e and not test_sdk" --ignore=tests/test_sdk --verbose


test-e2e: build-dev ## Run E2E tests headless
Expand Down Expand Up @@ -57,6 +61,8 @@ clean: ## Clean up test artifacts
rm -rf playwright-report/
rm -rf tests/e2e/screenshots/
rm -rf __pycache__/
rm -rf dist/
rm -rf *.egg-info/
find . -name "*.pyc" -delete
find . -name "*.pyo" -delete

Expand All @@ -72,3 +78,20 @@ docker-falkordb: ## Start FalkorDB in Docker for testing
docker-stop: ## Stop test containers
docker stop falkordb-test || true
docker rm falkordb-test || true

# SDK Testing
docker-test-services: ## Start all test services (FalkorDB + PostgreSQL + MySQL)
docker compose -f docker-compose.test.yml up -d
@echo "Waiting for services to be ready..."
@sleep 10

docker-test-stop: ## Stop all test services
docker compose -f docker-compose.test.yml down -v

test-sdk: ## Run SDK integration tests (requires docker-test-services)
uv run python -m pytest tests/test_sdk/ -v

test-sdk-quick: ## Run SDK tests without LLM (models and connection only)
uv run python -m pytest tests/test_sdk/test_queryweaver.py::TestModels tests/test_sdk/test_queryweaver.py::TestQueryWeaverInit -v

test-all: test-unit test-sdk test-e2e ## Run all tests
118 changes: 118 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,124 @@ Notes & tips
- The streaming response includes intermediate reasoning steps, follow-up questions (if the query is ambiguous or off-topic), and the final SQL. The frontend expects the boundary string `|||FALKORDB_MESSAGE_BOUNDARY|||` between messages.
- For destructive SQL (INSERT/UPDATE/DELETE etc) the service will include a confirmation step in the stream; the frontend handles this flow. If you automate destructive operations, ensure you handle confirmation properly (see the `ConfirmRequest` model in the code).

## Python SDK

The QueryWeaver Python SDK allows you to use Text2SQL functionality directly in your Python applications **without running a web server**.

### Installation

```bash
# SDK only (minimal dependencies)
pip install queryweaver
Comment thread
galshubeli marked this conversation as resolved.

# With server dependencies (FastAPI, etc.)
pip install queryweaver[server]

# Development (includes testing tools)
pip install queryweaver[dev]
```

### Quick Start

```python
import asyncio
from queryweaver import QueryWeaver

async def main():
# Initialize with FalkorDB connection
qw = QueryWeaver(falkordb_url="redis://localhost:6379")

# Connect a PostgreSQL or MySQL database
conn = await qw.connect_database("postgresql://user:pass@host:5432/mydb")
print(f"Connected: {conn.database_id}") # "mydb"

# Convert natural language to SQL and execute — pass the database_id
# returned by connect_database (un-prefixed; namespacing is internal).
result = await qw.query(conn.database_id, "Show me all customers from NYC")
print(result.sql_query) # SELECT * FROM customers WHERE city = 'NYC'
print(result.results) # [{"id": 1, "name": "Alice", "city": "NYC"}, ...]
print(result.ai_response) # "Found 42 customers from NYC..."

await qw.close()

asyncio.run(main())
```

### Context Manager

```python
async with QueryWeaver(falkordb_url="redis://localhost:6379") as qw:
conn = await qw.connect_database("postgresql://user:pass@host/mydb")
result = await qw.query(conn.database_id, "Count orders by status")
# close() runs automatically, awaiting any in-flight background memory writes.
```

### Multiple Instances

Multiple `QueryWeaver` instances can run side-by-side in the same process.
Each holds its own FalkorDB connection and passes it explicitly through
every call, so there is no shared global state to collide over.

```python
async with QueryWeaver(falkordb_url="redis://host-a:6379", user_id="tenant_a") as a, \
QueryWeaver(falkordb_url="redis://host-b:6379", user_id="tenant_b") as b:
sales = await a.connect_database("postgresql://user:pass@host-a/sales")
ops = await b.connect_database("postgresql://user:pass@host-b/ops")
await a.query(sales.database_id, "Show top customers")
await b.query(ops.database_id, "Count open tickets")
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

### Available Methods

| Method | Description |
|--------|-------------|
| `connect_database(db_url)` | Connect PostgreSQL/MySQL and load schema |
| `query(database, question)` | Convert natural language to SQL and execute |
| `get_schema(database)` | Retrieve database schema (tables and relationships) |
| `list_databases()` | List all connected databases |
| `delete_database(database)` | Remove database from FalkorDB |
| `refresh_schema(database)` | Re-sync schema after database changes |
| `execute_confirmed(database, sql)` | Execute confirmed destructive operations |

### Advanced Query Options

For multi-turn conversations, custom instructions, or per-request LLM overrides:

```python
from queryweaver import QueryWeaver, QueryRequest

request = QueryRequest(
question="Show their recent orders",
chat_history=["Show all customers from NYC"],
result_history=["Found 42 customers..."],
instructions="Use created_at for date filtering",
# Optional per-request LLM overrides — bypass env-based config
custom_api_key="sk-...",
custom_model="openai/gpt-4.1",
)

result = await qw.query("mydb", request)
```

### Handling Destructive Operations

INSERT, UPDATE, DELETE operations require confirmation:

```python
result = await qw.query("mydb", "Delete inactive users")

if result.requires_confirmation:
print(f"Destructive SQL: {result.sql_query}")
# Execute after user confirms
confirmed = await qw.execute_confirmed("mydb", result.sql_query)
```

### Requirements

- Python 3.12+
- FalkorDB instance (local or remote)
- OpenAI or Azure OpenAI API key (for LLM)
- Target SQL database (PostgreSQL or MySQL)

## Development

Expand Down
14 changes: 13 additions & 1 deletion api/core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,14 @@

from .errors import InternalError, GraphNotFoundError, InvalidArgumentError
from .schema_loader import load_database, list_databases
from .text2sql import MESSAGE_DELIMITER
from .pipeline import (
MESSAGE_DELIMITER,
graph_name,
get_database_type_and_loader,
sanitize_query,
sanitize_log_input,
is_general_graph,
)

__all__ = [
"InternalError",
Expand All @@ -17,4 +24,9 @@
"load_database",
"list_databases",
"MESSAGE_DELIMITER",
"graph_name",
"get_database_type_and_loader",
"sanitize_query",
"sanitize_log_input",
"is_general_graph",
]
25 changes: 25 additions & 0 deletions api/core/db_resolver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Resolve a FalkorDB handle, falling back to the server-side singleton.

Core text2sql functions accept an optional ``db`` parameter so the SDK can
inject its own connection without mutating process globals. When ``db`` is
None (route handlers that haven't threaded it yet), we lazily import the
module-level singleton from ``api.extensions``. The import is deferred so
the SDK can use this module without triggering ``api.extensions``'s
import-time FalkorDB connect.
"""

from typing import TYPE_CHECKING, Optional

if TYPE_CHECKING:
# Import only for type checking — avoids pulling falkordb at runtime when
# callers pass an explicit handle and never need the server default.
from falkordb.asyncio import FalkorDB


def resolve_db(db: Optional["FalkorDB"] = None) -> "FalkorDB":
"""Return the given ``db`` handle, or lazily import the server default."""
if db is not None:
return db
# pylint: disable=import-outside-toplevel
from api.extensions import db as _default_db
return _default_db
Loading
Loading