Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mcp-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ deploy(mdl_file_path="/workspace/my_mdl.json")
```

**Output — save a generated MDL:**
The agent writes YAML project files and compiled MDL JSON back to your host via the `mdl-project` skill:
The agent writes YAML project files and compiled MDL JSON back to your host via the `wren-project` skill:
```
# agent writes: mcp-server/workspace/my_project/wren_project.yml, models/*.yml, ...
# agent compiles to: mcp-server/workspace/my_project/target/mdl.json
Expand Down
46 changes: 41 additions & 5 deletions skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,29 +32,65 @@ npx openskills add Canner/wren-engine
```bash
cp -r skills/generate-mdl ~/.claude/skills/
# or all at once:
cp -r skills/generate-mdl skills/mdl-project skills/wren-sql skills/wren-quickstart ~/.claude/skills/
cp -r skills/generate-mdl skills/wren-project skills/wren-sql skills/wren-mcp-setup skills/wren-quickstart ~/.claude/skills/
```

Once installed, invoke a skill by name in your conversation:

```text
/wren-quickstart
/generate-mdl
/mdl-project
/wren-project
/wren-sql
/wren-quickstart
/wren-mcp-setup
```

## Available Skills

| Skill | Description |
|-------|-------------|
| [wren-quickstart](wren-quickstart/SKILL.md) | End-to-end quickstart — install skills, generate MDL, save project, start MCP server, and verify setup |
| [generate-mdl](generate-mdl/SKILL.md) | Generate a Wren MDL manifest from a live database using ibis-server introspection |
| [mdl-project](mdl-project/SKILL.md) | Save, load, and build MDL manifests as version-controlled YAML project directories |
| [wren-project](wren-project/SKILL.md) | Save, load, and build MDL manifests as version-controlled YAML project directories |
| [wren-sql](wren-sql/SKILL.md) | Write and correct SQL queries for Wren Engine — types, date/time, BigQuery dialect, error diagnosis |
| [wren-quickstart](wren-quickstart/SKILL.md) | Set up Wren Engine MCP via Docker and connect to Claude Code or other MCP clients |
| [wren-mcp-setup](wren-mcp-setup/SKILL.md) | Set up Wren Engine MCP via Docker, register with Claude Code or other MCP clients, and start querying |

See [SKILLS.md](SKILLS.md) for full details on each skill.

## Updating Skills

Each skill automatically checks for updates when invoked. If a newer version is available, the AI agent will notify you with the update command before continuing.

To update manually at any time:

```bash
# Update all skills
curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force

# Update a specific skill
curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force generate-mdl
```

## Releasing a New Skill Version

When updating a skill, two files must be kept in sync:

1. Update `version` in the skill's `SKILL.md` frontmatter:
```yaml
metadata:
author: wren-engine
version: "1.2" # bump this
```

2. Update the matching entry in [`versions.json`](versions.json):
```json
{
"generate-mdl": "1.2"
}
```

Both files must have the same version number. The `SKILL.md` version is what users have installed locally; `versions.json` is what the update check compares against.

## Requirements

- A running [ibis-server](../ibis-server/) instance
Expand Down
55 changes: 44 additions & 11 deletions skills/SKILLS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,37 @@ Skills are instruction files that extend AI agents with Wren-specific workflows.

---

## wren-quickstart

**File:** [wren-quickstart/SKILL.md](wren-quickstart/SKILL.md)

End-to-end onboarding guide for Wren Engine. Orchestrates the full setup flow — from installing skills and creating a workspace, to generating an MDL, saving it as a versioned project, starting the MCP Docker container, and verifying everything works.

### When to use

- Setting up Wren Engine for the first time
- Onboarding a new data source from scratch
- Getting a new team member started with Wren MCP

### Workflow summary

1. Install required skills via `install.sh`
2. Create a workspace directory on the host machine
3. Generate MDL from the database (`@generate-mdl`)
4. Save as a YAML project and compile to `target/` (`@wren-project`)
5. Start the Docker container and register the MCP server (`@wren-mcp-setup`)
6. Run `health_check()` to verify — then start a new session and query

### Dependent skills

| Skill | Purpose |
|-------|---------|
| `@generate-mdl` | Introspect database and build MDL JSON |
| `@wren-project` | Save MDL as YAML project + compile to `target/` |
| `@wren-mcp-setup` | Start Docker container and register MCP server |

---

## generate-mdl

**File:** [generate-mdl/SKILL.md](generate-mdl/SKILL.md)
Expand Down Expand Up @@ -33,14 +64,14 @@ Generates a complete Wren MDL manifest by introspecting a live database through
5. Optionally sample data for ambiguous columns
6. Build the MDL JSON (models, columns, relationships)
7. Validate via `mdl_validate_manifest`
8. Optionally save as a YAML project (see `mdl-project`)
8. Optionally save as a YAML project (see `wren-project`)
9. Deploy via `deploy_manifest`

---

## mdl-project
## wren-project

**File:** [mdl-project/SKILL.md](mdl-project/SKILL.md)
**File:** [wren-project/SKILL.md](wren-project/SKILL.md)

Manages Wren MDL manifests as human-readable YAML project directories — similar to dbt projects. Makes MDL version-control friendly by splitting the monolithic JSON into one YAML file per model.

Expand Down Expand Up @@ -112,11 +143,11 @@ Comprehensive SQL authoring and debugging guide for Wren Engine. Covers core que

---

## wren-quickstart
## wren-mcp-setup

**File:** [wren-quickstart/SKILL.md](wren-quickstart/SKILL.md)
**File:** [wren-mcp-setup/SKILL.md](wren-mcp-setup/SKILL.md)

Sets up Wren Engine MCP server via Docker and connects it to Claude Code (or another MCP client) over streamable-http.
Sets up Wren Engine MCP server via Docker, registers it with an AI agent (Claude Code or other MCP clients), and starts a new session to begin interacting with Wren.

### When to use

Expand All @@ -128,10 +159,10 @@ Sets up Wren Engine MCP server via Docker and connects it to Claude Code (or ano
### Workflow summary

1. Ask user for workspace mount path
2. Create `docker/.env` with `MDL_WORKSPACE`
3. `docker compose up -d`
4. Rewrite `localhost` → `host.docker.internal` in connection credentials
5. Add `wren` MCP server to Claude Code using streamable-http on port 9000
2. `docker run` with workspace mounted at `/workspace`, MCP server enabled on port 9000
3. Rewrite `localhost` → `host.docker.internal` in connection credentials
4. Add `wren` MCP server to Claude Code using streamable-http on port 9000 (`claude mcp add`)
5. Start a new session so the MCP tools are loaded
6. Run `health_check()` to verify

---
Expand All @@ -150,6 +181,8 @@ Then invoke in your AI client:

```
/generate-mdl
/mdl-project
/wren-project
/wren-sql
/wren-mcp-setup
/wren-quickstart
```
114 changes: 86 additions & 28 deletions skills/generate-mdl/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
---
name: generate-mdl
description: Generate a Wren MDL manifest from a database using ibis-server metadata endpoints. Use when a user wants to create or set up a new Wren MDL, scaffold a manifest from an existing database, or onboard a new data source without installing any database drivers locally.
compatibility: Requires a running ibis-server and the Wren MCP server with tools: setup_connection, list_remote_tables, list_remote_constraints, mdl_validate_manifest, mdl_save_project, deploy_manifest
compatibility: Requires a running ibis-server (default port 8000). No local database drivers needed.
metadata:
author: wren-engine
version: "1.0"
version: "1.1"
---

# Generate Wren MDL

## Version check

Silently fetch `https://raw.githubusercontent.com/Canner/wren-engine/main/skills/versions.json`.
Compare the `generate-mdl` key with this skill's version (`1.1`).
If the remote version is newer, notify the user before proceeding:

> A newer version of the **generate-mdl** skill is available (remote: X.Y, installed: 1.1).
> Update with:
> ```bash
> curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force generate-mdl
> ```

Then continue with the workflow below regardless of update status.

---

Generates a Wren MDL manifest by using ibis-server to introspect the database schema — no local database drivers required. All schema discovery goes through ibis-server, which already has drivers for all supported data sources.

## Workflow
Expand All @@ -20,77 +36,119 @@ Follow these steps in order. Do not skip steps or ask unnecessary questions betw
Ask the user for:
1. **Data source type** — one of: `POSTGRES`, `MYSQL`, `MSSQL`, `DUCKDB`, `BIGQUERY`, `SNOWFLAKE`, `CLICKHOUSE`, `TRINO`, `ATHENA`, `ORACLE`, `DATABRICKS`
2. **Connection credentials** — see [Connection info format](#connection-info-format) below
3. **Schema filter** (optional) — if the database has many schemas, ask which schema(s) to include

Do not ask for a SQLAlchemy connection string. Use the structured `conn_info` dict instead.
Do not ask for a SQLAlchemy connection string. Use the structured `connectionInfo` dict instead.

### Step 2 — Register connection

```
setup_connection(datasource=<type>, conn_info=<dict>)
```
> **Important:** If the database runs on the host machine and ibis-server runs inside Docker, replace `localhost` / `127.0.0.1` with `host.docker.internal` in the host field.

If this fails, report the error and ask the user to correct the credentials.
### Step 2 — Fetch table schema

### Step 3 — Fetch table schema
Call the ibis-server metadata endpoint directly:

```
list_remote_tables()
POST http://localhost:8000/v3/connector/<data_source>/metadata/tables
Content-Type: application/json

{
"connectionInfo": { <credentials dict> }
}
```

ibis-server returns a list of tables with their column names and types. Parse the response to understand the full database schema.
ibis-server returns a list of tables with their column names and types. Each table entry has a `properties.schema` field — use it to filter to the user's target schema if specified.

If this fails, report the error and ask the user to correct the credentials.

### Step 4 — Fetch relationships
### Step 3 — Fetch relationships

```
list_remote_constraints()
POST http://localhost:8000/v3/connector/<data_source>/metadata/constraints
Content-Type: application/json

{
"connectionInfo": { <credentials dict> }
}
```

ibis-server returns foreign key constraints. Use these to build `Relationship` entries in the MDL.
Returns foreign key constraints. Use these to build `Relationship` entries in the MDL. If the response is empty (`[]`), infer relationships from column naming conventions (e.g. `order_id` → `orders.id`).

### Step 5 — Sample data (optional)
### Step 4 — Sample data (optional)

For columns where purpose is unclear from the name and type alone:
For columns where purpose is unclear from the name and type alone, query a few rows using the raw table name with schema prefix:

```
query("SELECT * FROM <table_name> LIMIT 3")
POST http://localhost:8000/v3/connector/<data_source>/query
Content-Type: application/json

{
"sql": "SELECT * FROM <schema>.<table> LIMIT 3",
"manifestStr": "",
"connectionInfo": { <credentials dict> }
}
```

Note: use the raw table name (not model name) at this stage, since MDL is not yet deployed.
Note: use the raw `schema.table` reference at this stage, since the MDL is not yet deployed.

### Step 6 — Build MDL JSON
### Step 5 — Build MDL JSON

Construct the manifest following the [MDL structure](#mdl-structure) below.

Rules:
- `catalog`: use `"wren"` unless the user specifies otherwise
- `schema`: use the database's default schema (e.g. `"public"` for PostgreSQL, `"dbo"` for MSSQL)
- `schema`: use the target schema name (e.g. `"public"` for PostgreSQL default, `"jaffle_shop"` if user specified)
- `dataSource`: set to the enum value from Step 1 (e.g. `"POSTGRES"`)
- `tableReference.catalog`: set to the database name (not `"wren"`)
- Each table → one `Model`. Set `tableReference.table` to the exact table name
- Each column → one `Column`. Use the exact DB column name
- Mark primary key columns with `"isPrimaryKey": true` and set `primaryKey` on the model
- For FK columns, add a `Relationship` entry linking the two models
- Omit calculated columns for now — they can be added later

### Step 7 — Validate
### Step 6 — Validate

Validate the MDL by running a dry-plan against a simple query. Base64-encode the manifest first:

```python
import json, base64
manifest_b64 = base64.b64encode(json.dumps(mdl).encode()).decode()
```

Then call:

```
mdl_validate_manifest(mdl=<manifest dict>)
POST http://localhost:8000/v3/connector/<data_source>/dry-plan
Content-Type: application/json

{
"manifestStr": "<base64-encoded manifest>",
"sql": "SELECT * FROM <any_model_name> LIMIT 1"
}
```

This calls ibis-server's dry-plan endpoint. If validation fails, fix the reported errors and validate again.
If validation succeeds, the response is the planned SQL string. If it fails, fix the reported errors and validate again.

### Step 8 — Save project (optional)
> **Note:** Use the `/v3/` endpoint, not `/v2/`. The v2 dry-plan requires a separate Wren Engine Java process (`WREN_ENGINE_ENDPOINT`) which is not part of the standard Docker setup.

### Step 7 — Save project (optional)

Ask the user if they want to save the MDL as a YAML project directory (useful for version control).

If yes, follow the **mdl-project** skill (`skills/mdl-project/SKILL.md`) to write the YAML files.
If yes, follow the **wren-project** skill (`skills/wren-project/SKILL.md`) to write the YAML files and build `target/mdl.json` + `target/connection.json`.

### Step 8 — Deploy

### Step 9 — Deploy
**If Wren MCP tools are available** (i.e., Claude Code has the `wren` MCP server registered):

```
deploy_manifest(mdl=<manifest dict>)
```

**If MCP tools are not available**, deploy by writing the MDL to the workspace file that the container watches:

1. Build `target/mdl.json` from the YAML project (see wren-project skill)
2. Ensure the container was started with `-e MDL_PATH=/workspace/target/mdl.json`
3. Restart the container to reload — or call the `deploy` MCP tool after connecting

Confirm success to the user. The MDL is now active and queries can run.

---
Expand Down Expand Up @@ -188,7 +246,7 @@ When in doubt, use `VARCHAR` as a safe fallback.

## Connection info format

Pass to `setup_connection(datasource=..., conn_info={...})`:
Pass to `setup_connection(datasource=..., connectionInfo={...})`:

```
POSTGRES : {"host": "...", "port": "5432", "user": "...", "password": "...", "database": "..."}
Expand Down
6 changes: 5 additions & 1 deletion skills/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ set -euo pipefail
REPO="Canner/wren-engine"
BRANCH="${WREN_SKILLS_BRANCH:-main}"
DEST="${CLAUDE_SKILLS_DIR:-$HOME/.claude/skills}"
ALL_SKILLS=(generate-mdl mdl-project wren-sql wren-quickstart)
ALL_SKILLS=(generate-mdl wren-project wren-sql wren-mcp-setup wren-quickstart)

# Parse --force flag and skill list from arguments
FORCE=false
Expand Down Expand Up @@ -113,3 +113,7 @@ echo "Done. Invoke skills in your AI client:"
for skill in "${SELECTED_SKILLS[@]}"; do
echo " /$skill"
done
echo ""
echo "To update skills later, re-run with --force:"
echo " curl -fsSL https://raw.githubusercontent.com/Canner/wren-engine/main/skills/install.sh | bash -s -- --force"
echo "Or check for updates: each skill notifies you automatically when a newer version is available."
7 changes: 7 additions & 0 deletions skills/versions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"generate-mdl": "1.1",
"wren-project": "1.1",
"wren-sql": "1.0",
"wren-mcp-setup": "1.1",
"wren-quickstart": "1.0"
}
Loading