Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased

`GET /version` with a capability list (new `taosmd.capabilities` module). The server now publishes what the running build actually supports, because neither a status code nor a version number could answer that. `taosmd serve` renders the dashboard SPA on unknown non-API paths, so `GET /collections` returns `200 text/html` on a build with no collections code, and an integrator who "verified" a route by checking for a 200 got a confident yes from a server that could not do the thing (this really happened, against the wrong service). Semver does not close the gap either: features land continuously between bumps, and a production box sat a month stale without anyone noticing even though `GET /health` already reported a version. `/version` returns `{"version", "commit", "commit_source", "built_at", "built_at_source", "capabilities"}` and `GET /health` gains the same `capabilities` list alongside its existing `status` and `version` keys, which are unchanged (taOS and the dashboard consume both). Both endpoints are unauthenticated by design, joining `/health` in `_PUBLIC_PATHS`, so monitoring and drift probes keep working on a token-secured box; they expose build identity and capability identifiers only (no paths, no tokens, no configuration). Capabilities are **stable contract identifiers with an explicit version suffix** (`collections.v1`, `grants.v1`, `temporal.v1`, `a2a.v1`, `tasks.v1`, `ingest.v1`, `search.v1`, `graph.v1`, `shelves.v1`), not feature names: a breaking change to a wire contract becomes `collections.v2`, so a client pinned to `collections.v1` sees the capability disappear (a visible break it can act on) rather than `collections` silently meaning something new; additive changes keep the identifier. The list is derived at request time by probing the running build (each identifier is declared next to the module and symbols that implement it, and is advertised only if they resolve), so deleting or renaming an implementation deletes the claim instead of leaving a stale boast, and a divergence test asserts every declared capability's routes exist in the real dispatcher. The commit sha is resolved once at first call and cached, never per request and never by shelling out: `git rev-parse` in a request path can block on a lock or a slow filesystem, so the git plumbing is read directly from the filesystem (`.git/HEAD` -> loose ref or `packed-refs`, including the `gitdir:` indirection used by worktrees and submodules), with an optional packaged `taosmd/_build_info.py` stamp taking precedence for wheel and container builds. Every step degrades to `null` rather than raising, so a pip install with no checkout and no stamp still gets a working endpoint.

Collections, Phase 1 (docs MVP, per `docs/specs/codebase-indexing-collections-design.md`): named containers of content indexed from a folder, queryable by granted agents alongside conversation memory. A collection is a first-class row (`created -> indexing -> ready | error`, plus reversible `archived`) with typed project links (`{type: taos|git, id}`, metadata only, never access-granting) and per-agent grants (`(canonical_id, scope='collection', collection_id)` unique rows, enforced at search time). Indexing wires the previously-unwired loader framework into a real ingest path: a gitignore-aware walker (stdlib rules; VCS/dependency/hidden dirs, binaries, oversized files, and symlink escapes skipped) feeds files a registered loader claims through a zero-dep paragraph chunker into `ingest_batch` under the collection's own agent namespace, with per-chunk content-hash ids so re-index dedups unchanged files; changed and deleted files have their old rows superseded (`valid_to` + marker), never deleted. The feature is off by default: the new `collections.allowed_roots` config list (or `TAOSMD_COLLECTIONS_ALLOWED_ROOTS`) must name the directories collections may index, and `source_path` is containment-checked (`resolve_within`) at create and at every index. Surfaces: HTTP (`POST /collections` and `POST /collections/{id}/index` admin-gated with async 202+poll indexing, `DELETE /collections/{id}` archives; list/get/link/unlink/grants on the data plane; `collection`/`collections`/`collections_only` on search), CLI (`taosmd collections list|create|index|link|unlink|grant|revoke`), and MCP (`memory_list_collections`, `collection` on `memory_search`). Collection hits carry `collection_id`/`file_path`/`source` metadata. A per-collection `embedder` field is stored and returned now (the mechanism for the code-embedder bake-off); Phase 1 always indexes with the global default. `benchmarks/collections_eval.py` pre-registers the file-level Recall@5 eval over the repo's own docs.

Admin token separation (#154, phase 1). Admin operations are now gated by a dedicated `admin_token`, distinct from the data-plane `server_token`. Previously the server token gated every data and A2A endpoint AND the admin surface, so on a token-less deployment the only way to authorize an admin op was to set a server token, which locked out every agent on the data plane for the duration of the admin window (this hit the Pi bus in production for about three minutes during a channel cleanup). Now the admin write routes (`POST /shelves`, `POST /shelves/{id}/archive|unarchive`, `POST /a2a/admin/delete-channel|rename-channel|supersede-message`) are exempt from the data-plane token gate and enforce the admin token themselves. Resolution prefers `admin_token` and falls back to `server_token`: existing token-secured installs keep working unchanged; setting only `admin_token` gates admin while leaving data and A2A endpoints open; with both set the data plane is gated by `server_token` and admin by `admin_token`, so a caller holding only the server token cannot run admin ops; with neither set the admin surface still fails closed (403). Configure via `admin_token` in config, the `TAOSMD_ADMIN_TOKEN` env var, or `taosmd config set-admin-token`. Phase 2 (isolating admin operations from the single service loop so a slow admin op cannot stall data reads/writes) is not part of this change and is tracked separately.
Expand Down
46 changes: 44 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -560,7 +560,7 @@ taosmd config set-token <your-secret-token>
export TAOSMD_TOKEN=<your-secret-token>
```

The token is sent as `Authorization: Bearer <token>` on every request. `GET /health` and the web inspector (`GET /`) are always public so monitoring probes keep working. Never commit the token to version control.
The token is sent as `Authorization: Bearer <token>` on every request. `GET /health`, `GET /version`, and the web inspector (`GET /`) are always public so monitoring and capability probes keep working. Never commit the token to version control.

### How the Python API and MCP server interact with remote mode

Expand Down Expand Up @@ -698,7 +698,8 @@ ollama pull qwen3:4b # Same model as the smaller node, same quality

| Method | Path | Request | Response |
|--------|------|---------|----------|
| `GET` | `/health` | (none) | `{"status": "ok", "version": <str>}` |
| `GET` | `/health` | (none) | `{"status": "ok", "version": <str>, "capabilities": [<str>]}` |
| `GET` | `/version` | (none) | `{"version", "commit", "commit_source", "built_at", "built_at_source", "capabilities"}` |
| `POST` | `/ingest` | `{"text": str, "agent": str, "project"?: str}` | `{"archived": int, "agent": str, "project": str\|null, "data_dir": str, ...}` (adds `"vector_failures": int` and `"degraded": true` when the embedder fails) |
| `POST` | `/ingest/batch` | `{"items": [{"text": str, "id"?: str, "metadata"?: obj}], "agent": str, "project"?: str}` | `{"ingested": int, "skipped": int, ...}` |
| `POST` | `/search` | `{"query": str, "agent": str, "limit"?: int, "project"?: str, "also_include"?: [str], "mode"?: "bm25"}` | `{"hits": [...]}` |
Expand All @@ -718,6 +719,47 @@ Each hit in `/search` results has the agent-rules contract shape: `{text, source

`/ingest/batch` is the bulk-import path: each item can carry a stable `id` (your content hash), preserved as `source_id` and used to skip already-imported items, so the whole batch can be re-POSTed safely after a partial migration. `mode=bm25` on `/search` skips query embedding entirely and returns keyword-ranked hits in about 10ms, built for search-as-you-type UIs over short-form memory; the default mode remains the full recipe-driven retrieval.

### Version and capability discovery

Do not probe for a feature with a status code. `taosmd serve` answers unknown non-API paths with the dashboard SPA, so `GET /collections` returns `200 text/html` even on a build that has no collections code at all. Ask `GET /version` instead:

```bash
curl -s http://127.0.0.1:7900/version
```

```json
{
"version": "0.4.0",
"commit": "76f72ffef139a9cc08c76d7348b9b25849c845a6",
"commit_source": "git",
"built_at": "2026-07-21T11:38:52Z",
"built_at_source": "install",
"capabilities": [
"a2a.v1", "collections.v1", "grants.v1", "graph.v1",
"ingest.v1", "search.v1", "shelves.v1", "tasks.v1", "temporal.v1"
]
}
```

`capabilities` is a list of **stable contract identifiers**, not feature names. The `.vN` suffix is the contract: when a wire contract changes in a way that breaks existing callers, the identifier becomes `collections.v2`, so a client pinned to `collections.v1` sees the capability disappear (a visible break it can act on) instead of `collections` quietly meaning something new. Additive changes keep the same identifier. The right client check is membership:

```python
caps = set(requests.get(f"{base}/version").json()["capabilities"])
if "collections.v1" not in caps:
raise RuntimeError("this taOSmd build does not speak collections.v1")
```

The list is derived at runtime by probing the running build (see `taosmd/capabilities.py`), so it cannot advertise a feature whose code is absent. `commit` and `built_at` are best-effort and may be `null` (see the table below); they are there so an operator can spot a box running a stale build. `/version` is unauthenticated and cheap, like `/health`, and exposes nothing beyond build identity and capability identifiers (no paths, no tokens, no configuration).

`GET /health` returns the same `capabilities` list alongside its existing `{"status", "version"}` keys, which are unchanged.

| Field | Meaning |
|-------|---------|
| `commit` | 40-char sha of the build, or `null` |
| `commit_source` | `"git"` (resolved from the checkout), `"build-stamp"` (packaged `taosmd/_build_info.py`), or `null` |
| `built_at` | ISO 8601 UTC build or install time, or `null` |
| `built_at_source` | `"build-stamp"`, `"install"` (dist-info mtime), or `null` |

### Agent-to-agent (A2A) bus

`taosmd serve` also exposes a lightweight message bus for agent-to-agent communication on the same port:
Expand Down
33 changes: 33 additions & 0 deletions docs/collections.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,39 @@ Over MCP: `memory_list_collections` lists them; `memory_search` takes a

## HTTP surface

### Check the server actually speaks collections first

Collections landed after several releases of `taosmd serve`, so an integrator
pointing at an existing deployment cannot assume the routes are there. Do not
check by requesting `/collections` and looking at the status code: the server
answers unknown non-API paths with the dashboard SPA, so `GET /collections`
returns `200 text/html` on a build with no collections code at all. That check
has already sent one integration to the wrong service.

Ask `GET /version` (public, no token needed) and test capability membership:

```bash
curl -s http://127.0.0.1:7900/version | jq -r '.capabilities[]'
# a2a.v1
# collections.v1
# grants.v1
# ...
```

```python
caps = set(httpx.get(f"{base}/version").json()["capabilities"])
if "collections.v1" not in caps:
raise RuntimeError("server does not speak collections.v1")
if "grants.v1" not in caps:
raise RuntimeError("server cannot grant collection access")
```

`collections.v1` covers list/get/create/index/link/unlink/archive; `grants.v1`
covers the per-agent grant and revoke routes. The identifiers are derived from
the running build, so a server cannot advertise a capability it lacks, and a
breaking change to the collections wire contract will appear as
`collections.v2` rather than silently redefining `collections.v1`.

Data plane (bearer token when one is configured):

```text
Expand Down
23 changes: 20 additions & 3 deletions docs/serve-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,9 @@ the same machine can reach the API. Authentication is **off by default**, but
a bearer token is available: when `server_token` is set in the server config
(via `taosmd config set-token`) or the `TAOSMD_TOKEN` environment variable is
set, every data and A2A endpoint requires an `Authorization: Bearer <token>`
header and returns `401` otherwise. The `/health`, `/`, and `/ui` endpoints
always stay public so monitoring probes and the browser dashboard keep working.
header and returns `401` otherwise. The `/health`, `/version`, `/`, and `/ui`
endpoints always stay public so monitoring probes, capability probes, and the
browser dashboard keep working.

On a trusted private network (a home LAN, a Tailscale network), the network
boundary is typically sufficient as the access control. For any public-facing or
Expand All @@ -188,7 +189,23 @@ whether you run `taosmd serve` in the foreground or as a background service.

```bash
curl http://127.0.0.1:7900/health
# Expected: {"status": "ok", "version": "..."}
# Expected: {"status": "ok", "version": "...", "capabilities": [...]}
```

To check which build is running and what it actually supports (useful for
spotting a box left on a stale build):

```bash
curl http://127.0.0.1:7900/version
# {"version": "...", "commit": "...", "commit_source": "git",
# "built_at": "...", "built_at_source": "install",
# "capabilities": ["a2a.v1", "collections.v1", ...]}
```

`capabilities` holds stable contract identifiers derived from what the running
build implements. Test membership (`"collections.v1" in capabilities`) rather
than probing a route's status code: unknown non-API paths serve the dashboard
SPA, so a `200` there says nothing about what the server supports. See the
"Version and capability discovery" section of the README for the `.vN` contract.

The read-only inspection UI is at `http://127.0.0.1:7900/` in your browser.
Loading