Mobile chat tooling pass: expanded tools, reliable navigation, and token/context optimization

README.md

@@ -404,11 +404,11 @@ Unit/integration tests (`electron/**/*.test.ts`) cover SQLite queries, file I/O,
 
 ## Star History
 <p align="center">
-<a href="https://www.star-history.com/?repos=ddutchie%2Fcairn&type=date&legend=top-left">
+<a href="https://www.star-history.com/?repos=ddutchie%2Fcairn&type=date&legend=bottom-right">
  <picture>
-   <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&theme=dark&legend=top-left" />
-   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&legend=top-left" />
-   <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&legend=top-left" />
+   <source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&theme=dark&legend=bottom-right&sealed_token=gaTjy9B4KXJYY2tjt1lWuTxZHL2F-HZBa4drw4LZiG8HyHu4VItVT3Milarsc7LQd56kRacKeYnNFdtus8dp6WMn6dNB61zTm1oYMWCgPIJkREq8xf93-0jzRXGoTTqbW4NxE2LQ12oR6c_dsvoExhpXri_zIcQftFid2HwSMUcrJephPz4pnlUx-5iz" />
+   <source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&legend=bottom-right&sealed_token=gaTjy9B4KXJYY2tjt1lWuTxZHL2F-HZBa4drw4LZiG8HyHu4VItVT3Milarsc7LQd56kRacKeYnNFdtus8dp6WMn6dNB61zTm1oYMWCgPIJkREq8xf93-0jzRXGoTTqbW4NxE2LQ12oR6c_dsvoExhpXri_zIcQftFid2HwSMUcrJephPz4pnlUx-5iz" />
+   <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=ddutchie/cairn&type=date&legend=bottom-right&sealed_token=gaTjy9B4KXJYY2tjt1lWuTxZHL2F-HZBa4drw4LZiG8HyHu4VItVT3Milarsc7LQd56kRacKeYnNFdtus8dp6WMn6dNB61zTm1oYMWCgPIJkREq8xf93-0jzRXGoTTqbW4NxE2LQ12oR6c_dsvoExhpXri_zIcQftFid2HwSMUcrJephPz4pnlUx-5iz" />
  </picture>
 </a>
 </p>

changelogs/v2.4.8.md

@@ -0,0 +1,13 @@
+## What's new
+
+The built-in AI assistant now uses a much leaner system prompt with no change to what it can do.
+
+### Under the hood
+
+- **Trimmed the AI Chat system prompt (~960 → ~240 tokens).** The prompt previously restated per-tool guidance (Notes / Tasks / Dashboards / Idea Flow / Knowledge Graph sections) that the model already receives from each tool's own description. A live tool-selection experiment showed the assistant picks the right tool at least as reliably — actually slightly better — with a prompt that keeps only the cross-cutting rules (obtaining IDs, confirming after writes, the suggest-connections behaviour, and markdown/mermaid rendering) and leans on the tool descriptions for the rest. An opt-in regression test (`electron/lib/prompt-optimization.test.ts`, skipped unless an LLM endpoint is configured — no secrets in source) guards that the trimmed prompt keeps selecting tools at least as well as the old one.
+- **Slimmer AI tool definitions (~5,090 → ~4,630 tokens per request).** The JSON tool schemas sent to the model on every chat turn carried mechanical validation noise that the model never uses — giant ISO-8601 date regexes, integer min/max bounds, defaults, and `additionalProperties` flags (server-side validation is unchanged and still enforces all of it). Stripping that noise from the emitted schemas shrinks every request with no change to tool selection or argument correctness — verified by a live experiment across single-turn and multi-turn (chained) tool sequences. Field descriptions are kept, since those genuinely help the model fill arguments.
+- **Leaner project-context tool responses.** `get_project_context_pack` (used by the AI assistant and MCP clients) previously embedded up to 1,000 characters for *every* pinned note with no limit on how many, plus every open task's description — a busy project could return 5,000+ tokens in a single call that then rode along in the conversation for the rest of the session. It now returns the 5 most-recent pinned notes with a short excerpt (and a `pinnedNotesTotal` count) and caps open tasks with short previews, cutting a busy project's response ~70%. The assistant fetches full content on demand via `get_note` / `get_task`.
+- **Pinned notes are summarised by their outline.** Instead of a raw first-few-hundred-characters excerpt, a pinned note that has section headings is now represented by its heading outline (e.g. `Auth Design › Token rotation › Refresh flow`) — a more useful summary in roughly half the tokens. Flat notes with no headings still get a short excerpt. (This is the readable, structured way to convey a note's meaning — vector embeddings can't be turned back into text.)
+
+
+

electron/ipc/chat-executor.test.ts

@@ -927,13 +927,20 @@ describe("get_project_context_pack", () => {
     expect(allIds).not.toContain("done-card");
   });
 
-  it("pinnedNotes includes full content", async () => {
+  it("pinnedNotes carry a digest — outline for structured notes, excerpt for flat ones", async () => {
     const db = makeDb();
     seed(db);
-    createNote(db, { id: "pinned", projectId: "proj1", workspaceId: "ws1", title: "Pinned", content: "# Overview\nDetails here", isPinned: true });
+    // Structured note (multiple sections) → outline digest.
+    createNote(db, { id: "pinnedStruct", projectId: "proj1", workspaceId: "ws1", title: "Design", content: "# Design\n\n## Goals\ng\n\n## Approach\na", isPinned: true });
+    // Flat note (title only) → excerpt digest.
+    createNote(db, { id: "pinnedFlat", projectId: "proj1", workspaceId: "ws1", title: "Note", content: "just some prose here", isPinned: true });
     const result = await exec(db, "get_project_context_pack", { projectId: "proj1" }) as Record<string, unknown>;
-    const pinned = result.pinnedNotes as Array<{ id: string; content: string }>;
-    expect(pinned.find((n) => n.id === "pinned")?.content).toContain("Overview");
+    const pinned = result.pinnedNotes as Array<{ id: string; outline?: string[]; excerpt?: string }>;
+    const struct = pinned.find((n) => n.id === "pinnedStruct");
+    const flat = pinned.find((n) => n.id === "pinnedFlat");
+    expect(struct?.outline).toContain("Goals");
+    expect(flat?.excerpt).toContain("prose");
+    expect(result).toHaveProperty("pinnedNotesTotal");
   });
 
   it("returns { error } for unknown project", async () => {

electron/lib/context-audit.test.ts

@@ -1,7 +1,7 @@
 /**
  * Context Audit — real token counts for every LLM surface: chat, pi-agent, MCP.
  *
- * Uses gpt-tokenizer (cl100k_base) which matches gpt-4o / gpt-4-turbo encoding.
+ * Uses gpt-tokenizer's default encoder (o200k_base) — the gpt-4o encoding.
  * OpenAI serialises each tool as:
  *   {"type":"function","function":{"name":"...","description":"...","parameters":{...}}}
  * We count that exact JSON string, which matches what the API receives.
@@ -154,7 +154,7 @@ function grand(label: string, tokens: number) {
 
 // ── Tests ────────────────────────────────────────────────────────────────────
 
-describe("Context Audit (real tokens via gpt-tokenizer / cl100k_base)", () => {
+describe("Context Audit (real tokens via gpt-tokenizer / o200k_base)", () => {
 
   it("EXECUTE MODE — pi-agent", () => {
     const defs = executeDefs();