Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
a472bf3
feat(mobile): formatting/AI toolbar in the new-note composer
ddutchie Jul 7, 2026
9749008
Mobile Styling Pass
ddutchie Jul 7, 2026
d0b7677
perf(mobile): batch tag queries + lazy-load WebView bundles
ddutchie Jul 7, 2026
30db00c
perf(mobile): drive board drag hover-highlight on the UI thread
ddutchie Jul 7, 2026
f70fb35
fix(mobile): crash on card drag — withAlpha() called inside worklet
ddutchie Jul 7, 2026
c1aa285
perf(mobile): memoise chat bubbles + hoist tag-chip styles
ddutchie Jul 7, 2026
ddeb964
perf(mobile): virtualize the notes list + fix defeated row memo
ddutchie Jul 7, 2026
2314bca
fix(mobile): add tab-bar safe-zone bottom padding to notes list + not…
ddutchie Jul 7, 2026
4c08898
feat(mobile): experimental on-device Apple Intelligence provider (dev…
ddutchie Jul 7, 2026
5c14cf5
Fix Status Bar
ddutchie Jul 7, 2026
f0ff1be
PR Review
ddutchie Jul 7, 2026
f918ab6
refactor(mobile): extract useNoteFormattingToolbar shared hook
ddutchie Jul 8, 2026
6b30ee8
fix(mobile): lock note body while an AI action is pending
ddutchie Jul 8, 2026
5224dfe
feat(mobile): on-device semantic search via Apple NLContextualEmbedding
ddutchie Jul 8, 2026
5188ca2
fix(mobile): always show Semantic search scope + explain when unavail…
ddutchie Jul 8, 2026
414c0d1
fix(mobile): move search scope switch into the native Stack.Toolbar
ddutchie Jul 8, 2026
d1d3759
fix(mobile): correct search results scrolling (header + keyboard insets)
ddutchie Jul 8, 2026
3acd1fe
style(mobile): use standard (non-large) header on Search tab
ddutchie Jul 8, 2026
1867f74
feat(mobile): embeddings SwiftPM tests, sync-modal reindex, pinned gl…
ddutchie Jul 8, 2026
92f06b9
polish(mobile): search empty-state hints + scope bar position
ddutchie Jul 8, 2026
705b3e9
refactor(mobile): clear all lint warnings (setState-in-effect + dup i…
ddutchie Jul 8, 2026
ce4bc07
fix(mobile): centre embeddings by corpus centroid so search scores di…
ddutchie Jul 8, 2026
04cdb6a
fix(mobile): drop BGE query prefix from Apple embeddings search
ddutchie Jul 8, 2026
36d9b35
fix(mobile): address PR review — race guards, error handling, stale s…
ddutchie Jul 8, 2026
2cf0e4e
fix(mobile): hybrid lexical+semantic search — real-corpus test expose…
ddutchie Jul 8, 2026
f031d4c
feat(mobile): zoom-to-fit button on the graph legend row
ddutchie Jul 8, 2026
44be613
fix(mobile): sort semantic results by hybrid rank, not raw cosine
ddutchie Jul 8, 2026
1927d28
fix(mobile): semantic search % reflects rank; fix first result behind…
ddutchie Jul 8, 2026
8d27169
fix(mobile): dismiss keyboard before opening a search result
ddutchie Jul 8, 2026
ed1b1d2
feat(mobile): semantic search in Chat + Related Notes on note detail
ddutchie Jul 8, 2026
c1ff5dc
feat(mobile): table of contents on note detail (floating glass button…
ddutchie Jul 8, 2026
9e05b6a
fix(mobile): TOC button uses plain insets.bottom (tab bar already in …
ddutchie Jul 8, 2026
abc4603
fix(mobile): drop double-counted TAB_BAR_BASE from note/project scrol…
ddutchie Jul 8, 2026
4067838
feat(mobile): shared branded EmptyState (Cairn icon + version) across…
ddutchie Jul 8, 2026
7a68cba
feat(mobile): branded EmptyState on the chat tab
ddutchie Jul 8, 2026
41e0879
feat(mobile): Cairn Breakout easter egg + expo-haptics feedback
ddutchie Jul 8, 2026
9d54559
feat(mobile): haptic feedback on native-route modals
ddutchie Jul 8, 2026
6eac49e
feat(mobile): tactile feedback for taps and toolbar actions
ddutchie Jul 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 210 additions & 0 deletions docs/plans/apple-foundation-models-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
# Plan: On-device / Private Cloud Compute Apple Intelligence provider

**Status:** shelved. On-device (iOS 26) implementation exists in the repo but is
**not viable** for Cairn's agentic tool use. Re-implement against **Private Cloud
Compute (PCC)** once the managed entitlement is granted and an **iOS 27 SDK** is
available.

---

## Current gating (dev-only, hidden from users)

The on-device provider is **hidden from end users** but kept for local iteration,
gated on a build-time flag:

- `EXPO_PUBLIC_APPLE_LLM_DEV=1` in the local, git-ignored `mobile/.env` exposes the
"On-device" provider (Settings → AI, `resolveProvider`, `hasProvider`).
- EAS `preview`/`production` builds don't receive `.env` and the flag isn't set in
`eas.json`, so the var is `undefined` → `isAppleDevEnabled()` is false → the
provider never appears or resolves in shipped builds.
- Central gate: `isAppleProviderAvailable()` = `isAppleDevEnabled() && isAppleLlmAvailable()`
in `src/chat/providers/apple.ts`. All consumers already route through it.
- The native module (`modules/apple-llm`) still **compiles** in EAS (autolinked,
tiny, dormant — never invoked when the flag is off). We deliberately did NOT
exclude it from the native build, to keep local/EAS prebuild identical and avoid
a fragile conditional-autolinking setup.

To fully remove it later (if PCC supersedes it), delete `modules/apple-llm`,
`src/chat/providers/apple.ts`, the `"apple"` `ProviderPref` value, the settings
segment, and the `resetAppleSession` wiring — git history + this doc are the recipe.

## TL;DR

- We built a complete on-device Apple Foundation Models provider (`modules/apple-llm`
+ `src/chat/providers/apple.ts`) using native tool-calling via guided generation.
It compiles and runs, but the **~3B on-device model is too weak** for Cairn's
multi-tool agent flow and the **4096-token context window** is too small once the
system prompt + tool schemas are included. It could not reliably answer even a
single-project question.
- **PCC is the fix**: same unified `LanguageModelSession` API, one-line model swap
(`PrivateCloudComputeLanguageModel()`), **32K context**, stronger reasoning, no
API keys, no server to run. Requires iOS 27+, a **managed entitlement**
(`com.apple.developer.private-cloud-compute`, applied for), and a network
connection.
- The **entire tool-bridge architecture we built carries over unchanged** — tools,
instructions, streaming, and the `respond`/`streamResponse` calls are identical
across on-device and PCC models. Only the model instance and availability/quota
handling differ.

---

## What already exists in the repo (the reusable foundation)

All of this is compiled/verified against the iOS 26.5 SDK (Xcode 26.6) and can be
reused directly for PCC:

| File | Role | Reuse for PCC |
|------|------|---------------|
| `mobile/modules/apple-llm/ios/AppleLlmModule.swift` | Expo module: session cache, streaming, tool bridge, `generate`/`resetSession`/`prewarm`/`resolveToolCall`, availability | **Reuse**; swap the model instance + add quota handling |
| `mobile/modules/apple-llm/ios/AppleLlmSchemaParser.swift` | JSON Schema → `DynamicGenerationSchema` → `GenerationSchema` | **Reuse as-is** |
| `mobile/modules/apple-llm/index.ts` + `AppleLlm.types.ts` | Typed JS bridge | **Reuse**; add quota events/fields |
| `mobile/src/chat/providers/apple.ts` | `ChatProvider` — native tools, `tool-executed` events, lean schemas, persistent per-thread session, short system prompt | **Reuse**; relax the 4096 tuning for 32K |
| `mobile/src/chat/providers/types.ts` | `tool-executed` StreamEvent (provider ran the tool; agent must not re-run) | **Reuse** |
| `mobile/src/chat/providers/index.ts`, `ai-config.ts` | `ProviderPref = "rork" \| "openai" \| "apple"`, resolve/fallback, availability gating | **Reuse** |
| `mobile/src/components/AiSettingsForm.tsx` | "On-device" provider segment, availability gate | **Reuse**; relabel + quota UI |

### Key architecture decisions (already made, keep them)

1. **Native tool-calling, not prompt-injection.** Each Cairn tool's JSON Schema
becomes a native `GenerationSchema` via `DynamicGenerationSchema` (guided
generation), so the model is *constrained* to emit valid arguments. An earlier
text-parsing approach (`<tool_call>{…}</tool_call>`) was unreliable and was
removed.
2. **Tools execute in JS, not Swift.** `BridgedTool: Tool` (`Arguments =
GeneratedContent`, `Output = String`). `call()` suspends on a
`CheckedContinuation`, fires an `onToolCall` event to JS; JS runs the local
SQLite tool (`TOOL_MAP`) and calls `resolveToolCall(callId, resultJson)` to
resume the native turn. State lives in a `ToolBridge: @unchecked Sendable`
(lock-guarded continuation dict) so the tool stays `Sendable`.
3. **`tool-executed` StreamEvent.** Because Apple runs the multi-tool loop
*inside one `generate()`*, the provider emits `tool-executed` (with output) so
`agent.ts` shows the tool in the UI trail but does NOT re-run it (Rork/OpenAI
still use `tool-input-available` + the agent's own loop).
4. **Persistent session per chat thread.** Swift caches a `LanguageModelSession`
by a JS `sessionId`; `generate` sends only the newest user turn (transcript
carried natively). `resetAppleSession()` (JS) on chat-clear bumps the id.
5. **NativeModule typing gotcha.** The native module type must be
`declare class X extends NativeModule<Events>` with a **value** import of
`NativeModule` from `expo` — an interface-extends + type-only import does not
inherit `addListener`.

### Verified FoundationModels API (from the SDK `.swiftinterface`)

- `SystemLanguageModel.default.availability` → `.available` / `.unavailable(UnavailableReason)`;
`UnavailableReason`: `.deviceNotEligible`, `.appleIntelligenceNotEnabled`, `.modelNotReady`.
- `LanguageModelSession(model:tools:instructions:)`; `.streamResponse(to: String, options:)`
→ `ResponseStream<String>`; iterating yields `Snapshot` with `.content: String` (cumulative).
- `GenerationOptions(sampling:temperature:maximumResponseTokens:)`.
- `LanguageModelSession.GenerationError.exceededContextWindowSize` / `.assetsUnavailable`.
- `Tool` protocol; `DynamicGenerationSchema` (objects/arrays/enum/pattern/number bounds/anyOf),
`GenerationSchema(root:dependencies:)`, `GeneratedContent.jsonString` / `init(json:)`.

---

## Why on-device failed (Apple's own guidance)

From `managing-the-context-window` and `prompting-an-on-device-foundation-model`:

- Fixed **4096-token** window covers instructions + transcript + **tool definitions +
schemas** + output. Cairn's ~11 tools' schemas + system prompt leave too little for
reasoning.
- On-device model is small; "may get confused with long/indirect instructions",
struggles with multi-step orchestration, "split into simpler requests".

We already mitigated (persistent session, short prompt, lean schemas, native tools) —
still not enough. This is a model-capability ceiling, not a wiring bug. **PCC's 32K
window + stronger reasoning is the actual fix.**

---

## PCC re-implementation plan

### Prerequisites
- [ ] Managed entitlement `com.apple.developer.private-cloud-compute` granted
(applied for). Add it via a config plugin (mirror `plugins/withICloudContainer.js`).
- [ ] Xcode with an **iOS 27 SDK** (PCC types are iOS 27+; not present in 26.5).
- [ ] Test device on iOS 27 with Apple Intelligence enabled.

### Native changes (`AppleLlmModule.swift`)
1. **Model selection.** Add a `useServer: Bool` (or a `modelKind` string) to
`generate`/`ensureSession`. Build the session with the right model:
```swift
if useServer, #available(iOS 27.0, *) {
session = LanguageModelSession(model: PrivateCloudComputeLanguageModel(),
tools: bridgedTools, instructions: instructions)
} else {
session = LanguageModelSession(model: SystemLanguageModel.default,
tools: bridgedTools, instructions: instructions)
}
```
Everything else (streaming, tool bridge, schema parser) is unchanged — this is
Apple's whole selling point ("change a single line of code… tools and
instructions carry over").
2. **Availability.** Add a PCC availability probe using
`PrivateCloudComputeLanguageModel().availability` with the new
`.unavailable(.systemNotReady)` / `.deviceNotEligible` cases (all `@available(iOS 27)`).
3. **Reasoning level (optional, high value for Cairn).** PCC supports
`ContextOptions(reasoningLevel: .standard/.enhanced/.deep)`. Pass a level from JS
options; use `.deep` for hard planning, `.standard` default. Note: deeper reasoning
consumes more of the (32K) window and adds latency.
4. **Quota handling.** Surface `model.quotaUsage` (status: below/approaching/exceeded,
`limitIncreaseSuggestion`, reset date) and the `exceededQuota` error. Emit a new
`onQuota` event or expose a `quotaStatus()` function. Map `exceededQuota` to a new
`AppleLLMErrorCode` (e.g. `QUOTA_EXCEEDED`).
5. **Network fallback.** If a PCC request fails due to no network, retry on-device
(or surface a clear "needs a connection" error, like the existing cloud providers).

### JS/provider changes
1. **New ProviderPref value** or a sub-mode: e.g. `"apple"` stays on-device,
add `"apple-cloud"` — OR keep one `"apple"` pref with an availability-driven
auto-upgrade to PCC when on iOS 27 + entitled. Recommended: a single "Apple
Intelligence" provider that prefers PCC when available and falls back to
on-device, exposed in settings with a "Use cloud (Private Cloud Compute)" toggle.
2. **Relax the 4096 tuning for PCC.** With 32K, we can:
- Restore richer tool descriptions (the `leanSchema()` stripping is only needed
for the tiny window; keep it for on-device, relax for PCC).
- Use the full agent system prompt instead of the terse `appleSystemPrompt()`.
- Raise `maxTokens`.
Gate these on the model kind.
3. **Quota UX.** Add a small status line + "Show options" (iCloud+ upgrade) in the
chat/composer when `quotaUsage` is approaching/exceeded; use Apple's
`limitIncreaseSuggestion.show()` via a native function.
4. **Consider re-enabling the full multi-tool agent flow.** With PCC's reasoning +
32K, the model should handle the whole 11-tool set reliably. Keep the native
tool-bridge (Apple runs the loop internally, emits `tool-executed`).

### Settings UI
- Relabel the "On-device" segment; when PCC is available show a sub-toggle
"Private Cloud Compute (larger, cloud, uses daily quota)".
- Show availability reason + quota status.

### Testing (Xcode helpers)
- Xcode Scheme → Run → Options → "Simulated Apple Foundation Models Availability":
"Approaching Quota Usage Limit" / "Quota Usage Limit Reached" to test quota UI
without burning real quota.

---

## Open questions / risks
- **iOS 27 minimum for PCC** — our current on-device path is iOS 26. Decide whether
PCC is an iOS-27-only enhancement layered on the iOS-26 on-device base, or whether
we drop on-device entirely and make Apple = PCC-only (simpler, but iOS 27+ only and
online-only).
- **Daily quota** — PCC has per-user daily limits (iCloud+ upgradable). For a
power-user tool like Cairn this may be limiting; keep Rork/OpenAI as the primary
and Apple/PCC as a privacy-preserving option.
- **Entitlement gating** — the managed entitlement means third-party builds of Cairn
can't use PCC unless separately entitled; keep OpenAI-compatible as the BYO path.

## Doc-fetching note
Apple's Foundation Models docs are a JS SPA that WebFetch/defuddle can't read. Fetch
the JSON backend instead:
`https://developer.apple.com/tutorials/data/documentation/foundationmodels/<page>.json`
(e.g. `adding-server-side-intelligence-with-private-cloud-compute.json`).

## References
- Adding server-side intelligence with Private Cloud Compute
- Expanding generation with tool calling
- Managing the context window
- Prompting an on-device foundation model
- Entitlement: `com.apple.developer.private-cloud-compute`
15 changes: 15 additions & 0 deletions mobile/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,18 @@ APPLE_TEAM_ID=
# NOTE: EXPO_PUBLIC_* vars are inlined into the shipped JS bundle, so this value
# is extractable from a built app. Env only keeps it out of git/source.
EXPO_PUBLIC_TOOLKIT_URL=

# ─────────────────────────────────────────────────────────────────────────────
# Apple on-device AI (experimental, dev-only)
# ─────────────────────────────────────────────────────────────────────────────
# The Apple Foundation Models (on-device Apple Intelligence) provider is
# EXPERIMENTAL and not yet viable for Cairn's agentic tool use — the on-device
# model is too small / context too tight (see
# docs/plans/apple-foundation-models-provider.md; the plan is to redo it against
# Private Cloud Compute on iOS 27). It is therefore HIDDEN from end users.
#
# Set to 1 in your LOCAL .env.local to expose the "On-device" provider in
# Settings → AI so you can keep iterating on a device. Leave BLANK/unset for EAS
# and any shipped build — the provider then never appears or resolves. (The
# native module still compiles but stays dormant unless this flag is on.)
EXPO_PUBLIC_APPLE_LLM_DEV=
12 changes: 11 additions & 1 deletion mobile/app/(tabs)/_layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,21 @@ export default function TabsLayout() {
accent instead of the platform default blue. minimizeBehavior
collapses the tab bar as you scroll down for more content room. */}
<NativeTabs tintColor={t.accent} minimizeBehavior="onScrollDown">
<NativeTabs.Trigger name="index">
<NativeTabs.Trigger name="projects">
<NativeTabs.Trigger.Icon sf="folder.fill" md="folder" />
<NativeTabs.Trigger.Label>Projects</NativeTabs.Trigger.Label>
</NativeTabs.Trigger>

<NativeTabs.Trigger name="calendar">
<NativeTabs.Trigger.Icon sf="calendar" md="calendar_month" />
<NativeTabs.Trigger.Label>Calendar</NativeTabs.Trigger.Label>
</NativeTabs.Trigger>

<NativeTabs.Trigger name="graph">
<NativeTabs.Trigger.Icon sf="point.3.connected.trianglepath.dotted" md="hub" />
<NativeTabs.Trigger.Label>Graph</NativeTabs.Trigger.Label>
</NativeTabs.Trigger>

<NativeTabs.Trigger name="search" role="search">
<NativeTabs.Trigger.Icon sf="magnifyingglass" md="search" />
<NativeTabs.Trigger.Label>Search</NativeTabs.Trigger.Label>
Expand Down
7 changes: 7 additions & 0 deletions mobile/app/(tabs)/calendar/_layout.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { TabStack } from "@/components/TabStack";

// Regular (non-large) title so the fixed calendar grid isn't pushed off-screen
// by an oversized large-title header the way a scroll-driven list tab is.
export default function CalendarLayout() {
return <TabStack largeTitle={false} />;
}
73 changes: 73 additions & 0 deletions mobile/app/(tabs)/calendar/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import { useCallback, useState } from "react";
import { Stack, useFocusEffect, useRouter } from "expo-router";
import { useSafeAreaInsets } from "react-native-safe-area-context";
import { listCardsWithDueDates, type CalendarCard } from "@/db/queries";
import { CalendarView, type CalendarLayout } from "@/components/CalendarView";
import { TabScreen } from "@/components/TabScreen";
import { ICON_VIEW_MONTH, ICON_VIEW_WEEK } from "@/components/toolbar-icons";
import { useDataChanged } from "@/sync/useSyncStatus";

/**
* Workspace-wide Calendar: every live task with a due date across all projects,
* on a month grid with a per-day agenda. Project names are shown on each row so
* cross-project due dates are distinguishable.
*
* The primary controls — "Today" and the Month/Week switch — live in the native
* Stack.Toolbar (right) so the tab reads like the rest of the app; the grid's
* in-body toolbar keeps only prev/next + the period label. This screen owns the
* `layout` state and a `todayNonce` it bumps to jump the grid back to today.
*/
export default function CalendarScreen() {
const router = useRouter();
const insets = useSafeAreaInsets();
const [cards, setCards] = useState<CalendarCard[]>([]);
const [layout, setLayout] = useState<CalendarLayout>("month");
const [todayNonce, setTodayNonce] = useState(0);

const load = useCallback(() => setCards(listCardsWithDueDates()), []);
useFocusEffect(useCallback(() => load(), [load]));
useDataChanged(load);

return (
<TabScreen>
<Stack.Screen options={{ title: "Calendar" }} />
<Stack.Toolbar placement="right">
<Stack.Toolbar.Button
accessibilityLabel="Jump to today"
onPress={() => setTodayNonce((n) => n + 1)}
>
Today
</Stack.Toolbar.Button>
<Stack.Toolbar.Menu
icon={layout === "month" ? ICON_VIEW_MONTH : ICON_VIEW_WEEK}
accessibilityLabel="Calendar span"
>
<Stack.Toolbar.Label>{layout === "month" ? "Month" : "Week"}</Stack.Toolbar.Label>
<Stack.Toolbar.MenuAction
icon={ICON_VIEW_MONTH}
isOn={layout === "month"}
onPress={() => setLayout("month")}
>
Month
</Stack.Toolbar.MenuAction>
<Stack.Toolbar.MenuAction
icon={ICON_VIEW_WEEK}
isOn={layout === "week"}
onPress={() => setLayout("week")}
>
Week
</Stack.Toolbar.MenuAction>
</Stack.Toolbar.Menu>
</Stack.Toolbar>
<CalendarView
cards={cards}
showProject
bottomInset={insets.bottom}
layout={layout}
onLayoutChange={setLayout}
todayNonce={todayNonce}
onOpenCard={(id) => router.push({ pathname: "/card/[id]", params: { id, back: "Calendar" } })}
/>
</TabScreen>
);
}
Loading