feat(onboarding): track AI and CLI instruction copies

docs/superpowers/specs/2026-08-13-onboarding-copy-analytics-design.md

@@ -0,0 +1,149 @@
+# Onboarding Copy Analytics Design
+
+## Goal
+
+Simplify AI-assisted onboarding by always copying instructions that contain the
+real Capgo API key, and measure the two sensitive copy actions without exposing
+the key, copied command, or AI prompt to analytics.
+
+The AI-instructions copy must reach both PostHog and Bento. The CLI-command copy
+must reach PostHog only. Both PostHog events must remain associated with the
+active browser session.
+
+## User Experience
+
+Remove the API-key choice dialog from `AppOnboardingFlow.vue`. Clicking **Copy
+AI instructions** first ensures an API key exists and then immediately copies
+the AI onboarding prompt with the real-key CLI command.
+
+If API-key provisioning fails, do not copy a redacted fallback. Keep the
+existing error toast and stop. If clipboard writing fails, keep the existing
+manual-copy dialog and do not emit a successful-copy analytics event.
+
+The existing CLI command remains clickable when its API key is ready. This is
+the onboarding action described as copying the API key because the copied CLI
+command contains that key; there is no separate API-key-only button in this
+component.
+
+## Event Contract
+
+Add two canonical events:
+
+- `onboarding_ai_instructions_copied`
+- `onboarding_cli_command_copied`
+
+Both frontend PostHog captures include safe, structured onboarding context:
+
+- `onboarding_attempt_id`: the existing ID generated by the onboarding progress
+  tracker for this run through the flow;
+- `onboarding_version`: the existing onboarding analytics version;
+- `flow`: `pre_org` or `existing_org`;
+- `app_id` and `org_id` when available;
+- `intent` when selected;
+- `existing_app` when known;
+- `resumed`: whether the flow resumed a pending app;
+- `setup_command`: `builder` or `ota`.
+
+Do not include the API key, CLI command, prompt, app name, email address, store
+URL, or other free-form user input.
+
+Events fire only after `navigator.clipboard.writeText` succeeds. Tracking is
+best-effort and must never make a successful copy look unsuccessful.
+
+## PostHog Session Preservation
+
+Capture both PostHog events directly in the browser with the existing
+`pushEvent` service. This preserves PostHog's native browser session and normal
+client context. Reuse the `onboarding_attempt_id` already generated by
+`createOnboardingProgressTracker` as the explicit onboarding-flow correlation
+property, and include it in the Bento-forwarding request.
+
+Do not rely on the server-side PostHog capture for the AI-copy event because it
+would create a second event and would not naturally carry the browser SDK's
+session context.
+
+## Bento Forwarding
+
+After the browser captures `onboarding_ai_instructions_copied`, send the same
+event name and safe metadata through the existing authenticated
+`/private/events` frontend helper.
+
+The backend recognizes this exact event as an allowlisted frontend-captured
+event. It builds a server-owned Bento payload and sends it through the existing
+Bento tracking path, while disabling the backend PostHog provider for this
+request. LogSnag may continue to receive the backend event.
+
+Implement backend PostHog suppression as an internal option on the shared
+tracking dispatcher. The client cannot select arbitrary providers: the
+`/private/events` route applies the option only after matching the exact
+allowlisted event name. All other event behavior remains unchanged.
+
+The Bento event is `app:ai_instructions_copied`, uses the onboarding email
+preference, and includes the verified organization, app, onboarding attempt ID,
+and safe onboarding context. The backend verifies the caller's
+organization/app access using the existing tracking-v2 flow before constructing
+the Bento payload.
+
+Use a per-session uniqueness key so repeated retries or double delivery do not
+duplicate the Bento signal for the same onboarding session and app.
+
+`onboarding_cli_command_copied` never calls `/private/events`; it remains
+PostHog-only.
+
+## Implementation Shape
+
+Change `copyText` to return whether clipboard writing succeeded. On success:
+
+1. `copyCliCommand` emits `onboarding_cli_command_copied` directly to PostHog.
+2. `copyAiInstructions` emits `onboarding_ai_instructions_copied` directly to
+   PostHog, then sends the allowlisted backend event asynchronously for Bento.
+
+Remove the redacted CLI-command computed value, the conditional API-key prompt
+guidance, and the dialog-specific translations and tests. Keep the prompt's
+with-key safety guidance, rewritten as unconditional copy.
+
+Centralize the safe event properties in one helper inside the onboarding
+component or its existing analytics helper so both copy actions use identical
+session and flow context.
+
+## Failure And Security Boundaries
+
+- API-key provisioning failure stops AI-instruction copying and tracking.
+- Clipboard failure emits neither successful-copy event.
+- PostHog or Bento tracking failure never blocks or reverses a successful copy.
+- The Bento request is fire-and-forget from the UI.
+- The backend derives organization authorization and Bento routing; the browser
+  cannot provide an arbitrary Bento event or disable providers for other events.
+- Analytics payloads must never serialize `apiKey`, `cliCommand`, or the prompt.
+
+## Testing
+
+Frontend tests verify:
+
+- AI copying no longer opens the API-key choice dialog;
+- the real-key command is always used after successful key loading;
+- key-loading and clipboard failures emit no success events;
+- both copy actions use the same safe session/context builder;
+- the AI event calls browser PostHog and `/private/events` once;
+- the CLI-command event calls browser PostHog only;
+- no analytics payload contains the key, command, or prompt.
+
+Backend tests verify:
+
+- the allowlisted AI-copy event builds the expected Bento payload;
+- backend PostHog is skipped only for that exact event;
+- the event still uses tracking-v2 organization/app authorization;
+- arbitrary events cannot request provider suppression or Bento forwarding;
+- the session uniqueness key produces one Bento signal per app/session.
+
+Run frontend and backend lint, focused unit tests, TypeScript type checking, and
+the production frontend build. Keep the production-mode frontend server alive
+for browser verification throughout implementation.
+
+## Scope
+
+This change removes the copy-choice dialog, always includes the API key in AI
+instructions, adds the two copy events, and adds the minimum backend allowlist
+needed for Bento delivery without duplicate PostHog capture. It does not change
+API-key generation, other API-key pages, onboarding navigation, database schema,
+or unrelated tracking events.

messages/en.context.json

@@ -388,13 +388,8 @@
   "app-not-found-description": "Used in Capgo web console areas: components, components/dashboard, pages/app. Role: helper or description text. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-button": "Used in Capgo web console areas: components/dashboard. Role: button or action label. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-caption": "Used in Capgo web console areas: components/dashboard. Role: UI sentence. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
-  "app-onboarding-ai-help-copy-description": "Used in Capgo web console areas: components/dashboard. Role: dialog explanatory text. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
-  "app-onboarding-ai-help-copy-title": "Used in Capgo web console areas: components/dashboard. Role: dialog title. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
-  "app-onboarding-ai-help-copy-with-key": "Used in Capgo web console areas: components/dashboard. Role: button or action label. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
-  "app-onboarding-ai-help-copy-without-key": "Used in Capgo web console areas: components/dashboard. Role: button or action label. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-prompt": "Used in Capgo web console areas: components/dashboard. Role: UI sentence. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-with-key": "Used in Capgo web console areas: components/dashboard. Role: AI prompt instruction confirming the user intentionally included their API key. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
-  "app-onboarding-ai-help-without-key": "Used in Capgo web console areas: components/dashboard. Role: AI prompt instruction for safely replacing a redacted API key. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-status-existing": "Used in Capgo web console areas: components/dashboard. Role: UI label. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-status-new": "Used in Capgo web console areas: components/dashboard. Role: UI label. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",
   "app-onboarding-ai-help-title": "Used in Capgo web console areas: components/dashboard. Role: section or dialog title. Translate for UI; keep Capgo product names, code, and placeholders unchanged.",

messages/en.json

@@ -379,17 +379,12 @@
   "app-onboarding-appid-taken-pick-another": "App ID {appId} is already used. Pick another one or use one of the suggestions.",
   "app-onboarding-appid-taken-switched": "App ID {original} was already taken, so Capgo switched to {replacement}.",
   "app-onboarding-ai-help-button": "Copy AI instructions",
-  "app-onboarding-ai-help-caption": "Choose whether to include your API key before copying instructions for ChatGPT, Claude, Cursor, or another assistant.",
-  "app-onboarding-ai-help-copy-description": "Including it gives your assistant a ready-to-run command. Only include it with an assistant you trust.",
-  "app-onboarding-ai-help-copy-title": "Include your API key?",
-  "app-onboarding-ai-help-copy-with-key": "Copy with API key",
-  "app-onboarding-ai-help-copy-without-key": "Copy without API key",
+  "app-onboarding-ai-help-caption": "Let your AI assistant guide you through setting up Capgo. Copy the onboarding instructions to get started.",
   "app-onboarding-ai-help-prompt": "I am setting up Capgo for my mobile app and I need step-by-step help with the CLI install.\n\nCapgo context:\n- App name: {appName}\n- App ID: {appId}\n- App status: {appStatus}\n- The Capgo app is already created in the dashboard.\n- Command:\n{command}\n\nPlease help me step by step.\n1. Tell me exactly where I should run this command.\n2. Ask me one question at a time if you need to know my stack (Capacitor, Ionic, React Native, native iOS/Android, etc.).\n3. Help me verify the installation succeeded.\n4. {apiKeyGuidance}",
   "app-onboarding-ai-help-status-existing": "Existing published app",
   "app-onboarding-ai-help-status-new": "New app not published yet",
   "app-onboarding-ai-help-title": "Need help from an AI assistant?",
   "app-onboarding-ai-help-with-key": "I explicitly consented to include my Capgo API key in this message so you can use the command as provided. Continue without warning me about the pasted key, and do not repeat the API key in your response.",
-  "app-onboarding-ai-help-without-key": "The command contains [YOUR_CAPGO_API_KEY]. Tell me where to replace it locally, and do not ask me to paste my real Capgo API key into this chat.",
   "app-onboarding-badge": "Create your app",
   "app-onboarding-choice-title": "Choose the best next step",
   "app-onboarding-choice-subtitle": "Install Capgo in the real project now, or add disposable demo data first and explore the dashboard.",

src/components/dashboard/AppOnboardingFlow.vue

@@ -1,6 +1,7 @@
 <script setup lang="ts">
 import type { Database } from '~/types/supabase.types'
 import type {
+  OnboardingCopyEvent,
   OnboardingDetailsEvent,
   OnboardingDetailsEventProperties,
   OnboardingIntent,
@@ -31,6 +32,7 @@ import { getCapgoApiErrorCode, invokeCapgoApi } from '~/services/capgoApi'
 import { pushEvent } from '~/services/posthog'
 import { createSignedImageUrl, getImmediateImageUrl } from '~/services/storage'
 import { getLocalConfig, isLocal, useSupabase } from '~/services/supabase'
+import { sendEvent } from '~/services/tracking'
 import { useDashboardAppsStore } from '~/stores/dashboardApps'
 import { useDialogV2Store } from '~/stores/dialogv2'
 import { useMainStore } from '~/stores/main'
@@ -139,12 +141,6 @@ const cliCommand = computed(() => {
 
   return `npx @capgo/cli@latest i ${key}${localCommand}`
 })
-const redactedCliCommand = computed(() => {
-  if (usesBuilderSetupCommand.value)
-    return `npx @capgo/cli@latest build init -a [YOUR_CAPGO_API_KEY]${localCommand}`
-
-  return `npx @capgo/cli@latest i [YOUR_CAPGO_API_KEY]${localCommand}`
-})
 const cliCommandArgs = computed(() => {
   const args: string[] = []
 
@@ -194,10 +190,9 @@ const suggestedAppId = computed(() => {
   return `com.${orgSlug}.${appSlug}`
 })
 const generatedAppId = computed(() => createdApp.value?.app_id || manualAppId.value.trim() || suggestedAppId.value)
-function createAiHelpPrompt(command: string) {
+function createAiHelpPrompt() {
   const resolvedAppId = createdApp.value?.app_id || generatedAppId.value || '[APP_ID]'
   const resolvedAppName = createdApp.value?.name?.trim() || appName.value.trim() || resolvedAppId
-  const apiKeyGuidance = t(command.includes('[YOUR_CAPGO_API_KEY]') ? 'app-onboarding-ai-help-without-key' : 'app-onboarding-ai-help-with-key')
   let appStatus = t('app-onboarding-ai-help-status-new')
   if (props.preOrg)
     appStatus = t('app-onboarding-v2-ai-help-status')
@@ -208,8 +203,8 @@ function createAiHelpPrompt(command: string) {
     appName: resolvedAppName,
     appId: resolvedAppId,
     appStatus,
-    apiKeyGuidance,
-    command,
+    apiKeyGuidance: t('app-onboarding-ai-help-with-key'),
+    command: cliCommand.value,
   })
 }
 const appOnboardingSteps = computed<Array<{ id: OnboardingFlowStep, label: string }>>(() => {
@@ -997,6 +992,7 @@ async function copyText(text: string) {
   try {
     await navigator.clipboard.writeText(text)
     toast.success(t('copied-to-clipboard'))
+    return true
   }
   catch (error) {
     console.error('Failed to copy text', error)
@@ -1011,14 +1007,43 @@ async function copyText(text: string) {
       ],
     })
     await dialogStore.onDialogDismiss()
+    return false
   }
 }
 
+function trackSuccessfulCopy(event: OnboardingCopyEvent) {
+  const orgId = currentOrg.value?.gid
+  const appId = createdApp.value?.app_id || generatedAppId.value || undefined
+  const properties = progressTracker?.trackCopyEvent(event, {
+    ...(appId ? { app_id: appId } : {}),
+    ...(existingApp.value !== null ? { existing_app: existingApp.value } : {}),
+    ...(selectedIntent.value ? { intent: selectedIntent.value } : {}),
+    ...(orgId ? { org_id: orgId } : {}),
+    setup_command: usesBuilderSetupCommand.value ? 'builder' : 'ota',
+  })
+
+  if (event !== 'onboarding_ai_instructions_copied' || !properties || !orgId || !appId)
+    return
+
+  void sendEvent({
+    channel: 'onboarding',
+    event,
+    icon: '🤖',
+    nonPersonTags: properties,
+    notify: false,
+    org_id: orgId,
+    tags: { app_id: appId },
+    tracking_version: 2,
+  }).catch(() => {})
+}
+
 async function copyCliCommand() {
   if (!apiKey.value)
     return
 
-  await copyText(cliCommand.value)
+  const copied = await copyText(cliCommand.value)
+  if (copied)
+    trackSuccessfulCopy('onboarding_cli_command_copied')
 }
 
 async function copyAiInstructions() {
@@ -1028,26 +1053,17 @@ async function copyAiInstructions() {
   catch (error) {
     console.error('Cannot ensure API key', error)
     toast.error(t('app-onboarding-toast-apikey-error'))
+    return
   }
 
-  dialogStore.openDialog({
-    id: 'app-onboarding-ai-help-copy-dialog',
-    title: t('app-onboarding-ai-help-copy-title'),
-    description: t('app-onboarding-ai-help-copy-description'),
-    buttons: [
-      {
-        text: t('app-onboarding-ai-help-copy-without-key'),
-        role: 'secondary',
-        handler: () => copyText(createAiHelpPrompt(redactedCliCommand.value)),
-      },
-      {
-        text: t('app-onboarding-ai-help-copy-with-key'),
-        role: 'primary',
-        disabled: !apiKey.value,
-        handler: () => copyText(createAiHelpPrompt(cliCommand.value)),
-      },
-    ],
-  })
+  if (!apiKey.value) {
+    toast.error(t('app-onboarding-toast-apikey-error'))
+    return
+  }
+
+  const copied = await copyText(createAiHelpPrompt())
+  if (copied)
+    trackSuccessfulCopy('onboarding_ai_instructions_copied')
 }
 
 function goToInstallStep() {

src/services/tracking.ts

@@ -47,6 +47,10 @@ interface TrackOptions {
    * example: { username: "mattie" }
    */
   tags?: Tags
+  /**
+   * Per-event metadata that must not become PostHog person properties.
+   */
+  nonPersonTags?: Tags
   /**
    * Send push notification
    */