Skip to content

Add validation-phase analytics tracking - #87

Merged
Gamius00 merged 5 commits into
mainfrom
analytics
Jul 9, 2026
Merged

Add validation-phase analytics tracking#87
Gamius00 merged 5 commits into
mainfrom
analytics

Conversation

@FleetAdmiralJakob

@FleetAdmiralJakob FleetAdmiralJakob commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds validation-phase PostHog analytics for the Expo app and wires identified student profiles through the existing Clerk/Convex identity flow.

Changes include:

  • PostHog provider setup with analytics disabled unless EXPO_PUBLIC_POSTHOG_API_KEY is set.
  • Identified-only tracking after sign-in, with autocapture, lifecycle events, anonymous pre-auth events, and session replay intentionally disabled.
  • Validation events for homework/exam creation, material upload, and successful study plan generation.
  • Clerk unsafe metadata sync for validationStudentCode into Convex so PostHog profiles can link back to the Confluence validation student table.
  • Domain/context docs for Validation Phase terminology and PostHog environment setup.
  • Small cleanup for an unused exported UI type so knip remains green.

Related Issues

Foundation for #76.

Related follow-up slices: #77, #78, #79, #80, #81, #82, #83, #84.

This PR intentionally does not close those issues; they cover the remaining execution/recovery tracking work.

Why

The current validation plan needs behavioral data tied to real validation students. This PR establishes the analytics and identity foundation without broadening product scope beyond the Validation Phase.

Validation

  • pnpm check
  • pnpm check:unused
  • git diff --check
  • wsl coderabbit doctor
  • wsl coderabbit review --agent --type uncommitted
  • wsl coderabbit review --agent --type committed --base origin/main

CodeRabbit completed with 0 findings for both committed and uncommitted review scopes.

Notes

validationStudentCode is synced from Clerk unsafe metadata as a temporary analytics join key. It is not used for authorization or permissions.

Summary by CodeRabbit

  • New Features

    • Added analytics tracking for key validation flows, including plan generation, material uploads, and entry creation.
    • Improved identity-based analytics so events are tied to signed-in users.
    • Added support for an optional student validation code in user profiles.
  • Bug Fixes

    • Fixed app configuration formatting so the app loads settings correctly.
  • Documentation

    • Expanded environment variable and integration guidance for analytics and validation terminology.

@coderabbitai

coderabbitai Bot commented May 30, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR integrates PostHog analytics into the app: adds an analytics library and configuration, wraps the app with PostHogProvider and a new AnalyticsIdentity component that identifies/resets PostHog based on Convex/Clerk auth state, and adds capture events in entry, learning-plan-generation, and material-upload flows. It also introduces a validationStudentCode field propagated through Convex schema/mutations and Clerk metadata, plus documentation, dependency, and minor config/cleanup updates.

Changes

PostHog Analytics Integration

Layer / File(s) Summary
Analytics library and config
src/lib/analytics.ts, .env.example, docs/contexts/platform/CONTEXT.md, docs/contexts/integrations/CONTEXT.md
Adds PostHog config constants, event/property types, definedAnalyticsProperties, useValidationAnalytics hook, and documents PostHog env vars and integration terminology.
PostHogProvider and AnalyticsIdentity wiring
src/app/_layout.tsx, src/components/analytics-identity.tsx
Wraps app providers with PostHogProvider and adds AnalyticsIdentity, which identifies or resets PostHog based on Convex auth and user data.
Capture events in feature screens
src/app/entry/new.tsx, src/app/learning-plans/[planId]/generating.tsx, src/app/learning-plans/new.tsx
Adds capture calls for entry creation, plan generation, and material upload events via useValidationAnalytics.
validationStudentCode field propagation
convex/schema.ts, convex/users.ts, src/context/AuthContext.tsx
Adds an optional validationStudentCode field to the users schema, profileFields/syncCurrentUser args, and AuthUser/sync payload.
Supporting config, docs, dependencies, cleanup
app.json, package.json, docs/contexts/product/CONTEXT.md, src/components/ui/surface.tsx
Fixes plugin array syntax, adds Expo/PostHog dependencies, updates product glossary docs, and removes the SurfaceVariant type export.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Screen
  participant useValidationAnalytics
  participant PostHog

  User->>Screen: complete action (upload/create/generate)
  Screen->>useValidationAnalytics: capture(eventName, properties)
  useValidationAnalytics->>PostHog: identify(clerkId) if needed
  useValidationAnalytics->>PostHog: capture(eventName, definedAnalyticsProperties)
Loading
sequenceDiagram
  participant RootLayout
  participant AnalyticsIdentity
  participant ConvexAuth
  participant PostHog

  RootLayout->>AnalyticsIdentity: mount with auth/onboarding context
  AnalyticsIdentity->>ConvexAuth: check auth state, query getMe
  ConvexAuth-->>AnalyticsIdentity: user or null
  alt user authenticated
    AnalyticsIdentity->>PostHog: identify(clerkId, properties)
  else no user
    AnalyticsIdentity->>PostHog: reset()
  end
Loading

Possibly related issues

Possibly related PRs

  • Dayova/dayova-mvp#135: Both PRs modify src/app/learning-plans/new.tsx's upload flow, with this PR adding a material_uploaded capture tied to the upload completion logic changed there.

Suggested labels: ready-for-agent

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding analytics tracking for the validation phase.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch analytics
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch analytics

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Collaborator Author

Superseded by #144, which brings the validation analytics foundation onto the current codebase and completes the broader analytics execution/founder-observability work for #76-#84.

Keeping this PR only as historical context while #144 is under review; it should not be merged. Once #144 lands, this PR can be closed as superseded.

@Gamius00
Gamius00 marked this pull request as ready for review July 8, 2026 23:23

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/contexts/integrations/CONTEXT.md`:
- Around line 23-26: The note currently reads like the Validation Student Code
already lives only in Dayova data, but the current sync path still pulls
validationStudentCode from Clerk unsafeMetadata and copies it into Convex.
Reword the PostHog/Validation Phase note in CONTEXT.md to clearly distinguish
the target state from the present implementation, or add a brief migration
caveat that mentions the Clerk unsafeMetadata to Convex boundary so readers
understand what is current versus planned.

In `@src/components/analytics-identity.tsx`:
- Around line 35-48: Limit the payload passed to posthog.identify in
analytics-identity.tsx to the stable properties that are actually documented,
keeping clerkId and validationStudentCode in definedAnalyticsProperties and
removing the extra person fields unless there is a documented need for them. If
any additional properties must remain, update the surrounding analytics-identify
logic or integrations guidance to explicitly justify each one and explain how it
is handled.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 70ff7b18-c055-4e50-91c2-d78c59d62441

📥 Commits

Reviewing files that changed from the base of the PR and between b8571fa and 95ead04.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (16)
  • .env.example
  • app.json
  • convex/schema.ts
  • convex/users.ts
  • docs/contexts/integrations/CONTEXT.md
  • docs/contexts/platform/CONTEXT.md
  • docs/contexts/product/CONTEXT.md
  • package.json
  • src/app/_layout.tsx
  • src/app/entry/new.tsx
  • src/app/learning-plans/[planId]/generating.tsx
  • src/app/learning-plans/new.tsx
  • src/components/analytics-identity.tsx
  • src/components/ui/surface.tsx
  • src/context/AuthContext.tsx
  • src/lib/analytics.ts
💤 Files with no reviewable changes (1)
  • src/components/ui/surface.tsx

Comment on lines 23 to +26
## Notes

- PostHog identifies validation-phase students with Clerk's stable user ID. The Validation Student Code should live in Dayova data, not Clerk auth metadata, and should be sent to PostHog as a profile property when present.
- PostHog tracking is identified-only during the Validation Phase. Anonymous pre-auth app activity, autocapture, lifecycle events, and session replay are intentionally out of scope unless login or onboarding becomes a measured blocker.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify how validationStudentCode is sourced and synced.
rg -nC3 'validationStudentCode' src/context/AuthContext.tsx convex/users.ts convex/schema.ts

Repository: Dayova/dayova-mvp

Length of output: 3207


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '1,60p' docs/contexts/integrations/CONTEXT.md | cat -n

Repository: Dayova/dayova-mvp

Length of output: 2218


🏁 Script executed:

#!/bin/bash
set -euo pipefail
sed -n '330,410p' src/context/AuthContext.tsx | cat -n

Repository: Dayova/dayova-mvp

Length of output: 3407


Clarify whether this is the target state or the current sync path. validationStudentCode is still read from Clerk unsafeMetadata and copied into Convex today, so this note reads like a completed state. Add a short migration caveat or reword it to describe the current boundary.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/contexts/integrations/CONTEXT.md` around lines 23 - 26, The note
currently reads like the Validation Student Code already lives only in Dayova
data, but the current sync path still pulls validationStudentCode from Clerk
unsafeMetadata and copies it into Convex. Reword the PostHog/Validation Phase
note in CONTEXT.md to clearly distinguish the target state from the present
implementation, or add a brief migration caveat that mentions the Clerk
unsafeMetadata to Convex boundary so readers understand what is current versus
planned.

Comment on lines +35 to +48
posthog.identify(
user.clerkId,
definedAnalyticsProperties({
clerk_id: user.clerkId,
convex_user_id: convexUser?._id,
email: user.email,
name: user.name,
birth_date: user.birthDate,
grade: user.grade,
school_type: user.schoolType,
state: user.state,
avatar_url: user.avatarUrl,
validation_student_code: convexUser?.validationStudentCode,
}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check if consent or data processing agreements are documented anywhere.
rg -niC2 'consent|gdpr|ccpa|data.process|privacy.policy' docs/ .env.example README.md --type=md || echo "No consent/privacy docs found"

Repository: Dayova/dayova-mvp

Length of output: 185


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== Files ==\n'
git ls-files | rg 'src/components/analytics-identity\.tsx|integrations|privacy|consent|posthog|analytics' || true

printf '\n== analytics-identity.tsx ==\n'
sed -n '1,220p' src/components/analytics-identity.tsx

printf '\n== docs mentions ==\n'
rg -n -C 3 'posthog|clerk_id|validation_student_code|identify|analytics|person properties|email|birth_date|grade|school_type|state|avatar_url' . --glob '!**/node_modules/**' --glob '!**/.git/**' || true

Repository: Dayova/dayova-mvp

Length of output: 50373


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== File list for docs ==\n'
git ls-files | rg '(^|/)(README|docs|integrations|privacy|policy).*\.(md|txt)$|(^|/)README\.md$' || true

printf '\n== Candidate docs content ==\n'
for f in $(git ls-files | rg '(^|/)(README|docs|integrations|privacy|policy).*\.(md|txt)$|(^|/)README\.md$'); do
  echo "--- $f ---"
  sed -n '1,220p' "$f"
done

Repository: Dayova/dayova-mvp

Length of output: 23292


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path('src/components/analytics-identity.tsx')
print(p.exists(), p)
if p.exists():
    text = p.read_text()
    for needle in ['posthog.identify', 'definedAnalyticsProperties', 'clerk_id', 'validation_student_code']:
        print(f'\n## {needle}')
        idx = text.find(needle)
        if idx != -1:
            start = text.rfind('\n', 0, idx-200)
            end = text.find('\n', idx+400)
            print(text[max(0, idx-200):idx+500])
PY

Repository: Dayova/dayova-mvp

Length of output: 2778


Limit the PostHog identify payload
src/components/analytics-identity.tsx:35 sends email, name, birth_date, grade, school_type, state, and avatar_url, but the integrations doc only calls out clerk_id and validation_student_code. Trim this to the stable identifiers, or add explicit guidance for why each extra person property is needed and how it’s handled.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/components/analytics-identity.tsx` around lines 35 - 48, Limit the
payload passed to posthog.identify in analytics-identity.tsx to the stable
properties that are actually documented, keeping clerkId and
validationStudentCode in definedAnalyticsProperties and removing the extra
person fields unless there is a documented need for them. If any additional
properties must remain, update the surrounding analytics-identify logic or
integrations guidance to explicitly justify each one and explain how it is
handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants