Skip to content

fix(admin): add activation stages to onboarding funnel - #2690

Merged
riderx merged 4 commits into
mainfrom
codex/onboarding-funnel-activation-stages
Jul 15, 2026
Merged

fix(admin): add activation stages to onboarding funnel#2690
riderx merged 4 commits into
mainfrom
codex/onboarding-funnel-activation-stages

Conversation

@riderx

@riderx riderx commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary\n\n- make Production plugin device and Completed update download sequential stages of the Onboarding Funnel\n- restrict activation cohorts to apps that already reached the bundle stage in their seven-day onboarding window\n- keep subscription as a separate commercial conversion because it is not a downstream product-activation step\n\n## Root cause\n\nThe merged implementation rendered activation metrics below the funnel rather than as funnel stages.\n\n## Visual\n\nOnboarding Funnel\n\n## Validation\n\n- bun lint\n- bun run lint:backend\n- bun run typecheck:backend\n- bun run typecheck:frontend\n- bunx vitest run tests/admin-onboarding-funnel.unit.test.ts\n- bun run test:unit


Note

Medium Risk
Changes admin onboarding funnel SQL and conversion math, which can shift reported metrics; scope is admin analytics only, not auth or customer-facing flows.

Overview
Reworks the admin onboarding funnel so product activation is measured as sequential funnel stages instead of a separate “activation signals” block below the chart.

When activation telemetry is available, the funnel now includes Production plugin device and Completed update download after bundle upload, with step-to-step conversion tiles (Bundle → Production device, Production device → Update download). Subscription stays outside the product funnel as a single commercial conversion (Bundle → Subscribed).

Backend metrics now use adjacent-stage denominators (e.g. production device % of orgs with bundle, update download % of orgs with production device). Bundle eligibility is tightened via shared onboardingBundleEligibility: orgs count as having uploaded a bundle when they have an app in the 7-day window, a channel, and a non-builtin bundle uploaded in that window—not only when that bundle is the channel’s current version (fixes orgs dropping out after a later channel promotion).

Activation cohorts for telemetry use the same bundle eligibility. Copy and i18n strings were updated to match the new funnel story.

Reviewed by Cursor Bugbot for commit 8a3ab12. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features
    • Enhanced the admin onboarding funnel to present step-by-step conversion rates from bundle creation to production device and completed update download, with conditional stages and tiles when telemetry is available.
  • Bug Fixes
    • Corrected onboarding funnel eligibility and conversion calculations to use adjacent-stage denominators and improved cohort filtering for more consistent reporting.
  • Documentation
    • Updated onboarding funnel messaging text and refreshed the related funnel screenshot/asset.
  • Tests
    • Expanded admin onboarding funnel and admin stats coverage to reflect the updated funnel metrics.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 34 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 637bfdfc-230a-4298-a5ae-107c909f69d7

📥 Commits

Reviewing files that changed from the base of the PR and between e2dec0c and 8a3ab12.

📒 Files selected for processing (3)
  • messages/en.json
  • supabase/functions/_backend/utils/pg.ts
  • tests/admin-stats.test.ts
📝 Walkthrough

Walkthrough

The activation funnel now enforces app-version activation windows, calculates nested stage conversions, and conditionally displays telemetry-dependent dashboard stages. English messaging, screenshot assets, and test expectations were updated.

Changes

Activation funnel

Layer / File(s) Summary
Activation cohort metrics and rates
supabase/functions/_backend/utils/onboardingFunnel.ts, supabase/functions/_backend/utils/pg.ts
Activation cohorts now require eligible app versions within the creation window, exclude out-of-window production-device cohorts, and use stepwise conversion denominators.
Telemetry-aware dashboard funnel
src/pages/admin/dashboard/users.vue, messages/en.json, docs/pr-screenshots/admin-onboarding-activation-milestones.webp
The dashboard conditionally renders production-device and update-download stages, shows unavailable telemetry messaging, updates funnel wording, and refreshes the screenshot asset.
Nested funnel validation
tests/admin-onboarding-funnel.unit.test.ts, tests/admin-stats.test.ts
Tests validate nested stage counts, date trends, bundle-version filtering, and subscription results.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AdminDashboard
  participant getAdminOnboardingFunnel
  participant activationCohortQuery
  participant getAdminOnboardingActivationMetrics
  AdminDashboard->>getAdminOnboardingFunnel: request onboarding funnel data
  getAdminOnboardingFunnel->>activationCohortQuery: fetch time-constrained activation cohorts
  activationCohortQuery-->>getAdminOnboardingFunnel: return eligible cohorts
  getAdminOnboardingFunnel->>getAdminOnboardingActivationMetrics: calculate nested stage metrics
  getAdminOnboardingActivationMetrics-->>getAdminOnboardingFunnel: return stage counts and trends
  getAdminOnboardingFunnel-->>AdminDashboard: render telemetry-aware funnel stages
Loading

Suggested reviewers: dalanir, wcaleniewolny

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately reflects the main change to add activation stages to the onboarding funnel.
Description check ✅ Passed It includes a clear summary, visual screenshot, and validation steps, and is sufficiently complete despite lacking an explicit checklist section.

Comment @coderabbitai help to get the list of available commands.

@codspeed-hq

codspeed-hq Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will degrade performance by 26.57%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 42 untouched benchmarks
⏩ 2 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
/updates payload validation 94.3 µs 128.4 µs -26.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/onboarding-funnel-activation-stages (8a3ab12) with main (cc975eb)2

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (660ac14) during the generation of this report, so cc975eb was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@riderx
riderx marked this pull request as ready for review July 15, 2026 11:16
@riderx

riderx commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_f31d7e4f-74b1-4a6d-8813-b7dcb0b96623)

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@cursor
cursor Bot requested review from Dalanir and WcaleNieWolny July 15, 2026 11:17

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Cursor Bugbot did not complete (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Requesting human review for the admin onboarding funnel changes.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@coderabbitai coderabbitai Bot added the codex label Jul 15, 2026
@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_b79175a5-b1be-4348-81c5-08a052b82da5)

@coderabbitai coderabbitai Bot removed the codex label Jul 15, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers are already assigned for the admin onboarding funnel changes.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

@riderx

riderx commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
supabase/functions/_backend/utils/pg.ts (1)

3163-3180: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Align orgs_with_bundles with the activation cohort. supabase/functions/_backend/utils/pg.ts:3163-3180 public.transfer_app() preserves apps.created_at when moving an app to a new org, so transferred apps can be counted by orgs_with_bundles but excluded by activationCohortQuery’s a.created_at window. That skews production_device_conversion_rate by comparing different populations.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@supabase/functions/_backend/utils/pg.ts` around lines 3163 - 3180, Update the
orgs_with_bundles CTE to apply the same app-created-at activation window as
activationCohortQuery, using each app’s created_at relative to the organization
cohort dates. Ensure transferred apps with created_at outside the organization’s
window are excluded so both conversion-rate populations align.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@supabase/functions/_backend/utils/pg.ts`:
- Around line 3167-3179: The shared channels/app_versions eligibility predicates
are duplicated across orgs_with_bundles, daily_bundles, daily_subscriptions, and
activationCohortQuery. Define one reusable sql fragment for this EXISTS check
and interpolate it into each query, preserving the existing aliases, filters,
and seven-day creation window so all funnel queries use identical rules.

In `@tests/admin-stats.test.ts`:
- Around line 992-1052: Create dedicated uniquely named organization, app, and
channel data within the test instead of using ONBOARDING_ORG_ID,
ONBOARDING_APP_ID, and the shared production channel. Insert the initial bundle
and subscription state required by the onboarding funnel scenario, then update
only the test-owned channel to the later promoted version. Adjust the assertions
to reflect exclusively this test’s data so the test is isolated and safe for
concurrent execution.

---

Outside diff comments:
In `@supabase/functions/_backend/utils/pg.ts`:
- Around line 3163-3180: Update the orgs_with_bundles CTE to apply the same
app-created-at activation window as activationCohortQuery, using each app’s
created_at relative to the organization cohort dates. Ensure transferred apps
with created_at outside the organization’s window are excluded so both
conversion-rate populations align.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 36dd866d-faa4-46b0-8221-bb73a9c34e48

📥 Commits

Reviewing files that changed from the base of the PR and between 24368af and e2dec0c.

📒 Files selected for processing (2)
  • supabase/functions/_backend/utils/pg.ts
  • tests/admin-stats.test.ts
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • Cap-go/capacitor-updater (manual)

Comment thread supabase/functions/_backend/utils/pg.ts Outdated
Comment thread tests/admin-stats.test.ts Outdated
@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_96c4c40f-df8f-4a91-bb56-2422a1c2f743)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers are already assigned for the admin onboarding funnel changes.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers are already assigned for the admin onboarding funnel changes.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@riderx
riderx merged commit 332b1a4 into main Jul 15, 2026
2 of 3 checks passed
@riderx
riderx deleted the codex/onboarding-funnel-activation-stages branch July 15, 2026 15:40
@cursor

cursor Bot commented Jul 15, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_70f4cd8a-e9d2-4124-8240-ff853042490d)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: medium. Cursor Bugbot did not complete successfully (check skipped, usage limit reached), so automated review is incomplete and I am not approving. Human reviewers were already assigned; this PR is now merged.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant