Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update experiment slug for goals-first experience #99133

Merged
merged 2 commits into from
Feb 2, 2025

Conversation

p-jackson
Copy link
Member

@p-jackson p-jackson commented Jan 30, 2025

Related to #99011

Proposed Changes

  • Switches to the new calypso_signup_onboarding_goals_first_flow_holdout_v2_20250131 experiment slug
  • Stop forcing the goals-first experience in development

It implements the first branch in this flow chart

Screenshot 2025-01-30 at 1 34 31 PM

p1738272885652059/1738193377.949839-slack-C085HCWCEDN

Why are these changes being made?

We are adjusting weightings for the goal-first experience. It's now 10% control and 90% with the goals-first experience. That 90% is split into holdout and cumulative cohorts, but that doesn't make any difference for the useGoalsFirstExperiment() hook. Both treatment cohorts get the goals-first experience.

wpcalypso will continue to be forced onto the goals-first experience for stakeholders. But now that we have the new experiment slugs, developers should explicitly choose the cohorts they want to test with.

Testing Instructions

  • Use Explat to set the calypso_signup_onboarding_goals_first_flow_holdout_v2_20250131 experiment to your desired cohort 22180-explat-experiment
  • Clear your local storage and visit /setup/onboarding
    • control sees the domains step (or account creation) as the first step
    • treatment_frozen and treatment_cumulative see the goals screen as the first step

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
    • For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

Developers can start using the Explat interface for the
calypso_signup_onboarding_goals_first_flow_holdout_v2_20250131
experiment.
@p-jackson p-jackson requested a review from a team as a code owner January 30, 2025 22:25
@p-jackson p-jackson self-assigned this Jan 30, 2025
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Jan 30, 2025
@p-jackson p-jackson requested a review from a team January 30, 2025 22:25
@p-jackson p-jackson requested review from a team and samacart and removed request for a team January 30, 2025 22:26
@matticbot
Copy link
Contributor

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

  • notifications

To test WordPress.com changes, run install-plugin.sh $pluginSlug update/new-experiment-slug on your sandbox.

@matticbot
Copy link
Contributor

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~22 bytes added 📈 [gzipped])

name           parsed_size           gzip_size
entry-stepper        +47 B  (+0.0%)      +22 B  (+0.0%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

Copy link
Contributor

@vykes-mac vykes-mac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As described:

✅ Control group see account creation
✅ Treatment cumulative and Treatment frozen both the goals screen

Copy link

@necronet necronet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@p-jackson p-jackson merged commit 3e12267 into trunk Feb 2, 2025
22 of 23 checks passed
@p-jackson p-jackson deleted the update/new-experiment-slug branch February 2, 2025 21:03
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Feb 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants