Big Sky GF: Add AI Copy to big sky plans step #99127

vykes-mac · 2025-01-30T19:18:23Z

Closes #99009

Proposed Changes

Add copy Unlimited AI Website Builder edits to big sky plans grid
Copy is rendered conditionally

Why are these changes being made?

Pre-req

Change setPlansListExperiment( EXPERIMENT_NAME, variationName ); in the use-goals-first-experiment.ts file and set the variation to treatment_cumulative

wp-calypso/client/landing/stepper/declarative-flow/helpers/use-goals-first-experiment.ts

Line 35 in 4635b1d

setPlansListExperiment( EXPERIMENT_NAME, variationName );

Go to setup/onboarding goals first flow
Chose big sky on design choice screen
Continue to plans step and you should see the copy in the plans grid
Check the copy is also displayed on the checkout page
Test the copy is not shown for non goals first users

Testing Instructions

Pre-merge Checklist

Has the general commit checklist been followed? (PCYsg-hS-p2)
Have you written new tests for your changes?
Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
Have you checked for TypeScript, React or other console errors?
Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
- For UI changes, have we tested the change in various languages (for example, ES, PT, FR, or DE)? The length of text and words vary significantly between languages.
For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

github-actions · 2025-01-30T19:18:34Z

Calypso Live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-131744

Jetpack Cloud live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-131744&env=jetpack

Automattic for Agencies live (direct link)

	https://calypso.live?image=registry.a8c.com/calypso/app:build-131744&env=a8c-for-agencies

matticbot · 2025-01-30T19:25:54Z

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

App Entrypoints (~340 bytes added 📈 [gzipped])

name         parsed_size           gzip_size
entry-main       +1250 B  (+0.1%)     +340 B  (+0.1%)
entry-login      +1190 B  (+0.1%)     +333 B  (+0.1%)

Common code that is always downloaded and parsed every time the app is loaded, no matter which route is used.

Sections (~6133 bytes added 📈 [gzipped])

name                                parsed_size           gzip_size
checkout                               +14344 B  (+0.8%)    +5610 B  (+1.1%)
onboarding-flow                         +1426 B  (+0.7%)     +397 B  (+0.7%)
site-setup-wg                           +1289 B  (+0.4%)     +358 B  (+0.4%)
site-setup-flow                         +1289 B  (+0.4%)     +358 B  (+0.4%)
write-flow                              +1190 B  (+0.1%)     +335 B  (+0.1%)
update-design-flow                      +1190 B  (+0.1%)     +335 B  (+0.1%)
plugin-bundle-flow                      +1190 B  (+0.2%)     +335 B  (+0.2%)
new-hosted-site-flow-user-included      +1190 B  (+0.7%)     +335 B  (+0.7%)
new-hosted-site-flow                    +1190 B  (+0.7%)     +335 B  (+0.7%)
link-in-bio-tld-flow                    +1190 B  (+0.1%)     +335 B  (+0.1%)
import-hosted-site-flow                 +1190 B  (+0.1%)     +335 B  (+0.1%)
import-flow                             +1190 B  (+0.7%)     +335 B  (+0.7%)
hundred-year-plan                       +1190 B  (+0.6%)     +335 B  (+0.6%)
hundred-year-domain                     +1190 B  (+0.5%)     +335 B  (+0.6%)
copy-site-flow                          +1190 B  (+0.1%)     +335 B  (+0.1%)
connect-domain                          +1190 B  (+0.6%)     +335 B  (+0.7%)
build-flow                              +1190 B  (+0.1%)     +335 B  (+0.1%)
stepper-user-step                        +299 B  (+0.1%)     +103 B  (+0.1%)

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~335 bytes added 📈 [gzipped])

name                                        parsed_size           gzip_size
async-load-calypso-my-sites-checkout-modal      +1190 B  (+0.1%)     +335 B  (+0.1%)
async-load-automattic-help-center-stepper       +1190 B  (+0.1%)     +335 B  (+0.1%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

vykes-mac · 2025-01-30T19:35:44Z

@p-jackson From the way the issue is worded it sounded like these copy should be rendered conditionally, is that the case?

matticbot · 2025-01-30T19:39:10Z

This PR modifies the release build for the following Calypso Apps:

For info about this notification, see here: PCYsg-OT6-p2

blaze-dashboard
command-palette-wp-admin
happy-blocks
help-center
notifications
odyssey-stats
whats-new
wpcom-block-editor

To test WordPress.com changes, run install-plugin.sh $pluginSlug update/plan-step-copy on your sandbox.

a8ci18n · 2025-01-30T22:21:34Z

This Pull Request is now available for translation here: https://translate.wordpress.com/deliverables/17220117

Some locales (Hebrew) have been temporarily machine-translated due to translator availability. All other translations are usually ready within a few days. Untranslated and machine-translated strings will be sent for translation next Monday and are expected to be completed by the following Friday.

Thank you @vykes-mac for including a screenshot in the description! This is really helpful for our translators.

p-jackson · 2025-01-30T23:32:22Z

@vykes-mac yeah this would be conditional on being in the onboarding flow, and have the onboarding/big-sky-before-plans feature flag (or this experiment #99096)

p-jackson

Yes, checking this now, I think these new featured features do need to be conditional.

I don't think it makes sense to display them prominently in the standard non-big-sky flow like this:

I also see they're semibold in the mockup. I'm not sure what the best way to customise this sort of thing on a per-flow basis. Especially the changes to the checkout 🤔

vykes-mac · 2025-01-31T18:52:02Z

I also see they're semibold in the mockup. I'm not sure what the best way to customise this sort of thing on a per-flow basis. Especially the changes to the checkout 🤔

I think simplest thing right now is to add the same string twice, the bold one for the plan features and the not bold one for checkout

vykes-mac · 2025-01-31T21:29:23Z

packages/calypso-products/src/features-list.tsx

+			} ),
+		getDescription: () => i18n.translate( 'Build your site with our AI Website Builder.' ),
+	},
+	[ FEATURE_BIG_SKY_WEBSITE_BUILDER_CHECKOUT ]: {


The copy on the plans page should be bold, so we create a non bold one for the checkout page, maybe there is a better way to do this?

vykes-mac · 2025-01-31T21:30:15Z

client/landing/stepper/declarative-flow/helpers/use-goals-first-experiment.ts

@@ -31,5 +32,7 @@ export function useGoalsFirstExperiment(): [ boolean, boolean ] {
 	 */
 	const variationName = experimentAssignment?.variationName ?? 'control';

+	setPlansListExperiment( EXPERIMENT_NAME, variationName );


This depends on this #99133 PR as the experiment name is changed there

This will actually go in useBigSkyBeforePlans, so it depends on #99096

vykes-mac · 2025-01-31T21:32:22Z

packages/calypso-products/src/is-goals-first.ts

+
+export function isGoalsFirst(): boolean {
+	return (
+		getPlansListExperiment( 'calypso_signup_onboarding_goals_first_flow_holdout_20241220' ) ===


We are only showing the copy to the treatment_cumulative group. @p-jackson can you confirm this? Also the experiment name when this PR is merged

😮
Technique makes sense! Seems like we're basically showing the new copy if the experiment assignment happens to be be loaded. If it hasn't nothing catastrophic happens, but following the usual flow the test group assignment should have been made by now.

We are only showing the copy to the treatment_cumulative group

This is one of the goals-first experiment cohorts. But this copy change shouldn't actually be part of the goals-first experiment. It's for the big sky experiment

getPlansListExperiment

I think there's actually another explat function we can use for this: dangerouslyGetMaybeLoadedExperimentAssignment

It should reach into explat and get the user's current assignment, returning null if it hasn't been assigned.

I'll make the changes so that we're using the correct experiment. Or maybe even I'll use the feature flag for this PR so I can get it merged.

Oh I see now. getPlansListExperiment isn't a new thing we've added; it's an existing library function. If this is how previous experiments have been done, it makes sense for us to do it the same way. So I won't switch to dangerouslyGetMaybeLoadedExperimentAssignment.

p-jackson · 2025-02-02T21:43:15Z

client/my-sites/checkout/checkout-main-wrapper.tsx

@@ -80,6 +81,7 @@ export default function CheckoutMainWrapper( {
 	const translate = useTranslate();
 	const locale = useSelector( getCurrentUserLocale );
 	const selectedSiteId = useSelector( getSelectedSiteId ) ?? undefined;
+	useGoalsFirstExperiment();


We need this to ensure the experiment assignment is stored in memory and we can retrieve it to render the feature list. But I don't love that it implicitly assigns the user to a cohort if they haven't been assigned yet.

Who knows where checkout-main-wrapper is being included from, causing an assignment to occur.

I think the hook's internal flow === 'onboarding' check will be enough to save us though.

…d on plans page should be bold

p-jackson · 2025-02-02T23:08:18Z

packages/calypso-products/src/is-big-sky-onboarding.ts

+	if ( new URLSearchParams( location.search ).has( 'big-sky-checkout' ) ) {
+		return true;
+	}


@vykes-mac I added a query parameter for the checkout screen case, so that the message only appears when the user actually chooses Big Sky

p-jackson · 2025-02-02T23:09:37Z

packages/calypso-products/src/is-big-sky-onboarding.ts

+	if ( typeof window === 'undefined' ) {
+		return false;
+	}
+
+	return Boolean( window.__a8cBigSkyOnboarding );


@vykes-mac I discovered that the Global Styles code was doing a hack like this to conditionally show the text

wp-calypso/packages/calypso-products/src/is-global-styles-on-personal-enabled.ts

Line 6 in 246305e

export function isGlobalStylesOnPersonalEnabled(): boolean {

So I've done something similar 🙈

I did this approach also, similar to global styles, but I opted for the experiment name approach so I didn't have to pass a parameter to the checkout page, but that works as well.

a8ci18n · 2025-02-03T00:31:01Z

Translation for this Pull Request has now been finished.

vykes-mac self-assigned this Jan 30, 2025

vykes-mac added [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. Reviewer Can Merge PR author indicates the reviewer is free to merge/deploy if they want to own the change. labels Jan 30, 2025

vykes-mac added the [Status] String Freeze Add the [Status] String Freeze label to your PR to ensure new strings are translated before merging label Jan 30, 2025

p-jackson requested changes Jan 31, 2025

View reviewed changes

vykes-mac requested review from a team as code owners January 31, 2025 21:28

vykes-mac commented Jan 31, 2025

View reviewed changes

p-jackson reviewed Feb 2, 2025

View reviewed changes

vykes-mac and others added 6 commits February 3, 2025 11:24

add AI copy to plans grid for big sky plans

67b08fb

remove strong element

550526d

Add a separate feature to display in checkout since the copy displaye…

29c02bc

…d on plans page should be bold

show the copy only when in the goals first experiment

cbfacc9

Switch from goals-first to big sky experiment

7b98cbc

Add hacks to get AI copy to only appear when user chooses Big Sky

246305e

p-jackson force-pushed the update/plan-step-copy branch from 4635b1d to 246305e Compare February 2, 2025 23:06

p-jackson reviewed Feb 2, 2025

View reviewed changes

fix typescript

fdfc8ab

p-jackson approved these changes Feb 2, 2025

View reviewed changes

p-jackson merged commit 2c0643a into trunk Feb 2, 2025
13 checks passed

p-jackson deleted the update/plan-step-copy branch February 2, 2025 23:38

github-actions bot removed [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. [Status] String Freeze Add the [Status] String Freeze label to your PR to ensure new strings are translated before merging labels Feb 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Big Sky GF: Add AI Copy to big sky plans step #99127

Big Sky GF: Add AI Copy to big sky plans step #99127

vykes-mac commented Jan 30, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited by matticbot

Loading

matticbot commented Jan 30, 2025 •

edited

Loading

vykes-mac commented Jan 30, 2025

matticbot commented Jan 30, 2025 •

edited

Loading

a8ci18n commented Jan 30, 2025 •

edited

Loading

p-jackson commented Jan 30, 2025

p-jackson left a comment •

edited

Loading

vykes-mac commented Jan 31, 2025

vykes-mac Jan 31, 2025

vykes-mac Jan 31, 2025

p-jackson Feb 2, 2025

vykes-mac Jan 31, 2025

p-jackson Feb 2, 2025

p-jackson Feb 2, 2025

p-jackson Feb 2, 2025

p-jackson Feb 2, 2025

p-jackson Feb 2, 2025

vykes-mac Feb 3, 2025

a8ci18n commented Feb 3, 2025

Big Sky GF: Add AI Copy to big sky plans step #99127

Big Sky GF: Add AI Copy to big sky plans step #99127

Conversation

vykes-mac commented Jan 30, 2025 • edited Loading

Proposed Changes

Why are these changes being made?

Pre-req

Testing Instructions

Pre-merge Checklist

github-actions bot commented Jan 30, 2025 • edited by matticbot Loading

matticbot commented Jan 30, 2025 • edited Loading

vykes-mac commented Jan 30, 2025

matticbot commented Jan 30, 2025 • edited Loading

a8ci18n commented Jan 30, 2025 • edited Loading

p-jackson commented Jan 30, 2025

p-jackson left a comment • edited Loading

Choose a reason for hiding this comment

vykes-mac commented Jan 31, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

a8ci18n commented Feb 3, 2025

vykes-mac commented Jan 30, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited by matticbot

Loading

matticbot commented Jan 30, 2025 •

edited

Loading

matticbot commented Jan 30, 2025 •

edited

Loading

a8ci18n commented Jan 30, 2025 •

edited

Loading

p-jackson left a comment •

edited

Loading