Implement annotations for Gutenberg #10265

atimmer · 2018-07-05T09:15:51Z

Summary

This PR can be summarized in the following changelog entry:

Add support for the 'eye' markers in Gutenberg using the experimental annotations API in Gutenberg. They will work for the paragraph, quote, heading and list blocks.

Relevant technical choices:

Annotations are available under the wp-annotations handle so I added that as a dependency. To make sure our scripts still load in older versions of Gutenberg I added a no-op script in our WP dependency registration.
The text annotations get a annotation-text-[source] class, so I've added a annotation-text-yoast style to get our desired mark color.
The eye markers will only be present if the API is present. So in Gutenberg <4.2 the eye markers will simply be hidden. In Gutenberg >4.3 the eye markers will be present and useable.
Annotations work for the following blocks: core/quote, core/paragraph, core/list and core/heading.
I've implemented adding the annotations with window.requestIdleCallback, this is to prevent freezing the browser while we are adding annotations. This can mean that it takes a bit longer before all annotations are rendered. This is expected behavior. It should never take longer than 1 second per annotation.
The calculation for annotations is done by taking the Mark objects and calculating where the the offsets within the block are.
I use richText to convert the HTML to be able to purely match the text inside block attributes. This breaks down if there is HTML in our marks, so I strip the marks of HTML. This can be made more sophisticated, but in my testing this was not necessary.

Test instructions

This PR can be tested by following these steps:

Gutenberg <4.2

When you open the Gutenberg editor the eye marks should stay hidden.
The content analysis should work as expected.

Gutenberg >4.3

This is not yet released and master doesn't have the relevant branch yet until this PR is merged: WordPress/gutenberg#7718. So for now, you need to check out that Gutenberg branch to be able to test this.

Open your editor.
Type a piece of content that has transition words.
See the 'eye' button with the transition word assessment.
Click this button.
You should see the sentence with the transition word be highlighted.
Obviously the editor shouldn't crash.

It should look something like this (I have updated the colors to match the classic editor since then):

Quality assurance

I have tested this code to the best of my abilities
I have added unittests to verify the code works as intended

Fixes #11444

; Conflicts: ; js/dist/analysis-80-RC1.js ; js/dist/analysis-80-RC1.min.js ; js/dist/commons-80-RC1.js ; js/dist/commons-80-RC1.min.js ; js/dist/configuration-wizard-80-RC1.js ; js/dist/configuration-wizard-80-RC1.min.js ; js/dist/search-appearance-80-RC1.js ; js/dist/search-appearance-80-RC1.min.js ; js/dist/wp-seo-admin-80-RC1.js ; js/dist/wp-seo-admin-80-RC1.min.js ; js/dist/wp-seo-admin-global-80-RC1.js ; js/dist/wp-seo-admin-global-80-RC1.min.js ; js/dist/wp-seo-admin-gsc-80-RC1.js ; js/dist/wp-seo-admin-gsc-80-RC1.min.js ; js/dist/wp-seo-admin-media-80-RC1.js ; js/dist/wp-seo-admin-media-80-RC1.min.js ; js/dist/wp-seo-api-80-RC1.js ; js/dist/wp-seo-api-80-RC1.min.js ; js/dist/wp-seo-bulk-editor-80-RC1.js ; js/dist/wp-seo-bulk-editor-80-RC1.min.js ; js/dist/wp-seo-dashboard-widget-80-RC1.js ; js/dist/wp-seo-dashboard-widget-80-RC1.min.js ; js/dist/wp-seo-edit-page-80-RC1.js ; js/dist/wp-seo-edit-page-80-RC1.min.js ; js/dist/wp-seo-featured-image-80-RC1.js ; js/dist/wp-seo-featured-image-80-RC1.min.js ; js/dist/wp-seo-filter-explanation-80-RC1.js ; js/dist/wp-seo-filter-explanation-80-RC1.min.js ; js/dist/wp-seo-help-center-80-RC1.js ; js/dist/wp-seo-help-center-80-RC1.min.js ; js/dist/wp-seo-metabox-80-RC1.js ; js/dist/wp-seo-metabox-80-RC1.min.js ; js/dist/wp-seo-metabox-category-80-RC1.js ; js/dist/wp-seo-metabox-category-80-RC1.min.js ; js/dist/wp-seo-modal-80-RC1.js ; js/dist/wp-seo-modal-80-RC1.min.js ; js/dist/wp-seo-network-admin-80-RC1.js ; js/dist/wp-seo-network-admin-80-RC1.min.js ; js/dist/wp-seo-post-scraper-80-RC1.js ; js/dist/wp-seo-post-scraper-80-RC1.min.js ; js/dist/wp-seo-quick-edit-handler-80-RC1.js ; js/dist/wp-seo-quick-edit-handler-80-RC1.min.js ; js/dist/wp-seo-recalculate-80-RC1.js ; js/dist/wp-seo-recalculate-80-RC1.min.js ; js/dist/wp-seo-reindex-links-80-RC1.js ; js/dist/wp-seo-reindex-links-80-RC1.min.js ; js/dist/wp-seo-replacevar-plugin-80-RC1.js ; js/dist/wp-seo-replacevar-plugin-80-RC1.min.js ; js/dist/wp-seo-shortcode-plugin-80-RC1.js ; js/dist/wp-seo-shortcode-plugin-80-RC1.min.js ; js/dist/wp-seo-structured-data-blocks-80-RC1.js ; js/dist/wp-seo-structured-data-blocks-80-RC1.min.js ; js/dist/wp-seo-term-scraper-80-RC1.js ; js/dist/wp-seo-term-scraper-80-RC1.min.js ; js/dist/wp-seo-wp-globals-backport-80-RC1.js ; js/dist/wp-seo-wp-globals-backport-80-RC1.min.js ; js/src/edit.js ; js/src/wp-seo-post-scraper.js

designsimply · 2018-11-09T19:27:47Z

@atimmer I checked https://github.com/Yoast/wordpress-seo/releases/tag/9.2-RC1 and I don't see any mention of the eye markers. Does this mean it's not yet ready for testing in 9.2-RC1 but may be included in an 9.2-RC2 or in the final 9.2 release?

IreneStr · 2018-11-12T08:10:40Z

@designsimply We're planning on including this PR in at least one RC before the final 9.2 release. The first RC including it will either be RC2 or RC3. You can find the changelog for each RC on our releases page.

The error was "Cannot read property 'getBlocks' of undefined". We need to actually check if the `core/editor` store is registered before checking if a certain method is available on that object.

Missing () made the code always try the Gutenberg method for marking.

afercia · 2018-11-12T09:55:33Z

Worth noting in the Classic Editor I'm getting 2 JS errors:

Uncaught TypeError: Cannot read property 'getBlocks' of undefined
    at isAnnotationAvailable (gutenberg.js?0011:90)

Uncaught ReferenceError: YoastSEO is not defined
    at hasPresenter (getIndicatorForScore.js?bad3:14)

atimmer · 2018-11-12T10:02:01Z

@afercia Should be fixed with
d0eb99d and 8259c5c.

IreneStr

CR 👍 Only some documentation questions/requests. All other issues I found were already fixed in the previous 3 commits.

IreneStr · 2018-11-12T10:30:14Z

js/src/decorator/gutenberg.js

+	const startOffset = marked.indexOf( startMark );
+	let endOffset = marked.indexOf( endMark );
+
+	endOffset = endOffset - startMark.length;


You are using endOffset in two different 'meanings'* here , which is a little bit confusing.

* offset from beginning of the text, and offset from the beginning of the mark

Could be more clear with:

const endOffset = marked.indexOf( endMark ) - startMark.length;

Or:

Suggested change

endOffset = endOffset - startMark.length;

const matchedEndMark = marked.indexOf( endMark );

const endOffset = matchedEndMark - startMark.length;

IreneStr · 2018-11-12T10:30:26Z

js/src/decorator/gutenberg.js

+}
+
+/**
+ * Calculates an annotation if the given mark is applicable tot he content of a block.


typo: to the

IreneStr · 2018-11-12T10:32:43Z

js/src/decorator/gutenberg.js

+	 * So we try to find a mark index based on the mark content with all tags stripped.
+	 */
+	if ( foundIndex === -1 ) {
+		original = mark.getOriginal().replace( /(<([^>]+)>)/ig, "" );


What's the difference between stripping all tags using this, and using stripHTMLTags like above? Please comment.

stripHTMLTags in YoastSEO.js replaces HTML tags with one space, but we want to do another try where we replace HTML tags without replacing them with one space.

IreneStr · 2018-11-12T10:35:04Z

js/src/decorator/gutenberg.js

+
+	const offsets = getOffsets( mark );
+
+	const startOffset = foundIndex + offsets.startOffset;


Why do you need to add an offset to the foundIndex? Please add inline comment.

The offsets.startOffset and offsets.endOffset are offsets of the <yoastmark> relative to the start of the Mark object. The foundIndex is the index form the start of the RichText until the matched Mark, so to calculate the offset from the RichText to the <yoastmark> we need to add those two indexes/offsets.

Edit: This can probably be refactored to be more readable.

IreneStr · 2018-11-12T10:35:27Z

js/src/decorator/gutenberg.js

+	const startOffset = foundIndex + offsets.startOffset;
+	let endOffset = foundIndex + offsets.endOffset;
+
+	// If the marks are at the beginning and the end we can use the length


Please turn into multline comment

IreneStr · 2018-11-12T10:45:52Z

js/src/decorator/gutenberg.js

+ *
+ * @param {Paper} paper The paper that the marks are calculated on.
+ * @param {Mark[]} marks The marks to annotate in the text.
+ * @returns {void}


Newline before returns

IreneStr · 2018-11-12T10:46:49Z

js/src/decorator/gutenberg.js

+ * @returns {void}
+ */
+export function applyAsAnnotations( paper, marks ) {
+	// Do this always to allow people to select a different eye marker.


... while/when another eye marker is still active

IreneStr · 2018-11-12T10:47:23Z

js/src/decorator/gutenberg.js

+
+	// For every block...
+	const annotations = flatMap( blocks, ( ( block ) => {
+		// We go through every annotate able attribute.


annotatable

IreneStr · 2018-11-12T10:48:22Z

js/src/decorator/gutenberg.js

+ * @param {string} blockTypeName The name of the block type.
+ * @returns {string[]} The attributes that we can annotate.
+ */
+function getAnnotateAbleAttributes( blockTypeName ) {


Annotatable

IreneStr · 2018-11-12T10:57:18Z

js/src/edit.js

+	 */
+	initializeAnnotations() {
+		if ( isAnnotationAvailable() ) {
+			this._store.dispatch( setMarkerStatus( "enabled" ) );


Does this interfere with other places where we dis-/enable the markers? For example, when we detect a pagebuilder

It shouldn't. We check if the state for annotations is available and If so we put our marks as annotations in the annotation state. This is the most direct feature checking that's possible.

abotteram · 2018-11-12T13:36:57Z

Acceptance:

General testing (4.9.8 with Gutenberg plugin):

When updating the Keyphrase, the keyphrase markers are not updated.
When typing inside a marker, the marker isn't removed, even if it is no longer relevant for the assessment.
When collapsing and reopening a readability or seo analysis collapsible, the marker button is disabled, but the markers are still present in the editor. Clicking the marker button again will re-mark the content again.

5.0.0-beta 3 with Gutenberg plugin development version (that should have annotations).

Annotations do not work. window.wp.annotations is not available.

No breaking issues. 👍

klihelp · 2019-05-13T10:50:23Z

Not working on classic editor using Gutenberg Manager plugin.

Ref: #10265 (comment)

klihelp · 2019-05-13T11:54:24Z

The above issue is part of the withSelect() Gutenberg 5.6.1 which is not in core yet.
wp.data.select(....) returns null

atimmer added 2 commits July 5, 2018 11:14

Implement Gutenberg marking PoC

646d1dc

Remove console.logs

a95e74e

atimmer mentioned this pull request Jul 5, 2018

Annotation API WordPress/gutenberg#7718

Merged

4 tasks

atimmer added 19 commits July 5, 2018 11:36

Build assets for good measure

6e2f6f5

Merge branch 'trunk' into stories/gutenberg-annotations

182c21d

Wait for a frame before applying marks

0987ba0

Merge branch 'trunk' into stories/gutenberg-annotations

6b4d089

Merge branch 'trunk' into stories/gutenberg-annotations

359c171

Keep track of text nodes for better marking in Gutenberg

d4c4e26

Merge branch 'trunk' into stories/gutenberg-annotations

b2c94ae

Implement support for the refactored RichText structure

dfb6d52

Make annotations work in more cases

28608e9

Fix annotations and give them a color

cdb7726

Merge branch 'trunk' into stories/gutenberg-annotations

1be6297

Merge branch 'trunk' into stories/gutenberg-annotations

2ac7cda

Merge branch 'trunk' into stories/gutenberg-annotations

b001d4d

Improve annotations for Gutenberg

bdc944f

Enable annotation conditionally

6916b17

Cleanup annotation application

9f60b68

Add rich-text as an external

d7ccb1b

Fix annotations if the first thing isn't text

987f23a

atimmer changed the title ~~Implement Gutenberg marking PoC~~ Implement Annotations for Gutenberg Nov 5, 2018

atimmer changed the title ~~Implement Annotations for Gutenberg~~ Implement annotations for Gutenberg Nov 5, 2018

atimmer added 4 commits November 5, 2018 17:27

Implement annotations for multiline blocks

743121f

Drop XPath calculation

e27b7f9

Change annotation store

45ed0b7

Change annotation API usage to new __experimental

7822d1f

atimmer changed the base branch from trunk to release/9.2 November 9, 2018 10:52

Merge branch 'release/9.2' into stories/gutenberg-annotations

9ba1cbd

atimmer added 3 commits November 9, 2018 12:14

Add wp-annotations as a dependency

51e1f79

Remove hash duplicate tracking

62ba637

Refactor annotation applying to be more readable

8194099

atimmer added the needs-code-review label Nov 9, 2018

IreneStr added this to the 9.2 milestone Nov 9, 2018

atimmer added 3 commits November 12, 2018 09:47

Fix error in classic editor

d0eb99d

The error was "Cannot read property 'getBlocks' of undefined". We need to actually check if the `core/editor` store is registered before checking if a certain method is available on that object.

Fix mark buttons in classic editor

8259c5c

Missing () made the code always try the Gutenberg method for marking.

Allow the user to select a different eye marker

43b7e55

afercia mentioned this pull request Nov 12, 2018

Annotations markers tooltips cut-off #11556

Closed

IreneStr reviewed Nov 12, 2018

View reviewed changes

IreneStr and others added 2 commits November 12, 2018 13:39

Add and clarify comments and docblocks in gutenberg.js

7d103d2

Increase marker color specificity

1a78fd2

abotteram merged commit b214bb6 into release/9.2 Nov 12, 2018

abotteram deleted the stories/gutenberg-annotations branch November 12, 2018 14:49

abotteram removed the needs-code-review label Nov 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement annotations for Gutenberg #10265

Implement annotations for Gutenberg #10265

atimmer commented Jul 5, 2018 •

edited

Loading

designsimply commented Nov 9, 2018

IreneStr commented Nov 12, 2018

afercia commented Nov 12, 2018

atimmer commented Nov 12, 2018

IreneStr left a comment

IreneStr Nov 12, 2018

atimmer Nov 12, 2018

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

atimmer Nov 12, 2018

IreneStr Nov 12, 2018

atimmer Nov 12, 2018 •

edited

Loading

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

IreneStr Nov 12, 2018

atimmer Nov 12, 2018

abotteram commented Nov 12, 2018 •

edited

Loading

klihelp commented May 13, 2019

klihelp commented May 13, 2019 •

edited

Loading

-	endOffset = endOffset - startMark.length;
+     const matchedEndMark = marked.indexOf( endMark );
+	const endOffset = matchedEndMark - startMark.length;


		const offsets = getOffsets( mark );

		const startOffset = foundIndex + offsets.startOffset;

Implement annotations for Gutenberg #10265

Implement annotations for Gutenberg #10265

Conversation

atimmer commented Jul 5, 2018 • edited Loading

Summary

Relevant technical choices:

Test instructions

Gutenberg <4.2

Gutenberg >4.3

Quality assurance

designsimply commented Nov 9, 2018

IreneStr commented Nov 12, 2018

afercia commented Nov 12, 2018

atimmer commented Nov 12, 2018

IreneStr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atimmer Nov 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abotteram commented Nov 12, 2018 • edited Loading

klihelp commented May 13, 2019

klihelp commented May 13, 2019 • edited Loading

atimmer commented Jul 5, 2018 •

edited

Loading

atimmer Nov 12, 2018 •

edited

Loading

abotteram commented Nov 12, 2018 •

edited

Loading

klihelp commented May 13, 2019 •

edited

Loading