Validate and protect corrupted block content #1929

aduth · 2017-07-17T19:57:12Z

Closes #1735
Closes #1736
Related: #391

This pull request seeks to validate blocks parsed from post content. It does so by regenerating the expected save output upon the initial parse and comparing it against the actual content. If there is a mismatch, the block is flagged as invalid and the user is barred from editing the block (aside from moving or deleting) with a visual indicator of the invalid state:

Implementation notes:

Yes, there's likely a performance impact of creating an initial serialization of the block
Yes, there's likely false positives that can occur, particular on order of attributes or class names (which in reality should not have an impact on behavior of content). To reduce false positives, both contents are run through the same HTML beautifier we use for serialization. False positives from reordering attributes seems acceptable to me, as we should discourage manual markup modifications, or at least transition the user to a "manual mode". Enhancements could be made here. Editing block markup in Text mode should be unaffected.

Testing instructions:

I've left the initial Cover Image block in Demo content as invalid, because I'd like to revisit whether we really want nested section tags in the serialized output. Verify that the first block in post content is flagged as invalid and cannot be edited.

Navigate to Gutenberg > Demo
Note that the first block has a "Corrupt" warning like the one shown above

afercia · 2017-07-17T21:34:44Z

Worth noting that, while clicking the block is disabled, using a keyboard it's still possible to focus the cover image title and edit it. Tabbing backwards from the title it's also possible to focus the formatting toolbar buttons and use them:

aduth · 2017-07-18T20:00:04Z

using a keyboard it's still possible to focus the cover image title and edit it

Great point, and a challenging problem to solve if we want to keep the edit render visible in the background (a nice effect I think). A few possibilities that come to mind:

Intercept the focus event within the block and "skip" to the next valid focus-able element
Query for all editable inputs within the block after render and disable them
Wait for something to come of an intertness property, or simulate effect with a component wrapper
Show only the warning notice, no edit preview

Other ideas welcome.

mtias · 2017-07-24T12:25:24Z

editor/modes/visual-editor/invalid-block-warning.js

+	<div className="editor-visual-editor__invalid-block-warning">
+		<Dashicon icon="warning" size={ 40 } />
+		<p>{ __(
+			'Uh oh! This block is corrupt and has been locked to protect ' +


"This block has been modified externally and has been locked to protect against content loss." cc @jasmussen

Yep, that sounds good!

mtias · 2017-07-24T12:27:20Z

We should consider adding these buttons:

[ Convert to Freeform block ] [ Overwrite changes ] [ Do nothing ]

jasmussen · 2017-07-24T12:37:25Z

We should consider adding these buttons:
[ Convert to Freeform block ] [ Overwrite changes ] [ Do nothing ]

I dig that. Though it beckons yet another name than "Classic Text" for the fallback block. I mean, I do like the "Classic Text" name, I think it's good for when you mean to insert it via the Inserter, and it's more descriptive than "Freeform", but in this context it's obvious that it can contain more than just "text". Should we simply call it "Classic Editor"?

mtias · 2017-07-24T12:40:44Z

Yes, any name works for me. Maybe "convert to generic block"? Also we could offer the option to handle as HTML block.

mtias · 2017-07-24T12:42:14Z

This looks like a great start to me. @nylen what do you think of measuring performance impact with and without validation on a long-ish post? We may consider making this something you can disable via config or filter.

aduth · 2017-07-24T16:04:48Z

Rebased to resolve conflicts and update warning text.

I tried a couple ideas relating to disabling keyboard focus for the preview. Both of which are not particularly ideal from a performance point-of-view. Each implements an Inert component which, when wrapping content, enforces that children cannot receive focus.

The approach in 7ad484f intercepts global tab keydown events and, if the key would result in the node receiving focus, prevents the default event behavior and instead shifts focus to the next focusable node it finds
The approach in f452481 finds and disables all inputs and contenteditables within the node every time it renders

mtias · 2017-07-25T10:08:48Z

@aduth I'm thinking it'd be cool to be able to switch between "old" and "new" renders to visually diff. It could be two tabs next to the warning message.

aduth · 2017-07-25T12:54:32Z

I'm thinking it'd be cool to be able to switch between "old" and "new" renders to visually diff. It could be two tabs next to the warning message.

Should we do this in a follow-up pull request?

mtias · 2017-07-25T15:50:01Z

Should we do this in a follow-up pull request?

Yes, just wondered if it could help with the keyboard issues, because we could try not rendering the default view.

aduth · 2017-07-25T16:14:46Z

The issue is that if preview uses edit rendering, it's cumbersome to find and disable all inputs within that render.

Alternatively, we could render the save representation for corrupt blocks, which unlike the Edit form should include no editable fields, and is arguably more appropriate to show in case "corrupt" means "user made their own edits that may appear in preview".

mtias · 2017-07-25T16:21:10Z

Yes, I was thinking of rendering save and raw as comparison, not edit.

aduth · 2017-07-25T18:15:04Z

5328e23 shows the save representation as the preview, which shouldn't have any editable fields, and avoids the need for any special tab behavior (well... now that I think of it, if the block is a contact form, maybe not so much 😬 ).

mtias · 2017-07-25T18:21:03Z

If it's a contact form, it is likely the save method is null.

aduth · 2017-07-25T18:24:09Z

Sure, it's a rare case that the save representation have focusable fields, but still technically possible and not very well handled here.

jasmussen · 2017-07-26T07:27:09Z

I took the liberty of pushing a little visual polish:

Matías points about some action buttons seems really worth addressing, though okay to do it in a separate PR. In the mean time, the message feels like it should be rephrased to be less "this is what's up" and more like "this is what it means". I'm not entirely sure how to phrase this without being overly verbose, as it's a rather complex thing to explain.

Perhaps this begs general help text in some way, or like linking to external documentation. A simple "What does this mean?" link at the end could address it.

aduth · 2017-07-26T16:29:48Z

Since this touches many different files, I don't want to let this linger pending finer details. I assume we'll need to do some revisions in subsequent pull requests, notably of course fixing the "corrupt" demo Cover Image block.

In the meantime, I've rebased and force-pushed with intent to merge.

One additional change I made is allow the root node block list item to be focused so that it can be "selected", but only for arrangement / deletion (no toolbar shown). See feb105e.

aduth · 2017-07-26T16:32:04Z

Actually, I did find one remaining item that should probably be addressed here: If the user tries to edit an invalid block in Text mode, those user edits will not be respected on save.

aduth · 2017-07-26T17:06:12Z

I believe the behavior described in #1929 (comment) is actually working correctly, and the issue I'm seeing is caused by the wrong URL being assigned when saving a post from the Demo.

Should be: ?page=gutenberg&post_id=568
Actually: ?page=gutenberg-demo&post_id=568

...Resulting in demo content being shown after a refresh, not the saved post's.

Likely a regression introduced in #1967, where $is_demo check no longer accounts for whether a post ID is present in query parameters. Will open a separate pull request shortly.

aduth · 2017-07-26T17:17:28Z

I've verified that editing an invalid block in Text mode works as expected: Changes will be respected, but switching back to Visual will still show the block as invalid so long as its reserialization doesn't match.

See also #2032 for demo content save fix.

nylen · 2017-07-26T17:58:32Z

blocks/test/fixtures/core__gallery.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<div class=\"wp-block-gallery\">\n\t<figure class=\"blocks-gallery-image\">\n\t\t<img src=\"https://cldup.com/uuUqE_dXzy.jpg\" alt=\"title\" />\n\t</figure>\n\t<figure class=\"blocks-gallery-image\">\n\t\t<img src=\"http://google.com/hi.png\" alt=\"title\" />\n\t</figure>\n</div>"


This needs to be fixed.

nylen · 2017-07-26T17:58:43Z

blocks/test/fixtures/core__image__center-caption.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<figure class=\"wp-block-image\"><img src=\"https://cldup.com/YLYhpou2oq.jpg\" class=\"aligncenter\"/><figcaption>Give it a try. Press the &quot;really wide&quot; button on the image toolbar.</figcaption></figure>"


This needs to be fixed.

nylen · 2017-07-26T17:58:54Z

blocks/test/fixtures/core__list__ul.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<ul><li>Text & Headings</li><li>Images & Videos</li><li>Galleries</li><li>Embeds, like YouTube, Tweets, or other WordPress posts.</li><li>Layout blocks, like Buttons, Hero Images, Separators, etc.</li><li>And <em>Lists</em> like this one of course :)</li></ul>"


This needs to be fixed.

nylen · 2017-07-26T17:59:02Z

blocks/test/fixtures/core__preformatted.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<pre class=\"wp-block-preformatted\">Some <em>preformatted</em> text...<br>And more!</pre>"


This needs to be fixed.

nylen · 2017-07-26T17:59:09Z

blocks/test/fixtures/core__pullquote.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<blockquote class=\"wp-block-pullquote\">\n<p>Testing pullquote block...</p><footer>...with a caption</footer>\n</blockquote>"


This needs to be fixed.

nylen · 2017-07-26T17:59:14Z

blocks/test/fixtures/core__table.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<table><thead><tr><th>Version</th><th>Musician</th><th>Date</th></tr></thead><tbody><tr><td><a href=\"https://wordpress.org/news/2003/05/wordpress-now-available/\">.70</a></td><td>No musician chosen.</td><td>May 27, 2003</td></tr><tr><td><a href=\"https://wordpress.org/news/2004/01/wordpress-10/\">1.0</a></td><td>Miles Davis</td><td>January 3, 2004</td></tr><tr><td>Lots of versions skipped, see <a href=\"https://codex.wordpress.org/WordPress_Versions\">the full list</a></td><td>&hellip;</td><td>&hellip;</td></tr><tr><td><a href=\"https://wordpress.org/news/2015/12/clifford/\">4.4</a></td><td>Clifford Brown</td><td>December 8, 2015</td></tr><tr><td><a href=\"https://wordpress.org/news/2016/04/coleman/\">4.5</a></td><td>Coleman Hawkins</td><td>April 12, 2016</td></tr><tr><td><a href=\"https://wordpress.org/news/2016/08/pepper/\">4.6</a></td><td>Pepper Adams</td><td>August 16, 2016</td></tr><tr><td><a href=\"https://wordpress.org/news/2016/12/vaughan/\">4.7</a></td><td>Sarah Vaughan</td><td>December 6, 2016</td></tr></tbody></table>"


This needs to be fixed.

nylen · 2017-07-26T17:59:20Z

blocks/test/fixtures/core__verse.json

-        }
+        },
+        "isValid": false,
+        "originalContent": "<pre class=\"wp-block-verse\">A <em>verse</em>…<br>And more!</pre>"


This needs to be fixed.

nylen · 2017-07-26T18:00:37Z

There are now several examples of block fixtures with "isValid": false in the test suite. All of these should probably be valid, and we need to add test cases for the ways we expect this feature to detect invalid blocks.

aduth · 2017-07-26T18:35:55Z

See #2038 for updated Cover Image implementation to resolve lingering warning on Demo content screen.

aduth added the [Feature] Parsing Related to efforts to improving the parsing of a string of data and converting it into a different f label Jul 17, 2017

aduth mentioned this pull request Jul 20, 2017

Handle unrecognized blocks without modifying their markup #1735

Closed

mtias reviewed Jul 24, 2017

View reviewed changes

mtias added [Feature] Blocks Overall functionality of blocks General Interface Parts of the UI which don't fall neatly under other labels. labels Jul 24, 2017

aduth force-pushed the try/validate-parse branch from eeb0450 to f452481 Compare July 24, 2017 16:00

aduth force-pushed the try/validate-parse branch from f452481 to 5328e23 Compare July 25, 2017 18:12

aduth added 5 commits July 26, 2017 12:15

Validate and protect corrupted block content

295e860

Add inert component to redirect unfocusable

5cbe8d7

Refactor inert to explicitly disable fields

f8733d1

Use save rendering for invalid post preview

74be312

Allow invalid block focus for movement / deletion

feb105e

aduth force-pushed the try/validate-parse branch from 5132df5 to feb105e Compare July 26, 2017 16:27

aduth mentioned this pull request Jul 26, 2017

Initialize demo content only for new posts #2032

Merged

aduth merged commit a8b39fb into master Jul 26, 2017

aduth deleted the try/validate-parse branch July 26, 2017 17:17

nylen mentioned this pull request Jul 26, 2017

Ensure that all block fixtures end with newlines #2034

Merged

aduth mentioned this pull request Jul 26, 2017

Create new blocks as valid #2033

Merged

nylen reviewed Jul 26, 2017

View reviewed changes

nylen mentioned this pull request Jul 27, 2017

Add "More" block. #1440

Merged

This was referenced Jul 27, 2017

Fix and regenerate invalid block fixtures #2056

Closed

Provide options for invalid block resolution #2132

Closed

youknowriad mentioned this pull request Aug 2, 2017

Content is lost if you remove P tags #1526

Closed

aduth mentioned this pull request Aug 4, 2017

Validation: Detect block validity by tokenizing content #2210

Merged

nylen mentioned this pull request Aug 4, 2017

Add HTML schema for separator block #914

Closed

5 tasks

aduth mentioned this pull request Aug 10, 2017

Cover Image: Extract title as children array (fix "modified externally") #2341

Merged

nylen mentioned this pull request Aug 10, 2017

Parser: Start tokenizing HTML within the blocks #2286

Closed

aduth mentioned this pull request Sep 29, 2017

Validation: Improve developer messaging for invalid blocks #2837

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Validate and protect corrupted block content #1929

Validate and protect corrupted block content #1929

aduth commented Jul 17, 2017 •

edited

Loading

afercia commented Jul 17, 2017

aduth commented Jul 18, 2017

mtias Jul 24, 2017

jasmussen Jul 24, 2017

mtias commented Jul 24, 2017

jasmussen commented Jul 24, 2017

mtias commented Jul 24, 2017

mtias commented Jul 24, 2017

aduth commented Jul 24, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017 •

edited

Loading

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

jasmussen commented Jul 26, 2017

aduth commented Jul 26, 2017

aduth commented Jul 26, 2017

aduth commented Jul 26, 2017 •

edited

Loading

aduth commented Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen Jul 26, 2017

nylen commented Jul 26, 2017

aduth commented Jul 26, 2017

Validate and protect corrupted block content #1929

Validate and protect corrupted block content #1929

Conversation

aduth commented Jul 17, 2017 • edited Loading

afercia commented Jul 17, 2017

aduth commented Jul 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtias commented Jul 24, 2017

jasmussen commented Jul 24, 2017

mtias commented Jul 24, 2017

mtias commented Jul 24, 2017

aduth commented Jul 24, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017 • edited Loading

aduth commented Jul 25, 2017

mtias commented Jul 25, 2017

aduth commented Jul 25, 2017

jasmussen commented Jul 26, 2017

aduth commented Jul 26, 2017

aduth commented Jul 26, 2017

aduth commented Jul 26, 2017 • edited Loading

aduth commented Jul 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nylen commented Jul 26, 2017

aduth commented Jul 26, 2017

aduth commented Jul 17, 2017 •

edited

Loading

mtias commented Jul 25, 2017 •

edited

Loading

aduth commented Jul 26, 2017 •

edited

Loading