axe-core emits errors instead of frame-tested violations for some types of iframes

[dbjorge]

Product: axe-core

Summary: When axe-core's default frameMessenger hits a postMessage error, it results in an Error leaking from await axe.run(), where it would ideally instead result in the scan completing with a frame-tested violation for the frame in question.

Some specific variants that can trigger the issue:

• A file:// url that embeds an iframe which axe-core could normally reach will emit an error with message allowedOrigins value "null" is not a valid origin
• Any page that embeds an iframe which the browser prevents cross-origin access to (eg, a file:// url that embeds an https:// page) will emit a SecurityError with message Blocked a frame with origin "null" from accessing a cross-origin frame.
  • In Chromium, the message is Blocked a frame with origin "null" from accessing a cross-origin frame.
  • In Firefox, the message is Permission denied to access property "DOMException" on cross-origin object

Repro steps:

1. Download this self-contained repro gist to a local html file
2. Open it via a file:/// url
3. Click the two "scan" buttons
4. Observe the errors displayed in the <textarea>

Expectation: if the scan context includes iframes which the default postMessage implementation cannot communicate with, it should result in scans which complete with frame-tested violations, not errors emitted from await axe.run()

Actual: await axe.run() emits Errors in the two variants from the repro file

Motivation: Caused a regression in one of our sample projects when upgrading from 4.1.x to 4.2.x, see TroyWalshProf/SeleniumAxeDotnet#154 and microsoft/axe-pipelines-samples#552.

axe-core version: 4.2.2
axe-webdriver, extension or other integration version: n/a

Browser and Assistive Technology versions:
* Browser versions: repros under each of the following:
    * Microsoft Edge Version 91.0.864.48 (Official build) (64-bit)
    * Google Chrome Version 91.0.4472.106 (Official Build) (64-bit)
    * Firefox 89.0 (64-bit)
        * **Note: SecurityException variant works manifests slightly differently in Firefox vs Chromium**
* AT versions: n/a

[straker]

I can see what's going on. While I agree with you that if axe-core can't ping the iframe it shouldn't result in an error but rather a frame-tested failure, which should probably contain the error message at best. However, we just modified the frame-tested rule to no longer ping iframes at all, so it would have no knowledge of why the iframe didn't get tested.

We'll have to think on this to figure out the best way to handle errors when trying to send iframe messages while still informing the user as to why the frame wasn't tested.

[WilcoFiers]

I think the first thing to fix is to prevent axe.run from throwing an error. We can work on improving messaging later, but axe.run definitely shouldn't error over being unable to communicate with iframes.