Thoughts on/review of CloseWatcher proposal and relation to synthesized events

[domenic]

Hi AOM folks,

I'm working on a proposal called close watchers which is intended to allow web developers to more easily handle "close signals" such as the Esc key or the Android back button in a uniform, cross-platform way.

My current explainer indicates that accessibility-tech "dismiss" gestures (so far I know about VoiceOver's) might want to trigger this path. But as pointed out in w3ctag/design-reviews#594 (comment), this seems to conflict with the plans in https://github.com/WICG/aom/blob/gh-pages/explainer.md#user-action-events-from-assistive-technology to have such gestures synthesize an Esc key press.

I'd be happy to remove the idea of AT directly firing close signals at CloseWatcher instances, and instead have the synthesized Esc key presses trigger the close signal indirectly instead. In both cases developers could use the CloseWatcher interface just fine to uniformize their handling of such close signals across platforms. But I wanted to get your take, as the experts in the space.

Concretely, the difference would be:

• Direct variant: making a dismiss gesture triggers any active CloseWatcher. No esc key presses are synthesized.
• Indirect variant: making a dismiss gesture fires a key event for the Esc key press. If that is not canceled, then the key event in turn triggers any active CloseWatcher.

For privacy my initial guess is that either option is reasonable, but a determined attacker could still distinguish AT users from non-AT users in some situations:

• For the direct variant, they would notice that CloseWatcher is triggered without any corresponding key event. Since iOS has no other situation where this could occur, except for AT users, the site could detect AT usage
• For the indirect variant: any Esc keypress events on iPhones automatically indicate to the site an AT user, since Esc keypresses are otherwise impossible there

I don't know enough about dismiss gestures on other operating systems or devices to comment on those, but I suspect there's a similar tradeoff.

[cookiecrook]

@domenic wrote:

For the indirect variant: any Esc keypress events on iPhones automatically indicate to the site an AT user, since Esc keypresses are otherwise impossible there

This could be triggered by full keyboard access users on all systems, not just AT users.

[domenic]

Hmm, I guess I'm not understanding the distinction; everything I read classifies full keyboard access as an accessibility feature and seems like it's something a smaller class of users turns on. I'd love to learn more about the distinction between full keyboard access and accessibility technology.

[cookiecrook]

Web Platform Design Principle 2.9 states, “Don’t reveal assistive technologies are being used.”

Because of this, AOM direction has been to avoid any distinguishing event difference between AT uses and mainstream uses. if CloseWatcher (or an equivalent event) becomes available in a mainstream context, it would likely be used in addition to the key event, rather than instead of.

[cookiecrook]

everything I read classifies full keyboard access as an accessibility feature

Presumably this mention is referring to FKA in a specific context, such as iOS? There’s nothing about the feature that precludes its use by people with or without disabilities.

[domenic]

Yes. Is there something about other AT that prohibits its use by people without disabilities?

Regarding the design principles, I'm trying to understand the implications with regard to FKA and the dismiss gesture. I can guess a few readings based on what you've said so far:

• It is not OK to reveal that any technologies useful by people with disabilities are being used, including FKA and the dismiss gesture.
• It is not OK to reveal that "assistive technologies" are being used, with some definition that excludes FKA from "assistive technologies" so it is OK to reveal that FKA is being used. I'd like to learn more about this definition and, e.g., what side of the line the VoiceOver swipe gesture falls on.
• It is not OK to reveal assistive technologies in general, but it is OK to reveal any assistive technology which a non-disabled user could also turn on and use. Again I'd like to learn more about the distinctions here.

Are any of these close to your interpretation? Or something else?

[cookiecrook]

Yes. Is there something about other AT that prohibits its use by people without disabilities?

No, but since the AT for severe disabilities is optimized for those with a specific disability, outside the testing or occasional power user case, it's rare for someone without the target disability to user a screen reader or switch.

Regarding the design principles, I'm trying to understand the implications with regard to FKA and the dismiss gesture. I can guess a few readings based on what you've said so far:
It is not OK to reveal that any technologies useful by people with disabilities are being used, including FKA and the dismiss gesture.
It is not OK to reveal that "assistive technologies" are being used, with some definition that excludes FKA from "assistive technologies" so it is OK to reveal that FKA is being used. I'd like to learn more about this definition and, e.g., what side of the line the VoiceOver swipe gesture falls on.
It is not OK to reveal assistive technologies in general, but it is OK to reveal any assistive technology which a non-disabled user could also turn on and use.

I might rephrase this: "it is OK to reveal use of technology which is commonly used by those with and without disabilities." Privacy caveats apply. In particular, where the use of which does not statistically infer a disability. For example, the dynamic type "bold text" and "font size" settings on iOS started out in the Accessibility settings, but are some of the most commonly used features on iOS.

Again I'd like to learn more about the distinctions here.

The distinction is probably one to be decided by the TAG, or individually by each WG. For example, CSS WG decided to incorporate reduce motion and other relevant media features in part because they were more widely used than just by those with vestibular disorders. You can't reasonably infer that its use implies a "disability."

That's not the case with other AT designed for more severe disabilities like screen readers and switch software, whose use may be by a tester or power user, but much more often than not, means the user is blind or has a severe motor impairment.

[cookiecrook]

Thanks @domenic for attending today. Rehashing the key points as I recall them while still fresh.

We didn't come around to a specific path forward on whether something like CloseWatcher is necessary, but Dominic did raise two scenarios that the straight Esc key synthesis doesn’t handle:

• There’s no unambiguous registering of “the web app (or specific element) can and will handle the close”… So it’s potentially ambiguous whether the “back” button (or AT gesture, for example) should send an Esc key to the web view or a native “back” (such as iOS’s accessibilityPerformEscape()) to the native app that contains the web view. The engine could wait for the Esc key event to be cancelled or not, but potential downsides are performance implications, and the following malicious actor case.
• The malicious case (interrelated with above) that an ad banner should not be able to hijack the Android “back” button (or AT gesture) unless there is user interaction with that web view. For example, a page can’t create a close watcher for a dialog until after the user first clicked a button to open that dialog.

[domenic]

Yep, thanks for the summary! There's also the developer ergonomics benefit that the stack of close watchers provides. See https://github.com/WICG/close-watcher#a-single-event for some discussion.