[AriaNotify] Should notifications be HTML rather than plaintext or SSML?

[minorninth]

It feels like a missed opportunity to invent a new API that's built around plaintext, or a speech-only markup language like SSML. I'm fine with not prioritizing braille in v1 but I don't think we should build an API that's not rich enough to support rich notifications in the future.

Screen readers already don't just read the text of the web as plaintext, some screen readers support things like:

• Inline markup like bold/italic/emphasis
• Pronunciation hints (CSS speak-as)
• Different output for speech vs braille
• Math
• Abbreviations
• Various markup to distinguish between phone numbers vs math, etc.
• Switching languages in the middle of one sentence

More importantly, these are all web features (HTML, CSS, etc.) - and as they're added to web standards, browsers and screen readers can adopt them to provide richer speech and braille output.

Do we really want to move to an API that leaves out all of that potential? Do we want to say that we're never going to have an API that provides a way to annotate whether something is a phone number or an equation, or stresses one word - or that if we're ever going to support those that we'd have to reinvent them, rather than building on existing support for reading HTML?

[alisonmaher]

This is an interesting suggestion, although it may be overcomplicating the use cases we foresee ariaNotify solving.

Regarding speech markup and additional braille configurations, we decided to hold off on these for a future version of the API, and we believe that these would be possible to add on seamlessly in the future (as noted in #2334 and #2335).

However, given that ariaNotify is meant to provide notifications for visual cues that a non-AT user would be able to pick up from visual context (example, bolding a word actually resulted in the word being bolded, or knowing that a file failed to upload, or knowing if someone entered a virtual meeting, etc), I would imagine that HTML may be more than what an author needs to accomplish this use case.

More specifically, today, authors are using hidden dummy HTML nodes and live regions to accomplish these use cases. From what we have found, authors want a simpler and more ergonomic way of accomplishing this. I worry that using HTML instead of a simple textual based notification would make this more cumbersome for the use cases we foresee this solving.

Additionally, if the author is wanting to portray something that would require multiple languages, inline markup, math, etc, they should likely indicate that some way in HTML so that it is accessible to AT and non-AT users. I have a hard time thinking of use cases for these scenarios (other than what was mentioned in #2334 and #2335) that would require additional markup for the notification itself, as opposed to just including it in the normal content of the page.

Do you have use cases in mind for some of these scenarios you mentioned where an author would specifically want to portray information to an AT user using HTML markup that would be implied to a non-AT user through some sort of visual cue that wouldn't be accomplished with the current ariaNotify design?

[minorninth]

Let's take the example of bolding a word. Let's say the document is in French but the UI is in English. The selected word is "pain" (French for "bread"). They want to speak: "pain bolded", but the word "pain" should be pronounced in French and "bolded" in English.

In HTML, it's just:

pain bolded

[alisonmaher]

Thanks for the example. In this particular scenario, we expect authors to more likely to announce "bold on" or "bold off" instead of including the word itself. I also expect that more likely than not, most authors would be ok with an announcement using a single language, even in this particular example.

I am sure there certainly be these kinds edge cases, though. However, based on the use cases we have seen, we expect a string to cover the vast majority of authoring asks that we've heard so far. As such, I'd be inclined to consider this proposal for a future version of the API, similar to what was discussed in #2334 and #2335.

I do think your point around HTML being the way to provide markup makes a lot of sense, although we would need to consider what limitations that structured payload could take. I could see adding something like this on in the future in one of two ways:

1. Add a new parameter to ariaNotify() that takes the markup-based version of the notification. The benefit of this approach is that if an OS notification API doesn't yet support the markup-based addition, then it would fall back to the plain string.
2. Create a separate "sister" API called something like ariaNotifyElement() that takes an HTML element instead of a string to make the announcement.

[cookiecrook]

Whether it’s HTML or not, I concur with the need for a structured payload in some format. Perhaps a DOM node, which could be a string node or element node containing other strings or elements.

What kind of limits should the “structured notification” payload have? E.g. would you expect to be able to pass a link or other interactive control in the notification payload?