Accessibility: Voiceover assisted use of iOS app

[ThomDietrich]

Hey iOS developers,
this issue is a summary of problems related to accessibility of the iPhone app. While I am sure some of these are related to the frontend project, I want to suggest using this issue here, to track the overall progress to eventually enabling disabled users to use the Home Assistant app on their iPhone. I need your help to test the progress here, as I am an android user and wouldn't be able to assist in testing. If another tracking format is prefered by you guys, I am happy to adapt.

iOS device model, version and app version

Model Name: iPhone 12
Software Version: latest
App version: latest

The following was tested with the iPhone of my brothers. Happy to provide exact versions and further input as needed!

Home Assistant Core Version

core-2021.3.2

---

Preamble

Smart Home systems are an incredible addition to the life of an impaired person. My brother is blind and uses his smart home to take care of all kinds of things and to have a clear view on current states, e.g. that all lights are off when he leaves the apartment.

For those less familiar with accessibility: My brother uses the factory-provided assistive function VoiceOver on his iPhone to interact with all apps and features. Normally he is not restricted in what he is able to do with his smartphone. See this introduction video: https://www.youtube.com/watch?v=qDm7GiKra28

We recently moved over from openHAB, and while Home Assistant has some fresh ideas, the iOS is sadly not accessible on multiple levels.

Describe the bug

The Home Assistant Companion app has accessibility issues during initial setup and in daily use. Following, I will give a list of eminent problems and I hope you can help me to address them in the right repository or in new issues here. We are eager to provide further input. The list could be longer but the following are the critical show stoppers.

To reproduce, simply enable voiceover in your test environment. All reported issues are easy to reproduce.

Initial setup

1. Login page: Unclear jump

   • After entering username and password, swiping right should bring the user to the "Continue" button
   • Instead the "make password visible" eye icon is selected and the VoiceOver focus is moved back to the password field in a loop
   • The blind user is not able to "make password visible", nor does he get to the "Continue" button
   • The blind user is confused by the irrational behavior of the app
   • The blind user can manually swipe over his screen to find the Continue button

2. Login page: Continue button doesn't work

   • The blind user would normally double-tap the "Continue" button to trigger its action
   • This simply does not work. The button seems to have no function

Daily use

3. The hamburger menu doesn't open

   • Already reported in iOS Accessibility: cannot tap on sidebar when using voiceover #456, sadly closed by stale bot
   • Clicking the hamburger menu doesn't do anything
   • The blind user can't switch to another dashboard, nor settings or any other of the valuable views
   • Even with the sidebar opened up (done by temporary disabling VoiceOver) no element in the sidebar can be selected

4. Menu to the top right doesn't open

   • same issue as for the hamburger menu

Further issues

5. Many cards and elements are read by VoiceOver in a bad way
   • not a big issue now, not solely related to the iOS app
   • we will address these in the frontend repo as soon as general access is possible :)

Please let me know how we can help to solve these issues.
All the best, Thomas

[zacwest]

Unfortunately these are all in the WebView, which is displaying the frontend. See home-assistant/frontend#8178.

[ThomDietrich]

Hello @zacwest,
that's what I expected and indicated in the beginning of my message. Thanks for pointing me to the issue you have created for the issue. This is indeed a good summary of the hamburger menu issue, surprised I didn't find it before.

Your ticket does not cover the other mentioned issues. Shouldn't these also be addressed before closing the ticket? Where are we going to track them. Happy to create individual issues if that is the better way to take this forward. What I want to avoid is unlinked individual tickets losing traction and the bigger picture. Complete accessibility of the iOS app is an important topic and I would be happy to know it enjoys the attention it should.

On a personal note: I believe it was inconsiderate to pass of the ticket with one sentence and directly close it. It is my humble personal opinion that every aspect of a ticket and one's intention should be considered, and confirmation should be requested before closing. Your work as open source contributor is precious, so is my time typing the ticket. Let's be mutually respectful :)

All the best

[zacwest]

I read your ticket, and I'm sorry I did not include "please open tickets in the frontend repo for this" -- that was the message I was trying to get across. This repo tracks issues related to the iOS app, but not all of the iOS app from a user perspective. The frontend repo contains the code and issues related to dashboards, the login form, the sidebar and all other things referenced in your ticket that I can see. Feel free to create individual tickets in the frontend repo, but keeping tickets for the same issues doesn't help get them fixed.