NVDA and Plover pplication.

[ossossosso]

Hi!
My name is Daniele. I'm from Italy. I'm a stenotyper: using a special keyboard I transcribe bery fast recorded audio into text. I'm new in this job so I need some help to fix the workflow. I don't know any other blind person who do it, so I have nobody to ask to.
Thank you in advance for reading, and sorry for my basic English.

The application that I use for that is called "Plover":
http://www.openstenoproject.org/

It is free and opensource. It simply translates keys combinations that I press on my keyboard into words or syllables, it isn't a word processor, so I can write in any application (MS Word, Wordpad, Google Documents, Notepad, etc.). Everywhere there is a Text Field.

NVDA reads correctly the word that I type. The problem is when I make a mistake and I have to delete the word.

Using a standard qwerty keyboard if I press "Backspace" NVDA reads the characters I delete. one by one.
And if I press "Ctrl + Backspace" NVDA reads the word I delete, one by one.

Using my stenokeyboard and Plover I can simulate that combinations of keys, but NVDA doesn't read characters or words that I delete.

To fix it there's an option under NVDA preferences-->keyboard settings-->handle keys from other applications.
If it is checked the problem is solved.
But unfortunatley another problem appears.

Let me explain in short how stenomachine and plover work together for better understand.

Plover (like any other steno applications) has a dictionary text file, with this kind of entry:
"Vctpie": "inevitabilmente",
On the left side of the ":" there is the combination of keys I press at once. On the right side the output that appears if I press that combination. Basically stenotype is a long list of shortcuts that I can press to type longer words. Once you memorize all of them, you can write extremely fast.

On the right side I insert what I want, also a sentence. So if I press that specific combinations of keys, the whole sentence appears.

Here's is the problem. With entries like this one:
"I/Pshr": "Pubblico Ministero -"

If "handle keys from other applications" is ON, the output is modified by NVDA, randomly:
Pubblico Minister -
or
Pubblico MINISTERO _
or
Pubblico Minisero -

Another example. The dictionary contains this entry:
"I/PRcte": "Avv. Parte Civile, Carosso -"

The uncorrect output could be:
Avv. Parte Civile, Caross -
or
Avv. Parte Civile, Carosso
or
AVV. PARTE CIVILE; CAROSSO _

I notice the problem only with longer output. Sometimes it appears, sometimes not. The last example:

"Pstp/CHRh": "Pubblico Ministero",
sometimes can output
pPubblico Minister

I hope I have been clear. I did my best.
I really hope can be found a solution.

I can give more informations if need.

Thank you very much for reading until here.

Ps.
Plover works also with a standard qwerty keyboard. Once Plover is run and Enabled, the qwerty keyboard changes layout into a stenomachine.

[Adriani90]

@ossossosso how is it working with NVDA 2019.1 and the last version of plover? Do you still have this issue?

[ossossosso]

Thank you for your question.
A usable workflow between a SCreen Reader and Plover needs a specific plugin to make them dialogue correctly. Unfortunatly an update, unless dedicated to it, cannot fix all the issues.

Thanks again.

[josephsl]

Hello,

Do you have any updates on this situation in 2024?

Thanks.

[ossossosso]

Hi Joseph,
To ensure that NVDA's vocal feedback works correctly during transcription with Plover, a plugin is required that enables communication between Plover and NVDA. Without this plugin, there will be inaccuracies.

The developers at Freedom Scientific have created a plugin for Plover that works correctly.
The plugin is available among the Plover plugins.

The plugin has open-source code, and I imagine it can be modified to be used with other screen readers as well.
Best regards,
Daniel

[Adriani90]

Thanks for the update. Is ther a link to the plugin and its source code?Von meinem iPhone gesendetAm 20.07.2024 um 14:47 schrieb ossossosso ***@***.***>: Hi Joseph, To ensure that NVDA's vocal feedback works correctly during transcription with Plover, a plugin is required that enables communication between Plover and NVDA. Without this plugin, there will be inaccuracies. The developers at Freedom Scientific have created a plugin for Plover that works correctly. The plugin is available among the Plover plugins. The plugin has open-source code, and I imagine it can be modified to be used with other screen readers as well. Best regards, Daniel —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[josephsl]

Hi,

Link to that plugin (at least the source code) please. Even though Vispero has created a JAWS script or a plugin, it may not be adoptable across screen readers due to internal differences. If it turns out an add-on from NVDA is needed, then I think it would be best to take care of it from the add-ons community first.

Thanks.

[ossossosso]

Running Plover, you can download from the plugin manager.
Otherwise you can download the code from one of the two links.

https://pypi.org/project/plover-jaws/

https://www.wheelodex.org/projects/plover-jaws/

P.S.
I am a blind stenographer, and I began this career in 2017 after losing my sight.
Every two years, international competitions are organized by a federation called Intersteno, which brings together transcribers from around the world.
My first participation in Intersteno was in 2017, and this July, I competed for the fourth time.
The competitions concluded just a few days ago, with around 400 participants from 30 countries across all five continents.
In the competition where I used the shorthand keyboard, Plover, and speech synthesis, I secured second place.
I was the only blind participant.
Beyond sharing my personal achievement, I want to highlight the career opportunities available for blind individuals.
The same tools I used in the competition are those I rely on in my daily work as a court stenographer.
Achieving these results would not have been possible without software development that made my workflow accessible, including the Plover plugin.
Below is the link to the results of the Audio Transcription competition, where I achieved second place:
https://intersteno2024.org/resultlistAT.php
You can find me listed in the second position in the first table on the page; my name is Daniele Casarola.
This year, I also achieved first place in another competition called Real Time:
https://intersteno2024.org/resultlistRT.php
In this competition, I used Dragon Naturally Speaking for voice recognition to transcribe the text.
Once again, without accessibility tools, participating would not have been possible.
I hope my experience serves as an inspiration for other blind individuals seeking a fully accessible profession that is still relatively unknown.
I also hope it encourages developers to continue improving the accessibility of this profession.
Thank you

[josephsl]

Hi,

Thank you for the link to download the JAWS plugin. According to PyPI (Python Package Index), it is listed as GPL version 2, so it passes license checks. However, when I attempt to install it on my local Python installation (via pip), it says the package satisfying the Python version cannot be found because the plugin is written for Python 3.6 which is end of life. Besides, the plugin is marked "alpha", which is a bit concerning - it could be buggy and may not owrk well with newer Python versions and/or Plover.

Code analysis: the plugin is essentially a bridge between Plover and JAWS screen reader. It employs Plover Python modules (apparently based on the import statement), uses pywin32 223 and pythoncom to invoke COM (Component Object Model) facilities,, and communicates with JAWS through its COM API. The key feature is detecting and speaking inserted/deleted text, and part of that involves checking to see if JAWS should announce text (a series of "if" statements are used to determine things such as digits, whitespace, and what not).

I think one way to implement NVDA support for Plover is NVDA controller client where the ap actually tells the screen reader what to say. For example, Plover could send inserted text to NVDA when text is inserted or inform NVDA about deleted text. An NVDA add-on might work but it must be distributed outside of Plover's plugin infrastructure or outside PyPI (JAWS scripts require local Python installation).

About software versions: in short, you are using alpha level code, a software version that could be prone to bugs and may include unfinished features. In open-source communities, it is okay to distribute and use alpha software as long as you know what you're doing. In a more professional environment, use of alpha versions is discouraged, especially if the environment demands accuracy and timeliness like stenography (exceptions are possible). For instance, NVDA's own alpha-level code (called master branch snapshots) are meant for developers and testers and is not used often in professional environments.

Thanks.

[ossossosso]

Ehm,
I'm not a developer, so I can't follow all the technical details, but I generally understood the meaning of your message.
I spoke with the developer of the plugin when it was created. Once we saw that everything worked, we stopped. I'm not sure why he referred to it as alpha.
Regarding the functionality of the plugin, I also uploaded a video on YouTube:

https://www.youtube.com/watch?v=oNlLTbTCRj8&t=42s

There are different tasks to satisfy to make everything fully accessible.
If you believe it is a project to invest a bit of time, I will be really happy to prepare a document detailed about it.

From the code I see you have already get some points.
The video on youtube also gives you some extra informations.

Here another video of a demo typing with a steno keyboard. I type pretty slow to better let hear the audio of the dication and of the screen reader. In a normal job is faster:

https://www.youtube.com/watch?v=emiCbuMhFxs

Thanks for your interest.

[Adriani90]

@jenchanws, @sammdot, @nvdaes, @benoit-pierre sorry for tagging you on this one but it seems NVDA users are still trying to use this software and there are some difficulties. Can anyone give some details on whaht needs to be improved?
From what I read in the comments above a similar method like for Jaws needs to be used for communicating with NvDA as well, ideally over the controlerClient, providing the coresponding dll file with the plover installation. Has this been already considered?

[nvdaes]

Hello. Years ago, I read the code of a JAWS plugin and thought about trying to implement a similar feature for NVDA. After that, I didn't work on it.
I work with Plover daily in my job, and I'm very grateful with the person, an NVDA user, who talked me about this software, that now is my main tool to work, since I was be able to stop using a proprietary software that may not be maintained properly, I don't know but I think that maintenance was not very appropriate, and anyway I needed to deppend on companies to receive software updates.
I'm working with Plover during more than 3 years, and I don't need a plugin to improve the experience with NVDA while typing via my stenograph machine. If someone needs something, I'll pay attention.

[ossossosso]

Hi Noelia, it's a pleasure to hear from you again.

Adriani90, I'm reaching out to share some issues I've encountered while using Plover to type and listening with NVDA. Here are the details of my setup: I’m currently using Plover RC2 and NVDA 2024.2.
Some of the text is vocalized by NVDA during transcription, and as a Court Reporter, I need to transcribe precisely what I hear. If NVDA provides inaccurate or delayed audio feedback, it can slow down my workflow.
To explain more clearly: when I transcribe, I listen primarily to the speaker's audio, and in the background—almost like an echo—I hear the screen reader reading what I'm typing. My attention is mainly focused on the speaker, but at times, I need to rely on the screen reader’s audio.
Since the speaker naturally pauses to breathe, the screen reader doesn’t, and during those pauses, my focus shifts entirely to the screen reader, which vocalizes slightly after the speaker. Sometimes, when I'm unsure if I've pressed the right key combination, I look to the screen reader for confirmation. That's why I need NVDA to vocalize everything, not just part of the text. Otherwise, I have to slow down and use the QWERTY keyboard to manually check what I've typed.
Here are the main issues I'm experiencing:
1.
NVDA doesn't read punctuation marks when I insert them using Plover.

NVDA doesn't read aloud when I delete a word or punctuation marks using Plover.

NVDA doesn’t let me review the last word, line, or sentence I’ve just written using a key combination on the steno keyboard.

If the "Handle keys from other applications" setting is enabled, some text written with Plover gets jumbled, especially when one stroke on the steno keyboard inserts more than one word at a time.
For example, on the steno keyboard, I press a combination of two keys, W and T.
This combination should translate as:
\n{^}WITNESS JAMES SMITH -{-|}
The string should create a new line, remove any leading spaces, insert the text, and capitalize the next character. However, instead of reading:
WITNESS JAMES SMITH -
I sometimes hear:
Wi JAMES SMITH TNESS -

Muting parts of the output
It isn’t necessary for the screen reader to vocalize all the text I type. For example, in an interrogation, before each question and answer, there’s often the phrasing:
QUESTION OF THE JUDGE -
ANSWER OF THE WITNESS, JAMES SMITH -
These are very common phrases, especially in tense interrogations, but they can interfere with the screen reader, slowing it down and increasing the delay between what I write and what I hear. In fact, I don’t really need these specific parts to be vocalized.
It would be helpful if Plover had the option to mute or unmute certain outputs. For instance, something like:
{Plover meta command to mute NVDA} QUESTION OF THE JUDGE - {Plover meta command to unmute NVDA}
This way, I could choose when to mute certain text that’s not essential for transcription.

These are the main issues I’ve encountered so far when using Plover and NVDA as a Court Reporter.

Daniele

[nvdaes]

Hi Daniele, happy to see you here. Perhaps this should be a discussion instead of an issue.
I'm not a court report but mainly a captioner in real time, using a platform so that my text is shown in internet, for example in Youtube. I've finished a session about 15 minutes ago, and I have a colleague that some part of the time read my texts and tells me if I'm making a frequent mistake. Also I configure NVDA to report bad typed words in document formatting, and sometimes in Firefox I listen a sound and then I check my braille display quickly.
Having a command to read a paragraph from Plover would be great, a paragraph or a line etc.
I would like to work on this but I don't have a machine at home and here in job I have some free time deppending on events. If I can, I will investigate about this too.

[nvdaes]

I'm thinking in a possible add-on trying to read the file where keystrokes are saved, so we eventually can consult what we wrote on demand. Tomorrow I may have some time. I know that Plover can be used with a conventional keyboard, but some combinations may not be recognized, so I prefer to test at job.

[nvdaes]

@ossossosso , for the punctuation issue, that is, that NVDA doesn't speak it, have you tried to install the reportSymbols add-on?
I'm testing now in Notepad setting NVDA to speak typed words, with this add-on installed, and punctuation like comma, numbers and other symbols are reported.
After installing the add-on, available in NVDA's store, you need to go to NVDA's menu, Preferences menu, settings, reportSymbol and mark the desired checkboxes, for example, to speak punctuation, check printable symbols.
I may provide more feedback while I perform more tests.

[nvdaes]

@ossossosso , in case this is helpful, I se that we can create commands in Plover like:

"Tpa*": "{#Insert(l)}". NVDA key is not recognized, but if handle keys from other applications is enabled, insert+l, using laptop keyboard in my case, will read the current line and we can handle this from the stenograph machine.
We can create multiple commands, for example, control+t, then control+v. I did this to insert the current time in transcriptions of pre-recorded audio. So we may create a keystroke to handle keys from other applications, then read the current line, word, etc, and then disable it.

[nvdaes]

@ossossosso , I've been successful creating a command to delete a word and then make NVDA to read the current line.
Personally, I use the undo macro to delete the last translated text, i.e., the whole text produced after a stroke, which may delete a whole sentence. In this case I haven't been suscessful. This may require a plugin for Plover itself, and I'm not sure if the result will be accurate or possible, maybe.
I have an event in an hour, so I won't mame more tests today, and tomorrow I'll be typing during about 3 or 4 hours, so I'll test the next week.
If something of this is helpful to you, let me know. I don't rely in speech output while typing since it disturbes me and I need to be focused just in speakers, just using the error sounds when something is not typing correctly.

[nvdaes]

Hi again, I think that an NVDA add-on would be better than a Plover plugin, since Plover plugins may handle keystrokes and translations, but if for any reason the text is replaced in the text processor, due to a macro or autofix, the real text may not be reported.
As mentioned, personally I don't rely on simultaneous audio from the screen reader. But if someone wants my help to create an NVDA add-on for plover, I may test again.