Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speech Input: General Implementation + Find Feature Completed #1346

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

NeilCresswell
Copy link

Adds general HTML5 speech input capability to ChronoZoom for browsers that are capable. Is safely ignored by browsers that are not capable. Also has option to enable/disable speech input in web config. Please note that this version defaults to disabled state in web.config for now since:

  1. This is only usable in Chrome at present, until IE12 launches.
  2. This first version is for demo/proof of concept purposes that can be expanded upon, and only the ability to search has been completed. Other verbal controls such as "next item" in a tour are stubbed but not implemented.

If you'd like to enable this on your personal copy of CZ to experiment (or demo) then this line should be added to your web.config:

<add key="SpeechInputEnabled" value="true" />

Uses general HTML5 speech input capability to drive verbal searches of ChronoZoom (fully implemented) using the find verb in the format "find *". For example, from the home page of CZ (anon user) you can say "find fish" and the view will switch to the canvas of timelines and the first timeline/exhibit/content item with "fish" in it will be located and you'll be navigated to that item. You can then search for something else such as "find humanity", "find Roman history" or "find cosmos". If an item can't be found, e.g. "find manatees" then you're told "Unable to find manatees".

Including now so work is not lost for potential future use. There was a discussion about possible finishing up a verbal tour navigation feature. The next item/previous item verbage for tours is already captured and passed to the appropriate empty functions (stubbed.) All that would be required would be to add code to make sure that a tour window was up and if so, fire the next or previous buttons. This is quite trivial to implement.

Can now say "find something" where something is any item you like
belonging to the current collection. e.g. "find fish", "find roman
history", "find cosmos", "find humanity". You will then be taken to the
canvas (if on home page view) and journeyed to the first item (timeline,
exhibit or content item) that matches what you're looking for. If the
item cannot be found, you'll be told. Also has option to toggle speech
input/control on/off via web config. Currently only works in Chrome. It
has been stated that IE12 will also support speech input so might be
worth installing the preview of IE12 to test.
@alterm4nn
Copy link
Owner

Nice work, @NeilCresswell!

While testing, I figured out some things that restrict me from publishing it to master branch.

  1. Voice control now works only in Chrome.
  2. Browser popup asking to use you microphone appears immediately after closing previous one (windows), and right after navigation to any element (mac os).
  3. There's no notification for user that voice input is being processed.
  4. Result is not obvious. 'find life' navigates me to 'sea life' content item, which event not appear in first page of searching results by word 'life'
  5. Since functionality is not completed, it's recommended to form it a feature (see https://github.com/alterm4nn/ChronoZoom/blob/master/Source/Chronozoom.UI/scripts/cz.js#L70).

Thanks!

@luuxx
Copy link

luuxx commented Mar 24, 2016

Can one of the admins verify this patch?

1 similar comment
@luuxx
Copy link

luuxx commented Jul 18, 2017

Can one of the admins verify this patch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants