Category Archives: Accessibility

Using speech recognition for passwords

By Kimberly Patch

I get a lot of questions about how to use speech recognition software for passwords.

Speech is inherently different from the keyboard because people can tell what your password is when you say it out loud. And when the password is unpronounceable you end up spelling it, which is both nonsecure and tedious.

I see a lot of people using the not-so-great solution of mapping a cryptic password to something pronounceable using the Dragon vocabulary manager or the Utter Command Enter List facility (UC Enter List lets you combine words with the Enter key). Neither method is very secure, because the mapping is in a utility that someone can simply look at.

The easiest good solution is to use what’s already there — check “Remember Password” on your browser and when you type your username the password will fill in automatically. Set a master password in your browser to protect the list of passwords (in Firefox click tools/options/security and check “Use a Master Password”).

Once you have Remember Password set up, put your username(s) in the Utter Command Enter List (say “Add Enter” to open the Enter List) and you’ll be able to say your username plus “Enter” in a single phrase. With “Remember Password” checked the password will fill in automatically and you’ll be able to log on using a single speech command.

Another good solution is a password manager like Roboform, which manages all your passwords (there’s a free version). All you need to do is enter a master password when you turn on your computer. Roboform also automatically fills in forms for you. It takes some set up, but in the end it makes things easier.

Dragon 11.5 free upgrade finally here

By Kimberly Patch

If you already use Dragon NaturallySpeaking 11, you’re entitled to a free upgrade to 11.5.

You must access the upgrade from within Dragon 11 — Click on “Help\ Check for Updates”, select “Dragon NaturallySpeaking 11.5” and follow the instructions to download the upgrade. Once it’s downloaded, click on the upgrade to install.

There’s more information here: nuance.custhelp.com/app/answers/detail/a_id/6213

We strongly recommend upgrading. The upgrade fixes several bugs and gives you the ability to use your iPhone as a wireless microphone over a wifi network.

Utter Command is fully compatible with Dragon 11.5.

Quick Hotmail control by speech

By Kimberly Patch

I got a question today about controlling Hotmail by speech. Here’s a short answer.

The good news about web programs is more of them now have keyboard shortcuts, and more of the shortcuts are standard. This makes it easier to use speech control without customization.

Do a web search for “Hotmail keyboard shortcuts” and you’ll find several lists. Here’s one from about.com.

Hotmail has a pretty good set of shortcuts, including some defacto standards. Utter Command e-mail commands like “New Message”, “This Reply”, “Reply All” and “This Save” work in Hotmail because Hotmail uses the common shortcuts for these functions (see UC Lesson 8.3).

For the less standard functions you can speak keyboard. Here are a few that are particularly useful:
“Letter fi” or “Foxtrot India” goes to the Inbox folder
“Letter fs” or “Foxtrot Sierra” goes to the Sent folder
“Control Dot” goes to the next message
“Control Comma” goes to the previous message
“Control Enter” sends

Tip: Make sure to say “Shift” before “Control “ if you use any of the shift control commands.

Probably the best way to control drop-down menus that you use frequently in web programs is to use the naming-a-mouse-click ability (see UC Lesson 10.24). You can say two mouse clicks in a row to control a drop-down using a single speech command.

Google+: will keyboard access improve beyond jk Enter Tab?

It’s obvious that Google+ is a powerful tool for personal and business use, and that we’ll see much more of it as time goes on. Circles are a brilliant way to organize contacts and share information, and hangouts makes video conferencing easy. What’s not to like?

Well, have you ever been in a situation where you’re looking at food that looks great and smells great and you’re hungry and would very much like to have some, but there’s some reason you can’t?

I have to give Google+ very mixed reviews on accessibility.

The more important a tool like Google+ turns out to be, the more important that everyone have access to it — including a couple of large communities who find it difficult or impossible to use a mouse: folks who have repetitive strain injuries and folks who are blind.

The good news is it’s relatively simple to make a program accessible to everyone: include keyboard controls. Better yet, provide a way to reconfigure keyboard shortcuts and share configurations. Enable the keyboard, and alternative controls like speech recognition can translate keyboard shortcuts to give users whatever type of access they need.

So how accessible is Google+ on this basic — keyboard shortcuts?

I couldn’t easily find shortcuts documentation.

So I tried some things out.

It failed on one basic requirement — you can’t use the arrow keys to move up and down conversations and drop-down menu items.

There’s a little good news, however.
– You can use the the “j” and “k” keys to move between conversations, just like in Gmail.
– The Enter key sets you up to write a comment.
– And a combination of the right number of Tabs and Enter lets you post, delete or cancel from the comment field.

So if you use Dragon plus Utter Command speech software to control the computer you can say “Letter j” or “Letter k” to move back or forward through entries. You can also skip forward or back in a single utterance, e.g. “k Times 3”. And you can open a comment field by saying “Enter”, and, once the comment is in, say, “Tab Enter” to post it, or “2 Tab Enter” to cancel it.

But that’s about it for useful direct keyboard control.

If you use the Firefox Mouseless browsing extension, you can go directly to most elements for a small penalty — numbers taking up space on your screen.

And if you use Dragon’s speak links ability you can click links and buttons that contain words, but this breaks down with less pronounceable names like +1’s and unpronounceable things like the home, pictures, profile and circles icons. Dragon’s speak links ability is also a little fragile — it’s all too easy to accidentally say a word that clicks a link.

And both solutions require you to identify something by sight before you take action, which can make things slower or a showstopper depending on your abilities. This is where keyboard shortcuts should be filling in the blanks.

Given the mixed situation, the easiest way for Dragon plus Utter Command users to access commonly clicked items like the search field, unpronounceable icons like profile, and unpronounceable symbols like the little drop-down list on the top right corner of each entry might be to use the UC Touch List to set up named mouse clicks. It takes a little set up, but will get you to the meal in the end.

Google+ clearly needs more keyboard shortcuts.

Better yet, how about a tool to allow us to easily configure keyboard shortcuts in Google+? Or even better, how about a tool to allow us to easily configure keyboard shortcuts across Google products? This would allow more people into the circle. It also has a lot of potential to improve the experience for folks who are already in.

It’s obvious that Google+ is a powerful tool for personal and business use. We’ll see much more of it as time goes on. Circles are a brilliant way to organize contacts and share information, and hangouts makes video conferencing easy. What’s not to like?

I have to give Google+ very mixed reviews on accessibility.

So how accessible is Google+ on this basic — keyboard shortcuts?

I couldn’t easily find shortcuts documentation.

So I tried some things out.

It failed on one basic requirement — you can’t use the arrow keys to move up and down conversations and drop-down menu items.

But that’s about it for useful direct keyboard control.

If you use the Firefox Mouseless browsing extension, you can go directly to most elements for a small penalty — numbers taking up space on your screen. And if you use Dragon’s speak links ability you can click links and buttons that contain words, but this breaks down with less pronounceable names like +1’s and unpronounceable things like the home, pictures, profile and circles icons. Dragon’s speak links ability is also a little fragile — it’s all too easy to accidentally say a word that clicks a link. And both solutions require you to identify something by sight before you take action, which can make things slower or a showstopper depending on your abilities. This is where keyboard shortcuts should be filling in the blanks.

Given the mixed situation, the easiest way for Dragon plus Utter Command users to access commonly clicked items like the search field and profile buttons might be to use the UC Touch List to set up named mouse clicks. It takes a little set up, but will get you to the meal in the end.

Google+ clearly needs more keyboard shortcuts.

7/14/11
All too often software vendors act like they’re the only ones with a software vendor relationship with the user.
it’s no big deal if an update automatically downloads every once in awhile. It is a big deal if you use 20 pieces of software and an update from each downloads every once in awhile.
This is why standards or good user control is important for communicating with software.

Posting to Word Press by speech

I get a lot of inquiries about how I carry out particular computer tasks by speech.

Here are the gory details on what I do to write a blog item and post it to WordPress:

Getting ready to write

When I think of an idea for a Patch on Speech blog post I say
– “Blog Pending Site” to bring up the Google document I write the blog in. Then I say
– “Find Mark 1”, then “Another Graph” to position the cursor. I have “MARK 1” written at the top of my working section. The first command selects “MARK 1”, and the second one positions the cursor two lines below it at the top of the section. Then I say
– “Today Short Enter” to add the current date and move the cursor to the next line

Writing

I either jot down an idea, or write a whole post.

When I’m writing I make heavy use of “1-20 Befores” to select the last few words I said and change them. A key point about this technique is I don’t count how many words I want to select back. I just make sure to select more words than I need to change, then look to see what is selected and resay what I need to.

I also make use of the Dragon inline commands, which allow you to say punctuation like “Open Quote” and “New Paragraph” without pausing. I use “Another Graph” to start a new paragraph when I’m not at the very end of a line. I occasionally find myself speaking keyboard to fix something, for instance “Left Backspace Right” to correct “two” to “to”.

We’ve just been testing a series of commands that lets you use a mouse without clicking, and I’ve been experimenting with commands like “Touch Word” and “Touch 3 Words” to select text.

Posting

After I’ve written and edited a piece, I say
– “Find Mark 1”, then “2 Down Home” to put the cursor at the beginning of the headline
Then I use several “1-100 Up\Downs” commands combined with a copy command to select the story, e.g. “50 Downs”, “20 Downs”, “5 Ups Copy”

Then I open the page where I post by saying
– “WordPress Site”
If I’m not already logged on it prompts me for my username. I have my username in the UC Enter list so I can say it and hit the Enter key in one utterance. Since my password is stored I can login in a single utterance:
“<username> Enter”
Once I’m in I say
– “31 Go” to click the “New Post” link
– “Tab Paste” to tab to the body field and paste the text
– “Go Top” to move the cursor to the top of the file
– “Line Cut” to cut the headline
– “2 Delete” to remove the extra lines
– “49 Go” to move to the headline field
– “This Paste” to paste the headline

Categories and Publish

I add categories using the Go numbers, one or two at a time , e.g. “31 Go” to add one category and “38 Go 41 Go” to add two categories in a single utterance, and use a Go number to hit the “Preview” button.

Then I look over the post, say “Doc Close” to close the preview, and use a Go number to hit “Publish”.

Avoid having to remember commands

This way you don’t have to remember commands. Eventually, after using the guide a bunch of times, you’ll have the sequence memorized without having to consciously memorize it.

If you have a way of carrying out a task by speech that you’re particularly proud of — or if there’s something you’re struggling with — drop me a line at kim @ this web address.

I get a lot of inquiries into how I carry out particular computer tasks by speech.

Here are the gory details on what I do to write a blog item and post it to WordPress.

Getting ready to write

When I think of an idea for a Patch on Speech blog post I say

– “Blog Pending Site” to bring up the Google document I write the blog in. Then I say

– “Find Placeholder”, then “Another Graph” to position the cursor. I have “MARK 1” written at the top of my working section. The first command selects “MARK 1”, and the second command positions the cursor two lines below it, so the new ideas are always at the top of the section. Then I say

– “Today Short Enter” to add the current date and move the cursor to the next line

Writing

I either jot down an idea, or write a whole post.

When I’m writing I make heavy use of “1-20 Befores” to select the last few words I said and change them. A key point about this technique — I don’t count how many words I want to select back — I just make sure to go over the number I want to change, then I look to see what is selected and resay what I need to. I also make use of the Dragon Inline commands, which allow you to say punctuation like “Open Quote” and “New Paragraph” without pausing. I use “Another Graph” to start a new paragraph when I’m not at the very end of a line. I occasionally find myself speaking keyboard to fix something, for instance “Left Backspace Right” to correct “two” to “to”. We’ve just been testing out a series of commands that lets you use a mouse device without clicking, and I’ve found that commands like “Touch Word”and ”

Posting

After I’ve written and edited a piece, I select the blog text and say

– “Copy to 1 File” to copy story to the use the clipboard “1 File” so I can paste it later

– “2 Up” to unselect and put the cursor on the headline, and

– “Line Copy” to copy the headline

Once I have the blog and headline loaded up, I open the page where I post by saying

– “Word Press Site”

If I’m not already logged on it it prompts me for my username. I have my username in the UC Enter list so I can say it and hit the Enter key in one utterance. Since my password is stored This is all I need to say to login:

“<username> Enter”

Once I’m in I say

– “31 Go” to click the “post” link

– “Paste Tab” to paste the headline and tab to the next field

– “1 File Paste” to paste the blog text.

I think the key to enabling a program for efficient speech control is to take the time to look at what you want to do in detail and plot it out — take the time to write out the steps. Make a game of figuring out just how efficient you can be. Then take the steps and put them in one of the UC custom guides, so you can call up instantly and simply read the set of commands to carry out the task, e.g. “Custom 3 Guide”. This way you don’t have to remember commands. Eventually, from the repetition and saying and picturing the commands in the guide, you’ll have the memorized. But you won’t have to spend extra energy while you’re trying to do your work memorizing them.

If you have a way of carrying out a task by speech that you’re particularly proud of — or if there’s something you’re struggling with — drop me a line.

Long web documents at a glance

There are two ways to speed up a computer task: Carry out the same steps you always have, but go faster, or find an easy-to-use tool that requires fewer steps.

If you need to navigate through long documents on the Internet — papers, standards documents, patent documents etc. — the Firefox HeadingsMap extension will save you a lot of time. It lets you navigate using a map of the headings in a document. The headings map also gives you a great overview — a quick mental map of the document. It works especially well with speech. And it shows errors in headings, which is useful when you’re putting together a long document.

HeadingsMap shows up as a small symbol containing an “h” on the Status bar at the bottom left corner of the Firefox window. If your Firefox window is maximized the “h” appears immediately above the “Start” button.

Click the “h” and a narrow window appears on the left containing all the headings and subheadings in a document. Click the “h” again and the window disappears. Right-click on the “h” and you’re presented with configuration options. I usually uncheck the “levels” checkbox, which makes the headings map a little cleaner looking.

In general, there are three different ways to navigate among items on tree views like the headings map:
– the mouse
– the Up/Down arrows
– the letter keys

The most efficient way to implement letter keys navigation is to let the user type more than one key of a selection, say “d o” to select “dove” rather than “dinosaur”. A less efficient way is to treat every letter as a new navigation event and jump to the next instance beginning with that letter.

Fortunately, HeadingsMap has implemented all three methods, including the efficient letter key method.This method works especially well with speech because you simply say the whole word to navigate to it, e.g. “dove”.

You can download the HeadingsMap extension here: addons.mozilla.org/en-us/firefox/addon/headingsmap/

And here are a couple of especially long documents you can try it out on:
A paper on the effects of climate change on birds from the Public Library of Science:
www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000585
The public draft of a World Wide Web Consortium standards document:
www.w3.org/TR/2010/WD-UAAG20-20100617/

Friday Tip: Creative New Line

Although you can use the Utter Command Folders List utility to directly open any folder on your folders list using a single command, you’re also likely to at least occasionally navigate ad-hoc through the file system. A common way to navigate is to say a folder name to navigate to the folder (e.g. “Financials”), say “Enter” to go into the folder, say the name of a subfolder (e.g. “Budget”), then say “Enter” to go into the subfolder, etc.

Here’s a tip from Jacob Cole, an MIT student I’ve been training on Utter Command.

Navigate folders using the Dragon “New Line” in-line command, which was originally conceived as a text command. In-line commands are used within a text phrase. They’re mostly punctuation marks like “comma”. “New Line” is a little different. It literally hits the Enter key to give you a new line. The classic “New Line” example is saying a grocery list without pausing between lines, e.g. “Avocados New Line Eggs New Line Flour”.

Jacob pointed out that you can also use “New Line” to reduce the number of phrases you have to say when navigating through folders. Using the above example, instead of having to say four separate utterances to go two folder layers deep, you can say “Financials New Line” to navigate to the financial folder and go into it, then “Budget New Line” to navigate to the budget folder and go into it. Or even “Financials New Line Budget New Line”.

Happy navigating.

Have any good tips or pet peeve’s about using speech input? Let me know at info@ this website address.

Tip: Easier scrolling with mouse-speech combination

If you use a mouse to scroll, have you noticed how much fine motor control you use to keep the arrow on the scroll bar as you move the page? You’re doing a fair bit of work to do this. It’s akin to keeping on a balance beam.

If you can move your mouse, you can use an Utter Command touch/speech combination that’ll show you just how hard you have to work to use just the mouse to control the scroll bar.

Next time you use the mouse to scroll, place the mouse arrow on the scroll bar, then say “Touch Hold”. This command holds down the left mouse button. Now you can scroll by simply moving the mouse up and down. There’s no need to click, and there’s no need to keep inside the narrow confines of the width of the scroll bar. This command is especially effective when you’re reading and can leave the left mouse button down between moves. It’s also especially effective when you’re skimming quickly through a document — you can concentrate more on what you’re reading because there’s no need to take your eyes off it to make sure the mouse is on the scroll bar. When you’re done using this command make sure to release the mouse button: “Touch Release”.

You can use the same method in a drawing program to draw without having to have the pen touch the tablet.

There are more details on the “Touch Hold/Release” command in UC Lesson 4.5.

Keep in mind that the Touch Hold/Release method is one of several ways to control the scroll bar using Utter Command — if the combination is comfortable for you it’s a good one. If you need to be completely hands-free, see UC Lesson 1.8, which details all the ways you can use speech to navigate documents, and UC Lesson 9.5, which details Web navigation.

Happy navigating.

Discover, Adjust, Organize and Share

By Kimberly Patch

Keyboard shortcuts have a lot of potential. They’re fast.

For example, cutting and pasting by

– Hitting “Control x”
– Moving the cursor to the paste location
– Then hitting “Control v”

is speedier than

– Moving the mouse to the “Edit” menu
– Clicking “Edit ”
– Clicking “Cut”
– Moving the cursor to the paste location
– Moving back up to click “Edit ”
– Then clicking “Paste”.

Add this up over many tasks and you have a big difference in productivity.

So why don’t we see more people using keyboard shortcuts?

Ask someone who uses the mouse for just about everything and you’re likely to get a compelling answer — it’s easier. And it is — it’s cognitively easier to choose a menu item than to remember a shortcut.

Given a choice, people generally do what’s easier. On a couple different occasions I’ve heard people say that, all else being equal, they’d hire a blind programmer over a sighted one because the blind programmer is faster. The blind programmer must use keyboard shortcuts.

This is a common theme — we have something potentially better, but human behavior stands in the way of adoption.

In the case of keyboard shortcuts there’s a little more to the story, however.

As a software community we haven’t implemented keyboard shortcuts well.

Many folks know keyboard shortcuts for a few very common actions like cut, paste and bold, but it’s more difficult to come up with keyboard shortcuts for actions like adding a link or a hanging indent because they are used less often and are less likely to be the same across programs.

So the user is often stuck with different shortcuts for the same tasks in different programs, requiring him to memorize and keep track of multiple sets of controls. This is cognitively difficult for everyone, and more so for some disabled populations and the elderly.

This type of implementation is akin to asking someone to speak different languages depending on who they are speaking to. Depending on how motivated and talented they are, some folks may be able to do it, but not many. And if there’s an easier way, even those capable of doing it either way will often choose easier even if it’s less efficient.

So we aren’t letting keyboard shortcuts live up to their potential.

There’s a second keyboard shortcuts issue that’s getting worse as Web apps become more prevalent: clashing shortcuts. If you hit “Control f” in a Google document, do you get the Google Find facility or the browser Find facility? Go ahead and try it out. It’s messy.

This is already an issue in the assistive technology community, where people who require alternate input or output must use software that runs all the time in conjunction with everything else. For example, a speech engine must be on all the time listening for commands, and screen magnifier software must be running all the time to enlarge whatever you’re working in.

So there are two problems: keyboard shortcuts aren’t living up to their potential to increase efficiency, and, especially on the Web, keyboard shortcuts are increasingly likely to clash.

I think there’s a good answer to both problems: a cross-program facility to easily discover, adjust, organize and share shortcuts.

– We need to easily discover shortcuts in order to see them all at once so we can see patterns across programs and conflicts in programs/apps that may be opened at once.

– We need to easily adjust shortcuts so we can choose common shortcuts and avoid clashes. We need to organize so we can remember what we did.

– We need to easily arrange commands and add headings so we can find commands quickly and over time build a good mental map of commands.. Lack of ability to organize is the Achilles’ heel of many macro facilities. It’s like asking people to play cards without being able to rearrange the cards in their hand. It’s possible, but unless there’s a reason for it, makes things unnecessarily difficult.

– We need to share the adjustments because it makes us much more efficient as a community. My friend Dan, for instance, is very logical. He uses many of the same programs I do, and we both use speech input. So if there were a facility to discover, adjust, organize and share keyboard shortcuts, I’d look to see if Dan had posted his changes, and I would adjust to my needs from there.

The organizing and sharing parts are the most important, because they allow for crowdsourcing.

Over the past few decades the computer interface ecosystem has shifted from single, unrelated programs to separate programs that share information, to programs so integrated that users may not know when they are going from one to another. This has increased ease-of-use and efficiency but at the same time complicated program control.

At the same time programs have grown more sophisticated. There’s a lot of wasted potential in untapped features.

If we give users the tools to discover, adjust, organize and share, I bet we’ll see an increase in speed and efficiency and an uptick in people discovering nifty new program features.

Share this post:

Share this post:

Share this post:

Share this post:

Share this post:

Share this post:

Share this post:

Share this post:

Share this post:

Share this post: