Category Archives: Keyboard shortcuts

Getting Gmail working well with speech commands

By Kimberly Patch

If you haven’t used speech commands to control a computer, it might not be obvious that single character commands, for instance “y” to archive a message in Gmail, can present a challenge.

Single-character commands seem like a great idea, especially for Web programs, because your Web browser already takes up some common keyboard shortcuts. Gmail has a lot of single-character commands, and once you get to know them you can fly along using the keyboard. In general I’m all for more keyboard shortcuts because it’s easy to enable them using speech.

Command conundrum

Single-character commands that can’t be changed, however, can get speech users in a lot of trouble. Say a command or make a noise that’s misheard as text in a program that doesn’t use single-character shortcuts and either nothing happens or you get some stray text you can easily undo. Do the same thing in a single-character-command program and you can cause many actions to happen at once.

A stray “Kelly” in your Gmail inbox, for instance, will move the cursor up one message (single-character command “k”) and archive it (single-character command “y”). “Bruno” causes even more damage.

Turn off the keyboard shortcuts, though, and the program becomes fairly inaccessible for speech users. We need the shortcuts, and we can combine multiple keystrokes into single utterances to make things even better. It’s having little control over them that presents a problem.

Speech-safe single character shortcuts

Google Labs has a nifty extension that presents a simple fix. It lets you change the characters you use for keyboard shortcuts, including using two characters rather than one. Add a plus sign (+) to the beginning of every shortcut and they all become speech-safe.

Here are step-by-step instructions.
– go to your Gmail account, click the settings gear icon at the top right of the screen
– click “Labs”
– search for the “Custom Keyboard Shortcuts” extension and click to download. This will add a ”Keyboard Shortcuts” tab to your Gmail settings
– now, click the settings gear icon at the top right of the screen
– click Keyboard Shortcuts
– add “+“ to the beginning of every command

If you’re using Utter Command 2.0 you’re now all set. Say “Plus” and any one- or two-character command. Say, for instance “Plus j” or “Plus Juliet” to move down one item. You can also say a command multiple times in a single utterance. Say “Plus j Repeat 5” to move down five items, for instance. And you can combine two commands: “Plus j Plus y” moves down one item, then archives that item (say “Question Mark” to call up the keyboard shortcuts list.)

Raising the bar

The Google Labs add-on enables Gmail for speech users, but there are many other programs out there that use single-character shortcuts, including other Google programs, and other Web-based programs like Twitter. Message for Google: How about one facility that would let us control keyboard shortcuts across Google programs?

It would also improve things if we could have a larger number of characters available for a given character shortcut, the ability to also control control-key shortcuts, the ability to save and share different sets, and the ability to apply at least some shortcuts across applications

Important Note: If you were a beta tester or received the Utter Command 2.0 pre-release, you might not have the “Plus” set of commands. If this is the case, send e-mail to “Info” at this web address, and we’ll make sure you have the release version. The release version shows 15 new sets of commands on the “New commands for 2.0” list you can open from the Taskbar icon menu.

Tips, tricks, productivity, accessibility, usability and all things speech recognition.

Quick Hotmail control by speech

By Kimberly Patch

I got a question today about controlling Hotmail by speech. Here’s a short answer.

The good news about web programs is more of them now have keyboard shortcuts, and more of the shortcuts are standard. This makes it easier to use speech control without customization.

Do a web search for “Hotmail keyboard shortcuts” and you’ll find several lists. Here’s one from about.com.

Hotmail has a pretty good set of shortcuts, including some defacto standards. Utter Command e-mail commands like “New Message”, “This Reply”, “Reply All” and “This Save” work in Hotmail because Hotmail uses the common shortcuts for these functions (see UC Lesson 8.3).

For the less standard functions you can speak keyboard. Here are a few that are particularly useful:
“Letter fi” or “Foxtrot India” goes to the Inbox folder
“Letter fs” or “Foxtrot Sierra” goes to the Sent folder
“Control Dot” goes to the next message
“Control Comma” goes to the previous message
“Control Enter” sends

Tip: Make sure to say “Shift” before “Control “ if you use any of the shift control commands.

Probably the best way to control drop-down menus that you use frequently in web programs is to use the naming-a-mouse-click ability (see UC Lesson 10.24). You can say two mouse clicks in a row to control a drop-down using a single speech command.

Google+: will keyboard access improve beyond jk Enter Tab?

It’s obvious that Google+ is a powerful tool for personal and business use, and that we’ll see much more of it as time goes on. Circles are a brilliant way to organize contacts and share information, and hangouts makes video conferencing easy. What’s not to like?

Well, have you ever been in a situation where you’re looking at food that looks great and smells great and you’re hungry and would very much like to have some, but there’s some reason you can’t?

I have to give Google+ very mixed reviews on accessibility.

The more important a tool like Google+ turns out to be, the more important that everyone have access to it — including a couple of large communities who find it difficult or impossible to use a mouse: folks who have repetitive strain injuries and folks who are blind.

The good news is it’s relatively simple to make a program accessible to everyone: include keyboard controls. Better yet, provide a way to reconfigure keyboard shortcuts and share configurations. Enable the keyboard, and alternative controls like speech recognition can translate keyboard shortcuts to give users whatever type of access they need.

So how accessible is Google+ on this basic — keyboard shortcuts?

I couldn’t easily find shortcuts documentation.

So I tried some things out.

It failed on one basic requirement — you can’t use the arrow keys to move up and down conversations and drop-down menu items.

There’s a little good news, however.
– You can use the the “j” and “k” keys to move between conversations, just like in Gmail.
– The Enter key sets you up to write a comment.
– And a combination of the right number of Tabs and Enter lets you post, delete or cancel from the comment field.

So if you use Dragon plus Utter Command speech software to control the computer you can say “Letter j” or “Letter k” to move back or forward through entries. You can also skip forward or back in a single utterance, e.g. “k Times 3”. And you can open a comment field by saying “Enter”, and, once the comment is in, say, “Tab Enter” to post it, or “2 Tab Enter” to cancel it.

But that’s about it for useful direct keyboard control.

If you use the Firefox Mouseless browsing extension, you can go directly to most elements for a small penalty — numbers taking up space on your screen.

And if you use Dragon’s speak links ability you can click links and buttons that contain words, but this breaks down with less pronounceable names like +1’s and unpronounceable things like the home, pictures, profile and circles icons. Dragon’s speak links ability is also a little fragile — it’s all too easy to accidentally say a word that clicks a link.

And both solutions require you to identify something by sight before you take action, which can make things slower or a showstopper depending on your abilities. This is where keyboard shortcuts should be filling in the blanks.

Given the mixed situation, the easiest way for Dragon plus Utter Command users to access commonly clicked items like the search field, unpronounceable icons like profile, and unpronounceable symbols like the little drop-down list on the top right corner of each entry might be to use the UC Touch List to set up named mouse clicks. It takes a little set up, but will get you to the meal in the end.

Google+ clearly needs more keyboard shortcuts.

Better yet, how about a tool to allow us to easily configure keyboard shortcuts in Google+? Or even better, how about a tool to allow us to easily configure keyboard shortcuts across Google products? This would allow more people into the circle. It also has a lot of potential to improve the experience for folks who are already in.

It’s obvious that Google+ is a powerful tool for personal and business use. We’ll see much more of it as time goes on. Circles are a brilliant way to organize contacts and share information, and hangouts makes video conferencing easy. What’s not to like?

Well, have you ever been in a situation where you’re looking at food that looks great and smells great and you’re hungry and would very much like to have some, but there’s some reason you can’t?

I have to give Google+ very mixed reviews on accessibility.

The more important a tool like Google+ turns out to be, the more important that everyone have access to it — including a couple of large communities who find it difficult or impossible to use a mouse: folks who have repetitive strain injuries and folks who are blind.

The good news is it’s relatively simple to make a program accessible to everyone: include keyboard controls. Better yet, provide a way to reconfigure keyboard shortcuts and share configurations. Enable the keyboard, and alternative controls like speech recognition can translate keyboard shortcuts to give users whatever type of access they need.

So how accessible is Google+ on this basic — keyboard shortcuts?

I couldn’t easily find shortcuts documentation.

So I tried some things out.

It failed on one basic requirement — you can’t use the arrow keys to move up and down conversations and drop-down menu items.

There’s a little good news, however.
– You can use the the “j” and “k” keys to move between conversations, just like in Gmail.
– The Enter key sets you up to write a comment.
– And a combination of the right number of Tabs and Enter lets you post, delete or cancel from the comment field.

So if you use Dragon plus Utter Command speech software to control the computer you can say “Letter j” or “Letter k” to move back or forward through entries. You can also skip forward or back in a single utterance, e.g. “k Times 3”. And you can open a comment field by saying “Enter”, and, once the comment is in, say, “Tab Enter” to post it, or “2 Tab Enter” to cancel it.

But that’s about it for useful direct keyboard control.

If you use the Firefox Mouseless browsing extension, you can go directly to most elements for a small penalty — numbers taking up space on your screen. And if you use Dragon’s speak links ability you can click links and buttons that contain words, but this breaks down with less pronounceable names like +1’s and unpronounceable things like the home, pictures, profile and circles icons. Dragon’s speak links ability is also a little fragile — it’s all too easy to accidentally say a word that clicks a link. And both solutions require you to identify something by sight before you take action, which can make things slower or a showstopper depending on your abilities. This is where keyboard shortcuts should be filling in the blanks.

Given the mixed situation, the easiest way for Dragon plus Utter Command users to access commonly clicked items like the search field and profile buttons might be to use the UC Touch List to set up named mouse clicks. It takes a little set up, but will get you to the meal in the end.

Google+ clearly needs more keyboard shortcuts.

Better yet, how about a tool to allow us to easily configure keyboard shortcuts in Google+? Or even better, how about a tool to allow us to easily configure keyboard shortcuts across Google products? This would allow more people into the circle. It also has a lot of potential to improve the experience for folks who are already in.

7/14/11
All too often software vendors act like they’re the only ones with a software vendor relationship with the user.
it’s no big deal if an update automatically downloads every once in awhile. It is a big deal if you use 20 pieces of software and an update from each downloads every once in awhile.
This is why standards or good user control is important for communicating with software.

Long web documents at a glance

There are two ways to speed up a computer task: Carry out the same steps you always have, but go faster, or find an easy-to-use tool that requires fewer steps.

If you need to navigate through long documents on the Internet — papers, standards documents, patent documents etc. — the Firefox HeadingsMap extension will save you a lot of time. It lets you navigate using a map of the headings in a document. The headings map also gives you a great overview — a quick mental map of the document. It works especially well with speech. And it shows errors in headings, which is useful when you’re putting together a long document.

HeadingsMap shows up as a small symbol containing an “h” on the Status bar at the bottom left corner of the Firefox window. If your Firefox window is maximized the “h” appears immediately above the “Start” button.

Click the “h” and a narrow window appears on the left containing all the headings and subheadings in a document. Click the “h” again and the window disappears. Right-click on the “h” and you’re presented with configuration options. I usually uncheck the “levels” checkbox, which makes the headings map a little cleaner looking.

In general, there are three different ways to navigate among items on tree views like the headings map:
– the mouse
– the Up/Down arrows
– the letter keys

The most efficient way to implement letter keys navigation is to let the user type more than one key of a selection, say “d o” to select “dove” rather than “dinosaur”. A less efficient way is to treat every letter as a new navigation event and jump to the next instance beginning with that letter.

Fortunately, HeadingsMap has implemented all three methods, including the efficient letter key method.This method works especially well with speech because you simply say the whole word to navigate to it, e.g. “dove”.

You can download the HeadingsMap extension here: addons.mozilla.org/en-us/firefox/addon/headingsmap/

And here are a couple of especially long documents you can try it out on:
A paper on the effects of climate change on birds from the Public Library of Science:
www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1000585
The public draft of a World Wide Web Consortium standards document:
www.w3.org/TR/2010/WD-UAAG20-20100617/

Thunderbird tabs and consistency

Thunderbird now has tabs for open messages, which is very convenient. You can have three messages open and see where they are from the tabs — this is similar to tabbed browsing in programs like Firefox and Internet Explorer. And you can move among tabs using the same commands you use to move among tabs in your browser: “Tab Back”, “Tab Forward” and “1-20 Tab Back/Forward”.

Unfortunately, however, the keyboard shortcut to close a message tab is different from the standard close document/tab command used in most programs including Firefox, even though Thunderbird is developed by the same organization as Firefox. The usual command “Control Function 4” logically mirrors the common “Alternate Function 4” that’s used to close a window.

If the standard keyboard shortcut were enabled like it is in programs like Microsoft Word and Firefox, you could say the shortcut or “Document Close” to close a document or tab. And if you wanted to close more than one you could say “Document Close Times 3”, for instance.

If you dig through the keyboard shortcuts for Thunderbird, you’ll find that there is a nonstandard keyboard shortcut to close a message tab: “Control w”. So you can train yourself to say “Control w” to close a message when you’re in Thunderbird. Also keep in mind you can also say “Control w Times 3” to close three open messages. But it would be far better to not have to think about which program you are in when closing a tab or document. Feel free to complain to Thunderbird about this oversight at the Thunderbird support forum.

Here’s another Thunderbird tip: If you want to move a message rather than just closing it try “Move Recent”, “1-10 Down Enter”.
There’s more Thunderbird strategy on the Redstart Wiki: http://redstartsystems.com/Wikka/wikka.php?wakka=UCandThunderbird

Discover, Adjust, Organize and Share

By Kimberly Patch

Keyboard shortcuts have a lot of potential. They’re fast.

For example, cutting and pasting by

– Hitting “Control x”
– Moving the cursor to the paste location
– Then hitting “Control v”

is speedier than

– Moving the mouse to the “Edit” menu
– Clicking “Edit ”
– Clicking “Cut”
– Moving the cursor to the paste location
– Moving back up to click “Edit ”
– Then clicking “Paste”.

Add this up over many tasks and you have a big difference in productivity.

So why don’t we see more people using keyboard shortcuts?

Ask someone who uses the mouse for just about everything and you’re likely to get a compelling answer — it’s easier. And it is — it’s cognitively easier to choose a menu item than to remember a shortcut.

Given a choice, people generally do what’s easier. On a couple different occasions I’ve heard  people say that, all else being equal, they’d hire a blind programmer over a sighted one because the blind programmer is faster. The blind programmer must use keyboard shortcuts.

This is a common theme  — we have something potentially better, but human behavior stands in the way of adoption.

In the case of keyboard shortcuts there’s a little more to the story, however.

As a software community we haven’t implemented keyboard shortcuts well.

Many folks know keyboard shortcuts for a few very common actions like cut, paste and bold, but it’s more difficult to come up with keyboard shortcuts for actions like adding a link or a hanging indent because they are used less often and are less likely to be the same across programs.

So the user is often stuck with different shortcuts for the same tasks in different programs, requiring him to memorize and keep track of multiple sets of controls. This is cognitively difficult for everyone, and more so for some disabled populations and the elderly.

This type of implementation is akin to asking someone to speak different languages depending on who they are speaking to. Depending on how motivated and talented they are, some folks may be able to do it, but not many. And if there’s an easier way, even those capable of doing it either way will often choose easier even if it’s less efficient.

So we aren’t letting keyboard shortcuts live up to their potential.

There’s a second keyboard shortcuts issue that’s getting worse as Web apps become more prevalent: clashing shortcuts. If you hit “Control f” in a Google document, do you get the Google Find facility or the browser Find facility? Go ahead and try it out. It’s messy.

This is already an issue in the assistive technology community, where people who require alternate input or output must use software that runs all the time in conjunction with everything else. For example, a speech engine must be on all the time listening for commands, and screen magnifier software must be running all the time to enlarge whatever you’re working in.

So there are two problems: keyboard shortcuts aren’t living up to their potential to increase efficiency, and, especially on the Web, keyboard shortcuts are increasingly likely to clash.

I think there’s a good answer to both problems: a cross-program facility to easily discover, adjust, organize and share shortcuts.

– We need to easily discover shortcuts in order to see them all at once so we can see patterns across programs and conflicts in programs/apps that may be opened at once.

– We need to easily adjust shortcuts so we can choose common shortcuts and avoid clashes. We need to organize so we can remember what we did.

– We need to easily arrange commands and add headings so we can find commands quickly and over time build a good mental map of commands.. Lack of ability to organize is the Achilles’ heel of many macro facilities. It’s like asking people to play cards without being able to rearrange the cards in their hand. It’s possible, but unless there’s a reason for it, makes things unnecessarily difficult.

– We need to share the adjustments because it makes us much more efficient as a community. My friend Dan, for instance, is very logical. He uses many of the same programs I do, and we both use speech input. So if there were a facility to discover, adjust, organize and share keyboard shortcuts, I’d look to see if Dan had posted his changes, and I would adjust to my needs from there.

The organizing and sharing parts are the most important, because they allow for crowdsourcing.

Over the past few decades the computer interface ecosystem has shifted from single, unrelated programs to separate programs that share information, to programs so integrated that users may not know when they are going from one to another. This has increased ease-of-use and efficiency but at the same time complicated program control.

At the same time programs have grown more sophisticated. There’s a lot of wasted potential in untapped features.

If we give users the tools to discover, adjust, organize and share, I bet we’ll see an increase in speed and efficiency and an uptick in people discovering nifty new program features.

Speaking to Excel

I’ve gotten a lot of inquiries lately about using speech recognition in Excel.

The fastest way to learn to apply Utter Command to Excel is to read UC Lesson 10.9: Navigating, numbers, functions, selecting and formatting in tables and spreadsheets, and UC Lesson 10.10: Putting it all together in any program (say “UC Lesson 10 Point 9” and “UC Lesson 10 Point 10” to call them up). Then take a look at the Top Excel Guide, which opens a list of useful shortcuts along the right edge of your screen.

Here are some basics:

  • “Cell” followed by a letter and number jumps to any cell, e.g. “Cell B 2” or “Cell Bravo 2”
  • “Control Space” selects the row the cursor is on
  • “Shift Space” selects the column the cursor is on

Here are some particularly useful combinations:

  • A number followed by a direction selects cells — keep in mind you can select in two directions at once, e.g. “3 Rights · 5 Downs” to select 3 columns to the right and 5 rows down
  • “3 Downs · Control d” selects 3 rows down, then invokes the fill function to copy whatever was in the first row to the selected rows

And here’s a method that will save you time whether you use one formula or many:

Add the formulas you use to the Vocabulary Editor, with a comfortable spoken form. For instance “Equals Sum” to type “=SUM(”

To add a formula, say

  1. “NatSpeak Vocabulary”
  2. Speak the formula using “spell” to put the written form in the Written Form text box, e.g. “spell equals caps Sierra Uniform Mike close paren” to type “=SUM(“
  3. “1 Tab”
  4. Put a comfortable and memorable written form in the spoken Form dialog box, e.g. “equals sum”.
  5. “Enter”
  6. “Escape” to exit or “Written Form” to add another.

Now every time you want to type “=SUM(“, say “equals sum”.

Speeding Web navigation: single-step deep menu access

Utter Command speech-enables the Firefox Mouseless Browsing extension, which puts a unique number on every clickable item on a Web page. UC lets you click every item on a page, including links, by saying the number plus the word “Go”, for instance “7 Go”.

This works pretty well, but it gets even better when you discover that an item doesn’t have to be visible for you to click it.

This lets you click items that are off-screen. Better yet, it lets you click items on drop-down menus without having to first drop-down the menu. This lets you use a single step to get to any menu item in a Web application once you know the number.

For instance, to insert a horizontal line in a Google Document you can click the “Insert” menu, then click the menu item “Horizontal Line”. There’s no direct keyboard shortcut for horizontal line, so it’s usually a two-step task.

Using numbers you can say “7 Go” to drop-down the Insert menu, then “84 Go” to click  Horizontal Line. But if, like me, you add horizontal lines often enough to remember the number, you can cut straight to the chase and say “84 Go” anytime you want a horizontal line.

Tip: Rudolf Noe’s Customize Your Web



Rudolf Noe, creator of the Mouseless Browsing add-on, is beta testing a new add-on that gives nonprogrammers extensive control of the Web.Noe’s Customize Your Web Firefox add-on allows you to specify that certain things happen every time a given webpage comes up. You can control where the focus is, click a button automatically, change how webpage elements look, and even change how they’re arranged on the page. Customize Your Web also contains a macro facility that allows you to attach keystrokes to elements on a given webpage. The key thing about the extension is it provides extensive control without having to program.Two of the simplest abilities — controlling where the focus is and clicking buttons, are fairly easy to implement. The focus ability lets you, for instance, open the Google Documents Site with the focus in the search bar. The click ability allows you to automatically login to any site.

To set up a focus change or button click on a webpage you go to that webpage, click the tiny Configure Your Web button in the bottom right corner of the screen right above the toolbar, click the element you want to affect, choose an action, then save what you have done.

You can name a Mouse Touch to click the Customize your Web button (see UC Lesson 10.24).

With just a little more effort you can specify keystrokes to do things like going down one search result, or click “Previous” or “Next” at the bottom of a search page.

If you assign the up and down arrows to go up and down by search result in a Google search, and Enter to click a selected result, you can then use the Utter Command speech command “3 Down · Enter”, for instance, to open the third search result down.

Noe’s video shows you how to use the extension in detail.

Also see UC Exchange page UCandFirefox.

Have you found Firefox or Thunderbird add-ons that make things easier when you’re using speech? Tell me about them – reply here or let me know at info@ this website address.

Tip: Rudolf Noe’s Customize Your Web

Rudolf Noe, creator of the Mouseless Browsing add-on, is beta testing a new add-on that gives nonprogrammers extensive control of the Web.Noe’s Customize Your Web Firefox add-on allows you to specify that certain things happen every time a given webpage comes up. You can control where the focus is, click a button automatically, change how webpage elements look, and even change how they’re arranged on the page. Customize Your Web also contains a macro facility that allows you to attach keystrokes to elements on a given webpage. The key thing about the extension is it provides extensive control without having to program.Two of the simplest abilities — controlling where the focus is and clicking buttons, are fairly easy to implement. The focus ability lets you, for instance, open the Google Documents Site with the focus in the search bar. The click ability allows you to automatically login to any site.

To set up a focus change or button click on a webpage you go to that webpage, click the tiny Configure Your Web button in the bottom right corner of the screen right above the toolbar, click the element you want to affect, choose an action, then save what you have done.

You can name a Mouse Touch to click the Customize your Web button (see UC Lesson 10.24).

With just a little more effort you can specify keystrokes to do things like going down one search result, or click “Previous” or “Next” at the bottom of a search page.

If you assign the up and down arrows to go up and down by search result in a Google search, and Enter to click a selected result, you can then use the Utter Command speech command “3 Down · Enter”, for instance, to open the third search result down.

Noe’s video shows you how to use the extension in detail.

Also see UC Exchange page UCandFirefox.

Have you found Firefox or Thunderbird add-ons that make things easier when you’re using speech? Tell me about them – reply here or let me know at info@ this website address.