Discover, Adjust, Organize and Share

August 31st, 2010

Keyboard shortcuts have a lot of potential. They’re fast.

For example, cutting and pasting by

- Hitting “Control x”
- Moving the cursor to the paste location
- Then hitting “Control v”

is speedier than

- Moving the mouse to the “Edit” menu
- Clicking “Edit ”
- Clicking “Cut”
- Moving the cursor to the paste location
- Moving back up to click “Edit ”
- Then clicking “Paste”.

Add this up over many tasks and you have a big difference in productivity.

So why don’t we see more people using keyboard shortcuts?

Ask someone who uses the mouse for just about everything and you’re likely to get a compelling answer — it’s easier. And it is — it’s cognitively easier to choose a menu item than to remember a shortcut.

Given a choice, people generally do what’s easier. On a couple different occasions I’ve heard  people say that, all else being equal, they’d hire a blind programmer over a sighted one because the blind programmer is faster. The blind programmer must use keyboard shortcuts.

This is a common theme  — we have something potentially better, but human behavior stands in the way of adoption.

In the case of keyboard shortcuts there’s a little more to the story, however.

As a software community we haven’t implemented keyboard shortcuts well.

Many folks know keyboard shortcuts for a few very common actions like cut, paste and bold, but it’s more difficult to come up with keyboard shortcuts for actions like adding a link or a hanging indent because they are used less often and are less likely to be the same across programs.

So the user is often stuck with different shortcuts for the same tasks in different programs, requiring him to memorize and keep track of multiple sets of controls. This is cognitively difficult for everyone, and more so for some disabled populations and the elderly.

This type of implementation is akin to asking someone to speak different languages depending on who they are speaking to. Depending on how motivated and talented they are, some folks may be able to do it, but not many. And if there’s an easier way, even those capable of doing it either way will often choose easier even if it’s less efficient.

So we aren’t letting keyboard shortcuts live up to their potential.

There’s a second keyboard shortcuts issue that’s getting worse as Web apps become more prevalent: clashing shortcuts. If you hit “Control f” in a Google document, do you get the Google Find facility or the browser Find facility? Go ahead and try it out. It’s messy.

This is already an issue in the assistive technology community, where people who require alternate input or output must use software that runs all the time in conjunction with everything else. For example, a speech engine must be on all the time listening for commands, and screen magnifier software must be running all the time to enlarge whatever you’re working in.

So there are two problems: keyboard shortcuts aren’t living up to their potential to increase efficiency, and, especially on the Web, keyboard shortcuts are increasingly likely to clash.

I think there’s a good answer to both problems: a cross-program facility to easily discover, adjust, organize and share shortcuts.

- We need to easily discover shortcuts in order to see them all at once so we can see patterns across programs and conflicts in programs/apps that may be opened at once.

- We need to easily adjust shortcuts so we can choose common shortcuts and avoid clashes. We need to organize so we can remember what we did.

- We need to easily arrange commands and add headings so we can find commands quickly and over time build a good mental map of commands.. Lack of ability to organize is the Achilles’ heel of many macro facilities. It’s like asking people to play cards without being able to rearrange the cards in their hand. It’s possible, but unless there’s a reason for it, makes things unnecessarily difficult.

- We need to share the adjustments because it makes us much more efficient as a community. My friend Dan, for instance, is very logical. He uses many of the same programs I do, and we both use speech input. So if there were a facility to discover, adjust, organize and share keyboard shortcuts, I’d look to see if Dan had posted his changes, and I would adjust to my needs from there.

The organizing and sharing parts are the most important, because they allow for crowdsourcing.

Over the past few decades the computer interface ecosystem has shifted from single, unrelated programs to separate programs that share information, to programs so integrated that users may not know when they are going from one to another. This has increased ease-of-use and efficiency but at the same time complicated program control.

At the same time programs have grown more sophisticated. There’s a lot of wasted potential in untapped features.

If we give users the tools to discover, adjust, organize and share, I bet we’ll see an increase in speed and efficiency and an uptick in people discovering nifty new program features.

Suggestion for Dragon: Easier Correction

July 6th, 2010

In the last couple of months I’ve had a couple occasions to suggest to the folks at Nuance, the company that makes the Dragon NaturallySpeaking speech engine, that their “Resume With” command is under advertised. The command is very useful, but I keep meeting people who don’t know about it.

“Resume With” lets you change text on the fly. For instance, if you say “The black cat jumped over the brown dog”, then — once you see it on the screen — change your mind about the last bit and say “Resume With over the moon”, the phrase will change to “The black cat jumped over the moon.”

This is a particularly useful command for doing something people do a lot — change text as they dictate.

Now I have a suggestion that I think would make the command both better and more often used. Split “Resume With” into two commands: “Try Again” and “Change To”. The two commands would have the same result as “Resume With”, but “Try Again” would tell the computer that the recognition engine got it wrong the first time and you are correcting the error. “Change To” would tell the computer that you are simply changing text.

This would be a less painful way to correct text than the traditional correction box. Users are tempted to change text rather correct it because it’s easier. This would make it equally easy to correct and change using what is arguably the fastest and easiest way to make a change.

Easy correcting is important because NaturallySpeaking learns from correcting and because it’s annoying when the computer gets things wrong. Correcting improves recognition. Minimizing the interruption reduces frustration and lets users concentrate on their work rather than spending time telling Dragon how to do its job. From my observations, many users are tempted to change text rather than correct it when the computer gets something wrong simply because it’s easier.

It would be great to have these commands both in Dragon NaturallySpeaking on the desktop and in Dragon Dictation, the iPhone application. This would enable truly hands-free dictation in Dragon Dictation.

Guest Blog: Remembering Touch Commands

April 16th, 2010

So you’ve mastered Touch commands, in fact you’ve mastered them so well that you’ve created dozens and now you can’t remember them all. To access them, you can say “UC List Touch”, but that draws your focus away from the screen that you are working on.

If you incorporate your Touch commands into your Custom Files, you will always have them at your fingertips — or tongue tips, as it were.

Some of you are already using Custom Files (e.g.,“Custom 1 Guide”, “Custom 2 Guide”). For those of you who aren’t, these are handy guides that can come up on the right side of your screen, providing a “cheat sheet” while you continue to work. They take up some space on your screen, but they are frequently worth it. Especially if — like me — you have a hard time remembering all those great Touch commands!

Right now I am using Custom Files for the two programs I use most frequently, Outlook and Firefox. In each of these programs, I have created a number of Touch commands to move the mouse and click at various spots on the screen. I decided that I wanted to have easy access to my touch commands so I could maximize my use of this powerful tool. I also wanted to reorganize my touch commands by program (when you add a new Touch command, it is added chronologically to the bottom of your list ).

Here’s how to do it:

  1. Say “UC List Touch” to call up your list of touch commands.
  2. Say “All Copy to Excel”, which will copy all of your touch commands into an Excel spreadsheet.  It will also copy all of the number coordinates that go with each touch command you’ve created, but don’t worry about that, the next step will take care of that.
  3. Select your touch commands by saying “40 Downs” or some number that will capture them (if you have a lot of touch commands, you might need to say “100 Downs”, for example.)
  4. Say “All Copy to Notepad”. This will copy your touch commands into Notepad, which will remove all of the formatting from Excel (you don’t want to have the lines that separate each Excel.
  5. You now have a clean list of all of your touch commands that you can organize. Rearrange them by program, or any way that you would like. Once you have them in an order that makes sense to you, you can easily copy and paste them into your custom files.

Now your touch commands are part of your cheat sheet. So the next time you are working in a program and you want to remember all of those handy dandy touch commands you created for it, just call up your custom file and there they are.

ONE BIG CATCH: Since the Custom File does take up a portion of your screen on the right side, some of your touch commands may be “off” since part of your screen is now filled by the Custom File itself. So you can look at the file to remind yourself of your touch commands, then close the file and continue. It’s still easier than scrolling through the list you see when you call up UC List Touch.

- Big Talker

Speaking to Excel

April 13th, 2010

I’ve gotten a lot of inquiries lately about using speech recognition in Excel.

The fastest way to learn to apply Utter Command to Excel is to read UC Lesson 10.9: Navigating, numbers, functions, selecting and formatting in tables and spreadsheets, and UC Lesson 10.10: Putting it all together in any program (say “UC Lesson 10 Point 9″ and “UC Lesson 10 Point 10″ to call them up). Then take a look at the Top Excel Guide, which opens a list of useful shortcuts along the right edge of your screen.

Here are some basics:

  • “Cell” followed by a letter and number jumps to any cell, e.g. “Cell B 2″ or “Cell Bravo 2″
  • “Control Space” selects the row the cursor is on
  • “Shift Space” selects the column the cursor is on

Here are some particularly useful combinations:

  • A number followed by a direction selects cells — keep in mind you can select in two directions at once, e.g. “3 Rights · 5 Downs” to select 3 columns to the right and 5 rows down
  • “3 Downs · Control d” selects 3 rows down, then invokes the fill function to copy whatever was in the first row to the selected rows

And here’s a method that will save you time whether you use one formula or many:

Add the formulas you use to the Vocabulary Editor, with a comfortable spoken form. For instance “Equals Sum” to type “=SUM(”

To add a formula, say

  1. “NatSpeak Vocabulary”
  2. Speak the formula using “spell” to put the written form in the Written Form text box, e.g. “spell equals caps Sierra Uniform Mike close paren” to type “=SUM(“
  3. “1 Tab”
  4. Put a comfortable and memorable written form in the spoken Form dialog box, e.g. “equals sum”.
  5. “Enter”
  6. Escape” to exit or “Written Form” to add another.

Now every time you want to type “=SUM(“, say “equals sum”.

Tip: Finding a command

December 31st, 2009

Here’s a very quick tip.

If you know the name of a command, or even part of it, and want to look it up in the Utter Command documentation, say

   – “UC Index” to bring up the Utter Command Index
   “Find Open” to put the cursor in the find dialog box
   – type a keyword you want to look for, for instance “Wait”, “Drag” or “Before”
   “Enter” to find the first instance
   – if necessary, “Enter” again to find subsequent instances

Once you find what you’re looking for, use the reference number to call up the full lesson on the command, e.g. “UC Lesson 4 Point 5″. This is also a good way to see the consistent patterns in the Utter Command speech command set.

Tell me what you think – reply here or let me know at info@ this website address.

Happy new year!

Trying out Dragon Search for the iPhone

December 23rd, 2009

Dragon Search is a nice app. Here’s how it works: open the app, hit one button, speak the phrase you want to search for. By default the app stops listening and starts the search when you pause so you don’t have to hit another button when you’re done.

The app comes up quickly, which from a practical standpoint is extremely important. And in my experience so far the search has been fast. There’s also a button you can push to cancel out of the search. The big plus of this application is the different search channels: Google, iTunes, Twitter, Wikipedia, and YouTube. You can search for something, like green apples, and the results will come up in the channel you used last. Once you’ve done a search you can switch channels easily to see results across channels.

I have a couple of practical suggestions.

1. The history list is just three items long — I’d like a much longer scrolling history list. Google Voice Search has a long scrolling list that includes dates. I would’ve liked to have seen Nuance improve on that.

2. I’d also like to be able to add my own channel.

I’ll also take the opportunity to repeat what I said a couple of days ago. I appreciate the progress on speech apps — don’t get me wrong. But speech on the iPhone is still not what I really want, which is system-level speech control of a mobile device that would give me the option to use speech for anything. These new apps are steps in the right direction — making the iPhone more hands-free. But there’s still a long way to go.

A few more thoughts on Dragon Dictation

December 23rd, 2009

I’ve been using Dragon Dictation on the iPhone a little more over the past few days and have a couple more thoughts for improvement.

1. If you select text in the full-screen application, then switch to the keyboard the text doesn’t stay selected. The text should stay selected. If you’ve selected an incorrect word or phrase, found there are no correct choices, and are proceeding to the keyboard to correct it. It’s frustrating to have to select again.

2. I’ve lost dictation a couple of times because I’ve switched out of the app — this is unexpected because writing apps like Notepad tend to stay where you left them. I suspect that Dragon Dictation maker Nuance made this choice in order to limit the number of steps for new dictation. I think there are ways to provide this valuable option without increasing steps. The quick solution would be a “remember last dictation option” in settings that would let the user decide which way to do it. Maybe a better solution would be adding a “continue” button to the bottom of the initial screen that would give you the option to continue. So if you wanted to start fresh you would press the main button in the middle of the screen, but if you wanted to continue you could press the smaller “continue” button at the bottom of the screen.

Trying out Dragon Dictation for the iPhone

December 16th, 2009

I’ve been trying out the Dragon Dictation iPhone app. It’s still not what I really want, which is system-level speech control of a mobile device that would give me the option to use speech for anything. But it’s a step in the right direction of making the iPhone more hands-free.

Here’s how Dragon Dictation for the iPhone works: open the app, hit one button, speak up to 30 seconds of dictation, then hit another button to say you’re done. Your dictation shows up on the screen a few seconds later. Behind the scenes the audio file you’ve dictated is sent to a server, put through a speech-recognition engine, and the results sent back to your screen. Now you can add to your text by dictating again, or hit an actions button that gives you three choices: send what you’ve written to your e-mail app, send it to your text app, or copy it to the clipboard so you can paste it someplace else.

The recognition is usually fairly accurate in quiet environments. Not surprisingly, you get a lot of errors in noisy environments. To its credit, on a mobile device the built-in microphone is not optimal for speech-recognition. It does pretty well given these constraints.

Here’s a practical suggestion that should be easy to implement: Add a decibel meter so people can see exactly how much background noise there it is at any given time. This would make people more aware of background noise so they could set their expectations accordingly.

The interface for correcting errors is reasonable. Tap on a word and there are sometimes alternates available or you can delete it. Tap the keyboard button and you can use the regular system keyboard to clean things up.

I have two interface suggestions:

1. You can’t use the regular system copy and paste without going into the keyboard mode. You should be able to. I suspect this is fairly easy to fix.

2. There is no speech facility for correcting errors. I think there’s a practical fix here as well.

First, some background. Full dictation on a mobile device is tricky. Full dictation speech engines take a lot of horsepower. Dragon Dictation sidesteps the problem by sending the dictation over the network to a server running a speech engine. The trade-off is it’s difficult to give the user close control of the text — you must dictate in batches and wait briefly to see the results. This makes it more difficult to offer ways to correct using speech. But I think there is a good solution already in use on another platform.

Although it’s difficult to implement most speech commands given the server setup, the “Resume With” command that’s part of the Dragon NaturallySpeaking desktop speech application is a different animal. This command lets you start over at any point in the phrase you last dictated by picking up the last couple of words that will remain the same and dictating the rest over again.

This would make Dragon Dictation much more useful for people who are trying to be as hands-free as possible. It would also lower the frustration of misrecognitions and subtly teach people to dictate better.

It’s nice to see progress on mobile speech. I’m looking forward to more.

Tip: Scrolling by speech

November 24th, 2009

I’ve gotten several questions lately about scrolling by speech, which is key to comfortable hands-free operation. Utter Command gives you several ways to scroll by speech. The best way depends on the situation.

To quickly look something over, use the speech command that allows you to see successive screens with a pause between changes. For example, “3 Screen Down Wait” moves down a screen, then after a default wait of two seconds moves down another screen, then two seconds later moves down a third screen. If you want a longer wait, add a specific number of seconds, e.g. “3 Screen Down Wait 5″ (UC Lesson 7.23). 

To directly control the scroll bar by speech, place the mouse pointer on the scroll bar using a command like “99 by 10″ and use the vertical drag command to move the scroll bar to a given point. For example “Drag By 50″ moves the scroll bar to the middle. Then, if you then want to go three quarters of the way down say “Drag By 75″. You can also control the scroll bar incrementally, for instance, “Drag 3 Down” (UC Lesson 4.2, 4.5).

In some programs, including some versions of Word, the cursor moves to the page you scrolled to when you use an arrow command like “5 Down”. And in some programs, like Firefox, you can say a link number to move the cursor. In these cases you can leave the arrow parked on the scroll bar, edit the text, than say another drag command to move the scrollbar without having to move the mouse to the scrollbar again. In some programs, including WordPad, you have to move the cursor to the new page by clicking. In this case, keep the right ruler open on your screen so you can easily click back to the scroll bar when you’re ready to scroll again.

- If you use this method a lot, try naming a mouse click to move the arrow to the scroll bar at the home position (UC Lesson 10.24).

- You can also use this method to control horizontal scrollbars — use the “Drag 1-100 By” command.

- If you’re a ZoomText user, you can use this method even when the scrollbar is not showing on the screen.

Tell me what you think about scrolling by speech – reply here or let me know at info@ this website address.

Highlighting and hot water

November 13th, 2009

Have you ever used a faucet that had a hot water knob on the right side instead of the left?

Even if it’s well labeled, chances are you’ll turn the wrong handle a good percentage of the time. This is because controlling the faucet is something you usually do without thinking and your habit is to turn with your left hand when you want hot, not your right.

Consistency allows for habit, which saves time. Do a consistent navigation task a few times and after that you don’t have to think about it. It’s become habit, which means you can use more of your brain to think. The system backfires, however, when you unconsciously expect consistency, use habit, and are caught by surprise.

I often talk about the importance of consistent keyboard shortcuts across programs, because I use keyboard shortcut navigation more than mouse/toolbar navigation.

But consistency is just as important in toolbars.

The default order for many common groups of items is consistent across programs. For instance, Bold, Italic and Underline are commonly shown in that order. Left Justify, Center and Right Justify are commonly shown in that order. Style, Font and Size are commonly shown in that order. There’s a glaring problem, however, when it comes to the highlight and text color icons.

Microsoft Office toolbars put the highlight on the left and the text color icon on the right, while Google Docs and OpenOffice defaults put the highlight button on the right and the text color icon on the left.

The inconsistency makes it impossible to form a habit that’s useful across programs. If you get used to one way you’ll inevitably pick the wrong button when you’re in the program you’re not used to. If you regularly use a mix of inconsistent programs you’re likely to get things wrong fairly often.

In a world where people use multiple programs, inconsistent default order in groups of icons puts a larger-than-necessary cognitive load on folks. Worse, it makes habit a liability rather than an advantage.

It would be good for people if we had a standard order for related icons like Highlight and Text Color just as we have a standard order for faucet controls. The exact order matters much, much less than consistency across programs. Software is complicated enough already — we need to give people all the easy breaks we can.