Auto-correct a touch-screen problem

For the past few months, I’ve been taking an average of 1.6 flights per week on commercial airplanes. Most of these offered seatback entertainment, so I could watch the TV show or movie of my choice, or listen to satellite radio while reading. Touch-screen controls are easy to use because they let me touch—or tap—the item or the control that I want. By using the touch screen, I can select a program, adjust the volume, skip the next song, and so on.

One thing I’ve noticed is that about ¼ of seatback touch screens are poorly registered. By registration I mean that the system and the user agree on where the user is tapping or touching the screen:

An illustration of registration

I recorded a video of two common tasks for a seatback entertainment system: selecting the language and adjusting the volume. As you can see, the registration is off, so I initially get the French interface instead of the English, and I must press an unrelated button to adjust the sound:

The registration error is significant. My fingertip tapped about 2 cm left of the centre of the EN button. The larger the registration error, the harder to tap a small target—as was the case with the volume controls in the video, above, where I appear to be tapping the Fast-Forward button. On more than one flight I have unintentionally increased the sound to painful levels while attempting to lower the volume!

A system such as this could be made to detect and auto-correct poor registration. If we assume that repeat taps on a blank location indicates poor registration, the software could:

  1. After several repeat taps, select the nearest target—a reasonable guess—even if it is a centimetre or two away from the user’s tap.
  2. Ask the user to confirm the guess. “Did you mean [this one]?”
  3. If the user confirms, calculate the amount by which to correct the registration, and then fix the registration error.

This solution requires a screen—perhaps the start screen—whose choices are spaced far apart, so the system can detect when the user appears to be tapping a blank space:

Tapping a blank space (at right)

If user testing were to show that auto-correction needs human involvement, after calculating the registration error, the system could ask the user to check the corrected registration. For example:

Confirming that the registration is correct
Are you there? Please tap the green circle.

I haven’t done any testing of this idea, nor have I given this much thought, so I’m certain there are many more and better ways to auto-correct a registration problem on a touch screen. I merely wanted to identify one possible solution in order to get to the next point: the need to consider the business drivers when deciding to address (or deciding not to address) a usability problem.

Everything costs money

Fixing this problem—it’s a real problem, you’ve seen the video—would cost money. If the following can be quantified and evaluated within a framework of passenger-experience goals, there may be a convincing business case:

  • Not every passenger can work around a registration problem. Those who cannot would be unable to use the entertainment system. When everyone else gets a movie, how does the passenger with a failing system feel?
  • If a failed entertainment system is perceived as a negative experience, will passengers blame the touch-screen/software manufacturer or blame the airline? I’m sure you can imagine the complaint: “I sat there for hours without a movie! It’s the airline’s fault.” What’s the likelihood that this will cause churn (passenger switches to another brand next time)?
  • Based on the screens I’ve seen, some frustrated passengers must use hard objects that scratch and even gouge the touch screen. Are they trying to force the screen to understand what they want? Are they vandalising the screen? What’s the cost of replacing a damaged or vandalised screen?
  • A scratched screen is like graffiti. It affects every subsequent passenger in that seat. Do vandalised screens affect the airline’s goal of attaining a particular passenger rating for perceived quality or aesthetic experience?
  • The in-flight entertainment system was implicated in a catastrophic Swiss Air crash near Peggy’s Cove about a decade ago. Would a fix to the touch-screen registration problem incur prohibitive safety-testing costs?

What “standard” GUI means

If you decide to forego the design stage and reuse or copy an existing GUI or interaction design, your life is easier.

But how do you know when you have a standard to follow?

An obvious place to find standards is in the precedents set by your own Development team. If your standards are not documented, consider a BarnRaising afternoon, once a month, until the high priority standards are documented.

And while many of us have enjoyed the comfort of a style guide to follow, there’s more flexibility now, as Milan Guenther wrote in Photos for interaction:

© Boxes and ArrowsThe barrier between web pages and desktop software is beginning to disappear, and modern rich client user interface technologies such as Silverlight/WPF, Air, or Java FX enables designers to take the control over the whole user experience of a software product. Style guides for operating systems like MacOS or Windows become less important because software products are available on multiple platforms, incorporating the same custom design independently from OS-specific style guides. Software companies and other parties involved begin to use the power of a distinct visual design to express both their brand identity and custom interactive design solutions to the users.

But not everything needs to be reinvented. There are plenty of common user tasks that the operating system supports with default GUI, including Open, Save, Print, Search/Replace, Font, Color, and Page Setup tasks. These also defaults serve as patterns for the custom GUI that you design—the visible objects, and the invisible ones, such as:

  • mental models
  • workflows

The two traditional operating systems give us plenty of standard controls to meet any design challenge, including buttons, boxes, sliders, progress indicators, notifications, and much more. If you develop for MacOS, Apple publishes various guidelines, including for user experience. If you develop for Windows, the Vista UX Guide is updated regularly, with new sections for ribbons, touch, pen, and printing.

 sample-controls

Despite everything I just said about following standards, I’m also a big believer in changing things that are broken. An example: on my desk I can have two sheets of paper, copies of the same document. On my desk, they don’t need to have different names—even if each sheet has different comments written on it by different people. On my computer, the same two documents must either have different names or be in different folders. Surely the computer can keep track of both, uniquely, without forcing the user to choose different names? The unique-name requirement is an example of architecture dictating the interface. Alan Cooper wrote about this and other examples of tail wagging the dog in About Face: The essentials of user-interface design.

If you’re wondering when to copy/reuse, and when to design, check out my blog post, GUI: copy it or design it?