Auto-correct a touch-screen problem

For the past few months, I’ve been taking an average of 1.6 flights per week on commercial airplanes. Most of these offered seatback entertainment, so I could watch the TV show or movie of my choice, or listen to satellite radio while reading. Touch-screen controls are easy to use because they let me touch—or tap—the item or the control that I want. By using the touch screen, I can select a program, adjust the volume, skip the next song, and so on.

One thing I’ve noticed is that about ¼ of seatback touch screens are poorly registered. By registration I mean that the system and the user agree on where the user is tapping or touching the screen:

An illustration of registration

I recorded a video of two common tasks for a seatback entertainment system: selecting the language and adjusting the volume. As you can see, the registration is off, so I initially get the French interface instead of the English, and I must press an unrelated button to adjust the sound:

This video shows a touch screen that detects a tap in a different location than the one I tapped.

The registration error is significant. My fingertip tapped about 2 cm left of the centre of the EN button. The larger the registration error, the harder to tap a small target—as was the case with the volume controls in the video, above, where I appear to be tapping the Fast-Forward button. On more than one flight I have unintentionally increased the sound to painful levels while attempting to lower the volume!

A system such as this could be made to detect and auto-correct poor registration. If we assume that repeat taps on a blank location indicates poor registration, the software could:

  1. After several repeat taps, select the nearest target—a reasonable guess—even if it is a centimetre or two away from the user’s tap.
  2. Ask the user to confirm the guess. “Did you mean [this one]?”
  3. If the user confirms, calculate the amount by which to correct the registration, and then fix the registration error.

This solution requires a screen—perhaps the start screen—whose choices are spaced far apart, so the system can detect when the user appears to be tapping a blank space:

Tapping a blank space (at right)

If user testing were to show that auto-correction needs human involvement, after calculating the registration error, the system could ask the user to check the corrected registration. For example:

Confirming that the registration is correct
Are you there? Please tap the green circle.

I haven’t done any testing of this idea, nor have I given this much thought, so I’m certain there are many more and better ways to auto-correct a registration problem on a touch screen. I merely wanted to identify one possible solution in order to get to the next point: the need to consider the business drivers when deciding to address (or deciding not to address) a usability problem.

Everything costs money

Fixing this problem—it’s a real problem, you’ve seen the video—would cost money. If the following can be quantified and evaluated within a framework of passenger-experience goals, there may be a convincing business case:

  • Not every passenger can work around a registration problem. Those who cannot would be unable to use the entertainment system. When everyone else gets a movie, how does the passenger with a failing system feel?
  • If a failed entertainment system is perceived as a negative experience, will passengers blame the touch-screen/software manufacturer or blame the airline? I’m sure you can imagine the complaint: “I sat there for hours without a movie! It’s the airline’s fault.” What’s the likelihood that this will cause churn (passenger switches to another brand next time)?
  • Based on the screens I’ve seen, some frustrated passengers must use hard objects that scratch and even gouge the touch screen. Are they trying to force the screen to understand what they want? Are they vandalising the screen? What’s the cost of replacing a damaged or vandalised screen?
  • A scratched screen is like graffiti. It affects every subsequent passenger in that seat. Do vandalised screens affect the airline’s goal of attaining a particular passenger rating for perceived quality or aesthetic experience?
  • The in-flight entertainment system was implicated in a catastrophic Swiss Air crash near Peggy’s Cove about a decade ago. Would a fix to the touch-screen registration problem incur prohibitive safety-testing costs?

Up and down the TV channels

My television lets me step through the channels. To do this, I use the remote control’s CH button. Similarly, my television lets me page through the list of programs, five channels at a time. To do this, I use the remote control’s PG button. In fact, it’s one button for the stepping and paging functions.

My remote control

The programs in the list are shown in numeric order, so smaller numbers are higher in the list. Pressing “+” will page the list up, so “+” leads to smaller numbers. Similarly, pressing “–” will page the list down, to larger numbers. This follows the same mental model as scrolling in a computer window, including the one you’re reading in, now.

Scrolling up

In contrast, when I’m watching one channel (full-screen, so with the program guide hidden), the same two buttons have the inverse effect. The “+” button increases the number of the channel (which is like moving down in the programs list, not up). This follows the same mental model as a spin control in many computer programs.

Spinning up

Imagine using the one button in succession for the two functions:

first as PG to page through the menu
  and then, after selecting a channel,
as CH to step through the channels.

I see in this an excellent problem for a practicum student or as a class assignment that’s combining user research, design, GUI, and handheld devices. Possible questions:

  • What research would confirm that this is, in fact, a problem?
  • If you confirm the problem, is it entirely on the hardware side? How many people are affected?
  • Is there a business case to fix the problem?
  • How could you fix it? What design methods and processes would you use? Why?
  • How could you demonstrate that your design fixes the problem? Is there a lower-cost way to validate the design, and, if so, what are the trade-offs?