Put usability in your Agile backlog

We’ve all seen it: waterfall projects that deliver the wrong thing, too late. So we understand the appeal of the Agile method, delivering working software sooner, so the intended users—our clients and their customers—can provide feedback that steers us to deliver the right thing. Agile reporting tools also help us estimate how long the work will take, which makes it possible to deliver on time.

But there’s a tension between delivering the right thing and delivering on time. And as a UX practitioners, we sometimes see usability sacrificed in the rush to release on time. This happens despite the first of the Agile Manifesto’s principles:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Valuable software is usable software, among other things. Data about what’s usable comes from testing. And much of that testing can’t take place until after the development—done in Agile stories—is completed and signed off. Development teams—consisting of analysts, developers, testers, usability researchers, and interface designers—often consider an Agile story to be completed despite the lack of feedback from the intended customers about its usability—or unusability. We can do better. Continue reading “Put usability in your Agile backlog”

Cumulative cost of a few seconds

Currently, I’m on a project team that’s designing, building, and implementing call-centre software. You can probably imagine the call-centre experience from the customer side—we’ve all had our share of call-centre experiences. I’ve been looking at call centres from the other side—from the perspective of the customer-service agents and their employer.

I started by observing customer-service agents on the job. At the site I visited, the agents were using a command-line system, and the agents typed so fast that I couldn’t make sense of their on-screen actions. I signed up for several weeks of training to become a novice customer-service agent. This allowed me to make sense of my second round of observations, and appreciate how efficiently the agents handle their customer calls. It also helped me to identify tasks where design might improve user performance.

Wrap-up choicesFor example, after each call the agent decides why the customer called, and then, by scanning lists of main reasons and detailed reasons, “wraps up” the call, as illustrated. I measured the time on task; the average wrap-up task is nine seconds in duration.

It’s only nine seconds

Nine seconds may not seem long, but let’s make a few (fictitious but reasonable) assumptions, and then do a little math.

If the average call-handling time is five minutes, or 300 seconds, the 9 seconds spent on call wrap-up is 3% of the total handling time. A full-time agent could spend 202,500 seconds—that’s 56¼ hours per year—on call wrap-ups, assuming a 7½-hour workweek and no lulls in incoming calls. Since call volumes vary, there will be times when call volumes are too low to keep all agents taking calls. The customer-service agents have other tasks to complete during such lulls, but if we assume this happens about a third of the time, we need to round down the 56¼ hours accordingly. Let’s choose a convenient number: 40 hours, or one workweek per agent per year.

One workweek is 2% of the year.

Based on this number, a redesigned call wrap-up that takes only half the time would save one percent of the labour. Eliminating the wrap-up entirely would save two percent. That frees a lot of hours for other tasks.

A similar calculation on the cost side (n hours to design and implement changes) leaves us with a simple subtraction. Projected saving minus cost is the return on investment, or ROI. Comparing that number to similar numbers from other projects that we could tackle instead—the opportunity costs—makes it easy to decide which design problem to tackle.

Leaner, more agile

This week, I’m attending a few days of training in agile software development, in an Innovel course titled Lean, Agile and Scrum for Project Managers and IT Leadership.

My first exposure to agile was in Desiree Sy‘s 2005 presentation, Strategy and Tactics for Agile Design: A design case study, to the Usability Professionals Association (UPA) annual conference in Montreal, Canada. It was a popular presentation then, and UPA-conference attendees continue to be interested in agile methods now. This year, at the UPA conference in Portland, USA, a roomful of usability analysts and user-experience practitioners discussed the challenges that agile methods present to their practice. One of the panellists told the room: “Agile is a response to the classic development problem: delivering the wrong product, too late.” There was lots of uncomfortable laugher at this. Then came the second, thought-provoking sentence: “Agile shines a light on the rest of us, since we are now on the critical path.” Wow! So it’s no longer developers, but designers, usability analysts, etc, who are holding up the schedule?

An agile loadDuring this week’s training, I’m learning lots while looking for one thing in particular: how to ensure agile methods accommodate non-developer activities, from market-facing product management activities, to generative product design, to early prototype testing, to usability testing, and so on.

I’m starting to suspect that when agile methods “don’t work” for non-developers, it’s because the process is wagging the dog (or that its “rules” are being applied dogmatically). I think I’m hearing that agile isn’t a set of fixed rules—so not a religion—but a sensible and flexible method that team members can adapt to their specific project and product.

Train yourself in frustration, confusion, and inefficiency

For professional reasons, I like to mess around with software. It’s a form of training, because some of the messing around leads to frustration, confusion, and inefficiency. And that’s good.

My hope is that my experiences will help me to better understand what I put various groups of software users through when they use the software I helped design and build.

An easy way to mess around is by changing default settings. For example, my iTunes isn’t set to English. This helps me understand the experience of users who learned one language at home as children and now use another language at work as adults. It’s not just beneficial to experience the initial pain of memorising where to click (as I become a rote user in a GUI I cannot read), but also the additional moments of frustration when I must do something new—an occasional task whose command vector I haven’t memorised.

Relating to the language challenges that some users face

Another easy way to mess around is to switch between iMac and Windows computers. It’s not just the little differences, such as whether the Minimise/Maximise/Close buttons are on the left or right sides of the title bar, or whether that big key on the keyboard is labelled Enter or Return.

Switching between operating systemsIt’s also the experience of inefficiency. It’s knowing you could work faster, if only the tool weren’t in your way. This also applies to successive versions of “the same” operating sytem. This is the frustration of the transfer user.

It’s noticing how completely arbitrary many design standards are—how arbitrarily different between operating systems—such as the End key that either does or doesn’t move the insertion point to the end of the line.

Another easy way to mess around is to run applications in a browser that’s not supported. I do it for tasks that matter, such as making my travel bookings.

All this occasional messing around is about training myself. The experiences I get from this broaden the range of details I ask developers to think about as they convert designs into code and into pleasing, productive user experiences.

In a separate IxDA discussion thread, a few people reacted to this blog post:

  • Try a Dvorak keyboard instead of a Qwerty keyboard (Johnathan Berger).
  • Watch children’s first use of a design (Brandon E.B. Ward).
  • Use only the keyboard, not the mouse (CK Vijay Bhaskar).
  • Sit in at the Customer Support desk for a day (Adrian Howard).
  • Search Twitter to find out how people feel about a product (Paul Bryan).

See also the comment(s) below, directly in this blog.

Design requires courage and trust, not just user involvement

Designing is usually a rewarding activity, but the path from start to finish can be filled with frustration and even panic. I’ve seen design processes work—and come to the realisation that “My own designs benefited from rapid iteration!”

The benefit of designThese humbling experiences helped me learn to trust the process, even in the face of frustration or panic. It’s these experiences that give me the courage to follow the design process, even when it isn’t clear how to resolve the tension between conflicting design constraints.

In the face of an unknown, individuals and especially teams tend to turn to knowns. If needed, they’ll manufacture the known data, by deferring the choice to users. Here’s part of what Larry Constantin wrote about courage in software design, in a paper that advocates for user involvement at the right time:

Most damning and least recognized among the limitations of user-centered design is the way it subtly discourages courage. Courage is one of the central tenets of extreme programming and agile development methods. […] User-centered design makes it too easy for designers to abdicate responsibility in deference to user preference, user opinion, and user bias. In truth, it is hard to stick with something you know works when users are screwing up their faces at it. What if you are wrong? What if you are not as good a designer as you thought you were? It takes real courage and conviction to stand up for an innovative design in the face of users who complain that it is not what they expected or who want it to work just like some other software or who object to certain sorts of features as a matter of course. It takes responsible judgment to know when to listen to users and when to ignore them.

In the many design sessions I have facilitated, three times I’ve seen that lack of courage expressed by a participant. Each time, it sounded like a mix of panic and frustration:

The solution has been on the wall since the first round!

The design sessions I facilitate ask participants to saturate the design space with lots of ideas. They each bring five sketches—five substantially different ideas—and then, after sharing their ideas with the other participants, they rapidly iterate the first 15 or 20 sketches to develop even more. All this takes place before any analysis.

When the goal is to saturate the design space—to identify as many solutions as possible in a short time—there’s more to influence the design once the analysis begins. Inevitably, the design that the team decides on was not already on the wall. Motivated design participants quickly learn this, and—in most cases—become advocates of the process.

For most development teams, the Five Sketches™ process I introduce is a departure from the status quo, so it takes courage for their team members to take a stand, to say “I will use this process” for design problems that need it.

How many user personas?

If you’re creating user personas, How-To articles often tell you that you only need two or three personas at most. That’s fine for most web-design projects. However, if you are working on an enterprise-wide system that has modules for different types of professionals who each perform distinct and substantial tasks, then you will have a larger number of user profiles.

How big is the feature set? Imagine a product suite the size of Microsoft® Office that actually consists of very different pieces: Excel, Word, Access, PowerPoint, and more. Usually, the only persona who is involved with every module of a suite is someone like Ivan the IT administrator, whose tasks are very different from most users.

That may sound obvious, but I really struggled for a while with the notion that I was “doing it wrong” because I couldn’t squeeze the user roles and needs into a mere three user personas—or five, or seven, for that matter. When you have a dozen user personas, it’s challenging to keep them all apart, but most teams only need a few at a time. Likely, the only people who need to know all the user personas are on the user-experience and product-management teams. And the alternative—to have a catch-all user persona whose role is to “use the software”—is of no help at all.

If, by chance, you find you need many user personas, then beware once the projects wrap up and the teams turn to their next projects. They may have incorrectly learned to begin a project by creating user personas (“…well, that’s what we did last time…!”) instead of re-using and tweaking the existing user personas. [Not everyone agrees with re-use; see the comments.]

User personas will influence your product design and affect how people throughout Development and Marketing think, strategically and tactically, about their work. So you need to get the user personas right. Getting a series of user personas reviewed—not rubber-stamped but mindfully critiqued—is a challenge; nobody ever makes time to do it well. Here’s my solution: after you research users and then write your draft user personas, review them together with some subject-matter experts, Marketing staff, and developers, in a barn-raising exercise. Pack them all into one afternoon for the initial review. Then meet the first Friday of the month until you have agreement about each user persona. During one such meeting, one of my subject-matter experts said to another: “Oh, this user persona is just like [a customer named H—]!” The user persona was so on-target that that it reminded her of someone I had never met or researched. That was a nice way to learn that I got it right.

For detailed How-To advice on developing user personas, try these readings:

Developers can learn ¾ of Design

Microsoft’s Bill Buxton recently wrote an article for Business Week, titled On Engineering and Design: An open letter. In it, Buxton explains that developers can improve the user experience of the product that they’re building by learning three of the four layers that engage designers:Four layers of design
  • Design awareness.
  • Design literacy.
  • Design thinking.

Buxton also mentions a fourth layer, design practice. He explains that design practice represents a fulltime job for highly trained professionals, and that it’s rare to find a developer who straddles both. (In my experience, small- and mid-sized companies may get by without a design practitioner, if their designs are constrained by the rules and standards of the operating system and hardware, and if their competitors do no better.)

Buxton thinks non-designers can easily learn about and appreciate the first three layers of design. On the third layer, design thinking, Buxton writes:

Cognitive science makes it clear that the strategies designers use in approaching problems or questions are different (not “better”) than those employed by those trained in engineering disciplines. Both strategies are complementary. Given the complexity of the problems that confront us, it seems to me that expanding our collective arsenal of techniques is something we could all benefit from.

This difference in problem-solving strategies is the ideation-judgement axis that I wrote about in Please exit your comfort zone. Learning to use these different strategies—at the right time in the design and development process—is what Five Sketches™ teaches to developers and other non-designers.

Speed sketching vs. art/perfectionism

For a Five Sketches™ design session, I ask design participants to bring at least five substantially different ideas to the table, in the form of sketches. A common initial reaction is: “…But I can’t sketch!”

Many design participants believe they cannot draw. To be honest, I believe that about myself. People who feel they cannot draw tend to extend that belief to encompass design sketching, as well. However, I have learned to distinguish between drawing (below, left) and sketching a software design (below, right).

Drawing versus sketching

Software design does not require drawing, only sketching. From experience, I can tell you that sketching is easy once you realise that the goal is to produce only rough and low-fidelity ideas, not decorative and high-fidelity pictures.

Occasionally, I encounter a design participant who has trouble with sketching. I once had a participant who was so self-conscious of her sketching ability that she was paralyzed during the rapid iteration phase, when the design participants are quickly mashing up the sketches with borrowed and new ideas to produce additional sketches. I noticed that this participant was always busy—talking or wiping the board or replacing the snacks—but never sketching. This avoidance, and the underlying mental block, was an impediment to the whole team. Like all generative design processes, Five Sketches™ relies on its participants to saturate the design space with ideas. If a participant isn’t participating, it increases the project risk.

To pre-empt this mental block and to reduce the project risk, during Five Sketches™ training, I now introduce a quick speed-sketching exercise. Speed sketching is a race against the clock. With pen and paper ready, before each round, I give the participants five seconds to think, and then:

  • Speed sketching in seconds!10 seconds to sketch a cell phone.
  • 8 seconds to sketch a sandwich.
  • 6 seconds to sketch an airplane.
  • 4 seconds to sketch a ship.
  • 2 seconds to sketch a house.

After each sketch, the participants look at the other sketches. I ask them to identify the details that make the sketched object recognisable. There isn’t time to draw a high-fidelity rendering, so the sketches are rough and have few details.

Interestingly, with few details the sketched objects are still clearly recognisable. Participants learn that detail and fidelity aren’t needed. By the last round of sketches—the “2 seconds to sketch a house” round—participants have seen that everyone’s ability to produce sketches is about equal, and they usually conclude: “I sketch well enough for others to grasp my ideas.” Generative design requires lots of ideas—disposable ideas—not fancy drawings.

Jason Santa Maria says a similar thing in his Pretty Sketchy blog post, in which he declares that sketching is not about being a good artist; sketching is about being a good thinker.

This sugar packet is a movie

Whether it’s ethnographic research, usability research, or marketing research, I’ve learned that the best insights aren’t always gleaned from scheduled research.

Here’s a photo of impromptu research, conducted by Betsy Weber, TechSmith’s product evangelist. I was her research subject. Betsy recorded me pushing sugar packets around a table as I explained how I’d like Camtasia to behave.

Jerome demos an idea to Betsy. Photo by Mastermaq

Betsy takes information like this from the field back to the Camtasia team. There’s no guarantee that my idea will influence future product development, but what this photo shows is that TechSmith listens to its users and customers.

The ongoing stream of research and information that Betsy provides ensures better design of products that will be relevant and satisfying for TechSmith customers down the line.

Sketch, wireframe, prototype

Over the past month, I’ve come across the same discussion several times: “When designing a website or product, do you use wireframing or prototyping?”

The first part of my answer is: “Make sure you sketch, first.”

At the design stage, sketching, wire-framing, and prototyping are not equal. Sketching is useful at the divergent phase of design because it lets the design participants express and capture lots of different ideas quickly and anywhere that pen and paper will work. Nothing is as fast as running a pen across a sheet of paper to capture an idea—and then another, and another. And since sketching is intentionally rough, everyone can do it.


Responding to Œ the problem statement, first  saturate the design space with lots of ideas, and then Ž analyse and rapidly iterate them to  a design solution.

I also believe sketching is great for the convergent phase of design, but there are potential hurdles that design participants may encounter. It can be challenging to convey complex interaction, 3D manipulation, transitions, and multi-state or highly interactive GUI in sketches without learning a few additional techniques. This is unfortunate, because having to learn additional techniques reduce the near-universal accessibility of sketching.

The second part of my answer, therefore, is that “if you need to learn additional techniques to make sketching work, feel free to choose wireframes or prototyping as alternatives when there are compelling reasons to do so.”

I should point out that the three techniques—sketching, wireframing, and prototyping—are not mutually exclusive. Wireframes and paper prototypes can both be sketched—especially for simple or relatively static GUI designs.

There are no validity concerns with the use of low-fidelity sketches, as these readings show: