We’ve all seen it: waterfall projects that deliver the wrong thing, too late. So we understand the appeal of the Agile method, delivering working software sooner, so the intended users—our clients and their customers—can provide feedback that steers us to deliver the right thing. Agile reporting tools also help us estimate how long the work will take, which makes it possible to deliver on time.
But there’s a tension between delivering the right thing and delivering on time. And as a UX practitioners, we sometimes see usability sacrificed in the rush to release on time. This happens despite the first of the Agile Manifesto’s principles:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Valuable software is usable software, among other things. Data about what’s usable comes from testing. And much of that testing can’t take place until after the development—done in Agile stories—is completed and signed off. Development teams—consisting of analysts, developers, testers, usability researchers, and interface designers—often consider an Agile story to be completed despite the lack of feedback from the intended customers about its usability—or unusability. We can do better. Continue reading “Put usability in your Agile backlog”
This week, I’m attending a few days of training in agile software development, in an Innovel course titled Lean, Agile and Scrum for Project Managers and IT Leadership.
My first exposure to agile was in Desiree Sy‘s 2005 presentation, Strategy and Tactics for Agile Design: A design case study, to the Usability Professionals Association (UPA) annual conference in Montreal, Canada. It was a popular presentation then, and UPA-conference attendees continue to be interested in agile methods now. This year, at the UPA conference in Portland, USA, a roomful of usability analysts and user-experience practitioners discussed the challenges that agile methods present to their practice. One of the panellists told the room: “Agile is a response to the classic development problem: delivering the wrong product, too late.” There was lots of uncomfortable laugher at this. Then came the second, thought-provoking sentence: “Agile shines a light on the rest of us, since we are now on the critical path.” Wow! So it’s no longer developers, but designers, usability analysts, etc, who are holding up the schedule?
During this week’s training, I’m learning lots while looking for one thing in particular: how to ensure agile methods accommodate non-developer activities, from market-facing product management activities, to generative product design, to early prototype testing, to usability testing, and so on.
I’m starting to suspect that when agile methods “don’t work” for non-developers, it’s because the process is wagging the dog (or that its “rules” are being applied dogmatically). I think I’m hearing that agile isn’t a set of fixed rules—so not a religion—but a sensible and flexible method that team members can adapt to their specific project and product.
Do you ever wonder how effective expert reviews and usability tests are? Apparently, they can be pretty good.
Rolf Molich and others have conducted a series of comparative usability evaluation (CUE) studies, in which a number of teams evaluate the same version of a web site or application. The teams chose their own preferred methods—such as an expert review or a usability test. Their reports were then evaluated and compared by a small group of experts.
What the first six CUE studies found
About 30% of reported problems were found by multiple teams. The remaining 70% were found by a single team only. Fortunately, of that 70%, only 12% were serious problems. In one CUE study, the average number of reported problems was 9, so a team would be overlooking 1 or 2 serious problems. The process isn’t perfect, but teams found 80% or more of the serious problems.
Teams that used usability testing found more usability problems than expert reviewers. However, expert reviewers are more productive-they found more issues per hour-as this graph from the fourth CUE study illustrates:
Teams that found the most usability problems (over 15 when the average was 9) needed much more time than the other teams, as illustrated in the above graph. Apparently, finding the last few problems takes up the most time.
The CUE studies do not consider the politics of usability and software development. Are your developers sceptical of usability benefits? Usability studies provide credibility-boosting video as visual evidence. Are your developers using an Agile method? Expert reviews provide quick feedback for each iteration.
To learn more about comparative usability evaluation, read about the findings of all six CUE studies.