We’ve all seen it: waterfall projects that deliver the wrong thing, too late. So we understand the appeal of the Agile method, delivering working software sooner, so the intended users—our clients and their customers—can provide feedback that steers us to deliver the right thing. Agile reporting tools also help us estimate how long the work will take, which makes it possible to deliver on time.
But there’s a tension between delivering the right thing and delivering on time. And as a UX practitioners, we sometimes see usability sacrificed in the rush to release on time. This happens despite the first of the Agile Manifesto’s principles:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Valuable software is usable software, among other things. Data about what’s usable comes from testing. And much of that testing can’t take place until after the development—done in Agile stories—is completed and signed off. Development teams—consisting of analysts, developers, testers, usability researchers, and interface designers—often consider an Agile story to be completed despite the lack of feedback from the intended customers about its usability—or unusability. We can do better.
A disconnect caused by timing
The collection of feedback—usability research that provides quantitative performance data and qualitative observations—typically occurs during other iterations—before and after the iteration in which developers work on a story. Formative testing of prototypes occurs during a previous iteration. Summative testing of as-built features occurs during a subsequent iteration.
Since summative testing takes place after an Agile story is completed and signed off, there’s resistance to re-opening a story for defects, because that wreaks havoc on the team’s ability to predict how on time the project or the current epic will be. Consider this simplified burn-down chart:
Development teams focus on speed, and management’s reporting tools reinforce this. That’s why teams often cite these two principles of the Agile Manifesto:
- Deliver working software frequently [in short sprints that last] from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Working software is the primary measure of progress.
It’s the short duration of sprints that pushes UX research about one sprint’s stories into other sprints. So usability defects show up later. And that would be acceptable if teams tended to follow this Agile Manifesto principle:
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Linking usability defects back to the story that gave rise to the defect brings into doubt the completeness of every functional story and nonfunctional story that affects usability. Adding new stories to fix usability defects could work, though teams tend to frame such “new” stories as “scope creep” that puts the release date at risk. Unexpected new stories also make it seem like the goal posts are moving, which demotivates a goal-oriented team. And did you notice that word: unexpected? Usability defects are only unexpected if the team expects designers and developers to deliver perfectly usable software on their first attempt, in every case, for every story.
That’s an unrealistic expectation, isn’t it?
If we expect some usability-related design failures and some usability-related coding failures, then the problem is simple: plan for the unexpected.
Account for usability defects in the projected release date
It’s possible to include unknown usability defects in our release-date projections. Here are some ideas—some of which might seem radical:
- Use reporting tools that track new scope, so that new scope can be factored into the projected release date.
- If the team generates many usability defects, is it systemic? Remember the last principle in the Agile Manifesto: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”
- Reduce the likelihood of major changes in the mental model, the workflow design, and the interaction design. Ensure the designers understand the entire set of features in sprint zero—before development begins. Consider possible design solutions for key workflows, then share this vision with the whole development team, and test it with customers. If needed, to finish this work faster, involve developers and QA in the design process. (The Five Sketches™ method is one highly efficient way to do that.)
- Allow stories that deliver a “working” prototype as equal to a “working” piece of code. This allows formative testing on paper prototypes or interactive prototypes before developers write code. The Agile principle that demands working software from every iteration allows some wiggle room: “Working software is the primary measure of progress” but not the only measure. This allows team members—all team members on an Agile team, including developers, QA, UI designers, and UX researchers—to focus on design, prototyping, and validation together instead of making a designer or usability work one sprint ahead or one sprint behind the rest of the team.
- Put usability deliverables in their own stories, rather than in the acceptance criteria of stories that build functions. Especially when you need work done that substantially shapes the user experience but that is not related to a specific feature, regard this user-experience work just as you regard database-, architectural-, or other foundational work.
- Reduce known future usability defects. Sometimes, when a series of stories are taken together, their individual deliverables don’t combine into a good user experience. Sometimes the Agile principle of simplicity—which pushes us to maximize the amount of work not done—must be balanced with the Agile principle of technical excellence—which posits that good design enhances agility. In other words, deliver something a little less simple for individual stories that, collectively, will require an additional story to clean up a predictable usability deficiency.
- If management likes waterfall release planning, add placeholder stories for usability fixes. If you’re not fully Agile, don’t set yourself up to drop usability fixes that were unknown at the start of the release cycle. At the start of the project, pad the backlog to allow for unknown but expected usability-defect fixes. That way, usability won’t be seen to cause delays to the release date.