Tag: Usability

Posted on

Epistemology of usability studies

Currently, I’m conducting research on usability analysis and on how Morae software might influence that. My research gaze is rather academic, in that I’m especially interested in the epistemology of usability analysis.

One of my self-imposed challenges is to make my research relevant to usability practitioners. I’m a practitioner and CUA myself, and I have little time for academic exercises because I work where the rubber hits the road. This blog post outlines what I’m up to.

At Simon Fraser University, I learned that epistemological approaches have different assumptions about what is knowable. On one side (below, left), it’s about numbers, rates, percentages, graphs, grids, tables, proving absolute truths. On the other side, (below, right) it’s about seeking objectivity while knowing that it’s impossible because everything has a cultural context. The epistemology you choose, when doing research, depends on what you believe. And the epistemology dictates what methods you use, and how you report your results.

You can be
certain of
what you know.		 You cannot be
objective about
what you know.

Let’s look at some examples.

Study 1 fits with the view (above, left) that “you can be certain of what you know.” I plan and conduct a quantitative study to measure the time it takes a series of users to complete two common tasks in a software package: upgrading to the latest version of the software, and activating the software. I make appointments with users. In my workplace, I give each user a scenario and a computer. I observe them and time them as they complete the tasks by using the software package. My hope is that statistical analysis will give me results that I can report, including the average time on task with error bars, as the graph (right) illustrates.

Study 2 fits with the view (above, right) that “you cannot be objective about what you know” because all research takes place within a context. To lessen the impact of conducting research, I contact users to ask if I can study their workplace. I observe each user for a day. My hope is to analyse the materials and interaction that I’ve observed in context—complete with typical interruptions, distractions, and stimuli. Since a new software version has just been released, my hope is that I’ll get to observe them as they upgrade. I’ll report any usability issues, interaction-design hurdles, and unmet needs that I observe.

The above are compilations of studies I conducted.

  • Study 1 revealed several misunderstandings and installation problems, including a user who abandoned the installation process because he believed it was complete. I was able to report the task success rate and have the install wizard fixed.
  • Study 2 revealed that users write numbers on paper and then re-enter them elsewhere, which had not been observed when users visited our site for usability testing. One user told me: “I never install  the latest version because the updates can be unstable,” and another said: “I only upgrade if there’s a fix for a feature I use” to avoid unexpected new defects. I was able to report the paper-based workaround and the users’ feelings about quality, for product managers to reflect in future requirements.

Clearly, there’s more than one way to conduct research, and not every method fits every team. That’s an idea that can be explored at length.

This has me wondering: which method fits what, when, where? Is there a relationship between a team’s development process and the approach to user research (epistemology) that it’s willing to embrace? …between its corporate usability maturity and the approach?

Those are two of the lines of inquiry in my research at Simon Fraser University.

If you liked this post, you may also like Are usability studies experiments?

Posted on

Standard OK-Cancel button order

I have two stories about command buttons.

Quite a few years ago, a team member walked me through a new dialog box. He entered some data, and then unintentionally clicked the Cancel button. He made this error twice in a row, thus losing his changes twice in a row. I pointed out that the OK and Cancel buttons were in the wrong order. The developer switched the buttons to the Windows-standard layout (below, right), and the user-performance problem was solved.

A few years later, on a different project, not only were the buttons in non-standard order, they used non-standard wording and they used coloured icons. My request to follow the Windows standard was met only half-way and then sent for Beta testing before I saw it again. The buttons were now in the correct order, but the button names were changed, and the names and icons were still non-standard. Beta testers loudly protested the change. (Beta testers are often expert users, and experts abhor any change that slows them down.) At the time, the company was only a few steps up the Neilsen Corporate Usability Maturity model, so instead of completing the change to Windows-standard OK and Cancel buttons, the buttons were rolled back, to appease the protesting Beta users. I found out too late to retest with Windows-standard buttons, so there was no data to convince the developers. For me, it was an opportunity to learn from failure. :)

Why is non-standard so hard?

Try this Stroop test (right). Ignore the words. Instead, identify the colours, out loud. No doubt, the second panel went slower and took more effort.

Try the variation, at left. Find the first occurrence of the word Blue. Next, find the first occurrence of the colour █.

Just as mismatches between text and colour slow your Stroop-test performance, mismatches between standard and non-standard OK and Cancel buttons slow user performance. Our Beta users clicked the wrong buttons—a huge waste of their time—because the new solution didn’t follow any standard. The Beta testers were right to protest, but wrong in their demand to revert to the original non-standard state. (See: Customers can’t do your job.)

Users learn GUI patterns—patterns that are widely reinforced by user experience—and users expect GUI to behave predictably, so it’s unwise to deviate radically from the standards, unless there are product-management reasons to do so.

I’ll write more about following standards versus designing something new in the coming few posts.

P.S. It looks like Jakob Nielsen got here before me.

Posted on

Users are not used to it

For several years, I did usability testing on CAD-style software that was full of legacy code, some of which preceded Windows 98.

Some of that legacy code dealt with CAD objects that displayed on screen. To work with these objects, users had a choice of menu commands and toolbar buttons, supplemented by dialog boxes. For example, to move an object, users could not simply click and drag it; they would choose a command, click the object, and then, in a dialog box, enter the distance to move the object.

That’s the way CAD programs worked when that legacy code was originally written.

Over the years, during my usability testing of various features, I noticed a growing trend toward direct manipulation. That is, to work with an object, users would try to click it or drag it. They would do this without thinking. Even long-time users, faced with a new feature (studies from 2005-2006), would try direct manipulation first:

  • 100% of the test subjects clicked a cube, trying to select it.
  • 100% of the test subjects dragged a point or line, trying to move it.
  • 100% of the test subjects clicked in the window, trying to create a point.
  • 100% dragged across points, trying to select them.

But the new features were built on the legacy code, so had the command-driven interaction style. A simple click on the object was usually a dead end for users.

And the users would say: “Darn,” and then look for another vector—another pathway—to complete the task.

The reasons we didn’t provide direct manipulation:

  • “Our users are used to the way it is now.” Clearly, usability-test results negated that argument. Users are not so accustomed to old-style interaction, because their first instinct for new CAD tasks in an existing product was direct manipulation.
  • “There’s not enough bang for the buck” because the opportunity cost (the cost of skipping other possible projects) was deemed too great. It’s hard to argue with this, as a usability analyst. The company opted for more features, and may have increased its risk of being leapfrogged by the competition, as discussed in an earlier blog.
Posted on

Your usability advantage

When businesses buy software, rather than choose the software with the lowest purchase price, they ought to consider the total cost of ownership—including the added productivity and enjoyment that usability and user-experience provide.

Every software company will say “our product is usable,” so how can you prove to prospective customer that you’ve really got usability?

Your product has a usability advantage if:

  1. Your development team’s motivation is right. Software meets customer business needs if came out of a design and development process that considers stakeholders beyond the development team.

    2. Incidentally, getting the motivation right is what Five Sketches™ was designed to help development teams to do.

  2. Trials quickly reveal product effectiveness. In a hands-on trial, you want users to try common tasks, figure them out, and say they liked the experience. A good hands-on trial reduces a competitor’s vendor demo to an infomercial.
  3. It’s about information more than data. Data requires cognitive transformation in the user’s head to become information. Information is ready now to support insight and appropriate action.
  4. Change management is minimal. Your mental model is clearly evident and the user experience is pleasant, so resistance to change is lower. Employees will see evidence of leadership rather than another “solution” imposed on them.
  5. Your training teaches skills. Pick one: training that leads users through a maze (an unusable-interface), or training that teaches users smarter ways to work toward their goals.
  6. You have metrics. If you tell customers how long it will take new users to start performing, you show your respect for their total cost of ownership.
  7. You have references. A product reference is as close as a Google search. In a web-2.0 world, your best “reference” could be an engaged, loyal user community.

The first 4½ to 5 points, above, require the Development team’s involvement, and the last few benefit from Dev involvement. Clearly, a usability advantage requires the involvement of other departments, directed by a product manager who works the Marketing, Sales, Support, and Development teams in concert. :)

This post was inspired by a Howard Hambrose article in Baseline Magazine, which recommends that IT professionals question software usability before they buy and implement.

Posted on

Heuristics at the design stage

On the IxDA’s discussion list for interaction designers, Liam Greig posted his “human friendly” version of a heuristics checklist based on Nielson’s originals and the ISO’s ergonomics of human-system interactions.

Here are  just the headings and the human-friendly questions, which are useful at a project’s design stage.

A design should be…

  • transparent. Ask: Where am I? What are my options?
  • responsive. Ask: What is happening right now? Am I getting what I need?
  • considerate. Ask: Does this make sense to me?
  • supportive. Ask: Can I focus on my task? Do I feel frustrated?
  • consistent. Ask: Are my expectations accurate?
  • forgiving. Ask: Are mistakes easy to fix? Does the technology blame me for errors?
  • guiding. Ask: Do I know where to go for help?
  • accommodating. Ask: Am I in control? Am I afraid to make mistakes?
  • flexible. Ask: Can I customize my experience?
  • intelligent. Ask: Does the technology know who I am? Did the technology remember the way I left things?

Sometimes, at the analysis end of a Five Sketches™ ideation-design session, the design participants see more than one path forward. Use heuristics to frame the discussion of how each path rates, to reach a decision faster.

The heuristics can be about more than just usability. You can also assess coding costs, maintenance costs, code stability…. And, of course, you must also assess potential designs against the project’s requirements.

Posted on

From napkin to Five Sketches™

In 2007, a flash of insight hit me, which led to the development of the Five Sketches™ method for small groups who need to design usable software. Looking back, it was an interesting journey.

The setting. I was working on a two-person usability team faced with six major software- and web products to support. We were empowered to do usability, but not design. At the time, the team was in the early stages of Nielsen’s Corporate Usability Maturity model. Design, it was declared, would be the responsibility of the developers, not the usability team. I was faced with this challenge:

How to get usable products
from software- and web developers
by using a method that is
both reliable and repeatable.

The first attempt. I introduced each development team to the usability basics: user personas, requirements, paper prototyping, heuristics, and standards. Some developers went for usability training. In hindsight, it’s easy to see that none of this could work without a formal design process in place.

The second attempt. I continued to read, to listen, and to ask others for ideas. The answer came as separate pieces, from different sources. For several months, I was fumbling in the metaphorical dark, having no idea that the answer was within reach. Then, after a Microsoft product launch on Thursday, 18 October, 2007, the light went on. While sitting on a bar stool, the event’s guest speaker, GK Vanpatter, mapped out an idea for me on a cocktail napkin:

  1. Design requires three steps.
  2. Not everyone is comfortable with each of those steps.
  3. You have to help them.
Some key design ideas, conveyed to me on a napkin sketch in a Vancouver bar.

The quadrants are the conative preferences or preferred problem-solving styles.

I recognised that I already had an answer to step 3, because I’d heard Bill Buxton speak at the 2007 UPA conference, four months earlier. I could help developers be comfortable designing by asking them to sketch.

It was more easily said than done. Everyone on that first team showed dedication and courage. We had help from a Vancouver-based process expert who skilfully debriefed each of us and then served us a summary of remaining problems to iron out. And, when we were done, we had the beginnings of an ideation-and-design method.

Since then, it’s been refined with additional teams of design participants, and it will be refined further—perhaps changed significantly to suit changing circumstances. But that’s the story of the first year.

Posted on

Functional sophistication, not complexity

Some software companies add ever more features to their software as a way to differentiate it from its competitors. Lucinio Santos’ lengthy analysis of sophistication versus complexity includes this graphic:

functional-sophistication-not-complexity

An excellent example of simplification is the Microsoft Office ribbon. Many users who upgrade dislike the ribbon for months because of the sheer amount of GUI change it imposes, but the ribbon successfully simplifies and makes existing features more discoverable.

Incidentally, the Office ribbon was designed by a design team using generative design. I facilitated a ribbon-design project that used a team of developers Five Sketches™—a method that incorporates a generative design.

Posted on

Choose: usability or features?

I was talking to a B2B product manager who told me “The industry we target sees little difference between our product and our competitors.” Their plan is to differentiate their product from its competitors. My question: to make your software different from that of the competition, should you mainly add new functionality or mainly improve the usability?

LeapfroggingBob Holt addresses this question in his article, Death by 1000 cuts. He asks: “As the worlds of our customers and our own business models continue to change and evolve, should we be changing the balance between improving the usability of our current products and adding new functionality?” Holt answers his own question: “I say absolutely yes. After all, it wouldn’t it be a shame if our revenues bled out through a thousand little cuts while we are rushing around trying to build the next Big Thing?”

But the same product manager I mentioned above also told me: “I firmly believe that you cannot win only by addressing usability. You must also look for the future of the market—that is, you won’t innovate past your current box—to avoid getting leapfrogged.”