Tag: software development

Posted on

Which user involvement works

User-centred design (UCD) advocated involving users in the design process. Have you wondered what form that user involvement could take, and which forms lead to the most successful outcomes?

I recently came across data that Mark Keil published a while ago. He surveyed software companies and correlated project outcomes with the type of user access that designers and developers had.

Type of contact with users Effectiveness
  For custom software projects
  Facilitate teams, hold structured workshop with users, or use joint-application development (JAD). ██████████
  Expose users to a UI prototype or early version to uncover any UI issues. ██████
  Expose users to a prototype or early version to discover the system requirements. ████
  Hold one-on-one semi-structured or open-ended interviews with users. ████
  Test the product internally (acceptance testing rather than QA testing for bugs) to uncover new requirements. ██
  Use an intermediary to define user goals and needs, and to convey them to designers and developers. ██
  Collect user questions, requirements, and problems indirectly, by e-mail or online locations. █
  For packaged or mass-market software projects
  Listen to live/synchronous phone support, tech-support, or help-desk calls. ████████
  Hold one-on-one semi-structured or open-ended interviews with users. ██████
  Expose users to a UI prototype or early version to uncover any UI issues. ████
  Convene a group of users, from time to time, to discuss usage and improvements. ████
  Expose users to a prototype or early version to discover the system requirements. ██
  Test the product internally (acceptance testing rather than QA testing for bugs) to uncover new requirements. ██
  Consult marketing and sales people who regularly meet with and listen to customers. ██
  At trade shows, show a mock-up or prototype to users and get their feedback. █
Not reported as effective in this 1995 source
  Conduct a (text) survey of a sample of users.
  Conduct a usability test to “tape and measure” users in a formal usability lab. (This study precedes such products as TechSmith Morae.)
  Observe users for an extended period, or conduct ethnographic research.
  Conduct focus groups to discuss the software.

Although Keil’s article includes quantitative data, his samples are small. I opted to show only the relative usefulness of various methods. My descriptions, above, are long because the original article uses 1995 terms that have shifted in meaning. I believe some of the categories now overlap, due to changes in technology and method. For example, getting users to try a prototype of the UI in order to uncover UI issues sounds like the early usability testing that I do with TechSmith Morae, yet the 1995 results gave these activities very different effectiveness ratings.

For details, see the academic article by Mark Keil (Customer-developer links in software development). Educational publishers typically require a fee for access.

These methods also relate to research I’m doing on epistemology of usability analysis.

Posted on

Internet Explorer leapfrogs Firefox?

Previously, I wrote about GUI—when to copy it and when to design it. When your competition has something better, I recommended you design, to leapfrog your competitor. Here’s an example of two competing web browsers:

Click to enlarge

At first glance, the new Internet Explorer 8 address bar looks like a copy of Firefox’s existing awesome bar, but click the image for an enlarged view. You’ll see that:

  • The on-the-fly suggestions are grouped as History and Favourites.
  • Each group lists only five items, by default.
  • To remove an item (think stale links and mistyped URLs), highlight the item and then click the  ×  that appears.

Compared to Internet Explorer 7, Firefox had a better address bar. And just as clearly, the additional features of the Internet Explorer 8 address bar are an attempt to leapfrog Firefox. After the public has used IE8 for three months, it’ll be interesting to hear whether users think Microsoft succeeded.

Read the related post, GUI: Copy it or design it.

Posted on

Epistemology of usability studies

Currently, I’m conducting research on usability analysis and on how Morae software might influence that. My research gaze is rather academic, in that I’m especially interested in the epistemology of usability analysis.

One of my self-imposed challenges is to make my research relevant to usability practitioners. I’m a practitioner and CUA myself, and I have little time for academic exercises because I work where the rubber hits the road. This blog post outlines what I’m up to.

At Simon Fraser University, I learned that epistemological approaches have different assumptions about what is knowable. On one side (below, left), it’s about numbers, rates, percentages, graphs, grids, tables, proving absolute truths. On the other side, (below, right) it’s about seeking objectivity while knowing that it’s impossible because everything has a cultural context. The epistemology you choose, when doing research, depends on what you believe. And the epistemology dictates what methods you use, and how you report your results.

You can be
certain of
what you know.		 You cannot be
objective about
what you know.

Let’s look at some examples.

Study 1 fits with the view (above, left) that “you can be certain of what you know.” I plan and conduct a quantitative study to measure the time it takes a series of users to complete two common tasks in a software package: upgrading to the latest version of the software, and activating the software. I make appointments with users. In my workplace, I give each user a scenario and a computer. I observe them and time them as they complete the tasks by using the software package. My hope is that statistical analysis will give me results that I can report, including the average time on task with error bars, as the graph (right) illustrates.

Study 2 fits with the view (above, right) that “you cannot be objective about what you know” because all research takes place within a context. To lessen the impact of conducting research, I contact users to ask if I can study their workplace. I observe each user for a day. My hope is to analyse the materials and interaction that I’ve observed in context—complete with typical interruptions, distractions, and stimuli. Since a new software version has just been released, my hope is that I’ll get to observe them as they upgrade. I’ll report any usability issues, interaction-design hurdles, and unmet needs that I observe.

The above are compilations of studies I conducted.

  • Study 1 revealed several misunderstandings and installation problems, including a user who abandoned the installation process because he believed it was complete. I was able to report the task success rate and have the install wizard fixed.
  • Study 2 revealed that users write numbers on paper and then re-enter them elsewhere, which had not been observed when users visited our site for usability testing. One user told me: “I never install  the latest version because the updates can be unstable,” and another said: “I only upgrade if there’s a fix for a feature I use” to avoid unexpected new defects. I was able to report the paper-based workaround and the users’ feelings about quality, for product managers to reflect in future requirements.

Clearly, there’s more than one way to conduct research, and not every method fits every team. That’s an idea that can be explored at length.

This has me wondering: which method fits what, when, where? Is there a relationship between a team’s development process and the approach to user research (epistemology) that it’s willing to embrace? …between its corporate usability maturity and the approach?

Those are two of the lines of inquiry in my research at Simon Fraser University.

If you liked this post, you may also like Are usability studies experiments?

Posted on

Generative design vs. Five Sketches™

© Leah Buley, from her presentation on SlideShareLeah Buley talked about generative design at the South by Southwest Interactive conference today [in March 2009]. Buley feels design methods are lacking in the set of professional tools we use for software development: “We don’t have so many good, reliable, repeatable design techniques.” I agree with her.

Buley tells how, in her first design session at Adaptive Path, she was handed a pen and paper, and told to sketch. The point was “to crank out a lot of ideas in a short time.” That’s what generative design is all about: saturating the design space with ideas. Design programs teach generative design, but there’s no generative design taught in programs for developers, QA staff, technical communicators, product management, or marketing.

There are already several design methods for software development teams to choose from, including Buley’s wonderful grab bag. Others I know of are Five Sketches™ (obviously) and Design Studio, both of which focus on the complete process of producing a design, start to finish, with all its challenges. Microsoft’s Bill Buxton told the UPA 2007 conference that he insists on generative design, and I’d love to see that in action. Each method differs slightly, but they all work because….

Why do they all work? Because of generative design. Generative design addresses what Buley calls her “dirty secret.” She freely confesses, about her design work before Adaptive Path: “I had very little confidence that what I was presenting as the design was in fact the one, optimal solution to the problem.” My experience teaching Five Sketches™ tells me that once you’ve participated in a generative-design process, you’ll know that you can have that confidence.

A related question: when will computer science programs teach Basic Design Methods to developers?

Posted on

Your usability advantage

When businesses buy software, rather than choose the software with the lowest purchase price, they ought to consider the total cost of ownership—including the added productivity and enjoyment that usability and user-experience provide.

Every software company will say “our product is usable,” so how can you prove to prospective customer that you’ve really got usability?

Your product has a usability advantage if:

  1. Your development team’s motivation is right. Software meets customer business needs if came out of a design and development process that considers stakeholders beyond the development team.

    2. Incidentally, getting the motivation right is what Five Sketches™ was designed to help development teams to do.

  2. Trials quickly reveal product effectiveness. In a hands-on trial, you want users to try common tasks, figure them out, and say they liked the experience. A good hands-on trial reduces a competitor’s vendor demo to an infomercial.
  3. It’s about information more than data. Data requires cognitive transformation in the user’s head to become information. Information is ready now to support insight and appropriate action.
  4. Change management is minimal. Your mental model is clearly evident and the user experience is pleasant, so resistance to change is lower. Employees will see evidence of leadership rather than another “solution” imposed on them.
  5. Your training teaches skills. Pick one: training that leads users through a maze (an unusable-interface), or training that teaches users smarter ways to work toward their goals.
  6. You have metrics. If you tell customers how long it will take new users to start performing, you show your respect for their total cost of ownership.
  7. You have references. A product reference is as close as a Google search. In a web-2.0 world, your best “reference” could be an engaged, loyal user community.

The first 4½ to 5 points, above, require the Development team’s involvement, and the last few benefit from Dev involvement. Clearly, a usability advantage requires the involvement of other departments, directed by a product manager who works the Marketing, Sales, Support, and Development teams in concert. :)

This post was inspired by a Howard Hambrose article in Baseline Magazine, which recommends that IT professionals question software usability before they buy and implement.

Posted on

Heuristics at the design stage

On the IxDA’s discussion list for interaction designers, Liam Greig posted his “human friendly” version of a heuristics checklist based on Nielson’s originals and the ISO’s ergonomics of human-system interactions.

Here are  just the headings and the human-friendly questions, which are useful at a project’s design stage.

A design should be…

  • transparent. Ask: Where am I? What are my options?
  • responsive. Ask: What is happening right now? Am I getting what I need?
  • considerate. Ask: Does this make sense to me?
  • supportive. Ask: Can I focus on my task? Do I feel frustrated?
  • consistent. Ask: Are my expectations accurate?
  • forgiving. Ask: Are mistakes easy to fix? Does the technology blame me for errors?
  • guiding. Ask: Do I know where to go for help?
  • accommodating. Ask: Am I in control? Am I afraid to make mistakes?
  • flexible. Ask: Can I customize my experience?
  • intelligent. Ask: Does the technology know who I am? Did the technology remember the way I left things?

Sometimes, at the analysis end of a Five Sketches™ ideation-design session, the design participants see more than one path forward. Use heuristics to frame the discussion of how each path rates, to reach a decision faster.

The heuristics can be about more than just usability. You can also assess coding costs, maintenance costs, code stability…. And, of course, you must also assess potential designs against the project’s requirements.

Posted on

Designing *with* developers

Today, Joel Spolsky blogged about development process and design. He makes a couple of points I agree with. As an example, he says that developers don’t know how to do everything. He says it first by describing his lack of skills in an early job at Microsoft, and later by describing the lack of skills in very experienced developers:

Joel SpolskyYour garden-variety super-smart programmer is going to come up with a completely baffling user interface that makes perfect sense IF YOU’RE A VULCAN (cf. git). The best programmers are notoriously brilliant, and have some trouble imagining what it must be like not to be able to memorize 16 one-letter command line arguments. These programmers then have a tendency to get attached to their first ideas, especially when they’ve already written the code.

At this point, reading Spolsky’s blog, I’m like the eager school kid with his hand up: “Oh, oh, I know the answer!” Five Sketches™ was specifically developed to prevent the people-attached-to-their-first-ideas problem and to ensure collaboration between developers and other members of the team.

Software products are developed by teams, but, as Spolsky goes on to say—about developers—some team members have more power those who work alongside them, because developers control the code. He tells this story:

Joel Spolsky       A programmer asks me to intervene in some debate he is having with a [non-developer peer; in this story it’s a Program Manager who performs a design function].
       “Who is going to write the code?” I asked.
       “I am….”
       “OK, who checks things into source control?”
       “Me, I guess, ….”
       “So what’s the problem, exactly?” I asked. “You have absolute control over the state of each and every bit in the final product. What else do you need? A tiara?”

Spolsky’s solution is to rely on the ability of others [in this case, the Program Manager] to convince the developer to make necessary changes. But requiring stakeholders and colleagues to engage in persuasion is risky; the variables include not only technical skill and experience, but also credibility, and rhetoric, persuasion, and interpersonal communication skills of both parties.

I recall when we first developed our ideation-design process, on an interdisciplinary team. We had Sharon to debrief each of us, to find out what had worked and what hadn’t, and then we addressed the shortcomings and frustrations in the process. This is how we developed Five Sketches™. And this is why it works … with developers … and with people from QA, Marketing, Tech-Comm, and so on.

By the way, you can read Joel Spolsky’s entire blog post here.

Posted on

Customers can’t do your job

Agile methodology can produce usable products, as long as you know what you’re doing. A common pitfall in agile is this incorrect assumption that you’ll get a usable product simply by building what the client tells you to build.

When there is some question about how to make a feature usable, customers may have something to say, but their answers are more likely based on opinion and emotion rather than on design experience and behavioural observation.

Brazilian blogger and computer-science- and HCI expert, Francisco Trindade, gives this illustration: If you had asked people how they would like to search for Web pages in the pre-Google era, how many would have asked for a blank page with a text box on it…?

Can customers design? Probably not.Trindade says that, regardless of whether this ask-the-client behaviour is laziness or a responsibility-avoidance strategy, people who design software need to “stop pretending that the client has all the answers, and trust a little bit more in themselves to create solutions.”

Creating solutions? That’s a job for developers and the Five-Sketches™ method, or any other design method they’re comfortable with.

Posted on

From napkin to Five Sketches™

In 2007, a flash of insight hit me, which led to the development of the Five Sketches™ method for small groups who need to design usable software. Looking back, it was an interesting journey.

The setting. I was working on a two-person usability team faced with six major software- and web products to support. We were empowered to do usability, but not design. At the time, the team was in the early stages of Nielsen’s Corporate Usability Maturity model. Design, it was declared, would be the responsibility of the developers, not the usability team. I was faced with this challenge:

How to get usable products
from software- and web developers
by using a method that is
both reliable and repeatable.

The first attempt. I introduced each development team to the usability basics: user personas, requirements, paper prototyping, heuristics, and standards. Some developers went for usability training. In hindsight, it’s easy to see that none of this could work without a formal design process in place.

The second attempt. I continued to read, to listen, and to ask others for ideas. The answer came as separate pieces, from different sources. For several months, I was fumbling in the metaphorical dark, having no idea that the answer was within reach. Then, after a Microsoft product launch on Thursday, 18 October, 2007, the light went on. While sitting on a bar stool, the event’s guest speaker, GK Vanpatter, mapped out an idea for me on a cocktail napkin:

  1. Design requires three steps.
  2. Not everyone is comfortable with each of those steps.
  3. You have to help them.
Some key design ideas, conveyed to me on a napkin sketch in a Vancouver bar.

The quadrants are the conative preferences or preferred problem-solving styles.

I recognised that I already had an answer to step 3, because I’d heard Bill Buxton speak at the 2007 UPA conference, four months earlier. I could help developers be comfortable designing by asking them to sketch.

It was more easily said than done. Everyone on that first team showed dedication and courage. We had help from a Vancouver-based process expert who skilfully debriefed each of us and then served us a summary of remaining problems to iron out. And, when we were done, we had the beginnings of an ideation-and-design method.

Since then, it’s been refined with additional teams of design participants, and it will be refined further—perhaps changed significantly to suit changing circumstances. But that’s the story of the first year.

Posted on

Functional sophistication, not complexity

Some software companies add ever more features to their software as a way to differentiate it from its competitors. Lucinio Santos’ lengthy analysis of sophistication versus complexity includes this graphic:

functional-sophistication-not-complexity

An excellent example of simplification is the Microsoft Office ribbon. Many users who upgrade dislike the ribbon for months because of the sheer amount of GUI change it imposes, but the ribbon successfully simplifies and makes existing features more discoverable.

Incidentally, the Office ribbon was designed by a design team using generative design. I facilitated a ribbon-design project that used a team of developers Five Sketches™—a method that incorporates a generative design.