Usability research and analysis – Page 8 – Five Sketches™ Consulting helps your development team with user-experience research and design to meet your goals and your customers' needs

8 April 200913 April 2009

How to test earlier

Involving users throughout the software-development cycle is touted as a way to ensure project success. Does usability testing count as user contact? You bet! But since most companies test their products later in the process, when it’s difficult to react meaningfully to the user feedback, here are two ways to get your testing done sooner.

Prioritise. Help the Development team rank the importance of the individual programming tasks, and then schedule the important tasks to complete early.

If a feature must be present in order to have meaningful interaction, then develop it sooner.

For example, email software that doesn’t let you compose the message is meaningless. To get meaningful feedback from users, they need to be able to type an e-mail.

Developers often want to start with the technologically risky tasks. Addressing that risk early is good, but it must be balanced against the risk of a product that’s less usable or unusable.

If a feature need not be present or need not be working fully in order to have meaningful interaction, then provide hard-coded actions in the interim, and add those features later.

For example, if the email software lets users change the message priority from Standard to Important, hard-code it for the usability test so the priority is always Standard.

If a less meaningful feature must to be tested because of its importance to the business strategy, then develop it sooner.

For example, email software that lets users record a video may be strategically important for the company, though users aren’t expected to adopt it widely until most laptops ship with built-in cameras.

Schedule. For each feature to be tested, get the Development team to allocate time to respond to usability recommendations, and then ensure this time is neither reallocated to problem tasks, nor used up during the initial development effort of the to-be-tested features. Engage the developers by:

Sharing the scenarios in advance.
Updating them on your efforts to recruit usability-study participants.
After developers incorporate your recommendations, retesting and then reporting improvements in user performance.

Development planning that prioritises programming tasks based on the need to test, and then allows time in the schedule to respond to recommendations, is more likely to result in usable, successful products.

6 April 20096 July 2015

Please exit your comfort zone

People approach business problems in different ways. Some people want to identify the many possible solutions, up front. Others want to fully understand and assess one solution before considering the next solution. And still others want to start implementing a reasonable solution immediately, to see how well it works, and then, in learning from experience, modify that idea to make it work better. This difference in people’s natural tendency or impulse is called conative style—it’s the default way they direct their effort.

But a person’s default behaviour isn’t the only way they can direct their effort. People can adopt a different conative style—for a while. Most people in the workplace have done this successfully. However, when the pressure is on, people tend to resort to their preferred conative style, as they instinctively seek some comfort. And workplace problems often come with pressure, don’t they?

The other thing workplace problems often come with are project teams, inevitably made up of people who have different conative styles. That’s actually a benefit—Dr Meredith Belbin claims that teams team diversity is essential to success for projects in general. For design projects, specifically, team diversity is also important. But consider these pitfalls that design teams naturally face:

Time-and-cost constraints and other pressures mean the team members naturally adopt their default conative styles.
Team members have different conative styles, so they unintentionally work at cross-purposes. One want to generate candidate solutions, another wants to start building, and a third wants to compare and assess.
The team members with the strongest personalities have disproportionate influence.
The contributions of quieter voices get missed, causing disengages team members and overlooked ideas.

This natural dynamic can actually put great design outcomes out of reach. That’s because:

Design is more likely to succeed when many ideas are generated up front, and analysis is delayed until after all ideas are generated.
Idea generation is an ideation conative style, but analysis is a judgement conative style.
Ideation and judgement are opposite conative styles.

Design is stressful for people who have a judgement conative style and haven’t had previous success with ideation. Since, under pressure, people resort to their preferred conative style, Five Sketches™ has specific things to encourage non-designers to engage in generative activity (to generate ideas without analysing them). Developers are great at analysis, logic, reason, examination, which are typical of a judgement conative style, so during Five Sketches training, we present a model of the conative styles that appeals to their judgement: we plot the styles, we provide data from of previous projects (as social proof), and we provide a timeline that shows them ideation will be a temporary activity. During the ideation stage, there are specific actions and reminders to keep people on the ideation side until it’s time for everyone to move to the judgement side of the ideation-judgement axis. Over the space of several hours, this is what happens:

Experience shows that it works. After a brief foray outside their comfort zones, developers and other non-designers get to return to their preferred conative styles. The benefits: design outcomes are better; stakeholders are included and heard; requirements are met; and the development schedule is the same or shorter.

4 April 20096 April 2009

Design can change the culture

I was reading a comment by Creative Sage CIO and CEO, Cathryn Hrudicka, about corporations that want to use social media to reach their market:

The process of trying to become a more “transparent” organization, so they can use social media tools, pushes many organizations—companies, nonprofits, trade associations—into change, especially if they have had a very bureaucratic and non-transparent culture before. That can mean many levels of changes, which usually bring up issues between executives or managers and their teams, and it can affect relationships between staff or team members. Individuals often need to be coached to adapt to the new practices or even to drive them more effectively. Some leadership and teamwork issues come up, too.

It reminded me of a company that changed after I was asked to introduce an ideation-design method. The management team wanted the method to do double duty—a way to empower and encourage the development team.

Prior to this, it seemed team members had been discouraged from taking design initiative. Half the team was disheartened or miserable, the other half of the team was disengaged to the point where they wouldn’t offer a design suggestion even when asked by their boss. Initially, some team members took a wait-and-see approach, but after training and several successful project experiences, the majority of developers became increasingly comfortable with involvement in product design.

It took commitment from key managers to invigorate the team, and it took considerable time. They still use Five Sketches™ today.

If you liked this post, check out a recent post about Google’s corporate culture.

2 April 200918 April 2009

Ideas are disposable

Having a good design idea is not an amazing event. Sketching works People have good design ideas all the time. What is amazing is having lots of good design ideas, so that they can be combined and iterated into the best possible design.

Here’s why ideas are disposable:

Sketching lets you capture lots of ideas, quickly and inexpensively.
The more ideas you try for, the sooner you learn that it’s easy to come up with five or more ideas.
The easier it gets to have lots of ideas, the sooner you realise the value of any one idea is minimal.

Minimal value = disposable.

This is one of the keys to the success of Five Sketches™. Once you learn how easy ideas are to have, you can saturate the design space, iterate the design, and produce excellent design outcomes at will.

31 March 200927 March 2009

Usability, not drowning

Is it a usability experiment? Are they testing the theory that a baby in a bucket won’t fall over as easily as a baby in a bath?

Photo from Yahoo.co.jp news.

Actually, it’s a photo that accompanies a news story about babies in IJmuiden, The Netherlands, whose parents were learning baby massage. The warm bath in advance is supposed to make them feel like when they were in the uterus.

29 March 20094 April 2009

Design and engineering culture

I’m sure Douglas Bowman’s blog last week was widely read. His post was a kind of public exit interview, titled Goodbye Google.

Goodbye Google As Bowman left Google, he pointed out the pro-engineering bias in its approach to problem solving—including problems of design. Two of several examples he gives:

[…] a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. […]

Bowman would like to see more weight put on design principles. His blog includes a link to Wikipedia’s article on design elements and principles, which lists:

•	unity	•	harmony	•	contrast
•	balance	•	repetition, rhythm, pattern	•	variety, alternation
•	emphasis, dominance, focal point	•	proportion, scale	•	functionality
•	attraction	•	artistic unity	•	genuineness in media and form
•	proximity	•	colour

I don’t think design principles are beyond an engineer. I do think engineers need to be taught how—and when—to think about these details.

As we discovered during the development of Five Sketches™, this kind of detail is often outside the comfort zone of an engineer. However, I can affirm that even engineers who initially produce work with little design insight or creativity have managed to astonish me with amazing design results within a year. And that’s after only occasional participation in Five Sketches™ design sessions.

So, yes, engineers can learn to participate in design, with success and predictability, though I would caution that Five Sketches™ works for engineers because it was developed for and with them, to meet their needs.

At Google, to bring about such a cultural change, Bowman would have needed the unwavering support of at least one senior executive. Even then, as Bowman himself acknowledged in his goodbye message, a company as large as Google does not change direction easily.

If you liked this post, read about how a company’s use of social media influences its corporate culture.

27 March 200922 March 2009

User mismatch: discard data?

When you’re researching users, every once in a while you come across one that’s an anomaly. You must decide whether to exclude their data points in the set or whether to adjust your model of the users.

Let me tell you about one such user. I’ll call him Bob (not his real name). I met Bob during a day of back-to-back usability tests of a specialised software package. The software has two categories of users:

Professionals who interpret data and use it in their design work.
Technicians who mainly enter and check data. A senior technician may do some of the work of a professional user.

When Bob walked in, I went through the usual routine: welcome; sign this disclaimer; tell me about the work you do. Bob’s initial answers identified him as a professional user. Once on the computer, though, Bob was unable to complete the first step of the test scenario. Rather than try to solve the problem, he sat back, folded his hands, and said: “I don’t know how to use this.” Since Bob was unwilling to try the software, I instead had a conversation (actually an unstructured interview) with him. Here’s what I learned:

My observation		For this product, this is…
•	Bob received no formal product training. He was taught by his colleagues.	Typical of more than half the professionals.
•	Bob has a university degree that is only indirectly related to his job.	Atypical of professionals.
•	He’s young (graduated 3 years ago).	Atypical of professionals, but desirable because many of his peers are expected to retire in under a decade.
•	Bob moved to his current town because his spouse got a job there. He would be unwilling to move to another town for work.	Atypical. Professionals in this industry typically often work in remote locations for high pay.
•	Bob is risk averse.	Typical of the professionals.
•	He is easily discouraged, and isn’t inclined to troubleshoot.	Atypical. Professionals take responsibility for driving their troubleshooting needs.
•	Bob completes the same task once or several times a day, with updated data each time.	Atypical of professionals. This is typical of technicians.

I decided to discard Bob’s data from the set.

The last two observations are characteristic of a rote user. Some professionals are rote users because they don’t know the language of the user interface, but this did not apply to Bob. There was a clear mismatch between the work that Bob said he does and both his lack of curiosity and non-performance in the usability lab. These usability tests took place before the 2008 economic downturn, when professionals in Bob’s industry were hard to find, so I quietly wondered whether hiring Bob had been a desperation move on the part of his employer.

If Bob had been an new/emerging type of user, discarding Bob’s data would have been a mistake. Imagine if Bob had been part of a new group of users:

What would be the design implications?
What would be the business implications?
Would we need another user persona to represent users like Bob?

26 March 200926 March 2009

Usability testing distant users

When a product’s users are scarce and widely dispersed, and your travel budget is limited, usability testing can be a challenge.

Remote testing from North America was part of the answer, for me. I’ve never used UserVue because the users I needed to reach were in Africa, Australia, South America, and Asia—continents that UserVue doesn’t reach. Even within North America, UserVue didn’t address the biggest problems I faced:

My study participants commonly face restrictive IT policies, so they cannot install our pre-release product and prerequisites.
I need to prevent study participants from risking their data by using it with a pre-release product.
There’s no way to force an uninstall after the usability test. Who else will see our pre-release?

Instead, I blended a solution of my own with Morae, Skype, Virtual Audio Cable, and GoToMeeting. I Testing that's really remote used GoToMeeting to share my desktop, which addresses all three of the problems listed above. I used Skype to get video and audio. I used Virtual Audio Cable to redirect the incoming voice from Skype to Morae Recorder’s microphone channel. Morae recorded everything except the PIP video. It worked. However, my studies were sometimes limited by poor Internet bandwidth to the isolated locations of my study participants.

Amateur facilitators. I realise this is controversial among usability practitioners, but beggars in North America can’t be choosers about how they conduct usability tests on other continents. I developed a one-hour training session for the team of travelling product managers. Training included a short PowerPoint presentation about the concepts, followed by use of Morae Recorder with webcam and headset while role-playing in pairs. The main points I had to get across:

Between study participants, reset the sample data and program defaults.
When you’re ready to start recording, first check that the video and audio are in focus and recording.
While you facilitate, do not lead the user. Instead, try paraphrasing and active listening (by which I mean vernacular elicitation). Remember that you’re not training the users, so task failure is acceptable, and useful to us.

I had a fair bit of influence over the quality of the research, since I developed the welcome script and test scenarios, provided the sample data, and analysed the Morae recordings once they arrived in North America. Due to poor Internet bandwidth to the isolated areas of my study participants, the product managers had to ship me the Morae recordings on DVD, by courier.

It worked. I also believe that amateur facilitation gave the product managers an additional opportunity to learn about customers.

25 March 200927 March 2009

Why products stay pre-chasm

I’ve spent some time working with legacy products—software for which the core code was written before “usability” was a term developers had heard of, back when developers were still called programmers.

I remember my first conversation—held last century—with a developer about his users and the usability of his legacy product. The product-adoption curve, with the chasm I used Geoffrey Moore’s book, Crossing the chasm, to introduce the idea that there are different types of users. The chasm (illustrated) shows five groups of users. The area under the curve represents the quantity of potential users, in the order in which they will typically adopt the product. Users who are technology enthusiasts and visionaries either enjoy or don’t mind being on the bleeding edge because of the benefits they get from using the product, according to Moore. Usability isn’t one of the typical benefits to the left of the chasm, where new products are first introduced. Wider adoption follows only after the product is made to cross the chasm—but this takes a concerted effort on the part of the development team.

What delays a product from crossing the chasm?

Revenues that are sufficient to let the company putter along.
Team members who themselves are tech enthusiasts or visionaries. This occurs, for example, when a nutritionist who also knows how to program develops software for nutritionists.
Team members who have infrequent exposure to newer user experiences. This could include someone who still uses Microsoft Office 2003, eschews social-networking applications on the web, or uses Linux.
Product managers whose roles are weakly defined or absent, making it more likely that new features get developed for the technical challenge of it, rather than for the user need or the business strategy.
Sales reps who ask for more functions in the product in order to make a sale—and a development culture that goes along with this.
Managers whose vision and strategic plan leave out user experience and usability.
Team members who have been on the team for decades.
Organisations that don’t use business cases to help them decide where to apply business resources.
Organisations with poor change-management practices—because moving a product across the chasm is more likely dramatic and disruptive than smooth and gradual.
Designers who are weak or absent in the process, so that “design” happens on the fly, during development.

If you recognise your work environment or yourself in this list, do you want to change things? If you do, but don’t know how, what actions can you take to learn the answer?

24 March 200924 March 2009

Common tasks losing usability

There’s been a loss of usability for people who type text.

Like me, you may have experienced these unwelcome experiences:

After typing a long message in Facebook, when I click send, I get a page error and my entire message is lost.
After typing a post in WordPress, if the server has gone down or I press an unintended keyboard shortcut, I lose my entire post.

By comparison, if Word stops responding, it will (try to) offer me my unsaved changes when I reopen the document.

With users increasingly doing their work in browsers, why don’t browsers remember the text we were typing? Or why doesn’t the operating system? Or why doesn’t the weblication make backups? Or the server?

When my actions in a browser fail, imagine a message such as this:

This hints at just one possible solution. In the spirit of Five Sketches™, I bet you can come up with at least five more ways to support users after they’ve lost the work they typed in a browser.