Entries from October 2008 ↓

YOU may have access to free usability testing

Yes, free usability testing is alive and well and even you, dear reader, can find it and use it. Free usability testing is also called “customer feedback” and is readily available in the form of comment cards, web forms, or telephone transcripts. At my company, I read transcripts from chat conversations we have with our customers. It clearly puts into perspective what issues our customers are dealing with, what products or services they are trying to acquire to help them with their issues, and any comments or feedback they have about usability issues, such as difficulty using web pages or web forms when trying to purchase services.

Now I’ll be the first to tell you that customer feedback in and of itself is not the same as conducting actual usability testing with users in a 1-on-1 environment. However, it is a great source of general usability feedback, and in the sense that it’s real users, trying to accomplish real tasks on your real applications, can be a treasure chest of helpful usability clues which point you in the direction of further sleuthing usability issues.

Customer Feedback is free usability testing

This customer feedback is a form of free usability testing, in the sense that your firm probably already collects customer feedback from a variety of sources, and customers will tell you (sometimes in painful detail) what’s bugging them or what’s making them unhappy.

The first trick is finding the feedback, and then the real tricky part is analyzing the feedback and knowing what actions to take based on any suspected usability issues found. Much of the customer feedback you read may not necessarily align with usability issues. Don’t fret, keep on digging. You may not be able to act on much of the customer feedback you read, again, don’t fret, simply note any issues or concerns that others in your firm may need to know about, and forward as necessary. Eventually, if you read enough, you’ll soon begin seeing a pattern of recurring comments about a particular issue or problem that is usability related, and THAT my friends is the free usability testing you’ve been searching for.

Analyzing customer feedback

When analyzing customer feedback, I try to bucket the recurring comments into three or so groups of usability related items:

The first group is critical usability issues, for example a large number of comments about a task in a web form that gives error messages. Critical issues are usually highly specific to a process or task and are narrow in focus. These are issues that if resolved, could add a larger amount of completed transactions for your form. Now please don’t expect the comments from users to provide detailed usability explanations of what the task was, and what the error was that causes the task to fail. Customers don’t understand the details of a process and more than likely are just going to say something like “your web site sux, it crashed on me.” You’ll have to do some sleuthing, but should be able to idenfity the issue fairly quickly.

The second group is important usability issues, which as the name implies are usability issues that should be addressed, but are not deemed “critical.” These issues may or may not be easy to fix, and may appear to be more broad natured. They are important in the sense that by addressing them, you can either fix usability issues or add functions or features that provide a better customer experience. For example, you may see many feedback comments from customers that are complaints about not having certain information available when they are reviewing their customer service account information online. You may not be able to immediately acton on this type of information, but by tracking the number of requests for this function over time, you can build a case for adding this new feature to your existing customer service application.

The third group is general usability issues, which is a bucket of either one-off usability items, or modest usabilty issues that you already know about. It helps to track these issues, if only to put forth a case of potential usability enhancements in the future. Examples of this type of usability feedback may be issues about the speed of the web site, or the inability to change the formatting of displayed information, or any other item that has to do with usability, but is not repeated in other customer feedback comments.

Usability Feedback in Comments

Although it takes time, pouring through the comments received from your customers is a great source of free usability testing. Don’t have feedback forms on your web site or in your internal applications? Get them! Make a case for why comments are so important to your organization. By using real customers, and their real feedback about your products, services or applications, you’ll be making use of a great form of usability information.

What is Useful Usability?

I like to think of a definition of “useful usability” as:

“Helping to make things easier to use in a quick & easy manner”

“Helping”

I think helping in this useful usability definition refers to the fact that the vast majority of usability practitioners do not actually code and make their recommended usability changes. As usability experts, all the usability practitioners I’ve ever met or known typically test, then suggest or identify improvements to teams that actually do the work. This is not a negative, it’s the same concept as an architect who does not grab hammer and nails to build the design he or she developed. What this means is that usability practitioners must be experts at propagating ideas and concepts, and must work very well with teams, both business and IT, to communicate ideas, concepts and specific recommendations for change.

“Make things easier to use”

Making things easier is what useful usability is all about. Whether it’s a ball-point pen, a company web site or a cell phone, making something easy to use provides increased user satisfaction, which ultimately provides businesses with a product or service that can be sold well. Making things easier from the business’ perspective means making things more profitable. If we do our job well, then companies make more money. We do that by making things easier for the end customer, thus more satisfying, resulting in a more rewarding customer experience which ultimately drives sales (and thus profits) for firms.

“But,” you say, “I don’t build external stuff, I only work on employee applications, or I work for a not-for-profit.”

Doesn’t matter. By making internal employee applications easier to use you are helping to drive more profits, in the form of increased employee production, for your firm. Not-for-profits are the same, your efforts make your not-for-profit more efficient, resulting in better production for the firm, which can take the savings gained and use it elsewhere to better fund operations.

Smart usability practitioners always summarize their work in terms of revenue gained, or operational savings earned for a firm. If you always communicate the business value your efforts are helping to create, you’ll pretty much always have a value for the firm and thus a job!

“Quick and easy manner”

Useful usability is about getting usability done. I once conducted an amazingly long and expensive usability test of the Blue Cross of California web site. We brought users in representing the typical Personas of our users. We set up a location near downtown LA with a full usability lab with multiple cubes, computers, video cameras and usability analysts. It took me about 9 months, from inception of the project to final delivery of the analysis document. In a massively attended meeting with all the VP “stakeholders” of each division of the company present, we delivered our analysis of where there were usability issues with the site. Half way through the presentation, one of the VP stakeholders interrupted the presentation. “OK, we get it, the web site sucks” he said. “So what are your doing to fix it?” I blinked several times, at a loss for words. The reality was I had done nothing to fix it, I had just spent almost a year simply analyzing it and pointing out what needed to be fixed. I had entered the “analysis paralysis” zone.

From that point forward, I realized that to be effective, usability should be useful, meaning efficient. Useful usability then became my mantra. Do usability testing and optimization simply, quickly, and deliver results to make changes. Usability does not have to be a mind-numbing exercise of massive proportions. It can be as simple as paper and pencil wireframes, card-sorts using 3×5 index cards and observing real users over their shoulder as they interact with a ball-point pen, web site or a cell phone.

What Useful Usability Means

So, the definition of useful usability is really about doing your usability tasks simply, quickly and delivering results that can be acted upon almost immediately. By making usability useful, you make yourself useful, and ultimately you help your firm and it’s customers, which in my opinion is one of the most satisfying and rewarding aspects of being a usability practitioner.

Contextual research and observation are crucial to good usability, including the usability of everyday objects that we love to hate.

I hate my coffee maker. No, I mean I really hate it. It’s a new one, I got it as a gift because my old one (which I quite liked) “broke” when it apparently leapt off the shelf of the cabinet and crashed down onto the floor. Or so my wife would have me believe.

The new one is pretty, but it’s big. My wife does NOT want the coffee maker on the kitchen countertops, so I dutifully make my morning coffee, sit down and dutifully write out this blog (which I think only my mom reads – Hi Mom!) and sip my coffee.

When I’m done blogging, I dutifully put the coffeemaker away, back up into the kitchen cabinet where it lives (out of the way of my wife) and go about my normal business.

Contextual Research and Your Kitchen

So why should I hate my coffee maker? Because I like to program it the night before to have my hot, steaming, nutritious coffee ready for me when I come downstairs the next morning. Which means each night I have to re-set the time, and then re-set the brew time. Each night. Every night. You see, my coffee maker forgets what time it is each time I unplug it from the wall and put it away in it’s wife-friendly hiding space in the cabinet. There’s no battery, so each time I unplug it it “forgets”

I don’t know if usability research was conducted on my coffee maker, but imagine with me that it was, and that you and I are flys on the wall of the usability research design center at the coffee maker manufacturer’ headquarters. We’re watching the usability team conduct usability research with some typical subjects who match coffee-drinker Personas. The usability researchers are busy conducting observations to see exactly what features and functions of coffee makers the users most liked, or thought critical to coffee making success.

Labs Don’t Offer Contextual Observation

As we buzz around the room, we note that the subjects are dutifully using several brands and versions of coffee makers, all of which are plugged into the walls of the lab. What’s missing however is the kitchen environment, and most importantly the wife who refuses to let the coffee maker live on said kitchen countertop. Our imaginary usability team does a good job observing the users, and determines the several features and functions that are critical, such as easy to load coffee filter holders, easy to pour water access, and easy to pull in and out coffee pots.

If our imaginary design team sends this information to the manufacturing team, and they build the coffee maker to specs, then theoretically they’ve achieved coffee maker design success. Or have they? They’ve actually left out a crucial part of coffee maker usability. What about all those people who have to unplug their coffee makers each morning? How will the coffee maker “remember” the time? Many coffee makers use a battery that keeps track of the time when unplugged from the power at the wall. Sadly, this coffeemaker has no such convenience, because our imaginary team did not conduct contextual observation and so did not realize the importance of the battery-kept time.

Contextual Research Offers New Design Opportunities

If our imaginary team had left their design lab, and observed real coffee maker users out in their kitchens, they more than likely would have observed a household or two where the coffee maker was required to be unplugged and put away into a hiding space in a cabinet each morning. They also would have observed that the coffee maker time had to be reset each night in order to brew the coffee at the appropriate time each morning. Seeing the struggle each night our imaginary usability team would realize the importance of a coffee maker that remembers the time.

The moral of this story? Observation in the lab is not contextual observation, and thus does not reflect a user’s environment and their “reality” of usage. Lab observation is helpful, but without contextual observation critical tasks can be overlooked, resulting in a design that is less than perfect, and users who are less than satisfied.

As for me? my coffee is gone (yum, it was good!), and this usability post is about over. My wife is eyeing the coffee maker on the countertop, so I better get going and put my coffee maker away to preserve our happy family environment. I’ll have to re-program the time tonight [sigh].