Usability Testing Overview

An Overview of Usability Testing

A recent Question posted to LinkedIn caught my eye, the question was “How to conduct a usability review of a web product?”  Feeling like I might be able to help, I decided to respond with an Answer.  Unfortunately, LinkedIn has a character limit on their Answers, and as any of my long-term useful usability blog readers will tell you I’m pretty much a failure at responding with anything less than about 3 to 4 thousand characters!

Ravi's Usability Question

So, here then is my “expanded” answer that LinkedIn wouldn’t let me post.  By the way, although the term “usability review” was mentioned, I felt that from the description Ravi provided it is probably more likely that the usability help needed is actually a “usability test.”  What’s the difference?  Here’s my definitions:

“Usability Review” – A review of a web site or application by a usability expert who critiques the usability based on common best practices.  This is also known as a Heuristic Review.

“Usability Test” – A test of a web site or application in which a user conducts a task or tasks while being observed and/or recorded by a usability expert.

Hello Ravi,

It’s good that you wish to conduct usability testing on your website!

From the sound of it, you may be interested in having a usability test conducted on your web site. Although you could try to do this yourself by reading articles, I highly recommend using a trained professional. Especially if the web site in question is business-critical (in other words, your business’ main means of acquiring revenue).

It can be easy to receive poor or worse, bad usability testing results if proper usability testing methodology is not utilized. A usability consultant should be able to help you very quickly establish the correct way to conduct the test. Assuming you are interested in a usability test, I suggest an in-person 1-on-1 usability test, to properly identify task-flow problems among your web site’s critical tasks.

Here’s an overview of the steps necessary to conduct a non-biased 1-on-1 usability test:

1. Identify Personas:

You must conduct usability testing with people who represent your typical users. A fictional representative user known as a “Persona” is developed based on the core attributes of your typical users. This Persona will be used for multiple purposes, including the purpose of creating testing scenarios and for finding usability test participants.

This is a critical step because obtaining usability testing results from people who do not match your typical users (your Persona or Personae) could negatively impact results. Any testing and subsequent results analysis that are used to determine usability “improvements,” without correct test participants, might actually hurt your application, because the test was flawed with non-representative participants.

This is a critical step which sometimes is overlooked, resulting in bad usability test data!

2. Identify Critical Tasks:

There are many tasks users must go through on a typical website, but certain of those tasks are critical – meaning if the task is not accomplished the user’s goals will not be met. Certainly Registration and Paying are critical tasks, but are there others that should be evaluated?

There are 2 types of critical tasks, the first being the users, and the second being the company’s. Often, where both of these task groups meet is a good place where you should focus your usability testing. Usability testing using 1-on-1 performance based tests are constructed by understanding the existing method that should be used to accomplish the critical task, then testing those methods to see if there are any task-flow errors.

3. Create a Usability Testing Protocol.

By identifying your Personas and Critical Tasks, you can now develop a detailed testing protocol, that contains the script a moderator will use to conduct each test. The script is critical because it helps ensure that each test is non-biased, by offering each test participant the same explanation for testing scenarios.

Typically usability testing protocols also provide a scoring sheet the moderator or other observers can use to score each task. I like to use the 0, 1, 2 scoring method – a “0″ representing a task that was not completed successfully, a “1 representing a task that was muddled through or was completed but with errors along the way, and a “2″ for a task that was successfully completed with no errors. Typically observers or the moderator will use the protocol as a notepad, writing down any task-flow issues and other important observations from the test, along with scoring each task.

4. Conduct a Non-Biased 1-on-1 Usability Test.

Often usability testing is conducted in person, with a moderator observing and administering the test, and a test participant seated at a computer with the software to be tested.  This 1-on-1 performance based testing is an excellent way to capture usability testing data, including verbal and non-verbal information from the test participant. 

There are other ways to conduct usability testing. For example, remote testing can also be used, and is a great way to test participants who might be geographically disperse.

In addition to the moderator, often usability testing software will be installed on the test machine.  This software records all interactions the test participants have with the software (mouse-clicks, dialog boxes, screen and cursor movements, etc.) along with audio and video of the participant.  Some popular usability testing software includes Morae from Techsmith and Silverback from Clearleft.

As I mentioned above, the usability test protocol is very important because it helps ensure a non-biased test is given to each testing participant. Also important when conducting the test is making sure the moderator probes the test participant – but without offering bias. A proper technique is critical to ensure bias does not influence the results.

For example, asking “How you would register on this site, which button would you press to start the process?” is an example of a biased question – note the use of the words “which button” which lets the test participant know a button needs to be pressed to start the process.

Instead, an experienced moderator might use a less-biased question, for example, “Can you show me how you would register on this site, how would you proceed?” does not offer any clues about the process, and has the added benefit of using an open-ended question. 

Open-ended questions are important for usability testing as they are a means with which an on-going narration can be established from the test participant.  By recording and noting the vocalized thoughts of the test participant as he or she goes through the test, the moderator or observers can, among other things, better understand the users expectations for how they believe the task should be preformed vs how they are actually required to do the task.

Typically anywhere from 5 to 10 test participants are needed to conduct usability testing.  This is because unlike surveys or focus groups, usability testing is not gathering opinions about what a tester believes is happening, or should be happening.  Rather, the usability test is recording what the tester actually does, the actions that had successful conclusions, and the actions that were not successful. 

It usually only takes a few users going through a task flow to identify where the task flow has errors. Subjecting thousands of testers to the same tasks is expensive, needless and frankly, would be extremely painful to watch!

5. Analyze Results.

Using the notes and protocol scoring from each test, a pattern of task failures can be identified, and subsequent improvements recommended.   Although it might seem straightforward, the analysis and recommendations for usability improvements should be done carefully, and with full consideration of the needs of the Personas.  Often, what might work well for one group of users might actually hinder or worse, cause complete failure of tasks among another group.

Conclusion – Usability Can Help Any Web Site or Application – If Done Correctly

Of course, as with any overview there’s lots of detail and quite a few smaller steps left off of this list. If conducting usability testing sounds dishearteningly complex, that’s because it can seem that way sometimes to people who haven’t been trained in how to conduct usability tests. A trained usability expert can take care of the complexity and provide a test and analysis that can improve your web site – which ultimately is your goal.

Of course, there are multiple ways to conduct usability testing and the above overview is just one example. However, it’s pretty fair to say that the above example is a fairly common example of a usability test.

The point to all this is; if your goal is to improve the results of a web site or application that is critical for business success, then using a trained expert to conduct your usability testing will benefit you and your company.