The Usability Vendor Checklist – A Handy Vendor Selection Tool
I’ve been asked from time to time how a business should go about finding and selecting a usability vendor. It seems to me there’s a fair amount of confusion in the marketplace about what usability is, and how to choose the best vendor for usability services. So, to help address this need I’ve created a checklist that any business can use to help in evaluating a usability vendor.
You can download your copy of the Usability Vendor Checklist at the Usability Resources page.
Usability Vendor Checklist
I should add here that this can also be a handy tool for usability vendors as well, to ensure their responses to a Request for Information (RFI) or Request for Proposal (RFP) include all the pertinent information about their usability services.
The checklist is in PDF format, so that it can be easily printed and used as a handy tool when going through the process of assessing usability vendors.
The benefits of using a usability vendor checklist
I believe that a useful approach for fairly and accurately evaluating potential usability vendors is to incorporate a checklist of common usability practices and procedures with information about the project in a RFI or RFP that any competent usability vendor can respond to.
By using this checklist as a part of the early information-gathering process, and requiring potential vendors to provide this information as part of a RFI or RFP response, firms will have the information they need with which to compare usability vendors in a fair and impartial manner.
In addition to this checklist, a company can request a sample report or deliverable. Although it’s not always possible to compare apples to apples reviewing a sample report, the sample report coupled with the checklist can provide a better picture of the capabilities and type of deliverables a usability vendor typically produces.
You can download the Usability Vendor Checklist at the Usability Resources page.
I hope you find this tool a helpful addition to your usability vendor selection process.
Similar Posts:
An Overview of Usability Testing
A recent Question posted to LinkedIn caught my eye, the question was “How to conduct a usability review of a web product?” Feeling like I might be able to help, I decided to respond with an Answer. Unfortunately, LinkedIn has a character limit on their Answers, and as any of my long-term useful usability blog readers will tell you I’m pretty much a failure at responding with anything less than about 3 to 4 thousand characters!
Ravi's Usability Question
So, here then is my “expanded” answer that LinkedIn wouldn’t let me post. By the way, although the term “usability review” was mentioned, I felt that from the description Ravi provided it is probably more likely that the usability help needed is actually a “usability test.” What’s the difference? Here’s my definitions:
“Usability Review” – A review of a web site or application by a usability expert who critiques the usability based on common best practices. This is also known as a Heuristic Review.
“Usability Test” – A test of a web site or application in which a user conducts a task or tasks while being observed and/or recorded by a usability expert.
Hello Ravi,
It’s good that you wish to conduct usability testing on your website!
From the sound of it, you may be interested in having a usability test conducted on your web site. Although you could try to do this yourself by reading articles, I highly recommend using a trained professional. Especially if the web site in question is business-critical (in other words, your business’ main means of acquiring revenue).
It can be easy to receive poor or worse, bad usability testing results if proper usability testing methodology is not utilized. A usability consultant should be able to help you very quickly establish the correct way to conduct the test. Assuming you are interested in a usability test, I suggest an in-person 1-on-1 usability test, to properly identify task-flow problems among your web site’s critical tasks.
Here’s an overview of the steps necessary to conduct a non-biased 1-on-1 usability test:
1. Identify Personas:
You must conduct usability testing with people who represent your typical users. A fictional representative user known as a “Persona” is developed based on the core attributes of your typical users. This Persona will be used for multiple purposes, including the purpose of creating testing scenarios and for finding usability test participants.
This is a critical step because obtaining usability testing results from people who do not match your typical users (your Persona or Personae) could negatively impact results. Any testing and subsequent results analysis that are used to determine usability “improvements,” without correct test participants, might actually hurt your application, because the test was flawed with non-representative participants.
This is a critical step which sometimes is overlooked, resulting in bad usability test data!
2. Identify Critical Tasks:
There are many tasks users must go through on a typical website, but certain of those tasks are critical – meaning if the task is not accomplished the user’s goals will not be met. Certainly Registration and Paying are critical tasks, but are there others that should be evaluated?
There are 2 types of critical tasks, the first being the users, and the second being the company’s. Often, where both of these task groups meet is a good place where you should focus your usability testing. Usability testing using 1-on-1 performance based tests are constructed by understanding the existing method that should be used to accomplish the critical task, then testing those methods to see if there are any task-flow errors.
3. Create a Usability Testing Protocol.
By identifying your Personas and Critical Tasks, you can now develop a detailed testing protocol, that contains the script a moderator will use to conduct each test. The script is critical because it helps ensure that each test is non-biased, by offering each test participant the same explanation for testing scenarios.
Typically usability testing protocols also provide a scoring sheet the moderator or other observers can use to score each task. I like to use the 0, 1, 2 scoring method – a “0″ representing a task that was not completed successfully, a “1 representing a task that was muddled through or was completed but with errors along the way, and a “2″ for a task that was successfully completed with no errors. Typically observers or the moderator will use the protocol as a notepad, writing down any task-flow issues and other important observations from the test, along with scoring each task.
4. Conduct a Non-Biased 1-on-1 Usability Test.
Often usability testing is conducted in person, with a moderator observing and administering the test, and a test participant seated at a computer with the software to be tested. This 1-on-1 performance based testing is an excellent way to capture usability testing data, including verbal and non-verbal information from the test participant.
There are other ways to conduct usability testing. For example, remote testing can also be used, and is a great way to test participants who might be geographically disperse.
In addition to the moderator, often usability testing software will be installed on the test machine. This software records all interactions the test participants have with the software (mouse-clicks, dialog boxes, screen and cursor movements, etc.) along with audio and video of the participant. Some popular usability testing software includes Morae from Techsmith and Silverback from Clearleft.
As I mentioned above, the usability test protocol is very important because it helps ensure a non-biased test is given to each testing participant. Also important when conducting the test is making sure the moderator probes the test participant – but without offering bias. A proper technique is critical to ensure bias does not influence the results.
For example, asking “How you would register on this site, which button would you press to start the process?” is an example of a biased question – note the use of the words “which button” which lets the test participant know a button needs to be pressed to start the process.
Instead, an experienced moderator might use a less-biased question, for example, “Can you show me how you would register on this site, how would you proceed?” does not offer any clues about the process, and has the added benefit of using an open-ended question.
Open-ended questions are important for usability testing as they are a means with which an on-going narration can be established from the test participant. By recording and noting the vocalized thoughts of the test participant as he or she goes through the test, the moderator or observers can, among other things, better understand the users expectations for how they believe the task should be preformed vs how they are actually required to do the task.
Typically anywhere from 5 to 10 test participants are needed to conduct usability testing. This is because unlike surveys or focus groups, usability testing is not gathering opinions about what a tester believes is happening, or should be happening. Rather, the usability test is recording what the tester actually does, the actions that had successful conclusions, and the actions that were not successful.
It usually only takes a few users going through a task flow to identify where the task flow has errors. Subjecting thousands of testers to the same tasks is expensive, needless and frankly, would be extremely painful to watch!
5. Analyze Results.
Using the notes and protocol scoring from each test, a pattern of task failures can be identified, and subsequent improvements recommended. Although it might seem straightforward, the analysis and recommendations for usability improvements should be done carefully, and with full consideration of the needs of the Personas. Often, what might work well for one group of users might actually hinder or worse, cause complete failure of tasks among another group.
Conclusion – Usability Can Help Any Web Site or Application – If Done Correctly
Of course, as with any overview there’s lots of detail and quite a few smaller steps left off of this list. If conducting usability testing sounds dishearteningly complex, that’s because it can seem that way sometimes to people who haven’t been trained in how to conduct usability tests. A trained usability expert can take care of the complexity and provide a test and analysis that can improve your web site – which ultimately is your goal.
Of course, there are multiple ways to conduct usability testing and the above overview is just one example. However, it’s pretty fair to say that the above example is a fairly common example of a usability test.
The point to all this is; if your goal is to improve the results of a web site or application that is critical for business success, then using a trained expert to conduct your usability testing will benefit you and your company.
Similar Posts:
Good Designers Who Know Usability Are REALLY Good Designers
I recently decided I needed a redesign of my personal web site wcraigtomlin.com and used the design services of a really good designer, Deborah Edwards-Onoro of Lireo Designs.
As is often the way with web redesign projects, my personal little web site redesign became something of a “project,” including all the usual issues such as scope creep (guilty), long delays by the client (guilty) then a sudden rush and pressure to get everything done ASAP with a modest amount of badgering (guilty, guilty and guilty).
As with other good designers, Deborah took it all in stride and demonstrated skills and patience that helped get the project accomplished in a very satisfactory manner. Most good designers pretty much know this comes with the turf, and do a pretty good job about handling such “problem” clients (like me).
However, I like the fact that Deborah also exhibits and understands the importance of applying usability to design. In my case, she used her knowledge of usability to produce a clean and simple user experience. As a side note, it’s also quite impressive that she’s actively involved in getting a new Michigan usability group together (now that’s REALLY believing in usability!).
Design & Usability
Designers have a tough job, and usability is not always top-of-mind among all designers. Add to this the pressure all designers face from “problem” clients (like me) as to critiquing design and you’ve got a extra handful of issues to deal with.
Let’s be honest, very few of us “clients” will critique or criticize CSS coding for example, but almost all of us will critique and criticize design colors, shapes and placements.
Humans, all of us, believe we “know” design and can judge between good and bad. Senior vice presidents of Finance, industrial cleaning product salesmen and even plumbers will offer free advice about a design they see, if offered a chance.
So it’s impressive when a designer will include the extra considerations of user experience issues as part of their process when developing a design. When this comes from a knowledge based on an understanding and appreciation of usability the design is going to be that much better. In my opinion this moves a designer from “good” to “really good,” because it means the designs will incorporate usability criteria to improve the user experience well before a web site is coded or goes live.
Really Good Designers
So it’s with gratitude that I say thanks to Deborah for all her work, she’s a really good designer. And if you are looking for a designer that includes usability as part of her process I suggest you contact Deborah.
And if for any reason Deborah can’t do the project (because she’s so busy with all the referrals I hope to send her way) then my suggestion is to seek other designers who take usability to heart. The end product these designers provide will mean far less re-work down the road, and an optimized web site that meets your business goals.
Similar Posts:
I Am Surprised and Honored to Make Mashable’s List of Top 10 Must-Follow Usability Experts
Wow! I am very surprised and very humbled to be considered as a top 10 Must-Follow usability expert by Mashable. My name’s listed right up there with the likes of legends like J.M. Spool and the Nielsen Norman Group.
Of course, Mashable didn’t tell me this or warn me or anything – so this was a complete surprise, and a very nice surprise at that – once I figured out what was going on.
Actually, it was such a surprise that at first I thought a hacker had infiltrated my Twitter account and was having EVERYONE follow me! Follows were coming in at the rate of 10 to 20 a minute for a while! Since the coffee hadn’t taken full effect (it was after all about 6:30AM) I prepared to go “private” on my twitter profile – and change the password and alert Twitter to my hacked account!
Good thing I paused to actually read the Tweets that were coming in – which is when it dawned on me that I had received this great compliment, and the follows were “legit.”
Useful Usability
So, here we are, talking about useful usability. As my readers will know (you and my mom – hi mom!) I’ve been going on and on blogging about the benefits that usability can provide, and how easy it is to add usability to a web site or application project. I guess there’s now more reason than ever to blog about useful usability – as I like to call it.
What is “Useful Usability?”
In my mind, useful usability is three things:
- WHAT: Making usability useful by educating people about what usability is (and isn’t), the benefits of incorporating usability in web and application design, and why it’s good for business.
My belief is usability is about improving a business by increasing revenue or decreasing costs, or both!
Although there’s a great sense of satisfaction in knowing that usability improvements can help the human condition, the reality is usability is most typically consumed by companies looking to sell, or help support the selling of their products.
If the usability is not directly involved in improving revenue generation (as part of the product, or application used to sell the product), it is used to improve applications (think employee intranet tools) that are used to support the business that sells the product.
The often-mentioned good usability of the iPod, Shuffle or iPhone IS the product, in the sense that it absolutely reflects the brand and the user experience that many consumers expect and are glad to pay for.
- WHERE: Providing useful usability best practices, tips and sometimes war-stories of where and how I and others actually go about adding usability to a project.
Just like going to the dentist, conducting usability testing and optimization early and often is the simplest and easiest way to eliminate yucky plaque build-up web site problems and get the biggest bang for your application development buck. Depending on where you are in the software (or hardware) development life cycle, differing methods of usability testing are needed to provide actionable and valid data.
Applying usability testing data to make real improvements can and should be validated using key performance indicators (KPIs). And make no mistake, usability will permanently improve conversion of web sites.
- HOW: Demonstrating that usability does not have to be cumbersome or expensive. Usability can be done quickly, efficiently and for low or no cost.
Saving money by using low cost usability methods is a great idea. Saving money by not using a usability expert to conduct usability testing and optimization is a very bad idea.
That’s because usability testing is not about gathering lots of opinions (that’s the world of surveys and error rates and statistical significance). Usability testing is not about asking a few users to pretend to use the site and tell you what they like or don’t like (that’s using one person’s opinion to base business-critical decisions on – ouch).
Usability testing is typically about applying detailed Personas to a specific set of critical tasks using a carefully controlled 1-on-1 performance based test to evaluate precisely where issues are in a task flow.
Only by consistently observing the same task flow errors being repeated again and again can a verifiable usability error be uncovered – and optimizations recommended.
Useful Usability
So that’s my take on usability and what I believe useful usability is all about.
I sincerely hope my usability posts will help you in your goal of using or understanding usability. And please add your comments, suggestions or rants thoughts about usability – together we can all grow more informed about usability!
Similar Posts:
Customer Experience Matters
now i'm eating soup
What Makes Them Click