How to Conduct a Usability Review

Conducting a Usability Review:

In the world of usability, nothing seems to confuse my clients more than trying to determine exactly what a usability review is.  And it’s difficult to purchase something if you don’t know what it is!

Usability reviews don’t hurt (physically)

You can think about a usability review this way, it’s kind of the same as going to a doctor for a check-up, your web site will be examined to find usability issues (ailments) and you’ll be provided with recommended optimizations (prescriptions) for improvements.

Usability reviews are not generally well known or understood because the usability field itself does not have a single, consistent, standardized definition of “usability review.”

It’s an interesting and ironic truth that usability professionals who pride themselves on utilizing standards for testing and optimizing web sites can’t create their own set of standard definitions of common usability terms.  Go figure.

So, what is a usability review and how do you do one?

Since there is no consistent standardized definition of a usability review (also known as an expert review, expert usability review, usability audit, heuristic evaluation, etc. etc. etc.) I’ll go ahead and give you mine:

Craig’s definition of a usability review:

“A usability review is an evaluation of a user interface versus common usability best practices and heuristics by a trained usability professional.”

So in the spirit of sharing and giving, here are the steps I use when conducting a review.  By following these steps, you will have all the information necessary to conduct your own usability review.

And if you would like me to send you a sample usability review (not a complete review as this was for a client, but you’ll see how to do it) so you can see how to create your own usability review, I am selling the sample for a pittance at just $9.95 (please note that due to it being a digital download I can’t provide a refund, as well, you have the document!).

You can purchase your copy safely and securely through PayPal:

Add to Cart

View Cart

Step 1 – Become a trained usability professional, or save time and hire one.

As with most any other professional such as a doctor, lawyer or elephant-trainer, it’s important to have an education and experience in the practice.  Usability testing of critical tasks and usability reviews are not about providing opinions about a design, conducting focus groups or deploying a satisfaction survey on a web site.

Rather, usability testing and reviews are a scientific approach to analyzing a user interface and task-flow to determine where (and why) there are problems that cause users to have difficulty completing their tasks.

Using a trained usability professional for your usability review means taking the guess-work out of conducting the evaluation, and ensuring that a non-biased approach is used.

Step 2 – Identify critical goals for the web site or application:

More than likely, there are multiple goals for your web site or application.  The important aspect of a usability review is to focus on the most critical goals.  This is because narrowing down the focus enables evaluation of specific tasks associated with that critical goal or goals, and helps shape the subsequent to-do list of potential optimizations derived from the review.

For an eCommerce site like Blue Nile it’s probably selling diamond engagement rings.

For an informational site like the State library of Kansas it’s probably helping you find the literature you are looking for.

Whatever your web site or application has for critical goals, those are the ones that you should focus on first when conducting a usability review.

Step 3 – Define typical users via a Persona:

The vast majority of web sites and applications have typical users who share a common set of domain expertise (knowledge of the field) and critical tasks.  Identifying the Persona (a fictional representation of the typical user) is critical.  This is because the usability review must take into consideration the type of person who is interacting most frequently with the user interface.  It must consider their familiarity or lack thereof with the terminology, information architecture, navigation schema and related user interface systems they interact with.

For example, the Persona used for a usability review of a web site that deals with precision electronic measurement probes for the engineering industry, such as on MicronTesa.com may be quite different from that of a Persona who visits VirginAtlantic looking to book a flight to Heathrow.

Basic usability standards apply across all users, but specific “mental maps” (expectations of labeling and information architecture – groupings of information) must be considered when conducting a usability review.

Step 4 – Conduct the critical tasks:

With the above steps completed, now the actual “review” of the web site or application can take place.  The identified critical tasks are conducted one at a time, yes even down to purchasing that airline ticket – make sure you purchase a fully-refundable ticket of course!

As each task is conducted, the usability review identifies specific task issues as well as general usability issues as defined by usability heuristics (best practices).

What are the usability heuristics?

According to Jakob Nielsen the 10 usability heuristics are:

  1. Visibility of system status
  2. Match between system and the real world
  3. User control and freedom
  4. Consistency and standards
  5. Error prevention
  6. Recognition rather than recall
  7. Flexibility and efficiency of use
  8. Aesthetic and minimalist design
  9. Help users recognize, diagnose, and recover from errors
  10. Help and documentation

Typically each critical task has to be conducted many times, and each time a separate usability heuristic is evaluated against the task.  Issues are noted, typically with screen shots captured and detailed information about the usability issue found.

NOTE: It’s important at this point to NOT list out every single issue you find with each of the above usability heuristics in a review. Why? Because you would end up with a very large and overly detailed document (probably hundreds of pages long) that nobody would read or use! My reviews do not list out the 10 Heuristics above in detail. Instead, I make sure to summarize groupings of heuristics into the key issues I’ve found into a short, readable Usability Review. And call it a ‘Review’ so that your readers understand you are providing them actionable information for the key issues found, not every single issue as that would be overwhelming and well, not usable!

Step 5 – Compile the analysis

By now, you have a great amount of information about specific usability issues in the task flows.  Although you *could* list each one out separately, but imagine the size of that massive document! A better way to compile the analysis is with a set of grouped issues that highlight the key opportunities for improvement you found.  In addition, I like to provide screen shots documenting the issues, with commentary provided.

It’s important to also include suggestions or recommendations to improve the usability issues found.  After all, nobody wants to receive a laundry-list of problems with no hope of improvement.

As mentioned, there should be several issues that all share common traits, these can be grouped together.  I like to group issues into buckets of commonality, such as those involving:

  • Information architecture
  • Navigation
  • Labeling (taxonomy)
  • Layout
  • Functional flow
  • Form function
  • Error handling and messaging

Step 6 – Present the analysis

More than likely the client will be faced with a litany of problems, issues and snafus.  When presenting the results of the usability review, I always like to start with some positives.  I typically will provide (as best I can) screen shots and commentary of the good points of the web site or application.  This has two purposes:

  1. If provides the recipient of the bad news with a glimmer of hope
  2. It reminds everyone that the web site or application has benefits, benefits that can and should be mentioned

There’s one other important point to consider about a usability review, and that is there’s no such thing as the perfect usability review.  I always like to remind my recipients that a usability review, because it’s conducted by one person, will not catch all the usability issues that might be present.

But by focusing on the critical tasks of the web site or application, hopefully the most significant usability issues are identified, and recommendations for improvements made available.

Conclusion: How to conduct a usability review

So there you have it, those are the steps I use in developing a usability review.  Whether you call it a heuristic review, a usability audit or an expert review, the point is it’s a great way to learn about issues and opportunities to improve a web site or application.

And if you would like me to send you a sample usability review (not a complete review as this was for a client, but you’ll see how to do it) so you can see how to create your own usability review, I am selling the sample for a pittance at just $9.95 (please note that due to it being a digital download I can’t provide a refund, as well, you have the document!).

You can purchase your copy safely and securely through PayPal:

Add to Cart

View Cart

This entry was posted in Methodology and tagged , , , . Bookmark the permalink.

25 Responses to How to Conduct a Usability Review

  1. Yagnesh Ahir says:

    Quite helpful resource this is. Keep up the good work!

  2. Very interesting that you use personas in heuristic evals. I’m getting ready to do an a/b comparison of a think aloud study with representative users where the b group will be given a persona. Since they’re already representative, will it help? Will it just confuse them? Will it make any difference at all? I don’t know, that’s why I’m doing the study. :)

  3. Sounds like an interesting study Derek, hopefully you’ll share the results with us.

    My personal philosophy is that there’s no such thing as a usability review WITHOUT a persona being considered, whether the usability practitioner realizes it or not.

    Consider this; how much different is a usability review of a web site that is focused on Seniors (let’s define that as people over age 65) who are looking for social security information, vs a web site that is focused on teen gamers (let’s say ages 15-19) who are looking to kill zombies online? Issues such as font size, background color, contrast, navigation schema, terminology, use of graphics etc. will (or should be) very different between the two reviews.

    Usability heuristics applied in a review without consideration of the typical user is simply not possible, it’s trying to apply standards for interface use without the “user.” Sort of goes against that whole “user-centric design” thingy. :-)

    I will be curious to see how the results of your study come out!

  4. Craig,

    I have so many comments to make about this post! I’ll try to be brief. First, so funny that you mention that we talk about standards all the time, and then we can’t standardize on the name of these!

    You talk in the post and in your comment above about the fact that the evaluation must be done with personas in mind, and you talk about doing an evaluation with the user and user tasks in mind, and yet, your definition you give at the beginning does not mention that… so I suggest you refine the definition a little bit to include these ideas?

    And to complicate things further, what do you do with a Persuasion Review… Is that part of this or would you consider that a different animal?

    can you send me a sample review too? Thanks for the great post.

  5. Thanks for your comments!

    Yes, funny that there’s no standard definitions, you’d think that would be a page on the UPA site – something useful like “usability definitions.” Oh well.

    I think my definition is pretty good – mostly because I thought of it – and partly because it’s simple and easy for clients to remember. Us usability practitioners pretty much already know what a usability review is (or so you’d think) but we don’t really care – because we’re not selling usability reviews to each other. We’re selling usability reviews to clients who don’t know a heuristic usability review from a Hollywood movie review. That’s why I think our standard definitions for what things are should be simple, easy to remember and – oh yes – user friendly! :-)

    I think since a ‘usability review’ uses the term ‘usability’ it implies tasks, and user-centric design principles, like knowing who your typical user is, understanding their mental map of a process, and aligning your user interface to meet those expectations. Thus, a usability review of an interface in relation to accomplishing critical tasks should and must – by the natural law of selection and Craig’s law of simplicity – include the elements of Personas, critical tasks, and evaluating the interactions based on those.

    Persuasion? In my mind, usability is about action – how easy or difficult it is to use something, was the task accomplished – yes or no? And for me, Persuasion is how well (or poorly) persuasive elements are used to stimulate trust, desire and the decision to take action. I see the two being different. But I think you could also argue that using persuasive elements correctly in let’s say an order flow would help the user accomplish a task – so perhaps my black-and-white, binary world is a bit more shades of gray and hexadecimal.

    Your sample usability review is on its way, I hope you find it interesting – and usable! :-)

  6. I am glad you used the term “mental map” instead of mental models.

    What intrigues me is your persona use.

    What is your definition about persona?
    Why do you need persona if you have people?

  7. Hello Fabio,

    A Persona is a fictional representation of the typical user of a web site or application. The Persona contains information about the critical tasks the typical user needs to accomplish, along with information about their domain expertise, their computer or application usage, etc. This Persona is the foundation of conducting usability testing, because you use the Persona to find a representative user with which to conduct your tests.

    My point about conducting a usability review is it’s impossible to do one WITHOUT some sort of assumption about the typical users (Persona) of the application. This is because best practices for interaction must by necessity include an understanding of the typical user of the application.

    For example, the best practices and standards applied for a group of seniors (say over the age of 65) looking for information on a Medicare web site will by necessity need to be quite different vs. those of teenagers wishing to use World of Warcraft to play fantasy games. Without including the considerations of the Persona, a usability review is simply applying ‘standards’ in a vacuum of information about users – a dangerous thing and NOT part of User-Centric design – which is the core principal of usability and usability reviews in the first place.

    I hope that helps explain why you must ALWAYS include the Persona when evaluating an interface with a usability review.

    For more information about Personas – try:
    WikiPedia – Persona
    Usability.gov – Develop Personas

  8. Ahmed Bassel says:

    You’ve simplified the process of conducting this exercise “Usability review”.
    It is shame that we still don’t have a solid or clear definition for such process, plus the available standards are very vague.

  9. Oh! I have bookmarked this baby to S T U D Y! Thanks so much for sharing this!!

  10. WallMountedHDD says:

    “Usability expert?” Okaaaay. Did they study UX for four years at at UI School? Are they up there with SEO Gurus and Website UI Emotional Response Masters? Just study UI design like any other programmer/artist and use the site. If the nav or layout feel funky then change it so it’s not. Badda boom.

  11. Pingback: links for 2010-03-06 « Mandarine

  12. jo says:

    the review was very helpfull.

  13. Shannon says:

    In response to WallMountedHDD : The fact is there are usability experts for a reason. I hate to exploit this fact, but generally when programmers are designing websites, they design them with their own expertise in mind, not necessarily the expertise of the user. Therefore, the job of a usability expert is extremely important in the process so that we don’t end up with websites that require a graduate degree in software engineering to operate. :) Badda boom.

  14. Andy says:

    Can I ask your opinion on creating Persona’s without user research to back them up?

    Making assumptions about typical users could contradict the clients understanding and you would have no evidence to back up your statement.

  15. craig says:

    Hi Andy,

    My opinion is creating a Persona without some form of contextual data about typical users and their habits is like fishing without a hook. Sure, you can do it, but it would be pure luck if you got it right and caught a fish captured the essence of typical users.

    The question then becomes, “what’s enough user research to move forward with creating a Persona?” I’m not sure I can answer that. I think it has to do with the amount of data available, the type of application being examined, and what information a firm may have about their typical users already.

  16. Sean Byrne says:

    Hi Craig,
    Thanks for the interesting post – first time I’ve read all the way to the bottom in a long time ! very useable :-)

    regards
    Seán

  17. Owen Cooney says:

    Great Article! Thanks Craig.

    Do you see usability reviews being independent of the app/website design team or as a key part of that team?

  18. craig says:

    Thanks Owen!

    ABSOLUTELY it is best to conduct reviews as early in the process as possible. Early course corrections cost far less in terms of man-hours than corrections that happen well after the code is written. Thus, review with the app/website design team early and often is the key to success.

  19. Ivette Litt says:

    Hi Craig:
    This a very nice and concise document and I certainly intend to keep it as reference for my projects. I am about to initiate my first “official” usability analysis and will welcome a sample of a Usability Review.
    I understand you wrote this post two years ago an you might had change your mind about the offer, but… again, I could use any help.
    I have to choose between two pretty interesting project, one is a Usability Analisys of a Map application and the other is HR portal on developed in SharePoint. ( I know….)
    Thank you so much
    Keep up your post… Great reference material

  20. Pravaal says:

    Really awesome article Craig !!!!
    Rather than writing a whole set of complex words and terms related to usability, you chose to write in a more understandable fashion. That’s the best part “It’s more usable than more informative”. Atleast it managed to push out the fear of the term “USABILITY REVIEW” from my mind.
    Ofcourse there are articles more informative than this on web, but what I like most about your’s is its simplicity and to the point nature. Good to start or refresh the basics.

    thanks again..

  21. craig says:

    Thanks Pravaal (I think),

    The point of a Usability REVIEW is just that, it’s a review. A review is not a comprehensive listing of each and every Heuristic. As I mentioned in my article, that would be a very large document that frankly, nobody would use. I know this because I’ve seen it time after time. So, thanks for the compliment (I think)!

    :-)

  22. Pingback: Usability: Reviews schreiben | Denkwerkzeuge im Wissensmanagement

  23. Pingback: Expert Review VS Usability Testing | Bite & Chew

  24. Pingback: Content Marketing in the Age of UX | Useful Usability

  25. Pingback: Debunking common UX myths

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>