Review of the usability book ‘Remote Research’ by Nate Bolt and Tony Tulathimutte

For those of you who’ve been following my blog for a while (hi Mom! hi @kailualisa!) you know the risk Nate, Tony and Lou Rosenfeld took asking me if I’d like to read and potentially review their new usability book, “Remote Research.”

As you know, I don’t pull any punches, and I calls ‘em like I sees ‘em.

Although I was provided by the authors with a free evaluation copy of the book - I am fully prepared to skewer it like a hot knife through butter if I think it’s a waste of your and my time (and money!).

The news for you, Nate and company is: I have read the book cover to cover and I can state in no uncertain terms that this is a usability book that usability practitioners, or those learning about or just interested in usability, should buy and can use over and over.

Simply put: You’ll use this book, a lot.

The reason?

For a nominal fee of US $36 (softcover and DRM-Free PDF, or $22 for 2 editions of PDFs) you will have a very handy how-to / reference guide for conducting remote research you will use again and again.

Here‘s some of the key points covered in the book:

• What remote usability research is (and what it isn’t)
• Pros and Cons of conducting remote research
• Step by step how to (for first timers)
• Detailed list of tools (both moderated and un-moderated)
• Tips and tricks for each tool (experienced remote researchers take note!)
• A companion web site with additional tools and information is included

Gosh, Nate and Tony even go so far as to include a rebuttal from remote research h8r opponent Andy Budd, creative director at Clearleft, as to why remote research should NOT be used.

I think Nate and Tony have done a good job of being fair about this subject, sharing both the pros as well as the cons of remote usability testing.

Review of ‘Remote Research’

The writing is light and easy to read, you won’t feel at all like you are reading a manual or dictionary.  Jargon is kept to a minimum, and for each new term they use Nate and Tony explain what it means in plain English.

Rather than putting me to sleep at night (I’m looking at you, ‘The Complete History of Taxation’) I found the writing to be fun, engaging and sprinkled with humorous yet relevant dialog.

Reading this book is very much like having a fun yet informative lunch with a remote research guru – you learn a lot, and enjoy the experience along the way.

Remote usability testing for first timers:

For some of you, it’s probably not necessary to read the entire book cover to cover (but you should, because you will be missing some humorous comments and very helpful pointers).  Those who have not yet conducted remote usability testing should read Chapters 1-5 and 7 (I’ll break out more information about each chapter below).

Those chapters are perfect for first timers, as they provide the overview of what remote testing is, along with step by step instructions for conducting your first remote usability test.  Included is a detailed listing of all the tools (there’s not many) you’ll need.

Remote usability testing for old pros:

For those of us who’ve already been conducting remote usability testing sessions, you’ll probably want to focus more on Chapters 7-9, and potentially Chapter 6 (Automated tools).  That’s not to say you shouldn’t read the other chapters, I learned some new ideas and tips in the overviews that I’m anxious to try, just that if you are pressed for time you may find those the most actionable.

So here then is a brief overview of each chapter with some of my observations about it included:

Chapter 1 - Why Remote Research?

The book starts with a very helpful overview of the various types of remote usability research methods.  It also includes a very helpful case study.  I think this chapter provides some useful tools you can use to “sell” remote usability testing internally, if you are in an organization that is resistant or unaware of the benefits.  Likewise, if you are selling remote usability research to prospective clients I think you will want to read this and use the information.

However, I do have one point of contention.  In Chapter 1, and later in the book Nate and Tony refer to the cost of remote researching being almost equal to in-person moderated testing, thus not having a big cost savings.  This in my opinion is not exactly accurate.

They were apparently basing their cost from the perspective of a consulting agency using remote vs. in-person testing, and thinking mostly about the billable hours of moderation and analysis - not from a Corporate (i.e. Client) infrastructure perspective.

At several companies I worked at, I created detailed studies comparing in-person vs remote testing costs if the company were to:

• Build and maintain a usability lab
• Rent usability labs
• Use remote testing technology and methods

My analysis revealed that remote testing was the clear winner for lowest cost, by a landslide.

Building a lab for in-person testing vs. using remote methods only made sense if the company would either use the lab every day (none of the companies was big enough to do that) or rent the lab out to 3^rd parties during down time (none thought that was a good idea).

Renting usability labs for in-person testing was far more expensive than using remote methods.  When you look at the total costs after just one year of usability testing remote easily won.  And if you compared both after several years of usability testing, the costs of paying for travel, hotels, per diem and lab rental fees for rental usability labs was astronomically more expensive than the costs for using remote research methods.

My personal opinion?  There’s almost no difference in results for conducting usability test sessions remotely vs. in-person with a usability lab - the methodology and tools for conducting the testing are almost identical, and the task-error results will be the same.  My belief is unless usability labs can add additional value beyond straight usability testing, over time they will go the way of 8-track tapes and VCRs (technologies that became cumbersome and too expensive vs. newer tools).

Again, that’s just my opinion and I hasten to add is not mentioned in the book.

Chapter 2 - Moderated Research: Setup

In this chapter Nate and Tony provide a detailed step-by-step process for actually conducting a real remote usability test.  I believe the information is perfect for beginners, and even seasoned pros may find some of the information new or different, I know I did.  I found the table of screen sharing tools at a glance (on page 34) to be very handy, as well as the detailed scripts and observer instructions.  You’ll probably dog-ear the example script pages here, and use them later, I did.

Chapter 3 - Recruiting for Remote Studies

Recruiting for remote usability testing is potentially one of the trickiest parts of the whole process - and must be done right - else the study can be flawed.  Nate and Tony do a great job of providing plenty of detail and tips on how to recruit, and how to avoid using bad recruits (fakers and professional survey takers).

Chapter 4 - Privacy and Consent

Legal stuff is boring.  But I think you should really, really read this chapter - you’ll thank me later when you avoid a multi-million dollar privacy lawsuit.  The legality and privacy issues of recording a participant is a critical element of usability testing, and there’s very little information out there about what to watch out for.  This chapter provides excellent information and should, after you read it, make you reach for the phone to call your lawyer and get more information about the dos and don’ts of legal language for your consent forms.

Chapter 5 - Moderating

Moderating is tricky, and Nate and Tony do a good job providing details and insight into how to be a better remote testing moderator - along with very handy example scripts.  Unless you do a lot of moderating (and by a lot I mean doing it every day) I think you’ll really appreciate the guidance and tips in this chapter.

Chapter 6 - Automated Research

Nate and Tony do a great job explaining what automated research is (also called un-moderated), what it’s not, how to use it and when not to use it.  I’ve dog-eared plenty of pages in this chapter - especially page 128 which contains a really great chart of the major automated research tools and where they sit in the dimensions of Qualitative vs. Quantitative and Concrete vs. Conceptual.  I think I may blow this chart up and place it on my wall.

Chapter 7 - Analysis and Reporting

It’s a funny thing about usability testing, there are lots of standards for how to do it, but almost no standards for how to analyze and especially report on it to clients.  This chapter covers both moderated and un-moderated analysis and reporting, and provides some really nice examples of ways to report effectively on all the data captured.  Also included are examples and tips of how other usability experts report results to their clients - good stuff.

Chapter 8 - Remote Research Tools

This chapter is worth the cost of the book all by itself.  It provides the best list I’ve seen of remote tools, including screen-sharing, recording and automated tools and services.  And here’s the really great part, Nate and Tony provide tips and tricks for how to use each of these tools.  From Adobe Connect to Webnographer, this is a comprehensive list and cheat-sheet of testing tools.  I am almost certain that you’ll dog-ear almost every page of this chapter, just like I did.

Chapter 9 - New Approaches to User Research

Nate and Tony provide some very interesting new tools and techniques, including one of my favorites, reverse screen sharing.  There’s also some good advice and guidance on mobile device research and research on video games.  They’ve even included information on conducting remote testing in cars.  Considering the rapid expansion of multi-modal mobile devices I fully expect in the next year or two to see a lot more research and research information about testing these devices in their “natural” environment such as the car, train, home or bleachers at the kid’s softball game.

Chapter 10 - The Challenges of Remote Testing

This chapter is an excellent listing with details about the issues and potential problems associated with remote usability testing.  Everything from legitimacy to getting the right recruits to persistent negativity is covered.  I think Nate and Tony did a good job keeping it real, so to speak, with being up-front about when to use, and when NOT to use, remote testing, and what some of the problems and pitfalls are.

Conclusion - Book review of “Remote Research” by Nate Bolt and Tony Tulathimutte

So, with “Remote Research” I believe you have a very useful and practical guide to remote usability testing.  This book is easy to read, is written in a open, honest and humorous way, and will provide you with a large amount of useful information you’ll be able to use again and again.  I have a special place for books I refer to often (down low on the wall bookshelf, where I can reach them) and that’s where this book is going.

I suspect you’ll place this book in an easy-to-reach place as well.

You can buy the book “Remote Research” at Rosenfeld Media, or at your favorite online bookseller.

Book Details:

• “Remote Research” By Nate Bolt and Tony Tulathimutte
• Copyright 2010, Rosenfeld Media, LLC
• First published, February 2010
• ISBN: 1-933820-77-2
• 266 Pages
• Available in paperback and DRM-free PDF (US $36.00)
• 2 digital editions (screen-optimized and print it yourself PDFs) (ISBN 1-933820-44-6) (US $22.00)

Note:

In case you were wondering if I’m making any profit from this review, I’m not.  The authors/publisher of this book provided a free evaluation copy of this book for my personal use.  I was not compensated for this review nor required to write this review.   The views, content and opinions expressed in this book review are mine alone.  Links from this review to the book web site are not affiliate links and I receive no money for traffic sent from this site to the book site nor for sales of the book.  I’m honest and non-biased.  Broke, but non-biased.  Hey, that’s how I roll.

A list of 12 Really Useful Usability Books Worth Reading, and Re-Reading

On my bookshelf in my office (ok, it’s a cube, but it’s kind of a big cube) is a rather largish accumulation of usability books. There’s other books thrown in there too, such as Customer Experience, online marketing, and even “The Handbook of Employee Benefits” which is definitely NOT in my opinion a “handbook,” being upwards of 1,300 pages of really, really small font size text. Anyway, as I was saying, I have lots of usability books on my bookshelf.

In reality, most of the books on my bookshelf, although interesting, I’ve only read once. However, there is a small group of them that I consider to be really useful, and I’ve actually re-read them, and refer to them from time to time when going about my usability work. Not all of these really useful books could strictly be classified as “usability” books, but in one way or form they all have significance to usability principles. Therefore, I present them to you and hope you find them interesting or helpful too! And by the way, don’t be shy about adding your own favorite books to this list as well!

So, here with no further ado is Craig’s…

Top 12 list of really useful usability books

1. The Loyalty Effect: The Hidden Force Behind Growth, Profits, and Lasting Value by Frederick F. Reichheld and Thomas Teal

Usability is all about making things easier and thus more satisfying for the people that have to use a web site or web application. But why should we care about satisfying people, what’s the business value for “satisfaction?” This book provides the arguments for why a satisfied and happy customer is a loyal customer, and why loyal customers are so amazingly valuable to a business. After executives read this book at one of my former companies, all of a sudden usability (and satisfaction) took on much more value, and projects to improve our web experience were initiated. This book still provides excellent arguments for why customer satisfaction and usability projects should be prioritized very high in an enterprise.

2. Customers.com: How to Create a Profitable Business Strategy for the Internet and Beyond by Patricia B. Seybold and Ronni T. Marshak

This book in my opinion was (and still is) brilliant, and way ahead of its time. Years before social media was born, and the voice of the consumer became omnipresent, this book explained the rationale for providing a good customer experience online. Patricia Seybold explained that customers more than ever before have the power to influence each other, and thus influence the profitability of a company like never before. Thus providing a good user / customer experience is critical to the health of an enterprise. She provides great examples of good vs bad experience that still apply today.

3. Don’t Make Me Think by Steve Krug

If you only read one usability book in your entire lifetime, read this one. I consider this the epitome of how an educational book should be written. It’s very easy to read, has lots of extremely useful information, uses visual examples to brilliantly explain good vs bad concepts, and is funny. It was written specifically to be read in one sitting, for example when flying from New York to Los Angeles, thus is an Executive’s best friend. Give this book to any executive at any firm and if they read it I guarantee they will seek to initiate usability improvement projects as soon as they can turn their cell phones on after landing.

4. Information Architecture for the World Wide Web by Rosenfeld & Morville

The second best book ever written about usability and web design. Who would have thought that librarians (librarians of all people!) would have the ultimate secret for how the web works, and how to design a perfect web site? Turns out the web and your web site is exactly like a library. Like a library, you have lots and lots of content. Your job is to make this content fit into categories that people expect it to be in. You have to use a labeling system that is simple, accurate and consistent to help people find those categories and content. That’s it! That’s the ultimate secret of a perfect web site. And the good news is if you read this book Rosenfeld and Morville will exactly explain how to go about creating those categories and that navigation.

5. The One to One Fieldbook by Peppers & Rogers

A prerequisite for this book is “Enterprise One to One” which defines how a company can and should be focused at the individual customer level, and why in this online age that is critical for success. Assuming you get the fact that your enterprise should be managing customer relationships at the individual level, how do you actually go about making it happen? This book explains how to make that vision happen. You can’t move a hill with a teaspoon yourself, but give every single person in your company a teaspoon and the will to dig, and all of you, working together, can. Companies that move to managing individual customer relationships (Apple and Zappos come to mind) seem to do quite well.

6. The Elements of Style by Strunk and White

Your single most important tool on the web is language. Without using thought-out, clear and easy to understand language, nothing else matters. I use this guide to remind myself of the dos and don’ts of proper writing style. Writing for the web is different to writing for print or other media, that’s true, but the basics of writing style and the core rules of well constructed language apply to the web just the same. Perhaps even more so! This is your guide (at least for the English language anyway) for the rules of the road of well-written content. PS – If your content writer has no clue what this book is – fire them and find someone who does!

7. Designing Interfaces: Patterns for Effective Interaction Design by Jenifer Tidwell

I originally was influenced to buy this book because I liked the picture of the duck on the cover, true story! I had no idea that this would be such a useful and usable book! Ever wonder why some web sites, even though they have complex subject matter, are so easy to use? This book explains the details of why, using best practices and patterns of design to help reinforce concepts. I think your book, like mine, will become very dog-eared over time! And that duck is just so darn cute!

8. Forms that Work: Designing Web Forms for Usability (Interactive Technologies) by Caroline Jarrett and Gerry Gaffney

I’m kind of surprised that this book didn’t sell out shortly after printing. By reading and using this book, any web site manager that has forms on his or her web site could probably double conversion rate! It’s a well-known fact that the vast majority of your form visitors will at some point abandon your form. Why? This book will help you easily answer that question. By applying the knowledge you gain from the best practices and principles in this book, you’ll decrease the number of abandonments, and increase the number of conversions. You’ll probably get a raise, or at least a bonus, and can finally take that trip you’ve always wanted to go on to Paris, the South of France and Italy (unless of course you happen to live in Paris, the South of France or Italy in which case you’re probably going to Walt Disney World).

9. The Design of Everyday Things by Donald A. Norman

Why is it so darn hard to change the time on a digital watch? Because the digital watch designers did not read this book! I consider this the grand-daddy of usability books. If every developer of a new device or software application was required to read and comprehend this book prior to development, our world would instantly become a much happier place to be. Read and refer to this book when you begin a design project and you will absolutely create a better design.

10. Letting Go of the Words: Writing Web Content that Works (Interactive Technologies) (Interactive Technologies) by Janice (Ginny) Redish

Are you looking for a way to double your online sales or transactions? Usability of content is critical to web site success, because content and language are your primary tools and thus critical for success (hmmm, I think I heard that someplace before)! Simply read this book, apply the concepts to your content, and viola! Your web site sales or transactions will increase right away! This is another one of those books that I refer to again and again, because the writing tips and guidance Ginny Redish provides are universal, meaningful and impactful. I can’t find the words to describe how helpful this book is for designing content that sells (reaching for the book), but I know where to go to get some advice!

11. Web Analytics: An Hour a Day by Avinash Kaushik

Usability is not just about Persona’s and usability testing, it’s about metrics and numbers and analysis of the those numbers as well. A smart usability practitioner is always watching the numbers. What are the trends? Where are the good numbers, where is the web site working well? Where are the bad numbers, what’s not working well? By understanding and analyzing web metrics, a good usability practitioner will know where the problem areas are, or if a new problem area pops-up. In addition, because metrics are the lifeblood of most of the rest of the company, you’ll be speaking the same language and thus able to communicate with your co-workers much more effectively. This book provides excellent guidance and advice into what metrics to analyze, how to analyze them and what to do with that analysis.

12. Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions by Tim Ash

If your company uses landing pages for online lead generation activity, and you conduct usability testing for a company, you may not have focused heavily on landing pages. That’s not a good thing. As with online forms, landing pages can have very high abandonment rates, which means lost revenue for a company. Stated in a more positive way, any usability improvements made to landing pages will almost certainly mean improvements in conversion, which means improvements in revenue. However, landing pages are tricky, they do not have the same purpose or function as a content-laden marketing page, and must be designed and tested differently. I like this book because in it Tim Ash provides clear and easy to understand guidance on how landing pages work. He provides design best practices and defines what analytics to measure and how to analyze them. This is another dog-eared book I refer to from time to time.

So that’s it! That’s my list of 12 really useful usability books. I hope you find it helpful from time to time.

If you have your own useful books please do share that by posting it in a Comment. That way, we can all have more helpful and useful usability information at our fingertips!