I read an interesting article recently on the UIE web site, “Avoiding Demographics When Recruiting Participants: An Interview with Dana Chisnell.” The article presents a situation like this; what do you do when a client or Company’s internal research group requests that usability participants be gathered based on the demographic data that the client has gathered?
Finding Usability Testing Participants:
Finding usability testing participants is not all that difficult, assuming you know whom to look for. The theory is that you first identify the Persona or Personae that will typically be conducting the critical tasks on the web site, and then based on that find usability test participants that match the Persona. However, there are those in an organization who may try to help out by offering you the clustered demographic groups of users to recruit. They may suggest that “typical” users are age ranges 23-35 years old, work in the financial vertical, etc.
Age Vs. Behavior:
As Dana Chisnell points out in her article, this “help” is actually not all that helpful when recruiting for usability testing. The main reason why is the demographic data does not help you understand the behavior of the typical users. Remember those Personae? They define who the user is based on their behavior, and based on their critical tasks, not on age, gender or location. Also, demographic data does not describe how much domain expertise the typical users have, and doesn’t provide any insight into the typical performance associated with the Persona:
“For example, age is not necessarily an indicator of behavior, performance, or expertise — the attributes that make a difference in the results of a usability results. Just because someone is in her 60s, doesn’t mean that she’s more or less technologically savvy or more or less security conscious than someone in his 30s or 40s. Being 20 doesn’t make you an expert computer user. Being 70 doesn’t mean that you don’t know how to use a computer.If we wanted to see how different behaviors, performance, and expertise affects how people will use an application, we needed to look to something other than age. Professions by the way won’t help either. The client gave us a law enforcement category, but otherwise it’s unlikely that someone who is a teacher is more likely to be security conscious than someone who is in real estate.”
Demographics: Good for Ads, Not For Usability:
Demographic clustering is helpful when creating advertising or marketing communications that seek to reach a specific target audience, with a specific target message. Recruiting usability participants however has little to do with age, or profession, or even geographic location. Instead, usability testing participants should be recruited based on matching the behaviors, needs relative to specific critical tasks, and a base of knowledge about the topic the user is expected to have.
Find Usability Participants Via Behaviors:
What then should you do if presented with the task of finding usability test participants by demographics? Dana Chisnell’s advice is to change the discussion by asking several questions back to the demographic-loving clients. I won’t steal her thunder, but I will say it comes back to the basics of why are they wanting to do a usability test in the first place, what is testing supposed to find, and what behaviors are supposed to be changed or improved by using the results of the usability test.
