Using Pretend in UX Research is a bad thing, here’s why.
If you find yourself using ‘pretend’ or ‘assume’ when conducting UX research, it means there are potentially major problems with your study. Watch this video to learn what to watch for, and how to fix it.
Using pretend or assume in your UX research should be a warning flag that your research may be flawed. That’s because when we use the words ‘pretend’ or ‘assume’ we are in fact forcing someone to consider a task or situation that they may have limited or no experience with. And that’s a bad thing. Why? Because flawed research leads to flawed findings, which leads to flawed experiences and poor performance.
Does this story sound familiar? You (or someone you hired) are testing your website with a usability study where your testers are asked to conduct a few critical tasks as you observe them. The session starts by asking a few warm up questions to your testers. After this warm-up, the actual test begins, with something along the lines of…
“Let’s assume you want to ________ [fill in the blank with your task] on this website, can you show me how you would do that?”
Here’s another variation…
“Pretend you want to ________ [fill in the blank with your task] using this app, can you show me how you would do that?”
Seems innocent, right? It’s not. Pretend and assume are red-alert warning words in your testing protocol that should make you aware of a hidden danger. The danger is your research might be flawed.
Now I’m not suggesting that you can NEVER use the words assume or pretend in your scenarios or protocols. In certain situations it actually may be alright to use those words. But only if the tester matches your personas AND your tester actually needs to do whatever the task is that you want to test.
But if the tester does not need to do those tasks, or has never done those tasks, you may be forced to ask them to pretend or assume they want to do those tasks. You are now using pretend or assume to motivate your testers to do tasks that more than likely they may have little or no motivation to actually do. What’s the danger?
The actions they do to complete the task may not be even close to what truly motivated people do trying to complete the same task. So the words pretend or assume should be a major warning sign to you that you may be going down a research path that may lead to bad data.
Using Pretend in Your UX Research
So what can you do if you find yourself using the words ‘pretend’ or ‘assume’ in your UX studies? Watch my brief video on using pretend in UX research to learn more about why those are major warning sign words, and what to do to fix your research and ensure a more positive outcome.
A Story of Using Pretend and Assume
Recently I was contacted by an eCommerce company who was just about to give up on usability testing and UX research. They had tried usability testing because no matter how many A/B tests they ran in their buy-flow, they couldn’t increase conversion. But the usability testing they had done didn’t improve conversion either.
I reviewed their prior testing. They had conducted four unmoderated tests with test participants recruited through an online user testing service. The task was “assume you want to buy a nice watch for your birthday, show us what you would do.”
I watched each test video and the problem was obvious. The testers had no inclination, no desire nor any need to conduct this task. Because they didn’t have any skin in the game, their interactions on the site and on the buy flow were not even close to what someone who was really shopping for a watch would do.
I’ve seen this over and over and over again with people saying unmoderated testing doesn’t work. The test is flawed because it’s using pretend and assume, so the results are bad, so everyone blames the unmoderated testing service.
Here’s a clue:
And here’s another clue:
So how did I help that eCommerce company? Simple. First, I recruited people who were actually shopping online for gifts. Second, I asked those testers to include my client’s eCommerce site as part of their shopping. Third, I wrote my testing protocol WITHOUT the words assume or pretend.
The happy ending to this little story is that we uncovered some issues with the product pages, and the filtering of products on their eCommerce site. As it turns out, this was actually the root cause for the poor conversion. The shoppers were entering the buy flow not because they were ready to buy, but because they were confused about how to find the products they were interested in. Fixing those issues increased the number of qualified purchasers entering the funnel and increased conversion.
Moral of the story?
If you are using assume or pretend in your UX research it is a warning sign that your research is in danger of producing bad results. Watch my brief video on using pretend or assume in UX research to learn more about how to diagnose the issues and fix the potential problems.