Authors Posts by Craig Tomlin

Craig Tomlin

I've been improving revenue with online marketing, usability & conversion optimization for start-ups, small businesses and Fortune 500 firms since 1996. I'm a Certified Usability Analyst and multi-award winning marketer. Contact me for website usability testing and audits!

0 123

Interview with UserZoom Co-CEO Alfonso De La Nuez with Thoughts on the Future of UX

UserZoom-CEO-Alfonso-de-la-Nuez-Interview-UsefulUsabilityThis is one in the continuing series of interviews with UX and Usability thought-leaders. This interview series focuses on bringing you up close and personal to those individuals in our industry who are responsible for moving user experience, usability and design in a new and exciting direction.

Alfonso de la Nuez is the Co-Founder and Co-CEO of UX and Usability research firm UserZoom. UserZoom is an enterprise-level all-in-one set of research tools for obtaining UX data across multiple user touchpoints.

Q1. What’s your background? Where did you go to school, what subjects interested you?

I have a BA in Business Administration from San Jose State University, with emphasis on International Business with Marketing. I’ve always been fascinated by the Business world, especially with Marketing and Branding strategies.

I lived the birth of the Internet while in college so I’m really passionate about the use of technology within Marketing. I believe at the end of the day, UX is nothing but great Marketing and Branding.

Q2. How did you get into usability and the UX testing field?

Since I had already connected technology with Marketing, UX came to me really easily and spontaneously. Back in 1998 I worked at a pioneering Web Design & Consulting company called Icon Medialab, out of Sweden. Even back then, the concept of HCI was highlighted and used for pitches and, although not that often, offered as part of the service.

As a Project Manager at Icon Medialab, I actually defined the information architecture and conceptual design of many of our customers. I realized then that, no matter how great the design team, we were literally making wild decisions and totally neglecting the end consumer.

I always felt wrong about it and so I ended up leaving the company and, eventually, started my own UX Consultancy, called Xperience Consulting. Xperience Consulting became UserZoom a few years later, and here we are today!

Q3. What is it about UX and usability that you most enjoy, or find most rewarding?

Since I went through quite a lot of ‘painful work experiences’ back in the late 1990s building million-dollar websites without real focus on the end user, what I enjoy the most is the ability to test things with a target audience before making an important decision.

Today, thanks to technology and the cloud, you can do this so easily, quick and cost-effectively, that it’s almost a crime not to conduct usability testing! :-)

Q4. You founded the usability testing tool and service UserZoom, what was the inspiration for developing it?

I’ve pretty much already summarized it in the previous questions, but there is one more important thing: We needed to make usability testing and research more easy and, most importantly, we wanted to get out of the lab!

Before founding UserZoom we’d been in the Lab hundreds of times. Don’t get me wrong, we enjoyed the ride and delivered solid value. We actually had 6 labs between 2 offices and 2 cities. However, this was not efficient, nor scalable.

We felt there had to be a way to collect data more cost-effectively, easily and also internationally.

Q5. As a business owner and start-up founder, what do you find were pitfalls with creating your business and finding clients interested in usability testing?

UserZoom has been in business since 2007. I can honestly say that, with the exception of a few power researchers who adopted this unmoderated testing method from the start, it took most of the UX community until 2012 to really go for it and adopt it in their organizations. That’s when we realized we had a real business opportunity ahead of us.

UserZoom-Team-Interview-UsefulUsabilityToday we find that a lot more people are interested in usability testing and UX measurement, not just the UX Pros. The remote testing method we offer makes it much more efficient and accessible to them.

Q6. From your experiences with managing UserZoom, what are the most important uses of your tool for businesses, how does it help them be more successful?

Without a doubt, it’s all about scaling their research. It’s all about Agile testing and REALLY making it part of the whole product development process.

It’s clear that iterative design is the best way to go, so UserZoom allows businesses to conduct multiple types of online research projects (task-based, card sorting, click testing, surveys, etc.) in a quick and cost-effective way, while having the consumers participating in their natural context.

Q7. There seem to be lots of competitors in the online UX testing and research tool category, what do you think makes UserZoom special or unique?

There are several things to consider. Starting with the fact that we’re an Enterprise-focused platform, which means you get:

  1. High-end, feature rich functionality
  2. All-in-one, multi-method approach under one single platform (again, from task-based tests, qualitative and quantitative data, card sorting, tree testing, click testing, 5-second tests, UX benchmarking, true-intent surveys and more), all pretty much for web and mobile.
  3. Lots of customization and ability to adjust according to our client’s needs. This includes recruiting with; sampling companies and custom panels, live visitor intercepts, Email links, QR codes, feedback tabs and more.
  4. Local UX research and consulting services and tech support, with offices in 4 countries
  5. Toughest data security standards in the market

Q8. What do you think the next year to two years will bring for UX and for usability research?  Do you see them growing or changing, if so why?

First of all, I see more companies than EVER doing more TESTING and EXPERIMENTATION than ever before. I see this resulting in much better overall DESIGN.

Then I also see great opportunities in mobile, and the power of testing in-the-wild.

Finally I see a lot of consolidation and convergence happening in the market. I think we’ll see a lot of Merger & Acquisition activity because the C-level executives, starting with the Chief Marketing Officer, are already valuing the strategic importance of combining business strategy with UX, CX (Customer Experience) and design strategy.

Q9. What’s next for you personally in your career, and for UserZoom in the next year or two, what would you like to focus on?

We’re building a truly GREAT business here. We’re 100% focused and more excited than ever before.

UserZoom-CEO-Alfonso-de-la-Nuez-UsefulUsabilityI personally would like to focus on doing all this pretty quickly, with the highest quality possible, and preserving the culture that’s made UserZoom a well-respected and recognized player within the space, particularly among the Enterprise market.

Q10. Finally, what advice do you have for those who want to get into the usability and UX field?

To understand that to be really great at UX today, you need to cover various areas of knowledge, including not just the classic sociology, psychology or anthropology, not just design, but also business and marketing.


Thank you Alfonso!

For more information on Alfonso’s all-in-one usability and UX research services or to request a free personalized Demo visit UserZoom.

Note: Companies mentioned in this article may or may not be sponsors of this site. However, in no way does their sponsorship or lack thereof impact the results of this or any other editorial content on the site. The editorial content of this and all other articles written by the Editorial team is determined without regard to whether a company is a sponsor of the site or not.

0 141

Should I usability test my own site is a question I am often asked.

Should-I-Usability-Test-My-Own-Website-UsefulUsability-The answer to the question “should I usability test my own site” is not as clear nor as obvious as you might think.

There are multiple reasons why a quick “yes” or “no” doesn’t necessarily apply.

In this article I will explain the issues of testing your own website so you can be more informed when it’s time for you to conduct usability testing.

Usability Testing Your Own Site Definition:

So what exactly is usability testing a website? My definition is:

What I mean by that definition is “usability testing” is conducting performance-based tests on a website’s critical tasks using people who match the website’s Personas. What’s a Persona? A Persona is a representation of the most common website visitors who all share a set of critical tasks.

It is very important to remember that it is the critical tasks that are being tested, not the tester’s opinions about those tasks.

Likewise, usability testing is not about capturing survey data, or voice of the customer information (although both of those can be helpful additions to the performance-based testing).

So when people ask me,

“Should I usability test my own website myself?”

I reply with,

“If you mean you being the test participant? No. Because you can’t usability test your own site yourself conducting performance based tasks. You probably don’t match the Persona, and you are already too biased and know the correct task flow.”

Running But Not Participating In Tests:

Some of you may be wondering a similar question, which is,

“Should I set up and run a usability test on my website only (ie. not participate in the testing)?”

Here the answer is a bit more cloudy.

In general, I would caution against usability testing your own website, there are 5 reasons why:

1. Usability Training Matters:

Unless you are trained in usability testing, it’s dangerous to assume you have the knowledge and expertise necessary to very carefully write a non-biased testing protocol.

It is amazingly easy to find tools that anyone with or without training can use to conduct usability testing. But it is also amazingly easy for those without usability training to accidentally create a biased test that provides bad data.

When I was in the process of becoming certified in usability analysis I attend many classes prior to taking and passing my certification test. I would say about half the classes were focused on helping students understand biases; where they can creep in, how to uncover them, and how to write protocols that carefully eliminate them.

Without that advanced training, conducting your own testing can introduce hidden biases in the test protocol, which could make results invalid and cause resulting website usability changes to make things worse, not better.

2. Internal Usability Teams Can Be Biased:

Although less common, I’ve come across situations in which I was surprised to find that even internal company usability teams had hidden bias’ in protocols. Not always of course, but often enough over the past 20 years of my experience that I’ve decided the longer you spend time with a website or company, the more difficult it becomes for you to remove your own biases from the process.

That’s not to say that internal company usability teams can’t write non-biased protocols. It’s just that it becomes more and more difficult as domain expertise rises AND testing patterns become standardized.

Let’s face it, as humans, we are all creatures of habit. And for internal teams this can sometimes unfortunately mean picking up biased habits and sticking to those habits when writing protocols and creating tests.

3. Usability Testing Protocols Are Mandatory:

Sadly there’s a common misconception out there that anyone can usability test simply by walking down a hallway or using software to ask people to try to do something on a website and observe them.

I’m here to tell you that’s a dangerous oversimplification of how to actually conduct non-biased tests.

To ensure that bias is not in the usability test, it’s critical that a usability testing protocol be developed AND be reviewed in advance. Without the protocol, random “guerrilla usability testing” or “agile testing” can accidentally introduce bias into the process.

Some usability practitioners, entrepreneurs and usability testing software services may disagree with me.

But I’m here to tell you the primary mission of a protocol is to ensure you’re reducing bias and that you’re testing across a standard set of criteria for each test you conduct. Without it, test results may vary due to the unintended introduction of biases.

4. Moderating Usability Testing is Harder Than It Seems:

Closely associated with needing a protocol, moderating the actual usability tests without bias is not easy.

The test participant is looking for clues (after all, it is a test). They will instinctively read a moderator’s body language and other verbal or non-verbal queues. Being on guard to not generate those clues while at the same time gathering think aloud feedback plus keeping the tester focused takes skill.

The common misconception that anyone can usability test without training or bias is false.

It is best to let trained usability professionals handle moderating usability testing, else risk bad data entering the process.

Again, entrepreneurs and do-it-yourselfers may scoff, but I’ve seen plenty of bad websites made worse by erroneous data coming from badly moderated testing sessions.

5. Analyzing and Prioritizing Usability Testing Results Requires Expertise:

Analyzing usability testing results and prioritizing them also takes skill that I caution should not be handled by untrained usability practitioners.

Here’s a common scenario:

You’ve conducted usability testing and found 8 places in a task-flow with errors causing poor performance.

What’s the severity, and thus priority, of those errors?

Analyzing issues and identifying possible solutions requires a fair amount of user experience and usability best practices knowledge.

Going with gut feelings about the best way to solve an issue can sometimes work, and sometimes not. It’s better to analyze the results and make recommendations based on extensive experience in understanding UX and usability best practices.

Associated with this, I have seen plenty of bias in prioritizing results even by internal usability teams. That’s because being so familiar with their organization, they knew what could and could not easily be worked on by developers. They also knew what would and wouldn’t be easily supported in terms of workload by business owners.

How many times did the “hard to do stuff” take a back seat to the “easy to do stuff?” And, did the “hard to do stuff” have a much greater impact on usability than the “easy to do stuff?” Often it does, but often it’s not prioritized that way due to internal bias.

Conclusion: Should I Usability Test My Own Site?

So should I test my own site?

Now that you have a clearer picture of the issues around testing your own website, I think you’ll agree a quick answer of ‘yes’ is not always appropriate.

This is NOT to say that usability testing your own site cannot or should not be done!

Instead, it’s a warning that when testing your own site it is very easy to let bias into the results, thus causing bad data and potentially bad results from subsequent optimization.

Being aware of these issues, my recommendation is to consider your resources, and if it makes sense contact a professional usability consultant like me or others in our industry anytime you’re considering usability testing on your own website.

Further Usability Testing Resources: – Usability Testing Methods – 14 Usability Testing Tools

Human Factors International – Certified Usability Analyst (CUA) Training

Nielsen Norman Group – Full Day Usability Testing Training

Interview with Steven Cohn CEO Validately, the remote moderated usability testing and research tool


This interview with Steven Cohn, CEO of Validately is one in a continuing series of interviews with the thought leaders of the usability and UX world. If you have not heard of Steven Cohn and Validately, you are in for a treat!

Steven reminds us why remote moderated usability testing and research is so powerful, and why it can literally save, or curse, a new start-up or existing established brand.

Q1. What’s your background? Where did you go to school, what subjects interested you?

Steven-Cohn-CEO-Validately-from-UsefulUsability-200x200I am a multiple-time entrepreneur. Validately is my third startup. My first start-up had a successful exit to LivingSocial. My second startup was a failure. We sold it for very little to TripAdvisor. I went to George Washington University for undergrad, where I met my wife. I graduated from Harvard Business School as well.

I am deeply passionate about starting companies and building great products. The challenge consumes me mentally. I love the journey of building a new product from a concept to a business.

Q2. How did you get into usability and the UX testing field?

I was very frustrated after the failure of my second startup. I was angry at myself and the time that I wasted chasing false positives. So I spent about a year interviewing Product and UX teams at great companies. I asked them how they build great products and the answer was clear and consistent…a lot of User Research and Testing! So I threw myself into that topic. I read everything that I could and met a lot of thought leaders on the topic.

Q3. What is it about UX and usability that you most enjoy, or find most rewarding?

Watching the usage chart go up and to the right. I deeply believe in the value of testing a prototype to learn. But I am always skeptical that the test learning is valid until we see customer’s reaction in real life. The chart has to move up and the to right or we go back to testing new potential solutions.

Q4. You recently created the remote usability user research & testing tool and service called Validately, what was the inspiration for developing it?

Validately team: Natalie Hollier, Mark Bathie, Steven Cohn, Jessica Ogilvie

I believe deeply that extensive user research and testing is critical building products that customers love. Specifically, I believe that moderated test sessions are invaluable.

Yet, moderated testing is still so manual and time consuming:

  • Recruiting real people who match your persona (not accessing a panel) is still too costly and time consuming.
  • Handling the manual tasks of scheduling and paying testers is still to manual.
  • Tools to run and record the tests are not tied to the proper reporting to stakeholders, so often it takes longer to create a post test report than conduct the actual research.

As a power user of user research, I wanted this to be fixed. I wanted a seamless product that took the pain out of user research. That’s why we started Validately.

Q5. As a business owner and start-up founder, what do you find are common issues or misunderstandings clients have about testing in general and UX testing specifically?

The one that I hear the most, which kills me, is, “I don’t have time to test.” Candidly, I believe that any organization that says “we don’t have time to test” are building product the wrong way. If you go into any fast growing company, you will see a significant investment in user research. Product and development teams are not meant to be feature factories.

More features ! = better product!

Companies that invest the time and resources into User Research have cleaner, more usable products, and customers reward these companies with increased usage and word of mouth marketing.

I haven’t quantified it yet, but anecdotally, I believe that there is a high correlation between investment in User Research and growth. Much more than any “growth hack” that any company can employ.

Q6. From your experiences with managing start-ups, what key aspects seem to be the most critical for success, and why do you think that is?

I frequently give workshops for companies and speeches at conferences around Demand Validation. Specifically, how to avoid false positives in early product development by requiring early customers to prove what they say. Specifically, I tell PMs, UX’ers or User Researchers, that if you are validating demand, do not be satisfied with the answer to:

“Would you use this product/feature?”

The problem with asking this questions is that it does not cost the tester anything to saying “yes, I would use it.” It is literally a no-brainer question. Would I use that? “Sure, I can see that happening.” But that is not validation. The better questions to ask are:

  1. Time Commitment: Would you spend several hours testing designs and early coded versions of this product/feature for us?
  2. Social Proof: Would you introduce us to 3 other people who would use this product/feature?
  3. Budget validation: Would you agree to buy this product at this price when we release this? If yes, will you sign this LOI and provide your credit card?

I go in more detail on this topic on this post, How to Validate Demand with Prototypes. But I am happy to discuss with anyone interested too. Just contact me on the Contact Us page on Validately.

Q7. You’ve created other start-ups, why get into remote testing and research with Validately, why do you feel it’s important and what makes you different versus your competitors?

The value of remote testing is that it tests the user in her native environment. Don’t get me wrong, there is value in testing labs. But let’s be honest, no one acts normally in a testing lab. So philosophically, I believe in the value of remote testing. Specifically in remote moderated testing.

Validately is different than anything that I have seen in a few ways:

    1. Seamless, lightweight moderated testing in both desktop and mobile. Our technology does not require any download or plugin. Just click on the testing link and you are live with the moderator.
    2. Recruiting to match your persona, not from a trained panel. We emphasize testing on real people, not trained panelists for moderated testing. Thus, we recruit testers for our customers from LinkedIn, Facebook, Google AdWords and other sources that match a specific persona. Don’t get me wrong, there is value in having a testing panel for unmoderated usability tests, which is why we offer a panel for un-moderated. But when you are doing moderated research, you need to talk to real people who match your specific persona.
    3. Reduce the post-session work for the researcher. We are releasing an entire suite of features aimed specifically at the pain point of tying videos to analysis. Our goal is that by the end of your moderated session(s), you will automatically have a complete report compiled that you can send to stakeholders with your notes and clips already queued up.
    4. Flexible, reasonable pricing model. We are Lean to our core and so we wanted to build a product that doesn’t let cost get in the way of learning. We believe that our pricing model is the most reasonable and flexible in the industry, so that you can test more!

Q8. What do you think the next year to two years will bring for UX and for remote research?  Do you see them growing or changing, if so why?

Trend 1: The UX & UX Research fields are exploding. Check out the graph of UX designer job trends from Indeed. That is exponential growth! I think this trend is going to continue. User Research is going to be table stakes, not a unique advantage.


Trend 2: Technology and automation is going to continue to improve.

  1. Anything that you do manually right now and hate, is going to be automated soon.
  2. Anything that slows you down from doing more research is going to be automated soon.
  3. Prototyping technology is going to continue to improve decreasing the difference between live code testing and prototype testing
  4. Longer term: Technology is going to help identify pro-actively usability issues that you have in the product.

Trend 3: I think the prices are going to come down. You won’t be able to charge $49 for a basic un-moderated usability test.

Q9. What’s next for your career and for Validately in the next year or two, what would you like to focus on?

What’s next for my career? :) I plan on being CEO of Validately for the next decade or so. My focus throughout that time will be the same as it is now… building a product that customers love.

Thank you Steven!

For More Information on Remote Moderated Usability Testing:

See the article: Remote Moderated Usability Testing Tools which contains description, benefits and tools including Validately.

Validately: The remote usability user research and testing tool and service founded by Steven Cohn and Co-Founder Mark Bathie.

Note: Companies mentioned in this article may or may not be sponsors of this site. However, in no way does their sponsorship or lack thereof impact the results of this or any other editorial content on the site. The editorial content of this and all other articles written by the Editorial team is determined without regard to whether a company is a sponsor of the site or not.


Why Pop-ups Work So Damn Well

Why do pop-ups work so damn well? Let’s face it, you hate them, I hate them, we all hate them, yet they are everywhere. Damn those POP-UPS. You can’t read an article on a website without getting at least one, sometimes more than one pop-up asking you to join a newsletter or download a free report! EVEN MINE!

Why-Do-Popups-Work-So-Damn-Well-UsefulUsabilityBut have you asked yourself, WHY are they everywhere?

My friends, it’s because those damn pop-ups WORK, and they work damn well.

I didn’t truly realize how well they work until I tried one on this very website to generate sign-ups for my monthly eNewsletter.

Prior to enabling this pop-up, I was lucky if I received one or two new members a week. I was “doing the right thing” by placing my offer to join the newsletter on the right sidebar or down at the bottom of the article. But guess what? I didn’t realize it until later, they didn’t work then, and they don’t work now!

So I turned on the pop-up (feeling just a tad that I was signing a pact with the Devil), and voila! I was getting two or three or six sign-ups a day! Wow!

The UX guy in me felt bad about the user experience. The marketer in me LOVED being able to finally reach my audience with a message they paid attention to.

Here’s the numbers just to prove it…


So WHY do pop-ups work?

Because size, interruption, and something I call “Cognitive Concordance” (opposite of Cognitive Dissonance) matter! I’m simplifying this heavily, so all you HCI PhDs don’t judge me, thanks.

Sorry, but size matters.

As humans, we tend to notice larger things before smaller things. The larger the thing, the greater the amount of awareness we tend to focus on it, at least momentarily.

For example: A very large mommy Mastodon is charging straight at you with her small baby at her side. You’re typically going to notice the very large mommy Mastodon first. That’s because if it were the other way around you’re less likely to be around to share your DNA with future generations.

This is a pre-wired function in all of us.

So, a very large pop-up that takes over a very large area of your display WILL attract much more of your attention, at least momentarily, than the relatively smaller sidebar or bottom sign up forms.

Interruptions are great for attracting attention.

That’s why when you’re watching TV (until the birth of Tivo that is) you are forced to sit through a [Squirrel!] whole bunch of commercials that interrupt your favorite show (unless you are watching HBO – hi all you Game of Thrones fans – Red Wedding! Holy cow! Right?).

Interruptive marketing forces you to at least momentarily notice the thing marketers want you to notice, like their commercial.

So if a big ‘ol pop-up pops-up 10 seconds after you start consuming content on a site, you’re going to notice it, at least momentarily. And you noticed that Squirrel! too, didn’t you?

Cognitive Concordance is another reason pop-ups work (my term, don’t bother looking it up in Wikipedia, it doesn’t exist).

By placing the offer for you to get even MORE valuable information from my emails while you are already consuming valuable content on my website, the offer is more likely to be in cognitive concordance or harmony with your feelings that this information is good, and so you are more inclined to sign up for more.

So you signed up.

And for the naysayers of you who don’t think pop-ups can improve your sign-up and conversion rates?

Well, I suggest you test one for a month or two and judge for yourself if you think they work.

So what about the future? Will pop-ups someday get worse?

Sadly (at least for non-marketers), this is now happening.

Pop-up firms are rolling out even more “in your face” pop-ups such as the one called the “Welcome Mat.” The Welcome Mat is a complete take-over of your website (i.e., an entire page of a pop-up) and forces visitors to either agree or decline an offer before they ever get to the page they wanted to visit!

Oy Vey!

Unlike you, I’m old(er).

And I remember the “dark times” of the early internet waaaaay back in the 1990s when all designers thought that a “Tunnel” was a really great thing.

They thought…

“Hey, why not force my visitors through a tunnel of stuff I want them to see before actually letting them get to the stuff they want to see?”

Friends, Tunnels were a really bad idea then, and I’m going to go out on a limb here and say their cousins the Welcome Mats will be a really bad idea now (if not done in very precise behaviorally targeted ways).

There IS a right way to properly use a full page Pop-up (or even the dreaded Tunnel). But it requires knowing much more data about your visitor and triggering the pop-up and its content based on well-specified business rules and behavioral triggers. I won’t go into the details now, but it does work IF implemented the proper way. And that is MUCH harder to do than most designers and marketers realize.

But I’ll warn you right now, you’re more than likely going to see a LOT more full page Pop-ups in your near future.

Brace yourself!

Conclusion: Why do pop-ups work

Pop-ups, even though we all say we hate them, actually work very effectively to attract our attention and cause us to take action. The primary reasons why pop-ups work so well is because they use large size, interruption and cognitive concordance to hyper-focus our attention and take action. In the future, variations of pop-ups will make the design rounds, including full page pop-ups such as the “Welcome Mat” and others.

And just like commercials, pop-ups won’t be leaving anytime soon.

0 701

The MagicBand UX is the Future of UX, Here’s Why

The MagicBand UX is the future of UX, of this I am certain. What’s a MagicBand? It’s a device, website, app and user experience that the Walt Disney Company created to enable seamless interaction and eliminate friction across a variety of systems. Here’s why it’s the future of an excellent UX.

MagicBand-the-future-of-ux-UsefulUsability“Future of UX” seems like a rather bold statement, right?

But before you dismiss me as a blithering idiot, allow me to explain and show you the future via a great user experience.

First, let’s briefly define what we mean by a great UX (since there’s no such thing as a common definition for “UX,” a term everyone overuses to death).

Craig’s definition of great UX:

“Great user experience equals interactions that seamlessly cross boundaries of channels, systems and/or divisions with no friction.”

So what does this have to do with Disney’s MagicBand? I can definitely tell you that the user experience of MagicBand, or to be more specific the ability to have a seamless user experience across widely-differing systems and tasks, makes MagicBand an example of a great UX. It does this by making multiple parts of your life vastly easier without you having to even think about it.

For those of you who have no clue what a Disney MagicBand is, here’s a brief summary:

MagicBand-UsefulUsabilityThe MagicBand is an RF device that you wear like a watch or fitness band, and it simplifies your life by…

  • Opening your DisneyWorld hotel room door (so it’s a key)
  • Giving you access to a Disney airport bus and all DisneyWorld parks (so it’s a ticket)
  • Enabling you to instantly pay for food, services and merchandise (so it’s a credit card)
  • Facilitating making dinner or event registrations months in the future (so it’s a reservation pass)

MagicBand-CloseUp-UsefulUsabilityAdd to this the ability to swim with it (it’s water resistant – but I wouldn’t dive to the bottom of the great barrier reef with it), the ability to personalize it (because the bands can be customized and you can add your name to it, mine is “Craigster.” Don’t judge me), and you can drop it and step on it without breaking it (because it’s rubber and plastic and yes, been there, dropped that, didn’t break it).

The MagicBand is coupled with the My Disney Experience website and app, which enables you to create your itinerary, reserve FastPass+ and restaurant times and book other activities. Together, they make a powerful system that eliminates much of the friction that formerly spoiled a fun vacation.

For those of you who have been to a Walt Disney park without the MagicBand, you know the friction that was and is inherent in the system.

DisneyWorld without MagicBand:

You had to show a ticket at the turnstile to enter the park. You had to pay for every meal, trinket, or Mickey Mouse ears hat by whipping out your credit card from your wallet, paying for it, and remembering to put your credit card back. You had to wait in line for your FastPass and you had to show your FastPass ticket at the turnstile when it was time to use it (assuming you hadn’t misplaced it – been there done that). Finally, exhausted after a long day of having fun you had to use your room key (assuming you hadn’t already lost it) to get into your room and drop like a bag of dirty clothes on the bed.

DisneyWorld with MagicBand:

MagicBand-in-use-usefulusabilityBut with MagicBand? Everything is better. That entire set of user experiences is now made frictionless by simply using your MagicBand.

You use your MagicBand to take the free shuttle bus from the airport to the Disneyworld hotel, and back.

You get into the turnstile at the park entrance with it.

You purchase your meals, trinkets and Mickey Mouse ears hats with it (I should mention here that you enter a 4 digit PIN as a security measure along with using your MagicBand, because the MagicBand is coupled to your credit card account).

Using MagicBand, there is no need to wait in line for obtaining a FastPass at a FastPass station, it’s now part of your mobile app which is connected to your MagicBand ID, so changing FastPass times is a breeze. and you go through the FastPass turnstyle by using your band (and yes, it knows if you’re trying to cheat by trying to show up too early).

Finally, your MagicBand lets you into your DisneyWorld park hotel room so you can fall gracefully onto your bed like a freshly cleaned and crisply ironed shirt (because you’ve enjoyed the seamless no-friction day in the park using your MagicBand).

MagicBand-in-use-UsefulUsabilityUsing the MagicBand eliminated a ton of the friction that typically comes with visiting a Disney park.

It made for a much more enjoyable and stress-free visit.

Instead of waiting in 2 hour lines or running across the park to get a space at a ride, we were able to actually enjoy our visit, just like the old days when we first started visiting DisneyWorld waaaaay back in the mid 1970s. (Yes, I’m that old, don’t judge us old-farts).

Why MagicBand represents the Future of UX

So why does MagicBand represent the future of an excellent UX? Simple!

Using the same scenario, you could in the near future be using your personalized “Band” (probably your Apple Watch or another Smart Watch or even an app on your phone – no need for a custom and extra band) to…

  • Pay for everything you buy whether at the grocery store, a retail store or a gift store
  • Grant you access to your front door, your car door, your apartment building front gate, your office door at work
  • Enable you to enter your venue whether a movie theater, a Broadway theater or your kids school theater
  • Track your body stats (heartbeat, steps, altitude, GPS, map your journey, etc.)
  • Be your watch and phone and personal alert system

Think about all the friction that a MagicBand type of UX could eliminate throughout the many tasks that make up the user experience of your day!

Until you’ve tried a MagicBand, the raves I’m lauding on it may not impress you. But I can assure you the friction a MagicBand eliminates, multiplied by the many, many use cases in which it could be utilized in everyday life, will revolutionize your daily user experience and eliminate a ton of your annoying friction.

Security and MagicBands:

And for those of you who are a bit paranoid, you may be asking…

“What about security? What if lost my band or someone steals it?”

In my vision of the near future, security is handled just like with your credit cards today. The security is provided via a two-step process.

  1. You must touch your device to a reader (it’s an RF device so near proximity ensures that only the device holder is using it), which reduces the chances of scammers using it virtually.
  2. You are required to use a 4 digit PIN (that you select) in conjunction with your Band, just like your credit card today.

And just like losing a credit card or having one stolen, losing a Band in the near future will not be that big a deal. The systems and processes are already in place to handle that. As with a credit card you may have lost, you’ll call the bank, notify them so they can kill your device (and don’t you just know that as soon as that’s done you’ll find the darn thing again?!). You’ll then go to your nearest bank branch to get your replacement, enter your PIN, and away you go.

Does MagicBand Mean Big Brother is Watching Me?

Wondering if the MagicBand means big brother is watching you? For those of you “Big Brother is Watching Me” types, if you are concerned about being tracked and observed by potentially dubious companies, individuals or your Government, welcome to the world of using your smartphones and credit cards (they are just as easily tracked my friends).

My attitude is, unless you’re completely living off the grid (in which case you’re not even reading this), only use cash for all purchases (you really feel comfortable carrying around a big enough wad of cash to buy a car or house? Really?), or live incognito with a fake name and Persona for nefarious reasons then don’t worry about it. Yes the MagicBand can be tracked, however this tracking is limited as the RF device only broadcasts to about 40 feet. Leave the proximity of local towers and tracking systems and you’re not being tracked. But really, big brother is most likely already tracking you via your other devices and using this device isn’t going to make that any better or worse.

In reality, the advantages of seamlessly being able to interact with multiple systems using just one Band handling all the tasks you typically do in a day in many ways makes up for the fact that yes, you could be tracked.

As a matter of fact, tracking might be a good thing, as any parent who’s stayed up at night worrying about where their son or daughter is could potentially have that information if there were a way to selectively turn on personal tracking.

MagicBand Drawbacks

MagicBand does have a few drawbacks. First, it’s a very expensive and extremely complex system behind the scenes. We’re talking Billions of Dollars that Disney had to invest to get this system running.

The MagicBand requires multiple networks, readers, antennae and tons of integrated back-end data systems and points that all have to play very nice together by connecting seamlessly. Just wiring DisneyWorld was a major endeavor requiring massive funds and investments of resources over multiple years.

Another obvious drawback is the MagicBand itself. Like the Nike Fuel or a Fitbit it is a device that has to be manufactured so there’s a cost there. It’s an extra “thing” you have to wear, and it costs you money. All negatives. Plus, because it is sealed the battery and RF transmitter have a shelf life which means after about 2 years you have a dead piece of plastic to further recycle or heaven forbid fill an already full landfill. The easier and lower-cost solution would be to just couple the apps that drive it into some other device like your Apple Watch, a smartphone or existing bands (like a Fuel or Fitbit).

Another major drawback to consider is who is going to pay for all this. Since so many systems are involved the investment to “MagicBand the U.S.” or even a large city is vast indeed. The powers-that-be in large and small companies may object to paying for the expense. After all, the systems to be integrated could potentially include vehicle manufacturers, door lock manufacturers, financial institutions, retail firms, restaurants, event centers, mobile networks, hotels, etc. etc. etc. Would everyone be willing to chip in to help make this vision a reality? I kind of doubt it.

In its current form the MagicBand requires special touch devices that you must touch your MagicBand against in order to enable your sales transaction or entrance to the park gate. Those are probably not cheap, and it’s highly unlikely that small mom-and-pop retailers or other establishments would be willing to pay for one. Although for all you Canadians you already are ahead of the curve on this, think about the user experience today of using your mobile credit card readers at restaurants to swipe your credit card, enter your PIN and approve your transaction.

Finally, these MagicBands and touch devices are pretty awkward to use from a heuristics perspective. I consistently found it difficult to touch the “sweet spot” to activate my transaction at turnstiles and shops without some deft manipulation of my wrist to get the Band in just the right spot. People with physical disabilities most likely have issues as well.

Conclusion: MagicBand is the Future of UX

Still, for all its issues, MagicBand does reflect the future of UX.

Most likely, further refinement could eliminate many of the flaws currently inherent in the MagicBand UX today, making it an even better user experience in the future.

The primary and most amazing thing however is the incredible freedom and friction-less way a MagicBand enables people to conduct multiple tasks for a variety of needs without even thinking about it. And in that regard, Disney has indeed created the future of UX.

Reducing friction is good UX, but eliminating it is great UX, and that’s what Disney’s MagicBand does at Disneyworld.

Put another way, the MagicBand user experience demonstrates how interactions that seamlessly cross boundaries of channels, systems and divisions with no friction can empower people and make for a much more enjoyable world.

MagicBand Additional Reading and Resources:

Wired Magazine’s excellent story on MagicBand with more details into the functionality and features.

FastCompany’s dark but fascinating inside story on the wars involved in the making of MagicBand.

Forbes story covering the announcement of the MagicBand rollout including listing functions that never happened (Characters greeting you by name for one).

New York Times original article announcing the rollout of the MagicBands, also including details on functions that eventually did, and did not happen.

Disney MagicBand page with more information on how the device and applications work.


0 221

Remote Moderated Usability Tools Description, Benefits and Listings

Remote-Moderated-Usability-Tools-UsefulUsabilityRemote moderated usability tools offer a better way to identify user experience task flow issues. Learn the description, benefits, method and tools for conducting remote moderated usability testing.

Remote moderated usability testing tools are powerful, they are almost equal to the ultimate best-case usability testing method, which is in-person moderated usability testing.

There are several online remote moderated tools that enable remote moderated testing. These include some older tried-and-true do-it-yourself tools, along with several all-in-one services that provide additional functionality and features.

I’ll briefly touch on how remote moderated usability testing works, the benefits of remote moderated testing, and then focus on several remote moderated usability tools that enable you to use this powerful technique to optimize your website or app.

How Remote Moderated Usability Tools Work

Remote moderated usability testing tools work by enabling a moderator (researcher) to be able to connect through real-time two-way communications with a test subject who is participating in the test. Because it is a two-way connection, the moderator can follow-up or probe the tester as the tester goes through the usability test.

Why is follow-up or probing during testing so important?

Because many times clues that testers are having difficulty completing tasks are non-verbal. Things like lifted eyebrows, head movements, stopping to pause or other non-verbal clues let a moderator know the user is having difficulty. Being able to follow-up or probe is critical to enabling the moderator to more deeply understand exactly what that usability issue is.

In addition, being able to follow-up and probe during testing can elicit a wealth of information from the tester, typically around what they were expecting to happen or find, versus what they actually had happen or found. This is crucial information for understanding the WHY of usability task flow errors. And without the WHY, it is difficult to determine what to do to improve the usability of that task.

Following-up and probing during testing ensures that the tester’s memory is fresh, enabling the tester to fully share their expectations, thoughts and feelings as they are being tested.

Remote unmoderated usability testing tools are very popular today, but they simply do not allow this essential data to be captured during testing. That is why remote or in-person moderated testing is better, it enables the moderator to capture all the information happening during testing.

Remote Moderated Usability Testing Method

Remote moderated usability testing methods are centered on creating a two-way communication between the moderator and the test participant. In addition, most methods use a recording and screen-capture tool to document both the tester’s screen interaction as well as the verbal “think aloud” information the tester is sharing during the test.

Benefits of Remote Moderated Usability Testing:

  1. Enables the moderator to follow-up and probe the tester while testing is occurring
  2. Adds critical WHY data to the usability test, by eliciting expectations versus reality information directly from the user
  3. Allows moderated testing of remote testers who may be in another city, State or even Country which expands the tester pool
  4. Saves large amounts of money and time versus in-person moderated testing
  5. Scaleable in enabling multiple tests in a minimal amount of time
  6. Can leverage website intercepts to enable testing of actual website visitors who match Personas

Remote Moderated Usability Tools:

There are two types of remote moderated usability testing tools:

  • Do-it-yourself tools
  • All-in-one remote moderated service
Do-It-Yourself Remote Moderated Tools

The do-it-yourself remote moderated usability testing tools are typically an assortment of three types of online tools:

  1. Two-way real-time online connection tool. These tools are widely available, think Skype, Webex, GoToMeeting, JoinMe etc. The purpose of this tool is connecting the moderator in real-time with the tester, enabling both audio and visual communication between the moderator and tester. Some of these tools enable still others to join in to watch the session, which is useful for allowing design team members to also observe the test.
  2. Screen capture and recording tool. These tools are also widely available, think Camtasia, SnagIt, Adobe Captivate and a host of others. The purpose of this tool is to record the session including the audio and the video of the tester’s screen as he or she conducts the test, along with the voice of the moderator. Although it’s not mandatory to have a recording of the session, most researchers like to do so to go back and analyze specific times in the test when usability issues were uncovered. It is also helpful to have these sessions recorded for sharing the tests with others (such as design team members) after the test is complete.
  3. Video editing tool. These are also widely available tools, consider everything from Camtasia, Movie Maker, Adobe Premiere and more to Mac based tools like iMovie. The purpose of this tool is to edit the testing sessions into highlight reels that specifically identify common usability problems. This enables the moderator to be able to concisely point out what’s causing the issue without the observers having to sit through watching all of the usability tests from beginning to end.

There are other tools that make life easier for a usability practitioner, including website intercept tools like Ethnio or survey pop-ups to find actual website visitors to use as test participants.

A mixture of the three types of tools listed above enables the researcher to be able to conduct their own do-it-yourself remote moderated usability testing in a cost-effective and scalable manner.

All-In-One Remote Moderated Usability Services

There are several all-in-one remote moderated usability services now available for the usability practitioner that eliminate the hassle of having to have all the tools above, while also offering valuable additional features designed specifically for remote moderated testing.

List of remote moderated usability services include:

  1. Validately – A relative newcomer with an easy to use interface that makes remote moderated usability testing a snap. Includes an intercept and screener with unlimited recording. Includes a scheduler to make scheduling sessions easy, as well as a built-in payment system for incentives to testers after their tests are complete. Highlight reel creation is not currently available, but is coming soon. Recordings cannot be downloaded. Finally, Validately is a very affordable solution when compared to the competition.
  2. UserTesting Pro Version – An all-in-one solution that includes the ability to have moderated remote usability testing with either the panel or your own participants.

I spoke with Steven Cohn, the CEO and a co-founder of Validately, along with Jessica Ogilvie, the CMO. Here’s what they have to say about Validately and why they chose to focus on a remote moderated usability testing service:

“We created Validately because we couldn’t find a remote moderated testing service that we really liked for developing and refining our own prototypes. With Validately, we simplify every aspect of remote moderated testing; finding testers who match Personas, scheduling sessions, paying incentives, storing and sharing large testing video files and lots more. And we worked very hard to make it all happen with NO downloads or plugins, which simplifies the user experience for the moderator AND the tester.”

To summarize the benefits of using a remote moderated usability testing service versus do-it-yourself: A service saves you from having to juggle multiple software tools yourself, it also provides the ability to utilize large panels (or better yet your own website visitors via their built-in intercepts), the ability to handle payment of incentives, scheduling sessions, and scaleability in the event you need to conduct continuous testing or testing at larger volume.

Conclusion: Remote Moderated Usability Tools

Remote moderated usability tools are almost equal to the best method of usability testing, which is in-person moderated testing.

The method for conducting remote moderated usability testing consists of connecting the moderator and the test participant in real-time using a two-way online communication tool, using a recording software tool to capture the audio and video of the tester session, and an editing tool to create highlight reels of the recorded tests to share the results with design teams.

The benefits of remote moderated usability testing include; enabling the moderator to follow-up or probe the tester as the test is occurring, the relatively low cost and time required for testing versus conducting in-person moderated tests, and the ability to intercept actual visitors of a site to use as testers.

Remote moderated usability testing tools come in two varieties, the do-it-yourself tools that are widely available, or an all-in-one remote moderated usability testing service that provides all the tools necessary plus additional functionality and features such as access to large panels.


For information on remote unmoderated usability testing tools read 14 Usability Testing Tools which includes comprehensive reviews of each usability tool and a comparison matrix.