14 Usability Testing Tools Matrix and Comprehensive Reviews

14 Usability Testing Tools Matrix and Comprehensive Reviews

16 521

A Comprehensive Review and Matrix of 14 Usability Testing Tools

Comprehensive review and matrix of 14 usability testing toolsIn 2009 I wrote a popular article on 24 usability testing tools. Since that time, there has been tremendous growth in the quantity and capability of usability testing tools. An update to the article is due, but because of the broad array of tools now available a more formalized definition and a matrix of usability tools is needed.  To that end, the following matrix and comprehensive review of 14 usability testing tools will help practitioners who are looking for tools for specific testing tasks.

Usability Testing Tools Matrix

For purposes of classifying tools to evaluate in this matrix, the tools must have three critical functions necessary to conduct a classically defined usability test:

  1. Audio and video recordings of testers (ala Think Out Loud method)
  2. Ability to edit recordings of tester sessions into highlight reels
  3. Capability to test using mobile devices (Smartphones and tablets)

The classical definition of usability testing is a systematic observation under controlled conditions to evaluate how well testers can complete critical tasks.

Thus, having the ability to record testers, to see their interactions and hear their thoughts (ala the Think Out Loud method) is critical to the observation process. It is also important to be able to edit those recordings, so that key insights can be shared with others in the form of highlight reels. Finally, the increasing adoption and usage of mobile devices to interact with websites and applications makes it necessary to conduct usability tests of the mobile user experience. Thus, tools should be able to test the mobile as well as PC based experience.

Not included in this review are Feedback or Click Tracking tools. This is because they only provide feedback or opinions about user interfaces, and lack the ability to observe testers conducting tasks. I will conduct the same comprehensive review of Feedback and Click Tracking tools in a future article.

Contact Us to advertise here

Or...
Contact Us to advertise here
Usability testing tools matrix from UsefulUsability.com

Four Quadrants of the Usability Testing Tools Matrix

Usability testing tools are divided into four quadrants on the matrix, based on where they fall when evaluated against two critical attributes:

  • Depth of Technology: What functionality and features are available with the tool, and how deep does the array of features run? Tools that score higher going up the vertical Depth of Technology axis have more functions and features.
  • Tool Impact: This is a measure of the ability of the tool to impact overall testing, and thus the observations and eventual recommendations for usability improvements. Tools that score higher (going to the right) on the Tool Impact axis have the ability to have a greater impact on overall testing results.

Based on these definitions, tools can fall into one of four quadrants on the matrix: Authority, Contender, Innovator and Niche.

  • Authority tools are those that provide a greater amount of usability testing functionality as well as a higher potential impact on discovering, diagnosing and ultimately optimizing the user experience.
  • Contender tools have the ability to provide a greater impact in terms of their depth of technology, but may have more limited potential impact in discovering, diagnosing and optimizing the user experience
  • Innovator tools have a more limited depth of technology, but have the potential to have a greater impact in usability testing and recommendations
  • Niche tools have a relatively limited depth of technology as well as a lower potential impact on usability testing and recommendations results, but can still be very useful for the specific tasks they were created for

All of these tools have specific features that make them unique.  Depending on your testing needs, any of these tools may be good choices and can compliment your toolkit.

Mobile Usability Testing Still Rudimentary

The upper right corner (Authority) of the matrix is blank for a very good reason. It represents the current lack of a fully integrated mobile usability testing solution. In my opinion we still lack a comprehensive, fully integrated mobile device usability testing tool, something that records both mobile device screen interaction and the audio/video of testers as they try to accomplish tasks. Because they are mobile devices, testing in this manner should be possible anywhere the tester goes, at any time, and using any smartphone or tablet (as long as it has a camera). Real time streaming of the screen and audio/video of the tester should be available for remote teams.

Mobile usability testing is currently in a neophyte stage. Yes, there are cludgy and cumbersome ways to test usability with a mobile device, typically requiring testers to place their phone on a desk with a camera hovering over it (thus not being at all ‘mobile’). Most methods today lack the sophistication of being a truly integrated and mobile experience.

Consider how just a few years ago new tools like Camtasia, Morae, UserTesting and others made usability testing websites an anytime, anywhere, quick and cost effective solution versus using expensive and cludgy labs or home grown equipment. We need that same fully integrated experience for mobile, but we do not have it, yet.

Authority Usability Testing Tools:

Usability testing tools matrix from UsefulUsability.com

Ovo Logger

Ovo Logger image from UsefulUsability.comOvo Logger by Ovo Studios can be considered to be the Rolls Royce of usability testing solutions. This is the tool for large Fortune 1000 enterprises or usability testing studios that conduct a high volume of testing and have teams spread around the globe. Typically these situations call for the need to have a robust tool to capture, edit and share results. It should be noted that Ovo Studios creates custom usability testing labs, and the Ovo Logger fits in well with those labs. The tool scores at the top of comparable tools for depth of technology, but is slightly limited in terms of impact due to requiring a dedicated PC, and being somewhat expensive to deploy.

Pros: Enterprise level usability testing software and hardware that records multiple inputs from PCs or Macs (cameras, microphones, screen interaction) and streams the output to observers. Editing tools enable real-time analysis of tests with the ability of observers to document important points. Can capture un-tethered iPads, non-jail broken iPhones and Android devices if using the Logger mobile app.

Cons: Designed specifically for larger firms with multiple locations and or usability teams. Requires a dedicated PC to run the software. Pricing is more expensive versus other usability testing tools and is based on number of sources, associated lab requirements, etc.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? Yes

Price: $3,000 and up depending on scope of associated usability lab

Camtasia plus a Screen Sharing Tool

Camtasia image from UsefulUsability.comA very powerful Do It Yourself approach for remote usability testing is to use a screen capture tool such as Camtasia (from TechSmith) coupled with a screen sharing tool such as GoToMeeting, WebEx or similar applications. The major benefit of this approach is the ability to conduct moderated remote tests, because the practitioner has a direct connection to the participant. In addition, the recording of the screen interaction with the voice and or video of the tester is immediately available for editing on the practitioners computer. The downside to this approach is lack of a simple mobile solution.

Pros: Enables remote moderated usability testing, because the practitioner has a direct connection to the participant. Sessions can be shared with other remote observers using the screen sharing tool. Audio and video recordings of the screen interaction with the voice of the tester are immediately available for editing on the practitioners computer.

Cons: For part-time or ad-hoc uses, this approach may be more expensive than using some of the other remote unmoderated tools available from vendors. Screen sharing tools are notorious for occasionally having technical problems that cause poor connection performance, or worst case dropping the connection entirely. This is not an optimal solution for conducting mobile usability testing, although cludgy methods of using a webcam pointed at the testers mobile device can be tried.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? No

Price: $299 for Camtasia, $19/month for GoToMeeting or $24/month for Webex

UserTesting

UserTesting image from UsefulUsability.comUserTesting provides a network of pre-screened testers who can meet several specified demographics. Or, practitioners can invite their own testers using the tool. Practitioners create a set of tasks for the testers to complete, either on your website or even competitor sites. Testers screen interaction and voices are recorded. UserTesting includes powerful and easy to use video editing tools to create highlight reels. UserTesting also provides mobile usability testing, which is a major feature considering the increasing utilization of mobile devices. Written responses to questions are included. UserTesting is one of the most robust tools to use for remote unmoderated usability testing and thus scores higher  for depth of technology and testing impact on the matrix.

Pros: Relatively low cost way to conduct remote unmoderated usability testing. Audio and video of the tester are recorded, and editing tools enable highlight reels to be created easily and quickly. Mobile device usability testing is available. The large network of testers typically means testing session results are available in an hour or two. Demographic filtering tools enable finding the appropriate testers for most Persona situations.

Cons: Because UserTesting pays small fees to the testers for each completed test, it is critical to carefully create non-biased tasks, else you risk the tester trying to cut corners to accomplish the test as quickly as possible. As a practitioner, you must watch for this kind of activity and request a re-do of that test if you are concerned about the quality of the tester. As an alternative, using your own testers can alleviate much of this. Mobile testing is available, but requires the tester to be at a desk so a webcam can be pointed down to the screen, which is not typically how most people use their mobile devices.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? Yes

Price: $49 per test with bulk pricing discounts available

Morae

Morae image from UsefulUsability.comMorae from Techsmith can be considered the Granddaddy of usability testing tools. This is because it was the first tool specifically created for usability testing. Even today it is still one of the best. It provides a comprehensive set of features available to record, analyze, edit and share usability testing sessions. Morae is an application that resides on the practitioners computer (PC only, sorry Macs). Morae records all screen interaction, including the testers voice and or face if using a webcam. Testers can be on any Mac or PC if testing websites or applications. Morae includes very powerful documentation, editing and sharing tools. Morae scores fairly high for depth of technology, and also fairly high for testing impact, the limiters being lack of a Mac practitioner solution and the issue of cludgy mobile testing.

Pros: Powerful screen capture and editing tools make this the choice for larger scale usability testing. Morae includes additional tools that can track a variety of screen and click interactions, plus has the ability to incorporate survey/opinion based data. Sharing test sessions with other practitioners and testing results with teams is relatively easy and quick.

Cons: Desktop & Laptop based system with no ability to record Mobile device screen interaction directly. Relatively expensive and PC-only (not Mac compatible) for the Manager and Recorder tools.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? No

Price: $1,995

Contender Usability Testing Tools:

Usability testing tools matrix from UsefulUsability.com

Userlytics

Userlytics image from UsefulUsability.comUserlytics provides both PC and Mobile based remote unmoderated usability testing. For mobile testing, the more expensive Gold or Enterprise plans are required. Testers can come from the Userlytics panel, or from testers you provide. Screen interaction, tester voices and webcam video images of testers can be recorded. Written answers to post test questions are also included.  The depth of technology is good, but because of the requirement to use the more expensive Gold plans for downloading and editing of videos, capturing of sessions longer than 15 minutes and testing mobile, this tools has somewhat limited testing impact versus some competitors. Practitioners that conduct larger volumes of tests should consider Userlytics.

Pros: Userlytics is relatively low in cost, and is definitely easy to set up and use. The more expensive plans allow download of recordings for editing and sharing. In addition, images, videos and other assets can be uploaded to the Userlytics secure servers for early prototype testing. White label testing results sharing is possible for Gold and above plans.

Cons: Userlytics has the same issues as any other remote unmoderated usability testing tool, with the burden being on the test creator to ensure non-biased questions. Userlytics has a rather confusing set of test functionality based on plans (Basic, Silver, Gold, Enterprise). For example, tests recordings are less than 15 minutes for sub-Gold plans, and mobile testing is only available on Gold and above plans. Depending on the plan (Silver or above), each monthly subscription price includes one or more ‘free’ tests, with each additional test costing $33 or less per test. There are also limits in terms of number of post test questions that can be asked based on plans.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? Yes

Price: $39 per month or more with bulk discount pricing available

Userfeel

UserFeel image from UsefulUsability.comUserFeel.com provides an assortment of testers, many of whom speak languages other than English, for remote unmoderated usability testing. Having the ability to choose a language other than English (many are offered) is a great benefit for testers of websites that are targeted for other countries. Written responses to post test questions are also available. Videos are available of the test on the website (but not available for download for editing purposes), and a white label playback option of the video is free. The depth of technology is good, but due to the lack of mobile testing and inability to download test recordings the tool impact is lower than some competitors. For those conducting usability testing in languages other than English, this could be a good solution.

Pros: A good tool for non-English language remote unmoderated usability testing. Demographic and language filtering tools are available. Pricing is relatively low and post test questionnaire functionality is available.  The free white label playback of the test recordings is a plus.

Cons: Like other remote unmoderated usability tools, practitioners must be careful to create non-biased tasks or else risk having testers try to quickly finish the test, causing poor test results. The advanced video editing capabilities of some competitor tools are not available. Videos cannot be downloaded and edited. Mobile testing is not available.

Tester Audio and Video Recordings? Yes

Recordings can be edited? No

Mobile testing capable? No

Price: $39 per month with bulk discount pricing available

Innovator Usability Testing Tools

OpenHallway

OpenHallway image from UsefulUsability.comOpenHallway is a remote unmoderated usability testing tool for PC or Mac-based research. Testers come from the practitioner (no panel is offered) via a link that is generated by OpenHallway when the scenario and tasks are set up. OpenHallway requires no code to use, which makes it a great tool for testing applications, websites and even competitor websites. Screen interaction is recorded, along with voice and or the testers face, if the tester has a webcam. Plans (Basic, Plus and Premium) determine the amount of video storage available. Unlimited number of test sessions are available, however the video storage amount will act as the limiter to how many sessions are actually conducted.  Because of video storage limits, and videos being MP4 only, and no mobile solution, OpenHallway has somewhat more limited depth of technology than some competitors, however the tool impact is good in that it can be used on almost any website with no code required.

Pros: OpenHallway is simple to use, easy to set up and therefore is a handy tool for recording unmoderated remote usability tests. As of the writing of this article there is a free trial.

Cons: OpenHallway is PC or Mac compatible, but does not support mobile.  Although no code is required, Java must be present on the tester computer for the tool to work. Videos are MP4 only, and only the Premium plan ($199 per month) allows downloading (and thus editing) of the video.

Tester Audio and Video Recordings? Yes

Recordings can be edited? Yes

Mobile testing capable? No

Price: $49 per month or higher, depending on the plan

UsabilityTools

UsabilityTools image from UsefulUsability.comUsability provides a variety of usability, feedback and click tracking tools, with the primary advantage being they all use a single practitioner interface. This makes creating tests that require multiple tools much easier. There are two versions, a UX Suite and Conversion Suite. The UX Suite contains click testing, web testing, survey, card sorting and a Persona Creator. Practitioners can set up web tests that provide scenarios to website visitors, who then try to accomplish the tasks and click success or abandon buttons based on whether they thought they could accomplish the task. Results are provided in data and charts. No code is required unless you choose to utilize their website intercept request on your site. The lack of tester audio and video recordings for PC and mobile is a limiter for depth of technology.  However, having the ability to utilize the tools efficiently and quickly provides a good tool impact.

Pros: A toolkit of the most common tools needed to fully understand the user experience. Having all tools in a single practitioner interface streamlines the process of creating and running multiple tool tests. In larger quantity testing scenarios, this set of tools can be more economical than using separate usability tools.

Cons: UsabilityTools does not provide audio and or video recordings of tester sessions. Mobile interaction using audio and video recordings is also not available. Pricing is not available on their website, although as of Sept. 2013 they had one time fees ($300) and monthly fees starting at $69 for the Professional version. Free panels of testers are not offered, although paid sources are available.

Tester Audio and Video Recordings? No

Recordings can be edited? No

Mobile testing capable? No

Price: Pricing information is not available on their website.

Loop11

Loop11 image from UsefulUsability.comLoop11 provides the ability to conduct unmoderated remote usability testing on PCs, Macs or mobile devices. The tool is easy to set up and fairly easy to administer (no code required). A key advantage is it can provide real-time data. Loop11 provides useful information for evaluating task flow on websites, including competitor websites. No audio and video recordings of unique test sessions are available, and because of this the depth of technology, and tool impact are lower than some competitors.

Pros: Loop11 requires no code, thus is useful for conducting comparison tests of user experience on competitor or other websites. Setting up and running tests is relatively quick and easy. Real time information can be advantageous for situations where quick results are needed.

Cons: As with other unmoderated tools Loop11 does not allow interaction with the testers to probe or ask follow-up questions. The test creator has to find his or her own test participants. No ability to record the tester via ‘think out loud’ methodology, something that exists with competitor tools. Somewhat expensive versus competitors.

Tester Audio and Video Recordings? No

Recordings can be edited? No

Mobile testing capable? Yes

Price: $350 per project or $158 per month

UserZoom

UserZoom image from UsefulUsability.comUserZoom includes multiple tools including remote unmoderated testing on PCs and mobile devices (via UZ Mobile), along with other tools such as card sorts, tree testing, screenshot click testing and more. For the remote unmoderated usability testing tool UserZoom enables tasks to be created and deployed to either 3rd party panels or the practitioner’s own panel. Intercepts are also available, a handy feature when trying to reach actual visitors to a website. UserZoom does include video recordings of screen interaction, but only at the pricey $9,000 per year or higher plans, and no voice capture of testers are available, thus Think Out Loud recordings of users explaining their actions are not captured. This limits the depth of technology and tool impact when compared to some competitors.

Pros: UserZoom includes powerful reporting and analytics tools that can greatly help researchers synthesize and analyze post test data, including heatmaps, time on task reports and much more. Plans allow practitioners to pick the best price point and functionality for their needs. Plans include a free Pilot, UZ Proof of Concept at $1,000, Basic plan at $9,000, Pro plan at $29,000 and an Enterprise plan that requires a custom quote. As of the writing of this article UserZoom offers a free pilot study.

Cons: UserZoom does not enable voice and or video capture of the tester as they try to accomplish a task, a major differentiator compared to competitor services. UserZoom does not have its own panel of testers available for free, as other competitors do. Practitioners can view and edit recordings of testers while they were conducting tasks to make highlight reels, but there is no audio nor video capture of the actual tester. UserZoom is rather expensive versus its competitors, and lacking the ability to hear and see testers via session recordings while they were conducting the task is another major differentiator between UserZoom and other remote unmoderated usability testing tools.

Tester Audio and Video Recordings? No

Recordings can be edited? Yes, but only recordings of screen interaction (no tester voice is available)

Mobile testing capable? Yes

Price: $9,000 for the Basic plan and up

Niche Usability Testing Tools

Usability testing tools matrix from UsefulUsability.com

TryMyUI

TryMyUI image from UsefulUsability.comTrymyUI.com is another of the many SaaS-based remote unmoderated usability testing tools. Testers create tasks, and videos are provided that include screen interaction and the testers voice. Written answers to post questions are also included. As of the writing of this article a free trial is included. Practitioners can use their own testers, but at a license cost of $99 for a single use test. Because video editing and mobile testing are not provided, the depth of technology and testing impact are lower than some competitors.

Pros: TryMyUI provides an alternative to other remote unmoderated tools that is slightly cheaper than many competitors. As a tool that provides video and audio recording of testers as they try to complete tasks it fits the classical definition of a usability testing tool.

Cons: All the same issues with remote unmoderated usability testing apply, with the burden being on the practitioner to create non-biased questions, and to carefully analyze the results to ensure the tester was not trying to use short cuts to complete the test. Video editing is not available, nor is mobile testing.

Tester Audio and Video Recordings? Yes

Recordings can be edited? No

Mobile testing capable? No

Price: $35 per test with bulk discount pricing available

WebEffective

WebEffective image from UsefulUsability.comWebEffective by Keynote Systems provides remote, unmoderated usability testing via tasks that are assigned and recorded. A panel is available, or practitioners can use their own testers. Data including click stream information is captured and displayed in a variety of reports, which can be downloaded. Individual sessions showing screen interaction are captured, however there is no audio nor video commentary captured of the tester as they go through the tasks.  Mobile testing is available. The lack of audio and video of testers as they try to complete tasks means WebEffective has a lower depth of technology and testing impact than some of its competitors.

Pros: A powerful tool to capture user interaction on websites or mobile devices, with a wide variety of reports available.

Cons: WebEffective does not provide audio and or video recordings of tester sessions. In this regard, WebEffective operates more as a click stream tool than a true usability testing tool.

Tester Audio and Video Recordings? No

Recordings can be edited? No

Mobile testing capable? Yes

Price: Pricing is not available on the Keynote WebEffective website

Ovo Solo

OvoSolo image by UsefulUsability.comThe Ovo Solo tool by Ovo Studios provides a small business version of Ovo Logger that enables screen interaction video capture along with the testers face and voice.  Simple logging controls enable analysis of the resulting test videos. However the tool is PC only. The relatively low price point makes this a good candidate for practitioners on a tight budget. Because of the PC only nature of the tool, the basic editing functions and the inability to record mobile sessions, this tool has lower depth of technology and testing impact than some of its competitors.

Pros: The relatively low price point makes this a very affordable way to capture, edit and display usability testing sessions.

Cons: This tool is PC only and is not able to record screen interaction on mobile devices. Editing tools are basic and may not meet more advanced usability tester needs.

Tester Audio and Video Recordings? Yes (but tool is PC only)

Recordings can be edited? Yes

Mobile testing capable? No

Price: $99

Silverback

Silverback image by UsefulUsability.comFor Mac users, Silverback from Clearleft enables remote and in-person moderated usability testing. Silverback is not available for PC based practitioners. Session screen interaction and audio and or video of the tester are recorded, and simple editing features like task tagging are available. Recordings are saved in QuickTime, which makes editing of highlight videos relatively easy for Mac users. The lack of PC functionality and video editing capability means this tool has lower depth of technology and testing impact than some other competitors. Mac practitioners will find Silverback a very easy to use tool at a very affordable price.

Pros: A relatively low cost and easy to use tool for Mac users that provides the basics in terms of classical usability testing functionality.

Cons: Mac desktop and laptop only, does not work with PCs. No ability to record mobile device screen interaction directly. As of the writing of this article currently does not support FaceTime camera with newer Macbooks, but the app is being re-written to fix this. Lacks powerful editing tools available in other applications.

Tester Audio and Video Recordings? Yes (but tool is Mac only)

Recordings can be edited? No

Mobile testing capable? Yes

Price: $69.95

14 Usability Testing Tools Conclusion

Usability testing tools matrix from UsefulUsability.com

Any of the 14 usability testing tools reviewed here can provide excellent results if applied in a manner which best fits their unique functionality and features.

If Your Favorite Tool is Missing

Not seeing your tool here? It could be because I’m classifying it as a Feedback or Click Tracking analytics type of tool. My original 24 usability testing tools article included many of these tools, but also included some tools that can better be defined as Feedback or Click Tracking analytics tools. In future articles I will provide a matrix and reviews separately for those types of tools. And if your tool is not a feedback or click tracking tool, let me know by leaving a comment.

Note: Companies mentioned in this article may or may not be advertisers on this site. However, in no way does their sponsorship or lack thereof impact the results of this or any other editorial content on the site. In all cases, the same rubric for evaluation is used to compare tools or services and all results reflect the outcome of the comparison without regard to whether a company is advertising on the site or not.

Hi! I'm Craig, and I'm glad you stopped by! Did you find the information useful? If so, why not stay up-to-date with my handy-dandy Useful Usability eNewsletter! It has tips, tools and other great content that will make your day easier (from a UX perspective that is). And NO, I won't spam you or share your email, because let's face it, Spammers Suck!
Join over 3.000 visitors who are receiving our newsletter and learn how to optimize your blog for search engines, find free traffic, and monetize your website.
We hate spam. Your email address will not be sold or shared with anyone else.

SIMILAR ARTICLES

16 COMMENTS

  1. Thanks so much for including UserTesting in your lineup, Craig. I just forwarded your overview to our product team, I know they always appreciate feedback – pros and cons alike!

  2. You’re welcome Stef, I hope my comments help. Thanks to you and your product team for continuing to push the capabilities of your tool. Good work!

  3. You’re welcome Bartosz. Hopefully product teams can refer back to the core of what a good usability testing experience should be, adding functionality and features on a prioritized basis. Best of luck! Cracking the mobile usability testing experience is definitely where the greatest opportunity lies, at least at this point.

  4. Hi Steve,

    Thanks for reaching out. You might have missed it, but in the article I’ve defined tools by their primary function:
    Usability Testing
    Feedback
    Click Tracking

    It appears your tool falls primarily into the Feedback category, where users provide feedback either on information architecture, static designs or related types of content.

    I’ll be happy to include it when I conduct my evaluation of Feeback tools!

  5. Thanks for an interesting read. Would be great to see a post where only mobile app usability testings tools would be covered.

  6. Thanks for your comment and I agree, sounds like a good article to do. With the growth in mobile apps there has also been growth in mobile testing tools. Right now, as I mentioned in my article there’s no seamless mobile usability testing tool that works they way researchers really need. What’s that use case look like?
    It should…
    Be able to record the testers audio/video as the tester uses the Think Out Loud method to try to accomplish tasks.
    It should be able to record screen interaction (swipes, button presses, screen movement).
    It should enable the tester to conduct the test anywhere, after all, it’s a “Mobile” device, right?
    It should be able to stream the information, or at least enable upload of the session, in near real time
    It should not require any customized, fiddling, or jail-breaking of the OS.
    It should run in the background and perhaps other than pressing a “Start Test” button at the very beginning should not require the tester to do anything else. Well, maybe click a “Stop Test” button.

    So there you go. As soon as someone develops that, and they will, we will have a truly seamless mobile usability testing tool. Until then, it’s all variations of Cludgy-ness.

    But that’s just my two cents. What do YOU think? Is there something out there that you feel does the trick?

  7. Craig, I enjoyed your article on testing. This is an area that requires many of us to try to better understand the capabilities of the large number of ‘testing’ tools. For your upcoming articles, you may benefit from a slightly different organization of the available tools at webusability.com. Thanks, Bob

  8. Thanks for the nice comment Bob. Yes, trying to determine how to categorize the numerous tools is half the battle. In my mind, the three main buckets associated with Usability testing come down to:
    Tools that help acquire user behavior and comments ala the Think Out Loud method
    Tools that capture user feedback (no matter whether it’s a live site, screen shot, etc.)
    Tools that monitor and measure activity (click tracking, quantitative data capture, etc.)

    It’s good to note you also included some design and technical measurement tools as well, such as those used for creating wireframes or mockups.

    But it’s funny, now that I think about it, that the one usability tool I forgot to mention in my article is probably the oldest and easiest to deploy: Ask someone to conduct a task and watch over their shoulder while they try to do it. Simple, quick and effective. Sometimes those tools are the ones that seem to be easiest to set up and the most effective in terms of capturing data! :-)

  9. Hi Craig,

    Have you taken a look at the uTest Usability testing offering? It would be great to see them compared to the others here.

    (Disclaimer: I just started at uTest)

  10. Thanks for your comment Andrew, and no, I was not aware uTest included usability testing tools. As a side note, the site did not come up at all for Organic results for various “usability testing” types of terms. In the future update of this article and matrix I’ll include them. However, it would be VERY helpful if your pricing was made much clearer. The app for pricing seems unusual considering the vast majority of your competitors clearly state their pricing right on their home page [just say'n].

  11. Hi Craig. Great read, i wish i would have stumbled upon it earlier!

    So I have one question. We are applying the lean startup methodology in our organization. that means we test and iterate then test again and iterate. so far we have conducted the tests whitin our weekly usability tests (using morae btw – i wouldnt buy it again but given the fact we own a license i have nothing to complain about). With the focus on testing early its often too much long of a wait for the next testing session or its really just a short test we need to conduct so not worth inviting someone over. So here’s my question: are there any tools someone can recommend to test prototypes online quick and easy, cheap and hassle free? we have a large customer base to test so we dont need participants. what we need is a good hint on which tool to use. i know most of them somehow fulfill my criteria but im looking for some subjective advice. i cant test all 14 :) cheers Mark

  12. Hi Mark, thanks for the nice comments.

    For testing prototypes I typically use a few tools. Camtasia plus a screen sharing tool like GoToMeeting or even Skype are great ways to capture user feedback on low fidelity prototypes quickly and efficiently. Also, if I need to capture more sessions I’ll send out a UserTesting invitation to my pool of testers. Once I use them though, I never use the same tester again, because they are pre-biased based on the knowledge gained from the prior testing sessions.

    Best of luck with your testing!

  13. Hi Craig,

    Thanks a lot for including UserZoom in your matrix. I just wanted to let you know that UserZoom recently launched Recorder, a new feature that records users’ audio, video (facial expressions) and screen interaction.

    Now UserZoom enables researchers to collect both quantitative (effectiveness & efficiency ratios, clickstreams, heatmaps and user satisfaction levels) and qualitative (user’s screen interaction, voice and face) data in the same platform.

Leave a Reply