Laboratory or Remote: Which type of usability testing should you use?

What comes to mind when you hear the phrase ‘Usability Testing’? Like many frequently used terms in Conversion Rate Optimisation this phrase is used to refer both to specific activities and broad categories of analysis. In this blog, we’ll look at the three different types of usability testing and answer the question “which type of usability testing should you use?”

These three types of usability testing offer different levels of insight into the issues and sources of confusion users may face when using your website. Understand the difference between these types of usability testing, and the benefits of each one, to learn which type is best for your specific requirements.

Laboratory usability testing

In a laboratory usability test, participants are studied in person while interacting with your website to monitor how they are using it. In addition, many laboratory studies also have the capabilities to gather data on eye movement, which provides additional insight into how they view your website, as well as recording the participants face, which often show some expressions which show participant’s real feelings whilst they’re using your site. Laboratory testing sessions are usually filmed for in-depth analysis and insight generation at a later date.

The advantages of laboratory usability testing

The biggest advantage of laboratory testing is its depth. Lab tests can have both qualitative and quantitative components, and factors such as eye movement, user stumbling blocks and satisfaction can be tracked and monitored with ease.

The disadvantages of laboratory usability testing

The disadvantages of laboratory usability testing are its cost and artificiality. Since test participants need to be examined in a unique environment using special equipment, it’s far more expensive than performing a remote test in which participants carry out test actions from their homes. In addition, the artificial environment of laboratory testing can sometimes produce artificial behaviour. Since participants aren’t in their natural work or living environment, they may not behave as they would at home.

Remote unmoderated usability testing

While laboratory usability testing is strictly controlled and expensive to track and monitor, remote unmoderated usability testing is inexpensive and easy for even the smallest of businesses to perform thanks to software like UserTesting or WhatUsersDo.

Remote unmoderated usability testing involves test participants completing specific, pre-defined tasks on your website, without additional guidance from a facilitator. They are completed at home on the tester’s own device and their screen and audio are recorded and sent to you to watch and analyse.

---

---

Some software providers offer the option of using their experts to analyse the videos for you but we recommend that, if you have time, you should watch the videos yourself to make sure you don’t miss out on any valuable insight.

While laboratory testing is often broad, with test participants completing big tasks and testers monitoring their interactions with your website, we find the best results from remote unmoderated testing tend to be when delving deeper into a single, highly specific area of your website. Providing testers with small, clearly defined tasks, such as ordering a certain product on your website or finding out more about a particular product or service, keeps them on topic and delivers usable insight time and again.

The advantages of remote unmoderated usability testing

The biggest advantages of remote unmoderated testing are its speed, its depth and its low cost. You can gain insights into specific aspects of your website’s performance in as little as 48 hours. They are also great if you have one specific area of your site which you’re interested in finding out more about. For example, if you’re just about to introduce a new navigation or filtering option and want to make sure it’s intuitive for real users, not just your design team.

The disadvantages of remote unmoderated usability testing

The biggest disadvantage of remote unmoderated testing is its test audience – many of the people performing remote tests through popular platforms think of the tests as “work” and often complete multiple tests each week, or they’re web professionals trying to earn a bit of extra dough doing what they know. As remote unmoderated tests are often small in scale and extremely specific, they can also feel “false” for participants. This means that their results may not be as accurate and actionable as those generated by a moderated test.

Note: WhatUsersDo have since clarified that, unlike the majority of remote unmoderated usability testing sites, they limit testers to 3 desktop tests per month to address the disadvantages of people becoming ‘professional’ usability testers.

Remote moderated usability testing

Remote moderated usability testing is a testing method that involves participants completing their usual customer journey on your website whilst sharing screens with a facilitator using tools such as join.me, GoToMeeting or Skype. In a remote moderated test, participants and test facilitators work remotely alongside each other while the test occurs. The test facilitator monitors the actions of the participant, communicating with them as and when required via chat or phone as they complete the test.

Since the test facilitator can communicate with the participant during the test, a moderated remote test allows the facilitator to ask follow-up questions for the test participant to gain additional insight into why they made the decisions they made while interacting with your website. It also allows the test participant to ask for clarification on their instructions – an ability that prevents misunderstandings from occurring. This can lead to test data that’s more reliable and actionable than data from an unmoderated test.

The advantages of remote moderated usability testing

The biggest advantage of remote moderated testing is its combination of low costs and natural results. Moderated tests can be completed by real users you’ve recruited from your website, not the anonymous, often semi-professional testers you get completing unmoderated usability tests. In addition, the large scope of a remote moderated test, combined with its natural setting, often produces highly accurate and insightful data about how visitors are using your website, not how you expect or want them to use it.

The disadvantages of remote moderated usability testing

Despite these advantages, remote moderated testing does have some downsides. It can be a time-consuming process, and the amount of traffic required to find people that match your target customer profile is often substantial. But we have found that the positives by far outweigh these downsides,

Which type of usability testing should you use?

From laboratory to remote, there’s no “best” form of usability testing – only the best form of testing for your specific purpose or goal. Each testing option offers advantages, disadvantages and unique insights into the way people use your website.

What all three types of usability testing share is one advantage: the ability to gain insight into your website’s usability and performance unavailable from other Conversion Rate Optimisation tools and techniques. These can guide future design changes, A/B tests and identify specific areas for further researching.

Is your website preventing users from becoming customers? Are users dropping out of your checkout and never returning? Usability testing can fill the gaps in your CRO process and give you the insight you need to make huge conversion improvements.

If you’d like to learn more about usability testing to increase conversion rates, and need expert help, read our ebook for more information.