What types of usability testing are used in Conversion Rate Optimisation?

There’s three types of usability testing in conversion rate optimisation: Laboratory, Remote unmoderated and Remote moderated. Here we have a look at each in a bit more detail.

Laboratory usability testing

Laboratory usability testing is the most expensive method, as it requires special premises and equipment and the testers need to make a special journey to get to the lab, rather than being in their own home.

In qualitative laboratory testing, users are studied interacting with the website in person. In-lab usability testing usually works with small and specific sample sizes to better obtain qualitative data.

Typically, the user sits in front of a PC, mobile or tablet alongside a facilitator who gives the user tasks to perform. A one-way mirror is often used, so that a number of observers can watch the interaction, make notes, and ensure the activity is recorded. Sessions are usually filmed and the software used logs interaction details. This enables a number of stakeholders to observe how a customer interacts with the site first-hand.

The kind of tasks might be ‘Please buy a gift of a teddy bear, to be delivered to your niece in Gloucester on her birthday next Wednesday’.

The observers can see for themselves if participants are able to complete specified tasks successfully and any stumbling blocks they encounter along the way. It also gives a flavour of how long it takes to complete specified tasks and helps generate ideas for the changes required to improve user performance and satisfaction.

In laboratory usability tests, a facilitator gives the user tasks to perform whilst other team members observe.

Remote unmoderated usability testing 

This inexpensive usability testing method is deep as opposed to broad because it’s designed to look at small, specific areas. It is ideal if the scope of the testing is limited to a specific feature, your participants are dispersed, or you need results fast.

For unmoderated user testing, you’ll need a software application package, such as whatusersdo or UserTesting, to administer the tasks and questions. This plays the role of a session facilitator to guide the participants through the session and record what happens.

In an unmoderated remote session, the participant can complete the study in his or her own time, on their own PC, mobile or tablet whilst recording the session for later review by the usability expert.

Although there is no real-time interaction with the participant, some remote testing tools allow pre-defined follow-up questions to be built into the study, to be shown after each task or at the end of the session.

An example video you would get from UserTesting with annotations added.

Remote moderated usability testing

By contrast, remote moderated usability testing is broad as opposed to deep. It’s a way of observing users throughout the entire customer journey, although, when appropriate, it can also be used to look at specific issues in more detail.

In moderated remote testing, users and facilitators operate in the same ‘virtual’ space at the same time. The facilitator watches the usability test remotely as it happens and communicates directly with the participant via the telephone, email, chat, or a combination of methods.

To carry out remote usability testing, you need to recruit website visitors willing to take part. We use a simple software tool called Ethnio, although you can also email a subset of your database to find willing participants. In order to carry out the usability testing, you also need screen sharing and recording software such as GoToMeeting or join.me. One of the great strengths of this method is that you are speaking to real users, while the experience of using the website is fresh in their minds.

Moderated sessions allow for interaction between the participant and facilitator, enabling the facilitator to ask questions for clarification or dive deeper into issues that crop up during the session.

However, the success of this method lies in the skill of the moderator. That person should, ideally, be the same person who is analysing the website using quantitative tools such as heat maps and analytics, so that they already have a good understanding of where the problems are. In addition, the moderator needs to be a good listener, and open to genuinely finding out what the issues are, rather than leading the tester towards a conclusion.

Case Study – Bettys 

Bettys is a prestigious and popular chain of tea shops in the North of England, with a loyal and devoted following. It is famous for its cakes, which it also sells online.

During one usability session the facilitator overheard the tester’s friend in the background saying “I didn’t know Bettys sold cakes”. This was strange, as the website clearly showed a basket symbol at the top and a large banner offering free delivery.

Rather than ignoring this chance remark, the researcher explored it in more depth. As a result, a new web page was developed which just added ten words to the home page. “Shop online now – direct from our bakery to your door.” This was tested and led to a double-digit improvement in sales.

Pros and cons of the different website usability testing methods

Laboratory Usability Testing

Pros

• Can include eye tracking which adds an extra dimension to data and analysis
• Tests can include both qualitative & quantitative components

Cons

• It’s expensive
• Participants aren’t in their natural environment

Remote unmoderated usability testing

Pros

• Cost effective
• Timely – results usually arrive within 48 hours
• Can observe participants in their natural environment
• Can effectively target specific areas of the site, such as search or navigation features.
• A useful and easy method of doing competitor analysis, by asking panellists to visit competitor websites
• Some companies also offer an analysis service, although we recommend doing that yourself so that you really understand the user’s experience and issues
• Some platforms integrate with recruitment tools like Ethnio, enabling the research team to enjoy the benefits of outsourced usability testing without the drawbacks of using a panel with no real connection to the site.

Cons

• As a default, uses a panel of people who may not accurately reflect your visitors
• Varying degrees of ability to select visitors matching specific criteria
• Responses often feel “canned” as the panellists may be doing these reviews semi-professionally. Feedback becomes opinionated judgements rather than reflections on an authentic user experience.
• Have to pre-set tasks which may seem ‘false’
• No opportunity to get a ‘flavour’ of your visitors
• No ability to steer testers back oncourse if they go off on a tangent (although a bad result can be rejected)

Remote moderated usability testing

Pros

• Cost effective
• Can be run with your real visitors, not a panel
• Can watch them use the website naturally rather than using constructed scenarios
• Participants are observed in their natural environment
• A moderator or facilitator keeps sessions on track
• There is the potential to ask more general questions about, for example, needs and buying behaviour which can help to get in the user’s mind and construct personas

Cons

• Time consuming
• Needs lots of traffic given the response rate and selection process of users to participate
• Some sessions may not work out for technical reasons

To find out more about usability testing, read our full usability testing ebook.