5 tips on mobile website usability testing

As the number of smartphones increases and the more retail businesses invest in offering a mobile version of full-screen website, usability testing becomes crucial.

In this post, I talk about our experience of carrying out a usability study for Go Outdoors on their mobile website and offer 5 tips for those who are thinking about doing this themselves.

Tip 1: Don’t do this yourself

If time and budget are an issue (when are they not?), then using a usability service makes sense, such as WhatUsersDo. Typically, these usability testing services offer:-

• A panel of usability testers – which you can segment for the required age, gender, socio-economic groups and special interest (normally an extra charge)
• Testers are provided with webcams on stands and then video their actions and comments on the mobile website
• The ability to tag up these videos with your comments
• Create video excerpts of significant moments
• Export these tags into Excel to analyse and include in your report

We used WhatUsersDo mobile website testing service which costs £100 per usability test. This cost also includes an analysis and recommendation service. There are other usability testing services that rely on you to analyse the videos yourself.

Tip 2: Write the test script with the mobile user in mind

There were two tasks we asked our testers to complete:

 

1st test script: Imagine you are at a festival, and you realise you have left all your tent pegs as well as the mallet at home.

The weather forecast is not great so you realise you need to go and buy a new set of tent pegs and mallet quickly. You think that you will be able to buy these items from Go Outdoors.

We then asked testers to Reserve and Collect the pegs and mallet at their nearest Go Outdoors store and tell us what they thought of the experience

The second, separate task was about using the mobile website in-store to help with a purchase decision.

2nd test script: Imagine this time you are already in a Go Outdoors store. You want to buy a decent heart rate monitor as part of the training you are doing for a half-marathon.

Rather than ask the sales assistant you decide to use the Go Outdoors website to provide you with more information about the range of good quality heart rate monitors. Your budget is up to £120

Now tell us which one you would buy and why, and tell us what you think of the information on the product and what we can do to improve it.

The first task was a good test of the website and we learned a lot:

• The mobile website was easily found using a brand search
• Testers overwhelmingly used search rather than the navigation
• They liked the ability to find the store using the phone location
• They found the website fast to load, and easy to navigate
• They missed having the ability to sort long lists of products by price and popularity

Unfortunately, we were not specific enough in our instructions to ensure that testers entered their details to do a Reserve and Collect. In any event, we completed this part of the test and found issues with the clarity of stock availability information on confirmation emails.

Upon reflection, the second test of using a mobile website to help choose a high-value item like a HR monitor whilst in-store was not such a realistic use of a mobile device.

We did learn a few things though:-

• The quality of product description is key – it must help visitors distinguish between products and their features, as well as explain what type of person might be this product e.g. “this HR monitor is perfect for….”
• On higher-value items the importance of customer reviews increases dramatically – if the product description can’t help, visitors hope other customers will guide them to the right product
• Complex tasks like this are best done in one single test rather than combined with another task
• Mobile usability testers are just like mobile users – short of time and often skimming – this means some won’t follow your instructions precisely – or even at all (We had to reject 20% of the testers on this task for not following the script).

Tip 3: Do more than 5 tests, preferably 10

The accepted wisdom from Jakob Nielsen is that five usability testers is sufficient to capture 85% of the usability issues.

Whilst that may be the case, mobile website are sufficiently new to retailers, usability testers and reviewers that there is a case for looking at more than five testers’ videos.

One of the issues is the speed at which testers, and their fingers, move up and down and through the screens.

The webcams used by the testers are of a quality that means going up and down the screen is often just a blur.

Having watched lots of full-screen website usability videos, my conclusion was that I needed to watch more testers’ videos to be 100% certain of the recommendations we were making to Go Outdoors.

Tip 4: Be kind and acknowledge praise

Normally, when doing full-screen usability studies we focus on where the improvements can be made. Rarely do we include plaudits or positive remarks. Clients want us to focus on what’s not working well and what they can to address these issues.

With Go Outdoors, we included much of the positive feedback from testers. I felt it was important when evaluating mobile websites to provide more context, e.g “Overall, testers thought it was really quick to load and simple to use.”

My impression, not backed up by any empirical evidence, was that mobile website testers were more likely to offer positive feedback and explain what was working well, and this should be included as part of the study.

Tip 5: Now, focus on the data Usability testing is a great way to identify way the problems visitors are likely to face – whether it’s mobile or full-screen.

Given the propensity of mobile users to use on-site search to navigate, our focus will be to work with Go Outdoors to identify those search terms that have high exit rates, and those search terms that are yielding no matches.

We believe there are substantial improvements we can make by analysing the performance of Go Outdoors on-site search data.

So in summary

#1 – save time and hassle – use a testing service
#2 – think about mobile users when writing your test script
#3 – this stuff is new – do more than 5 tests
#4 – listen out for what’s working as well as what’s not
#5 – use your web analytics data to drill down into the problems you saw

At the time of writing, Go Outdoors are implementing many of our recommendations, and we will provide an update to this post on the results from these improvements.

With the ever-increasing importance of mobile website usability testing, read our ebook below to find out how a focus on three power metrics can double your CRO success.

{{cta(’37f87ce6-80ae-4432-badc-3154b5ac4571′)}}

 

If you have performed mobile website usability testing, tell us how you got on – and what did you find out. Leave a comment below.