How To Generate CRO Test Ideas That Make A Difference

---

CRO can sometimes become kind of shallow. And when we say shallow, we’re talking about the CTA optimizations, the button placements, or the navigation menu refinement. Whilst those might be the right thing to focus on at the time, the difference between sustainable results and instant results is working to a strategy – instead of picking from a menu of tactics.

We prefer to consider CRO as if you’re conducting a sales conversation: Your website or app acts as a shop assistant in your physical store, able to offer immediate assistance and answers to enquiries from customers. 

If a customer has questions about things like delivery, pricing, trust and a range of themes that might be unique to you, the likelihood of closing a transaction will be determined by how well your site addresses their questions and concerns.

By taking this approach, CRO will make a long-lasting meaningful difference to your business outcomes, unlike ‘best practice’ conversion gimmicks which will inevitably run out of steam quickly. 

The true power of CRO lies in offering you the opportunity to learn more about your customers than you ever thought possible by conducting experiments that help to discover and refine your sales conversation.

So, how do you move beyond the typical, shallower A/B testing ideas and generate ideas that have a better chance at making a real difference? Here are a few of our favourite methods.

Start by finding good data sources

1. Data Sources

Quantitative

Quantitative data, which is all about numbers, will tell you what is happening. A good source is your web analytics platform e.g. Google Analytics. For example, the ‘Shopping Behaviour’ report in Google Analytics gives you a view of how users progress through the shopping journey. You can see what proportion of sessions drop off at which point. . This is quantitative data telling you what is happening.

Qualitative 

Qualitative data meanwhile helps you understand why certain things are happening, for example you can start forming informed theories about why 95% of mobile sessions don’t add to cart after seeing products. . Qualitative research can provide you with a wealth of valuable information about your customers, their reasons for buying, wants and pain points. Without these theories, it is impossible to come up with hypotheses to test.

2. Click Visualisation

When your visitors use your site, they leave behind traces that may be used to map their click behaviour. These data can be visualised using heat mapping tools which show where users clicked – and, importantly, didn’t click.

Heatmaps might reveal surprises, like users not clicking where you want them to click or being distracted and navigating a less profitable path. Even if there are no surprises, an understanding of behavioural patterns is an important piece of the customer knowledge puzzle.

Sometimes you’ll immediately spot an improvement opportunity in an area of your website, other times it will help you learn more about your visitors’ goals, needs and intentions.

3. Customer Facing Team Members

Team members from internal departments, particularly those who deal with your customers on a regular basis, are often the source of the greatest ideas. 

Internal sources, such as customer service teams, can provide you with extremely valuable information. They often see the types of questions people ask, what problems they express and whether or not you can resolve their issues quickly enough. 

Sales teams may offer previously overlooked insights into the exact wants and needs of your customers, and may even be able to provide solutions as to how to best address those needs.

How to define the problem

Ideas are the lifeblood of CRO, but ideas should target specific opportunities, linked to bigger goals. Your optimisation goals should be linked directly to the higher level organisational goals.

Before coming up with ideas, understand which goal you are targeting with what opportunity and what customer problem you’re trying to solve.

To help narrow down the definition of your problem, there are 2 methodologies in particular that can be utilized. They are:

Before coming up with ideas, understand which goal you are targeting with what opportunity and what customer problem you’re trying to solve.

To help narrow down the definition of your problem, there are 2 methodologies in particular that can be utilized. They are:

1. The 5 WHYs technique 

In the 1930s, Japanese industrialist, inventor, and founder of Toyota Industries Sakichi Toyota invented the 5 Whys technique. It became popular in the 1970s, and Toyota continues to employ it to find solutions today.

Toyota’s philosophy is to “go and see”. This means that Toyota’s decision-making is informed by in-depth knowledge of what’s really going on on the shop floor, rather than what someone in a boardroom believes may be taking place.

The 5 Whys approach is consistent with this practice, and it’s most successful when the responses come from individuals who have direct knowledge of the process or issue.

It’s quite straightforward: you ask “Why?” five times when a problem arises to get to the source of the issue. Explain what you’re observing or experiencing, then ask why it’s the case. Continue asking questions until you’ve reached a probable source of the problem. You may realize that more data is required to give an informed response as you ask the next question. This might be a push to consider another data source or perhaps revisit an old one, with a fresh perspective.

Then, after seeing an apparent counter-measure, you must implement it in order to avoid its recurrence.

The 5 Whys focuses on “counter-measures” rather than “solutions.” A countermeasure is a technique or set of strategies that aims to prevent the issue from recurring, whereas a solution may simply deal with the symptom. 

As such, counter-measures are much more robust, and will more likely prevent the problem from recurring, which makes it perfect for both generating proactive CRO test ideas, and discovering the very nucleus of your problem.

2. How Might We questions (HMW)

The How Might We template was originally designed by Procter & Gamble in the 1970s and implemented by IDEO. The approach has grown in popularity among design thinkers and is utilized by design teams across the world.

At the conclusion of an experiment, a team should have multiple ideas to treat a problem, with the intention of testing those potential solutions.

Because How Might We (HMW) questions are open-ended, they can generate a plethora of unique, refreshing and creative ideas and provide insight into what is causing a particular problem. Some examples of How Might We questions include:

• How might we do a better job of showcasing breadth of range
• How might we articulate the value of premium delivery
• How might we best display a large array of colour swatches on a mobile screen

How to generate successful CRO test ideas

1. Make use of different data sources

Triangulate a number of data sources to form a richer picture of the world of your customer, using this to come up with theories about how to improve things. Use what the numbers are telling you in your quantitative data and then apply what the qualitative data is telling you about why certain things might be occurring. 

Bear in mind the difference between what people say and what people do. Those are not the same, and is a primary reason for testing in the first place. Use what people say to inform test hypotheses, and always drill a level or two deeper to get to the core user problem. 

You might start top-down, using the problem as a starting point and then substantiating it with the data, or bottom-up, deciding to take the lead and analyse the product page to discover how it could be improved to capture more conversions.

2. Use ideation sessions to make it a communal activity 

Ideation sessions are effective for two reasons. Firstly, they are collaborative efforts that can help to excite the whole organization and spread the CRO culture. Secondly, it’s a good way to tap into multiple perspectives of the same problem. Who’s to say my idea is the best one? Quantity breeds quality, and that’s the key objective of an ideation session. 

Ideation sessions are used at their maximum effectiveness when they are focused on a specific objective. This way it prevents a large group of people from muddying waters with too many generalized ideas. Start with a solid definition of the problem you are tackling, then ask participants to generate as many ideas as possible in a short time frame. Wacky ideas often prompt breakthroughs, so encourage them to think wild. .

Plus because everyone is contributing, everyone also has increased buy-in into the experiment – heightening the chances of overall programme success.

3. Formulate an idea generation matrix

Start with high level organisational strategy and goals at the top. Break those down into lower-tier optimisation strategies, followed by specific concepts that can be tested in each area. The last step is to generate individual hypotheses for each concept.

Combining all of the tests you can run into one big matrix then makes it easy to remember what variables may be tested and what kinds of tests you have at your disposal.

This allows you to quickly generate tests and discover testing categories that you may have overlooked.

4. Use a value/effort prioritization matrix

A simple Value/Effort prioritization grid is shown below. The top left quadrant contains ideas with the highest return on investment. These are tests that cost less to execute but have the highest potential impact since they target segments or areas of the site with the strongest links to revenue generation.

When thinking about the value in a value/effort prioritization matrix, there are two important perspectives to consider: 

1. Value to your business
2. Value to your users

After ideation sessions and data analysis, you should have a lot of thoughts about what test you can run. 

5. Submit ideas

Your final option to generate test ideas is to distribute an Ideation form throughout your business.

These forms might include sections that pertain to which website, or particular website area, the idea relates to, whether it replicates on multiple devices or just one, as well as a main section where the problem the idea looks to solve can be described.

The description section is arguably the most important section, and should ideally include data or evidence that the problem really exists. It’s good practice to guide respondents to think from a problem-mindset, rather than simply throwing ideas into a pot. 

In summary

Coming up with CRO test ideas may seem like a daunting task at first, but hopefully, after reading this you should have a good idea of where to start and some clever ways you can generate ideas that can make an impact.

Using a combination of the above methods will provide you with a wealth of ideas you can begin to prioritise and plan into your CRO strategy.

For help generating innovative CRO test ideas for your business, why not get in touch with our experts? Trusted by the likes of Canon, Interflora and Nike, you can be sure that our advice will help your business to achieve next level growth.