5 Ways To Generate CRO Test Ideas That Make A Difference

---

Not every AB experiment is built equal and this can be traced to the quality of the CRO test ideas powering them.

CRO can sometimes become kind of shallow. And when we say shallow, we’re talking about the CTA optimizations, the button placements, or the navigation menu refinement. Whilst those might be the right things to focus on at the time, the difference between sustainable results and instant results is working to a strategy – instead of picking from a menu of tactics. 

The true power of CRO lies in offering you the opportunity to learn more about your customers than you ever thought possible by conducting experiments that help to discover and refine your sales conversation.

So, how do you move beyond the typical, shallower A/B testing ideas and generate ideas that have a better chance at making a real difference? Here are a few of our favourite methods.

Start by finding good data sources

1. Data Sources

Quantitative

Quantitative data, which is all about numbers, will tell you what is happening. A good source is your web analytics platform e.g. Google Analytics. For example, the ‘Shopping Behaviour’ report in Google Analytics gives you a view of how users progress through the shopping journey. You can see what proportion of sessions drop off at which point. . This is quantitative data telling you what is happening.

Qualitative 

Qualitative data meanwhile helps you understand why certain things are happening, for example you can start forming informed theories about why 95% of mobile sessions don’t add to cart after seeing products. . Qualitative research can provide you with a wealth of valuable information about your customers, their reasons for buying, wants and pain points. Without these theories, it is impossible to come up with hypotheses to test.

2. Click Visualisation

When your visitors use your site, they leave behind traces that may be used to map their click behaviour. These data can be visualised using heat mapping tools which show where users clicked – and, importantly, didn’t click.

Heatmaps might reveal surprises, like users not clicking where you want them to click or being distracted and navigating a less profitable path. Even if there are no surprises, an understanding of behavioural patterns is an important piece of the customer knowledge puzzle.

Sometimes you’ll immediately spot an improvement opportunity in an area of your website, other times it will help you learn more about your visitors’ goals, needs and intentions.

Here’s an example of what a heatmap looks like:

3. Customer Facing Team Members

Team members from internal departments, particularly those who deal with your customers on a regular basis, are often the source of the greatest ideas. 

Internal sources, such as customer service teams, can provide you with extremely valuable information. They often see the types of questions people ask, what problems they express and whether or not you can resolve their issues quickly enough. 

Sales teams may offer previously overlooked insights into the exact wants and needs of your customers, and may even be able to provide solutions as to how to best address those needs.

How to define the problem

Ideas are the lifeblood of CRO, but ideas should target specific opportunities, linked to bigger goals. Your optimisation goals should be linked directly to the higher level organisational goals.

Before coming up with ideas, understand which goal you are targeting with what opportunity and what customer problem you’re trying to solve.

To help narrow down the definition of your problem, there are 2 methodologies in particular that can be utilized. They are:

Before coming up with ideas, understand which goal you are targeting with what opportunity and what customer problem you’re trying to solve.

To help narrow down the definition of your problem, there are 2 methodologies in particular that can be utilized. They are:

1. The 5 WHYs technique 

In the 1930s, Japanese industrialist, inventor, and founder of Toyota Industries Sakichi Toyota invented the 5 Whys technique. It became popular in the 1970s, and Toyota continues to employ it to find solutions today.

Toyota’s philosophy is to “go and see”. This means that Toyota’s decision-making is informed by in-depth knowledge of what’s really going on on the shop floor, rather than what someone in a boardroom believes may be taking place.

The 5 Whys approach is consistent with this practice, and it’s most successful when the responses come from individuals who have direct knowledge of the process or issue.

It’s quite straightforward: you ask “Why?” five times when a problem arises to get to the source of the issue. Explain what you’re observing or experiencing, then ask why it’s the case. Continue asking questions until you’ve reached a probable source of the problem. You may realize that more data is required to give an informed response as you ask the next question. This might be a push to consider another data source or perhaps revisit an old one, with a fresh perspective.

Then, after seeing an apparent counter-measure, you must implement it in order to avoid its recurrence.

The 5 Whys focuses on “counter-measures” rather than “solutions.” A countermeasure is a technique or set of strategies that aims to prevent the issue from recurring, whereas a solution may simply deal with the symptom. 

As such, counter-measures are much more robust, and will more likely prevent the problem from recurring, which makes it perfect for both generating proactive CRO test ideas, and discovering the very nucleus of your problem.

Let’s make this practical:

Here’s a framework you can use, just swap out the details:

5 Whys Template:

1. State the specific problem (e.g. high cart abandonment rate)
2. Ask “Why is this happening?” (e.g. checkout process is too long)
3. Ask “Why is that?” (e.g. customers have to fill in too many fields)
4. Ask “Why is that the case?” (e.g. we require a phone number and address before checkout)
5. Ask “Why do we do that?” (e.g. to reduce fraud risks)
6. Root cause identified (fraud prevention measures are frustrating customers)
7. Brainstorm countermeasures (e.g. reduce fields, offer guest checkout, highlight security)
8. Test the most promising countermeasure (e.g. A/B test guest checkout)
9. Monitor results and iterate until the problem resolved

2. How Might We questions (HMW)

The How Might We template was originally designed by Procter & Gamble in the 1970s and implemented by IDEO. The approach has grown in popularity among design thinkers and is utilized by design teams across the world.

At the conclusion of an experiment, a team should have multiple ideas to treat a problem, with the intention of testing those potential solutions.

Because How Might We (HMW) questions are open-ended, they can generate a plethora of unique, refreshing, and creative ideas and provide insight into what is causing a particular problem. Some examples of How Might We questions include:

• How might we do a better job of showcasing breadth of range
• How might we articulate the value of premium delivery
• How might we best display a large array of colour swatches on a mobile screen

HMW questions foster creative ideation by removing constraints that limit imagination. Reframing issues from different angles stimulates innovative solutions. The open-ended nature aligns the process to the customer’s perspective – how might we solve their problem? This client focus often reveals non-obvious possibilities.

To leverage HMW questions, frame problems as open questions like “How might we improve our checkout process?” Generate wild ideas without judging initial quality. Reframe the question from multiple angles to spur new solutions. Evaluate the ideas and test the most promising ones based on potential business impact. Continue iterating until the issue is resolved.

Here’s a framework you can use:

How Might We Framework:

1. How might we [make it easier for] [target audience] to [complete a key action]?

2. How might we [help] [target audience] [achieve goal] through [touchpoint]?

3. How might we [remove friction] during [touchpoint] for [target audience]?

4. How might we [improve metric] by [enhancing] [touchpoint] for [target audience]?

Example HMW for CRO:

1. How might we make it easier for mobile users to complete checkout?

2. How might we help customers feel more secure entering payment info?

3. How might we remove friction during checkout for time-pressed users?

4. How might we reduce cart abandonment by enhancing the checkout flow?

Together, these techniques mitigate two key idea-generation pitfalls. First is solutionizing – jumping to solutions without properly defining the problem. Second is design fixation – relying on existing paradigms versus creative thinking. The 5 Whys and HMW questions disrupt these tendencies.

How to generate successful CRO test ideas

1. Make use of different data sources

Triangulate a number of data sources to form a richer picture of the world of your customer, using this to come up with theories about how to improve things. Use what the numbers are telling you in your quantitative data and then apply what the qualitative data is telling you about why certain things might be occurring. 

Bear in mind the difference between what people say and what people do. Those are not the same, and is a primary reason for testing in the first place. Use what people say to inform test hypotheses, and always drill a level or two deeper to get to the core user problem. 

You might start top-down, using the problem as a starting point and then substantiating it with the data, or bottom-up, deciding to take the lead and analyse the product page to discover how it could be improved to capture more conversions.

Here’s a step-by-step guide to follow:

1. Gather quantitative data like analytics, surveys, and heatmaps to identify underperforming areas or pain points. Look for correlations, anomalies, and changes over time.

2. Collect qualitative data through user interviews, session recordings, and support tickets to understand why problems occur. Listen for frustrations, confusion, and desires.

3. Combine insights from both sources to form hypotheses about the root causes of issues. Leverage qualitative data to explain the “why” behind quantitative trends.

4. Be aware of disconnects between what people say and do. Use qualitative data to inform ideas, while relying on quantitative data to validate effectiveness.

5. Take both top-down (start with goals, find data to support) and bottom-up (analyze the product, find improvement opportunities) approaches.

2. Use ideation sessions to make it a communal activity 

Ideation sessions are effective for two reasons. Firstly, they are collaborative efforts that can help to excite the whole organization and spread the CRO culture. Secondly, it’s a good way to tap into multiple perspectives of the same problem. Who’s to say my idea is the best one? Quantity breeds quality, and that’s the key objective of an ideation session. 

Ideation sessions are used at their maximum effectiveness when they are focused on a specific objective. This way it prevents a large group of people from muddying waters with too many generalized ideas. Start with a solid definition of the problem you are tackling, then ask participants to generate as many ideas as possible in a short time frame. Wacky ideas often prompt breakthroughs, so encourage them to think wild. .

Plus because everyone is contributing, everyone also has increased buy-in into the experiment – heightening the chances of overall programme success.

Here’s a guide you can follow for CRO ideation sessions:

1. Define a specific problem statement or opportunity area to focus the session.

2. Gather cross-functional team members together(strategists, designers, developers, project managers, etc)

3. Set a timer for 10-15 minutes and challenge the group to generate as many ideas as possible.

4. Encourage wild, creative ideas to spur new connections and perspectives.

5. Capture all ideas and refrain from judging initially. Quantity breeds quality.

6. Review, shortlist, and combine ideas afterward to identify the strongest options to test.

3. Formulate an idea generation matrix

Start with high level organisational strategy and goals at the top. Break those down into lower-tier optimisation strategies, followed by specific concepts that can be tested in each area. The last step is to generate individual hypotheses for each concept.

Combining all of the tests you can run into one big matrix then makes it easy to remember what variables may be tested and what kinds of tests you have at your disposal.

This allows you to quickly generate tests and discover testing categories that you may have overlooked.

4. Use a value/effort prioritization matrix

A simple Value/Effort prioritization grid is shown below. The top left quadrant contains ideas with the highest return on investment. These are tests that cost less to execute but have the highest potential impact since they target segments or areas of the site with the strongest links to revenue generation.

When thinking about the value in a value/effort prioritization matrix, there are two important perspectives to consider: 

1. Value to your business
2. Value to your users

After ideation sessions and data analysis, you should have a lot of thoughts about what test you can run. 

5. Submit ideas

Your final option to generate test ideas is to distribute an Ideation form throughout your business.

These forms might include sections that pertain to which website, or particular website area, the idea relates to, whether it replicates on multiple devices or just one, as well as a main section where the problem the idea looks to solve can be described.

The description section is arguably the most important section, and should ideally include data or evidence that the problem really exists. It’s good practice to guide respondents to think from a problem-mindset, rather than simply throwing ideas into a pot. 

Frequently Asked Questions About How To Generate CRO Test Ideas.

1. Question: How do you come up with ideas for CRO tests?

Answer: Look at your analytics to identify underperforming pages and conversion bottlenecks. Survey users to pinpoint areas of friction. Analyze competitors for features lacking on your site. Brainstorm with team members to get different perspectives.

2. Question: What kinds of page elements can you test?

Answer: Headlines, copy, layouts, images, calls-to-action, forms, navigation, content, trust factors like security seals, etc. Nearly every component of a page is testable.

3. Question: Should you test radical redesigns or incremental changes?

Answer: Start with smaller incremental tests rather than dramatic makeovers. Making focused tweaks often yields bigger results than overhauling everything at once.

4. Question: How much lift should you aim for with CRO tests?

Answer: Aim for at least a 10% lift in your target metric. If results are under 5%, the variation may not be worth implementing due to potential test error.

5. Question: Can you run the same test on multiple pages?

Answer: Absolutely. Once you validate an improvement, roll it out across relevant pages to maximize impact.

6. Question: How often should you be testing?

Answer: Top companies like Booking, Netflix, Google, and Microsoft run ongoing testing programs with multiple simultaneous tests instead of one-off projects. Continual testing builds incremental gains over time.

In summary

Coming up with CRO test ideas may seem like a daunting task at first, but hopefully, after reading this you should have a good idea of where to start and some clever ways you can generate ideas that can make an impact.

Using a combination of the above methods will provide you with a wealth of ideas you can begin to prioritise and plan into your CRO strategy.

For help generating innovative CRO test ideas for your business, why not get in touch with our experts? Trusted by the likes of Canon, Interflora and Nike, you can be sure that our advice will help your business to achieve next level growth.