Financial services CRO and Experimentation strategies for growth

With many challenger banks being created after the last global financial crisis, the “Big Five” of the financial services industry no longer dominate the market. Banks such as Monzo, Starling Bank and Revolut are having a significant impact.

The top 8 challenger banks of 2019 include:

• N26
• Revolut
• Starling Bank
• Monese
• Cashplus
• Monzo
• Tandem
• Atom Bank

There may be more to join this list throughout 2020, with the likes of Goldman’s ‘Marcus’, who have higher interest rates on account. There’s others that may have hoped to be on the list, but have struggled to maintain growth and so have shut down, like Bo, RBS’s failed challenger who closed in May 2020, after just 6 months.

Challenger Bank – Revolut

As N26 have now decided to exit the UK following Brexit, we’ll focus on the second challenger bank on the list, Revolut. They have on average just over 4 million total visits each month (source: SimilarWeb), with over 12 million customers in less than 5 years (Financial Times).

We’ve taken a look at Revolut’s homepage with our AI-powered perception mapping software. The lighter areas are what engaged visitors see within the first 3 seconds when they visit their site:

Our analysis shows that their value proposition – ‘getting more from your money’ is getting eyeballs, but is it resulting in lots of people signing  up for a new account?. Certainly Revolut, according to FinanceMagnets appear to be doing well, claiming to be signing up 2,000 new customers per month.

So how can you grow your challenger bank or financial services business like Revolut?

Perhaps you’re getting large amounts of visitors to your website, but you’re just not converting them?

Or are you wasting time and hindering your growth by spending time debating next steps and the direction for your website?

We may just have the answer… CRO and Experimentation.

CRO and Experimentation

First things first. You may have heard about CRO (Conversion Rate Optimisation) but Experimentation could be a new thing for you. What’s the difference? Let me explain.

In simple terms, experimentation is a mindset, and CRO is an application of this mindset.

Both involve running controlled experiments to identify which variation gives the best outcome, but CRO is often limited to answering conversion questions, e.g. Did page layout A or page layout B result in more account openings?.

Experimentation can help you to answer broader business questions. For example, how can I reduce the volume of unnecessary calls and emails to the contact centre? Reducing the number of emails and calls you are having to answer will save you money but it won’t necessarily increase your conversion rate, e.g. the number of new accounts  opened.

Experimentation can be used to help a number of business units – HR, Finance, Operations as well as marketing and product development.

Business Question	Suggested success metrics
How can I increase the number of people who start and finish the application process?	Number of applications started, completion rate, application approval rate
How can I reduce my Cost Per Acquisition?	CPA, landing page conversion rate, volume of applications
What is the impact of a new feature e.g. finance calculator?	Conversion rate, usage of feature, quality of applications
How can I test the demand and viability of a new product before it is launched?	Visits to new products page, number of leads generated
What is the impact of varying interest rate on new applications?	Conversion rate, volume of applications, ratio of approved to total applications
How can I reduce the number of calls and emails to the contact centre?	Volume of queries, cost savings, visitor satisfaction
How can I increase the quality of applicants applying for jobs?	Ratio of suitable applicants to total applicants

We recently ran an experiment where we tested reducing the product range – using the hypothesis that too much choice can paralyse customers. It increased revenue by 6% , but it also allowed the business to simplify logistics with less SKUs to manage, have greater purchasing power with suppliers and reduce complexity.

Experimentation can also help you to develop new products or services in a  way that reduces the chance of an embarrassing and costly failure. Since you’re testing your new service – and it might be just a mock-up of a new service page – with paying customers you are reducing the risk of launching that no one wants. As a finance person, you know the importance of managing risk.

CRO and Experimentation have a number of benefits for both you, your customers and your business. For example,

1. Increase the speed that you launch new product, services or features
2. Improve user experience to get more people to complete application forms, manage their accounts and find out about more about other service your offer
3. Reduce costly calls and emails to contact centres and get users to the information they need faster
4. Increase the ROI of your marketing spend and reduce the risk of wasting advertising expenditure
5. Dominate your market – the more you test the more you learn allowing you to outexperiment your competitors.
6. Create an experimentation mindset in your business, moving away from deploy-and-pray to step-by-progress progress based on real results

So to grow your financial services business you need to adopt and ‘experimentation mindset, but most important of all, is a structured approach.

According to Econsultancy almost two-thirds (63%) of companies surveyed in their most recent Conversion Optimisation Report, don’t have a structured approach to optimisation, despite acknowledging the importance of this to their success.

A Structured Approach to CRO and Experimentation

One of our core values is ‘evidence over opinion’. Our adherence to facts and data – rather than guesses, hunches and well-meaning opinions – is fundamental to the successes and sales uplifts we have delivered our clients.

Our unique approach to CRO and Experimentation puts Research and Analysis at the nexus between the user and the business.

Research and Analysis

Our aim is to get inside the head of your customers and website users. This will help you to understand the ‘sales conversation’ going on in the head of your users. Focusing  on the webpage, rather than this conversation, is like looking at conversion through the wrong end of the telescope.

Our research is a variety of approaches. Simply put, the more insight you have about your users the more powerful your hypothesis will be. Insight-driven hypotheses are a key component of a high-performing CRO and Experimentation programme

Research is divided into qualitative and quantitative, behavioural (what people do) and attitudinal (what people say).

Some of the research methods below require you to add tools to your website. Some tools like Qualaroo which is a survey tool can be added to your website via a tag management system, like Google Tag Manager.

A/B testing and personalisation platforms – like Optimizely, Convert, VWO, Monetate, Qubit and Adobe’s Test and Target – need to be added directly into the website code.

This body of knowledge is supplemented by the valuable research you may already have done on your customers, e.g. personas, market research, interviews etc. It is also useful to review your marketing plan and development queue in light of what you discover about your customers. For example, it may be that you are planning to roll out a new feature – and this will take dev team months of hard work to build and launch – but is there evidence that your users will find this new feature? Is there a way of testing this before the build stage starts?

Often, CRO and  Experimentation programmes help you to identify ways to avoid making investments in new features that no-one wants. Testing allows you to check your assumptions before spending huge sums on development.

Triage / Prioritisation

The research and analysis phase will generate dozens of possible hypotheses you will want to test. Triage is the name we give to our systemic approach to ranking and prioritizing these testing opportunities.

It’s important to get this prioritisation right. Building and launching tests have an opportunity cost. This means you want to run tests that have a good chance of a positive outcome.

There are a number of frameworks for prioritizing which tests to run firsts.

Some include factors like Potential, Importance and Ease, others use Evidence and Impact. See below for a framework we regularly use.

The most  important factor in the success of your programme  is that you have – and use – a prioritization framework. Without one you might be at the mercy of ‘those  who shout the loudest’.

We sometimes call  this approach the HIPPO way. HIPPO stands for “Highest Paid  Person’s Opinion”. Letting opinion determine which test to run is to put it mildly, ‘sub-optimal’.

Your CRO and Experimentation Roadmap

After completing the Triage (Priorisation) step, you are now in a position to create your roadmap; which hypotheses you want to test first and in which part of the website.

There are a number of ‘slots’ on your website – key pages in the online conversion journey. An effective roadmap ensures that you are running tests on this simultaneously. This approach gives your programme high-velocity. As soon as you have declared a test in one ‘slot’ you need to be ready to run another test in that now empty slot.

However, until you have run a test you’re not sure of the results. That’s why you’re testing after all. If you get a positive outcome it might be worth  running further tests to see if you can ‘beat’ that initial result. Conversely,  if you don’t get a positive outcome you should proceed to the next test in your plan.

Your test results will help to steer the direction of your programme, and  this means you are continuously updating your roadmap.

Creative Work and Testing

Testing is not delayed until the Optimisation Plan is completed. We start the testing cycle as early as possible in the programme, in parallel with research and analysis.

The first stage of this process is to create a wireframe, incorporating revised copy, if necessary. This wireframe is then reviewed and amended based on comments from customer-facing colleagues, AWA’s whole optimisation team and results of eye-tracking studies.

Once the wireframe is approved, it is developed on your testing platform in line with the brand tone and design style of your site. The test, or experiment, is then QA’d and launched.

Finally, an important part of the optimisation process, is to, where possible, hard code the winning variation into the native website code. Some testing platforms, such as Google Optimize, allow you to deploy your winning variations at a click of a button, thereby reducing the demand on developer resources.

Measuring your CRO and Experimentation Programme

The three most important numbers to monitor are

1. Velocity – how many tests you have launched per month.

This is the only number you have direct control over. According to Econsultancy, only 10% of those surveyed are running  more than 50 tests per year, that’s just over 4 per month. There is a positive correlation between test velocity and performance.

2. Win rate – what proportion of your tests result in a positive outcome.

A win rate of 90% indicates you are not being particularly innovative in your testing. A win rate of 5% may point to you needing better quality research in order to develop more relevant hypotheses.

3. Effect – this is the uplift you observe in your chosen metric

Your chosen metric might be Conversion Rate or Revenue Per User, but you should ensure you are capturing this metric, along with a range of user-based metrics that will help you understand the performance of the main metric. It’s important to decide and agree on stopping rules in advance; this will form part of your experimentation manifesto. Declare a test too early and you could easily have a false positive.

Case Study – Capital One

Every industry, including financial services, is making a shift towards CRO and Experimentation, including Amazon in retail, Booking.com in travel and the Wall Street Journal in media.

And the gap is opening up between those companies who are using CRO and Experimentation to grow their business. See the difference below in the share prices of those experimenting and those who aren’t.

Since 1988 Capital One has built up a reputation for testing and experimentation. Their CEO and cofounder, Rich Fairbank, is an  avid proponent of this approach. He says what  he likes about the credit card industry is its “ability to turn a business into a scientific laboratory where every decision about product design, marketing, channels of communication, credit lines, customer selection, collection policies and cross-selling decisions could be subjected to systematic testing using thousands of experiments.” (Source: Harvard Business Review)

Fairbank cites this ‘information-based strategy as one of the key reasons for being the US’s fifth largest provider of credit cards.

Need some help?

AWA is an experienced CRO and experimentation agency, with a track record of  helping financial services innovate and grow their business.

If you’d like to find out more about how we could help you with your CRO or experimentation programme, contact us today.