Skip to main content

Insights

3 reasons why your business needs server-side experimentation

As part of my job, I speak to businesses about experimentation every day. Almost everyone associates it with a/b-testing changes to the website user interface (UI). Few people are familiar with experimentation beyond that. 

Jeff Bezos famously said the success of Amazon could be attributed to the number of  experiments they ran. Booking is an often-quoted case study about achieving exponential growth through experimentation. Bloomberg found a link between experimentation and share price performance.  

What many people don’t realise is that these success stories involve more than simply testing changes to the UI. While this type of testing is powerful and remains an important part of an experimentation strategy, the big league incorporates server-side experimentation. 

What is server-side experimentation?

Changes to the website interface are usually made in the end user’s browser – Chrome, Internet Explorer, Safari etc. This is called client-side testing, client referring to the software used to view the website. 

The scope of client-side testing is limited to what you’re able to manipulate in the browser, using front-end coding languages like HTML, CSS and JavaScript. For instance, you cannot use it to test different search algorithms, pagination algorithms, free delivery thresholds and product discounts. With server-side testing, also called full-stack experimentation, the field is wide open. Since changes are made on the server (see figure below) anything can be tested anywhere on the site as well as across channels. Think mobile, OTT (over the top), IOT (internet of things) etc.

The difference between client-side and server-side experimentation. Client-side means that changes are made in the end user’s browser. This is perfect for testing website UI changes, but presents certain technical limitations. Server-side means that experimental changes are made directly on the server. You can run experiments across the entire stack, testing new product features and functionality. Source: Optimizely

Server-side experimentation can boost your business in 3 ways:

1. Validate new features and functionality

“For product teams, one of the pitfalls of the feature development life cycle is the risk of investing time, effort, and money in building features that your customers won’t use,” says Optimizely. The only way to know whether a new feature will deliver business value is to test it on real customers. Server-side experimentation lets you find your way to a successful product or feature by discovering what doesn’t work and implementing only the wins. 

Feature flagging makes it possible to toggle bits of code on or off, so you enable or disable new product features without having to do additional code releases. This way your product team can gradually introduce new functionality – or experiment winners – to a percentage of users, ramping up slowly. You monitor key business metrics and technical performance and, if required, rollback or switch off a feature without the need to redeploy code.

Server-side experimentation is a natural extension of modern back-end development practices such as Continuous Integration and Continuous Deployment (CI/CD). This means constantly making small changes, validating each one, and rolling back or iterating as required. 

2. Advanced optimisation across channels

Some optimisation experiments cannot be achieved client-side. Typical examples are product recommendation recipes and price testing. Since the logic resides in the back-end, the variation has to be coded on the server itself. 

Page redesigns and checkout funnels are sometimes possible, but always limited, with client-side testing. This is because addressing a series of different URLs at the same time can be difficult via the front end. Any significant checkout changes, like introducing new payment methods or changing web forms, would be very challenging to do client-side.

Whereas client-side experiments target one client at a time, for example a webpage, server-side tests can be run across channels. For example, a retailer could cover web, mobile, app and POS (point of sale) with one experiment.

3. No page load delays  

Client-side tests are susceptible to flicker – a noticeable delay between the original content and changes being displayed. For example, if you were to test different product images on a PDP, you may see the old images briefly before the new ones load. It can be annoying and potentially even skew test results in severe cases. It’s one of the biggest criticisms of client-side testing. 

With full-stack experimentation this problem is completely avoided. There is no delay when changes in the variation load, because everything arrives at more or less the same time from the server. This also means you can test the same hypothesis across multiple digital channels and platforms. It will give you universal insights to make informed decisions and apply the right solutions for each channel. 

Towards a culture of experimentation

Harvard Business School Professor Stefan Thomke in his book Experimentation Works points out that “an ‘everything is a test’ mentality yields surprisingly large payoffs and competitive benefits, and may even help stock performance.” 

This mentality doesn’t come about overnight. It’s a journey that often starts with relatively simple client-side experiments, increasing in scale and complexity over time. Gradually, it becomes a way of doing things. However, ‘everything is a test’ cannot be a reality unless the limitations of client-side testing are removed.

Full-stack experimentation is credited for driving innovation at Google, Netflix, Facebook and other market leaders. These companies have hundreds or even thousands of online tests happening every day, checking almost every code change according to various product and business metrics.  

Discussions around server-side testing can be quite technical, but essentially it’s the same conversation about data-driven learning through experimentation. Professor Thomke explains: “What has changed is that we now have the tools that allow us to learn about innovation performance – new product, customers experiences, and business models – rapidly, inexpensively, and at an unprecedented scale. Large-scale experimentation means that companies can systematically kiss a lot of frogs to (hopefully) find a prince.”

Server-side experimentation is not for everyone. But if your optimisation efforts are restricted by technical limitations, or your business is doing new product development, or your development team has already embraced a CI/CD mindset – chances are, you can’t afford not to do it. 

By Stella Chatzianagnostou, Experiment Engineer, AWA digital 

Keep up-to-date

People from Facebook, FarFetch and RS Components receive our newsletter. You can too. Subscribe now.

Interested in turning experimentation and testing into an advantage for your entire business?