Skip to main content

Our new CRO team member: ChatGPT

That moment arrived. We are now talking to computers in natural language.

This is BIG.

I’m old enough to remember the start of the internet. I was at university; we were early adopters who understood its significance. This feels similar.

“Helpful assistants that talk to you, answer questions, and give advice…” That’s how OpenAI CEO Sam Altman describes ChatGPT in his announcement (see below). Within days, over 1 million people were using it.

Not perfect. But pretty damn good.

From what I see on Twitter, many people are still having fun throwing simple questions at it. Like this one:

Ok, it’s not perfect. Pretty damn good though.

It waffled its way through a definition of statistical significance, which more or less made sense, but I don’t think it’s ready to assist Stats101 students with assignments. I submitted feedback with references, which is the objective of the current “free research preview”. At some point, there’ll be a charge. Well, I’m ready and willing to pay.

ChatGPT in Conversion Rate Optimisation

Of course I asked my new AI-assistant, now always open in a Chrome tab, to suggest a title for this post. “ChatGPT: The Future of Conversion Rate Optimisation“. Ambitious, I like it. But no.

It’s only been a week, so we’re seeing mere glimpses of potential. Here are three examples of how we’ve been experimenting with it.

Copywriting

VWO hosted a “friendly competition” between professional human copywriters and their bot counterparts. You already know where this is going… AI-generated copy performed at least as well as humans, and that’s being generous.

Out of 7 concluded A/B-tests:

  • AI won 3
  • Humans won 1
  • 3 were tied

This was a while ago, before the ChatGPT revolution. Not only that, but every attempt is an opportunity for the AI to improve itself. That is what it does, by definition

We’ve used it to help brainstorm copy ideas when creating wireframes. To be clear, we treat it as a member of a team – not a replacement. It’s one voice in ideation.

Does it replace copywriters? I don’t believe it does. If anything, it probably raises the bar and helps to separate the cream from the rest. If AI can do an average job, “average” just isn’t good enough anymore. Put the right tool in the right hands though, magic happens.

Text analysis

If you’ve worked with us, you know that Voice of Customer data is central in our methodology. We want to get into the hearts and minds of your customers. Understand what made them buy, not buy, buy from a competitor or do nothing.

We’re big fans of customer interviews, open-text surveys, helpline emails and customer reviews. That’s a lot of text to analyse and we use different tools for that, including actually reading through much of it!

My personal favourite is using a script I wrote in R, a popular statistical programming language. It provides a good summary, word clouds, top likes and dislikes, sentiment analysis etc.

This weekend I tested that script against ChatGPT. It was close enough for me to be impressed. For now I’ll stick to my R script, but I see the potential.

Writing and debugging code

Some coders have been using ChatGPT for debugging and to ask questions they’d normally post on dev forums. Well-known CRO Lorenzo Carreri (@lorenzocarreri) made it build him an entire landing page.

Marketers and analysts often use languages like SQL, R and Python. We may use these for example to do advanced analysis, calculate statistical significance of A/B-tests etc.

I tested its ability to write Python code to run a simulation of A/B-test results. With minor edits, it worked perfectly. Five minutes max, if that.

The “threat” of AI

How worried should we be about this? Are computers finally coming after your job?

Perhaps we should take advice from the man behind ChatGPT. Here’s Sam Altman again:

It’s there to help you. Assign it tasks so you have more time to do what humans should be doing. What are those?

Well, who better to ask than ChatGPT itself:

  • Develop new ideas and innovations
  • Communicate effectively
  • Make ethical decisions
  • Strategic thinking
  • Understand cultural differences

I haven’t been this excited and buzzing about new possibilities since that sound of the dial-up modem back in the 90s.

How are you using ChatGPT? What potential do you see? Let me know.

Is your CRO programme delivering the impact you hoped for?

Benchmark your CRO now for immediate, free report packed with ACTIONABLE insights you and your team can implement today to increase conversion.

Takes only two minutes

Don’t miss the next event

Experimentation Works hosts some of the most interesting and knowledgeable speakers and practitioners on experimentation and advanced CRO, leave your email address to get advance notification of the next event

If your CRO programme is not delivering the highest ROI of all of your marketing spend, then we should talk.

Share via
Copy link
Powered by Social Snap