The ABCs of A/B Testing
You may have read the phrase “A/B testing” numerous times before and simply glossed over it. Or perhaps it’s a strategy that you implement on a daily basis. Either way, it never hurts to have a refresher course on the basic fundamentals.
What is A/B Testing?
A/B testing (also known as split testing) is beneficial for marketing and design decisions. It is a method of comparing two versions or elements of a webpage or app to see which performs better. (Note: the term is often used even when testing more than the two variables A and B.) After showing ‘A’ and ‘B’ to similar users at the same time, you compare the conversion rates to see what performs best. You can find out what works and what doesn’t, all before you implement or publish content.
Why should you run an A/B test?
As a marketer, I’m always looking for experiments to run. In the digital work, optimizing conversion rates on websites is a must. A/B testing plays a big role in my CRO plans. It allows me to run a controlled hypothesis test and capitalize on my existing user base. Attracting paid traffic can be costly but the price of increasing conversions is low. The ROI on A/B tests can be significant; even small changes to a landing page can lead to an increase in lead generation and ulitmately sales.
What can you test? Anything on your website that affects the user’s experience (i.e. headlines, subheadlines, paragraph text, links, images, CTAs) can be tested.
Common Mistakes:
Are you running an A/B experiment but getting bogus or misleading results? There are a few things you might the doing wrong.
- No process/strategy: The best way to run an A/B test is to follow the scientific process. You need a trajectory and end goal to guide your efforts, such as business goals, web goals, or key success metrics. It’s a science experiment so it needs to be rigorous. Procedures ned to be in place to ensure what you are doing is measurable and repeatable.
- No theorizing: Study your website data, research your personas’ behaviors, craft your hypothesis, test said hypothesis, analyze the results and repeat as needed until you have something work implementing. Imagine you’re the customer/intended audience and think about why they didn’t do what you intended for them to do.
- No hypothesis: C’mon, what’s the first step in any experiment? Remember learning the scientific method in school? If you’re like me, it might be one of the few things you remember from elementary school science class. Step one (after initial research) is to construct your hypothesis = By {doing this}, {KPI, A, B} will improve {this much} because of {theory, data}.
- No optimization: Or, at least, you are optimizing for the wrong KPIs. A/B is all about conversions, micro and macro.
- You’ve started with a complicated test : A good general rule of thumb is if you’re new to something, start with a simple endeavor, not something grandiose. Test one variable at a time.
There you have it; you can now put a stop to all that guesswork. Instead of jumping off the deep end with no clear path, use web analytics to your advantage to get insights based on real user feedback. Lastly, don’t give up if/when your first A/B tests fail. Try, try again!
Resources:
