Why A/B Testing Can Be a Waste of Time (And What to Do Instead)
A/B testing is useful. To a point.
Here’s why, and what you should do instead.
When A/B Testing Doesn’t Make $en$e
The lure of a single A/B test skyrocketing conversions 10% overnight is powerful.
But it’s also mostly a myth.
The problem? Most fail. Or at the very least, fail to deliver significant benefits.
They also have a few inherent problems, such as:
- You’re testing your own assumptions. So allowing bias to distort your A/B tests is both common and frequent.
- They take a long time to see results (minimum of 3-4 weeks). That opportunity cost could be better spent somewhere else (like the three tips below).
- Results are going to be completely dependent on a large sample size. So it might take awhile to attain true statistical significance (see above).
But don’t take my word for it.
And that magic number is…
Wait, there’s more!
To put faith in the results, every individual test should have at least 250 conversions (minimum) too.
Simply put, that rules out A LOT of people reading this. Or it should.
But fear not, because you can STILL greatly increase low conversions. You just need to turn your attention towards capitalizing on the big wins that happen on your site.
Avoid the tunnel vision of A/B testing minutia, and focus on improving how people flow through your website, converting at each step, to raise sitewide conversions across the board.
Step #1. Where are Your Visitors Coming From?
The quickest way to increase conversions is to understand who’s coming to your website, why they’re coming, and what they’re looking for.
By starting with where they’re coming from and how they got there.
Sounds catchy and overcomplicated. That’s why consultants get paid the big bucks. 🙂 To tell you what you already know.
Fortunately, it’s pretty simple.
All you’re doing is piecing together bits of basic information from your site analytics to make assumptions or hypotheses about your web traffic.
Let’s start with the easiest thing you probably already know: traffic channels. Start by pulling up results for the past 30 days, and the past six months to a year to rule out any sudden spikes or changes.
In the example above, you’ll notice that A LOT of this website’s traffic is coming through Search, both Organic and Paid, and Direct.
(For now, let’s ignore Direct. The TL;DR version is that Direct tends to be overstated with visits that are actually from emails or social – while those are understated proportionally.)
The fact that this site receives a TON of visits through Search and Direct, but very little from other sources, begins to hint at how people are finding them.
But to really make any educated guesses, we need to see where this traffic is going. Because that tells you A LOT about who’s coming to these pages, and why.
Here’s …read more
Source:: Kiss Metrics Blog