Note: this is an updated version of a previous post on my own blog.

By now, you should know that A/B testing has the power to increase conversions and grow your revenue. Even a seemingly small lift from a 2% conversion rate to 3% conversion rate does wonders when you look closely at the math.

Let’s say you have 500,000 unique monthly visitors and you’re currently converting at 2% with an average order value of $20. That’s a monthly revenue of $200,000.

But if you started optimizing and created a better user experience for your shoppers, you could increase that conversion rate. On average, companies running 5+ tests per month will see at least a 5x ROI from their conversion rate optimization efforts. Even a 1% conversion lift affects revenue greatly:

With the same traffic of 500,000 unique monthly visitors and average order value of $20, you improve your ecommerce site and increase your conversion rate to 3%. Now, you’re looking at a monthly revenue of $300,000. That’s $100,000 more per month or $1,200,000 more per year.

These potential revenue gains didn’t come from acquiring more visitors to your ecommerce store; it came from your existing users. If you focus on converting current traffic, you’ll grow profits fast. All you need to do is optimize and A/B test your way to an improved site experience.

Just one problem – you’ve never A/B tested your ecommerce site before. Don’t worry.


Here’s how to get your first ecommerce A/B test up and running in under 15 minutes.

1st, Choose a platform. (2 minutes)

You’ll need a platform to execute your A/B tests and push experiments live. There are many options to choose from:

  1. Optimizely – The most adopted A/B testing platform in the world that helps users easily create and implement experiments. (Also integrates seamlessly with Experiment Engine)
  2. VWO – Another popular A/B testing platform that makes optimization more accessible using an editor and other tools.
  3. Other options include Adobe Target, Unbounce, Maxymiser, and more, depending on your needs.

2nd, Form a Hypothesis. (4 minutes)

Now it’s time to start thinking about ways to improve user experience for your store. Maybe that means easier navigation to your most popular items or emphasizing your deals. Try to gather the insight you currently have about your customers and form a hypothesis to enhance their buying experience.

For example, we looked at the TicketCity site, which sells tickets to popular events, to optimize their mobile experience.

TicketCity A/B Test Original

A hypothesis was drawn that users needed more information about event seating on the listings page before clicking into it. Customers needed an obvious way to find tickets by seating, which should drive more users towards the product page.

3rd, Start Simple (4 minutes)

If you have a strength, use it to your testing advantage. Perhaps you’re a whiz at writing copy or know how to create awesome graphics. Whatever it is, see if you can create something that aligns with your hypothesis. Looking at our example for TicketCity, a strong writer could add copy describing seating at the venue. A designer might think of ways to show a map. These types of variations on your original page become testable elements in your experiment.

Here are two A/B test variations we tested.

For TicketCity, experts from Experiment Engine’s network of optimizers submitted numerous variations. The original page you saw became our control (left) and our newly created version became our variation (right).

TicketCity Experiment Ecommerce

The variation aligns with the hypothesis by adding a map of the venue to the listings page. Showing the map by default allows users to look available seating first while considering their ticketing options.

4th, Set A Goal (2 minutes)

In order to know whether your A/B test resulted in a winning variation, you’ll need to determine the metric by which to measure success. If you’re working on a product details page, it’ll likely be how many users “Add to Cart”. If it’s the checkout page, it’ll be how many users complete purchases. Look at the action you’re trying to drive users towards.

For the TicketCity listings page, the goal was for more users to click into a ticket product page.

5th, Install The Tracking Code (2 minutes)

To get experiments technically running on your site, you’ll need to install code provided by the platform you’re using. Luckily with tools, like Optimizely, VWO, and Experiment Engine, it’s quick and easy. For Optimizely users, use this guide about implementing their code snippet. For VWO users, this will help you add their smart code. Experiment Engine customers are provided special instructions from our customer success team.

After this, you’re ready to click “run experiment”.


That’s it! You are now A/B testing.

After 14 or so minutes, you’re running your first true A/B test and on the way to optimizing your ecommerce store like a pro. Keep it up and you’ll continue to improve the process.

Of course, your first test doesn’t guarantee a win and occasional testing won’t consistently get you lifts. To continuously improve your ecommerce site, you’ll want to scale your A/B testing into a full-fledged optimization program that gets you to that average of 5+ tests per month. Because if you’re not maximizing testing capacity, you’re not getting those potential gains in monthly revenue. This means you should be investing more time and resources to manage workflow, gain customer insights, design & implements tests, and collect data. Your company should think about hiring a CRO or using an all-in-one solution like Experiment Engine.

To find out if our TicketCity A/B test won, check out the full case study for results and data.

To learn more about Experiment Engine’s platform and services, request a demo.

About the Author

Luiz Centenaro has worked with hundreds of eCommerce companies to help increase traffic and conversions and is an avid conversion rate optimization student. When he isn't optimizing websites he can probably be found climbing a mountain and hanging a hammock. If you love eCommerce you should follow Luiz on Twitter or Google +
  • http://ecommercecosmos.com/ Luiz Centenaro

    If anyone has questions about getting their A/B testing program started I would love to chat!

  • Katherine Rosenkranz

    Hey @Luiz_Centenaro:disqus — my organization has a few A/B tests we want to run through user research groups. We have everything built out but are curious how to approach the testing set up. Since we are doing user group testing instead of a live A/B test, how do you recommend we present the options to users? One user sees A and B for a test? or a user only sees A and we compare that to a different user’s impression of B?

    Huge thanks!
    Katherine

    • http://luizcentenaro.com/ Luiz Centenaro

      Hey Katherine, that is a great question and something I’ve never done before! The stats should stay the same though, if you have a significant sample size and randomize the groups you should get a valid test :D.

      Hope that helps!