Important A/B Testing Process You Want To Practice

In order to improve your website’s performance, you always want to be testing things.  It is very important that we marketers have an A/B testing process in place.

In that A/B testing process, you’ll need to apply your data analysis and make your data work for you.  The optimization phase would allow your learnings from what you have learned from digital data and take action on it.

In optimization, web metrics data turns into an actionable data.  What was analyzed is put into action to promote enhancements to better align the website to the business objective and goal.

Every change to the website, every design decision, every new feature should be analyzed in comparison to what the actual state of the website is and what the expected future impact on the site will be.

Once the change is made for optimization, it is necessary to measure the impact of the change and determine how it compares with expectations.  This concept will allow optimization phase to be the final point in the digital analytics life cycle, and act as a foundation for the next version of the life cycle.

Here is a process that worked really well for me when I took zero testing to dozens of tests in a month for a major brand in Silicon Valley.

The figure shown below, it is a typical conversion funnel seen in online business. In this example, the overall conversion rate for this funnel is 0.09%.

The kind of question to ask yourself is:

• Is this conversion rate meet your expectation?  In other words, are you meeting your sales goal?
• How can we improve this conversion rate?
• How much more return are we looking at when we raise the conversion rate?

There are so many questions to be asked before you work on website optimization. Hence analyzing before you start testing is super key.  Many marketers run ads and tests without strategies or proper planning and target.

Forecasting the impact and return for website optimization is important.  You really want to test an idea that you think that is going to work, and after you test it and you hypothesis fails, then you go and come back with a better idea.

While starting my journey on A/B testing, I’ve learned a lot of key points that I’d like to share.

The first thing to really think about prior to setting up a test is obviously to really understand what is the goal or outcome that you want to improve.

For example, a common goal for anyone who is selling stuff online is number of converisons, or lift in revenue.The important parts of the planning process for testing are:

1) Understanding the size of the issue

2) Prioritizing the issue

3) Come up with a great hypothesis to test

Understanding the size of the issue

It is very easy for a marketer or an executive to point out an issue that he or she feels very strongly about a particular problem on the website.

However, that may not be (probably in most of the cases) the biggest problem that the website has.  Depending on the problem you’re trying to solve, there would be various tactics to identify the size of the problem.

Taking bounce rate as a KPI example, you’ll probably want to sort the page by highest entrance visit (or landing page by visits) and find the page with the highest bounce rate and traffic volume.

Example…

This is a very important part of testing, as you don’t want to be solving a very small problem and wasting precious resource and time.

Prioritizing the issue

Now that you have a list of pages with big problems, but your team has more than one KPI (key performance indicator).

When you analyze the data, now you have multiple big problems that seem to be equally important. This is why sorting your order of work by prioritizing what to test is important.

One way to do that is analyzing the opportunities in common unit or metric. So if the problem is bounce rate, then converting to the traffic lost in terms of dollars could be done so that you can compare that to another conversion opportunity to sales in dollars.

Figuring out your methodology to prioritize the problem is going to be the 2nd most critical process.

Come up with a great hypothesis to test

Now that you know what page you want to test before you jump into involving the design folks and copywriters, do your analysis. You’ll need to start to understand what and why things suck and see if there are opportunities to improve.

What tests will give you a better success rate (lift in sales)?

Here are few things you can look at as you analyze your data:

• Traffic source and any particular things that pop out that are driving bad or even a great result
• Heatmap analysis to see what content is being engaged and interacted or not
• VOC: What are consumers saying about that page that could be improved

Don’t try too many things at once.
If you’re trying to start optimization testing for the first time, here is something to seriously take into consideration. Hopefully, your a/b testing is not just a one-time test, but rather a program that is part of digital marketing’s DNA.  Look at the Ready -> Aim -> Fire -> Debrief process above.  ;)

That said, you’ll probably want to find out some kind of pattern that works or pattern that highlights what makes your website sucks most of the time.

This is something worth trying at first and continues trying because it’ll give you a good leverage and organized thought process when you’re trying to solve a more complex problem later on in your optimization journey.

Check out this (above) quick diagram I drew. Say you’ll be performing 4 tests in upcoming weeks or months.

You know you’ll be testing or wanting to test several different test ideas, but you’re going to keep it simple by performing an A/B split testing, one at a time.

What this diagram highlight are, if you come up with randomly different new content per test while keeping the method consistent (consecutively using Circle for version A and Triangle for version B), you may find a particular pattern contributing to a win.

Let’s say Circles and Triangles represent a certain layout, and Content A…H are the actual writings within those different layouts.

So in this example, 3 out of 4 tests had version A winning the A/B split tests.  Something will tell you that maybe there is something about the Circle layout that keeps on contributing to Version A winning.  Key

The key takeaway for many A/B tests that I ran in the past experience:

Keep your tests simple!  Test one thing at a time for A/B split testing.

That said, before you head into your “Test #5”, perhaps you can conclude that there is something fundamentally wrong with the Triangle layout version.

With this in mind, then you may set up Test #5 with circle Vs. circle with different content to hone in on the real content that works, or do a same Content A Vs. Content A, but having Circle layout for Test A and Triangle layout for Test B.

Now, using circles or triangles to represent web page’s layout was pretty rudimentary, but if these could be some real-life examples you may encounter:

• Short page (Circle) Vs. Long Page (Triangle)
• Marketing fluffy content Vs. Tech Spec heavy content
• Black background Vs. White background
• 3 images Vs. 5 images

You can use a test method called MVT (multivariate testing), which is actually more complex, but such test method could accomplish testing various variables in one test.

That’s something to learn as you start running more tests.

What is your A/B testing practice?  How and what do you do to scale and making sure it leads to higher performing web business?

Thank you so much for taking the time to read my content.  If you've liked what I've had to say please subscribe!