Using visitor goal, department goal, and analytics in optimization planning

I was in a webinar hosted by Kissmetrics and Optimizely, and it was really insightful. I’d thought I share the key things I’ve learned that when I applied to your optimization practice will help generate a better result.

First off, as most people who are in this optimization practice are aware that great test results start with good planning and understanding of customers. You have to know your customers first.

According to their advice which I agree is, you’ll need to do the following.

1. Get qualitative insights. Find the biggest customer problems.
2. Predict how to improve. Use your analytics and mathematics.
3. Confirm prediction with an a/b test or MVT (multi-variate testing)
4. Iterate the tests fast, as testing is not a one time thing.

Wrong tests start from colors and photos testing randomly. For those who know what I’m talking about, many marketers, when they hear testing they immediately want to try different colored links, random photos, etc. It may work at first, but that method will not scale.

You have to have a good test plan so that you’re not wasting human resource and time. Good or bad test, it still needs time and potentially money to get the results back.

Don’t be surprised if the test is not an uplift, that means all that time could have been spent on good planning to reduce your risks of running bad test or test that doesn’t perform. Hence, gaining more chances of getting test results that impact business outcomes.

1. Get qualitative insights. Find the biggest customer problems.

So Where is qualitative data? Survey’s are great way to capture qualitative data. There are many services online that will enable you to capture voice of customer. I’m currently using qualaroo (formerly Kiss Insights) and it is doing well for me, but there are other services out there that are gerat as well. Essentially with the survey data, you’ll need to synthesize the results to understand what are the customer’s biggest pain points on your web site.

You can also use your web analytics data and synthesize quant data to hone in on personas or why people are visiting the site and what are their pain points. Search terms and landing page behavior and bounce rate data are great place to profile what customer pain points are.

2. Predict how to improve. Use your analytics and mathematics.

Say the customer’s pain point is…”not able to find the product manual”, then you’ll know the conversion rates and the traffic drivers of product manual through analytics. The challenging part is to profile who are on the site for the product manual vs. people are aren’t on the site for the product manual.

That challenge could be mitigated by running a page level survey to hone in your tests to particular page if you knew what exact product manual consumers find challenging to find. In addition, running segmentation on web analytics data could tell you more details about the target audience.

I personally use heatmap tool on top of web analytics to really make my hypothesis a good one for setting a test.

So if you know your conversion rate, segmented data around manual downloads action event, then the exercise will be around testing to make that traffic or ratio higher or less. Ideally those efforts will correlate with VOC (voice of customer), hence improving qualitative data and desirable outcome.

3. Confirm prediction with an a/b test or MVT (multi-variate testing)

Once you derived your hypothesis and strategy to reduce the product manual download find-ability issue, then you’ll need to run the tests to see what works vs. don’t work.

This whole testing thing is a big cycle of test changes, so you may end up with many tests to really start moving the needle. That means your team really need to move fast, think fast, act fast, and be ready to iterate.

In order to run great tests, the folks at Kissmetrics and Optimizely highlighted a great point.

4. Iterate the tests fast, as testing is not a one time thing.

Start testing with understanding your Goals, Visitors Goals, Analytics.

This point was really important to me, because I know many folks involved in marketing are always focused on marketing/department goals, and analytics or data outcome, but many marketers are guilty of forgetting to put customers first. Marketers love to target, reach, and convert… right?!

Here is a screen grab of what they talked about. (courtesy of Optimziely and Kissmetrics)

This is awesome, because it really highlights that not only analysts should strive to analyze what’s working from department goal or company’s goal stand point, but we should also try to understand customer’s pain points and fix those problems.

For example, in eCommerce we obsess with conversion rate and revenue, so analysts focus on those metrics all day long. However, if you happen to learn that customers are finding the product image sucks because it doesn’t show the size against the avg hand size, then it is not about improving funnel or other acquisition ad placements.

It is fixing the damn photo so customers can feel comfortable buying the product in the first place. So you go test to see if that new image improves conversions and feedback thru VOC.

I think the breadth of data to analyze will become fun and challenging as you start to incorporate visitors goal into department goal & analytics. It seems many companies are doing this already and it is amazing what you can learn from them. Thanks to Kissmetrics and Optimizely for the awesome webinar.

Some slides…

How to Run AB Tests that Get REAL Results from KISSmetrics on SlideShare

Thank you so much for taking the time to read my content.  If you've liked what I've had to say please subscribe!