3 benefits to combining A/B testing with digital experience analytics

When conducting A/B testing, it’s common for people to only look at the beginning and end result to come to a conclusion. Did users convert or didn’t they convert? You have the beginning and the end result, but the devil is in the details, which is why this type of traditional A/B testing falls short.

Let’s break down an example of why the full picture of the user journey is important when conducting an A/B test. Since the pandemic hit, many road races have switched to virtual, meaning runners can participate in the race wherever and whenever they choose during a specified time period. Runners now are responsible for timing themselves as well. Say runner A and runner B, who are both just as fast as one another, both chose to run the Boston Marathon virtually. They’re being measured on who the better runner is. Runner A chose to run it on Tuesday and Runner B chose to run it on Saturday. Runner A finished in 3:50 and runner B finished in 3:57. Knowing the finish times, you can draw the conclusion that runner A is a better than runner B. But is that really accurate?

Other facts need to be taken into consideration to determine who is actually faster, and those details lie within the journey of each runner:

• How much sleep did each runner get the night before?
• What were the weather conditions?
• What speed did they each run at?
• Was there construction along the route?
• Did one run through a busy city while the other ran through a rural town?

The same considerations apply with A/B testing and understanding the digital experience (DX).

Combining A/B testing with digital experience analytics drives better business results

There’s an obvious gap in traditional A/B testing. This is where digital experience analytics comes into play. It can compliment your A/B testing and provide insight on the full customer journey, helping you make data driven decisions. Below are three business benefits to combining the two.

1. Analyze your users behavior throughout the journey

A/B testing helps you analyze if users are converting or aren’t. Glassbox compliments the A/B test to see why users are or aren’t converting. You may have created this great webpage and are shocked to see it’s not converting. Maybe there are errors on the page causing customers to struggle. Or, it’s possible the conversion button was placed in a bad spot and users are continuously scrolling up and down the page to find it and abandoning the page out of frustration. It could also be that the image on the page is taking too long to load for that specific test. Glassbox helps you understand the full page journey so you can see where users are getting frustrated and what’s causing their struggles so that you can make better informed decisions.

2. Visualize exactly where your users are spending their time

With the ability to replay your sessions back, you can see exactly what your users are doing on the specific webpage you’re testing. Glassbox gives you the opportunity to look at each test as a perspective on an interaction map and pinpoint exactly where users are spending their time versus where they’re not.

This comes in handy when you have something that you thought would be very engaging on your webpage, but it turns out that no one is interacting with it. After reviewing the data you can see that it’s placed below the fold and the bulk of the people aren’t spending their time there. With this data, you can fix the UX and increase engagement and conversions.

3. Prioritize revenue

With each A/B test you perform that’s combined with DX, you have insight into how much revenue it’s generating or not generating (or even how much you’re losing). Based on that, you can better understand which journeys result from a certain test, helping you prioritize which tests you should go with.

Glassbox helps you understand the full customer experience

When it comes to A/B testing, if you’re just looking at the beginning and end, you may be reaching the wrong conclusions. You need to factor in things about each individual user and their unique journey, which might not necessarily be directly related to your A/B test or you might not be aware they were related. Introducing DX into play with your A/B test provides you with the credibility behind your theory.

At Glassbox, we’re committed to giving you the full insights to interpret your A/B testing results and optimize your digital experience. If you have an A/B testing tool already, our platform integrates with the best, including Optimizely, AB Tasty, Adobe Target and more. For those getting started with A/B testing, we recently acquired Convertize’s A/B testing platform and are integrating it into Glassbox to make A/B testing a core component of our digital experience analytics platform.