Why AB testing?
A/B testing is a simple and effective way to test new ideas and understand what works and what doesn’t. It helps you learn more about your customers, while also providing insight into what they want from your website.
Let's look at an example: You have two versions of an email signup form on your website: one with just text and one with both text and graphics. Which one should you use?
Of course, it’s very difficult to know which method will work best for getting people on board. Often when deciding on an approach, a lot of opinions get thrown about and it’s hard to think objectively about what will best help you reach your end-goal. So, instead of debating endlessly and going back-and forth, you can set up an A/B test.
1. Pick what you want to test
When it comes to A/B testing, the first step is understanding what you want to test. Think about what kind of knowledge you could gain that could add significant value. One of the most important elements of a good A/B test is a clear, concise hypothesis that can be objectively tested and will link directly to your overarching business goals.
This is a good time to be creative and think of something that will ‘move the needle’. It’s also important that you can measure it—so if you want to test whether buttons are more effective when they are green or blue, for example, make sure you have some way of measuring how many people clicked on each button.
When you're trying to find the right A/B test to run, you need to first consider what your website is trying to accomplish. Is it a SaaS tool? An e-commerce store? A blog that wants traffic and social media shares? You can use an A/B test as an opportunity to improve several different things:
- Navigation: Do you find that people have trouble finding information on your website? Could they be saving time by accessing certain resources through different menus? Try creating new navigation menus and see how they perform compared with the old ones.
- Content: Do users understand what your site is about or do they get confused by terms or language used? If so, consider using different text and visuals on your website to see what performs better on your key metrics.
- CTA (Call To Action): Does your page layout encourage visitors into taking action and converting into customers? Try changing up where call-to-action buttons are placed and how you guide users to them.
If the goal of your website is lead generation, for example, then an A/B test might involve changing the copy on one page from "Submit" to "Contact us" with information about how many people have already contacted them so far this month in order to encourage visitors who haven't yet done so yet (but may be interested). This is an effect known as social proof. Or perhaps if you’re an ecommerce site, you might want to experiment with using the scarcity effect. The scarcity effect works by creating the feeling that people need to take action now, otherwise they will miss out. This could be implemented with something as simple as an indicator that shows which products are almost out of stock.
2. Determine your testing methodology
Now you’ve determined what you want to test and why it’s important to your business. It’s time to begin the next step - how are you going to implement the test? Let’s say for example that you want to change some of the product visuals on your e-commerce website. You predict that by making the image of the product larger that it will receive more attention and therefore increase product sales. Your next step is to define which images you want to change, and of which product(s). If you change all the product images on your site, then you’re of course not going to be able to tell if making the visuals bigger actually made a difference to your sales. So you could decide to keep it simple and make the images bigger of a few key products that are listed on your site on separate pages.
In A/B testing, it’s crucial to ensure that you only change one variable at a time. If you change multiple elements of your website and/or page, you won’t be able to distinguish between which change actually affects your conversion metrics. Therefore making it harder to derive any meaningful conclusions that you can use in the future. How long should you run an experiment for?
When deciding how long to run an experiment for, you'll need to consider a few things. The longer you run an experiment for, the more reliable your results will be. A good rule of thumb is that if your site has more than 100 visitors per day, then it should be at least 14 days long. This allows enough time for users to see both variations of your site and gives enough time for data to accumulate in order to get accurate results.
Finding the exact right length can be tricky, as longer tests also create an opportunity cost. If you’re testing one small feature for 2 months, that also means that you cannot test other features on that page for 2 months until the first experiment is over. Otherwise you might affect the validity of your experiment results. However, running too many tests can also be expensive - you need to research and find ideas for things to test, design the changes and implement them on the live site. This can mean several hours from developers and designers in the process. And of course, not every experiment results is a positive one, so you need to pick wisely!
So what’s the best length for your A/B test? It depends. If you're looking for a quick answer, then you should be aiming for at least 14 days. This will give you enough time to gather data and get accurate results while also minimizing costs. However, this does depend greatly on the regular traffic to your website.
3. Choose your A/B testing tool
Having a good hypothesis and methodology is great, but they’re not much use if you don’t have the tools to implement them! Fortunately, there are many solid A/B testing tools on the market. In order to choose the right one for your needs, you must first look at the following factors:
Firstly (and most importantly) does the functionality meet the needs of your experiment? Let’s say that our hypothetical A/B tester wants to change the navigation of their website, will the tool allow them to edit this with minimal development assistance? Or is it more targeted towards changing text and images? It’s also good to keep in mind what kind of experiments you will want to use in the future. This can be hard to anticipate, but keeping this in mind, it’s good to find a tool that has the flexibility to test multiple different kinds of variables in different settings.
How easy is it to use? Will you be able to set up an A/B test in minutes, or will it take hours of work? This is a very important factor to consider, especially considering the resources and expertise you have at your disposal. If you’re a small company or startup, you probably want to start with something more on the simple side.
How easy is it to integrate with your website? Does this tool require technical knowledge that is above your skill level? Again, if you have developers to help you implement your A/B testing tool, that’s great. You can customize your setup to meet the exact needs of your organization. But if you don’t have the luxury of having developers at your disposal, look for something easy with low or no-code required to set up.
What kind of customer support does this tool offer? Do they have a live chat feature or simply email responses only? If something goes wrong with your A/B test and there’s no one you can talk to immediately about fixing it, then there might not be much point in using this solution at all. There’s nothing more frustrating than something going wrong with your experiment data and having nobody to respond to you when you need them most!
Finding the right tool for the job is crucial to get the most out of the tests you want to run. It can be hard to choose the right tool with so many options on the market, but as long as you consider the 4 points above you should be sufficiently informed to make a solid choice for your website.
4. Setting up an A/B test
Setting up an A/B test can be simple, but it very much depends on what you want to test, and what tools you have available. If you want to test things like changing the colour of an element or the text on a page, most no/low-code tools can help you with this. But if you want to change more complex things like the navigation structure of your site you might need a developer to help you. Once you've decided on your goals, chosen the best metric and found the right time to test, you'll be ready to get started.
The first step is creating two versions of a webpage—the control and the experimental variation. For example, if you're testing whether or not your messaging has improved conversion rates in comparison to the original version of your homepage, then you would create two different homepages with different messaging concepts displayed on them: one with "Free Shipping" as its headline; another with "Savings." Your goal would be for visitors who see each page (control group vs variant group) to convert into customers at similar rates as those who previously visited this page without any changes made (i.e., no free shipping vs no savings).