CRO In the Wild – 5 real world A/B tests we’ve seen this week

Great marketers are always looking for inspiration and what better way than to see what some of the biggest brands worldwide are testing on their sites. In this new series of posts, we’re looking at real tests on real websites? These aren’t published case studies or carefully chosen examples. We’ve looked at sites running split tests and captured the variations so that you can see what other brands are testing.

These aren’t our clients so we don’t know the hypotheses and we can’t see the data, but for each test I’ve added a few comments on what we’ve seen.

Zipjet – Trustpilot Badge

We’ll kick off with London based laundry provider, Zipjet. They’re currently running a homepage test where they’re adding a Trustpilot badge to the top right of the page:

This is a great test and one that any business can try. Social proof is an extremely strong persuasion principle and this is an effective way to get it in front of as many visitors as possible. A similar test for a leading healthcare company increased sales by 13.6%. This could be even more effective if Zipjet had a higher Trustpilot rating, but I’m confident that this will be a winner for them. Do you agree? Let us know in the comments section below.

Follow up tests could include making the Trustpilot score even more prominent on the homepage, or possibly even including it in the headline.

Hidden Hearing – New Landing Page

Our next example comes from UK hearing aid company, Hidden Hearing. This test is a major redesign of their landing page:

 

There’s a lot going on here, but I’ll break it down into the main changes:

• Imagery – the variation features a clear image of the main target audience for Hidden Hearing. Images such as this help people to identify with the brand and confirm that it is right for them, however can risk alienating non-core customers. Hidden Hearing may have many younger customers who could be put off by this.
• Key Benefits – these have been reduced from five in a graphic to three main points. Having a large number of benefits is good for conveying the Unique Selling Points (USPs) of a brand, but can overwhelm users. The original small text could easily be missed, whereas focusing on three key points makes sure these are clearly visible.
• Headline and form header focuses on “local branch” – this can work well in increasing trust (brands with a real-world local presence are seen as more trustworthy), however people may be concerned that there isn’t a branch local to them.
• Focus on Experience – the expert testimonial has been changed and the qualifications and certifications Hidden Hearing have received have been promoted at the bottom of the page. Clearly experience and trust is important to visitors, and by promoting these, Hidden Hearing aim to reassure them.
• Testimonial Quote – adding the quote “The free hearing test … has made a significant difference to our lives” is a strong message as it focuses on outcomes. Users of products such as these are looking to solve a problem and this aims to address that. It could be even more effective with a more specific outcome “I can hear the birds when I wake up in the morning now and I never could before”. Additionally, giving a name and location of the person who said it would help to add credibility

This is a big change on a very significant page of the site and I would expect it is giving some very clear results. How would you have done this differently? Let us know below!

Wageday Advance – Button Colours

Unfortunately this is an example of one of those conversion optimisation myths that just won’t go away. For many years sites have been trying to get small increases in conversion rates through testing button colours and Wageday Advance is no exception:

‍

Wageday Advance might get a small result from this test, although it could be due to chance so I’d strongly recommend testing it to 99% confidence. Button colours are unlikely to strongly influence users. Ultimately there are three key things a button should do:

1. Be clear and obvious on the page (in this case the orange button is already)
2. Explain what the next action is (this does a good job)
3. Connect to the user’s end goal (it may be worth adding something to connect to the user’s goal of getting approved for a loan for example)

Button colour tests can be effective if the button isn’t clear on the page, but Wageday Advance could get much bigger increases from testing the copy of the button, or even better the surrounding content to ensure that users’ concerns are overcome.

Have you had success running button colour tests? Let us know in the comments.

Wufoo – Form On Mobile Landing Page

Wufoo is a web form provider owned by SurveyMonkey. They are currently running a test on mobile:

This is a nice, effective test from Wufoo with a clear goal. In Variation #1, the form is shown by default, making it clearer what the next step is, removing an interaction from the signup process (no need to click the button) and reducing the number of options available (just sign up for free without a pro option). This is risky, however, as there is no information at the top of the page to make it clear what the service is.

Often for SaaS products such as this, getting the user signed up and using the product is the key step, so this may work well, however I would want to look at scrolmaps for this page to understand exactly which features people are reading about in both versions.

Has this approach worked for you before? Let us know in the comments.

FreeAgent – Homepage

FreeAgent are running a very big test at the moment with three very different variations:

This is a great test as it uses three very different ways to convince the user:

• Control – This focuses on the things that users want to achieve (the text “VAT returns” is dynamic and changes regularly to show other use cases for the tool). This is a really strong approach to use if your tool or service has many different uses and has been one I’ve used successfully with a financial services client.
• Variation #1 – This is centred around social proof. The large statistic of “50,000 small businesses” is very compelling to show the scale of the service and adding in real tweets is an extremely effective way to make the social proof more effective. People identify more strongly with social proof when it’s presented as being from identifiable individuals and this should work well. Free Agent could make this stronger by using a specific figure for the number of small businesses, which could give more credibility.
• Variation #2 – This focuses on the individual user’s frustrations. A good approach to convincing someone they need your service is to focus on the pain you’re helping them to avoid.

This test really aims to understand what motivates FreeAgent’s users. It’s not just about adding something to the site. When they have the results they will know clearly which approach is best to use with their customers.

Have you run a test like this before? Let us know in the comments below.

Summary

This week we’ve seen five very different tests from across the web including some great examples of brands running tests to really understand their users, and some which are going to be less effective. Check back next week for five more real world test examples.