ShareWhat do these brands have in common?
They are not afraid to experiment. These companies have created a culture where everyone in the business is aligned and committed to testing. As a result, these businesses are thriving.
Transforming teams and companies to a point where everyone cares and respects experimentation is not only commendable, it’s necessary.
The power of testing
According to Forrester Research, companies that experiment grow 8 times faster.
Testing allows you to learn at a faster rate and innovate in the process. However, this is easier said than done. It’s not uncommon for businesses to leave their CRO experimentation efforts down to one specialist.
More often than not there is a discontent between conversion rate optimisation and the business. It’s often seen as an add-on or something that does not add value opposed to being an integral part of an organisation’s core strategy.
Testing is not about having the best tools or the largest volumes of traffic. It’s more about attitude, ideas, and practice – things that are accessible to anyone, therefore there is no excuse not to do it. If you are struggling to prove the value of your experimentation strategy or struggling to get internal buy-in the key is to test the things that matter.
When you do that, you, your colleagues and the business will start to learn and understand your customers. You start to understand what motivates them, what they buy, why they are likely to buy, and so on. This continual understanding of your customer sometimes is enough to spark and interest in CRO and thus the experimentation begins.
However often companies are put off testing because of two key reasons:
1. Fear of failing
Of course, nobody wants to fail, so it’s no surprise that this is one of the biggest barriers in introducing the concept of experimentation within your business.
Apple Newton
In 1993, Apple wasn’t the global super giant it is now. In fact, Apple wouldn’t go on to make a successful product until 2001 when it released the first iPod. The apple Newton was deemed “ahead of its time” and started at $700 dollars.
The device’s text recognition feature was so bad that even the Simpsons made fun of it. But fast forward 26 years and Apple is now the most valuable company in the world.
Watch the 1993 Apple Newton ad.
Amazon Fire phone
In 2014, Amazon released the Amazon Fire phone off the back of the success of their Kindle Fire tablet. I have never met anyone who owned the phone, did you? I think we can all agree it was a flop. The product made a loss of $170 million. To put that into perspective $170 dollars is what it cost to produce Iron Man 2 & the Hunger games movies.
Wrong.
The Amazon Echo.
Development of the Fire phone and the Echo began around the same time. While the Fire phone was a failure, Amazon was able to take their learnings and accelerate their efforts when building Echo and Alexa.
By 2019, more than 100 million Alexa-enabled devices have been sold. What does this prove? That Amazon certainly isn’t afraid to innovate and test, and they are reaping the benefits because of it.
In a letter to Amazon investors, Jeff Bezos wrote:
“As a company grows, everything needs to scale, including the size of your failed experiments. If the size of your failures isn’t growing, you’re not going to be inventing at a size that can actually move the needle. Amazon will be experimenting at the right scale for a company of our size if we occasionally have multibillion-dollar failures.
Of course, we won’t undertake such experiments cavalierly. We will work hard to make them good bets, but not all good bets will ultimately pay out. This kind of large-scale risk taking is part of the service we as a large company can provide to our customers and to society. The good news for shareowners is that a single big winning bet can more than cover the cost of many failed ones.”
2. Testing the wrong things or not testing at all
The Domino’s winter survivor deal is a traditional winter offer and has always been enormously popular. But in 2016, the company made a change to the deal:
- Reducing the price from £16.96 to £14.99
- Removing the cookie from the deal
These are small changes. The customer pays less, however if they want a cookie, they can add it back in and it would cost the same as the original deal. All of this sounds rational, right? The problem is, consumers are not always rational.
Unfortunately, the change had a very negative effect on the general perception of the deal. It was less popular and as a result the company made less money because fewer customers were taking part.
This is a great example of the sort of change that should be tested first before being implemented. To limit the risk, we test, measure and iterate. In this scenario, Domino’s should have created and tested multiple deals and gone with the deal/products that customers respond well to.
Test what matters
Practitioners often get too tied up in the minor tweaks: copy or layout changes, testing something to be safe before deploying it any way and so on. But these changes are often not radical enough to drive real change in user behaviour which is why so often CRO programmes fail or halter.
This differs greatly from how those who do it well. When you compare yourself to the likes of Amazon, or Spotify, or Air B&B they use their programmes to introduce new concepts, new bundles, new pricing structures, in summary radical changes.
This goes back to my point earlier. Test what matters – but how do you do that?
Here are a few questions to think about when trying to work out what to test and what matters most to your business:
- What are the goals, assumptions, risks, investments that my business is making?
- What processes are in place just because “that’s the way it’s always been done”?
- What are the risks you’re taking in the next year?
- What are you going to be doing / what are you not going to be doing?
You could also go around your business and start asking teams and people what is useful to them and being to assess whether experimentation in the business can provide advantage over your competition.
Experimentation is a chance to make better and bolder choices. It allows you to see what is behind a door before deciding to go through.
Be radical, but not always
When you start to take a more radical approach to experimentation, you will begin to realise that your standard A/B test perhaps doesn’t offer as much value as you once thought. Not all changes need to be radical, there is always room in your experimental strategy for small changes, but they should not make up the majority of your testing strategy as this is where your testing begins to suffer.
Stakeholders lose interest, and it doesn’t add value and testing halters. When you focus on changes that matter and start to experience results that propel the business you will quickly see a surge in investment in your experimentation strategy
Running experiments
This is likely to be the biggest challenge for most. Getting a concept to test can be hard work for many reasons:
- You can’t find developers to carry out your concepts
- It may be too technically demanding
- Improvement ideas may be deemed ‘too radical’ to be tested
If you ever come across this barrier you need to start looking at Minimum viable experiments. Think of MVE as a proof of concept. So, you have an idea of new radical navigation that is going to totally change how users interact with your website. Rather than spending hours coding and designing the new navigation, going through a round of amends, getting stakeholder approval etc, you, in essence, build a proof of concept to test the idea. The “minimal” part of MVE means it should test one (and only one) idea. The mechanism to test the concept as I’m going to demonstrate in a few examples below can vary however the premise is the same.
Zalando
Zalando a famous German clothing retailer founded in 2008. Before starting they proved their concept with just an online storefront consisting of pictures of shoes from shoe stores. When a shoe was a sold, they would run down to the local shoe shop, pop them in a box and post them to the customer.
Dropbox
A more famous case study comes from Dropbox. Before even having a working product, they created a video 3-minute explainer video highlighting the value of what they wanted to offer. People loved it and overnight it had 75,00 subscribers.
Testing your concept
You can test your concept in a number of ways, including:
- Biometric testing
- Users interviews
- Remote testing
- Landing pages
- Explainer video
- Digital prototypes
- Paper prototypes
- Virtual reality
TL;DR
Driving the internal perception of testing isn’t always easy, but experimentation is a key opportunity for growth. Changing your mentality to testing and the makeup of your experimentation strategy can have a profound impact on business success and internal buy-in.
- Think radically and innovate
- Switch up your testing programme, test big, test what is important
- Be prepared to fail sometimes but don’t let the fear put you off
- Don’t let technical hurdles be a barrier and consider different ways to prove a concept
- Simplify experiments where possible to keep up momentum
- Keep innovating!
If you’d like to talk to us about CRO and testing, get in touch today!
Or if you’d like to learn more about CRO, we have a free Conversion Rate Optimisation guide, and a prerecorded CRO webinar you can listen to.
