A/B TEST CASE STUDY SUMMARY
Website: A wilderness camping and backpacking blog called Slick & Twisted Trails.
Offer: Gear checklist and instructional video
Hypothesis validated: Using unexpected language (mild swearing) that matches how the audience describes their problem will result in a higher conversion rate.
Version A results in 108% more email opt-ins than Version B.
Statistical Significance
The big takeaway: Do your research, understand your audience and then take calculated risks with your copy. It's a great way to stand out from competitors who are addicted to using bland 'best practice' copywriting techniques, which are often overused and ignored by consumers.
A Deeper Look: Why did the ‘crap’ version convert better?
A 108% lift in conversions due to copy changes alone is pretty significant. But it didn’t happen by chance: a fair bit of customer research and psychology went into getting this result.
The website: Slick & Twisted Trails is an eight-year-old website about wilderness camping and backpacking. It primarily features gear reviews, but also general tips and trip reports. Visitors typically ended up at this site after searching for some variation of “best <gear type> for backpacking.”
The audience: A HotJar poll showed that about 30% of visitors struggled to reduce the weight of their backpacks — a problem that my business partner and I were uniquely suited to solve. These folks typically fell into two camps:
- 1People who were interested in the practice of ultralight backpacking, which emphasizes carrying the lightest and least amount of gear. These were typically younger folks (age 25-50)
- 2Retired people who wanted to reduce the weight of their backpacks to ease the strain on their body. That way, they could hike longer and with less fatigue. These folks didn’t necessarily know about the practice of ultralight backpacking.
From my experience hanging out with backpackers, I knew this crowd typically valued a minimalist lifestyle, was skeptical of consumerism and had zero tolerance for marketing bullshit.
The funnel: I created a free ultralight backpacking checklist and instructional video to help visitors solve their ‘pack weight’ problem. This also allowed me to build my newsletter and gauge demand for a potential ultralight backpacking course. A slide-in form was shown to all website visitors offering the checklist.
The strategy: Slick marketing phrases wouldn’t work on this audience (it doesn’t work on most people these days, actually). So I considered two approaches:
VERSION 1: Use sticky language to surprise the brain and grab attention
Not only is the word ‘crap’ attention-snatching, some research shows that swearing (even mild swearing) can help add a persuasive punch to your copy.
In one study, testimonies with profanity were considered more believable. Also, psychologists from Northern Illinois University found that using swear words to emphasize a point can improve the persuasiveness of your argument.
But most of all, the headline “Hauling too much crap in your backpack?” matched how the audience thought about their problem. Surveys showed that many people knew they were carrying unnecessary ‘crap’ in their backpack, but they weren’t sure how to identify which items to bring on the trail and which ones to leave behind. The CTA button played off the saying ‘cut the crap’ to add another level of humor to the copy.
VERSION 2: Tell ‘em what they get: plain and simple
I figured the no-frills approach could resonate with this audience as well. These folks had an especially low tolerance of marketing BS, so hitting them with a big ol' benefit might come across as salesy.
In this CXL case study from 2017, the CROs found that using a simple and straightforward headline worked best when marketing to aspiring truck drivers. The benefit-laden copy they tested fell flat. So I wondered if this approach would work with my backpacker audience as well.
The results:
Test duration: 40 days
VERSION A: 1.13%
VERSION B: 0.54%
Winner: Version A increased email opt-ins by 108%
The test reached 99% statistical significance based on a two-sided test. Results verified using CXL’s A/B test calculator.
‘Whoa — isn’t 1% a pretty low conversion rate for a freebie like this?’
It sure is and I was fine with that. Because I didn’t want to convert everyone who came to the site — only those who were interested in reducing the weight of their backpack, which meant they’d be a fit for the ultralight backpacking course that was offered to subscribers later on.
Final thoughts:
Don’t be afraid to take (calculated) risks in your copy. Pay attention to the words your audience uses and think about their mindset at the exact moment they encounter your offer or brand. Then don't be shy: go ahead and test a hypothesis that might seem a little wild, a little crazy or even a little risky.
By taking a 'breakthrough or bust' approach to your messaging, you'll stand out by not falling into the 'best practices' trap that most companies default to. And who knows: you might even connect with your audience in a deeper, more meaningful way.