Growth Newsletter #132

Everyone loves to say, "Let's run an A/B test!"

But it's actually not the right move in many cases—particularly for startups. Make sure you can answer “yes” to these 7 questions before and after running an experiment:

Before

#1. Is your hypothesis testable?

Testable: “If we change our landing page header, our CTR will increase.”

Not testable: “If we change our landing page header, site visitors will like it better.”

That’s subjective and not measurable.

#2. Have stakeholders committed to abiding by the results?

We tend to reject outcomes that contradict our beliefs (called the Semmelweis Reflex, for gross reasons). That can be a problem if an experiment’s results are at odds with HIPPOs: highest-paid people’s opinions.

So ask: Are the HIPP actually willing to change it?

#3. Is the test doable?

Be realistic about the amount of time and resources it’ll take to reach statistical significance. If you don't have high volume, it probably isn't. Use this calculator to find out

#4. How can we make sure results are reliable?

Pay attention to Twyman's Law: Any figure that looks interesting or different is usually wrong.

Be very skeptical of the results, particularly if they're surprising.

After

#5. Are there clear cause and effect?

People shop more when it’s cold in the UK. That doesn’t mean they shop more because it’s cold. It probably has more to do with the holidays.

Correlation is not causation. Drill into the deeper reason.

#6. Have we gotten the most value out of it?

Your results could affect your market, product roadmap, future experiments, and entire growth strategy. One way to increase experiment value is to share your findings with your entire team.

#7. Are our experiments truly driving decision making at our company?

Are you just doing them to checking them off of a to-do list? Or is it actually something that's driving decision making? If not, they’re not worth the resources you’re putting into them.

And as a handy reference, here's a graphic!

‍

If you're older than 30, you probably remember late-night informercials for knives and exercise equipment. 

"You might pay $100. You might even pay $200! But we're selling it for just $39.99 for the next 20 minutes.

BUT WAIT, THERE'S MORE! If you order in the next 10 minutes, we'll give you a second knife, and a knife sharpener worth $20 for free." 

Here's how that worked:

1. They price anchored you at $100 or even $200.
2. $39.99 seems cheap in comparison—what a deal.
3. But they "quantity anchored" you at 1 being 39.99. Now they offer 2 for the same price!
4. Oh wait, AND a $20 knife sharpener too? I'd be crazy to not order several. 
5. For good measure, they added time pressure to get you to order right away—at 1 AM while your partner is asleep and not there to talk sense into you.

These were really effective.

We can use similar sales tactics, just in a less obvious and cringe way. 

Here's what Alex Hormozi recommends for creating effective "bonus offers" on sales calls:

Psychologically, if there’s all these bonuses, the buyer will think: “Well, the core offer has to be more valuable than all these bonuses."

And if you assign a price point to the bonuses that exceed that of the core offer, it’ll become a no-brainer—like an extra $40 knife and a $20 knife sharpener.

If you've never seen it, the Will It Blend? YouTube channel is one of the viral greats. For years they'd use their blenders to rip apart different everyday objects:

But what exactly makes a campaign like this spread and get 293,926,912 views? And for a boring old blender company, Blendtec, no less.

Jonah Berger (professor at the Wharton School) outlines in his book, Contagious, that we share content because of:

1. Social currency. Sharing it makes us look smart, cool, or interesting to our peers. For example: the ridiculousness of a blender tearing up a brand new iPad.
2. Triggers. A stimulus in our environment prompts us to talk about it. For example: a friend busts out their blender to make a smoothie—or mentions their breakfast.
3. Emotion. When we care, we share. Activating emotions like anger, excitement, amusement, and awe drive action more than happiness, sadness, and contentment. For example, the amusement and awe we get watching marbles get pulverized.
4. Public. Can others see that people are using or engaging with the product? It's hard to emulate the behavior of others if we can’t see it. For example, we see the millions of views on the videos.
5. Practical value. We believe it to be legitimately useful to someone—even better if it's niche and we can think of "just the right person this'll help." For example, knowing that this blender is more than enough to grind up your smoothie.
6. Story. What broader narrative can this be wrapped in? Embed ideas and products into stories that people want to tell. Make your message so integral to the narrative that they can’t tell the story without it. Like the blender that's able to rip apart a crowbar.

Contrast this instead to Blendtec making videos that simply talked about the tech specs (5 speeds, 1hp motor, etc) and showed it effortlessly make a mango smoothie.

BORING! Who cares, right?

If you want people to talk about your product, infuse your marketing campaigns with these elements to drastically increase the chance of it going viral (and driving sales).