The causal mindset every marketer must know (and quietly skips)

Almost everything this blog ends up writing about (does this ad actually work? does changing the price move sales? will this new feature pull users deeper? and so on) comes down to one question: "If I do this, does that happen?" If we can't ask and answer that cleanly, every later post sits on sand. So the first post is the sand.

Causal thinking isn't a field. It's a way of thinking.

The phrase "causal inference" sounds like heavy statistics. It isn't. It's just "the habit of pinning down cause and effect correctly." Something we do every day without putting a name on it.

Imagine a store owner. Last quarter, revenue dropped 20%. In a panic:

"We're not advertising enough. Double the ad budget!"

Ad spend doubles. Next quarter's revenue? Flat. Why didn't it move? A few months later the real answer surfaces. The drop wasn't about ads at all. A competitor cut prices 15%. People just went where it was cheaper. Doubling ad spend changed nothing.

The owner trusted one wrong cause-and-effect: "not enough ads → revenue down." So they burned millions on ads. Meanwhile the actual cause (competitor pricing) got no response.

That's why pinning down cause matters. If you want to know what to do to change the outcome, you have to pin the cause down first. Otherwise you spend money in the wrong place and get nothing. Sometimes worse than nothing.

A pharma company runs clinical trials (splitting people into two groups, giving one the real drug and the other a placebo, then comparing) for years before launching a drug for exactly this reason. If "does this drug cure the illness?" gets the wrong answer, people die. The ad industry runs lift tests (comparing revenue between a group shown the ad and a group not shown) for the same reason. All the same question: "If I do this, does that happen?"

Hold on. "Correlation" vs "causation" first.

Two words first. The rest of this post hinges on the difference.

Correlation just means "two numbers move together." That's it. Nothing more. In summer, ice cream sales go up. Also in summer, temperatures go up. The two numbers move together, so we call them correlated. But that absolutely does not mean "buying ice cream makes temperatures go up."

Causation is different. It means "one thing produces the other." There's a mechanism. Changing one actually changes the other. "Taking the medicine lowers my fever." That's causation. Without the medicine, no fever drop. With it, drop.

In one line:

Correlation is easy. It's everywhere.
Causation is hard. It's rare.

Having a correlation doesn't mean you have causation. Mix the two up and you get the store owner doubling ad spend to no effect.

The world is full of "fake causation"

A few famous ones. All of them: two numbers move almost in lockstep, with zero real relationship behind them.

Example 1. Ice cream vs drownings

In summer, ice cream sales climb. In the same months, drownings also climb. So should we conclude:

"Eating ice cream causes drowning."

Obviously no. Both are being pushed by "summer," a common cause behind each. Summer means more ice cream. Summer also means more swimming.

Reveal the hidden cause

Example 2. Nicolas Cage films vs pool drownings

Here's a famous one from a US stats site. From 1999 to 2009, in the years Nicolas Cage was in more films, more Americans drowned in swimming pools. Plot the two lines and they overlap almost perfectly.

Is this causation? "Nicolas Cage making more movies drowns people in pools"? Obviously no. The two numbers happened to move together. Nothing more. There's no reason to call Nic and say "please stop acting, people are dying."

Example 3. Internet Explorer vs US murder rate

Another famous one. From 2006 to 2011, Internet Explorer's market share dropped. In the same window, the US murder rate also dropped. The two lines fall almost in lockstep. That doesn't mean "if we all ditch Explorer, murder goes down." Two numbers just happened to decline in the same window. That's all.

Why fake correlations happen

The technical name for this is "spurious correlation." "Spurious" means "fake" or "bogus." So: "fake correlation."

There are basically two reasons it happens. One is a hidden common cause (ice cream and drownings are both being pushed from behind by "summer"). The other is pure coincidence (Nic Cage and pool drownings have no shared cause; the lines just happened to look alike). The common-cause side gets its own post. For now hold onto this: fake correlations are everywhere, and the common-cause kind is the one that fools you most.

The real trouble starts the moment you look at data and say "ah, that's the cause!" That's how you end up banning ice cream to stop drownings, or asking Nic Cage to retire.

So when you're making decisions off data, always ask the question twice: "Is this real causation? Or just correlation?"

Okay, so how do you actually get at causation?

The examples above were the easy kind. Common sense screams "obviously no" right away. Ice cream doesn't drown people. Nic Cage doesn't drown people. But there's another kind of trap, the one where common sense doesn't save you. Those are the dangerous ones. Here's one.

Alex drinks half a bottle of wine with steak every night. Smokes, too.
Mike doesn't drink. Eats vegetables, jogs every morning, meditates sometimes.
A few years later: Alex gets cancer. Mike is healthy.

Now the question. Did Alex get cancer because of the wine and smoking? Is Mike healthy because of the clean living?

Sounds like an obvious "yes, of course" at first. But concluding that would be wrong. Why? Because Alex and Mike aren't the same person to begin with.

Different genes. Different upbringing. Different jobs, sleep, ancestry. The difference between them isn't just "drinking vs not drinking." It's thousands of other differences, all tangled together. So when Alex gets cancer, there's no way to tell which of those thousands of things actually caused it.

Plenty of people who've never touched a drink or cigarette get cancer. Plenty of lifelong heavy drinkers live to ninety. Which means Alex might have gotten cancer even without the wine, and Mike might have stayed healthy even if he'd smoked.

So comparing Alex vs Mike cannot get you "wine caused the cancer." Could be genes. Could be environment. Could be luck. There's no clean way to separate them.

So what are you supposed to compare? "The same Alex, in two parallel worlds."

The real way to pin down causation: compare the same person in two states. One side treated (wine), the other not. Everything else: identical.

To know if wine caused Alex's cancer, you'd have to compare "Alex who drank" and "Alex who didn't." Same genes, same environment, same ancestry: the same person, two versions. One with the drink and one without.

Is that possible? Obviously no. There's no parallel universe. Alex lives one life. Either he drinks or he doesn't. He can't do both.

Try it on yourself. Say you took the startup job out of college instead of the big-tech one. A few years later you wonder, "what would I be doing if I'd gone to big tech instead?" You can't know. The parallel-universe version of you is unobservable. The you that exists is the one who took the startup. The big-tech you lives only in your head.

Academics have a name for the Alex problem: the Fundamental Problem of Causal Inference. Grand name, but it means something simple. For a single person, you can never observe both "the outcome if treated" and "the outcome if not treated" at the same time. Nobody on earth can open parallel universes yet.

So we really can't ever get at true causation?

Honestly: yes.

A real example. A few weeks ago I dropped my phone on the street. The corner of the screen shattered. Glass dust still coming out of it (AppleCare already expired, truly painful).

Now the question. If you kicked a rock on your way out that morning, did that cause my phone to break? Obviously you'd say no.

But how do you prove it's "no"? Picture this: the rock you kicked clipped a passing retriever's tail. The startled dog's owner, instantly livid, posted something on Twitter. My friend saw the tweet and sent me a text. I pulled out my phone to read the text and dropped it. Each link in that chain has a tiny probability, sure, but it isn't zero.

The classic version of this: "a butterfly flapping its wings can cause a storm on the other side of the world." In theory, every event in the world connects to every other. To say two events are definitely unrelated, you'd have to trace every causal link in the entire universe. No one on earth can do that. Which means "true causation" is impossible in the strict sense. Not because the statistics aren't good enough. Because of how the world is built.

So, give up? No. We switch to "groups."

By now you might be thinking "so why bother studying causal inference at all?" Fair question. Luckily, there's one route that doesn't ask us to give up.

At the individual level, fine, impossible. At the group level, possible. How?

Here's the move. We can't make "the same Alex in two parallel worlds." But at the group level, we can make something close: two groups that are, on average, identical.

Think of it like splitting dodgeball teams in PE

Remember splitting dodgeball teams in gym class.

Method 1: let people pick. Teacher says "split yourselves into two teams." Friends clump up. The strong players bunch up on one side. Team skill levels drift apart.

Method 2: roll a die. Teacher says "roll a die; odd goes team A, even goes team B." Strong and weak get mixed into both sides. Team strengths come out roughly even.

That's the whole trick. Random assignment (coin-flip style) makes the two groups look similar. Each person's traits (skill, personality, whatever) end up sprinkled evenly across both sides instead of piling on one.

Scale it up to people

Now try the same thing with 1,000 people. 500 in group A, 500 in group B. Pure random, like flipping a coin per person.

Do that, and the two groups end up roughly identical on average. The phrase "on average" matters. It doesn't mean the two groups are literally identical. If you line up 500 vs 500 one-to-one, they're obviously different people. But the group-level averages come out similar. Average age. Average income. Average genetic predisposition. Even average "fondness for drinking" comes out close. The bigger the sample, the closer the match. Easiest way to say it: "draw enough and the biases wash out."

The real magic

Here's the part that matters most. Even variables we can never measure get balanced automatically.

Take 1,000 lottery tickets, mix them, split into two boxes of 500. Each ticket has a serial number, a prize amount, odds of winning, a print date, and we never look at any of those. Still, the two boxes come out roughly identical on average. Similar average serial, similar average prize, similar average odds. All we did was shuffle and split.

That's the real magic of random assignment. Even "third variables we don't know about" get handled for us. Genes, constitution, personality, sleep, ancestry, whatever. Shuffle and split, and the averages even out.

Randomize into two groups Sample size: 100

Now the comparison. And why it's so powerful.

The two groups are, on average, the same. Now treat one, not the other. Group A gets the treatment. Group B doesn't.

Why is this so powerful? Let's say it again, flat and clear. Before treatment, the two groups were, on average, identical. Average age, average income, average predisposition to buy. Even variables we can't measure came out equal on average (that was the whole point of the lottery-box analogy). So if the two groups end up with different outcomes after the treatment, there's exactly one thing that could have caused that difference: the treatment. Nothing else holds up as an explanation.

Someone might push back: "isn't that difference actually about genes? about environment? about who was more likely to buy to begin with?" All of those die on the same point. The two randomly-assigned groups were, on average, identical on every one of those variables. The one thing that was different is the treatment itself. Alex vs Mike doesn't bite here. The two random groups started the same, so an outcome gap can only be the treatment.

Another way to put it: random assignment creates "parallel universes at the group level." We can't make two parallel universes of a single person, but we can make two parallel universes of a crowd (two random, similar-on-average groups). Treat one, leave the other, and the average effect of the treatment falls out cleanly. That's how we get around the "Fundamental Problem of Causal Inference" at the group level.

This is the basic logic of an experiment. Clinical drug trials. A/B tests. Policy evaluations. All the same move. Random split, treat one side, compare. Simple, but powerful.

What happens when you take this to advertising

One idea we've built up this whole time: "The treated and untreated groups have to start out the same for the comparison to work." Otherwise you can never tell whether an outcome gap is from the treatment or from the fact that the two groups were different to begin with. Dodgeball teams. Lottery boxes. Alex vs Mike. All the same point.

Ad reports almost never obey this. It's less "they don't bother" and more "there's basically no way to." In advertising, random assignment is close to impossible. Ad platforms don't flip a coin to decide "show this user the ad or not." They use an algorithm to pick the users most likely to buy and push the ads toward them. Which means from the start, "people who saw the ad" and "people who didn't" are completely different kinds of people, all mixed together. Same shape as the PE class where the teacher said "split yourselves." Why that's especially hard in ads is its own post, so for this one, hold onto this: the numbers in the report can lie to you. (Lots more to come on ads. Stay tuned.)

Two of those numbers you see every day: "ad revenue" and "coupon revenue." Different-looking traps, same root. Both fail the "two groups that start out alike" test. Here's how each one goes.

Trap 1. The "$10M in ad revenue" line in your report

Everyone running ads sees this kind of number daily. "This quarter, this campaign brought in $10M in ad revenue." It's the sum of revenue from everyone who clicked the ad. (ROAS, which stands for return on ad spend, is just this number divided by what you spent on the ad. For this post we're skipping the ratio and looking at the plain $10M instead.)

People read this number roughly as: "The ad generated $10M. Turn the ad off and that $10M goes away." Which assumes the causal effect of the ad is $10M. In the back of your head you've already set up a comparison: "world with the ad: $10M in revenue" vs "world without the ad: $0."

Three things to pin down.

1. What is this number actually counting? Easy. Everyone who clicked the ad, their post-click revenue, summed. The gate is "ad click." Everything through the gate goes into the $10M.

2. What's the implicit comparison when we read this number? The report only shows "clickers spent $10M." But the moment you read it, your head jumps to the other side: "what would their revenue have been without the ad? $0." That's what makes "the ad made $10M" feel like it holds together.

Quick vocabulary stop. That imaginary other side, the "what would've happened if the ad hadn't shown" world, has a name: a counterfactual. Sounds academic, but we've already been using one. Remember Alex's "didn't drink" parallel universe earlier in the post? That was a counterfactual. Parallel universe = counterfactual. So right now, we're quietly setting the counterfactual revenue at "$0."

3. Why is that wrong? Because without the ad, plenty of those same people would've bought anyway. Modern ad delivery is designed to put ads in front of people already likely to buy. Retargeting (showing ads to people who already visited your site) pulls those visitors back. Branded search ads (ads tied to your brand name on search) catch people who already typed your name in. All of it. The very condition that put the ad in front of this person is "likely to buy soon." They'd have come through organic search (clicking the free Google result instead of the paid ad), direct traffic, email, whatever. Our "world without ad = $0" assumption is wrong from the start. In a world without ads, some of those people buy and some don't. How many? The report never checks.

So the $10M actually contains three very different kinds of purchases mixed together. The report tags them all as "ad revenue." But if you split them by "what would've happened without the ad" (= the counterfactual), three types show up:

• (a) Someone who would've bought nothing without the ad. Saw the ad, bought $100. → $100 is fully the ad's doing. The textbook (a). The version everyone has in their head.
• (b) Someone who would've bought $100 anyway through organic search. The ad happened to be in their face, they clicked it, and bought $100. → The ad caused $0 of this. It inserted itself into their path, but the purchase would've happened. Still, the report counts the full $100 as "ad revenue."
• (c) Someone who would've bought $100 without the ad, but after seeing the ad threw an extra item in the cart and spent $200. → The ad caused only the extra $100. The original $100 isn't the ad's doing. But the report counts the full $200 as "ad revenue."

Important point. This doesn't mean the ad causes no revenue at all. (a) is 100% the ad. The extra in (c) is also the ad. The ad has real effect: extra revenue that exists only because of the ad. There's a word for this extra: incremental lift. But the $10M in the report contains all three types rolled into one. Breaking it apart:

• What the ad actually caused = all of (a) + the extra in (c)
• What the ad didn't cause, but got credited for anyway = all of (b) + the original half of (c)

The report can't tell these apart. It bundles everything into "ad revenue." The $10M in your head is a world made of only (a). The real world is a mix of (a), (b), and (c). Which is why the ad's actual effect is smaller than the report number (the number inflates the effect).

Same shape as the ice cream and drownings story earlier. Behind ice cream and drowning there's a hidden common cause: "summer." Behind ad clicks and purchases there's a hidden common cause too: "the person was going to buy anyway." The same people who click ads also buy. The ad didn't produce the purchase. A shared intent-to-buy pushed both. Case (b) is exactly that shape.

The real question of ad measurement, then, is: "Of the $10M in ad revenue the report shows, how much is the share the ad actually caused (= all of (a) + the extra in (c), the incremental lift)?" How to pull that apart is what the rest of this blog is going to be about. One early note: anyone claiming they can separate it perfectly is lying to you. The "true causation is impossible" line from earlier applies here too. We're building "less-wrong estimates," not reaching an answer key.

Trap 2. Coupons are the same thing underneath

If the ad-revenue trap was "quietly assuming the counterfactual = $0," the coupon trap is "using a totally different group as the counterfactual." Same root, different shape.

Imagine an e-commerce site that puts a "10% off, code: SUMMER10" banner on the checkout page and pulls the data at the end of the quarter:

• Orders using the coupon: average basket $500 (3,000 orders)
• Orders not using the coupon: average basket $250 (10,000 orders)

Coupon orders are twice the size. The team's conclusion: "The coupon doubled average basket size. Next quarter, push coupons harder."

The mental model: coupon → basket size 2x. Turn it off and that extra revenue vanishes.

What the report is actually comparing: coupon-using orders vs non-coupon orders. Are those two groups the same kind of people? Nope. Alex vs Mike all over again.

People who type a coupon code at checkout are almost always people who came in with the intent to buy a lot. They checked reviews. They searched for discount codes. They stacked up a big cart. People who don't use the coupon are a mix of "didn't know a coupon existed," "wandered in, grabbed one thing," "doesn't bother with discounts," very different kinds of shoppers. The two groups differ far more by "how determined to buy they were when they arrived" than by "did they type a code or not."

Simplest way to say it: "the trap where only the people who were going to do the thing anyway are the ones doing it." The using side and the non-using side are two very different crowds from the start. There's a name for this: self-selection (people selecting themselves in). The treated group (coupon users) was never the same group as the untreated, so you can't tell whether the gap came from the treatment or from who was in which group to begin with.

The same hidden common cause shows up: "intent to buy." People who intend to buy a lot both grab coupons and spend more. The coupon didn't produce the bigger basket. A lot of it is shared push from "intent to buy." Ice cream and drownings in coupon form.

Both of them in one line

• Ad revenue trap: we read the number and quietly assume "revenue without ads = $0." Our counterfactual number is wrong.
• Coupon revenue trap: the report hands us "revenue of non-coupon users" as the counterfactual, and we accept it. The counterfactual group is a different kind of person altogether.

Both are "no proper counterfactual" problems: we're interpreting a number with no right-hand side to compare against. That doesn't mean ads or coupons have no real effect. They do. The point is "not all of the report's number is that effect." The moment we stop obeying the earlier rule ("treated and untreated have to start the same"), the number overstates the real effect.

Okay, how do we actually measure it then? The eBay experiment.

"How do we separate the real effect of a treatment out of the report number?" This is what causal inference researchers do for a living. There are dozens of methods, and the right one depends on the field. One thing they share: none of them is "perfect measurement." Not because the methods are sloppy. Because there's no parallel universe to check against, so no one knows the exact right answer. Even comparing methods against each other is often hard. (Another face of the "true causation is impossible" point we laid down earlier.)

Still, the cleanest single method is the one we've been building up this whole post: split two similar groups randomly, apply the treatment to only one, and compare the results. The famous real example that closes the picture:

It's a 2015 eBay paper.

eBay is a big US marketplace for new and used goods. When someone Googles "ebay," the result page shows a paid ad at the top and the eBay organic result right below it. Organic result = the free listing Google surfaces on its own. Paid ad = eBay paid to sit above that organic listing. eBay pays Google a few cents every time someone clicks the ad.

In the dashboard, that ad had a crazy-high return. The exact kind of thing we just unpacked in the "ad revenue" trap. The eBay team asked themselves the question we've been asking: "Did the ad actually cause the revenue, or would those people have clicked the organic result and come anyway?" The dashboard can't answer that. Ad-clickers and non-ad-clickers are two different crowds to start with.

So eBay ran a randomized experiment. The US is sliced into 210 regions based on which local TV channels people watch (ad teams call them DMAs and use them to buy ads geographically). eBay randomly picked 68 of them and fully turned off the "ebay" branded search ads in those regions (= treatment: no ad shown). The other 142 regions kept ads running normally (= no treatment). The experiment ran about two months. Random split for the reasons we built up: with enough regions picked at random, the two groups of regions end up similar on average across every variable (income, shopping behavior, whatever). Same shuffle-the-lottery-tickets logic. Only difference: a lottery ticket is now a region, not a person. Regions got shuffled, split.

Two months in, they compared revenue across the two groups of regions:

Revenue in the ad-off regions was essentially the same as revenue in the ad-on regions. People in ad-off regions just clicked the organic eBay link and came in anyway.

Nearly all the clicks the report was crediting to the paid ad would've come in through organic search even if the ad didn't exist. Which means the extra revenue the ad "actually caused" was close to zero. Nearly all of the enormous "ad revenue" number the dashboard was showing was credit for people who were going to come anyway. A pure illusion. Correlation, not causation.

Source: Blake, Nosko, Tadelis (2015), Econometrica, "Consumer Heterogeneity and Paid Search Effectiveness." The paper that keeps performance marketers up at night.

Why this matters

You can't find this out from the dashboard alone. The ad looks like a golden goose. Suggest turning it off and everyone panics about "losing the revenue." But the revenue wasn't leaving. The money was.

The method that caught it is the one we've been building: two groups split at random, treatment on one, compare the outcomes. Group-level causal estimation. The store owner's "double the ad budget" call. "Did the coupon lift revenue?" All of it. This is the frame that gets the answer right.

One honest caveat. Even this kind of experiment isn't "true causation." Even with a careful split you can land on weirdly non-representative regions. External events during the window (holidays, competitor promos, news cycles) can shift the result. If people in ad-off regions cross over and shop in ad-on regions, the effect gets diluted (spillover). Just like we said in the middle of this post, true causation is strictly impossible. But a randomized experiment is vastly closer to the truth than any single number pulled from a dashboard. That's the compromise worth making.

Almost everything this blog ends up writing is a variation of one line: "The number I'm looking at. Is it causation, or just correlation?" Attribution. Personalization. Ad lift tests. All different shapes of the same question: "how do we pull real causation out of numbers that only look causal."

So, the next post

That's the mindset of causal inference. True causation is impossible. At the group level, randomize and compare, and you can get something close.

Next post will pick up where this one leaves off: "so how do we actually design these random experiments in advertising?" The world of A/B tests, lift tests, geo experiments, that whole family.

(Quick note on A/B tests. If you think of them as "which of these two campaigns performs better," that's only half the story. The real heart of an A/B test is reconstructing the "world without the treatment" on average, which is how we sidestep the "Fundamental Problem" above. More on that next time.)

P.S. No guarantee the next post is actually the "experiment design" one. If something else grabs me more in the meantime, that one goes first. Apologies in advance.

One last line. Anyone who tells you they can pin causation down perfectly is either lying, delusional, or divine. We are none of those three. So "give up perfect, take smart compromise" is the better path. Basically every post this blog ever runs (measurement, personalization, ad experiments) is a different shape of that compromise.