.webp)

.webp)

.webp)

.webp)

.webp)

.webp)

Marketing Incrementality Testing: A Complete Guide to Measuring What Matters

Published

June 2, 2025

Updated

October 3, 2025

In today's complex marketing landscape, understanding the true impact of your marketing efforts has never been more challenging or more important. Traditional attribution models often paint an incomplete and sometimes misleading picture of campaign performance. That's where incrementality testing comes in.

What is Incrementality in Marketing?

At its heart, incrementality testing answers one fundamental question: Would this conversion have happened anyway, without my marketing?

Incrementality is defined as the lift in desired outcomes that wouldn't have occurred without a specific marketing intervention. In other words, it's the true impact of your marketing. We can think of it as a simple equation:

Incrementality = (Results with marketing) - (Results without marketing)

Correlation vs. Causation

We've all heard this one before, but it's worth repeating: Correlation is not causation! Just because someone saw your ad and then made a purchase doesn't mean the ad caused the purchase.

Marketing Has an Attribution Problem

The Multi-Touch Reality

Today's customer journey is anything but simple. Before making a purchase, customers might see your search ad, open your email, view a social media post, click on a display ad, visit your store... the list goes on.

The Credit-Claiming Game

A customer often interacts with your brand multiple times before converting. Their journey could look something like this:

• Day 1: Person sees a Meta ad, clicks, but doesn't convert.
• Day 3: Person does some comparison shopping and clicks on a Google ad, but doesn't convert, though you capture their email address.
• Day 5: Person clicks on an email and converts with an offer.

In traditional attribution, each of these channels might claim credit for that conversion.

Several Attribution Models Attempt to Solve the Problem

Over the years, marketers have developed various attribution models to address this problem:

• Last-click gives all credit to the final touchpoint before conversion—it is simple but deeply flawed.
• First-click does the opposite, crediting the touchpoint that started the journey.
• Linear attribution spreads credit equally.
• Time decay gives more weight to more recent touchpoints.

These models give you some idea of what's working, but they're still just rule-based guesses and don’t take into account channels that aren’t click-based.

Why Incrementality Testing?

Beyond Attribution Models

So why should we move beyond attribution models to incrementality testing? Attribution models are fundamentally backward-looking—they take events that already happened and assign credit after the fact. Incrementality testing is forward-looking and uses scientific experiments to determine what actually causes conversions.

The Business Value

Incrementality testing delivers three big business benefits:

1. Better Budget Allocation: When you know which channels are truly driving incremental conversions, you can shift money toward what's working and away from what's not.
2. True ROAS Measurement: You'll see the genuine return on your spending.
3. Focus on What Works: You can focus your strategy on what actually moves the needle.

Methodologies & Approaches

Testing Overview

There's no one-size-fits-all approach to incrementality testing; you can select the right methodology based on the channels you're running and each platform's specific capabilities:

• Digital Advertising: Randomized experiments (PSA/Ghost ads)
• Email Marketing: Audience holdouts (RCT implementation)
• Paid Search: Geo testing

Randomized Experiments

Randomized experiments are considered the gold standard of incrementality testing. You randomly divide your audience into two groups, a test group that sees your marketing and a control group that doesn't.

Audience Holdout Tests

Audience holdout tests are one of the most practical implementations of randomized experiments. The idea is simple: You reserve a small portion of your audience—maybe 10-20%—and don't show them your marketing.

Interpreting Results

Key Metrics to Measure

1. Incremental lift: The percentage improvement in your test group compared to your control group.
2. Incremental conversions: The actual number of extra conversions generated by your marketing.
3. Incremental ROAS: Your return on ad spend based on true incremental revenue.

Understanding Statistical Significance

One of the most important concepts when interpreting test results is statistical significance.

From Insights to Action

The whole point of incrementality testing is to drive better marketing decisions. Here's a simple framework for turning your test insights into action:

1. Identify high-incremental channels: These are your marketing workhorses.
2. Shift budget toward what works: Gradually move spending from low-incremental to high-incremental channels.

Getting Started

Ready to embark on your incrementality testing journey? Here are some practical next steps:

1. Identify one area of your marketing where you suspect traditional metrics might be misleading.
2. Choose a simple methodology like a holdout test to start with.
3. Set clear metrics for what success looks like.
4. Learn from your results and iterate.