Incrementality Testing
What is incrementality testing?
Incrementality testing is a measurement methodology that determines whether a marketing activity — an ad campaign, a channel, a placement — actually caused a consumer to take an action they would not have taken otherwise. Rather than measuring total conversions, incrementality testing isolates the causal effect of advertising by comparing the behavior of exposed consumers against a matched control group that did not see the ad.
How does incrementality testing work?
There are two main types of incrementality testing:
- Holdout testing: This type of testing involves randomly assigning users to either the test group or the control group. The test group is then exposed to the marketing campaign, while the control group is not. The performance of the two groups is then compared to calculate the incremental impact of the campaign.
- Matched cohort testing: This type of testing involves identifying two groups of users who are similar in terms of their demographics, interests, and behaviors. One group is then randomly assigned to be the test group, while the other group is assigned to be the control group. The test group is then exposed to the marketing campaign, while the control group is not. The performance of the two groups is then compared to calculate the incremental impact of the campaign.
Types of incrementality testing:
The two main types of incrementality testing are described above. In addition, there are a number of other methods that can be used to measure incrementality, such as:
- Multi-touch attribution (MTA): MTA is a method of tracking the impact of different marketing channels on conversions. It can be used to estimate the incremental impact of a particular channel by comparing the performance of users who were exposed to the channel to the performance of users who were not.
- Causal impact analysis: This is a more advanced method of incrementality testing that uses statistical methods to isolate the impact of a marketing campaign from other factors that may be affecting performance.
How to measure incrementality testing:
Incrementality testing can be measured using a variety of metrics, depending on the specific goals of the marketing campaign. Some common metrics include:
- Sales: The incremental increase in sales generated by the campaign.
- Leads: The incremental increase in leads generated by the campaign.
- App installs: The incremental increase in app installs generated by the campaign.
- Website traffic: The incremental increase in website traffic generated by the campaign.
Why is incrementality testing important to marketers? Incrementality testing is important to marketers because it helps them understand the true value of their marketing campaigns. By measuring the incremental impact of their campaigns, marketers can identify which campaigns are generating the most value and make adjustments to their budgets and strategies accordingly.
Who needs to know what incrementality testing is:
- Performance marketer
- Digital marketer
- Marketing manager
- Brand manager
- Product manager
- CEO
Why incrementality testing matters for commerce media
In commerce media, where ads appear at moments of already-high intent, incrementality testing is essential to proving that post-transaction placements drive net-new revenue rather than simply capturing consumers who would have converted anyway. An incrementality test answers the question: did this ad actually change behavior, or did it just happen to appear near a consumer who was going to act regardless?
In the Fluent context
Fluent's measurement framework is built around incrementality — using ROMO (Return on Measured Outcome) to verify that advertising on the Fluent network produces verified, incremental business outcomes for brand partners.