Measurement · Pillar guide
Incrementality testing in 2026: methods, costs, and when to run one
Platform ROAS is a correlation number. Incrementality is the causation number. This is the pillar guide I hand every client who asks whether their Meta, Google, or LinkedIn spend is actually causing revenue, or just claiming credit for it.
Updated 2026-12-19. Covers conversion lift, geo experiments, marketing mix modelling, and modelled conversions with real cost and duration bands.
TL;DR
- Below USD 15k monthly spend: skip formal tests, fix tracking first.
- USD 15k to 40k: one platform conversion lift study per quarter on the biggest channel.
- USD 40k to 150k: add quarterly geo experiments and an annual open-source MMM.
- Above USD 150k: continuous measurement roadmap, 1 to 2 percent of media spend on measurement.
- Median reallocation across accounts I have tested: 18 percent of paid budget.
Why platform ROAS lies
Meta, Google, and LinkedIn each take credit for a conversion whenever they touched the user. So does GA4, HubSpot, and every affiliate pixel. Add them up on a busy account and reported ROAS across channels sums to 4x actual revenue. The gap is cannibalisation: brand search taking credit for organic demand, retargeting taking credit for repeat buyers, Advantage Plus taking credit for both.
Incrementality testing exists because the only way to know what a channel is actually causing is to switch it off, sometimes, on purpose. Everything else is a story.
The four methods, compared
Signal: Randomised holdout inside the platform, results in 2 to 4 weeks.
Best for: Single-channel validation: is my Meta or Google spend causing incremental conversions right now?
Cost: Free from the platform. Requires eligible spend (USD 25k+ over 30 days on Meta, similar on Google).
Limits: Only measures the platform running the test. Cannot compare Meta vs Google vs Organic. Runs on the platform's ID graph, so post-ATT signal is thinner than pre-2021.
Signal: Hold out one or more geographies for 4 to 8 weeks; compare to synthetic control markets.
Best for: Cross-channel questions: does incremental spend on Meta cannibalise organic or brand search?
Cost: 10 to 20 percent of test-cell spend for the test window, plus 1 to 2 weeks of analyst time.
Limits: Needs 8+ comparable markets. Small countries or single-city businesses cannot run clean geo tests. Bangladesh-only brands usually cannot; regional or global brands can.
Signal: Econometric regression on 18+ months of weekly spend, sales, and control variables.
Best for: Board-level budget allocation across paid, organic, PR, and offline. Answers 'how much should we spend on channel X next quarter?'
Cost: USD 8k to 40k per model refresh. Open-source options (Meta Robyn, Google Meridian) cut cost to internal-analyst time.
Limits: Backward-looking. Recommends yesterday's optimal budget. Pair with conversion lift for a forward-looking check.
Signal: Platform ML fills in unobservable conversions (iOS ATT opt-out, cross-device, cookie loss).
Best for: Daily optimisation signal when direct observation is broken. Keeps bid strategies fed.
Cost: Free, on by default in Google Ads and Meta.
Limits: Not an incrementality method. Estimates attribution, not causation. Treat as a bid signal, never as a scale-or-cut decision input.
Deep dives: incrementality test, geo experiment, marketing mix model, modelled conversions.
First-test playbook, 7 steps
- Pick one hypothesis: usually 'brand search is cannibalising organic' or 'Advantage Plus is duplicating retargeting spend'.
- Choose the smallest method that fits: platform lift study for single-channel, GeoLift for cross-channel, MMM only if you already have 18 months of clean weekly data.
- Write the test plan in one page: hypothesis, method, test and control cells, minimum detectable effect, duration, decision rule.
- Get finance and platform buy-in on the holdout cost before you start. Nothing kills a test faster than a founder pausing the holdout in week two.
- Run the test to its planned end date. Do not eyeball early results and stop.
- Read out with confidence interval, not point estimate. 'Lift 18 percent, 95 percent CI 6 to 30 percent' is decision-grade. 'Lift 18 percent' alone is not.
- Convert the finding into a budget shift in the next planning cycle. If the test does not change a decision, it was not worth running.
Cost and duration bands by spend
Method: Skip formal tests. Use directional brand-holdout for 2 weeks on brand search only. Cost: near zero.
Expected ROI: Not decision-grade at this spend. Focus on tracking hygiene and CRM-tied reporting first.
Method: One platform conversion lift study per quarter on the largest channel.
Expected ROI: Typical finding: 15 to 40 percent of reported conversions are non-incremental. Payback in one quarter.
Method: Quarterly geo experiment across 2 channels + annual open-source MMM.
Expected ROI: Reallocation of 10 to 25 percent of budget typical. Payback in 6 to 10 weeks.
Method: Continuous test roadmap: rolling lift studies + biannual MMM + quarterly geo experiments.
Expected ROI: Measurement is a line item, not a project. Budget 1 to 2 percent of media spend for measurement.
Six common mistakes
- Stopping the holdout early because 'the number looks good'. Early stopping inflates false positives.
- Reading modelled conversions as incrementality. They are attribution with a hole-filler, not causation.
- Running a geo test on 3 markets. Not enough statistical power; the confidence interval will span zero.
- Testing brand search off with a 2-week holdout during a peak sales window. Seasonality swamps the effect.
- Using MMM output more than 6 months old to justify a scale decision. Budget elasticity drifts fast.
- Not writing the decision rule before the test. Post-hoc rationalisation is what killed the last three tests I audited.
Incrementality testing FAQ
What is incrementality testing in performance marketing?
Incrementality testing measures the causal lift a campaign delivers over a scenario where the campaign did not run. It answers: how many of these conversions would have happened anyway? Standard methods include geo holdouts, conversion lift studies inside Meta and Google, matched-market tests, and marketing mix models. Platform-reported ROAS is a correlation number; incrementality is the causation number.
When should a brand run its first incrementality test?
Once monthly ad spend crosses roughly USD 15k across all paid channels, or when brand search and direct traffic together account for more than 25 percent of platform-reported conversions. Below that spend, the noise band on any incrementality test is wider than the effect size, so results are not decision-grade. Above it, unmeasured cannibalisation across brand search, retargeting, and Advantage Plus starts costing real money.
Which incrementality method should I start with?
Start with a Meta or Google conversion lift study if the platform offers it in your market: setup is free, the RCT design is clean, and you get a report in 2 to 4 weeks. Move to geo experiments once you have 8+ comparable markets and can hold one out for 4 weeks. Marketing mix modelling comes last: it needs 18+ months of weekly data across channels and only pays back at spend above USD 40k monthly.
How long does a geo experiment need to run?
Minimum 4 weeks of holdout, ideally 6 to 8. Shorter windows mix seasonality noise with the causal signal. Pair test and control markets on population, seasonality, and category demand, not on geographic proximity. Use Google's open-source GeoLift or Meta's geo-lift template, not eyeball comparisons.
What is the difference between conversion lift and modelled conversions?
Conversion lift is a controlled experiment: some users see the ad, some do not, and the difference in conversion rate is the lift. Modelled conversions fill in gaps where the platform cannot observe a conversion directly (iOS ATT opt-out, cookie loss, cross-device) using machine learning on similar observable users. Lift measures causation; modelled conversions estimate attribution when observation fails.
Does incrementality testing replace attribution modelling?
No. Attribution (data-driven, last-click, MTA) allocates credit across touchpoints for daily optimisation and reporting. Incrementality validates whether the channel is actually causing conversions, on a quarterly cadence. Run both. Use attribution for bid signal and campaign optimisation. Use incrementality to decide whether to scale, cut, or restructure a channel.
How much budget should be held out for an incrementality test?
Rule of thumb: expect to leave 10 to 20 percent of test-cell spend on the table for the duration of the test. That is the cost of causal learning. Compare it to the annualised waste from optimising on the wrong number: on a USD 30k monthly account with 25 percent brand-search cannibalisation, that is USD 90k per year in mis-allocated spend, versus USD 900 to 1,800 in test cost.
What do you deliver on an incrementality engagement?
One written test plan with hypothesis, method (lift study, geo-lift, or MMM), test and control definition, minimum detectable effect and duration. One dashboard tying platform-reported to incremental. One post-test readout with lift, confidence interval, and recommended budget shift by channel. And one integration into the media plan so the next quarter's decisions use the number, not last quarter's story.
Frequently asked questions
What is incrementality testing in performance marketing?
Incrementality testing measures the causal lift a campaign delivers over a scenario where the campaign did not run. It answers: how many of these conversions would have happened anyway? Standard methods include geo holdouts, conversion lift studies inside Meta and Google, matched-market tests, and marketing mix models. Platform-reported ROAS is a correlation number; incrementality is the causation number.
When should a brand run its first incrementality test?
Once monthly ad spend crosses roughly USD 15k across all paid channels, or when brand search and direct traffic together account for more than 25 percent of platform-reported conversions. Below that spend, the noise band on any incrementality test is wider than the effect size, so results are not decision-grade. Above it, unmeasured cannibalisation across brand search, retargeting, and Advantage Plus starts costing real money.
Which incrementality method should I start with?
Start with a Meta or Google conversion lift study if the platform offers it in your market: setup is free, the RCT design is clean, and you get a report in 2 to 4 weeks. Move to geo experiments once you have 8+ comparable markets and can hold one out for 4 weeks. Marketing mix modelling comes last: it needs 18+ months of weekly data across channels and only pays back at spend above USD 40k monthly.
How long does a geo experiment need to run?
Minimum 4 weeks of holdout, ideally 6 to 8. Shorter windows mix seasonality noise with the causal signal. Pair test and control markets on population, seasonality, and category demand, not on geographic proximity. Use Google's open-source GeoLift or Meta's geo-lift template, not eyeball comparisons.
What is the difference between conversion lift and modelled conversions?
Conversion lift is a controlled experiment: some users see the ad, some do not, and the difference in conversion rate is the lift. Modelled conversions fill in gaps where the platform cannot observe a conversion directly (iOS ATT opt-out, cookie loss, cross-device) using machine learning on similar observable users. Lift measures causation; modelled conversions estimate attribution when observation fails.
Does incrementality testing replace attribution modelling?
No. Attribution (data-driven, last-click, MTA) allocates credit across touchpoints for daily optimisation and reporting. Incrementality validates whether the channel is actually causing conversions, on a quarterly cadence. Run both. Use attribution for bid signal and campaign optimisation. Use incrementality to decide whether to scale, cut, or restructure a channel.
How much budget should be held out for an incrementality test?
Rule of thumb: expect to leave 10 to 20 percent of test-cell spend on the table for the duration of the test. That is the cost of causal learning. Compare it to the annualised waste from optimising on the wrong number: on a USD 30k monthly account with 25 percent brand-search cannibalisation, that is USD 90k per year in mis-allocated spend, versus USD 900 to 1,800 in test cost.
What do you deliver on an incrementality engagement?
One written test plan with hypothesis, method (lift study, geo-lift, or MMM), test and control definition, minimum detectable effect and duration. One dashboard tying platform-reported to incremental. One post-test readout with lift, confidence interval, and recommended budget shift by channel. And one integration into the media plan so the next quarter's decisions use the number, not last quarter's story.
Want a written incrementality test plan for your account?
Send me your last 90 days of platform reports plus one CRM export. I reply with a one-page test plan (hypothesis, method, duration, decision rule) inside four working hours, no sales pitch attached.