Incrementality Testing: How to Measure the Real Impact of Your Ads

Incrementality? What does it mean and why is it the hot new buzzword on the marketing scene?

We’ve put together a guide for business owners and directors to cut through the noise and understand the essentials around this topic.

We don’t want to get into a blog with lots of buzzwords and complexity that can’t be digested and transformed into actions. This is something that comes very easily when we start looking into any content around ads that contain the words marginal or incremental!

What do we mean by incrementality in ads?

Incrementality from an ads or marketing perspective means we can see that activity has actually delivered a measurable impact on the business.

This isn’t as straightforward as it sounds as any ad channel will have its measured sales, leads and return right?

The issue we have now is that there are activities which cannibalise sales and leads which would have happened anyway or certainly without ads themselves having an interaction.

This means that we can be spending on activity and measuring in channel, however, we’re not actually feeling any uplift or change in the P&L and broader business (outside of an increase in cost).

I don’t believe there is a way to get to perfection in this area. We can work on reducing the overlap though and understanding the core activities that drive business impact and those which support or overly take credit.

Why has incrementality become a topic of conversation?

You’ve no doubt had a scenario where revenue is reported like this:

Facebook Ads: £10,000

Google Ads: £10,000

Klaviyo: £10,000

Shopify actual revenue: £20,000

As digital marketing has advanced and cookie based measurement has become the norm, we find ourselves in this sort of scenario whereby each of these channels has interacted with a user on the journey from first site visit to customer, so each channel credits itself with the full sale and revenue.

As a business though we only transacted one customer and that’s the source of truth.

You can have your team or partners jumping up and down about the 10x ROAS they’re delivering for you, posting screenshots on Linkedin and celebrating like they’ve just won an F1 race.

You’re left with a P&L at the end of the month though which looks a little more sorry for itself and a sales team which aren’t running off their feet answering the phones.

Of course every advertising platform wants to showcase the maximum value that it provides as it’s in their own interest to show you the ROI you’re getting from their service.

I’m optimistic in these scenarios that the platforms themselves, Google, Meta, Klaviyo aren’t outwardly trying to inflate numbers. It’s our interpretation of the data that just needs to evolve in 2025.

Your advertising agency and other marketing partners may not be aware themselves so it’s good to be in the know.

Common areas to review

There are areas where the highest percentage of overlap happens and as such we see the lowest incremental impact on the business. We can identify these areas by looking at the attribution type in the channel and also where the channel interacts in the customer journey.

Attribution types

The biggest inflation of performance which doesn’t drive incremental business impact is what’s called view-through (or open in the case of email marketing) conversions.

View-through / open attribution

In a channel like Meta, a full sale and revenue can be counted against your campaign if an ad is served to the user, not clicked but the user then buys within 7 days.

This is very problematic as the ad being served doesn’t mean that the user actually saw it. Took in the copy, content, took influence or anything else.

This is especially more problematic when you haven’t excluded website visitors from your campaign. It’s standard buying behaviour to visit a website for the first time and then open other tabs and come back to buy later.

Showing an ad to this person, them not clicking on it and then buying is unlikely to be driving incremental business impact.

This isn’t just Meta, it applies to any type of display based advertising. Amazon DSP, Google Display, YouTube, TikTok and more.

We have the same scenario in Klaviyo and other email platforms. A user can open but not click any links in an email, within 5 days if they come and make a purchase, this can be attributed as a full sale.

Depending on where this email occurred in the customer journey, you can start to see where there’s more likelihood for double counting. A welcome flow pop-up email that’s opened but not clicked and then a customer transacts, is in that same standard consideration period so is likely to have come to buy anyway.

Solution: Whilst you can’t rule out fully that the ad didn’t impact any users, you can see your performance split by view and click attribution. You want to see that your spend is weighted toward those who saw an ad, clicked it and then took an action directly.

On Meta you can now optimise for click attributed sales whilst still seeing the split on view and click. This is advisable.

When spend is scaled on click attributed sales, you will see the store uplift and P&L follow suit.

Ensuring that remarketing lists are excluded will avoid overlap of those who have visited and are now just in the consideration period.

Review and adjust your email attribution settings, If they are on a five day open, you need to discuss if someone is likely to have recalled an email they opened more than a day ago enough to fairly allocate a sale against that marketing touchpoint.

If there’s a question around is it impacting brand awareness, you can look for correlations in your brand search in Google and Microsoft keyword planner to see if it’s driving an uplift. If there’s no business impact and no measurable brand uplift then you need to consider this likely isn’t adding real business impact.

Brand search in pMax

Another common issue we see with ads incrementality is brand search being captured in pMax campaigns.

pMax is a campaign format which is there to primarily use shopping ads to drive new customer acquisition for you.

What can happen if a brand exclusion isn’t in place is that you will start showing against searches for your brand name. These are users who already know who you are, either because they’ve bought before or another marketing touchpoint has brought them to the website and they’re now coming back to close.

The issue for ads is that if brand isn’t excluded, your campaign level ROAS may look on target, however, the high ROAS/conversion rate of the brand terms is the thing weighting up the actual performance which is under where it needs to be.

As you spend more 80% of the budget is going onto sales which aren’t profitable, a small amount is making the campaign look like it’s on target so the underperformance flies under the radar.

Sometimes bidding on brand in Google Ads is necessary. If competitors are hijacking those super high intent users who are looking for you, then you need to block them out.

This has to be in its own campaign though so you can control the budget you spend on it, ensure that you aren’t overpaying for clicks and then most importantly see a true picture of your generic product or service term performance.

Solution: Add a brand exclusion list to your pMax campaigns, ensure that brand is excluded as a negative keyword list from search and shopping.

Run a separate brand campaign if you have competitors running activity on your term so that you can see reporting segmented, bid for these terms in a controlled way and manage budgets.

Affiliate marketing

Running activity on channels such as Impact or AWIN has many benefits, you’re only paying a commission when someone has bought from you which paired with the reach of some partners can be a significant opportunity.

We have another dynamic here though in terms of customer journey.

A large number of these partners work by offering a discount. With lots of backlinks and a high domain authority coupon sites rank instantly and on page one for your brand name + discount code.

This can mean that your hard won customers from core channels like paid search, organic social and search etc can get hijacked when they’re ready to buy when they search in a new tab for brand + discount code.

The channel again will attribute a full sale and revenue but it’s unlikely to move the dial in the business.

You’ve also devalued your brand here with coupon codes which over the long term can have a more detrimental impact.

It’s not all doom and gloom though. There are some huge reach content sites and partners which have traffic which you wouldn’t otherwise access. Being selective in partners is essential to making this channel add real value.

Solution: Be critical and selective about who you on-board to your affiliate programme. Coupon code sites might have mass reach but are you just giving away margin to those who would have bought anyway?

Make a page on your website which isn’t in the navigation which contains your welcome offer. This will ensure that you rank organically number one for brand + discount code which can pull sales attribution into an existing channel and away from one where you will pay platform fee, commission and give a discount.

Playing devil’s advocate

To be unbiased, let's just flip what we’ve said.

Yes there are brand benefits to display advertising activity which across a longer period of time shouldn’t be measured on shorter term commercial measures like ROAS.

You can’t prove in a negative that someone who got served an ad impression and then took influence and wouldn’t have otherwise bought. We also can’t measure long term brand impact or value in less performance based activities. This doesn’t make it a 50/50 likelihood though. It’s far more 90%+ unlikely.

We have to be reason based with where we invest month-to-month especially for SME.

The ideal in between is that our click based, first touch, new customer acquisition is profitable. If it isn't, let's figure out the hard problems which are blocking it from being so. Usually that’s conversion rate optimization, a deep focus on user experience and proposition.

This way we know that there is a huge amount of incremental revenue, sales, leads in addition to what’s reported on the platform.

Importantly though we’re not burning cash on pie in the sky silver bullet activity which looks too good to be true!

Summary

When you scale spend on a generic product or service term campaign in Google Ads which is profitably returning, you will see the correlation directly in the P&L and feel it in the business.

When this happens you are also driving new customers into your email list, referral programme, to your social channels and much more.

The incremental real impact of generic product or service campaigns in Google Ads or cold targeted click attributed campaigns in paid social, adds far more value than just the numbers they deliver which in themselves are true commercial value.

Email marketing and referral schemes can’t do what they do best without something feeding the pipeline on the front end.

The thing to note here with Shopping or search based ads is that they are click attributed by default so whilst there can still be multiple touchpoints on a customer journey, there’s far less overlap.

Subjectively, you know at least that the user was relevant, they were presented with an ad, clicked it, came to the website and then made a purchase. There is a lot less speculation around impact here as we know that the user took action, that’s not always the case with view or open attributed activity.

We can use the check and balance that was given to us by Christopher Hitchens:

“Extraordinary claims, require extraordinary evidence”