A lot of brands are still in the dark about the real impact of their ad spend, particularly on Connected TV (CTV). The problem is that old-school last-touch attribution models keep giving CTV credit for conversions that were going to happen anyway, which wastes a ton of money and leaves everyone clueless about what actually works. To get past correlation and find real lift, you have to run proper incrementality testing to isolate the causal effect of a campaign. So how do you actually do that?

The Attribution Illusion: Why Traditional Models Fail CTV

The CTV ad market is on track to blow past $30 billion in the US in 2026, based on eMarketer reports, but a huge chunk of that money is spent blind. Marketers are measuring success with last-touch attribution, which is the core of the problem. When someone sees a CTV ad and then downloads your app, last-touch gives 100% of the credit to the ad, completely ignoring the messy reality of the user’s journey and the fact many conversions are just organic.

Think about it: a user was already going to download your app. They see your CTV ad, then grab their phone, search the app store, and install. Last-touch says the CTV ad did all the work. But what if they were going to do that anyway? That’s the whole ballgame right there, separating what you caused from what just happened to happen. Without that clarity, you end up in meetings defending a budget with bogus numbers, and you have no idea how to actually scale the things that are genuinely working.

The Hidden Costs of Misattribution

I’ve seen it firsthand: clients will dump millions into CTV campaigns because the reports look amazing, but a real analysis shows their incremental return on ad spend (ROAS) is in the toilet. Wasting money this way also means you’re missing huge opportunities. Every dollar spent on a non-incremental CTV placement is a dollar you could have put into a channel that actually finds new customers, and that opportunity cost can absolutely tank your growth and hand an advantage to your competitors.

On top of that, skewed data makes it impossible to forecast or plan with any confidence. If you think your CTV ads are bringing in 100,000 conversions a month when only 30,000 are actually incremental, your whole growth model is a house of cards. This gap between reported metrics and real business impact is exactly what keeps marketing leaders up at night, because they’re the ones on the hook for results. It’s just a classic case of measuring the easy stuff instead of the important stuff.

The Failed Attempts: What Went Wrong First

Before getting to a good incrementality testing setup, most people (us included) stumble around a lot. Our first attempts with clients used methods that seemed smart but were just scientifically weak. The classic mistake is just comparing performance before and after a campaign. It’s simple, sure, but it’s also completely useless because it ignores everything else happening in the world, seasonality, what your competitors are doing, a new product launch, the economy. Any of those things could be driving the change you see, not your ads.

We also see people A/B testing creative variations thinking it’s an incrementality test, but it isn’t. All that tells you is which ad is better, not whether the campaign as a whole is bringing in new business you wouldn’t have gotten anyway. People try to use proxy metrics or historical data too, but the results are always a mess because the data’s too noisy. These early failures all taught us the same thing: if you don’t have a properly designed experiment, you’re just guessing.

I remember one client who tried a geo-test, just turning off their CTV campaigns in a few regions to see what happened. They wanted to compare the ‘off’ regions to the ‘on’ ones, but it was a disaster. The regions weren’t even remotely comparable in terms of baseline user behavior, market saturation, or competitor ads. The data that came back was garbage, wildly inconsistent numbers that told us nothing. That kind of chaotic approach is exactly why you need a structured, scientific method. It’s just not optional.

The Solution: Implementing Rigorous Incrementality Testing with AppsFlyer

So the right way to figure out your real ad impact is with structured incrementality testing, using a mobile measurement partner (MMP) like AppsFlyer to manage the process. The whole idea is to run a controlled experiment: you intentionally don’t show ads to a ‘holdout’ or ‘control’ group, while a similar ‘exposed’ group gets the ads. By comparing how the two groups behave, marketers can finally see the actual lift their campaign is creating.

Step 1: Define Clear Hypotheses and Success Metrics

First, the team has to be crystal clear about what it’s trying to prove. Are you trying to show that CTV ads drive new app installs? Or maybe that they boost in-app purchases from current users? A good hypothesis is specific, like: ‘Our new CTV campaign will drive 10% more app installs than we see from the control group.’ Then the team defines the primary success metric (app installs, first-time buys) and any secondary ones (retention, ARPU). Getting this straight from the start is what keeps the whole experiment on track.

Step 2: Design the Experiment: Holdout Groups and Segmentation

The make-or-break part of this whole thing is creating the control and exposed groups and making sure they’re statistically solid and actually comparable. This means working with your CTV platform partners to set up a holdout, where some platforms will let you segment an audience and just exclude a percentage from getting the ad. The absolute key here is that the groups have to be random and have similar baseline behaviors, if a control group was never going to convert in the first place, the whole test is a wash. That’s why we always analyze historical data on both groups to make sure they’re statistically similar *before* the test goes live.

With CTV, segmentation usually happens at the household or device ID level. Marketers might target specific device IDs with the campaign while holding back a random percentage of similar IDs as the control. It’s absolutely essential that the holdout group gets zero exposure to the test campaign, which is admittedly tough in the fragmented CTV world but it’s doable if your platform integrations are tight. We usually tell clients to aim for a holdout group that’s 10% to 20% of the target audience, though the right number depends on how big you think the lift will be and your total audience size. If the holdout is too small, your results won’t be statistically significant and you can’t conclude anything.

Step 3: Use AppsFlyer for Data Collection and Analysis

Once the test is running, AppsFlyer is what you use to collect and unify all the data. It tracks installs, in-app events, and the whole user journey for mobile apps. The trick is to get your experiment’s group assignments, who’s in the control vs. exposed group, into AppsFlyer. You can do this with custom attribution links, by passing parameters when an impression is tracked, or by just uploading segment data. Its deep linking and measurement tools are what let you connect an app install or a later event back to a specific user’s group assignment.

Inside the AppsFlyer dashboard, you can then slice your data by these experimental groups, which lets you directly compare the key metrics side-by-side. For instance, you can look at the organic install rate for the control group and compare it to the rate for the exposed group. After you account for any background noise, that difference is your incremental lift. Because AppsFlyer’s data is so granular, you can drill down even further by geo, device type, or creative to see exactly where your campaign is having the biggest incremental impact.

Step 4: Analyze Results and Calculate Incremental Lift

The analysis has to be statistically sound. The process involves comparing the conversion rates, like app installs per 1,000 users, between your exposed and control groups. The formula for lift is simple enough: (Exposed Group Conversion Rate - Control Group Conversion Rate) / Control Group Conversion Rate. But just seeing a difference isn’t good enough. It’s necessary to determine if that difference is statistically significant, which is where you calculate confidence intervals and p-values to make sure the lift you’re seeing isn’t just random luck. Any stats calculator or analytics platform can help with the math.

Don’t stop at conversion rates, either. You have to look at downstream metrics. Do the users you acquired incrementally from CTV have higher retention or LTV than your normal organic users? That kind of deeper analysis gives you the full picture of the campaign’s value. The goal is to acquire higher-quality installs, which are often more valuable in the long run.

Measurable Results: From Guesswork to Growth

Putting a real incrementality testing framework in place has produced some huge wins for our clients. A perfect example is a subscription streaming service that was getting squeezed on budget. Their own reports showed CTV as a rockstar channel, supposedly driving millions of installs every month. When we ran a proper test using this methodology, we found a very different story.

We ran a four-week incrementality test across their three biggest CTV platforms and found that almost 40% of the app installs being credited to CTV were not incremental at all, those people were going to install the app regardless. For the test, we held out 15% of the target audience using device IDs to make sure they saw no CTV ads. Comparing the install rates between the exposed and control groups gave us the true incremental lift, and the numbers were blunt: CTV was working, but it was way less efficient than their last-touch reports made it seem.

With this data in hand, the client finally made some sharp, informed decisions. They immediately shifted 15% of their CTV budget away from the junk platforms and into channels with proven incremental ROAS, and they also doubled down on the specific CTV publishers and creative that *did* show real lift. The result? A 22% jump in their total marketing ROAS the very next quarter, even though their total number of installs dipped slightly. This was all about optimizing every single dollar for incremental impact, which allowed them to keep investing in CTV and spend more effectively.

The improved ROAS was a great outcome, but the real win was the change in mindset. Their marketing team went from defensively justifying spend with correlation to proactively optimizing it with causation. That kind of insight builds huge trust with the CFO and the execs, and it gives the marketing team the confidence to experiment because they finally have a reliable way to measure what’s actually working.

If you want to understand the real value of your marketing in a channel as complex as CTV, you have to get past basic attribution. It’s time to run rigorous incrementality testing to find the genuine impact of your campaigns and make decisions that drive sustainable growth and give you a clear strategic direction. The money you put into a solid testing framework will pay for itself over and over in optimized spend.