Validating incrementality in baseline+lift models for linear TV

TV Ad Measurement

Smart advertisers will reach their customers through a (wide) variety of channels: digital, radio, TV, mail, catalog, out of home, viral (aka word of mouth), etc. As people get introduced to a brand from all angles, measuring the contribution from a single marketing channel becomes increasingly challenging. To deal with this, marketers have mostly fallen back to multi-touch attribution models i.e. each channel will get partial credit, often by applying complex (borderline artsy) formulae.

Testing TV Ad Measurement Through Incrementality

Incrementality refers to the notion that a TV ad will only get credit or attribution IF it was the sole contributor to a visit, install or sale. It is somewhat “unfair and harsh” (i.e. if a customer is exposed to a TV and a Facebook ad, the latter will get all the credit); however, industry pundits will agree that incrementality is a stronger approach to identify campaign optimizations.

Incrementality is typically measured in one-off (and sometimes expensive) test-control experiments. At Tatari, we do this continuously (without the need for one-off tests), as first covered in AdExchanger (we strongly recommend reading this article first). In this blog post, we will put Tatari’s approach for incrementality in linear TV (dynamic baselining) to the test by comparing it with a pure test-control experiment. We will do so using an example of a Tatari client which is a design and commerce retailer.

Our test market consisted of three DMAs: Denver, Raleigh, and Austin. This choice was, amongst other things, part based on the client’s strong correlation with overall national sales and geographical dispersion. The same was done for the control market. 

The experiment itself ran over four (consecutive) weeks, with an identical media-buy in each test market. The results (summarized below) suggested 20% incremental new-to-site (or first session) visits in the test market.

Before we could compare the resulting Cost-per-Visitor and Cost-per-Acquisition (learn more about measuring conversion from TV advertising) against the client’s performance as measured by Tatari’s dynamic baselining methodology (in dashboard), we needed to adjust for CPM differences. The local CPM in this experiment was 3.5x higher than the historical national CPM.

The comparison results are summarized below. The Cost-per-Visitor (CPV) and Cost-per-Acquisition (CPA) derived from the incrementality test ($7.79 and $488 respectively) line up well with the weekly values for each presented in Tatari’s dashboard.

Testing and experimenting are always good, but can be expensive, whether from a setup (e.g. high local CPMs) or opportunity cost (e.g. paused national campaign) perspective. TV advertisers who do not have this luxury of time or cost (not to mention patience) can, however, take great comfort in Tatari’s dynamic baselining methodology (i.e. as shown above, the numbers line up).  And there’s always an inbetween: while this test was “pure” (or about as well-constructed as one can for linear TV), alternative approaches exist. Feel free to contact us to discuss.

Last but not least: let us not forget about incrementality in streaming TV. There is a much higher overlap between digital and streaming TV eyeballs. Therefore, as brands tend to max out their digital campaigns, view-through measurement for CTV/OTT (as widely applied in our industry) tends to inflate the results (or: make them look better than what they really are). At Tatari, we have always offered both view-through and incremental TV measurement, whether streaming or linear. As such, our clients have the ability to select the right metric depending on the type of measurement required (and interchange).