Difficulties of measuring conversions on linear TV

In advertising, conversion is a step in the marketing funnel during which website (or app-store) visitors perform a desired action (e.g. they purchase a product, become subscribers, etc.). It is usually expressed a percentage. For example, a 5% conversion rate means that for every 100 website visits (or app downloads), 5 people will purchase a product. Because it directly shows how many people became customers as a result of being exposed to an ad, it allows for calculation of the customer acquisition cost (CAC), and every marketer therefore wants to measure it correctly.

When people are driven to a website or an app-store through an online channel (e.g search engine marketing or Facebook), measuring conversions is a relatively straightforward process. Because there is a online session associated with that channel, marketers can tell whether that specific session resulted in conversion. For instance, if a user was browsing the internet, clicked on an online ad, landed on the advertiser’s website, and then purchased a product, that means that the user converted during that session. Simply put, with online advertising channels, marketers can “trace” the clicks.

It is much trickier to measure conversions through offline mediums like linear TV and radio. TV ads, for example, also drive consumers to the internet by supplying a call-to-action (e.g. a website link in the end-tile), but if the consumers convert during that online session, there is no obvious way to tell whether this conversion was actually driven by TV. In other words, it is impossible to directly trace a purchase or subscription back to a TV ad.

As a result, measuring TV-driven conversions is an incredibly difficult process and is one of the more complex features of Tatari’s product. We fear, however, that not everyone in the industry is aware of all the details that need to be considered when dealing with this topic. This article therefore lays out the difficulties of measuring TV-driven conversions and shows the key aspects of our methodology. We hope it will help advertisers understand how their conversions are measured, and enable them to demand a similar standard of quality and rigor from their existing analytics partner.

Main difficulties of measuring TV-driven conversions

To best understand all the complexities of measuring TV-driven conversions, it is useful to first look at a hypothetical example.  Let’s say that the timeline of a conversion event looks like this:

(1) user is offline → (2) user views a TV ad → (3) user comes online and starts a session → (4) user converts

We can assess whether TV drove traffic to the advertiser’s website or app by measuring immediate lift using a baseline-lift model. For example, if we notice an increase in the number of visitors or app installs above the baseline few minutes after a TV ad aired, we can attribute the increase in traffic to the TV ad.

If the user converts, however, it is hard to immediately know whether this conversion was driven by TV due to two issues:

• Timeline of active and inactive sessions: A user might come to the advertiser’s website or app after the ad aired and immediately convert during the active session. But, what if they wait several hours, days, or even months? This can be further complicated if more ads are aired during these inactive sessions. In that case, how to tell which session drove the conversion event?

• Difference between TV and non-TV users: Naturally, some of the users who come to the advertiser’s website or app will not have seen the advertiser’s TV ad. For instance, they might have heard about the advertiser’s product through friends or on the radio. This means that not all users in the cohort can be attributed to TV. Therefore, it is important to develop a model that distinguishes TV conversions from non-TV conversions in these calculations.

Solving the timeline issues

To solve the first issue, Tatari attributes conversions only to those sessions that were initiated after five minutes of the ad airing. Furthermore, Tatari does not over-attribute conversions to the last active session if the user had multiple active sessions before converting.

Here are some examples, from simpler to more complex:

The first example is a simple scenario in which a user converted during an active session that was initiated within 5 minutes of a TV spot airing. Conversion is therefore assigned to that session. In the second example, a user converted during the second active session, which occurred five minutes within 5 minutes of a TV spot airing. Conversion is therefore assigned to the second session. Finally, in the third example, the active session was initiated before a TV spot aired, so the conversion that occurred at the end is not attributed to TV.

There were no timeout periods (inactive sessions) in these examples. But, what if it takes several days for the user to convert? In these cases, Tatari does not over-attribute to the last active session.

In the first case, despite the second active session during which the user converted, conversion is attributed to the active session that was initiated within 5 minutes of the TV spot airing. In the second case, conversion is again assigned to the first active session even though a second TV spot aired during the second active session during which the user converted. That’s because the second active session was initiated before the second TV spot aired.

Finally, what if the inactive session is really long (e.g. several weeks to months)? In that case, the conversion that occurred at the end of the last active session is not assigned to TV, as shown in the figure below.

These are only some examples that illustrate how intricate the process of measuring TV-driven conversions can be. In reality, there are dozens of distinct scenarios that need to be accounted for, and Tatari ensures to correctly classify each conversion event.

Solving the issue of differences between TV conversions and non-TV conversions

To distinguish TV conversions from non-TV conversions, Tatari looks at every minute-by-minute cohort of converting users. We assign a likelihood of coming from TV to each cohort based on an aggregated statistical lift approach. Furthermore, when there is enough evidence for such difference, we do not assume that the conversion rate for both populations is the same. Instead, our model adjusts for the possibility that, within each cohort, there might be users who convert after seeing a TV ad and users who convert but who did not see a TV ad. This adjustment is crucial, otherwise conversions might be over-attributed to TV. From there, the final conversion rate is calculated as the sum of all minute-by-minute cohort conversions.

The importance of underlying assumptions

As can be seen, measuring conversions from an offline medium like TV is a daunting process. Tatari’s methodology offers a robust way of doing this by accounting for the timeline of active and inactive sessions, as well as by distinguishing TV conversions from non-TV conversions. That said, it is important to understand that there are many ways of “correctly” calculating TV-driven conversions and that each methodology will largely depend on the different underlying assumptions. Tatari’s approach is primarily centered on ensuring that conversion events are neither under- nor over-attributed to TV, and that our numbers allow for apples-to-apples comparison to conversion rates from other marketing channels.