Going dark: Why pausing TV doesn’t always prove or disprove its results

Going dark on TV refers to an advertiser temporarily pausing its TV campaigns.  There is often good reason for this e.g. consumer demand outstrips inventory availability.  Sometimes, however, TV advertising is put on hold as a means for a company to gain certainty about the ROI of its TV dollars.  By going dark, the company expects a falloff in demand and in that, discern the value of TV from the reduced orders.

Going dark on TV is not always a good strategy. Logic arguments aside, we will demonstrate this with some real data.

In this particular case study, our client had just concluded a $220K pilot campaign. Whilst this investment yielded valuable insights (e.g. around creative and networks) and a good response, the conversion to a sale (from TV responders) was questionable. Sales from TV, plotted in green, were not as strong as hoped.

Based on data from a (well-constructed!) survey during the pilot period, our client measured TV conversion to be 1.7%.

Tatari, on the other hand, calculated conversion to be more in the 3% range (see chart below). One of Tatari’s approaches to measuring TV conversion was laid out earlier; could it be that wrong? Regardless, the decision was made to put TV advertising on hold as the company found better (and more immediate) success on some other marketing channels. 

Remarkably, once TV advertising was halted in December, sales credited to TV kept coming in (i.e. the green line going up in the absence of TV!). There was an additional tiny bump in mid-February from a small Hulu campaign. At the time of writing, the same survey now suggested that conversion from TV was more like 2.5%. And clearly, the end was not yet in sight. Assuming a steady or gradual tapering off over the next 3-4 months, we expect conversion (from the survey) to clock in at 3.3%, which is precisely what we had originally measured in the dashboard. 

The astute reader may correctly point out that the survey results may not be accurate (after all, it relies on human memory and honesty). In this case, however, the construct of the survey was particularly strong, and we were able to further back up the conversion results with an IP-level analysis (i.e. matching the IP addresses of the impressions to the IP address of buyers, fully anonymously).

We see this scenario happen quite often with first-time TV advertisers, who often have good intentions in making sure TV advertising is working for them. The truth is, when going dark on TV, sales only partially soften right away and marketers may incorrectly conclude that TV wasn’t a great investment. Only after considerable time will sales start to soften significantly (or outright collapse). At Tatari we believe easily six to nine months; according to Kantar, a six-month absence in TV advertising could result in a 39% reduction in total brand awareness.  

And then the reverse process begins: revamping spend doesn’t make sales snapback immediately. The impact from TV, whether on the way up or down, goes beyond immediate and delayed response. There can also be a significant halo effect that confounds all of the above. Halo refers to people unconsciously recognizing the company (a.k.a. awareness) in other marketing channels, leading to stronger performance. For example, TV may drive stronger click-through on Facebook or higher quality scores on Google.

Shortly after the end of the pilot campaign, sales from other channels (as measured by the survey: friends, Instagram, blog, and search) increased from 50% to 200%. This did not stem from higher investment in those channels; it is either seasonality or television halo (and most likely a combination of both). In fact, TV generates brand awareness that persists and is actualized during high-seasonal periods. To slam dunk the case, note how friends (or word-of-mouth) goes up. As the old saying goes “money can’t buy you friends …”. 

Going dark is possibly one of the most expensive TV tests a company can pursue. Whilst it saves TV dollars, the opportunity cost in sales is (too) significant, especially considering the lag it takes, first on the way down, then again on the way up.