Measuring viewer engagement with television is one of my favorite topics, partly because it’s messy, and partly because it’s a sign that something really needs to change to make the economics behind the television ecosystem sustainable.  So, on Friday, when Optimedia U.S. released their third annual study, Content Power Ratings 2.0, that assesses how engaged people are with television shows, I knew I had to write something about the topic.

Although I think this type of thing is a step in the right direction when it comes to assessing the appeal of a program, I can’t help but wonder if we’re really measuring the right things.  Maybe people are “engaging” with these shows, but are they actually advocating and/or buying the brands that advertise on them more than the non-engaged viewers?

If that’s the real question we need to answer, then we might be measuring media engagement backwards.

Measurement transparency

One of the reasons it’s hard to assess what “engagement” really means in the context of TV is few organizations release details on ranking methodology.  A post on TV Decoder, the NY Times blog on television, says that the Optimedia study factors in viewership on TV, online, and mobile devices, as well as “buzz” and “advocacy”.  Although these are probably some of the right things to look at to assess engagement with a show, how you add them up makes a difference.

For example, how long a show runs in broadcast, as well as how readily available online and on mobile device platforms, and how much access the core audience has to those platforms, could have an impact.  There is also no mention of consumption via DVR, or DVD rentals or sales, or through P2P file sharing, which cold be significant in some cases.  Also, you could argue that driving out to buy a DVD, taking the time to find a show on Netflix, or wading through a P2P site to watch a poor quality copy say more about someone’s interest in a program than searching for it on hulu.

Then there’s the “buzz” and “advocacy” problem.  There are several companies that measure online audience buzz and “word of mouth”, but how to you measure it offline?  One of my friends, whenever we meet up,  encourages me to try Damages.  How do you account for that?  And how do you know which type of buzz or advocacy was actually effective to get people to watch and advocate the show to more people?

It’s hard to say much about methodology since we don’t have more details, but those are some potential issues, at least questions, I would ask the authors of the study.

But, to me,  the much more salient issue, when we’re looking at the economic viability of the current TV ecosystem is, are engaged viewers actually more likely than non-engaged viewers (or non-viewers) to buy or promote the products advertised on the show?

Advertising agency

There have been several studies on engagement that consistently seem to find that people who are “engaged” with a show tend to remember the names of the advertisers whose commercials are featured, and sometimes have a better overall impression of the brand.

The problem here is that there often isn’t a causal link, and there’s no way to get it from TV viewing alone, or even brand attitudes or recall around the show.  It’s one thing to ask an American Idol viewer if he or she remembers a Coke ad, and how they feel about Coke, quite another to link those good feelings to the crucial moment in a store when they choose or don’t choose a Coke over Pepsi.   We also don’t tend to know much about their purchase habits or behaviors after seeing the show.  Did American Idol lead them to buy a Coke even though they were regular Pepsi drinkers?  Did they suggest Coke to their friends?

Full reverse

What if we measured engagement the other way around, starting with that crucial moment in the store, seeing what Coke buyers know about American Idol, not what American Idol viewers feel about Coke?  What if we asked people who were standing, bottle of soda in hand in the grocery store (or survey recent Coke buyers online or over the phone) about how they made their choice, and if they recommended the brand to friends when they were making a choice?

Chances are, most people wouldn’t say “because Randy was drinking it on American Idol last night” – they might feel that response would make it look like they were “taken in” by product placement, or they may not even be aware that seeing Randy’s tumbler was a trigger for their choice.  However, we could also ask which programs they watch regularly, which platforms they use to watch them, how much they enjoy them, and if they encourage their friends to watch, to see if there’s a relationship between their purchase behaviors when it comes to an advertiser’s brand and how much they like and view a show.

So, approaching the question this way, we could also ask about how they advocate the advertiser’s brand and find out if people who watch a show are actually more regular, stronger or better champions of the products.  With that data, we could look at traditional TV measurement, whether it’s straight ratings or engagement or both, and work out the degree to which engagement with the program as we define it now actually translates into sales among the viewers and their networks.

If the current economics surrounding TV are to continue, engagement is ultimately about brands, not shows.  If not, the door is wide open.