This executive understands how to use research

One of my favorite podcasts, movie-biz junkie that I am, is The Business, a half-hour program airing on Santa Monica, Calif., public radio station KCRW that features fascinating, long-form interviews with entertainment-industry types, often those who are behind-the-scenes.

A recent episode contained an interview with John Landgraf, president of the FX TV network. Now, Landgraf is already a personal hero of sorts for giving my beloved Louis CK the freedom and the money to make one of the best shows on television. (Word of warning: his humor is definitely not for everyone, so if you’re easily offended, save us both a hassle and don’t click on the link.) Of course, he also is helping Charlie Sheen resuscitate his career – which proves that his taste is exquisite but not impeccable.

Anyway, during his interview on The Business, Landgraf spelled out his views on research in general and focus groups in particular, in response to host Kim Masters asking him about the veracity of an anecdote from a New York Times article in which he pooh-poohed focus group reactions to an early Louis episode. Quoting from the Times:

After a focus group agreed that the death of a dog at the episode’s end was a “bummer,” Louis C. K. recalled, Mr. Landgraf told him, “Here’s the strongest data we’ve got, and as the network president, I’m urging you to ignore it.”

Landgraf confirmed that the story was true and offered a view of focus groups that at first sounded like an echo of Steve Jobs:

“People don’t always know what they want. That’s the problem with research and with focus groups. What group of people could tell you, ‘What we really want is Avatar or South Park or The Simpsons.’? Those things didn’t exist in anything like that form before they [were created].”

My heart sank as I listened – here was another blinkered, generalized slamming of the qualitative process. But seconds later Landgraf redeemed himself:

“People can’t always tell you what they want and they are often confused by really innovative work – until they’re not. On the other hand, they can definitely tell you what works for them and what doesn’t and why. Whether or not they know the language of filmmaking or of writing, they’re very articulate about their own feelings and their own experiences.

“And so I not only do research, I actually watch the research. I don’t read somebody’s summary of how people reacted. I sit, oftentimes with our executives, and I actually watch [through the one-way mirror] or we stream them on the Internet if they are taking place in a remote town. And sometimes they clarify or sharpen my understanding of the work. Because what happens is, we get too close to it, once we’ve seen it and once we’ve provided notes [to the creators]. [The focus groups] provide objectivity.”

In other words, it’s all in how you use them. I know focus groups are not perfect. And I agree that they shouldn’t be used to test truly groundbreaking product ideas or television shows. People really don’t know what they want or how to react when shown or told about something that they can’t easily wrap their brains around.

But, “the focus groups hated it” is not a good reason to make a go/no-go decision. Data from any kind of research – qualitative, quantitative, social media listening – should never be taken on its own. All forms of consumer inquiry are complementary. Analyzed together, they can clarify and sharpen, as Landgraf said, and be interwoven to form a more complete picture. Making decisions based on information gathered from one form of research is like guessing what an entire jigsaw puzzle looks like based on a single piece.





This entry was posted in Concept Research, Focus Groups, Market Research Best Practices, Media Research, Qualitative Research, Quantitative Research. Bookmark the permalink.

2 Responses to This executive understands how to use research

  1. Bill Tanner says:

    Often, people can articulate outcomes they want. They may not know how to best achieve the outcomes but just knowing what they want [faster, more convenient, sweeter, shinier, brighter, simpler, etc.] can stimulate invention/ keep projects on the right track. They can also at times judge solutions … not always directly but just by talking about the solutions, reveal undiscovered aspects of them that lead to even better solutions.

    BTW Have had many research companies tell me that Apple buys a lot of research. Steve Jobs also did a lot of his own .. talking to Apple store customers.

    • Jenny Irons says:

      I have spent the last twenty years listening to people tell me what they think they think and for about nineteen of those years it has been quite apparent to me that they can only tell us “what” and “how” but really very few can tell us “Why”….that is our job as qualitative researchers – to observe, listen, question and give meaning and context around the “Why” in order to tell our clients “what next”.