One of my favorite podcasts, movie-biz junkie that I am, is The Business, a half-hour program airing on Santa Monica, Calif., public radio station KCRW that features fascinating, long-form interviews with entertainment-industry types, often those who are behind-the-scenes.
A recent episode contained an interview with John Landgraf, president of the FX TV network. Now, Landgraf is already a personal hero of sorts for giving my beloved Louis CK the freedom and the money to make one of the best shows on television. (Word of warning: his humor is definitely not for everyone, so if you’re easily offended, save us both a hassle and don’t click on the link.) Of course, he also is helping Charlie Sheen resuscitate his career – which proves that his taste is exquisite but not impeccable.
Anyway, during his interview on The Business, Landgraf spelled out his views on research in general and focus groups in particular, in response to host Kim Masters asking him about the veracity of an anecdote from a New York Times article in which he pooh-poohed focus group reactions to an early Louis episode. Quoting from the Times:
After a focus group agreed that the death of a dog at the episode’s end was a “bummer,” Louis C. K. recalled, Mr. Landgraf told him, “Here’s the strongest data we’ve got, and as the network president, I’m urging you to ignore it.”
Landgraf confirmed that the story was true and offered a view of focus groups that at first sounded like an echo of Steve Jobs:
“People don’t always know what they want. That’s the problem with research and with focus groups. What group of people could tell you, ‘What we really want is Avatar or South Park or The Simpsons.’? Those things didn’t exist in anything like that form before they [were created].”
My heart sank as I listened – here was another blinkered, generalized slamming of the qualitative process. But seconds later Landgraf redeemed himself:
“People can’t always tell you what they want and they are often confused by really innovative work – until they’re not. On the other hand, they can definitely tell you what works for them and what doesn’t and why. Whether or not they know the language of filmmaking or of writing, they’re very articulate about their own feelings and their own experiences.
“And so I not only do research, I actually watch the research. I don’t read somebody’s summary of how people reacted. I sit, oftentimes with our executives, and I actually watch [through the one-way mirror] or we stream them on the Internet if they are taking place in a remote town. And sometimes they clarify or sharpen my understanding of the work. Because what happens is, we get too close to it, once we’ve seen it and once we’ve provided notes [to the creators]. [The focus groups] provide objectivity.”
In other words, it’s all in how you use them. I know focus groups are not perfect. And I agree that they shouldn’t be used to test truly groundbreaking product ideas or television shows. People really don’t know what they want or how to react when shown or told about something that they can’t easily wrap their brains around.
But, “the focus groups hated it” is not a good reason to make a go/no-go decision. Data from any kind of research – qualitative, quantitative, social media listening – should never be taken on its own. All forms of consumer inquiry are complementary. Analyzed together, they can clarify and sharpen, as Landgraf said, and be interwoven to form a more complete picture. Making decisions based on information gathered from one form of research is like guessing what an entire jigsaw puzzle looks like based on a single piece.