All Market Research Is Wrong!

October 28, 2010 Digital Media Solutions

You're Doing It WrongCatchy headline, huh? Well, it’s not mine. It actually graces a delightful manifesto published by Faris Yakob, chief innovation officer at MDC Partners and the former chief technology strategist at McCann Erickson New York.

In his blog post, Faris presents two reasons that relying on self-reported information is at best inaccurate and, at worst, a complete waste of money.

The first is that we don’t know why we do what we do. According to him, most of our decisions are made at a subconscious level, making our explanations of those decisions no more accurate a depiction of our decision-making process than the shadows dancing on the wall in Plato’s Allegory of the Cave are an accurate depiction of reality.

The second reason is that “the gulf between claimed attitudes [and intentions] and actual behavior is vast.” Here, Faris particularly calls out focus groups as providing poor data by creating “utterly artificial data as a response to utterly artificial situations and social dynamics.” I don’t think any movie fans would be surprised by this. It’s no scientific measure, but if you consider the movies that have had their endings changed because of a focus group’s suggestions, and how well they were received, you can see this principal at work (points to Deep Blue Sea, though, which is my go-to film whenever I need background noise while working).

Faris points to some very interesting behavioral economics research and draws some examples from Philip Graves’ “Consumer.ology.”

  • preference for identical Nike running shoes increasing across the board in test due to the smell of flowers being introduced to the room;
  • taste tests indicating that people have a clear preference between identical drinks depending on context;
  • the fact that people perform the same tests differently after being subliminally exposed to different stimuli, like logos, but have no knowledge of how this influence as taken place

[people exposed to the Apple logo found more uses for an everyday object in creativity tests, people exposed to the Disney logo behaved more honestly than the control in subsequent tests];

  • and of course the obvious fact that, despite rigorous testing, 80% of new products fail, and for a product like Red Bull researchers concluded “that no other product had ever performed so poorly in consumer testing; the look, taste, and mouth-feel were regarded as “disgusting” and the idea that it “stimulates mind and body” didn’t persuade anyone the taste was worth tolerating.”

So, what’s the solution? Well, he presents a couple ideas, including relying more heavily on observed behavior and revealed preference information instead of emphasizing focus groups and survey results. (Though, in fairness, Ilya Vedrashko points out that those can be venues for discrete choice modeling, which may be applicable.)

About the Author

Digital Media Solutions

Founded by a team of lifelong athletes, Digital Media Solutions (DMS) is an industry leader in providing end-to-end customer acquisition solutions that help clients win in their business ventures and realize their marketing goals. The company’s set of proprietary assets and capabilities in the world of performance marketing and marketing technology allow clients to meticulously target and acquire the right customers. DMS relentlessly pursues flawless execution for top brands within highly complex and competitive industries including mortgage, education, insurance, consumer brands, careers and automotive.

Follow on Twitter More Content by Digital Media Solutions
Previous Article
LinkedIn Launches New Personalized Product Pages
LinkedIn Launches New Personalized Product Pages

Expanding upon the “Company Pages” feature recently implemented, where members are able to follow companies...

Next Article
Dotting the I & T of Pixel Tracking
Dotting the I & T of Pixel Tracking

A class-action lawsuit against Facebook in 2009 for its “Beacon” advertising program that was settled last ...


Get Free Marketing Tips in Your Inbox

Thank you!
Error - something went wrong!