Tuesday, March 31, 2009

A Basic Question All Clients Wonder And Most Never Ask

I am counting on plenty of good discussions around the Q&A below. And the Q&A should be a straight forward one to explain and understand. But it isn't. No one I have spoken to has been able to convince (or should I say enlighten) me about whether or not qualitative research can ever be representative.

So late last week I fired off an email to my good friend, John Griffiths, who I always contact when I want to know what my opinion should be on any given subject. His answer follows, and I hope your comments will follow to.

Q: "Hi John, hope you are well. I was wondering if you could help me with something. In both Brazil and Dubai we are being asked the same question by inexperienced clients (potential clients): 'how representative is qualitative research?' Now I know and you know that it isn't. I was wondering if you had a set piece response we could use? An explanation which is more than an apology - I usually talk about how we try to understand mechanisms and triggers rather than how many times who does what...

Any thoughts/ideas would be greatly appreciated."

A: "ooh good question - there must be papers on qualitative representativeness as opposed to quantitative representativeness. The point as you are saying is that qualitative representativeness is of a different order. Its not just quantitative representiveness with not enough people. WE know that over a certain number of interviews/ groups that findings stabilise because the models of thinking and feeling start to repeat at well below the levels of quantiative validation.

here's a def of qual I used with the Debenham's team last year: A disciplined approach to gathering and analysing information using a repertoire of open ended interviewing techniques and formal and informal interviewing methods

And here's the quant by comparison: Quantitative research supplies a number to anything that can be measured. And there is a large body of researchers who argue that measurement can be applied to anything. It produces ‘hard’ data that is more than just opinion. Based on sizeable surveys. However it cannot stand alone as a technique. It is often appropriate to gauge people’s opinions first using qualitative techniques before determining exactly what should be measured

Here's the point. Until you have assessed the quality of what you are counting there is little point in counting it. Qualitative research is the identification of those units before you then count them.

Without the qualiatitive part you have no idea what you are counting. Counting heads isnt good enough if you don't understand what those heads are thinking and feeling. And that can only be done using qualitative assessment.

Another way of talking about it is that knowledge starts with the unthought unspoken. And is created when it is captured and made explicit. You cannot count it until you have made it explicit. Some one has to turn behaviours and attitudes into concepts and words before we can aggregate and weight them. The problem with quant oriented clients is that they think the currency exists. So all we have to do is practice accountancy. In reality we have to invent the currency first. Then we count it."

Looking forward to your comments.


  1. Personally, I think that qual really gets at issues in a way that quant rarely does. If you want to know how many people buy x toothpaste versus y, or which one of six reasons is the most popular, you use quant. But if you really want to get into why people feel loyal to x toothpaste, how they fit toothpaste into their lives, really get into the meat of their toothpaste buying then you need qual. I feel quant can miss a lot, because of the mass of data and the lack of options (people chose the closest option, not necessarily what the would say).

    The marketing research classes that I've taken usually say that qual is for prelim research and to examine particular subgroups (say, motorbike riders) or a super loyal fanbase.

  2. Love the currency metaphor. I’d build on it by saying that qualitative is the only way to get at the essence of something. In this way, it can be more “representative” because it is reflecting true substance, rather than simply measuring dimensions. I spent 5 years developing best practices for the Clorox Company on getting to the big, strategic insights. For brand positioning, retail activation and product development, the most essential input is good ethnography. What good ethnography delivers is an understanding of the meaning behind our behaviors. And this meaning (within the frame of culture) is universal and therefore “representative”. What most companies miss is the distinction between discovery research and validation research. For discovery, qualitative is really the only way to go. Lori Wahl, BigInsightz

  3. Great comments. So to continue the build...
    I love the use of "representativeness" compared to projectability. What qual does is help to represent human behavior, needs, motivations, desires, etc. the kind of stuff that leads to product innovation, communication and branding.
    Here paying homage to the outlier is important. Quant usually dismisses the the outliers but qual can learn from them. How is someone adapting product for their own use? Example is the camera phone. When it first came out, marketers weren't sure of the value but the audience quickly taught the manufacturers applications and attachments not previously imagined.
    In terms of projectability, an issue is how homogeneous is the audience being observed? If completely homogeneous, then you only need to interview one person. But that is never the case. So John Griffiths comments about starting with the "unthought unspoken" (Love that!) also apply in helping to understand how wide is the range of diversity. -- dorothy deasy, www.ddeasy.com