Wednesday, December 17, 2008

Merry Christmas & a Happy New Year



I made a pledge that I would transparently and honestly feedback to you any ethnographic experiments we have conducted. Here is one we completed the other day. Using our panel. Asking 250 people to tell us about how they snacked. Whatever you might think about the process, please let me know, it demonstrates to me that qualitative and quantitative can come together even where ethnography is concerned.

Enjoy the clip. It's long-ish at nearly 10 minutes but well worth a watch over a cup of coffee (or tea). I would love to hear your thoughts. Share/Bookmark

9 comments:

  1. I liked the experiment. It may be less naturalistic, but it's more focus. Although the sample gets more robust i don't think it becomes "quantitative" or statistically representative, and i don't think that's a big problem.

    Even if Qualitative an Quantitative research can (and "should") come together, the approach is different, don't you think?

    If the cost is reduce and we achieved understanding, the client may appreciate the shift.
    glemusmtz@gmail.com

    ReplyDelete
  2. I appreciate the sample won't become statistically representative... how do we achieve that though? A larger sample size? My guess is most clients will want us to explore specific segments anyway.

    ReplyDelete
  3. First, let me say how much I love your blog and appreciate what you are doing for the "commercial ethnography" community. On this latest experiment, I think the use of a panel is great and I love how you finally got engagement in the study by supplying the right stimulus. BUT calling this a quantitative sample is taking it too far. I think you need to introduce more controls to ensure that you are getting consistency of response before you can call it quantitative. Also, I'm not sure why you want this sample to be quantitative. Put otherwise, is ethnography the right methodology to dimensionalize the size of / interaction between observations or should you be using survey techniques (which if done right, can be quite cheap and effective).

    ReplyDelete
  4. Pure ethnography falls over (with some clients) in two main ways: 1) They don't get qual research. And you will be amazed at how many large consumer facing organisations just don't get it. 2) clients don't get the point of understanding the importance of context. And before you say, 'it's your fault for not explaining it...' I have and money can b a huge barrier.

    So, film carefully targeted events, more of them, disentangled from a large sample. I think this is another way cutting the consumer cake.

    ReplyDelete
  5. Cutting the consumer cake differently by getting more ethnographic slices of roughly the same process totally makes sense. I still wouldn't call this a quantitative sample. I think it can mislead a client into think the research is projectable.
    p.s. This is Lori Wahl writing ... I'm still figuring out how to get my own "handles" linked to this web site vs. my husband's g-mail account. :)

    ReplyDelete
  6. Siamack, Quite fun and interesting. I grew up in quantitative, but over the years bridged into qualitative more and more to help me / my clients / their brands get a better feel for what was really driving some of the target events we were trying to understand.

    Relevant to this point, I once added a set of laddering questions to a quant survey and got back 225 or so usable responses ... in contrast to the 25 or so we normally gathered. The result was that I was able to determine, at a far more granular level, what was driving the DIY activity we were trying to understand. Based on prior research, we believed "pride," "price" and "control" were the three main drivers of the DIY activity, but with the large sample I determined that there was actually a fourth, more rare driver, and within "pride" alone, 12 different pathways (ladders) emerged. As a result, we were able to develop communications and messaging that nailed the feelings of many different groups within the target audience base.

    So to get back to the topic at hand, keep at it! The line between qualitative and quantitative is imaginary, and with some quality controls in place, there is no reason consumers cannot self capture target events ... after all, there are still some events we cannot be there to witness (or they do not want us there to witness), and doing this would help lessen some of the concerns we / our clients might have regarding the impact of local market variances / variables on what we are observing.

    Feel free to post this on your site if you'd like.

    Cheers!
    John Thomas
    Managing Partner
    H. I. Thomas Consulting Group, LLC

    ReplyDelete
  7. It has been really great to have the constant stream from you – even telling about the disappointing results but then hearing how you tried again and were able to get the response you were looking for when the film enthusiasts had some stimuli to react to. Wonderful! I have always found this to be true with consumer research. When we did the project with you a few years ago (looking at the different shopping channels)- it worked well to get the panelists to do a task before we identified the individuals to go and film. So- that is really not new, but using the film as stimuli really was a new and wonderful addition to the process. The one person who filmed themselves did not do too bad of a job- you can always ask them to do another from a different angle or to zoom in on something you were not able to see (or just have them talk about it like in your co-discovery phase).
    My emphasis these days is Authenticty- your work provides plenty of that!
    Hope your holiday season is full of Joy-
    Carolyn Grant
    Information & Knowledge Design, P&G

    ReplyDelete
  8. Dear Siamack
    I've been watching the ethnosnacker series with real interest - and great to see people getting engaged and sometimes exercised. I've been a passenger, I know - my parents health issues took a turn for the worse over the summer.
    Just recently I watched the real versus fake ethnography, and re-read some of the ethography without theory is nothing, and then today I looked at the new hybrid ethnography or whatever we might call it. And your questions are, as ever, rather challenging. So instead of doing the things I'm supposed to be doing (which include tying down my Oxford outline), I'm reflecting.

    I like a good theory - bound to in my job really - but I still think we can find ourselves up our own backsides. For market research, theory is surely supposed to ensure that we get representative and reliable results. Yet ethnography in its original format (go and live with the other for a year type idea) is essentially unreliable (even by virtue of perspective even before you get to possible consequences of enactment) and in its form used today, unrepresentative by design - it is not about representative samples of people although perhaps it aims to be about representative behaviour. Seems to me we are in the essence of the innovators dilemma - the generation of a good idea for something new. Ethnography offers the chance for this to be generated by immersion in customer behaviour (something acadmic studies tell us is particularly helpful), rather than by, say, idea generation techniques where creative people dream up possible ideas - perhaps then to be floated past possible target customers, or by good ideas I saw on holiday, came up with in the shower, thought of while I was talking to my mother-in-law, or, indeed by customers. So where does the new hybrid fit? Well, I think ethnography has all its faults on the outside (they can be easily seen compared with other market research techniques which superficially look more rigorous) so possible problems of self selection seem more of an issue there (and here). But I think we are fooling ourselves if we believed that is not always the case. Then there is the priming - with the question and the film. That definitely frames the response. But it sounded from your categories as if a number of respondents had thrown off the framing of the Mars bar film, and used it only as a prompt for the question. Perhaps two different examples of film clips would have reduced the framing effect, if this is a concern. Are the responses valid - well it is possible that people are inventing things but I doubt if they are inventing more than they might in response to a Q&A survey (more work here). But is it representative (a goal of quant research) and so we can generalise from it. That's more heroic. But I think it is more generalisable than more usual ethnography. Is it new? New to me. Will EDL commit Seppuku introducing it? I'd be careful (actually a bit worried) that people may think they are getting the best of both worlds where I think they are getting a good compromise between them. So it could cannibalise the other work where this might provide a better result. What and who is it good for? I can see this in the idea generation space for new consumer products (and maybe services - but this might be harder) where usage is explored - sort of lead user research for consumer products, with the ability to get a feel for the likely size of response.

    Be a good conversation this (it's turning into a book as it stands) - if I was remotely competent with my computer I'd have filmed this response - oh well.

    I was going to ask you something about the session I hope you are still happy to do on my marketing innovation class. We talked about taking the full session (2*1.5 hours) and I have in my head the idea that we might do some discovery. But I think you had an idea for live client work and I'm not sure it was that. Are we still on for it - whatever it is? I'm sure it will be fun and I know they will learn so I think we should keep it suitable for the 'new to ethnography' but otherwise do what interests us.

    Be in touch in the New Year? Meanwhile have a good break (you do take time off don't you?)
    Vey best wishes
    Laura Cousins
    Oxford University SIAD

    ReplyDelete
  9. This comment has been removed by a blog administrator.

    ReplyDelete