Awareness is Everything

Monday, September 21, 2009

Social Media and Market Research

Earlier this month Brad Bortner posed this question in The Forrester Blog For B2B Market Research Professionals:
"Are MROCs [Market Research Online Communities] the next big thing in market research, and will they eventually take measurable share from traditional qualitative research?"
Sentient Services has been working in online qualitative for a few years now through asynchronous bulletin board focus groups. While you give up a lot in moving away from a face-to-face interaction (body language, vocal intonation, etc.), in an asynchronous online group you have a lot of different strengths.
  1. Less time restraint – respondents have more time to think, they can look up notes and do “homework” assignments. Additionally, we can let side conversations go and see if the tangent provides additional insight
  2. Broader coverage – asynchronous participation means that respondents aren’t locked into 6-8pm ET, making time zones a non-issue. This translates to breaking down some geographic boundaries.
  3. Bigger groups – we’re not limited to the capacity of a conference room, meaning that we “seat” at least 12 participants per group (vs. 6-8 participants in a traditional group).
While we’ve previously scrutinized sample goodness when using social networks for market research and the value of polling features in LinkedIn, I believe MROCs (private online communities focused on research) are just an extension of online methodologies we already see. Instead of recruiting participants to one online discussion or survey, they are being recruited for continuing feedback on a variety of topics. And MROCs will impact both qualitative and quantitative research – it’s just as easy to host a survey in an online community as it is a forum discussion.

What are your thoughts on MROCs? What other evolutions do you foresee in the research industry?

Labels: , , , , ,

diigo it

Friday, February 13, 2009

Google and Eye-Tracking

A post from the Official Google Blog caught my eye last week: Eye-tracking Studies: more than meets the eye.

As Google points out, most people are not conscious of their eye movement, especially when doing something as mundane as a web search. Eye-tracking data lets you identify which elements of a webpage (or other stimulus) are viewed, and in what order. Just as importantly, you can identify elements that are not viewed - which may be the reason why task completion, ad recall or messaging breaks down.

In my opinion, eye-tracking is most powerful when it is combined with traditional think-aloud usability protocol. At Sentient, we do this with a little bit of a twist - first we start by allowing the user to complete a series of tasks without interruption from us to capture task completion and eye-tracking data without interference from trying to hold a conversation as well. Then we have the user walk us through what they were thinking and doing in a qualitative debrief.

By delving into a qualitative debrief after a user completes a task while their eyes are tracked we can learn the why behind what they did. For example:
  1. Did they linger on an element because it intrigued them or confused them?
  2. Why did they look at one navigation element, but then move to other navigation elements and click on them?

By adding eye-tracking to the usability arsenal, you get a rich interaction between the quantitative eye-tracking metrics and the qualitative insights derived from traditional usability methods.

Labels: , , ,

diigo it

Thursday, January 8, 2009

Stats, Media and a new book

I was listening to XM Public Radio with Bod Edwards this morning and heard a fascinating interview with Joel Best. Joel is a professor at the University of Deleware and just published a book titled "Stat-Spotting: A Field Guide to Identifying Dubious Data". A timely book if there ever was one - on the heels of an election with data thrown everywhere, financial markets that are either melting down or poised to rebound right about now depending upon the numbers you read that day and of course all of the numbers released daily on growth/no growth by industry/sector and just about any other segment one might be interested in. And, of course, market research and all of our clients and colleagues that deal with data, its interpretation and implications.

The book is available on and here is the description from, that summarizes it nicely:

Are four million women really battered to death by their husbands or boyfriends each year? Does a young person commit suicide every thirteen minutes in the United States? Is methamphetamine our number one drug problem today? Alarming statistics bombard our daily lives, appearing in the news, on the Web, seemingly everywhere. But all too often, even the most respected publications present numbers that are miscalculated, misinterpreted, hyped, or simply misleading. Following on the heels of his highly acclaimed Damned Lies and Statistics and More Damned Lies and Statistics, Joel Best now offers this practical field guide to help everyone identify questionable statistics. Entertaining, informative, and concise, Stat-Spotting is essential reading for people who want to be more savvy and critical consumers of news and information.

Happy New Year!

Labels: ,

diigo it

Monday, December 1, 2008

Usability Survival Kit - Be prepared

We all know the scout motto: be prepared. And having grown up in Girl Scouts, I like to think that I stay prepared for whatever life might send my way. For example, when traveling to conduct usability sessions, I always drag a copy of the project folder from the server to the local drive on my laptop. I also bring hard copies of critical documents for the study.

However, during usability sessions last month, life threw me a curveball I wasn’t ready for: The power went out to the entire building. There were no lights. No microphones. No recording. My laptop had limited battery supply, and there was no internet because the routers had no power.

Luckily this happened on the first day of a two day study, and we were able to reschedule the remaining participants for the next day. We had a marathon second day, but we successfully completed our study.

Having gone through this experience, I’ve compiled a “survival kit” for usability sessions that should get you through a power outage.
  1. Mobile broadband card – plug it into your computer, and you have internet access
  2. Portable power supply – power your computer through the rest of the sessions
  3. Digital audio recorder – hit record and capture the conversation from the session
Those three pieces of equipment should get you through usability sessions without power. Now, how you would travel with a portable power supply in addition to everything else you’re already carting around is another story.

What about you? What unexpected events have you encountered while doing research? How did you cope with them or resolve them?

Labels: , , ,

diigo it

Tuesday, September 2, 2008


More money is being spent online, serving up ads and an ad revenue model continue to be the driving force behind new start-ups, Microsoft purchases (or planned ones) and Google product development such as the new browser Chrome - built to further deliver targetting for those that buy via Google and potentially shut out others like Microsoft from lucrative profiling data The big question becomes how do you measure such terms as "immersion", "product placement", "gaming", "social media" and so forth.

One interesting idea we came up with at Sentient was in regards to measuring brand interaction in virtual worlds for market research (at the bottom of this post). How are you measuring brand interaction on emerging platforms?

Virtual Worlds activity is measured with specific metrics that are different from web metrics

These are the areas it makes most sense to measure:
– Sim Traffic
Sim = server
Sim traffic is the total amount of users that have visited the respective presence in a given time frame

Currently virtual worlds can accommodate 65-100 users per sim
– Concurrency
Average number of users on a sim at the same time

– Sustainability
Average time experience per user (in hours)

– Experiential Value (EV)
((Total Traffic/(Concurrency/10))*Sustainability= VE ratio

Benchmark - WBHV, 12/12/06 launch - ((200/(40/10))*40 = 533.33
– WBHV Rave Party was considered a success by Second Life standards

Labels: , , ,

diigo it

Tuesday, May 20, 2008

Austin is where the digitally savvy things are

That’s from a Scarborough Research report, which found that Austin has the highest concentration of consumers that own certain high tech items (such as DVRs, satellite radio, VoIP), engage in certain internet behaviors (including blogging, downloading music, online gaming) and use leading-edge cell phone features (email, text messaging, etc.). Scarborough Research terms these consumers the digitally savvy, and nationally, 6% of the population is classified as digitally savvy. While Austin boasts 12% of its population as being digitally savvy. Yet another reason why Austin is the coolest place to live! (Ok, perhaps I’m slightly biased as I call Austin home…)

As both the report and this recent article in Ad Age point out, the digitally savvy are leading edge digital consumers. Historically, this demographic has provided marketers a glimpse into the future in terms of cell phone and third screen behaviors. These behaviors are what enable the lifestyle of the digitally savvy – they are entrepreneurs and business decision makers that tend to have a longer commute, plus they like to travel. Thus they seem to prefer to “pull” information at their convenience instead of having it “pushed” to them. For example, they are more likely than the general population to download TV and video programs online.

The digitally savvy make an ideal target for a variety of market research engagements since they are more likely to be heavy and diverse online spenders, entrepreneurial, business decision makers and hungry for information (among other things).

  1. Ethnography could be used to further define how and from where this demographic pulls their information and to discover how a relationship model of advertising might be incorporated into the digitally savvy’s daily habits.
  2. Usability tests, especially on e-commerce sites, could yield tweaks to your site that greatly improve conversion rates. The digitally savvy, through their own tendencies, will have explored many sites and thus have developed a sense of best-of-breed on which they base their expectations of where certain parts of a site to be.
  3. Ideation would also be a great way to harness the strengths of the digitally savvy. Their entrepreneurship and hunger for information point to creative thinking processes that are just waiting to be tapped.

I am interested in hearing your thoughts – how else can we tap the digitally savvy? And let us know if you want to take a trip to Austin to visit the digitally savvy in person.

Labels: , , , ,

diigo it

Monday, October 15, 2007

Virtual Worlds Expo: Wrap-Up

Back home now, with the luxury of a little time to think about it, these are my main takeaways from Virtual Worlds Expo:

Interoperability/open standards: Everyone was talking about this, in all it's iterations:

* A shift from "walled gardens" to interoperability, which could mean the ability to move your avatar and/or identity from one world to another, from a virtual world to a traditional website and back again, from virtual world to mobile phone and back

* Making virtual worlds work like the Web

* Having a common client for virtual worlds

* Having greater accessibility to your online and in-world friends from any world, site, or phone

Measurement and research:

Who's there? What drives them? What keeps them coming back? There was a lot of discussion about the "early adopters" in virtual worlds, although to mind this is wrong -- the early adopter are just *now* getting there. According to the original Everett Rogers adoption curve, it's the Innovators who are currently best represented in virtual worlds populations.

Some thought-provoking numbers were put forward by those on the Demographics panel:

* Michael Cai from Parks Research had numbers (from a 9,500 user study) on what people do less of in the real world while they participate in virtual worlds. The runaway winner was "don't watch as much TV" with 60% -- implications for advertising and brands are very clear there. The number I found thought-provoking as well as amusing was "16% don't know what they do less of" which seems to indicate that a pretty large percentage of people just don't really know how they spend their time!

* Mary Ellen Gordon of Market Truths had some interesting research results (though from a small sample) that indicate Second Life is a good place for a brand, if done right: 57% of respondents considered buying a real-life product as a result of a recommendation they received from someone in Second Life (which actually speaks as much to the power of word of mouth as it does to the value of using Second Life as a marketing and branding platform). Additionally, her research showed that:

55% recommended a real-life product to someone they were chatting to in Second Life.
25% have gone to look at a product in real life after seeing it in Second Life.
9% have purchased a product in real life after seeing it in Second Life.
8% have bought a real-life product in Second Life.

Segmentation (or not) between entertainment and "serious" purposes

Across a number of different panels, but particularly in the Community and Customer Service panel, there was discussion of just what it is that "drives" virtual worlds, that makes them so compelling. The aggregate answer seems to be that virtual worlds' growth is fueled by community and narrative (or story). This would seem to suggest what at least panelist Raph Koster of Areae said out loud, which is that there is no real differentiation in virtual worlds between entertainment and "serious purposes". This is actually also true in the non-virtual world as well, but the virtual world, like gaming, is perhaps the first platform that started out that way.

To read all Awareness Is Everything posts on the Virtual World Expo, go here.

This link goes to a Virtual Worlds News wrap-up post that links out to a ton of conference coverage, including ours.

Labels: , , ,

diigo it

Wednesday, October 10, 2007

Demographics in virtual worlds: Numbers and charts and predictions, oh my!

I've stayed with the Community track at the Virtual Worlds Expo. The featured morning panel on that track was "Demographics and Numbers: Where Things Are Heading".

First up was Nic Mitham, managing director of K-Zero. He forecasts growth for virtual worlds. One big reason -- the number of kid-based groups. Pretty soon children will be outgrowing the kids' worlds, and they will look for new worlds to get into. There's a huge marketplace already of children using virtual worlds who will migrate up the food chain as they grow older.

Said Mitham, growth in virtual worlds will come from Western Europe, from Russia, Eastern Europe, South America, and Asia. He said he doesn't yet see much virtual worlds activity for and from "silver surfers" or Baby Boomers. He said he also feels that corporate communities are prime for growth in virtual worlds, as is the educational space.

Mitham said the most exciting area for him was product development -- new virtual worlds, and new interfaces, that he feels will help grow the overall virtual worlds space. He says the typical profile of today's virtual worlds user is an early adopter (though I actually think this is wrong -- we're just now getting early adopters in virtual worlds. According to the original Everett Rogers adoption curve, it's the Innovators who are currently best represented in virtual worlds populations).

Mitham said easier interfaces will trigger growth for early adopters, and serve as a bridge to get new people into virtual worlds. One thing that will engage people is to make the overall experience easier. Also helping will be Web-based remote viewers, using a browser to access virtual worlds, including mobile devices. New worlds will help growth as well. A Google My World would be a huge source of growth.

Mitham also sai that diversification would be helpful in growing overall vw user numbers. He cited the rise of category-centric vertical worlds such as Football Superstars (currently in development). There's potential for growth in community-based worlds where the draw is the content, not just "early adopters" (or Innovators!) looking for the next new thing. Finally he cited cross-world avatars as having the potential to grow the total population, though not the number of unique users. But cross-world avatars could help more people get engaged in vws.

Mitham put forward some growth projections for the period Q407 to Q408 in selected virtual worlds. These are increases in the numbers of registered accounts, where the first number is the Q407 number of registered accounts and the second number is Mitham's prediction of the number of registered accounts for Q408:

Second Life 10m to 20m; 1 m to 7 m, Kaneva 0.6m to 3m, HiPiHi 0m to 10m, Whyville 3m to 10m, Club Penguin 15m to 30m, Football Superstars 0m to 3m

Next up was Parks Associates' Michael Cai, who put forward numbers form a recent Parks survey:

6% of US broadband users visit virtual worlds on a weekly basis.
18% have tried a virtual world at least once.
Second Life is the number 1 virtual worlds, followed by teen worlds and kid worlds. Most regular virtual worlds users visit more than virtual world.

He also had numbers on virtual worlds vs social networks -- he reported huge gaps betwen social network and virtual worlds usage among various demographics. Such as -- 40% of 25- to 34-year-olds participate in social networks, vs 12% of that age group participating in virtual worlds; 71% of 18- to 24-year-olds participate in social networks, vs 10% of hat age group participating in virtual worlds; 35% of females participate in social networks vs 5% females participating in virtual worlds; and 29% of males participate in social networks vs 7% of males participating in virtual worlds.

Cai also had numbers (from a 9,500 user study) on what people do less of in the real world while they participate in virtual worlds:

60% don't watch as much TV
22% don't sleep as much
18% don't read as much
16% don't know what they do less of (I found that illuminating -- people just don't really know how they spend their time!)
15% work less
15% spend less time with friends and family
12% don't do sports as much
11% don't shop as much
7% other

Finally, Cai said the majority of users think Second Life is a good medium to promote a brand or product

Last up was Market Truths' Mary Ellen Gordon, who presented an update of research whose first wave was done in the first quarter of this year (n=201 Second Life users in Q! and 190 SL users in Q3, after excluding questionable respondents)

60% of respondents have positive perceptions of real-life brands in Second Life, which is an improvement over the Q1 numbers.

Between quarters, the perceptions of brand effects and consequences hasn't changed. Brands bringing in new resources to virtual worlds are considered a good thing, and people are not as afraid that real-life brands will damage small content creators in Second Life.

Gordon said that as the Second Life population has grown, some concerns abotu brands "taking over" Second Life are still there, but the new people coming in too SL now are more mainstream and don't have as many concerns.

Here are the tactics Gordon's research says work best for brands in Second Life:

Give away SL versions of real-life products; co-create real-life products; sponsor brand-related events; customize real-life products for SL use. People tend to like things that link Second Life and real life, that offer two-way interactions.

What doesn't work: advertising via IM in-world; advertising using notecards for neutrally perceived brands (brands that are not already high-profile in Second Life). Most tactics are perceived more positively when undertaken by brands for which pre-existing attitudes are positive.

Gordon also presented numbers on consumer behavior in and out of Second Life:

57% considered buying a real-life product as a result of a recommendation they received from someone in Second Life.
55% recommended a real-life product to someone they were chatting to in Second Life.
25% have gone to look at a product in real life after seeing it in Second Life.
9% have purchased a product in real life after seeing it in Second Life.
8% have bought a real-life product in Second Life.

Gordon's research on brand types: High-awareness brands in Second Life are currently concentrated in five categories: Of 24 brands meantioned unaided, they were concentrated in information technology, athletic shoes, soft drinks, cars, and media

Gordon said that given the word-of-mouth numbers form her research, it's important for brands to start looking at soft measures as well as metrics -- measures such as word of mouth, engaging with the brand/product in real life, in-world effects vs real world effects vs cross-channel effects, and buying process.

Both active and passive measures have issues in terms of user acceptability. People in focus groups have said they value their privacy and don't really want to be tracked in Second Life as closely as is now possible, let alone will be possible in future.

Finally, Gordon said that some of the brands with the highest awareness do not have an official presence, but their awareness is the result of people making unofficial versions of their products available in Second Life. There's a concern among brand people about whether or not these products represent their brands as they would like. Brands need to work with small content creators in a way that works for the brand yet doesn't cause a backlash.

Questions from the floor:

What's the most underserved demographic in virtual worlds? Mid-40s and up, said KZero's Mitham, although he doesn't know whether the demand is there yet. He said, we know these people are active web and email users. Cai from Parks said the social networking crowd isn't well-served yet, and he doesn't understand why and such sites haven't been in virtual worlds yet. He also said females are underserved.

How many virtual world users are there today worldwide, and projections through 2011? If someone knows the answer to that, I'd ask them to buy my lottery ticket, said Mitham.

Labels: , , ,

diigo it

Virtual World Guide white paper available for free download

We've compiled a guide to existing virtual worlds that's available for free download. This guide offers info and screen shots on dozens of worlds from Second Life to Habbo Hotel. There's also more information in the paper about our Virtual Awareness research offering, designed to help companies understand more about how, when, and where to take their brand into virtual worlds.

To get a free copy of Sentient's Guide to Virtual Worlds, go here..

Labels: , , , ,

diigo it

Thursday, October 4, 2007

User research for Second Life, virtual worlds in general

As of yet there's not a whole lot of research available publicly on the user experience in Second Life and virtual worlds overall. That's starting to change. Just this week a couple of reports have crossed my desk, one free and the other relatively inexpensive. Both are thought-provoking.

User Acceptance of Virtual Worlds bills itself as an "exploratory report" and with 250 respondents, that seems right. It's also fairly timely, having been fielded last spring. The authors, academics at Rollins College and Potsdam University, present this summary of their findings, a sort of snapshot of their respondents:
· 90% of respondents have less than a year experience on Second Life.
· 70% access Second Life from home.
· 67% of respondents are not afraid of giving personal information.
· Almost 60% are very likely to buy virtual goods from Second Life, and 42% are willing to use their credit card.
· 70% perceive Second Life to improve collaboration, 69% say it improves communication, and 61% say it improves cooperation between people.
· 56% of respondents perceive Second Life as easy to use.
· Finally, people are using Second Life not to change their identity, but rather to explore and visit new places and meet people.
The authors also asked their respondents what other social networks they are on, and got this response: 72% also use YouTube, 47% use Flickr, 40% use MySpace, 39% use FaceBook and 33% use LinkedIn.

The reason I found this interesting was that I read this report just after I read Social Technographics, a Forrester report from last spring that attempt to segment consumers of all kinds of social media, describing them by their placement on the "participation ladder" (see image here, in a thoughtful blog post on the Business Communicators of Second Life blog that discusses the Social Technographics report in detail).

According to report author Charlene Li, the ladder represents "six increasing levels of a participation in social technologies. Participation at one level may or may not overlap with participation at other levels." The six rungs are (in descending order of intensity of participation):
· Creators. Online consumers who publish blogs, maintain Web pages, or upload videos to sites like YouTube at least once per month. Creators, an elite group, include just 13% of the adult online population. Creators are generally young — the average age of adult users is 39 — but are evenly split between men and women.

· Critics. These online consumers participate by commenting on blogs or posting ratings and reviews on sites like Critics represent 19% of all adult online consumers and on average are several years older than Creators.

· Collectors. Users who save URLs on a social bookmarking service like or use RSS feeds on Bloglines, thus creating metadata that’s shared with the entire community. Collectors represent 15% of the adult online population and are the most male-dominated of all the Social Technographics groups.

· Joiners. This unique group has just one defining behavior — using a social networking site like or Facebook. They represent only 19% of the adult online population and are the youngest of the Social Technographics groups. They are highly likely to engage in other Social Computing activities — 56% also read blogs, while 30% publish blogs.

· Spectators. This group of blog readers, video viewers, and podcast listeners, which represents 33% of the adult online population, is important as the audience for the social content made by everyone else. The most common activity for Spectators is reading blogs.

· Inactives. Today, 52% of online adults do not participate at all in social computing activities. These Inactives have an average age of 50, are more likely to be women, and are much less likely to consider themselves leaders or tell their friends about products that interest them.
The way I am reading the results from these studies, Second Life is full of Creators and Joiners. The Social Technographics report also offers profile information on users with different kinds of motivations. Social media users who are entertainment-motivated (which would be many Second Life users) tend to "participate in greater numbers," says the report (as opposed to those whose use of social media is motivated by career or family). As more job fairs and other recruiting activities start to take place in Second Life, we may see greater numbers of career-motivated people there.

These studies are interesting for the snapshots they offer, but overall there is a dearth of research available on virtual worlds users -- what motivates them, what interests them, who they are, what their real-life and virtual-worlds habits are, how those are different and how those are similar.

There's a move afoot to start a Metaverse Market Index, which will offer an independent, welcome source for tracking data and for virtual worlds user research. Once the MMI gets started, the virtual worlds world will be less like the Wild West and more like....the Wild West with a couple of badged sheriffs, maybe . Data should begin to come from that effort by mid-2008.

Meanwhile, companies that would like to explore the possibilities of virtual worlds for their brands would be advised to do primary research specifically tailored to their brand (both in-world and real-life). Call us if you need help with that. That we can do!

Labels: , ,

diigo it

Tuesday, September 18, 2007

Sometimes It Counts To Count -- And Sometimes It Doesn't

A couple of very good posts on the uses of qualitative vs. quantitative data have come my way over the past few weeks. Both of these are well worth looking into:

Andrew Hargadon, author of How Breakthroughs Happen: The Surprising Truth About How Companies Innovate, wrote this on his blog about “the virtues of qualitative research”:

“Qualitative research is, at its heart, an attempt to understand how people (or fish) interpret their reality and as a result make it. Anyone who has both looked at manufacturing statistics and wandered the factory floor knows that you can learn a lot by watching and talking to the workers about their work and their lives. And so, when you decide you want your company to be more innovative--and you decide to reward those who are ‘innovative’--you need to be very careful how you are measuring innovation.”
In an unrelated post, Bob Sutton, author of The No-Asshole Rule, discusses on his blog the current focus of many management gurus (including Sutton himself), “evidence-based management”:

“Managers and the business press seem to automatically assume that quantitative evidence is always the best evidence....The message seems to be that evidence-based management means management by quantitative data. I reject that thought, and have always believed that there are times when qualitative data are more powerful, valid, and useful for guiding action than quantitative data.”

Sutton goes on to describe three areas he feels it essential that companies use qualitative data:

1. When you don’t know what to count.

My take: This is what we often use qualitative techniques for in market research. If you don’t know what specific kinds of answers you're looking for, you can’t even construct a questionnaire with closed-end questions. So, often the first step is a qualitative study that will allow you to understand what’s important and what even begin to understand what can and should be measured.

2. When you can count it, but it doesn’t stick.

My take: It seems to me that this is the least compelling of Sutton's reasons, if only because so many companies seem to want numbers of some kind before they’ll make a decision. But it is true that compelling stories and images, which can only come from qualitative research, can be very persuasive.

3. When what you can count doesn’t count.

I don’t have anything to add here that’s better than Sutton’s observation: “In the hunt for and obsession with what can be counted, the most important evidence is sometimes overlooked. As Einstein said, ‘Not everything that counts can be counted, and not everything that can be counted counts.’ ”

Labels: , , ,

diigo it

Friday, September 14, 2007

Ford and JWT blur the line between marketing and research

"This is a marketing research project, and you won't be marketed to as a result of this project. And we won't use your image or words to market anything."

We say this, or words to this effect, at the beginning each and every focus group we do. Words like this appear somewhere in the recruiting script of every quant survey we do. We live by these words. And we're not the only ones -- the "church and state separation" between interviewing people for research and interviewing people to solicit marketing endorsements is a wide gap that's observed by most ethical agencies, and spelled out in the standards and ethics statements of groups such as CASRO (Council of American Survey Research Organizations).

Apparently the folks at Ford and JWT did not get that memo about the ethics of market research. A couple of weeks ago the Ford "Swap My Ride" campaign launched, complete with testimonials solicited under the guide of marketing research:

"In New York, Miami, Los Angeles and Dallas, its advertising agency, JWT, had workers pretend to be from a fake market-research firm, track down owners of cars made by Toyota Motor Corp., Honda Motor Co. and other competitors, and ask them to drive new Ford models for a supposedly impartial weeklong test." (from the Wall Street Journal, sub reqd)

Not only did JWT gather and film the research "respondents" under a ruse, they *never* told some what was really going on. Again, from the Sept. 14 Wall Street Journal article:

"Reached yesterday, Mr. Campos [one of the participants] was surprised to learn In-Home Test Drive Experience isn't a real company and was linked to Ford. 'I had no idea,' he said. He added it doesn't change his high opinion of the Focus, but that it would be better for the company to be 'more straight-forward.' "

The WSJ calls this an "aggressive marketing tactic" and seems to suggest that Ford has no choice, as the quality perception of American cars is so low among buyers of foreign (especially Japanese) cars that it's virtually impossible to get past that perceptual screen and get people to really "see" the cars and the quality improvements that have been made over the years.

“We wanted raw, unbiased opinions,” said Toby Barlow, co-president and executive creative director at JWT Team Detroit, Ford’s longtime creative shop. “We didn’t want them to think they were in a TV commercial. We needed a trick to get real objectivity and honest responses.” (from The New York Times, sub reqd)

They wanted honest responses, so they had to lie to get them? If people didn't know that their video was going to be used, I wonder if they told them that their words and images were going to end up on this Ford website?

This is a story that's just now starting to throw off a backlash (see here and here). I predict that the backlash will grow. In the end all that will have happened is that once again those of us who are actually practicing market research according to ethical guidelines will again have to defend our industry to consumers who are quite rightly skeptical. And Ford will simply come off looking desperate. People, this is not how social media works. The point of consumer-generated media is that you let the consumers generate the media, not trick them into starring in media that you generate.

Labels: ,

diigo it

Monday, August 13, 2007

Is it a case of the chicken and egg?

Virtual worlds, they are all the buzz. CEO's of Fortune 50 companies are claiming they are the next evolution of the internet. Gartner has publicly announced that by 2011 they believe 80% of the internet will have a "second life" (though they also warned about security issues this past week as well). Lastly, over 100 articles have been written about virtual worlds in just the past 6 months.

If you were to see all of this, you might immediately run down to your web team and say "I NEED A VIRTUAL WORLD PRESENCE." Which is exactly what a 100 or so major companies have already done, only to see that a nascent market is precisely that: a nascent market.

Don't get me wrong, I believe just like Sal Palmisano, IBM CEO, that virtual worlds are the next evolution of the internet, however we must first learn how to market within virtual worlds, how users of virtual worlds like to be communicated to, and how best to message our brand and position our company, products, and services. We must iron out the kinks and nail down security issues. We must bolt down the hatches, and hoist the sales.

I don't blame the companies completely though, as so many creative developers quickly jumped on the bandwagon and started spitting out their show-stopping creative and telling brands exactly how and why they should be in a virtual world. All of this without a single statement, quote, remark, or comment from an actual customer of (insert brand).

Did we not just go through this around the turn of this century when everyone had a website not because they knew how to leverage and use it, but because the guy next door did and the creative was just too cool for school? Did we also not go through one of the worst industry busts of all time, when the deck of cards on the Dot.Com era came crashing down? Did we not learn that we must first research, ask questions, talk to customers, understand usability, etc. prior to building the coolest creative ever?

I just got done spending a couple of hours on reading all the case studies about how companies have been successful designing and re-designing their websites because they took the time to poll customers, to run usability tests, and to understand the psychology of the users. Virtual world presences are basically 3-D websites, so why should we buck the process we have been using the past 5 years or so just because there is a new cool gadget out on the market?

If you are a marketer of a brand and you are reading this, I urge you to ask the question which comes first, the research/usability or the creative. If you think it is research/usability, then give me a ring. I can’t guarantee that virtual worlds are ready for your brand, however I can help you make a decision, keep you from second guessing, and allow you to focus on the things that matter. If you think it is creative go talk to the corporations already in virtual worlds. My guess is that a good many of them can't explain the current strategic direction of their presence (though they might have been able to when it launched).

Virtual worlds are definitely the next evolution of the internet, but until we as marketers start to understand that virtual worlds are not a chicken and egg proposition, that they are clearly research/usability first, creative second, then we will be stuck with 1 step forward and 2 steps back as big media has a heyday debating the viability of virtual worlds.

Labels: ,

diigo it

Wednesday, August 1, 2007

Austin AdFed and Virtual Worlds

I recently had the pleasure of writing an article for the Austin AdFed newsletter about virtual worlds and marketing. Take a look here and let me know what you think.

Labels: , , ,

diigo it

Sunday, July 22, 2007

A sideways smack at the focus-group methodology

Focus groups -- the most picked-on market research methodology of them all -- got a sideways smack from James Surowiecki in the New Yorker's July 9 edition. I say "sideways" because the point of the article was not to dump on focus groups, but to tout the much more fashionable prediction markets as a potential method for identifying hits and misses in the consumer market space. The slap at focus groups was just a by-product:
"Prediction markets avoid many of the faults of focus groups, which tend to be dominated by the loudest and most opinionated people, to be driven toward consensus decision, and to discourage disagreement, making them of limited usefulness. ('Seinfeld,' famously, was a complete bust with focus groups.)"
First, surely someone can come up with something newer than Seinfeld about which focus group respondents were wrong. That example is aging.

Second, note that when people complain about focus groups they always complain that they were wrong at predicting a product's success (usually) or failure (occasionally). Yet among professional qualitative researchers, the idea that you might use focus groups to predict *anything* is ludicrous. So, once more, repeat after me: Focus groups help you discover why and how -- not how many. The job of quantifying and predicting falls to quantitative research.

Here's a similar example from another New Yorker writer, Malcolm Gladwell. In his book Blink he tells the story of the Aeron chair: Apparently in the early 1990s, when the concept for the Herman Miller Aeron chair was being researched, focus groups were held with "facility managers and ergonomic experts" -- the target market. These group participants mostly did not like the odd aesthetics of the chair. Gladwell gives the Herman Miller folks a big pat on the back for going forward with the launch anyway. And what happened? Quoting Gladwell:
"In the beginning, not much. The Aeron, after all, was ugly....however, in California and in New York, in the advertising community and in Silicon Valley, it became a kind of cult object....It began to appear in films and television commercials....and by the end of the 1990s....was the best-selling chair in the history of the company."
I'd like to point out here that this is *exactly* what the focus group research indicated. Not predicted -- but indicated. The target market didn't like the chair and wasn't likely to adopt it. That is a finding -- not an implication, insight, or recommendation. (The fact that the focus group did not like Seinfeld was also a finding, not a recommendation.)

A smart qualitative researcher would have been able to dig deep and get a sense for what those findings meant. It's *not* about whether the groups hate the Aeron and Seinfeld -- it's about gaining insights that help Herman Miller put together a product-launch strategy for the Aeron that....about understanding the kinds of reactions Seinfeld elicits from a specific small demographic. The mistake comes with premature closure. The findings are not insights or recommendations. They are not quantified. They are not predictions.

The purpose of focus groups is not to predict or quantify. The purpose is to understand how and why, to get the "lay of the land," to come to an understanding of the opinions of a particular, very small group. There are lots of ways to do this, of course -- observational research, such as traditional ethnography or an online hybrid; one-on-one interviews; interviews and groups online, in virtual worlds; interviews and groups with target markets, with existing customers. Gathering 8 to 12 strangers in a room and asking previously written, non-directed questions of them, leading them into a group discussion, is a perfectly legitimate method -- as long as it's done correctly.

The prediction market, as cool and up-to-the-moment as it is, won't give you the depth of insight and understanding in people's likes, dislikes, and motives that qualitative research will. But I can totally see how it would be way better for predicting who'll win the Oscars, and may also be better at predicting how well a given book will sell (the subject of the Surowiecki article).

I'll give Surowiecki the last word, because despite his sideways slap at focus groups, this indicates that he *does* "get" consumer research -- but like a lot of others, he just can't resist smacking the focus-group methodology around a little.
"The collective intelligence of consumers isn’t perfect—it’s just better than other forecasting tools. The catch is that to get good answers from consumers you need to ask the right kinds of questions; asking the market to predict how many copies a book will sell, which requires predicting how a wide readership will behave, is better than asking the market to predict which manuscript will get a book deal, which requires predicting the decisions of a small number of editors. (The Simon & Schuster experiment with MediaPredict, unfortunately, focusses more on the latter.) And you need a critical mass of people to participate."

Labels: , ,

diigo it