Dec 302012

I am currently working on a project for Vision Critical’s University, creating an introduction to market research, which should appear in a month or so. As part of that project, I’d like to share some of my thinking about why companies conduct market research, to see what my peers think, for better, worse, or different?

The four main uses of market research, by commercial organisations, in descending order of importance (in terms of spend) are:

  1. Monitoring performance, for example ad tracking, brand awareness, viewing figures, usage, customer satisfaction, mystery and shopping.
  2. Finding things out, for example the size of a market, current usage patterns, and market opportunities.
  3. To test ideas and products, for example ad testing, pack testing, and pricing research.
  4. To help create new products, ideas, campaigns etc.

Monitoring studies tend to be ongoing (as opposed to ad hoc) and they tend to be quantitative. The 2012 ESOMAR Global Market Research report shows this is the largest category of research spend, accounting for between one-third and half of all research dollars (depending on which category you assign some of the items to). As well as large-scale quant trackers, other approaches include automated and online measurements, mystery shopping, community panels, and social media research.

To find things out
This type of research is often called exploratory and is normally conducted ad hoc (i.e. not as part of an ongoing study). A wide range of research techniques are used for this purpose, for example: qual, quant, research communities, social media research, and semiotics. The specific aims of the research will often dictate, or at least suggest, a specific approach. For example, a market sizing study is typically going to require a large scale quantitative study with a careful sampling plan. By contrast, understanding the role of tea in a community’s life could be tackled with a wide range of approaches, from ethnography at one end of the spectrum to analysing loyalty card data at the other.

Testing ideas and products
This category includes ad testing, concept testing, product testing, much of NPD (new product development) research; it also includes testing whether a model or idea is true. This type of research is usually conducted ad hoc, utilizing qual quant, and/or research communities. Some of the newer brain and biometric techniques find their home in this category, in particular in ad testing. For example: fMRI, EEG, implicit association, eye-tracking, and facial coding.

Creating ideas and products
Although qual research, especially focus groups, have been used for many years to ‘ideate’ new products and services, this category of research has grown considerably since the arrival of social approaches, in particular research communities. Of the four categories listed here, creation is by far the smallest, but perhaps one of the most exciting.

Your thoughts?
I’d love to hear your thoughts. Do you think these four categories broadly encompass why commercial organisations conduct market research? Would you add, delete, or change any categories? What about the examples within the categories, do they seem about right to you?

Dec 182012

The BBC is reporting that Instagram, recently bought by Facebook, is altering its privacy policy to allow third-parties, such as FaceBook and advertisers, to access its members’ information and to use their pictures without permission or recompense. Yes, it is being reported that Instagram is going to be selling your photos, without asking you, without paying you, and without an opt-out!

Instagram/Facebook are not proposing to offer an opt-out of this commercialisation of people’s pictures. They have said, reportedly, that people who are not happy must leave instagram by January 16, 2013 – just under a month from now. CNET has a story titled ‘Instagram says it now has the right to sell your photos’ and Wired is offering advice on how to protect yourself with its article ‘How to Download Your Instagram Photos and Kill Your Account’.

If the reports are accurate and if Facebook does not change its mind I would expect Instagram’s, already weak, user base to largely disappear and for Facebook to face a backlash from users and perhaps legislators.

Facebook paid $1Billion for Instagram, it could soon be worth very, very much less than this.

In a recent post on Vision Critical University I showed that double jeopardy applies in the social media world, i.e. sites with fewer users tend to be less loyal. Instagram is a smaller site, smaller than FaceBook, Twitter, LinkedIn – so I would expect it suffer badly from letting its users down.

Dec 182012

I am in the process of writing an introductory statistics book for market researchers. This post and some of the following posts are taken from that book, in an attempt to field test the style, approach, and depth I am employing. All comments welcome.

My recommendation is that most numbers in presentations and reports should be presented as 2 or 3 significant digits. I feel that the issue of significant digits is more important than the more frequently discussed issue of decimal places.

In a number, the significant digits are those that carry the key details. If a bank robber steals $56 million, the 5 and the 6 are the significant digits – and the million gives the scale of the number. If we say that PI is 3.1416 then we are showing it to four decimal places and five significant digits.

Table 1 shows the number of internet users in five key, original, members of the EU; showing the raw numbers and the same numbers using two significant digits.

Column B shows the estimates in the format they were downloaded from the InternetWorldStats website. These raw numbers contain 7 or 8 digits, and commas are used to help make the numbers more readable. These values, presumably, represent the best estimates for each country, but they require an active act to read and interpret. By contrast, Column C shows the numbers using just two significant digits.

The use of two significant digits in Column C has two advantages, when compared with Column B.

  1. It is much easier see the relationships in Column C, compared with Column B. For example, in Column C, it is easy to see that Italy has just over twice as many internet users as the Netherlands, and about half as many as Germany. This information is harder to see at a glance in Column B.
  2. Almost all numbers have errors in them, and they tend to relate to a specific moment in time. Statisticians talk of spurious accuracy when too many digits are displayed, for example when saying 37.67% plus or minus 10%. If we use all of the digits, as in Column B, then we are implying (to most readers) that all the digits are equally accurate. By using just the two most significant digits, Column C gives a message to the reader that these are approximations.

Methods of utilising 2 or 3 significant digits
Here are some tips for different situations:
  1. Percentages. Only use round numbers, e.g. 36% rather than 35.67%.
  2. Salaries. Round them to the nearest thousands, for example $136K, rather than $135,670.
  3. 7-point rating scales. One decimal place, for example 4.6 rather than 4.634.
  4. Sales. Round the numbers to the nearest thousands, million, or billions. For example, numbers like 36,785 and 76,230 could be expressed as 37K and 76K (two significant digits). However, 36,785, 76,230 and 148, 102 would need to be shown as 37K, 76K, and 148K (three significant digits).

Ralph Waldo Emmerson said “A foolish consistency is the hobgoblin of small minds”, and it would be foolish to think that every set of numbers can be shown to two or three significant digits. Background documents, notes, and tables are often better with more digits.

However, in most cases, and in most presentations and reports, two or three significant digits are going to help the audience/reader understand the message better than showering them with digits.

Dec 112012

One of the questions I get asked quite often is whether or not research communities, such as MROCs and Community Panels, are possible inside the rules of market research? The answer is caveated, it depends on which community and which country’s research rules. In the post below I will set out my layman’s (i.e. it could be wrong) view of where communities sit in terms of the rules.

Why communities might NOT be market research?
There are three main areas of concern:
  1. Many communities use client based incentives, e.g. shop vouchers, air miles, telephone minutes etc. This tends to be against societies’ guidelines as they (and some legislators) feel that this is either distorting the market or a form of sales promotion. Brands are keen to use these sorts of incentives because community members tend to prefer them and they increase the bonding of the community members and the community.
  2. If the community is intensive, for example a long term, qualitative/ideation community, the community members tend to become advocates for the brand. The view of societies’ tends to be that this is market distorting and can be seen as a form of marketing. Brands are keen on this element of communities because it helps develop the brands word of mouth.
  3. If community members use their own names, or their own photos, their anonymity can be compromised. The anonymity is further compromised if the brand is involved in running the community themselves.
The choices for agencies
In essence, and under most societies’ guidelines, research agencies have two choices.
  1. Try to fit communities into the existing framework. For example, don’t use client products or services for incentives, use larger communities and panel management to minimise the impact on the respondents (for example using a community panel rather than an MROC), and insist that members do not use their real names and images. This option is easier in a market where other agencies choose the same option.
  2. Don’t call communities market research. Most societies’ allow market research companies to do things that are not market research, provided they do not describe them as market research. They often talk about using market research methods for non-research purposes. In Europe this is a common route, particularly in the UK. Within this framework, researchers are still bound to act honourably, e.g. respecting respondents, using appropriate techniques, etc – but avoiding saying things like ‘research conducted under the ESOMAR rules’.

There is of course a third option, one that is actually quite common. Many agencies seem to operate their communities utilising all three of the problem areas, but still describe it as market research, still flaunt their abidance with societies’ rules, in blissful ignorance of what the rules are.

One other question that comes up fairly often is ‘who owns the ideas generated by the community?’ The answer, if you have written your terms and conditions sensibly, is the brand. Not the members

Dec 012012
Gartner Hype Cycle

At the Festival of NewMR, Wednesday 5th December, I will be presenting a summary of where social media research is at the moment and where it is going next. As part of that presentation I will be exploring why I think social media research is heading into what Gartner have termed the Trough of Disillusionment. This post explores what I mean by this prediction.

The term social media research has two definitions, a broad one and a narrow one. The broad definition includes social media mining and listening, netnography, communities, smartphone ethnography, research into social media, and social media as a sample source. The narrow definition refers just to the seeking out and collection of large amounts of naturally occurring social media conversations and comments – this definition includes social media listening and mining. It is this narrow definition of social media research that I think is heading into the Trough of Disillusionment.

It should be noted that the Gartner Hype Cycle is a loose description of a repeatedly seen phenomenon – it is not a law and it is certainly not a specific mathematical formula. But it can help understand what often happens to new technologies.

Social media research’s rise was spectacular, with a massive growth in new tools and companies offering a variety of methods to collect data. But, as well as new tools and new companies there was way too much hype. The claims about the share of voice that could be captured online was inflated, the ability to segment the comments by simple demographics (such as country, gender, and age) was massively over-stated, and the claims made for automated sentiment analysis verged on the ludicrous in some cases.

As I have noted in another blog, 2012 seems to have been the year that manual coding of open-ends made a come-back, to the disappointment of most of us. This has been matched by papers looking at combining social media research with ‘old’ research (to compensate for social media research’s weaknesses), concerns about privacy, and a decline in the excitement over what can be found by passively listening to tweets, blogs, and status updates.

This decline in the expectations of social media research will reverse. We will get a better fix on what it can and can’t do, prices will come down, and tools and algorithms will improve. But before that happens, there will be more disillusionment to come.

To hear the whole social media research update, which includes both the broad and narrow definitions of social media research, join me online at the Festival of NewMR, Wednesday 5th December – registering is free.