Aug 262014
 
No More Surveys

Back in March 2010, I caused quite a stir with a prediction, at the UK’s MRS Conference, when I said that in 20 years we would not be conducting market research surveys. I followed my conference contribution with a more nuanced description of my prediction on my blog.

At the time the fuss was mostly from people rejecting my prediction. More recently there have been people saying the MR industry is too fixated on surveys, and my predictions are thought by some to be too cautious. So, here is my updated view on why I think we won’t be conducting ‘surveys’ in 2034.

What did I say in 2010?
The first thing I did was clarify what I meant by market research surveys:

  • I was talking about questionnaires that lasted ten minutes or more.
  • I excluded large parts of social research; some parts of which I think will continue to use questionnaires.

Why no more surveys?
In essence there are three key reasons that I think surveys will disappear

  1. The decline in response rates means that most survey research is being conducted with an ever smaller proportion of the population, who are taking very large numbers of surveys (in many cases several per week). This raises a growing number of concerns that the research is going to become increasingly unrepresentative.
  2. There are a growing number of areas where researchers feel that survey responses are poor indicators of true feelings, beliefs, priorities, and intentions.
  3. There are a growing number of options that can, in some cases, provide information that is faster, better, cheaper – or some combination of all three. Examples of these options include: passive data, big data, neuro-stuff, biometrics, micro-surveys, text processing of open-ended questions and comments, communities, and social media monitoring.

Surveys are the most important thing in market research!
There is a paradox, in market research, about surveys, and this paradox is highlighted by the following statements both being true:

  1. The most important data collection method in market research is surveys (this is because over half of all research conducted, in terms of dollars spent) is conducted via surveys.
  2. The most important change in market research data collection is the move away from surveys.
Because surveys are currently so important to market research there is a vast amount of work going on to improve them, so that they can continue to deliver value, even whilst their share of MR declines. The steps being taken to improve the efficiency and efficacy of surveys include:
  • Mobile surveys
  • Device agnostic surveys
  • Chunking the survey into modules
  • Implicit association
  • Eye-tracking
  • Gamification
  • Behavioural economics
  • Biometrics
  • In the moment research
  • Plus a vast rage of initiatives to merge other data, such as passive data, with surveys.

How quickly will surveys disappear?
When assessing how quickly something will disappear we need to assess where it is now and how quickly it could change.

It is hard to know exactly how many surveys are being conducted, especially with the growth of DIY options. So, as a proxy I have taken ESOMAR’s figures on market research spend.

The table below shows the proportion of global, total market research spend that is allocated to: Quant via surveys, Quant via other routes (e.g. people meters, traffic, passive data etc), Qual, and Other (including secondary data, consultancy and some proportion of communities).

The first three rows show the data reported in the ESOMAR Global Market Research reports. Each year reflects the previous year’s data. The data show that surveys grew as a proportion of research from 2007 to 2010. This was despite a reduction in the cost of surveys as F2F and CATI moved to online. From 2010 to 2013 there was indeed a drop in the proportion of all research spend that was devoted to surveys. However, given the falling cost of surveys and the continued growth of DIY, it is likely that the absolute number of surveys may have grown from 2010 to 2013.

Other quant, which covers many of the things that we think will replace surveys, fell from 2007 to 2010. In many cases this was because passive collection techniques became much cheaper. For example the shift from expensive services to Google Analytics.

The numbers in red are my guess as to what will happen over the next few years. My guess best on 35 years in the industry, talking to the key players, and applying what I see around me.

I think surveys could lose 9 percentage points in 3 years – which is a massive change. Does anybody seriously think it will be much faster? If surveys lose 9 percentage points they will fall below 50% of all research, but still be the largest single method.

I am also forecasting that they will fall another 11 percentage points by 2019 – trends often accelerate – but again, does anybody really think it will be faster? If that forecast is true, by 2019 about one-third of paid for research will still be using surveys. Other quant will be bigger than surveys, but will not be a single approach; there will be many forms of non-survey research.

I also think that Other (which will increasingly mean communities and integrated approaches) and qual will both grow.

What do you think?
OK, I have nailed my flag to the mast, what do you think about this issue? Are my forecasts too high, about right, or too low? Do you agree that the single most important thing about existing data collection methods is the survey process? And, that the most important change is the movement away from surveys?


 

Aug 172014
 
iBeacon

Beacons are devices that send a signal to people’s mobile phones, which identifies when somebody is close to a specific beacon, which in turn allows locations (e.g. a Starbucks) to know that somebody is in their store, which allows a range of marketing and market research options to be enabled. Beacons are the key that unlocks location-based research, which is one of the key requirements of ‘in the moment’ research. OK, there is a lot of jargon in that sentence, so let’s unpick it.

In the moment research. Traditional research was based on asking respondents to remember what they had seen and done. This was not too much of a challenge with really big things, like how many cars or refrigerators did you buy this month, and if you bought one, what brand was it and roughly how much did you pay. But if you want to know where somebody bought their bottled water, how they felt about the way they were served by a barista, or how their commute into work today went, the interactions need to be investigated during the activity, or immediately afterwards. This sort of research is called in the moment research. Location-based research. This means that the location triggers the research. For example, somebody visits a supermarket and the supermarket research activity is triggered. Later they visit a station and a travel related activity is triggered.

Beacons and iBeacons. The first beacons that were used for location-based activities were based on sound. The beacons were fitted in locations, such as a store, and emitted a very high frequency signature. The sound was too high pitched for humans to hear it, but mobile devices running the appropriate app could detect it. The beacon was in a location such as a store, when a participating respondent entered the store the beacon, the store, the company running the service, and the participant’s phone all know the participant is in the store. Which means the location can be recorded and an action initiated.

Sonic beacons have been superseded by beacons using Bluetooth LE (the LE stands for low energy), with the key example being Apple’s iBeacon. Although iBeacon is an Apple device it can work with iPhones and Android devices.

ShopKick. The most high profile user of beacons is not a market research company, but a marketing company, ShopKick. ShopKick persuade people to sign up to their system so they can earn awards for visiting participating sites. ShopKick can then use their system to send messages, coupons, request for actions etc.

The near future? Beacons are unlocking location-based research and in the moment research, which is going to increase the reach, sensitivity, and validity of market research.

What’s Hot in Research? Beacons are one example of something that is hot in market research. To find out more about what is hot, check out this post on the NewMR website.


 

Aug 142014
 
Godzilla

One of the questions I get asked most often is “What’s hot in market research?”. I will be broadcasting my update as NewMR lecture next Wednesday, August 20, (you can register for it here).

But here is a sneak peek into what is hot, still hot, bubbling under the surface, and not so hot.

Still Hot
It is important when looking at the ‘new stuff’ not to ignore stuff that has been around for a while, but which is still growing in market share, importance, and usage:

  • Mobiles in traditional research. Mobile is a big and growing part of CATI, online surveys, and F2F – this trend has a long way to go yet.
  • Communities. Communities (including Insight Communities and MROCs) have been the fastest growing major new research approach for a few years now, and this is going to continue.
  • DIY. We hear less about DIY these days, that is probably because it has become normal, this sector is growing, both in terms of part of being a key part of existing MR and partly because it is growing the scope of market research.

Hot!
These are three of the items that I think are the hottest topics in MR, in terms of their growth and potential. All three of these are going to be game changers.

  • Beacons. For example iBeacons, which use geofencing and allow location-based services (including research) to be offered in much easier and more practical ways than is offered by methods such as GPS.
  • In the moment research. Research using mobiles and research using participants to capture information as people go about their normal day, including qual, quant, and passive, is making research more valid and sensitive.
  • Micro surveys. The most high profile micro (or nano or very short) provider is Google Consumer Surveys, but there are a variety of other providers, such as RIWI. Also, Beacons, In the Moment, and Communities are all leveraging Micro Surveys.

Bubbling
These three are going to make a major impact soon, but not quite yet.

  • Text analytics. The technology is not quite here yet, but when it clears the last few hurdles it will hit market research like a freight train – for example shifting the balance from closed questions to open questions, and finally driving more value out of social media discourses.
  • Web messaging. Apps like WhatsApp, WeChat, and Line are growing faster than anything else globally. A few people are looking at how to leverage these for market research, and more will follow.
  • Research bots. One of the key factors limiting the use of social media, communities, and the use of video is the requirement to use people to do the moderation and analysis. Bots (software applications short for robots) are going to change this and open a new, vast range of options.

Not So Hot
These three are all interesting niches, some people are making a good living from them, but they are not scaling in a way that makes a difference to most brands or researchers.

  • Facial Coding. It answers some questions, but is limited in terms of its range of uses, delays, scalability, and cost.
  • Webcam qual. The benefits are usually too small and the resistance from potential participants are too high to make this a generally useful approach.
  • Social Media Research. Whilst social media research, especially monitoring, has become essential, it has not grown into what was expected.

What about?

  • Big Data
  • Behavioural Economics
  • Gamification
  • Smartphone ethnography
  • Neuroscience
  • Geotracking
  • Wearbles
  • Quantified Self
  • Biometrics

Want to know where these items fit in this picture? Tune in to our webinar next Wednesday, 10am New York time, which is 3pm London time. Click here to register.


 

Jul 282014
 

During the last week I have made three separate presentations in Tokyo on the topic of the key trends in mobile market research and I have found that creating a big picture has helped get my message across. I use the big picture as the first and last slide of the presentation to show where I am heading and to sum up the message. The audience seem to feel that a big picture makes the ideas clearer and helps provided an integrated understanding of what is happening and why.

So, in this post I share my Big Picture of the Key Trends in Mobile Market Research.

Why is mobile so interesting?
The key reasons are all shown inside the phone. Ubiquity refers to the fact that about 70%-80% of the world’s adults have a mobile phone, and the penetration is growing. Mobile phones are the most widely owned device on the planet, and they are changing the way humans communicate.

Because people have their phone with them all the time, 24/7, they provide a better way of contacting people. Better than waiting for people to answer the phone, open their email, or answer the door. The phone is with people ‘in the moment’, i.e. when they are actually doing things.

Increasingly phones are smartphones and connected to the internet. In 2014 we can’t assume that enough people have smartphones connected to the internet to ignore other methods and options, but the trend is towards most of the people that we tend to research being fully smartphone and tablet connected.

Mobile devices can do so much more than just surveys, passive data, push notification, and location based services are just the start. The phone is becoming a window into people’s lives.

Mobile is already a major part of ‘traditional’ research
Marketers, insight professionals, and market researchers need to be aware that mobile is already a major part of existing research methods. In terms of CATI, the amount that is conducted via mobile phone has been rising in most developed markets and is currently around 40%-60% in many markets. In the emerging markets, such as most of Africa, the mobile percentage is much HIGHER. In countries where incomes are lower, it tends to be just the well-off who have a fixed-line telephone, most people have just mobile phones.

20%-30% of online surveys are being attempted by people using mobile devices (phones, phablets, and tablets), so most people doing online are already doing mobile.

In terms of face-to-face and qual research, mobile devices are increasingly being used, for example in mCAPI where the interviewer has a tablet or phone instead of a clipboard.

Mobile is creating/expanding other forms of market research
Participative, or WE-research, is enlisting the person previously known as the respondent to be an active player in the research process. Participants are seeking out experiences, capturing photos and videos, and suggesting commentary. This reaches places that researchers could not reach and empowers customers and citizens.

Passive data collection adds objective information to the subjective picture the participant can supply, and with no effort required by the participant, and no reliance on their memory. Passive data is telling us what people do, when then do, how long they do it, and in many cases where and with whom they do it.

In the moment research is reminding us of how bad our memories are. Collecting fresh insights, in the moment when products and services are being experienced is opening the door to much more accurate, detailed, and relevant research. Location based services are allowing us to follow people through their day and ‘push’ requests to them based on where they are and what they are doing – giving us the best possible in the moment data.

Want more?
Of course, whilst a big picture is all some people need, a detailed picture is what others want. For mobile market research the detailed picture is going to be available in about a month with the release of our new book, “The Handbook of Mobile Market Research”, which is being published by Wiley and is available from Amazon. You can download a free chapter from the NewMR website, click here.


 

Jul 162014
 

Sometimes when I run a workshop or training session people want detail, they want practical information about how to do stuff. However, there are times when what people want is a big picture, a method of orientating themselves in the context of the changing landscape around them. Tomorrow I am running a workshop for #JMRX in Tokyo and we are looking at emerging techniques, communities, and social media research – so a big picture is going to be really useful to help give an overview of the detail, and to help people see where things like gamification, big data, and communities all fit.

So, here is my Big Picture of NewMR (click on it to see it full size), and I’d love to hear your thought and suggestions.

Big Picture

The Big Picture has five elements

The heart of the message is that we have reached an understanding that surveys won’t/can’t give us the answers to many of the things we are interested in. People’s memories are not good enough, many decision are automatic and opposed to thought through, and most decision are more emotion that fact. Change is needed, and the case for this has been growing over the last few years.

The four shapes around the centre are different strands that seek to address the survey problem.

In the top left we have big data and social media data, moving away from working with respondents, collecting observations of what people say and do, and using that to build analyses and predictive models.

In the top right we have a battery of new ways of working with respondents to find out why they do things, going beyond asking them survey questions.

In the bottom left we have communities, which I take as a metaphor for working with customers, co-creating, crowdsourcing, treating customers and insiders, not just users.

The bottom right combines elements from the other three. ‘In the moment’ is perhaps, currently, the hottest thing in market research. Combining the ability to watch and record what people do, with interacting with them to explore why and what they would do the options changed.

Thoughts?
So, that is my big picture. Does it work for you? What would you add, change, delete, or tweak?


 

Jun 212014
 
Nissam Small Car

A very large part of market research is based on asking people questions, for example in surveys, focus groups, depth interviews, and online discussions. In general, people are very willing to answer our questions, but the problem is that they will do it even when they can’t give us the right answer.

At IIeX last week Jan Hofmeyr shared the results of some research where respondents had been asked about which brand they buy most often and he compared it to their last 3 and last 6 purchases from audit data. He found that in the last 3 purchases 68% of people had not bought the product they claimed to buy ‘most often’, and in the last 6 purchases 58% of people had not bought their ‘most often’ brand.

The video below is designed for entertainment, but it illustrates the bogus answer problem really well:

There are two key reasons why asking questions can produce bogus answers:

  1. Social desirability bias. People are inclined to try to show themselves in the best possible light. Ask them how often they clean their teeth and they are going to want to give an answer that makes them look good, or at least does not imply they are lazy or dirty. In the video, many of the people know that music fans are supposed to know about music, so they don’t want to appear dumb.
     
  2. Being a poor witness to our own motivations and actions. Writers like Daniel Kahneman, Dan Ariely, and Mark Earls, have written about how people tend to be unaware of how they make decisions. Some of the people in the video are being primed in the question to assume that they know about the brand may possibly be deceived by their own thought processes, with what they do know being used a s pattern generator to produce plausible thoughts.

Of course, in addition to these two reasons, some people simply lie – but in my experience that is a tiny proportion (when seeking the views of customers and the general public) compared with the two reasons listed above. However, the problem of conscious lies increases if incentives are offered.

One way to reduce the number of false answers is to make it much easier for people to not answer a question, ideally by not having to say “I don’t know”, and letting people guide you to the strength of their answer. Look at the video again and you will see that many of the people being interviewed are trying to signal they don’t really know about the bands, for example “I don’t know any of their music but I’ve heard from my friends that ….”. For the sake of the interview and the comedy situation the interviewer presses them into appearing to know more. In an information gathering process we should take that as a cue to back off and make it safe or even ‘wise’ to avoid going any further.

Another important step is to avoid asking questions that most people won’t ‘know’ the answer to, such as “What is the most important factor to you when selecting a grocery store?”, “How many cups of coffee will you drink next week?”, “How many units of alcohol do you drink in an average week?”.

If you’d like to know more about asking questions, check out this presentation from Pete Cape.

The problems with direct questions are one of the major reasons that market researchers are looking towards techniques that use one or more of the following:

  • • Implicit or ‘neuro ‘techniques, such as facial coding, implicit association, and voice analytics.
  • • Passive observations, i.e. recording what people actually do.
  • • In the moment research, where people give their feedback at the time of an event, not at a later date via recall.


May 102014
 
MRMW London 2013

I am lucky enough to be invited to take part in a wide range of market research events, including those organised by the trade bodies, by specialist conference groups, and by innovators. For many years many market research conferences had been getting more and more similar, and less and less interesting as they sought to play to a wide audience.

However, in recent times that has changed, and one of the best examples of the change is the MRMW (Market Research in the Mobile World) series of conferences, organised by Merlien. I have been lucky to be involved in several of the MRMW events and I think what they offer is:

  • Focus, the MRMW events focus on mobile, this has helped develop the industries focus on developing new mobile approaches.
  • Series, the MRMW has several events a year and the series dates back to 2010. This has created an ongoing dialogue. Ideas are developed over time, rather than assuming they will appear fully fledged.
  • Global, the MRMW conferences are held in North America, Europe, Africa, and Asia. This enables the series to draw on regional strengths, and then take those strengths to new markets.
  • Innovations, for a small organisation Merlien have been very successful in introducing new faces and ideas, for example David ZaKariaie the founder and CEO of Glassic who brought copious sets of Google Glass for attendees to experience.

Chicago, May 27-30
The next MRMW is in Chicago, May 27-30, click here to find out more. Unfortunately, I can’t be there, but my hot picks from the agenda are:

  • The session on where investments are going in mobile MR, with Simon Chadwick.
  • The market research on trial session, with Dan Foreman.
  • Google Glass, from gimmick to tool, with David Zakariaie.
  • Device agnostic research, with Zoe Dowling.
  • Privacy and engagement – how far can you go, with Reg Baker.

I look forward to watching their contributions by video reply – check out the MRMW site to access their videos.

If you can’t make Chicago in May, perhaps you can make Berlin in September and/or Cape Town in November – the call for papers for both of these events is still open, so consider presenting something.

 

Apr 012014
 
Larnaca April 2014

I am currently at an academic conference on mobile research in Cyprus, a WebDataNet event. I am a keynote speaker and my role is to share with the delegates the commercial market research picture.

I really enjoy mixing with the academic world, and I am intrigued and fascinated by the differences between the academic and commercial worlds. This post looks at some of the key differences that I have noticed.

Timelines
In the academic world, timelines are usually longer than in market research. For example, an ethnographic project might be planned for 8 months, in the field for 4 months, and spend 12 months being analysed and written up. A commercial ‘ethnography’ might spend 4 weeks in design and set-up, the fieldwork might be wrapped up in 2 weeks, and the analysis and ‘write up’ conducted in 2 weeks.

In many ways the differences in the timelines result from differences in the motivation for doing a research project. Commercial market research is often conducted to answer a specific business question, which means the research has to be conducted within the timeline required by the business question – which is typically rapid. Academic research is typically conducted to advance the body of knowledge, which means there is often not a specific time constraint. However, there is a need to establish what is already known (the literature review) and a need to spend time creating a write up that embeds the new learning in the wider canon of knowledge.

The balance between preparation, action, analysing, and writing up

In the commercial world the answer is the point of the study; the method, providing it is acceptable, is less relevant.

In an academic study, the value of the specific answer is sometimes almost the least important feature of the project. For example, a commercial project looking at five possible ads for a new soft drink would seek to find the winner. An academic project would normally find that sort of result too specific (i.e. not an addition to the canon of knowledge). An academic project might be more interested in questions such as, what is the relationship between different formats of ad and the way they are evaluated, or the extent to which short-term and long-term effects can be identified. Indeed, in academic project the brands and the specific ads tested will often be obscured, because the study is about the method and the generalizable findings, not (usually) about which ad did best.

The definition of quality
Academic and market researchers have a hierarchy of types of validity but the hierarchy is not the same. Market researchers tend to value Criterion validity (does the measure correlate with or predict something of interest) as their ‘best’ measure.

By contrast, the academic world tends to prioritise Construct Validity, which relates to how well new findings relate to an accepted theory of how things work. This again probably relates to the specificity of the objectives. Market researchers need something that works well enough to solve a particular business problem. The academic is seeking to build knowledge and to connect that research to a wider framework.

The difference in samples
Most market research is conducted with a sample drawn from the target population and usually the sample is constructed to be similar to the target population in terms of simple variables such as age and gender – although it usually falls well short of being a random probability sample. By contrast, a large proportion of academic research appears to be conducted with convenience samples, often students.

The most common reason, for using convenience samples, is lack of resources. In some cases there is a belief that the phenomenon being researched is equally distributed across the population, such as preference for using left or right hand.

Access to the results
In commercial research the results are normally private to the client, unless they are for PR purposes. Traditionally, the results of academic research have been made available to the wider academic world. The future of access to academic research is subject to two contradictory trends. Firstly, commercially sponsored research is tending to be more secretive, because of the commercial interests. Secondly, Governments (who are often a major funder) are pushing the Open Data agenda, making research less secretive.

Which is better?
Academic research and market research differ in several ways, but that is mostly because they have different objectives. If you wanted to use a market research project for academic purposes you would need to add a literature review, add a comprehensive write-up, and be prepared to mount a robust defence of your method. If you wanted to use an academic project for a commercial project you would need to check the ethical clearance, check the timelines were going to be relevant, and check whether the study was likely to give an actionable result.

Mar 082014
 
Marina Bay Sands

This post is written as I reach the end of the first week of a three week Vision Critical trip to the Asia Pacific Region. For the last few years I have been spending about ten weeks a year in the APAC region, typically spread over three or four separate trips – because I am convinced that this is where much of the future (especially in terms of commerce, marketing, and insights) is being made.

Singapore Client Round Table
I arrived in Singapore Monday evening and the week got off to a flying start with breakfast with my Vision Critical colleagues from Sydney and from our newly opened Singapore office, followed by a meeting with the CEO of Indian partner, Majestic and lunch with an insight community client, Google. The afternoon was devoted to a client round-table meeting where several of Vision Critical’s clients gather to hear a keynote presentation (from me on this occasion) and then spend time sharing their learning with each other. This event was hosted by Google in their superb offices overlooking the Marina area, with key contributions from SingTel, Sony and others. Client roundtable sessions are a great way for clients to share their experiences with insight communities.

MRMW – Market Research in a Mobile World
Wednesday and Thursday was the APAC incarnation of MRMW, the leading global series of conferences on mobile market research, organised and promoted by Merlien. The keynote presentation was given by SingTel’s Melissa Gil, talking about how their three Vision Critical Insight Communities (Indonesia, Australia, and Singapore) provide them with rapid and cost-effective insight into digital consumers. One of the key points that SingTel made was that the speed and usefulness of the insights they produce mean that the SingTel insights team are involved in meetings and decisions at all levels of the business.

One of the key topics at the Conference was the evolving data protection picture in Asia and on the Tuesday Sue York from the University of Queensland (and curator of content at NewMR) moderated a panel on Data Protection, with Derek Ho (Senior Counsel from MasterCard), Dan Foreman (President of ESOMAR), Martin Tomlinson (Vice President of the Market Research Society of Singapore), and Stephen Jenke (Global Head of Data Collection at Kantar). The key points being made was that the picture on Asia was developing quickly, rules are becoming more onerous, and different countries have different rules.

Google Ray

One of the high points of the Conference was a presentation by David Zakariaie of Glassic who had brought ten sets of Google Glass with him to the event and who co-ran a session with me looking at the technology and the opportunities for market research to utilise this technology. Other key elements of the conference included: using feature phones as well as smartphones, utilising automated techniques for facial coding, video processing, and image processing (in all three cases the main theme was limited, but impressive, success), and moves towards geolocation and geofencing.

Effective Presentation Workshop
On the Friday I ran my “Secrets of Effective Presentations” workshop, which seemed to go down really well. I love workshops in multicultural situations as I am sure I learn as much as the attendees. Some of the secrets of creating and giving great presentations are global, but having a group from a wide range of countries (in this case Singapore, Cambodia, Saudi Arabia, and Australia) and with people who have a variety of first languages (and with a mixture of clients, suppliers, and academics) means that nothing can be taken for granted.

Key Singapore Takeaways
Compared with Europe and even with North America, Singapore embodies a ‘can do’ attitude, where the expectation is that tomorrow will be better than today, and that we are on a rapid path to a better, more technical, more insightful, richer society. Singapore also embodies the strength of cultural diversity. Most meetings with clients include people from a wide variety of countries. In order to get to Singapore, and in order to do well, most people have something special about them, and this tends to be blended to create something greater than the parts.