Jan 262013

At the moment I am working on two exciting projects with Navin Williams looking at mobile market research (MMR), a book and course – more on these soon.

As part of these projects I have been taking another look at the phenomenon of ‘accidental MMR’.

Most people who are using online surveys are already using MMR, even if they have not decided to, and even if they are not aware of it. Any online survey, even if not designed for a mobile device, runs the risk of being completed on a mobile device by some respondents – unless specific measures are taken to avoid it.

Surveys sent to members of online access panels or to customers sourced from client databases are frequently completed via mobile devices – unless the survey has been designed to recognize the mobile device and designed to prevent a mobile device being used. This form of MMR is referred to as accidental mobile research, and the prevalence of accidental MMR is one of the reasons that it is safe to say that MMR, in 2013, has ‘arrived’.

As with most things mobile, reliable and consistent figures, about the prevalence of accidental MMR, are hard to obtain. In Europe and North I hear figures that vary from 5% to 20% , and a recent project from Hong Kong produced 20% using mobile devices – with about half using tablets.

Do you have estimates about the prevalence of MMR? All sources and figures submitted will be thanked and cited in our book and course.

Jan 162013

One of the questions I get asked fairly often is when should an answer list, in a survey, be randomised and when should it be presented in the same order to everybody.

In my opinion, the key issue is to think about how the respondent is answering the question in terms of:

  1. Does the respondent ‘know’ the answer? In which case the questionnaire needs to help them find their answer.
  2. Is the respondent looking at the answer list and picking the most applicable option or options? In which case randomising the list is highly desirable.

Examples of the first category, where people already know the answer are: How old are you (show the answers in ascending age), gender (nobody randomises male/female do they?), and where do you live (organise list alphabetically or regionally). I would also include, in this category, questions like what make of car do you own, what type of phone do you have, and I might include which of these supermarkets is your main one – if the list is short.

Examples of the second category, where the list guides the selection, include, which of these statements best describes your attitude to …., which of the following beers have you seen advertising for in the last month, which supermarkets do you ever use?

However, randomising can require more thought that simply clicking the randomise button in a survey scripting tool. If I want to ask which soft drinks have you drunk at least once in the last month, I want Coke, to appear next to Coke Light, Coke Zero etc. Sometimes, to help the participant, it is necessary to break the list into several lists, each randomised. So, when asking about which magazines has somebody read in the last three months, you might show a list of weekly titles, monthly titles, and online titles, rather than randomising them as one list.

Of course, even in a randomised list, key options, such as: Other, None of these etc, should remain fixed, at the bottom of the list.

One pedantic point is that researchers should avoid saying they have randomised the list to remove order effects. Every randomised list, the list as seen by a specific respondent, has order effects. The items near the top of that version of the list will be more likely to be picked and specific adjacency effects will exist (where seeing one item next to another changes the chance of it being selected). However, randomising flattens the order effects out, distributing them across all the items in the list and across all the participants, instead of focusing them on the items at the top of the list.

Do you have any guiding rules for when to randomise and when not to?

Jan 112013

At the moment, NewMR is running a survey asking the #NewMR community, amongst other things, what they would like to see us concentrate on in 2013. The survey is still running (if you have not given us your views yet you can find it by clicking here, but the results so far are interesting.

In the study we offered participants an array of 21 potential topics for NewMR events and asked them to select up to 4 that they would like us to run. The chart shows the results so far.

As the chart shows, the top three, at least at the moment, are Data Visualisation, Mobile Research, and Social media research – with about one-third of the participants selecting them.

The next grouping in the data comprises four choices, Big Data, Text Analytics, Behavioural Economics, and Presenting Results. The good news for NewMR is that our next event, on February 15th, is a Big Data event (and there are still spaces available). The inclusion of Presenting Results in this group, combined with the top ranking for Data Visualisation brings home the message that this is a topic that people are really interested in, and one that we will organises as soon as we can.

The next group of four have perhaps two important messages in them. The group includes Gamification, Online Qual, Advanced Quant, and Research Communities. The first lesson is that whilst there is some appetite for major mainstream topics, such as Online Qual and Advanced Quant, the interest is more niche than the top items. The second message is that perhaps the interest in presentations on Gamification and Research Communities may be less than it was.

The bottom ten contain all the regional specific ideas, such as Asia, Latin America, and Africa; Basic Training; and all of the established but specific areas, from Pricing to B2B, and from Conjoint to Semiotics.

NewMR will certainly take this feedback into account in planning our schedule for 2013. You can expect to see most of the items that came at the top of the list making an appearance. However, we will not forget the people voted for the items further down the list. NewMR’s role is to lead as much as it is to follow, so expect to see some new ideas, perhaps ‘Mobile Qual’ or a combination of ‘Semiotics, Ethnography, and Discourse Analysis’.

Jan 072013

Market research is being deluged with new sources of data, from social media, from electronic communications, and from research communities. Whilst some of this information is suitable for quantitative analysis, large parts of it are unstructured, for example tweets, posts, comments, and uploads. Whilst this data presents an interesting opportunity for market research, it presents a sequence of inter-connected problems and challenges:

  1. Many of the market researchers who are most proficient with unstructured data, the qualitative researchers, are not instinctively drawn towards online data, preferring to deal with people in a face-to-face environment.
  2. Many of the researchers most attracted to large amounts of online data, the hard core quantitative researchers, have little appreciation of the different epistemologies of quantitative and qualitative research.
  3. Many of the software vendors, perhaps in a rush to market, have released products before they were really ready and with massive over-claims.

In order for market research to fully leverage the potential benefits of the discourses being generated, market research needs to address the qualitative deficit. The qualitative deficit is the shortage of talent, software, and approaches designed to utilise massive amounts of qualitative data.

Some of the new approaches and skillsets that are needed relate to the process of quantifying messages within discourses – Google’s Flu Trends being a straightforward example (where Google use phrases that indicate people are searching for flu remedies to quantify the incidence of flu). This use of unstructured data will be of value in areas such as brand and ad tracking.

However, the bigger deficit relates to taking qualitative information and extracting qualitative findings. In online discussions the meaning is often unrelated to the frequency of words and phrases, the meaning rests in the structure of the conversation and the outcomes of the conversation. Feedback for product design, the identification of opportunities, core reasons for product dissatisfaction are likely to be found in the meaning of discourses, as opposed to counts of terms and phrases.

Whilst part of the answer will be new software, my feeling is that research urgently needs to expand the number of researchers with a good understanding of qualitative methods and epistemology.

Jan 022013

There are currently a wide range of discussions and blog posts talking about 2013 and beyond (you can join an interesting example in the #NewMR Linked group).

In my opinion (and the opinion of plenty of other people too), 2013 is going to be a big year for Asia. This will partly be in comparison to the poor state of the European economy and the challenges in the USA. However, Asia is making its own future. Many of the economies in Asia Pacific are growing strongly, the pool of talented researchers and research users is growing, and there is a tremendous energy about the way people do business in the Asia Pacific market in general, and in South-East Asia in particular.


From a personal point of view I am certainly putting my time where my mouth is. In October/November I was in Australia, Singapore, Hong Kong and Japan. In 2013 I am going to be in Asia Pacific more than any other region (other than home). In terms of meetings and sessions already booked in January to April 2013 I am going to be running workshops in India, Australia, and Hong Kong, and giving presentations in Australia, India, Hong Kong, Japan, Malaysia, Singapore, and Vietnam – I hope to add mainland China, probably Shanghai to that list.

NewMR is going to be increasing its focus on Asia in 2013, so stay tuned for more information – for example we are a media partner of the MRMW conference in Kuala Lumpur, in January, where I will be chairing one of the two days. My work colleagues at Vision Critical have clearly come to the same view about Asia Pacific, with offices in Australia, Hong Kong, and Japan, and with new offices just about to open in Singapore and Shanghai.

Of course, it can’t all be work, so I spent (part of) New Year’s day in the Brisbane Gallery of Modern Art (GOMA) viewing their 7th Asia Pacific Triennial of Contemporary Art (APT7), which is where the picture comes from – after all there is a chance that there is more to life than market research!

What are your thoughts? Is 2013 going to be a big year for Asia? Is the term Asia or Asia Pacific meaningful, or should we be talking about China more than the rest?