Mar 082014
Marina Bay Sands

This post is written as I reach the end of the first week of a three week Vision Critical trip to the Asia Pacific Region. For the last few years I have been spending about ten weeks a year in the APAC region, typically spread over three or four separate trips – because I am convinced that this is where much of the future (especially in terms of commerce, marketing, and insights) is being made.

Singapore Client Round Table
I arrived in Singapore Monday evening and the week got off to a flying start with breakfast with my Vision Critical colleagues from Sydney and from our newly opened Singapore office, followed by a meeting with the CEO of Indian partner, Majestic and lunch with an insight community client, Google. The afternoon was devoted to a client round-table meeting where several of Vision Critical’s clients gather to hear a keynote presentation (from me on this occasion) and then spend time sharing their learning with each other. This event was hosted by Google in their superb offices overlooking the Marina area, with key contributions from SingTel, Sony and others. Client roundtable sessions are a great way for clients to share their experiences with insight communities.

MRMW – Market Research in a Mobile World
Wednesday and Thursday was the APAC incarnation of MRMW, the leading global series of conferences on mobile market research, organised and promoted by Merlien. The keynote presentation was given by SingTel’s Melissa Gil, talking about how their three Vision Critical Insight Communities (Indonesia, Australia, and Singapore) provide them with rapid and cost-effective insight into digital consumers. One of the key points that SingTel made was that the speed and usefulness of the insights they produce mean that the SingTel insights team are involved in meetings and decisions at all levels of the business.

One of the key topics at the Conference was the evolving data protection picture in Asia and on the Tuesday Sue York from the University of Queensland (and curator of content at NewMR) moderated a panel on Data Protection, with Derek Ho (Senior Counsel from MasterCard), Dan Foreman (President of ESOMAR), Martin Tomlinson (Vice President of the Market Research Society of Singapore), and Stephen Jenke (Global Head of Data Collection at Kantar). The key points being made was that the picture on Asia was developing quickly, rules are becoming more onerous, and different countries have different rules.

Google Ray

One of the high points of the Conference was a presentation by David Zakariaie of Glassic who had brought ten sets of Google Glass with him to the event and who co-ran a session with me looking at the technology and the opportunities for market research to utilise this technology. Other key elements of the conference included: using feature phones as well as smartphones, utilising automated techniques for facial coding, video processing, and image processing (in all three cases the main theme was limited, but impressive, success), and moves towards geolocation and geofencing.

Effective Presentation Workshop
On the Friday I ran my “Secrets of Effective Presentations” workshop, which seemed to go down really well. I love workshops in multicultural situations as I am sure I learn as much as the attendees. Some of the secrets of creating and giving great presentations are global, but having a group from a wide range of countries (in this case Singapore, Cambodia, Saudi Arabia, and Australia) and with people who have a variety of first languages (and with a mixture of clients, suppliers, and academics) means that nothing can be taken for granted.

Key Singapore Takeaways
Compared with Europe and even with North America, Singapore embodies a ‘can do’ attitude, where the expectation is that tomorrow will be better than today, and that we are on a rapid path to a better, more technical, more insightful, richer society. Singapore also embodies the strength of cultural diversity. Most meetings with clients include people from a wide variety of countries. In order to get to Singapore, and in order to do well, most people have something special about them, and this tends to be blended to create something greater than the parts.


Jan 272014

As I and others (see this great post by Reg Baker in 2011) have been pointing out, people have been saying mobile is the next big thing for over 15 years, even in the days when that meant SMS, or WAP, or writing 100s of apps for different types of phones. At conferences and client sessions I keep being asked “So, when will mobile be the big thing?”

The answer is that it is now a big thing, and it has been for probably 18 months or more. There are lots of interesting aspects to why mobile is so important, its role in mobile ethnography, its role in audience measurement, its use in location based services, and the gloss that is the quantified self.

But there are two massive areas that outweigh all others. The first is online and the second is CATI.

I have been talking to a wide range of survey providers as part of the book on mobile market research I am writing with Navin Williams and Sue York (due out in September) and there is a consensus that about 20% to 30% of online surveys are being taken (or attempted) by people using mobile devices – and the number is still climbing. If you are doing online research you are almost certainly doing mobile research – the question is whether you are doing it well?

The second big indicator is the way that CATI centres are having to change their calling patterns to recruit more mobile phone users. The US Pew Center (a body that still try to do research right) recently changed its ratio so that 60% of its calls will be with people on mobile phones in the US. The picture is even stronger in the developing markets, interviewer face-to-face is moving to mCAPI, CATI is dominated by mobile (because that is what people have), and companies like Jana and TNS are conducting masses of research on feature phones using SMS or USSD.

So, passive data, iBeacon triggering, app collecting, HTML5 friendly, tap and squeeze mobile research is very interesting, and growing, but mobile becoming the dominant element in CATI and perhaps one-third of all online by the end of this year means it has arrived and it is very big – very big.

So, at last, we can all, me included, stop asking when mobile is going to arrive and focus more on the best uses today, and the best options for tomorrow.

Oh, and if you want to start wondering about the next big thing after mobile, start wondering when messaging apps, like WhatsApp, SnapChat and especially WeChat become the dominant form of interpersonal communication (displacing SMS on the way).


Jan 142014
Navin Headshot Square

Posted by Navin Wlliams, CEO MobileMeasure Consultancy Ltd, Shanghai, China.

Editors note: Navin is known as a leading expert of mobile market research and has been posting his forecasts for a few years and kindly shared his 2014 forecast with us.

2013 was a year of consolidation of the market in the mobile space. With Android settling into its leadership position and Apple’s iOS having a relatively good year too. The other operating systems have largely fallen by the wayside. With mobile finally settling the fight with online and finally being recognized as a tour de force too, it’s a good time to put on my soothsayer hat and share my vision of how 2014 is going to pan out.

OS sales shares

Source: Kantar Worldpanel ComTech, via CNET:

Winner Takes All – Android

In last year’s post I talked about Android dominating the mobile landscape and eventually by getting prices down to a sub US$100 mark make analysts relook at the Feature phone and Smartphone definition by the end of 2013. This has not really happened as we still have these definitions widely in use. However no one is obsessing over these definitions as it is well recognized that the formerly dominant Feature phones are a dying breed. From a competing set point of view the only recognized feature phone is Nokia’s S40 series which are now restricted to just the existing installed base, can only lose from here on as Nokia by adopting and then being sold to Microsoft becomes a Windows phone led platform. Other than the S-40 feature phones, the other feature phones in the market, dominated by the proprietary systems built by Chinese manufacturers disappeared overnight as they didn’t need to develop their own OS to save license costs, but could adopt an established OS in the form of the Android. So not only did Android eat into shrinking Nokia OS share, but the other feature phone OSs and the other dying OSs like Blackberry as well.

Android closed 2012 as the No. 1 player and in 2013 they helped grow the Phablet phenomena. And though in 2013 the Android Tablet share did grow, it’s their Phablets that got much more airplay. In 2014 expect to see Android phablets to dominate the landscape and in turn help them grow the share of Android Tablets as well.

Too Big to Fail Still ? – Apple & iOS

Considering the market dynamics in 2013, Apple and its iOS did admirably well. They didn’t :

  • grow their screen size significantly;
  • deliver a true low cost phone to take on the bottom half of the mobile handset pyramid;
  • come up with any serious ground breaking innovations.
Screen size: Apple has long maintained that the size of their screen was optimum and anything bigger will get too big to handle. They did however reduce the frame area and grow the screen diagonally larger. This however paled in comparison with the competing phone manufacturers churning out significantly larger screens and bringing to the fore a whole new category in “Phablets”.

Low Cost Phone: to be fair, Apple did bring out the 5C, which is US$649, which for most of the world is not exactly “C for Cheap” and still keeps it in the premium smart phone bracket. This phone though did not set the market on fire, it definitely went a long way in keep its Smartphone share up.

Innovation: Apple built its persona around product innovation, however in 2013 we haven’t seen much of it. Though wearable technology which syncs to your iphone has been talked about for a while now, the iwatch never materialized.

In 2012, Apple was a dominant force in the Smartphone market, in 2013 they fought valiantly and maintained their position in key markets but also lost share to a rampaging Android in 2013. In 2014, they will have to address all three issues of Screen size, Low Cost and Innovation or will it be the year that they lose a lot of the shine they have built around themselves. Will Apple play by the conventional rules or look to reinvent themselves and the industry as they often have in the past?

Mend and Grow year – Nokia & Windows Mobile

While 2013 was labeled as the make or break year for Nokia and Windows, 2014 is going to be mend and grow. If you take away the declining numbers of the final years of Nokia and look only at Windows as a Greenfield entry they have done admirably well in 2013. With the buying out of Nokia by Microsoft, it becomes the second non hardware technology company after Google –Motorola to enter the mobile hardware arena.

Though currently in the Smartphone market, Windows has done admirably well by being the third player often edging out Blackberry or a close forth behind Blackberry. 2014 is when we can truly evaluate its performance. In 2014 we can expect a flurry of launches of new models at multiple price points and screen sizes. Coupled with the formidable marketing arm of Microsoft and the former Nokia teams, expect them to at least double their market share across markets.

The New King – Samsung

A couple of years ago, Samsung was the challenger. Today, they are dominating the manufacturer’s sweepstakes trying to keep the other manufacturers at bay. After having introduced the Phablet with the Note phone series, they have grown the category exponentially and dominated it as well. In 2013 they showed that they are not just riding on Android and large screens to dominate the market place. They introduced the Galaxy Gear which included a watch phone with the top end Samsung Galaxy phones purchased. The much talked about iWatch in the blogsphere seems to have been missed by Apple and was picked up and dished out by Samsung instead. So Samsung is looking to take the lead in the innovation stakes to create a persona all its own. The Samsung Galaxy gear smart watch may turn out to be more of an interesting gimmick than a serious utility tool, but it’s too early to call.

In 2014 we can expect Samsung to continue to dominate the Smartphones especially the Phablets, larger screens and with their Galaxy Gear added to their arsenal, they also have a serious differentiator. On the not so bright side, of all the manufacturers Samsung has the most to lose as they like Nokia before them play across the price and product spectrum. They should have a tough year but hold onto their leadership position.

The Dead Dead – Blackberry

The highlight of 2013 for BB was the announcement of its all new Blackberry 10 OS which was delayed constantly and when it finally got launched was a little to late and didn’t introduce anything innovative to retain anyone’s attention. So from living dead to dead dead, is where BB finds itself.

Though to be fair to them they still have a large corporate business stuck at the hip and for many of these corporates wanting to migrate their secure mobile systems moving to another system is a tough choice. Something like self-amputation vs. rebuilding the Pyramids – both tough decisions. Many have bitten the bullet and others are still building plans. So there’s still an opportunity for BB to revive itself.

Out of the Shadows – Made in China

If we saw the Chinese manufacturers coming of age in 2013, expect 2014 to be their graduation party. The biggest impact is going to be from the Chinese manufacturers with the likes of Huawei, Lenovo and ZTE all making both affordable and top end Smartphones for the global market. Taiwanese manufacturer HTC is a fallen star hoping to revive their fortunes. And there are the Indian manufacturers who until now were content with the Indian market are getting ambitious and looking for foreign shores as well.

Expect at least 2-3 of the top 5 manufacturers to be Chinese manufacturers in 2014. It would be pretty safe to assume all 5 will be “made in China”.

Market Research and Mobile Market Research

Accidents Happen: If 2012 was for probing and experimenting, 2013 saw the term accidental mobile research being coined. Accidental mobile research is research which takes place from a mobile phone but is meant to happen online. And enough of these accidents happened for online survey providers to take notice and tweak their online surveys to identify if a user was indeed trying to fill out the survey from a mobile phone and therefore should see a more mobile friendly mobile rendered survey. However this did not really help as these may have been rendered for mobile but were not designed for mobile. They were designed for online and force fed on mobile, though it was within the confines and legible on the mobile screen. This is borne out of the bench mark tests done across online survey providers showing a consistent pattern of mobile exceeding online survey completion times by 50%. So while this number makes me very unhappy, from an industry point of view maybe it’s a point to rejoice as it’s a step ahead from “mobile is too small” to acknowledging that surveys need to be rendered appropriately for the mobile phone. In 2014, expect to see surveys designs to be mobile specific. A question is asked, which is answered via a drag and drop online, while the same question is answered using taps and selections on mobile enabling the times to be close or match when done either on the mobile or online by the survey respondent.

More Than a Talking Point: I was recently at a Qualitative conference (not a Mobile Market Research Conference) in Singapore and I noticed that only 2 speakers in the entire 2 days could get through without referring to Mobile in their presentations. Many like myself, presented specific mobile cases and examples. I actually spoke on the first day and a lot of what I was presenting was mentioned or covered before I came into speak later in the day. And often on most conferences I speak at, either I am the lone speaker (or niche) on a mobile topic be it a mobile or any other Research conference. Expect more informed discussion on the same in 2014.

Norms: One of the biggest barriers to entry often for research products and services is norms. Expect this to be re-evaluated across providers, as mobile is another animal. We will get different results as the context for mobile data makes the data different. So it’s not that the consumer is lying or the world has changed so much, just that mobile can contextualize a survey and give results that also differ. For instance when we do a pantry check survey on mobile, we request they go to the pantry and upload a picture as well and then fill the survey giving added insights from the mobile picture as well. The same survey done online though it had the option to take a photo and upload, no one did it as it was too cumbersome. Most in fact didn’t even visit the pantry and reported on memory. So eventually at the broad level (key brands) we got the same results, however it was the things that got left out in the responses of the online survey were the most revealing in the study. One of the additional insights was that some product packaging (like cereal boxes) was too big for standard pantries of the target group we were seeing a lot of when conducting the research.

Wearable Tech: 2013 saw Google Glass and other wearable tech like Samsung Galaxy Gear Smart Watch make headlines and showcased across the world. In 2014 expect both these technologies and other similar ones to penetrate more and more lives of consumers. As researchers we will get a ring side seat to the numerous market research applications and how the smartest (or fools) of us dives into it to extract the insightful gems that we all seek.

Messaging Apps: SMS is in decline, that was the big story of 2013. This came as Messaging Apps like WeChat and WhatsApp thrived. Though WhatsApp was the pioneer, it has taken a back seat to the innovative copy cat being WeChat. Not only did WeChat copy WhatsApp (or can you call building another messaging app as truly copying?), either way they didn’t stop there. They made it bigger and better, growing to 700+ million globally at last count. In 2014, expect more and more market researchers to dive in enhance the power of market research via mobile messaging Apps.

And finally a word of advice …

Youth: While the benefit of grey cells is never in doubt, especially in a knowledge industry like market research. In Mobile Market Research youth is a huge scoring point, with even non technical kids wield a mobile like it were putty in their hands. In 2014, go engage and learn from the young who are using the mobile intuitively as an extension of themselves and not as a device to be used – it’s part of them.

Go forth and mobilize. Enjoy your 2014!

Navin has close to two decades of experience spread across Market Research, Technology, Media & Telecom sectors. Having worked with the top global agencies spread over 4 countries and 2 continents; he’s had the opportunity to be part of MR technological adoptions over the years in diverse developing environments. Navin has spent the better part of the last decade on new media technology and his quest to drive Mobile adoption in Market Research led him to form MOBILEMEASURE CONSULTANCY.

A pioneer in mobile enabled MR, Navin is widely regarded as a thought-leader in the evolution of digital technology in MR, he has written a number of Whitepapers and is a frequent speaker at industry conferences. Recently he was involved in the designing and writing of the curriculum for University of Georgia’s Mobile Marketing Research course and is currently working on a book on Mobile MR.

Navin is based in Shanghai, China with his wife and two Children. He can be reached at


Jan 012014

Ray Poynter, Navin Williams, and Sue York are writing a book on mobile market research, which will be published in August/September by Wiley, with the support of ESOMAR. The book has been provisionally titled, The Handbook of Mobile Market Research, and is a companion to The Handbook of Online and Social Media Research.

Are you or one of your colleagues, or your organisation, interested in helping us in any of the following ways?

  1. 1. Reviewing one or more chapters and letting us have your thoughts and suggestions?
  2. 2. Supplying case studies or Research on Research to help illustrate points in the book? Ideally, material that has already been published on your website, at a conference, or in articles.

We will, of course, fully cite and credit any help you and your organisation are able to offer.

Timelines are horrendous, of course! We’ve finished the first draft of the book and sent it to the publisher. This draft is very rough, if you have a look at any of the chapters you will spot errors and notes to ourselves in the text. The final text is being sent to the publisher 31st January, so we’d need any feedback or help before then.

So, if you are able to help, please email and tell me which draft chapters you’d like us to send you. You are welcome to ask for as many or as few as possible. In doing so you are of course agreeing to not make these draft chapters widely available, as they are the copyright of Wiley, the publisher.

The chapters are: (the titles will change a bit)

Dec 232013
The material below is an excerpt from a book I am writing with Navin Williams and Sue York on Mobile Market Research, but its implications are much wider and I would love to hear people’s thoughts and suggestions.

Most commercial fields have methods of gaining and assessing insight other than market research, for example testing products against standards or legal parameters, test launching, and crowd-funding. There are also a variety of approaches that although used by market researchers are not seen by the market place as exclusively (or even in some cases predominantly) the domain of market research, such as big data, usability testing, and A/B testing.

The mobile ecosystem (e.g. telcos, handset manufacturers, app providers, mobile services, mobile advertising and marketing, mobile shopping etc) employs a wide range of these non-market research techniques, and market researchers working in the field need to be aware of the strengths and weaknesses of these approaches. Market researchers need to understand how they can use the non-market research techniques and how to use market research to complement what they offer.

The list below cover techniques frequently used in the mobile ecosystem which are either not typically offered by market researchers or which are offered by a range of other providers as well as market researchers. Key items are:

  • Usage data, for example web logs from online services and telephone usage from the telcos.
  • A/B testing.
  • Agile development.
  • Crowdsourcing, including open-source development and crowdfunding.
  • Usability testing.
  • Technology or parameter driven development.

Usage data

The mobile and online worlds leave an extensive electronic wake behind users. Accessing a website tells the website owner a large amount about the user, in terms of hardware, location, operating system, language the device is using (e.g. English, French etc), and it might make an estimate of things like age and gender based on the sites you visit and the answers you pick. Use a mobile phone and you tell the telco who you contacted, where you were geographically, how long the contact lasted, what sort of contact was it (e.g. voice or SMS). Use email, such as Gmail or Yahoo, and you tell the service provider who you contacted, which of your devices you used, and the content of your email. Use a service like RunKeeper or eBay or Facebook and you share a large amount of information about yourself and in most cases about other people too.

In many fields, market research is used to estimate usage and behaviour, but in the mobile ecosystem there is often at least one company who can see this information without using market research, and see it in much better detail. For example, a telco does not need to conduct a survey with a sample of its subscribers to find out how often they make calls or to work out how many texts they send, and how many of those texts are to international numbers. The telco has this information, for every user, without any errors.

Usage data tends to be better, cheaper, and often quicker than market research for recording what people did. It is much less powerful in working out why patterns are happening, and it is thought (by some people) to be weak in predicting what will happen if circumstances change. However, it should be noted that the advocates of big data and in particular ‘predictive analytics’ believe that it is possible to work out the answer to ‘what-if’ questions, just from usage/behaviour data.

Unique access to usage data
One limitation to the power of usage data is that in most cases only one organisation has access to a specific section of usage data. In a country with two telcos, each will only have access to the usage data for their subscribers, plus some cross-network traffic information. The owner of a website is the only company who can track the people who visit that site (* with a couple of exceptions). A bank has access to the online, mobile and other data from its customers, but not data about the users of other banks.

This unique access feature of usage data is one of the reasons why organisations buy data from other organisations and conduct market research to get a whole market picture.

* There are two exceptions to the unique access paradigm.
The first is that if users can be persuaded to download a tracking device, such as the toolbar, then that service will build a large, but partial picture of users of other services. This is how is able to estimate the traffic for the leading websites globally.

The second exception is if the service provider buys or uses a tool or service from a third party then some information is shared with that provider.

A complex and comprehensive example of this type of access is Google who sign users up to their Google services (including Android), offer web analytics to websites, and serve ads to websites, which allows them to gain a large but partial picture of online and mobile behaviour.

Legal implications of usage data
Usage data, whether it is browsing, emailing, mobile, or financial, is controlled by law in most countries, although the laws tend to vary from one jurisdiction to another. Because the scale and depth of usage data is a new phenomenon and because the tools to analyse it and the markets for selling/using it are still developing the laws are tending to lag behind the practice.

A good example, of the challenges that legislators and data owners face is determining what is permitted and what is not, are the problems that Google had in Spain and Netherlands towards the end of 2013. The Dutch Government’s Data Protection Agency ruled in November 2013 that Google had broken Dutch law by combining data together from its many services to create a holistic picture of users. Spain went one step further and fined Google 900,000 Euros for the same offence (about $1.25 million). This is unlikely to be the end of the story, the laws might change, Google might change its practices (or the permissions it collects), or the findings might be appealed. However, they illustrate that data privacy and protection are likely to create a number of challenges for data users and legislators over the next few year.

A/B testing

The definition of A/B testing is a developing and evolving one; and it is likely to evolve and expand further over the next few years. At its heart A/B testing is based on a very old principle, create a test where two offers only differ in one detail, present these two choices to matched but separate groups of people to evaluate, and whichever is the more popular is the winner. What makes modern A/B testing different from traditional research is the tendency to evaluate the options in the real market, rather than with research participants. One high profile user of A/B testing is Google, who use it to optimise their online services. Google systematically, and in many cases automatically, select a variable, offer two options, and count the performance with real users. The winning option becomes part of the system.

Google’s A/B testing is now available to users of some of its systems, such as Google Analytics. There are also a growing range of companies offering A/B testing systems. Any service that can be readily tweaked and offered is potentially suitable for A/B testing – in particular virtual or online services.

The concept of A/B testing has moved well beyond simply testing two options and assessing the winner, for example:

  • Many online advertising tools allow the advertiser to submit several variations and the platform adjusts which execution is shown most often and to whom it is shown to maximise a dependent variable, for example to maximise click through.
  • Companies like Phillips have updated their direct mailing research/practice by developing multiple offers, e.g. 32 versions of a mailer, employing design principles to allow the differences to be assessed. The mailers are used in the market place, with a proportion of the full database, to assess their performance. The results are used in two ways. 1) The winning mailer is used for the rest of the database. 2) The performance of the different elements are assessed to create predictive analytics for future mailings.
  • Dynamic pricing models are becoming increasingly common in the virtual and online world. Prices in real markets, such as stock exchanges have been based for many years on dynamic pricing, but now services such as eBay, Betfair, and Amazon apply differing types of automated price matching.
  • Algorithmic bundling and offer development. With services that are offered virtually the components can be varied to iteratively seek combinations that work better than others.

The great strength of A/B testing is in the area of small, iterative changes, allowing organisations to optimise their products, services, and campaigns. Market research’s key strength, in this area, is the ability to research bigger changes and help suggest possible changes.

Agile development

Agile development refers to operating in ways where is it easy, quick, and cheap for the organisation to change direction and to modify products and services. One consequence of agile development is that organisations can try their product or service with the market place, rather than assessing it in advance.

Market research is of particular relevance when the costs of making a product are large, or where the consequences of launching an unsatisfactory product or service are large. But, if products and services can be created easily and the consequences of failure are low, then ‘try it and see’ can be a better option than classic forms of market research. Whilst the most obvious place for agile development is in the area of virtual products and services, it is also used in more tangible markets. The move to print on demand books has reduced the barriers to entry in the book market and facilitated agile approaches. Don Tapscott in his book Wikinomics talks about the motorcycle market in China, which adopted an open-source approach to its design and manufacture of motorcycles, something which combined agile development and crowdsourcing (the next topic in this section).


Crowdsourcing is being used in a wide variety of way by organisations, and several of these ways can be seen as an alternative to market research, or perhaps as routes that make market research less necessary. Key examples of crowdsourcing include:

  • Open source. Systems like Linux and Apache are developed collaboratively and then made freely available. The priorities for development are determined by the interaction of individuals and the community, and the success of changes is determined by a combination of peer review and market adoption.
  • Crowdfunding. One way of assessing whether an idea has a good chance of succeeding is to try and fund it through a crowdfunding platform, such as Kickstarter. The crowdfunding route can provide feedback, advocates, and money.
  • Crowdsourced product development. A great example of crowdsourcing is the T-shirt company People who want to be T-shirt designers upload their designs to the website. Threadless displays these designs to the people who buy T-shirts and asks which ones people want to buy. The most popular designs are then manufactured and sold via the website. In this sort of crowdsourced model there is little need for market research as the audience get what the audience want, and the company is not paying for the designs, unless the designs prove to be successful.

Usability testing

Some market research companies offer usability testing, but there are a great many providers of this service who are not market researchers and who do not see themselves as market researchers. The field of usability testing brings together design professionals, HCI (human computer interaction), ergonomics, as well market researchers.

Usability testing for a mobile phone, or a mobile app, can include:

  • Scoring it against legal criteria to make sure it conforms to statutory requirements.
  • Scoring it against design criteria, including criteria such as disability access guidelines.
  • User lab testing, where potential users are given access to the product or service and are closely observed as they use it.
  • User testing, where potential users are given the product or given access to the service and use it for a period of time, for example two weeks. The usage may be monitored, there is often a debrief at the end of the usage period (which can be qualitative, quantitative, or both), and usage data may have been collected and analysed.

Technology or parameter driven

In some markets there are issues other than consumer choice that guide design and innovation. In areas like mobile commerce and mobile connectivity, there are legal and regulatory limits and requirements as to what can be done, so the design process will often be focused on how to maximise performance, minimise cost, whilst complying with the rules. In these situations, the guidance comes from professionals (e.g. engineers or lawyers) rather than from consumers, which reduces the role for market research.

Future innovations

This section of the chapter has looked at a wide range of approaches to gaining insight that are not strengths of market research. It is likely that this list will grow over time as technologies develop and it is likely to grow as the importance of the mobile ecosystem continues to grow.

As well as new non-market research approaches being developed it is possible, perhaps likely, that areas which are currently seen as largely or entirely the domain of market research will be shared with other non-market research companies and organisations. The growth in DIY or self-serve options in surveys, online discussions, and even whole insight communities are an indication of this direction of travel.

So, that is where the text is at the moment. Plenty of polishing still to do. But here are my questions?
  1. Do you agree with the main points?
  2. Have I missed any major issuies?
  3. Are there good examples of the points I’ve made that you could suggest highlighting/using?

Jul 172013

Following the discussion on tablets in mobile market research, this post addresses the wider issue of why somebody would want to conduct a study that is mobile only.

Having spoken to a wide cross section of clients and researcher, typical reasons for a mobile only study seem to include:

  1. Because the data needs to be collected, or is better if collected, ‘in the moment’. Where ‘in the moment’ typically means as people are making a decision, whilst using something, or immediately after using something.
  2. To collect passive data, as people go about their everyday lives.
  3. Because mobile gives a more appropriate sample than other similar methods. For example, in a country where 80% of economically active adults have a phone and 50% have internet access, mobile can provide the better sample.
  4. In order to change CAPI to mCAPI, re-energising CAPI.
  5. To add items like photos and videos to traditional survey responses.
  6. Where the mobile device can assist or improve data logging and collection.
  7. Research on the mobile ecosystem, for example of mobile advertising and campaigns.
  8. To research mobile data collection, part of what researchers call RoR, research-on-research.

Another example of point 3, a more appropriate sample, is provided by French company ELIPSS who have created a panel of people, selected via random probability, to whom they have given an internet connected tablet, creating a sample source that is both internet enabled and broadly representative of the group they are seeking to represent.

Two items which are currently not on the list are a) to be cheaper, b) to be faster. This may change in the future and faster and perhaps cheaper could become drivers of mobile usage. But here is why we don’t see them as drivers at the moment

There is doubt that mobile will be cheaper than online surveys in the foreseeable future for like-for-like surveys. The cost of programming a study for online and mobile is, at its best, the same. Testing for online and mobile is, at its best, the same. Incentives, are at their best, the same. And, the processing costs are typically the same. In fact, at the moment, mobile studies typically cost more to program and test since there are more contingencies to consider and to deal with.

However, if the desire to use mobile drives researchers to use shorter surveys, the net effect could be cheaper studies – as well as better and faster.

Faster is a plausible benefit for mobile, although this is a matter of degree. When online research, coupled with online access panels, burst on the scene, one of the key benefits was speed, days instead of weeks. In terms of mobile the speed difference for data collection is likely to be hours instead of days. However, this does not mean that most project turnarounds will reduce by the same factor. A project includes, design, scripting (the writing and testing of the survey), the fieldwork, and the analysis. Reducing the fieldwork from, say, 48 hours to 4 hours might reduce a project from five days to four days – good, but only crucial sometimes.

However, mobile data collection may come into its own when researchers start requesting, near, instant results. Consider the launch of a campaign, or assessing an open-air event, or dealing with the impact of a product disaster like a recall, a mobile survey could be sent out within minutes and a broad, cross-section of people could reply within minutes, potentially allowing real-time management of the campaign, event, or news.

In many cases there are methodological reasons to want the fieldwork to last at least 24 hours, and potentially longer. Different times of day attract different sorts of respondents. Researchers have reported that responses in the morning can be different from responses collected in the evening, and quite often that the first third of responses are different from the last third – although this may be due to more than just speed as the last third is often the part of a sample where there is a struggle to fill quotas – i.e. the last third are often demographically different from the first third.

Big shout of thanks to Frankie Johnson for highlighting mCAPI in relation to the previous post, and to Gerry Nicolaas for bring Elipss to my attention.

So, what are your thoughts? Would you make any major additions, deletions, or amendments to our list? Are you aware of interesting examples of people doing some of the less common alternatives?

Jul 122013

As mentioned before, I am in the midst of co-writing a book on mobile research and today I have been working my through the contrasting roles of phones, PCs, and tablets in quantitative research, specifically with respect to surveys.

The discussion about phones was relatively straightforward, covering both studies designed specifically for phones, and studies where phones might be used by some of the respondents, whilst others used, for example, a PC.

However, when I came to write the section on tablets, I came to the realisation that not only are surveys not normally written for tablets at the moment, they are unlikely to be written specifically and solely for tablets in the future.

Tablets can be used for surveys written for phones, tablets can be used for surveys written for PCs, but why would a survey be written for tablet in such a way that it was not suitable for a smartphone AND not suitable for a PC?

Possible reasons for writing a tablet only study might focus on having a large touch screen, but that seems like a niche. A tablet only survey could be written for a specific game, but again, that would be a niche. Of course, a survey about tablets might be designed to be tablet only, but once again a niche.

It is easy to see mCAPI (mobile CAPI) and qualitative uses that are specifically tablet. But unless somebody can see something I am missing, there seems to be little need for a genre that is tablet only surveys?

Unless you know better?

ps The title “The tablet that didn’t bite” is a loose reference to a phrase used in a Sherlock Holmes story (The Sliver Blaze), where the non-appearance of something revealed a vital clue.

Jul 052013

As I have mentioned before, I am involved in writing a book on mobile market research, with Navin Williams and Sue York. As part of that process we will be posting elements of our thinking and snippets of the book to NewMR in order to crowd-source improvements. Here is one such snippet, it is the first page of a chapter on mobile qualitative research. We would love to hear your thoughts.

Mobile Specific Qualitative Research
This chapter looks at qualitative market research techniques that have been created by, or heavily impacted by, the arrival and utilisation of mobile devices. A separate chapter looks at how mobile devices are being incorporated into other, more traditional, forms of qualitative research (for example, in online focus groups and discussions, or in connection with face-to-face qualitative approaches).

Topics covered in this chapter include:

  • Mobile ethnography: where participants captures slices of their lives, or the lives of people around them, as an input to an ethnographic analysis.
  • Mobile diaries: where participants record their activity in relation to a specific topic, for example during the purchase of a mortgage, or whilst on a journey.
  • Triggered recording: where participants record their interactions with some external factor, for example, every time they see and advert for a particular category.
  • Qualitative tracking: This approach uses passive tracking, i.e. the phone uses its features and sensors to record where the participants go, what they do, etc; without any moment-to-moment intervention from the participants. These traces are then reviewed by the researcher as an input to their qualitative analysis.

To some extent, some of these approaches show a degree of overlap. For example, in mobile ethnography, mobile diaries, and triggered recording a participant might be asked to create a message when a specific event happens, they might be asked to take a photo or record a video, or to record how they feel. The difference tends to be the balance between the activities, the reason for the research, and how the research will be analysed. For example, in a mobile diary project the participants’ descriptions may be the key deliverable, in an ethnography it is the analysis and write-up that is the key element of the project.

Several of these mobile qualitative approaches use data collection methods that are similar to mobile quantitative techniques. For example, qualitative mobile diaries might be used to follow 20 participants, capturing their thoughts and experiences in relation to some activity, such as every time they have a drink during the day, capturing open-ended comments and images. A quantitative mobile diary study might be based on 400 participants and be based on the answers to closed questions, captured every time the participants drink something. Similarly, qualitative tracking might look at twelve people for several days, and the analysis might include sitting with the participants and reviewing the trace information to build a rich picture what has happened. A quantitative project might be based on 600 people and the analysis based on using software to find patterns in the data, e.g. sequences of actions, or typical routes.

This chapter reviews each of these approaches, providing practical advice, case studies, and methodological notes.


  1. I would love to hear from people with case studies they would like to share, either in the book or on our mobile resources page.
  2. Is mobile specific qualitative research a suitable term for this collection of approaches?
  3. Would you add any techniques to this list?
  4. Would you change the names of any of these four approaches?
  5. Do the first three really constitute three different approaches, or would they be better rolled into a single item?

Jun 292013

The ITU (the International Telecommunication Union, the UN agency that looks after ICT – information and communication technologies) has produced a useful update on ICT facts and figures.

The report is well worth reading and shows, amongst other things:

  • As more and more mobile phones are bought, the growth is slowing. In 2005/6 the global growth rate in cellular subscriptions was just under 25%. In 2012/13 it was down to just over 5%. In the developing world the growth has fallen from over 30% in 2005/6 to just over 6% now. None of which is surprising, but it is nice to know the numbers.
  • The internet continues to grow in all regions and globally. With 77% in the developed world having internet access, and 31% in the developing world.
  • Globally just under 3 billion people are using the internet, almost 40% of the population.
  • About 50% of the households with access to the internet are in the developing world (although that is a much lower penetration rate than in the developed world, 28% in the developing world and 78% in the developed world).
  • Fixed-broadband is much cheaper in the developed world than the developing world, although the price has been falling in the developing world. Costs in the report are measured as a percentage of GNI (Gross National Income – roughly, the amount the whole country earns) per person. In the developed world fixed broadband costs under 2% of average monthly income, in the developing world it costs over 30% of average monthly income.
  • Fixed broadband in the developing world is growing, but is still only 6%, compared with 27% in the developed markets. However, over 50% of the households with fixed-broadband are in the developing world, because it is larger.
  • The four countries with the highest percentage of their fixed-broadband being high-speed are: South Korea, Hong Kong, Japan, and Bulgaria.
  • Mobile broadband subscriptions have grown from under 300 million in 2007, to 2 billion in 2013.
  • In the developing countries mobile broadband is more expensive than it is in the developed markets, but cheaper than fixed broadband in the developing markets.
  • In Africa mobile broadband subscriptions cost about 50% of average income, compared with less than 2% in Europe.

The ITU is 100% wrong on penetration
So, it is a pity that the ITU refer to a highly misleading statistic in their report, which challenges the value the way that data from the ITU will be considered. And, it is a pity that some people in and around the market research world have picked up on this misleading number.

What is this misleading statistic? I am referring to the part of the report where the ITU says that the penetration of mobile-cellular is 96% globally and approaching 100%. It then compounds its dodgy use of language when it describes the penetration in the developed world as 128%, and describes mobile-cellular penetration as 170% in the CIS (a subset of the countries that used to be in Soviet Union, including Russia).

Let’s just think about 100% for one moment. In the way we normally use the phrase (for products, diseases, education, services) 100% would mean every baby, every prisoner, every homeless person would have one. For example, when we estimate the penetration of a TV show we interview a representative sample and gross up to the population. Clearly, it would be a nonsense to claim that 100% of people have a mobile phone. By the time we get to 170%, we can see that the ‘normal’, or useful definition of penetration is not the one they are using.

So, what do the reports of 100% penetration mean? Read the non-nonsense bits of the ITU report and you will notice that the team who have produced the charts (as opposed to the copy) refer to mobile-cellular subscriptions, and mobile-cellular subscriptions per 100 people. It is a pity that the copywriters did not follow the lead of the ITU people who worked on the charts.

What are mobile-cellular subscriptions? Very roughly, the number of subscriptions is the number of sims in use. If somebody has two phones, that is two sims, two subscriptions. If somebody has a dual-sim phone, that is two sims, and is often two subscriptions. If somebody has two phones, a tablet, and a mobile modem, they have four sims.

Am I just being pedantic, or does it matter? Yes, in my opinion it matters. Because people are quoting these super high ‘penetration’ rates there is an assumption that catering for mobile phone users, in and of itself, avoids excluding people. We can use the UK as a good example. The ITU figures for the UK, in 2011, says there were 131 subscriptions per 100 people – a figure the ITU copywriters and careless MR tweeters would call 131% penetration. However, the UK’s General Lifestyle Survey found that in 2011 one-in-seven households had zero mobile phones (i.e. 86% of households had at least one person in it who had at least one mobile phone). Data collected in the UK by the communication regulator (Ofcom) estimate that at the end of 2012 92% of adults owned or had the use of a mobile phone.

In the developed markets, such as the UK, the difference between a penetration rate of 131-132% of the total population (babies and all) and a real rate 0f 86-92% of adults is not particularly important. But if the ratio in the UK is typical, the ITU figure of 100% global could mean about two-thirds of adults have the use of a mobile phone, and that does matter. For example, it means research projects requiring a good representation of people, in some countries, cannot assume that mobile is currently a safe option.

Jun 102013

As I mentioned in earlier posts. NewMR is involved in the creation of a new book, provisionally called the Handbook of Mobile Market Research. We will be publishing a lot of our work online, as the book progresses, to share our learning, to invite comments, and hopefully elicit extra material. Much of the material we are gathering is available via our Mobile Market Research Resources page.

The post published on this page is a piece of ‘work in progress’ from one of the chapters in the new book. The chapter will look at key debates in mobile market research, and this post addresses the question “How do clients move 20 to 30 minute tracking studies onto smartphones?“. We have access to some raw data and studies to back up the points in this post, but we’d love to have more, and I have flagged up in the post where we are particularly looking for more material. So, if you’d like to contribute: comment here, comment in the NewMR LinkedIn group or email us via

Note, this work remains our copyright, or at least until it is transferred to the publishers. If you use it, or quote from it, please cite the source.

How do clients move 20 to 30 minute tracking studies onto smartphones?

There is a general view that one of the things that has slowed down the development of mobile marketing research has been the problem of how to move a 20 to 30 minute survey onto a phone. This problem seemed insurmountable in the days of feature phones, but even now, with smartphones becoming ever more common, there are no clear answers to the problem.

The problem appears to have two key elements:

  1. The belief that people will not take part in 20 to 30 minute surveys on their mobile phone.
  2. The belief that many research projects require long surveys, for example brand trackers, ad tracking, some U&A studies, and many customer satisfaction studies.
If both of these elements are true, then a substantial part of market research will not transfer to smartphones (it may, of course, still transfer to mobile via tablets). So, it is worth examining both of these elements in more depth.

People won’t do long interviews on their smartphones

Some of the people who put forward this proposition base it on common sense and personal experience. If we interrupt people during their busy day, to do a survey on their ever-present smartphone, they won’t have the time or the inclination to complete a long survey. Also, since the smartphone screen is small and the interface fiddly, it will be too onerous to do a long survey. However, common sense and personal experience are often a bad guide to what people actually do. Long surveys would not need to be synonymous with people completing them whilst busy, and plenty of people seem very happy with using their smartphone for extended periods of time – as any journey on a train will confirm.

One point that researchers should bear in mind is that when CATI appeared on the seen the consensus was that interviews needed to be short, but over time they became longer. When online research appeared (in the mid-1990s), early movers such as Pete Comley [REF:] said that interviews needed to be short, about five minutes, and certainly not longer than eight. However, both CATI and online went on to be used for longer and longer studies, and 40 minute studies are not rare these days.

If the experience of CATI and online are reviewed, the picture seems to be:

  1. There are a large number of people who are not willing to take part in market research surveys at all
  2. There are a large number of people who will sometimes take part in market research surveys, typically because the survey is short, not too boring, and they have been asked at the right time (asking nicely helps too).
  3. There are a group of people who will do a large number of surveys, and some of them will do quite long surveys, in return for incentives. For example, these are the people who sign-up to online access panels.

The evidence?
The evidence, so far, falls, broadly, into two groups. The first relates specifically to projects conducted to test mobile market research. The second relates to respondents who have used their mobile devices to take part in surveys that were solely, or mainly, intended for people using PCs – the type of mobile market research that is often referred to as unintentional mobile.

Mobile specific studies into drop-off rates
Many studies have reported that there are few problems in finding respondents willing to do ten minute surveys on the smartphone. With several studies indicating a severe increase in dropout at a point between ten and fifteen minutes.

Unintentional mobile market research
It would appear that depending on the panel, some 5% to 15% of surveys intended for online via PC are being completed on smartphones, including surveys over 20 minutes in length.

Summary of whether people will do long surveys on their smartphones
Although most research pundits and opinion leaders believe that mobile surveys should be short (indeed they tend to believe that online, CATI, and face-to-face should be relatively short too), the evidence suggests that researchers are faced with a choice.

Long mobile surveys are possible, if researchers are willing to reduce the population who are prepared to take part in their surveys. This is the decision they have made when dealing with CATI and online, so it is perfectly possible that some, perhaps the majority, of researchers will be willing (over a period of time) to trade of the breadth of the population they are surveying in order to ask the sorts of surveys that they or their clients think are necessary. It is likely that the growth in access panels with large numbers of mobile users will facilitate this choice.

Many projects require long surveys

Many types of market research surveys have become longer over the years. There seem to be several forces driving this process:

  • There are more brands and brands have more variants than in the past.
  • Different parts of the business want to add their questions to existing studies, especially as budgets become tighter.
  • Techniques such as driver analysis tend to require a wide range of topics to be measured – often resulting in grids in the surveys.
  • Legacy issues mean it is easier to add a new measurement than to remove an old one.
  • KPIs are often linked to specific questions in surveys, meaning they take on a life of their own.
But perhaps the main reason that surveys have become longer is that market researchers have found ways of persuading, some, people to do longer surveys. By creating convenient samples, e.g. online access panels, of people who will take part in long surveys for incentives, market research has created the opportunity for long surveys.

The alternatives to long surveys

Several alternatives to long surveys have been put forward, but most of them have not become widely popular, and none are yet the norm.

Review: this is the simplest and probably most common way of shortening a survey. All of the stakeholders are interviewed to find out what their current priorities are. The survey is subject to analysis, for example to identify correlations between the measures and to identify which measures are not measuring anything useful. The intended result is a shorter survey. However, even when this method is employed successfully, it tends to be a temporary solution, as the survey often starts to grow again.

Partial Data: Partial data refers to asking different participants different questions, in order to build a total picture. One method of doing this is to split answer lists, and even whole questions, across different respondents. Normally, in these cases, the researcher preserves a core that is the same, to allow the data set to be analysed meaningfully. One implication of this approach is that the sample size needs to be increased, if the sample size per question is to remain at the pre partialising level.

Another technique for working with partial data is to use Hierarchical Bayes (HB). For example, HB is often used in Discrete Choice Modelling (DCM) studies. Each respondent sees a subset of the tasks, and HB is used to calculate the utilities for each respondent. Note, in these cases HB is not used to estimate the stated values, it is used to estimate the implied or revealed values.

Splitting the survey over time: In this approach, the respondent completes the survey over a period of time, as a set of smaller surveys. This method underlies some of the popularity of insight communities, where the longitudinal nature of the relationship means that questions can be asked in short surveys, without the need to re-ask things like demographics.

One issue for the researcher to keep in mind, if using this approach, is that the key sample size issue is how many participants complete all the steps. If a survey is broken into, say, three units. The base for some of the analysis will be people who completed the first, second, and third elements, and there is normally some drop off between the stages.

Re-thinking: For some researchers the most attractive option to shorter surveys is to re-think the whole process. The question, in this case, becomes not how can we make this shorter, but what do we really need to do to answer the clients business needs?

Different researchers and thinkers have come up with different ideas. Probably, the idea that has had the greatest impact on the research industry was Fred Reichheld’s proposal that the NPS (Net Promoter Score) was the one number that companies needed to measure, however, and perhaps perversely, market research has tended to incorporate NPS measurements into long surveys, rather than replacing them. Some researchers have looked at replacing grids and attribute batteries with open-ended questions, seeking to analyse them with automated text analytics. Other researchers have looked to reduce their tracking studies by collecting more information from social media, and restricting the survey to just those elements that their social medial monitoring does not capture. However, none of these routes has systematically and widely reduced the length of surveys to date.

One interesting contribution to the thinking about shorter surveys was presented to the 2012 ESOAMR 3D Conference, when Alice Louw and Jan Hofmyer, showed how flawed much of the thinking behind long surveys was, and proposed focusing on just those elements where respondents could provide meaningful information. Although this paper has not turned into specific research approaches, at least not ones that the industry has adopted, it perhaps shows the degree of radical thinking that is required to create real change?

Summary of whether longer surveys are needed?
This question about whether longer surveys are needed almost misses the point. Whilst longer surveys are possible, it is likely that clients will continue to use them. This may change is providers can come up with something which is either cheaper or which is dramatically better.

Overall Summary

The common view is that a large part of client’s research spend is on long surveys, that these surveys can’t readily be made shorter, and that smartphone based mobile surveys need to be short. People who believe this to be true will tend to keep their surveys as PC based online for the foreseeable future. For these people, mobile will be a method of tacking other problems, but not the problems which they currently associate with long surveys.

However, it is likely that some respondents will, for a fee, be willing to do long surveys on their mobiles, so this will probably happen. Researchers, and the users of research, should note that it is likely that these populations will be even more dissimilar from the total population than the population of people willing to be an active member of an online access panel.

The hunt is still on for a method of shortening long surveys that is cheaper than long surveys, as fast as long surveys, and good enough. Time will tell whether such a solution is found.

We’d love to hear your thoughts. Is this a useful review of a key question? Do you have a different view? Do you have data or studies you’d like to share?

p.s. I like to include a relevant image for each post, but given that there are likely to be 50+ posts on mobile research over the next few months, posts of different pictures of mobile phones are likely to become tedious. So, in this series of posts, the images will be ones I have taken with my mobile phone.