How data capture is changing the way we research

From 2Bs to bots

90% of all the data in the world has been created in the last two years. With the arrival of automated data capture, search marketing, user testing, online polls, chatbots and community forums, forms and 2B pencils are a thing of the past. But what form of data capture is best suited to your business?

As a teenager, one of my summer jobs was product testing for a market research company. My mum would drop me off at the corporate headquarters where I’d spend the next two hours sitting in a sterile white booth, tasting chips delivered to me through a hatch in the wall, and filling out a 50 page survey with a 2B pencil describing the level of saltiness of the tiny morsel of food I’d been given. The job bored me to death, I wanted more chips, but the $40 payout seemed worth it at the time!

Reflecting on it now, if I thought completing the survey was bad, what about the poor person who needed to decipher my handwriting and enter it into a spread sheet at the other end? What a laborious task.

It’s little wonder 90% of all the data in the world has been created in the last two years. With the arrival of automated data capture, search marketing, user testing, online polls, chatbots and community forums, teenagers with 2B pencils are a thing of the past.

With more options available, more companies are performing studies – not just the guys who can afford it. What this means is a higher demand on users to take part in surveys.

But just because capturing data has become easier, it doesn’t mean filling out a survey is more enjoyable. So where the focus of market research used to be on writing the perfect set of questions, it now includes creating an experience people want to take part in. Because there’s no point in writing questions if no one wants to answer them.

So, how do you get creative with data capture? To give you a starting point, we’ve reviewed some of the most popular digital formats for market research from a researcher and a customer perspective:


First off, you need to determine if you’re after specific insights or a big picture summary. If it’s quantitative data, a survey will be the easiest and most efficient way to get to the point quickly, as customers are more likely to enter information in one sitting than return frequently to a discussion page to check what has been said.

Surveys are the most comfortable choice for non-tech-savvy users, because they are similar to the paper forms we’ve been filling out for years. However, if you want to avoid boring your audience to tears, a custom experience is a must. There are plenty of platforms that allow you to build surveys with animation and images.

But beware: the upside of surveys – familiarity – can also be their downfall. Based on past experiences, customers might not even open your survey at all if they think they know what it will be like. This is where incentives, time estimates, progress bars and games will help persuade users to give the survey a go. For example, the QLD government created the Health and Fitness Age calculator which collects information about health and eating habits, but, as an incentive, at the end calculates an age to represent the user’s health.





Forums are like the online version of a focus group. They are a great way of gathering opinions and feedback on an idea or product. While it can be hard to get measurable data for a report out of a group discussion, allowing the conversation to take an organic journey can be helpful for uncovering unexpected insights.

The biggest challenge in creating a successful forum is making it a tool people actually want to use. Usually forums are created using an existing template, re-skinned to match your brand. Unfortunately most templates out there at the moment have clunky user interfaces, making it hard for people with low technical abilities to use, and frustrating for tech-savvy audiences. Here lies a great opportunity for designers to rethink the forum format and create a new model that works – let me know how you go with that!

Another variable on forums is user profiling. If you ask users to sign up to your forum and fill in a short form, you can gather data about their demographics, adding an extra layer of insight to their answers. But this process might deter some users from joining the forum. So my recommendation would be, if user demographics aren’t particularly relevant to your project, allow anonymity so that anyone can join. Just be sure to set some rules to make sure people behave in an appropriate manner online.

Community Groups and Social Media

If you’ve only got a short timeframe and not much money, hosting research on social media could be the way to go, as it’s free and has the functionality to post a variety of content for discussion.

For more organic data collection, using a closed Facebook or LinkedIn group allows you to invite your users to a confidential page where you can ask them to respond to pictures, videos, questions and polls, and of course engage in lively discussion. You can plan all your content in advance and publish it using a scheduler like Hootsuite to avoid the toll of constant maintenance.

Or, if you want a quick answer, mirco-surveys have become increasingly popular on social media. This means simply posting a single question poll to your company page to engage your followers interest on a topic. For example, Bill Gates recently posted a poll to Twitter asking “what has happened to extreme poverty over the last 25 years?” After answering the question, users could see the results and learn about his charity work. This micro-survey gathered data, taught users about poverty statistics and promoted his cause at the same time.




One common argument people make against social media market research is “what if our customers don’t have an account?” I say don’t let this stop you, the rewards outweigh the risk in this case. If you choose a platform your target group already uses, this can work in your favour, as the research will be built into their daily routine.

Bots and Conversational UX

A UX trend currently taking off is artificial intelligence, with one example being ‘conversational UX’. Conversational UX basically means that instead of a webpage being made out of a navigation, buttons, links and content, the whole online experience appears as one dynamic conversation.

These formats are sometimes called “bots”, because the user is talking to a form of artificial intelligence. Some bots exist in a free-form style, where the user can ask anything they like. Other bots control the conversation by running on a set of cleverly devised questions, like the news site UX Bear.




This second format lends itself perfectly to market research, because as the user answers each question, data is being collected. Bots can collect users’ names and demographics, ask them what they’re interested in, offer them information to read and ask their opinion on it, without it feeling like a survey.

The experience doesn’t have to be a single linear journey either. You can collect ideas from people who’ve already responded and then replay them to new users to gain deeper insights.


With a well-devised content strategy, a bot could gain some interesting insights without customers even knowing they were taking part in market research.

In fact, this is where market research is already heading.

Apple keeps voice data from Siri requests for two years and uses it to improve their products. Google tracks our locations, browsing history, YouTube viewing and emails to create advertising profiles and deliver us personalised content. With the progression of the Internet of Things, all aspects of our everyday lives are being documented, such as how many steps we’ve taken in a day, how many calories we’ve eaten, how many hours we’ve spent watching Netflix and even how many hours we’ve slept in a night. In years to come we may not have to conduct surveys at all. It will simply be a matter of buying data that’s already been collected.

But in the meantime, remember no 2B pencils. Whatever format you choose to capture data, make it enjoyable, seamless and integrated into your customers’ digital lives.



By Clare Maxwell