Can AI play a leading role in SA consumer market research?

According to PwC, AI could contribute up to $15.7-trillion to the global economy in 2030, more than the current output of China and India combined.

Within the field of market research, there is huge scope to integrate AI solutions and machine learning (a subset of AI) into traditional research processes which are often time consuming and labour intensive.

In the 2019 Grit Report, the leading survey of the market research industry, AI was the clear winner in the Buzz Topics section of the report — underscoring that there is strong interest in AI applications and solutions within the global industry. Notably, however, the report also flagged the risk of “artificial stupidity” and the potential for market research professionals to be led astray by this emerging field of technology.

“Although the use of AI in market research is a very young concept, it undoubtedly presents an opportunity to innovate and explore different ways of gathering and analysing consumer insights,” says Caitlin Bauristhene, head of research at KLA, a SA intelligence and insights agency.

“In SA, we have to overcome specific challenges in order to leverage the many benefits of AI in market research, and it’s important to emphasise that you cannot leave the human out of the process altogether … whether that is the human respondent or the human researcher or analyst.”

Locally, access to digital data collection presents somewhat of a roadblock for AI applications, because AI typically runs live on digital data collection platforms. According to Bauristhene, the (confident) use of online-based data collection has been dependent on market research agencies convincing buyers that their target markets are accessible via online or mobile methodologies.

This has tracked along with increased smartphone penetration, however, there are some groups that still cannot be easily accessed digitally (such as the very lower-end segments, the very affluent and B2B segments). In addition, market research agencies have traditionally been set up to put the more technical skills (programming surveys/platforms, data processing and analytics) in the back office — and even outsourced — while the client-facing analyst role is the typical “hero” position within the field.

“As it stands, it is hard to find a researcher who has a combination of analytical, strategic and client-service skills — as well as technical data skills,” says Bauristhene. “Beyond the skills question, there is also the local challenge of capital investment into AI innovation. Often, global agencies are unable to convince local clients that what is done overseas can work here, and local agencies have limited capital for investment into research & development on their own.”
Yet investment is arguably critical for the local industry to be able to harness developments within AI technology — particularly given SA’s unique diversity. Indeed, the multitude of languages and cultures within SA poses a major challenge for NLP (natural language processing) engines, for example.
“Even beyond our multilingual consumers, the nuance and turn of phrase in our digitally written text language is hard for a human to interpret, never mind a computer,” explains Bauristhene. “For NLP engines to become mainstream, the systems would need intensive iterative training of the machine learning, and this takes both time and investment, which are seldom available today.”

Yet even in the face of such formidable challenges, there are still ways to integrate AI and machine-learning systems into local solutions — something which KLA was able to achieve in partnership with GroupSolver, a US-based survey research company. When working on a project for a large FMCG company, KLA sought a way to incorporate AI and machine learning in such a way that it could overcome the fundamental challenge of SA’s diverse population.

Working with GroupSolver’s innovative technology platform, KLA was able to acquire high-quality data while also providing an engaging respondent experience.

“With this platform, the respondent answers questions which are analysed in real time, looking for links and themes in the answers, before the next question is presented — thus personalising the question for the respondent using AI and machine learning,” says Logan Seegmiller, at GroupSolver. “The process is very engaging for the respondent, who is constantly answering questions that are relevant to him or her, and it even becomes conversational.”

As a market-research solution, the turnaround time is quick because the researcher can access results almost immediately — and doesn’t have to wade through loads of data before key themes and insights emerge.

“This is a great step forward for AI implementation within local market research,” says Bauristhene. “This solution enabled KLA to acquire qualitative, respondent-ratified consensus by harnessing a quantitative platform, at scale. In addition, the tool’s ability to qualify the relationships that exist between the different open-ended takeouts adds immense richness and insight.”

“Looking ahead, we believe that together with GroupSolver, we will be able to make this an invaluable tool in our arsenal — and can begin to add differential insights to quantitative research projects.”