The Great Rebranding of Market Research
How a lack of standards is killing survey research
I often used the Patagonian Toothfish as an analogy to describe the survey research industry. As a name, Patagonian Toothfish doesn’t sound too appetizing and I’m sure if most of us saw it on a menu we might be reticent to give it a try. Yet a couple decades ago the reality was that the fish had potential as a delicious high priced menu item given the difficulty to overcook, the buttery white flesh, and a mild, non-fishy flavor. To fishermen it was known as white gold, and to ensure consumers saw it the same way the fish needed a rebrand; this is how we got the Chilean Sea Bass.
I think of market research in a lot of the same ways. With the proliferation of Big Data and Analytics in the early 2000’s, the idea that insights could be garnered from the data of everyday living vs. coming from consumer surveys started an industry that’s now bigger than the market research industry ever was. While peak data may have been 5 years ago before GDPR reigned in the industry, we’re now in a second evolution with the advent of machine learning, computer vision, conversational AI, and other advanced data solutions. Despite all this innovation, surveys have remained roughly the same. Sure, there’s been a massive migration to online, there’s been a proliferation of DIY tools and services, but the general concept of surveys has remained as it was two decades ago.
This lack of change is both a good sign and a bad sign. On the good end, the survival of surveys is proof still that observing people through data and what they say in posts on social media doesn’t give you an accurate view of how people think. If we go back to middle school critical thinking, we remember that to fully answer a question we need to understand the who, what, when, where, why and how of any problem. Analytics and data directly tackle the who, what, when, where and how problems, but answering the why question is still the biggest problem facing businesses. If you can’t understand why something is happening how can you effectively understand your business or know when it’ll grow or shrink. Why is still critical and as such, so too are surveys.
While answering this question has kept surveys in the mainstream, the survey industry has evolved in a different direction to that of big analytics. Companies across the Fortune 1000 spectrum are investing capital into developing analytics and data science functions. As a result, Amazon, Google, and Microsoft are doubling down on enabling tech in the cloud, startups are turning academic ideas into real world solutions and digital assistants are fueling a whole new level of AI development driven off of helping consumers. On the other hand, it feels like the survey industry has evolved based on cost reduction rather than a progressive controlled innovation agenda. The journey has been from pencil and paper collection, to call center RDD, to online surveys, and now to the ultimate cost saver: DIY (or you do all the work).
So, what’s different about the fascinating world of big data and the world of survey research. I think it comes down to a key enabler that’s been missing in the survey research space for some time — common standards. When you look at the evolved state of several of the big data industries you see collective agreement on a standard operating procedure; one that encourages efficiency yet still enables competitive differentiation.
As an example, in the world of cloud services, AWS, GCP, and Azure all support containerization of applications. This means you can write one piece of software and run it effectively on any of the big cloud platforms (as well as your own data center). This drives competitiveness amongst the various platforms, but also encourages them to ensure they work together.
The same adoption in standards exists everywhere and it simplifies B2B and B2C experiences. Advertising on TV is standardized to 30 and 15 second spots because they’re easier for programmers to implement and easy to buy. The USB C port is becoming ubiquitous for charging mobile devices, apparently even now possibly including Apple who’ve been holding out for some time. We can move from our homes to an office or internet café and still retain our needed connections because of the Wi-Fi standards. Simply said, standards make life easier.
Yet looking at the survey research industry we have a woeful lack of standardization. It’s my belief that this lack of standardization has led to further disintermediation and declining interest in the field overall — fundamentally it’s too complicated to navigate. Don’t believe me? Here’s some examples:
If you’ve ever joined a consumer survey panel, you’ll find you are asked to complete extensive demographic profiling including such basics as Age, Gender, and Income. Yet, every survey you take as a member of that panel will undoubtedly ask you again for Age, Gender, and Income. As that panelist you undoubtedly wonder why you must answer the same question repeatedly. What it comes down to is a lack of standards. Across all surveys and panels that exist no two companies have adopted the same standard ways of collecting core demography for consumers. As a result, while the panel company may know your Income, the income bands the panel company used might not match the income bands used in the survey. On top of it all there are no standard APIs for passing along the known data from the panel companies into the survey platforms. Sure, there are one off APIs to make this possible, but that puts a burden on the survey platforms to build a significant number of integrations to make this work and those companies would rather spend effort building cool new features for clients. Starting with a set of standards for demography and a standard set of API protocols to move data into surveys would eliminate a serious source of annoyance for consumers and enable more passive data to be populated into surveys enriching the client experience. Even in the case where a client absolutely must ignore the standard that’s fine, there will always be an exception; standards aren’t a mandate, they are the easiest path to create convenience and efficiency.
One thing we all notice when we open an app on our iPhones or Android devices, is that they all tend to work the same way. On iPhones this means the menu icons are usually along the bottom, and that you can swipe to delete, etc. Google employs similar patterning in their Material Design toolkit. This is a great example of a soft standard, a standard that is not mandated but heavily encouraged to increase performance and engagement due to the high degree of adoption by developers. Both Apple and Google encourage these standards as having a common pattern of behavior that makes it easier for consumers to know what to do. Yet there are no soft standards for survey design. In fact, the survey authoring platforms go out of their way to try and make their designs unique to appeal to their clients. Yet as consumers the lack of standards across survey platforms is jarring. Some clients use drag and drop for ranking; some require numerical ordering with text boxes. Some show scales vertically and some horizontally. Trust me, join a panel, and try navigating from one company’s survey to the next. Underlying all of this is the assumption that the survey is designed for the client, they can request question styles, colors, images to fit their fancy. The reality is that in designing a survey, the ultimate user is the end consumer that’s going to take that survey. We want all the consumers globally that take surveys online to know what’s expected of them. We want them to know that “disagree” is always on the left and “agree” is always on the right and not the other way around, etc. This idea of training consumers on a survey pattern should result in an increase in data reliability and happier survey takers. As an industry we need a publicly adopted survey design framework, a standard that is implemented across platforms to drive greater efficiency and productivity. If surveys get better, we all win.
It’s amazing to me to see the level of technology that goes into high volume trading systems and the challenges they face in clearing transactions. While you may think of financial trading systems when thinking of this, I tend to think of ad exchanges and ad buying platforms. Everyday billions of ads are shown to consumers and those billions of transactions must be reconciled and turned into an invoice. The online ad space has created several standard billing practices and technologies to automate clearing transactions. Those technologies have had to evolve over time to stay on top of changes to the billing model such as dealing with fraud, considering data used, and deciding if the consumer saw the ad or not. Yet those systems have evolved and flexed based off a standard published by the Interactive Advertising Bureau (IAB) called OpenRTB (Real time bidding). Those standards facilitate mass scale buying and selling of advertising in a way that enables a multibillion-dollar industry. Today, technology used to settle transactions in online surveys generally revolves around simple redirect URLs that tell the source of the consumer what the outcome was for the survey (complete, over quota, etc.). That simple URL defines a transaction for the consumer sourcing partner and is easily co-opted by bad actors to generate numerous false positives in an effort to defraud the industry of incentive dollars. Ironically the URL is a standard. It’s the method supported by most players in the industry however, the one thing worse than no standard is an unsupported standard. One that’s left to flounder and not evolved to the changing nature of the environment. As an industry we need a well-supported standard settlement mechanic supported by panels and survey platforms.
Don’t get me wrong, there are some glimmers of hope out there. One of the biggest barriers to getting better utilization out of survey data is standardizing the outputs of survey data collection systems. The freedom to move data into a processing platform of choice is something that unlocks the power of survey data and will drive innovation in tools supporting survey data sets. A small team based in the UK started a project known as TSAPI intended to create a standard set of APIs to enable survey data to be moved from platform to platform. This is a great project that raises the bar for how the survey research industry should operate. With a lot of the initial work having been done the team now face the uphill battle to encourage adoption of the standards across the major players in the industry.
And that’s what it comes down to, the industry. The survey research industry like any other industry is competitive, everyone is looking to find their niche, or edge over the competition. Creating standards works against this ethos. Therefore, we need the ability to delegate the process of standardization to industry associations that will bring the innovation and standards to the group. In most industry standardization initiatives industry organizations act as the mediators to make sure standard work well and those same organizations promote the health and success of the overall industry. We need to evolve our industry organizations. Most of them are born from an academic point of view, acting as peer review organizations to evaluate the science of market research and forums for evaluating new cutting-edge methodologies. But at this stage in our industry evolution, we need an association that cares less about promoting the academic side of research but rather is more of a trade association focused on what is needed to promote greater utilization of surveys, after all, that’s what makes member companies more money. Our associations should be helping make surveys easier, more accessible to clients, and helping the industry grow. This should be possible with a focus on developing hard and soft standards as well as the establishment of a mechanism to ensure standards are current and kept up to date.
Because of the Why question surveys remain relevant. But in a hyper connected, API driven world, standards rule and without support survey research will stay the Patagonian Toothfish, completely underappreciated. As someone who’s been in the space for quite some time, I’d like to see survey research have its Chilean Sea Bass moment.