As phones become increasingly less ubiquitous, it becomes difficult to conduct telephone-based surveys. But the good news is that there’s a new way to conduct surveys! Random device engagement (RDE) is an innovative method of conducting research that uses a respondent’s unique identifier as the keystone of their participation.
It is important to note that by observing the identifier of device world ads, survey firms can prevent fraud related to SUMAs (single users, multiple accounts). RDE samples are also entirely random and non-biased. An improvement on random digit dialing, is used to engage with landline (and now cell) phones randomly.
This flexibility has a number of advantages over traditional survey methods: RDEs are less expensive because they don’t require travel or other costs associated with conducting interviews; they provide more privacy for respondents because no one sees who has or hasn’t completed the survey; and they offer more control over when respondents have access to the survey since there is no need for interviewers or other personnel at collection sites (Tucker 1983; West and Blom 2017).
What is random device engagement?
Let us introduce Random Device Engagement (RDE); the natural successor of RDD (Random Digit Dialing), in terms of orthography, philosophy, and quality.
Random device engagement (RDE) polling relies on advertising networks or other portals on devices to engage random people where they are. One of the most common versions of this is within smartphone advertising modules, but it can easily be placed in gaming, virtual reality, etc.
The principle behind Random Device Engagement polling is that people who have been selected by a polling company are more likely to participate in polls than those who have not been selected. Advertisers can take advantage of RDE polling in this way to better understand what customer experience is within their products or services and how they might be able to improve them.
Random Device Engagement represents a huge benefit, especially when compared to RDD. While surveys can be completed in a matter of days, it is not uncommon for RDD to take weeks or even months. With social networks and assisted crowdsourcing, surveys can be completed a little faster than with RDE, but still lack the speed that RDE offers.
Online panels are comparable in speed if you pay extra for respondents from merged panels (online panels charge extra to get respondents from other panels because it increases their speed).
When it comes to the accuracy of RDE, it’s important to know that coverage is one of the main factors. Within the US market, the major RDE companies can reach more than 5,000,000 unique respondents. RDE is still lagging behind RDD in terms of coverage at this time, but it will catch up soon. Crowdsourced polling using social media is similar to social media-assisted crowdsourcing, and coverage is superior to online panels.
Panels that are available online have a very small footprint, which affects their ability to gather detailed information about the population.
Random Device Engagement Methods
Let’s review the work published in both Goel, Obeng, and Rothschild (2015) and Konitzer, Corbett-Davies and Rothschild (N.d.) to showcase how effective RDE samples can be. And, add examples from the 2017-2018 special congressional elections.
The first study was conducted by Goel et al. (2015) who used a population-based sample of 1,200 registered voters in general elections in Michigan. The sample was drawn from official voter files from the Board of Canvassers in Michigan and included voters who were eligible to vote but did not cast a ballot in 2004.
This study revealed that nonvoters were more likely than voters to favor Kerry over Bush in 2004, with a relative advantage of 8 percentage points for Kerry among nonvoters compared with only 3 percentage points among voters (p<0.05).
The second study was conducted by Konitzer et al., (2016) who used the Cooperative Congressional Election Study (CCES), which interviewed 1,068 respondents via RDDs during the 2008 presidential election cycle. The CCES dataset includes information about respondents’ political attitudes as well as their past voting behavior, which allowed Konitzer et al., to examine how past voting behavior influences future turnout.
The researchers were able to demonstrate how the use of RDE, through Pollfish, was able to closely match gold-standard polling such as the General Social Survey. The researchers found that this method could be used to replace house calls in surveys since house calls are often too expensive for most research projects to afford, so this is a useful benchmark.
The results showed that their predictions based on a single poll are not significantly worse than the predictions from poll aggregators such as Huffington Post Pollster (HPP). When comparing their state-by-state estimates against the actual outcome, they found that compared to HPP’s estimates, their RMSE was only slightly higher: 4.24% points vs 3.62% points (for 50 states excluding DC)
Method #1 – Random Digit Dialing (RDD)
The first method, Random Digit Dialing (RDD), is the traditional method. It works fine, but it is doomed in the next few years. Thus, this is really about which of the new online survey sampling methods will replace it: online panels, Assisted Crowdsourcing, or Random Device Engagement (RDE). We believe that RDE is the future.
During the course of 2017 and 2018 polling firms have employed all three new methods in predicting Congressional election outcomes: RDE comes out way above the other two.
Polling firms have used three new methods to forecast Congressional election results during 2017 and 2018. RDE performs better than the others.
In this article, we provide a detailed analysis of the method’s strengths, including surveys conducted by the Pew Research Center.
Method #2 – Telemetry Data
The use of telemetry data in survey research is not new. In fact, the most famous example of this technique is the American National Election Studies (ANES), which has been collecting household-level telephone and face-to-face surveys since 1948. The ANES collects this information on a rolling basis so that researchers can track changes over time and across geographic locations.
More recently, RDE has begun supplementing collected attitudinal data with a variety of parametric or telemetric data.
Survey-taking people are fundamentally different than those who don’t take surveys, as we all know. As the progressive analytics shop, CIVIS has argued recently, a battery of nearly 30 additional demographic, attitudinal, and lifestyle questions that get at notions of social trust and cosmopolitanism are necessary to be able to weigh and correct all the ways in which survey respondents are unusual.
As Konitzer, Eckman, and Rothschild (2016) argue, telemetry data is a much more cost-effective (and unobtrusive) way to collect these variables. Home and work location, commuting or mobility patterns, or the political makeup of one’s neighborhood or social network, derived from satellite-based (read: extremely accurate) longitudinal location-coordinate data predict demographic variables well.
Method #3 – River sampling
River sampling is a method of recruiting respondents that use banner ads. It’s a common practice in market research and polls, but it has some major drawbacks.
Depending on the river sampling method, banner ads could be used for polling, or engagement could take place through legacy websites or locations where Rapid Data Enumeration recruits respondents. RDE has access to account numbers, but river sampling does not, introducing two serious disadvantages: River sampling is incapable of detecting SUMA – fraudsters might cheat by participating twice in the same survey, especially if there is a financial incentive for participating. And, any degree of demographic/geographic targeting seems impossible.
The process is simple: an ad request comes to the server, which then redirects it to the ad network for processing directly. The ad network determines if there are any available ads that are appropriate for that particular user and sends them along to the server.
Good RDE polling is done with the cooperation of the publisher, providing a native experience, while banner ads are pushed through the ad network.
Random Device Engagement Examples
As outlined previously, we firmly believe Random Digit Dialing (RDD) is doomed. Online panels are effective but expensive, Assisted Crowd-sourcing works well but takes too long, and Random Device Engagement (RDE) is much faster and more cost-effective.
The future of RDE is bright. Penetration of devices will further increase in the future, increasing reach of RDE in the US and making RDE the only viable alternative in less developed markets. Take Africa: the smartphone penetration rate is projected to grow at 52.9% year on year.
Currently, there are 293 million smartphone users across the continent, meaning that taking into account current growth rates, there will be 929.9 million smartphones by the year 2021 in Africa. But the rosy future for RDE is not just about penetration—advancements in bridging Ad IDs with other known identifiers in the American market mean that individual targeting based on financial history or credit card spending patterns will be possible.
And while we’re talking about bridging data sources: political survey firms can now poll directly from a voter file of 250 million Americans.
Conclusion
We started by discussing how traditional RDD worked and how it was doomed by a lack of participation from younger people who are less likely to have landlines. We reviewed two new techniques: online panels and assisted Crowdsourcing. Online panels are similar to traditional RDD but with more modern technology; they do not require participants to answer any questions themselves or even speak with a human being.
Instead, they use automated scripts and algorithms for data collection from internet users who agree to participate. Assisted Crowdsourcing uses human beings as well as computers; humans are used for tasks like tagging images or transcribing voice recordings while computers do other things such as analyzing text documents for sentiment analysis purposes.
Finally, we discussed our preferred method: Random Device Engagement (RDE). This method uses sophisticated algorithms to automatically identify devices.
Learn how to unlock the power of research with QuestionPro’s Audience. Our products are used by over 3 million people in over 50 countries and we’re always on the lookout for new ways to use our technology to help you gather more meaningful insights from your customers and clients.
Use QuestionPro’s Audience to help you create more accurate surveys that will provide you with better data.
