• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
QuestionPro

QuestionPro

questionpro logo
  • Products
    survey software iconSurvey softwareOur flagship survey solution. Sophisticated tools to get the answers you need.research edition iconResearch SuiteTuned for researchers. Get more insights. Response based pricing.CX iconCXExperiences change the world. Deliver the best with our CX management software.WF iconWorkforceEmpower your work leaders, make informed decisions and drive employee engagement.
  • Solutions
    IndustriesGamingAutomotiveSports and eventsEducationGovernment
    Travel & HospitalityFinancial ServicesHealthcareCannabisTechnology
    Use CaseNPS+CommunitiesAudienceContactless surveysMobile
    LivePollsMember ExperienceGDPRPositive People Science360 Feedback Surveys
  • Resources
    BlogeBooksSurvey TemplatesCase StudiesTrainingHelp center
  • Features
  • Pricing
Language
  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
Call Us
+1 800 531 0228 +1 (647) 956-1242 +52 999 402 4079 +49 301 663 5782 +44 20 3650 3166 +81-3-6869-1954 +61 2 8074 5080 +971 529 852 540
Log In Log In
SIGN UP FREE

Home CX Consumer Insights

What Happens When You Change Survey Methods

imageMy last article about changing rating scales prompted this e-mail from a reader.

Over many years of surveying customers and guests I have had to grapple many times with changing methodology: not just changing scales but in changing data collection methods.  Now, more than ever, many clients are thinking about, if not already, switching to data collection through email surveys and internet based surveying.  As the saying goes “Everyone else is doing it so we should too”.

What faces us is the question of how to explain the differences in scores achieved by the surveying methods. For example, having conducted large scale guest satisfaction surveys across many European countries using intercept surveying we must now convert our collection methods to internet services.  Using the exact same questionnaire but administering the survey by email we witness an average drop in satisfaction rating of 9-10% across the board.  For companies firmly focused on customer satisfaction this drop is very disconcerting.

Yes, the “halo effect” may explain some of the change but a drop of 10%?  Does primacy and recency affect the scores but by how much?  Having intercepted the guest before leaving the premises we now mail them a day or two later and get the survey completed sometimes up to 5 to 7 days later.  Does the level of satisfaction just drop off as time passes?

In previous years we have tested the change in methodology between face to face, postal and telephone interview and can clearly see that the face to face scores are always the highest, not only because we set the parameters on how respondents can be selected and can to some extent control all the environmental variables, but once we lose this control the data changes substantially.

I just want to pose the question to other interested marketers  and ask how they understand this variance in scores, if in evidence and how best can it be explained.

— Madeleine O’Carroll, consultant researcher working in the EMEA region, specialising in hospitality research.

I’ve done some looking on the internet, but haven’t found a lot of information on this topic.  What’s been your experience?  Does anyone have resource or advice for Madeleine?  Leave a comment here or send me any links or resources that you have and I will post them.

Better yet – if you’ve had experience on this and would like to send a guest post on this topic – please e-mail it to me [email protected] and I will post it.  Be sure to include a one-sentence bio and I link where I can send our readers for more information about you and your organization.

 

SHARE THIS ARTICLE:

About the author
Ivana Taylor
View all posts by Ivana Taylor

Primary Sidebar

Gain insights with 80+ features for free

Create, Send and Analyze Your Online Survey in under 5 mins!

Create a Free Account

RELATED ARTICLES

HubSpot - QuestionPro Integration

Data and Emotions, People and Machines: Contextual Business in the Age of AI

Nov 18,2021

HubSpot - QuestionPro Integration

A complete guide to staff surveys

Apr 01,2020

HubSpot - QuestionPro Integration

[Webinar Recap] Datapad Sneak Peek

May 13,2020

BROWSE BY CATEGORY

  • Academic
  • Academic Research
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Brand Awareness
  • Branding
  • Business
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Experience IN
  • Customer Loyalty
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • Customer Satisfaction
  • CX
  • Decision Making
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Intercept
  • klantervaring
  • [email protected]
  • LivePolls
  • Market Research
  • Market Research
  • Marketing
  • Marketing
  • Marktonderzoek
  • medewerkersonderzoek
  • Mercadotecnia
  • Mobile
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • QuestionPro Products
  • Release Notes
  • Research
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Surveys
  • Tech News
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Uncategorized
  • Video Learning Series
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce
  • Workforce
  • Workforce Intelligence
  • Workforce Intelligence
  • Workforce Intelligence

Footer

MORE LIKE THIS

Audience engagement tools can be utilized in meetings, conferences, events, training sessions, and educational settings. Learn more.

Audience Engagement Tools: What it is, why to use + Top 5 tools

Feb 4, 2023

Organizational assessment examines procedures, environment, and structure. It helps businesses identify areas of opportunity. Learn more.

Organizational Assessment: What It Is, Benefits + How to Conduct

Feb 3, 2023

Text mining is essential for analyzing + processing unstructured data. It allows to track how and when its products and brand are mentioned.

Text Mining: What is It, Methods + How is It Used in Business

Feb 2, 2023

Bullying: Types, Example Survey Questions and More

Feb 1, 2023

Other categories

  • Academic
  • Academic Research
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Brand Awareness
  • Branding
  • Business
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Experience IN
  • Customer Loyalty
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • Customer Satisfaction
  • CX
  • Decision Making
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Intercept
  • klantervaring
  • [email protected]
  • LivePolls
  • Market Research
  • Market Research
  • Marketing
  • Marketing
  • Marktonderzoek
  • medewerkersonderzoek
  • Mercadotecnia
  • Mobile
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • QuestionPro Products
  • Release Notes
  • Research
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Surveys
  • Tech News
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Uncategorized
  • Video Learning Series
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce
  • Workforce
  • Workforce Intelligence
  • Workforce Intelligence
  • Workforce Intelligence

Copyright © 2023 · Magazine Pro on Genesis Framework · WordPress · Log in

questionpro-logo-nw
Help center Live Chat SIGN UP FREE
  • Sample questions
  • Sample reports
  • Survey logic
  • Branding
  • Integrations
  • Professional services
  • Security
  • Survey Software
  • Customer Experience
  • Workforce
  • Communities
  • Audience
  • Polls Explore the QuestionPro Poll Software - The World's leading Online Poll Maker & Creator. Create online polls, distribute them using email and multiple other options and start analyzing poll results.
  • Research Edition
  • LivePolls
  • InsightsHub
  • Blog
  • Articles
  • eBooks
  • Survey Templates
  • Case Studies
  • Training
  • Webinars
  • Coronavirus Resources
  • All Plans
  • Nonprofit
  • Academic
  • Qualtrics Alternative Explore the list of features that QuestionPro has compared to Qualtrics and learn how you can get more, for less.
  • SurveyMonkey Alternative
  • VisionCritical Alternative
  • Medallia Alternative
  • Likert Scale Complete Likert Scale Questions, Examples and Surveys for 5, 7 and 9 point scales. Learn everything about Likert Scale with corresponding example for each question and survey demonstrations.
  • Conjoint Analysis
  • Net Promoter Score (NPS) Learn everything about Net Promoter Score (NPS) and the Net Promoter Question. Get a clear view on the universal Net Promoter Score Formula, how to undertake Net Promoter Score Calculation followed by a simple Net Promoter Score Example.
  • Offline Surveys
  • Customer Satisfaction Surveys
  • Employee Survey Software Employee survey software & tool to create, send and analyze employee surveys. Get real-time analysis for employee satisfaction, engagement, work culture and map your employee experience from onboarding to exit!
  • Market Research Survey Software Real-time, automated and advanced market research survey software & tool to create surveys, collect data and analyze results for actionable market insights.
  • GDPR & EU Compliance
  • Employee Experience
  • Customer Joruney
  • About us
  • Executive Team
  • In the news
  • Testimonials
  • Advisory Board
  • Careers
  • Brand
  • Media Kit
  • Contact Us

QuestionPro in your language

  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)

Awards & certificates

  • survey-leader-asia-leader-2023
  • survey-leader-asiapacific-leader-2023
  • survey-leader-enterprise-leader-2023
  • survey-leader-europe-leader-2023
  • survey-leader-latinamerica-leader-2023
  • survey-leader-leader-2023
  • survey-leader-middleeast-leader-2023
  • survey-leader-mid-market-leader-2023
  • survey-leader-small-business-leader-2023
  • survey-leader-unitedkingdom-leader-2023
  • survey-momentumleader-leader-2023
  • bbb-acredited
The Experience Journal

Find innovative ideas about Experience Management from the experts

  • © 2022 QuestionPro Survey Software | +1 (800) 531 0228
  • Sitemap
  • Privacy Statement
  • Terms of Use