• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
QuestionPro

QuestionPro

questionpro logo
  • Products
    survey software iconSurvey softwareEasy to use and accessible for everyone. Design, send and analyze online surveys.research edition iconResearch SuiteA suite of enterprise-grade research tools for market research professionals.CX iconCustomer ExperienceExperiences change the world. Deliver the best with our CX management software.WF iconEmployee ExperienceCreate the best employee experience and act on real-time data from end to end.
  • Solutions
    IndustriesGamingAutomotiveSports and eventsEducationGovernment
    Travel & HospitalityFinancial ServicesHealthcareCannabisTechnology
    Use CaseAskWhyCommunitiesAudienceContactless surveysMobile
    LivePollsMember ExperienceGDPRPositive People Science360 Feedback Surveys
  • Resources
    BlogeBooksSurvey TemplatesCase StudiesTrainingHelp center
  • Features
  • Pricing
Language
  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
  • Hebrew IL (Hebrew)
  • ไทย (Thai)
  • Deutsch (German)
  • Portuguese de Portugal (Portuguese (Portugal))
Call Us
+1 800 531 0228 +1 (647) 956-1242 +52 999 402 4079 +49 301 663 5782 +44 20 3650 3166 +81-3-6869-1954 +61 2 8074 5080 +971 529 852 540
Log In Log In
SIGN UP FREE

Home Market Research

Correlation isn’t Causality

Business Graph

I came across a published report recently that made me wonder why people persist in reporting that there is a causal research relationship when the data doesn’t justify the assertion. Actually, the reasons aren’t all that hard to figure out. Usually, it’s because the relationship seems obvious, and sometimes it is when the person writing the report has a bias they wish to share.

But I’m getting ahead of myself.  Let’s start with a couple of definitions:

A correlation is simply the test of the relationship between two variables.  Pearson’s coefficient, commonly used to test linear relationships between scale variables, will be 1 (or -1) for perfect correlation.  Other coefficients are used for different types of variables. Tools such as SPSS that calculate correlation coefficients generally provide some guidance as to whether the relationship is significant – the strength of the correlation.

What correlation tells you is given the value of the one variable, what to expect for the value of another variable.

Causality, on the other hand, is a statement that if the value of one variable is changed then the value of the second variable will change accordingly.  Correlation is necessary, but not sufficient, for a cause-and-effect relationship.

It is easy to find good examples of correlations where assuming a causal relationship would be absurd. The Wikipedia article on the topic shows a chart of Mexican lemons imported from Mexico to the US plotted against total US highway fatalities. This is an example of a coincidental correlation.

Another type of misinterpretation occurs when the order of the cause and effect is reversed. Daniel Huff’s excellent “How To Lie With Statistics” discusses the relationship between smoking and college grades.  Apparently the results were used to promote the idea that giving up smoking would lead to improved grades.  But it is equally feasible that lower grades caused students to take up smoking.

Create memorable experiences based on real-time data, insights and advanced analysis. Request Demo

We can get into trouble by using more sophisticated statistical techniques without paying enough attention to the meaning of the data and the variables being used to express results. Regression analysis is a powerful tool, but look at the correlations first.  Even the jargon can encourage misinterpretation and misstatements; when you are performing analysis for the ‘dependent’ variable it is easy to conclude causality where none exists.

More subtle problems can occur when some other factor is the cause for both the correlated variables.   This article describes a study where eating breakfast was correlated with elementary school success.  This could have resulted in the conclusion that breakfast eating caused them to be better learners. The article continues, “It turns out, however, that those who don’t eat breakfast are also more likely to be absent or tardy — and it is absenteeism that is playing a significant role in their poor performance. When researchers retested the breakfast theory, they found that, independent of other factors, breakfast only helps undernourished children perform better.” The article is from the Statistical Assessment Service – STATS – which is a non-partisan resource whose mission is to provide education on the use and abuse of science and statistics in the media.

I can’t be sure which of the fallacies were behind the ill-considered statements that were the inspiration for this article without access to the raw data.  The Kauffman Foundation does some excellent work studying entrepreneurship.  But their report on “The Use of Credit Card Debt by New Firms” draws some conclusions that are not justified by the data shown. The report states that “credit card debt reduces a firm’s probability of survival” (emphasis mine).  It appears that the authors want to warn entrepreneurs to avoid using credit cards. All the more surprising then that two positive examples for credit card funding (Spike Lee and the Blair Witch Project movie) are named in the report. I don’t want to be hypercritical of Kaffman or the report, as there are some interesting and useful results presented.  But from the data shown it seems equally likely that the businesses that failed were going to fail anyway, regardless of taking on credit debt.  In fact, businesses that failed during the three years of the study actually had lower credit card debt at the end of the first year.  Perhaps they did not borrow aggressively enough!

How then do you avoid drawing the wrong conclusions about cause-and-effect?  And how can you deliver results from research that provide useful guidance for actions that forward the organizational goals?

First, avoid making statements that imply the correlations imply causality.  Consider the other possibilities such as reverse causality or another variable that wasn’t measured.  However, don’t be too pedantic or academic either.  It is often fair to say that there may be a cause-and-effect relationship.  And frequently the changes that will positively impact one variable will be beneficial to the organization as long as they make sense on the face of it.

If you really need to confirm causality, you’ll generally need to do some sort of study that is repeated over time.  By including the same people in the sample, you’ll have good assurance that changes you see in Overall Satisfaction can be connected with the changes you make from one wave to the next – such as for Speed of Connecting to a Customer Service Representative.  If you don’t use the same people, you’ll have to take more care to make sure the samples are the same as far as possible.

For more examples that will help you critically review your own and others’ work, check out this great list of correlation/causality fallacies.

And finally, I couldn’t resist this cartoon on the topic from XKCD:
image

Idiosyncratically,

Mike Pritchard

Mike Pritchard is President of 5 Circles Research, a Seattle-based firm specializing in helping people conduct their own surveys through consulting and training. More information on services, including training classes for do-it-yourself surveyors and the SurveyTips blog, can be found atwww.5circles.com.

SHARE THIS ARTICLE:

About the author
Mike Pritchard

View all posts by Mike Pritchard

Primary Sidebar

Research what's on your mind. Find out what's on theirs!

A suite of tools to leverage research and transform insights.

Discover our insight platform

RELATED ARTICLES

HubSpot - QuestionPro Integration

Top 11 Data Management Software in 2024

Sep 01,2022

HubSpot - QuestionPro Integration

Target Population: What It Is + Strategies for Targeting

Aug 29,2024

HubSpot - QuestionPro Integration

Educational Evaluation: What Is It & Importance

Jan 01,2023

BROWSE BY CATEGORY

  • Academic
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • CX
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Life@QuestionPro
  • LivePolls
  • Market Research
  • Marketing
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • Release Notes
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Tuesday CX Thoughts (TCXT)
  • Uncategorized
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce Intelligence

Footer

MORE LIKE THIS

ken tcxt

Experience – What’s Included? | Tuesday CX Toughts

May 20, 2025

artificial-data

What is Artificial Data & How It’s Shaping Research

May 20, 2025

wells-fargo-nps-2025

Wells Fargo NPS 2025: What Businesses Can Learn

May 19, 2025

word-cloud

Word Cloud: What it is & How to Use QuestionPro Word Cloud?

May 16, 2025

Other categories

  • Academic
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • CX
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Life@QuestionPro
  • LivePolls
  • Market Research
  • Marketing
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • Release Notes
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Tuesday CX Thoughts (TCXT)
  • Uncategorized
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce Intelligence

questionpro-logo-nw
Help center Live Chat SIGN UP FREE
  • Sample questions
  • Sample reports
  • Survey logic
  • Branding
  • Integrations
  • Professional services
  • Security
  • Survey Software
  • Customer Experience
  • Workforce
  • Communities
  • Audience
  • Polls Explore the QuestionPro Poll Software - The World's leading Online Poll Maker & Creator. Create online polls, distribute them using email and multiple other options and start analyzing poll results.
  • Research Edition
  • LivePolls
  • InsightsHub
  • Blog
  • Articles
  • eBooks
  • Survey Templates
  • Case Studies
  • Training
  • Webinars
  • All Plans
  • Nonprofit
  • Academic
  • Qualtrics Alternative Explore the list of features that QuestionPro has compared to Qualtrics and learn how you can get more, for less.
  • SurveyMonkey Alternative
  • VisionCritical Alternative
  • Medallia Alternative
  • Likert Scale Complete Likert Scale Questions, Examples and Surveys for 5, 7 and 9 point scales. Learn everything about Likert Scale with corresponding example for each question and survey demonstrations.
  • Conjoint Analysis
  • Net Promoter Score (NPS) Learn everything about Net Promoter Score (NPS) and the Net Promoter Question. Get a clear view on the universal Net Promoter Score Formula, how to undertake Net Promoter Score Calculation followed by a simple Net Promoter Score Example.
  • Offline Surveys
  • Customer Satisfaction Surveys
  • Employee Survey Software Employee survey software & tool to create, send and analyze employee surveys. Get real-time analysis for employee satisfaction, engagement, work culture and map your employee experience from onboarding to exit!
  • Market Research Survey Software Real-time, automated and advanced market research survey software & tool to create surveys, collect data and analyze results for actionable market insights.
  • GDPR & EU Compliance
  • Employee Experience
  • Customer Journey
  • Synthetic Data
  • About us
  • Executive Team
  • In the news
  • Testimonials
  • Advisory Board
  • Careers
  • Brand
  • Media Kit
  • Contact Us

QuestionPro in your language

  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
  • Hebrew IL (Hebrew)
  • ไทย (Thai)
  • Deutsch (German)
  • Portuguese de Portugal (Portuguese (Portugal))

Awards & certificates

  • survey-leader-asia-leader-2023
  • survey-leader-asiapacific-leader-2023
  • survey-leader-enterprise-leader-2023
  • survey-leader-europe-leader-2023
  • survey-leader-latinamerica-leader-2023
  • survey-leader-leader-2023
  • survey-leader-middleeast-leader-2023
  • survey-leader-mid-market-leader-2023
  • survey-leader-small-business-leader-2023
  • survey-leader-unitedkingdom-leader-2023
  • survey-momentumleader-leader-2023
  • bbb-acredited
The Experience Journal

Find innovative ideas about Experience Management from the experts

  • © 2022 QuestionPro Survey Software | +1 (800) 531 0228
  • Sitemap
  • Privacy Statement
  • Terms of Use