• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
QuestionPro

QuestionPro

questionpro logo
  • Products
    survey software iconSurvey softwareOur flagship survey solution. Sophisticated tools to get the answers you need.research edition iconResearch SuiteTuned for researchers. Get more insights. Response based pricing.CX iconCXExperiences change the world. Deliver the best with our CX management software.WF iconWorkforceEmpower your work leaders, make informed decisions and drive employee engagement.
  • Solutions
    IndustriesGamingAutomotiveSports and eventsEducationGovernment
    Travel & HospitalityFinancial ServicesHealthcareCannabisTechnology
    Use CaseNPS+CommunitiesAudienceContactless surveysMobile
    LivePollsMember ExperienceGDPRPositive People Science360 Feedback Surveys
  • Resources
    BlogeBooksSurvey TemplatesCase StudiesTrainingHelp center
  • Features
  • Pricing
Language
  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)
Call Us
+1 800 531 0228 +1 (647) 956-1242 +52 999 402 4079 +49 301 663 5782 +44 20 3650 3166 +81-3-6869-1954 +61 2 8074 5080 +971 529 852 540
Log In Log In
SIGN UP FREE

Home Market Research

Can text analytics become more analytical? (part 1)

The term text analytics sounds highly technical. Yet in articles and discussions about this field, the two forms of output that get the most mentions are little more than forms of counting. What are these outputs and why do they pass the cut as “analytical” in connection with text? Let’s start with the output, and then move on to the reasons.

Perhaps the single type of image that represents text analytics is an image showing words of various sizes stacked and piled into a shape. Most often this is a simple rectangle, although some can be strikingly free-form. This display is sometimes called a word cloud and sometimes a wordle.

edtech-wordle-04 2

For all its visual appeal, this display simply shows the relative frequencies of words in a document. The bigger words are more prevalent, smaller ones less so. While this is often represented as a slick analytical feat, the only good trick is the computer’s skill at fitting the words together.

Additional computer processing takes place in the background, starting with the removal of so-called stop words. These are the smaller, frequent words that typically do not bear the meaning of a document, but rather serve to stitch it together. In the last sentence, stop words would include “these,” “are,” “the,” “that,” “but,” “to,” and so on. The computer eliminates these by consulting a dictionary. This often is a simple text file that the user can modify. Wordles typically also ask the computer to do stemming, the process of removing tenses and regularizing endings so that different forms of the same word do not get counted as two distinct entities.

Some programs go further and perform lemmatization, which tries to recognize the part of speech of a word, so that (for instance), “busy” (the verb) does not get lumped in with “business” (the noun) or “busy” (the adjective). However, stop word removal, stemming and even lemmatization are computational problems, not analytical ones. From your perspective, once you just ask for a wordle, one quickly appears. The computer may work hard, but you need not do any analytical thinking.

Another prevalent form of output which is not quite analytical is called sentiment analysis. As in the removal of stop words, this involves the use of a dictionary. And as in the creation of the wordle, it is centered on counting. The analysis typically refers to dictionaries of positive and negative words and phrases. Looking at a document or set of documents, the computer counts the occurrences of each type. The balance of plus and minus gives the “sentiment score.” Dictionaries of sentiment words can be found online, and even downloaded for use. You can easily find one online with about 2000 positive phrases and about 6000 negative ones.

Once again, the computational challenges can be ferocious. For instance, would make a mistake about sentiment if you just counted positive and negative in this sentence: “My new SoggyOs breakfast food is so good, it makes Sorghum Sweeties taste bad by comparison, and Kardboard Krunchies just terrible.” There are two negatives and one positive, so a simple count would peg this sentence as negative.

Yet clearly, the writer is singing the praises of his/her new breakfast-like substance. Indeed, unusual locutions, neologisms, slang, sarcasm and awful sentence structure often prove highly confusing to computer. But resolving these requires better algorithms and faster machines, not more thought from the analyst.

We can expect counting and enumeration to continue getting more accurate. But still, these will remain counting. What are the reasons for this? Stayed tuned for the next installment where we discuss probable causes.

About the Author: Dr. Steven Struhl has been involved in marketing science, statistics and psychology for 30 years. Before founding Converge Analytic, he was Sr. Vice President at Total Research/ Harris Interactive for 15 years, and earlier served as director of market analytics and new product development at statistical software maker SPSS.

SHARE THIS ARTICLE:

About the author
Steven Struhl
View all posts by Steven Struhl

Primary Sidebar

Gain insights with 80+ features for free

Create, Send and Analyze Your Online Survey in under 5 mins!

Create a Free Account

RELATED ARTICLES

HubSpot - QuestionPro Integration

Propensity Score: Construction & Evaluation

Jan 06,2023

HubSpot - QuestionPro Integration

Audience Engagement Tools: What it is, why to use + Top 5 tools

Feb 04,2023

How to build a product focusing on customer experience

Jun 30,2021

BROWSE BY CATEGORY

  • Academic
  • Academic Research
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Brand Awareness
  • Branding
  • Business
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Experience IN
  • Customer Loyalty
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • Customer Satisfaction
  • CX
  • Decision Making
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Intercept
  • klantervaring
  • Life@QuestionPro
  • LivePolls
  • Market Research
  • Market Research
  • Marketing
  • Marketing
  • Marktonderzoek
  • medewerkersonderzoek
  • Mercadotecnia
  • Mobile
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • QuestionPro Products
  • Release Notes
  • Research
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Surveys
  • Tech News
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Uncategorized
  • Video Learning Series
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce
  • Workforce
  • Workforce
  • Workforce Intelligence
  • Workforce Intelligence
  • Workforce Intelligence

Footer

MORE LIKE THIS

A positive work environment improves employee morale, retention, and productivity. It helps making a workplace better for everyone.

Work Environment: What it is, Types & Elements to Create it

Mar 23, 2023

People science is an approach to market research that goes beyond traditional methods to understand the people behind the data.

People Science: What it Means for the Workforce

Mar 22, 2023

Let's learn the importance of personalization in a successful customer experience program from my recent ski trip experience.

Ski Days and CX: Defining Fun For Everyone — Tuesday CX Thoughts

Mar 21, 2023

Adaptive conjoint analysis is a market research method that adapts to each person's answers to determine which product features are liked.

Adaptive Conjoint Analysis: What is it, Types & User Cases

Mar 19, 2023

Other categories

  • Academic
  • Academic Research
  • Academic Research
  • Artificial Intelligence
  • Assessments
  • Audience
  • Brand Awareness
  • Brand Awareness
  • Branding
  • Business
  • Business
  • Case Studies
  • Communities
  • Consumer Insights
  • Consumer Insights
  • Customer effort score
  • Customer Engagement
  • Customer Experience
  • Customer Experience IN
  • Customer Loyalty
  • Customer Loyalty
  • Customer Research
  • Customer Satisfaction
  • Customer Satisfaction
  • CX
  • Decision Making
  • Employee Benefits
  • Employee Engagement
  • Employee Engagement
  • Employee Engagement
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Employee Retention
  • Enterprise
  • Events
  • Forms
  • Friday Five
  • General Data Protection Regulation
  • Guest Post
  • Insights Hub
  • Intercept
  • klantervaring
  • Life@QuestionPro
  • LivePolls
  • Market Research
  • Market Research
  • Marketing
  • Marketing
  • Marktonderzoek
  • medewerkersonderzoek
  • Mercadotecnia
  • Mobile
  • Mobile
  • Mobile App
  • Mobile diaries
  • Mobile Surveys
  • New Features
  • non-profit
  • NPS
  • Online Communities
  • Polls
  • Question Types
  • Questionnaire
  • QuestionPro
  • QuestionPro Products
  • QuestionPro Products
  • Release Notes
  • Research
  • Research Tools and Apps
  • Revenue at Risk
  • Startups
  • Survey Templates
  • Surveys
  • Surveys
  • Tech News
  • Tech News
  • Tips
  • Training
  • Training Tips
  • Trending
  • Uncategorized
  • Video Learning Series
  • VOC
  • Webinar
  • Webinars
  • What’s Coming Up
  • Workforce
  • Workforce
  • Workforce
  • Workforce
  • Workforce Intelligence
  • Workforce Intelligence
  • Workforce Intelligence

questionpro-logo-nw
Help center Live Chat SIGN UP FREE
  • Sample questions
  • Sample reports
  • Survey logic
  • Branding
  • Integrations
  • Professional services
  • Security
  • Survey Software
  • Customer Experience
  • Workforce
  • Communities
  • Audience
  • Polls Explore the QuestionPro Poll Software - The World's leading Online Poll Maker & Creator. Create online polls, distribute them using email and multiple other options and start analyzing poll results.
  • Research Edition
  • LivePolls
  • InsightsHub
  • Blog
  • Articles
  • eBooks
  • Survey Templates
  • Case Studies
  • Training
  • Webinars
  • Coronavirus Resources
  • All Plans
  • Nonprofit
  • Academic
  • Qualtrics Alternative Explore the list of features that QuestionPro has compared to Qualtrics and learn how you can get more, for less.
  • SurveyMonkey Alternative
  • VisionCritical Alternative
  • Medallia Alternative
  • Likert Scale Complete Likert Scale Questions, Examples and Surveys for 5, 7 and 9 point scales. Learn everything about Likert Scale with corresponding example for each question and survey demonstrations.
  • Conjoint Analysis
  • Net Promoter Score (NPS) Learn everything about Net Promoter Score (NPS) and the Net Promoter Question. Get a clear view on the universal Net Promoter Score Formula, how to undertake Net Promoter Score Calculation followed by a simple Net Promoter Score Example.
  • Offline Surveys
  • Customer Satisfaction Surveys
  • Employee Survey Software Employee survey software & tool to create, send and analyze employee surveys. Get real-time analysis for employee satisfaction, engagement, work culture and map your employee experience from onboarding to exit!
  • Market Research Survey Software Real-time, automated and advanced market research survey software & tool to create surveys, collect data and analyze results for actionable market insights.
  • GDPR & EU Compliance
  • Employee Experience
  • Customer Journey
  • About us
  • Executive Team
  • In the news
  • Testimonials
  • Advisory Board
  • Careers
  • Brand
  • Media Kit
  • Contact Us

QuestionPro in your language

  • English
  • Español (Spanish)
  • Português (Portuguese (Brazil))
  • Nederlands (Dutch)
  • العربية (Arabic)
  • Français (French)
  • Italiano (Italian)
  • 日本語 (Japanese)
  • Türkçe (Turkish)
  • Svenska (Swedish)

Awards & certificates

  • survey-leader-asia-leader-2023
  • survey-leader-asiapacific-leader-2023
  • survey-leader-enterprise-leader-2023
  • survey-leader-europe-leader-2023
  • survey-leader-latinamerica-leader-2023
  • survey-leader-leader-2023
  • survey-leader-middleeast-leader-2023
  • survey-leader-mid-market-leader-2023
  • survey-leader-small-business-leader-2023
  • survey-leader-unitedkingdom-leader-2023
  • survey-momentumleader-leader-2023
  • bbb-acredited
The Experience Journal

Find innovative ideas about Experience Management from the experts

  • © 2022 QuestionPro Survey Software | +1 (800) 531 0228
  • Sitemap
  • Privacy Statement
  • Terms of Use