{"id":10839,"date":"2014-12-22T06:00:45","date_gmt":"2014-12-22T14:00:45","guid":{"rendered":"http:\/\/www.questionpro.com\/?p=10839"},"modified":"2026-10-07T05:40:36","modified_gmt":"2026-10-07T12:40:36","slug":"problems-with-surveys","status":"publish","type":"post","link":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/","title":{"rendered":"Problems with Surveys: Types, Examples, and How to Avoid Them"},"content":{"rendered":"\n<p>Problems with surveys are flaws in who you ask, what you ask, or how you analyze the answers. They make results look reliable when they are not. A survey can launch smoothly, collect thousands of responses, and still point a team in the wrong direction.<\/p>\n\n\n\n<p>The software is rarely the cause. More often it is a leading question, a missing answer option, a sample that skips the people who matter, or a survey so long that respondents stop reading.<\/p>\n\n\n\n<p>In this blog, we break down the main types of survey problems, show before-and-after examples, and cover how to measure and prevent each one.<\/p>\n\n\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What are problems with surveys?<\/strong><\/h2>\n\n\n\n<p>Problems with surveys are design, sampling, collection, or analysis flaws that make survey data an inaccurate picture of the people you want to understand. Researchers also call them survey errors. They come in two kinds:<\/p>\n\n\n\n<ul>\n<li><strong>Preventable mistakes,<\/strong> such as a double-barreled question.<\/li>\n\n\n\n<li><strong>Built-in limits,<\/strong> such as the random variation that appears when you survey a sample instead of everyone.<\/li>\n<\/ul>\n\n\n\n<p>Bad survey data rarely looks bad. Percentages add up, charts look clean, and teams act on the results.<\/p>\n\n\n\n<p>Small wording changes can move those results a lot. An<a href=\"https:\/\/www.pewresearch.org\/writing-survey-questions\/\" target=\"_blank\" rel=\"noreferrer noopener\"> experiment found that<\/a> 51% of respondents favored letting doctors give terminally ill patients the means to end their lives. Only 44% supported the same policy when the question used the word suicide. Same policy, seven points apart.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Survey bias, survey error, and survey mistakes: What is the difference?<\/strong><\/h2>\n\n\n\n<p>Survey error is any gap between your results and reality. Survey bias is an error that pushes results in one direction, and a survey mistake is a preventable cause of either. The terms overlap, so the table separates them.<\/p>\n\n\n\n<div style=\"margin:1.5rem 0;width:100%;max-width:100%;\">\n  <table style=\"border-collapse:collapse;width:100%;max-width:100%;table-layout:auto;\">\n    <thead>\n      <tr>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Term<\/th>\n        <th style=\"background:#162450;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">What it means<\/th>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Example<\/th>\n      <\/tr>\n    <\/thead>\n    <tbody>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Sampling error<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Random variation that occurs because you surveyed a sample, not the whole population<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Two polls of 500 customers give slightly different scores<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\"><a class=\"wpil_keyword_link\" href=\"https:\/\/www.questionpro.com\/blog\/sampling-bias\/\"   title=\"Sampling bias\" data-wpil-keyword-link=\"linked\"  data-wpil-monitor-id=\"609\">Sampling bias<\/a><\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">A sample that systematically leaves out or over-represents certain groups<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Surveying only app users about overall brand perception<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Non-sampling error<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Any error from design, collection, or processing rather than from sampling itself<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">A confusing question or a data coding slip<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Response bias<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Respondents answer inaccurately for a consistent reason<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Overstating how often they vote<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Survey mistake<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">A preventable human error in design or setup<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">An answer list with overlapping ranges<\/td>\n      <\/tr>\n    <\/tbody>\n  <\/table>\n<\/div>\n\n\n\n<p>Sampling error shrinks as your sample grows. Bias does not, so a bigger sample never rescues a biased one.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Where do problems with surveys come from?<\/strong><\/h2>\n\n\n\n<p>Problems with surveys come from four places: who you ask, what you ask, how you ask, and who answers. Before online tools, most companies hired a research firm to run surveys. Software removed that cost but not the need for expertise, so many teams now build surveys without a blueprint. Here is where each problem starts:<\/p>\n\n\n\n<ul>\n<li><strong>Who you ask:<\/strong> Coverage gaps, biased samples, and people who never respond.<\/li>\n\n\n\n<li><strong>What you ask:<\/strong> Leading wording, double-barreled questions, jargon, and weak answer options.<\/li>\n\n\n\n<li><strong>How you ask:<\/strong> Length, question order, broken logic, and poor mobile design.<\/li>\n\n\n\n<li><strong>Who answers and what you do next:<\/strong> Social desirability, fake responses, and open-text answers that are hard to analyze.<\/li>\n<\/ul>\n\n\n\n<p>The sections below follow that order. Each problem comes with a definition, an example, and a fix.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do sampling and coverage problems skew survey results?<\/strong><\/h2>\n\n\n\n<p>A survey can only describe the people it reaches and hears from. Three problems break that link: <em>coverage error, sampling bias, and non-response bias.<\/em> The 1936 Literary Digest poll shows all three. The magazine mailed 10 million ballots,<a href=\"https:\/\/arxiv.org\/pdf\/2107.14166\" target=\"_blank\" rel=\"noreferrer noopener\"> received about 2.4 million back<\/a>, and predicted that Alf Landon would beat Franklin Roosevelt. Roosevelt won with roughly 61% of the vote.<\/p>\n\n\n\n<p>The ballots went to car and telephone owners, and the people who replied differed from those who did not. Volume could not fix either gap.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Coverage error<\/h3>\n\n\n\n<p>Coverage error happens when part of your target population has no chance of being included. The list or channel you use, called the sampling frame, leaves them out. The Literary Digest&#8217;s lists of car and telephone owners did exactly that.<\/p>\n\n\n\n<p>Common examples:<\/p>\n\n\n\n<ul>\n<li>An email-only survey misses customers with no email address on file.<\/li>\n\n\n\n<li>A web-only survey misses people with limited internet access.<\/li>\n\n\n\n<li>An English-only survey misses respondents who prefer another language.<\/li>\n<\/ul>\n\n\n\n<p>Fix it by defining the full population first. Then add channels, such as SMS, in-app prompts, or mail, until every major group can respond.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Sampling bias and self-selection<\/h3>\n\n\n\n<p><a href=\"https:\/\/www.questionpro.com\/blog\/sampling-bias\/\">Sampling bias<\/a> is a sample that systematically over-represents some groups and under-represents others. Self-selection is a common cause. People who volunteer to respond differ from people who do not.<\/p>\n\n\n\n<p>Picture a feedback pop-up on a pricing page. It attracts visitors with strong feelings about price, and it misses everyone else.<\/p>\n\n\n\n<p>A larger sample does not fix this problem. Random selection from a defined list does, along with quotas for key groups such as age, region, and customer type. When gaps remain, weighting helps. Weighting adjusts results so each group counts in proportion to its real share of the population.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Non-response bias<\/h3>\n\n\n\n<p><a href=\"https:\/\/www.questionpro.com\/blog\/non-response-bias-what-it-is-how-to-avoid-it\/\">Non-response bias<\/a> appears when the people who skip your survey differ from the people who answer. It is not the same as a low response rate. Research shows that the typical telephone response rate was<a href=\"https:\/\/www.pewresearch.org\/short-reads\/2015\/07\/21\/the-challenges-of-polling-when-fewer-people-are-available-to-be-polled\/\" target=\"_blank\" rel=\"noreferrer noopener\"> 36% in 1997<\/a> and<a href=\"https:\/\/www.pewresearch.org\/methods\/2019\/02\/27\/response-rates\/\" target=\"_blank\" rel=\"noreferrer noopener\"> 6% in 2018<\/a>. It also notes that low rates alone do not make a poll inaccurate, but they raise the risk of error.<\/p>\n\n\n\n<p>The real question is whether non-responders differ in ways that matter. Three steps help:<\/p>\n\n\n\n<ul>\n<li>Send one or two reminders.<\/li>\n\n\n\n<li>Compare respondents with your full list on known traits such as age, plan type, and region.<\/li>\n\n\n\n<li>Follow up with a small sample of non-responders through another channel.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do badly written questions distort survey results?<\/strong><\/h2>\n\n\n\n<p>Badly written questions distort results by nudging answers, bundling ideas, or confusing respondents. Wording is the most controllable source of survey problems. Small wording differences can substantially change answers. Four patterns cause most of the damage. For more worked examples, see our guide to<a href=\"https:\/\/www.questionpro.com\/blog\/common-survey-mistakes\/\"> common survey mistakes<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Leading and loaded questions<\/h3>\n\n\n\n<p>A leading question uses wording that nudges respondents toward an answer. A loaded question builds in an assumption the respondent may not share.<\/p>\n\n\n\n<ul>\n<li><strong>Leading, before:<\/strong> How helpful was our award-winning support team?<\/li>\n\n\n\n<li><strong>Leading, after:<\/strong> How would you rate your support experience?<\/li>\n\n\n\n<li><strong>Loaded, before:<\/strong> What frustrated you about your recent hospital visit?<\/li>\n\n\n\n<li><strong>Loaded, after:<\/strong> How would you describe your recent hospital visit?<\/li>\n<\/ul>\n\n\n\n<p>Read each question aloud. If a stranger could guess the answer you want, rewrite it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Double-barreled questions<\/h3>\n\n\n\n<p>A double-barreled question asks about two things but allows only one answer. Ask one question at a time. Consider this one: &#8220;How satisfied are you with our pricing and customer support?&#8221; A customer who loves the support but resents the price cannot answer honestly.<\/p>\n\n\n\n<p>Split it into two questions. Whenever you see &#8220;and&#8221; or &#8220;or&#8221; in a question, check whether it hides a second idea.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Confusing wording, jargon, and double negatives<\/h3>\n\n\n\n<p>Respondents answer the question they understand, not the one you meant. These four wording problems show up most often.<\/p>\n\n\n\n<div style=\"margin:1.5rem 0;width:100%;max-width:100%;\">\n  <table style=\"border-collapse:collapse;width:100%;max-width:100%;table-layout:auto;\">\n    <thead>\n      <tr>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Problem<\/th>\n        <th style=\"background:#162450;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Before<\/th>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">After<\/th>\n      <\/tr>\n    <\/thead>\n    <tbody>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Jargon<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">How satisfied are you with our omnichannel touchpoints?<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">How satisfied are you with the ways you can contact us?<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Double negative<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Do you disagree that the app should not require a password?<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Should the app require a password?<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Too broad<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">What problems do you have with our product?<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Which feature gave you the most trouble this month?<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Too wordy<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Considering everything you experienced with us this year, including purchases, support, and billing, how do you feel overall?<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Overall, how satisfied are you with us?<\/td>\n      <\/tr>\n    <\/tbody>\n  <\/table>\n<\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do answer options and rating scales cause survey problems?<\/strong><\/h2>\n\n\n\n<p>Respondents can only pick from the choices you show them, so weak answer options distort data even when the question is sound. In an experiment on voting priorities, 58% chose the economy when it appeared in a list. Only 35% named it on their own when the question was open-ended.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Missing and overlapping answer options<\/h3>\n\n\n\n<p>Answer lists fail in two ways. They overlap, or they leave people out.<\/p>\n\n\n\n<ul>\n<li><strong>Overlapping:<\/strong> 0-1 years, 1-3 years, 3-5 years. A customer of exactly three years fits two boxes.<\/li>\n\n\n\n<li><strong>Missing:<\/strong> The same list has no box for customers of five years or more.<\/li>\n<\/ul>\n\n\n\n<p>Fix it with options that are mutually exclusive, meaning no overlap, and exhaustive, meaning everyone fits somewhere. A better list reads: less than 1 year, 1 to under 3 years, 3 to under 5 years, 5 years or more. Add &#8220;Other (please specify)&#8221; or &#8220;None of the above&#8221; when you cannot list every case.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Ambiguous rating scales<\/h3>\n\n\n\n<p>A rating scale is ambiguous when respondents define the numbers differently. One customer gives an 8 after a flawless visit because nothing is ever perfect. Another gives a 10 after a poor visit because a low score might get an employee in trouble. The two scores mean different things, but your dashboard treats them as equal.<\/p>\n\n\n\n<p>Four fixes:<\/p>\n\n\n\n<ol>\n<li>Label every point, not only the endpoints, such as &#8220;Very dissatisfied&#8221; through &#8220;Very satisfied.&#8221;<\/li>\n\n\n\n<li>Keep the direction identical across the whole survey.<\/li>\n\n\n\n<li>Use four or five options for opinion questions. People struggle to hold more choices in mind at once.<\/li>\n\n\n\n<li>Add a follow-up question for the lowest and highest scores.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do you capture the reasons behind survey answers?<\/strong><\/h2>\n\n\n\n<p>Capture the reasons behind survey answers by adding targeted follow-up questions and coding the responses consistently. A rating tells you what happened but not why. Two respondents with different reasons can give the same score, and a closed list cannot capture reasons you did not think of. In the same experiment, 43% of open-ended respondents gave an answer that was missing from the closed list. Only 8% of closed-list respondents went beyond it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Add targeted follow-up questions<\/h3>\n\n\n\n<p>Open-ended questions help, but too many tire respondents. Add them where an explanation changes a decision.<\/p>\n\n\n\n<ul>\n<li><strong>After a rating, ask:<\/strong> <em>What was the main reason for your score?<\/em><\/li>\n\n\n\n<li><strong>After a multiple-choice question about the most important feature, ask:<\/strong> <em>What made that feature important to you?<\/em><\/li>\n\n\n\n<li><strong>After a low score, ask: <\/strong><em>What would have improved your experience?<\/em><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Code open-ended responses consistently<\/h3>\n\n\n\n<p>Coding means tagging each comment with a theme so you can count it. Hundreds of unique comments are hard to analyze by reading alone, and inconsistent tagging adds a new error. Follow these steps:<\/p>\n\n\n\n<ol>\n<li>Read a first batch and draft a short list of themes, called a codebook.<\/li>\n\n\n\n<li>Have two people tag the same sample and compare results. Resolve differences and tighten the theme definitions.<\/li>\n\n\n\n<li>Apply the codebook to every response. Add a theme only when a pattern repeats.<\/li>\n\n\n\n<li>Report theme counts next to representative comments.<\/li>\n<\/ol>\n\n\n\n<p>Text analysis tools and AI can speed up tagging. Spot-check a sample of their output against human judgment before you trust the counts.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do survey length, order, and design hurt response quality?<\/strong><\/h2>\n\n\n\n<p>Respondents rarely quit because of one bad question. They quit because the survey feels like work. Length, order, logic, and device design decide whether people finish and whether they answer carefully.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Survey length and fatigue<\/h3>\n\n\n\n<p><a href=\"https:\/\/www.questionpro.com\/blog\/survey-fatigue\/\">Survey fatigue<\/a> is the drop in effort and completion that happens when a survey feels too long, repetitive, or irrelevant. Watch for these signs:<\/p>\n\n\n\n<ul>\n<li>Skip rates that climb in the second half of the survey<\/li>\n\n\n\n<li>Straight-lining, which means choosing the same answer down a whole grid<\/li>\n\n\n\n<li>Answers that get faster from page to page<\/li>\n\n\n\n<li>Sharp drop-off on a specific page<\/li>\n<\/ul>\n\n\n\n<p>Cut every question you cannot tie to a decision. Give respondents an honest time estimate and show progress as they go.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Question order and demographic placement<\/h3>\n\n\n\n<p>Earlier questions can change how people answer later ones. Researchers call this an order effect. In the same previous experiment, 88% said they were dissatisfied with the way things were going in the country when that question followed a presidential approval question. Only 78% said so without the approval question first.<\/p>\n\n\n\n<p>Group questions by topic and randomize lists where order carries no meaning. Place demographic questions near the end unless you need them to screen respondents. Opening with personal questions makes the survey feel like a form instead of a conversation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Broken logic and poor mobile design<\/h3>\n\n\n\n<p>Skip logic sends respondents to different questions based on earlier answers. Broken logic creates dead ends, loops, and irrelevant questions. Wide grids that look fine on a laptop can become unreadable on a phone. Test every survey with these steps:<\/p>\n\n\n\n<ol>\n<li>Sketch the survey as a flowchart before you build it.<\/li>\n\n\n\n<li>Click through every branch and confirm each path reaches the finish page.<\/li>\n\n\n\n<li>Check that no path sends respondents back to a question they already answered.<\/li>\n\n\n\n<li>Complete the survey on a phone, not only a desktop.<\/li>\n\n\n\n<li>Replace wide matrix grids with one question per screen where you can.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Why do respondents give inaccurate answers even when questions are well written?<\/strong><\/h2>\n\n\n\n<p>Respondents give inaccurate answers because people want to look good, agree politely, and forget details. Clear questions still pick up those habits.<\/p>\n\n\n\n<div style=\"margin:1.5rem 0;width:100%;max-width:100%;\">\n  <table style=\"border-collapse:collapse;width:100%;max-width:100%;table-layout:auto;\">\n    <thead>\n      <tr>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Bias<\/th>\n        <th style=\"background:#162450;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">What happens<\/th>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">How to reduce it<\/th>\n      <\/tr>\n    <\/thead>\n    <tbody>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Social desirability bias<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Respondents give answers that make them look good, especially on sensitive topics<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Offer anonymity where possible and word options so honest answers feel normal<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Acquiescence bias<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Respondents tend to agree with any statement<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Replace agree\/disagree items with a choice between two statements<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Recall bias<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">People forget or misremember past events<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Shorten the time window and survey soon after the event<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Central tendency bias<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Respondents avoid extreme scale points<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Label every point and write options that describe real behavior<\/td>\n      <\/tr>\n    <\/tbody>\n  <\/table>\n<\/div>\n\n\n\n<p>For a fuller catalog, read our guide on<a href=\"https:\/\/www.questionpro.com\/blog\/how-to-avoid-survey-bias\/\"> how to avoid survey bias<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How are bogus and AI-generated responses changing survey problems?<\/strong><\/h2>\n\n\n\n<p>Bogus and AI-generated responses add false data that looks real, which makes quality checks essential. Bogus respondents are people who answer without sincerity to collect rewards with minimal effort. Research shows that studies have documented large errors from bogus respondents in online opt-in surveys, concentrated among adults under 30 and Hispanic adults. Generative AI tools can make low-effort open-text answers harder to spot by eye, which raises the value of systematic checks.<\/p>\n\n\n\n<p>Build these checks into every survey:<\/p>\n\n\n\n<ul>\n<li>Add one attention check, such as an item that tells respondents which answer to select.<\/li>\n\n\n\n<li>Flag respondents who finish in under half the median completion time. Treat this as a starting rule and tune it for your survey.<\/li>\n\n\n\n<li>Look for straight-lining and contradictions, such as an age of 22 paired with 30 years of work experience.<\/li>\n\n\n\n<li>Read a sample of open-text answers for copied, generic, or off-topic text.<\/li>\n\n\n\n<li>Buy samples from sources that verify respondents.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do you measure survey quality and spot problems early?<\/strong><\/h2>\n\n\n\n<p>Measure survey quality by tracking five signals: completion rate, completion time, skip rate, straight-lining, and the gap between your sample and your population. Check them while the survey is live and again after it closes. The table shows what each one usually points to.<\/p>\n\n\n\n<div style=\"margin:1.5rem 0;width:100%;max-width:100%;\">\n  <table style=\"border-collapse:collapse;width:100%;max-width:100%;table-layout:auto;\">\n    <thead>\n      <tr>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Signal<\/th>\n        <th style=\"background:#162450;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">What it may reveal<\/th>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">What to do<\/th>\n      <\/tr>\n    <\/thead>\n    <tbody>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Completion rate (finished divided by started)<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">A long survey, a confusing page, or a technical fault<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Find the page with the biggest drop-off and fix that question first<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Median (middle) completion time vs. your pilot time<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Speeders or an overlong survey<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Review anyone under half the median time<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Skip rate by question<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">A sensitive, unclear, or irrelevant question<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Rewrite it or make it optional<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Straight-lining share<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Fatigue or low effort<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Shorten grids and add attention checks<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Sample vs. population gap<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Coverage gaps or non-response bias<\/td>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Compare respondents with your customer list or US Census Bureau data, then recruit or weight to close the gap<\/td>\n      <\/tr>\n    <\/tbody>\n  <\/table>\n<\/div>\n\n\n\n<p>Here is an example with illustrative numbers. Customers aged 18 to 34 make up 30% of your list but only 12% of your respondents. Their views count for far less than they should. Weighting gives each of them more influence, but it cannot add views you never collected.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do you decide which survey problems to fix first?<\/strong><\/h2>\n\n\n\n<p>Fix the problems that make data invalid before the ones that only make it less efficient. Not every problem costs the same, so work through this order:<\/p>\n\n\n\n<ol>\n<li><strong>Can the right people answer?<\/strong> Fix coverage and sampling first. No wording change rescues data from the wrong audience.<br><\/li>\n\n\n\n<li><strong>Does each question measure one thing?<\/strong> Remove leading, loaded, and double-barreled wording.<br><\/li>\n\n\n\n<li><strong>Can respondents answer accurately?<\/strong> Repair answer options, scales, and jargon.<br><\/li>\n\n\n\n<li><strong>Will enough of them finish?<\/strong> Cut length, repair logic, and test on mobile.<br><\/li>\n\n\n\n<li><strong>Can you trust the responses?<\/strong> Add quality checks and compare the sample with a benchmark.<\/li>\n<\/ol>\n\n\n\n<p>When time is short, stop after step three. Steps one to three decide whether the data means anything.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How do you test a survey for mistakes before launch?<\/strong><\/h2>\n\n\n\n<p>Test a survey by running a pilot, which is a small trial run with people like your real respondents, and then reviewing both their feedback and their data. A pilot catches problems the author can no longer see.<\/p>\n\n\n\n<ul>\n<li><strong>Read every question aloud:<\/strong> Awkward phrasing shows up fast.<\/li>\n\n\n\n<li><strong>Test with 5 to 10 people from your target audience:<\/strong> Ask each person to explain every question in their own words.<\/li>\n\n\n\n<li><strong>Run a split test on wording<\/strong> when two versions seem equal. Give half the sample to each version to see whether wording changes the results.<\/li>\n\n\n\n<li><strong>Review pilot data:<\/strong> Look for skipped questions, long pauses, and logic dead ends.<\/li>\n\n\n\n<li><strong>Confirm privacy basics:<\/strong> State why you collect data, collect only what you need, and check which privacy and consent rules apply to your audience, including state laws in the US.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How does QuestionPro help you catch survey problems before launch?<\/strong><\/h2>\n\n\n\n<p>QuestionPro&#8217;s<a href=\"https:\/\/www.questionpro.com\/blog\/survey-score\/\"> Survey Score<\/a> reviews a survey inside the Survey Builder before you publish. It flags issues across five categories and gives a readiness score, so problems surface before respondents find them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What Survey Score checks<\/h3>\n\n\n\n<p>Each category maps to a problem covered above. AI reviews also read question wording and tag their findings as AI-judged, with a confidence label.<\/p>\n\n\n\n<ul>\n<li><strong>Logic and flow:<\/strong> Dangling branches, unreachable questions, and loops<\/li>\n\n\n\n<li><strong>Methodology:<\/strong> Leading wording, too few answer options, and double-barreled questions<\/li>\n\n\n\n<li><strong>Accessibility:<\/strong> Checks derived from the Web Content Accessibility Guidelines (WCAG), such as alt text, input labels, and contrast<\/li>\n\n\n\n<li><strong>Respondent experience:<\/strong> Excessive length and drop-off pressure<\/li>\n\n\n\n<li><strong>Compliance:<\/strong> Consent, handling of personal information, and regional disclosures<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">How readiness scores and fixes work<\/h3>\n\n\n\n<p>Every survey starts at 100, and each finding lowers the score by severity. A Blocker means respondents will hit an error or dead end. A Warning means data quality will suffer. An Advisory is optional.<\/p>\n\n\n\n<div style=\"margin:1.5rem 0;width:100%;max-width:100%;\">\n  <table style=\"border-collapse:collapse;width:100%;max-width:100%;table-layout:auto;\">\n    <thead>\n      <tr>\n        <th style=\"background:#1a2b5e;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Band<\/th>\n        <th style=\"background:#162450;color:#fff;padding:10px 14px;border:1px solid #C5CFE8;font-size:18px;line-height:1.4;text-align:left;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Score<\/th>\n      <\/tr>\n    <\/thead>\n    <tbody>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Ready to publish<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">90 and above<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Good shape<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">75 to 89<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Needs work<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">50 to 74<\/td>\n      <\/tr>\n      <tr>\n        <td style=\"background:#ffffff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;font-weight:600;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Not ready<\/td>\n        <td style=\"background:#f0f4ff;padding:10px 14px;border:1px solid #E5E7EB;font-size:16px;line-height:1.5;vertical-align:top;white-space:normal;word-break:normal;overflow-wrap:break-word;\">Below 50<\/td>\n      <\/tr>\n    <\/tbody>\n  <\/table>\n<\/div>\n\n\n\n<p>You can apply a one-click fix, review a suggested rewrite, or preview an AI-proposed change. Nothing AI-generated applies until you confirm it. Survey Score is in open beta and runs on Advanced, Team Edition, and Research Edition licenses. It supports your accessibility and compliance reviews without replacing them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reaching the right respondents<\/h3>\n\n\n\n<p>Problems that start with the sample need a different fix. When your own list leaves gaps,<a href=\"https:\/\/www.questionpro.com\/audience\/\"> QuestionPro Audience<\/a> gives you access to targeted respondents instead of a general population list. You can then build and field the questionnaire in<a href=\"https:\/\/www.questionpro.com\/survey-software\/\"> survey software<\/a> that handles logic and reporting.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Reliable survey data starts with clear questions and the right people<\/strong><\/h2>\n\n\n\n<p>Most problems with surveys share one root cause. The author built the survey from their own point of view, not the respondent&#8217;s. The fix is to read each question as if you were seeing it for the first time.<\/p>\n\n\n\n<p>Keep three habits:<\/p>\n\n\n\n<ul>\n<li>Reach the right people before you polish the wording.<\/li>\n\n\n\n<li>Ask one clear thing at a time, and let people explain when it matters.<\/li>\n\n\n\n<li>Pilot every survey and watch the quality signals after launch.<\/li>\n<\/ul>\n\n\n\n<p>Good surveys do not need perfect statistics. They need fewer, clearer questions and honest checks on who answered.<\/p>\n\n\n\n\n\t<div class=\"banner-section wf-section\" lang=\"\" >\n\t\t<div class=\"right-column-container\">\n\t\t\t<div class=\"bannerbg white\">\n\t\t\t\t<span class=\"h1-2\">Create memorable experiences based on real-time data, insights and advanced analysis.<\/span>\n\t\t\t\t<a href=\"#userliteForm\" data-toggle=\"modal\" class=\"button w-button\">Request Demo<\/a>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n\t<div class=\"userlite-modal modal fade\" id=\"userliteForm\" tabindex=\"-1\" role=\"dialog\" style=\"display: none;\">\n\t\t<div class=\"modal-dialog\" role=\"document\">\n\t\t\t<div class=\"modal-content\" role=\"document\">\n\t\t\t\t<div class=\"modal-body\">\n\t\t\t\t\t<div class=\"modal-header\">\n\t\t\t\t\t\t<button type=\"button\" class=\"close\" data-dismiss=\"modal\" aria-label=\"Close\">\n\t\t\t\t\t\t\t<i class=\"material-icons\">close<\/i>\n\t\t\t\t\t\t<\/button>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t<div class=\"contact-us-form-wrapper contact-box\">\n\t\t\t\t\t\t<div class=\"userlite-form-wrapper\">\n\t\t\t\t\t\t\t<iframe src=\"https:\/\/www.questionpro.com\/userlite-form-blog-en.html?product=Surveys&amp;referralurl=https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/10839&amp;lang=en&amp;cat=enquetes-fr|etude-de-marche-fr|survey-software\" style=\"display: block;\" ><\/iframe>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"demo-form-wrapper success-message-div\" style=\"display:none\">\n\t\t\t\t\t\t\t<p class=\"success-message-para\"><\/p>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently Asked Questions (FAQs)<\/h2>\n\n\n\n<div class=\"schema-faq wp-block-yoast-faq-block\"><div class=\"schema-faq-section\" id=\"faq-question-1791376353117\"><strong class=\"schema-faq-question\"><strong>Can weighting fix a biased survey sample?<\/strong><\/strong> <p class=\"schema-faq-answer\">Weighting adjusts results so each group counts in proportion to the population, which helps when the gaps are known, such as age or region. It cannot fix differences you never measured, so pair it with better recruitment instead of relying on it alone.<\/p> <\/div> <div class=\"schema-faq-section\" id=\"faq-question-1791376377797\"><strong class=\"schema-faq-question\"><strong>Do incentives make survey problems worse?<\/strong><\/strong> <p class=\"schema-faq-answer\">Incentives can raise response rates, but they also give bogus respondents a reason to join.<a href=\"https:\/\/www.pewresearch.org\/short-reads\/2024\/03\/05\/online-opt-in-polls-can-produce-misleading-results-especially-for-young-people-and-hispanic-adults\/\"> <\/a>They are the people who rush through surveys to earn rewards. Pair incentives with attention checks and verified sources.<\/p> <\/div> <div class=\"schema-faq-section\" id=\"faq-question-1791376385178\"><strong class=\"schema-faq-question\"><strong>What survey problems are most common in employee surveys?<\/strong><\/strong> <p class=\"schema-faq-answer\">A common one is fear of being identified, which pushes people toward safe, positive answers. Promise anonymity only when you can deliver it, and set a minimum group size before you report results for any team.<\/p> <\/div> <div class=\"schema-faq-section\" id=\"faq-question-1791376393126\"><strong class=\"schema-faq-question\"><strong>Can AI tools write survey questions without causing problems?<\/strong><\/strong> <p class=\"schema-faq-answer\">AI can draft questions quickly, but drafts often reuse leading phrasing or bundle two ideas into one question. Treat the output as a first draft, review every question against the checks above, and pilot it with real respondents.<\/p> <\/div> <div class=\"schema-faq-section\" id=\"faq-question-1791376400681\"><strong class=\"schema-faq-question\"><strong>How do you compare survey results over time if the wording changes?<\/strong><\/strong> <p class=\"schema-faq-answer\">Keep wording and question order identical between waves.<a href=\"https:\/\/www.pewresearch.org\/writing-survey-questions\/\"> <\/a>Use the same wording and a similar context when tracking trends. If you must change a question, mark a break in the trend line.<\/p> <\/div> <\/div>\n","protected":false},"excerpt":{"rendered":"<p>Problems with surveys are flaws in who you ask, what you ask, or how you analyze the answers. They make [&hellip;]<\/p>\n","protected":false},"author":51,"featured_media":807296,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_yoast_wpseo_focuskw":"problems with surveys","_yoast_wpseo_title":"%%title%%","_yoast_wpseo_metadesc":"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.","_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[1442,1443,187],"tags":[284,377],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Problems with Surveys: Types, Examples, and How to Avoid Them<\/title>\n<meta name=\"description\" content=\"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Problems with Surveys: Types, Examples, and How to Avoid Them\" \/>\n<meta property=\"og:description\" content=\"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\" \/>\n<meta property=\"og:site_name\" content=\"QuestionPro\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/questionpro\" \/>\n<meta property=\"article:published_time\" content=\"2014-12-22T14:00:45+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-10-07T12:40:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2023\/08\/problems_with_surveys.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2100\" \/>\n\t<meta property=\"og:image:height\" content=\"1254\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Anas Al Masud\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@questionpro\" \/>\n<meta name=\"twitter:site\" content=\"@questionpro\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Anas Al Masud\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\"},\"author\":{\"name\":\"Anas Al Masud\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/9eea0e42df379be31b78fff9d6d0ade3\"},\"headline\":\"Problems with Surveys: Types, Examples, and How to Avoid Them\",\"datePublished\":\"2014-12-22T14:00:45+00:00\",\"dateModified\":\"2026-10-07T12:40:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\"},\"wordCount\":3481,\"publisher\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\"},\"keywords\":[\"Best Practice\",\"Top10\"],\"articleSection\":[\"Enqu\u00eates\",\"\u00c9tude de march\u00e9\",\"Surveys\"],\"inLanguage\":\"en-US\"},{\"@type\":[\"WebPage\",\"FAQPage\"],\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\",\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\",\"name\":\"Problems with Surveys: Types, Examples, and How to Avoid Them\",\"isPartOf\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#website\"},\"datePublished\":\"2014-12-22T14:00:45+00:00\",\"dateModified\":\"2026-10-07T12:40:36+00:00\",\"description\":\"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#breadcrumb\"},\"mainEntity\":[{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117\"},{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797\"},{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178\"},{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126\"},{\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681\"}],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.questionpro.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Surveys\",\"item\":\"https:\/\/www.questionpro.com\/blog\/category\/survey-software\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Problems with Surveys: Types, Examples, and How to Avoid Them\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#website\",\"url\":\"https:\/\/www.questionpro.com\/blog\/\",\"name\":\"QuestionPro\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.questionpro.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\",\"name\":\"QuestionPro\",\"url\":\"https:\/\/www.questionpro.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg\",\"contentUrl\":\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg\",\"caption\":\"QuestionPro\"},\"image\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/questionpro\",\"https:\/\/twitter.com\/questionpro\",\"https:\/\/www.linkedin.com\/company\/questionpro\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/9eea0e42df379be31b78fff9d6d0ade3\",\"name\":\"Anas Al Masud\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f6a7635b41d5d7d93f424df5177347b8?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f6a7635b41d5d7d93f424df5177347b8?s=96&d=mm&r=g\",\"caption\":\"Anas Al Masud\"},\"description\":\"Digital Marketing Lead at QuestionPro. SEO-driven content strategist specializing in content that ranks, engages, and converts, while boosting online visibility through hands-on digital marketing expertise.\",\"url\":\"https:\/\/www.questionpro.com\/blog\/author\/anas-al-masud\/\"},{\"@type\":\"Question\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117\",\"position\":1,\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117\",\"name\":\"Can weighting fix a biased survey sample?\",\"answerCount\":1,\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"Weighting adjusts results so each group counts in proportion to the population, which helps when the gaps are known, such as age or region. It cannot fix differences you never measured, so pair it with better recruitment instead of relying on it alone.\",\"inLanguage\":\"en-US\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Question\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797\",\"position\":2,\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797\",\"name\":\"Do incentives make survey problems worse?\",\"answerCount\":1,\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"Incentives can raise response rates, but they also give bogus respondents a reason to join.<a href=\\\"https:\/\/www.pewresearch.org\/short-reads\/2024\/03\/05\/online-opt-in-polls-can-produce-misleading-results-especially-for-young-people-and-hispanic-adults\/\\\"> <\/a>They are the people who rush through surveys to earn rewards. Pair incentives with attention checks and verified sources.\",\"inLanguage\":\"en-US\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Question\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178\",\"position\":3,\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178\",\"name\":\"What survey problems are most common in employee surveys?\",\"answerCount\":1,\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"A common one is fear of being identified, which pushes people toward safe, positive answers. Promise anonymity only when you can deliver it, and set a minimum group size before you report results for any team.\",\"inLanguage\":\"en-US\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Question\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126\",\"position\":4,\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126\",\"name\":\"Can AI tools write survey questions without causing problems?\",\"answerCount\":1,\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"AI can draft questions quickly, but drafts often reuse leading phrasing or bundle two ideas into one question. Treat the output as a first draft, review every question against the checks above, and pilot it with real respondents.\",\"inLanguage\":\"en-US\"},\"inLanguage\":\"en-US\"},{\"@type\":\"Question\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681\",\"position\":5,\"url\":\"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681\",\"name\":\"How do you compare survey results over time if the wording changes?\",\"answerCount\":1,\"acceptedAnswer\":{\"@type\":\"Answer\",\"text\":\"Keep wording and question order identical between waves.<a href=\\\"https:\/\/www.pewresearch.org\/writing-survey-questions\/\\\"> <\/a>Use the same wording and a similar context when tracking trends. If you must change a question, mark a break in the trend line.\",\"inLanguage\":\"en-US\"},\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Problems with Surveys: Types, Examples, and How to Avoid Them","description":"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/","og_locale":"en_US","og_type":"article","og_title":"Problems with Surveys: Types, Examples, and How to Avoid Them","og_description":"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.","og_url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/","og_site_name":"QuestionPro","article_publisher":"https:\/\/www.facebook.com\/questionpro","article_published_time":"2014-12-22T14:00:45+00:00","article_modified_time":"2026-10-07T12:40:36+00:00","og_image":[{"width":2100,"height":1254,"url":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2023\/08\/problems_with_surveys.jpg","type":"image\/jpeg"}],"author":"Anas Al Masud","twitter_card":"summary_large_image","twitter_creator":"@questionpro","twitter_site":"@questionpro","twitter_misc":{"Written by":"Anas Al Masud","Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#article","isPartOf":{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/"},"author":{"name":"Anas Al Masud","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/9eea0e42df379be31b78fff9d6d0ade3"},"headline":"Problems with Surveys: Types, Examples, and How to Avoid Them","datePublished":"2014-12-22T14:00:45+00:00","dateModified":"2026-10-07T12:40:36+00:00","mainEntityOfPage":{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/"},"wordCount":3481,"publisher":{"@id":"https:\/\/www.questionpro.com\/blog\/#organization"},"keywords":["Best Practice","Top10"],"articleSection":["Enqu\u00eates","\u00c9tude de march\u00e9","Surveys"],"inLanguage":"en-US"},{"@type":["WebPage","FAQPage"],"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/","url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/","name":"Problems with Surveys: Types, Examples, and How to Avoid Them","isPartOf":{"@id":"https:\/\/www.questionpro.com\/blog\/#website"},"datePublished":"2014-12-22T14:00:45+00:00","dateModified":"2026-10-07T12:40:36+00:00","description":"Problems with surveys explained: sampling gaps, leading questions, fatigue, and fake responses, with before-and-after fixes and a pre-launch checklist.","breadcrumb":{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#breadcrumb"},"mainEntity":[{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117"},{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797"},{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178"},{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126"},{"@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681"}],"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.questionpro.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Surveys","item":"https:\/\/www.questionpro.com\/blog\/category\/survey-software\/"},{"@type":"ListItem","position":3,"name":"Problems with Surveys: Types, Examples, and How to Avoid Them"}]},{"@type":"WebSite","@id":"https:\/\/www.questionpro.com\/blog\/#website","url":"https:\/\/www.questionpro.com\/blog\/","name":"QuestionPro","description":"","publisher":{"@id":"https:\/\/www.questionpro.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.questionpro.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.questionpro.com\/blog\/#organization","name":"QuestionPro","url":"https:\/\/www.questionpro.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg","contentUrl":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg","caption":"QuestionPro"},"image":{"@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/questionpro","https:\/\/twitter.com\/questionpro","https:\/\/www.linkedin.com\/company\/questionpro\/"]},{"@type":"Person","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/9eea0e42df379be31b78fff9d6d0ade3","name":"Anas Al Masud","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/f6a7635b41d5d7d93f424df5177347b8?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f6a7635b41d5d7d93f424df5177347b8?s=96&d=mm&r=g","caption":"Anas Al Masud"},"description":"Digital Marketing Lead at QuestionPro. SEO-driven content strategist specializing in content that ranks, engages, and converts, while boosting online visibility through hands-on digital marketing expertise.","url":"https:\/\/www.questionpro.com\/blog\/author\/anas-al-masud\/"},{"@type":"Question","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117","position":1,"url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376353117","name":"Can weighting fix a biased survey sample?","answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"Weighting adjusts results so each group counts in proportion to the population, which helps when the gaps are known, such as age or region. It cannot fix differences you never measured, so pair it with better recruitment instead of relying on it alone.","inLanguage":"en-US"},"inLanguage":"en-US"},{"@type":"Question","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797","position":2,"url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376377797","name":"Do incentives make survey problems worse?","answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"Incentives can raise response rates, but they also give bogus respondents a reason to join.<a href=\"https:\/\/www.pewresearch.org\/short-reads\/2024\/03\/05\/online-opt-in-polls-can-produce-misleading-results-especially-for-young-people-and-hispanic-adults\/\"> <\/a>They are the people who rush through surveys to earn rewards. Pair incentives with attention checks and verified sources.","inLanguage":"en-US"},"inLanguage":"en-US"},{"@type":"Question","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178","position":3,"url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376385178","name":"What survey problems are most common in employee surveys?","answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"A common one is fear of being identified, which pushes people toward safe, positive answers. Promise anonymity only when you can deliver it, and set a minimum group size before you report results for any team.","inLanguage":"en-US"},"inLanguage":"en-US"},{"@type":"Question","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126","position":4,"url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376393126","name":"Can AI tools write survey questions without causing problems?","answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"AI can draft questions quickly, but drafts often reuse leading phrasing or bundle two ideas into one question. Treat the output as a first draft, review every question against the checks above, and pilot it with real respondents.","inLanguage":"en-US"},"inLanguage":"en-US"},{"@type":"Question","@id":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681","position":5,"url":"https:\/\/www.questionpro.com\/blog\/problems-with-surveys\/#faq-question-1791376400681","name":"How do you compare survey results over time if the wording changes?","answerCount":1,"acceptedAnswer":{"@type":"Answer","text":"Keep wording and question order identical between waves.<a href=\"https:\/\/www.pewresearch.org\/writing-survey-questions\/\"> <\/a>Use the same wording and a similar context when tracking trends. If you must change a question, mark a break in the trend line.","inLanguage":"en-US"},"inLanguage":"en-US"}]}},"featured_image_src":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2023\/08\/problems_with_surveys.jpg","featured_image_src_square":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2023\/08\/problems_with_surveys.jpg","author_info":{"display_name":"Anas Al Masud","author_link":"https:\/\/www.questionpro.com\/blog\/author\/anas-al-masud\/"},"_links":{"self":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/10839"}],"collection":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/users\/51"}],"replies":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/comments?post=10839"}],"version-history":[{"count":6,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/10839\/revisions"}],"predecessor-version":[{"id":1119168,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/10839\/revisions\/1119168"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/media\/807296"}],"wp:attachment":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/media?parent=10839"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/categories?post=10839"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/tags?post=10839"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}