{"id":38745,"date":"2017-03-01T04:21:01","date_gmt":"2017-03-01T12:21:01","guid":{"rendered":"https:\/\/www.questionpro.com\/blog\/?p=38745\/"},"modified":"2017-03-01T04:21:01","modified_gmt":"2017-03-01T12:21:01","slug":"counter-order-bias-online-surveys-questionpro","status":"publish","type":"post","link":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/","title":{"rendered":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-38812 aligncenter\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/Survey-List-Order.jpg\"alt=\"image\" width=\"646\" height=\"386\" \/><\/p>\n<p>Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely presented first. This usually happens when the answer between a \u201cyes\u201d or a \u201cno\u201d may be a thin line.<\/p>\n<p>The same phenomenon occurs when we conduct surveys and present a question to the respondent with a rather sizable list of options in a particular order. At this stage the user may tend to click on the first few options because they were presented to them first and hence there is a \u201cbias\u201d. It\u2019s called the \u201cOrder Bias\u201d, and much like any other bias, it can be poisonous to a research survey, invariant of the type of survey you are running.<\/p>\n<p>To counter this particular bias, QuestionPro has a powerful functionality \u2013 Advanced Randomization. This feature is built with the purpose of randomizing multiple choice question\u2019s (MCQ) options in such an advanced way that the possibility of biased answers are highly minimised.<\/p>\n<p>Here is a step-by-step walkthrough for using Advanced Randomization in QuestionPro survey environment:<\/p>\n<p><strong>Step 1. <\/strong>Once you have your survey question with the lists ready, click on Edit.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38746\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/1.jpg\"alt=\"image\" width=\"744\" height=\"423\" \/><\/p>\n<p><strong>Step 2. <\/strong>Now go to the Settings tab and click on Validation &amp; Display Settings.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38752\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/2.jpg\"alt=\"image\" width=\"1028\" height=\"363\" \/><\/p>\n<p><strong>Step 3. <\/strong>Turn On Randomization and select Advanced Randomization.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38758\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/3.jpg\"alt=\"image\" width=\"403\" height=\"475\" \/><\/p>\n<p>A new screen will now pop up.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38764\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/new.jpg\"alt=\"image\" width=\"716\" height=\"421\" \/><\/p>\n<p>The next steps we will study with a few Case Studies to show how changes in settings affect the survey that is sent to respondents.<\/p>\n<p><strong>Case 1:<\/strong> Constant + Randomising<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38770\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/4.jpg\"alt=\"image\" width=\"716\" height=\"422\" \/><\/p>\n<p><strong>Result<\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38776\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/5.jpg\"alt=\"image\" width=\"742\" height=\"386\" \/><\/p>\n<p>As illustrated, all 4 items that we had selected have been randomized. The item in the middle continued to hold its position as we did not include it in the list.<\/p>\n<p><strong>Case 2:<\/strong> Randomising + Selective Randomization with Subsets<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38782\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/case-2.jpg\"alt=\"image\" width=\"716\" height=\"421\" \/><\/p>\n<p>Result<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38806\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/new-1.jpg\"alt=\"image\" width=\"742\" height=\"384\" \/><\/p>\n<p>Here, 2 items were randomized as per the first set. Only 2 items were selected at random from the second set. &#8220;Discover&#8221; was not randomized.<\/p>\n<p><strong>Case 3:<\/strong> Randomising + Subsets + Hide Items<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38794\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/8.jpg\"alt=\"image\" width=\"720\" height=\"425\" \/><\/p>\n<p><strong>Result<\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-38800\" src=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/9.jpg\"alt=\"image\" width=\"744\" height=\"357\" \/><\/p>\n<p>Now we have included a new set \u2013 one from where NO items are picked up. This is particularly useful when you have difference categories of survey respondents and you wish to withhold some options.<\/p>\n<p>As we just saw, Advanced Randomization is all about Permutations and Combinations. It allows you to have complete control over how and how many items of the list you would like to randomize.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise [&hellip;]<\/p>\n","protected":false},"author":70,"featured_media":38812,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[203],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro<\/title>\n<meta name=\"description\" content=\"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro\" \/>\n<meta property=\"og:description\" content=\"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\" \/>\n<meta property=\"og:site_name\" content=\"QuestionPro\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/questionpro\" \/>\n<meta property=\"article:published_time\" content=\"2017-03-01T12:21:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/Survey-List-Order.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"843\" \/>\n\t<meta property=\"og:image:height\" content=\"504\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Harshad Deshpande\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@questionpro\" \/>\n<meta name=\"twitter:site\" content=\"@questionpro\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Harshad Deshpande\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\"},\"author\":{\"name\":\"Harshad Deshpande\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/443b1c2aeec1dd398b91c1be333580c8\"},\"headline\":\"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization\",\"datePublished\":\"2017-03-01T12:21:01+00:00\",\"dateModified\":\"2017-03-01T12:21:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\"},\"wordCount\":400,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\"},\"articleSection\":[\"Market Research\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\",\"url\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\",\"name\":\"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro\",\"isPartOf\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#website\"},\"datePublished\":\"2017-03-01T12:21:01+00:00\",\"dateModified\":\"2017-03-01T12:21:01+00:00\",\"description\":\"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely\",\"breadcrumb\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.questionpro.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Do it yourself\",\"item\":\"https:\/\/www.questionpro.com\/blog\/tag\/do-it-yourself\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#website\",\"url\":\"https:\/\/www.questionpro.com\/blog\/\",\"name\":\"QuestionPro\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.questionpro.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#organization\",\"name\":\"QuestionPro\",\"url\":\"https:\/\/www.questionpro.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg\",\"contentUrl\":\"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg\",\"caption\":\"QuestionPro\"},\"image\":{\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/questionpro\",\"https:\/\/twitter.com\/questionpro\",\"https:\/\/www.linkedin.com\/company\/questionpro\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/443b1c2aeec1dd398b91c1be333580c8\",\"name\":\"Harshad Deshpande\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5534268e892a0e479dfc74f8954da9c8?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5534268e892a0e479dfc74f8954da9c8?s=96&d=mm&r=g\",\"caption\":\"Harshad Deshpande\"},\"description\":\"Director, Sales at QuestionPro. Empowering Organizations and Higher Ed with Research Survey Software and CX Solutions.\",\"url\":\"https:\/\/www.questionpro.com\/blog\/author\/harshad\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro","description":"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/","og_locale":"en_US","og_type":"article","og_title":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro","og_description":"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely","og_url":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/","og_site_name":"QuestionPro","article_publisher":"https:\/\/www.facebook.com\/questionpro","article_published_time":"2017-03-01T12:21:01+00:00","og_image":[{"width":843,"height":504,"url":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/Survey-List-Order.jpg","type":"image\/jpeg"}],"author":"Harshad Deshpande","twitter_card":"summary_large_image","twitter_creator":"@questionpro","twitter_site":"@questionpro","twitter_misc":{"Written by":"Harshad Deshpande","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#article","isPartOf":{"@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/"},"author":{"name":"Harshad Deshpande","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/443b1c2aeec1dd398b91c1be333580c8"},"headline":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization","datePublished":"2017-03-01T12:21:01+00:00","dateModified":"2017-03-01T12:21:01+00:00","mainEntityOfPage":{"@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/"},"wordCount":400,"commentCount":0,"publisher":{"@id":"https:\/\/www.questionpro.com\/blog\/#organization"},"articleSection":["Market Research"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/","url":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/","name":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization | QuestionPro","isPartOf":{"@id":"https:\/\/www.questionpro.com\/blog\/#website"},"datePublished":"2017-03-01T12:21:01+00:00","dateModified":"2017-03-01T12:21:01+00:00","description":"Human psychology has so much influence over what we do and the choices we make. Often we don\u2019t even realise we have made a choice because it was merely","breadcrumb":{"@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.questionpro.com\/blog\/counter-order-bias-online-surveys-questionpro\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.questionpro.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Do it yourself","item":"https:\/\/www.questionpro.com\/blog\/tag\/do-it-yourself\/"},{"@type":"ListItem","position":3,"name":"Counter Question Order-Bias in Online Survey with QuestionPro Advanced Randomization"}]},{"@type":"WebSite","@id":"https:\/\/www.questionpro.com\/blog\/#website","url":"https:\/\/www.questionpro.com\/blog\/","name":"QuestionPro","description":"","publisher":{"@id":"https:\/\/www.questionpro.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.questionpro.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.questionpro.com\/blog\/#organization","name":"QuestionPro","url":"https:\/\/www.questionpro.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg","contentUrl":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2022\/10\/questionpro-logo.svg","caption":"QuestionPro"},"image":{"@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/questionpro","https:\/\/twitter.com\/questionpro","https:\/\/www.linkedin.com\/company\/questionpro\/"]},{"@type":"Person","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/443b1c2aeec1dd398b91c1be333580c8","name":"Harshad Deshpande","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.questionpro.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5534268e892a0e479dfc74f8954da9c8?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5534268e892a0e479dfc74f8954da9c8?s=96&d=mm&r=g","caption":"Harshad Deshpande"},"description":"Director, Sales at QuestionPro. Empowering Organizations and Higher Ed with Research Survey Software and CX Solutions.","url":"https:\/\/www.questionpro.com\/blog\/author\/harshad\/"}]}},"featured_image_src":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/Survey-List-Order.jpg","featured_image_src_square":"https:\/\/www.questionpro.com\/blog\/wp-content\/uploads\/2017\/03\/Survey-List-Order.jpg","author_info":{"display_name":"Harshad Deshpande","author_link":"https:\/\/www.questionpro.com\/blog\/author\/harshad\/"},"_links":{"self":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/38745"}],"collection":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/users\/70"}],"replies":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/comments?post=38745"}],"version-history":[{"count":0,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/posts\/38745\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/media\/38812"}],"wp:attachment":[{"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/media?parent=38745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/categories?post=38745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.questionpro.com\/blog\/wp-json\/wp\/v2\/tags?post=38745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}