Well, if the PhD survey scientists have said it — then it must be true: Size DOES Matter — at least when it comes to comment boxes. In this article, we’re going to share some fun facts that will not only astound and amaze your colleagues at the next cookout or cocktail party, but they might actually improve the quality of your survey results.
Here’s your first fun fact: Academics didn’t really start studying how visual layout of surveys effects responses until the early 1990’s! In fact, it was Tom Smith from the National Opinion Research Center in Chicago who wrote a groundbreaking paper entitled “Little Things Matter” that got the ball rolling on understanding what the impact of visual design is on response rates.
Since then several studies have been completed and now, a recent one entitled “Visual Design, Order Eﬀects, and Respondent Characteristics in a Self-Administered Survey” that comes from a team of researchers from Oklahoma State, University of Nebraska and Washington State University did several experiments in survey design and one of them included this idea of varying the size of the comment box to see if the size affected the number of words left in the comment box.
The short answer is YES — the bigger the box, the more words that respondents wrote in the comment box.
A second fun fact, was that some groups were more likely to provide longer and more detailed responses; this included respondents over
60 years of age, with less than a college degree, and women who all provided responses that were one to two words longer than their comparison group, regardless of the size of the box. Now that should lead to some very interesting conversation!
What to do with this information?
Not every survey requires extensive open-ended responses, but if you are doing exploratory research or collecting preliminary qualitative data be sure to create a larger comment box.
How to change the size of your comment box in QuestionPro