When conducting an investigation we always want it to be efficient, to meet the goals and for the process to be carried out with the fewest number of difficulties. If you are determined to use surveys as a research method, one way to make sure that everything will go well is to do a pilot survey to help you detect any unforeseen events that may arise.
Every research project involves money, time and effort, so carrying out various tests before starting the process will be of great help, especially when a large number of participants are involved. Let’s get into the world of pilot surveys.
LEARN ABOUT: Candidate Experience Survey
What is a pilot survey?
People think that testing a survey takes a long time and requires a lot of resources to do it. The pilot survey is a strategy that helps to evaluate or test a questionnaire using a smaller sample size than the planned sample.
Any test is better than none, so if you are going to do a survey it is best to test it with the resources you have available, keep in mind that even applying it to a very small group of people you can make significant improvements to your research.
LEARN ABOUT: Survey Mistakes And How to Avoid
Uses of a pilot survey
You can make use of pilot surveys to:
- Apply it to a small group that will not be evaluated in the original survey.
- Consider the responses of a selected group as the first results obtained from the project.
- You can also use a pilot survey as the definitive survey of your research and deliberately ask participants for feedback, for example asking them how clear the instructions are or what questions are difficult to answer.
After obtaining and analyzing results from the pilot survey, logistical, technical and any other issues can be addressed. You can correct the questions on your survey or choose the most appropriate types of questions if, for example, you are going to carry out an online survey.
A pilot survey can be used to detect the lack of training of the personnel to be surveyed, issues with the logistics of distribution and collection of the survey as well as errors in the data recording. These problems can be fixed before taking the actual survey.
Advantages of a Pilot Survey
Conducting a pilot survey before the final survey has great benefits for the researcher. One of the advantages is that they help to detect the elements that could cause a negative impact on the research, such as a bad design that causes abandonment or poorly asked questions that confuse the respondent, causing them to answer incorrectly or to simply not give any answer. Avoid these common survey mistakes.
A pilot survey also helps to assess the accuracy of the instructions in advance, to see if all participants understand them.
Pilot surveys also save financial resources, because if errors are found early, there will be less chance of obtaining unreliable results that could harm our business or having to start from scratch after applying the real survey.
In short, the main goal of a pilot survey is to determine whether a large-scale survey is worth conducting.
Disadvantages
When doing a preliminary survey, there is the possibility of making inaccurate predictions or hypotheses based on the data from the pilot test.
Successfully completing a pilot survey is not a guarantee of the success of a large-scale survey. Although the results of a pilot test may offer some index of the response rate of your main survey, this does not guarantee that this will be the case, as there are other factors to take into account.
We invite you to follow these recommendations to increase the response rate of your surveys.
Sample questions for your pilot survey
A pilot survey is an excellent opportunity to evaluate your mistakes and directly know your respondents’ opinions about your questionnaire. That’s why it’s a common practice to include a few questions at the end of your main questionnaire to identify areas for improvement in your survey collection tool.
Here are some examples of questions that you can use; these can be answered by the interviewer or by the person being surveyed:
- How long did it take you to complete the survey? (Enumerator)
- 1-5 minutes
- 5- 10 minutes
- +10 minutes
Knowing the length of your survey and how much time it takes for people to answer them is very important. It will allow you to make adjustments such as adding or removing questions, separating them into segments, or identifying if time is perceived as negative or positive to the participants.
- How difficult was it to answer the questions in that survey?
- Very easy
- Easy
- Neither very easy nor very difficult
- Hard
- Very difficult
This question will help you assess the degree of comfort that people are having when participating in your study and answering your questionnaire. The stress level caused can cause bias in your research, so it is essential to monitor it at an earlier stage.
- Were there any terms or words you did not know or thought were misused?
- No
- Yes
- Which? ________
When designing your survey, using far-fetched or strange words for the target audience is usually a common mistake. The pilot phase is a great time to identify them and replace them with more familiar ones to avoid bias or misinterpretation of the question.
- What did you think of the survey design?
- Very good
- Fine
- Neither very bad nor very good
- Bad
- Very bad
Your survey’s design is fundamental. Knowing the opinion of those who are responding will allow you to validate if it is appropriate or if something needs to be adjusted to improve the response rates and quality of the data collected.
- Did you like answering this survey?
- Yes
- No
With this question, you will be able to know in general terms if the interaction was positive or negative. It can also be helpful to identify possible candidates to participate in another study, a positive perception favors them collaborating again in your surveys in the future.
- Was the applicator precise with the instructions, the purpose of the study, and how they will handle their responses?
- Yes
- No
- Barely
Sometimes the mistakes of a study are not only linked to things like sampling bias or other aspects of its design. Human error can be a damaging agent, so a direct evaluation in your pilot survey will be beneficial to better train the people involved in the data collection, especially when conducting a face-to-face survey.
How to do a pilot survey?
An easy way to do pilot surveys is by applying a test survey to a small group of 5 to 10 people who are representative of your target group. If this group of people is not available to you, you can select a group that is as similar as possible. Even with this small number of people, you should be able to identify the main problems with your survey.
Another good practice is to ask these people to answer the pilot survey out loud to find out what they are thinking about when they are answering it or where they have an obstacle. Take the surveys separately, not in groups.
Participants must answer as if they were answering the real survey, that is, if you are going to carry out an online survey, it is important that you apply it, both on a desktop computer, and test it on some devices such as tablets or cell phones, in order that you can be sure that your mobile surveys are adapted to this type of format.
Another point that can be helpful is to observe them while they answer the survey. Look for the parts where they make mistakes, this is an indicator that the questions and/or the survey design is not clear enough and requires you to make improvements. Take note of everything you observe.
Likewise, if a field survey is to be carried out, where the interviewers are going to ask the questions, you must have the respondents trained to formulate the questions of your survey correctly. I share some survey tips for interviewers that will be useful when carrying out the investigation.
Once the participants in the pilot survey have finished, do an analysis of the observations, your notes, and the results of the survey and make the necessary changes. This is generally all you will need unless you make very significant changes that warrant redoing other pilot surveys before you start doing the real research. If the issues have been resolved, go ahead with your project! Good luck!
Are you ready to conduct a pilot survey before starting your big research project? We can help you. Use QuestionPro and access our many features to obtain and analyze valuable data.
