When using surveys for data collection, you are facing multiple decisions. One of them is to choose a delivery medium. The evolution of surveys has kept up with the development of technology: going from door to door, mailing questionnaires out, calling people to complete phone interviews, using portable computers – and finally online surveys.
Online surveys provide some obvious advantages such as reduced paper and administration costs. However, it makes sense to look a bit deeper into the differences between the two general options, online and offline surveys, especially with respect to the quality of the collected data.
Quality of data
The quality of survey results is made up of the right questions, representativeness of the examined group of interest, and the validity and depth of the answers. Error in your data can occur systematically due to biases, like researcher bias, survey bias, respondent bias and non-response bias, or randomly by misrepresentation.
But it’s not just the accuracy of the data: Especially when surveying feedback on customers or an event experience or gathering new ideas and suggestions for improvement, the quality of free text fields are crucial for usable data. All those aspects are notably influenced by your choice of survey format.
But it’s not just the accuracy of the data: Especially when surveying feedback on customers or an event experience or gathering new ideas and suggestions for improvement, the quality of free text fields are crucial for usable data. All those aspects are notably influenced by your choice of survey format.
Response time and rates
Online surveys most certainly produce quicker responses, since you can share the link to your survey immediately after creating it without having to call people, mail it to them or meet them in person. Offline surveys, on the other hand, usually produce higher response rates with in-person surveys being the most effective, followed by mail surveys. Online and email surveys still have a better response rate than telephone surveys.
The height of response rates doesn’t necessarily translate to representation: If you want to represent the whole population of one country, you’ll need a much lower response rate than when you wish for feedback from a defined group. In a smaller group, you’ll get much smaller sample sizes and it’s much more likely to miss neutral responses and get more positive and negative results.
The height of response rates doesn’t necessarily translate to representation: If you want to represent the whole population of one country, you’ll need a much lower response rate than when you wish for feedback from a defined group. In a smaller group, you’ll get much smaller sample sizes and it’s much more likely to miss neutral responses and get more positive and negative results.
Target group selection
Most of the time, the lower response rates of online surveys can be compensated by being able to reach more people – even worldwide if you wish. With the right software you can run a survey in multiple languages of which the participants can select.
If the survey has a specific target group or needs a certain frequency distribution in order to be representative, quota setting is a solution that can be used in survey software.
A downside to online surveys is that random sampling is more difficult, since generating random email addresses like with phone numbers is not possible. That is not a problem if the group of interest consists of the members of a club, the students of a certain university, or registered customers to a shop. Otherwise panel companies can offer a solution to getting the wished sample size.
The benefits of offline interviews include going to specific places after an event to reach specific people and also reaching the computer-illiterate population. Though the “online population” used to have special characteristics like being younger, better educated and more tech-savvy, the computer literacy has drastically increased in the last 15 years. Online surveys are thus reaching more and more people, whereas phone surveys, being still a popular medium of choice, are reaching less and less people, since households that have no landline are increasing, in particular with younger people.
If the survey has a specific target group or needs a certain frequency distribution in order to be representative, quota setting is a solution that can be used in survey software.
A downside to online surveys is that random sampling is more difficult, since generating random email addresses like with phone numbers is not possible. That is not a problem if the group of interest consists of the members of a club, the students of a certain university, or registered customers to a shop. Otherwise panel companies can offer a solution to getting the wished sample size.
The benefits of offline interviews include going to specific places after an event to reach specific people and also reaching the computer-illiterate population. Though the “online population” used to have special characteristics like being younger, better educated and more tech-savvy, the computer literacy has drastically increased in the last 15 years. Online surveys are thus reaching more and more people, whereas phone surveys, being still a popular medium of choice, are reaching less and less people, since households that have no landline are increasing, in particular with younger people.
(Perceived) anonymity
Especially upon addressing sensitive topics like smoking, drinking, sexual behavior, cheating on taxes or gambling, self-protective answering can be observed. This means the participant is not answering honestly, downplays or exaggerates his or her answer. It occurs especially with an interviewer present and even depends on factors like age, gender and ethnicity.
But also when asking less sensitive questions, and instead about the experience with a product or event, the results might be biased by the presence of the interviewer in in-person interviews, since people may be more polite and less honest. This also applies to situations with an existing relationship of dependence, as often observed at universities, where the professor to be rated is present at the evaluation.
Oftentimes people mistrust online surveys, when they are unsure of their data safety. Particularly if you want to prevent multiple participation, you either have to select setting cookies, saving the IP address or only allow access with a personalized token. All those options might prevent the participants from answering upfront or from answering at all, if you neglect assuring the participants of their anonymity and also choosing a software tool that allows anonymization as needed.
This not only concerns the data accessible to the survey creators but also to the software provider. This is why the hosting and data storage should be at best on servers within countries with high data protection legislation like Iceland, Germany, the Netherlands or Spain.
But also when asking less sensitive questions, and instead about the experience with a product or event, the results might be biased by the presence of the interviewer in in-person interviews, since people may be more polite and less honest. This also applies to situations with an existing relationship of dependence, as often observed at universities, where the professor to be rated is present at the evaluation.
Oftentimes people mistrust online surveys, when they are unsure of their data safety. Particularly if you want to prevent multiple participation, you either have to select setting cookies, saving the IP address or only allow access with a personalized token. All those options might prevent the participants from answering upfront or from answering at all, if you neglect assuring the participants of their anonymity and also choosing a software tool that allows anonymization as needed.
This not only concerns the data accessible to the survey creators but also to the software provider. This is why the hosting and data storage should be at best on servers within countries with high data protection legislation like Iceland, Germany, the Netherlands or Spain.
Other technical advantages
Although the readability of black ink on white paper goes unchallenged, online surveys still win the victory in the design category. By not being restricted in space, it is not only possible to create a more appealing appearance with more coloring and implementation of pictures and videos, but also to provide unlimited space for free text entries.
To keep participants interested, online surveys allow many options for personalization, including branching to only show relevant questions, piping to reuse answers or defining different end URLs.
Though readability might be better in paper surveys, online surveys grant access to people with disabilities such as poor eyesight by being compatible with screen readers. On the researcher’s side deciphering handwriting is being skipped and less errors in data entry occur with online surveys.
To keep participants interested, online surveys allow many options for personalization, including branching to only show relevant questions, piping to reuse answers or defining different end URLs.
Though readability might be better in paper surveys, online surveys grant access to people with disabilities such as poor eyesight by being compatible with screen readers. On the researcher’s side deciphering handwriting is being skipped and less errors in data entry occur with online surveys.
Interaction
A special benefit of personal interviews arises when you want feedback to an event taking place or to ask for customer feedback immediately after the experience: Since the experience (and the excitement at best) is still fresh in their mind, the motivation to fill in a questionnaire might be higher, particularly in regards to open text fields.
The interviewer bias, that has already been mentioned, can be utilized in a good way: From my own experience, a face to face interaction can be beneficial when you are looking for answers in form of free text fields. I learned that people often do not think of their opinion or thoughts as relevant, and so they’d rather skip a question than entering something they deem wrong.
In personal interviews you can encourage those to share their experiences. If required, you can give further explanations or ask further questions when fitting.
Although offline surveys immediately after the examined experience make a lot of sense, online surveys are still superior when it comes to time management. People are often reluctant to answering at all or fill out their questionnaire as quickly as possible if they feel pressured.
Did you know? When creating questionnaires with LimeSurvey, you can have respondents take your questionnaire offline with no need for internet availability. The free Android app OfflineSurveys enables caching any LimeSurvey survey and collecting your data offline. Once the Internet is available again, the app can automatically synchronise the collected responses with your LimeSurvey system. Feel free to check it out here.
The interviewer bias, that has already been mentioned, can be utilized in a good way: From my own experience, a face to face interaction can be beneficial when you are looking for answers in form of free text fields. I learned that people often do not think of their opinion or thoughts as relevant, and so they’d rather skip a question than entering something they deem wrong.
In personal interviews you can encourage those to share their experiences. If required, you can give further explanations or ask further questions when fitting.
Although offline surveys immediately after the examined experience make a lot of sense, online surveys are still superior when it comes to time management. People are often reluctant to answering at all or fill out their questionnaire as quickly as possible if they feel pressured.
Did you know? When creating questionnaires with LimeSurvey, you can have respondents take your questionnaire offline with no need for internet availability. The free Android app OfflineSurveys enables caching any LimeSurvey survey and collecting your data offline. Once the Internet is available again, the app can automatically synchronise the collected responses with your LimeSurvey system. Feel free to check it out here.
Conclusion
The question whether to choose an online or offline survey cannot be answered conclusively – and it does not need to be. The choice has to be made based on the particular cause depending on the survey goal and type, employee or customer feedback, academic research or feedback to an event.
In many cases a combination of both might make the most sense, because that way you can also combine the benefits of both: broadly speaking, the quantity of online surveys with the quality of paper surveys and the combined accessibility of different target groups.
LimeSurvey provides a convenient way to combine both: questionnaires that have been created online can be easily converted into a word document or a PDF file. The question display is thereby adjusted in order to be used in a written or personal interview. Afterwards the collected data can be entered manually into LimeSurvey, so that in the end both data sets can be analyzed collectively. All aforementioned features like multiple languages, data security, choosing the degree of anonymity and personalization of the survey design are also provided by LimeSurvey.
In many cases a combination of both might make the most sense, because that way you can also combine the benefits of both: broadly speaking, the quantity of online surveys with the quality of paper surveys and the combined accessibility of different target groups.
LimeSurvey provides a convenient way to combine both: questionnaires that have been created online can be easily converted into a word document or a PDF file. The question display is thereby adjusted in order to be used in a written or personal interview. Afterwards the collected data can be entered manually into LimeSurvey, so that in the end both data sets can be analyzed collectively. All aforementioned features like multiple languages, data security, choosing the degree of anonymity and personalization of the survey design are also provided by LimeSurvey.
