Beginners Guide to LimeSurvey Part 2: What counts as a response in LimeSurvey Cloud

English

Details: Category: Blog; 22 July 2020

3 years ago

How are responses counted in LimeSurvey Cloud? How you can minimize survey break-off? What can you do if you are running short of responses? How many responses do you need for a certain net sample? Check out this blog post and learn how to get as many complete questionnaires out of your package as possible. Watch out for our pro tips! Responses and how they are counted in LimeSurvey Cloud In LimeSurvey Cloud the term "response" basically translates to completed questionnaire. In the best case, a respondent starts your questionnaire and completes it until the end page. Each finished questionnaire counts as one response and lowers your remaining responses available in your LimeSurvey Cloud package by one point. But in web surveys – like in all self-administered survey modes – we frequently see respondents who start a questionnaire but break off participation during the survey-taking process. Dropout that happens during survey-taking counts as half a response. In LimeSurvey one response can be used for really extensive questionnaires. However the longer the questionnaire, the higher the risk of incompletes. Many dropouts take place on the first page of a survey – for example, when participants find out that the topic is not relev.ant to them. That is when the welcome screen should be used. The welcome screen is a special page where you can describe your research and show your survey data policy. Once enabled, dropouts on the welcome screen of a survey won’t lower your response balance Pro Tip 1: Consider using the welcome screen with your survey as break-off on that page does not lower your available response contingent. Besides regular survey participation, manually editing the entries of the survey response table counts as response. If you create a new entry directly in the survey response table – e.g., if you do a mixed-mode study – it counts as one response. Also if you change the status of a survey from partially completed to completed, your response balance will be lowered by one response instead of half a response. What you can do to lower break-off Break-off is inevitable in surveys. But you can keep it as low as possible when considering the following: - Make sure you target a relevant audience. - Make your survey not too long. How long? Well, actually that depends on your audience and the survey’s topic. In any case be honest about the survey’s topics and its length in the invitation, as well as on the welcome page. - Show that you are a serious survey researcher and that you respect data protection and privacy. Use LimeSurvey's built-in feature to display your data policy on the welcome page. - Offer an incentive – some kind of reward for your respondents – for complete participation. Incentives are either an per capita incentive (everyone gets it), the participation in a lottery drawing (e.g., of money or a voucher), or indirect motivation (like the donation to charity or that respondents get access to certain information). - Follow the general guidelines and rules for questionnaire design and for good survey practice (e.g., question wording, response scales, task difficulty, etc.). - Test your survey logic and question relevance. - Make mandatory as few questions as possible. - Take advantage of LimeSurvey’s feature to save a survey and resume it later. - If you have the possibility, use email not only for invitations but also for reminders (we recommend up to two reminders). - Before you field your actual survey, do a pretest – e.g., with colleagues – to check if there are any problems. Consider paradata like response times to detect difficult questions. Pro Tip 2: Thoroughly design your survey following established guidelines and test it with a small sample in advance. This will help to get higher completion rates and higher data quality. Tracking your responses in LimeSurvey Cloud The most obvious difference between the LimeSurvey Cloud packages is how many questionnaires (aka responses) can be completed. The number of responses available in LimeSurvey Cloud starts from 25 per month (Free package) and for regular packages up to 1.000.000 responses per year (Corporate package) – which can be even topped with individual offers. At the end of each subscription period unused responses expire and the response limit is reset to the start value. There are two places where you can find your current responses balance. In your survey installation (also called survey instance) you see your available responses in the top menu. The other place is inside your LimeSurvey profile in the menu survey application. If you choose edit in that menu, you can set a limit for remaining responses. As soon as your remaining responses fall below this limit, you will receive an alert email. Running out of responses? Read our following tips Sometimes more people participate in your survey than you expect or high break-off rates lower your response balance. Before you run out of responses you will receive an alert email, but as soon as your response balance hits zero, people who try to access your survey will get an error message. So what can you do if you run out of responses? 1) You can either wait until your subscription renews or 2) You can renew your subscription early - just log in on our platform, access the survey application menu, and renew your package earlier..3) If you exceed your responses not only once in a while, upgrading to a higher package is the recommended solution. Pro Tip 3: For more flexibility, you should opt for a yearly subscription. We also offer multi-year packages for companies - just contact our sales team. Estimation of response volume To avoid running short of responses during your survey, we recommend you do a rough estimation of the responses you need in advance. Let’s think it from the end: Given the case you want to analyze a net sample of 500 complete questionnaires. However, frequently not all complete questionnaires are really suited for data analysis. When checking the answers for missing values, strange response patterns (e.g., always selecting the first response), illogical answers, response duration (if a participant is too fast, this can be an indicator for low seriousness), you may have to remove 10% of the cases. So you need about 550 completes. If the break-off rate of your survey is around 30%, you need to invite another 240 persons (120 responses) to fill it out. Moreover, you should also plan for your pretest another 30 responses. Overall, you need approximately 700 responses to end with a net sample of 500 cases. Please keep in mind that the numbers in this example may greatly vary! Especially the dropout rate which can be considerably lower or higher. Pro Tip 4: Estimate the responses you need in advance – but always monitor your response numbers while your survey is online. Further reading You can find additional information in the online manual: https://manual.limesurvey.org/Responses_&_statistics https://manual.limesurvey.org/Survey_quotas Ready for Part 3? Now that you are really into our "Beginners Guide" Series, you should take a look into our last blog post from that series. Part 3 but if you haven´t read part 1, then you should go ahead and read that first. Part 1

How are responses counted in LimeSurvey Cloud? How you can minimize survey break-off? What can you do if you are running short of responses? How many responses do you need for a certain net sample? Check out this blog post and learn how to get as many complete questionnaires out of your package as possible. Watch out for our pro tips!

Responses and how they are counted in LimeSurvey Cloud

In LimeSurvey Cloud the term "response" basically translates to completed questionnaire. In the best case, a respondent starts your questionnaire and completes it until the end page. Each finished questionnaire counts as one response and lowers your remaining responses available in your LimeSurvey Cloud package by one point.

But in web surveys – like in all self-administered survey modes – we frequently see respondents who start a questionnaire but break off participation during the survey-taking process. Dropout that happens during survey-taking counts as half a response. In LimeSurvey one response can be used for really extensive questionnaires. However the longer the questionnaire, the higher the risk of incompletes.

Many dropouts take place on the first page of a survey – for example, when participants find out that the topic is not relev.ant to them. That is when the welcome screen should be used. The welcome screen is a special page where you can describe your research and show your survey data policy. Once enabled, dropouts on the welcome screen of a survey won’t lower your response balance

Pro Tip 1: Consider using the welcome screen with your survey as break-off on that page does not lower your available response contingent.

Besides regular survey participation, manually editing the entries of the survey response table counts as response. If you create a new entry directly in the survey response table – e.g., if you do a mixed-mode study – it counts as one response. Also if you change the status of a survey from partially completed to completed, your response balance will be lowered by one response instead of half a response.

What you can do to lower break-off

Break-off is inevitable in surveys. But you can keep it as low as possible when considering the following:

- Make sure you target a relevant audience.

- Make your survey not too long. How long? Well, actually that depends on your audience and the survey’s topic. In any case be honest about the survey’s topics and its length in the invitation, as well as on the welcome page.

- Show that you are a serious survey researcher and that you respect data protection and privacy. Use LimeSurvey's built-in feature to display your data policy on the welcome page.

- Offer an incentive – some kind of reward for your respondents – for complete participation. Incentives are either an per capita incentive (everyone gets it), the participation in a lottery drawing (e.g., of money or a voucher), or indirect motivation (like the donation to charity or that respondents get access to certain information).

- Follow the general guidelines and rules for questionnaire design and for good survey practice (e.g., question wording, response scales, task difficulty, etc.).

- Test your survey logic and question relevance.

- Make mandatory as few questions as possible.

- Take advantage of LimeSurvey’s feature to save a survey and resume it later.

- If you have the possibility, use email not only for invitations but also for reminders (we recommend up to two reminders).

- Before you field your actual survey, do a pretest – e.g., with colleagues – to check if there are any problems. Consider paradata like response times to detect difficult questions.

Pro Tip 2: Thoroughly design your survey following established guidelines and test it with a small sample in advance. This will help to get higher completion rates and higher data quality.

Tracking your responses in LimeSurvey Cloud

The most obvious difference between the LimeSurvey Cloud packages is how many questionnaires (aka responses) can be completed. The number of responses available in LimeSurvey Cloud starts from 25 per month (Free package) and for regular packages up to 1.000.000 responses per year (Corporate package) – which can be even topped with individual offers. At the end of each subscription period unused responses expire and the response limit is reset to the start value.

There are two places where you can find your current responses balance. In your survey installation (also called survey instance) you see your available responses in the top menu. The other place is inside your LimeSurvey profile in the menu survey application. If you choose edit in that menu, you can set a limit for remaining responses. As soon as your remaining responses fall below this limit, you will receive an alert email.

Running out of responses? Read our following tips

Sometimes more people participate in your survey than you expect or high break-off rates lower your response balance. Before you run out of responses you will receive an alert email, but as soon as your response balance hits zero, people who try to access your survey will get an error message.

So what can you do if you run out of responses?

1) You can either wait until your subscription renews or 2) You can renew your subscription early - just log in on our platform, access the survey application menu, and renew your package earlier..3) If you exceed your responses not only once in a while, upgrading to a higher package is the recommended solution.

Pro Tip 3: For more flexibility, you should opt for a yearly subscription. We also offer multi-year packages for companies - just contact our sales team.

Estimation of response volume

To avoid running short of responses during your survey, we recommend you do a rough estimation of the responses you need in advance. Let’s think it from the end: Given the case you want to analyze a net sample of 500 complete questionnaires. However, frequently not all complete questionnaires are really suited for data analysis. When checking the answers for missing values, strange response patterns (e.g., always selecting the first response), illogical answers, response duration (if a participant is too fast, this can be an indicator for low seriousness), you may have to remove 10% of the cases. So you need about 550 completes. If the break-off rate of your survey is around 30%, you need to invite another 240 persons (120 responses) to fill it out. Moreover, you should also plan for your pretest another 30 responses. Overall, you need approximately 700 responses to end with a net sample of 500 cases. Please keep in mind that the numbers in this example may greatly vary! Especially the dropout rate which can be considerably lower or higher.

Pro Tip 4: Estimate the responses you need in advance – but always monitor your response numbers while your survey is online.

Ready for Part 3?

Now that you are really into our "Beginners Guide" Series, you should take a look into our last blog post from that series.

Part 3

but if you haven´t read part 1, then you should go ahead and read that first.

Part 1

To download the RC2 of LimeSurvey 4, click here. You can also access the live RC2 release of LimeSurvey 4 here. The new question editor panel A new question editor has been implemented (based on CKEditor 5 and VueJS 2). A script editor has also been added to help users increase the complexity of their surveys by entering JavaScript, HTML or CSS code. In terms of question attributes, you will see only the general ones next to the question and help text boxes. All the other question attributes have been moved below the text boxes, being located next to the subquestion and answer option menus. Together with the live preview feature, users can now see how the edits impact the question design live. Global & group survey settings In LimeSurvey 4 you can edit the survey settings at the global and group level. The participant and publication & access settings, presentation information, and notifications can be easily set up from the global and survey group menu: The inheritance level works in the following way: Global settings -> Survey group settings -> Survey settings Any change done in one of the upper levels will immediately be inherited by the survey as long as the inheritance attribute is activated. New file management system The panel for resource management has been replaced with a file management system that provides a complete overview of the files and folders from your instance. The ability to copy and move files from one place to another has also been implemented. Data encryption Data encryption is a new feature provided in LimeSurvey 4. It has been created to help survey administrators better cope with the new data-protection regulations adopted in different parts of the world (e.g., GDPR or HIPAA). This new capability permits the encryption of personal data located in the central participant database and/or survey participants table. Moreover, you can also encrypt the data you collect from your respondents if the encryption question attribute is enabled. User management panel and user roles The user management panel has been completely redone offering you the possibility to add dummy users, as well as importing/exporting users from other LimeSurvey instances. However, the biggest news is the appearance of roles – now you can simply create roles within your organisations (e.g., students, professors, research), and easily provide user groups the same role, which in turn assigns to each user group the same set of global permissions. What is missing? At the moment, the “question themes” feature is being integrated into the GUI. It will be much easier to add customized questions into your instance - you will be no longer limited to the default question types. In the meantime, we are also working at the development of our LimeStore - a market for plugins, themes, and question themes created by our community members. For more information, access the LimeStore page. Testing and testing! We are getting closer to the release of the first stable version of LimeSurvey 4. To fasten the process, join us in testing RC2 and report the potential issues on our bugs tracker. If you have questions about LimeSurvey 4 or you just simply want to show the gratitude to the community and developers who contributed to this release, join our forum discussion topic here.

Blog

4 years ago

LimeSurvey Authorized Partners - Get Your Lime Job Done

Did you know that you can hire a professional for survey creation or customization? Or for support? Or to generate autom ...

LimeSurvey GmbH is a growing organization that governs the development of the LimeSurvey software. Two types of packages are provided on our platform: LimeSurvey Cloud and LimeSurvey ComfortUpdate. The former provides users a hosted software solution supported by us, while the latter comes with ComfortUpdate, premium plugins, survey templates, and themes. To better understand the difference between LimeSurvey Cloud and LimeSurvey ComfortUpdate, check out our blog post on different hosting solutions for LimeSurvey. However, the growth of our user base increases the number of special requests we receive that cannot always be fulfilled by us. Therefore, we have actively started to expand the services we provide via the creation of a group of service companies that provide a wide range of optional commercial services. These services may include individual support, consulting or training, installation and maintenance of LimeSurvey on your own server, integration of LimeSurvey in your software environment, custom code development, survey creation, and the design of individual themes. Who Are Our Partners? The partners network consist of individual and organizations located all over the globe: Africa, Australia, Asia, Europe, North and South America. Services & Support The services and support provided by our partners cover the following areas: Hosting: If case self-hosting is not for you, and LimeSurvey Cloud does not meet your demands, our partners can accommodate any time your individual LimeSurvey installation. Support: It covers many areas of LimeSurvey such as consulting services, and plugins. To better understand what each partner provides, visit their personal website. Training: While our manual and forum activity is high, sometimes you just need to find out about tricks and workarounds from professionals. That gives you the possibility to outperform your competition when it comes to survey implementation and analysis. Installation: Sometimes, you just wish to self-host your instance on the hosting of your organization. However, due to the existing server configuration of your organization, the installation process of LimeSurvey may need some additional tweaks. Moreover, you may also have to configure it properly in case you work with confidential data. Therefore, the possibility of hiring a professional for configuring web servers should definitely be taken into account. Integration: Automatizing workflows or the integration of LimeSurvey in the platforms used by your organization is a must-have for increased efficiency. After all, the usage of API and scripts may not be for everyone. Customization: LimeSurvey is a flexible software, feature acknowledged by everyone. With direct access to LimeSurvey developers, our partners are always informed about the development of the software, as well as about the features that will be implemented in the upcoming releases. Survey creation: Time is a scarce commodity. Not everyone has time to go through the manual and forum, and learn how to construct a survey. As a result, outsourcing this task may allow you to concentrate on more urgent things. Theme design: An eye-catching survey heavily increases the rate of completeness for your survey. Our professionals are always ready to meticulously work on the survey theme to provide you the best design possible or to create a layout in line with your corporate identity. Finding the Right Partner Now that you have an overview of the different services that can be contracted, you can start looking for the right partner for your organization. To do that, check first the list of partners list of partners from limesurvey.com, and explore the links on the left menu to find your matching partner for the service and country you are looking for. If a partner lives in the same country as you, (s)he will be displayed as featured partner on the same web page, being displayed at the top. As in any other Open Source project, the community continues to represent the most important part of LimeSurvey. However, with the growth of the project and its user base, many requests need to be fulfilled in a timely manner. Therefore, users seeking quick professional help can contact anytime LimeSurvey professionals who are eager to satisfy their project requirements anytime. We will introduce our partners on our blog in the upcoming weeks. In the meantime, you may start contacting them directly or you can write us via the limesurvey.org platform support tracker.

Blog

4 years ago

Workshop report 5/2019 - What's up with LimeSurvey 4

We are constantly improving LimeSurvey, that is why we are working on a new edition that will come up with a new redesig ...

The new question editor Starting with LimeSurvey Developer Edition 4.0.0beta1, a new question editor will be provided. It has been completely redesigned, as it was split up and not very easy to use. The new system provides a live-preview mode to help users easily edit questions, as well as subquestions and answer option in the same window. The new design is based on VueJS2 and CKEditor 5. We have also added a script field to the question editing, where you can add scripts that should be run with the question. It is automatically wrapped in the necessary on:ready functions. New text- and source editor The popup editor has been removed. We have added in exchange a pure source mode based on the ACE-editor, the same one used in our theme editor. Now it is easier to work with the source code of your questions, subquestions, question groups, and any other text edition fields in LimeSurvey. The source mode can be set to the default mode in the personal options. Global survey options With LimeSurvey 4 you are able to change your global survey options. The global options will be inherited by any survey in the system. Moreover, you can see the inherited value in the overviews. This goes the same way as the design theme options introduced in LimeSurvey 3: Global survey settings -> Survey group settings -> Survey settings Other notable changes The most important changes are hidden from sight. The database has been restructured, languages are now fully normalized, providing us the opportunity to build the foundations of a limitless survey software.The first steps to a fully modularized survey rendering are also done with this version. The question types now consist of a data renderer, a view renderer, and a twig package, making it easy to correct and fix question-related issues. We’ve added a few other extra options, like question-based encryption settings, as well as soft-mandatory question attributes. Future development We are currently removing papercuts in the newly created functionalities and recreating the functions that were available before the changes. We will also add in the upcoming weeks a way to upload and use custom question themes in LimeSurvey 4.Some parts are still not working perfectly and we are in heavy testing to find all possible edge cases. If you wish to help us, please download the beta release, test it, and report all the issues found on our bugs tracker. Join our forum discussion about the new release and stay tuned for more information about the development of LimeSurvey 4!