- Posts: 48
- Thank you received: 2
Welcome to the LimeSurvey Community Forum
Ask the community, share ideas, and connect with other LimeSurvey users!
Three-way interlocking quotas, 204 groups
- modernity4r
- Topic Author
- Offline
- Senior Member
Your LimeSurvey version: 6.3.9+
Own server or LimeSurvey hosting: Both
Survey theme/template: Fruity
==================Survey Design Issue with 204 groups (quotas?)I need to conduct a survey that connects age groups (6 groups), gender (2 groups), and regions (17 groups), targeting 2,040 respondents. This results in 204 distinct response groups.This scenario can be understood as a kind of three-way interlocking quota, where age, gender, and region are interlinked to form complex quota groups. It seems almost absurd to create 204 quota groups in quotas menu; setting up so many quotas appears practically impossible. Faced with the reality of having to stratify a sample of 2,040 respondents into 204 groups in a three-way table, I am unsure how to solve this problem.Conducting an oversized survey with more than 2,040 respondents and then readjusting it to a scale of 2,040 seems likely to be rejected by the committee.Furthermore, considering separate quotas for age groups, gender, and regions is not an option, as this approach could lead to over- or under-representation in some response groups.I am considering using the Expression Manager with functions like
if()
sum()
{sum(if(Agegroup=="AO01", if(Gender=="AO01", if(Region=="AO01", 1, 0), 0), 0))}
{sum(if(Agegroup=="AO01", if(Gender=="AO02", if(Region=="AO01", 1, 0), 0), 0))}
...
{sum(if(Agegroup=="AO01", if(Gender=="AO01", if(Region=="AO02", 1, 0), 0), 0))}
{sum(if(Agegroup=="AO01", if(Gender=="AO02", if(Region=="AO02", 1, 0), 0), 0))}
...
{sum(if(Agegroup=="AO06", if(Gender=="AO01", if(Region=="AO17", 1, 0), 0), 0))}
{sum(if(Agegroup=="AO06", if(Gender=="AO02", if(Region=="AO17", 1, 0), 0), 0))}
Thank you in advance.
Please Log in to join the conversation.
- Joffm
- Offline
- LimeSurvey Community Team
- Posts: 12940
- Thank you received: 3979
Why? How long does it take to set one quota?It seems almost absurd to create 204 quota groups in quotas menu; setting up so many quotas appears practically impossible.
I should say: Less than a minute.
So it will take only about three hours.
To be honest: I saw such things here already.
But now to your expressions.
Yes if you do it this way, there will be 204 equations.
You can shorten this by creating a question of type "multiple short text" (Q1) with 294 subquestions and in only one equation you assign the value to the subquestions, like
{Q1_SQ001=sum(if(Agegroup=="AO01", if(Gender=="AO01", if(Region=="AO01", 1, 0), 0), 0))}
{Q1_SQ002=sum(if(Agegroup=="AO01", if(Gender=="AO02", if(Region=="AO01", 1, 0), 0), 0))}
BTW: Why the "sum()" function. This is only one nested IF.
But I do not see the goal of this approach.
Now my idea.
I'll never understand why people do not use numerical codes.
It is simple arithmetic.
One simple equation (eqCombi)
{34*(QAge-1)+17*(QSex-1)+QRegion} will return a unique number (1-204) for each combination
In a next group you may use the function "statCountIF" to check the number of responses with that special combination code
If there are already 10, set an equation (eqQuota) to 1 and screen out.
like
{if(statCountIf(eqCombi.sgqa, eqCombi))>10,1,0)}
Now you set one single quota on "eqQuota"
Joffm
Volunteers are not paid.
Not because they are worthless, but because they are priceless
Please Log in to join the conversation.
- holch
- Offline
- LimeSurvey Community Team
- Posts: 11660
- Thank you received: 2742
I find the idea of interlocking quotas based on 3 different variables quite absurd, You will have about n=10 people in each of the 204. Do you really expect to be able to analyse any differences between the 204 quota groups? Probably not, given that each quota group has only n=10 respondents in it. I don't think you would be able to calculate any significant differences between
Male - Age group 1 - Region 1 (n=10) and
Male - Age group 2 - Region 2 (n=10)
Conducting an oversized survey with more than 2,040 respondents and then readjusting it to a scale of 2,040 seems likely to be rejected by the committee.
The committee should be aware, that with all these quotas, you WILL run a study with far more people. At least you will need to invite a lot more people, and you will have a LOT of screen outs because of very small and very specific quotas. At the end of the day, based on probability, the number of people that you need to invite to get the result is more or less the same in both scenarios. Probably the same amount of people will have to start the survey and answer the questions about gender, age and region.
At the end, probably not much of a difference. But I would highly recommend to "untangle those quotas". With the sample size you will probably get a pretty good distribution overall anyway and with n=10 in each quota group you can't do indepth analysis based on the quota groups anyway. So not worth the effort, in my opinion.
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.
Please Log in to join the conversation.
- sens5900
- Offline
- New Member
- Posts: 2
- Thank you received: 0
I am modernity4r, the person who posted this message. I am encountering continuous errors during the reply process, so I am posting this using my sens5900 account, which I use for educational purposes.I find the idea of interlocking quotas based on 3 different variables quite absurd, You will have about n=10 people in each of the 204. Do you really expect to be able to analyse any differences between the 204 quota groups? Probably not, given that each quota group has only n=10 respondents in it. I don't think you would be able to calculate any significant differences between
Male - Age group 1 - Region 1 (n=10) and
Male - Age group 2 - Region 2 (n=10)
Each of the 204 groups has a different number of respondents, meaning they have different quotas. My previous statement that I need to survey 2,040 people led to a misunderstanding that I have to survey 10 people in each of the 204 groups. However, the total size of the survey is 2,040, but the size of respondents in each group varies. This is because there are significant differences in population by age group and region.
In fact, designing quotas for 204 groups based on age group, gender, and region is a common requirement for quality assurance of sample extraction in national-scale political surveys in a country. Of course, having 204 quotas would mean screening out many respondents, which incurs significant costs. Therefore, it's common to apply weight adjustments within a certain range.
Please Log in to join the conversation.
- Joffm
- Offline
- LimeSurvey Community Team
- Posts: 12940
- Thank you received: 3979
So there are 204 different quotas.Each of the 204 groups has a different number of respondents, meaning they have different quotas.
On the other side:
Do you pay your respondents?
If not:
Set a quota - as I showed before - to the maximum limit.
Later in your data it is easy to remove all individual overquotas.
Joffm
Volunteers are not paid.
Not because they are worthless, but because they are priceless
Please Log in to join the conversation.
- holch
- Offline
- LimeSurvey Community Team
- Posts: 11660
- Thank you received: 2742
My previous statement that I need to survey 2,040 people led to a misunderstanding that I have to survey 10 people in each of the 204 groups.
Yes, this lead me to this conclusion. But having different sizes (this means some of these quotas will be even lower than n=10, while others will be higher), this aggravates the situation even more. I don't see how so small quota groups will be good for the quality of the survey. I'd rather have the same sized groups and then weigh data based on at least n=10 (which is already absurdly low and I wouldn't trust to make predictions on quota groups of n=10, but this is what you do when you put weights to it. You take this small sample and assume that they are representative for this target group within the population. However, given the small sample size of 10, there is a good chance that your sample is way off of the real universe of this target group. Let's say you have an extremist party in your country and usually only 1% of the population would vote for them, but by chance among your n=10 or maybe even less for a specific quota group you have interviewed one that would vote for this party. Suddenly you would assume that 10% of this target group will vote for the extremist party. Now imagine if you are unlucky and in that specific target group you interview 2 of those?
To be honest, this sounds like a very dangerous process to political research.
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.
Please Log in to join the conversation.
- sens5900
- Offline
- New Member
- Posts: 2
- Thank you received: 0
Please Log in to join the conversation.
- holch
- Offline
- LimeSurvey Community Team
- Posts: 11660
- Thank you received: 2742
You could think of creating a couple of quota and then have a look at the .LSS file and continue creating the rest within the LSS file. You can at least give it a try and see if it speeds up the process. But all in all, it can't take much more than a couple of hours to prepare those 204 quota. Testing them will be the fun part though...
Regarding the methodological position on specifying detailed quotas, I have a slightly different view from holch.
Which is totally fine. I just wanted to highlight, in case you were planning on analyzing by comparing responses from these 204 different quota groups, you might run into big issues. If you are not weighting based on these tiny sample portions, you should be fine. But I would highly advise against weighing based on quota groups with the size you are creating. Because you are saying that some of the quota groups are bigger than n=10. Good. But that also means that other quota groups must be even smaller than n=10. I wouldn't even think about weighting data based on samples smaller than n=50, better n=100+. But all my comments are just a side note. I know that this was not the question and you are more interested in the technical feasibility. But I just feel "obliged" to highlight these points. What you make out of it is up to you of course.
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.
Please Log in to join the conversation.