Counterbalanced design--help!

5 years 2 months ago #77215 by HCI_Guy
HCI_Guy created the topic: Counterbalanced design--help!
Hello again!

Let's say that I have a 1x2 design, with two questions, "1" and "2".

I want to counterbalance my design--in other words, I need to have 2 "forms" of my survey: one form that goes 1-2, and one that goes 2-1.

Is there any way to automate this, and record which form a particular user is in? Also, is there a way to round-robin form assignment?

I ask because the alternative is to bang out 36 different forms (18 questions each), and then create some (more) bolt-on code to handle the round-robin assignment.


Please Log in to join the conversation.

5 years 2 months ago - 5 years 2 months ago #77242 by tpartner
tpartner replied the topic: Counterbalanced design--help!
You should be able to handle the 1-2/2-1 problem as follows:

1) Generate a random number between 1 and 2 and load it into a hidden question (You can do this with Expression Manager in 1.92 or this workaround in 1.91)

2) Create a copy of Q1 and place it after Q2 (we'll call it Q1-2)

3) Use conditions or relevance to:
- Show Q1 and hide Q1-2 IF the hidden question = 1
- Hide Q1 and show Q1-2 IF the hidden question = 2

This will effectively randomize the display order of Q1 and Q2.

I'm afraid I don't understand the reference to round-robin and 18 forms.

Tony Partner
Solutions, code and workarounds presented in these forums are given without any warranty, implied or otherwise.
Last Edit: 5 years 2 months ago by tpartner.

Please Log in to join the conversation.

5 years 2 months ago - 5 years 2 months ago #77253 by HCI_Guy
HCI_Guy replied the topic: Counterbalanced design--help!
I ask two questions, which I'll sum up here:
1. Can Limesurvey generate multiple survey forms (versions) from a single survey?
2. Can Limesurvey be coerced into automatically assigning participants to one of these surveys based upon completion statistics for the different survey forms (versions)?

Thank you for your response, but as we increase complexity, this approach becomes unwieldy. This is because, for chains longer than two elements, you'll have to track questions that have come before as well as those that will come after your random number generation.

Counterbalanced design eliminates the impact of unwanted effects (e.g. training, etc.). Here's a quick primer , with some great graphics to help visualize what I'm talking about.

First question:
Consider a 3x3 design: there would be 9 individual "cells" to fill. The questions could be arranged in any of the following orders, and a counterbalanced design would have us use all of these:
Cell 1: 1-2-3-4-5-6-7-8-9
Cell 2: 1-3-4-5-6-7-8-9-2
Cell 3: 1-4-5-6-7-8-9-2-3
Cell 9: 9-8-7-6-5-4-3-2-1

Second question:
With respect to round-robin assignment: if I have something like this:

Cell 1: 8 people
Cell 2: 2 people
Cell 3: 0 people
Cell 4: 0 people
Cell 9: 1 person

Then I cannot use the data, as there might be something in the way that questions are ordered in cells 3 and 4 that throw the entire experiment off.

What I am looking for--or might have to build--is a "front-end" that could see someone coming in to the survey, "look" at the cell counts, and say, "aha! I'll give you the version of this described by cell 3!" (question order 1-4-5-6-7-8-9-2-3). It doesn't have to be exact (e.g. I'm okay with not looking up entries until after a survey has been 100% completed), but it should still work to shunt people into an approximately-appropriate survey form.

It feels like this has become a more open question, and I fear I might be writing some more software (tell me it ain't so!). :blink:
Last Edit: 5 years 2 months ago by HCI_Guy.

Please Log in to join the conversation.

5 years 2 months ago #77268 by TMSWhite
TMSWhite replied the topic: Counterbalanced design--help!
The round robin assignment sounds like an interesting idea for the Central Participants Database. There is already a quota feature which does cell counts within a survey, but is used to exclude people. However, perhaps it could be adapted / extended for round robin.

For counter-balanced or Latin-square ordering of questions, there might be an easy temporary solution:
1. First design your survey in Cell 1 order.
2. Export the survey structure as Excel .
3. Save the Excel as a tab separated value (TSV) file.
4. Write a perl (or similar) script to read that TSV file and shuffle the questions into each of the counterbalanced orders, saving it with a unique filename that makes the counterbalanced type clear.
5. Open those TSV files with Excel and save them as .xls
6. Import each of your 9 surveys.

Then, you could do your round robin assignment based upon completion counts of those surveys.
The following user(s) said Thank You: HCI_Guy

Please Log in to join the conversation.

5 years 2 months ago #77276 by HCI_Guy
HCI_Guy replied the topic: Counterbalanced design--help!

Good suggestion; I knew there was a faster way than within the UI itself to do this!

As for the round-robin assignment, I'll start looking at the quota feature to figure out if this is something I can adapt or not.


Please Log in to join the conversation.

Start now!

Just create your account and start using Limesurvey today.

Register now