It's not so much that the admins would pick up the bug; it'd be the other users, who do much the same kind of stuff as I do.

Thanks for the release history. I've looked through it and there doesn't seem to be anything to do with randomisation that was subsequently fixed.

I guess this is starting to look like a statistical fluke, rather than a bug!

@Jelo: Wasn't too bad and I let an iMacros macro do a few clicks as well...

@Lodore: Would be interesting to see the total structure of the survey. Maybe there is something else that influences it. And: Are those responses real responses or tests that you do?

@ Holch: The responses are real, if you're asking are they done by actual people. I can't, because of research ethics, circulate the actual results. However, I've attached the survey logic file here if that's any good (PDF too big to attach, so used jpeg instead).

I also looked at the pattern of randomisation, and it looks, well, random. As in, it's not like there's three participants assigned to condition one and then the fourth always assigned to condition two; instead, it jumps all over the place like you'd expect. Maybe it really is just chance!

I was asking about the structure, because when you do research with real people, there could be other reasons and obviously you are showing different things. Let's assume that one path is longer, more tedious, the questions are more annoying, the text is more complicated or something you might have more drop outs. But of course this wouldn't explain if you do an analysis over all started interviews.

I see what you mean now. But, the two conditions are equally tedious (and man, are they tedious); in any event, the replies I'm talking about are all compete complete replies, so the point is moot anyway.

Hehehehe, in this case it is probably just chance, though slightly skewed.