Error: Reaction Times are Measured Incorrectly?

Mehr
2 Monate 2 Wochen her - 2 Monate 2 Wochen her #179994 von ultimatte
COM_KUNENA_MESSAGE_CREATED_NEW
Hi everyone,

I have been successfully (imho) using Limesurvey for three years and I love it. Thank you for your great work.
However, now, I have encountered an issue with which, please, maybe you could help me:

I used Limesurvey Version 3.15.6+190108 locally on XAMPP (Windows 10, 64-bit) to design a survey.
Limesurvey, of course, measures, reaction time for each page that is answered.
In my survey, I created several pages with text that forward automatically to the next page after 3 seconds. There is no option to interact with these pages.
I was hoping to measure a general latency of the hardware and internet setup of future online respondents to my survey.
Unfortunately, when I activated and tested the survey I realized that the reaction time measures vary considerably for these pages.
I tested three times and the reaction times varied between 3.69 and 4.91 seconds, although, of course, I tested the survey locally with the exact same setup and the pages forwarded each after 3 seconds.

Do you know if this is an error in Limesurvey?
Is there a way to prevent this, i.e. to suppress this variation of reaction time measurement?

Thank you so much,
Daniel
Last edit: 2 Monate 2 Wochen her by ultimatte.

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

LimeSurvey Partners
Mehr
2 Monate 2 Wochen her #179996 von holch
COM_KUNENA_MESSAGE_REPLIED_NEW
Webpages never load within the same time. There are so many factors that influence that (Browser, webserver, database server, OS, etc.).

I find 3.69 - 4.91 actually estonishingly good for your setup (even if you are running things locally, there are so many things that play into this.

I think if you want to run tests with the level of accuracy that you expect, you probably can only do it within a native application (not web based) and after a lot of testing.

I would always take the times in Limesurvey (or any other webbased survey system) with a grain of salt and I would definitely not rely on this if you need accurace of 1 second or less.

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #179997 von holch
COM_KUNENA_MESSAGE_REPLIED_NEW
By the way: LS does not measure "reaction" time. But the time between when the page is loaded to when the respondend moves on from the page.

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180000 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
Thank you very much for your reply.
I realize that reaction times are an issue with online experiments. However, I was not expecting the maximum delay to be almost 200% higher than the lowest if the study is run locally with browser, server, OS, etc. all the same (and nothing else running in background). That's a bummer :/
Studies seem to suggest that they are more reliable (cf. URL: peerj.com/articles/1058/ )
Thanks again,
Daniel

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180001 von holch
COM_KUNENA_MESSAGE_REPLIED_NEW
I am not an expert on this, so wait until some more experienced users (like Tpartner & Denis) will skip in. However, in my experience, everything that has to do with web technology is generally always quite "unstable" when it comes to response time. You still have http and database connections, that might take sometimes longer, some times are quicker.

I guess when experiments are time sensitive, they should not be dependend on database connections, etc. You might be able to run them purely in the browser (e.g. via Javascript, Flash, etc) and only at the end transfer the timings etc. This is probably better when treating time sensitive experiments. But even then, with Javascript, at least a couple of years ago there were significant differences between the speek of how browsers rendered javascript code (if I remember right, Chrome was especially quick in doing so), etc.

Of course, with the same machine and browser, etc. this shouldn't impact too much, but I just wouldn't want to rely on it.

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180011 von DenisChenu
COM_KUNENA_MESSAGE_REPLIED_NEW
For showing a page during X seconds,

I think i use a
- Hide page during load
- Use learn.jquery.com/using-jquery-core/document-ready/ to show the page and start the counter
- Click next (and hide the page) when counter get 0


For body loading/loaded see
javascript : gitlab.com/SondagesPro/SurveyThemes/skel...s/skelvanilla.js#L85
css : gitlab.com/SondagesPro/SurveyThemes/skel.../skelvanilla.css#L31
html : gitlab.com/SondagesPro/SurveyThemes/skel...yout_global.twig#L25

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180017 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
Hi, thank you for your reply. However, I'm sorry, I can't follow.
Do you suggest that I should not use the timer provided by Limesurvey but jquery?
Unfortunately, I'm not familiar with jquery. I have to write the code in the question's source code, right?
I would appreciate a lot if you could elaborate.
Thank you so much

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180018 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
I couldn't find an option in Limesurvey to 'hide the during loading' :/ (only 'hide question'?)

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180020 von DenisChenu
COM_KUNENA_MESSAGE_REPLIED_NEW

ultimatte schrieb: …
Do you suggest that I should not use the timer provided by Limesurvey but jquery?

Sorry, i didn't know you use LimeSurvey timer :).

Maybe you can test importing skelvanilla theme and test with it.

gitlab.com/SondagesPro/SurveyThemes/skelvanilla
dl.sondages.pro/skelvanilla.zip

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .
Folgende Benutzer bedankten sich: ultimatte

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180022 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
Thank you!
I tested it. Unfortunately, I still get a considerable variance in page load times with your theme although your theme seems slightly better: 3.78-4.82s with Vanilla; 4.2-5.2s with Skelvanilla.
Please, do you have another suggestion?

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180023 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
Apart from this, your theme looks great and I would like to use it. However, I need the grey background from Vanilla as I present images with white background. Could you please tell me how to get the grey background back in your theme?
Thanks a lot

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180028 von DenisChenu
COM_KUNENA_MESSAGE_REPLIED_NEW
Loook at theme option : you have whole boostswatch included.
And here : you can choose "well" for question wrapper.

And you can make your own theme with gitlab.com/SondagesPro/SurveyThemes/less-skelvanilla (but need css/less compentency)

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180044 von ultimatte
COM_KUNENA_MESSAGE_REPLIED_NEW
Thank you! That worked <3

Do you or someone else maybe have another suggestion for the response time measurement?

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Mehr
2 Monate 2 Wochen her #180046 von DenisChenu
COM_KUNENA_MESSAGE_REPLIED_NEW
Maybe this need to be reported : response time must be done after HTML is shown ?

Didn't know …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Bitte Anmelden oder Registrieren um an der Konversation teilzunehmen.

Jetzt loslegen!

Melden Sie sich jetzt an, und erstellen Sie in wenigen Minuten Ihre erste Umfrage.

Account einrichten

Abonnieren Sie unseren Newsletter

Abonnieren Sie unseren Newsletter für alle Neuigkeiten rund um LimeSurvey
captcha