Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

Double response database entries

  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206027 by Selcal
Double response database entries was created by Selcal
Hi all,

Since I've been using LimeSurvey, I've run into a problem where on occasion, when a response is submitted, two database entries appear in the survey_xxxxxx database. They each have the same token, but a different seed. The first one will have all null answers/datetime/etc, the other contains the correct actual submitted information.

Background:
  • I run a backend system that uses the API to add invites. In the past I had this issue also when directly adding invites in the database.
  • This has occurred from V3.x onwards in several versions, up to now when I have updated everything to 4.3.18 (200928). I had several instances installed, all exhibited this behaviour from time to time, with no appearent pattern to it (different invited users, etc)
  • The webserver logs show no errors
  • Using MySQL, PHP 7.4 now, but the issue arose from PHP7.0.x onwards. All required and recommended modules are installed/enabled.
  • SQL Servertype: MariaDB, 10.3.23-MariaDB-log-cll-lve - MariaDB Server, Protocol v10
  • Have used Installatron install packages, and now manually installed packages, still the same result

The problem is, that when using Limesurvey, it reads the first entry from the database which usually is the null entry. So the respondent seems in the result to not have submitted any answers. When in fact, when I delete the null entry, the correct answers appear. Same behaviour when I read the results by token using the API.

It's difficult to troubleshoot, and even difficult to detect. I've now written a PHP script that detects double token entries to alert me, so I can keep track of it. Of about 80-100 invites per day, about 1 will have such a problem. It's not always the same respondent.

I'm looking for ways to find more information. I've found the option to increase debugging, but I read everywhere this will output to the screen? I don't want that for my users, can it be logged to file as well? I'm sure it can, but I haven't found it (I really did try -- sorry about that!).
Any possible other options I could try to improve database queries or the likes? To prevent this?
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206029 by Selcal
Replied by Selcal on topic Double response database entries
Forgot to add, running on a shared host, Linux (3.10.0-962.3.2.lve1.5.32.el6h.x86_64), and Apache though it refuses to report Apache version anywhere. PHP current version is 7.4.9.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 6 months ago #206032 by DenisChenu
Replied by DenisChenu on topic Double response database entries
Without any plugins ?

With the last 3.X version ?

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206033 by Selcal
Replied by Selcal on topic Double response database entries
Thanks, no plugins loaded, all setup with default configuration. It happened with the last V3 installation I had, 3.22.25, now I have migrated all to 4.3.18 and it still happens with that.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 6 months ago #206034 by DenisChenu
Replied by DenisChenu on topic Double response database entries

Selcal wrote: I have migrated all to 4.3.18 and it still happens with that.

… bad idea …

Best is to stay on stable … if you can revert (have a DB backup) , i rellay think it's best.

Else :
I never see this … can you reproduce with a simple survey ?
If yes : can you report the issue with this simple survey
If not : can you report the issue with the survey as lss and explain you try to reproduce.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206035 by Selcal
Replied by Selcal on topic Double response database entries
I can go back though v4 offers some advantages I use. I'll have a think about reverting, but it won't help me with this unfortunately.

Most of the surveys are quite simple. The problem is that it's hard to reproduce as it occurs with no clear pattern. I'd like to investigate to be able to make a more clear bug report.

Any possibility of logging debug info to a file instead of screen? It could help me. Otherwise I'd have to sit and do surveys about 100 times manually before it possibly happens :).
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 6 months ago - 3 years 6 months ago #206036 by DenisChenu
Replied by DenisChenu on topic Double response database entries
You can debug easily on a file
manual.limesurvey.org/Optional_settings#Logging_settings

You can add tracevar function to trace some variables

Then:
Code:
'log' => array(
      'routes' => array(
        'fileTrace' => array(
          'class' => 'CFileLogRoute',
          'categories' => 'vardump',
        ),
      ),
    ),

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
Last edit: 3 years 6 months ago by DenisChenu.
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206038 by Selcal
Replied by Selcal on topic Double response database entries
Thanks, I really can't believe I didn't find that section on files before. Too much staring at similar pages made me go crosseyed I guess. I'll set that up and then see if I can catch anything!
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 6 months ago #206041 by DenisChenu
Replied by DenisChenu on topic Double response database entries
The place to put some trace : github.com/LimeSurvey/LimeSurvey/blob/31...ger_helper.php#L5654

in my opinion.

And check what happen to session : srid.
github.com/LimeSurvey/LimeSurvey/blob/31...ger_helper.php#L5499

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206044 by Selcal
Replied by Selcal on topic Double response database entries
Thanks -- I'll check those.
I'm going to try to catch some of these errors. I already record time/date in the responses so with logs should be able to get something. Hopefully back soon!
The topic has been locked.
  • Selcal
  • Selcal's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
3 years 6 months ago #206052 by Selcal
Replied by Selcal on topic Double response database entries
I think I managed to locate the issue. It occurs when a survey link is opened twice at virtually the same time. The entry in the database is created twice at this point. Only one gets filled in and the other remains orphaned in the database.

I've raised a bug in the tracker.

Background:

I mostly provide invite/survey for third parties making it hard to get in touch with end users. But the issue actually re-arose this afternoon. Nothing in the logs so far -- but I got permission to ask the end-user filling in the survey how they did it.

Turns out they have Outlook, and sometimes use web outlook. On Outlook, a link is opened by double-clicking (security setting). In web outlook, it is a single click. The user, out of habit, double-clicked in web outlook this afternoon; opening the survey twice at virtually the same time. It then occurred to me that this may trigger the issue. And then I could reproduce it.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
3 years 6 months ago #206055 by DenisChenu
Replied by DenisChenu on topic Double response database entries
I have some issue with reloadAnyReponse and double load.

I create a botRegexp to disable block response edition.
gitlab.com/SondagesPro/coreAndTools/relo...ponse.php#L1011-1021

This seems to happen with some proxy. But outlook (single) didn't open the link (excel do it … try to open the link …)

Else : if you didn't activate Enable token-based response persistence : each time user click on the link : it create a new response.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.

Lime-years ahead

Online-surveys for every purse and purpose