Duplicate Tokens Being Generated

4 years 9 months ago #81122 by brian
Hello friends -

One of my dashboards started showing some flaky behavior; when I poked around I found that I've got a bunch of duplicate tokens in my token table! I have 2552 rows in lime_tokens_##### and 2432 distinct token fields in lime_tokens_#####. I am running lime Version 1.91+ Build 10951.

I do not have tokens set to expire, i.e., validfrom and validuntil is null in all rows.

I am not importing survey recipients via spreadsheet or adding people manually, but rather, an programmatically inserting values into lime_tokens_##### and then invoking the lime 'generate tokens' functionality through automated web mimicry; i.e., HTTP posts generated from the java space.

This is a big, big problem for me. Has anyone else run into something like this? If I need to, I can change up my routine and generate my own random keys and skip out on the component that invokes lime "random" token generation, but wanted to see if anyone out here could give me some insight.


Please Log in to join the conversation.

4 years 9 months ago #81124 by brian
Note: Somehow a 80kb .png violated a maximum length for attachments, but I had a screenshot of my database explorer and duplicate token action.

Please Log in to join the conversation.

4 years 9 months ago #81126 by holch
There is also a limit for the size in Pixel. I think it is something around 800px. You could upload the screenshot somewhere else and send a link though.

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Please Log in to join the conversation.

4 years 9 months ago #81127 by Mazi
I have never read about such a problem and I'm pretty sure this is related to your customized code. Does it include the check for exisiting tokens like it is done internally when tokens are created by Limesurvey?

Best regards/Beste Grüße,
Dr. Marcel Minke
(Limesurvey Head of Support)
Need Help? We offer professional Limesurvey support
Contact: marcel.minke(at)survey-consulting.com'"

Please Log in to join the conversation.

4 years 9 months ago - 4 years 9 months ago #81224 by brian
Hi Mazzi -

I'd be the first to blame my code too, but I can't see how. (?) In any case, I've modified my code to generate my own unique tokens and skipped the component that was invoking the lime piece to handle this.

Even crazier, when I extract the .ddl from lime_tokens_##### it tells me that there is a unique index attached to the token column. See the attached screenshot. That isn't my code!

Last Edit: 4 years 9 months ago by brian. Reason: changed 'spreadsheet' to 'screenshot' / sorry

Please Log in to join the conversation.

4 years 9 months ago #81307 by lucianoes
I'm having the same issu working with remote control. There are duplicate tokens. I'm generating tokens from perl(Request Tracker).

Please Log in to join the conversation.

1 year 4 months ago #128818 by FortyEightK
I know this thread is a bit ancient but I've had the duplicate token issue too so I created a little script for myself that helps in the process of getting rid of them and others may find it useful. Duplication isn't a massive issue as they don't occur regularly so it's just a manual process of removing the offending records. The following is based on SQL Server but could be easily modified for MySQL. This script will generate some more T-SQL that's currently running surveys that have duplicate tokens. Run this in SSMS or your favoured database IDE:

use limesurvey
SELECT sid, lang.surveyls_title,
'SELECT ''' + lang.surveyls_title + ''' as title,' + CAST(sid as varchar(20)) + ' as LsId, token, COUNT(token) as cnt FROM lime_survey_' + CAST(ls.sid as varchar(20)) + ' GROUP BY token HAVING COUNT(token) > 1 UNION ' as query
FROM lime_surveys ls INNER JOIN lime_surveys_languagesettings lang ON ls.sid = lang.surveyls_survey_id
WHERE ls.active = 'Y'
ORDER BY lang.surveyls_title

Copy the results of the [query] column into another database IDE window, remove the trailing UNION and Execute. It will list the offending duplicate tokens along with the Survey ID which is easily translatable to the appropriate answer table. It's then just a case of searching for the dupe tokens and removing the records that are mostly NULLs (always check the records you're deleting first though!)

Please Log in to join the conversation.