Welcome to the LimeSurvey Community Forum

Ask the community, share ideas, and connect with other LimeSurvey users!

Humongous sessions

  • jschreuder
  • jschreuder's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
7 years 6 months ago - 7 years 6 months ago #140917 by jschreuder
Humongous sessions was created by jschreuder
We're running a very big survey with 173 questions in 25 groups with many conditions. After opening it up to about 25000 potential participants it crashed after about 900 had tried filling it out. It turned out that the session table exploded, each session being about 2 megabytes in size. I've had to truncate the sessions table to get the site running again. I've switched to PHP's own session driver instead of MySQL and looked in the generated session file, it turns out it stores the complete survey (questions, conditions, intros, etc.) in there for each and every user. Is that how it is intended to work? Our table exploded to about 2 gigabytes in size because of this. Is there any way I can switch this off? Because there's no way I can run the survey like this...
Last edit: 7 years 6 months ago by jschreuder.
The topic has been locked.
More
7 years 6 months ago #140945 by Moradzadeh
Replied by Moradzadeh on topic Humongous sessions
Looking at my own database (running 7 small surveys), I see nothing of the kind of storage you report. What table is the complete survey appearing in for each session?
The topic has been locked.
  • jschreuder
  • jschreuder's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
7 years 6 months ago #140949 by jschreuder
Replied by jschreuder on topic Humongous sessions
I've put it on another server now. And with file-based storage, and each individual session still takes about 1.8 megabytes. But it won't crash my DB anymore. It was stored in the lime_sessions table.

Still when I look into each session file PHP's native session handler generates it has the entire survey in there. With everything. So I'm not surprised you won't see as much with a small survey (we haven't had trouble up till now with smaller surveys). But the sessions should still be surprisingly big, even for small surveys. As they take up much more space than just the keys + answers should.
The topic has been locked.
More
7 years 6 months ago #140950 by Moradzadeh
Replied by Moradzadeh on topic Humongous sessions
Glad you found a solution. Odd, when running a survey response, I still do not see that, but I am probably trying to replicate your issue the wrong way.
The topic has been locked.
  • jschreuder
  • jschreuder's Avatar Topic Author
  • Offline
  • New Member
  • New Member
More
7 years 6 months ago #140953 by jschreuder
Replied by jschreuder on topic Humongous sessions
I was actually hoping for a misconfiguration on my part, but I found the following in the documentation this morning:

"When using database sessions, make sure then MySQL setting max_allowed_packet is set large enough, some surveys generate over 2 MB of session data."
- manual.limesurvey.org/Global_settings#General_description (last bit of "Session lifetime")

So this seems to be intended behaviour. I fear it makes LimeSurvey unfit for large-scale surveys with big questionnaires. Not complaining, I'm very happy with it for the small stuff. But it seems a pity as I think it's unnecessary to make sessions this huge.
The topic has been locked.
More
7 years 6 months ago #140954 by Moradzadeh
Replied by Moradzadeh on topic Humongous sessions
Looks like an opportunity to optimize with a plug-in or a re-code.
The topic has been locked.
More
7 years 6 months ago #140990 by jelo
Replied by jelo on topic Humongous sessions

jschreuder wrote: So this seems to be intended behaviour. I fear it makes LimeSurvey unfit for large-scale surveys with big questionnaires. Not complaining, I'm very happy with it for the small stuff. But it seems a pity as I think it's unnecessary to make sessions this huge.

Sessionconcept of Limesurvey is still a big malus point.
Even when using php session files you will have to keep an eye on the session generation with big surveys with a lot of users in a small timeframe. Each time someone hits the survey a 2MB file is generated in the php session directory. The default session directory is tmp on UNIX machines. With some hosting providers TMP is put on a secured, locked down partition with a size limit. So there might be a barrier hit, when having a lot of session files generated in a small timeframe.
The I/O burden can also be a pity. Having a SSD for the sessionfiles can help since the 2 MB file is directly written when the user hits the first surveypage.

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
The topic has been locked.
More
7 years 6 months ago #140992 by Moradzadeh
Replied by Moradzadeh on topic Humongous sessions
I'm guessing that the original developers figured that the pre-written file would be a performance improvement over generating the survey on the fly for each user's visit to the site. For smaller surveys, they may be right.

For your case, I wonder whether breaking your humongous survey into perhaps three smaller ones might serve your purposes.
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
7 years 6 months ago #141051 by DenisChenu
Replied by DenisChenu on topic Humongous sessions

Moradzadeh wrote: ..... a re-code.

Yes for some part, this must needed.
But without breaking existing ....

And we must use Cache and not session for some other part ... session must only keep the answers.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
More
7 years 3 months ago #145788 by bruce78
Replied by bruce78 on topic Humongous sessions

DenisChenu wrote:

Moradzadeh wrote: ..... a re-code.

And we must use Cache and not session for some other part ... session must only keep the answers.


Hi Denis, are there any plans on adding caches/session caches to v. 3.0 at all?
The topic has been locked.
  • DenisChenu
  • DenisChenu's Avatar
  • Offline
  • LimeSurvey Community Team
  • LimeSurvey Community Team
More
7 years 3 months ago #145843 by DenisChenu
Replied by DenisChenu on topic Humongous sessions

bruce78 wrote: ....
Hi Denis, are there any plans on adding caches/session caches to v. 3.0 at all?

Not currently. And remind : we still use $_SESSIOn : it's one of the reason the first pacge can take a long long long time to call.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand , plugin development .
I don't answer to private message.
The topic has been locked.
More
7 years 3 months ago #145851 by bruce78
Replied by bruce78 on topic Humongous sessions

DenisChenu wrote: Not currently. And remind : we still use $_SESSIOn : it's one of the reason the first pacge can take a long long long time to call.


Interesting... btw, do you think disk sessions are faster/more reliable than db sessions, all things being equal?
The topic has been locked.

Lime-years ahead

Online-surveys for every purse and purpose