Humongous sessions

More
10 months 1 week ago #145922 by bruce78
bruce78 replied the topic: Humongous sessions
Cool... I'll let you know if I check this out...
The following user(s) said Thank You: DenisChenu

Please Log in to join the conversation.

More
10 months 1 week ago #146009 by bruce78
bruce78 replied the topic: Humongous sessions

DenisChenu wrote: But think the best solution is to save session in RAM only or SSD. Maybe in a SQLite DB on RAM is the better way.
Denis


Ok, so I've been trying to get the sessions info to write to a sqlite db... I've been editing the config file but I've not had much luck, even though sqlite3 is running fine on the machine. Extracts from the config file are below but in brief...

If I leave
'class' => 'application.core.web.DbHttpSession',
as is under the
'db_sqlite' => array(
section, I get this error 500 message

Property "DbHttpSession.file" is not defined.


However, if I comment out
//'class' => 'application.core.web.DbHttpSession',
, the error message changes to

Object configuration must be an array containing a "class" element.


Below is the relevant extract of config.php and I've attached the full version in case I've missed anything else.

Do you know what I'm doing wrong or is LS not compatible with SQLite?

This thread suggests SQLite can be made to work as a core DB but these other threads ( one , two ) suggest SQLite isn't compatibe, but I suspect both are talking about using SQLite as the primary db and not for the session table...

Here's the extract from the config file...
return array(
	'components' => array(
		'db' => array(
			'connectionString' => 'mysql:host=localhost;port=3306;dbname=dbname;',
			'emulatePrepare' => true,
			'username' => 'user',
			'password' => 'pass',
			'charset' => 'utf8mb4',
			'tablePrefix' => 'lime_',
		),
 
		// Uncomment the following line if you need table-based sessions
		'session' => array (
		'class' => 'application.core.web.DbHttpSession',
		'connectionID' => 'db_sqlite',
		'sessionTableName' => '{{sessions}}',
		),
 
 
		'db_sqlite' => array(
			'file' => '/home/test.db',
			'connectionString' => 'sqlite:/home/test.db',
		//	'class' => 'application.core.web.DbHttpSession',
		    'emulatePrepare' => true,
			'username' => '',
			'password' => '',
		 // 'charset' => 'utf8mb4',
		 //	'tablePrefix' => 'lime_',
		),
Attachments:

Please Log in to join the conversation.

More
10 months 1 week ago #146011 by DenisChenu
DenisChenu replied the topic: Humongous sessions
Hi,

github.com/LimeSurvey/LimeSurvey/blob/ma...bHttpSession.php#L20

Did you try a var dump/die here ? To see if it's the good session ?

Seems OK according to Yii doc : www.yiiframework.com/doc/api/1.1/CDbHttp...#connectionID-detail

I don't see 'file' => '/home/test.db', in Yii doc about www.yiiframework.com/doc/api/1.1/CDbConnection : what did you add it ? You already have it in connectionString

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Please Log in to join the conversation.

More
10 months 3 days ago #146126 by bruce78
bruce78 replied the topic: Humongous sessions

DenisChenu wrote: github.com/LimeSurvey/LimeSurvey/blob/ma...bHttpSession.php#L20
Did you try a var dump/die here ? To see if it's the good session ?


I don't understand how to try a var dump/die - can you help me out?

DenisChenu wrote: I don't see 'file' => '/home/test.db', in Yii doc about www.yiiframework.com/doc/api/1.1/CDbConnection : what did you add it ? You already have it in connectionString


I've removed 'file' => '/home/test.db' and I still get the same errors...

Would you suggest anything else at all?

Please Log in to join the conversation.

More
10 months 3 days ago #146128 by DenisChenu
DenisChenu replied the topic: Humongous sessions
Hi,

I do some test : OK, seem to using another db is broken with this configuration. I do some more test.

BUT : there are a solution :).
'session' => array (
			'class' => 'application.core.web.DbHttpSession',
			'connectionID' => null,
			'sessionTableName' => '{{sessions}}',
		),

Unsure you need application.core.web.DbHttpSession, maybe you can test with system.web.CDbHttpSession too here.

Actually : if 'connectionID' => null, : Yii app create a sqlite db in /tmp/runtime/session-1.1.17.db

To put it elsewhere, currently usage of symbolic link but can be try with this solution.

I think LS don't try to create the db componenent, then use it but don't create . I take a look.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Please Log in to join the conversation.

More
10 months 3 days ago #146133 by jelo
jelo replied the topic: Humongous sessions

bruce78 wrote: That's a neat idea... do you think something like that might reduce my first page load time from 10 seconds to something less?

10 seconds? Stick to SSD storage in combination with sessions-files. Wouldn't use Limesurvey without SSD. Till LimeSurvey is mostly use in small surveys with not many concurrent probands the session misdesign will be last for a long time.
There are other code parts which won't be technically challenged for a long time since not many users use the stuff. E.g. export functions crumble when handling bigger amount of data. Still waiting to see a big LSA export working. Perhaps I need 128 GB RAM on a webserver to do it ;-)

Please Log in to join the conversation.

More
10 months 3 days ago #146137 by bruce78
bruce78 replied the topic: Humongous sessions

DenisChenu wrote:

'session' => array (
			'class' => 'application.core.web.DbHttpSession',
			'connectionID' => null,
			'sessionTableName' => '{{sessions}}',
		),
Actually : if 'connectionID' => null, : Yii app create a sqlite db in /tmp/runtime/session-1.1.17.db


Ok, thanks Denis, this worked as you said but sqlite didn't speed up the loading of the survey... if anything it was a bit slower and I'm also worried about using RAM for sessions as this won't allow people to save their responses if the machine needs to be rebooted, so I'm back to square one in that regard.

SSD is the next idea, as is going back to the 2.06 version....

Please Log in to join the conversation.

More
10 months 3 days ago #146138 by bruce78
bruce78 replied the topic: Humongous sessions

jelo wrote: 10 seconds? Stick to SSD storage in combination with sessions-files. Wouldn't use Limesurvey without SSD.


This is the plan... the survey was also noticeably faster on 2.06 on the same hardware...

jelo wrote: Till LimeSurvey is mostly use in small surveys with not many concurrent probands the session misdesign will be last for a long time.


Sure, I can imagine that... I guess there are no plans to speed up the session design in version 3?

Please Log in to join the conversation.

More
10 months 3 days ago #146147 by DenisChenu
DenisChenu replied the topic: Humongous sessions
My /tmp/ and php/temp are already mounted on RAM : then use the default sqlite db : this is very longer for me ...

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Please Log in to join the conversation.

More
10 months 3 days ago #146166 by DenisChenu
DenisChenu replied the topic: Humongous sessions

bruce78 wrote:

jelo wrote: Till LimeSurvey is mostly use in small surveys with not many concurrent probands the session misdesign will be last for a long time.


Sure, I can imagine that... I guess there are no plans to speed up the session design in version 3?

Big session is here since a lot of year then hard to remove (because a lot of functionality depend from this session).

Using cache seems a good idea to do some part of session : but
1. all this part must be cut before with User dependant VS not user dependant (we must cache only 'not user dependant')
2. A good obsolete system for caching

More than 2 week of complete work just for Expression Manager StartSurvey, and surely a lot of bug after.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .

Please Log in to join the conversation.

More
10 months 2 days ago #146186 by bruce78
bruce78 replied the topic: Humongous sessions

DenisChenu wrote: My /tmp/ and php/temp are already mounted on RAM : then use the default sqlite db : this is very longer for me ...


I'm guessing your talking about using one of ramfs or tmpfs and mounting the directory in fstab? These two posts seem to go over this idea... is this what you're referring to?

DenisChenu wrote: Using cache seems a good idea to do some part of session : but
1. all this part must be cut before with User dependant VS not user dependant (we must cache only 'not user dependant')
2. A good obsolete system for caching
More than 2 week of complete work just for Expression Manager StartSurvey, and surely a lot of bug after.


I can imagine it would be a lot of work... I guess for larger surveys, the trick is to optimise the machine first which is probably easier right now...

Please Log in to join the conversation.

More
10 months 2 days ago - 10 months 2 days ago #146190 by DenisChenu
DenisChenu replied the topic: Humongous sessions

bruce78 wrote: I'm guessing your talking about using one of ramfs or tmpfs and mounting the directory in fstab?

/tmpfs, yes.

But sorry : on this test server : on a hd, then seems sqlite is slower than files ?

Currently i use for this server, default config of debian7 : session.save_path /var/lib/php5/sessions
I have memcache enable, maybe i can test a memcache session :)

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .
Last Edit: 10 months 2 days ago by DenisChenu. Reason: fix quote

Please Log in to join the conversation.

More
10 months 2 days ago #146193 by bruce78
bruce78 replied the topic: Humongous sessions

DenisChenu wrote: But sorry : on this test server : on a hd, then seems sqlite is slower than files ?


Correct, the sqlite is slower than the HDD

DenisChenu wrote: Currently i use for this server, default config of debian7 : session.save_path /var/lib/php5/sessions
I have memcache enable, maybe i can test a memcache session :)


I suspect memcache would be a great improvement for large surveys - what changes need to be made to the LS code to test this/implement this?

Please Log in to join the conversation.

More
10 months 2 days ago #146195 by bruce78
bruce78 replied the topic: Humongous sessions
This guide for using memcache with Magento seems to also rely on a yii-syle config file... I wonder if it's really as simple as editing the config file as per the example...
'session' =>
   array (
      'save' => 'memcache',
      'save_path' => 'tcp://<memcache ip or host>:<memcache port>'
),

Please Log in to join the conversation.

More
10 months 2 days ago - 10 months 2 days ago #146196 by DenisChenu
DenisChenu replied the topic: Humongous sessions
I test directly with
php_value session.save_handler memcache
php_value session.save_path tcp://127.0.0.1:11211?persistent=1&weight=1&timeout=1&retry_interval=15
in my virtualhost file (and validate if it's OK)

Not really more speed, maybe the gap is not big due to session contruction and not session save/load system.

For Yii config : i don't see save_handler in www.yiiframework.com/doc/api/1.1/CHttpSession , then unsure you can set it in config.

PS : looking at memcache log : save /load are really speed, but no real HTML construction improvment.

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand (or search sondages pro).
An error happen ? Before make a new topic : remind the Debug mode .
Last Edit: 10 months 2 days ago by DenisChenu.

Please Log in to join the conversation.

Start now!

Just create your account and start using Limesurvey today.

Register now