- Posts: 1223
- Thank you received: 351
Welcome to the LimeSurvey Community Forum
Ask the community, share ideas, and connect with other LimeSurvey users!
Server capacity for many simultaneous survey takers
- Ben_V
- Offline
- Platinum Member
Less
More
7 years 4 months ago - 7 years 4 months ago #144512
by Ben_V
Benoît
EM Variables => bit.ly/1TKQyNu | EM Roadmap => bit.ly/1UTrOB4
Last Releases => 2.6x.x goo.gl/ztWfIV | 2.06/2.6.x => bit.ly/1Qv44A1
Demo Surveys => goo.gl/HuR6Xe (already included in /docs/demosurveys)
Replied by Ben_V on topic Server capacity for many simultaneous survey takers
@jelo => info about if LS 3.0 is coming with some improvements about session handling
@holch => Thank you! I had big doubts if the new LS 3.0 version will be finally based on the rewrite (described here ) intented by @sammousa
@ddrmoscow => Thank you a lot for sharing your successful experience... I think you just provided the most useful post written in this forum about server configuration for wide respondent panels & LS extreme use.
I agree about the use of CDN hosted files. This said, the use of third-party servers may create some issues when strict privacy rules compliance is required...
@holch => Thank you! I had big doubts if the new LS 3.0 version will be finally based on the rewrite (described here ) intented by @sammousa
@ddrmoscow => Thank you a lot for sharing your successful experience... I think you just provided the most useful post written in this forum about server configuration for wide respondent panels & LS extreme use.
I agree about the use of CDN hosted files. This said, the use of third-party servers may create some issues when strict privacy rules compliance is required...
Benoît
EM Variables => bit.ly/1TKQyNu | EM Roadmap => bit.ly/1UTrOB4
Last Releases => 2.6x.x goo.gl/ztWfIV | 2.06/2.6.x => bit.ly/1Qv44A1
Demo Surveys => goo.gl/HuR6Xe (already included in /docs/demosurveys)
Last edit: 7 years 4 months ago by Ben_V.
The topic has been locked.
- ddrmoscow
- Offline
- Junior Member
Less
More
- Posts: 21
- Thank you received: 10
7 years 4 months ago #144528
by ddrmoscow
Replied by ddrmoscow on topic Server capacity for many simultaneous survey takers
@Ben_V => thank you, hope it 'll be helpfull
Forgotten to mention in previous post : described server was https with valid SSL certificate, strict privacy compliant; without any third-party server issues, as all links, including cdn links were https too.
Forgotten to mention in previous post : described server was https with valid SSL certificate, strict privacy compliant; without any third-party server issues, as all links, including cdn links were https too.
The following user(s) said Thank You: Ben_V
The topic has been locked.
- holch
- Offline
- LimeSurvey Community Team
Less
More
- Posts: 11746
- Thank you received: 2750
7 years 4 months ago #144532
by holch
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.
Replied by holch on topic Server capacity for many simultaneous survey takers
@ddrmoscow: Thank you very much for your real life example. Very interesting.
What cloudhosting plattform did you use for this, if I may ask? I remember that not all scalable cloud hosting plattforms play nice with Limesurvey.
What I found very interesting was this:
I am currently struggeling with chaning the new templates, because they are full of code. I got a little rusty with my CSS and it is very hard to find the right places to make changes. Now I imagine someone with little to no CSS/HTML skills, it becomes more and more difficult to work with those templates for laymen.
And then of course there is the load issue, more CSS/HTML/Javascript, more load. Not so relevant for small surveys, but of course for those big surveys it is.
IMHO we should get simpler standard themes, with as little CSS styling as possible. Then from there more advanced users can make their changes. As I said, I tried to make a few simple tweeks to a standard template to adapt the colors, etc. but there are parts that I just can't find on how to change. E.g. there is some stupid hover effect that I just can't get rid off, which should be pretty simple.
What cloudhosting plattform did you use for this, if I may ask? I remember that not all scalable cloud hosting plattforms play nice with Limesurvey.
What I found very interesting was this:
optimize survey template - remove all unused css and jscripts, optimize + minify all resource files (images, .js, .css, e.t.c)
I am currently struggeling with chaning the new templates, because they are full of code. I got a little rusty with my CSS and it is very hard to find the right places to make changes. Now I imagine someone with little to no CSS/HTML skills, it becomes more and more difficult to work with those templates for laymen.
And then of course there is the load issue, more CSS/HTML/Javascript, more load. Not so relevant for small surveys, but of course for those big surveys it is.
IMHO we should get simpler standard themes, with as little CSS styling as possible. Then from there more advanced users can make their changes. As I said, I tried to make a few simple tweeks to a standard template to adapt the colors, etc. but there are parts that I just can't find on how to change. E.g. there is some stupid hover effect that I just can't get rid off, which should be pretty simple.
I answer at the LimeSurvey forum in my spare time, I'm not a LimeSurvey GmbH employee.
No support via private message.
The topic has been locked.
- ddrmoscow
- Offline
- Junior Member
Less
More
- Posts: 21
- Thank you received: 10
7 years 4 months ago #144541
by ddrmoscow
Replied by ddrmoscow on topic Server capacity for many simultaneous survey takers
@holch : always welcome ;
about platform used in decribed case - it's CloudStack platform combined 3 data-centers (Germany + Russia + Holland) + KVM instance (most reliable and fast according to my tests and experience) + local Russian provider Maxiplace (they don't even have english version site)
also succeed earlier with smaller scale servers on digitalocean, aws.cloud and few other Russian providers -> no problems with LS;
regarding template optimization (css and js)
- agree, basic , very light-weight minified template could be very usefull in such a case; the problem is - you never know which javascript functions would be required for a particular survey; thus, basically, ther're all included, as well as css elements;
- if you struggle with hover elements from "user agent stylesheet" - most cases the rule for that element doesn't exist in your .css/.less file
about platform used in decribed case - it's CloudStack platform combined 3 data-centers (Germany + Russia + Holland) + KVM instance (most reliable and fast according to my tests and experience) + local Russian provider Maxiplace (they don't even have english version site)
also succeed earlier with smaller scale servers on digitalocean, aws.cloud and few other Russian providers -> no problems with LS;
regarding template optimization (css and js)
- agree, basic , very light-weight minified template could be very usefull in such a case; the problem is - you never know which javascript functions would be required for a particular survey; thus, basically, ther're all included, as well as css elements;
- if you struggle with hover elements from "user agent stylesheet" - most cases the rule for that element doesn't exist in your .css/.less file
The following user(s) said Thank You: Ben_V
The topic has been locked.
- jelo
- Offline
- Platinum Member
Less
More
- Posts: 5070
- Thank you received: 1263
7 years 4 months ago #144562
by jelo
The issues with Limesurvey and cloudservices was mostly related to application-scaleout. Where you got the stack provided.
As long as the cloudplatform is starting with the OS, there is no real issue with Limesurvey.
The sessions issues when using sessions is not that big when using SSD storage. You used tokens in the invitations, right?
Would you mind to post the hours you invested upfront for the survey?
The scaling was done manually or did you use some rules to let the system adapt?
Is MariaDB (BTW version 12, really?) recognizing the scaling that good? With MySQL 5.x the memory scaleout isn't working that nice.
"peak load duration: ~75 mins" means that this was the longest interview duration you had in your survey project?
Which not automatically means that any respondent had lags caused by load issues when filling out the survey.
Your setup seems way above the infrastructure Limeservice is using for projects. Since Louis proposed the platform to teracomp it would be nice to get some specs about the Limeservice-Platform as well.
The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
Replied by jelo on topic Server capacity for many simultaneous survey takers
Ben perhaps meant that the CDN provider will get the IP of the survey respondents, when filling out the survey.ddrmoscow wrote: without any third-party server issues, as all links, including cdn links were https too.
The issues with Limesurvey and cloudservices was mostly related to application-scaleout. Where you got the stack provided.
As long as the cloudplatform is starting with the OS, there is no real issue with Limesurvey.
The sessions issues when using sessions is not that big when using SSD storage. You used tokens in the invitations, right?
Would you mind to post the hours you invested upfront for the survey?
The scaling was done manually or did you use some rules to let the system adapt?
Is MariaDB (BTW version 12, really?) recognizing the scaling that good? With MySQL 5.x the memory scaleout isn't working that nice.
"peak load duration: ~75 mins" means that this was the longest interview duration you had in your survey project?
Which not automatically means that any respondent had lags caused by load issues when filling out the survey.
Your setup seems way above the infrastructure Limeservice is using for projects. Since Louis proposed the platform to teracomp it would be nice to get some specs about the Limeservice-Platform as well.
The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users
The topic has been locked.
- ddrmoscow
- Offline
- Junior Member
Less
More
- Posts: 21
- Thank you received: 10
7 years 4 months ago #144582
by ddrmoscow
Replied by ddrmoscow on topic Server capacity for many simultaneous survey takers
@jelo: used "jsdelivr" CDN not for the first time; so far didn't notice any negative feedback + used for a lot of well-known web-resources;
yes, used tokens;
Didn't count hours to start the survey, so can evaluate approximatelly:
~ 5-6 hours survey creation, testing, cleaning and optimization on a regular server with the same LS version;
~ 5-6 hours installation LAMP + php + nginx + phpmyadmin + LS + SSL, basic configuration, DB optimization, LS tweak, testing components compatibility on a small scale server config (2Gb/2proc/10Gb SSD);
~ 4-5 hours on a high scale server -> mpm_prefork, load and stress-tests;
- scale-back to small config;
- ~2 hours next day prior to expected avalanche -> half-scale high config, prewarm server with simulated dummy traffic;
1 hour before moment "X" scaled to high config, load some more dummy traffic;
X-time => as usual: cross fingers, prepare pills and :whistle:
It makes 17-20 hours investment upfront in total . Wow, just wonder myself, is it a lot or isn't? Your opinion?
As you understood, scaling was totally manual, moreover I scaled when there're minimum to no users. Can't juge if scaling workd perfect, as I don't have enought experience, described one was the greatest.
Yea, you got me , my mistake - (no idea where 12 come from ...) Maria was 10.1.17 , sorry ;
No, by "Peak load duration" I mean that highest server load lasted only 75 mins ;
yes, used tokens;
Didn't count hours to start the survey, so can evaluate approximatelly:
~ 5-6 hours survey creation, testing, cleaning and optimization on a regular server with the same LS version;
~ 5-6 hours installation LAMP + php + nginx + phpmyadmin + LS + SSL, basic configuration, DB optimization, LS tweak, testing components compatibility on a small scale server config (2Gb/2proc/10Gb SSD);
~ 4-5 hours on a high scale server -> mpm_prefork, load and stress-tests;
- scale-back to small config;
- ~2 hours next day prior to expected avalanche -> half-scale high config, prewarm server with simulated dummy traffic;
1 hour before moment "X" scaled to high config, load some more dummy traffic;
X-time => as usual: cross fingers, prepare pills and :whistle:
It makes 17-20 hours investment upfront in total . Wow, just wonder myself, is it a lot or isn't? Your opinion?
As you understood, scaling was totally manual, moreover I scaled when there're minimum to no users. Can't juge if scaling workd perfect, as I don't have enought experience, described one was the greatest.
Yea, you got me , my mistake - (no idea where 12 come from ...) Maria was 10.1.17 , sorry ;
No, by "Peak load duration" I mean that highest server load lasted only 75 mins ;
The following user(s) said Thank You: pcerny
The topic has been locked.