LS on a server cluster. Was: What is the max response-id limesurvey can handle?

More
8 months 3 weeks ago #175972 by hvgard
Hi,

I 'm more sure this is the right category for this Q. So apologies when this one is on the wrong place.

What is the max response-id limesurvey can handle? I'm not much of a database guy. So looking up the datatype is beyond me. Let alone find the answer.

We are on Version 2.6.7-lts Build 171212

Thanks, Harold

Harold van Garderen

Please Log in or Create an account to join the conversation.

LimeSurvey Partners
More
8 months 2 weeks ago #175995 by holch
What do you mean with max response-id? The maximum length of digits a response ID can have or the maximum number of response-Id.

And what do you refer to as "response-id" exactly?

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #175997 by hvgard
Hi, thanks for you response.

We will soon have a project where a Limesurvey survey can get over 10 million "visits". I want to make sure we don't run out of the allowed ID range.

So I mean the maximum (largest) number the ID field can get. A million, 10 milllion? 2^16, 2^32? PS: I guess ID is has unsigned type in the MySQL database.

Thanks, Harold

Harold van Garderen

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #175999 by DenisChenu
int : (AUTO_INCREMENT) : 2^(31) more than 2 billion …

Assistance on LimeSurvey forum and LimeSurvey core development are on my free time.
I'm not a LimeSurvey GmbH member, professional service on demand .
An error happen ? Before make a new topic : remind the Debug mode .

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176003 by holch

We will soon have a project where a Limesurvey survey can get over 10 million "visits".


Wow, that is some serious stuff. Because in my experience, you get a return of max. 10% (maybe if you have a very special relationship with your audience you can get more, but with these numbers I doubt that there is a very special relationship).

So this would mean you would invite about 100 Million people?

I guess you have other problems than just the number of lines in the MySQL database or the potential length of the id for each response.

I guess you have heaving machinery ready, with a lot of memory, CPU power and SSD disk space to handle a load like this. Report your experience here please!

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176012 by jelo

holch wrote: Wow, that is some serious stuff. Because in my experience, you get a return of max. 10%

The project with 10 million "visits" could be very small in terms of server resources. Depends on the timeframe. The question about the ID could be the biggest issues.

@Harold: Can you share two lines about the project scope? Classic survey? Transaction feedback?

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176017 by hvgard
Thanks Denis. That should suffice for a while :-).

Harold van Garderen

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176019 by hvgard
Hi Jelo, it is a "stories" project on education for parents, teachers and children/youth. We will gather responses for 6 months. But in january there will be a campaign via regular media and we expect up to 40k visitors per hour.

Harold van Garderen

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176020 by hvgard
The potential audience is indeed 5.5 million. Even when 10% responds this means a half million responses. And people are invited to come back to share experiences.

We indeed have "thick" database servers and so far 5 webservers in a cluster to handle peak loads. We can expand the # of webservers (hardware or CPU's of memory) in 10 tot 15 mins. We can currently scale up to 12 if I remember well.

Harold van Garderen

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176021 by jelo
Thanks for the insight.

hvgard wrote: we expect up to 40k visitors per hour.

Here comes the flood. Good
luck with the project. Some kind of horizontal or vertical scaling seems to be needed, to handle that amount of visitors.

The meaning of the word "stable" for users
www.limesurvey.org/forum/development/117...ord-stable-for-users

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176023 by hvgard
:-). For that we have hired professional server managers and there will be an appropriate service level agreement. From the top of my head: 16 hours/day with 2 hour response time.

Harold van Garderen

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176060 by holch

The potential audience is indeed 5.5 million.


So the whole population of Finland...

Sounds interesting. However, a campaign via regular media probably means a response rate far below my pretty high estimate of 10%. I'd rather go towards 1%. I assume that potential audience means you can theoretically reach 5.5 Million. However, if regular media means TV, newspapers, etc, then the actual audience is probably a lot lower (not everyone will watch TV or read the newspapers), and people, if not targeted directly are less likely to respond than when invited directly. Anyway, as I don't know anything about your actual target group I might be totally wrong.

Sounds like you have put some thoughts into this with professional server managers and I guess you and them have made some stress tests already.

Good luck! And please report back once the project is running or has finished.

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176066 by hvgard
Hi "holch",

I should explain a bit. It's an and/and situation. We will have 6 "regular" invitation that go to areas of the country. Each area has around 100 to 150k people that will be invited. We expect response rates around 10% for that. There will also be a campaign early next year that will be television and/or social media based. Indeed response rates will be low. Still hitrates can be high during TV appearances or when it goes viral on f.e. facebook.

We have stress tested the system up to 4000 hits/minute. Response time/page was 1.1 seconds max. Thus far we are happy with that. As said earlier, we can add webserver boxes or CPU or memory almost on the fly.

Thanks for all your support. This project will run for months. Will keep you posted.

Harold van Garderen
The following user(s) said Thank You: DenisChenu, holch

Please Log in or Create an account to join the conversation.

More
8 months 2 weeks ago #176089 by holch

should explain a bit. It's an and/and situation.

You don't need to explain anything! But of course, it is appreciated. Every major use case of LS is always a good base when questions come up in the forum. Thank you for sharing!

I'm not a LimeSurvey GmbH member. I answer at the LimeSurvey forum in my spare time. No support via private message.
Some helpful links: Manual (EN) | Question Types | Workarounds
The following user(s) said Thank You: hvgard

Please Log in or Create an account to join the conversation.

More
3 months 3 weeks ago #182310 by hvgard
Hi, this post is to report back on running LS on a server cluster. For 4 months now we run LS on a cluster of 1 load balancer, 5 servers (4 web with 2 cores each and a database server).

I can be very short: no hiccups at all.

We had over 45.000 visitors and got almost 5000 responses (spending on average 20 mins).

We stress tested the cluster with 5000 "hits" per minute and it showed just 10% decrease in response time.

During X-mas we had moments with between 250 and 300 concurrent users.

So in case you need one, or need to build one .....

Harold van Garderen

Please Log in or Create an account to join the conversation.

Start now!

Just create your account and start using Limesurvey today.

Register now
Join our Newsletter!