Check out the LimeSurvey source code on GitHub!
Welcome, Guest
Username: Password:

TOPIC: GSoC 2011: Central Participants Database

GSoC 2011: Central Participants Database 5 years 4 months ago #60319

  • jcleeland
  • jcleeland's Avatar
  • Offline
  • Moderator Lime
  • Posts: 27
  • Thank you received: 1
  • Karma: 8
This forum is a place to discuss and comment on Aniessh Sethh's Google Summer of Code Proposal, "Central Participants Database".

You can read the proposal here:

docs.limesurvey.org/Central+participants...ase+%28User+panel%29
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60329

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 5176
  • Thank you received: 769
  • Karma: 230
First of all: I am really looking forward to this feature.

It would help a lot to built custom panels, which would be a great help.

From what I could see on the proposal page, this goes already in the right direction. However, as you asked for feedback, here it comes:

There is a software called phpPanelAdmin (www.goeritz.net/panelware/). I never tried it, I never used it, but you might wanna have a look at it.

History of participation:
A very important (and probably not very compliated feature) would be a history of participation / invitation. In a panel it is very important not to "stress" your participants, but you also should send them a survey from time to time, otherwise they will not participate anymore. So it would be necessary to store when and to which survey a person was invited, and which survey they completed.

The locking feature:
Sounds good, but is not flexible enough for my taste. Because this kind of database is to use the same contacts more than once. Locking them based on the sole number of participations will not do the job in most cases. What we would need would be a feature where you could say "Only invite people who have participated / have been invited in 'less than X' surveys since dd/mm/yyy", something like this.

Source of contact:
It would be good to register the source of contact and if there was a double opt in (important in many countries) when the person registered itself for example. Often a panel is recruited from different sources (client database, free recruitment via telephone, web, etc.).

Attributes:
I like the possibility of free attributes, so everyone can create it's own panel with the characteristics the person needs. Ideally it would be possible to fill the database via a questionnaire, so that the participants can register and fill the attributes themselves.

So those where my first ideas coming up. If you did not understand some of what I was describing (sorry, had to write this quickly, but did not want to skip it, as this is really a feature that can make a big difference for Limesurvey), just get in touch.

I am really looking forward to see the first results. :-)
Have a look at the manual! It is a really valuable source for information. Here some helpful links:
Manual (EN) | Question Types | Question Attributes | Workarounds

If you found this answer helpful and it saved you some time please consider a donation to the project to keep Limesurvey going!
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60361

  • jcleeland
  • jcleeland's Avatar
  • Offline
  • Moderator Lime
  • Posts: 27
  • Thank you received: 1
  • Karma: 8
Hi Holch,

I reckon the free attributes could hold things like opt-in histories and so forth. Would you be able to describe what a "double opt-in" means in practise?

Jason
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60385

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 5176
  • Thank you received: 769
  • Karma: 230
Well, I am not sure how you would save the history of invitations and participations in the same table as the rest, but I am not a database expert. Because let's say you need to be able to see in which surveys the person has participated and when this was.

The double opt-in is a technique to guarantee that the participant really wanted to register. It is often uses for newsletters, etc. It means basically that the person has to agree twice to participate or to be registered.

Usually it is a form first, so people fill this out to register. However, anyone that knows your email could do that, right? So what happens is that there is sent a confirmation email, where the owner of the email account has to click a link if he agrees with the registration. You can see that quite often when you register for any website, etc. In email marketing this is considered best practice and for market research (panel management) I think this is minimum requirement.

Please feel free if you have any further questions. While I am not a developer (i can do some basic javascript/php stuff) at least I have some practical experience in online market research and am willing to share information with the developers. Trying to contribute at least a little bit to this great open source softare. ;-)
Have a look at the manual! It is a really valuable source for information. Here some helpful links:
Manual (EN) | Question Types | Question Attributes | Workarounds

If you found this answer helpful and it saved you some time please consider a donation to the project to keep Limesurvey going!
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60771

Well, I am not sure how you would save the history of invitations and participations in the same table as the rest, but I am not a database expert. Because let's say you need to be able to see in which surveys the person has participated and when this was.
Please see the changes in the DB Schema on the wiki, it will be linked using a combination of token_id and survey_id, and of-course participant_id.
The double opt-in is a technique to guarantee that the participant really wanted to register. It is often uses for newsletters, etc. It means basically that the person has to agree twice to participate or to be registered.
I don't know if it is required, will discuss it in the next meeting.
Usually it is a form first, so people fill this out to register. However, anyone that knows your email could do that, right? So what happens is that there is sent a confirmation email, where the owner of the email account has to click a link if he agrees with the registration. You can see that quite often when you register for any website, etc. In email marketing this is considered best practice and for market research (panel management) I think this is minimum requirement.
I think we can do it either way, form first, or checking the e-mail first. I mean a user can only fill in the form only if his e-mail is found to be valid or the participant is added to database only if he confirms his e-mail address. I think these are not that big an issues and can be handled in either manner. :)

Yeah,I always believe that experience counts a lot, so please if you have any suggestion/recommendations feel free to share and comment, it's better if I have to change the documentation now rather than change the code at a later stage :)
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60773

  • Mazi
  • Mazi's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 6009
  • Thank you received: 371
  • Karma: 260
Hi Holger,

thanks a lot for your valuable feedback!

We had a first meeting this week to discuss the details of this feature (see docs.limesurvey.org/CPDB+Meeting+Logs for outcome) and there will be another meeting next Thursday at 14:00 German time. You - and everybody else interested - are invited to join our meetings and share your opinion because we code this feature for our users, it must fit your requirements.

We have also created a special section at our wiki which deals with the context in which the Central Participant DataBase (CPDB) might be used. Please have a look and post about any shortcomings you can think of: docs.limesurvey.org/Central+participants...r+panel%29#Scenarios

Some comments (also for Aniessh to discuss them at the next meeting):

1. I like the idea of having a double opt-in feature as described by holch. We should have a setting to either use the old style (just sending email invitations) or the new double opt-in.

2. I think it was holch who suggested to also use telephone numbers (for telephone interviews) instead of email addresses as primary key. This probably won't be implemented becase of technical difficulties.
The alternative is to use fake email addresses and enter the telephone number as token or additional attribute.

3. History of participation: "So it would be necessary to store when and to which survey a person was invited, and which survey they completed" (holch).
-> This information is already available at the token table of each survey. We just need to sum this up and create a nice overview for it. Or what else do you need?

4. Source of contact: Currently we plan to store the owner ID (the user who aded the participant details) at the databse. Would this be sufficient? Where should this information be displayed later on?

5. Attributes/registration: "Ideally it would be possible to fill the database via a questionnaire, so that the participants can register and fill the attributes themselves." (holch).
-> At the curent LimeSurvey versions user can register themselves by entering first name, alst name and email. Even if there are additional atributes these can't be entered yet. Therefore we are planning to extend the registration screen to show all availavble additional atributes of a curent survey.
There might also be a "show/hide at registration" setting for each attribute if the admin only wnats to show some of them and add additonal information to some atrributes later.
What do you think about this approach?

Best regards/Beste Grüße,
Dr. Marcel Minke
(Limesurvey Head of Support)
Need Help? We offer professional Limesurvey support
Contact: marcel.minke(at)limesurvey.org'"
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60778

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 5176
  • Thank you received: 769
  • Karma: 230
Hi all!

Thank you Aniessh and Mazi for the comments. I already thought I am the only one interested in this. ;-)

I like the idea of Aniessh to register with email first and then fill in the form. Makes much more sense. On the other hand, there is probably less spam when you have to fill in more fields, I don't know. ;-)

I'll have a look at the updates in the wiki.

Unfortunately the meetings are not very convenient for me. 14:00 German time is 9:00 here and this is when I get to the office. No chance to participate, unfortunately. However, I keep posting here.

The idea with the telephone numbers wasn't mine. I don't even see much of an advantage for this. Because you can really make an automatic confirmation via telephone. And someone who registers via the internet should be very likely to have an email address. So at the moment I don't see the use of that, but maybe someone will show me a scenario where this makes sense.

Yeah, the information that we need about the survey history is already available. But, what happens when a survey is deactivated, for example? I think for having such a central database of users, the history of surveys and invitations is crucial to manage the database/panel. You should never send too many surveys in a certain timeframe and you need to send a certain minimum amount to keep them interested. This is why it would be nice, when searching for possible participants to a survey, we could exclude people that had received too many invitations, etc. Ideally we would be able to check on the frequency of surveys (e.g. 15 sent, none taken) to check if the user is still "alive" or has already quit the participation. I understand that this might be already quite advanced and I don't know if this can be part of this project. I am just giving the ideas, you guys will have to decide if it is within the scope of GSoC 2011.

Source of contact is nothing major. It would be a field where it is written for example the user who included it, or "from webform XYZ", etc. But I don't think that this is a major thing.

For making good use of such a central user database there should be more information stored about the user then pure contact information. Because then you can target your surveys a lot better. Why invite (and screen out) someone with the age of 16 if you are looking for participants with 30 and above? (just an example). Or for example, you need car owners, why invite someone who has already said he has no car? So it would be good if there would be custom attributes in the tables and those tables could be filled by a questionnaire (or varios questionnaires). For those where you don't have a certain information it stays blank, that should be a problem.
Have a look at the manual! It is a really valuable source for information. Here some helpful links:
Manual (EN) | Question Types | Question Attributes | Workarounds

If you found this answer helpful and it saved you some time please consider a donation to the project to keep Limesurvey going!
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60780

  • Mazi
  • Mazi's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 6009
  • Thank you received: 371
  • Karma: 260
Thanks for your feedback. I'm not sure if we already said this: The CPDB will be put on top of the current token system. Each survey will still have it's own token system, we don't want to break things.
Besides that there will be a central participant and attributes section. Data can be copied from CPDB -> tokens and vice versa.

Some comments:

1. Can you list all features/details you'd like to see at a central overview? We can then try to create some advanced filters on:
- number of invites received
- last survey taken on YYYY-MM-DD
- relationship invitations/answered surveys
-...

2. Larger overviews will be sortable and searching is possible, see Demo here: aniessh.sethhs.com/displayparticipant/
This should make it easy to group users or filter users by age, gender, ...

Best regards/Beste Grüße,
Dr. Marcel Minke
(Limesurvey Head of Support)
Need Help? We offer professional Limesurvey support
Contact: marcel.minke(at)limesurvey.org'"
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60782

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 5176
  • Thank you received: 769
  • Karma: 230
Where the hell do my posts go? I have this on a regular basis here in the forum. I submit a post, I even can see it posted and later on they are just gone! (by the way, the website is really, really slow for me and the documentation is even worse).

I just wanted to say that I have added two scenarios to the wiki. Feel free to update/change/delete them. However, I think that they are pretty common scenarios in market research.
Have a look at the manual! It is a really valuable source for information. Here some helpful links:
Manual (EN) | Question Types | Question Attributes | Workarounds

If you found this answer helpful and it saved you some time please consider a donation to the project to keep Limesurvey going!
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60787

Yeah, the information that we need about the survey history is already available. But, what happens when a survey is deactivated, for example? I think for having such a central database of users, the history of surveys and invitations is crucial to manage the database/panel. You should never send too many surveys in a certain timeframe and you need to send a certain minimum amount to keep them interested. This is why it would be nice, when searching for possible participants to a survey, we could exclude people that had received too many invitations, etc. Ideally we would be able to check on the frequency of surveys (e.g. 15 sent, none taken) to check if the user is still "alive" or has already quit the participation. I understand that this might be already quite advanced and I don't know if this can be part of this project. I am just giving the ideas, you guys will have to decide if it is within the scope of GSoC 2011.

I have already taken this point into consideration (i.e. Deactivated surveys) and this is handled using the lime_survey_links table, that will contain the history even if the survey is deactivated.
Source of contact is nothing major. It would be a field where it is written for example the user who included it, or "from webform XYZ", etc. But I don't think that this is a major thing.

Can be implemented using attribute, no need of a seperate field as I see it.You can set it in free form, or something like option, so that it will be easier to sort them, for example between the one's who got to know from a particular website to those who got to know from a friend.
For making good use of such a central user database there should be more information stored about the user then pure contact information. Because then you can target your surveys a lot better. Why invite (and screen out) someone with the age of 16 if you are looking for participants with 30 and above? (just an example). Or for example, you need car owners, why invite someone who has already said he has no car? So it would be good if there would be custom attributes in the tables and those tables could be filled by a questionnaire (or varios questionnaires). For those where you don't have a certain information it stays blank, that should be a problem.”

I am not very sure about what you are trying to say in the second point, but if you are talking about taking attribute with the sign-up control, then , yeah we are doing that :).
Where the hell do my posts go? I have this on a regular basis here in the forum. I submit a post, I even can see it posted and later on they are just gone! (by the way, the website is really, really slow for me and the documentation is even worse).

That sucks, happened to me once, but it just starting working again just fine :)
I just wanted to say that I have added two scenarios to the wiki. Feel free to update/change/delete them. However, I think that they are pretty common scenarios in market research.

->A user creates a new survey and wants to invite only participants that have not taken part in a survey for the last 2 months. He might also want to exclude all that have participated in a survey with a similar topic in the last 6 months.

Last two months scenario can be easily handled by a complex SQL query, however checking similar topics is something that has to be done manually.

->Taking care of the panel health
The manager of the central participant list wants to check on the panel health. He wants to know if all participants have been invited to surveys in a certain period of time, how often they have been invited, the ration between invitation and participation, etc. Ideally there would be a system to attribute "points" to each survey and those points would be added to each participant (something like the total km count for a car). Ideally there would also be another field with points that can be reset (e.g. like the day km in a car). This would help to remunerate participants for their participations, without having to "pay" them for every single survey.

I think lime_survey_links along with tokens table will do the needfull.

Please feel free to ask any question/clarification or some feedback/opinions.

Thanks
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60792

  • holch
  • holch's Avatar
  • Offline
  • LimeSurvey Team
  • Posts: 5176
  • Thank you received: 769
  • Karma: 230
Yes, I assume that similar topics is nothing that can be done automatically. But maybe if you can exclude people who took certain surveys, that will do the trick.

"The making good use..." part is handle by the custom attributes that can be filled by surveys I think. I had written it before I saw Mazis post, even if it is before my post.

So far this sounds very promising to me.
Have a look at the manual! It is a really valuable source for information. Here some helpful links:
Manual (EN) | Question Types | Question Attributes | Workarounds

If you found this answer helpful and it saved you some time please consider a donation to the project to keep Limesurvey going!
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 4 months ago #60794

Yes, I assume that similar topics is nothing that can be done automatically. But maybe if you can exclude people who took certain surveys, that will do the trick.

I think just joining the surveys field in the display participant will do the needful, but we do need to discuss how to show multiple surveys, I think it will be best to discuss it in the next meeting :)
So far this sounds very promising to me.

Thank you :).
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 3 months ago #61994

  • jelo
  • jelo's Avatar
  • Offline
  • Platinum Lime
  • Posts: 1363
  • Thank you received: 176
  • Karma: 52
What is the current status of the Closed-Loop Opt-in AKA Double Opt-in AKA
Confirmed Opt-In (COI) feature? If a registration is done by user via email and he/she receives emails the Closed-Loop Opt-in process is a must.

In the chat of the first meeting the question
but isn't registration related to surveys, not the cpdb?
was asked.

The Confirmed Opt-In has to be done by every kind of registration via email.

Beside from the compliance with law it is the only way to prevent wrong emails and keep a basic quality. To comply with law a easy Opt-Out function is a must too.

A big issue with collecting participants is keeping up with the source of the contact data. If you import data from different source it would useful to have a few attributes which contain information about the source of the participant data.

Would be interesting to know if a new unified registration process will be considered or survey (token) registration and central participants database
will implemented in a "mashed" way.

Is the aim of central participants database to act as access panel with quotation and sampling?

The current token management could be seen as the participants database (surveywide). The difference is between a personalized survey and a panelbased (central participants database) survey.
The main difference between these two is that the central participants database allows sampling and quotation.

Access to surveys can be restricted via token and with a participant databases (not important if central or on the surveylevel) via username/password. This allows new kind of participant types like tester or translator which usually need to browse through the survey many times.
New features could be based checking that participant status. Translator gets a textbox beside every item to translate the questionnaire. The "tester" gets a textbox to give feedback for the questionnaire designer. The feedback of different "testers" will be aggregated and shown in the questionnaire designe

Thanks for reading. Just a bit of loud thinking.
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 3 months ago #61999

What is the current status of the Closed-Loop Opt-in AKA Double Opt-in AKA
Confirmed Opt-In (COI) feature? If a registration is done by user via email and he/she receives emails the Closed-Loop Opt-in process is a mus

It is on our list, but we are concentrating on making the initial system work first and these feature will come in the later duration of the project.
A big issue with collecting participants is keeping up with the source of the contact data. If you import data from different source it would useful to have a few attributes which contain information about the source of the participant data.
We can always use additional attribute for it.
Would be interesting to know if a new unified registration process will be considered or survey (token) registration and central participants database will implemented in a "mashed" way.

The central registration system is not on the list, but extending the token registration to take additional attribute is. Yeah we are supporting the participants to be added to and fro from the central table to token table.
Is the aim of central participants database to act as access panel with quotation and sampling?

Yeah and sharing.
Access to surveys can be restricted via token and with a participant databases (not important if central or on the surveylevel) via username/password. This allows new kind of participant types like tester or translator which usually need to browse through the survey many times.
New features could be based checking that participant status. Translator gets a textbox beside every item to translate the questionnaire. The "tester" gets a textbox to give feedback for the questionnaire designer. The feedback of different "testers" will be aggregated and shown in the questionnaire designers
Nice idea, but I don't think it is on the list for now, but later if time permits, yeah.

Thank you :)
The administrator has disabled public write access.

GSoC 2011: Central Participants Database 5 years 2 months ago #63907

  • jelo
  • jelo's Avatar
  • Offline
  • Platinum Lime
  • Posts: 1363
  • Thank you received: 176
  • Karma: 52
A quote from the meetinglogs
docs.limesurvey.org/CPDB+Meeting+Logs
about my request form recording the source of the contact for every panel member.
[14:05] <Mazi> jasebo, do you think we need this additional information?
[14:05] <jasebo> why would it matter? What's the difference between knowing someone was added by CSV, and someone else was added manually?
[14:05] <jasebo> apart from knowing the second one took longer
[14:05] <jasebo> ?
[14:06] <Mazi> that's what I think, too. I vote for putting it on the "we do that if we really have some time left" list

It such a field important?
It depends on how you import the contact data.
Is the contact data always prepare in a csv file where every contact is from the same source? If yes, than such a field is not needed.

If contacts are from mixed sources and sometime only a few participants are added and not presorted by source such a field is very useful.

E.g. in German law you need to able to tell the participant where you got his contact details from? ( §34 I Nr.1 BDSG ).

I hope we will see a broader discussion when the first release is ready for testing. Thanks for your hard work (Aniessh and mentors).
The administrator has disabled public write access.
Time to create page: 0.404 seconds
Imprint                   Privacy policy         General Terms & Conditions         Revocation information and revocation form