Welcome, Guest
Username: Password: Remember me

TOPIC: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes

purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63260

  • TMSWhite
  • TMSWhite's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 759
  • Thank you received: 82
  • Karma: 36
I'm working on extending [url=http://http://bugs.limesurvey.org/view.php?id=5103]ExpressionManager [/url]to hide/display questions - like conditions, but supporting more complex equations.

In the generated Javascript, there are the following types of xxxxxSGQA codes. Please let me know if I am understanding their purpose correctly and/or point me to documentation describing how they should be used.

(1) questionQ - a <div> around each question; controls whether the question is visible/hidden
(2) displayQ - set to 'on' if visible - but what is this really used for?
(3) javaSQGA - these match the column names from the deployed survey, and can hold a value (but don't seem to). What are they used for? Does this hold the response for each question?
(4) answerSGQA - all types except X, R, and | have answerSGQA-<value> IDs. How are these supposed to be used? Sometimes they store values, but there are many more of these than there are database fields.
(5) javatbdSGQA - what are these for?
(6) tbdispSGQA - what are these for? I'm seeing them for all question types except for :;P5H!LD|GIKRQ

If the javaSGQA fields are truly storing the current responses, then I can generate Javascript equations from ExpressionManager that control the visibility for each qestionQ based on mathematical equations using those javaSGQA values.

However, most of the custom JavaScript solutions for dealing with dynamic sums and display seem to use the answerSGQA fields, so I'm confused as to which I should be using.

Any pointers? Thanks.

/Tom
The administrator has disabled public write access.

Re: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63262

  • TMSWhite
  • TMSWhite's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 759
  • Thank you received: 82
  • Karma: 36
Below is the reply that Thibault wrote on the dev mailing list. I'm replicating it here so that he doesn't have to :)
TMSWhite wrote:
In the generated Javascript, there are the following types of xxxxxSGQA codes. Please let me know if I am understanding their purpose correctly and/or point me to documentation describing how they should be used.

(1) questionQ - a <div> around each question; controls whether the question is visible/hidden

Exactly.
TMSWhite wrote:
(2) displayQ - set to 'on' if visible - but what is this really used for?

I would love to know the current use of this field as well.
In fact I would love to have this field recorded to DB and used to efficiently implement the chaining-conditions thing.
TMSWhite wrote:
(3) javaSQGA - these match the column names from the deployed survey, and can hold a value (but don't seem to). What are they used for? Does this hold the response for each question?
(4) answerSGQA - all types except X, R, and | have answerSGQA-<value> IDs. How are these supposed to be used? Sometimes they store values, but there are many more of these than there are database fields.

At the time I rewrote the conditions-code the answerSGQA element was used to store the answer for a question in the current page. However, for some question types the true value to be recorded to DB is computed from the HTML elements manipulated by the participants (see for instance the Ranking question type). In this case the element used to store the value my be named by the javaSQGA convention.

The above statement was true for questions displayed in the same page, but since conditions can use answers from questions in previous page, javaSGQA hidden-input elements are used to store answer from previous page.

The only way to know in which element you can get an answer is to use the retrieveJSidname() php function.
TMSWhite wrote:
(5) javatbdSGQA - what are these for?
(6) tbdispSGQA - what are these for? I'm seeing them for all question types except for :;P5H!LD|GIKRQ
TMSWhite wrote:
If the javaSGQA fields are truly storing the current responses, then I can generate Javascript equations from ExpressionManager that control the visibility for each qestionQ based on mathematical equations using those javaSGQA values.

However, most of the custom JavaScript solutions for dealing with dynamic sums and display seem to use the answerSGQA fields, so I'm confused as to which I should be using.

Any pointers? Thanks.

I'm afraid that the only pointer is the PHP code of the retrieveJSidname() function.

Hope this helps a little bit,
Thibault
Last Edit: 3 years 2 weeks ago by TMSWhite. Reason: fix typo
The administrator has disabled public write access.

Re: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63266

  • TMSWhite
  • TMSWhite's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 759
  • Thank you received: 82
  • Karma: 36
Thibault wrote:
At the time I rewrote the conditions-code the answerSGQA element was used to store the answer for a question in the current page. However, for some question types the true value to be recorded to DB is computed from the HTML elements manipulated by the participants (see for instance the Ranking question type). In this case the element used to store the value my be named by the javaSQGA convention.

The above statement was true for questions displayed in the same page, but since conditions can use answers from questions in previous page, javaSGQA hidden-input elements are used to store answer from previous page.

The only way to know in which element you can get an answer is to use the retrieveJSidname() php function.

I did try to use retrieveJSidname(), but it expects and array of Conditions, which poses two problems:
(1) ExpressionManager has access to $fieldMap, which has all of the valid SGQA codes and provides easy access to the currently stored values for those questions (in $_SESSION[SGQA]) - so generating the Conditions array from each $fieldMap element seems like overkill.
(2) As part of this enhancement, I'm going to be advocating for eliminating the Conditions table entirely, and simplifying all of the SQL queries that use it, since ExpressionManager should be able to provide backwards-compatible support to all existing Conditions.

So, I'm trying to find a way to generate the needed JavaScript "name" and "id" fields directly from $fieldMap. I can get, from that, the info that retrieveJSidname() wants, but a key, underlying question, is which fields do I really need? I want to support the following:
(1) Make question visible/invisible based upon equations processed by ExpressionManager.
(2) Ensure that I have access to all of the values that ExpressionManager needs on each page (in the form of hidden JavaScript values).

For #1, changing question visibility should be easy via the questionQ attributes, assuming I can compute the Boolean result of the equation. ExpressionManager should be able to generate an appropriately formatted JavaScript equation within noop_checkconditions(), so we could change question visibility on the fly.

For #2, the question is, which JavaScript variable name will reliably store the current value for each question?

According to retrieveJSidname(), I should be using "javaSGQA", except for:
(a) QuestionType is (one of D,N,S,T,U,Q,K) AND the question is being displayed on the current page (in which case use "answerSGQA")
(b) QuestionType is R (ranking) in which case use "fvalue_QA" if on the same page, or "javaSGQA" on a prior page.

However, what part of the code ensures that hidden values for each of those values actually get sent to the HTML page? In my current tests, all of the "value" attributes for the "javaSGQA" and "answerSGQA" nodes are empty.

Does the current Conditions management system only write values for SGQA fields that it "knows" are needed on conditional processing?
If so, I could emulate that with ExpressionManager, since it does know all the variables it uses; and it could do a cascascade search of whether any of those variables depend upon other variables.

Rather than going to the extreme of such a cascading search, would there be any harm or privacy risk in simply writing all of the hidden fields for all current responses to the HTML page? If that is a risk, another option would be to identify all of the fields within a survey which are part of an equation or tailored message, and just write all of their values as hidden fields on each page.

I wonder, since we're talking about making ExpressionManager optional, maybe I should have it take that later approach:
(1) Identify all fields within a survey that are part of equations, assessments, tailoring, or validation
(2) Write the current value of each to hidden fields on each page - perhaps using a new naming schema like "exprMgrSGQA" to avoid name clashes with the current javaSGQA, answerSGQA, etc. fields.
(3) Extend noop_checkconditions() so that the epxrMgrSGQA value gets updated in sync with the javaSGQA or answerSGQA values (e.g. each time a user changes a response) - that way this doesn't affect data saving
(4) Create an optional exprMgr_checkconditions() function (called from noop_checkconditions()) that would process the ExpressionManager-managed "relevancy" conditions that control question visibility.

If it's preferable, I could continue to try the retrieveJSidname() approach to avoid steps 2 and 3 above. However, do we plan to continue to require that somewhat obtuse naming system long-term? Would this be a good time to start thinking about simplifying the naming scheme?

Thoughts?
The administrator has disabled public write access.

Re: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63274

  • lemeur
  • lemeur's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 31
  • Karma: 15
retrieveJSidname(), but it expects and array of conditions
It expects an array describing only one condition (not all conditions).
Basically it only needs to know: the fieldname in DB (leads to know the qid and gid), the question type. Another important information is the group for which the Javascript evaluation code is generated.

There is also a very specific case: for type M or P questions (multiple choice), because the current conditions table stores conditions such as "SGQ = A" (whereas the true DB fieldname is a set o SGQA columns) we need to know the "SGQ" virtual quesiton code and all possible comparizon values (A1, A2, ...) this helps rewrite the condition exactly like it must be evaluated which is "SGQA = 'Y'". This very specific case will disapear in the future because I think your Expression Manager will directly write conditions using the true DB fieldname an no "virtual" SGQ name for these types of questions.
(1) ExpressionManager has access to $fieldMap, which has all of the valid SGQA codes and provides easy access to the currently stored values for those questions (in $_SESSION[SGQA])

Not always true: answers are stored in $_SESSION[SGQA] only after moving to the next page. This raises the issue of conditions using answers from questions on the same page.
(2) As part of this enhancement, I'm going to be advocating for eliminating the Conditions table entirely

Yes, I agree.
since ExpressionManager should be able to provide backwards-compatible support to all existing Conditions.

Yes, conditions from the old conditions table will be translated to ExpressionManager conditions when updating LimeSurvey.
However this will also require that we design a new GUI, because we can't reqauire all our users to learn the ExpressionManager syntax. My opinion: we should have a basic graphical condition editor able to cover 70% of survey conditions (very basic conditions). For complex conditions, teh survey designer can switch to advanced mode inwhich the GUI dispears (it's not possible to get back to simple mode without reseting conditions). In advanced mode we use the ExpressionManager, and as you proposed, it would be fantastic to have some kind of auto-completion while writing the ExpressionManager code in the Editor (I don't know how to do this, but this would be a fantastic asset).
For #2, the question is, which JavaScript variable name will reliably store the current value for each question?

According to retrieveJSidname(), I should be using "javaSGQA", except for:
(a) QuestionType is (one of D,N,S,T,U,Q,K) AND the question is being displayed on the current page (in which case use "answerSGQA")
(b) QuestionType is R (ranking) in which case use "fvalue_QA" if on the same page, or "javaSGQA" on a prior page.

Correct.
However, what part of the code ensures that hidden values for each of those values actually get sent to the HTML page?

Sorry I don't understand.
The hidden values returned by retrieveJSidname() are always filled with teh answer value (if the question was already asked, and if there was no resetting of the answer in case the participant got back and changed his answers, causing the conditions to hide previous answers).
In my current tests, all of the "value" attributes for the "javaSGQA" and "answerSGQA" nodes are empty.

This sounds weird, I might not have catched your meaning.
Does the current Conditions management system only write values for SGQA fields that it "knows" are needed on conditional processing?

Unfortunately, this is not teh case yet, but I'm assigned a ticket about this: bugs.limesurvey.org/view.php?id=5068
If so, I could emulate that with ExpressionManager, since it does know all the variables it uses; and it could do a cascascade search of whether any of those variables depend upon other variables.

That would be interresting indeed.
Rather than going to the extreme of such a cascading search, would there be any harm or privacy risk in simply writing all of the hidden fields for all current responses to the HTML page?

This is already done this way (at least it should be).

The only case where we want to hide vgariable used in conditions when possible is when we use Token-attributes in conditions (for instance the token of the participant contains data that should remain confidential ). This very rare case is one of the reasons why generating Javascript from the conditions-table ended up with a tricky code.

If that is a risk, another option would be to identify all of the fields within a survey which are part of an equation or tailored message, and just write all of their values as hidden fields on each page.

This would avoid writing unusefull values to the HTML pages.
I wonder, since we're talking about making ExpressionManager optional

Well in fact we're talking about making ExpressionManager mandatory since it will replace existing conditions (and possible future quotas and assesments).
(1) Identify all fields within a survey that are part of equations, assessments, tailoring, or validation
(2) Write the current value of each to hidden fields on each page - perhaps using a new naming schema like "exprMgrSGQA" to avoid name clashes with the current javaSGQA, answerSGQA, etc. fields.
(3) Extend noop_checkconditions() so that the epxrMgrSGQA value gets updated in sync with the javaSGQA or answerSGQA values (e.g. each time a user changes a response) - that way this doesn't affect data saving
(4) Create an optional exprMgr_checkconditions() function (called from noop_checkconditions()) that would process the ExpressionManager-managed "relevancy" conditions that control question visibility
.

That would be the safiest way to implement ExpressionManager without touching the other part of the code, but I feel like it will add extra complexity while maintaining the code: teh HTML pages would become full with duplicated hidden fields, we won't knwo which field is used for which purpose, ...

I vote against this proposal.
However, do we plan to continue to require that somewhat obtuse naming system long-term? Would this be a good time to start thinking about simplifying the naming scheme?

Very good question, and I've got a very good answer. I really think that it is time to simplify the naming scheme. this is a very courageous work but I think it is time for several reasons:
* LimeSurvey is muting this summer with GSoC, though we do not plan to rewrite eevrything from scratch, some things will change, so it is time to think about it for conditions too
* You are an excellent software designer, you're a fighter and I know you can help us on this. Morover I will certainly have more time to work on LS soon (I'm on holidays at the end of next week).
* I'm sure that simplifying the naming scheme will help the community maintain the code, even if you're not always as available as you are now (though I would love to keep you on board as long as possible) ;-)

My 2 cents,
Thibault
Last Edit: 3 years 2 weeks ago by lemeur.
The administrator has disabled public write access.

Re: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63279

  • lemeur
  • lemeur's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 31
  • Karma: 15
Just a reminder for me:
For the ExpressionManager expression editor I think that codemirror ould do the trick (codemirror.net/manual.html#modeapi). It even has support for autocomplete though I'm not sure it would be easy to implement.
The administrator has disabled public write access.

Re: purpose of the JavaScript xxxxxxxSGQA "id" and "name" attributes 3 years 2 weeks ago #63293

  • TMSWhite
  • TMSWhite's Avatar
  • OFFLINE
  • LimeSurvey Team
  • Posts: 759
  • Thank you received: 82
  • Karma: 36
lemeur wrote:
For the ExpressionManager expression editor I think that codemirror ould do the trick (codemirror.net/manual.html#modeapi). It even has support for autocomplete though I'm not sure it would be easy to implement.

Neat idea. The code coloring module (Javascript.js) is only 350 lines long, and we could remove about half of it to show highlighting of just the valid content. Since you can pre-define the valid syntax and functions, we could probably pre-define the valid functions, variables, and operators. That way auto-completion would only show allowable values. So, you'd know that it was a mostly valid expression if everything was in color (e.g. black text for variable names or functions would mean an unknown value).

Not that I want to volunteer to do that work yet - need to get the rest of the ExpressionManager suite of functions working first :) .
The administrator has disabled public write access.
Moderators: ITEd
Time to create page: 0.134 seconds
Donation Image