Welcome, Guest
Username: Password: Remember me

TOPIC: Problem with char set when acquring a question group

Problem with char set when acquring a question group 3 years 9 months ago #56186

  • sitawit
  • sitawit's Avatar
  • OFFLINE
  • Fresh Lemon
  • Posts: 9
  • Karma: 0
I have a problem when acquiring data from limesurvey directly by using stored procedure in MS SQL Server 2008 R2.

Here is my situation.


  • I build another application to use data of limesurvey.
    I use stored procedure to acquire data from limesurvey's database.
    I create web service which will handle all request to my stored procedure.
    I use JSON as a response of web service.
    I use PHP to create a front end application to request those web services.

  • Here is my problem


    My problem is when I display a question group name using above method I have problem with special character. My question group name contains German's special character and when it saves to database it has some character like ä „ ü instead of the actual German special character. So when I try to acquire it, it will present these unknown character instead of the right character.

    My survey set up as English as main language but I need to present some German char too (imagine teaching German using English :P)

    I wonder how can limesurvey itself can get these character correctly? My application is quite complicated but do someone has some guide for this or give me a clue how can limesurvey do this.

    This is my develop environment.
    - MS SQL Server 2008 R2 as limesurvey database
    - Limesurvey Version 1.90+ Build 9596
    - C# .Net 4.0 as web service
    - JSON for .Net from sourceforge.net/projects/csjson/
    - PHP 5.2.6 + apache 2.2 as frontend application
    The administrator has disabled public write access.

    Re: Problem with char set when acquring a question group 3 years 9 months ago #56214

    • Mazi
    • Mazi's Avatar
    • NOW ONLINE
    • LimeSurvey Team
    • Posts: 5425
    • Thank you received: 307
    • Karma: 252
    If Limesurvey displays these characters correctly at the survey then they should be stored correctly at the database.
    In this case check if your scripts use UTF-8 charset and if outputted data is escaped correctly.

    Best regards/Beste Grüße,
    Dr. Marcel Minke
    (Limesurvey Head of Support)
    Need Help? We offer professional Limesurvey support
    Contact: marcel.minke(at)limesurvey.org'"
    The administrator has disabled public write access.

    Re: Problem with char set when acquring a question group 3 years 9 months ago #56230

    • sitawit
    • sitawit's Avatar
    • OFFLINE
    • Fresh Lemon
    • Posts: 9
    • Karma: 0
    Thanks for reply :laugh:

    Is this one do the trick? :huh:
    html_entity_decode($group_name, ENT_COMPAT, 'UTF-8')

    I've tried it but not all of them decode correctly. (or I did something wrong?)

    Actually one of the main problem is JSON decode in PHP. Lets say I "select * from lime_groups" and encode as JSON after that I decode it but fail. The special char that keep in "lime_groups" is the problem so I used this to remove some unsafe char.
    	function safeJSONChars($data) {
    	    $aux = str_split($data);
    	    foreach($aux as $a) {
    	        $a1 = urlencode($a);
    	        $aa = explode("%", $a1);
    	        foreach($aa as $v) {
    	            if($v!="") {
    	                if(hexdec($v)>127) {
    	                $data = str_replace($a,"&#".hexdec($v).";",$data);
    	                }
    	            }
    	        }
    	    }
    	    return $data;
    	}

    Above function I believe it is the cause of problem. It might remove some necessary data so the UTF-8 decode cannot function normally.

    So do you have suggestion of dealing with JSON and these special char? :unsure:
    The administrator has disabled public write access.

    Re: Problem with char set when acquring a question group 3 years 9 months ago #56233

    • sitawit
    • sitawit's Avatar
    • OFFLINE
    • Fresh Lemon
    • Posts: 9
    • Karma: 0
    Got another clue.

    JSON seem not to be the problem if we do the "escape" thing right.
    But now it is how limesurvey encode and decode those special char can someone show me just the function (for the question group of cause).
    The administrator has disabled public write access.

    Re: Problem with char set when acquring a question group 3 years 9 months ago #56239

    • sitawit
    • sitawit's Avatar
    • OFFLINE
    • Fresh Lemon
    • Posts: 9
    • Karma: 0
    FINALLY got the solution.

    The problem of all thing is C# .net (As web service) and PHP (As front end).
    I use URL escape for transferring data to JSON.

    the URL encode function PHP is not identical to C# .net URL encode.
    So when it decodes, it fails.

    I need to set some encoding type in C# .net to make it works with PHP URL encode.

    URL encode in C# .net
    System.Web.HttpUtility.UrlEncode("some special char", Encoding.GetEncoding(1252));

    I make C# .net encode URL using Western European (Windows) and its represent number is 1252.

    And in PHP you can URL decode with
    urldecode("some url encoded char");

    This way I can get the right question group name!

    Don't know if this is a good way to program but it works for me.
    The administrator has disabled public write access.

    Re: Problem with char set when acquring a question group 3 years 9 months ago #56252

    • Mazi
    • Mazi's Avatar
    • NOW ONLINE
    • LimeSurvey Team
    • Posts: 5425
    • Thank you received: 307
    • Karma: 252
    Thanks for posting your solution!

    Best regards/Beste Grüße,
    Dr. Marcel Minke
    (Limesurvey Head of Support)
    Need Help? We offer professional Limesurvey support
    Contact: marcel.minke(at)limesurvey.org'"
    The administrator has disabled public write access.
    Moderators: ITEd
    Time to create page: 0.310 seconds
    Donation Image