[Glass] UTF8 character encoding translation between Pharo and Gemstone

[Dale K. Henrichs]

I am curious how the Unicode strings are being encoded on the Pharo side? You are transferring the strings from Pharo to GemStone correct? 

I'm not sure what Pharo does with the WideStrings when creating the STON output, but it sounds wrong to be getting DoubleByeStrings from Pharo ...

Dale



----- Original Message -----
| From: "Paul DeBruicker" <pdebruic at gmail.com>
| To: glass at lists.gemtalksystems.com
| Sent: Tuesday, October 1, 2013 12:55:12 PM
| Subject: [Glass] UTF8 character encoding translation between Pharo and	Gemstone
| 
| I've got some unicode stings in my Pharo image that were taken as
| input
| from a Seaside form.  I'm trying to use STON to get them into
| Gemstone.
|  STON parses those strings as DoubleByteString and sends
| #decodeFromUTF8.  DoubleByteString does not implement
| #decodeFromUTF8.
| If I implement #decodeFromUTF8 as
| 
| DoubleByteString>>decodeFromUTF8
| 	^self asByteArray decodeFromUTF8
| 
| I get a spurious null character (codePoint = 0) inserted by the
| primitive and the string I'm attempting to decode doubles in size.
|  Then
| when emitting that string as JSON for a webservice the null
| characters
| are encoded to UTF8 (e.g. \u0000) and sent along.
| 
| 
| So that makes me think my implementation of #decodeFromUTF8 is
| probably
| not the way to do it.
| 
| 
| Is there a better way that won't result in the extra null characters?
| 
| Should I do something to the string in Pharo before sending it along
| to
| Gemstone?
| 
| Thanks
| 
| Paul
| _______________________________________________
| Glass mailing list
| Glass at lists.gemtalksystems.com
| http://lists.gemtalksystems.com/mailman/listinfo/glass
|