[Glass] Files and UTF8 but not using String #encodeAsUTF8

[Dale Henrichs via Glass]

Mariano,


Hmmm, this is a bit of a sticky wicket ....

I'm afraid that the best way to solve this on is make a major change to 
SIXX and force all output and input to be utf8 ... but if you are using 
SIXX to move data between pharo and gemstone, then you'll have to make 
sure thatSIXX on the pharo size will properly decode utf8 ...

Alternatively, we could try porting the the whole TextConverter scheme 
to GemStone ...

I suppose it's about time we did something in this area ...

Dale


On 2/26/15 7:03 AM, Mariano Martinez Peck via Glass wrote:
> Hi guys,
>
> I am trying to implement the solution provided by Dale for exporting 
> and importing large objects with SIXX: 
> https://github.com/glassdb/SIXX?files=1
>
> In his example he does:
>
>  strm := WriteStream on: String new.
>  #( 1 2 3) sixxOn: strm persistentRoot: (UserGlobals at: 
> #'MY_SIXX_ROOT_ARRAY')
>
> That stream 'strm' is in memory. I need files. And I want those files 
> to be encoded with UTF8. In addition, in my experience, I have been 
> trying to use GsFile as much as possible since it was way faster than 
> other classes when I tested it. So...so far I was using the following 
> approach to write a UTF8 file:
>
> file := GsFile openWrite: aFilename.
> file nextPutAll: aString encodeAsUTF8.
>
> However, I cannot use that approach in the SIXX scenario. Why? Because 
> I cannot easily hook in the parts where sixx gets the string of an 
> object and writes it to the stream. So I kind of need to create the 
> File stream with UTF8 from the beginning.
>
> I do have UT8TextConverter, but GsFile dnu #converter:. I tried:
>
> | stream |
> *stream := MultiByteBinaryOrTextStream on: (GsFile openWrite: aFilename).*
> *stream converter: UTF8TextConverter new. *
> *stream text. *
> MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [
> UserGlobals at: #'MY_SIXX_ROOT_ARRAY' put: Array new.
>  #( 1 2 3) sixxOn: stream persistentRoot: (UserGlobals at: 
> #'MY_SIXX_ROOT_ARRAY')
> ].
> stream close.
>
> But it doesn't work. Ok..I did see GsFile >> contentsAsUtf8   so I 
> could write all the file first, then grab the contents as UT8 and then 
> do what I did above (a new file doing #nextPutAll: of the UTF8). 
> But...since I am doing all this code because the object graph I am 
> trying to serialize is big I am afraid I will run out of memory while 
> trying to have all the contents as UTF8. So I would really like the 
> "streaming" possibility.
>
> Any ideas how can I do that?
>
> Thanks,
>
>
>
> -- 
> Mariano
> http://marianopeck.wordpress.com
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150226/33bf4c01/attachment-0001.html>