[Glass] Files and UTF8 but not using String #encodeAsUTF8

[Dale Henrichs via Glass]

What package is that class located in?

Dale

On 2/26/15 9:42 AM, Mariano Martinez Peck wrote:
>
>
> On Thu, Feb 26, 2015 at 2:31 PM, Dale Henrichs via Glass 
> <glass at lists.gemtalksystems.com 
> <mailto:glass at lists.gemtalksystems.com>> wrote:
>
>     Mariano,
>
>
>     Hmmm, this is a bit of a sticky wicket ....
>
>     I'm afraid that the best way to solve this on is make a major
>     change to SIXX and force all output and input to be utf8 ... but
>     if you are using SIXX to move data between pharo and gemstone,
>     then you'll have to make sure thatSIXX on the pharo size will
>     properly decode utf8 ...
>
>     Alternatively, we could try porting the the whole TextConverter
>     scheme to GemStone ...
>
>
> Hi Dale,
>
> This is already ported, Johan did it. I just don't know how to use the 
> MultiByteBinaryOrTextStream (to which I can set a #converter:) 
> together with a GsFile backend.. but i guess Johan did something 
> because I cannot imagine he did everything in memory, right?
>
>     I suppose it's about time we did something in this area ...
>
>     Dale
>
>
>
>     On 2/26/15 7:03 AM, Mariano Martinez Peck via Glass wrote:
>>     Hi guys,
>>
>>     I am trying to implement the solution provided by Dale for
>>     exporting and importing large objects with SIXX:
>>     https://github.com/glassdb/SIXX?files=1
>>
>>     In his example he does:
>>
>>      strm := WriteStream on: String new.
>>      #( 1 2 3) sixxOn: strm persistentRoot: (UserGlobals at:
>>     #'MY_SIXX_ROOT_ARRAY')
>>
>>     That stream 'strm' is in memory. I need files. And I want those
>>     files to be encoded with UTF8. In addition, in my experience, I
>>     have been trying to use GsFile as much as possible since it was
>>     way faster than other classes when I tested it. So...so far I was
>>     using the following approach to write a UTF8 file:
>>
>>     file := GsFile openWrite: aFilename.
>>     file nextPutAll: aString encodeAsUTF8.
>>
>>     However, I cannot use that approach in the SIXX scenario. Why?
>>     Because I cannot easily hook in the parts where sixx gets the
>>     string of an object and writes it to the stream. So I kind of
>>     need to create the File stream with UTF8 from the beginning.
>>
>>     I do have UT8TextConverter, but GsFile dnu #converter:. I tried:
>>
>>     | stream |
>>     *stream := MultiByteBinaryOrTextStream on: (GsFile openWrite:
>>     aFilename).*
>>     *stream converter: UTF8TextConverter new. *
>>     *stream text. *
>>     MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [
>>     UserGlobals at: #'MY_SIXX_ROOT_ARRAY' put: Array new.
>>      #( 1 2 3) sixxOn: stream persistentRoot: (UserGlobals at:
>>     #'MY_SIXX_ROOT_ARRAY')
>>     ].
>>     stream close.
>>
>>     But it doesn't work. Ok..I did see GsFile >> contentsAsUtf8   so
>>     I could write all the file first, then grab the contents as UT8
>>     and then do what I did above (a new file doing #nextPutAll: of
>>     the UTF8). But...since I am doing all this code because the
>>     object graph I am trying to serialize is big I am afraid I will
>>     run out of memory while trying to have all the contents as UTF8.
>>     So I would really like the "streaming" possibility.
>>
>>     Any ideas how can I do that?
>>
>>     Thanks,
>>
>>
>>
>>     -- 
>>     Mariano
>>     http://marianopeck.wordpress.com
>>
>>
>>     _______________________________________________
>>     Glass mailing list
>>     Glass at lists.gemtalksystems.com  <mailto:Glass at lists.gemtalksystems.com>
>>     http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>     _______________________________________________
>     Glass mailing list
>     Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>     http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
>
> -- 
> Mariano
> http://marianopeck.wordpress.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150226/ed1a0439/attachment-0001.html>