[Glass] Files and UTF8 but not using String #encodeAsUTF8

[Mariano Martinez Peck via Glass]

On Thu, Feb 26, 2015 at 2:31 PM, Dale Henrichs via Glass <
glass at lists.gemtalksystems.com> wrote:

>  Mariano,
>
>
> Hmmm, this is a bit of a sticky wicket ....
>
> I'm afraid that the best way to solve this on is make a major change to
> SIXX and force all output and input to be utf8 ... but if you are using
> SIXX to move data between pharo and gemstone, then you'll have to make sure
> thatSIXX on the pharo size will properly decode utf8 ...
>
> Alternatively, we could try porting the the whole TextConverter scheme to
> GemStone ...
>
>
Hi Dale,

This is already ported, Johan did it. I just don't know how to use the
MultiByteBinaryOrTextStream (to which I can set a #converter:) together
with a GsFile backend.. but i guess Johan did something because I cannot
imagine he did everything in memory, right?



> I suppose it's about time we did something in this area ...
>
> Dale
>
>
>
> On 2/26/15 7:03 AM, Mariano Martinez Peck via Glass wrote:
>
> Hi guys,
>
>  I am trying to implement the solution provided by Dale for exporting and
> importing large objects with SIXX: https://github.com/glassdb/SIXX?files=1
>
>  In his example he does:
>
>   strm := WriteStream on: String new.
>  #( 1 2 3) sixxOn: strm persistentRoot: (UserGlobals at:
> #'MY_SIXX_ROOT_ARRAY')
>
>  That stream 'strm' is in memory. I need files. And I want those files to
> be encoded with UTF8. In addition, in my experience, I have been trying to
> use GsFile as much as possible since it was way faster than other classes
> when I tested it. So...so far I was using the following approach to write a
> UTF8 file:
>
>   file := GsFile openWrite: aFilename.
>  file nextPutAll: aString encodeAsUTF8.
>
>  However, I cannot use that approach in the SIXX scenario. Why? Because I
> cannot easily hook in the parts where sixx gets the string of an object and
> writes it to the stream. So I kind of need to create the File stream with
> UTF8 from the beginning.
>
>  I do have UT8TextConverter, but GsFile dnu #converter:. I tried:
>
>  | stream |
>  *stream := MultiByteBinaryOrTextStream on: (GsFile openWrite:
> aFilename).*
> * stream converter: UTF8TextConverter new. *
> * stream text. *
>   MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [
>     UserGlobals at: #'MY_SIXX_ROOT_ARRAY' put: Array new.
>         #( 1 2 3) sixxOn: stream persistentRoot: (UserGlobals at:
> #'MY_SIXX_ROOT_ARRAY')
>   ].
>   stream close.
>
>  But it doesn't work. Ok..I did see GsFile >> contentsAsUtf8   so I could
> write all the file first, then grab the contents as UT8 and then do what I
> did above (a new file doing #nextPutAll: of the UTF8). But...since I am
> doing all this code because the object graph I am trying to serialize is
> big I am afraid I will run out of memory while trying to have all the
> contents as UTF8. So I would really like the "streaming" possibility.
>
>  Any ideas how can I do that?
>
>  Thanks,
>
>
>
>  --
> Mariano
> http://marianopeck.wordpress.com
>
>
> _______________________________________________
> Glass mailing listGlass at lists.gemtalksystems.comhttp://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>


-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150226/bfa6e237/attachment.html>