[Glass] Files and UTF8 but not using String #encodeAsUTF8

[Johan Brichau via Glass]

It’s in the PharoCompatibility project on github.

Though, I must say it merits some love.
It works (using it for years already) but we have sometimes done some hacks to make it work with the Stream classes of GS. Or at least that’s what I remember right off the top of my head.

But I have not used this with SIXX. My last experience with SIXX was when we moved the Vooruit database from Pharo+GOODS to Gemstone and I used the commitOnAlmostOutOfMemory trick.

Johan

On 26 Feb 2015, at 18:46, Dale Henrichs <dale.henrichs at gemtalksystems.com> wrote:
> 
> What package is that class located in?
> 
> Dale
> 
> On 2/26/15 9:42 AM, Mariano Martinez Peck wrote:
>> 
>> 
>> On Thu, Feb 26, 2015 at 2:31 PM, Dale Henrichs via Glass <glass at lists.gemtalksystems.com <mailto:glass at lists.gemtalksystems.com>> wrote:
>> Mariano,
>> 
>> 
>> Hmmm, this is a bit of a sticky wicket ....
>> 
>> I'm afraid that the best way to solve this on is make a major change to SIXX and force all output and input to be utf8 ... but if you are using SIXX to move data between pharo and gemstone, then you'll have to make sure thatSIXX on the pharo size will properly decode utf8 ...
>> 
>> Alternatively, we could try porting the the whole TextConverter scheme to GemStone ...
>> 
>> 
>> Hi Dale,
>> 
>> This is already ported, Johan did it. I just don't know how to use the MultiByteBinaryOrTextStream (to which I can set a #converter:) together with a GsFile backend.. but i guess Johan did something because I cannot imagine he did everything in memory, right?
>> 
>>  
>> I suppose it's about time we did something in this area ...
>> 
>> Dale
>> 
>> 
>> 
>> On 2/26/15 7:03 AM, Mariano Martinez Peck via Glass wrote:
>>> Hi guys,
>>> 
>>> I am trying to implement the solution provided by Dale for exporting and importing large objects with SIXX: https://github.com/glassdb/SIXX?files=1 <https://github.com/glassdb/SIXX?files=1>
>>> 
>>> In his example he does:
>>> 
>>>  strm := WriteStream on: String new.
>>>  #( 1 2 3) sixxOn: strm persistentRoot: (UserGlobals at: #'MY_SIXX_ROOT_ARRAY')
>>> 
>>> That stream 'strm' is in memory. I need files. And I want those files to be encoded with UTF8. In addition, in my experience, I have been trying to use GsFile as much as possible since it was way faster than other classes when I tested it. So...so far I was using the following approach to write a UTF8 file:
>>> 
>>>  file := GsFile openWrite: aFilename.
>>>  file nextPutAll: aString encodeAsUTF8.
>>> 
>>> However, I cannot use that approach in the SIXX scenario. Why? Because I cannot easily hook in the parts where sixx gets the string of an object and writes it to the stream. So I kind of need to create the File stream with UTF8 from the beginning. 
>>> 
>>> I do have UT8TextConverter, but GsFile dnu #converter:. I tried:
>>> 
>>> | stream |
>>>  stream := MultiByteBinaryOrTextStream on: (GsFile openWrite: aFilename).
>>> 
>>>                                 stream converter: UTF8TextConverter new. 
>>> 
>>>                                 stream text. 
>>>   
>>>                               MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [
>>>     
>>>                               UserGlobals at: #'MY_SIXX_ROOT_ARRAY' put: Array new.
>>>      
>>>                                  #( 1 2 3) sixxOn: stream persistentRoot: (UserGlobals at: #'MY_SIXX_ROOT_ARRAY')
>>>   
>>>                               ].
>>>   
>>>                               stream close.
>>> 
>>> But it doesn't work. Ok..I did see GsFile >> contentsAsUtf8   so I could write all the file first, then grab the contents as UT8 and then do what I did above (a new file doing #nextPutAll: of the UTF8). But...since I am doing all this code because the object graph I am trying to serialize is big I am afraid I will run out of memory while trying to have all the contents as UTF8. So I would really like the "streaming" possibility. 
>>> 
>>> Any ideas how can I do that?
>>> 
>>> Thanks, 
>>> 
>>> 
>>> 
>>> -- 
>>> Mariano
>>> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
>>> 
>>> 
>>> _______________________________________________
>>> Glass mailing list
>>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>>> http://lists.gemtalksystems.com/mailman/listinfo/glass <http://lists.gemtalksystems.com/mailman/listinfo/glass>
>> 
>> 
>> _______________________________________________
>> Glass mailing list
>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>> http://lists.gemtalksystems.com/mailman/listinfo/glass <http://lists.gemtalksystems.com/mailman/listinfo/glass>
>> 
>> 
>> 
>> 
>> -- 
>> Mariano
>> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150226/d5d27a59/attachment-0001.html>