[Glass] Processing large text files / statmonitor tips

[Hernán Morales Durand]

El 09/12/2013 16:32, Mariano Martinez Peck escribió:
> Hi guys,
>
> I am doing a quick bench to see how GemStone behaves for some use-cases
> I have in which lots of large csv (or similar text based) files are read
> and processed. Notice that so far this read is NOT a bulk load (I am not
> persisting this data in GemStone..I am just processing it).
>

It is one-time processing or data will be re-used?

> I am trying to see how can I make it a bit faster (I am sure I can make
> it).
> So...first question...is there anything I could do in FileStream and
> friends that could have a big impact in the performance of reading text
> files?
>

Can you post CSV details? like file size and number of lines?

Hernán

> I used ProfMonitor to see which places of my code were the bottleneck,
> and of course, the files reading is most of it. So I wonder if there is
> anything I could do in the Gem or Stone configuration parameters?
>
> Finally, I run statmonitor and I have the vsd file. But since I don't
> know much so far, I wanted to ask if there were some key statistics I
> should look at for my usecase.
>
> Thanks a lot in advance,
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>