[Glass] out-of-mem removing elements from indexed RcIdentityBags

[Dale Henrichs via Glass]

Johan,

I don't think that there's much that you can do to reduce the space 
consumption.

The discrepancy in space consumption that you're seeing is due to the 
fact that in a BtreeNode, we use 4 slots per object and RcIdentityBags 
only takes 1 slot per object.

Right out of the gate, it takes 4x as many page-sized objects to store 
the same number of objects.

Since you are looking at each object participating in 6 indexes, the 
back of the envelope calculation says that it will take about 24x as 
many page-sized objects to store the same set of objects in BtreeNodes ...

So 5MB to 100MB TOC size is in the ballpark for "expected memory 
consumption".

If TOC is an issue, I can imagine that there might be ways to handle the 
deletion of objcts from collections where you defer the removal to a 
batch process that has plenty of TOC available ...

Dale

On 02/05/2015 11:06 PM, Johan Brichau via Glass wrote:
> Hi there,
>
> I am debugging an out-of-temporary-objectspace issue we ran into because we are removing objects from many different RcIdentityBags that each have 6 equality indexes defined on them in a single transaction. The amount of temp object space required for that is 100 MB. I am sure the issue is because of the indexes because when I execute the same operation after removing all the indexes, it requires only 5 MB of temp object space. When I check which instances are occupying most of the space, I get the *BtreeNode* family of classes rising to the top in bytesize.
>
> Though the manual mentions some things about performance when modifying indexed collections, I am surprised about the size required to execute this operation. So, is there a way to investigate why so many large nodes need to be loaded when removing? How do I interpret the results of _indexStatistics? Is there anything I can try to investigate?
>
> Thanks for any considerations
> Johan
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass