[Glass] Large collection and common practice

[Paul Baumann via Glass]

Hi Bruno,

Multiple sessions can feed an RC collection with reduced commit conflicts,
but you don't always want to keep the objects in the RC collection. One
common technique is to have a manager session dedicated to moving objects
from RC collections into collections that can be accessed more efficiently.
Design so that the manager is the only session that will be updating the
collections (so that commit conflicts will not happen). The manager session
can do polling for new items and you can add gem-to-gem signaling to wake
the manager for more timely responses. The challenges with this kind of
design are related to update timing between sessions. The process involves
a commit to add to the RC collection, an abort for the manager session to
see the objects, a commit by the manager to update the root collection
(with RC collection removal, RcQueues are usually used BTW), and an abort
by the original session if it needs to see the indexed item was added to
the root collection. If there is timing sensitivity with this data then
you'll likely resort to searching first in your indexeded collection and
then also reviewing objects still in the queue waiting for the manager to
process them.

A variation of the manager session technique is to send data to the manager
session without doing a commit, this might be through communication between
gems or by using session-specific file updates that the manager gem reads.
Gem-to-gem signaling can be added to this approach later too if you need to
improve timing. This variation can avoid the intermediate commit, but you'd
still may need to #continueTransaction to see what the manager session
updated.

I wonder what kind of indexing you would need besides ID. If you don't need
to query for anything other than ID then a dictionary would be fine with
the ID as key. A dictionary can even use a key that is a custom object that
redefines equality and hash from attributes of what is searched for. Merkle
tree hashes might also be used as a way to test if some attribute is
contained, but that is a bit advanced to go into. Another advanced item
that I once implemented was a custom Dictionary where the key was derived
from the value by behavior (it was more efficient because it avoided the
cost of Association creation). So many cool tricks, I loved working with
GS/S.

Paul Baumann



On Tue, Jan 3, 2017 at 2:39 PM, Mariano Martinez Peck via Glass <
glass at lists.gemtalksystems.com> wrote:

>
> On Tue, Jan 3, 2017 at 3:14 PM, BrunoBB via Glass <
> glass at lists.gemtalksystems.com> wrote:
>
>> Hi All,
>>
>> I have a lot RcKeyValueDictionary where the key is the id of the object
>> and
>> the value is the object itself.
>> This id once assigned it does NOT change, so far so good :)
>>
>> The RcKeyValueDictionary is used intensively to add and remove objects (in
>> my case OrbeonFormInstance). The dictionary is very useful because the key
>> is always given as parameter.
>>
>> Also there are searchs by specific inst var of OrbeonFormInstance class
>> (like username,group, createdTime and so on).
>>
>> My problem is that i can NOT create an index on aRcKeyValueDictionary.
>> So which is the commom practice in these cases:
>> 1- Change the RcKeyValueDictionary to be an UnorderedCollection ?
>> 2- Add a new instance variable to the class that holds the
>> RcKeyValueDictionary and this new variable to be anUnorderedCollection ?
>>
>> 1) This will complicate my direct searchs using the ID.
>> 2) Extra computation when adding and removing objects (now there 2
>> collections to maintain)
>>
>> The general question will be something like:
>> When Dictionaries are very suitable to store large quantity of objects but
>> indexes are also needed which solution should be implemented ?
>>
>
>
> Assuming you do need or get benefits from the RC flavor (else it brings
> unnecessary overhead), then quickly analyzing the situation (until GemStone
> have indexed and rc-flavor Dictionary impl), I think I would use a
> RcIdentityBag. I would create a identity index for #id , and yes, you will
> have to modify your code that access the dict, to know detect on the
> collection using the identity index of ID.
>
> But...I am sure someone more experienced will come with a better approach!
>
> Cheers,
>
>
>
>> regards
>> bruno
>>
>>
>>
>> --
>> View this message in context: http://forum.world.st/Large-co
>> llection-and-common-practice-tp4928607.html
>> Sent from the GLASS mailing list archive at Nabble.com.
>> _______________________________________________
>> Glass mailing list
>> Glass at lists.gemtalksystems.com
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170103/e492f8a2/attachment.html>