[Glass] Should I be worried about symbols lookup performance?

[Paul DeBruicker via Glass]

Could you MD5 the strings and store the md5's in the table 'rows' then have a
lookup dictionary somewhere that lets you go from the md5 to the original
string?







GLASS mailing list wrote
> Hi guys,
> 
> I am storing a lot of data internally in GemStone that I get from a third
> party lib. Right now I have 1MM objects but likely soon there will some
> more MM for sure. These objects are kind of "rows" from huge files I
> receive from this lib as if it were a "database". Anyway, these objects
> have a string code. These string code WON'T fit in 61 bits or so, so these
> will NOT be immediate objects.  However...each code is repeated in average
> 20 times. So..in 1MM rows, I could be storing 1MM strings, or... 50000
> symbols....
> 
> I expect this imported data to increase and increase everyday. In fact I
> am
> not even 100% sure the best solution is to store this inside GemStone, but
> that's a story for another day.
> 
> My question is...if I use symbols I will be saving (I guess) a lot of
> space, reducing a lot the number of objects, and likely the number of
> objects needed in memory (hence I hope I will need less memory/spc).   My
> only  worry is about the symbol lookup performance. I don't know how the
> "Symbol table" is implemented in GemStone.   From what I understand, when
> I
> CREATE a symbol I pay the lookup in the table but then my object reference
> that points to the new symbol will directly point to the symbol and not to
> the entry in the symbol table...so I don't have a new indirection each
> time
> I try to access my symbol instance, right?
> 
> However...if I get a very large table of symbols, I am afraid that ever
> single #asSymbol I do in my app (from any other use case..fully decouple
> from this one) would be slowed down.
> I cannot find AllSymbols dictionary and the deeper I could get to
> understand was #_existingWithAll:
> 
> I tried to do some bench and it seems the #asSymbol is still fast even
> with
> a much larger SymbolTable (or whatever GemStone equivalent).
> 
> This may be a tradeoff, but my gut feelings tell me that storing these as
> symbols is worth.
> 
> BTW... I guess I must put this to true: STN_SYMBOL_GC_ENABLED  also..
> 
> Do you have any suggestion or recommendation?
> 
> Thanks in advance,
> 
> 
> -- 
> Mariano
> http://marianopeck.wordpress.com
> 
> _______________________________________________
> Glass mailing list

> Glass at .gemtalksystems

> http://lists.gemtalksystems.com/mailman/listinfo/glass





--
View this message in context: http://forum.world.st/Glass-Should-I-be-worried-about-symbols-lookup-performance-tp4816387p4816452.html
Sent from the GLASS mailing list archive at Nabble.com.