[Glass] [GS/SS Beta] Scale to multiple nodes sharing a DB?

[Norbert Hartl]

Am 15.11.2013 um 18:59 schrieb Dale K. Henrichs <dale.henrichs at gemtalksystems.com>:

> 
> 
> ----- Original Message -----
> | From: "BrunoBB" <smalltalk at adinet.com.uy>
> | To: glass at lists.gemtalksystems.com
> | Sent: Friday, November 15, 2013 9:17:13 AM
> | Subject: Re: [Glass] [GS/SS Beta] Scale to multiple nodes sharing a DB?
> | 
> | Hi Norbert,
> | 
> | "distributing an unpartitioned memory space over multiple hosts and
> | modifying state only within transactions has severe limits."
> | 
> | Can you elaborate on this sentence ? What do you have in mind ?
> | 
> | -unpartitioned- can have multiple interpretations. Unpartitioned at
> | hardware
> | level, at logical level ?
> | 
> | I think you are talking about Shared Page Cache,  a SPC in a host has
> | different objects that another SPC in other host (both attached to
> | the same
> | stone). In this case with your definition is partitioned or not ?
> 
> You are correct in surmising that the SPC does provide an additional level of partitioning that tends to organize references. GemStone also provides a clustering protocol for compacting object subgraphs on the pages ...
> | 
> | You will have a problem if your transactions are large in time (lot
> | of
> | changes without commits or aborts). You can solve this at application
> | level.
> 
> Our customers tend to towards larger and larger SPCs and more and more cpus on a single machine for the abosulte fastest performance ... As is always the case the fastest access paths are always memory to memory if you have to go over a wire or hit disk then the performance impact is not transparent ...
> 
> So in the end, Norbert is correct that there are limitations to what can be achieved by adding multiple machines as opposed to increasing the size of a given machine ... but in the end it does depend upon the characteristics of the application and the size of the working set required ….

Agreed. My statement was gemstone independent. There are things that need to be done. You need to distribute your objects in a way that finding the location of an object (and retrieving it) is fast. But still there is overhead while distributing and locating. For transactions a kind of lock or negotiation needs to be done to have fault-free transaction. And there is overhead for this, too. And these overheads are added with every added SPC. I would even assume that the overhead is non-linear so there is a maximum-not-too-big of instances you can have before the management overhead is multiple of your real data work. But you can organize your data in a way to mitigate the problem.  
So what I really wanted to say is: There are technical problems that exist regardless which technology you use. I think gemstone is really good in managing and distributing objects and assuring data consistency. But it is not alien technology so it is constrained by those general technical problems as well. 

Norbert