[Glass] taming growth during migration

[Otto Behrens via Glass]

Hi,

We're migrating a large number of objects in our systems in this release.
Our problem is that while this migration is running, the repository grows
too much and runs out of space.

We can just give it more space, but I feel it is a better solution to get
the garbage collector to clean up faster. So, if I'm not smoking my socks,
this boils down to tuning parameters so that the Gc keeps up with the
process that generates the garbage.

We did the following which helped a lot: we commit in batches while looping
over objects and after each commit, we monitor PagesNeedReclaimSize and
wait for it to drop below 250 before we continue with the next batch.

We played with the batch sizes. Too few objects in a batch increases the
commit rate so high it runs out of space very quickly. Too many creates a
larger number of objects to Gc in a epoch / reclaim. I found a batch size
that sortof works.

What I can figure out from the stats (using VSD) is that the
PagesNeedReclaimSize stays low.

After a restore, loaded new code, just before we start the migration
process:

SystemRepository fileSizeReport
   File size =       10334.00 Megabytes
   Space available = 55.67 Megabytes

Just after the stone stops allowings commits because of the free space
threshold (and the topaz session exits):

SystemRepository fileSizeReport
   Repository size = 14000.00 Megabytes
   Free Space =      2732.08 Megabytes

So, it appears as if there is enough free space to work with, but it does
not become available.

I see this in the reclaimgcgem log, and wondered if I should lower /
increase this parameter:
"Committing the transaction. Reason: #objsMovedPerCommitThreshold exceeded"

If you have any ideas on this, or if you need logs / statmonitor files or
something else, please let me know. Still investigating...

We are running GS 3.1.0.5.

Cheers
Otto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150205/755f10f2/attachment.html>