[Glass] taming growth during migration

[Bill Erickson via Glass]

Otto,

I just dealt with this same problem from another customer in Tech Support.

One thing that is essential is that you keep your commit record backlog
down as much as possible.  Make sure you don't have any sessions during
this time period that are in transaction and not periodically
committing/aborting, as this can cause serious DB growth using space to
hang onto the commit records.  These also fill up the cache, flushing out
needed data pages and slowing down other sessions as they will need to
re-load needed pages back into the cache, which tends to exacerbate the
problem.

Regards,
------------------------------------------------------------------------
Bill Erickson
GemTalk Systems Engineering
15220 NW Greenbrier Parkway #240, Beaverton OR 97006
------------------------------------------------------------------------

On Thu, Feb 5, 2015 at 5:55 AM, Otto Behrens via Glass <
glass at lists.gemtalksystems.com> wrote:

> Hi,
>
> We're migrating a large number of objects in our systems in this release.
> Our problem is that while this migration is running, the repository grows
> too much and runs out of space.
>
> We can just give it more space, but I feel it is a better solution to get
> the garbage collector to clean up faster. So, if I'm not smoking my socks,
> this boils down to tuning parameters so that the Gc keeps up with the
> process that generates the garbage.
>
> We did the following which helped a lot: we commit in batches while
> looping over objects and after each commit, we monitor PagesNeedReclaimSize
> and wait for it to drop below 250 before we continue with the next batch.
>
> We played with the batch sizes. Too few objects in a batch increases the
> commit rate so high it runs out of space very quickly. Too many creates a
> larger number of objects to Gc in a epoch / reclaim. I found a batch size
> that sortof works.
>
> What I can figure out from the stats (using VSD) is that the
> PagesNeedReclaimSize stays low.
>
> After a restore, loaded new code, just before we start the migration
> process:
>
> SystemRepository fileSizeReport
>    File size =       10334.00 Megabytes
>    Space available = 55.67 Megabytes
>
> Just after the stone stops allowings commits because of the free space
> threshold (and the topaz session exits):
>
> SystemRepository fileSizeReport
>    Repository size = 14000.00 Megabytes
>    Free Space =      2732.08 Megabytes
>
> So, it appears as if there is enough free space to work with, but it does
> not become available.
>
> I see this in the reclaimgcgem log, and wondered if I should lower /
> increase this parameter:
> "Committing the transaction. Reason: #objsMovedPerCommitThreshold exceeded"
>
> If you have any ideas on this, or if you need logs / statmonitor files or
> something else, please let me know. Still investigating...
>
> We are running GS 3.1.0.5.
>
> Cheers
> Otto
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150205/ef3f9a09/attachment-0001.html>