[Glass] gc settings

[Jon Paynter]

Another option is to allocate extra time for extra aggressive cleanup of
your shadow pages.  Then let the migration run all out for as long as it
needs.  When its done, kick everyone out of the stone and spin up a bunch
of reclaim gems.  Like Dale said, they quite fast.

Im not sure where the "couple thousand" number came from -- but we will
only pause a migration for cleanup if the shadowpages get above 4million.


On Fri, Feb 14, 2014 at 1:35 PM, Otto Behrens <otto at finworks.biz> wrote:

> Thanks a lot. Trying it out.
>
> On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
> <dale.henrichs at gemtalksystems.com> wrote:
> > Otto,
> >
> > I talked this over with one of the guys here and he suggests that the
> trick
> > is to not create shadow pages faster than the gc gem can handle them ...
> the
> > gcgems are pretty aggreessive by default ...
> >
> > He suggests that you  commit frequently and the  monitor the
> > PagesNeedsReclaiming stat in your migration loop and when it gets above a
> > certain threshold (a couple thousand pages?) stop migrating and go into
> an
> > "idle loop" where you abort (to avoid CR backlog and allow pages to run
> > through their normal life cycle) and check for the stat to drop below
> > another threshold (250 pages? once it's below your threshold get going
> again
> > ...
> >
> > regarding the epoch gc. the become operation does not itself generate a
> lot
> > of dead if you dow a become between a persistent object and a temp object
> > and commit, the temp object will be written as the persist object and no
> > garbage will be produced ... you will however create shadowpages as the
> new
> > versions of the objects are copied to new pages (page reclaim is why the
> > gcgems get revved up) ... now if you are creating short-lived persist
> > objects as some other part of your  migration process then an epoch will
> > help keep those dead boys in check ...
> >
> > Dale
> >
> >
> > On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <otto at finworks.biz> wrote:
> >>
> >> Hi,
> >>
> >> Do you have a set of GcUser settings that we can use for aggressive
> >> garbage collection during a migration process? We would like to do the
> >> garbage collection (epoch and reclaim) while the migration process is
> >> generating a lot of dead stuff. We don't mind if the migration takes a
> >> bit longer. We do mind if the migration generates a lot of free space
> >> in the extent.
> >>
> >> Thanks
> >> Otto
> >> _______________________________________________
> >> Glass mailing list
> >> Glass at lists.gemtalksystems.com
> >> http://lists.gemtalksystems.com/mailman/listinfo/glass
> >
> >
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20140214/376ace97/attachment-0001.html>