[Glass] gc settings

[Dale Henrichs]

Yeah a couple thousand was a very conservative figure :) ... depends upon
how much extra space you can afford in your repo ... at 4M (16k) pages,
that's 64G which might be bigger than a lot of GLASS data bases (not all,
though:)

Dale


On Fri, Feb 14, 2014 at 2:32 PM, Jon Paynter <kittle31 at gmail.com> wrote:

> Another option is to allocate extra time for extra aggressive cleanup of
> your shadow pages.  Then let the migration run all out for as long as it
> needs.  When its done, kick everyone out of the stone and spin up a bunch
> of reclaim gems.  Like Dale said, they quite fast.
>
> Im not sure where the "couple thousand" number came from -- but we will
> only pause a migration for cleanup if the shadowpages get above 4million.
>
>
> On Fri, Feb 14, 2014 at 1:35 PM, Otto Behrens <otto at finworks.biz> wrote:
>
>> Thanks a lot. Trying it out.
>>
>> On Fri, Feb 14, 2014 at 9:44 PM, Dale Henrichs
>> <dale.henrichs at gemtalksystems.com> wrote:
>> > Otto,
>> >
>> > I talked this over with one of the guys here and he suggests that the
>> trick
>> > is to not create shadow pages faster than the gc gem can handle them
>> ... the
>> > gcgems are pretty aggreessive by default ...
>> >
>> > He suggests that you  commit frequently and the  monitor the
>> > PagesNeedsReclaiming stat in your migration loop and when it gets above
>> a
>> > certain threshold (a couple thousand pages?) stop migrating and go into
>> an
>> > "idle loop" where you abort (to avoid CR backlog and allow pages to run
>> > through their normal life cycle) and check for the stat to drop below
>> > another threshold (250 pages? once it's below your threshold get going
>> again
>> > ...
>> >
>> > regarding the epoch gc. the become operation does not itself generate a
>> lot
>> > of dead if you dow a become between a persistent object and a temp
>> object
>> > and commit, the temp object will be written as the persist object and no
>> > garbage will be produced ... you will however create shadowpages as the
>> new
>> > versions of the objects are copied to new pages (page reclaim is why the
>> > gcgems get revved up) ... now if you are creating short-lived persist
>> > objects as some other part of your  migration process then an epoch will
>> > help keep those dead boys in check ...
>> >
>> > Dale
>> >
>> >
>> > On Fri, Feb 14, 2014 at 4:07 AM, Otto Behrens <otto at finworks.biz>
>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> Do you have a set of GcUser settings that we can use for aggressive
>> >> garbage collection during a migration process? We would like to do the
>> >> garbage collection (epoch and reclaim) while the migration process is
>> >> generating a lot of dead stuff. We don't mind if the migration takes a
>> >> bit longer. We do mind if the migration generates a lot of free space
>> >> in the extent.
>> >>
>> >> Thanks
>> >> Otto
>> >> _______________________________________________
>> >> Glass mailing list
>> >> Glass at lists.gemtalksystems.com
>> >> http://lists.gemtalksystems.com/mailman/listinfo/glass
>> >
>> >
>> _______________________________________________
>> Glass mailing list
>> Glass at lists.gemtalksystems.com
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20140214/bccadac9/attachment.html>