[Glass] Is it possible to disable tranlogs for some gems?

Fri Feb 10 03:33:06 PST 2017

On Fri, Feb 10, 2017 at 2:26 AM, Paul DeBruicker via Glass <
glass at lists.gemtalksystems.com> wrote:

> As part of your bulk loading process can you add a step/check to clean up
> the
> unneeded tranlogs?
>
> e.g. use
>
> SystemRepository oldestLogFileIdForRecovery
>
>
> To determine which can be discarded after each CSV file is loaded and then
> delete the outdated ones.
>
>

I thought about that too. But, in my case, I have a N number of backups of
the stone, and I keep all necessary tranlogs for the oldest backup (via
LAST_NEEDED_TRANLOG=`$GEMSTONE/bin/copydbf -i $file 2>&1 | grep tranlog |
cut -d ' ' -f 13` ) so that I am sure I can reapply logs to any of the N
backups.  I guess this has a price.

To do what you say, I should always remove all logs up to
#oldestLogFileIdForRecovery rather than oldest backup. Right?

>
> Or tidy up at the end?  Other than spending then discarding the effort of
> making the tranlogs in the first place are there undesirable consequences
> to
> that?
>
>
The fact is that I hate spending GB on tranlogs (and probably writing the
tranlogs impact performance of these bulk load cron jobs) which are mostly
generated from data which I don't care to recover. If the stone would
crash, I would simply re-run the background jobs and load the data again.

>
>
>
> GLASS mailing list wrote
> > On Mon, Aug 31, 2015 at 12:32 PM, Dale Henrichs via Glass <
>
> > glass at .gemtalksystems
>
> >> wrote:
> >
> >> Yeah, you have to stop the stone, change configs, start stone, do bulk
> >> load, stop stone, reset configs and the restart stones ... should
> disable
> >> customer access during that time as well ... there is a not-tranlogged
> >> data
> >> feature but that is a case where persistent data is not tranlogged and
> is
> >> not recoverable on a crash, but in this case you lose the ability to
> >> recover the persistent state altogether on a crash ... which is probably
> >> not appropriate in your case .... I am hoping to get the not-tranlogged
> >> feature hooked up for Seaside3.2 and GemStone3.3, but that is a pipe
> >> dream
> >> at this point:)
> >>
> >
> >
> > Hi Dale,
> >
> > What is the status of this not-tranlogged data? My question is because I
> > have some cron jobs that run at night, some processing text files and
> > filling some data. These processes are generating 13GB of tranglos every
> > day..
> >
> > Thanks!
> >
> >
> >>
> >> Dale
> >>
> >>
> >> On 8/31/15 8:13 AM, Mariano Martinez Peck via Glass wrote:
> >>
> >> OK the admin guide proposes to like:
> >>
> >> STN_TRAN_LOG_DIRECTORIES = /dev/null, /dev/null;
> >> STN_TRAN_FULL_LOGGING = TRUE;
> >>
> >> But that still implies files modification and stone stop/start right? At
> >> least I do not need to make a full backup? mmm sounds like....they
> forgot
> >> to say that there?
> >>
> >>
> >>
> >> On Mon, Aug 31, 2015 at 12:04 PM, Mariano Martinez Peck <
> >> <
>
> > marianopeck@
>
> > >
>
> > marianopeck@
>
> >> wrote:
> >>
> >>> Hi guys,
> >>>
> >>> I almost sure the answer is no. But I would like to comment my scenario
> >>> since there could be other workarounds. I have some daily cron jobs
> that
> >>> run at night that do quite large commits. Basically, they read huge
> >>> files
> >>> and do a kind of bulk load into GemStone.
> >>>
> >>> Once of the things I notice is that for every day, I have like 4 log
> >>> files of 1GB each. This makes much more space usage on disk (which in
> my
> >>> case is importante) and also, I can imagine there is quite a
> performance
> >>> downgrade while having to write logs.
> >>>
> >>> So since this jobs run at night, with little or none user connect, I am
> >>> fine with not writing logs while these jobs run. "I am fine" in the
> >>> sense
> >>> that if the system crashes while doing so, I am ok to recover from
> >>> previous
> >>> checkpoint and lost a few transactions if any.
> >>>
> >>> However, it seems it is not possible to disable tranlogs for particular
> >>> gems, right? In addition, I did not found an easy way to directly
> >>> enable/disable logs of the stone. To disable, the admin guide says I
> >>> must
> >>> run a full backup, then change file, then stop stone, start again
> >>> etc...So
> >>> it seems quite complicated...
> >>>
> >>> Is there anything that could help me here?
> >>>
> >>> I am using GemStone 3.1.0.6
> >>>
> >>> Thanks in advance,
> >>>
> >>> --
> >>> Mariano
> >>> http://marianopeck.wordpress.com
> >>>
> >>
> >>
> >>
> >> --
> >> Mariano
> >> http://marianopeck.wordpress.com
> >>
> >>
> >> _______________________________________________
> >> Glass mailing
>
> > listGlass at .gemtalksystems
>
> > ://lists.gemtalksystems.com/mailman/listinfo/glass
> >>
> >>
> >>
> >> _______________________________________________
> >> Glass mailing list
> >>
>
> > Glass at .gemtalksystems
>
> >> http://lists.gemtalksystems.com/mailman/listinfo/glass
> >>
> >>
> >
> >
> > --
> > Mariano
> > http://marianopeck.wordpress.com
> >
> > _______________________________________________
> > Glass mailing list
>
> > Glass at .gemtalksystems
>
> > http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
>
>
> --
> View this message in context: http://forum.world.st/Is-it-
> possible-to-disable-tranlogs-for-some-gems-tp4847185p4933756.html
> Sent from the GLASS mailing list archive at Nabble.com.
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>

-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170210/fa4ddf4c/attachment-0001.html>