[Glass] Backups, GC and removing translogs

[Dale Henrichs]

Mariano,

I think this sounds fine overall ... just a few minor comments in-lined


On Mon, Aug 11, 2014 at 1:33 PM, Mariano Martinez Peck <
marianopeck at gmail.com> wrote:

> Hi guys,
>
> I want to know what do you think about the following scenario. I want to
> start making some cleaning for my gemstones. After reading several emails
> etc I came to the following steps that I would like to run DAILY:
>
> 1) Stop all seaside gems (including maintainance etc)
>
> 2) Perform MFC and reclaim:
>
> [
> System beginTransaction.
>  SystemRepository markForCollection.
> SystemRepository reclaimAll.
> System commitTransaction.
>
> ] on: Halt,  Warning do: [:ex | ex resume]
>
> This step is not mandatory for me since I do run GLASS maintainance VM
> which does a MFC every hour...but...it might be a good idea anyway.
>

If you are going to run an mfc once a day, then I would suggest that you
can probably get away with eliminating the hourly mfc and run on epoch gc
instead. The mfc  introduces excessive disk i/o as  every page (with live
objects) is dragged through the SPC and at the end, you will need to pull
your "standard" working set of pages in off of disk..

>
> 3) Perform backup :
>
> SystemRepository startNewLog.
> SystemRepository fullBackupCompressedTo:
> '/opt/somewhere/backup-dateXXX.dbf.gz'.
>

It's a good idea to validate your backup after the backup is completed.
Take a look at the backup script that I use up on SS3[1] for an example
using copydbf. This at least avoids issues with file level corruption. I
believe that some folks do a restore from backup of the newly created
backup file along with some validation which covers any logical corruption.
Whether or not you go this far really depends upon how critical the backups
are ...

For super critical backups you will want to arrange for off-site storage of
backups and tranlogs.

In the backup script you will see that I force the creation of a new
tranlog right before the backup ... Frankly I forget why I put that trick
in there:), but the idea is that you will avoid having monster tranlogs
lying around ...

[1]
https://github.com/glassdb/webEditionHome/blob/master/bin/startGemStoneBackup.sh#L65-83

>
> 4) Clean:
>
> Now...from what I understand, this script:
> https://github.com/noha/stone-creator/blob/master/bin/delete-old-tranlogs.sh
>  does:
>
> LAST_NEEDED_TRANLOG=`$COPYDBF_EXE -i $FILE 2>&1 | grep tranlog | cut -d '
> ' -f 13`
>
> I have just checked copydbf and indeed, seems to be answer the same as
> #oldestLogFileIdForRecovery. So it seems safe to use this tool, correct?
>

I don't know offhand, but if Norbert put it in his script, then I trust it:)


> However...from what I understand..the critical part there is the $FILE
> which should be the last backup (in my example the latest in
> /opt/somewhere/). Is this correct?
>

If you are keeping several backups around you need to keep the tranlogs
that apply to the oldest backup that you are keeping... Using the latest
backup file will indeed give you the list of old and new tranlogs needed
for the latest, but there may be a tranlog or two that are still needed for
the oldest backup that you are keeping.

<http://lists.gemtalksystems.com/mailman/listinfo/glass>Dale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20140811/0de0b4bd/attachment.html>