[Glass] Backup procedure
Mariano Martinez Peck via Glass
glass at lists.gemtalksystems.com
Mon Jun 8 10:12:23 PDT 2015
On Mon, Jun 8, 2015 at 2:04 PM, Johan Brichau <johan at yesplan.be> wrote:
> Hi Mariano,
>
> Why are you stopping the Seaside gems during an MFC?
> If this is because you’ve noticed that some ‘garbage’ is retained by the
> gems: a simple recycle (stop/start) does the job as well.
>
>
Hi johan,
Thanks for asking. Yes, that was one of my concerns. Good to know a
stop/start would work too.
> Once your repository grows larger, it will take a considerable amount of
> time to complete a full backup. You don’t want your site to be offline that
> long.
>
>
Indeed, and I was already thinking about that. At some point, I remembered
having a problem in which the GC would not run because they were waiting
for the "vote" of some seaside gems (at least that's what I thought). I
THINK this is related to GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 90; thingy.
Right?
So..if I make sure my seaside gems do have such flag, I am safe now? I am
using 3.1.0.6 and latest GLASS.
Thanks Johan for asking!
> Johan
>
> On 08 Jun 2015, at 17:00, Mariano Martinez Peck via Glass <
> glass at lists.gemtalksystems.com> wrote:
>
> Hi Dario,
>
> Yes, I have to do that and another couple of things (like disabling monit
> so that it won't start my gems while they are down for GC)
>
> Here is the full script. While it won't work out of the box for you, I
> think you can really get an idea.
>
> Dale, you may want to take something from it too (in fact, this script was
> a fork of yours, but I added quite a few things).
>
> I am not a bash expert, so it's far from perfect.
>
> Cheers,
>
>
>
> #!/bin/bash
>
> # Actually I only need this for the gemstone.secret check..because the
> rest gets from the site source
> source /opt/gemstone/product/seaside/defSeaside
>
> if [ -s $GEMSTONE/seaside/etc/gemstone.secret ]; then
> . $GEMSTONE/seaside/etc/gemstone.secret
> else
> echo 'Missing password file $GEMSTONE/seaside/etc/gemstone.secret'
> exit 1
> fi
>
> SiteSubfolder="$1"
>
> # Requires a Sites subfolder password as a parameter
> if [ "a$1" = "a" ]; then
> echo 'Missing argument <Sites subfolder>'
> exit 1
> fi
>
> source $MYAPP_SITES_DIRECTORY/$SiteSubfolder/gemstone/env
>
>
> date=`date +%Y%m%d_%H%M%S`
>
> backupfile=${GEMSTONE_DATADIR}/backups/${GEMSTONE_NAME}_backup_${date}.dbf.gz
>
> su -m $GEMSTONE_USER -c "touch
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log"
> su -m $GEMSTONE_USER -c "touch
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log"
>
>
> echo "`date` ------------------------ Starting backup of ${GEMSTONE_NAME}
> ------------------------" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>
>
> if [ ! -d "${GEMSTONE_DATADIR}/backups" ]; then
> su -m $GEMSTONE_USER -c "mkdir -p '${GEMSTONE_DATADIR}/backups'"
> fi
>
>
>
> echo "`date` Adding backup to object log" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> (ObjectLogEntry trace: 'BACKUP: begin ' object: '${backupfile}') addToLog.
> %
> commit
> logout
> quit
> "
>
>
>
> echo "`date` Stopping monit" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> monit unmonitor -g ${GEMSTONE_NAME} >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
>
> echo "`date` Stopping seaside gems" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> sh $APPLICATION_DIR/scripts/stopSeasideGems.sh
>
> echo "`date` Performing #markForCollection and #reclaimAll" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> [
> [
> MCRepositoryGroup default repositoriesDo: [:rep | rep flushCache ].
> MCDefinition clearInstances.
> MCMethodDefinition cachedDefinitions removeKeys:
> (MCMethodDefinition cachedDefinitions keys).
> MCMethodDefinition shutDown.
> MethodVersionHistory uniqueInstance cleanUp.
> SystemRepository markForCollection.
> SystemRepository reclaimAll.
>
> ] on: Halt, Warning do: [:ex | ex resume]
> ] on: Error do: [:ex | ].
> %
> commit
> logout
> quit
> "
>
>
>
> echo "`date` Start new tranlog and perform backup" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> run
> | id |
> id := SystemRepository startNewLog.
> [ id < 0 ] whileTrue: [
> System sleep: 1.
> id := SystemRepository startNewLog ].
> SystemRepository fullBackupCompressedTo: '${backupfile}'
> %
> logout
> quit
> "
>
> echo "`date` Start again Seaside Gems" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> sh $APPLICATION_DIR/scripts/startSeasideGems.sh
> sleep 10
>
> echo "`date` Starting monit" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> monit monitor -g ${GEMSTONE_NAME} >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
>
>
>
>
> echo "`date` Writing object log" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> (ObjectLogEntry trace: 'BACKUP: completed ' object: '${backupfile}')
> addToLog.
> %
> commit
>
> logout
> quit
> "
>
>
> if [ $? -eq 0 ]
> then
> echo "`date` Successful backup ... starting validation of backup" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> echo "------------------------------------------------------------" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $GEMSTONE/bin/copydbf ${backupfile} /dev/null >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
> if [ $? -eq 0 ]
> then
> echo "`date` Successful validation " >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> echo "------------------------------------------------------------"
> >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> echo "------------------------------------------------------------"
> >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>
>
> echo "`date` Delete old backups" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
>
> run
> | autoGeneratedBackups backupsToKeep |
> autoGeneratedBackups := (FACompatibilityUtils current directoryFromPath:
> '${GEMSTONE_DATADIR}/backups/') faEntries
> select: [ :anEntry | ((anEntry faBasename beginsWith:
> '${GEMSTONE_NAME}_backup_')
> and: [ anEntry faBasename endsWith: '.dbf.gz' ])
> and: [anEntry faBasename size =
> '${GEMSTONE_NAME}_backup_${date}.dbf.gz' size ]
> ].
> (FileStream oldFileNamed:
> '$GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log')
> setToEnd;
> nextPutAll: ('There are ', ((FACompatibilityUtils current
> directoryFromPath: '${GEMSTONE_DATADIR}/backups/') faEntries) size
> asString, ' entries in backup folder'); crlf;
> nextPutAll: ('There are ', autoGeneratedBackups size asString, '
> autogenerated backups'); crlf;
> flush.
> autoGeneratedBackups := autoGeneratedBackups asSortedCollection:
> [:entryA :entryB | entryA faBasename < entryB faBasename].
> backupsToKeep := autoGeneratedBackups last: (7 min: autoGeneratedBackups
> size).
> (autoGeneratedBackups reject: [ :each | backupsToKeep includes: each ])
> do: [:anEntry |
> (FileStream oldFileNamed:
> '$GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log')
> setToEnd; nextPutAll: ('File to be deleted: ', anEntry faBasename);
> crlf; flush.
> anEntry faParentDirectory deleteFileNamed: anEntry faBasename.
> ].
> System performOnServer: 'ln -sf ''', backupsToKeep first fullName , '''
> ''', backupsToKeep last faParentDirectory pathName, '/OldestBackup'''.
> %
> commit
> logout
> quit
> "
>
>
> echo "`date` Delete old tranlogs file and only keep the needed ones for
> the oldest backup `readlink ${GEMSTONE_DATADIR}/backups/OldestBackup` " >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> sh $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/delete-old-tranlogs.sh -d
> ${GEMSTONE_DATADIR} -f ${GEMSTONE_DATADIR}/backups/OldestBackup -g
> /opt/gemstone/product -r >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> 2>&1
>
> echo "`date` ---------------------- Backup finihed
> -----------------------"
>
>
> exit 0
> else
> reason="Failed validatino"
> echo "`date` Failed validation" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> fi
> else
> reason="Failed backup"
> echo "`date` Failed backup" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> fi
>
>
>
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
>
> run
> 'For details about the failure, scan backward in log file to previous
> topaz session or copydbf sessions'
> %
> logout
> quit
>
> run
> (ObjectLogEntry fatal: 'BACKUP: failed ' object: '${reason}. See
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log for details') addToLog.
> %
> commit
> logout
> quit
>
> "
>
>
>
> echo "------------------------------------------------------------" >>
> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> exit 101 #failure
>
>
>
>
> On Mon, Jun 8, 2015 at 11:46 AM, Dario Trussardi via Glass <
> glass at lists.gemtalksystems.com> wrote:
>
>> Ciao,
>>
>> i need to setup procedure for backup my devKit glass environment.
>>
>> For now i use:
>>
>> https://github.com/glassdb/webEditionHome/blob/master/bin/startGemStoneBackup.sh
>>
>>
>>
>> This procedure do:
>> runs a GemStone full backup (compressed) and validates the resulting
>> backup file
>> It work fine and startNewLog before do the backup.
>>
>> Now i need to complete the backup.
>>
>> I think:
>>
>> 1) save the new backup into another system into local network ( named
>> backupSystem )
>>
>> 2) remove the old backup from the server and from the backupSystem
>>
>> 3) remove the old tranlog from the server
>> 4) remove other .... logs from the server
>>
>> Someone has already addressed this issue?
>>
>> The goal is to have a system which automatically manages all the step and
>> is ready for restoring in the break case.
>>
>> Thanks for considerations..
>>
>>
>> Dario
>>
>> _______________________________________________
>> Glass mailing list
>> Glass at lists.gemtalksystems.com
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
--
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150608/9f28768f/attachment-0001.html>
More information about the Glass
mailing list