[Glass] Backup procedure

[Dario Trussardi via Glass]

Ciao Mariano,

	thanks for sharing your work.

	I have some questions:   see below 

> Hi Dario,
> 
> Yes, I have to do that and another couple of things (like disabling monit so that it won't start my gems while they are down for GC)
> 
> Here is the full script. While it won't work out of the box for you, I think you can really get an idea.
> 
> Dale, you may want to take something from it too (in fact, this script was a fork of yours, but I added quite a few things).
> 
> I am not a bash expert, so it's far from perfect.
> 
> Cheers,
> 
> 
> 
> #!/bin/bash
> 
> # Actually I only need this for the gemstone.secret check..because the rest gets from the site source
> source /opt/gemstone/product/seaside/defSeaside
> 
> if [ -s $GEMSTONE/seaside/etc/gemstone.secret ]; then
>     . $GEMSTONE/seaside/etc/gemstone.secret
> else
>     echo 'Missing password file $GEMSTONE/seaside/etc/gemstone.secret'
>     exit 1
> fi
> 
> SiteSubfolder="$1"
> 
> # Requires a Sites subfolder password as a parameter
> if [ "a$1" = "a" ]; then
>         echo 'Missing argument <Sites subfolder>'
>         exit 1
> fi
> 
> source $MYAPP_SITES_DIRECTORY/$SiteSubfolder/gemstone/env
> 
> 
> date=`date +%Y%m%d_%H%M%S`
> backupfile=${GEMSTONE_DATADIR}/backups/${GEMSTONE_NAME}_backup_${date}.dbf.gz
> 
> su -m $GEMSTONE_USER -c "touch $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log"
> su -m $GEMSTONE_USER -c "touch $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log"
> 
> 
> echo "`date` ------------------------ Starting backup of ${GEMSTONE_NAME} ------------------------" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> 
> 
> if [ ! -d "${GEMSTONE_DATADIR}/backups" ]; then
> 	su -m $GEMSTONE_USER -c "mkdir -p '${GEMSTONE_DATADIR}/backups'"
> fi
> 
> 
> 
> echo "`date` Adding backup to object log" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> (ObjectLogEntry trace: 'BACKUP: begin ' object: '${backupfile}') addToLog.
> %
> commit
> logout
> quit
> "
> 
> 
> 
> echo "`date` Stopping monit"  >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> monit unmonitor -g ${GEMSTONE_NAME} >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
> 
> echo "`date` Stopping seaside gems" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> sh $APPLICATION_DIR/scripts/stopSeasideGems.sh
> 
> echo "`date` Performing #markForCollection and #reclaimAll" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> [
> [

	I don't know the next commands :

> MCRepositoryGroup default repositoriesDo: [:rep | rep flushCache ].
> MCDefinition clearInstances.
> MCMethodDefinition cachedDefinitions  removeKeys:
>     (MCMethodDefinition cachedDefinitions keys).
> MCMethodDefinition shutDown.
> MethodVersionHistory uniqueInstance cleanUp.

	But you run markForCollection every time ?

	I think to run markForCollection only at the end of a cycle ( week - month )  when i remove link of hold data 

> SystemRepository markForCollection.
> SystemRepository reclaimAll.
> 
> ] on: Halt,  Warning do: [:ex | ex resume]
> ] on: Error do: [:ex | ].
> %
> commit
> logout
> quit
> "
> 
> 
> 
> echo "`date` Start new tranlog and perform backup" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> run
> | id  |
> id := SystemRepository startNewLog.
> [ id < 0 ] whileTrue: [
>   System sleep: 1.
>   id := SystemRepository startNewLog ].
> SystemRepository fullBackupCompressedTo: '${backupfile}'
> %
> logout
> quit
> "
> 
> echo "`date` Start again Seaside Gems" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> sh $APPLICATION_DIR/scripts/startSeasideGems.sh
> sleep 10
> 
> echo "`date` Starting monit"  >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> monit monitor -g ${GEMSTONE_NAME} >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log  2>&1
> 
> 
> 
> 
> echo "`date` Writing object log" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> begin
> run
> (ObjectLogEntry trace: 'BACKUP: completed ' object: '${backupfile}') addToLog.
> %
> commit
> 
> logout
> quit
> "
> 
> 
> if [ $? -eq 0 ]
> then
>   echo "`date` Successful backup ... starting validation of backup" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>   echo "------------------------------------------------------------" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>   $GEMSTONE/bin/copydbf ${backupfile} /dev/null >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
>   if [ $? -eq 0 ]
>     then
>       echo "`date` Successful validation " >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>       echo "------------------------------------------------------------" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
>       echo "------------------------------------------------------------" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> 
> 
> 	  echo "`date` Delete old backups" >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> 	  $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/runTopazStringScript.sh $1 "
> 
> 	  run
> 	  | autoGeneratedBackups backupsToKeep |
> 	  autoGeneratedBackups := (FACompatibilityUtils current directoryFromPath: '${GEMSTONE_DATADIR}/backups/') faEntries
> 	  		select: [ :anEntry | ((anEntry faBasename beginsWith: '${GEMSTONE_NAME}_backup_')
> 	  			and: [ anEntry faBasename endsWith: '.dbf.gz' ])
> 	  			and: [anEntry faBasename size = '${GEMSTONE_NAME}_backup_${date}.dbf.gz' size ]
> 	  		].
> 	(FileStream oldFileNamed: '$GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log')
> 		setToEnd;
> 		nextPutAll: ('There are ', ((FACompatibilityUtils current directoryFromPath: '${GEMSTONE_DATADIR}/backups/') faEntries) size asString, ' entries in backup folder'); crlf;
> 		nextPutAll: ('There are ', autoGeneratedBackups size asString, ' autogenerated backups'); crlf;
> 		flush.
> 	  autoGeneratedBackups := autoGeneratedBackups asSortedCollection: [:entryA :entryB | entryA faBasename < entryB faBasename].
> 	  backupsToKeep := autoGeneratedBackups last: (7 min: autoGeneratedBackups size).
> 	  (autoGeneratedBackups reject: [ :each | backupsToKeep includes: each ]) do: [:anEntry |
> 	  	(FileStream oldFileNamed: '$GEMSTONE_LOGDIR/${GEMSTONE_NAME}_old_backups.log')
> 	  		setToEnd; nextPutAll: ('File to be deleted: ', anEntry faBasename); crlf; flush.
> 		anEntry faParentDirectory deleteFileNamed: anEntry faBasename.
> 		 ].
> 	  System performOnServer: 'ln -sf ''', backupsToKeep first fullName , ''' ''', backupsToKeep last faParentDirectory pathName, '/OldestBackup'''.
> %
> commit
> logout
> quit
> "
> 
	
	I don't understund what the system do in this steps.

> 
> 	  echo "`date` Delete old tranlogs file and only keep the needed ones for the oldest backup `readlink
> 	${GEMSTONE_DATADIR}/backups/OldestBackup` " >> $GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log
> 	  sh $MYAPP_GEMSTONE_ADDITIONS_SCRIPTS/delete-old-tranlogs.sh -d ${GEMSTONE_DATADIR} -f

	Can you sharing the	 delete-old-tranlogs.sh   code?

> 	${GEMSTONE_DATADIR}/backups/OldestBackup -g /opt/gemstone/product -r >>
> 	$GEMSTONE_LOGDIR/${GEMSTONE_NAME}_backup.log 2>&1
> 
> 	  echo "`date` ---------------------- Backup finihed -----------------------"
> 

	Thanks for any considerations.

		Dario