[Glass] upgrade speed sanity check

[Dale Henrichs]

On 9/23/19 12:36 PM, PAUL DEBRUICKER via Glass wrote:
> Hi -
>
> I'm trying to upgrade a 8 GB and 30GB stone from 3.3.1 to 3.5 on a mac first (while troubleshooting my upgrade process) then possibly on ubuntu 18.04 if need be.  Both machines have NVMe SSD's with plenty of room and with plenty of RAM and CPU.
>
> Right now the extent and tranlogs are both on the same device, in the default GsDevKit_home structure.  On the mac I could move the tranlogs to an SDHC hard, and on the Ubuntu machine I could move them to a SATA drive.
>
> While upgrading the 8GB stone I'm seeing speeds inline with the drives published specs of around 250-400MB/s i/o speeds on the Mac.  And it has read 1.35TB of data and written 105GB of data in about 40 minutes.
>
> Right now I assume things are fine and going smoothly but because it might be taking longer than it should I wanted to check:
>
> 1. Should it take more than 40 minutes to upgrade a 8GB stone?
> 2. Should the 30GB stone (that has different objects/code/etc) take ~4x as long?
> 3. Is there any way to tell ahead of time how long an upgrade will take?

No ... which should answer questions 1 and 2 ... however, it does seem 
that it is taking a long time ...

What scripts are you using to do the upgrade? I'll assume that you are 
using $GS_HOME/bin/upgradeStone to do the upgrade ... if you are running 
different scripts that would be useful to know ...

Assuming upgradeStone, there should be an upgradeLog directory in the 
directory of the stone that you are upgrading to ... in that directory 
there should be a bunch of log files in that directory and an `ls -l` 
will give us a clue of where you are in the upgrade process.

Here's an example directory from and upgrade of a 500M 3.4.3 extent to 
3.5.0:

    rogue:upgradeLog>ls -altr
    total 11456
    drwxr-xr-x 11 dhenrich smalltalk     4096 Sep 23 13:23 ..
    -rw-r--r--  1 dhenrich smalltalk        0 Sep 23 13:23 topazerrors.log
    -rw-r--r--  1 dhenrich smalltalk 11286737 Sep 23 13:24 upgradeImage.out
    -rw-r--r--  1 dhenrich smalltalk       33 Sep 23 13:24 conversion.conf
    -rw-r--r--  1 dhenrich smalltalk     1254 Sep 23 13:24 installGsDevKit_upgrade.topaz
    -rw-r--r--  1 dhenrich smalltalk      253 Sep 23 13:24 createGsDevKit_upgrade.topaz
    -rw-r--r--  1 dhenrich smalltalk     3696 Sep 23 13:24 createGsDevKit_upgrade.out
    -rw-r--r--  1 dhenrich smalltalk     5527 Sep 23 13:24 installGsDevKit_upgrade_topaz.log
    -rw-r--r--  1 dhenrich smalltalk   112384 Sep 23 13:24 installGsDevKit_upgrade.out
    -rw-r--r--  1 dhenrich smalltalk      533 Sep 23 13:24 prepareGsDevKitImage_pragma_user.topaz
    -rw-r--r--  1 dhenrich smalltalk     6942 Sep 23 13:24 prepareGsDevKitImage_pragma_user.log
    -rw-r--r--  1 dhenrich smalltalk      462 Sep 23 13:24 prepareImage_pragma_systemuser.topaz
    -rw-r--r--  1 dhenrich smalltalk     6874 Sep 23 13:24 prepareImage_pragma_systemuser_topaz.log
    -rw-r--r--  1 dhenrich smalltalk      551 Sep 23 13:24 prepareGsDevKitImage.topaz
    -rw-r--r--  1 dhenrich smalltalk   118768 Sep 23 13:25 prepareGsDevKitImage.log
    -rw-r--r--  1 dhenrich smalltalk      525 Sep 23 13:25 cleanupGsDevKitImage.topaz
    drwxr-xr-x  2 dhenrich smalltalk     4096 Sep 23 13:25 .
    -rw-r--r--  1 dhenrich smalltalk   118379 Sep 23 13:25 upgradeSeasideImage.out
    -rw-r--r--  1 dhenrich smalltalk     6436 Sep 23 13:25 cleanupGsDevKitImage.log
    -rw-r--r--  1 dhenrich smalltalk    13452 Sep 23 13:25 topaz.out

sending me the newest log file and the`ls -l` list should be enough 
information for me to figure out where you are in the upgrade process 
and we can go from there ...

> 4. Are there any optimizations or a pre-upgrade checklist I should look or go through at that isn't covered in the install guide?
no .... I'm guessing that you are getting hit by "excessive migrations", 
and that you _might_ be using an older version of GsDevKit_home (the SHA 
of $GS_HOME commit should answer that question ... I can also tell by 
the list of files in the upgradeLog directory)... the "modern" upgrade 
from 3.3.1 to 3.5.0 should be avoiding most if not all of the migrations 
that would cause long upgrade times, but that will actually depend upon 
whether or not you are running with a "modern" version of GLASS/TODE in 
your 3.3.1 stone or not (the SHAs of the various projects you have 
loaded in your image will help figure that out) ... let's start by 
looking at the log files and go from there ...
> 5. Would moving the tranlogs to a separate disk prior to upgrading help substantially?
Maybe, but first we want to determine what's happening ... if the 
migrations cannot be avoided, it might make sense to arrange for the 
repository scans to use multi-threaded list instances, before resorting 
to movint tranlogs around, but I first want to see if you are getting 
unnecessary upgrades.


Dale

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gemtalksystems.com/mailman/private/glass/attachments/20190923/542972f5/attachment.htm>