[Glass] Fwd: Re: Fwd: Metacello Preview / tODE Questions

Dale Henrichs via Glass glass at lists.gemtalksystems.com
Fri Mar 16 16:01:31 PDT 2018 

inadvertantly used private email ... I think that my response may 
contain generally useful information...



-------- Forwarded Message --------
Subject: 	Re: Fwd: [Glass] Metacello Preview / tODE Questions
Date: 	Fri, 16 Mar 2018 14:42:38 -0700
From: 	Dale Henrichs <dale.henrichs at gemtalksystems.com>
To: 	Ken Treis <ken at miriamtech.com>



Sorry Ken,

I've been pre-ocuppied with Rowan work and taking care of my wife and 
grand kids ... My wife spent a couple of weeks in the hospital back in 
February and while she has been getting better every day, she's still 
not all the way back and it may take a few more months before she's back 
to normal, so my email queue has been backing up ...

I appreciate the fact that you pinged me ...

My answers are embedded below ...


On 03/16/2018 12:21 PM, Ken Treis wrote:
> Dale,
>
> Just bumping this to the top of your inbox. If you have a minute or 
> two to point me in the right direction, I'd sure appreciate it. Or… if 
> there's a better way to load tODE into a pre-tODE repository, I'm open 
> to just about anything.
>
> Thanks!
>
> --
> Ken Treis
> Miriam Technologies, Inc.
> (866) 652-2040 x221
>
>
>> Begin forwarded message:
>>
>> *From: *Ken Treis via Glass <glass at lists.gemtalksystems.com 
>> <mailto:glass at lists.gemtalksystems.com>>
>> *Subject: **Re: [Glass] Metacello Preview / tODE Questions*
>> *Date: *March 1, 2018 at 2:05:24 PM PST
>> *To: *Dale Henrichs <dale.henrichs at gemtalksystems.com 
>> <mailto:dale.henrichs at gemtalksystems.com>>
>> *Cc: *glass at lists.gemtalksystems.com 
>> <mailto:glass at lists.gemtalksystems.com>
>> *Reply-To: *Ken Treis <ken at miriamtech.com <mailto:ken at miriamtech.com>>
>>
>> Dale,
>>
>> Thank you very much for the explanations. That helps me a lot.
>>
>> The way I installed tODE was definitely unorthodox, and I know it's 
>> not fair to ask you to support non-standard installations, so don't 
>> feel pressured to fix my problems here. I'd just appreciate some 
>> general guidelines.
>>
>> We've got lots of infrastructure built around the old way of running 
>> GLASS apps; the stone runs from /opt/gemstone/product/seaside and we 
>> have a number of helper scripts running from there. So I didn't want 
>> to dive head-on into the tODE-managed stone world if I didn't have 
>> to, and right now I'm trying to take baby steps into it. If I could 
>> continue to run my stone and scripts from the old tree, that'd be 
>> ideal… but if that's asking for trouble, I can give up on my ideals.
Well it is not directly asking for trouble, but the full tODE 
implementation is dependent upon directory structure and a set of 
environment variables that reference key points in the directory 
structure ... so the issues you are seeing are more than likely caused 
by one or more of the environment variables used by GsDevKit_home and 
tODE not being correctly set or pointing at chunks of the directory 
structure that are not present ...

Frankly it was hard enough to put together the set of scripts and 
directory structure in the first place, without trying to wrap my head 
around trying to figure out how to help you figure out which pieces of 
the infrastructure you cannot live without ...

I'm afraid that trying to get tODE to be fully functional outside the 
environment it was designed to work within, could turn out to be a slow 
process of re-creating the GsDevKit_home environment within the context 
of your structure and scripts.

OTOH, you might be one env var away from success... so I'll go ahead and 
start the process of trying to help you get things working ...

>> To load tODE, I first ran my repo through a full upgrade from 3.2.15 
>> to 3.3.6 (upgradeImage, upgradeSeasideImage, etc). Then, I downloaded 
>> GsDevKit_home, ran the `installServer` script, and loaded the tODE 
>> code using Metacello. Here's the script I used:
>>
>>     #!/bin/bash -e
>>     #
>>     # Run as normal user with sudo privileges
>>     export GEMSTONE=/opt/gemstone/product
>>     export GEMSTONE_NAME=${GEMSTONE_NAME:-seaside}
>>
>>     # Install tODE
>>     export GS_HOME=/opt/gemstone/GsDevKit_home
>>     export GS_SHARED_REPO="${GS_HOME}/shared/repos"
>>     export
>>     GS_SHARED_REPO_TODE="filetree://${GS_SHARED_REPO}/tode/repository
>>     <filetree://$%7BGS_SHARED_REPO%7D/tode/repository>"
>>
>>     sudo -u gsadmin -E $GEMSTONE/bin/topaz -lq <<EOF
>>     set user DataCurator gems $GEMSTONE_NAME pass redacted
>>     login
>>     run
>>     GsDeployer bulkMigrate: [
>>       Metacello new
>>         baseline: 'FileTree';
>>         repository: 'filetree://${GS_SHARED_REPO}/filetree/repository
>>     <filetree://$%7BGS_SHARED_REPO%7D/filetree/repository>';
>>         get.
>>       Metacello new
>>         baseline: 'Metacello';
>>         repository:
>>     'filetree://${GS_SHARED_REPO}/metacello-work/repository
>>     <filetree://$%7BGS_SHARED_REPO%7D/metacello-work/repository>';
>>         get.
>>       Metacello new
>>         baseline: 'Metacello';
>>         repository:
>>     'filetree://${GS_SHARED_REPO}/metacello-work/repository
>>     <filetree://$%7BGS_SHARED_REPO%7D/metacello-work/repository>';
>>         load.
>>     ].
>>     %
>>     commit
>>     run
>>     | todeRepo |
>>     todeRepo := GsFile _expandEnvVariable: 'GS_SHARED_REPO_TODE'
>>     isClient: false.
>>     Transcript
>>       cr;
>>       show: '-----Install tODE using ', todeRepo.
>>     GsDeployer bulkMigrate: [
>>       Metacello new
>>         baseline: 'Tode';
>>         repository: todeRepo;
>>         get;
>>         lock;
>>         onConflict: [:ex | ex disallow];
>>         load: 'GemStone Dev' ].
>>     %
>>     commit
>>     quit
>>     EOF
>>
>>
>> That all went fine, and at that point I was able to connect to the 
>> stone with a tODE client on my laptop via an SSH tunnel for the 
>> NetLDI port. Here's my session config:
>>
>>     TDSessionDescription {
>>     #name : 'Production',
>>     #stoneHost : 'localhost',
>>     #stoneName : 'seaside',
>>     #gemHost : 'localhost',
>>     #netLDI : '',
>>     #netLDIPort : '50380',
>>     #gemTask : 'gemnetobject',
>>     #userId : 'DataCurator',
>>     #password : 'redacted',
>>     #osUserId : nil,
>>     #osPassword : nil,
>>     #dataDirectory : nil,
>>     #backupDirectory : '/opt/gemstone/product/seaside/data/backups/',
>>     #snapshotDirectory : '/opt/gemstone/product/seaside/data/snapshots/',
>>     #gemstoneVersion : '3.3.6',
>>     #gciLibraryName : nil,
>>     #adornmentColor : nil,
>>     #serverGitRoot : nil,
>>     #serverTodeRoot : '/opt/gemstone/GsDevKit_home',
>>     #authorId : nil
>>     }
>>
>>
>> But then, as you saw, I couldn't perform certain operations that 
>> needed to talk to the GsDevKit_home file tree. Is the #serverTodeRoot 
>> spec enough, or does the stone process on the server need 
>> tODE-specific environment variables, or are there some class vars 
>> that need to be set?
If you run `$GS_HOME/bin/utils/gsDevKitEnvVars`[1]  you will see that 
there are about 80 environment variables that are defined by 
GsDevKit_home (I've listed the set of env var values that are set for my 
environment in the attached file). As I scan the list it seems to me 
that most of those env vars are likely to be used at one point or 
another while using tODE ...

... but that's not all. It is not quite enough to just install the tODE 
code into your stone. There are a set of tODE scripts[2] that are also 
run after tODE is installed to create a directory structure. The code 
that creates the directory structure is implemented in the Pharo 
devKitCommandLine image and is triggered by running the todeLoad[3] 
script ... so in theory, the whole sequence of steps can be reverse 
engineered, but teasing out the bits that you need to run so that you 
can install tODE in your environment will be very tedious ...

As I spend the time to think about your "problem". I think that you 
would be better off install tODE into your stone by transplanting your 
extents into a GsDevKit_home installation, installing tODE using the 
standard GsDevKit_home scripts, then moving your extents back into your 
old structure and then make sure that when you run tODE all of the 
GsDevKit_home env vars are set pointing at the GsDevKit_home/tODE 
directory structure under GS_HOME....

This is the sequence of steps that I think you would need to do:

#
# 1. Install tode into your extent (this should be the upgraded extent 
before you installed tODE using your process)
#     When this step is done, the expected directory structure for tODE 
will have been created under $GS_HOME,
#     but you should be able to cp the extent back into your 
/opt/gemstone location as long as you make sure the
#     env vars are set up correctly
#
$GS_HOME/bin/createStone -s <path-to-your-extent-without-tode-installed> 
tode_stone <gs-version>
$GS_HOME/bin/stopStone tode_stone
cp $GS_HOME/server/stones/tode_stone/extents/extent0.dbf 
<path-to-your-extent-without-tode-installed>
#
# 2. customize your netldi startup script to pick up the GsDevKit_home 
env vars
#     If you look at the startNetldi script[4] you will see the 
following bash code is being run
#     This sequence ensures that the appropriate environment variables 
are defined for
#     GsDevKit_home and the stone --- both of which may be used by tODE:
source "${GS_HOME}/bin/defGsDevKit.env"
stonePath="$GS_SERVER_STONES/${stoneName}"
pushd "$stonePath" >& /dev/null
   source "$stonePath/stone.env"
popd >& /dev/null

So at this point I am pretty optimistic that this approach will get you 
going ...

Dale

[1] 
https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/utils/gsDevKitEnvVars
[2] 
https://github.com/GsDevKit/GsDevKit_home/tree/master/sys/default/client/tode-scripts
[3] https://github.com/GsDevKit/GsDevKit_home/blob/master/sys/bin/todeLoad
[4] https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/startNetldi

>>
>> I couldn't figure out how to copy a stack from the tODE debugger, but 
>> I took a screenshot (attached). If I'm reading it right, my root node 
>> doesn't have a "sys" child at all… ?
>>
>> --
>> Ken Treis
>> Miriam Technologies, Inc.
>> (866) 652-2040 x221
>>
>>
>>> On Feb 28, 2018, at 9:46 AM, Dale Henrichs via Glass 
>>> <glass at lists.gemtalksystems.com 
>>> <mailto:glass at lists.gemtalksystems.com>> wrote:
>>>
>>>
>>>
>>> On 02/27/2018 02:08 PM, Ken Treis via Glass wrote:
>>>> I have an old 3.2.15 repository full of important data, and I 
>>>> managed to (mostly) install tODE in it, but now I have lots of 
>>>> (probably newbie) questions about tODE and Metacello Preview.
>>>>
>>>> 1. Are ConfigurationOf* classes obsolete? If so, what replaces 
>>>> them? Most of the intro docs just talk about baselines, which now 
>>>> live in separate classes — and often separate packages — thus 
>>>> apparently disconnecting baseline specs from version specs.
>>> ConfigurationOf* are not obsolete. You see the emphasis on 
>>> BaselineOf, because I think that git/github is superior to 
>>> Monticello files for multi-developer open source projects. When a 
>>> project is stored in a git repository it is not necessary to record 
>>> the versions of the individual mcz files in the ConfigurationOf* (a 
>>> very tedious process) so the "version specs" aren't necessary.
>>>
>>> It is still necessary to specify the package and project 
>>> dependencies so the BaselineOf was invented to specify only the 
>>> package/project dependencies....
>>>
>>> tODE has been specifically designed to support a development model 
>>> based on git and github and I am in the process of converting all of 
>>> the projects used by GLASS/GsDevKit to GitHub and BaselineOf*.
>>>
>>> ConfigurationOf* will continue to be supported "forever" however, I 
>>> want to encourage folks to move their projects to git, because the 
>>> ecosystem around git-based repositories continue to be improved.
>>>>
>>>> 2. Is it possible to load a specific version of Seaside (in my 
>>>> case, 3.0.7.1) using tODE? Everything I'm reading seems to be 
>>>> targeted at loading baselines.
>>>
>>> For Seaside 3.0 I think that you can only use ConfigurationOf* ... 
>>> there are ways to create tODE project entires for ConfigurationOf* 
>>> projects[1], but I think that for ConfigurationOf* development you 
>>> should continue to use Metacello load expressions and custom 
>>> Smalltalk expressions. I would suggest that you create tODE scripts 
>>> using `touch sh <script-name>` for your load scripts and other 
>>> maintenance tasks. Ask questions on this list if you need some more 
>>> pointers.
>>>
>>> [1] 
>>> https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/XMLSupport.ston
>>>>
>>>> 3. I'm able to connect to the old repository on my (headless) 
>>>> server with both tODE and GemTools, but I can't seem to `project 
>>>> install` Seaside:
>>>>
>>>>     tode 1 > project install
>>>>     --url=http://gsdevkit.github.io/GsDevKit_home/Seaside3.ston
>>>>     Topez ERROR: a TodeCommandError occurred (error 121000), No
>>>>     such node: '/sys/local/server/projects/'
>>>>
>>> This error sees to be a case where the stone may not have been setup 
>>> correctly ... tODE expect there to be a "standard directory 
>>> structure in well known places" and this error seems to indicate 
>>> that perhaps there is a problem in this area ... could you give me 
>>> more details about how you "managed to (mostly) install tODE" 
>>> because perhaps you are hitting errors because of the "(mostly)" 
>>> bits ...
>>>
>>> You can get a stack from a command error using `debugMode --set` 
>>> (see `debugMode -h` for more details) and the stack will help me 
>>> track down what is wrong ..
>>>>
>>>> I've got a $GS_HOME/sys/local/server/projects directory, both on 
>>>> the client and the server. Which of these is the one that matters, 
>>>> and what do I have mis-configured such that tODE can't find it? I 
>>>> tried both the `--local` and `--stone` arguments to no avail.
>>> These are all things that work when stones are created using the 
>>> GsDevKit_home `createStone` command, so I am wondering where you 
>>> might have departed from the standard installation ... "managed to 
>>> (mostly) install tODE"  makes me wonder ...
>>>
>>> Dale
>>>
>>>>
>>>> Thanks in advance for help with any of these.
>>>>
>>>> --
>>>> Ken Treis
>>>> Miriam Technologies, Inc.
>>>> (866) 652-2040 x221
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Glass mailing list
>>>> Glass at lists.gemtalksystems.com
>>>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>>
>>> _______________________________________________
>>> Glass mailing list
>>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>> _______________________________________________
>> Glass mailing list
>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20180316/a20fc22a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2018-03-01 at 2.01.49 PM.png
Type: image/png
Size: 237052 bytes
Desc: not available
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20180316/a20fc22a/attachment-0001.png>
-------------- next part --------------
GS_CLIENT_DEV_CLIENTS=/home/dhenrich/rogue/_homes/rogue/_home/dev/clients
GS_CLIENT_DEV=/home/dhenrich/rogue/_homes/rogue/_home/dev
GSDEVKIT_DEV_GIT_CHECKOUT=master
GSDEVKIT_DEV_GIT_REMOTE=origin
GSDEVKIT_HOME_GIT_CHECKOUT=master
GSDEVKIT_HOME_GIT_REMOTE=origin
GSDEVKIT_SERVER_GIT_CHECKOUT=master
GSDEVKIT_SERVER_GIT_REMOTE=origin
GSDEVKIT_SYS_LOCAL_GIT_CHECKOUT=master
GSDEVKIT_SYS_LOCAL_GIT_REMOTE=origin
GSDEVKIT_TODE_CLIENT_GIT_CHECKOUT=master
GSDEVKIT_TODE_CLIENT_GIT_REMOTE=origin
GS_DOCS=/home/dhenrich/rogue/_homes/rogue/_home/docs
GS_HOME=/home/dhenrich/rogue/_homes/rogue/_home
GS_SERVER=/home/dhenrich/rogue/_homes/rogue/_home/server
GS_SERVER_STONES=/home/dhenrich/rogue/_homes/rogue/_home/server/stones
GS_SHARED_DOWNLOADS=/home/dhenrich/rogue/_homes/rogue/_home/shared/downloads
GS_SHARED_GIT_CHECKOUT_GEMSTONE_FILETREE=gemstone2.4
GS_SHARED_GIT_CHECKOUT_GLASS1=master
GS_SHARED_GIT_CHECKOUT_GREASE=master
GS_SHARED_GIT_CHECKOUT_METACELLO=master
GS_SHARED_GIT_CHECKOUT_PHARO3_FILETREE=pharo3.0
GS_SHARED_GIT_CHECKOUT_PHARO_COMPAT=master
GS_SHARED_GIT_CHECKOUT_RB=dev
GS_SHARED_GIT_CHECKOUT_SMALLTALKCI=master
GS_SHARED_GIT_CHECKOUT_STON=gemstone
GS_SHARED_GIT_CHECKOUT_TODE=master
GS_SHARED_GIT_CHECKOUT_ZINC=gs_master
GS_SHARED_GIT_CHECKOUT_ZODIAC=gs_master
GS_SHARED_GITHUB_COMMIT_GEMSTONE_FILETREE=gemstone2.4
GS_SHARED_GITHUB_COMMIT_GLASS1=master
GS_SHARED_GITHUB_COMMIT_GREASE=v1.1.?
GS_SHARED_GITHUB_COMMIT_METACELLO=master
GS_SHARED_GITHUB_COMMIT_PHARO3_FILETREE=pharo3.0
GS_SHARED_GITHUB_COMMIT_PHARO_COMPAT=master
GS_SHARED_GITHUB_COMMIT_RB=dev
GS_SHARED_GITHUB_COMMIT_SMALLTALKCI=master
GS_SHARED_GITHUB_COMMIT_STON=v0.9.?
GS_SHARED_GITHUB_COMMIT_TODE=master
GS_SHARED_GITHUB_COMMIT_ZINC=2.4.3.?
GS_SHARED_GITHUB_COMMIT_ZODIAC=gs_master
GS_SHARED_GIT_REMOTE_GEMSTONE_FILETREE=origin
GS_SHARED_GIT_REMOTE_GLASS1=origin
GS_SHARED_GIT_REMOTE_GREASE=origin
GS_SHARED_GIT_REMOTE_METACELLO=origin
GS_SHARED_GIT_REMOTE_PHARO3_FILETREE=origin
GS_SHARED_GIT_REMOTE_PHARO_COMPAT=origin
GS_SHARED_GIT_REMOTE_RB=origin
GS_SHARED_GIT_REMOTE_SMALLTALKCI=origin
GS_SHARED_GIT_REMOTE_STON=origin
GS_SHARED_GIT_REMOTE_TODE=origin
GS_SHARED_GIT_REMOTE_ZINC=origin
GS_SHARED_GIT_REMOTE_ZODIAC=origin
GS_SHARED=/home/dhenrich/rogue/_homes/rogue/_home/shared
GS_SHARED_PHARO=/home/dhenrich/rogue/_homes/rogue/_home/shared/pharo
GS_SHARED_REPO_GEMSTONE_FILETREE=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/filetree/repository
GS_SHARED_REPO_GLASS1=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/glass/repository
GS_SHARED_REPO_GREASE=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/Grease/repository
GS_SHARED_REPO=/home/dhenrich/rogue/_homes/rogue/_home/shared/repos
GS_SHARED_REPO_METACELLO=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/metacello-work/repository
GS_SHARED_REPO_PHARO3_FILETREE=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/pharo3.0/filetree/repository
GS_SHARED_REPO_PHARO_COMPAT=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/PharoCompatibility/repository
GS_SHARED_REPO_RB=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/rb/repository
GS_SHARED_REPO_SMALLTALKCI=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/smalltalkCI/repository
GS_SHARED_REPO_STON=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/ston/repository
GS_SHARED_REPO_TODE_CLIENT=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/todeClient
GS_SHARED_REPO_TODE=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/tode/repository
GS_SHARED_REPO_ZINC=filetree:///home/dhenrich/rogue/_homes/rogue/_home/shared/repos/zinc/repository
GS_SHARED_REPO_ZODIAC=github://GsDevKit/zodiac:gs_master/repository
GS_SYS_DEFAULT_CLIENT=/home/dhenrich/rogue/_homes/rogue/_home/sys/default/client
GS_SYS_DEFAULT=/home/dhenrich/rogue/_homes/rogue/_home/sys/default
GS_SYS_DEFAULT_SERVER=/home/dhenrich/rogue/_homes/rogue/_home/sys/default/server
GS_SYS=/home/dhenrich/rogue/_homes/rogue/_home/sys
GS_SYS_LOCAL_CLIENT=/home/dhenrich/rogue/_homes/rogue/_home/sys/local/client
GS_SYS_LOCAL=/home/dhenrich/rogue/_homes/rogue/_home/sys/local
GS_SYS_LOCAL_SERVER=/home/dhenrich/rogue/_homes/rogue/_home/sys/local/server
GS_SYS_SESSIONS=/home/dhenrich/rogue/_homes/rogue/_home/sys/local/sessions
GS_SYS_STONES=/home/dhenrich/rogue/_homes/rogue/_home/sys/local/server/stones
GS_TODE_CLIENT=/home/dhenrich/rogue/_homes/rogue/_home/dev/todeClient

More information about the Glass mailing list