[Glass] Metacello Preview / tODE Questions
Ken Treis via Glass
glass at lists.gemtalksystems.com
Sat Mar 31 19:45:48 PDT 2018
Dale, thanks for setting me on the right track here. I hope your wife's health continues to improve.
This approach looks like it's going to work, but I ran into problems with the `createStone` because my repository is too big for the default free keyfile. Is it possible to specify an alternate keyfile when invoking `createStone`?
--
Ken Treis
Miriam Technologies, Inc.
(866) 652-2040 x221
> On Mar 16, 2018, at 4:01 PM, Dale Henrichs via Glass <glass at lists.gemtalksystems.com> wrote:
>
> inadvertantly used private email ... I think that my response may contain generally useful information...
>
>
> -------- Forwarded Message --------
> Subject: Re: Fwd: [Glass] Metacello Preview / tODE Questions
> Date: Fri, 16 Mar 2018 14:42:38 -0700
> From: Dale Henrichs <dale.henrichs at gemtalksystems.com> <mailto:dale.henrichs at gemtalksystems.com>
> To: Ken Treis <ken at miriamtech.com> <mailto:ken at miriamtech.com>
>
> Sorry Ken,
> I've been pre-ocuppied with Rowan work and taking care of my wife and grand kids ... My wife spent a couple of weeks in the hospital back in February and while she has been getting better every day, she's still not all the way back and it may take a few more months before she's back to normal, so my email queue has been backing up ...
> I appreciate the fact that you pinged me ...
>
> My answers are embedded below ...
>
> On 03/16/2018 12:21 PM, Ken Treis wrote:
>> Dale,
>>
>> Just bumping this to the top of your inbox. If you have a minute or two to point me in the right direction, I'd sure appreciate it. Or… if there's a better way to load tODE into a pre-tODE repository, I'm open to just about anything.
>>
>> Thanks!
>>
>> --
>> Ken Treis
>> Miriam Technologies, Inc.
>> (866) 652-2040 x221
>>
>>
>>> Begin forwarded message:
>>>
>>> From: Ken Treis via Glass <glass at lists.gemtalksystems.com <mailto:glass at lists.gemtalksystems.com>>
>>> Subject: Re: [Glass] Metacello Preview / tODE Questions
>>> Date: March 1, 2018 at 2:05:24 PM PST
>>> To: Dale Henrichs <dale.henrichs at gemtalksystems.com <mailto:dale.henrichs at gemtalksystems.com>>
>>> Cc: glass at lists.gemtalksystems.com <mailto:glass at lists.gemtalksystems.com>
>>> Reply-To: Ken Treis <ken at miriamtech.com <mailto:ken at miriamtech.com>>
>>>
>>> Dale,
>>>
>>> Thank you very much for the explanations. That helps me a lot.
>>>
>>> The way I installed tODE was definitely unorthodox, and I know it's not fair to ask you to support non-standard installations, so don't feel pressured to fix my problems here. I'd just appreciate some general guidelines.
>>>
>>> We've got lots of infrastructure built around the old way of running GLASS apps; the stone runs from /opt/gemstone/product/seaside and we have a number of helper scripts running from there. So I didn't want to dive head-on into the tODE-managed stone world if I didn't have to, and right now I'm trying to take baby steps into it. If I could continue to run my stone and scripts from the old tree, that'd be ideal… but if that's asking for trouble, I can give up on my ideals.
> Well it is not directly asking for trouble, but the full tODE implementation is dependent upon directory structure and a set of environment variables that reference key points in the directory structure ... so the issues you are seeing are more than likely caused by one or more of the environment variables used by GsDevKit_home and tODE not being correctly set or pointing at chunks of the directory structure that are not present ...
>
> Frankly it was hard enough to put together the set of scripts and directory structure in the first place, without trying to wrap my head around trying to figure out how to help you figure out which pieces of the infrastructure you cannot live without ...
>
> I'm afraid that trying to get tODE to be fully functional outside the environment it was designed to work within, could turn out to be a slow process of re-creating the GsDevKit_home environment within the context of your structure and scripts.
>
> OTOH, you might be one env var away from success... so I'll go ahead and start the process of trying to help you get things working ...
>
>>> To load tODE, I first ran my repo through a full upgrade from 3.2.15 to 3.3.6 (upgradeImage, upgradeSeasideImage, etc). Then, I downloaded GsDevKit_home, ran the `installServer` script, and loaded the tODE code using Metacello. Here's the script I used:
>>>
>>> #!/bin/bash -e
>>> #
>>> # Run as normal user with sudo privileges
>>> export GEMSTONE=/opt/gemstone/product
>>> export GEMSTONE_NAME=${GEMSTONE_NAME:-seaside}
>>>
>>> # Install tODE
>>> export GS_HOME=/opt/gemstone/GsDevKit_home
>>> export GS_SHARED_REPO="${GS_HOME}/shared/repos"
>>> export GS_SHARED_REPO_TODE="filetree://${GS_SHARED_REPO}/tode/repository <filetree://$%7BGS_SHARED_REPO%7D/tode/repository>"
>>>
>>> sudo -u gsadmin -E $GEMSTONE/bin/topaz -lq <<EOF
>>> set user DataCurator gems $GEMSTONE_NAME pass redacted
>>> login
>>> run
>>> GsDeployer bulkMigrate: [
>>> Metacello new
>>> baseline: 'FileTree';
>>> repository: 'filetree://${GS_SHARED_REPO}/filetree/repository <filetree://$%7BGS_SHARED_REPO%7D/filetree/repository>';
>>> get.
>>> Metacello new
>>> baseline: 'Metacello';
>>> repository: 'filetree://${GS_SHARED_REPO}/metacello-work/repository <filetree://$%7BGS_SHARED_REPO%7D/metacello-work/repository>';
>>> get.
>>> Metacello new
>>> baseline: 'Metacello';
>>> repository: 'filetree://${GS_SHARED_REPO}/metacello-work/repository <filetree://$%7BGS_SHARED_REPO%7D/metacello-work/repository>';
>>> load.
>>> ].
>>> %
>>> commit
>>> run
>>> | todeRepo |
>>> todeRepo := GsFile _expandEnvVariable: 'GS_SHARED_REPO_TODE' isClient: false.
>>> Transcript
>>> cr;
>>> show: '-----Install tODE using ', todeRepo.
>>> GsDeployer bulkMigrate: [
>>> Metacello new
>>> baseline: 'Tode';
>>> repository: todeRepo;
>>> get;
>>> lock;
>>> onConflict: [:ex | ex disallow];
>>> load: 'GemStone Dev' ].
>>> %
>>> commit
>>> quit
>>> EOF
>>>
>>> That all went fine, and at that point I was able to connect to the stone with a tODE client on my laptop via an SSH tunnel for the NetLDI port. Here's my session config:
>>>
>>> TDSessionDescription {
>>> #name : 'Production',
>>> #stoneHost : 'localhost',
>>> #stoneName : 'seaside',
>>> #gemHost : 'localhost',
>>> #netLDI : '',
>>> #netLDIPort : '50380',
>>> #gemTask : 'gemnetobject',
>>> #userId : 'DataCurator',
>>> #password : 'redacted',
>>> #osUserId : nil,
>>> #osPassword : nil,
>>> #dataDirectory : nil,
>>> #backupDirectory : '/opt/gemstone/product/seaside/data/backups/',
>>> #snapshotDirectory : '/opt/gemstone/product/seaside/data/snapshots/',
>>> #gemstoneVersion : '3.3.6',
>>> #gciLibraryName : nil,
>>> #adornmentColor : nil,
>>> #serverGitRoot : nil,
>>> #serverTodeRoot : '/opt/gemstone/GsDevKit_home',
>>> #authorId : nil
>>> }
>>>
>>> But then, as you saw, I couldn't perform certain operations that needed to talk to the GsDevKit_home file tree. Is the #serverTodeRoot spec enough, or does the stone process on the server need tODE-specific environment variables, or are there some class vars that need to be set?
> If you run `$GS_HOME/bin/utils/gsDevKitEnvVars`[1] you will see that there are about 80 environment variables that are defined by GsDevKit_home (I've listed the set of env var values that are set for my environment in the attached file). As I scan the list it seems to me that most of those env vars are likely to be used at one point or another while using tODE ...
>
> ... but that's not all. It is not quite enough to just install the tODE code into your stone. There are a set of tODE scripts[2] that are also run after tODE is installed to create a directory structure. The code that creates the directory structure is implemented in the Pharo devKitCommandLine image and is triggered by running the todeLoad[3] script ... so in theory, the whole sequence of steps can be reverse engineered, but teasing out the bits that you need to run so that you can install tODE in your environment will be very tedious ...
>
> As I spend the time to think about your "problem". I think that you would be better off install tODE into your stone by transplanting your extents into a GsDevKit_home installation, installing tODE using the standard GsDevKit_home scripts, then moving your extents back into your old structure and then make sure that when you run tODE all of the GsDevKit_home env vars are set pointing at the GsDevKit_home/tODE directory structure under GS_HOME....
>
> This is the sequence of steps that I think you would need to do:
>
> #
> # 1. Install tode into your extent (this should be the upgraded extent before you installed tODE using your process)
> # When this step is done, the expected directory structure for tODE will have been created under $GS_HOME,
> # but you should be able to cp the extent back into your /opt/gemstone location as long as you make sure the
> # env vars are set up correctly
> #
> $GS_HOME/bin/createStone -s <path-to-your-extent-without-tode-installed> tode_stone <gs-version>
> $GS_HOME/bin/stopStone tode_stone
> cp $GS_HOME/server/stones/tode_stone/extents/extent0.dbf <path-to-your-extent-without-tode-installed>
> #
> # 2. customize your netldi startup script to pick up the GsDevKit_home env vars
> # If you look at the startNetldi script[4] you will see the following bash code is being run
> # This sequence ensures that the appropriate environment variables are defined for
> # GsDevKit_home and the stone --- both of which may be used by tODE:
> source "${GS_HOME}/bin/defGsDevKit.env"
> stonePath="$GS_SERVER_STONES/${stoneName}"
> pushd "$stonePath" >& /dev/null
> source "$stonePath/stone.env"
> popd >& /dev/null
>
> So at this point I am pretty optimistic that this approach will get you going ...
>
> Dale
>
> [1] https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/utils/gsDevKitEnvVars <https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/utils/gsDevKitEnvVars>
> [2] https://github.com/GsDevKit/GsDevKit_home/tree/master/sys/default/client/tode-scripts <https://github.com/GsDevKit/GsDevKit_home/tree/master/sys/default/client/tode-scripts>
> [3] https://github.com/GsDevKit/GsDevKit_home/blob/master/sys/bin/todeLoad <https://github.com/GsDevKit/GsDevKit_home/blob/master/sys/bin/todeLoad>
> [4] https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/startNetldi <https://github.com/GsDevKit/GsDevKit_home/blob/master/bin/startNetldi>
>
>>>
>>> I couldn't figure out how to copy a stack from the tODE debugger, but I took a screenshot (attached). If I'm reading it right, my root node doesn't have a "sys" child at all… ?
>>>
>>> --
>>> Ken Treis
>>> Miriam Technologies, Inc.
>>> (866) 652-2040 x221
>>>
>>>
>>>> On Feb 28, 2018, at 9:46 AM, Dale Henrichs via Glass <glass at lists.gemtalksystems.com <mailto:glass at lists.gemtalksystems.com>> wrote:
>>>>
>>>>
>>>>
>>>> On 02/27/2018 02:08 PM, Ken Treis via Glass wrote:
>>>>> I have an old 3.2.15 repository full of important data, and I managed to (mostly) install tODE in it, but now I have lots of (probably newbie) questions about tODE and Metacello Preview.
>>>>>
>>>>> 1. Are ConfigurationOf* classes obsolete? If so, what replaces them? Most of the intro docs just talk about baselines, which now live in separate classes — and often separate packages — thus apparently disconnecting baseline specs from version specs.
>>>> ConfigurationOf* are not obsolete. You see the emphasis on BaselineOf, because I think that git/github is superior to Monticello files for multi-developer open source projects. When a project is stored in a git repository it is not necessary to record the versions of the individual mcz files in the ConfigurationOf* (a very tedious process) so the "version specs" aren't necessary.
>>>>
>>>> It is still necessary to specify the package and project dependencies so the BaselineOf was invented to specify only the package/project dependencies....
>>>>
>>>> tODE has been specifically designed to support a development model based on git and github and I am in the process of converting all of the projects used by GLASS/GsDevKit to GitHub and BaselineOf*.
>>>>
>>>> ConfigurationOf* will continue to be supported "forever" however, I want to encourage folks to move their projects to git, because the ecosystem around git-based repositories continue to be improved.
>>>>>
>>>>> 2. Is it possible to load a specific version of Seaside (in my case, 3.0.7.1) using tODE? Everything I'm reading seems to be targeted at loading baselines.
>>>>
>>>> For Seaside 3.0 I think that you can only use ConfigurationOf* ... there are ways to create tODE project entires for ConfigurationOf* projects[1], but I think that for ConfigurationOf* development you should continue to use Metacello load expressions and custom Smalltalk expressions. I would suggest that you create tODE scripts using `touch sh <script-name>` for your load scripts and other maintenance tasks. Ask questions on this list if you need some more pointers.
>>>>
>>>> [1] https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/XMLSupport.ston <https://github.com/GsDevKit/GsDevKit_home/blob/gh-pages/XMLSupport.ston>
>>>>>
>>>>> 3. I'm able to connect to the old repository on my (headless) server with both tODE and GemTools, but I can't seem to `project install` Seaside:
>>>>>
>>>>> tode 1 > project install --url=http://gsdevkit.github.io/GsDevKit_home/Seaside3.ston <http://gsdevkit.github.io/GsDevKit_home/Seaside3.ston>
>>>>> Topez ERROR: a TodeCommandError occurred (error 121000), No such node: '/sys/local/server/projects/'
>>>> This error sees to be a case where the stone may not have been setup correctly ... tODE expect there to be a "standard directory structure in well known places" and this error seems to indicate that perhaps there is a problem in this area ... could you give me more details about how you "managed to (mostly) install tODE" because perhaps you are hitting errors because of the "(mostly)" bits ...
>>>>
>>>> You can get a stack from a command error using `debugMode --set` (see `debugMode -h` for more details) and the stack will help me track down what is wrong ..
>>>>>
>>>>> I've got a $GS_HOME/sys/local/server/projects directory, both on the client and the server. Which of these is the one that matters, and what do I have mis-configured such that tODE can't find it? I tried both the `--local` and `--stone` arguments to no avail.
>>>> These are all things that work when stones are created using the GsDevKit_home `createStone` command, so I am wondering where you might have departed from the standard installation ... "managed to (mostly) install tODE" makes me wonder ...
>>>>
>>>> Dale
>>>>
>>>>>
>>>>> Thanks in advance for help with any of these.
>>>>>
>>>>> --
>>>>> Ken Treis
>>>>> Miriam Technologies, Inc.
>>>>> (866) 652-2040 x221
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Glass mailing list
>>>>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>>>>> http://lists.gemtalksystems.com/mailman/listinfo/glass <http://lists.gemtalksystems.com/mailman/listinfo/glass>
>>>>
>>>> _______________________________________________
>>>> Glass mailing list
>>>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>>>> http://lists.gemtalksystems.com/mailman/listinfo/glass <http://lists.gemtalksystems.com/mailman/listinfo/glass>
>>> <Screen Shot 2018-03-01 at 2.01.49 PM.png>
>>> _______________________________________________
>>> Glass mailing list
>>> Glass at lists.gemtalksystems.com <mailto:Glass at lists.gemtalksystems.com>
>>> http://lists.gemtalksystems.com/mailman/listinfo/glass <http://lists.gemtalksystems.com/mailman/listinfo/glass>
>>
>
> <envvars.txt>_______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20180331/4b69211a/attachment-0001.html>
More information about the Glass
mailing list