[GemStone-Smalltalk] GemStone-Smalltalk Digest, Vol 129, Issue 4

Thu May 29 19:40:20 PDT 2025

Sure James,

Server side:
gsadmin at debian ~
❯ gslist -vl
Status Version Owner Pid Port Started Type Name
------- --------- --------- -------- ----- ------------ ------ ----
OK 3.7.2 root 72588 50377 May 29 00:45 Netldi gs64ldi
OK 3.7.2 gsadmin 72793 34459 May 29 00:52 Stone gs64stone
OK 3.7.2 gsadmin 72794 42869 May 29 00:52 cache gs64stone~ada6cb43e22fbc5f

gsadmin at debian ~
❯ gslist -cvl
Status Version Owner Pid Port Started Type Name
------- --------- --------- -------- ----- ------------ ------ ----
OK 3.7.2 root 72588 50377 May 29 00:45 Netldi gs64ldi
OK 3.7.2 gsadmin 72793 34459 May 29 00:52 Stone gs64stoneOK 3.7.2 gsadmin 72794 42869 May 29 00:52 cache gs64stone~ada6cb43e22fbc5f

Client side:
gsadmin at 73a6df6448b8:/$ gslist -vl
Status Version Owner Pid Port Started Type Name
------- --------- --------- -------- ----- ------------ ------ ----
OK 3.7.2 root 10 50377 May 29 23:45 Netldi gs64ldi
OK 3.7.2 root 25 36973 May 29 23:46 cache gs64stone~ada6cb43e22fbc5f
gsadmin at 73a6df6448b8:/$ gslist -cvl
Status Version Owner Pid Port Started Type Name
------- --------- --------- -------- ----- ------------ ------ ----
OK 3.7.2 root 10 50377 May 29 23:45 Netldi gs64ldiOK 3.7.2 root 25 36973 May 29 23:46 cache gs64stone~ada6cb43e22fbc5f

The other client machine is the same,

Ian

Sent with [Proton Mail](https://proton.me/mail/home) secure email.

On Thursday, May 29th, 2025 at 11:05 PM, James Foster <Smalltalk at JGFoster.net> wrote:

> Ian,
>
> I’ve not studied your details yet, but I’d be interested in finding out the impact of a couple variations on gslist (https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/MAIN.htm):
>
> cslist -vl
> gslist -cvl
>
> In particular, the `-c` option “Removes locks left by servers that have been killed.”
>
> James
>
>> On May 29, 2025, at 6:50 PM, Ian <icjohnson at protonmail.com> wrote:
>>
>> James.
>>
>> My lab:
>>
>> Bare Meta Server:
>>
>> - 32GB RAM
>> - x86_64 12 Core Intel
>> - Stone: gs64stone
>> - NetLdi: gs64ldi
>> - Shared Memory 8 GB (Community 10GB x509 trial license - keeping cache in memory while minimal)
>> - Operation:
>>
>> - ❯ ls -la gem netldid pgsvr pgsvrmain stoned
>> -r-sr-xr-x 1 gsadmin gsadmin 118024 Dec 17 16:16 gem
>> -r-sr-xr-x 1 root root 116560 Dec 17 16:16 netldid
>> -r-sr-xr-x 1 gsadmin gsadmin 13029328 Dec 17 16:16 pgsvr
>> -r-sr-xr-x 1 gsadmin gsadmin 122168 Dec 17 16:16 pgsvrmain-r-sr-xr-x 1 gsadmin gsadmin 24165744 Dec 17 16:17 stoned
>>
>> - gslist:
>>
>> - ❯ gslist
>> Status Version Owner Started Type Name
>> ------- --------- --------- ------------ ------ ----
>> exists 3.7.2 root May 29 00:45 Netldi gs64ldi
>> exists 3.7.2 gsadmin May 29 00:52 Stone gs64stoneexists 3.7.2 gsadmin May 29 00:52 cache gs64stone~ada6cb43e22fbc5f
>>
>> Clients x2 (Lenovo x85_64 machines):
>>
>> - Docker containers:
>>
>> - HAProxy: reverse proxy -> gem to stone, stone to gem throughput
>> - GsClient; netldi gem rpc client running topaz (app launch via topaz)
>> - Dockerfiles create custom, Debian based images (All containers behave the same but for "startup.sh")
>> - Docker container: gslist:
>>
>> - root at x1:/opt/puzzleit/compose/gsclient# docker exec -it gsclient /bin/bash
>> gsadmin at 73a6df6448b8:/$ gslist
>> Status Version Owner Started Type Name
>> ------- --------- --------- ------------ ------ ----
>> exists 3.7.2 root May 29 23:45 Netldi gs64ldiexists 3.7.2 root May 29 23:46 cache gs64stone~ada6cb43e22fbc5f
>> - Docker container: ps aux:
>>
>> - gsadmin at 73a6df6448b8:/$ ps aux
>> USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
>> gsadmin 1 0.0 0.0 3928 3056 ? Ss May29 0:00 /bin/bash /opt/gemstone/startup.sh
>> root 10 0.0 0.0 20092 9300 ? S May29 0:00 /opt/gemstone/sys/netldid -s -D /opt/gemstone/log
>> gsadmin 11 0.0 0.0 4192 3152 pts/0 Ss May29 0:00 /bin/bash
>> gsadmin 18 0.0 0.0 18800 8528 pts/0 S+ May29 0:00 topaz -I /opt/gemstone/data/topazini
>> root 19 0.0 0.1 573212 48016 ? Ssl May29 0:00 /opt/gemstone/sys/gem TCP 6
>> root 23 0.0 0.0 249896 8000 ? Ssl May29 0:00 /opt/gemstone/sys/pgsvrmain gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33907 TCP
>> root 25 0.2 0.2 493412 87428 ? Sl May29 0:04 /opt/gemstone/sys/shrpcmonitor gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 128 1 1900 0 432 0 0 60 0 0gsadmin 103 0.0 0.0 4192 3248 pts/2 Ss 00:21 0:00 /bin/bash
>>
>> Plan for the stone sever:
>>
>> - Scale as per Gemstone best practice/admin guide
>> - Extents/Trans Logs running separate network-based disks, bare metal adding extents as needed
>> - Potential bare metal midlevel cache(s)
>>
>> Plan for remote clients:
>>
>> - Eventually K8s deployments
>> - Scaled using kubernetes deployments
>> - Proxied via non-ssl-terminated reverse proxy (igems will manage x509 certificate behavior)
>> - Kubernetes API gateway manages ingress/egress via all the needed dedicated client gem/server gem communication via the proxy
>> - Micro-services are gem centric containers with "startup.sh" files governing service behavior
>> - No need for volume management; Stone persistence...
>>
>> Client Reboots:
>>
>> - If I gracefully log out of the stone using topaz> logout in each of two clients and reboot (containers are set to restart if in not started) them both I am able to:
>>
>> - docker exec -it gsclient /bin/bash
>> - topaz -I /path/to/topazini and login without issue (both in startup.sh (as this script is run on container start) and on the command line of startiup.sh is commented out in the startup file.
>> - If there is a hard reset none of the above holds true.
>>
>> - Upon entering the container starting topaz manually of from within the start file fails with a stale cache error with the gem refusing to start
>> - It is this behavior that I was enquiring about.
>>
>> - Why is this terminal? Does it really need to be?
>> - My only throughs are very large pages caches taking a long time to repopulate/perhaps you guys have a way to rover so that cache repop is non-destructive?
>> - This behaviour is also evident if the network connection is terminated for any length of time (I can't say of short blips are recoverable)
>>
>> Client reboot points two and three are of concern and their behavior I am trying to understand more fully, cache management and the shared page cache.
>>
>> WRT foreground/background, I am having no issues the way I have it but wonder if there are differences in cache behavior using the methods suggested?
>>
>> I will be sure to try these methods and see where that takes me.
>>
>> Thank you for that and for your time,
>>
>> Ian
>>
>> Sent with [Proton Mail](https://proton.me/mail/home) secure email.
>>
>> On Thursday, May 29th, 2025 at 7:28 PM, James Foster <Smalltalk at JGFoster.net> wrote:
>>
>>> Ian,
>>>
>>> I would prefer a more explicit wait of not exiting than running netldi in the foreground. I’ve seen two recommendations:
>>> CMD ["tail", "-f", "/dev/null”]
>>> CMD ["sleep", "infinity"]
>>>
>>>> … the remote page cache becomes stale on system reboot ...
>>>> Note that the container lives in all the above cases ...
>>>
>>> Could you say more about your architecture? You have the stone in one Docker container and a gem in another Docker container? Which container lives and which one reboots?
>>>
>>> The idea of scaling gems with Docker sounds plausible, but I’ve not heard of anyone doing it.
>>>
>>> James
>>>
>>>> On May 27, 2025, at 7:34 PM, Ian via GemStone-Smalltalk <gemstone-smalltalk at lists.gemtalksystems.com> wrote:
>>>>
>>>> Hi Norm, James,
>>>>
>>>> I think this is worth exploring. Please bear with me.
>>>>
>>>> To answer your question Norm, container entry points, however you run them (docker run, compose entrypoint:, entry point in the dockerfile, whatever) runs as PID 1.
>>>>
>>>> I want to run a script, call it startup.sh, like this in my dockerfile:
>>>>
>>>> ENTRYPOINT: ["/some/container/path/startup.sh"]
>>>>
>>>> The scripts job is to:
>>>>
>>>> 1. startnetldi -s
>>>> 2. topaz -I topazini -S instructions.topaz (simplified)
>>>>
>>>> Once these two requirements are met, startup.sh exits. Meaning PID 1 also exits and the container shuts down (depending on you compose settings, an endless restart likely ensues).
>>>>
>>>> Now, when I run step one as 'netldid -s,' netldid runs on the foreground. PID 1 lives on and the container persists. Connections to the stone live, my application runs in the gem running in the container and all is good.
>>>>
>>>> The issue I am now trying to resolve is the fact that the remote page cache becomes stale on system reboot, system crashes or connection interruptions.
>>>>
>>>> Note that the container lives in all the above cases, netldid process persists and is happy but a stale page cache is terminal.
>>>>
>>>> I must bring down the containers by some manual method: docker compose down, or in terms of kubernetes, restart the deployment, to have the cache regenerated from scratch.
>>>>
>>>> This is fine as everything persists in the stone. However, provisions must be made to handle containers whose connection to the stone is terminated/interrupted for whatever the reason.
>>>>
>>>> From here:
>>>>
>>>> My goal is to run all connections to the stone using x509 enabled gems for all connecting entities of whatever type. The above is the beginnings of a POC to test what hurdles had to be overcome; Whether or not it would even be possible. It is doable, I think.
>>>>
>>>> Having started the full read of the x509 guide, I come to realize that the cache is created at the prompting from the server-side x509 enabled netldi through the host agent. In this case, it seem that it may be possible to build container resilience into what I a trying to do as the remote netldi must be alive, requesting a login from the server by saying "here I am, here is my port." What I don't know is that if the remote netldi sees a local cache will it still fail or will it say, you silly cache, you're busted, let's clean up and make a new one.
>>>>
>>>> If this can be made to work Gemstone/S remote gems could sit behind some kind of load balancer and run HA in docker, docker swarm or any kubernetes variety fully and robustly containerized and talk to a stone that sits as they do now, on a dedicated server managing connections, transactions and persisting data to extents until the cows come home.
>>>>
>>>> I feel like this has probably already been done but can't find a trailblazer on the web to guide me.
>>>>
>>>> I do know that I can run netldi on many nodes, bare metal or VM fronted by some form of load balancer for HA, scalability is simply much easier using containers and a container management system of some kind.
>>>>
>>>> Seems to me that remote gems are the perfect, secure and safe and scalable way to run Smalltalk based microservices in remote containerize gems with persistent object storage to boot!
>>>>
>>>> Long store short, to my question: Am I approaching this in the correct way? Is what I am trying to do even possible?
>>>>
>>>> Ian
>>>>
>>>> Sent with Proton Mail secure email.
>>>>
>>>> On Tuesday, May 27th, 2025 at 4:00 PM, gemstone-smalltalk-request at lists.gemtalksystems.com <gemstone-smalltalk-request at lists.gemtalksystems.com> wrote:
>>>>
>>>>> Send GemStone-Smalltalk mailing list submissions to
>>>>> gemstone-smalltalk at lists.gemtalksystems.com
>>>>>
>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>>>> or, via email, send a message with subject or body 'help' to
>>>>> gemstone-smalltalk-request at lists.gemtalksystems.com
>>>>>
>>>>> You can reach the person managing the list at
>>>>> gemstone-smalltalk-owner at lists.gemtalksystems.com
>>>>>
>>>>> When replying, please edit your Subject line so it is more specific
>>>>> than "Re: Contents of GemStone-Smalltalk digest..."
>>>>>
>>>>> Today's Topics:
>>>>>
>>>>> 1. netldid (Ian)
>>>>> 2. Re: netldid (James Foster)
>>>>> 3. Re: netldid (Norm Green)
>>>>> 4. Re: netldid (Ian)
>>>>>
>>>>> ----------------------------------------------------------------------
>>>>>
>>>>> Message: 1
>>>>> Date: Mon, 26 May 2025 19:33:08 +0000
>>>>> From: Ian icjohnson at proton.me
>>>>>
>>>>> To: "gemstone-smalltalk at lists.gemtalksystems.com"
>>>>> gemstone-smalltalk at lists.gemtalksystems.com
>>>>>
>>>>> Subject: [GemStone-Smalltalk] netldid
>>>>> Message-ID:
>>>>> PejD_sVleK6zxKRUkWsS3REI1mSSbkKihZU8MKvy-t-r1_b4wyM-WvvRxoRr8gIYVe5uQ4TXtf5G0BxUZ2zFNo-YwKfY6v2do6NGknySb7U=@proton.me
>>>>>
>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>
>>>>> Hi GemStoners,
>>>>>
>>>>> Are there plans in the works to allow netldid to run in the foreground?
>>>>>
>>>>> This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
>>>>>
>>>>> On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
>>>>>
>>>>> Thanks in advance,
>>>>> Ian
>>>>>
>>>>> Sent with Proton Mail secure email.
>>>>> -------------- next part --------------
>>>>> An HTML attachment was scrubbed...
>>>>> URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/67818374/attachment-0001.htm
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Message: 2
>>>>> Date: Mon, 26 May 2025 13:25:35 -0700
>>>>> From: James Foster Smalltalk at JGFoster.net
>>>>>
>>>>> To: Ian icjohnson at proton.me
>>>>>
>>>>> Cc: "gemstone-smalltalk at lists.gemtalksystems.com"
>>>>> gemstone-smalltalk at lists.gemtalksystems.com
>>>>>
>>>>> Subject: Re: [GemStone-Smalltalk] netldid
>>>>> Message-ID: 24CFB521-8477-4700-94F5-3E15CA35E192 at JGFoster.net
>>>>>
>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>
>>>>> Ian,
>>>>>
>>>>> A number of people have created Docker images for GemStone/S. This is fine as long as you map the database to something outside the container (else it will be lost when the container stops).
>>>>>
>>>>> See https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/B-CommandReference.htm#pgfId-857966 to let netldid run in the foreground.
>>>>>
>>>>> James
>>>>>
>>>>>> On May 26, 2025, at 12:33?PM, Ian via GemStone-Smalltalk gemstone-smalltalk at lists.gemtalksystems.com wrote:
>>>>>>
>>>>>> Hi GemStoners,
>>>>>>
>>>>>> Are there plans in the works to allow netldid to run in the foreground?
>>>>>>
>>>>>> This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
>>>>>>
>>>>>> On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
>>>>>>
>>>>>> Thanks in advance,
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>> Sent with Proton Mail https://proton.me/mail/home secure email.
>>>>>> _______________________________________________
>>>>>> GemStone-Smalltalk mailing list
>>>>>> GemStone-Smalltalk at lists.gemtalksystems.com
>>>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>>>>
>>>>> -------------- next part --------------
>>>>> An HTML attachment was scrubbed...
>>>>> URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/eebd20d9/attachment-0001.htm
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Message: 3
>>>>> Date: Mon, 26 May 2025 13:43:48 -0700
>>>>> From: Norm Green norm.green at gemtalksystems.com
>>>>>
>>>>> To: gemstone-smalltalk at lists.gemtalksystems.com
>>>>> Subject: Re: [GemStone-Smalltalk] netldid
>>>>> Message-ID: 7f994942-644d-4801-806e-a3c3f419962e at gemtalksystems.com
>>>>>
>>>>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>>>>>
>>>>> Hi Ian,
>>>>>
>>>>> I don't fully understand why you want to run netldi in the foreground.
>>>>> That said, startnetldi forks $GEMSTONE/sys/netldid with appropriate
>>>>> arguments, so you could run that in the foreground.
>>>>>
>>>>> Norm Green
>>>>>
>>>>> On 5/26/2025 12:33 PM, Ian via GemStone-Smalltalk wrote:
>>>>>
>>>>>> Hi GemStoners,
>>>>>>
>>>>>> Are there plans in the works to allow netldid to run in the foreground?
>>>>>>
>>>>>> This would allow the running of startnetldi -s in a container without
>>>>>> an infinite loop and a long running sleep statement (Better way?)
>>>>>>
>>>>>> On that note, I am able to containerize a running client application,
>>>>>> linked and RPC; Am I running down a road that will eventually bonk me
>>>>>> in the head?
>>>>>>
>>>>>> Thanks in advance,
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>> Sent with Proton Mail https://proton.me/mail/home secure email.
>>>>>>
>>>>>> _______________________________________________
>>>>>> GemStone-Smalltalk mailing list
>>>>>> GemStone-Smalltalk at lists.gemtalksystems.com
>>>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>>>>
>>>>> -------------- next part --------------
>>>>> An HTML attachment was scrubbed...
>>>>> URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/8b7b14ed/attachment-0001.htm
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Message: 4
>>>>> Date: Mon, 26 May 2025 21:05:20 +0000
>>>>> From: Ian icjohnson at proton.me
>>>>>
>>>>> To: James Foster Smalltalk at JGFoster.net
>>>>>
>>>>> Cc: "gemstone-smalltalk at lists.gemtalksystems.com"
>>>>> gemstone-smalltalk at lists.gemtalksystems.com
>>>>>
>>>>> Subject: Re: [GemStone-Smalltalk] netldid
>>>>> Message-ID:
>>>>> 2OoOBEiO6J-fYbKZwRSH68GLdy_pWxVzfCcdsPW6lpox5PVN-6Qp6m9ZUJHG8YC_xCOnX1pi-PJ2zrOgLQRC86SgPeSX2PT5hpACzicoPGM=@proton.me
>>>>>
>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>
>>>>> netldid -s
>>>>>
>>>>> The wording got me.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Ian
>>>>>
>>>>> Sent with Proton Mail secure email.
>>>>>
>>>>> On Monday, May 26th, 2025 at 5:25 PM, James Foster Smalltalk at JGFoster.net wrote:
>>>>>
>>>>>> Ian,
>>>>>>
>>>>>> A number of people have created Docker images for GemStone/S. This is fine as long as you map the database to something outside the container (else it will be lost when the container stops).
>>>>>>
>>>>>> See https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/B-CommandReference.htm#pgfId-857966 to let netldid run in the foreground.
>>>>>>
>>>>>> James
>>>>>>
>>>>>>> On May 26, 2025, at 12:33?PM, Ian via GemStone-Smalltalk gemstone-smalltalk at lists.gemtalksystems.com wrote:
>>>>>>>
>>>>>>> Hi GemStoners,
>>>>>>>
>>>>>>> Are there plans in the works to allow netldid to run in the foreground?
>>>>>>>
>>>>>>> This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
>>>>>>>
>>>>>>> On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
>>>>>>>
>>>>>>> Thanks in advance,
>>>>>>> Ian
>>>>>>>
>>>>>>> Sent with Proton Mail secure email.
>>>>>>> _______________________________________________
>>>>>>> GemStone-Smalltalk mailing list
>>>>>>> GemStone-Smalltalk at lists.gemtalksystems.com
>>>>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>>>>
>>>>> -------------- next part --------------
>>>>> An HTML attachment was scrubbed...
>>>>> URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/b228663a/attachment-0001.htm
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> Subject: Digest Footer
>>>>>
>>>>> _______________________________________________
>>>>> GemStone-Smalltalk mailing list
>>>>> GemStone-Smalltalk at lists.gemtalksystems.com
>>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>>>>>
>>>>> ------------------------------
>>>>>
>>>>> End of GemStone-Smalltalk Digest, Vol 129, Issue 4
>>>>> **************************************************
>>>>
>>>> _______________________________________________
>>>> GemStone-Smalltalk mailing list
>>>> GemStone-Smalltalk at lists.gemtalksystems.com
>>>> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250530/b597e2c0/attachment-0001.htm>