[GemStone-Smalltalk] GemStone-Smalltalk Digest, Vol 129, Issue 12
Ian
icjohnson at proton.me
Thu May 29 22:18:38 PDT 2025
James,
Removing the locks works.
Starts up every time.
Not sure what the downfalls are. Probably time it takes to regen the cache. Acceptable if in event of node/container crash.
Not sure how to figure out which locks belong to which container in the event that more than one container winds up on the same host.
Problem for another day.
Thanks for your time and the time on the list,
Ian
Sent with Proton Mail secure email.
On Friday, May 30th, 2025 at 12:43 AM, gemstone-smalltalk-request at lists.gemtalksystems.com <gemstone-smalltalk-request at lists.gemtalksystems.com> wrote:
> Send GemStone-Smalltalk mailing list submissions to
> gemstone-smalltalk at lists.gemtalksystems.com
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> or, via email, send a message with subject or body 'help' to
> gemstone-smalltalk-request at lists.gemtalksystems.com
>
> You can reach the person managing the list at
> gemstone-smalltalk-owner at lists.gemtalksystems.com
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of GemStone-Smalltalk digest..."
>
>
> Today's Topics:
>
> 1. Re: GemStone-Smalltalk Digest, Vol 129, Issue 4 (Ian)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 30 May 2025 03:42:40 +0000
> From: Ian icjohnson at protonmail.com
>
> To: James Foster Smalltalk at JGFoster.net
>
> Cc: gemstone-smalltalk at lists.gemtalksystems.com
> Subject: Re: [GemStone-Smalltalk] GemStone-Smalltalk Digest, Vol 129,
> Issue 4
> Message-ID:
> xLWF4C2eoIttSQcBV-Aa7p4kpuvTLrFp1bY4eQ8RN_XYUFjos5rkQKJbCE2tRsHJ8ciFPbGXawbzk_J3vHMSQ1LX0Tb-UN0NERpMzydwXiw=@protonmail.com
>
>
> Content-Type: text/plain; charset="utf-8"
>
> Just had a thought, if I manually remove all locks out of /opt/gemstone/locks I wonder if netldi will just regenerate the page cache?
>
> Ian
>
> Sent with Proton Mail secure email.
>
> On Friday, May 30th, 2025 at 12:38 AM, Ian icjohnson at protonmail.com wrote:
>
> > Just rebooted: Everything on the client running:
> >
> > Test 1:
> > prompt> systemctl shutdown -r now
> >
> > After startup:
> >
> > Client machine:
> >
> > Docker: Restart crash loop
> > ? sudo -i -u gsadmin
> > gsadmin at x1 ~
> > ? sudo -i
> > root at x1:~# docker ps -a
> > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
> > 73a6df6448b8 gsclient:latest "/opt/gemstone/start?" 3 hours ago Restarting (0) 4 seconds ago gsclient
> > d506351f8e64 nginx:latest "/docker-entrypoint.?" 3 hours ago Up About a minute 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp nginx
> > root at x1:~# docker ps -a
> > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
> > 73a6df6448b8 gsclient:latest "/opt/gemstone/start?" 3 hours ago Restarting (0) 17 seconds ago gsclient
> > d506351f8e64 nginx:latest "/docker-entrypoint.?" 3 hours ago Up About a minute 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp nginx
> > root at x1:~# docker ps -a
> > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
> > 73a6df6448b8 gsclient:latest "/opt/gemstone/start?" 3 hours ago Restarting (0) 23 seconds ago gsclient
> > d506351f8e64 nginx:latest "/docker-entrypoint.?" 3 hours ago Up About a minute 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp nginx
> > root at x1:~# docker ps -a
> > CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
> > 73a6df6448b8 gsclient:latest "/opt/gemstone/start?" 3 hours ago Restarting (0) 6 seconds ago gsclientd506351f8e64 nginx:latest "/docker-entrypoint.?" 3 hours ago Up About a minute 0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp nginx
> >
> > I should have set KEEP_LOG first: gs64ldi is empty and there are no other logs.
> >
> > root at x1:/srv/gsclient/logs# ll
> > total 8
> > drwxr-xr-x 2 5000 5000 4096 May 29 23:59 ./
> > drwxr-xr-x 5 5000 5000 4096 May 29 20:42 ../-rw-r--r-- 1 root 5000 0 May 29 23:59 gs64ldi.log
> > Restarting docker:
> > root at x1:/opt/compose/gsclient# docker compose down
> > [+] Running 3/3
> > ? Container gsclient Removed 0.0s
> > ? Container nginx Removed 0.3s
> > ? Network gsclient_default Removed 0.2s
> > root at x1:/opt/compose/gsclient# docker compose up -d --force-recreate
> > [+] Running 3/3
> > ? Network gsclient_default Created 0.1s
> > ? Container nginx Started 0.3s
> > ? Container gsclient Started 0.5sroot at x1:/opt/puzzleit/compose/gsclient#
> >
> > Enter the container:
> >
> > root at x1:/opt/puzzleit/compose/gsclient# docker exec -it gsclient /bin/bash
> > gsadmin at 7cb59490fa9f:/$ ps aux
> > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > gsadmin 1 0.0 0.0 3928 2932 ? Ss 03:04 0:00 /bin/bash /opt/gemstone/startup.sh
> > root 10 0.0 0.0 19480 8208 ? S 03:04 0:00 /opt/gemstone/sys/netldid -s -D /opt/gemstone/log
> > gsadmin 11 0.3 0.0 4192 3336 pts/0 Ss 03:05 0:00 /bin/bashgsadmin 18 0.0 0.0 8092 4020 pts/0 R+ 03:05 0:00 ps aux
> > Run topaz:
> >
> > gsadmin at 7cb59490fa9f:/$ topaz -I /opt/gemstone/data/topazini
> > [Info]: Loaded /opt/gemstone/lib/libgcirpc-3.7.2-64.so
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: topaz, Linear GemStone Interface (Remote Session) |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB |
> > | PROCESS ID: 22 DATE: 05/30/25 03:07:06 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=gsadmin (5000) |
> > |________________________________________________________________________________|
> > Reading initialization file /opt/gemstone/data/topazini
> > [05/30/25 03:07:08.445 UTC]
> > gci login: currSession 1 rpc gem processId 23 socket 5topaz 1>
> > It is not until I restart all the containers that I am able to access the systems again.
> >
> > Test 2:
> > Disconnect internet: systemctl stop NetworkManager
> >
> > [Panic; exiting Topaz]
> > Topaz pausing for About to exit topaz, type <return> to continue
> > gsadmin at 7cb59490fa9f:/$ <Press Enter - topaz tries to load>
> > gsadmin at 7cb59490fa9f:/$
> > [Info]: Loaded /opt/gemstone/lib/libgcirpc-3.7.2-64.so
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: topaz, Linear GemStone Interface (Remote Session) |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB |
> > | PROCESS ID: 49 DATE: 05/30/25 03:16:08 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=gsadmin (5000) |
> > |________________________________________________________________________________|Reading initialization file /opt/gemstone/data/topazini
> >
> > Hangs here. then after some seconds:
> > Buffered output from initialization file /opt/gemstone/data/topazini
> > topaz +> set gemstone !@debian!gs64stone
> > topaz +> set hostusername gsadmin
> > topaz +> set hostpassword ********
> > topaz +> set username DataCurator
> > topaz +> set password *********
> > topaz +> set gemnetid !@x1#netldi:gs64ldi!gemnetobject
> > topaz +> login
> > -----------------------------------------------------
> > GemStone: Error Fatal
> > Session lost connection to the Stone Repository monitor during login, remote cache startup timed out
> > Error Category: 231169 [GemStone] Number: 4038 Arg Count: 1 Context : 20 exception : 20
> > Arg 1: nil
> > End of initialization files
> > topaz>
> > topaz>topaz>
> >
> > Logs available:
> >
> > root at x1:/srv/gsclient/logs# ll
> > total 44
> > drwxr-xr-x 2 5000 5000 4096 May 30 00:16 ./
> > drwxr-xr-x 5 5000 5000 4096 May 29 20:42 ../
> > -rw-r--r-- 1 5000 5000 6838 May 30 00:13 gemnetobject237cb59490fa9f.log
> > -rw-r--r-- 1 5000 5000 6052 May 30 00:17 gemnetobject507cb59490fa9f.log
> > -rw-r--r-- 1 root 5000 2047 May 30 00:04 gs64ldi.log
> > -rw-r--r-- 1 5000 5000 3361 May 30 00:07 gs64stone_27cachepgsvr_7cb59490fa9f.log
> > -rw-r--r-- 1 root 5000 5450 May 30 00:13 gs64stone_29pcmon_7cb59490fa9f.log-rw-r--r-- 1 5000 5000 3024 May 30 00:16 gs64stone_54cachepgsvr_7cb59490fa9f.log
> >
> > gemnetobject237cb59490fa9f.log:
> >
> > root at x1:/srv/gsclient/logs# cat gemnetobject237cb59490fa9f.log
> > ________________________________________________________________________________
> > | NetLDI Child Task |
> >
> > NRS from client !#encrypted:gsadmin at password!gemnetobject
> > CLIENTHOST: 127.0.0.1
> > COMMAND: /opt/gemstone/sys/gemnetobject TCP 6
> > ||
> > [Info]: the hostname is:
> > GEMSTONE is: "/opt/gemstone"
> > gem's arguments are: TCP 6
> > [Info]: Loaded /opt/gemstone/lib/libgcilnk-3.7.2-64.so
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > | Covered by U.S Patents: |
> > | 6,256,637 Transactional virtual machine architecture (1998-2018) |
> > | 6,360,219 Object queues with concurrent updating (1998-2018) |
> > | 6,567,905 Generational Garbage Collector (2001-2021) |
> > | 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable |
> > | Database (2001-2021) |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: GEM, GemStone Session Process |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 23 DATE: 05/30/25 03:07:06 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > +--------------------------------------------------------------------------------+
> > | GEMSTONE_NRS_ALL = #dir:%D |
> > ||
> > ________________________________________________________________________________
> > Configuration Files
> > -z value is from a default rule
> > System File: /opt/gemstone/data/system.conf
> > Warning, file not found: (null)
> > ________________________________________________________________________________
> > Reading config file /opt/gemstone/data/system.conf
> > ________________________________________________________________________________
> > | Gem Configuration Options for process id 23 |||
> > | Gem Configuration Options for process id 23 |
> > ||
> >
> > CONFIG_WARNINGS_FATAL = FALSE;
> > DUMP_OPTIONS = TRUE;
> > GEM_ABORT_MAX_CRS = 100000;
> > GEM_CACHE_WARMER_ARGS = "";
> > GEM_CACHE_WARMER_MID_CACHE_ARGS = "";
> > GEM_COMPRESS_TRANLOG_RECORDS = TRUE;
> > GEM_ENV = "";
> > GEM_FREE_FRAME_CACHE_SIZE = -1;
> > GEM_FREE_FRAME_LIMIT = -1;
> > GEM_FREE_PAGEIDS_CACHE = 200;
> > GEM_HALT_ON_ERROR = -1;
> > GEM_KEEP_MIN_SOFTREFS = 0;
> > GEM_KERBEROS_KEYTAB_FILE = "";
> > GEM_KEYRING_DIRS = "";
> > GEM_LISTEN_FOR_DEBUG = FALSE;
> > GEM_MAX_SMALLTALK_STACK_DEPTH = 1000;
> > GEM_NATIVE_CODE_ENABLED = 2;
> > GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = TRUE;
> > GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
> > GEM_PGSVR_FREE_FRAME_LIMIT = -1;
> > GEM_PGSVR_UPDATE_CACHE_ON_READ = TRUE;
> > GEM_PGSVR_USE_SSL = FALSE;
> > GEM_READ_AUTH_ERR_STUBS = FALSE;
> > GEM_REPOSITORY_IN_MEMORY = FALSE;
> > GEM_RPC_KEEPALIVE_INTERVAL = 0;
> > GEM_RPC_USE_SSL = TRUE;
> > GEM_SMALLTALK_STACK_ERROR_PERCENT = 25;
> > GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
> > GEM_SOLO_EXTENT = "$GEMSTONE/bin/extent0.dbf";
> > GEM_STATMONITOR_ARGS = "";
> > GEM_STATMONITOR_MID_CACHE_ARGS = "";
> > GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
> > GEM_TEMPOBJ_CACHE_SIZE = 50000KB;
> > GEM_TEMPOBJ_CODE_SIZE = 0;
> > GEM_TEMPOBJ_CONSECUTIVE_MARKSWEEP_LIMIT = 50;
> > GEM_TEMPOBJ_MESPACE_SIZE = 0;
> > GEM_TEMPOBJ_OOMSTATS_CSV = FALSE;
> > GEM_TEMPOBJ_OOPMAP_SIZE = 0;
> > GEM_TEMPOBJ_PERMGEN_SIZE = 0;
> > GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
> > GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
> > GEM_TEMPOBJ_POMGEN_SIZE = 0;
> > GEM_TEMPOBJ_SCOPES_SIZE = 2000;
> > LOG_WARNINGS = TRUE;
> > SHR_NUM_FREE_FRAME_SERVERS = -1;
> > SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_POLICY = 0;
> > SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_SIZE_MB = 0;
> > SHR_PAGE_CACHE_LOCKED = FALSE;
> > SHR_PAGE_CACHE_NUM_PROCS = 15992;
> > SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
> > SHR_PAGE_CACHE_PERMISSIONS = 0660;
> > SHR_PAGE_CACHE_SIZE_KB = 204800KB;
> > SHR_SPIN_LOCK_COUNT = 5000;
> > SHR_TARGET_FREE_FRAME_COUNT = -1;
> > SHR_WELL_KNOWN_PORT_NUMBER = 0;
> > (vmGc spaceSizes: eden init 2048K max 9344K , survivor init 448K max 1600K old max 37440K
> > vmGc methods max 9408K,14016K doits max 3200K,4672K perm max 5056K, pom 10*4224K=42240K,
> > vmGc remSet 1156K, meSpace max 61708K oopMapSize 262144 max footprint 191M)
> >
> > [Info]: RPC client/gem GCI levels = 37000/37000
> > [Info]: Client PID: 22
> > ________________________________________________________________________________
> > | [Info]: Process ID of my page server is 76196 on stone's machine |
> > |________________________________________________________________________________|
> > --- 05/30/25 03:07:06.451 UTC Login
> > [Info]: User ID: DataCurator
> > [Info]: Repository: gs64stone
> > [Info]: Session ID: 5 login at 05/30/25 03:07:07.141 UTC
> > [Info]: GCI Client Host: 127.0.0.1
> > [Info]: Page server PID: 76196
> > [Info]: using libicu version 58.2
> > Overriding GEM_RPC_USE_SSL to FALSE, client on localhost
> > [tid 129917360973632]: FATAL ERROR 4009 , PageLocate failure pageId 2018 extentId 0 expectedKind 4, lost connection to pgsvr C
> > -----------------------------------------------------
> > GemStone: Error Fatal
> > The gem's connection to the local shared cache monitor was lost.
> > , PageLocate failure pageId 2018 extentId 0 expectedKind 4, lost connection to pgsvr C
> > Error Category: 231169 [GemStone] Number: 4009 Arg Count: 1 Context : 20 exception : 20
> > Arg 1: 20
> > --- 05/30/25 03:13:32.913 UTC Logging out
> >
> > *****************************************************
> > ****** Abnormal Shutdown at 05/30/25 03:13:32.914 UTC
> > *****************************************************
> > -----------------------------------------------------
> > GemStone: Error Fatal
> > The gem's connection to the local shared cache monitor was lost.
> > , PageLocate failure pageId 2018 extentId 0 expectedKind 4, lost connection to pgsvr C
> > Error Category: 231169 [GemStone] Number: 4009 Arg Count: 1 Context : 20 exception : 20Arg 1: 20
> >
> > gemnetobject507cb59490fa9f.log
> >
> > ________________________________________________________________________________
> > Configuration Files
> > -z value is from a default rule
> > System File: /opt/gemstone/data/system.conf
> > Warning, file not found: (null)
> > ________________________________________________________________________________
> > Reading config file /opt/gemstone/data/system.conf
> > ________________________________________________________________________________
> > | Gem Configuration Options for process id 50 |
> > |________________________________________________________________________________|
> >
> > CONFIG_WARNINGS_FATAL = FALSE;
> > DUMP_OPTIONS = TRUE;
> > GEM_ABORT_MAX_CRS = 100000;
> > GEM_CACHE_WARMER_ARGS = "";
> > GEM_CACHE_WARMER_MID_CACHE_ARGS = "";
> > GEM_COMPRESS_TRANLOG_RECORDS = TRUE;
> > GEM_ENV = "";
> > GEM_FREE_FRAME_CACHE_SIZE = -1;
> > GEM_FREE_FRAME_LIMIT = -1;
> > GEM_FREE_PAGEIDS_CACHE = 200;
> > GEM_HALT_ON_ERROR = -1;
> > GEM_KEEP_MIN_SOFTREFS = 0;
> > GEM_KERBEROS_KEYTAB_FILE = "";
> > GEM_KEYRING_DIRS = "";
> > GEM_LISTEN_FOR_DEBUG = FALSE;
> > GEM_MAX_SMALLTALK_STACK_DEPTH = 1000;
> > GEM_NATIVE_CODE_ENABLED = 2;
> > GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = TRUE;
> > GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
> > GEM_PGSVR_FREE_FRAME_LIMIT = -1;
> > GEM_PGSVR_UPDATE_CACHE_ON_READ = TRUE;
> > GEM_PGSVR_USE_SSL = FALSE;
> > GEM_READ_AUTH_ERR_STUBS = FALSE;
> > GEM_REPOSITORY_IN_MEMORY = FALSE;
> > GEM_RPC_KEEPALIVE_INTERVAL = 0;
> > GEM_RPC_USE_SSL = TRUE;
> > GEM_SMALLTALK_STACK_ERROR_PERCENT = 25;
> > GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
> > GEM_SOLO_EXTENT = "$GEMSTONE/bin/extent0.dbf";
> > GEM_STATMONITOR_ARGS = "";
> > GEM_STATMONITOR_MID_CACHE_ARGS = "";
> > GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
> > GEM_TEMPOBJ_CACHE_SIZE = 50000KB;
> > GEM_TEMPOBJ_CODE_SIZE = 0;
> > GEM_TEMPOBJ_CONSECUTIVE_MARKSWEEP_LIMIT = 50;
> > GEM_TEMPOBJ_MESPACE_SIZE = 0;
> > GEM_TEMPOBJ_OOMSTATS_CSV = FALSE;
> > GEM_TEMPOBJ_OOPMAP_SIZE = 0;
> > GEM_TEMPOBJ_PERMGEN_SIZE = 0;
> > GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 50;
> > GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
> > GEM_TEMPOBJ_POMGEN_SIZE = 0;
> > GEM_TEMPOBJ_SCOPES_SIZE = 2000;
> > LOG_WARNINGS = TRUE;
> > SHR_NUM_FREE_FRAME_SERVERS = -1;
> > SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_POLICY = 0;
> > SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_SIZE_MB = 0;
> > SHR_PAGE_CACHE_LOCKED = FALSE;
> > SHR_PAGE_CACHE_NUM_PROCS = 15992;
> > SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
> > SHR_PAGE_CACHE_PERMISSIONS = 0660;
> > SHR_PAGE_CACHE_SIZE_KB = 204800KB;
> > SHR_SPIN_LOCK_COUNT = 5000;
> > SHR_TARGET_FREE_FRAME_COUNT = -1;
> > SHR_WELL_KNOWN_PORT_NUMBER = 0;
> > (vmGc spaceSizes: eden init 2048K max 9344K , survivor init 448K max 1600K old max 37440K
> > vmGc methods max 9408K,14016K doits max 3200K,4672K perm max 5056K, pom 10*4224K=42240K,
> > vmGc remSet 1156K, meSpace max 61708K oopMapSize 262144 max footprint 191M)
> >
> > [Info]: RPC client/gem GCI levels = 37000/37000
> > [Info]: Client PID: 49
> > --- 05/30/25 03:17:38.242 UTC Logging out
> > -----------------------------------------------------
> > GemStone: Error Fatal
> > Session lost connection to the Stone Repository monitor during login, remote cache startup timed out
> > Error Category: 231169 [GemStone] Number: 4038 Arg Count: 1 Context : 20 exception : 20
> > Arg 1: 20
> >
> > *****************************************************
> > ****** Abnormal Shutdown at 05/30/25 03:17:38.244 UTC
> > *****************************************************
> > -----------------------------------------------------
> > GemStone: Error Fatal
> > Session lost connection to the Stone Repository monitor during login, remote cache startup timed out
> > Error Category: 231169 [GemStone] Number: 4038 Arg Count: 1 Context : 20 exception : 20Arg 1: 20
> >
> > gs64ldi.log
> >
> > root at x1:/srv/gsclient/logs# cat gs64ldi.log
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: NETLDI, GemStone Network Daemon |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 10 DATE: 05/30/25 03:04:28 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > |________________________________________________________________________________|
> >
> > synthesizing :: , listening on wildcard
> > Using OS translation of service name gs64ldi to port 50377
> > Created listening socket on :: port 50377
> >
> > Server socket gs64ldi created
> > Summary of netldi parameters:
> > The host name is '7cb59490fa9f'.
> > GEMSTONE is '/opt/gemstone'.
> > GEMSTONE_EXE_CONF is '/opt/gemstone/data/gem.conf'.
> > Authentication is required for ALL accesses.
> > Process creation is permitted through user's HOME directory.
> > Created processes belong to client's account.
> > The default directory for child log files is /opt/gemstone/log
> > The command line is:
> > /opt/gemstone/sys/netldid -s -D /opt/gemstone/log
> > current directory is /--- 05/30/25 03:04:28.368 UTC Entering Service Loop
> >
> > gs64stone_27cachepgsvr_7cb59490fa9f.log
> >
> > | NetLDI Child Task |
> >
> > NRS from client !#encrypted:gsadmin at password#dir:%D#log:%D/gs64stone_%Pcachepgsvr_%M.log#monitor!runcachepgsvr gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33883 TCP
> > CLIENTHOST: ::1
> > COMMAND: /opt/gemstone/sys/runcachepgsvr gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33883 TCP
> > ||
> > runcachepgsvrmainr[Info]: Description of arguments:
> > the hostname is: 7cb59490fa9f
> > GEMSTONE is: /opt/gemstone
> > pgsvr arguments are: gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33883 TCP
> > [Info]: Loaded /opt/gemstone/lib/libpgsvr-3.7.2-64.so
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: PGSVRSHR, GemStone Networked DBF I/O Service (shared library) |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 27 DATE: 05/30/25 03:07:06 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > +--------------------------------------------------------------------------------+
> > | GEMSTONE_NRS_ALL = #dir:%D#log:%D/gs64stone_%Pcachepgsvr_%M.log |
> > ||
> >
> > command line is:
> > /opt/gemstone/sys/pgsvrmain gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33883 TCP
> >
> > [Info]: The hostname is 7cb59490fa9f hostId 7608401484507935676
> >
> > [Info]: Main thread ID: 126267610625856
> > [Warning]: Write to /proc/27/oom_score_adj failed with EACCES , linux user does not have CAP_SYS_RESOURCE
> > [Warning]: No server process protection from OOM killer
> > 2025-05-30 03:07:06.265 UTC [tid 0x72d6f813f740]: finishNetConnection: no reply needed
> > 2025-05-30 03:07:06.265 UTC [tid 0x72d6f813f740]: Network connection has been inherited.
> > 2025-05-30 03:07:06.385 UTC [tid 0x72d6f813f740]: [Info]: Starting shrpcOobMain thread 0 tid 0
> > 2025-05-30 03:07:06.386 UTC [tid 0x72d6f813f740]: [Info]: Starting FFThread thread 1 tid 0
> > 2025-05-30 03:07:06.396 UTC [tid 0x72d6e95fe6c0]: [Info]: Starting FreeFrame thread 1 tid 126267363944128[tid 0x72d6f813f740] --- 05/30/25 03:07:06.397 UTC Entering Service Loop
> >
> > gs64stone_29pcmon_7cb59490fa9f.log
> >
> > root at x1:/srv/gsclient/logs# cat gs64stone_29pcmon_7cb59490fa9f.log
> > ________________________________________________________________________________
> > | GemStone Child Task |
> > | |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 29 DATE: 05/30/25 03:07:06 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > | COMMAND: /opt/gemstone/sys/shrpcmonitor gs64stone~ada6cb43e22fbc5f 12800 13 |
> > | 0 5000 128 1 1900 0 432 0 0 60 0 0 |
> > ||
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: SHRPCMON, GemStone SharedPageCache Monitor |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 29 DATE: 05/30/25 03:07:06 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > +--------------------------------------------------------------------------------+
> > | GEMSTONE_NRS_ALL = #dir:%D#log:%D/gs64stone_%Pcachepgsvr_%M.log |
> > ||
> > 2025-05-30 03:07:06.353 UTC [tid 0x7874119a9740]: hostId is 7608401484507935676
> >
> > Cache config is 12800 pages = 200 MB, total is 210 MB, overhead 5% of configured size
> > HashTableSize 16384 TotalPces 30016 freeListPces 13632
> > SHR_PAGE_CACHE_NUM_PROCS = 13
> > SHR_SPIN_LOCK_COUNT = 5000
> > SHR_TARGET_FREE_FRAME_COUNT = 128
> > SHR_NUM_FREE_FRAME_SERVERS = 1 (thread count)
> > SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900
> > SHR_WELL_KNOWN_PORT_NUMBER = 0 (listening port)
> > SHR_PAGE_CACHE_LOCKED = 0
> > SHR_PAGE_CACHE_PERMISSIONS = 0660
> > SHR_PUSH_TO_MIDCACHES_THREADS = 0
> >
> > NOTE: Shared cache is not locked into memory.
> > Shared memory region number 1 has shmid = 0
> > The shared semaphore array has semid = 0
> >
> > --- 05/30/25 03:07:06.375 UTC cache creation took 0.022 seconds
> >
> > Created listening socket on ::1 port 43133
> > synthesizing listen on 127.0.0.1
> > Created listening socket on 127.0.0.1 port 43133
> >
> > --- 05/30/25 03:07:06.376 UTC Listening for clients on port 43133
> >
> > --- 05/30/25 03:07:06.376 UTC Starting crashed slot recovery thread.
> > [Info]: Starting 1 hash lock recovery test thread.
> > [Info]: Starting 1 frame lock recovery test thread.
> > --- 05/30/25 03:07:06.380 UTC Starting clean slot recovery thread.
> > [Info]: Timeout for stone/cache pgsvr connect is 20 seconds
> > [05/30/25 03:13:32.919 UTC]: Client died: Slot 3, PID 23, LostOtFlags 0, sessionId 5 Name TopazR
> > [05/30/25 03:13:32.919 UTC]: Starting phase 1 crashed client recovery: Slot 3, PID 23, Name TopazR
> > Decremented refCount frame 8 new value 0x0 new count 0 page 6086 kind 5 beginId (26812 0) session 4 crashed process 23 otCachePinnedFrames 0
> > Decremented refCount frame 7 new value 0x0 new count 0 page 437 kind 5 beginId (73 19) session 3 crashed process 23 otCachePinnedFrames 1
> > Decremented refCount frame 6 new value 0x0 new count 0 page 460 kind 5 beginId (73 19) session 3 crashed process 23 otCachePinnedFrames 2
> > Decremented refCount frame 5 new value 0x0 new count 0 page 465 kind 5 beginId (73 19) session 3 crashed process 23 otCachePinnedFrames 3
> > Free frame cache for slot 3: recovering frame 800.
> > Free frame cache for slot 3: recovering frame 801.
> > Free frame cache for slot 3: recovering frame 802.
> > Free frame cache for slot 3: recovering frame 803.
> > Free frame cache for slot 3: recovering frame 804.
> > Free frame cache for slot 3: recovering frame 805.
> > Free frame cache for slot 3: recovering frame 806.
> > Free frame cache for slot 3: 7 frames 7 recoverable.
> > [05/30/25 03:13:32.919 UTC]: Finished phase 1 crashed client recovery: Slot 3, PID 23, Name TopazR
> > Returned 7 recovered frames to free list[Info]: Time waiting for lock test threads was 20078 microsecs
> >
> > gs64stone_54cachepgsvr_7cb59490fa9f.log
> >
> > root at x1:/srv/gsclient/logs# cat gs64stone_54cachepgsvr_7cb59490fa9f.log
> > ________________________________________________________________________________
> > | NetLDI Child Task |
> >
> > NRS from client !#encrypted:gsadmin at password#dir:%D#log:%D/gs64stone_%Pcachepgsvr_%M.log#monitor!runcachepgsvr gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 39617 TCP
> > CLIENTHOST: ::1
> > COMMAND: /opt/gemstone/sys/runcachepgsvr gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 39617 TCP
> > ||
> > runcachepgsvrmainr[Info]: Description of arguments:
> > the hostname is: 7cb59490fa9f
> > GEMSTONE is: /opt/gemstone
> > pgsvr arguments are: gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 39617 TCP
> > [Info]: Loaded /opt/gemstone/lib/libpgsvr-3.7.2-64.so
> > ________________________________________________________________________________
> > | GemStone/S64 Object-Oriented Data Management System |
> > | Copyright (C) GemTalk Systems 1986-2024 |
> > | All rights reserved. |
> > +--------------------------------------------------------------------------------+
> > | PROGRAM: PGSVRSHR, GemStone Networked DBF I/O Service (shared library) |
> > | VERSION: 3.7.2, Tue Dec 17 11:05:28 2024 (branch 3.7.2) |
> > | COMMIT: 2024-12-17T10:36:35-08:00 b01090c78195bf9fdff5f0ba518227ed7e5bb3fa |
> > | BUILT FOR: x86-64 (Linux) |
> > | RUNNING ON: 12-CPU 7cb59490fa9f x86_64 (Linux 6.11.0-26-generic #26~24.04.1-Ubuntu
> > | SMP PREEMPT_DYNAMIC Thu Apr 17 19:20:47 UTC 2) |
> > | PROCESSOR: 13th Gen Intel(R) Core(TM) i7-1355U (Raptor Lake) |
> > | MEMORY: 31757 MB , getauxval(AT_SECURE) = 1 |
> > | PROCESS ID: 54 DATE: 05/30/25 03:16:08 UTC (UTC +0:00) |
> > | USER IDS: REAL=gsadmin (5000) EFFECTIVE=root (0) |
> > +--------------------------------------------------------------------------------+
> > | GEMSTONE_NRS_ALL = #dir:%D#log:%D/gs64stone_%Pcachepgsvr_%M.log |
> > ||
> >
> > command line is:
> > /opt/gemstone/sys/pgsvrmain gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 39617 TCP
> >
> > [Info]: The hostname is 7cb59490fa9f hostId 7608401484507935676
> >
> > [Info]: Main thread ID: 135876414183232
> > [Warning]: Write to /proc/54/oom_score_adj failed with EACCES , linux user does not have CAP_SYS_RESOURCE
> > [Warning]: No server process protection from OOM killer
> >
> > Shared cache named 'gs64stone~ada6cb43e22fbc5f' (pid 29) already exists on this host.
> > Illegal attempt to connect to stale shared cache. Shutdown stale shared cache, remove lock file and retry
> >
> > I didn't do the last 2 tests:
> >
> > - where I logout of topaz and reboot. In this one the client reboots, the containers come up without the cache loop and I am able to login to topaz and access the stone.
> > - where I do a docker compose down and reboot. In this case I do a docker compose up and all works as normal
> >
> > On test one above, the continuous restarts are because the netldi can't reach the cache (when I do a KEEP_LOG they are the same as seen here) much like seen in test two.
> >
> > Ian
> >
> > Sent with Proton Mail secure email.
> >
> > On Thursday, May 29th, 2025 at 11:50 PM, James Foster Smalltalk at JGFoster.net wrote:
> >
> > > And if you hard-reset a machine, what do you get?
> > >
> > > > On May 29, 2025, at 7:40?PM, Ian icjohnson at protonmail.com wrote:
> > > >
> > > > Sure James,
> > > >
> > > > Server side:
> > > > gsadmin at debian ~
> > > > ? gslist -vl
> > > > Status Version Owner Pid Port Started Type Name
> > > > ------- --------- --------- -------- ----- ------------ ------ ----
> > > > OK 3.7.2 root 72588 50377 May 29 00:45 Netldi gs64ldi
> > > > OK 3.7.2 gsadmin 72793 34459 May 29 00:52 Stone gs64stone
> > > > OK 3.7.2 gsadmin 72794 42869 May 29 00:52 cache gs64stone~ada6cb43e22fbc5f
> > > >
> > > > gsadmin at debian ~
> > > > ? gslist -cvl
> > > > Status Version Owner Pid Port Started Type Name
> > > > ------- --------- --------- -------- ----- ------------ ------ ----
> > > > OK 3.7.2 root 72588 50377 May 29 00:45 Netldi gs64ldi
> > > > OK 3.7.2 gsadmin 72793 34459 May 29 00:52 Stone gs64stoneOK 3.7.2 gsadmin 72794 42869 May 29 00:52 cache gs64stone~ada6cb43e22fbc5f
> > > >
> > > > Client side:
> > > > gsadmin at 73a6df6448b8:/$ gslist -vl
> > > > Status Version Owner Pid Port Started Type Name
> > > > ------- --------- --------- -------- ----- ------------ ------ ----
> > > > OK 3.7.2 root 10 50377 May 29 23:45 Netldi gs64ldi
> > > > OK 3.7.2 root 25 36973 May 29 23:46 cache gs64stone~ada6cb43e22fbc5f
> > > > gsadmin at 73a6df6448b8:/$ gslist -cvl
> > > > Status Version Owner Pid Port Started Type Name
> > > > ------- --------- --------- -------- ----- ------------ ------ ----
> > > > OK 3.7.2 root 10 50377 May 29 23:45 Netldi gs64ldiOK 3.7.2 root 25 36973 May 29 23:46 cache gs64stone~ada6cb43e22fbc5f
> > > >
> > > > The other client machine is the same,
> > > >
> > > > Ian
> > > >
> > > > Sent with Proton Mail secure email.
> > > >
> > > > On Thursday, May 29th, 2025 at 11:05 PM, James Foster Smalltalk at JGFoster.net wrote:
> > > >
> > > > > Ian,
> > > > >
> > > > > I?ve not studied your details yet, but I?d be interested in finding out the impact of a couple variations on gslist (https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/MAIN.htm):
> > > > >
> > > > > cslist -vl
> > > > > gslist -cvl
> > > > >
> > > > > In particular, the `-c` option ?Removes locks left by servers that have been killed.?
> > > > >
> > > > > James
> > > > >
> > > > > > On May 29, 2025, at 6:50?PM, Ian icjohnson at protonmail.com wrote:
> > > > > >
> > > > > > James.
> > > > > >
> > > > > > My lab:
> > > > > >
> > > > > > Bare Meta Server:
> > > > > >
> > > > > > - 32GB RAM
> > > > > > - x86_64 12 Core Intel
> > > > > > - Stone: gs64stone
> > > > > > - NetLdi: gs64ldi
> > > > > > - Shared Memory 8 GB (Community 10GB x509 trial license - keeping cache in memory while minimal)
> > > > > > - Operation:
> > > > > >
> > > > > > - ? ls -la gem netldid pgsvr pgsvrmain stoned
> > > > > > -r-sr-xr-x 1 gsadmin gsadmin 118024 Dec 17 16:16 gem
> > > > > > -r-sr-xr-x 1 root root 116560 Dec 17 16:16 netldid
> > > > > > -r-sr-xr-x 1 gsadmin gsadmin 13029328 Dec 17 16:16 pgsvr
> > > > > > -r-sr-xr-x 1 gsadmin gsadmin 122168 Dec 17 16:16 pgsvrmain-r-sr-xr-x 1 gsadmin gsadmin 24165744 Dec 17 16:17 stoned
> > > > > >
> > > > > > - gslist:
> > > > > >
> > > > > > - ? gslist
> > > > > > Status Version Owner Started Type Name
> > > > > > ------- --------- --------- ------------ ------ ----
> > > > > > exists 3.7.2 root May 29 00:45 Netldi gs64ldi
> > > > > > exists 3.7.2 gsadmin May 29 00:52 Stone gs64stoneexists 3.7.2 gsadmin May 29 00:52 cache gs64stone~ada6cb43e22fbc5f
> > > > > >
> > > > > > Clients x2 (Lenovo x85_64 machines):
> > > > > >
> > > > > > - Docker containers:
> > > > > >
> > > > > > - HAProxy: reverse proxy -> gem to stone, stone to gem throughput
> > > > > > - GsClient; netldi gem rpc client running topaz (app launch via topaz)
> > > > > > - Dockerfiles create custom, Debian based images (All containers behave the same but for "startup.sh")
> > > > > > - Docker container: gslist:
> > > > > >
> > > > > > - root at x1:/opt/puzzleit/compose/gsclient# docker exec -it gsclient /bin/bash
> > > > > > gsadmin at 73a6df6448b8:/$ gslist
> > > > > > Status Version Owner Started Type Name
> > > > > > ------- --------- --------- ------------ ------ ----
> > > > > > exists 3.7.2 root May 29 23:45 Netldi gs64ldiexists 3.7.2 root May 29 23:46 cache gs64stone~ada6cb43e22fbc5f
> > > > > > - Docker container: ps aux:
> > > > > >
> > > > > > - gsadmin at 73a6df6448b8:/$ ps aux
> > > > > > USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
> > > > > > gsadmin 1 0.0 0.0 3928 3056 ? Ss May29 0:00 /bin/bash /opt/gemstone/startup.sh
> > > > > > root 10 0.0 0.0 20092 9300 ? S May29 0:00 /opt/gemstone/sys/netldid -s -D /opt/gemstone/log
> > > > > > gsadmin 11 0.0 0.0 4192 3152 pts/0 Ss May29 0:00 /bin/bash
> > > > > > gsadmin 18 0.0 0.0 18800 8528 pts/0 S+ May29 0:00 topaz -I /opt/gemstone/data/topazini
> > > > > > root 19 0.0 0.1 573212 48016 ? Ssl May29 0:00 /opt/gemstone/sys/gem TCP 6
> > > > > > root 23 0.0 0.0 249896 8000 ? Ssl May29 0:00 /opt/gemstone/sys/pgsvrmain gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 -1 1 1900 0 432 0 0 0 0 debian 34459 60 33907 TCP
> > > > > > root 25 0.2 0.2 493412 87428 ? Sl May29 0:04 /opt/gemstone/sys/shrpcmonitor gs64stone~ada6cb43e22fbc5f 12800 13 0 5000 128 1 1900 0 432 0 0 60 0 0gsadmin 103 0.0 0.0 4192 3248 pts/2 Ss 00:21 0:00 /bin/bash
> > > > > >
> > > > > > Plan for the stone sever:
> > > > > >
> > > > > > - Scale as per Gemstone best practice/admin guide
> > > > > > - Extents/Trans Logs running separate network-based disks, bare metal adding extents as needed
> > > > > > - Potential bare metal midlevel cache(s)
> > > > > >
> > > > > > Plan for remote clients:
> > > > > >
> > > > > > - Eventually K8s deployments
> > > > > > - Scaled using kubernetes deployments
> > > > > > - Proxied via non-ssl-terminated reverse proxy (igems will manage x509 certificate behavior)
> > > > > > - Kubernetes API gateway manages ingress/egress via all the needed dedicated client gem/server gem communication via the proxy
> > > > > > - Micro-services are gem centric containers with "startup.sh" files governing service behavior
> > > > > > - No need for volume management; Stone persistence...
> > > > > >
> > > > > > Client Reboots:
> > > > > >
> > > > > > - If I gracefully log out of the stone using topaz> logout in each of two clients and reboot (containers are set to restart if in not started) them both I am able to:
> > > > > >
> > > > > > - docker exec -it gsclient /bin/bash
> > > > > > - topaz -I /path/to/topazini and login without issue (both in startup.sh (as this script is run on container start) and on the command line of startiup.sh is commented out in the startup file.
> > > > > > - If there is a hard reset none of the above holds true.
> > > > > >
> > > > > > - Upon entering the container starting topaz manually of from within the start file fails with a stale cache error with the gem refusing to start
> > > > > > - It is this behavior that I was enquiring about.
> > > > > >
> > > > > > - Why is this terminal? Does it really need to be?
> > > > > > - My only throughs are very large pages caches taking a long time to repopulate/perhaps you guys have a way to rover so that cache repop is non-destructive?
> > > > > > - This behaviour is also evident if the network connection is terminated for any length of time (I can't say of short blips are recoverable)
> > > > > >
> > > > > > Client reboot points two and three are of concern and their behavior I am trying to understand more fully, cache management and the shared page cache.
> > > > > >
> > > > > > WRT foreground/background, I am having no issues the way I have it but wonder if there are differences in cache behavior using the methods suggested?
> > > > > >
> > > > > > I will be sure to try these methods and see where that takes me.
> > > > > >
> > > > > > Thank you for that and for your time,
> > > > > >
> > > > > > Ian
> > > > > >
> > > > > > Sent with Proton Mail secure email.
> > > > > >
> > > > > > On Thursday, May 29th, 2025 at 7:28 PM, James Foster Smalltalk at JGFoster.net wrote:
> > > > > >
> > > > > > > Ian,
> > > > > > >
> > > > > > > I would prefer a more explicit wait of not exiting than running netldi in the foreground. I?ve seen two recommendations:
> > > > > > > CMD ["tail", "-f", "/dev/null?]
> > > > > > > CMD ["sleep", "infinity"]
> > > > > > >
> > > > > > > > ? the remote page cache becomes stale on system reboot ...
> > > > > > > > Note that the container lives in all the above cases ...
> > > > > > >
> > > > > > > Could you say more about your architecture? You have the stone in one Docker container and a gem in another Docker container? Which container lives and which one reboots?
> > > > > > >
> > > > > > > The idea of scaling gems with Docker sounds plausible, but I?ve not heard of anyone doing it.
> > > > > > >
> > > > > > > James
> > > > > > >
> > > > > > > > On May 27, 2025, at 7:34?PM, Ian via GemStone-Smalltalk gemstone-smalltalk at lists.gemtalksystems.com wrote:
> > > > > > > >
> > > > > > > > Hi Norm, James,
> > > > > > > >
> > > > > > > > I think this is worth exploring. Please bear with me.
> > > > > > > >
> > > > > > > > To answer your question Norm, container entry points, however you run them (docker run, compose entrypoint:, entry point in the dockerfile, whatever) runs as PID 1.
> > > > > > > >
> > > > > > > > I want to run a script, call it startup.sh, like this in my dockerfile:
> > > > > > > >
> > > > > > > > ENTRYPOINT: ["/some/container/path/startup.sh"]
> > > > > > > >
> > > > > > > > The scripts job is to:
> > > > > > > >
> > > > > > > > 1. startnetldi -s
> > > > > > > > 2. topaz -I topazini -S instructions.topaz (simplified)
> > > > > > > >
> > > > > > > > Once these two requirements are met, startup.sh exits. Meaning PID 1 also exits and the container shuts down (depending on you compose settings, an endless restart likely ensues).
> > > > > > > >
> > > > > > > > Now, when I run step one as 'netldid -s,' netldid runs on the foreground. PID 1 lives on and the container persists. Connections to the stone live, my application runs in the gem running in the container and all is good.
> > > > > > > >
> > > > > > > > The issue I am now trying to resolve is the fact that the remote page cache becomes stale on system reboot, system crashes or connection interruptions.
> > > > > > > >
> > > > > > > > Note that the container lives in all the above cases, netldid process persists and is happy but a stale page cache is terminal.
> > > > > > > >
> > > > > > > > I must bring down the containers by some manual method: docker compose down, or in terms of kubernetes, restart the deployment, to have the cache regenerated from scratch.
> > > > > > > >
> > > > > > > > This is fine as everything persists in the stone. However, provisions must be made to handle containers whose connection to the stone is terminated/interrupted for whatever the reason.
> > > > > > > >
> > > > > > > > From here:
> > > > > > > >
> > > > > > > > My goal is to run all connections to the stone using x509 enabled gems for all connecting entities of whatever type. The above is the beginnings of a POC to test what hurdles had to be overcome; Whether or not it would even be possible. It is doable, I think.
> > > > > > > >
> > > > > > > > Having started the full read of the x509 guide, I come to realize that the cache is created at the prompting from the server-side x509 enabled netldi through the host agent. In this case, it seem that it may be possible to build container resilience into what I a trying to do as the remote netldi must be alive, requesting a login from the server by saying "here I am, here is my port." What I don't know is that if the remote netldi sees a local cache will it still fail or will it say, you silly cache, you're busted, let's clean up and make a new one.
> > > > > > > >
> > > > > > > > If this can be made to work Gemstone/S remote gems could sit behind some kind of load balancer and run HA in docker, docker swarm or any kubernetes variety fully and robustly containerized and talk to a stone that sits as they do now, on a dedicated server managing connections, transactions and persisting data to extents until the cows come home.
> > > > > > > >
> > > > > > > > I feel like this has probably already been done but can't find a trailblazer on the web to guide me.
> > > > > > > >
> > > > > > > > I do know that I can run netldi on many nodes, bare metal or VM fronted by some form of load balancer for HA, scalability is simply much easier using containers and a container management system of some kind.
> > > > > > > >
> > > > > > > > Seems to me that remote gems are the perfect, secure and safe and scalable way to run Smalltalk based microservices in remote containerize gems with persistent object storage to boot!
> > > > > > > >
> > > > > > > > Long store short, to my question: Am I approaching this in the correct way? Is what I am trying to do even possible?
> > > > > > > >
> > > > > > > > Ian
> > > > > > > >
> > > > > > > > Sent with Proton Mail secure email.
> > > > > > > >
> > > > > > > > On Tuesday, May 27th, 2025 at 4:00 PM, gemstone-smalltalk-request at lists.gemtalksystems.com gemstone-smalltalk-request at lists.gemtalksystems.com wrote:
> > > > > > > >
> > > > > > > > > Send GemStone-Smalltalk mailing list submissions to
> > > > > > > > > gemstone-smalltalk at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > To subscribe or unsubscribe via the World Wide Web, visit
> > > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> > > > > > > > > or, via email, send a message with subject or body 'help' to
> > > > > > > > > gemstone-smalltalk-request at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > You can reach the person managing the list at
> > > > > > > > > gemstone-smalltalk-owner at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > When replying, please edit your Subject line so it is more specific
> > > > > > > > > than "Re: Contents of GemStone-Smalltalk digest..."
> > > > > > > > >
> > > > > > > > > Today's Topics:
> > > > > > > > >
> > > > > > > > > 1. netldid (Ian)
> > > > > > > > > 2. Re: netldid (James Foster)
> > > > > > > > > 3. Re: netldid (Norm Green)
> > > > > > > > > 4. Re: netldid (Ian)
> > > > > > > > >
> > > > > > > > > ----------------------------------------------------------------------
> > > > > > > > >
> > > > > > > > > Message: 1
> > > > > > > > > Date: Mon, 26 May 2025 19:33:08 +0000
> > > > > > > > > From: Ian icjohnson at proton.me
> > > > > > > > >
> > > > > > > > > To: "gemstone-smalltalk at lists.gemtalksystems.com"
> > > > > > > > > gemstone-smalltalk at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > Subject: [GemStone-Smalltalk] netldid
> > > > > > > > > Message-ID:
> > > > > > > > > PejD_sVleK6zxKRUkWsS3REI1mSSbkKihZU8MKvy-t-r1_b4wyM-WvvRxoRr8gIYVe5uQ4TXtf5G0BxUZ2zFNo-YwKfY6v2do6NGknySb7U=@proton.me
> > > > > > > > >
> > > > > > > > > Content-Type: text/plain; charset="utf-8"
> > > > > > > > >
> > > > > > > > > Hi GemStoners,
> > > > > > > > >
> > > > > > > > > Are there plans in the works to allow netldid to run in the foreground?
> > > > > > > > >
> > > > > > > > > This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
> > > > > > > > >
> > > > > > > > > On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
> > > > > > > > >
> > > > > > > > > Thanks in advance,
> > > > > > > > > Ian
> > > > > > > > >
> > > > > > > > > Sent with Proton Mail secure email.
> > > > > > > > > -------------- next part --------------
> > > > > > > > > An HTML attachment was scrubbed...
> > > > > > > > > URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/67818374/attachment-0001.htm
> > > > > > > > >
> > > > > > > > > ------------------------------
> > > > > > > > >
> > > > > > > > > Message: 2
> > > > > > > > > Date: Mon, 26 May 2025 13:25:35 -0700
> > > > > > > > > From: James Foster Smalltalk at JGFoster.net
> > > > > > > > >
> > > > > > > > > To: Ian icjohnson at proton.me
> > > > > > > > >
> > > > > > > > > Cc: "gemstone-smalltalk at lists.gemtalksystems.com"
> > > > > > > > > gemstone-smalltalk at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > Subject: Re: [GemStone-Smalltalk] netldid
> > > > > > > > > Message-ID: 24CFB521-8477-4700-94F5-3E15CA35E192 at JGFoster.net
> > > > > > > > >
> > > > > > > > > Content-Type: text/plain; charset="utf-8"
> > > > > > > > >
> > > > > > > > > Ian,
> > > > > > > > >
> > > > > > > > > A number of people have created Docker images for GemStone/S. This is fine as long as you map the database to something outside the container (else it will be lost when the container stops).
> > > > > > > > >
> > > > > > > > > See https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/B-CommandReference.htm#pgfId-857966 to let netldid run in the foreground.
> > > > > > > > >
> > > > > > > > > James
> > > > > > > > >
> > > > > > > > > > On May 26, 2025, at 12:33?PM, Ian via GemStone-Smalltalk gemstone-smalltalk at lists.gemtalksystems.com wrote:
> > > > > > > > > >
> > > > > > > > > > Hi GemStoners,
> > > > > > > > > >
> > > > > > > > > > Are there plans in the works to allow netldid to run in the foreground?
> > > > > > > > > >
> > > > > > > > > > This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
> > > > > > > > > >
> > > > > > > > > > On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
> > > > > > > > > >
> > > > > > > > > > Thanks in advance,
> > > > > > > > > >
> > > > > > > > > > Ian
> > > > > > > > > >
> > > > > > > > > > Sent with Proton Mail https://proton.me/mail/home secure email.
> > > > > > > > > > _______________________________________________
> > > > > > > > > > GemStone-Smalltalk mailing list
> > > > > > > > > > GemStone-Smalltalk at lists.gemtalksystems.com
> > > > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> > > > > > > > >
> > > > > > > > > -------------- next part --------------
> > > > > > > > > An HTML attachment was scrubbed...
> > > > > > > > > URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/eebd20d9/attachment-0001.htm
> > > > > > > > >
> > > > > > > > > ------------------------------
> > > > > > > > >
> > > > > > > > > Message: 3
> > > > > > > > > Date: Mon, 26 May 2025 13:43:48 -0700
> > > > > > > > > From: Norm Green norm.green at gemtalksystems.com
> > > > > > > > >
> > > > > > > > > To: gemstone-smalltalk at lists.gemtalksystems.com
> > > > > > > > > Subject: Re: [GemStone-Smalltalk] netldid
> > > > > > > > > Message-ID: 7f994942-644d-4801-806e-a3c3f419962e at gemtalksystems.com
> > > > > > > > >
> > > > > > > > > Content-Type: text/plain; charset="utf-8"; Format="flowed"
> > > > > > > > >
> > > > > > > > > Hi Ian,
> > > > > > > > >
> > > > > > > > > I don't fully understand why you want to run netldi in the foreground.
> > > > > > > > > That said, startnetldi forks $GEMSTONE/sys/netldid with appropriate
> > > > > > > > > arguments, so you could run that in the foreground.
> > > > > > > > >
> > > > > > > > > Norm Green
> > > > > > > > >
> > > > > > > > > On 5/26/2025 12:33 PM, Ian via GemStone-Smalltalk wrote:
> > > > > > > > >
> > > > > > > > > > Hi GemStoners,
> > > > > > > > > >
> > > > > > > > > > Are there plans in the works to allow netldid to run in the foreground?
> > > > > > > > > >
> > > > > > > > > > This would allow the running of startnetldi -s in a container without
> > > > > > > > > > an infinite loop and a long running sleep statement (Better way?)
> > > > > > > > > >
> > > > > > > > > > On that note, I am able to containerize a running client application,
> > > > > > > > > > linked and RPC; Am I running down a road that will eventually bonk me
> > > > > > > > > > in the head?
> > > > > > > > > >
> > > > > > > > > > Thanks in advance,
> > > > > > > > > >
> > > > > > > > > > Ian
> > > > > > > > > >
> > > > > > > > > > Sent with Proton Mail https://proton.me/mail/home secure email.
> > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > GemStone-Smalltalk mailing list
> > > > > > > > > > GemStone-Smalltalk at lists.gemtalksystems.com
> > > > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> > > > > > > > >
> > > > > > > > > -------------- next part --------------
> > > > > > > > > An HTML attachment was scrubbed...
> > > > > > > > > URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/8b7b14ed/attachment-0001.htm
> > > > > > > > >
> > > > > > > > > ------------------------------
> > > > > > > > >
> > > > > > > > > Message: 4
> > > > > > > > > Date: Mon, 26 May 2025 21:05:20 +0000
> > > > > > > > > From: Ian icjohnson at proton.me
> > > > > > > > >
> > > > > > > > > To: James Foster Smalltalk at JGFoster.net
> > > > > > > > >
> > > > > > > > > Cc: "gemstone-smalltalk at lists.gemtalksystems.com"
> > > > > > > > > gemstone-smalltalk at lists.gemtalksystems.com
> > > > > > > > >
> > > > > > > > > Subject: Re: [GemStone-Smalltalk] netldid
> > > > > > > > > Message-ID:
> > > > > > > > > 2OoOBEiO6J-fYbKZwRSH68GLdy_pWxVzfCcdsPW6lpox5PVN-6Qp6m9ZUJHG8YC_xCOnX1pi-PJ2zrOgLQRC86SgPeSX2PT5hpACzicoPGM=@proton.me
> > > > > > > > >
> > > > > > > > > Content-Type: text/plain; charset="utf-8"
> > > > > > > > >
> > > > > > > > > netldid -s
> > > > > > > > >
> > > > > > > > > The wording got me.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > > Ian
> > > > > > > > >
> > > > > > > > > Sent with Proton Mail secure email.
> > > > > > > > >
> > > > > > > > > On Monday, May 26th, 2025 at 5:25 PM, James Foster Smalltalk at JGFoster.net wrote:
> > > > > > > > >
> > > > > > > > > > Ian,
> > > > > > > > > >
> > > > > > > > > > A number of people have created Docker images for GemStone/S. This is fine as long as you map the database to something outside the container (else it will be lost when the container stops).
> > > > > > > > > >
> > > > > > > > > > See https://downloads.gemtalksystems.com/docs/GemStone64/3.7.x/GS64-SysAdminGuide-3.7/B-CommandReference.htm#pgfId-857966 to let netldid run in the foreground.
> > > > > > > > > >
> > > > > > > > > > James
> > > > > > > > > >
> > > > > > > > > > > On May 26, 2025, at 12:33?PM, Ian via GemStone-Smalltalk gemstone-smalltalk at lists.gemtalksystems.com wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi GemStoners,
> > > > > > > > > > >
> > > > > > > > > > > Are there plans in the works to allow netldid to run in the foreground?
> > > > > > > > > > >
> > > > > > > > > > > This would allow the running of startnetldi -s in a container without an infinite loop and a long running sleep statement (Better way?)
> > > > > > > > > > >
> > > > > > > > > > > On that note, I am able to containerize a running client application, linked and RPC; Am I running down a road that will eventually bonk me in the head?
> > > > > > > > > > >
> > > > > > > > > > > Thanks in advance,
> > > > > > > > > > > Ian
> > > > > > > > > > >
> > > > > > > > > > > Sent with Proton Mail secure email.
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > GemStone-Smalltalk mailing list
> > > > > > > > > > > GemStone-Smalltalk at lists.gemtalksystems.com
> > > > > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> > > > > > > > >
> > > > > > > > > -------------- next part --------------
> > > > > > > > > An HTML attachment was scrubbed...
> > > > > > > > > URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250526/b228663a/attachment-0001.htm
> > > > > > > > >
> > > > > > > > > ------------------------------
> > > > > > > > >
> > > > > > > > > Subject: Digest Footer
> > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > GemStone-Smalltalk mailing list
> > > > > > > > > GemStone-Smalltalk at lists.gemtalksystems.com
> > > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
> > > > > > > > >
> > > > > > > > > ------------------------------
> > > > > > > > >
> > > > > > > > > End of GemStone-Smalltalk Digest, Vol 129, Issue 4
> > > > > > > > > **************************************************
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > GemStone-Smalltalk mailing list
> > > > > > > > GemStone-Smalltalk at lists.gemtalksystems.com
> > > > > > > > https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://lists.gemtalksystems.com/mailman/archives/gemstone-smalltalk/attachments/20250530/43f2d716/attachment.htm
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> GemStone-Smalltalk mailing list
> GemStone-Smalltalk at lists.gemtalksystems.com
> https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
>
> ------------------------------
>
> End of GemStone-Smalltalk Digest, Vol 129, Issue 12
> ***************************************************
More information about the GemStone-Smalltalk
mailing list