<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">David,<div class=""><br class=""></div><div class="">The error “socket read EOF” indicates that the Gem attempted to read from a socket and received an EOF response. <div class=""><br class=""></div><div class="">Given that the client and the server are in different Docker containers, they are effectively on separate hosts and there is a strong indication that the socket closed between them. Given the timing duration and consistency, my first guess is that a socket is being closed due to inactivity. If you added a commit every minute (say, the last time through the loop), would that change the behavior?</div><div class=""><br class=""></div><div class="">James<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Nov 14, 2021, at 10:40 AM, David Shaffer <<a href="mailto:shaffer@shaffer-consulting.com" class="">shaffer@shaffer-consulting.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">The host is an AWS EC2 instance running Ubuntu (20.04.2) running on AWS (kernel 5.11.0-1020-aws) with Docker (version 20.10.7, build 20.10.7-0ubuntu5~20.04.2). Gemstone and the VisualWorks client are running in separate Ubuntu containers (“latest” on Dockerhub which is 20.04/focal). Docker is running in “swarm mode” on this host and both client and server are swarm services.<div class=""><br class=""></div><div class="">The sleep is 1 second and there is not always work to do. In fact, most loop iterations complete without committing any changes.</div><div class=""><div class=""><br class=""></div><div class="">I’m currently pursuing some sketchy syslog entries on the EC2 host that seem to correspond to the network errors. I’ll share them as soon as I’ve pruned them down a bit. This same setup ran for 6 years <span style="caret-color: rgb(0, 0, 0);" class="">(this EC2 instance for 2 years, my transition to swarm mode was about 1 year ago) </span>with GOODS as backend without network-related errors, though.</div><div class=""><br class=""></div><div class="">-D<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Nov 14, 2021, at 1:29 PM, James Foster <<a href="mailto:smalltalk@jgfoster.net" class="">smalltalk@jgfoster.net</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">David,<div class=""><br class=""></div><div class="">Tell us a bit more about your configuration. Are you running Windows, macOS, or Linux? Is the client inside the Docker container (you mentioned the “entire system”)? How long is the sleep? Is there always work to do? </div><div class=""><br class=""></div><div class="">James</div><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Nov 14, 2021, at 10:22 AM, David Shaffer via GemStone-Smalltalk <<a href="mailto:gemstone-smalltalk@lists.gemtalksystems.com" class="">gemstone-smalltalk@lists.gemtalksystems.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">Hey folks:</div><div class=""><br class=""></div><div class="">I’ve (finally) deployed a server using GemStone 3.6.2, GemBuilder 8.5 and VW 9.0. My server’s main loop is essentially:</div><div class=""><br class=""></div><div class="">Abort</div><div class="">Do work</div><div class="">Commit</div><div class="">Sleep</div><div class=""><br class=""></div><div class="">Every two hours (I’m not sure it is exactly two hours but it seems pretty reliable), I get the following during the abort call:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class="">GS Server Error - GbsGsErrStnNetLost - The session has lost its connection to the Stone Repository monitor.</blockquote><div class=""><br class=""></div><div class=""><div style="caret-color: rgb(0, 0, 0);" class="">The entire system runs on a single host in Docker so it can’t possibly be a network hiccup. The gemnetobject logs are pasted below…they make it seem like a network error but, again, this is highly unlikely. Has anyone else run into this or have any troubleshooting advice?</div></div><div style="caret-color: rgb(0, 0, 0);" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0);" class="">-David</div><div style="caret-color: rgb(0, 0, 0);" class=""><br class=""></div><div style="caret-color: rgb(0, 0, 0);" class=""><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">--- 11/13/21 22:56:18.959 UTC Login</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: Gave this process preference for OOM killer: wrote to /proc/460/oom_score_adj value 250</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: User ID: Trader</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: Repository: gs64stone</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: Session ID: 5 login at 11/13/21 22:56:18.964 UTC</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: GCI Client Host: </span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: Page server PID: -1</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">[Info]: using libicu version 58.2</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">-----------------------------------------------------</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">GemStone: Error Fatal</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Network error - text follows:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">, socket read EOF</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Error Category: 231169 [GemStone] Number: 4137 Arg Count: 1 Context : 20 exception : 20</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Arg 1: 20</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">--- 11/13/21 23:03:16.322 UTC Logging out</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85); min-height: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85); min-height: 14px;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""></span><br class=""></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">*****************************************************</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">****** Abnormal Shutdown at 11/13/21 23:03:16.824 UTC</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">*****************************************************</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">-----------------------------------------------------</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">GemStone: Error Fatal</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Network error - text follows:</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">, socket read EOF</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Error Category: 231169 [GemStone] Number: 4137 Arg Count: 1 Context : 20 exception : 20</span></div><div style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: "MesloLGS NF"; color: rgb(124, 105, 56); background-color: rgba(255, 170, 161, 0.85);" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">Arg 1: 20</span></div><div class=""><span style="font-variant-ligatures: no-common-ligatures" class=""><br class=""></span></div></div></div>_______________________________________________<br class="">GemStone-Smalltalk mailing list<br class=""><a href="mailto:GemStone-Smalltalk@lists.gemtalksystems.com" class="">GemStone-Smalltalk@lists.gemtalksystems.com</a><br class=""><a href="https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk" class="">https://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk</a><br class=""></div></blockquote></div><br class=""></div></div></div></blockquote></div><br class=""></div></div></div></div></blockquote></div><br class=""></div></div></body></html>