<div dir="ltr"><div dir="ltr">Hi Lourens,<br clear="all"><div><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><br></div></div></div></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div lang="EN-US"><div>
<p class="MsoNormal">Least connect load balancing on nginx does not mean each upstream will only get one call and the rest will go to the other open ports. Underload and depending on below setting on each upstream, multiple calls will go to each upstream.</p></div></div></div></blockquote><div><br></div><div>I'm not quite following you. I did not intend to say each upstream will get one call, sorry if I was misleading. What I understand is that if an upstream is busy with "one call", other upstreams should receive subsequent calls. The upstream with the least number of connections waiting in the queue should receive more requests than others. At least, this is how I interpret how it works. If I don't get it, let me know.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><p class="MsoNormal"><u></u><u></u></p>
<p class="MsoNormal"><u></u> Each socket/port (say ie. port 8001) has its own queue see:</p><p class="MsoNormal"><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">makeListener: queueLength<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">"Turns the receiver into a listening socket.<u></u><u></u></p>
<p class="MsoNormal">The queueLength argument specifies the size of the listen backlog queue for<u></u><u></u></p>
<p class="MsoNormal">incoming connections.<u></u><u></u></p>
<p class="MsoNormal">Returns the receiver or nil if an error occurred."<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">My experience, nginx will give each port multiple requests until the above queue is full. Thus fast calls could be stuck behind slow calls. Making this param 1 will in essence force nginx to only give each port one request, thus if 10 port
are available and 20 requests comes in, things will start to go pear shape if some are slow requests.<br></p></div></blockquote><div><br></div><div>If there are many slow calls simultaneously requested, yes, I agree, the distribution can be unfortunate. If there are fewer slow calls and some upstreams manage to serve faster calls, at least some calls will go through. It is not perfect, yes.</div><div><br></div><div>This is not our problem. We see calls routed to an upstream that is unable to handle the call.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><div><p class="MsoNormal">
<u></u><u></u></p>
<p class="MsoNormal"><u></u> See: <a href="https://github.com/jgfoster/WebGS" target="_blank">https://github.com/jgfoster/WebGS</a> for a possible idea about the threading.</p></div></div></blockquote><div><br></div><div>Maybe</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><div><p class="MsoNormal"><u></u></p>
<p class="MsoNormal"><u></u> In regards to autoscaling, see Kubernetes concept about this.</p><p class="MsoNormal"><u></u></p>
<p class="MsoNormal"><u></u></p></div></div></blockquote><div><br></div><div>This is not in our scope and does not make sense for us as a solution to this problem</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-US"><div><p class="MsoNormal"> <u></u></p>
<p class="MsoNormal">Regards<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Lourens<u></u><u></u></p>
<p class="MsoNormal">
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Otto Behrens <<a href="mailto:otto@finworks.biz" target="_blank">otto@finworks.biz</a>> <br>
<b>Sent:</b> Thursday, December 21, 2023 9:01 AM<br>
<b>To:</b> Lourens van Nieuwenhuizen <<a href="mailto:LvNieuwenhuizen@momentum.co.za" target="_blank">LvNieuwenhuizen@momentum.co.za</a>><br>
<b>Cc:</b> Iwan Vosloo <<a href="mailto:iwan@finworks.biz" target="_blank">iwan@finworks.biz</a>>; <a href="mailto:glass@lists.gemtalksystems.com" target="_blank">glass@lists.gemtalksystems.com</a><br>
<b>Subject:</b> Re: [Glass] load balancer configuration<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Lourens,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks for the reply. <br clear="all">
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal">You probably need to look at a multi thread/auto scaling solution. Forcing a timeout will probably give the client a bad experience if it happens often.<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I think this is what happens anyway: there are timeouts (in the nginx setup, eg. proxy_read_timeout and proxy_connect_timeout) that will timeout the upstream connections. And this causes a bad experience for the user. We want to manage
this better.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal">Usually each socket has a queue to “hold new requests” while the first one is being processed. (Making that 1 will cause problems on systems under load).
<u></u><u></u></p>
<p class="MsoNormal">Thus GemStone need to be able process more than 1 request on that queue, so if there are 5 requests on the queue, and no. 1 is a slow one, no. 2 – 5 will wait until no. 1 is complete.<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">We start up a limited number of sessions and at this point it is ok if there are some requests waiting in a queue. Nginx decides which queues have the least number of connections and routes the requests to that one. So the idea is that
requests are routed to queues that get serviced quickly. Yes, under load it may happen that some requests get stuck behind a process that takes too long. We are working on those.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The particular problem that I'm trying to prevent is that requests are routed to a queue where the GemStone process is still busy (and unable to handle the request), while other queues are empty (and able to handle requests). This is caused
by the fact that a browser starts a heavy process and then navigates away (this is what I tried to explain in my original email).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal">Now if GemStone process can process 2 requests per socket requests 2-5 will not wait for call no.1, thus the fast calls will be processed as normal.
<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">If 2 slow requests sits in a row, you will have the same issue, 3-5 will need to wait. This is where autoscaling comes in. GemStone needs to note request 1 is slow thus start a
3<sup>rd</sup> thread up incase no. 2 also is a slow request. If no. 2 is slow then start a 4<sup>th</sup> thread etc. And if things speed up again reduce the threads. Or always keep 3 threads and scale up when 1 request show slowness. <u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">This will also mean each thread have it’s own logged in session so that one thread does not persist 50% of another call’s data.<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This may be something that we can investigate later. We currently start up multiple sessions that do not use threading. I have not encountered a setup where a GS thread could have its own logged in session (are there references that you
can give to this, please?). I have not encountered "autoscaling" in the documentation or other literature (for this kind of context) before. Can you refer me to where I can read more about this?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal">In the interim you could increase the ports on GemStone and nginx if possible. This will not remove the problem but hopefully reduce the occurrence.<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">We are running 4 to 8 GemStone sessions (topaz) for web users (depending on how busy the instance is). They seem to handle the load well, except in the situation that we're struggling with.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Regards<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Lourens<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Glass <<a href="mailto:glass-bounces@lists.gemtalksystems.com" target="_blank">glass-bounces@lists.gemtalksystems.com</a>>
<b>On Behalf Of </b>Otto Behrens via Glass<br>
<b>Sent:</b> Wednesday, December 20, 2023 2:04 PM<br>
<b>To:</b> <a href="mailto:glass@lists.gemtalksystems.com" target="_blank">glass@lists.gemtalksystems.com</a><br>
<b>Cc:</b> Iwan Vosloo <<a href="mailto:iwan@finworks.biz" target="_blank">iwan@finworks.biz</a>><br>
<b>Subject:</b> [Glass] load balancer configuration<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Hi,<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">We are using nginx to load balance in front of GemStone that runs a Seaside application. Some of our requests run too long (we are working hard to cut them down) and in general,
the time it takes to service a request in our application varies between 0.1 and about 4 seconds. We are improving and getting more towards the lower end of that. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Because of this, we use the least_conn directive and we persist session state so that we could use any of our GemStone upstream sessions to service a request. Requests are generally
load balanced to idle sessions and there are theoretically no requests that wait for another to get serviced. Perhaps this is not optimal and you have better suggestions. It has worked ok for a long time, but should we consider another approach?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">When our code misbehaves and a request takes let's say 60 seconds to handle, things go pear shaped (yes we want to eliminate them). The user clicks "back" on the browser or closes
the browser and nginx picks it up with: <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">"epoll_wait() reported that client prematurely closed connection, so upstream connection is closed too while sending request to upstream"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">We suspect our problem is: when this happens, it appears as if nginx then routes requests to that same upstream, which is unable to handle it because it is busy handling the previous
request (which is taking too long), even with some upstream sessions sitting idle. Some users then end up with no response.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Ideally, we would like to catch the situation in the GemStone session and stop processing the request (when nginx closes the upstream connection). Alternatively, we could set timeouts
long enough so that if the browser prematurely closes the connection, nginx does not close the upstream connection. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Do you have a suggestion to handle this? Does it make sense to get timeouts (which ones?) to align so that this does not happen?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks a lot<u></u><u></u></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr>
<td width="400" valign="bottom" style="width:300pt;padding:0cm">
<p style="margin:0cm"><b><span style="font-size:13.5pt;color:rgb(146,148,151)">Otto Behrens</span></b><u></u><u></u></p>
<p style="margin:0cm"><span style="font-size:10.5pt;color:rgb(146,148,151)">+27 82 809 2375</span><u></u><u></u></p>
</td>
<td width="200" style="width:150pt;padding:0cm">
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Times,serif;color:black"><img border="0" width="200" height="38" style="width:2.0833in;height:0.3958in" id="m_-6764486699726307452m_-8078184374782115618m_2374925886526739079_x0000_i1026" src="https://www.finworks.biz/signature/finworks-signature-logo.png" alt="FINWorks"></span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr style="height:3.75pt">
<td style="padding:0cm;height:3.75pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="1" cellspacing="0" cellpadding="0" width="600" style="width:450pt;border-top:none;border-right:none;border-left:none;border-bottom:1pt solid rgb(200,28,36)">
<tbody>
<tr style="height:11.25pt">
<td style="border:none;padding:0cm;height:11.25pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr style="height:15pt">
<td style="padding:0cm;height:15pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr>
<td width="15" valign="top" style="width:11.25pt;padding:0cm;display:inline-block">
<p class="MsoNormal"><a href="http://za.linkedin.com/in/waltherbehrens" target="_blank"><span style="font-size:13.5pt;font-family:Times,serif;color:rgb(17,85,204);text-decoration:none"><img border="0" width="15" height="15" style="width:0.1562in;height:0.1562in" id="m_-6764486699726307452m_-8078184374782115618m_2374925886526739079_x0000_i1025" src="https://www.finworks.biz/signature/finworks-linkedin-logo.png" alt="FINWorks"></span></a><u></u><u></u></p>
</td>
<td width="250" valign="top" style="width:187.5pt;padding:0cm;display:inline-block">
<p class="MsoNormal"><span style="font-size:13.5pt;font-family:Times,serif;color:black"><a href="http://www.finworks.biz" target="_blank"><span style="font-size:11pt;font-family:Calibri,sans-serif;color:rgb(200,28,36)">www.finworks.biz</span></a></span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr style="height:7.5pt">
<td style="padding:0cm;height:7.5pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
<table border="0" cellspacing="0" cellpadding="0" width="600" style="width:450pt">
<tbody>
<tr>
<td style="padding:0cm">
<p style="text-align:justify"><span style="font-size:7.5pt;color:rgb(146,148,151)">Disclaimer & Confidentiality Note: This email is intended solely for the use of the individual or entity named above as it may contain information that is confidential and privileged.
If you are not the intended recipient, be advised that any dissemination, distribution or copying of this email is strictly prohibited. FINWorks cannot be held liable by any person other than the addressee in respect of any opinions, conclusions, advice or
other information contained in this email.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12pt"><u></u> <u></u></p>
<p style="margin-bottom:12pt"><span style="font-size:8pt;font-family:Verdana,sans-serif;color:rgb(102,102,102)">This electronic message, including any attachments, is intended only for the use of the individual or entity to which it is addressed and may contain
information that is privileged or confidential. If you are not the intended recipient, please notify us immediately by replying to this message and then delete this message from your system. Any use, dissemination, distribution or reproduction of this message
or any attachments by unintended recipients is unauthorised and may be unlawful. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to perform your own virus checks on any attachment to this message. We do not
accept liability for any loss or damage caused by software viruses. <u></u><u></u></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<br><br>
<p style="font-family:Verdana;font-size:8pt;color:rgb(102,102,102)">
This electronic message, including any attachments, is intended only for the use of the individual or entity to which it is addressed and may contain information that is privileged or confidential. If you are not the intended recipient, please notify us immediately by replying to this message and then delete this message from your system. Any use, dissemination, distribution or reproduction of this message or any attachments by unintended recipients is unauthorised and may be unlawful. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to perform your own virus checks on any attachment to this message. We do not accept liability for any loss or damage caused by software viruses.
<br><br><br>
</p></div>
</blockquote></div>
</div>