<div dir="ltr">Johan, thank you very much.<div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td width="400" valign="bottom"><p style="margin:0px;padding:0px"><span style="font-size:18px;color:rgb(146,148,151);font-family:Calibri,sans-serif;font-weight:700">Otto Behrens</span><br></p><p style="font-size:18px;font-weight:700;color:rgb(146,148,151);font-family:Calibri,sans-serif;margin:0px;padding:0px"><span style="font-size:14px;font-weight:300;margin:0px;padding:0px">+27 82 809 2375</span></p></td><td width="200" valign="middle"><img src="https://www.finworks.biz/signature/finworks-signature-logo.png" width="200" height="38" alt="FINWorks" style="display:block;border:0px;width:200px;height:38px;margin:0px;padding:0px"></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td height="5"></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium;border-bottom:1px solid rgb(200,28,36)"><tbody><tr><td height="15"></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td height="20"></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td width="15" valign="top" style="display:inline-block"><a href="http://za.linkedin.com/in/waltherbehrens" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.finworks.biz/signature/finworks-linkedin-logo.png" width="15" height="15" alt="FINWorks" style="display:inline-block;border:0px;width:15px;height:15px;margin-top:1.5px;padding:0px"></a></td><td width="250" valign="top" style="display:inline-block"><a href="http://www.finworks.biz/" style="color:rgb(200,28,36);font-family:Calibri,sans-serif;margin-left:10px;margin-top:0px;padding-top:0px;font-size:11pt;display:inline-block" target="_blank">www.finworks.biz</a></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td height="10"></td></tr></tbody></table><table width="600" cellpadding="0" cellspacing="0" border="0" style="color:rgb(0,0,0);font-family:Times;font-size:medium"><tbody><tr><td><p style="font-size:10px;color:rgb(146,148,151);font-family:Calibri,sans-serif;text-align:justify">Disclaimer & Confidentiality Note: This email is intended solely for the use of the individual or entity named above as it may contain information that is confidential and privileged. If you are not the intended recipient, be advised that any dissemination, distribution or copying of this email is strictly prohibited. FINWorks cannot be held liable by any person other than the addressee in respect of any opinions, conclusions, advice or other information contained in this email.</p></td></tr></tbody></table></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, May 26, 2024 at 1:21 PM Johan Brichau via Glass <<a href="mailto:glass@lists.gemtalksystems.com">glass@lists.gemtalksystems.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>In the meantime, I found the real cause in ZnBivalentWriteStream>>#next:putAll:startingAt: in a change that was made 11 years ago in commit [1] ((by myself... :roll-eyes:) <div><br></div><div>The incoming collection argument is converted to a ByteArray on each call of this iteration.</div><div>The trouble is that this is the entire collection and not just the part that is to be written in the iteration.</div><div>That means that for every 16Kb of data, the entire collection of 55MB (in my testcase) is converted to a ByteArray….</div><div><br></div><div>I’m working on a fix for this in <a href="https://github.com/GsDevKit/zinc/pull/105" target="_blank">https://github.com/GsDevKit/zinc/pull/105</a></div><div>It looks like simply removing the conversion and aligning the code of ZnBivalentWriteStream>>#next:putAll:startingAt: with its original in Pharo fixes the problem altogether.</div><div><br></div><div>I’m not sure if the change was done inadvertently or if it did fix issues. I did some manual tests with Seaside in a GS 3.6.8 and it looked fine.</div><div><br></div><div><div>In any case: fixing this will yield significant performance improvements for anyone running Zinc in GemStone…. not just for very large responses where the problem is very apparent. </div></div><div><br></div><div><div><div>[1] <a href="https://github.com/GsDevKit/zinc/commit/cbddf8b52589ad2107feba8557e27db5cad5acbd#diff-b940261c40f3e320ede03be3b7ac8c7fe5ff81ec310656aa6b557b069cce1dc2" target="_blank">https://github.com/GsDevKit/zinc/commit/cbddf8b52589ad2107feba8557e27db5cad5acbd#diff-b940261c40f3e320ede03be3b7ac8c7fe5ff81ec310656aa6b557b069cce1dc2</a> <br id="m_2570919173568960740lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 25 May 2024, at 10:32, Otto Behrens <<a href="mailto:otto@finworks.biz" target="_blank">otto@finworks.biz</a>> wrote:</div><br><div><div dir="auto">Johan, this is magic, thanks a lot. Wow</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 25 May 2024, 09:11 Johan Brichau via Glass, <<a href="mailto:glass@lists.gemtalksystems.com" target="_blank">glass@lists.gemtalksystems.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Other observations (before I head off doing other worldly things one has to do on Saturdays ;-):<div><br></div><div>- Changing `ZnUtils>>streamingBufferSize` from 16384 to 163840 (i.e. x10) also improves the performance (as expected) to 4s response time for the 55MB file</div><div>- In Pharo, the same code is used and there it takes 3s for the same file, same code (not quite the same Zinc version though, but the methods I mentioned are still the same)</div><div><br></div><div>Johan<br id="m_2570919173568960740m_-9045131995505474928lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 25 May 2024, at 08:57, Johan Brichau <<a href="mailto:johan@yesplan.be" rel="noreferrer" target="_blank">johan@yesplan.be</a>> wrote:</div><br><div><div>The problem seems to be in `ZnUtils>>nextPutAll:on:` that splits the collection into chunks to write it to the Socket.<br>When I remove that method and just pass through the writing of the collection as `stream nextPutAll: collection`, I go from 39s to 1s (see below, before and after)<br><br>Now: understand why that piece of code is there and if it’s needed or not :-)<div><br>Johan<br><br><font face="Courier New">johanbrichau@JohansMacBookAir ~ % time curl <a href="http://localhost:8383/test" rel="noreferrer" target="_blank">http://localhost:8383/test</a> --output bla.txt<br> % Total % Received % Xferd Average Speed Time Time Time Current<br> Dload Upload Total Spent Left Speed<br>100 55.2M 100 55.2M 0 0 1465k 0 0:00:38 0:00:38 --:--:-- 1476k<br>curl <a href="http://localhost:8383/test" rel="noreferrer" target="_blank">http://localhost:8383/test</a> -output bla.txt 0.04s user 0.14s system 0% cpu 38.657 total</font><div><br></div><div><div><font face="Courier New"><br></font></div><div><div><font face="Courier New">johanbrichau@JohansMacBookAir ~ % time curl <a href="http://localhost:8383/test" rel="noreferrer" target="_blank">http://localhost:8383/test</a> --output bla.txt</font></div><div><font face="Courier New"> % Total % Received % Xferd Average Speed Time Time Time Current</font></div><div><font face="Courier New"> Dload Upload Total Spent Left Speed</font></div><div><font face="Courier New">100 55.2M 100 55.2M 0 0 45.0M 0 0:00:01 0:00:01 --:--:-- 45.0M</font></div><div><font face="Courier New">curl <a href="http://localhost:8383/test" rel="noreferrer" target="_blank">http://localhost:8383/test</a> --output bla.txt 0.01s user 0.07s system 6% cpu 1.255 total</font></div><br><br><blockquote type="cite">On 23 May 2024, at 23:45, Johan Brichau <<a href="mailto:johan@yesplan.be" rel="noreferrer" target="_blank">johan@yesplan.be</a>> wrote:<br><br>Hi Otto,<br><br><blockquote type="cite">On 20 May 2024, at 19:08, Otto Behrens <<a href="mailto:otto@finworks.biz" rel="noreferrer" target="_blank">otto@finworks.biz</a>> wrote:<br><br>Okay, I’ll give that a try and see what I can get when using the Zinc adaptor in GemStone…</blockquote><br><br>It took me a bit longer to actually get started since I wanted to debug this on my Mac using a GsDevKit_stones setup and I had to dig into another rabbit hole first :-)<br>Anyway, I setup the simple handler below and I can confirm also notice that a file of 60MB takes 40s to be served and 28MB takes 10s. <br><br>In Pharo, the 60MB file takes 3s.<br>I’ll dive into this a bit more tomorrow…..<br><br>get<br><br> <get><br> <path: '/'><br> <produces: 'application/json'><br><br> | file |<br> file := GsFile openReadOnServer: '/Users/johanbrichau/testfile'.<br> ^ file contents<br><br></blockquote><br></div></div></div></div></div></blockquote></div><br></div></div>_______________________________________________<br> Glass mailing list<br> <a href="mailto:Glass@lists.gemtalksystems.com" rel="noreferrer" target="_blank">Glass@lists.gemtalksystems.com</a><br> <a href="https://lists.gemtalksystems.com/mailman/listinfo/glass" rel="noreferrer noreferrer" target="_blank">https://lists.gemtalksystems.com/mailman/listinfo/glass</a><br> </blockquote></div> </div></blockquote></div><br></div></div></div></div>_______________________________________________<br> Glass mailing list<br> <a href="mailto:Glass@lists.gemtalksystems.com" target="_blank">Glass@lists.gemtalksystems.com</a><br> <a href="https://lists.gemtalksystems.com/mailman/listinfo/glass" rel="noreferrer" target="_blank">https://lists.gemtalksystems.com/mailman/listinfo/glass</a><br> </blockquote></div>