<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">In the meantime, I found the real cause in ZnBivalentWriteStream>>#next:putAll:startingAt: in a change that was made 11 years ago in commit [1] ((by myself... :roll-eyes:) <div><br></div><div>The incoming collection argument is converted to a ByteArray on each call of this iteration.</div><div>The trouble is that this is the entire collection and not just the part that is to be written in the iteration.</div><div>That means that for every 16Kb of data, the entire collection of 55MB (in my testcase) is converted to a ByteArray….</div><div><br></div><div>I’m working on a fix for this in <a href="https://github.com/GsDevKit/zinc/pull/105">https://github.com/GsDevKit/zinc/pull/105</a></div><div>It looks like simply removing the conversion and aligning the code of ZnBivalentWriteStream>>#next:putAll:startingAt: with its original in Pharo fixes the problem altogether.</div><div><br></div><div>I’m not sure if the change was done inadvertently or if it did fix issues. I did some manual tests with Seaside in a GS 3.6.8 and it looked fine.</div><div><br></div><div><div>In any case: fixing this will yield significant performance improvements for anyone running Zinc in GemStone…. not just for very large responses where the problem is very apparent. </div></div><div><br></div><div><div><div>[1] <a href="https://github.com/GsDevKit/zinc/commit/cbddf8b52589ad2107feba8557e27db5cad5acbd#diff-b940261c40f3e320ede03be3b7ac8c7fe5ff81ec310656aa6b557b069cce1dc2">https://github.com/GsDevKit/zinc/commit/cbddf8b52589ad2107feba8557e27db5cad5acbd#diff-b940261c40f3e320ede03be3b7ac8c7fe5ff81ec310656aa6b557b069cce1dc2</a> <br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 25 May 2024, at 10:32, Otto Behrens <otto@finworks.biz> wrote:</div><br class="Apple-interchange-newline"><div><div dir="auto">Johan, this is magic, thanks a lot. Wow</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 25 May 2024, 09:11 Johan Brichau via Glass, <<a href="mailto:glass@lists.gemtalksystems.com">glass@lists.gemtalksystems.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-break:after-white-space">Other observations (before I head off doing other worldly things one has to do on Saturdays ;-):<div><br></div><div>- Changing `ZnUtils>>streamingBufferSize` from 16384 to 163840 (i.e. x10) also improves the performance (as expected) to 4s response time for the 55MB file</div><div>- In Pharo, the same code is used and there it takes 3s for the same file, same code (not quite the same Zinc version though, but the methods I mentioned are still the same)</div><div><br></div><div>Johan<br id="m_-9045131995505474928lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 25 May 2024, at 08:57, Johan Brichau <<a href="mailto:johan@yesplan.be" target="_blank" rel="noreferrer">johan@yesplan.be</a>> wrote:</div><br><div><div style="line-break:after-white-space">The problem seems to be in `ZnUtils>>nextPutAll:on:` that splits the collection into chunks to write it to the Socket.<br>When I remove that method and just pass through the writing of the collection as `stream nextPutAll: collection`, I go from 39s to 1s (see below, before and after)<br><br>Now: understand why that piece of code is there and if it’s needed or not :-)<div><br>Johan<br><br><font face="Courier New">johanbrichau@JohansMacBookAir ~ % time curl <a href="http://localhost:8383/test" target="_blank" rel="noreferrer">http://localhost:8383/test</a> --output bla.txt<br> % Total % Received % Xferd Average Speed Time Time Time Current<br> Dload Upload Total Spent Left Speed<br>100 55.2M 100 55.2M 0 0 1465k 0 0:00:38 0:00:38 --:--:-- 1476k<br>curl <a href="http://localhost:8383/test" target="_blank" rel="noreferrer">http://localhost:8383/test</a> -output bla.txt 0.04s user 0.14s system 0% cpu 38.657 total</font><div><br></div><div><div><font face="Courier New"><br></font></div><div><div><font face="Courier New">johanbrichau@JohansMacBookAir ~ % time curl <a href="http://localhost:8383/test" target="_blank" rel="noreferrer">http://localhost:8383/test</a> --output bla.txt</font></div><div><font face="Courier New"> % Total % Received % Xferd Average Speed Time Time Time Current</font></div><div><font face="Courier New"> Dload Upload Total Spent Left Speed</font></div><div><font face="Courier New">100 55.2M 100 55.2M 0 0 45.0M 0 0:00:01 0:00:01 --:--:-- 45.0M</font></div><div><font face="Courier New">curl <a href="http://localhost:8383/test" target="_blank" rel="noreferrer">http://localhost:8383/test</a> --output bla.txt 0.01s user 0.07s system 6% cpu 1.255 total</font></div><br><br><blockquote type="cite">On 23 May 2024, at 23:45, Johan Brichau <<a href="mailto:johan@yesplan.be" target="_blank" rel="noreferrer">johan@yesplan.be</a>> wrote:<br><br>Hi Otto,<br><br><blockquote type="cite">On 20 May 2024, at 19:08, Otto Behrens <<a href="mailto:otto@finworks.biz" target="_blank" rel="noreferrer">otto@finworks.biz</a>> wrote:<br><br>Okay, I’ll give that a try and see what I can get when using the Zinc adaptor in GemStone…</blockquote><br><br>It took me a bit longer to actually get started since I wanted to debug this on my Mac using a GsDevKit_stones setup and I had to dig into another rabbit hole first :-)<br>Anyway, I setup the simple handler below and I can confirm also notice that a file of 60MB takes 40s to be served and 28MB takes 10s. <br><br>In Pharo, the 60MB file takes 3s.<br>I’ll dive into this a bit more tomorrow…..<br><br>get<br><br> <get><br> <path: '/'><br> <produces: 'application/json'><br><br> | file |<br> file := GsFile openReadOnServer: '/Users/johanbrichau/testfile'.<br> ^ file contents<br><br></blockquote><br></div></div></div></div></div></blockquote></div><br></div></div>_______________________________________________<br> Glass mailing list<br> <a href="mailto:Glass@lists.gemtalksystems.com" target="_blank" rel="noreferrer">Glass@lists.gemtalksystems.com</a><br> <a href="https://lists.gemtalksystems.com/mailman/listinfo/glass" rel="noreferrer noreferrer" target="_blank">https://lists.gemtalksystems.com/mailman/listinfo/glass</a><br> </blockquote></div> </div></blockquote></div><br></div></div></div></body></html>