<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hello Otto,<br>
      <br>
      I'm not using Seaside, but here my two cents:<br>
      <br>
      70MB is quite large, but should work fine if the relevant settings
      are adjusted (increased GEM memory, allowed nginx response size).
      I agree, that 26 seconds for 16MB is a very long time.</p>
    <p>On an anstract level, these are my thoughts:<br>
      <br>
      * Depending on your API it might be an good idea to think about
      pagination and split the data into some more api requests with
      smaller responses.<br>
      * I would configure nginx to do the response compression (gzip) to
      the client.<br>
      * I would repeat your local curl test with a connect to your
      Zinc-server port just to take nginx out of the equation.<br>
      * If nginx is not causing the trouble, I would try to use
      GemStone's ProfMonitor to get some more insights about the run
      time behaviour of your smalltalk code.<br>
      <br>
      Best regards,<br>
      Ralph</p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">Am 20.05.2024 um 09:48 schrieb Otto
      Behrens via Glass:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAOJutV6B-xS2T2V-eYuhRQoahGMJvTi=pD_u2iDEMUPk7cLDnA@mail.gmail.com">
      <meta http-equiv="content-type" content="text/html; charset=UTF-8">
      <div dir="ltr">
        <div>We have not managed to fix this yet. What is your opinion
          on the following ideas?</div>
        <div><br>
        </div>
        <div>1. Otto, you are an idiot. Why would you be sending a 70MB
          json response out on a REST API? This is not how you do an
          API. [Otto: that may well be true. How should I then get the
          data across to the user? Is there someone that can help me
          with a solution?]</div>
        <div>2. Otto, you have not kept up to date with things and you
          are the only one in the whole world that's using
          WAGsZincAdaptor serving as an nginx upstream. WTF. [Otto: Yes,
          we are here on the bottom tip of Africa where the internet is
          slow and we read even slower, sorry about that. Please help me
          with some sites, documents and any other material so that I
          can start reading.]</div>
        <div>3. Otto, have you heard of the idea of compression? You
          should know that JSON will compress to at least a 10th of the
          original size because it is text with a lot of repetition.
          [Otto: yes, I downloaded a zip file once and could not read it
          in vim. Is this what I should do: compress the connection
          between nginx and the Zinc adaptor? Or should I send the json
          back as a compressed zip file?]</div>
        <div>4. Otto, you should get to know nginx and its settings and
          understand all the stuff nginx spits out when debug logging is
          on. Better still, download the C source code; you should still
          be able to after only Smalltalking for 20 years. [Otto: Are
          you super human? Have you seen all of that? Please enlighten
          me as this will take me years.]</div>
        <div><br>
        </div>
        Of course I missed some ideas. Please feel free to add them to
        the list.
        <div><br clear="all">
          <div>
            <div dir="ltr" class="gmail_signature"
              data-smartmail="gmail_signature">
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td width="400" valign="bottom">
                            <p style="margin:0px;padding:0px"><span
style="font-size:18px;color:rgb(146,148,151);font-family:Calibri,sans-serif;font-weight:700">Otto
                                Behrens</span><br>
                            </p>
                            <p
style="font-size:18px;font-weight:700;color:rgb(146,148,151);font-family:Calibri,sans-serif;margin:0px;padding:0px"><span
style="font-size:14px;font-weight:300;margin:0px;padding:0px">+27 82 809
                                2375</span></p>
                          </td>
                          <td width="200" valign="middle"><img
src="https://www.finworks.biz/signature/finworks-signature-logo.png"
                              alt="FINWorks"
style="display:block;border:0px;width:200px;height:38px;margin:0px;padding:0px"
                              moz-do-not-send="true" width="200"
                              height="38"></td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td height="5"><br>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium;border-bottom:1px solid rgb(200,28,36)"
                      width="600" cellspacing="0" cellpadding="0"
                      border="0">
                      <tbody>
                        <tr>
                          <td height="15"><br>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td height="20"><br>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td style="display:inline-block" width="15"
                            valign="top"><a
href="http://za.linkedin.com/in/waltherbehrens"
                              style="color:rgb(17,85,204)"
                              target="_blank" moz-do-not-send="true"><img
src="https://www.finworks.biz/signature/finworks-linkedin-logo.png"
                                alt="FINWorks"
style="display:inline-block;border:0px;width:15px;height:15px;margin-top:1.5px;padding:0px"
                                moz-do-not-send="true" width="15"
                                height="15"></a></td>
                          <td style="display:inline-block" width="250"
                            valign="top"><a
                              href="http://www.finworks.biz/"
style="color:rgb(200,28,36);font-family:Calibri,sans-serif;margin-left:10px;margin-top:0px;padding-top:0px;font-size:11pt;display:inline-block"
                              target="_blank" moz-do-not-send="true">www.finworks.biz</a></td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td height="10"><br>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                    <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                      cellspacing="0" cellpadding="0" border="0">
                      <tbody>
                        <tr>
                          <td>
                            <p
style="font-size:10px;color:rgb(146,148,151);font-family:Calibri,sans-serif;text-align:justify">Disclaimer
                              & Confidentiality Note: This email is
                              intended solely for the use of the
                              individual or entity named above as it may
                              contain information that is confidential
                              and privileged. If you are not the
                              intended recipient, be advised that any
                              dissemination, distribution or copying of
                              this email is strictly prohibited.
                              FINWorks cannot be held liable by any
                              person other than the addressee in respect
                              of any opinions, conclusions, advice or
                              other information contained in this email.</p>
                          </td>
                        </tr>
                      </tbody>
                    </table>
                  </div>
                </div>
              </div>
            </div>
          </div>
          <br>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Fri, May 17, 2024 at
          7:09 AM Otto Behrens <<a href="mailto:otto@finworks.biz"
            target="_blank" moz-do-not-send="true"
            class="moz-txt-link-freetext">otto@finworks.biz</a>>
          wrote:<br>
        </div>
        <blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div dir="ltr">Hi,
            <div><br>
            </div>
            <div>We are running into a performance problem where our API
              returns about 70MB json content. We run a nginx web server
              which connects to a WAGsZincAdaptor that we start in a
              topaz session. Do you perhaps have the same kind of setup
              and can you please give me some advice on this?</div>
            <div><br>
            </div>
            <div>We found that converting objects to json (using Object
              >> asJson from Seaside-JSON-Core) was not
              performing great, but was eating loads of memory because
              of WABuilder >> render:. This is not the issue and
              we improved this a bit (by eliminating String
              streamContents: and streaming more directly).</div>
            <div><br>
            </div>
            <div>The problem seems to be that after producing the json
              content, transmitting the response takes a long time.</div>
            <div><br>
            </div>
            <div>As an experiment, I read a 16MB file from disk and
              returned that as the result of an API call to eliminate
              all json producing code. I used curl as a client on the
              same machine as the nginx server, stone and the topaz
              session and it takes 26 seconds. This eliminates most
              overhead (no network latency).</div>
            <div><br>
            </div>
            <div>The stack below is what I see most of the time:</div>
            <div><br>
            </div>
            <div>1  SocketStream >> nextPutAll:  @natCode+0x4d
               [GsNMethod 169113089]<br>
                            FP: 0x7f2c0fee9930=StackLimit[-218] ,
              callerFP: StackLimit[-212]<br>
                arg 1:0x7f2bee7f0de0 (cls:103425 ByteArray size:16384)<br>
                rcvr: 0x7f2bff68f670 (cls:144280577 SocketStream
              size:12)<br>
              2  ZnBivalentWriteStream >> next:putAll:startingAt:
               @natCode+0x2cf  [GsNMethod 158727169]<br>
                            FP: 0x7f2c0fee9960=StackLimit[-212] ,
              callerFP: StackLimit[-202]<br>
                arg 3:69337098 (SmallInteger 8667137)<br>
                arg 2:0x7f2c0064fe50 (cls:74753 String
              size:16627226)'(large_or_fwd (size 16627226))'<br>
                arg 1:131074 (SmallInteger 16384)<br>
                rcvr: 0x7f2c00675370 (cls:145086465
              ZnBivalentWriteStream size:2)<br>
              3  ZnUtils class >> nextPutAll:on:  @natCode+0x421
               [GsNMethod 175369473]<br>
                            FP: 0x7f2c0fee99b0=StackLimit[-202] ,
              callerFP: StackLimit[-196]<br>
                arg 2:0x7f2c00675370 (cls:145086465
              ZnBivalentWriteStream size:2)<br>
                arg 1:0x7f2c0064fe50 (cls:74753 String
              size:16627226)'(large_or_fwd (size 16627226))'<br>
                rcvr: 0x7f2c0c335750 oid:143053313 (cls:143054593
              ZnUtils class size:19)<br>
              4  ZnByteArrayEntity >> writeOn:  @natCode+0xdb
               [GsNMethod 269993473]<br>
                            FP: 0x7f2c0fee99e0=StackLimit[-196] ,
              callerFP: StackLimit[-186]<br>
                arg 1:0x7f2c00675370 (cls:145086465
              ZnBivalentWriteStream size:2)<br>
                rcvr: 0x7f2c00651f00 (cls:145996545 ZnByteArrayEntity
              size:3)<br>
              5  ZnEntityWriter >> writeEntity:  @natCode+0x382
               [GsNMethod 269988609]<br>
                            FP: 0x7f2c0fee9a30=StackLimit[-186] ,
              callerFP: StackLimit[-180]<br>
                arg 1:0x7f2c00651f00 (cls:145996545 ZnByteArrayEntity
              size:3)<br>
                rcvr: 0x7f2c00675398 (cls:145876737 ZnEntityWriter
              size:2)<br>
              6  ZnMessage >> writeOn:  @natCode+0x295  [GsNMethod
              158696193]<br>
                            FP: 0x7f2c0fee9a60=StackLimit[-180] ,
              callerFP: StackLimit[-174]<br>
                arg 1:0x7f2c00675370 (cls:145086465
              ZnBivalentWriteStream size:2)<br>
                rcvr: 0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)<br>
              7  ZnResponse >> writeOn:  @natCode+0x1f0
               [GsNMethod 155024025857]<br>
                            FP: 0x7f2c0fee9a90=StackLimit[-174] ,
              callerFP: StackLimit[-169]<br>
                arg 1:0x7f2bff68f670 (cls:144280577 SocketStream
              size:12)<br>
                rcvr: 0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)<br>
              8  ZnSingleThreadedServer >> writeResponse:on:
               @natCode+0xa3  [GsNMethod 169204737]<br>
                            FP: 0x7f2c0fee9ab8=StackLimit[-169] ,
              callerFP: StackLimit[-162]<br>
                arg 2:0x7f2bff68f670 (cls:144280577 SocketStream
              size:12)<br>
                arg 1:0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)<br>
                rcvr: 0x7f2bff5de528 oid:4763064833 (cls:144532225
              ZnManagingMultiThreadedServer size:9)<br>
            </div>
            <div><br>
            </div>
            <div>Kind regards<br clear="all">
              <div>
                <div dir="ltr" class="gmail_signature">
                  <div dir="ltr">
                    <div>
                      <div dir="ltr">
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td width="400" valign="bottom">
                                <p style="margin:0px;padding:0px"><span
style="font-size:18px;color:rgb(146,148,151);font-family:Calibri,sans-serif;font-weight:700">Otto
                                    Behrens</span><br>
                                </p>
                                <p
style="font-size:18px;font-weight:700;color:rgb(146,148,151);font-family:Calibri,sans-serif;margin:0px;padding:0px"><span
style="font-size:14px;font-weight:300;margin:0px;padding:0px">+27 82 809
                                    2375</span></p>
                              </td>
                              <td width="200" valign="middle"><img
src="https://www.finworks.biz/signature/finworks-signature-logo.png"
                                  alt="FINWorks"
style="display: block; border: 0px; width: 200px; height: 38px; margin: 0px; padding: 0px;"
                                  moz-do-not-send="true" width="200"
                                  height="38"></td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td height="5"><br>
                              </td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium;border-bottom:1px solid rgb(200,28,36)"
                          width="600" cellspacing="0" cellpadding="0"
                          border="0">
                          <tbody>
                            <tr>
                              <td height="15"><br>
                              </td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td height="20"><br>
                              </td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td style="display:inline-block"
                                width="15" valign="top"><a
href="http://za.linkedin.com/in/waltherbehrens"
                                  style="color:rgb(17,85,204)"
                                  target="_blank" moz-do-not-send="true"><img
src="https://www.finworks.biz/signature/finworks-linkedin-logo.png"
                                    alt="FINWorks"
style="display: inline-block; border: 0px; width: 15px; height: 15px; margin-top: 1.5px; padding: 0px;"
                                    moz-do-not-send="true" width="15"
                                    height="15"></a></td>
                              <td style="display:inline-block"
                                width="250" valign="top"><a
                                  href="http://www.finworks.biz/"
style="color:rgb(200,28,36);font-family:Calibri,sans-serif;margin-left:10px;margin-top:0px;padding-top:0px;font-size:11pt;display:inline-block"
                                  target="_blank" moz-do-not-send="true">www.finworks.biz</a></td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td height="10"><br>
                              </td>
                            </tr>
                          </tbody>
                        </table>
                        <table
style="color:rgb(0,0,0);font-family:Times;font-size:medium" width="600"
                          cellspacing="0" cellpadding="0" border="0">
                          <tbody>
                            <tr>
                              <td>
                                <p
style="font-size:10px;color:rgb(146,148,151);font-family:Calibri,sans-serif;text-align:justify">Disclaimer
                                  & Confidentiality Note: This email
                                  is intended solely for the use of the
                                  individual or entity named above as it
                                  may contain information that is
                                  confidential and privileged. If you
                                  are not the intended recipient, be
                                  advised that any dissemination,
                                  distribution or copying of this email
                                  is strictly prohibited. FINWorks
                                  cannot be held liable by any person
                                  other than the addressee in respect of
                                  any opinions, conclusions, advice or
                                  other information contained in this
                                  email.</p>
                              </td>
                            </tr>
                          </tbody>
                        </table>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </blockquote>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
      <pre class="moz-quote-pre" wrap="">_______________________________________________
Glass mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Glass@lists.gemtalksystems.com">Glass@lists.gemtalksystems.com</a>
<a class="moz-txt-link-freetext" href="https://lists.gemtalksystems.com/mailman/listinfo/glass">https://lists.gemtalksystems.com/mailman/listinfo/glass</a>
</pre>
    </blockquote>
  </body>
</html>