[Glass] rest api returning large result
Otto Behrens
otto at finworks.biz
Mon May 20 07:18:35 PDT 2024
Ralph, thanks a lot for your response.
> 70MB is quite large, but should work fine if the relevant settings are
> adjusted (increased GEM memory, allowed nginx response size). I agree, that
> 26 seconds for 16MB is a very long time.
>
Yes, we ran out of temporary object space and increased that. We also
changed the Rest API code a bit to avoid unnecessary buffer copying. It is
not streaming properly yet, but survives with the memory that we have
allocated for now.
> On an anstract level, these are my thoughts:
>
> * Depending on your API it might be an good idea to think about pagination
> and split the data into some more api requests with smaller responses.
>
Ok, that may help. How big should a page be? Is 10MB too much?
> * I would configure nginx to do the response compression (gzip) to the
> client.
>
We did that, but it made no significant difference. The problem is on the
server side.
> * I would repeat your local curl test with a connect to your Zinc-server
> port just to take nginx out of the equation.
>
That is a great idea. Will do that. We are using client certificates, but
for a test I can hard code the fingerprint.
> * If nginx is not causing the trouble, I would try to use GemStone's
> ProfMonitor to get some more insights about the run time behaviour of your
> smalltalk code.
>
The smalltalk code is reasonably expensive, but my tests (using kill -USR1)
revealed that it was mostly spending time in SocketStream >> nextPutAll:
>
> Best regards,
> Ralph
>
>
> Am 20.05.2024 um 09:48 schrieb Otto Behrens via Glass:
>
> We have not managed to fix this yet. What is your opinion on the following
> ideas?
>
> 1. Otto, you are an idiot. Why would you be sending a 70MB json response
> out on a REST API? This is not how you do an API. [Otto: that may well be
> true. How should I then get the data across to the user? Is there someone
> that can help me with a solution?]
> 2. Otto, you have not kept up to date with things and you are the only one
> in the whole world that's using WAGsZincAdaptor serving as an nginx
> upstream. WTF. [Otto: Yes, we are here on the bottom tip of Africa where
> the internet is slow and we read even slower, sorry about that. Please help
> me with some sites, documents and any other material so that I can start
> reading.]
> 3. Otto, have you heard of the idea of compression? You should know that
> JSON will compress to at least a 10th of the original size because it is
> text with a lot of repetition. [Otto: yes, I downloaded a zip file once and
> could not read it in vim. Is this what I should do: compress the connection
> between nginx and the Zinc adaptor? Or should I send the json back as a
> compressed zip file?]
> 4. Otto, you should get to know nginx and its settings and understand all
> the stuff nginx spits out when debug logging is on. Better still, download
> the C source code; you should still be able to after only Smalltalking for
> 20 years. [Otto: Are you super human? Have you seen all of that? Please
> enlighten me as this will take me years.]
>
> Of course I missed some ideas. Please feel free to add them to the list.
>
> Otto Behrens
>
> +27 82 809 2375
> [image: FINWorks]
>
>
>
> [image: FINWorks] <http://za.linkedin.com/in/waltherbehrens>
> www.finworks.biz
>
> Disclaimer & Confidentiality Note: This email is intended solely for the
> use of the individual or entity named above as it may contain information
> that is confidential and privileged. If you are not the intended recipient,
> be advised that any dissemination, distribution or copying of this email is
> strictly prohibited. FINWorks cannot be held liable by any person other
> than the addressee in respect of any opinions, conclusions, advice or other
> information contained in this email.
>
>
> On Fri, May 17, 2024 at 7:09 AM Otto Behrens <otto at finworks.biz> wrote:
>
>> Hi,
>>
>> We are running into a performance problem where our API returns about
>> 70MB json content. We run a nginx web server which connects to
>> a WAGsZincAdaptor that we start in a topaz session. Do you perhaps have the
>> same kind of setup and can you please give me some advice on this?
>>
>> We found that converting objects to json (using Object >> asJson from
>> Seaside-JSON-Core) was not performing great, but was eating loads of memory
>> because of WABuilder >> render:. This is not the issue and we improved this
>> a bit (by eliminating String streamContents: and streaming more directly).
>>
>> The problem seems to be that after producing the json content,
>> transmitting the response takes a long time.
>>
>> As an experiment, I read a 16MB file from disk and returned that as the
>> result of an API call to eliminate all json producing code. I used curl as
>> a client on the same machine as the nginx server, stone and the topaz
>> session and it takes 26 seconds. This eliminates most overhead (no network
>> latency).
>>
>> The stack below is what I see most of the time:
>>
>> 1 SocketStream >> nextPutAll: @natCode+0x4d [GsNMethod 169113089]
>> FP: 0x7f2c0fee9930=StackLimit[-218] , callerFP:
>> StackLimit[-212]
>> arg 1:0x7f2bee7f0de0 (cls:103425 ByteArray size:16384)
>> rcvr: 0x7f2bff68f670 (cls:144280577 SocketStream size:12)
>> 2 ZnBivalentWriteStream >> next:putAll:startingAt: @natCode+0x2cf
>> [GsNMethod 158727169]
>> FP: 0x7f2c0fee9960=StackLimit[-212] , callerFP:
>> StackLimit[-202]
>> arg 3:69337098 (SmallInteger 8667137)
>> arg 2:0x7f2c0064fe50 (cls:74753 String size:16627226)'(large_or_fwd
>> (size 16627226))'
>> arg 1:131074 (SmallInteger 16384)
>> rcvr: 0x7f2c00675370 (cls:145086465 ZnBivalentWriteStream size:2)
>> 3 ZnUtils class >> nextPutAll:on: @natCode+0x421 [GsNMethod 175369473]
>> FP: 0x7f2c0fee99b0=StackLimit[-202] , callerFP:
>> StackLimit[-196]
>> arg 2:0x7f2c00675370 (cls:145086465 ZnBivalentWriteStream size:2)
>> arg 1:0x7f2c0064fe50 (cls:74753 String size:16627226)'(large_or_fwd
>> (size 16627226))'
>> rcvr: 0x7f2c0c335750 oid:143053313 (cls:143054593 ZnUtils class size:19)
>> 4 ZnByteArrayEntity >> writeOn: @natCode+0xdb [GsNMethod 269993473]
>> FP: 0x7f2c0fee99e0=StackLimit[-196] , callerFP:
>> StackLimit[-186]
>> arg 1:0x7f2c00675370 (cls:145086465 ZnBivalentWriteStream size:2)
>> rcvr: 0x7f2c00651f00 (cls:145996545 ZnByteArrayEntity size:3)
>> 5 ZnEntityWriter >> writeEntity: @natCode+0x382 [GsNMethod 269988609]
>> FP: 0x7f2c0fee9a30=StackLimit[-186] , callerFP:
>> StackLimit[-180]
>> arg 1:0x7f2c00651f00 (cls:145996545 ZnByteArrayEntity size:3)
>> rcvr: 0x7f2c00675398 (cls:145876737 ZnEntityWriter size:2)
>> 6 ZnMessage >> writeOn: @natCode+0x295 [GsNMethod 158696193]
>> FP: 0x7f2c0fee9a60=StackLimit[-180] , callerFP:
>> StackLimit[-174]
>> arg 1:0x7f2c00675370 (cls:145086465 ZnBivalentWriteStream size:2)
>> rcvr: 0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)
>> 7 ZnResponse >> writeOn: @natCode+0x1f0 [GsNMethod 155024025857]
>> FP: 0x7f2c0fee9a90=StackLimit[-174] , callerFP:
>> StackLimit[-169]
>> arg 1:0x7f2bff68f670 (cls:144280577 SocketStream size:12)
>> rcvr: 0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)
>> 8 ZnSingleThreadedServer >> writeResponse:on: @natCode+0xa3 [GsNMethod
>> 169204737]
>> FP: 0x7f2c0fee9ab8=StackLimit[-169] , callerFP:
>> StackLimit[-162]
>> arg 2:0x7f2bff68f670 (cls:144280577 SocketStream size:12)
>> arg 1:0x7f2c0064fe20 (cls:145901313 ZnResponse size:3)
>> rcvr: 0x7f2bff5de528 oid:4763064833 (cls:144532225
>> ZnManagingMultiThreadedServer size:9)
>>
>> Kind regards
>>
>> Otto Behrens
>>
>> +27 82 809 2375
>> [image: FINWorks]
>>
>>
>>
>> [image: FINWorks] <http://za.linkedin.com/in/waltherbehrens>
>> www.finworks.biz
>>
>> Disclaimer & Confidentiality Note: This email is intended solely for the
>> use of the individual or entity named above as it may contain information
>> that is confidential and privileged. If you are not the intended recipient,
>> be advised that any dissemination, distribution or copying of this email is
>> strictly prohibited. FINWorks cannot be held liable by any person other
>> than the addressee in respect of any opinions, conclusions, advice or other
>> information contained in this email.
>>
>
> _______________________________________________
> Glass mailing listGlass at lists.gemtalksystems.comhttps://lists.gemtalksystems.com/mailman/listinfo/glass
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> https://lists.gemtalksystems.com/mailman/listinfo/glass
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/archives/glass/attachments/20240520/cb931da9/attachment-0001.htm>
More information about the Glass
mailing list