[Glass] Gemstone 3106 with Pier and crawler
Trussardi Dario Romano via Glass
glass at lists.gemtalksystems.com
Tue Oct 30 07:57:20 PDT 2018
Ciao,
the FASTCGI daemon_server-9060.log report:
topaz 1> topaz 1> topaz 1> topaz 1> [5621550081 sz:5 cls: 135169 GsFile] aGsFile
topaz 1> topaz 1> topaz 1> topaz 1> daemon Server started on port 9060
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
2285681153
2285750273
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
failure
Read-Write Conflicts...
Write-Write Conflicts...
374958081
Write-Dependency Conflicts...
Write-ReadLock Conflicts...
Write-WriteLock Conflicts...
Rc-Write-Write Conflicts...
Synchronized-Commit Conflicts...
----------- Commit failure - retrying LOG ENTRY: aSymbolDictionary-----------
----------- Unreportable ERROR Encountered: 2018-10-30T11:32:50.57347202301025+01:00a ImproperOperation occurred (error 2364), The object aGsSocket that has some associated session state has lost that transient state. Examples of classes that use session state are GsSocket and GsFile.-----------
----------- Unreportable ERROR Encountered: 2018-10-30T11:32:51.41707491874695+01:00a ImproperOperation occurred (error 2364), The object aGsSocket that has some associated session state has lost that transient state. Examples of classes that use session state are GsSocket and GsFile.-----------
----------- Unreportable ERROR Encountered: 2018-10-30T11:32:52.14350891113281+01:00a ImproperOperation occurred (error 2364), The object aGsSocket that has some associated session state has lost that transient state. Examples of classes that use session state are GsSocket and GsFile.-----------
----------- Unreportable ERROR Encountered: 2018-10-30T11:32:52.95211100578308+01:00a ImproperOperation occurred (error 2364), The object aGsSocket that has some associated session state has lost that transient state. Examples of classes that use session state are GsSocket and GsFile.-----------
----------- Unreportable ERROR Encountered: 2018-10-30T11:32:54.54405093193054+01:00a ImproperOperation occurred (error 2364), The object aGsSocket that has some associated session state has lost that transient state. Examples of classes that use session state are GsSocket and GsFile.-----------
dario at monviso:/mnt/ISTDataBase/opt3106/gemstone/GemStone64Bit3.1.0.6-x86_64.Linux/EnvGruppi/log$
In the GLASS mailing list i found:
[1] http://forum.world.st/Glass-Fastcgi-Socket-issues-td4816974.html
At a certain point we read:
Maybe the heavy load makes it reproduce it because it's only when the SPC gets full enough to flush the GsSocket instance to disk...
And I think it's the problem I had when the crawler made many requests, and the system is degraded.
Now as I write this email, the system has created 32 new WASessions in one minute and the system is degraded again,
and i need to restart ( kill ) the Fastcgi daemon entry.
The solution for my old environment Gemstone 3.1.0.6 is to comment the last lines **** of code in :
saveLogEntry: anObjectLogEntry shouldCommit: shouldCommit
"Does an abort and commit, if not already in transaction"
| stdout |
stdout := GsFile stdoutServer.
stdout nextPutAll: '----------- ', anObjectLogEntry labelString, ' LOG ENTRY: ', anObjectLogEntry objectString.
stdout nextPutAll: '-----------', DateAndTime now asString.
stdout cr.
stdout close.
"obi TODO"
" ****shouldCommit
ifTrue: [ self doTransaction: [ anObjectLogEntry addToLog ]]
ifFalse: [ anObjectLogEntry addToLog ]."
as reported at the end of [ 1 ] ?
Thanks for every consideration,
Dario
> GLASS mailing list wrote
>> Ciao, thanks.
>>
>>> You can block the IP with nginx (or apache if you're using it) if you
>>> want:
>>>
>>> https://help.dreamhost.com/hc/en-us/articles/216456127-Blocking-IPs-with-Nginx
>>>
>>> and I'm sure you could google an article that covers how to do it with
>>> Apache.
>>
>> OK, i use Lighttpd server and i found the relative data entry to do it.
>>
>> Would it be interesting, at a first level,
>>
>> to directly manage the request in Seaside without having to restart the
>> web server?
>
> I'm sure there is but I don't know exactly where and why you'd want Seaside
> checking every request to see if its from an IP if the webserver can do it
> faster. I'd probably start looking in WAServerAdaptor to figure out how.
>
>
> GLASS mailing list wrote
>>>
>>>
>>> For the "daemontools thinks its working but its really not" problem you
>>> could add an additional daemontools monitored process that checks a url
>>> that
>>> is served by each fastcgi process e.g. every 30 seconds or so and then
>>> kill
>>> the gem if it returns the wrong thing.
>>
>> Are you saying that as I configured the system does not work?
>>
>> When the system has been subjected to a greater load of requests,
>> everything is degraded!
>>
>> Does anyone have a Gemstone Pier site with a certain load of requests?
>
> I just think sometimes there are errors in the gems that aren't enough to
> kill them, so daemontools doesn't know they're faulty, but the gem
> would/could still benefit from a restart. It should be able to handle the
> extra load as what you've described isn't much
>
> I'm not sure why performance was degraded. Are you using SSDs ? Did you run
> out of disk where you're storing stats or tranlogs?
>
>
> GLASS mailing list wrote
>>>
>>> GLASS mailing list wrote
>>>> Ciao,
>>>>
>>>> i have a Gemstone 3.1.0.6 environment with Seaside and Pier framework.
>>>>
>>>> The Gemstone stone and the relative Fast-CGI service is managed by
>>>> Daemontools.
>>>>
>>>> Everything works for a few years.
>>>>
>>>> It's been a few days since the home server is continuously submitted to
>>>> requests (one every 5 seconds) made by ip 46.4.122.196 ( a crawler ?! )
>>>>
>>>> Now it happens that in some situations the service relative to Fast-CGI
>>>> go down but is not managed ( restart )from daemontools.
>>>>
>>>> These are the daemontool service entry
>>>> /etc/service/gs_seaside-9060: up (pid 25393) 142648 seconds
>>>> /etc/service/gs_seaside-9061: up (pid 7106) 15059 seconds
>>>> /etc/service/gs_seaside-9062: up (pid 20118) 194917 seconds
>>>>
>>>> Every Fast-CGI service create 3 entry.
>>>>
>>>> I report the 9062 entry with: lsof -n -i -P | grep 20118
>>>>
>>>> topaz 20118 dario 6u IPv6 236275 0t0 TCP
>>>> [::1]:40117->[::1]:42453 (ESTABLISHED)
>>>> topaz 20118 dario 8u IPv6 236276 0t0 TCP
>>>> [::1]:40503->[::1]:43399 (ESTABLISHED)
>>>> topaz 20118 dario 12u IPv4 236281 0t0 TCP *:9062 (LISTEN)
>>>>
>>>> When the 20118 Fast-CGI IPv4 go down the :
>>>>
>>>> topaz 20118 dario 6u IPv6 236275 0t0 TCP
>>>> [::1]:40117->[::1]:42453 (ESTABLISHED)
>>>> topaz 20118 dario 8u IPv6 236276 0t0 TCP
>>>> [::1]:40503->[::1]:43399 (ESTABLISHED)
>>>>
>>>> remain active and therefore i believe that the daemontools does not
>>>> consider service ( 20118 ) to be dead.
>>
>> So this consideration is true?
>
> Yes I think you're right. Daemontools just makes sure the gem process is
> running, not whether the gem can handle tcp connections (or do anything else
> for that matter).
>
> Hope this helps.
>
>
> GLASS mailing list wrote
>>>>
>>>> But the web request ( when all the Fast-CGI are down )is not managed by
>>>> any TCP *:906[ 0 - 2 ] (LISTEN) because all are down.
>>>>
>>>> Any idea about it?
>>>>
>>>> Another question:
>>>>
>>>> i can intercept the requests made by 46.4.122.196 before creating
>>>> the
>>>> related WASessions and replying ...????.... - do not reply?
>>
>> Thanks,
>>
>> Dario
>>
>> _______________________________________________
>> Glass mailing list
>
>> Glass at .gemtalksystems
>
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
>
>
> --
> Sent from: http://forum.world.st/GLASS-f1460844.html
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20181030/9f4c2aed/attachment-0001.html>
More information about the Glass
mailing list