[Glass] Further commits have been disabled for this session because: 'CorruptObj error'. This session must logout.
Dale Henrichs
dale.henrichs at gemtalksystems.com
Tue May 27 12:18:00 PDT 2014
I forgot to mention that internally we are tracking this as bug 44291...
Dale
On Tue, May 27, 2014 at 12:16 PM, Dale Henrichs <
dale.henrichs at gemtalksystems.com> wrote:
> Okay, I've talked to engineering and they agree that the corruptObj error
> in this case is too extreme. We are in the final stages of a 3.2.1 release,
> so it may be too late to include a fix in 3.2.1, but it will most likely be
> included in a 3.2.2, but we don't have a schedule for that.
>
> Until then I think that the following patch
> to GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: is the
> best we can do:
>
>
> ----- Method:
> GRGemStonePlatform>>seasideProcessRequest:adaptor:resultBlock: (in category
> '*seaside-gemstone-core') -----seasideProcessRequest: aNativeRequest
> adaptor: adaptor resultBlock: resultBlock
> | result |
> [
> | retryCount result count |
> retryCount := self retryCount.
> count := 1.
> retryCount
> timesRepeat: [
> (result := self
> seasideProcessRequestWithRetry: aNativeRequest
> resultBlock: resultBlock) ~~ nil
> ifTrue: [ ^ result ].
> System _sessionCacheStatAt: 2 put: (System _sessionCacheStatAt: 2)
> + 1. "requests retried"
> (Delay forMilliseconds: self retryDelay * count) wait.
> count := count * 10 ]. "exceeded retry limit"
> ^ adaptor
> internalServerErrorMessage:
> 'Too many retries: ' , (retryCount + 1) printString ]
> on: Error
> do: [ :ex |
> self doAbortTransaction.
> + result := adaptor internalServerErrorMessage: ex description.
> + ex gsNumber == 2261
> + ifTrue:
> + [
> + "see discussion in:
> http://forum.world.st/Glass-Further-commits-have-been-disabled-for-this-session-because-CorruptObj-error-This-session-must-td4760409.html
> + "
> + Transcript
> + cr;
> + show:
> + 'Session terminating due to CorruptObj error ... session
> must logout'.
> + System logout ]
> + ex
> + return: nil "Do an explicit return. Because of the abort above,
> the default action for an exception (resume) is set. see bug39246." ].
> - result := adaptor internalServerErrorMessage: ex description. "Do
> an explicit return. Because of the abort above, the default action for an
> exception (resume) is set. see bug39246."
> - ex return: nil ].
> ^ result
>
> If you use this patch, make sure that you have a system in place for
> automatically restarting gems that get fatal errors. You should be using
> monit or daaemontools in production since gems may crash occasionally and
> they need to be restarted. Other than the extra time/cpu/disk for
> restarting a gem, stopping/restarting gems is not a fatal problem as no
> persistent data is lost or corrupted ... see my post on "GLASS 101:
> Disposable Gems, Durable Data"[1].
>
> I'm on the fence about including this patch in the standard release of
> Seaside for GemStone ... on the one hand if you get this particular error
> (or any other "commit prohibiting error" the gem is effectively out of
> service and needs to be restarted...
>
> An alternative to logout would be to update a field in SessionTemps that
> indicates the the gem needs to be restarted ... that session state can be
> queried by a monit http request (presumably) ... so at the end of the day
> we should probably invent some sort of gem termination policy that can be
> controlled at the application level...
>
> Dale
>
>
> [1]
> http://gemstonesoup.wordpress.com/2008/03/08/glass-101-disposable-gems-durable-data/
>
>
>
> On Mon, May 26, 2014 at 10:33 AM, Dale Henrichs <
> dale.henrichs at gemtalksystems.com> wrote:
>
>> Johan,
>>
>> I think that a corruptObj error in this case may be a bit extreme. I have
>> asked engineering if there is a "session safe" method for finding the
>> obviously invalid utf8 ... Today is a holiday, so I may not get a response
>> until tomorrow.
>>
>> I hope that crashed vms is an acceptable outcome of the security audit?
>>
>> The corruptObj error is our response to the error thrown by ICU ... the
>> commit prohibiting error normally is thrown when we think that "memory
>> stomping" has occurred and we want to avoid persisting potentially corrupt
>> objects ... presumably the security audit folks knew that mishandling this
>> particular ICU error condition could lead to a security breach:)
>>
>> Dale
>>
>>
>> On Mon, May 26, 2014 at 9:44 AM, Johan Brichau <johan at yesplan.be> wrote:
>>
>>> Hi,
>>>
>>> Today, we had a security audit on a Seaside 3.0.10 application running
>>> in a GS 3.1.0.5 stone with FastCGI behind nginx.
>>>
>>> I have no idea what exactly they did to obtain this, but the system went
>>> unresponsive after the following error until I restarted the gems.
>>> - a InternalError occurred (error 2261), The object with object ID
>>> 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode'
>>> - a TransactionError occurred (error 2249), Further commits have been
>>> disabled for this session because: 'CorruptObj error'. This session must
>>> logout.
>>> - repeat last one
>>>
>>> The stacks from the gem log are attached.
>>>
>>> I am trying to trace what happened. Any clues?
>>>
>>> Johan
>>>
>>> ----------- Internal FASTCGI ERROR Encountered:
>>> 2014-05-26T13:37:41.31612992286682+02:00
>>> a InternalError occurred (error 2261), The object with object ID
>>> 'Hannes_Alfvén' is corrupt. Reason: 'carrysize > 0 at end of utf8 decode'
>>> 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4
>>> [GsNMethod 7469480705]
>>> 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod
>>> 7469494785]
>>> 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14
>>> [GsNMethod 9828254465]
>>> 4 [] in GRGemStonePlatform >>
>>> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod
>>> 10506396161]
>>> 5 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod
>>> 1394121473]
>>> 6 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1
>>> [GsNMethod 1360964097]
>>> 7 String >> decodeFromUTF8 (envId 0) @1 line 1 [GsNMethod 1064104705]
>>> 8 UTF8PrimitiveEncoding >> decode: (envId 0) @2 line 4 [GsNMethod
>>> 7470277121]
>>> 9 GRUtf8GemStoneCodec >> decode: (envId 0) @3 line 3 [GsNMethod
>>> 7468617985]
>>> 10 [] in WAUrl >> decodedWith: (envId 0) @3 line 17 [GsNMethod
>>> 9230396417]
>>> 11 SequenceableCollection >> collect: (envId 0) @9 line 16 [GsNMethod
>>> 1064183041]
>>> 12 WAUrl >> decodedWith: (envId 0) @22 line 17 [GsNMethod 8789933313]
>>> 13 WAFastCGIRequestConverter >> requestUrlFor: (envId 0) @6 line 4
>>> [GsNMethod 9828231425]
>>> 14 WAServerAdaptor >> requestFor: (envId 0) @3 line 6 [GsNMethod
>>> 8790261761]
>>> 15 WAFastCGIRequestConverter >> requestFor: (envId 0) @12 line 7
>>> [GsNMethod 9828215809]
>>> 16 WAFastCGIAdaptor >> requestFor: (envId 0) @2 line 4 [GsNMethod
>>> 9828250881]
>>> 17 WAServerAdaptor >> contextFor: (envId 0) @2 line 5 [GsNMethod
>>> 8790264577]
>>> 18 WAServerAdaptor >> process: (envId 0) @2 line 5 [GsNMethod
>>> 8790258433]
>>> 19 [] in WAFastCGIAdaptor >> process: (envId 0) @2 line 6 [GsNMethod
>>> 8794996737]
>>> 20 [] in GRGemStonePlatform >>
>>> seasideProcessRequestWithRetry:resultBlock: (envId 0) @2 line 13
>>> [GsNMethod 10501067521]
>>> 21 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 22 [] in GRGemStonePlatform >>
>>> seasideProcessRequestWithRetry:resultBlock: (envId 0) @14 line 14
>>> [GsNMethod 10506395905]
>>> 23 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769]
>>> 24 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod
>>> 6527748609]
>>> 25 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock:
>>> (envId 0) @3 line 5 [GsNMethod 10509761025]
>>> 26 [] in GRGemStonePlatform >>
>>> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod
>>> 10506396417]
>>> 27 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 28 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock:
>>> (envId 0) @3 line 23 [GsNMethod 10509761281]
>>> 29 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod
>>> 9828418049]
>>> 30 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4
>>> [GsNMethod 8795113729]
>>> 31 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 32 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5
>>> [GsNMethod 9828248321]
>>> 33 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457]
>>> 34 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329]
>>> 35 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod
>>> 8853951745]
>>> 36 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod
>>> 9557561601]
>>> 37 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 38 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod
>>> 9322044673]
>>> 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 40 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod
>>> 9258731777]
>>> 41 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769]
>>> 42 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913]
>>> 43 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889]
>>> 44 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod
>>> 9261209601]
>>> 45 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977]
>>> 46 <Reenter marker>
>>> -----------
>>> ----------- Internal FASTCGI LOG ENTRY: anArray-----------
>>> ----------- Internal FASTCGI ERROR Encountered:
>>> 2014-05-26T13:37:41.37823009490967+02:00
>>> a TransactionError occurred (error 2249), Further commits have been
>>> disabled for this session because: 'CorruptObj error'. This session must
>>> logout.
>>> 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line 4
>>> [GsNMethod 7469480705]
>>> 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod
>>> 7469494785]
>>> 3 WAFastCGIAdaptor >> internalServerErrorMessage: (envId 0) @20 line 14
>>> [GsNMethod 9828254465]
>>> 4 [] in GRGemStonePlatform >>
>>> seasideProcessRequest:adaptor:resultBlock: (envId 0) @4 line 26 [GsNMethod
>>> 10506396161]
>>> 5 AbstractException >> _executeOuterHandler: (envId 0) @3 line 7
>>> [GsNMethod 1394117633]
>>> 6 AbstractException >> _pass:with: (envId 0) @4 line 13 [GsNMethod
>>> 1393936385]
>>> 7 AbstractException >> pass (envId 0) @2 line 14 [GsNMethod 1393935361]
>>> 8 [] in System class >> _localCommit: (envId 0) @2 line 34 [GsNMethod
>>> 5944183041]
>>> 9 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod
>>> 1394121473]
>>> 10 AbstractException >> _signalFromPrimitive: (envId 0) @1 line 1
>>> [GsNMethod 1360964097]
>>> 11 System class >> _primitiveCommit: (envId 0) @1 line 1 [GsNMethod
>>> 1789434881]
>>> 12 System class >> __commit: (envId 0) @2 line 8 [GsNMethod 1789345025]
>>> 13 [] in System class >> _localCommit: (envId 0) @2 line 30 [GsNMethod
>>> 5944183297]
>>> 14 ExecBlock >> onException:do: (envId 0) @2 line 66 [GsNMethod
>>> 1064628225]
>>> 15 System class >> _localCommit: (envId 0) @8 line 31 [GsNMethod
>>> 1789345281]
>>> 16 TransactionBoundaryDefaultPolicy >> commit: (envId 0) @2 line 3
>>> [GsNMethod 5986577665]
>>> 17 System class >> _commit: (envId 0) @7 line 16 [GsNMethod 1789345537]
>>> 18 System class >> commitTransaction (envId 0) @5 line 7 [GsNMethod
>>> 1789402113]
>>> 19 System class >> _commitPrintingDiagnostics (envId 0) @2 line 8
>>> [GsNMethod 1700522241]
>>> 20 SystemCommitTransaction >> defaultAction (envId 0) @2 line 3
>>> [GsNMethod 7468825857]
>>> 21 AbstractException >> _signalWith: (envId 0) @5 line 25 [GsNMethod
>>> 1394122241]
>>> 22 AbstractException class >> signal (envId 0) @3 line 5 [GsNMethod
>>> 1172775681]
>>> 23 GRGemStonePlatform >> doCommitTransaction (envId 0) @4 line 3
>>> [GsNMethod 7469481473]
>>> 24 [] in GRGemStonePlatform >>
>>> seasideProcessRequestWithRetry:resultBlock: (envId 0) @47 line 54
>>> [GsNMethod 10506395905]
>>> 25 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769]
>>> 26 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod
>>> 6527748609]
>>> 27 GRGemStonePlatform >> seasideProcessRequestWithRetry:resultBlock:
>>> (envId 0) @3 line 5 [GsNMethod 10509761025]
>>> 28 [] in GRGemStonePlatform >>
>>> seasideProcessRequest:adaptor:resultBlock: (envId 0) @6 line 15 [GsNMethod
>>> 10506396417]
>>> 29 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 30 GRGemStonePlatform >> seasideProcessRequest:adaptor:resultBlock:
>>> (envId 0) @3 line 23 [GsNMethod 10509761281]
>>> 31 WAFastCGIAdaptor >> process: (envId 0) @3 line 4 [GsNMethod
>>> 9828418049]
>>> 32 [] in WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 4
>>> [GsNMethod 8795113729]
>>> 33 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 34 WAFastCGIAdaptor >> answerResponderRole: (envId 0) @2 line 5
>>> [GsNMethod 9828248321]
>>> 35 FSResponderRole >> answer (envId 0) @3 line 4 [GsNMethod 8854227457]
>>> 36 FSRole >> handleConnection (envId 0) @3 line 5 [GsNMethod 8854243329]
>>> 37 FSConnection >> unsafeServe (envId 0) @5 line 8 [GsNMethod
>>> 8853951745]
>>> 38 [] in FSConnection >> safeServe (envId 0) @2 line 8 [GsNMethod
>>> 9557561601]
>>> 39 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 40 [] in FSConnection >> safeServe (envId 0) @2 line 9 [GsNMethod
>>> 9322044673]
>>> 41 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 1064627457]
>>> 42 [] in FSConnection >> safeServe (envId 0) @2 line 12 [GsNMethod
>>> 9258731777]
>>> 43 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 1064640769]
>>> 44 FSConnection >> safeServe (envId 0) @2 line 15 [GsNMethod 8853958913]
>>> 45 FSConnection >> serve (envId 0) @2 line 4 [GsNMethod 8853957889]
>>> 46 [] in FSSocketServer >> listen: (envId 0) @3 line 15 [GsNMethod
>>> 9261209601]
>>> 47 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 1403422977]
>>> 48 <Reenter marker>
>>> -----------
>>> _______________________________________________
>>> Glass mailing list
>>> Glass at lists.gemtalksystems.com
>>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20140527/bb7a336a/attachment-0001.html>
More information about the Glass
mailing list