[Glass] Error in StackLoad
Dale Henrichs
dale.henrichs at gemtalksystems.com
Thu Jul 17 14:43:42 PDT 2014
Ken,
Sorry for the late reply ... I've been busy elsewhere ...
Our vm/continuations guy is on vacation this week, so I'll forward this
information to him and see what he has to say when he's back ...
You might want to think about upgrading to 3.1.0.6, because there were
quite a few bugs fixed in 3.1.0.5. With that said, I don't know if your
particular problem has been addressed or not ...
Dale
On Thu, Jul 17, 2014 at 12:41 PM, Ken Treis <ken at miriamtech.com> wrote:
> Look at me, I'm using the list like a blog. Sorry about that.
>
> To my untrained eye, it looks like this is an issue with switching between
> native and interpreted code. If we saved continuations with native code and
> then tried to run them non-native, that'd be a plausible explanation for
> this error.
>
> For now, I've re-enabled native code and expired all sessions by sending
> WAApplication>>clear. So far things seem OK.
>
> --
> Ken Treis
> Miriam Technologies, Inc.
> (866) 652-2040 x221
>
> On Jul 17, 2014, at 12:16 PM, Ken Treis <ken at miriamtech.com> wrote:
>
> OK, I have a little more detail on this. I misread the error, it turns out
> that the StackLoad error doesn't occur while saving a continuation for
> debug. Those are getting saved and I can debug them in GemTools. The error
> is apparently triggered when trying to run a callback.
>
> Here's the full stack, relevant section *emphasized:*
>
> ----------- Continuation saved to object log ERROR Encountered:
> 2014-07-17T11:43:56.08789706230164-07:00
> a ImproperOperation occurred (error 2252), Error 'in StackLoad' , 'saved
> stack is LinuxX86_64, process switch expects interpreted' . Execution
> cannot continue.
> 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line
> 4 [GsNMethod 67373805057]
> 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod
> 67373809409]
> 3 WAErrorHandler >> saveExceptionContinuation: (envId 0) @8 line
> 6 [GsNMethod 67530876161]
> 4 WECOProductionErrorHandler >> saveException: (envId 0) @5 line
> 5 [GsNMethod 67645394433]
> 5 WECOProductionErrorHandler >> handleDefault: (envId 0) @2 line
> 2 [GsNMethod 67645395713]
> 6 WAErrorHandler >> handleError: (envId 0) @2 line 2 [GsNMethod
> 67530874113]
> 7 WECOProductionErrorHandler >> handleError: (envId 0) @7 line
> 2 [GsNMethod 67645393921]
> 8 WAErrorHandler >> handleGemStoneException: (envId 0) @4 line
> 4 [GsNMethod 67530874369]
> 9 WAHtmlHaltAndErrorHandler >> handleException: (envId 0) @2 line
> 2 [GsNMethod 67547940609]
> 10 [] in WAExceptionHandler >> handleExceptionsDuring: (envId 0) @2 line
> 5 [GsNMethod 67615814145]
> 11 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 12 [] in WAExceptionHandler >> handleExceptionsDuring: (envId 0) @2 line
> 8 [GsNMethod 67589087233]
> 13 [] in ExecBlock >> on:do: (envId 0) @4 line 49 [GsNMethod 37709826305]
> 14 AbstractException >> _executeHandler: (envId 0) @3 line 8 [GsNMethod
> 129979137]
>
>
>
> *15 AbstractException >> _signalFromPrimitive: (envId 0) @1 line
> 1 [GsNMethod 129984257]16 WACallback >> evaluateWithFieldValues: (envId 0)
> @1 line 1 [GsNMethod 67530766081]17 [] in WACallbackRegistry >> handle:
> (envId 0) @4 line 10 [GsNMethod 67590116097]*18 Collection >> do: (envId
> 0) @5 line 10 [GsNMethod 121376513]
> 19 WACallbackRegistry >> handle: (envId 0) @9 line 9 [GsNMethod
> 67530772225]
> 20 [] in WAActionPhaseContinuation >> runCallbacks (envId 0) @5 line
> 4 [GsNMethod 67615798273]
> 21 ExecBlock >> onException:do: (envId 0) @2 line 66 [GsNMethod 125587969]
> 22 ExecBlock >> on:do: (envId 0) @5 line 47 [GsNMethod 125587457]
> 23 WAExceptionHandler >> handleExceptionsDuring: (envId 0) @2 line
> 3 [GsNMethod 67533463809]
> 24 [] in WARenderLoopContinuation >> withNotificationHandlerDo: (envId 0)
> @2 line 16 [GsNMethod 67591011585]
> 25 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 26 WARenderLoopContinuation >> withNotificationHandlerDo: (envId 0) @4
> line 17 [GsNMethod 67535573761]
> 27 [] in WAActionPhaseContinuation >> runCallbacks (envId 0) @2 line
> 3 [GsNMethod 67591003905]
> 28 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 29 WAActionPhaseContinuation >> runCallbacks (envId 0) @2 line
> 6 [GsNMethod 67535567105]
> 30 WAActionPhaseContinuation >> handleRequest (envId 0) @2 line
> 2 [GsNMethod 67535568129]
> 31 [] in WASessionContinuation >> basicValue (envId 0) @2 line
> 2 [GsNMethod 67591037697]
> 32 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 33 WASessionContinuation >> withUnregisteredHandlerDo: (envId 0) @2 line
> 3 [GsNMethod 67535609601]
> 34 WASessionContinuation >> basicValue (envId 0) @2 line 2 [GsNMethod
> 67535609345]
> 35 WASessionContinuation >> value (envId 0) @4 line 5 [GsNMethod
> 67535610369]
> 36 WASession >> handleFiltered: (envId 0) @9 line 10 [GsNMethod
> 67535603201]
> 37 [] in WARequestHandler >> handle: (envId 0) @3 line 4 [GsNMethod
> 67589540609]
> 38 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 39 WADynamicVariable class >> use:during: (envId 0) @2 line 4 [GsNMethod
> 67530848769]
> 40 [] in WARequestContext >> push:during: (envId 0) @2 line 5 [GsNMethod
> 67589523201]
> 41 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 42 WARequestContext >> push:during: (envId 0) @3 line 6 [GsNMethod
> 67534366721]
> 43 WARequestHandler >> handle: (envId 0) @2 line 4 [GsNMethod 67534726401]
> 44 WASession >> handle: (envId 0) @10 line 11 [GsNMethod 67535596545]
> 45 WARegistry >> dispatch:to: (envId 0) @2 line 5 [GsNMethod 67535205633]
> 46 WARegistry >> handleKeyed:with: (envId 0) @2 line 5 [GsNMethod
> 67535202305]
> 47 WARegistry >> handleFiltered: (envId 0) @33 line 19 [GsNMethod
> 67535204609]
> 48 WAApplication >> handleFiltered: (envId 0) @10 line 8 [GsNMethod
> 67535584257]
> 49 WARequestFilter >> handleFiltered: (envId 0) @3 line 4 [GsNMethod
> 67534418177]
> 50 [] in WAExceptionFilter >> handleFiltered: (envId 0) @2 line
> 7 [GsNMethod 67615813633]
> 51 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 52 WADynamicVariable class >> use:during: (envId 0) @2 line 4 [GsNMethod
> 67530848769]
> 53 [] in WAExceptionFilter >> handleFiltered: (envId 0) @2 line
> 6 [GsNMethod 67589083905]
> 54 ExecBlock >> onException:do: (envId 0) @2 line 66 [GsNMethod 125587969]
> 55 ExecBlock >> on:do: (envId 0) @5 line 47 [GsNMethod 125587457]
> 56 WAExceptionHandler >> handleExceptionsDuring: (envId 0) @2 line
> 3 [GsNMethod 67533463809]
> 57 WAExceptionFilter >> handleFiltered: (envId 0) @5 line 4 [GsNMethod
> 67530878977]
> 58 [] in WARequestHandler >> handle: (envId 0) @3 line 4 [GsNMethod
> 67589540609]
> 59 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 60 WADynamicVariable class >> use:during: (envId 0) @2 line 4 [GsNMethod
> 67530848769]
> 61 [] in WARequestContext >> push:during: (envId 0) @2 line 5 [GsNMethod
> 67589523201]
> 62 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 63 WARequestContext >> push:during: (envId 0) @3 line 6 [GsNMethod
> 67534366721]
> 64 WARequestHandler >> handle: (envId 0) @2 line 4 [GsNMethod 67534726401]
> 65 WADispatcher >> handleFiltered:named: (envId 0) @3 line 5 [GsNMethod
> 67530834177]
> 66 WADispatcher >> handleFiltered: (envId 0) @8 line 6 [GsNMethod
> 67530832129]
> 67 [] in WARequestHandler >> handle: (envId 0) @3 line 4 [GsNMethod
> 67589540609]
> 68 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 69 WADynamicVariable class >> use:during: (envId 0) @2 line 4 [GsNMethod
> 67530848769]
> 70 [] in WARequestContext >> push:during: (envId 0) @2 line 5 [GsNMethod
> 67589523201]
> 71 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 72 WARequestContext >> push:during: (envId 0) @3 line 6 [GsNMethod
> 67534366721]
> 73 WARequestHandler >> handle: (envId 0) @2 line 4 [GsNMethod 67534726401]
> 74 [] in WAServerAdaptor >> handleRequest: (envId 0) @3 line
> 4 [GsNMethod 67589601025]
> 75 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 76 WAServerAdaptor >> handleRequest: (envId 0) @2 line 5 [GsNMethod
> 67535027713]
> 77 [] in WAGsZincStreamingAdaptor >> handleRequestCheckingLock: (envId 0)
> @2 line 11 [GsNMethod 67655441409]
> 78 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 79 [] in WAGsZincStreamingAdaptor >> handleRequestCheckingLock: (envId 0)
> @13 line 12 [GsNMethod 67650054401]
> 80 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 81 TransientRecursionLock >> critical: (envId 0) @11 line 12 [GsNMethod
> 67077824257]
> 82 WAGsZincStreamingAdaptor >> handleRequestCheckingLock: (envId 0) @4
> line 5 [GsNMethod 67632024577]
> 83 WAGsZincStreamingAdaptor >> handleRequest: (envId 0) @5 line
> 5 [GsNMethod 67664767745]
> 84 WAServerAdaptor >> handle: (envId 0) @2 line 4 [GsNMethod 67535020801]
> 85 [] in WAServerAdaptor >> process: (envId 0) @2 line 6 [GsNMethod
> 67589604353]
> 86 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 87 WAServerAdaptor >> process: (envId 0) @4 line 7 [GsNMethod 67535024897]
> 88 [] in ZnZincStreamingServerAdaptor >> process: (envId 0) @4 line
> 8 [GsNMethod 67650065409]
> 89 ZnDeferredResponse >> writeOn: (envId 0) @2 line 2 [GsNMethod
> 67632026369]
> 90 ZnSingleThreadedServer >> writeResponse:on: (envId 0) @2 line
> 4 [GsNMethod 67548271873]
> 91 [] in ZnMultiThreadedServer >> writeResponseSafely:on: (envId 0) @2
> line 4 [GsNMethod 84589534721]
> 92 ExecBlock >> on:do: (envId 0) @3 line 42 [GsNMethod 125587457]
> 93 ZnMultiThreadedServer >> writeResponseSafely:on: (envId 0) @3 line
> 5 [GsNMethod 84589718785]
> 94 ZnMultiThreadedServer >> executeRequestResponseLoopOn: (envId 0) @11
> line 11 [GsNMethod 67548246273]
> 95 [] in ZnMultiThreadedServer >> serveConnectionsOn: (envId 0) @2 line
> 9 [GsNMethod 67629683969]
> 96 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 97 [] in ZnMultiThreadedServer >> serveConnectionsOn: (envId 0) @2 line
> 10 [GsNMethod 67615763457]
> 98 [] in ExecBlock >> ifCurtailed: (envId 0) @2 line 6 [GsNMethod
> 67381020929]
> 99 ExecBlock >> ensure: (envId 0) @2 line 12 [GsNMethod 125587201]
> 100 ExecBlock >> ifCurtailed: (envId 0) @3 line 8 [GsNMethod 67374152193]
> 101 [] in ZnMultiThreadedServer >> serveConnectionsOn: (envId 0) @2 line
> 11 [GsNMethod 67591244545]
> 102 GsProcess >> _start (envId 0) @7 line 16 [GsNMethod 130094081]
> 103 <Reenter marker>
>
> --
> Ken Treis
> Miriam Technologies, Inc.
> (866) 652-2040 x221
>
> On Jul 17, 2014, at 12:04 PM, Ken Treis <ken at miriamtech.com> wrote:
>
> We've recently moved our application to a new server, and my Zinc gems are
> crashing when continuations are saved:
>
> a ImproperOperation occurred (error 2252), Error 'in StackLoad' , 'saved
> stack is LinuxX86_64, process switch expects interpreted' . Execution
> cannot continue.
>
>
> This wasn't happening on the old server, and configs are identical as far
> as I can tell. I disabled native code in the gem, thinking that'd help, but
> it made no difference. In a way, I'm glad about that because I really want
> to use native code in production... but now I'm at a loss as to where this
> error is coming from.
>
> We're still on 3.1.0.4. The new server is Debian 7, while the old one was
> Ubuntu 12.04.4 LTS. Kernels are essentially the same (3.2.0). Both the old
> server and the new server are cloud instances, old was at Rackspace with
> AMD processors, and the new is at Amazon with Intel Xeons.
>
>
> --
> Ken Treis
> Miriam Technologies, Inc.
> (866) 652-2040 x221
>
> ====
>
>
> _____________________________________________________________________________
> | Configuration Files
> |
> |
> |
> | System File:
> /opt/gemstone/GemStone64Bit3.1.0.4-x86_64.Linux/seaside/data/system.conf
> |
> |
> | Executable File:
> /opt/gemstone/GemStone64Bit3.1.0.4-x86_64.Linux/seaside/etc/seaside30.conf
>
> |_____________________________________________________________________________|
>
> _____________________________________________________________________________
> | Gem Configuration Options for process id 15785
> |
>
> |_____________________________________________________________________________|
>
> DUMP_OPTIONS = TRUE;
> GEM_GCI_LOG_ENABLED = FALSE;
> GEM_ABORT_MAX_CRS = 0;
> GEM_FREE_FRAME_CACHE_SIZE = -1;
> GEM_FREE_FRAME_LIMIT = -1;
> GEM_FREE_PAGEIDS_CACHE = 200;
> GEM_HALT_ON_ERROR = 0;
> GEM_KEEP_MIN_SOFTREFS = 0;
> GEM_MAX_SMALLTALK_STACK_DEPTH = 1000;
> GEM_NATIVE_CODE_ENABLED = FALSE;
> GEM_PRIVATE_PAGE_CACHE_KB = 960;
> GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;
> GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;
> GEM_PGSVR_FREE_FRAME_LIMIT = -1;
> GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
> GEM_RPC_KEEPALIVE_INTERVAL = 0;
> GEM_RPCGCI_TIMEOUT = 0;
> GEM_RPC_USE_SSL = TRUE;
> GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;
> GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;
> GEM_TEMPOBJ_CACHE_SIZE = 400000;
> GEM_TEMPOBJ_MESPACE_SIZE = 0;
> GEM_TEMPOBJ_OOPMAP_SIZE = 0;
> GEM_TEMPOBJ_SCOPES_SIZE = 2000;
> GEM_TEMPOBJ_POMGEN_SIZE = 0;
> GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 100;
> GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
> GEM_TEMPOBJ_START_ADDR not used on this platform
> LOG_WARNINGS = TRUE;
> SHR_NUM_FREE_FRAME_SERVERS = -1;
> SHR_PAGE_CACHE_LOCKED = FALSE;
> SHR_PAGE_CACHE_NUM_PROCS = 4089;
> SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;
> SHR_PAGE_CACHE_PERMISSIONS = 660;
> SHR_PAGE_CACHE_SIZE_KB = 4000000;
> SHR_TARGET_FREE_FRAME_COUNT = -1;
> SHR_WELL_KNOWN_PORT_NUMBER = 0;
> (vmGc spaceSizes: eden init 2048K max 74944K , survivor init 448K max
> 12544K,
> vmGc old max 299968K, code max 80000K, perm max 40000K, pom 10 *
> 33344K = 333440K,
> vmGc remSet 8068K, meSpace max 382432K oopMapSize 2097152 max
> footprint 1250M)
>
> _____________________________________________________________________________
> | GemStone/S64 Object-Oriented Data Management System
> |
> | Copyright (C) VMware, Inc. 1986-2013
> |
> | All rights reserved.
> |
> | Covered by U.S Patents:
> |
> | 6,256,637 Transactional virtual machine architecture
> |
> | 6,360,219 Object queues with concurrent updating
> |
> | 6,567,905 Generational Garbage Collector.
> |
> | 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable
> Database
>
> +-----------------------------------------------------------------------------+
> | PROGRAM: topaz, Linear GemStone Interface (Linked Session)
> |
> | VERSION: 3.1.0.4, Mon Jun 17 13:16:00 2013
> |
> | BUILD: gss64_3_1_0_x_branch-31024
> |
> | BUILT FOR: x86-64 (Linux)
> |
> | MODE: 64 bit
> |
> | RUNNING ON: 4-CPU www.weco-write.com x86_64 (Linux 3.2.0-4-amd64 #1 SMP
> Debian
> | 3.2.57-3) 30488MB
> |
> | PROCESS ID: 15785 DATE: 07/17/2014 10:30:24 PDT
> |
> | USER IDS: REAL=gsadmin (1004) EFFECTIVE=gsadmin (1004)
> |
>
> |_____________________________________________________________________________|
> neither .topazini nor $HOME/.topazini were found
> topaz> topaz> topaz> topaz> topaz> topaz> topaz> topaz> topaz> topaz>
> topaz> topaz> topaz> topaz> [Info]: LNK client/gem GCI levels = 851/851
> [Info]: libssl-3.1.0.4-64.so: loaded
> [Info]: User ID: DataCurator
> [Info]: Repository: seaside
> [Info]: Session ID: 15
> [Info]: GCI Client Host: <Linked>
> [Info]: Page server PID: -1
> [Info]: Login Time: 07/17/2014 10:30:24.910 PDT
> [Info]: libicudata-3.1.0.4-64.so: loaded
> [Info]: libicuuc-3.1.0.4-64.so: loaded
> [Info]: libicui18n-3.1.0.4-64.so: loaded
> Gave this VM preference for OOM killer, Wrote to /proc/15785/oom_score_adj
> value 250
> [07/17/2014 10:30:24.941 PDT]
> gci login: currSession 1 rpc gem processId -1 OOB keep-alive interval -1
> successful login
> topaz 1> topaz 1> topaz 1> topaz 1> [71841854977 sz:5 cls: 135169 GsFile]
> aGsFile
> topaz 1> topaz 1> topaz 1> topaz 1> [76033 sz:19 cls: 779521 System class]
> System
> topaz 1> topaz 1> topaz 1> topaz 1> Zinc Server started on port 9008
> ----------- Continuation saved to object log LOG ENTRY: anArray-----------
> ----------- Continuation saved to object log ERROR Encountered:
> 2014-07-17T10:31:00.24015998840332-07:00
> a ImproperOperation occurred (error 2252), Error 'in StackLoad' , 'saved
> stack is LinuxX86_64, process switch expects interpreted' . Execution
> cannot continue.
> 1 GRGemStonePlatform >> logError:title:shouldCommit: (envId 0) @2 line
> 4 [GsNMethod 67373805057]
> 2 GRGemStonePlatform >> logError:title: (envId 0) @2 line 3 [GsNMethod
> 67373809409]
> 3 WAErrorHandler >> saveExceptionContinuation: (envId 0) @8 line
> 6 [GsNMethod 67530876161]
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
>
> _______________________________________________
> Glass mailing list
> Glass at lists.gemtalksystems.com
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20140717/29d2cfb6/attachment-0001.html>
More information about the Glass
mailing list