[Glass] [GLASS] Seaside - growing extent - normal?
Dale Henrichs via Glass
glass at lists.gemtalksystems.com
Tue Apr 7 15:20:30 PDT 2015
Larry,
Okay, I cannot think of a reason that the earlier scans could not work,
so I'm thinking that I'd like to pick a smaller subset of the whole
system that runs faster and then try to understand why the scan isn't
working ...
So I'll start by finding out if UserInformationInterface will make a
likely candidate, so let's find out how big the cache is:
| app cache objectsByKey assoc expired1 expired2 timeout
lastAccessTable entry |
app := WADispatcher default handlers at: 'UserInformationInterface'.
cache := app cache.
objectsByKey := cache instVarAt: 3.
objectsByKey size
I'll put together another workspace for doing a scan on a per
application basis in th emean time ...
Dale
On 04/07/2015 07:28 AM, Lawrence Kellogg wrote:
> Answers below….
>
>> On Apr 6, 2015, at 9:59 PM, Dale Henrichs
>> <dale.henrichs at gemtalksystems.com
>> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>>
>>
>>
>> On 4/6/15 5:30 PM, Lawrence Kellogg wrote:
>>>
>>>> On Apr 6, 2015, at 8:25 PM, Dale Henrichs
>>>> <dale.henrichs at gemtalksystems.com
>>>> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>>>>
>>>> Okay,
>>>>
>>>> It looks like I assumed that you were using Seaside 3.1 and you
>>>> must be using Seaside3.0 ... I don't think that things have
>>>> actually changed in this area since Seaside 3.0, but it _is_
>>>> something to be aware of ... now...
>>>>
>>>> Given that the code (in Seaside-GemStone-Core-dkh.70) in
>>>> WARcLastAccessEntry>>isExpired:updating: (could you verify that
>>>> your version is the same?):
>>>>
>>>> isExpired: timeout updating: updating
>>>> | value |
>>>> ^ (value := self counter value) > lastCount
>>>> ifTrue: [
>>>> "accessed since last scan"
>>>> updating
>>>> ifTrue: [
>>>> lastCount := value.
>>>> time := Time totalSeconds ].
>>>> false ]
>>>> ifFalse: [ Time totalSeconds - time > timeout ]
>>>>
>>>> it appears that we have only done a expiration scan (updating =
>>>> true) only once and I have been under the impression that we've
>>>> done at least two full scans (the scan before the backup and the
>>>> scan after we discovered that we had not recovered any data) ... oh
>>>> well ...
>>>>
>>>> I think that the `time` in this entry corresponds to the date:
>>>> `2014-11-11T22:42:08+00:00` (Time totalSeconds returns seconds
>>>> since January 1, 1901) and that implies that the last scan was done
>>>> a year ago? Unless the code is very different or ... What does
>>>> `date` return from the linux shell?
>>>>
>>>> The next step is to see if the code is working like I expect it to.
>>>> The code below simulates and expiration scan, and then see how the
>>>> entry has changed along with couple of other things:
>>>>
>>>> | app cache objectsByKey assoc expired1 expired2 timeout
>>>> lastAccessTable entry |
>>>> app := WADispatcher default handlers at: 'UserInformationInterface'.
>>>> cache := app cache.
>>>> objectsByKey := cache instVarAt: 3.
>>>> assoc := objectsByKey associationAt: '5jTYOuLPzkOYLv4d'.
>>>> lastAccessTable := cache expiryPolicy instVarNamed:
>>>> #'lastAccessTable'.
>>>> entry := lastAccessTable at: '5jTYOuLPzkOYLv4d'.
>>>> timeout := cache expiryPolicy.
>>>> expired1 := entry isExpiredUpdating: timeout.
>>>> expired2 := entry isExpiredUpdating: -1.
>>>> ^ {timeout.
>>>> Time totalSeconds.
>>>> expired1.
>>>> entry.
>>>> entry time.
>>>> entry lastCount.
>>>> (entry counter value).
>>>> expired2}
>>>>
>>>> I've got to head home, but I'll check in again this evening when I
>>>> get the chance ...
>>>>
>>>
>>> Thanks, the above code returns:
>>>
>>> anArray( aWARcLastAccessExpiryPolicy, 3605819381, false,
>>> aWARcLastAccessEntry, 3605819381, 1, 1, true)
>>>
>> The time, corresponds to 2015-04-07T00:29:41+00:00...so the date on
>> your server is probably correct...
>>
>> Oh shoot, the first slot should have been `cache expiryPolicy
>> timeout`, so let's try again, with an edited workspace:
>>
>> | app cache objectsByKey assoc expired1 expired2 timeout
>> lastAccessTable entry |
>> app := WADispatcher default handlers at: 'UserInformationInterface'.
>> cache := app cache.
>> objectsByKey := cache instVarAt: 3.
>> assoc := objectsByKey associationAt: '5jTYOuLPzkOYLv4d'.
>> lastAccessTable := cache expiryPolicy instVarNamed: #'lastAccessTable'.
>> entry := lastAccessTable at: '5jTYOuLPzkOYLv4d'.
>> timeout := cache expiryPolicy timeout.
>> expired1 := entry isExpiredUpdating: timeout.
>> expired2 := entry isExpiredUpdating: -1.
>> ^ {timeout.
>> Time totalSeconds.
>> expired1.
>> entry.
>> entry time.
>> entry lastCount.
>> (entry counter value).
>> expired2}
>>
>
> Ok, here is the result from the above code, substituting at: for
> associationAt:
>
> anArray( 600, 3605869347, true, aWARcLastAccessEntry, 3605819381, 1,
> 1, true)
>
>
>
>
>> From this little experiment it seems that we've only run one
>> expiration run against this particular session ... so after you get
>> me the results from this array and the timeout (first slot) seems to
>> be a reasonable number (less than 12,620,853), then we will try
>> running the expiration run again... BUT before we run anther
>> expiration run, I want to see the code that you are using in
>
>
>> WACache>>gemstoneReap,
>
>
> gemstoneReap
> "Iterate through the cache and remove objects that have expired."
>
> "In GemStone, this method is performed by a separate maintenance VM,
> so we are already in transaction (assumed to be running in
> #autoBegin
> transactionMode) and do not have to worry about acquiring the
> TransactionMutex.
> Since we are using reducedConflict dictionaries in the first
> place, we will remove the keys
> and values from the existing dictionaries without using the mutex."
>
> | expired count platform |
> expired := UserGlobals at: #'ExpiryCleanup' put: OrderedCollection new.
> platform := GRPlatform current.
> platform doCommitTransaction.
> count := 0.
> objectsByKey
> associationsDo: [ :assoc |
> (self expiryPolicy isExpiredUpdating: assoc value key: assoc key)
> ifTrue: [
> self notifyRemoved: assoc value key: assoc key.
> count := count + 1.
> Transcript cr; show: 'Scan progress: ' , count printString.
> expired add: assoc.
> count \\ 100 == 0
> ifTrue: [platform doCommitTransaction ].
> count \\ 1000 == 0
> ifTrue: [Transcript cr; show: 'Scan progress: ' , count
> printString.] ] ].
> Transcript cr; show: 'finished scan: ' , count printString.
> platform doCommitTransaction.
> count := 0.
> (UserGlobals at: #'ExpiryCleanup' )
> do: [ :assoc |
> count := count + 1.
> Transcript cr; show: 'Expire progress: ' , count printString.
> objectsByKey removeKey: assoc key ifAbsent: [].
> keysByObject removeKey: assoc value ifAbsent: [ ].
> count \\ 100 == 0
> ifTrue: [ platform doCommitTransaction ].
> count \\ 1000 == 0
> ifTrue: [ Transcript cr; show: 'Expire progress: ' , count
> printString.]].
> platform doCommitTransaction.
> UserGlobals removeKey: #'ExpiryCleanup'.
> platform doCommitTransaction.
> Transcript show: 'Leaving gemstoneReap: ',expired size printString.
> ^ expired size
>
>> WABasicDevelopment class>>reapHandlerCache:dispatchers: and
>
> reapHandlerCache: aHandler dispatchers: dispatchers
> | expired |
> expired := 0.
> (aHandler isApplication or: [ aHandler isSession ])
> ifTrue: [
> expired := aHandler cache gemstoneReap.
> aHandler sessionsDo: [:session | expired := expired + session
> continuations gemstoneReap ]]
> ifFalse: [
> aHandler isDispatcher
> ifTrue: [ dispatchers add: aHandler ]].
> ^expired
>
>
>> WABasicDevelopment class>>reapSeasideCache, just to see if I can spot
>> a transaction error before running a second (and third) expiration.
>
> reapSeasideCache
>
> | dps expired |
> expired := 0.
> dps := OrderedCollection with: WADispatcher default.
> [ dps isEmpty ] whileFalse: [ | dp |
> dp := dps removeFirst.
> dp isApplication
> ifTrue: [
> expired := expired + (self reapHandlerCache: dp dispatchers: dps) ]
> ifFalse: [
> dp handlers valuesDo: [:ep |
> expired := expired + (self reapHandlerCache: ep dispatchers: dps) ]]].
> ^expired
>
>
>>
>> Yes twice more ... for some unknown reason, the session we are
>> looking has apparently not been scanned before (which is why I want
>> to read the code that is in your image) the `lastCount` should have
>> been equal to the `counter value` and I'm not quite sure why the
>> lastCount hasn't been bumped ... unless the first time we touched the
>> session is when we just ran the above workspace ...
>>
>> I will have to drop offline for awhile, but I will try bringing up a
>> stone that has an older version of Seaside installed to see if there
>> are indeed differences in the expiration logic ...
>>
>> I assume that you've been running these tests in the stone that we
>> restored from backup ..... or have you been running these tests in a
>> stone in which you've never run the expiration code?
>>
>
> Yes, all of the test have been run from the stone that I restored from
> backup, and the reap code has been run multiple times, as I recall.
>
> Best,
>
> Larry
>
>
>
>> Dale
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150407/6bd26567/attachment-0001.html>
More information about the Glass
mailing list