[Glass] Migrating from 3.2.9 to 3.3.3 block compilation issue
Mariano Martinez Peck via Glass
glass at lists.gemtalksystems.com
Wed Feb 1 11:33:17 PST 2017
On Wed, Feb 1, 2017 at 2:24 PM, Dale Henrichs <
dale.henrichs at gemtalksystems.com> wrote:
>
>
> On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:
>
>
>
> On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <
> dale.henrichs at gemtalksystems.com> wrote:
>
>>
>>
>> On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:
>>
>>
>>
>> On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <
>> dale.henrichs at gemtalksystems.com> wrote:
>>
>>> On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:
>>>
>>>
>>>
>>> On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <
>>> glass at lists.gemtalksystems.com> wrote:
>>>
>>>> Mariano,
>>>>
>>>> 1. Find allInstances of ExecBlock
>>>>
>>>> 2. Select all instances that #needsRecompile.
>>>>
>>>> 3. Send #recompile to the selected instances and collect instances that
>>>> signal an error during #recompile. The #recompile method attempts to
>>>> recompile Simple blocks in-place if there is an error during compilation or
>>>> the block is not simple, an Error is signaled.
>>>>
>>>> 4. For the remaining instances of ExecBlock, you will have to track
>>>> down the reference to the block and reinstall a new, recompiled instance of
>>>> the ExecBlock.
>>>>
>>>> 1. look at the #blockSource and the #receiver for the block and you
>>>> might recognize the method needed to recreate the block
>>>>
>>>> 2. look at the #blockSource and pick out a likely chunk of source
>>>> that you can use to search in method source (`browse source --exact
>>>> `<source-chunk>`)
>>>>
>>>> 3. Use Repository>>findReferencePathToObject: or
>>>> Repository>>findAllReferencePathsToObject: to find which object(s) are
>>>> holding onto the block.
>>>>
>>>
>>> Hi Dale,
>>>
>>> OK, it seems I am not having good luck. I tried the following code:
>>>
>>> [
>>> | allBlocks allBlocksNeedingRecompile blocksFailedToRecompile
>>> correctlyRecompiled |
>>> blocksFailedToRecompile := OrderedCollection new.
>>> correctlyRecompiled := OrderedCollection new.
>>> System commit.
>>> allBlocks := ExecBlock allSubInstances.
>>> allBlocksNeedingRecompile := allBlocks select: [:each | each
>>> needsRecompile ].
>>> allBlocksNeedingRecompile do: [:each |
>>> [
>>> each recompile.
>>> correctlyRecompiled add: each.
>>> ] on: Error do: [:ex |
>>> blocksFailedToRecompile add: each.
>>> ]
>>> ].
>>> System commit.
>>> Transcript show: 'Correctly recompiled closures: ',
>>> correctlyRecompiled size asString.
>>> Transcript show: 'Erred recompiled closures: ', blocksFailedToRecompile
>>> size asString.
>>> ] on: Warning do: [:ex | ex resume ]
>>>
>>>
>>> And it outputs:
>>>
>>> Correctly recompiled closures: 0Erred recompiled closures: 797155
>>>
>>> Wow. That's a lot to fix.
>>>
>>> But things get strange when I analyze which are the failing closures.
>>> For example, I have an ExecBlock whose source is:
>>>
>>> *"This is source for a block. " *
>>> * ^ [ *
>>> * self error:*
>>> * 'No concrete db roles associated to '''*
>>> * , (FaDBAccessor common users row: username column:
>>> 'CONCRETE ROLE')*
>>> * , ''' of user ''' , username*
>>> * ,*
>>> * '''. Either the concrete db role id is wrong or the roles
>>> must be added to concreteRoles.csv' *
>>> * ]*
>>>
>>>
>>>
>>> That closure, is a closure of an ifEmpty: of the
>>> method #sessionMapFor:prepareAccessLimitingSessionsWith:
>>> See:
>>>
>>> sessionMapFor: username prepareAccessLimitingSessionsWith:
>>> aDictionaryOfPreparationBlocks
>>> | answer concreteRole abstractRoleID |
>>> answer := IdentityDictionary new.
>>> concreteRole := self concreteRoleForUsername: username.
>>> concreteRole
>>> ifEmpty: [
>>> self error:
>>> 'No concrete db roles associated to '''
>>> , (FaDBAccessor common users row: username column: 'CONCRETE
>>> ROLE')
>>> , ''' of user ''' , username
>>> ,
>>> '''. Either the concrete db role id is wrong or the roles
>>> must be added to concreteRoles.csv'
>>> ].
>>> .....
>>>
>>>
>>> What is going on? I re-load all my app code after I upgraded stone so I
>>> expect that method to have been recompiled. In fact, I just edited the
>>> method and saved it again (so that a recompilation was triggered) and the
>>> above closure still appears as needing compilation!
>>>
>>> Even worst.... the _method instVar of that example IS NOT IDENTICAL
>>> (same OOP) to the one installed in the method dictionary
>>> (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith:
>>> ). So it looks like if they were un-installed closures.
>>>
>>> I thought it could be some MC version history around or whatever. So I
>>> performed my typical GC and cleanup code which should collect all that
>>> garbage. Still same numbers.
>>>
>>> mmmm I am running out of ideas.
>>>
>>> Thoughts?
>>>
>>> I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith:
>>> method is in an older version of the class, which would explain the
>>> differences in oops and the fact that when you recompile the method in the
>>> current version of the class, you don't eliminate block that needs to be
>>> recompiled ...
>>>
>>> `GsDeployer new classesWithClassHistory` should give you the list of
>>> classes that have older versions ...
>>>
>>
>> Hi Dale,
>>
>> I tried with that and it answers an empty array.
>>
>> Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3)
>> and I also see quite some:
>>
>> Correctly recompiled closures: 4917. Erred recompiled closures: 30807
>>
>> And I still see the same scenario.. closures that come from ifEmpty or
>> ifAbsent or things like that that should be gone when method is
>> recompiled... Some of these closures/methods are not even from my app but
>> from tODE.
>>
>> With some code to fix the closures I faced while running some parts of
>> the app, I started to make the app to work. I cannot spend much longer
>> testing the general issue but instead try to make the app to work. It's bad
>> because I may lazily find more and more closures needing to recompile as
>> the app receives more testing.
>>
>> Anyway, I will do some more tests with findAllReferencePathsToObject: to
>> see if I found the issue...
>>
>> In case you think there could be something you may want to try / do let
>> me know (compressed backup of this extent is only 80MB).
>>
>> Thanks for your help!
>>
>> I assume that you haven't done an MFC after the upgrade. At this point
>> it seems likely that you are looking at the old versions of the methods...
>>
>> This is probably an important step to add to the instructions ... step 0:
>> run an mfc ...
>>
>>
> I did that, but I admit it took me some time to realize that could be a
> problem. In fact, in that very same stone, the 30k decreased to 600 (still
> quite some to track down!). So yeah, I think such a step 0 MFC would be
> nice to add to docs. I don't know how to open an issue for that.
>
> internal bug #46657 (I already opened one:)
>
>
> Nevertheless, in the other stone (the 50GB one) I keep having a bit less
> than a million closures even after MFC.
>
>
> I think we need to become a bit more methodical in tracking this down ...
> if you've done an all instances run, you should avoid persisting the
> collection and you should probably logout of the session before doing an
> MFC.
>
>
Yes, I do this.
> So I would like to know that there are no classes in class history ...
>
>
Well, here i just discovered an issue...
`GsDeployer new classesWithClassHistory` shows empty (as I told you
earlier).
But... tODE `cls history` shows a bunch! And in fact, the examples I found
in the small stone seem to be from the classes listed in `cls history` ...
*I have to admit that I am not sure I like #classesWithClassHistory to NOT
show me certain classes even if they are "core" from GemSTone. I pretty
much have them all (like `cls history`) and decide myself which ones are
important to me. *
> Given an MFC and no classes in class history, I would be surprised to see
> blocks that are of the #isEmpty style .. if you have these types of blocks
> still alive, I would think that we'd need to do an a reference path
> analysis on them ... I would hope that the methods that are returned and
> held onto in random spots would be recognizable and that the reference path
> could be tracked down by looking senders of the method where the block was
> created ...
>
> It just occurred to me that you probably should clear the object log or at
> a minimum delete all of the continuations in the object log, since they
> would be another place where old methods would be kept alive ...
>
>
Yes, my cleaning also purges the object log completly.
So..as for the "small " stone it looks like I get down to do those from
different class versions of GemStone classes. So it seems good enough.
As for the big stone (50GB) I am still having a million closures and many
are NOT from class history > 1 .. .so I am still investigating this one.
Cheers.
--
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170201/a37bc21c/attachment-0001.html>
More information about the Glass
mailing list