[Glass] Migrating from 3.2.9 to 3.3.3 block compilation issue
Dale Henrichs via Glass
glass at lists.gemtalksystems.com
Wed Feb 1 09:24:55 PST 2017
On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:
>
>
> On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs
> <dale.henrichs at gemtalksystems.com
> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>
>
>
> On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:
>>
>>
>> On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs
>> <dale.henrichs at gemtalksystems.com
>> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>>
>> On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:
>>>
>>>
>>> On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass
>>> <glass at lists.gemtalksystems.com
>>> <mailto:glass at lists.gemtalksystems.com>> wrote:
>>>
>>> Mariano,
>>>
>>> 1. Find allInstances of ExecBlock
>>>
>>> 2. Select all instances that #needsRecompile.
>>>
>>> 3. Send #recompile to the selected instances and collect
>>> instances that signal an error during #recompile. The
>>> #recompile method attempts to recompile Simple blocks
>>> in-place if there is an error during compilation or the
>>> block is not simple, an Error is signaled.
>>>
>>> 4. For the remaining instances of ExecBlock, you will
>>> have to track down the reference to the block and
>>> reinstall a new, recompiled instance of the ExecBlock.
>>>
>>> 1. look at the #blockSource and the #receiver for
>>> the block and you might recognize the method needed to
>>> recreate the block
>>>
>>> 2. look at the #blockSource and pick out a likely
>>> chunk of source that you can use to search in method
>>> source (`browse source --exact `<source-chunk>`)
>>>
>>> 3. Use Repository>>findReferencePathToObject: or
>>> Repository>>findAllReferencePathsToObject: to find which
>>> object(s) are holding onto the block.
>>>
>>>
>>> Hi Dale,
>>>
>>> OK, it seems I am not having good luck. I tried the
>>> following code:
>>>
>>> [
>>> | allBlocks allBlocksNeedingRecompile
>>> blocksFailedToRecompile correctlyRecompiled |
>>> blocksFailedToRecompile := OrderedCollection new.
>>> correctlyRecompiled := OrderedCollection new.
>>> System commit.
>>> allBlocks := ExecBlock allSubInstances.
>>> allBlocksNeedingRecompile := allBlocks select: [:each | each
>>> needsRecompile ].
>>> allBlocksNeedingRecompile do: [:each |
>>> [
>>> each recompile.
>>> correctlyRecompiled add: each.
>>> ] on: Error do: [:ex |
>>> blocksFailedToRecompile add: each.
>>> ]
>>> ].
>>> System commit.
>>> Transcript show: 'Correctly recompiled closures: ',
>>> correctlyRecompiled size asString.
>>> Transcript show: 'Erred recompiled closures: ',
>>> blocksFailedToRecompile size asString.
>>> ] on: Warning do: [:ex | ex resume ]
>>>
>>>
>>> And it outputs:
>>>
>>> Correctly recompiled closures: 0Erred recompiled closures:
>>> 797155
>>>
>>> Wow. That's a lot to fix.
>>>
>>> But things get strange when I analyze which are the failing
>>> closures. For example, I have an ExecBlock whose source is:
>>>
>>> /"This is source for a block. " /
>>> / ^ [ /
>>> / self error:/
>>> / 'No concrete db roles associated to '''/
>>> / , (FaDBAccessor common users row: username
>>> column: 'CONCRETE ROLE')/
>>> / , ''' of user ''' , username/
>>> / ,/
>>> / '''. Either the concrete db role id is wrong
>>> or the roles must be added to concreteRoles.csv' /
>>> /]/
>>>
>>>
>>>
>>> That closure, is a closure of an ifEmpty: of the
>>> method #sessionMapFor:prepareAccessLimitingSessionsWith:
>>> See:
>>>
>>> sessionMapFor: username prepareAccessLimitingSessionsWith:
>>> aDictionaryOfPreparationBlocks
>>> | answer concreteRole abstractRoleID |
>>> answer := IdentityDictionary new.
>>> concreteRole := self concreteRoleForUsername: username.
>>> concreteRole
>>> ifEmpty: [
>>> self error:
>>> 'No concrete db roles associated to '''
>>> , (FaDBAccessor common users row: username
>>> column: 'CONCRETE ROLE')
>>> , ''' of user ''' , username
>>> ,
>>> '''. Either the concrete db role id is wrong or
>>> the roles must be added to concreteRoles.csv'
>>> ].
>>> .....
>>>
>>>
>>> What is going on? I re-load all my app code after I upgraded
>>> stone so I expect that method to have been recompiled. In
>>> fact, I just edited the method and saved it again (so that a
>>> recompilation was triggered) and the above closure still
>>> appears as needing compilation!
>>>
>>> Even worst.... the _method instVar of that example IS NOT
>>> IDENTICAL (same OOP) to the one installed in the method
>>> dictionary (FaDBSessionsByRolesFactory >>
>>> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So it
>>> looks like if they were un-installed closures.
>>>
>>> I thought it could be some MC version history around or
>>> whatever. So I performed my typical GC and cleanup code
>>> which should collect all that garbage. Still same numbers.
>>>
>>> mmmm I am running out of ideas.
>>>
>>> Thoughts?
>> I imagine that the
>> #sessionMapFor:prepareAccessLimitingSessionsWith: method is
>> in an older version of the class, which would explain the
>> differences in oops and the fact that when you recompile the
>> method in the current version of the class, you don't
>> eliminate block that needs to be recompiled ...
>>
>> `GsDeployer new classesWithClassHistory` should give you the
>> list of classes that have older versions ...
>>
>>
>> Hi Dale,
>>
>> I tried with that and it answers an empty array.
>> Now, I tried in a much smaller stone (also migrated from 3.2.9 to
>> 3.3.3) and I also see quite some:
>>
>> Correctly recompiled closures: 4917. Erred recompiled closures: 30807
>>
>> And I still see the same scenario.. closures that come from
>> ifEmpty or ifAbsent or things like that that should be gone when
>> method is recompiled... Some of these closures/methods are not
>> even from my app but from tODE.
>>
>> With some code to fix the closures I faced while running some
>> parts of the app, I started to make the app to work. I cannot
>> spend much longer testing the general issue but instead try to
>> make the app to work. It's bad because I may lazily find more and
>> more closures needing to recompile as the app receives more testing.
>>
>> Anyway, I will do some more tests with
>> findAllReferencePathsToObject: to see if I found the issue...
>>
>> In case you think there could be something you may want to try /
>> do let me know (compressed backup of this extent is only 80MB).
>>
>> Thanks for your help!
>>
> I assume that you haven't done an MFC after the upgrade. At this
> point it seems likely that you are looking at the old versions of
> the methods...
>
> This is probably an important step to add to the instructions ...
> step 0: run an mfc ...
>
>
> I did that, but I admit it took me some time to realize that could be
> a problem. In fact, in that very same stone, the 30k decreased to 600
> (still quite some to track down!). So yeah, I think such a step 0 MFC
> would be nice to add to docs. I don't know how to open an issue for that.
internal bug #46657 (I already opened one:)
>
> Nevertheless, in the other stone (the 50GB one) I keep having a bit
> less than a million closures even after MFC.
>
I think we need to become a bit more methodical in tracking this down
... if you've done an all instances run, you should avoid persisting
the collection and you should probably logout of the session before
doing an MFC.
So I would like to know that there are no classes in class history ...
Given an MFC and no classes in class history, I would be surprised to
see blocks that are of the #isEmpty style .. if you have these types of
blocks still alive, I would think that we'd need to do an a reference
path analysis on them ... I would hope that the methods that are
returned and held onto in random spots would be recognizable and that
the reference path could be tracked down by looking senders of the
method where the block was created ...
It just occurred to me that you probably should clear the object log or
at a minimum delete all of the continuations in the object log, since
they would be another place where old methods would be kept alive ...
Dale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170201/41cdf337/attachment-0001.html>
More information about the Glass
mailing list