[Glass] Migrating from 3.2.9 to 3.3.3 block compilation issue

Wed Feb 1 09:09:25 PST 2017

On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs <
dale.henrichs at gemtalksystems.com> wrote:

>
>
> On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:
>
>
>
> On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs <
> dale.henrichs at gemtalksystems.com> wrote:
>
>> On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:
>>
>>
>>
>> On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via Glass <
>> glass at lists.gemtalksystems.com> wrote:
>>
>>> Mariano,
>>>
>>> 1. Find allInstances of ExecBlock
>>>
>>> 2. Select all instances that #needsRecompile.
>>>
>>> 3. Send #recompile to the selected instances and collect instances that
>>> signal an error during #recompile. The #recompile method attempts to
>>> recompile Simple blocks in-place if there is an error during compilation or
>>> the block is not simple, an Error is signaled.
>>>
>>> 4. For the remaining instances of ExecBlock, you will have to track down
>>> the reference to the block and reinstall a new, recompiled instance of the
>>> ExecBlock.
>>>
>>>     1. look at the #blockSource and the #receiver for the block and you
>>> might recognize the method needed to recreate the block
>>>
>>>     2. look at the #blockSource and pick out a likely chunk of source
>>> that you can use to search in method source (`browse source --exact
>>> `<source-chunk>`)
>>>
>>>     3. Use Repository>>findReferencePathToObject: or
>>> Repository>>findAllReferencePathsToObject: to find which object(s) are
>>> holding onto the block.
>>>
>>
>> Hi Dale,
>>
>> OK, it seems I am not having good luck. I tried the following code:
>>
>> [
>> | allBlocks allBlocksNeedingRecompile blocksFailedToRecompile
>> correctlyRecompiled |
>> blocksFailedToRecompile := OrderedCollection new.
>> correctlyRecompiled := OrderedCollection new.
>> System commit.
>> allBlocks := ExecBlock allSubInstances.
>> allBlocksNeedingRecompile := allBlocks select: [:each | each
>> needsRecompile ].
>> allBlocksNeedingRecompile do: [:each |
>> [
>> each recompile.
>> correctlyRecompiled add: each.
>>   ] on: Error do: [:ex |
>> blocksFailedToRecompile add: each.
>> ]
>> ].
>> System commit.
>> Transcript show:  'Correctly recompiled closures: ',  correctlyRecompiled
>> size asString.
>> Transcript show: 'Erred recompiled closures: ',  blocksFailedToRecompile
>> size asString.
>> ] on: Warning do: [:ex | ex resume  ]
>>
>>
>> And it outputs:
>>
>> Correctly recompiled closures: 0Erred recompiled closures: 797155
>>
>> Wow. That's a lot to fix.
>>
>> But things get strange when I analyze which are the failing closures. For
>> example,  I have an ExecBlock whose source is:
>>
>> *"This is source for a block.  " *
>> * ^ [ *
>> *      self error:*
>> *          'No concrete db roles associated to '''*
>> *            , (FaDBAccessor common users row: username column: 'CONCRETE
>> ROLE')*
>> *            , ''' of user ''' , username*
>> *            ,*
>> *              '''. Either the concrete db role id is wrong or the roles
>> must be added to concreteRoles.csv' *
>> * ]*
>>
>>
>>
>> That closure, is a closure of an ifEmpty:  of the
>> method #sessionMapFor:prepareAccessLimitingSessionsWith:
>> See:
>>
>> sessionMapFor: username prepareAccessLimitingSessionsWith:
>> aDictionaryOfPreparationBlocks
>> | answer  concreteRole abstractRoleID  |
>> answer := IdentityDictionary new.
>> concreteRole := self concreteRoleForUsername: username.
>> concreteRole
>>    ifEmpty: [
>>      self error:
>>          'No concrete db roles associated to '''
>>            , (FaDBAccessor common users row: username column: 'CONCRETE
>> ROLE')
>>            , ''' of user ''' , username
>>            ,
>>              '''. Either the concrete db role id is wrong or the roles
>> must be added to concreteRoles.csv'
>> ].
>> .....
>>
>>
>> What is going on? I re-load all my app code after I upgraded stone so I
>> expect that method to have been recompiled. In fact, I just edited the
>> method and saved it again (so that a recompilation was triggered) and the
>> above closure still appears as needing compilation!
>>
>> Even worst....   the _method  instVar of that example IS NOT IDENTICAL
>> (same OOP) to the one installed in the method dictionary
>> (FaDBSessionsByRolesFactory >> #sessionMapFor:prepareAccessLimitingSessionsWith:
>> ). So it looks like if they were un-installed closures.
>>
>> I thought it could be some MC version history around or whatever. So I
>> performed my typical GC and cleanup code which should collect all that
>> garbage. Still same numbers.
>>
>> mmmm I am running out of ideas.
>>
>> Thoughts?
>>
>> I imagine that the #sessionMapFor:prepareAccessLimitingSessionsWith:
>> method is in an older version of the class, which would explain the
>> differences in oops and the fact that when you recompile the method in the
>> current version of the class, you don't eliminate block that needs to be
>> recompiled ...
>>
>> `GsDeployer new classesWithClassHistory` should give you the list of
>> classes that have older versions ...
>>
>
> Hi Dale,
>
> I tried with that and it answers an empty array.
>
> Now, I tried in a much smaller stone (also migrated from 3.2.9 to 3.3.3)
> and I also see quite some:
>
> Correctly recompiled closures: 4917. Erred recompiled closures: 30807
>
> And I still see the same scenario.. closures that come from ifEmpty or
> ifAbsent or things like that that should be gone when method is
> recompiled... Some of these closures/methods are not even from my app but
> from tODE.
>
> With some code to fix the closures I faced while running some parts of the
> app, I started to make the app to work. I cannot spend much longer testing
> the general issue but instead try to make the app to work. It's bad because
> I may lazily find more and more closures needing to recompile as the app
> receives more testing.
>
> Anyway, I will do some more tests with findAllReferencePathsToObject: to
> see if I found the issue...
>
> In case you think there could be something you may want to try / do let me
> know (compressed backup of this extent is only 80MB).
>
> Thanks for your help!
>
> I assume that you haven't  done an MFC after the upgrade. At this point it
> seems likely that you are looking at the old versions of the methods...
>
> This is probably an important step to add to the instructions ... step 0:
> run an mfc ...
>
>
I did that, but I admit it took me some time to realize that could be a
problem. In fact, in that very same stone, the 30k decreased to 600 (still
quite some to track down!). So yeah, I think such a step 0 MFC would be
nice to add to docs. I don't know how to open an issue for that.

Nevertheless, in the other stone (the 50GB one) I keep having a bit less
than a million closures even after MFC.

-- 
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170201/e1988748/attachment-0001.html>