[Glass] Migrating from 3.2.9 to 3.3.3 block compilation issue
Dale Henrichs via Glass
glass at lists.gemtalksystems.com
Wed Feb 1 12:18:30 PST 2017
On 02/01/2017 11:33 AM, Mariano Martinez Peck wrote:
>
>
> On Wed, Feb 1, 2017 at 2:24 PM, Dale Henrichs
> <dale.henrichs at gemtalksystems.com
> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>
>
>
> On 02/01/2017 09:09 AM, Mariano Martinez Peck wrote:
>>
>>
>> On Wed, Feb 1, 2017 at 1:20 PM, Dale Henrichs
>> <dale.henrichs at gemtalksystems.com
>> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>>
>>
>>
>> On 02/01/2017 04:40 AM, Mariano Martinez Peck wrote:
>>>
>>>
>>> On Wed, Feb 1, 2017 at 2:20 AM, Dale Henrichs
>>> <dale.henrichs at gemtalksystems.com
>>> <mailto:dale.henrichs at gemtalksystems.com>> wrote:
>>>
>>> On 1/31/17 7:44 PM, Mariano Martinez Peck wrote:
>>>>
>>>>
>>>> On Tue, Jan 31, 2017 at 2:25 PM, Dale Henrichs via
>>>> Glass <glass at lists.gemtalksystems.com
>>>> <mailto:glass at lists.gemtalksystems.com>> wrote:
>>>>
>>>> Mariano,
>>>>
>>>> 1. Find allInstances of ExecBlock
>>>>
>>>> 2. Select all instances that #needsRecompile.
>>>>
>>>> 3. Send #recompile to the selected instances and
>>>> collect instances that signal an error during
>>>> #recompile. The #recompile method attempts to
>>>> recompile Simple blocks in-place if there is an
>>>> error during compilation or the block is not
>>>> simple, an Error is signaled.
>>>>
>>>> 4. For the remaining instances of ExecBlock, you
>>>> will have to track down the reference to the block
>>>> and reinstall a new, recompiled instance of the
>>>> ExecBlock.
>>>>
>>>> 1. look at the #blockSource and the #receiver
>>>> for the block and you might recognize the method
>>>> needed to recreate the block
>>>>
>>>> 2. look at the #blockSource and pick out a
>>>> likely chunk of source that you can use to search
>>>> in method source (`browse source --exact
>>>> `<source-chunk>`)
>>>>
>>>> 3. Use Repository>>findReferencePathToObject:
>>>> or Repository>>findAllReferencePathsToObject: to
>>>> find which object(s) are holding onto the block.
>>>>
>>>>
>>>> Hi Dale,
>>>>
>>>> OK, it seems I am not having good luck. I tried the
>>>> following code:
>>>>
>>>> [
>>>> | allBlocks allBlocksNeedingRecompile
>>>> blocksFailedToRecompile correctlyRecompiled |
>>>> blocksFailedToRecompile := OrderedCollection new.
>>>> correctlyRecompiled := OrderedCollection new.
>>>> System commit.
>>>> allBlocks := ExecBlock allSubInstances.
>>>> allBlocksNeedingRecompile := allBlocks select: [:each |
>>>> each needsRecompile ].
>>>> allBlocksNeedingRecompile do: [:each |
>>>> [
>>>> each recompile.
>>>> correctlyRecompiled add: each.
>>>> ] on: Error do: [:ex |
>>>> blocksFailedToRecompile add: each.
>>>> ]
>>>> ].
>>>> System commit.
>>>> Transcript show: 'Correctly recompiled closures: ',
>>>> correctlyRecompiled size asString.
>>>> Transcript show: 'Erred recompiled closures: ',
>>>> blocksFailedToRecompile size asString.
>>>> ] on: Warning do: [:ex | ex resume ]
>>>>
>>>>
>>>> And it outputs:
>>>>
>>>> Correctly recompiled closures: 0Erred recompiled
>>>> closures: 797155
>>>>
>>>> Wow. That's a lot to fix.
>>>>
>>>> But things get strange when I analyze which are the
>>>> failing closures. For example, I have an ExecBlock
>>>> whose source is:
>>>>
>>>> /"This is source for a block. " /
>>>> / ^ [ /
>>>> / self error:/
>>>> / 'No concrete db roles associated to '''/
>>>> / , (FaDBAccessor common users row: username
>>>> column: 'CONCRETE ROLE')/
>>>> / , ''' of user ''' , username/
>>>> / ,/
>>>> / '''. Either the concrete db role id is wrong or the
>>>> roles must be added to concreteRoles.csv' /
>>>> /]/
>>>>
>>>>
>>>>
>>>> That closure, is a closure of an ifEmpty: of the
>>>> method #sessionMapFor:prepareAccessLimitingSessionsWith:
>>>> See:
>>>>
>>>> sessionMapFor: username
>>>> prepareAccessLimitingSessionsWith:
>>>> aDictionaryOfPreparationBlocks
>>>> | answer concreteRole abstractRoleID |
>>>> answer := IdentityDictionary new.
>>>> concreteRole := self concreteRoleForUsername: username.
>>>> concreteRole
>>>> ifEmpty: [
>>>> self error:
>>>> 'No concrete db roles associated to '''
>>>> , (FaDBAccessor common users row: username
>>>> column: 'CONCRETE ROLE')
>>>> , ''' of user ''' , username
>>>> ,
>>>> '''. Either the concrete db role id is wrong or the
>>>> roles must be added to concreteRoles.csv'
>>>> ].
>>>> .....
>>>>
>>>>
>>>> What is going on? I re-load all my app code after I
>>>> upgraded stone so I expect that method to have been
>>>> recompiled. In fact, I just edited the method and saved
>>>> it again (so that a recompilation was triggered) and
>>>> the above closure still appears as needing compilation!
>>>>
>>>> Even worst.... the _method instVar of that example IS
>>>> NOT IDENTICAL (same OOP) to the one installed in the
>>>> method dictionary (FaDBSessionsByRolesFactory >>
>>>> #sessionMapFor:prepareAccessLimitingSessionsWith: ). So
>>>> it looks like if they were un-installed closures.
>>>>
>>>> I thought it could be some MC version history around or
>>>> whatever. So I performed my typical GC and cleanup code
>>>> which should collect all that garbage. Still same numbers.
>>>>
>>>> mmmm I am running out of ideas.
>>>>
>>>> Thoughts?
>>> I imagine that the
>>> #sessionMapFor:prepareAccessLimitingSessionsWith: method
>>> is in an older version of the class, which would explain
>>> the differences in oops and the fact that when you
>>> recompile the method in the current version of the
>>> class, you don't eliminate block that needs to be
>>> recompiled ...
>>>
>>> `GsDeployer new classesWithClassHistory` should give you
>>> the list of classes that have older versions ...
>>>
>>>
>>> Hi Dale,
>>>
>>> I tried with that and it answers an empty array.
>>> Now, I tried in a much smaller stone (also migrated from
>>> 3.2.9 to 3.3.3) and I also see quite some:
>>>
>>> Correctly recompiled closures: 4917. Erred recompiled
>>> closures: 30807
>>>
>>> And I still see the same scenario.. closures that come from
>>> ifEmpty or ifAbsent or things like that that should be gone
>>> when method is recompiled... Some of these closures/methods
>>> are not even from my app but from tODE.
>>>
>>> With some code to fix the closures I faced while running
>>> some parts of the app, I started to make the app to work. I
>>> cannot spend much longer testing the general issue but
>>> instead try to make the app to work. It's bad because I may
>>> lazily find more and more closures needing to recompile as
>>> the app receives more testing.
>>>
>>> Anyway, I will do some more tests with
>>> findAllReferencePathsToObject: to see if I found the issue...
>>>
>>> In case you think there could be something you may want to
>>> try / do let me know (compressed backup of this extent is
>>> only 80MB).
>>>
>>> Thanks for your help!
>>>
>> I assume that you haven't done an MFC after the upgrade. At
>> this point it seems likely that you are looking at the old
>> versions of the methods...
>>
>> This is probably an important step to add to the instructions
>> ... step 0: run an mfc ...
>>
>>
>> I did that, but I admit it took me some time to realize that
>> could be a problem. In fact, in that very same stone, the 30k
>> decreased to 600 (still quite some to track down!). So yeah, I
>> think such a step 0 MFC would be nice to add to docs. I don't
>> know how to open an issue for that.
> internal bug #46657 (I already opened one:)
>>
>> Nevertheless, in the other stone (the 50GB one) I keep having a
>> bit less than a million closures even after MFC.
>>
>
> I think we need to become a bit more methodical in tracking this
> down ... if you've done an all instances run, you should avoid
> persisting the collection and you should probably logout of the
> session before doing an MFC.
>
>
> Yes, I do this.
>
> So I would like to know that there are no classes in class history
> ...
>
>
> Well, here i just discovered an issue...
>
> `GsDeployer new classesWithClassHistory` shows empty (as I told you
> earlier).
>
> But... tODE `cls history` shows a bunch! And in fact, the examples I
> found in the small stone seem to be from the classes listed in `cls
> history` ...
>
> *I have to admit that I am not sure I like #classesWithClassHistory to
> NOT show me certain classes even if they are "core" from GemSTone. I
> pretty much have them all (like `cls history`) and decide myself which
> ones are important to me.
> *
I had forgotten that GsDeployer had the excluded Globals list ... sorry
about that ...
Dale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20170201/43a7ce2d/attachment-0001.html>
More information about the Glass
mailing list