[Glass] Git based repositories (package-cache) problem when moving extent to another stone
Mariano Martinez Peck via Glass
glass at lists.gemtalksystems.com
Tue Oct 27 10:50:37 PDT 2015
Dale,
While that more robust #exists does help in the sense that the user gets a
much better error message, I think it's not enough. So now I get this
message:
...RETRY->BaselineOfSeaside3
...RETRY->BaselineOfSeaside3
gofer repository error: 'GoferRepositoryError: UserDefinedError: Disk
error: for path:
/opt/gemstoneAdditions/GsDevKit_home/server/stones/gs_329/logs/github-cache/GsDevKit/Seaside31/v3.1.4.2-gs/GsDevKit-Seaside31-55f1bac/repository'...ignoring
...FAILED->BaselineOfSeaside3
But my code still fail to load. There are a couple of places that should be
changed. Search for the string "*MCFileTreeFileUtils current
directoryExists:*"
And you will find
--------------------
MCGitHubRepository class>>resetCacheDirectoryIfInvalid
MCGitBasedNetworkRepository>>directory
MCGitBasedNetworkRepository class>>resetCacheDirectoryIfInvalid
MCFileTreeRepository>>flushCache
I think those guys should handle the error and on error, the flush or reset
anyway. To avoid this (adding error handling in all those places), I
suggested adding:
MCFileTreeFileDirectoryUtils class >> directoryExistsAndValid: aDirectory
^ aDirectory exists on: Error do: [ false ]
And so, the above 4 methods should actually send #directoryExistsAndValid:
rather than #directoryExists
Thoughts? My worry is portability.
Thanks,
On Tue, Oct 27, 2015 at 2:04 PM, Dale Henrichs <
dale.henrichs at gemtalksystems.com> wrote:
> Here's the bug report[1]
>
> [1] https://github.com/GsDevKit/GsDevKit/issues/79
>
>
> On 10/27/2015 10:01 AM, Dale Henrichs wrote:
>
> Ah ... good analysis see if the following change to FileDirectory>>exists
> gives a better error:
>
> exists
> | result |
> result := GsFile _exists: self pathString onClient: self onClient.
> result ifNil: [ self error: 'Disk erro: ' , GsFile lastErrorString ].
> ^ result
>
> If so, I've got it queued up as a change on the glassdb/glass dev branch
> ...
>
> Dale
>
> On 10/26/2015 06:47 PM, Mariano Martinez Peck wrote:
>
> Dale,
>
> As you know, the original error was: *a ImproperOperation occurred
> (error 2085), Expected nil to be a Boolean*
>
> The problem is the code like this *"MCFileTreeFileUtils current
> directoryExists: " . *The real problem is that #directoryExists: is like
> this:
>
> ---------
> MCFileTreeFileDirectoryUtils class >> directoryExists: aDirectory
> ^ aDirectory exists.
> ------
>
> However, if the directory EXISTS but you do not have permissions, you get
> nil, hence the "*Expected nil to be a Boolean"*
>
> Of course, an example of that code is
>
> ---------
> MCGitHubRepository class >> resetCacheDirectoryIfInvalid
> "use class var to survive upgrade when MCGitHubRepository moved to
> subclass of MCGitBasedRepository"
>
> "Reset if invalid"
>
> CacheDirectory notNil
> and: [
> * (MCFileTreeFileUtils current directoryExists: CacheDirectory)*
> ifFalse: [ CacheDirectory := nil ] ]
> ----------
>
> *But again, this is only ONE of the senders. There are others of above
> kind of code *(sending #directoryExists: and then sending an ifFalse: or
> isTrue:)
>
> If the answering nil when not correct permisions WOULD be
> general/portable, then I think a solution could be to create MCFileTreeFileDirectoryUtils
> class >> directoryExistsAndValid: that does:
>
> ---------
> directoryExistsAndValid: aDirectory
> ^ aDirectory exists ifNil: [false]
> --------
>
> Anyway, now the problem is clear. Maybe you can come with a better
> approach?
>
> Cheers,
>
>
> On Mon, Oct 26, 2015 at 10:13 PM, Mariano Martinez Peck <
> <marianopeck at gmail.com>marianopeck at gmail.com> wrote:
>
>> Dale,
>>
>> Do not continue loosing time until my next email. I found the
>> problem/solution but I don't have the time to explain and detail until
>> tomorrow.
>>
>> Thanks!
>>
>> On Mon, Oct 26, 2015 at 9:22 PM, Dale Henrichs <
>> <dale.henrichs at gemtalksystems.com>dale.henrichs at gemtalksystems.com>
>> wrote:
>>
>>> I'm headed home
>>>
>>> If the MCGitHubRepository class>>resetCacheDirectoryIfInvalid works for
>>> a deleted directory (and you have found that that case works), then it
>>> should work if th cache is explicitly cleared ...
>>>
>>> HOWEVER, I recall that resetCacheDirectoryIfInvalid is triggered during
>>> image startup, so perhaps there are additional initialization steps that
>>> run at image startup ... perhaps the directory inst vars are nuked I didn't
>>> look closely ... anyway take a look down that path and see if you can find
>>> the additional inits that might be run at image startup (also be suspicious
>>> of SessionTemps being used, since SessionTemps don't survive across logins)
>>> ...
>>>
>>> Dale
>>>
>>>
>>> On 10/26/2015 05:09 PM, Mariano Martinez Peck wrote:
>>>
>>>
>>> On Mon, Oct 26, 2015 at 7:58 PM, Dale Henrichs via Glass <
>>> <glass at lists.gemtalksystems.com>glass at lists.gemtalksystems.com> wrote:
>>>
>>>>
>>>>
>>>> On 10/26/2015 03:03 PM, Mariano Martinez Peck via Glass wrote:
>>>>
>>>>> Dale,
>>>>>
>>>>> I am finding problems with git-based repositories. Basically, let's
>>>>> say I load some stuff from git under the stone called 3.2.9. Then I copy
>>>>> the extent and I restore such a extent in another stone, called
>>>>> 'otherStone'.
>>>>>
>>>> What scripts steps do you use when you"copy and restore in another
>>>> stone" ....
>>>
>>>
>>> I do a snapshot with tODE then a cp of the extent.
>>>
>>>
>>>>
>>>>
>>>>> I now try to update code in my 'otherStone' and I get an exception
>>>>> while loading Seaside from metacello. The problem ends up being that in
>>>>> fact it is trying to get the stuff from here:
>>>>>
>>>>>
>>>>> '/opt/gemstoneAdditions/GsDevKit_home/server/stones/gs_329/logs/github-cache/GsDevKit/Seaside31/v3.1.4.2-gs/GsDevKit-Seaside31-55f1bac/repository'
>>>>>
>>>> Ah okay ... I've been working with local clones of all of the projects
>>>> (the mode that I think everyone should follow) ... the problem you are
>>>> seeing has to do with using github:// repositories ... git repositories use
>>>> filetree:// and the expectation is that you are moving extents within the
>>>> same $GS_HOME environment ... nonetheless this should work ...
>>>
>>>
>>> Yes, the problem I found is when using github repos, not filetree (at
>>> least the error I had), and yes, using same $GS_HOME.
>>>
>>>
>>>>
>>>>
>>>>> All those files have a different OS owner and file permissiones etc
>>>>> etc.
>>>>>
>>>>> Of course, if I remove
>>>>> /opt/gemstoneAdditions/GsDevKit_home/server/stones/gs_329/logs/github-cache
>>>>> it suddenly starts to work in 'otherStone'.
>>>>>
>>>> This seems to be a bug in MCGitHubRepository ... specifically
>>>> MCGitHubRepository class>>resetCacheDirectoryIfInvalid will reset the
>>>> CacheDirectory if it's no longer "valid" but it's test for "valid" is that
>>>> the directory exists ... it should also worry about permissions ... I guess
>>>> ... alternatively I could catch an error on access and reset the
>>>> CacheDirectory and try again ... I don't know if there is platform
>>>> independent protocol in Metacello for doing permission testing ... no quick
>>>> fix there, I think ....
>>>>
>>>
>>> Yes, that's exactly the point. In my case, the 'another' stone runs with
>>> a different OS user and hence there is a permission issue at file system
>>> level.
>>>
>>>
>>>
>>>>
>>>> In the mean time, you can clear the cache directory manually:
>>>>
>>>> MCGitHubRepository cacheDirectory: nil.
>>>>
>>>> If you are using $GS_HOME/bin/newExtent or $GS_HOME/bin/createStone
>>>> with the `-t` option (which you should be), then newExtent ends up running
>>>> the `rebuildServerTode` script:
>>>>
>>>> $GS_HOME/bin/todeIt $stoneName "script --script=rebuildServerTode"
>>>>
>>>> which itself runs $GS_HOME/sys/local/client/tode-scripts/rebuildSys ...
>>>> whose job is to do things like make sure that the freshly copied extent is
>>>> adapted correctly to it's new environment so I think that adding this line
>>>> to the script would be correct:
>>>>
>>>> eval `MCGitHubRepository cacheDirectory: nil`
>>>>
>>>>
>>> Thanks. I just tried it but it seems it is not yet enough. Look at the
>>> exception I get:
>>>
>>> Inspect
>>> MetacelloEnsureFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
>>> @31 line 39a GoferRepositoryError occurred (error 2710), a
>>> ImproperOperation occurred (error 2085), Expected nil to be a Boolean./
>>> --------------------
>>> . -> a GoferRepositoryError occurred (error 2710), a
>>> ImproperOperation occurred (error 2085), Expected nil to be a Boolean.
>>> .. -> aMetacelloEnsureFetchingMCSpecLoader(explicit load :
>>> 1.6 [ConfigurationOfIAM])
>>> (class)@ -> GoferRepositoryError
>>> (oop)@ -> 1767077889
>>> currGsHandler@ -> nil
>>> gsArgs@ -> nil
>>> gsDetails@ -> 'a ImproperOperation occurred (error 2085), Expected
>>> nil to be a Boolean.'
>>> gsNumber@ -> 2710
>>> gsReason@ -> nil
>>> gsResumable@ -> false
>>> gsStack@ -> nil
>>> gsTrappable@ -> true
>>> messageText@ -> 'a GoferRepositoryError occurred (error 2710), a
>>> ImproperOperation occurred (error 2085), Expected nil to be a Boolean.'
>>> repository@ ->
>>> aMCGitHubRepository(github://GsDevKit/Seaside31:v3.1.4.2-gs/repository)
>>> tag@ -> nil
>>>
>>>
>>> And when I inspect the 'repository' I get this:
>>>
>>> Inspect
>>> MetacelloEnsureFetchingMCSpecLoader(MetacelloCommonMCSpecLoader)>>retryingResolvePackageSpecReferences:gofer:
>>> @31 line 39a GoferRepositoryError occurred (error 2710), a
>>> ImproperOperation occurred (error 2085), Expected nil to be a
>>> Boolean./aMCGitHubRepository(github://GsDevKit/Seaside31:v3.1.4.2-gs/repository)/
>>> --------------------
>>> . ->
>>> aMCGitHubRepository(github://GsDevKit/Seaside31:v3.1.4.2-gs/repository)
>>> .. -> a GoferRepositoryError occurred (error 2710),
>>> a ImproperOperation occurred (error 2085), Expected nil to be a Boolean.
>>> (class)@ -> MCGitHubRepository
>>> (oop)@ -> 1226322689
>>> (committed)@ -> true
>>> cache@ -> nil
>>> creationTemplate@ -> nil
>>> directory@ -> aServerFileDirectory
>>> projectPath@ -> 'GsDevKit/Seaside31'
>>> projectVersion@ -> 'v3.1.4.2-gs'
>>> projectVersionPattern@ -> '3.1.?'
>>> readonly@ -> true
>>> repoPath@ -> 'repository'
>>> repositoryProperties@ -> aDictionary( 'packageExtension'->'.package',
>>> 'propertyFileExtension'->'.json')
>>> storeDiffs@ -> nil
>>>
>>>
>>> And as you can see the problematic is that 'directory' instVar. if I
>>> send #pathName to it, I get:
>>>
>>>
>>> '/opt/gemstoneAdditions/GsDevKit_home/server/stones/gs_329/logs/github-cache/GsDevKit/Seaside31/v3.1.4.2-gs/GsDevKit-Seaside31-55f1bac/repository'
>>>
>>> I tried sending #flushCache to the repository instance, but not
>>> luck..the 'directory' instVar still there...
>>>
>>> Note that #flushCache of github class DOES nil the 'directory' instVar.
>>> However, this is funny:
>>>
>>> MCGitHubRepository cacheDirectory: nil.
>>> MCRepositoryGroup default repositoriesDo: [:rep | rep flushCache ].
>>> MCRepositoryGroup default repositoriesDo: [:each | (each projectVersion
>>> = 'v3.1.4.1-gs') ifTrue: [Halt halt] ].
>>>
>>> And then it halts... 'each' continues to have again the wrong value in
>>> 'directory'. Shouldn't this be nil? Of course I made sure the
>>> #flushCache was really sent to the repository.
>>>
>>> I don't get it.
>>>
>>>
>>>
>>>> In fact, if you don't get there first, I'll submit a Metacello bug
>>>> report, and then add this patch (along with a reference to the Metacello
>>>> bug) to rebuildSys. I'm not planning a drop for a day or so, so if you want
>>>> me to do the patch sooner, I can slipstream the patch into master and then
>>>> create the bug and comment the patch with my next regular Early Access drop
>>>> ...
>>>>
>>>> I might be inclined to think that patch actually belongs there ... when
>>>> you move an extent ... all of the disk caches should be cleared ....
>>>>
>>>> Dale
>>>>
>>>> _______________________________________________
>>>> Glass mailing list
>>>> Glass at lists.gemtalksystems.com
>>>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>>>
>>>
>>>
>>>
>>> --
>>> Mariano
>>> http://marianopeck.wordpress.com
>>>
>>>
>>>
>>
>>
>> --
>> Mariano
>> http://marianopeck.wordpress.com
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
>
>
>
--
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20151027/99375584/attachment-0001.html>
More information about the Glass
mailing list