[Glass] GC on Gems only fired when temp space is over?
Mariano Martinez Peck via Glass
glass at lists.gemtalksystems.com
Tue Jun 23 06:51:47 PDT 2015
Thank you SO MUCH Dale! So this is what I did for all seaside gems in
case someone else is interested:
"This thread is needed to handle the SigAbort exception, when the primary
thread is blocked on an accept. Assuming default 60 second
STN_GEM_ABORT_TIMEOUT, wake up at 30 second intervals."
[
| count minutesToForceGemGC |
count := 0.
minutesToForceGemGC := 30.
[ true ] whileTrue: [
(Delay forSeconds: 30) wait.
count := count + 1.
(count \\\ (minutesToForceGemGC * 2)) = 0 ifTrue: [
System _generationScavenge_vmMarkSweep.
Transcript show: 'Gem GC forced'; cr.
count := 0.
].
].
] forkAt: Processor lowestPriority.
And yeah, it seems to work :)
And yeah, the GC seems to be super fast :) Good job!
*Also.... similar to this great idea, do you have something else in mind I
can do to minimize memory used when system is mostly "idle" ?*
I ask because I did not noticed much of a change if I shutdown all seaside
gems and start them again. However.....if I shut down all stones and start
them again (with all seaside gems again), then the memory used decreases A
LOT. While this may be obvious, it is not fully clear to me why. As a first
thought I would guess the main difference is the SPC. However, SPC is a
segment/shared and that memory is reserved from the very beginning. So I
don't expect that to change wether I reset the stone or not.
So...if it is not the SPC.... they are not the seaside gems, they are not
the admin/reclaim gc (those use very little temp space), what else could be
such a difference if memory consumption when I reset the stone?
Thanks in advance,
On Mon, Jun 22, 2015 at 8:49 PM, Dale Henrichs <
dale.henrichs at gemtalksystems.com> wrote:
>
>
> On 06/22/2015 01:22 PM, Mariano Martinez Peck wrote:
>
>
>
> On Mon, Jun 22, 2015 at 3:09 PM, Dale Henrichs <
> dale.henrichs at gemtalksystems.com> wrote:
>
>>
>>
>> On 06/22/2015 10:36 AM, Mariano Martinez Peck wrote:
>>
>>
>>
>> On Mon, Jun 22, 2015 at 2:16 PM, Dale Henrichs via Glass <
>> glass at lists.gemtalksystems.com> wrote:
>>
>>>
>>>
>>> On 06/19/2015 02:28 PM, Mariano Martinez Peck via Glass wrote:
>>>
>>
>>> You could schedule a periodic scavenge and mark sweep in your seaside
>>> vms ...
>>
>>
>> How can I do this if above you said *"scavenges and mark sweep are only
>> triggered under memory pressure" ?*
>>
>>
>>> Once an hour or so would make sure that your idle seaside gems are using
>>> the bare minimum of memory ... it may tak a mark sweep for the vm to return
>>> memory pages ... BTW, I assume that you are running on linux. On the mac
>>> the full memory footprint is allocated in memory ...
>>>
>>>
>> I would love to do that. Any idea how?
>>
>> These are the methods to use ... and as I mentioned in my last post, the
>> "sigabort process" would be a good place to this processing. Here are the
>> methods to manually trigger scavenges and mark sweeps ... the last one is
>> probably the one you want:
>> System class>>_vmMarkSweep
>> System class>>_generationScavenge
>> System class>>_generationScavenge_vmMarkSweep
>>
>>
> Dale,
>
> Thanks for such methods! Very useful. Now..I have a couple of questions
> :)
>
> In another email you said:
>
>
>
>
> *-------- We really try to make sure that idle gems stay idle and will
> only wake up when needed ... but there is at least one restless process
> that wakes up every 30 seconds to facilitate the handling of SigAborts ...
> the process itself doesn't touch much memory, but if a SigAbort is required
> then the gem will also do an abort and page in a bit more memory, but an
> abort is pretty inexpensive for a previously idle process ... BTW, this
> little "sigabort process" would be a spot where you could add the logic for
> running the periodic scavenge and mark sweep:)*
> --------
>
> The main question here is....are you that confident that
> #_generationScavenge_vmMarkSweep would be so fast that calling it every 30
> seconds won't impact much the system performance? I was thinking about
> calling it like...half an hour hahahaha.
>
> No I meant that you could put code into that process that ran the
> scavenge/marksweep once an hour or day or half hour or whatever ... every
> 30 seconds is probably too often ... the scavenge and marksweep _are_ very
> quick, but you only really need to run them often enough to keep idle gems
> from wasting swap space ... The real timing will be up to you as far as how
> long you think a gem can sit idle before it should drop any extra memory
> that it's using ...
>
>
> Anyway, if I watch the code:
>
> Transcript cr; show: 'New style SigAbort hanlder'.
> TransactionBacklog
> addDefaultHandler: [ :ex |
> "Run the abort in a lowPriority process, since we must acquire the
> transactionMutex."
> Transcript cr; show: 'handled sigabort: ', DateAndTime now printString.
> [
> GRPlatform current transactionMutex
> critical: [ GRPlatform current doAbortTransaction ].
> TransactionBacklog enableSignalling ]
> forkAt: Processor lowestPriority.
> ex resume ].
> RepositoryViewLost
> addDefaultHandler: [ :ex |
> GRPlatform current logError: ex description title: 'Lost OT'
> shouldCommit: false.
> ex pass ].
> TransactionBacklog enableSignalling.
>
> *it is not clear to me how can I hook. *
>
> That isn't the spot I was thinking of ... Here's the process that I
> was thinking could be piggy-backed:
>
> "This thread is needed to handle the SigAbort exception, when the primary
> thread is blocked. Assuming default 60 second STN_GEM_ABORT_TIMEOUT, wake
> up at 30 second intervals."
> [
> [ true ] whileTrue: [ (Delay forSeconds: 30) wait ].
> ] forkAt: Processor lowestPriority.
>
> The sigabort will only be triggered when smalltalk code is running ... if
> all of the processes in the vm are blocked on a socket read, the sigabort
> won't get through ... so this guy is there to make sure that the vm will
> wake up and check for a sigabort often enough ...
>
> Also..I was thinking to include it as part of seaside maintenance VM (a
> new task) to run every 30 minutes. But then the other question I had is
> that from the code/comment of #_generationScavenge_vmMarkSweep it is not
> clear to me if it is performed only in the current gem or in all gems. It
> seems to me in current gem. So..... I guess I cannot call this from
> maintenance VM, can I?
>
>
> The scavenge and sweep are only run in the current vm ... so each vm would
> have to have this code added ...
>
> Dale
>
--
Mariano
http://marianopeck.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gemtalksystems.com/mailman/private/glass/attachments/20150623/57f4c5b8/attachment.html>
More information about the Glass
mailing list