[Openais] [Corosync] The corosync shared memory keeps increasing

Andrew Beekhof andrew at beekhof.net
Thu Aug 5 02:42:09 PDT 2010


On Wed, Aug 4, 2010 at 8:24 PM, Steven Dake <sdake at redhat.com> wrote:
> On 08/03/2010 10:02 AM, hj lee wrote:
>> Hi,
>>
>> I tried the latest version corosync 1.2.7 rpms from clusterlabs. The
>> problem is still there. Actually the latest version gets worse. In old
>> 1.1.2 version, the shared memory increases only when cib exachanges
>> messages through corosync. In the 1.2.7 version, corosync shared memory
>> keeps increasing even when the cluster is idle. Simply run the corosync
>> and watch the memory in the top every a few minutes, the shared memory
>> just keeps increasing. Isn't this a memory leak?
>>
>> 1. Shared memory increases when the cluster is idle. I highly suspect it
>> leak comes from circular mmap in logsys.c.
>> [root at silverthorne4 epel]# top -b -n1 | egrep "coro|cib|attrd"
>> 16579 root      RT   0  207m 4200 1920 S  0.0  0.1   0:00.07 corosync
>> 16587 hacluste  -8   0 69044 4536 2544 S  0.0  0.1   0:00.35 cib
>> 16589 hacluste  -8   0 69808 2436 2024 S  0.0  0.1   0:00.00 attrd
>> ---- after a few minutes later
>> [root at silverthorne4 epel]# top -b -n1 | egrep "coro|cib|attrd"
>> 16579 root      RT   0  207m 4212 1932 S  0.0  0.1   0:00.07 corosync
>> 16587 hacluste  -8   0 69044 4536 2544 S  0.0  0.1   0:00.35 cib
>> 16589 hacluste  -8   0 69808 2436 2024 S  0.0  0.1   0:00.00 attrd
>>
>>
>> 2. Shared memory increases whenever pacemaker resource is started or
>> stopped. In this case, cib's shared memory also increases. I highly
>> suspect this leak comes from mmap in corosync ipc code.
>> [root at silverthorne4 epel]# top -b -n1 | egrep "coro|cib|attrd"
>> 16579 root      RT   0  207m 4316 2036 S  0.0  0.1   0:00.11 corosync
>> 16587 hacluste  -8   0 69048 4596 2584 S  0.0  0.1   0:00.39 cib
>> 16589 hacluste  -8   0 69808 2436 2024 S  0.0  0.1   0:00.00 attrd
>>
>> [root at silverthorne4 epel]# crm resource stop faultymon-clone
>>
>> [root at silverthorne4 epel]# top -b -n1 | egrep "coro|cib|attrd"
>> 16579 root      RT   0  207m 4336 2056 S  0.0  0.1   0:00.13 corosync
>> 16587 hacluste  -8   0 69068 4620 2596 S  0.0  0.1   0:00.41 cib
>> 16589 hacluste  -8   0 69808 2436 2024 S  0.0  0.1   0:00.00 attrd
>>
>> Thanks
>> hj
>>
>
> corosync doesn't leak without pacemaker.
>
> The major difference in pacemaker is the use of fork.  Angus, one thing
> to try is madvise (DONTFORK) on the mmap sections.

perhaps try the latest 1.1 code which does the forking outside of corosync:
   http://theclusterguy.clusterlabs.org/post/907043024/introducing-the-pacemaker-master-control-process-for


More information about the Openais mailing list