[PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock

[Greg Thelen]

On Wed, Apr 14, 2010 at 2:29 AM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu at jp.fujitsu.com> wrote:
> On Tue, 13 Apr 2010 23:55:12 -0700
> Greg Thelen <gthelen at google.com> wrote:
>> On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki <kamezawa.hiroyu at jp.fujitsu.com> wrote:
>> > On Fri, 19 Mar 2010 08:10:39 +0530
>> > Balbir Singh <balbir at linux.vnet.ibm.com> wrote:
>> >> * KAMEZAWA Hiroyuki <kamezawa.hiroyu at jp.fujitsu.com> [2010-03-19 10:23:32]:
>> >>
>> >> > On Thu, 18 Mar 2010 21:58:55 +0530
>> >> > Balbir Singh <balbir at linux.vnet.ibm.com> wrote:
>> >> >
>> >> > > * KAMEZAWA Hiroyuki <kamezawa.hiroyu at jp.fujitsu.com> [2010-03-18 13:35:27]:
>> >> >
>> >> > > > Then, no probelm. It's ok to add mem_cgroup_udpate_stat() indpendent from
>> >> > > > mem_cgroup_update_file_mapped(). The look may be messy but it's not your
>> >> > > > fault. But please write "why add new function" to patch description.
>> >> > > >
>> >> > > > I'm sorry for wasting your time.
>> >> > >
>> >> > > Do we need to go down this route? We could check the stat and do the
>> >> > > correct thing. In case of FILE_MAPPED, always grab page_cgroup_lock
>> >> > > and for others potentially look at trylock. It is OK for different
>> >> > > stats to be protected via different locks.
>> >> > >
>> >> >
>> >> > I _don't_ want to see a mixture of spinlock and trylock in a function.
>> >> >
>> >>
>> >> A well documented well written function can help. The other thing is to
>> >> of-course solve this correctly by introducing different locking around
>> >> the statistics. Are you suggesting the later?
>> >>
>> >
>> > No. As I wrote.
>> >        - don't modify codes around FILE_MAPPED in this series.
>> >        - add a new functions for new statistics
>> > Then,
>> >        - think about clean up later, after we confirm all things work as expected.
>>
>> I have ported Andrea Righi's memcg dirty page accounting patches to latest
>> mmtom-2010-04-05-16-09.  In doing so I have to address this locking issue.  Does
>> the following look good?  I will (of course) submit the entire patch for review,
>> but I wanted make sure I was aiming in the right direction.
>>
>> void mem_cgroup_update_page_stat(struct page *page,
>>                       enum mem_cgroup_write_page_stat_item idx, bool charge)
>> {
>>       static int seq;
>>       struct page_cgroup *pc;
>>
>>       if (mem_cgroup_disabled())
>>               return;
>>       pc = lookup_page_cgroup(page);
>>       if (!pc || mem_cgroup_is_root(pc->mem_cgroup))
>>               return;
>>
>>       /*
>>        * This routine does not disable irq when updating stats.  So it is
>>        * possible that a stat update from within interrupt routine, could
>>        * deadlock.  Use trylock_page_cgroup() to avoid such deadlock.  This
>>        * makes the memcg counters fuzzy.  More complicated, or lower
>>        * performing locking solutions avoid this fuzziness, but are not
>>        * currently needed.
>>        */
>>       if (irqs_disabled()) {
>>               if (! trylock_page_cgroup(pc))
>>                       return;
>>       } else
>>               lock_page_cgroup(pc);
>>
>
> I prefer trylock_page_cgroup() always.

What is your reason for preferring trylock_page_cgroup()?  I assume
it's for code simplicity, but I wanted to check.

I had though about using trylock_page_cgroup() always, but I think
that would make file_mapped accounting even more fuzzy that it already
it is.  I was trying to retain the current accuracy of file_mapped and
only make new counters, like writeback/dirty/etc (those obtained in
interrupt), fuzzy.

> I have another idea fixing this up _later_. (But I want to start from simple one.)
>
> My rough idea is following.  Similar to your idea which you gave me before.

Hi Kame-san,

I like the general approach.  The code I previously gave you appears
to work and is faster than non-root memcgs using mmotm due to mostly
being lockless.

> ==
> DEFINE_PERCPU(account_move_ongoing);

What's the reason for having a per-cpu account_move_ongoing flag?
Would a single system-wide global be sufficient?  I assume the
majority of the time this value will not be changing because
accounting moves are rare.

Perhaps all of the per-cpu variables are packed within a per-cpu
cacheline making accessing it more likely to be local, but I'm not
sure if this is true.

--
Greg