[PATCH v2 03/28] dcache: convert dentry_stat.nr_unused to per-cpu counters

Glauber Costa glommer at parallels.com
Mon Apr 8 13:18:40 UTC 2013


On 04/08/2013 01:14 PM, Glauber Costa wrote:
> On 04/05/2013 05:15 AM, Dave Chinner wrote:
>> On Thu, Apr 04, 2013 at 06:09:31PM -0700, Greg Thelen wrote:
>>> On Fri, Mar 29 2013, Glauber Costa wrote:
>>>
>>>> From: Dave Chinner <dchinner at redhat.com>
>>>>
>>>> Before we split up the dcache_lru_lock, the unused dentry counter
>>>> needs to be made independent of the global dcache_lru_lock. Convert
>>>> it to per-cpu counters to do this.
>>>>
>>>> Signed-off-by: Dave Chinner <dchinner at redhat.com>
>>>> Reviewed-by: Christoph Hellwig <hch at lst.de>
>>>> ---
>>>>  fs/dcache.c | 17 ++++++++++++++---
>>>>  1 file changed, 14 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/fs/dcache.c b/fs/dcache.c
>>>> index fbfae008..f1196f2 100644
>>>> --- a/fs/dcache.c
>>>> +++ b/fs/dcache.c
>>>> @@ -118,6 +118,7 @@ struct dentry_stat_t dentry_stat = {
>>>>  };
>>>>  
>>>>  static DEFINE_PER_CPU(unsigned int, nr_dentry);
>>>> +static DEFINE_PER_CPU(unsigned int, nr_dentry_unused);
>>>>  
>>>>  #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
>>>>  static int get_nr_dentry(void)
>>>> @@ -129,10 +130,20 @@ static int get_nr_dentry(void)
>>>>  	return sum < 0 ? 0 : sum;
>>>>  }
>>>>  
>>>> +static int get_nr_dentry_unused(void)
>>>> +{
>>>> +	int i;
>>>> +	int sum = 0;
>>>> +	for_each_possible_cpu(i)
>>>> +		sum += per_cpu(nr_dentry_unused, i);
>>>> +	return sum < 0 ? 0 : sum;
>>>> +}
>>>
>>> Just checking...  If cpu x is removed, then its per cpu nr_dentry_unused
>>> count survives so we don't leak nr_dentry_unused.  Right?  I see code in
>>> percpu_counter_sum_positive() to explicitly handle this case and I want
>>> to make sure we don't need it here.
>>
>> DEFINE_PER_CPU() gives a variable per possible CPU, and we sum for
>> all possible CPUs. Therefore online/offline CPUs just don't matter.
>>
>> The percpu_counter code uses for_each_online_cpu(), and so it has to
>> be aware of hotplug operations so taht it doesn't leak counts.
>>
> 
> It is an unsigned quantity, however. Can't we go negative if it becomes
> unused in one cpu, but used in another?
> 
> Ex:
> 
> nr_unused/0: 0
> nr_unused/1: 0
> 
> dentry goes to the LRU at cpu 1:
> nr_unused/0: 0
> nr_unused/1: 1
> 
> CPU 1 goes down:
> nr_unused/0: 0
> 
> dentry goes out of the LRU at cpu 0:
> nr_unused/0: 1 << 32.
> 
> That would easily be fixed by using a normal signed long, and is in fact
> what the percpu code does in its internal operations.
> 
> Any reason not to do it? Something I am not seeing?

Unless you have objections, I will fold the following patch into this one:


-------------- next part --------------
diff --git a/fs/dcache.c b/fs/dcache.c
index 8e166a4..c7cd9ee 100644
--- a/fs/dcache.c
+++ b/fs/dcache.c
@@ -118,7 +118,14 @@ struct dentry_stat_t dentry_stat = {
 };
 
 static DEFINE_PER_CPU(unsigned int, nr_dentry);
-static DEFINE_PER_CPU(unsigned int, nr_dentry_unused);
+/*
+ * The total counts for nr_dentry_unused are hotplug-safe, since we always loop
+ * through all possible cpus. It is quite possible, though, that the counters
+ * go negative.  That could easily happen for a dentry that is marked unused in
+ * one CPU but decrements that count after being preempted to another CPU.
+ * Therefore, we must use a signed quantity in here.
+ */
+static DEFINE_PER_CPU(long , nr_dentry_unused);
 
 #if defined(CONFIG_SYSCTL) && defined(CONFIG_PROC_FS)
 static int get_nr_dentry(void)


More information about the Containers mailing list