From patchwork Tue Feb 18 23:34:05 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nishanth Aravamudan X-Patchwork-Id: 321708 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id C406A2C03C7 for ; Wed, 19 Feb 2014 10:35:02 +1100 (EST) Received: from e8.ny.us.ibm.com (e8.ny.us.ibm.com [32.97.182.138]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id A6CC92C00D5 for ; Wed, 19 Feb 2014 10:34:15 +1100 (EST) Received: from /spool/local by e8.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 18 Feb 2014 18:34:13 -0500 Received: from d01dlp02.pok.ibm.com (9.56.250.167) by e8.ny.us.ibm.com (192.168.1.108) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 18 Feb 2014 18:34:10 -0500 Received: from b01cxnp22036.gho.pok.ibm.com (b01cxnp22036.gho.pok.ibm.com [9.57.198.26]) by d01dlp02.pok.ibm.com (Postfix) with ESMTP id 852FD6E803C for ; Tue, 18 Feb 2014 18:34:04 -0500 (EST) Received: from d01av01.pok.ibm.com (d01av01.pok.ibm.com [9.56.224.215]) by b01cxnp22036.gho.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id s1INY9e68651236 for ; Tue, 18 Feb 2014 23:34:09 GMT Received: from d01av01.pok.ibm.com (localhost [127.0.0.1]) by d01av01.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id s1INY9JO022543 for ; Tue, 18 Feb 2014 18:34:09 -0500 Received: from qbert.localdomain ([9.80.101.126]) by d01av01.pok.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id s1INY8NC022451; Tue, 18 Feb 2014 18:34:08 -0500 Received: by qbert.localdomain (Postfix, from userid 1000) id 1BD93480CAC; Tue, 18 Feb 2014 15:34:05 -0800 (PST) Date: Tue, 18 Feb 2014 15:34:05 -0800 From: Nishanth Aravamudan To: Michal Hocko Subject: Re: ppc: RECLAIM_DISTANCE 10? Message-ID: <20140218233404.GB10844@linux.vnet.ibm.com> References: <20140218090658.GA28130@dhcp22.suse.cz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20140218090658.GA28130@dhcp22.suse.cz> X-Operating-System: Linux 3.11.0-15-generic (x86_64) User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 14021823-0320-0000-0000-0000027E2D76 Cc: linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, Anton Blanchard , LKML X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Hi Michal, On 18.02.2014 [10:06:58 +0100], Michal Hocko wrote: > Hi, > I have just noticed that ppc has RECLAIM_DISTANCE reduced to 10 set by > 56608209d34b (powerpc/numa: Set a smaller value for RECLAIM_DISTANCE to > enable zone reclaim). The commit message suggests that the zone reclaim > is desirable for all NUMA configurations. > > History has shown that the zone reclaim is more often harmful than > helpful and leads to performance problems. The default RECLAIM_DISTANCE > for generic case has been increased from 20 to 30 around 3.0 > (32e45ff43eaf mm: increase RECLAIM_DISTANCE to 30). Interesting. > I strongly suspect that the patch is incorrect and it should be > reverted. Before I will send a revert I would like to understand what > led to the patch in the first place. I do not see why would PPC use only > LOCAL_DISTANCE and REMOTE_DISTANCE distances and in fact machines I have > seen use different values. > > Anton, could you comment please? I'll let Anton comment here, but in looking into this issue in working on CONFIG_HAVE_MEMORYLESS_NODE support, I realized that any LPAR with memoryless nodes will set zone_reclaim_mode to 1. I think we want to ignore memoryless nodes when we set up the reclaim mode like the following? I'll send it as a proper patch if you agree? Note, this won't actually do anything if CONFIG_HAVE_MEMORYLESS_NODES is not set, but if it is, I think semantically it will indicate that memoryless nodes *have* to reclaim remotely. And actually the above won't work, because the callpath is start_kernel -> setup_arch -> paging_init [-> free_area_init_nodes -> free_area_init_node -> init_zone_allows_reclaim] which is called before build_all_zonelists. This is a similar ordering problem as I'm having with the MEMORYLESS_NODE support, will work on it. Thanks, Nish diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5de4337..4f6ff6f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1853,8 +1853,9 @@ static void __paginginit init_zone_allows_reclaim(int nid) { int i; - for_each_online_node(i) - if (node_distance(nid, i) <= RECLAIM_DISTANCE) + for_each_online_node(i) { + if (node_distance(nid, i) <= RECLAIM_DISTANCE || + local_memory_node(nid) != nid) node_set(i, NODE_DATA(nid)->reclaim_nodes); else zone_reclaim_mode = 1;