From patchwork Thu Mar 23 14:51:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 742690 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3vpqJB2R4rz9s81 for ; Fri, 24 Mar 2017 01:51:37 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754187AbdCWOvh (ORCPT ); Thu, 23 Mar 2017 10:51:37 -0400 Received: from outbound-smtp04.blacknight.com ([81.17.249.35]:38477 "EHLO outbound-smtp04.blacknight.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751249AbdCWOvg (ORCPT ); Thu, 23 Mar 2017 10:51:36 -0400 Received: from mail.blacknight.com (pemlinmail04.blacknight.ie [81.17.254.17]) by outbound-smtp04.blacknight.com (Postfix) with ESMTPS id 4BC54987E8 for ; Thu, 23 Mar 2017 14:51:34 +0000 (UTC) Received: (qmail 1909 invoked from network); 23 Mar 2017 14:51:34 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[37.228.230.149]) by 81.17.254.9 with ESMTPSA (DHE-RSA-AES256-SHA encrypted, authenticated); 23 Mar 2017 14:51:34 -0000 Date: Thu, 23 Mar 2017 14:51:33 +0000 From: Mel Gorman To: Jesper Dangaard Brouer Cc: Tariq Toukan , "netdev@vger.kernel.org" , akpm@linux-foundation.org, linux-mm , Saeed Mahameed Subject: Re: Page allocator order-0 optimizations merged Message-ID: <20170323145133.twzt4f5ci26vdyut@techsingularity.net> References: <58b48b1f.F/jo2/WiSxvvGm/z%akpm@linux-foundation.org> <20170301144845.783f8cad@redhat.com> <83a0e3ef-acfa-a2af-2770-b9a92bda41bb@mellanox.com> <20170322234004.kffsce4owewgpqnm@techsingularity.net> <20170323144347.1e6f29de@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170323144347.1e6f29de@redhat.com> User-Agent: Mutt/1.6.2 (2016-07-01) Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Thu, Mar 23, 2017 at 02:43:47PM +0100, Jesper Dangaard Brouer wrote: > On Wed, 22 Mar 2017 23:40:04 +0000 > Mel Gorman wrote: > > > On Wed, Mar 22, 2017 at 07:39:17PM +0200, Tariq Toukan wrote: > > > > > > This modification may slow allocations from IRQ context slightly > > > > > > but the > > > > > > main gain from the per-cpu allocator is that it scales better for > > > > > > allocations from multiple contexts. There is an implicit > > > > > > assumption that > > > > > > intensive allocations from IRQ contexts on multiple CPUs from a single > > > > > > NUMA node are rare > > > Hi Mel, Jesper, and all. > > > > > > This assumption contradicts regular multi-stream traffic that is naturally > > > handled > > > over close numa cores. I compared iperf TCP multistream (8 streams) > > > over CX4 (mlx5 driver) with kernels v4.10 (before this series) vs > > > kernel v4.11-rc1 (with this series). > > > I disabled the page-cache (recycle) mechanism to stress the page allocator, > > > and see a drastic degradation in BW, from 47.5 G in v4.10 to 31.4 G in > > > v4.11-rc1 (34% drop). > > > I noticed queued_spin_lock_slowpath occupies 62.87% of CPU time. > > > > Can you get the stack trace for the spin lock slowpath to confirm it's > > from IRQ context? > > AFAIK allocations happen in softirq. Argh and during review I missed > that in_interrupt() also covers softirq. To Mel, can we use a in_irq() > check instead? > > (p.s. just landed and got home) Not built or even boot tested. I'm unable to run tests at the moment diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6cbde310abed..f82225725bc1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2481,7 +2481,7 @@ void free_hot_cold_page(struct page *page, bool cold) unsigned long pfn = page_to_pfn(page); int migratetype; - if (in_interrupt()) { + if (in_irq()) { __free_pages_ok(page, 0); return; } @@ -2647,7 +2647,7 @@ static struct page *__rmqueue_pcplist(struct zone *zone, int migratetype, { struct page *page; - VM_BUG_ON(in_interrupt()); + VM_BUG_ON(in_irq()); do { if (list_empty(list)) { @@ -2704,7 +2704,7 @@ struct page *rmqueue(struct zone *preferred_zone, unsigned long flags; struct page *page; - if (likely(order == 0) && !in_interrupt()) { + if (likely(order == 0) && !in_irq()) { page = rmqueue_pcplist(preferred_zone, zone, order, gfp_flags, migratetype); goto out;