From patchwork Wed Mar 9 00:24:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kamal Mostafa X-Patchwork-Id: 594517 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 7684214031D; Wed, 9 Mar 2016 11:26:40 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.76) (envelope-from ) id 1adRxV-00043Z-9m; Wed, 09 Mar 2016 00:26:37 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1adRvT-0002qD-F5 for kernel-team@lists.ubuntu.com; Wed, 09 Mar 2016 00:24:31 +0000 Received: from 1.general.kamal.us.vpn ([10.172.68.52] helo=fourier) by youngberry.canonical.com with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.76) (envelope-from ) id 1adRvS-0002pd-Vr; Wed, 09 Mar 2016 00:24:31 +0000 Received: from kamal by fourier with local (Exim 4.86) (envelope-from ) id 1adRvQ-0006K7-AN; Tue, 08 Mar 2016 16:24:28 -0800 From: Kamal Mostafa To: Tejun Heo Subject: [3.19.y-ckt stable] Patch "workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup" has been added to the 3.19.y-ckt tree Date: Tue, 8 Mar 2016 16:24:27 -0800 Message-Id: <1457483067-24272-1-git-send-email-kamal@canonical.com> X-Mailer: git-send-email 2.7.0 X-Extended-Stable: 3.19 Cc: Len Brown , "Rafael J . Wysocki" , Kamal Mostafa , Tang Chen , kernel-team@lists.ubuntu.com, Mike Galbraith X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.14 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: kernel-team-bounces@lists.ubuntu.com This is a note to let you know that I have just added a patch titled workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup to the linux-3.19.y-queue branch of the 3.19.y-ckt extended stable tree which can be found at: http://kernel.ubuntu.com/git/ubuntu/linux.git/log/?h=linux-3.19.y-queue This patch is scheduled to be released in version 3.19.8-ckt16. If you, or anyone else, feels it should not be added to this tree, please reply to this email. For more information about the 3.19.y-ckt tree, see https://wiki.ubuntu.com/Kernel/Dev/ExtendedStable Thanks. -Kamal ---8<------------------------------------------------------------ From 5d1fd405bbbc37ffa12d04a6d38b88ffa585de46 Mon Sep 17 00:00:00 2001 From: Tejun Heo Date: Wed, 3 Feb 2016 13:54:25 -0500 Subject: workqueue: handle NUMA_NO_NODE for unbound pool_workqueue lookup commit d6e022f1d207a161cd88e08ef0371554680ffc46 upstream. When looking up the pool_workqueue to use for an unbound workqueue, workqueue assumes that the target CPU is always bound to a valid NUMA node. However, currently, when a CPU goes offline, the mapping is destroyed and cpu_to_node() returns NUMA_NO_NODE. This has always been broken but hasn't triggered often enough before 874bbfe600a6 ("workqueue: make sure delayed work run in local cpu"). After the commit, workqueue forcifully assigns the local CPU for delayed work items without explicit target CPU to fix a different issue. This widens the window where CPU can go offline while a delayed work item is pending causing delayed work items dispatched with target CPU set to an already offlined CPU. The resulting NUMA_NO_NODE mapping makes workqueue try to queue the work item on a NULL pool_workqueue and thus crash. While 874bbfe600a6 has been reverted for a different reason making the bug less visible again, it can still happen. Fix it by mapping NUMA_NO_NODE to the default pool_workqueue from unbound_pwq_by_node(). This is a temporary workaround. The long term solution is keeping CPU -> NODE mapping stable across CPU off/online cycles which is being worked on. Signed-off-by: Tejun Heo Reported-by: Mike Galbraith Cc: Tang Chen Cc: Rafael J. Wysocki Cc: Len Brown Link: http://lkml.kernel.org/g/1454424264.11183.46.camel@gmail.com Link: http://lkml.kernel.org/g/1453702100-2597-1-git-send-email-tangchen@cn.fujitsu.com [ luis: backported to 3.16: adjusted context ] Signed-off-by: Luis Henriques Signed-off-by: Kamal Mostafa --- kernel/workqueue.c | 10 ++++++++++ 1 file changed, 10 insertions(+) -- 2.7.0 diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 82d0c8d..1cf1b72 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -552,6 +552,16 @@ static struct pool_workqueue *unbound_pwq_by_node(struct workqueue_struct *wq, int node) { assert_rcu_or_wq_mutex(wq); + + /* + * XXX: @node can be NUMA_NO_NODE if CPU goes offline while a + * delayed item is pending. The plan is to keep CPU -> NODE + * mapping valid and stable across CPU on/offlines. Once that + * happens, this workaround can be removed. + */ + if (unlikely(node == NUMA_NO_NODE)) + return wq->dfl_pwq; + return rcu_dereference_raw(wq->numa_pwq_tbl[node]); }