From patchwork Fri Dec 18 06:16:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 558750 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id EE76214029E for ; Fri, 18 Dec 2015 17:21:43 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id D94D71A1A51 for ; Fri, 18 Dec 2015 17:21:43 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from ozlabs.org (ozlabs.org [IPv6:2401:3900:2:1::2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 613461A1A1E for ; Fri, 18 Dec 2015 17:20:48 +1100 (AEDT) Received: by ozlabs.org (Postfix) id 2CCB51402A9; Fri, 18 Dec 2015 17:20:48 +1100 (AEDT) Delivered-To: linuxppc-dev@ozlabs.org Received: from authenticated.ozlabs.org (localhost [127.0.0.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPSA id 12EB514029E; Fri, 18 Dec 2015 17:20:48 +1100 (AEDT) From: Alistair Popple To: linuxppc-dev@ozlabs.org, mpe@ellerman.id.au Subject: [PATCH] powerpc/opal-irqchip: Fix deadlock introduced by "Fix double endian conversion" Date: Fri, 18 Dec 2015 17:16:17 +1100 Message-Id: <1450419377-29847-1-git-send-email-alistair@popple.id.au> X-Mailer: git-send-email 2.1.4 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alistair Popple , stable@vger.kernel.org, Ian Munsie , Andrew Donnellan , Daniel Axtens MIME-Version: 1.0 Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Commit 25642e1459ac ("powerpc/opal-irqchip: Fix double endian conversion") fixed an endian bug by calling opal_handle_events() in opal_event_unmask(). However this introduces a deadlock when an event is active during unmasking as opal_handle_events() calls generic_handle_irq() which may call opal_event_unmask() with the irq descriptor lock held. When generating multiple opal events in quick succession this would lead to the following stall warnings: EEH: Fenced PHB#0 detected, location: U78C9.001.WZS09XA-P1-C32 INFO: rcu_sched detected stalls on CPUs/tasks: 12-...: (1 GPs behind) idle=68f/140000000000001/0 softirq=860/861 fqs=2065 15-...: (1 GPs behind) idle=be5/140000000000001/0 softirq=1142/1143 fqs=2065 (detected by 13, t=2102 jiffies, g=1325, c=1324, q=602) NMI watchdog: BUG: soft lockup - CPU#18 stuck for 22s! [irqbalance:2696] INFO: rcu_sched detected stalls on CPUs/tasks: 12-...: (1 GPs behind) idle=68f/140000000000001/0 softirq=860/861 fqs=8371 15-...: (1 GPs behind) idle=be5/140000000000001/0 softirq=1142/1143 fqs=8371 (detected by 20, t=8407 jiffies, g=1325, c=1324, q=1290) This patch corrects the problem by queuing the work if an event is active during unmasking, which is similar to the pre-endian fix behaviour. Signed-off-by: Alistair Popple Reported-by: Andrew Donnellan Cc: stable@vger.kernel.org --- Michael, I'm quite confident this fixes the problem as it is just reverting to the previous behaviour only with the endian corrected, which was really the correct fix in the first place. Thanks. arch/powerpc/platforms/powernv/opal-irqchip.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) -- 2.1.4 diff --git a/arch/powerpc/platforms/powernv/opal-irqchip.c b/arch/powerpc/platforms/powernv/opal-irqchip.c index 0a00e2a..2f12cb9 100644 --- a/arch/powerpc/platforms/powernv/opal-irqchip.c +++ b/arch/powerpc/platforms/powernv/opal-irqchip.c @@ -83,7 +83,19 @@ static void opal_event_unmask(struct irq_data *d) set_bit(d->hwirq, &opal_event_irqchip.mask); opal_poll_events(&events); - opal_handle_events(be64_to_cpu(events)); + last_outstanding_events = be64_to_cpu(events); + + /* + * We can't just handle the events now with + * opal_handle_events() as opal_event_unmask() gets called + * from generic_handle_irq() which holds the irq descriptor + * lock leading to a deadlock if generic_handle_irq() gets + * called again from opal_handle_events(). Instead queue the + * events for later. + */ + if (last_outstanding_events & opal_event_irqchip.mask) + /* Need to retrigger the interrupt */ + irq_work_queue(&opal_event_irq_work); } static int opal_event_set_type(struct irq_data *d, unsigned int flow_type)