From patchwork Mon Sep 26 05:42:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682314 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=O632LASm; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWsF0ZJNz1yq7 for ; Mon, 26 Sep 2022 15:44:01 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWsF04SKz3bl0 for ; Mon, 26 Sep 2022 15:44:01 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=O632LASm; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::52c; helo=mail-pg1-x52c.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=O632LASm; dkim-atps=neutral Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrP0vSFz30NN for ; Mon, 26 Sep 2022 15:43:16 +1000 (AEST) Received: by mail-pg1-x52c.google.com with SMTP id bh13so5615492pgb.4 for ; Sun, 25 Sep 2022 22:43:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=fzntS4WX1i90+5q+5N/CxBpfx0CbxwrAKxxx53N9RCA=; b=O632LASm7es0XoQWWnlaV0jZXpOkevUkGYPI7jOzFnzIqJazBTo3/G3nB6GRX+8Tik gl5SAgQDRtLsnkSJugpOi6OMmYblbokbxEcWbvtwW73YX8pBsuvNvWvGgY+eQOSpfzej /5Tev82QT6TGGXVpgDrG1BFGkZ54pmH7auebBcoaeE0cdGKBZYiKPVjfVPISxv0TH3o6 VGzK/6TAZNtoAqoouG+31B5wZp0Qq/9+Aw6Hlng1YmVYC/zM7U9xZpBeHNfeyRe6FrDR KlEjnZTk324Fu9QvNFiKQxYU1pUCHJjKHz8spBz7tEylrznKtgY6IJwVLaBK2pgScIaI CdNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=fzntS4WX1i90+5q+5N/CxBpfx0CbxwrAKxxx53N9RCA=; b=4IX89uRIXFkEFPsbBpa4wRAJ0omN9ko+nPatEmhRqH+933eiab9OEGQsQEfkGhacg+ typpcMZzFIWDq2fj2Hp7ROZRe9bEQMceDSIqBCWbEKyjOxSyb6F2uAjyF2WSIbuXU9yr RsHsXKD4NvnrPZ0j+XlAGDEGN8KyjDfDPlvBIFbsluSaKfQDF46n88OalVvw8O5k7lU3 zBmrKX1gNGfVwUpXMr+BBRg5aHKifyNTLFGYzXO7wRTe9sHc7Dq+YxeyWsQXyDc2kq8O R0P4j2VdOypZi+O2S8dHMAplVF61f/8HYZTtGPpLHeOOd3R4IJ/ZnQxE+DqMxLdMpPJz mr2w== X-Gm-Message-State: ACrzQf1p9h+amaBFVkBlVqBOh+wcXaFU1WCgwZ7UDKw3StLrcsbgoqZq MyAQ50YFzGihy/Y5QJHGaELJ0yuUNyTrcA== X-Google-Smtp-Source: AMsMyM4LkAV0CPkzfamxhGm2WsOJB67oHCncfgp8dC/Y3XoesJ5AuWTnLujSB750+/s5Jg/gu8MJ3g== X-Received: by 2002:a05:6a00:22cb:b0:543:7003:21a1 with SMTP id f11-20020a056a0022cb00b00543700321a1mr21864402pfj.3.1664170994142; Sun, 25 Sep 2022 22:43:14 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:13 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 1/7] powerpc/64/interrupt: Fix false warning in context tracking due to idle state Date: Mon, 26 Sep 2022 15:42:59 +1000 Message-Id: <20220926054305.2671436-2-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Commit 171476775d32 ("context_tracking: Convert state to atomic_t") added a CONTEXT_IDLE state which can be encountered by interrupts from kernel mode in the idle thread, causing a false positive warning. Fixes: 171476775d32 ("context_tracking: Convert state to atomic_t") Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/interrupt.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/interrupt.h b/arch/powerpc/include/asm/interrupt.h index 8069dbc4b8d1..b61555e30c7c 100644 --- a/arch/powerpc/include/asm/interrupt.h +++ b/arch/powerpc/include/asm/interrupt.h @@ -195,7 +195,8 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs) * so avoid recursion. */ if (TRAP(regs) != INTERRUPT_PROGRAM) { - CT_WARN_ON(ct_state() != CONTEXT_KERNEL); + CT_WARN_ON(ct_state() != CONTEXT_KERNEL && + ct_state() != CONTEXT_IDLE); if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) BUG_ON(is_implicit_soft_masked(regs)); } From patchwork Mon Sep 26 05:43:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682315 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=CgxokaZV; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWsv6bmQz1yq7 for ; Mon, 26 Sep 2022 15:44:35 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWsv5jcpz3bmC for ; Mon, 26 Sep 2022 15:44:35 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=CgxokaZV; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=CgxokaZV; dkim-atps=neutral Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrR1Mwfz3bgC for ; Mon, 26 Sep 2022 15:43:18 +1000 (AEST) Received: by mail-pl1-x632.google.com with SMTP id b21so5235573plz.7 for ; Sun, 25 Sep 2022 22:43:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=LPFHi8omvCILxLZbqdRE+ytSu8T6h3A1h/9IS2BZUI4=; b=CgxokaZVKnURLKd6aDKekFl2kBvKdeGJSDkuPkJp0Qy8HSnZf9mEm4hOlDHnnMYuKd sZjmFiKL0toFgyFGIuzBS6GnZZFC7D9g3zJfiG+IZQMocdY4aUcwm6tVzweygcUBvnee /c/9ukhD2GF++ZevRY9tME8TmEXdUO18MS1Kxce5bS6IlEW4zObunQouITLmV1i+SlX6 T8hDyemF3Ex29AjYe4VclVq0NEP4n3FJ3pg13hB0LiMtwOuPKrZi3uaAj9AGgPDupmuB NqTYHJ1gBzF6yuqbxARNxVBjCWEsJQEp5Lqf1YHviP6bPSxjI8iV3sqcc55xc3LBsE4B 7gjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=LPFHi8omvCILxLZbqdRE+ytSu8T6h3A1h/9IS2BZUI4=; b=AM8R78NY+tnaxXFsrqhF7gCwZqTCK42QDv51efgJxlT27V/G6MgoWjKTk/c8rWQPwt MxhUWAcK/hTTmB3sfwv9yNqaQJyeK4RXPisd876PMtrz7yv3XgfGVMN7zSF5F4oQm1Pw Hj3FS4WRWPeEgMXT5jvNEzIO4d6/7hTLV+yvnEE/Y5hE87mM8WUVv5x03QW9VC0Vt2gA eO0KV1fxnkpIAVF1snyEVS3YbXe7gyK+D3Iz+d/ggaAhfCPAk2HFKSLUFzI9IlF7YHCJ KhT8VRCZcMIyvs3v8aGsX5EUjUTY7yWkGi01O1VCYTNXkYFFhtMGkhQeorPxs1NNhqvB H0zQ== X-Gm-Message-State: ACrzQf0DCKzRFelsmyCHji7cxKPcLSkj2d0qESxVguWMVmyU9MRvEiVp 7WizyHoLuwbbGAWVu234RMpR1j1JCpzKvg== X-Google-Smtp-Source: AMsMyM6YaFnqsc6HPDA3VKX1x+iyFi5KuhLpnmHqbwDbFGDCXpaYFUDjpv8GT2pE6NrMg60RtPIkeA== X-Received: by 2002:a17:90b:4f8d:b0:203:747c:7b7e with SMTP id qe13-20020a17090b4f8d00b00203747c7b7emr22661812pjb.98.1664170996377; Sun, 25 Sep 2022 22:43:16 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:15 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 2/7] powerpc/64: mark irqs hard disabled in boot paca Date: Mon, 26 Sep 2022 15:43:00 +1000 Message-Id: <20220926054305.2671436-3-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" This prevents interrupts in early boot (e.g., program check) from enabling MSR[EE], potentially causing endian mismatch or other crashes when reporting early boot traps. Fixes: 4423eb5ae32ec ("powerpc/64/interrupt: make normal synchronous interrupts enable MSR[EE] if possible") Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/setup_64.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c index 2b2d0b0fbb30..ce8fc6575eaa 100644 --- a/arch/powerpc/kernel/setup_64.c +++ b/arch/powerpc/kernel/setup_64.c @@ -182,8 +182,10 @@ static void __init fixup_boot_paca(void) get_paca()->cpu_start = 1; /* Allow percpu accesses to work until we setup percpu data */ get_paca()->data_offset = 0; - /* Mark interrupts disabled in PACA */ + /* Mark interrupts soft and hard disabled in PACA */ irq_soft_mask_set(IRQS_DISABLED); + get_paca()->irq_happened = PACA_IRQ_HARD_DIS; + WARN_ON(mfmsr() & MSR_EE); } static void __init configure_exceptions(void) From patchwork Mon Sep 26 05:43:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682316 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=i9rROWA7; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWtZ08gYz1ypH for ; Mon, 26 Sep 2022 15:45:10 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWtZ00ZSz3cFr for ; Mon, 26 Sep 2022 15:45:09 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=i9rROWA7; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::102f; helo=mail-pj1-x102f.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=i9rROWA7; dkim-atps=neutral Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrT51wMz3bmW for ; Mon, 26 Sep 2022 15:43:21 +1000 (AEST) Received: by mail-pj1-x102f.google.com with SMTP id g1-20020a17090a708100b00203c1c66ae3so5687721pjk.2 for ; Sun, 25 Sep 2022 22:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=3jB4Ii7xaomyhGENhnU8c09sqglMkNd9vUaTObmjEbw=; b=i9rROWA7MQSRV9lfc+HRWsTPufV6e+ohsffrhtXluiFCb1pqLQNbNAIOxnJjhrbkqj ZjKNzarrY+a6tgin2Lmpg8ENv8i5K0BCD5acCKJQkVUte+0rQ1r4swCOQ94eI3E91GdR 9PRT8TilGwkhxMfXwY2sTzCg33mxWgZrvQpbQ7XXqiSPA4Vduo9R3SL1BmV/tUnBnJiH /h0Nuno6K0gIBw/Xo0lvGy+FQcfuD9e3ITHQpF3czwejk7mZlFB83trGzect9orn4AIn ZV62wgUeWpOdoacC/kT9qELJfNK/iYe7QCgZ3muRTeqaBD1vLhW55v5LT5gZJ05t+i92 cCiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=3jB4Ii7xaomyhGENhnU8c09sqglMkNd9vUaTObmjEbw=; b=KY7FheW9SQ1iv19ZYWB0bzPm+ZamDJ4iB9tOs+5ipKTStGeooSBGENnjPnyhFs2w94 w0IllYTgjGgJ+ue2qwkrtK1S3nmqIBtkaJU4e7zoHBjQ0mKIGaCR1zWeUZKuCkgp2u0j DGvaC2o2Jy+SIy06cyc4PEKP3J9HqrJOHHUDeXrJ58daeLcjWe/unWjWGmhO1j5HLsOn bFTCqFNCSR+XJilhsyglzS5ayEO53wjezaEIUjC1pCei14veRLOqzIU5wRjB3NUaey0y gfNd1T13w6qkbiB8zEFED8fZYCQZWxgx1G9Ven/Oon2nnNc8KqVE68+ofdeWvttS889B eyTw== X-Gm-Message-State: ACrzQf0u7iA/j0lEGJ89xWV4xW8Zv8IvxVtosUemht1aOF0ysdA1hItF YVdnAsxRpMO/GHAegbMfua3oG144eiRADA== X-Google-Smtp-Source: AMsMyM4fmtELTs83PyV7uNFaHb9OBylupw+KzCIdPyMmkRownvCL77uNmO2rzbB4KEmniFE+wVpDLQ== X-Received: by 2002:a17:902:e54e:b0:178:5371:5199 with SMTP id n14-20020a170902e54e00b0017853715199mr20620773plf.59.1664170998779; Sun, 25 Sep 2022 22:43:18 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:18 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 3/7] powerpc/64/interrupt: Fix return to masked context after hard-mask irq becomes pending Date: Mon, 26 Sep 2022 15:43:01 +1000 Message-Id: <20220926054305.2671436-4-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" If a synchronous interrupt (e.g., hash fault) is taken inside an irqs-disabled region which has MSR[EE]=1, then an asynchronous interrupt that is PACA_IRQ_MUST_HARD_MASK (e.g., PMI) is taken inside the synchronous interrupt handler, then the synchronous interrupt will return with MSR[EE]=1 and the asynchronous interrupt fires again. If the asynchronous interrupt is a PMI and the original context does not have PMIs disabled (only Linux IRQs), the asynchronous interrupt will fire despite having the PMI marked soft pending. This can confuse the perf code and cause warnings. This patch changes the interrupt return so that irqs-disabled MSR[EE]=1 contexts will be returned to with MSR[EE]=0 if a PACA_IRQ_MUST_HARD_MASK interrupt has become pending in the meantime. The longer explanation for what happens: 1. local_irq_disable() 2. Hash fault interrupt fires, do_hash_fault handler runs 3. interrupt_enter_prepare() sets IRQS_ALL_DISABLED 4. interrupt_enter_prepare() sets MSR[EE]=1 5. PMU interrupt fires, masked handler runs 6. Masked handler marks PMI pending 7. Masked handler returns with PACA_IRQ_HARD_DIS set, MSR[EE]=0 8. do_hash_fault interrupt return handler runs 9. interrupt_exit_kernel_prepare() clears PACA_IRQ_HARD_DIS 10. interrupt returns with MSR[EE]=1 11. PMU interrupt fires, perf handler runs Fixes: 4423eb5ae32e ("powerpc/64/interrupt: make normal synchronous interrupts enable MSR[EE] if possible") Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/interrupt.c | 10 --------- arch/powerpc/kernel/interrupt_64.S | 34 +++++++++++++++++++++++++++--- 2 files changed, 31 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kernel/interrupt.c b/arch/powerpc/kernel/interrupt.c index 0e75cb03244a..f9db0a172401 100644 --- a/arch/powerpc/kernel/interrupt.c +++ b/arch/powerpc/kernel/interrupt.c @@ -431,16 +431,6 @@ notrace unsigned long interrupt_exit_kernel_prepare(struct pt_regs *regs) if (unlikely(stack_store)) __hard_EE_RI_disable(); - /* - * Returning to a kernel context with local irqs disabled. - * Here, if EE was enabled in the interrupted context, enable - * it on return as well. A problem exists here where a soft - * masked interrupt may have cleared MSR[EE] and set HARD_DIS - * here, and it will still exist on return to the caller. This - * will be resolved by the masked interrupt firing again. - */ - if (regs->msr & MSR_EE) - local_paca->irq_happened &= ~PACA_IRQ_HARD_DIS; #endif /* CONFIG_PPC64 */ } diff --git a/arch/powerpc/kernel/interrupt_64.S b/arch/powerpc/kernel/interrupt_64.S index ce25b28cf418..d76376ce7291 100644 --- a/arch/powerpc/kernel/interrupt_64.S +++ b/arch/powerpc/kernel/interrupt_64.S @@ -559,15 +559,43 @@ _ASM_NOKPROBE_SYMBOL(interrupt_return_\srr\()_kernel) ld r11,SOFTE(r1) cmpwi r11,IRQS_ENABLED stb r11,PACAIRQSOFTMASK(r13) - bne 1f + beq .Linterrupt_return_\srr\()_soft_enabled + + /* + * Returning to soft-disabled context. + * Check if a MUST_HARD_MASK interrupt has become pending, in which + * case we need to disable MSR[EE] in the return context. + */ + ld r12,_MSR(r1) + andi. r10,r12,MSR_EE + beq .Lfast_kernel_interrupt_return_\srr\() // EE already disabled + lbz r11,PACAIRQHAPPENED(r13) + andi. r10,r11,PACA_IRQ_MUST_HARD_MASK + beq 1f // No HARD_MASK pending + + /* Must clear MSR_EE from _MSR */ +#ifdef CONFIG_PPC_BOOK3S + li r10,0 + /* Clear valid before changing _MSR */ + .ifc \srr,srr + stb r10,PACASRR_VALID(r13) + .else + stb r10,PACAHSRR_VALID(r13) + .endif +#endif + xori r12,r12,MSR_EE + std r12,_MSR(r1) + b .Lfast_kernel_interrupt_return_\srr\() + +.Linterrupt_return_\srr\()_soft_enabled: #ifdef CONFIG_PPC_BOOK3S lbz r11,PACAIRQHAPPENED(r13) andi. r11,r11,(~PACA_IRQ_HARD_DIS)@l bne- interrupt_return_\srr\()_kernel_restart #endif - li r11,0 - stb r11,PACAIRQHAPPENED(r13) # clear out possible HARD_DIS 1: + li r11,0 + stb r11,PACAIRQHAPPENED(r13) // clear the possible HARD_DIS .Lfast_kernel_interrupt_return_\srr\(): cmpdi cr1,r3,0 From patchwork Mon Sep 26 05:43:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682317 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ja9aahLQ; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWvF1JTDz1ypH for ; Mon, 26 Sep 2022 15:45:44 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWvD5v0nz3br0 for ; Mon, 26 Sep 2022 15:45:44 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ja9aahLQ; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::102e; helo=mail-pj1-x102e.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Ja9aahLQ; dkim-atps=neutral Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrW6kW4z3bZ4 for ; Mon, 26 Sep 2022 15:43:23 +1000 (AEST) Received: by mail-pj1-x102e.google.com with SMTP id q35-20020a17090a752600b002038d8a68fbso11361636pjk.0 for ; Sun, 25 Sep 2022 22:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=vQgfl0s0CGmr/GOrxmh25LRPGto3tjJ8Zgh3Z+Ioj18=; b=Ja9aahLQmKZJPmSRVcxmzsdVXz0gvdN8z+XnXyvIJzv1Z7DTtIl5jrSNHHI2DbMBz/ jxFbh2VdDKKE0HM9DpsHAeHCgnh1YmKBylrxFqe+vBohI3d13JByQiJ+I0hYPfVQS0Aw UsvzuZyT3O8iE3taQARK/DHee2Xvei6zE5lnDQvODV5zHhn4wWq9gZCGqI0WtwREyNJd FX25uvRc3EmAsj89rF8n9okD5/NsOJEe/80+D8uF05gI9hX/SCO1cng3Bs8c6Zw+NPDo 0nzTOHDKyFYeYPLNo/tuTAUnpdMgfDwlSlSkx9/F9vb5WFxZHMhOt9+yCAM8dLT60i9R LPhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=vQgfl0s0CGmr/GOrxmh25LRPGto3tjJ8Zgh3Z+Ioj18=; b=1M/1GXdg2oRMud2OP9KP3IKR3GfhS6mB3mlZqgLCRV4azlvMQA1Ma27tAfPsjaqrnn XprBnid566RK1SSh+IIaI4MzBZb9YEWInrNnZk8EtIDBA4zBCg1sfWxecApyuythIgMG DYlWFfvYRUTAtEnjzLgoZBoj5+7jSl5ZZQUzs//Dn5uS8hyACMgSdpmruKPjJEA24lZ4 kLU8RgagIGlO28Qm8jv/wqaEZB9WHs1o82AXyWxQjcYK/E+rq3+2KaM25vvxVSsoTxXE 5/V5Kh6nR15tgIVn6sa5wrrKd7BSt0fv1vCNCo+Mf/VaTvD5bLw5dXw76/jAIFwjwgNn PJ9w== X-Gm-Message-State: ACrzQf22gaXIyBpzLQluUfsXoLwAjniKRuuxY/BF5wODOUA8phW8M+vE GkUVSbjgRxO3BoY9JeKq81Vpze51DiDMEA== X-Google-Smtp-Source: AMsMyM7b3VSWGxkAelT6SSutvHf5UgCxtiQ4L7eAudJrQQWNb9W6GZWjrzzV10NLv2e9bZ0NBl18kA== X-Received: by 2002:a17:902:c206:b0:176:a575:5ca8 with SMTP id 6-20020a170902c20600b00176a5755ca8mr20668496pll.74.1664171001136; Sun, 25 Sep 2022 22:43:21 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:20 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 4/7] powerpc/64s: Fix irq state management in runlatch functions Date: Mon, 26 Sep 2022 15:43:02 +1000 Message-Id: <20220926054305.2671436-5-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" When irqs are soft-disabled, MSR[EE] is volatile and can change from 1 to 0 asynchronously (if a PACA_IRQ_MUST_HARD_MASK interrupt hits). So it can not be used to check hard IRQ enabled status, except to confirm it is disabled. ppc64_runlatch_on/off functions use MSR this way to decide whether to re-enable MSR[EE] after disabling it, which leads to MSR[EE] being enabled when it shouldn't be (when a PACA_IRQ_MUST_HARD_MASK had disabled it between reading the MSR and clearing EE). This has been tolerated in the kernel previously, and it doesn't seem to cause a problem, but it is unexpected and may trip warnings or cause other problems as we tighten up this state management. Fix this by only re-enabling if PACA_IRQ_HARD_DIS is clear. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/runlatch.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/include/asm/runlatch.h b/arch/powerpc/include/asm/runlatch.h index cfb390edf7d0..ceb66d761fe1 100644 --- a/arch/powerpc/include/asm/runlatch.h +++ b/arch/powerpc/include/asm/runlatch.h @@ -19,10 +19,9 @@ extern void __ppc64_runlatch_off(void); do { \ if (cpu_has_feature(CPU_FTR_CTRL) && \ test_thread_local_flags(_TLF_RUNLATCH)) { \ - unsigned long msr = mfmsr(); \ __hard_irq_disable(); \ __ppc64_runlatch_off(); \ - if (msr & MSR_EE) \ + if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) \ __hard_irq_enable(); \ } \ } while (0) @@ -31,10 +30,9 @@ extern void __ppc64_runlatch_off(void); do { \ if (cpu_has_feature(CPU_FTR_CTRL) && \ !test_thread_local_flags(_TLF_RUNLATCH)) { \ - unsigned long msr = mfmsr(); \ __hard_irq_disable(); \ __ppc64_runlatch_on(); \ - if (msr & MSR_EE) \ + if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) \ __hard_irq_enable(); \ } \ } while (0) From patchwork Mon Sep 26 05:43:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682318 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MrC7/QSF; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWvv3bT0z1ypH for ; Mon, 26 Sep 2022 15:46:19 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWvv35Ppz3cB7 for ; Mon, 26 Sep 2022 15:46:19 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MrC7/QSF; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::62b; helo=mail-pl1-x62b.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=MrC7/QSF; dkim-atps=neutral Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrZ0wvmz3bc3 for ; Mon, 26 Sep 2022 15:43:25 +1000 (AEST) Received: by mail-pl1-x62b.google.com with SMTP id w13so5265081plp.1 for ; Sun, 25 Sep 2022 22:43:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=XJhviWUa1iDtZtz2IY3LCKwSVMIPPlufgF6evb/LdAI=; b=MrC7/QSFbkf98AhP2Lz05t1n+Z4m1U/fwMC1dlA/h8bbEjAud2wtIChY9uVFoHF7gj tqoTtr0xSn73DCJ0GRFoOPUKhPplFglSpWH2sqyNwhTVGOALjU1PwHLMBOYmV7SBiKb6 546OlOzT88gLfUWDJmhhQHYCPVjcd59jGNgaytuCLpB6ACbSfTWvUnV+gS0hAHLo4fZw UfaPPeiLwHJEslGgFN4bz/Vrj9VK6eJRpbtpHloKPVE5AB9W87nREN3PVZ4xlA9dFjL6 u8alyjOik33K649FA1+FqitIBrZNTW9IuyaRjRw7C1v/aBu0qVeIAxGm5PE08pn8zxYY fniw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=XJhviWUa1iDtZtz2IY3LCKwSVMIPPlufgF6evb/LdAI=; b=snBa91pV04+wbFnDaETD8dB9ss3qfZXgvGJh2If7GwdJ9KoWMgT1vAIa0rWK1oz+Pk TGpDm4cOWOCQ0iayUpvAt4a9ib8tPT7QNI1Wb/p66561reoHP9C74qmKIUOkd7pEIHxD nNb+fxyuhMXGkPDcg6T9N6TIZXHUJFHrWZrUPZOe11D1Pcay7/KM4ADQsq4mT7f0dOiH MQlFMKp9oQPQl+L1IlHnC+chvTjMhElAKY4CbUydNWQdqeM+yQRk8OEwCXj2rfvcp8d3 jKTQjSObI0LEunBMd+YeI2zCD3YTby+7/wP4nLVhfY3w2z6ITQAo+O6NOB2421c8W6ov 8jEQ== X-Gm-Message-State: ACrzQf0OlZRgkoWXBqxWC8MErzma+jdiOrUiEQc9dmsGCQa29PuIcGYP azM/LCU0oRmTz7yhrgOZFXC0FJQIto7bCA== X-Google-Smtp-Source: AMsMyM5WfQ47TSrJLX37GIMmm1oO/pB9A/AjJEjr6gnjPk3Cdg08TjBcY4Q6iZuc+SpDuN7JgRnlsA== X-Received: by 2002:a17:902:ce11:b0:172:6f2c:a910 with SMTP id k17-20020a170902ce1100b001726f2ca910mr20964384plg.156.1664171003358; Sun, 25 Sep 2022 22:43:23 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:22 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 5/7] powerpc/64s/interrupt: masked handler debug check for previous hard disable Date: Mon, 26 Sep 2022 15:43:03 +1000 Message-Id: <20220926054305.2671436-6-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Prior changes eliminated cases of masked PACA_IRQ_MUST_HARD_MASK interrupts that re-fire due to MSR[EE] being enabled while they are pending. Add a debug check in the masked interrupt handler to catch if this occurs. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/exceptions-64s.S | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index 3d0dc133a9ae..dafa275f18bc 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -2794,6 +2794,16 @@ masked_Hinterrupt: masked_interrupt: .endif stw r9,PACA_EXGEN+EX_CCR(r13) +#ifdef CONFIG_PPC_IRQ_SOFT_MASK_DEBUG + /* + * Ensure there was no previous MUST_HARD_MASK interrupt or + * HARD_DIS setting. + */ + lbz r9,PACAIRQHAPPENED(r13) + andi. r9,r9,(PACA_IRQ_MUST_HARD_MASK|PACA_IRQ_HARD_DIS) +0: tdnei r9,0 + EMIT_BUG_ENTRY 0b,__FILE__,__LINE__,0 +#endif lbz r9,PACAIRQHAPPENED(r13) or r9,r9,r10 stb r9,PACAIRQHAPPENED(r13) From patchwork Mon Sep 26 05:43:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682319 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Sd/FMCoT; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWwh6JkJz1ypH for ; Mon, 26 Sep 2022 15:47:00 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWwh67qzz3dpY for ; Mon, 26 Sep 2022 15:47:00 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Sd/FMCoT; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1029; helo=mail-pj1-x1029.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=Sd/FMCoT; dkim-atps=neutral Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrb6hCMz3bcw for ; Mon, 26 Sep 2022 15:43:27 +1000 (AEST) Received: by mail-pj1-x1029.google.com with SMTP id s90-20020a17090a2f6300b00203a685a1aaso5696916pjd.1 for ; Sun, 25 Sep 2022 22:43:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=nZy4uxEt5h7DKz5jIhv2KcDDKz2QLaUO1KgGLJtUmsg=; b=Sd/FMCoT9ojX59SYhGCwnk94Gcy/jAw4o+m8O3Q8XLbayhJwAqevHJLYZkxMCxPFr6 F8ydIzd1T0uy7jBhRxoMPM51EH8xKAePN52kfUzBXmuKXMX/e7mlFgNJkRuq3XiXGusO FHFqk17tjD+VfJBahVrwKwKFdSNJtaBpkVOInSwO6K+bkRunpFhqq9qxnGSs64A4X9xn RrpmXCrtEULjB2s1WVGffRk0AAplNhSu3Jp+hwBY+OBkUbIa5eWbnW7oJb/gi76xHMTt 2/pXfnW3Qc8CDcD7hs/uyCoSr4/LoCk2diCfa+E6VsP8B1IsxEwrJrbVtxuM+HL/MLYE v9hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=nZy4uxEt5h7DKz5jIhv2KcDDKz2QLaUO1KgGLJtUmsg=; b=n1pjYededeZMiduOkkB0i7b+jBjV+7lKaBpz8UpaABDmJa1Zd9jxOVoMJFHkr9zetj 7CzP2w22bGIsIndJjxUiFJGGlvFVBX38et2xjrzDCiPbUha8bueMPLBoX8UP0ESmYsO9 Jv1bBxNHUzB6uvFo/jZjAvGFb96jFslEpNnzUyvYKe32/nPKdVw/JCQgsP5SEX98KM3I 9VjrOqa1hyyUgeKNmLcUZQd0Sp67S+/fPgJ0KM0m4+U6jfKHdK2S8yYaDlVIXBBM1b0X p/naD1mI+Bh7Gv8+I4z3AcDN3mqtH+d/dOTSmUKCj5EbjZMF1nw5lJVcXtw0+pdlDo78 y2Mg== X-Gm-Message-State: ACrzQf1mfU1MO0CF9KrdL470CvhqNLmJPrjdVLAkozOu7jG+CoyV1d09 Wm5uUz1WJCjgOf4c5knC5F+yLiudiE2Y2w== X-Google-Smtp-Source: AMsMyM5H8hB2yDCTpktOWNvdES5sKRKHzPyPBJ8lpijpj3C36URsp+VJ2VXG0HKX9zYihNMWb+wEKw== X-Received: by 2002:a17:90b:3b81:b0:202:597a:c71d with SMTP id pc1-20020a17090b3b8100b00202597ac71dmr34167368pjb.105.1664171005696; Sun, 25 Sep 2022 22:43:25 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:25 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 6/7] powerpc/64/interrupt: avoid BUG/WARN recursion in interrupt entry Date: Mon, 26 Sep 2022 15:43:04 +1000 Message-Id: <20220926054305.2671436-7-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" BUG/WARN are handled with a program interrupt which can turn into an infinite recursion when there are bugs in interrupt handler entry (which can be irritated by bugs in other parts of the code). There is one feeble attempt to avoid this recursion, but it misses several cases. Make a tidier macro for this and switch most bugs in the interrupt entry wrapper over to use it. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/interrupt.h | 33 +++++++++++++++++----------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/include/asm/interrupt.h b/arch/powerpc/include/asm/interrupt.h index b61555e30c7c..7f3180047115 100644 --- a/arch/powerpc/include/asm/interrupt.h +++ b/arch/powerpc/include/asm/interrupt.h @@ -74,6 +74,19 @@ #include #include +#ifdef CONFIG_PPC64 +/* + * WARN/BUG is handled with a program interrupt so minimise checks here to + * avoid recursion and maximise the chance of getting the first oops handled. + */ +#define INT_SOFT_MASK_BUG_ON(regs, cond) \ +do { \ + if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG) && \ + (user_mode(regs) || (TRAP(regs) != INTERRUPT_PROGRAM))) \ + BUG_ON(cond); \ +} while (0) +#endif + #ifdef CONFIG_PPC_BOOK3S_64 extern char __end_soft_masked[]; bool search_kernel_soft_mask_table(unsigned long addr); @@ -170,8 +183,7 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs) * context. */ if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) { - if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) - BUG_ON(!(regs->msr & MSR_EE)); + INT_SOFT_MASK_BUG_ON(regs, !(regs->msr & MSR_EE)); __hard_irq_enable(); } else { __hard_RI_enable(); @@ -194,20 +206,15 @@ static inline void interrupt_enter_prepare(struct pt_regs *regs) * CT_WARN_ON comes here via program_check_exception, * so avoid recursion. */ - if (TRAP(regs) != INTERRUPT_PROGRAM) { + if (TRAP(regs) != INTERRUPT_PROGRAM) CT_WARN_ON(ct_state() != CONTEXT_KERNEL && ct_state() != CONTEXT_IDLE); - if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) - BUG_ON(is_implicit_soft_masked(regs)); - } - - /* Move this under a debugging check */ - if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG) && - arch_irq_disabled_regs(regs)) - BUG_ON(search_kernel_restart_table(regs->nip)); + INT_SOFT_MASK_BUG_ON(regs, is_implicit_soft_masked(regs)); + INT_SOFT_MASK_BUG_ON(regs, arch_irq_disabled_regs(regs) && + search_kernel_restart_table(regs->nip)); } - if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) - BUG_ON(!arch_irq_disabled_regs(regs) && !(regs->msr & MSR_EE)); + INT_SOFT_MASK_BUG_ON(regs, !arch_irq_disabled_regs(regs) && + !(regs->msr & MSR_EE)); #endif booke_restore_dbcr0(); From patchwork Mon Sep 26 05:43:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 1682320 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BWZnXz4k; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MbWxM6yqqz1ypH for ; Mon, 26 Sep 2022 15:47:35 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4MbWxM6SLDz3dxR for ; Mon, 26 Sep 2022 15:47:35 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BWZnXz4k; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1035; helo=mail-pj1-x1035.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=BWZnXz4k; dkim-atps=neutral Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4MbWrf3vYyz3c5v for ; Mon, 26 Sep 2022 15:43:30 +1000 (AEST) Received: by mail-pj1-x1035.google.com with SMTP id i15-20020a17090a4b8f00b0020073b4ac27so5658641pjh.3 for ; Sun, 25 Sep 2022 22:43:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=MYDXvHWVqlFaxJDDMXiyCj4gN/S0APV5VGQxUE3ephc=; b=BWZnXz4kgAC5AvUOIL7No/c7/ElDnDrlpWGppCkPUERaay6/serJrYbqA9waPSSORZ zr8jCRzVGo7558Uq6Sav9dl3pmdCiOj0yDSd/YaMIzzr93R+6KvIIR6NMZZlPdiSMPuk /6bO6MbhT/GIIrqjwLzeRcHOfqUSc3JImd+O86V20by9FwUCfW200ToCPRiJvxk/JUpK GRotMrEA62e7cSwcwyjaEhlwJQ5ovdXonc8dhT9Yz1hMTt6P4PsFZ6rIa9kESJ6P9v+Z XhkuIS2HvyGw5oGARH1u3bMvEwnc4OBJdKuSCTZZqyEadCSKrL2BEdMyq+ExDWRxoUKB flRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=MYDXvHWVqlFaxJDDMXiyCj4gN/S0APV5VGQxUE3ephc=; b=3wdvJdMBtCmNSVuaXAWsXOqNkFd+3+UzEfGvICsaauUJzr7fMvZDezE9F2+bY87ySQ 04GAUP/TXvXbnl7l+OY1xUHLapqtrX7Nx6INV5SNuANbwmUlYwvuANjjQGEhzPpTTUfX xIlLz+Cfb1d+apqwxLXcqNbFv9FG7A/Ij5izjoaSAJe9PDEiZ6ImH2k3JoMi4k+MFaIU cjqeVhoTaTzvVDF53rbKT8gHxX9CGNIDl26of0w8jBclmZNaMUqcRBlDhsbyCcd5V7tS ft92uNzO67/HWfdBPLTJxWJNhAIWTgxxExrG4JAJYZP+zA40rEMSGof84zW0MFjjLgnd 1Lfw== X-Gm-Message-State: ACrzQf1165bK9IGwfPNFMdpS0nJUvCdViAP9h7Rjjz4Awvf4NDify0ml as7ZNECcb2lXzZevBIIR2St29w09kYEkWA== X-Google-Smtp-Source: AMsMyM7XzO57FbdjTsEDGwmzhn/wtg5xseuZiN1WmfnmF0La0vdcg3hKsSyJ+RRRIKy4cy8oQ4e4LQ== X-Received: by 2002:a17:90b:4a43:b0:202:7706:73d7 with SMTP id lb3-20020a17090b4a4300b00202770673d7mr22613425pjb.137.1664171008015; Sun, 25 Sep 2022 22:43:28 -0700 (PDT) Received: from bobo.ozlabs.ibm.com (203-219-181-119.static.tpgi.com.au. [203.219.181.119]) by smtp.gmail.com with ESMTPSA id o90-20020a17090a0a6300b002001c9bf22esm5676553pjo.8.2022.09.25.22.43.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 25 Sep 2022 22:43:27 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH v3 7/7] powerpc/64/irq: tidy soft-masked irq replay and improve documentation Date: Mon, 26 Sep 2022 15:43:05 +1000 Message-Id: <20220926054305.2671436-8-npiggin@gmail.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220926054305.2671436-1-npiggin@gmail.com> References: <20220926054305.2671436-1-npiggin@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" irq replay is quite complicated because of softirq processing which itself enables and disables irqs. Several considerations need to be accounted for due to this, and they are not clearly documented. Refactor the irq replay code a bit to tidy and deduplicate some common functions. Add comments, debug checks. This has a minor functional change that irq tracing enable/disable is done after each interrupt replayed, rather than after a batch. It also re-sets state to IRQS_ALL_DISABLED after an interrupt, which doesn't matter much because interrupts are hard disabled at this point, but it is more consistent with how interrupt handlers are called. Signed-off-by: Nicholas Piggin --- arch/powerpc/kernel/irq_64.c | 93 +++++++++++++++++++++++------------- 1 file changed, 61 insertions(+), 32 deletions(-) diff --git a/arch/powerpc/kernel/irq_64.c b/arch/powerpc/kernel/irq_64.c index 01645e03e9f0..eb2b380e52a0 100644 --- a/arch/powerpc/kernel/irq_64.c +++ b/arch/powerpc/kernel/irq_64.c @@ -68,6 +68,35 @@ int distribute_irqs = 1; +static inline void next_interrupt(struct pt_regs *regs) +{ + /* + * Softirq processing can enable/disable irqs, which will leave + * MSR[EE] enabled and the soft mask set to IRQS_DISABLED. Fix + * this up. + */ + if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) + hard_irq_disable(); + else + irq_soft_mask_set(IRQS_ALL_DISABLED); + + /* + * We are responding to the next interrupt, so interrupt-off + * latencies should be reset here. + */ + trace_hardirqs_on(); + trace_hardirqs_off(); +} + +static inline bool irq_happened_test_and_clear(u8 irq) +{ + if (local_paca->irq_happened & irq) { + local_paca->irq_happened &= ~irq; + return true; + } + return false; +} + void replay_soft_interrupts(void) { struct pt_regs regs; @@ -79,18 +108,25 @@ void replay_soft_interrupts(void) * recurse into this function. Don't keep any state across * interrupt handler calls which may change underneath us. * + * Softirqs can not be disabled over replay to stop this recursion + * because interrupts taken in idle code may require RCU softirq + * to run in the irq RCU tracking context. This is a hard problem + * to fix without changes to the softirq or idle layer. + * * We use local_paca rather than get_paca() to avoid all the * debug_smp_processor_id() business in this low level function. */ + if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) { + WARN_ON_ONCE(mfmsr() & MSR_EE); + WARN_ON(!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)); + } + ppc_save_regs(®s); regs.softe = IRQS_ENABLED; regs.msr |= MSR_EE; again: - if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) - WARN_ON_ONCE(mfmsr() & MSR_EE); - /* * Force the delivery of pending soft-disabled interrupts on PS3. * Any HV call will have this side effect. @@ -105,56 +141,47 @@ void replay_soft_interrupts(void) * This is a higher priority interrupt than the others, so * replay it first. */ - if (IS_ENABLED(CONFIG_PPC_BOOK3S) && (local_paca->irq_happened & PACA_IRQ_HMI)) { - local_paca->irq_happened &= ~PACA_IRQ_HMI; + if (IS_ENABLED(CONFIG_PPC_BOOK3S) && + irq_happened_test_and_clear(PACA_IRQ_HMI)) { regs.trap = INTERRUPT_HMI; handle_hmi_exception(®s); - if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) - hard_irq_disable(); + next_interrupt(®s); } - if (local_paca->irq_happened & PACA_IRQ_DEC) { - local_paca->irq_happened &= ~PACA_IRQ_DEC; + if (irq_happened_test_and_clear(PACA_IRQ_DEC)) { regs.trap = INTERRUPT_DECREMENTER; timer_interrupt(®s); - if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) - hard_irq_disable(); + next_interrupt(®s); } - if (local_paca->irq_happened & PACA_IRQ_EE) { - local_paca->irq_happened &= ~PACA_IRQ_EE; + if (irq_happened_test_and_clear(PACA_IRQ_EE)) { regs.trap = INTERRUPT_EXTERNAL; do_IRQ(®s); - if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) - hard_irq_disable(); + next_interrupt(®s); } - if (IS_ENABLED(CONFIG_PPC_DOORBELL) && (local_paca->irq_happened & PACA_IRQ_DBELL)) { - local_paca->irq_happened &= ~PACA_IRQ_DBELL; + if (IS_ENABLED(CONFIG_PPC_DOORBELL) && + irq_happened_test_and_clear(PACA_IRQ_DBELL)) { regs.trap = INTERRUPT_DOORBELL; doorbell_exception(®s); - if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) - hard_irq_disable(); + next_interrupt(®s); } /* Book3E does not support soft-masking PMI interrupts */ - if (IS_ENABLED(CONFIG_PPC_BOOK3S) && (local_paca->irq_happened & PACA_IRQ_PMI)) { - local_paca->irq_happened &= ~PACA_IRQ_PMI; + if (IS_ENABLED(CONFIG_PPC_BOOK3S) && + irq_happened_test_and_clear(PACA_IRQ_PMI)) { regs.trap = INTERRUPT_PERFMON; performance_monitor_exception(®s); - if (!(local_paca->irq_happened & PACA_IRQ_HARD_DIS)) - hard_irq_disable(); + next_interrupt(®s); } - if (local_paca->irq_happened & ~PACA_IRQ_HARD_DIS) { - /* - * We are responding to the next interrupt, so interrupt-off - * latencies should be reset here. - */ - trace_hardirqs_on(); - trace_hardirqs_off(); + /* + * Softirq processing can enable and disable interrupts, which can + * result in new irqs becoming pending. Must keep looping until we + * have cleared out all pending interrupts. + */ + if (local_paca->irq_happened & ~PACA_IRQ_HARD_DIS) goto again; - } } #if defined(CONFIG_PPC_BOOK3S_64) && defined(CONFIG_PPC_KUAP) @@ -270,10 +297,12 @@ notrace void arch_local_irq_restore(unsigned long mask) trace_hardirqs_off(); replay_soft_interrupts_irqrestore(); - local_paca->irq_happened = 0; trace_hardirqs_on(); irq_soft_mask_set(IRQS_ENABLED); + if (IS_ENABLED(CONFIG_PPC_IRQ_SOFT_MASK_DEBUG)) + WARN_ON(local_paca->irq_happened != PACA_IRQ_HARD_DIS); + local_paca->irq_happened = 0; __hard_irq_enable(); preempt_enable(); }