From patchwork Thu Jun 15 04:44:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 1795191 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.a=rsa-sha256 header.s=selector2 header.b=ARLPWPZG; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4QhV9V3K0sz20WR for ; Thu, 15 Jun 2023 14:46:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243086AbjFOEqE (ORCPT ); Thu, 15 Jun 2023 00:46:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243150AbjFOEpu (ORCPT ); Thu, 15 Jun 2023 00:45:50 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2126.outbound.protection.outlook.com [40.107.101.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92965270F; Wed, 14 Jun 2023 21:45:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f3z2PZQ31jcgL3I2bi77OA7ph/g0bivyWsupDTZi1kDaQmATPYby5KUgqpsI/uqKNEbprjyQampYrYnx0eOWeYTEe/qSIu0CVDQdNSmdIAFD5A209C5dv+lfQmYDChJ9uwai1hMzB6V94j/+zICbEUdCOW6GVsbMlxAobHHOjw1x238jsY44hqpbJ8oo3K1enjAeL3utDWPoTwgthhPG9Ji6CggzQoVESqjdJ0ZVsQShoGtlZsqNrOJ6pa7JevH+JIis+8TUOZHk1XJU5P/EjMRx9hNI4fHpcdbD0fwV6e+AD75orc7J3BKg3GRGLQKD6PsZKyJmfe2aFKzDWaiRqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tj4M9fxqQNfCE5aujo7cGgHBSIYwBpKxgqAgVGTGBp8=; b=dvrZYMRnQpsS8KFhfW7jdcrbt3s6e592G88I+NNEq85Gt+VFiC4DY+1NUGj3x5LsKWdpLKcQage+LJBXN9Me1qRykRJuFzB85MuZ7iVfU3o/3aBXPjtrf+mqW1wCqDlBQCcusMfZiWIgjR8989Nwu4NmpbV5tB/piREL5H6hHLewv6hm8eIfb6Q7m9p6mst7E9p/A5VAM5CPaMOVsVKqRJ8Yy2qOK6B1KdKjIL7GoVM30NODP1ce8GoXwFQtcetDGm8cnXy1e6x631a0RATwYk6jWk01/cl+dwPo3EzM9yWANRI3ppgtLGqBFEiKvpgsBisD2M57W4mLWZiRU0TEgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tj4M9fxqQNfCE5aujo7cGgHBSIYwBpKxgqAgVGTGBp8=; b=ARLPWPZG4jupCbf53+rsVucLhEKk4b3LgU46PtVPLAVZ+sxnNC1x6i2KFxwKhUacNqEF/EW2p/0iduSv8RNY/5yrnFm9PJye6SAgTN+XhL+7v6hW7hVKB1piKRYafc87M+l1akSGv5CchntSS9JSGNNuPogIBi1Gz8DkuPAQUiA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) by PH7PR21MB3382.namprd21.prod.outlook.com (2603:10b6:510:1de::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.10; Thu, 15 Jun 2023 04:45:37 +0000 Received: from BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682]) by BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682%3]) with mapi id 15.20.6521.009; Thu, 15 Jun 2023 04:45:37 +0000 From: Dexuan Cui To: bhelgaas@google.com, davem@davemloft.net, decui@microsoft.com, edumazet@google.com, haiyangz@microsoft.com, jakeo@microsoft.com, kuba@kernel.org, kw@linux.com, kys@microsoft.com, leon@kernel.org, linux-pci@vger.kernel.org, lpieralisi@kernel.org, mikelley@microsoft.com, pabeni@redhat.com, robh@kernel.org, saeedm@nvidia.com, wei.liu@kernel.org, longli@microsoft.com, boqun.feng@gmail.com, ssengar@microsoft.com, helgaas@kernel.org Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, josete@microsoft.com, simon.horman@corigine.com, stable@vger.kernel.org Subject: [PATCH v4 1/5] PCI: hv: Fix a race condition bug in hv_pci_query_relations() Date: Wed, 14 Jun 2023 21:44:47 -0700 Message-Id: <20230615044451.5580-2-decui@microsoft.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230615044451.5580-1-decui@microsoft.com> References: <20230615044451.5580-1-decui@microsoft.com> X-ClientProxiedBy: CYZPR17CA0016.namprd17.prod.outlook.com (2603:10b6:930:8c::25) To BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL0PR2101MB1092:EE_|PH7PR21MB3382:EE_ X-MS-Office365-Filtering-Correlation-Id: e72a2ae8-3426-4843-736f-08db6d5b5ce3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3orImqG6Du5OYL0gbiciO6rGKclc1ELywVwcbxGUl5tHGbpNd6wOcrw6ammIqGW8vQBH42S59MGzx9r4r/mg91Er5zlWe4BislcMApT2dszW5TJ8Pw8E+mipOXZMsngbH9DCUBeLQye0MAOe0JFQtdyBWdJzAZt0nYrBMheNHnKdFSu2ChRcJ2DqOjsGp+sJtiwg+iOYRDv9ho4AzsT8GJAeh2k1T0/ZKD78ETAbjDTP25a/ZjyUJPqtMI34CXddb6jyeQWpJLxSdtYhq2n+Xp3fM05EkrPt1Ju+WMyv17OSZ21qk5FrdxkjwznD8UdGrbyPYgjghiSwKboM6I4m9OgjnAU5u+9At1/ulLHMTzflerxF6FAeK8zW2CCD8oLJrJPxcvLZ0Ab/kyufLiqMAx+NlutYtoYWV9tnO3Obhrk3kKGuvHfdDEZX2CAjJZkVMAu7oGVWwYFy2a8SKqfr/RBrMsLNyejUSqmd01+fDqEP5xqLopN3l3y4nArrB1Lj6+wDrbz34MAc75IA1fehoc9O2WY/8hFX4UeUsErTPPhpJu8JQ+nn3ig+HeXWrfe8LSaIt/P0L7M82zRofkdofxDi+T34x2CV2GjsGew2pUtvXHSIzptz0CsWsyBdPbCJP/RIn6axRU4ogVOdwpAuTQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR2101MB1092.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(82950400001)(921005)(478600001)(82960400001)(8676002)(41300700001)(8936002)(10290500003)(66946007)(4326008)(66476007)(316002)(38100700002)(2616005)(66556008)(83380400001)(186003)(6486002)(6666004)(52116002)(6506007)(5660300002)(6512007)(1076003)(86362001)(7416002)(2906002)(36756003)(66899021);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AmkbKPRSjqVrMtvDz4ak1hzj3VOnz6ypBzZG9TXzI3ySUqdqDnmlF4gBdWT2uf51w3a2JG9RZNkyuoVzjBwt4z9irqF42UGnAb0A31W8jzQ7HtgH4sCrj744RI+BCftmUJ3N7QSq1JAFof/VOZWuWGNAP0wYLkS1C3f6OIaAAnvv9u+f4rv9x070YwIMlEVJoMPUfrDGaXHOXaVK0lusvNPvLnv+S+A8d0UHWZQ8AWR+2WVxYQq3qAyswA29MJlh3z6l29IaVX6QHST6lkrQrlcmD9arFVvpQzOss3yZcXyPFrXeo4ZMpN732694MDEgGbuKIBsQr2H09Sl2fJ3YLdG/rJVppYgaCp5pMg6ntYvmcwJRDAoCuSkdDF8rIUfepM3XozdUUzsUYiIT+yQMV6ByhoQiYrzx5MvQAGbXgGEnb+Tdk0Q2+abXJPsAZiq2fEsOEkS3xIA7CaHCeP8yil+wP+DgqsOfo0oLnJ2qkW3gmIXGniy5MJYtGGkL7bh9cv4xqX0+adCja7ifvdBnWtsOX19Gjcu5ha3CR+e83LtcpA4lpFmcFxMHjxzpZqXjvMWtdw6ijPVlVsZ0e/ThiKY0fufDkwVyLB7DrCFFarHC1w7Z38N23mQPVof///Jpa59PMvK/Jd2P1Q8MFwPbo52t5PAeYATmmd/NsecYTnG85E+JQDLsI7SyfpKgRiZhcmyUw8ZtB4KNcsAWGeeCNGM23Dq2qMi25qE37NU5Mlkq8pTupITwbPCOq8enL4lYX9skZbjgeLR9OIZkT4GTIPVtLmuilgV/ycf3vDRN3bw4Qv8D9q/saht+KVi1gaFM2Wd4JHCZi5lxDG80hsUsCDwDGcqMExS4Yza8QaC0zefZLVfUCPhbzuO2KeM7gFsvnYO0f13d74LFwtRQejff4NqvmmEC9bi/bjO0Lqgqy7S1SiSIdiiHGF1NFOVesTej2hD0vS+BqDxRgB8afPvD5rp88ROwHYsvyqanSAo7l7ZXFIyJPHcM1SRyk2JYEvxbNVh3ZOwFNO3m8INxE7hHH6PeTWavdxNAWjQBrDL3omGP1r08A9RuNLOVsS5MmV+tvSZ9/fUQJt0IkDocFN0IT9TXSFzF3TtOZE+S8Zy+j5CR8bz/3Hvo6i4K3OH/V3K2dYtjGS24QPUuW9A7S87GFD+Mf+MirXbveMSBrkneI+w2vj07+WN83c8VCZzRF90HLwlfylUl/RW/y1Obrs7TDWXgUOZZjtNey50w2DmSFw0fWs4cpgiKaCQ1i72GC4nb9MbId1ncU2LtU5ikAhqPScAOK1AwyZDSGOuvLDNqlk3Gb4mZZuxYliOwgQxJddanwdBaNrlsJ4BGqMIgTRmGrCZ54VW3nGNHzH/PtES7KuX/Vn/UwFf+x53VuOsmwiAvxkZPzUO/zjqgEDsclVkBIYniSyGNSTJzmiVsoujOj+MSVzC5+8WMd7tJg+1aGwOyK4+lZ8MsrbaaSUgx1HphOrV4WWciIYjJpDm+Gt9JKP6gC9Y7pWioSBL3jMQMm3wS72v1gQvh/yDYYKCyU/q+2rgQscNASzR+f7i95OGDTZHefeWls+gBhP4ct7sJ5EBZEgAE0e+sOtpSfzLIU21KJHwqHMYGWAtejvyZF9jhUi933xA56Mo+O7N1N/u0kRfc X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: e72a2ae8-3426-4843-736f-08db6d5b5ce3 X-MS-Exchange-CrossTenant-AuthSource: BL0PR2101MB1092.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 04:45:37.4841 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yOGJFQ4zpHJbJ3OYryksj9sF+gL+kgwa4TZIVMBXBk0jso0/TqIfwsORxyUfGspvdEWeurtYLpmX9NrP+ULN4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR21MB3382 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Since day 1 of the driver, there has been a race between hv_pci_query_relations() and survey_child_resources(): during fast device hotplug, hv_pci_query_relations() may error out due to device-remove and the stack variable 'comp' is no longer valid; however, pci_devices_present_work() -> survey_child_resources() -> complete() may be running on another CPU and accessing the no-longer-valid 'comp'. Fix the race by flushing the workqueue before we exit from hv_pci_query_relations(). Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley Acked-by: Lorenzo Pieralisi Cc: stable@vger.kernel.org --- v2: Removed the "debug code". No change to the patch body. Added Cc:stable v3: Added Michael's Reviewed-by. v4: Provided more details of the issue in the comment and commit log. Added Lorenzo's Acked-by. drivers/pci/controller/pci-hyperv.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index bc32662c6bb7f..ea8862e656b68 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -3401,6 +3401,24 @@ static int hv_pci_query_relations(struct hv_device *hdev) if (!ret) ret = wait_for_response(hdev, &comp); + /* + * In the case of fast device addition/removal, it's possible that + * vmbus_sendpacket() or wait_for_response() returns -ENODEV but we + * already got a PCI_BUS_RELATIONS* message from the host and the + * channel callback already scheduled a work to hbus->wq, which can be + * running pci_devices_present_work() -> survey_child_resources() -> + * complete(&hbus->survey_event), even after hv_pci_query_relations() + * exits and the stack variable 'comp' is no longer valid; as a result, + * a hang or a page fault may happen when the complete() calls + * raw_spin_lock_irqsave(). Flush hbus->wq before we exit from + * hv_pci_query_relations() to avoid the issues. Note: if 'ret' is + * -ENODEV, there can't be any more work item scheduled to hbus->wq + * after the flush_workqueue(): see vmbus_onoffer_rescind() -> + * vmbus_reset_channel_cb(), vmbus_rescind_cleanup() -> + * channel->rescind = true. + */ + flush_workqueue(hbus->wq); + return ret; } From patchwork Thu Jun 15 04:44:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 1795192 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.a=rsa-sha256 header.s=selector2 header.b=ILjqORlQ; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4QhV9c1yb1z20WR for ; Thu, 15 Jun 2023 14:46:12 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243228AbjFOEqJ (ORCPT ); Thu, 15 Jun 2023 00:46:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243105AbjFOEpv (ORCPT ); Thu, 15 Jun 2023 00:45:51 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2126.outbound.protection.outlook.com [40.107.101.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3B94212B; Wed, 14 Jun 2023 21:45:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c/biEWcsmTCTqLammabYpFl15iT+OG/inM8YgfOuU1EOlpszV+iXH+HxeaG0uYL9ExF0/OLKrgvHHvesS0NngFLyMy91VKKAOTKtwdLnJIUgTurz+XFuXCLzJyCZGT9CR77loLUaV9jpTrWTYyLbYYuocdwXdNMRSZUME/PGvGWrwQYW2kvtNbbF+HP9htuFpM++2ck/pJjaQeakAJd1EXzmEptAxup1u64yT5J9NXm6om27iLbUwM7PaFsPm6tJ9+9sVl1rEwW3wZ1d4oQRhg0HgZ26Tc4XRTl9y1S8wKR9ESGkzXe9IFc68FOHj9ND/ic6aFLvLibKFdTqEeAg2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XB/ceB0iHCgRi4TRsvtgV9J8qIRTlPRi2g1VdUtXxNY=; b=E8Srw66dx5NtBcf77yLJmX7GX78xXpVnXsBCYB63JOCcVKOR3pObjLblYw91SzhwK8Bb0R/5APvDYR9+l1mVfZ6gprl/fTuQJjmQnrsRhjWeQvTWMcKH0VU7IfN9XtDcOl+C+AhSwueNzFUBrjzZkzEVjgTE6B4h/TB4h9hfIEdFOKErItagEn0jE9Ji+vmm2fdWZkiEmhI42UQGEYGA5/cfP8YK/W294wSE353235zOrT9SLIi4xJt8ZVBV9IKkOZygH/R+Naq6iRQNhBh9ASWUJykZJ6S+loMEqENUZtRKkoLQthgGTjLRTIDi6jS+/4bzItXcdLjfd8+c6CxZCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XB/ceB0iHCgRi4TRsvtgV9J8qIRTlPRi2g1VdUtXxNY=; b=ILjqORlQ42ZKWOR4IyhC1vwoph/UZTJh47qs+mIKFXwNTvz0J+/b1BoXKbmXyz+/7Qs9VfZQv/eljIudERVW57SDydHNbu/qRs+EpzyMqOLFR/CClWC0fGIUteV3s4U0f6kpmu5EA7QUuF7C+mWRIuhG/r0hLrSW0I5kslIjPxE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) by PH7PR21MB3382.namprd21.prod.outlook.com (2603:10b6:510:1de::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.10; Thu, 15 Jun 2023 04:45:40 +0000 Received: from BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682]) by BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682%3]) with mapi id 15.20.6521.009; Thu, 15 Jun 2023 04:45:40 +0000 From: Dexuan Cui To: bhelgaas@google.com, davem@davemloft.net, decui@microsoft.com, edumazet@google.com, haiyangz@microsoft.com, jakeo@microsoft.com, kuba@kernel.org, kw@linux.com, kys@microsoft.com, leon@kernel.org, linux-pci@vger.kernel.org, lpieralisi@kernel.org, mikelley@microsoft.com, pabeni@redhat.com, robh@kernel.org, saeedm@nvidia.com, wei.liu@kernel.org, longli@microsoft.com, boqun.feng@gmail.com, ssengar@microsoft.com, helgaas@kernel.org Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, josete@microsoft.com, simon.horman@corigine.com, stable@vger.kernel.org Subject: [PATCH v4 2/5] PCI: hv: Fix a race condition in hv_irq_unmask() that can cause panic Date: Wed, 14 Jun 2023 21:44:48 -0700 Message-Id: <20230615044451.5580-3-decui@microsoft.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230615044451.5580-1-decui@microsoft.com> References: <20230615044451.5580-1-decui@microsoft.com> X-ClientProxiedBy: CYZPR17CA0016.namprd17.prod.outlook.com (2603:10b6:930:8c::25) To BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL0PR2101MB1092:EE_|PH7PR21MB3382:EE_ X-MS-Office365-Filtering-Correlation-Id: 51762adb-e74f-476a-08ef-08db6d5b5e87 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lXSZXza3O2NuL/1PxOkl0LNDzssPJ2Fh7NWiDabgjNC2YgEwZZK+bMgB50nqIokvah7X0xKCgA9g4j/4gKW1G64XumE2GE2c2uUShCECy/UTRlzREHvtZIOMdAAx3thorxLejZZrhgJ44Zn2Zql8GG2ESvW/HV/kyAR5d8CBSMaBofn6+cY5fQn8EEjDRqD+A6G83IPUph82byUm1eySjMRIO6pkNNNiXqOeM2/azgty5AQpN8qZ0QhtZtFuBmhhvx9ZVUxhw99uFdarsv67z4YcTC9l3Pvh/w022lJ29KeeUpu+dky6vtD7GmDZBQQXmYFo/fpiI1bX4VhqXm6DCMnySbNWcMg3ylN9VEh1nrxRktYqXCZSAIc/NHP3H++jF3ICNw/kt+otF72pVqmtDq9xlBT8KY4SCGF93aHsKT+kwL1Y8u9qnP17WcsKD2grDB9+L2XjqeRHloi5h0oGASF5sVMJm+fLdUNzWWxW+zlqlGx7cf1INP5iFNp4uC2/ACMV3E4MdA7KjrGbh5LfZKcEHVcrak0/CQBV7YEpkLPYc8cmJPJBnlhdTr8TReCZZKFQh2pMT8mvEje9zMIe4NhrH3KRyZYCPG458P7qew7kUSdUsV7ndV+RGiRTaqP7VXuIAxP9I7RoyCNl7DAuyA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR2101MB1092.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(82950400001)(921005)(478600001)(82960400001)(8676002)(41300700001)(8936002)(10290500003)(66946007)(4326008)(66476007)(316002)(38100700002)(2616005)(66556008)(83380400001)(186003)(6486002)(52116002)(6506007)(5660300002)(6512007)(1076003)(86362001)(7416002)(2906002)(36756003)(66899021);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fqSWqSSzHFwQWZ4yMEUHfnI/0Mgy6PrEHgNZaoAg0nSm6MjEAHyyy9EVPbCBUfECwds449RK3noHNjxpPEx46w9RiLmWfLrXIRJclUb9BV668vQ38vmPdkZUdZ41NtmUKqvhPB4emyD5fPvZt2lnKW+AW3XT0i7Qyo9Q6kAByUWV5M7VjH3RsSZXfnQPahdlkzUekRtwNtqfhjRmsgHXaxBxFfaXYn5DJfEwTKXlQ8bI/JMJYNafNIyt2io389XGY+A+z6SgAR8w9ZYBGAD/+3w6Lq9DOBs4isu7WM9ydjk3EBdrX7lY2mUSp8xGBkbuYOaEJzQ0N5N/aUcIBauKFLCM1YtSenizmJ/EX4o3/MBHMVqn3OQYpRMZAdEURP0UdsDPafV2nS0+K9wwsd4GluJE0AvxqZflcvDx4qvEGiOAyqPxLmkMMA1deyytuZuq7BrwW12t2CI2prLTTcSXew4yazVpPYGg4fUjj1rTvkT3dIRlKO+VgQB1YOtGJ5ofFAXP4Z814bVRTLr3L3lTefzSUWWuSgxioV+YAh+y+YjJu3cnWZRdvkQSxEGH7UPRC0GyQ2vmMsAWsQWyeLzpqnf0EsfEWzMsGDqfp8ohO1uguZnsnEwyRLfGN+lW3Lm24mXfYwe7+pBMPsqD+10mavTwNOHCh15jGTDOg2xVau4SfSX5HEpgxWcJ/WOK5+2x2p9ZUC79gas/k/DdMEMFBORfNH/ue5U89uiY9U1CNHa03rC8GeD5JCGl5hQLsQy4qc7/PyIlhgA//vXDqhNtbE184urq8hGDTiqwSN6LtNxQwPpvDf9+yZjm2++SJxuBteFklwchu9R2iKK+lb+b5Sx89vLAVcGYyIw4uF8naBv+OrKbd4DrDmDVQ9/BW32CLmCivmGPmIA7LKY+KO/MFZ3oQlCm70E/90WcIzXW/1jOUVHryAHoJ7FRNg/Yc+B3QkWf7VltfFL/EHMW8iT13W+/kMb06KrG+USa4lLOF6felVnblP5tqXys/xjW/m72vJYbveTUMW+yG/Nn+27Dw2hLsys9nYKj/GL8uvwcdpK2o3cEwpEFPEgC41r5xvO1RHJsV2YvH/gz8g+90hFVB1xVbe/rWkOjxYnnCkmeA2ZZd5SJoMFaZppoZZzpikpmDEAR0eRUxKUo95xKunc2B7qad+Uv85PtGMvViqDQgaYeyrRykz+6s2y3iUGVZOpLeaHaNWL+SlymgxXyYOnlatp4tANNXouF6WwgK6AGT0asfJyWiMh9wrBRrt79u2504OscG+ygBytYpg2dHx1Ml/ZKSyQ99v8FuBdq1WWA1MAYbZA2ykd3XNiTE+ilb7Z1RJKsI6P/rv7piwQ+AHUq+vOwzMYw+S3BM3yJoZvovIXFTG8nh/oEqj2JzTQ91hjLo5nwFq8q1G0Tx9EgMVlGWnntlYUbYc3acI8ajEc1zxTQOX2rZfXCxwEKbrhGkQx6kvQVTeL9BpnZRG6tAhAM3q4CoEJdWyt5oHn3kwYjb0F/pv1tXYJQVlL1z1FWKyr7zRf/LssNHuWSaZ2uaftwcyzxaSEBVZphaRDIn09WV9zFyXtBApNGkEkH4NZUjUZPey9jOAtODmyLy5F2vcXOuczpHn2/SUl0WWSfpGXD0AaOOCplYvsdcExvILfnhbOW X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 51762adb-e74f-476a-08ef-08db6d5b5e87 X-MS-Exchange-CrossTenant-AuthSource: BL0PR2101MB1092.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 04:45:39.9982 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PiboE7hAa4C4GRVPTbTLdacuh6GorxpMadwtxmFK3yIzEtgDaO4QDOnpvB6kBj2CP7L5dS0E5G6m8BoSov7+MA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR21MB3382 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org When the host tries to remove a PCI device, the host first sends a PCI_EJECT message to the guest, and the guest is supposed to gracefully remove the PCI device and send a PCI_EJECTION_COMPLETE message to the host; the host then sends a VMBus message CHANNELMSG_RESCIND_CHANNELOFFER to the guest (when the guest receives this message, the device is already unassigned from the guest) and the guest can do some final cleanup work; if the guest fails to respond to the PCI_EJECT message within one minute, the host sends the VMBus message CHANNELMSG_RESCIND_CHANNELOFFER and removes the PCI device forcibly. In the case of fast device addition/removal, it's possible that the PCI device driver is still configuring MSI-X interrupts when the guest receives the PCI_EJECT message; the channel callback calls hv_pci_eject_device(), which sets hpdev->state to hv_pcichild_ejecting, and schedules a work hv_eject_device_work(); if the PCI device driver is calling pci_alloc_irq_vectors() -> ... -> hv_compose_msi_msg(), we can break the while loop in hv_compose_msi_msg() due to the updated hpdev->state, and leave data->chip_data with its default value of NULL; later, when the PCI device driver calls request_irq() -> ... -> hv_irq_unmask(), the guest crashes in hv_arch_irq_unmask() due to data->chip_data being NULL. Fix the issue by not testing hpdev->state in the while loop: when the guest receives PCI_EJECT, the device is still assigned to the guest, and the guest has one minute to finish the device removal gracefully. We don't really need to (and we should not) test hpdev->state in the loop. Fixes: de0aa7b2f97d ("PCI: hv: Fix 2 hang issues in hv_compose_msi_msg()") Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley Cc: stable@vger.kernel.org --- v2: Removed the "debug code". No change to the patch body. Added Cc:stable v3: Added Michael's Reviewed-by. v4: No change since v3. drivers/pci/controller/pci-hyperv.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index ea8862e656b68..733637d967654 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -635,6 +635,11 @@ static void hv_arch_irq_unmask(struct irq_data *data) pbus = pdev->bus; hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata); int_desc = data->chip_data; + if (!int_desc) { + dev_warn(&hbus->hdev->device, "%s() can not unmask irq %u\n", + __func__, data->irq); + return; + } local_irq_save(flags); @@ -2004,12 +2009,6 @@ static void hv_compose_msi_msg(struct irq_data *data, struct msi_msg *msg) hv_pci_onchannelcallback(hbus); spin_unlock_irqrestore(&channel->sched_lock, flags); - if (hpdev->state == hv_pcichild_ejecting) { - dev_err_once(&hbus->hdev->device, - "the device is being ejected\n"); - goto enable_tasklet; - } - udelay(100); } From patchwork Thu Jun 15 04:44:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 1795193 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.a=rsa-sha256 header.s=selector2 header.b=Q/oRgxGC; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4QhV9h5sBmz20WR for ; Thu, 15 Jun 2023 14:46:16 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243114AbjFOEqN (ORCPT ); Thu, 15 Jun 2023 00:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243116AbjFOEqA (ORCPT ); Thu, 15 Jun 2023 00:46:00 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2126.outbound.protection.outlook.com [40.107.101.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D44AE2684; Wed, 14 Jun 2023 21:45:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j5ekCV6HJ6iJpq17Iu64gq5hMA+iAej3fFDdqXyC5jXwLy0fFAijGSxk27o2tfRtYZMfNSX87N4D1jrEirWPB1ElmJkNOKPtJUPaFgb6SJ7NYD+53dAMFhDMfirF0CkoneAcnM7kaAwuf+9Cv/0mjOCdqpAOvJp3c/Bbpca5ti5dNvtyfKaACYl9T+QDZirYMeR2G3n1N3OglxtUei1KVKRctjfeCi1zcjmrfyBgxaV27hGWLDjmbA1GEKCSxOgh8atiT+Xcv74TVGLHP0Pftzpgv4I3uqaatVMPGEzBG+qsQcFAz2n1XkGN++uhH5ZZFkc46SoVXMFq7v2WBYs/Uw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OJ+hFj+wB2UrTRLkTBYFcRNUhFnHauukaAsov7l2qwU=; b=AZpcBatpnTVjYjB4SoSawINz2IqS8+VXBg9dWgZayjRw2UwEBzOowRtgB5c9dAe30ZBxuSXF/bSlJk9T/nz6gEUvaREBYjI9EZTxcNkpy+0ClG+dXzcqi0MPvA1+yi8aWpA84GcUJS2XOiLTtPymjiB2gu5BAF5DujaKJTWKWji0qWhQRyjjU8uj2TTvJEqZhpGsIw+gntNe2OoM4mTSxk3yvXgiWblksE9vw58VE0CIKC5g70D6w6boCWTp04pwfplI5yIBThWmBtVJ5h0N5A8KsCcDSZDRRsNNPYYo3ZIsDJr9NySPkl7GqeAQz7TINTwWxqgA4KmPYcL+clX73A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OJ+hFj+wB2UrTRLkTBYFcRNUhFnHauukaAsov7l2qwU=; b=Q/oRgxGCtWExhTY3uq8NBCHnUrlsVNFrkbEKBUxUD7GYpVE4hWnTm0h6dy922F4lW/m7gHRKdfeXqWzMDxBGHKlaI7AgpKgKMQFV0l85GgiKlGIyexAOonxZClAJXe9m6FPSLvE3eSk5CRSqLVlSgWYzTZHB3vHQPh4TJqvmLzw= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) by PH7PR21MB3382.namprd21.prod.outlook.com (2603:10b6:510:1de::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.10; Thu, 15 Jun 2023 04:45:42 +0000 Received: from BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682]) by BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682%3]) with mapi id 15.20.6521.009; Thu, 15 Jun 2023 04:45:42 +0000 From: Dexuan Cui To: bhelgaas@google.com, davem@davemloft.net, decui@microsoft.com, edumazet@google.com, haiyangz@microsoft.com, jakeo@microsoft.com, kuba@kernel.org, kw@linux.com, kys@microsoft.com, leon@kernel.org, linux-pci@vger.kernel.org, lpieralisi@kernel.org, mikelley@microsoft.com, pabeni@redhat.com, robh@kernel.org, saeedm@nvidia.com, wei.liu@kernel.org, longli@microsoft.com, boqun.feng@gmail.com, ssengar@microsoft.com, helgaas@kernel.org Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, josete@microsoft.com, simon.horman@corigine.com, stable@vger.kernel.org Subject: [PATCH v4 3/5] PCI: hv: Remove the useless hv_pcichild_state from struct hv_pci_dev Date: Wed, 14 Jun 2023 21:44:49 -0700 Message-Id: <20230615044451.5580-4-decui@microsoft.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230615044451.5580-1-decui@microsoft.com> References: <20230615044451.5580-1-decui@microsoft.com> X-ClientProxiedBy: CYZPR17CA0016.namprd17.prod.outlook.com (2603:10b6:930:8c::25) To BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL0PR2101MB1092:EE_|PH7PR21MB3382:EE_ X-MS-Office365-Filtering-Correlation-Id: 34955f6d-e823-4685-a080-08db6d5b600b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: m8pyY0EewRj8e3wtyiDfrbJhR770R7vkMOo8IPJ/DFgVoIflx8PWF/z6SqD5BsZhlxbRG/uPY7asTDtLIZdJA4ghDZydEM8Woq2lxTL5lk6mlXvXVeUQlgoZsKKcyjVhyAHPp4uXTvZ7LW65ATDrvuSRMW9bqFGiTlcru626bONd1CjE4MwgtYq4NlmfTVOVrD310iVJUK1eBN1CnCxpQ+XdvDCzBXIo/iIqsD4oPm676ZmDNAiyloSjZTXQEWyZ1l2OSQIIKhAESjZ7438xGehQYviPSjJsXBgA5nf6tgNpZOvZMxo9pnOJ1pDxsTqYu8JHYN13K3Y+upq4dFZJKSbMOcuGs5rmiXd98jherCmvBQsbWhdV4Y/m8qWWMrv8TsV3wVmNC1PCSSjugsk0oL84B/mfwkWKcPMwipfOe30jbiiYOZcOHM0UJX2GkGJJa1Xq7MMXwg5KyS2KDnnJege5RfAANxdW/czk0xd9PdhVax0o1OllQdg2x9UPjrXcdBv9nHJ1emCfnYYlZ+rNjFa0ibDVYqBAm8g8DksTvWsW6K/gl4WOjVtZktaSU9SwI6F2NrXPU1BqLsKc9Q7lnDQVj0G0GiMMetq1xlI4cvPByxqBOo/czgYqawL+t/hzJlaYXiawaN2VOUxdmzgYRw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR2101MB1092.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(82950400001)(921005)(478600001)(82960400001)(8676002)(41300700001)(8936002)(10290500003)(66946007)(4326008)(66476007)(316002)(38100700002)(2616005)(66556008)(83380400001)(186003)(6486002)(52116002)(6506007)(5660300002)(6512007)(1076003)(86362001)(7416002)(2906002)(36756003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kystszMxTz2sp2yzeIfJg9MvDVoNiDQWtiOhLyblR5FTPeDeLiAuD9jLCj/Mk3vLlNHvBH1uBeeibtN60L1inZx47f3B5ibdupTtTI5F2VKdsbkqjdWBdDgtmBnhd7pTiPX3GSH5XhshMhoRNVBMhbOPV3gc8/0jnlZQQUwFWP+PCGUvpUh9mK6kxY7XV4xaw+PUO6FMIu5oh5e8FRxSdh7yJZjZsVcKKsRdxArSgradFubqtHQax8jNOxs2e7fRTdjsUfQ7rhprIIsoyuyn/XzBa/LTwUL5zGHjuWn7Mkq5FIAv0lzNPUHz77zQ4z3zXqRMEoNzZYqX3gbSgCawUorfH8Y6knhD5MNWCYgZK/nyF/uFHrUBnVW5J8FF+mpSypOxrqKsHkpey1x36qDpo03ZWWZjI+Tu3cRk/Sl4ROa3mrYBQoh+rFp+V0qeS86S6fa/Q084ubLkwNy2ASFdqUEf0m7tmxCBpew1AsS9a11BYPFaPffD7dFG3S/SOVTxfwlMh1ItQpmEWWt112EBXzBCA1IXc3uq1Ns6mZiE3bNKagkgr/eagkD1jpWuWzXioqG2FH774F01atONI9AzEBSlYMqGZgM3/el61R2HT5ryG7/7CaKYQ23mmZV/CJLM9InB1mQbNTYkD+c4WzEETH0xbQbtbDZCu0msTxsd/4zMEKoVf9xyN4kU1aBQxvv4Qp7M9aKYLge6KsZ3DE5L/MZgHZ8v+dmNea8RK+Oohm9IGt7xVOW1mdy667/w+/TU22ybDSHiJBZOJmkaNk9Qww1RLaXnumdgHoUT/EXaHW2k1u1dUGDTA6qFzTuPnMqXQd8QOlmZ8qNVK7ILHJvZaHiq3ly0bW3bzp99mcyf/X+iTxgr00/zY7xJ9+i7/2xXq3l1jLeJruNrctukL0j39lKhhRxxSEvRlQguW8vkKcuXHQgvghOdNvdguhOuGaPmSdGa7SqhvhWGJNTom4btdgncuxbEqh38XTJGIBDX3i3jVBMfK7QgTPKBqu7tmf75mQoPUPth973qmnB4RH6Le9hqIrTg25evdPuVMW8sh9AgIbbsS1i0bPy3mi/FR4bj9+mkC8NLSk6r9LtHtENxzG9w+PxvznV4+z3EdaLErZygVWKNl/91miTcGLub9Y1U2gnwNHyeWQzP03Dcp8lVqhRMZQen1cCkLe3k7WMUB3EzWH6OsxTHFh/myVQ58TwZCqwgnVPFubVHmhdR7c2MNQ+M04qt6RCrdtlMsrBFzXaU81SkFweUN2trgykGXEP4GBV7zCJLmvJWkLqDxe6fQPcO0qtBvqDfAjxfgtdl5NBuPMEMoeYkV6LhszmBruCO87q8OJWr7Ckb0PdqAol9NtIX2qN82Lrgm5gBcKEUoUw/efx7o93MACoXtmqsx05zCRlXd8GyFud1LbvRcnkE05WqDnzErf6cPTzoALtOWzpR0dO2a4DVT9KDyKT3KfrhcTb3dPymK/ANG08s6s0vQ8doiA6YwwaljCeOvpNsfkr1aZkKJPJY2t3S1VMNYVIOsAj1J6WH+tCq9aOEiCLsNByPL9Jr2caF3rmSF2v2Lq3762fm4pelxVF/WFjJ/iLM21VUMsDWZR0PCFi38azbvmxTGZHwHT3QF3PUy6E/mJV4ezazUa4dX5uBlBGGhR+g X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 34955f6d-e823-4685-a080-08db6d5b600b X-MS-Exchange-CrossTenant-AuthSource: BL0PR2101MB1092.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 04:45:42.5075 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CTRX6MXnBV/p1FtTD3T83mqkZYxmTo7xdXWVvYOEwESr6tMj8jSdysTl9hX/g99KtkEEczAKSB/G2yCH3JW+5A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR21MB3382 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The hpdev->state is never really useful. The only use in hv_pci_eject_device() and hv_eject_device_work() is not really necessary. Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley Acked-by: Lorenzo Pieralisi Cc: stable@vger.kernel.org --- v2: No change to the patch body. Added Cc:stable v3: Added Michael's Reviewed-by. v3: Added Lorenzo's Acked-by. drivers/pci/controller/pci-hyperv.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 733637d967654..a826b41c949a1 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -545,19 +545,10 @@ struct hv_dr_state { struct hv_pcidev_description func[]; }; -enum hv_pcichild_state { - hv_pcichild_init = 0, - hv_pcichild_requirements, - hv_pcichild_resourced, - hv_pcichild_ejecting, - hv_pcichild_maximum -}; - struct hv_pci_dev { /* List protected by pci_rescan_remove_lock */ struct list_head list_entry; refcount_t refs; - enum hv_pcichild_state state; struct pci_slot *pci_slot; struct hv_pcidev_description desc; bool reported_missing; @@ -2843,8 +2834,6 @@ static void hv_eject_device_work(struct work_struct *work) hpdev = container_of(work, struct hv_pci_dev, wrk); hbus = hpdev->hbus; - WARN_ON(hpdev->state != hv_pcichild_ejecting); - /* * Ejection can come before or after the PCI bus has been set up, so * attempt to find it and tear down the bus state, if it exists. This @@ -2901,7 +2890,6 @@ static void hv_pci_eject_device(struct hv_pci_dev *hpdev) return; } - hpdev->state = hv_pcichild_ejecting; get_pcichild(hpdev); INIT_WORK(&hpdev->wrk, hv_eject_device_work); queue_work(hbus->wq, &hpdev->wrk); From patchwork Thu Jun 15 04:44:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 1795194 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.a=rsa-sha256 header.s=selector2 header.b=FI53SokU; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4QhVBJ2Mbcz20WR for ; Thu, 15 Jun 2023 14:46:48 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243145AbjFOEqi (ORCPT ); Thu, 15 Jun 2023 00:46:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243186AbjFOEqG (ORCPT ); Thu, 15 Jun 2023 00:46:06 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2126.outbound.protection.outlook.com [40.107.101.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A5D2273C; Wed, 14 Jun 2023 21:45:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WdIJxiE2Ux8r9YSheMNYy9315ZS5/Ti41WRsHQXPQOofo7U2eLihp3NxS0XglFz5IQSJZ26MuMrs/5Kx5GxPqV0TnDQ4Pz/D07QsC7zC4Ah3CRjx8rJJRarpHO7jIYIBQV/PSJmKgp6fAIcGU/Tb4X6zuS2vIo2A5bFIPcpMSE2AETT/8x9FwD6fdFdbft+e8u98I3ypnwMY0Cgc97tadj7GJDdNwFZuA1H5p0kpDm3pxDiqAnHOARTluVrAHu7m4B9bliWA65sef0JwyzZ37sFzrjMHZmaKCd5FWv0BGpT2RBdW5hdoC9FtAxU+0/f0NectqeLzXM+5ST1tnPAtag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fk05MFCoxRZll3A/9+GZob5QTR1AZHT/r3Axt7pKpss=; b=PUsSDeFJU5mIJ1XIjg4Hi+MjWb5+iu4HGqBRbMFwyfBmFo+z/k5x+X00Ktl0w8vKHXwXOrBV961r2ZnfQL/XcHtDP5lHR8b2x+Ng6NjSFCvRv3RPu8azRZeYFUGn+8GmN0Mkq/KKsUY1s3R36Js+nuvLhGyGr58s2IgjabdLDxsYkjkpvR+WJ8pWKRJDEhbs6UZvV/HLgkWQEpJcjIswAMbjoDstPSkv3wZk/UleMIrjhOgNm8IvhIA+4su2TmbFVbccg9HopIhNyIY3RQKTfDMWYsktVt6lUFxoSe9Yq83pTrpdSm+sTOPxHzD435CbZimX6b9haDIYmdGW4WNuiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fk05MFCoxRZll3A/9+GZob5QTR1AZHT/r3Axt7pKpss=; b=FI53SokUMM9cpctf4KwwVK0rzTook1SE+J80Tp8gf1uQe2D9PjlIOC9kUoxCMeOVTy8haDJD+NiqVelAMZYKqXInCmW+4AXSTeoOq1R1DLlUf+BL4D9p9XEZqt9ctjrFsyyieL0kLPmSBbhk0u348naaOA8t2PTXi5QCX0DX3X4= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) by PH7PR21MB3382.namprd21.prod.outlook.com (2603:10b6:510:1de::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.10; Thu, 15 Jun 2023 04:45:45 +0000 Received: from BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682]) by BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682%3]) with mapi id 15.20.6521.009; Thu, 15 Jun 2023 04:45:45 +0000 From: Dexuan Cui To: bhelgaas@google.com, davem@davemloft.net, decui@microsoft.com, edumazet@google.com, haiyangz@microsoft.com, jakeo@microsoft.com, kuba@kernel.org, kw@linux.com, kys@microsoft.com, leon@kernel.org, linux-pci@vger.kernel.org, lpieralisi@kernel.org, mikelley@microsoft.com, pabeni@redhat.com, robh@kernel.org, saeedm@nvidia.com, wei.liu@kernel.org, longli@microsoft.com, boqun.feng@gmail.com, ssengar@microsoft.com, helgaas@kernel.org Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, josete@microsoft.com, simon.horman@corigine.com, stable@vger.kernel.org Subject: [PATCH v4 4/5] Revert "PCI: hv: Fix a timing issue which causes kdump to fail occasionally" Date: Wed, 14 Jun 2023 21:44:50 -0700 Message-Id: <20230615044451.5580-5-decui@microsoft.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230615044451.5580-1-decui@microsoft.com> References: <20230615044451.5580-1-decui@microsoft.com> X-ClientProxiedBy: CYZPR17CA0016.namprd17.prod.outlook.com (2603:10b6:930:8c::25) To BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL0PR2101MB1092:EE_|PH7PR21MB3382:EE_ X-MS-Office365-Filtering-Correlation-Id: 922d3d6e-6079-473f-4c9e-08db6d5b6184 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: B0dhveHBDO6jtoApQcdMvuEgVSzUfGxk5edtyeQwOgNKJa7ZQAe4Ve8RW1iJUctESvJFFAZ9nvrs8fdod2FoUxDsH61CrNWjjDJqShs4+zfS4lm7DN/Nt3oEXosfA6bVBwOHh7//BoJ/wdvGIgfWyCGAaH/OD6M1HrZbnk/m5OJ38lnJPt4G76hcaPt+FWZWWvvMCeQ4gQMVumvuPwvdp7IFGuMHUXpxOSK8k6rfukuseSumr9rpg8VE7mb16cXzipLgsr836TneIss3kBcD0fixxplcy+9xLHGlsw8mv25ijjrYhD1lLw4eg+Obo/e1/LH316bkj4GDKm/rDP+nj28kvkHLO4NNu1dVj/tMXIPVDgVrcEPcE7xnUTBJEeB976+TSJ21jlANjauauNTOXuA+fDd7r7EL2xzyWie4VcCGxcpbBQlPiP82Y0nliSJzCIxESPkfCMjgBWVUCgBsH0bxnOgHATKPbOKUXBlKcOwj8w90XwtFkk0h+7PvRocJ6X016hXZukvsKPQsNiupxew70DiF5Vqd+mbkOci43i2YNJ0JwtCNtjkc/hVMiDxUHngrbC0sZ/LC7AYG2xrrBs7QAn9zeqo5iDUkv1+2DgWI6Q+sJc9S9dAMyqnnMQ/P8l8A+PRdzxde40xq91Yzig== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR2101MB1092.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(82950400001)(921005)(478600001)(82960400001)(8676002)(41300700001)(8936002)(10290500003)(66946007)(4326008)(66476007)(316002)(38100700002)(2616005)(66556008)(83380400001)(186003)(6486002)(6666004)(52116002)(6506007)(5660300002)(6512007)(1076003)(86362001)(7416002)(2906002)(36756003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A3Ffu1AIxQylLe1op6OQhu1vx9aWHAsXlzvgUxWBmTfstXfZmg0LlQgCpslN1Zm04eSy/hnzz1gJFvnZTMMXyMZ5E6cb2nU8Rh1M2yKgQrd42I1b0SqMUAztik2g+HezfIaZsnuIXLPCooNPgOsPMvgJJ+g/MQAPwXEh3ixt+nVA7ZFpUqaYoSstuDqUy2YV4l2udkawNAh4pMkXBzgIv98b3NdDoq8dDnEnyggEFeeXV22jVcXB9pDrR2SY0duwpqHJMU0UzJBgR6dlD97O4iSLqM6JrzhSR9ezBpcEo1XLzJ1uR90qhcsFrEOjZxLzTazIadUDcCZbg1oSMxKubGPb7l2XCU6sHUnNz9Tyj48z8vbCBgfb8GvnyFtAPWLlWLdVwRudrUjM5YGDDsR37Q3sarLWu4QU8ZdZh7EYoXd0vI6bjW3EOWnm1VCB1KwHROIn2u74XcW8zp32XK0jomQXAvCjd94MSXUPBDO1YYUefY5cIeU7JcZyNB7tgXPh7B9dsHA5yBsJVFvsOiOhowWzxrpxojw25+sDoyygEb6ab3RxOSja8MFfNUNEb95k+ilxAAeLFrDbaNgL3moT8JK3O+kcvsZva3ubWOPcvPHfZzMKFMvcXwtut2XbbQFvIbVVVek7ZMb5tlpvMxqp+oqwH124yvZTkdg896FHWxRqyslQT4N0GN8WFaSsX3hki9u2z03rvgkxVyYCB5nPwCobMxk4bPSfQxL9rGc5CtSjoza7bjrkttxpQGDsun0UJXkLc3HWYv18kY3SlOJPoyx7z15krR4Tfj0I1Kt9Wwbw6dJGnNRsjRlUE+GqdYBRFcygu5n7mJb8mKdT2mpVM2/KuBTxA5NBfCmPgkQ9XMKbxW4CFBYW+TAkNFsW/ltA+/wKhok5ZluIaqTv218ywHUzWp4lDL5Sn9Nf7WuGHeZR59oajXHOcyrkgdzFJ+lqFkjXsTuzu7XPFjB7g2kSvwy5jjNq8BUY6/IoQflBmcgBFpWlnhRN3GKCg84BgcCrs3I35WVhuJ+kxPHPh8624/6W3gesKrUNVYkE71JCww9Zi70/OfDwUHklrAZL+3QmdfliUTo3byt/uPSyG7kxXYyhJz4ytx7jE5+Z1ls1UFBlMFkJ3wBNEoeDfszSPLcOOUH3XJnHEhsjHMsSTlSG0XjReIbr/WNDeX91LMIHKkKQQHEcm2M5vBS0/lxatBBvfQtLqF/cjZzF6mrPlqrztDhSou9WOv8kaX61EQUibIwqm2zqV66Ko3K3A9aL+jnPZLrj9iqhNaonkfodK6tg7RKK1jvVDG0vCjl0VFobhD4bYL/mu8ZUoq4AUzXywLqxV9m3KIWNodofTyIC3H4oRFchmCc+1rDi6x2dxKpjALa8zCf2j0B089xB4BFdGrr9QuwVk18ldIckoyGUjn/ttuSQg9Z6m4Hd1UFE0wsK9eUqEZCpuYi1GefB8BV9c9io/bEHgYfJKGf/EuJy5cX5a6ZlQU/DNfzm1sxWdR431R5EakACmawZNiJri7ox6oBvolcBZ5OpqUk5THBuzgjILzIDn9LvQ0TbVC5yKWuabR5JUS8NAgCUstI81vk5i35TTEaXj3jhQ0WTKwMmp/7jJCpC1I42cm8vIOV+MjAVtsP4+rkeTYB/3yONX2qDoXtT X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 922d3d6e-6079-473f-4c9e-08db6d5b6184 X-MS-Exchange-CrossTenant-AuthSource: BL0PR2101MB1092.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 04:45:45.0344 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kOxOoJlYh5fZPvhNCqeiyk+YNPrzr0BCdAvvwUhnxCVOqga6GnIkJV9U+Q8JQeWmxeU7QrKNZ5XnyStmhikWRA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR21MB3382 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org This reverts commit d6af2ed29c7c1c311b96dac989dcb991e90ee195. The statement "the hv_pci_bus_exit() call releases structures of all its child devices" in commit d6af2ed29c7c is not true: in the path hv_pci_probe() -> hv_pci_enter_d0() -> hv_pci_bus_exit(hdev, true): the parameter "keep_devs" is true, so hv_pci_bus_exit() does *not* release the child "struct hv_pci_dev *hpdev" that is created earlier in pci_devices_present_work() -> new_pcichild_device(). The commit d6af2ed29c7c was originally made in July 2020 for RHEL 7.7, where the old version of hv_pci_bus_exit() was used; when the commit was rebased and merged into the upstream, people didn't notice that it's not really necessary. The commit itself doesn't cause any issue, but it makes hv_pci_probe() more complicated. Revert it to facilitate some upcoming changes to hv_pci_probe(). Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley Acked-by: Wei Hu Cc: stable@vger.kernel.org --- v2: No change to the patch body. Added Wei Hu's Acked-by. Added Cc:stable v3: Added Michael's Reviewed-by. v4: NO change since v3. drivers/pci/controller/pci-hyperv.c | 71 ++++++++++++++--------------- 1 file changed, 34 insertions(+), 37 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index a826b41c949a1..1a5296fad1c48 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -3318,8 +3318,10 @@ static int hv_pci_enter_d0(struct hv_device *hdev) struct pci_bus_d0_entry *d0_entry; struct hv_pci_compl comp_pkt; struct pci_packet *pkt; + bool retry = true; int ret; +enter_d0_retry: /* * Tell the host that the bus is ready to use, and moved into the * powered-on state. This includes telling the host which region @@ -3346,6 +3348,38 @@ static int hv_pci_enter_d0(struct hv_device *hdev) if (ret) goto exit; + /* + * In certain case (Kdump) the pci device of interest was + * not cleanly shut down and resource is still held on host + * side, the host could return invalid device status. + * We need to explicitly request host to release the resource + * and try to enter D0 again. + */ + if (comp_pkt.completion_status < 0 && retry) { + retry = false; + + dev_err(&hdev->device, "Retrying D0 Entry\n"); + + /* + * Hv_pci_bus_exit() calls hv_send_resource_released() + * to free up resources of its child devices. + * In the kdump kernel we need to set the + * wslot_res_allocated to 255 so it scans all child + * devices to release resources allocated in the + * normal kernel before panic happened. + */ + hbus->wslot_res_allocated = 255; + + ret = hv_pci_bus_exit(hdev, true); + + if (ret == 0) { + kfree(pkt); + goto enter_d0_retry; + } + dev_err(&hdev->device, + "Retrying D0 failed with ret %d\n", ret); + } + if (comp_pkt.completion_status < 0) { dev_err(&hdev->device, "PCI Pass-through VSP failed D0 Entry with status %x\n", @@ -3591,7 +3625,6 @@ static int hv_pci_probe(struct hv_device *hdev, struct hv_pcibus_device *hbus; u16 dom_req, dom; char *name; - bool enter_d0_retry = true; int ret; bridge = devm_pci_alloc_host_bridge(&hdev->device, 0); @@ -3708,47 +3741,11 @@ static int hv_pci_probe(struct hv_device *hdev, if (ret) goto free_fwnode; -retry: ret = hv_pci_query_relations(hdev); if (ret) goto free_irq_domain; ret = hv_pci_enter_d0(hdev); - /* - * In certain case (Kdump) the pci device of interest was - * not cleanly shut down and resource is still held on host - * side, the host could return invalid device status. - * We need to explicitly request host to release the resource - * and try to enter D0 again. - * Since the hv_pci_bus_exit() call releases structures - * of all its child devices, we need to start the retry from - * hv_pci_query_relations() call, requesting host to send - * the synchronous child device relations message before this - * information is needed in hv_send_resources_allocated() - * call later. - */ - if (ret == -EPROTO && enter_d0_retry) { - enter_d0_retry = false; - - dev_err(&hdev->device, "Retrying D0 Entry\n"); - - /* - * Hv_pci_bus_exit() calls hv_send_resources_released() - * to free up resources of its child devices. - * In the kdump kernel we need to set the - * wslot_res_allocated to 255 so it scans all child - * devices to release resources allocated in the - * normal kernel before panic happened. - */ - hbus->wslot_res_allocated = 255; - ret = hv_pci_bus_exit(hdev, true); - - if (ret == 0) - goto retry; - - dev_err(&hdev->device, - "Retrying D0 failed with ret %d\n", ret); - } if (ret) goto free_irq_domain; From patchwork Thu Jun 15 04:44:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dexuan Cui X-Patchwork-Id: 1795195 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=microsoft.com header.i=@microsoft.com header.a=rsa-sha256 header.s=selector2 header.b=T3Yhz42R; dkim-atps=neutral Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4QhVBL2QxPz20WR for ; Thu, 15 Jun 2023 14:46:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243314AbjFOEqq (ORCPT ); Thu, 15 Jun 2023 00:46:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237024AbjFOEqI (ORCPT ); Thu, 15 Jun 2023 00:46:08 -0400 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2126.outbound.protection.outlook.com [40.107.101.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2D4B26AA; Wed, 14 Jun 2023 21:46:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Bnt5tHR+LSy/uRf2edgSTHqzcHAVa0KMcAxgcUSPvasbopx3tfCHQXZc6HEGsf5tVuX/t8zwKeV5Nu1onp9uFcivlL4oByyOGhzVXxtluPg6RB+/eB+YtpwZfWCAhazCs+Uc3AynFLk4K/gCQNfXl+kgb5sYYKAHCAU0kFx4lfgc4qNli0/GfcoeF/ePf7s2P5vq6+n+1NhNoOHYDQNpkYCezzmt4+IWCfwjEmqFqzCwcECypj3JdKhPlM8mqVanxMro7w/MiI6T/aPUl1zjVcqvQ3EuVg7eEOsrW9tz/UPrxxsUgr8Tmn2zJI7L5Lv0Fic+WZwhECHJFHx7Aa2mWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=F7Zx7niXq8Uxv0dGXqhln9VHcxvy/tzKgtrgeexHazk=; b=OCsltjSAGJSpQ4k8MxwU7dxef0x9fPjvaomS5ljWh1vgBoywAHGn+usX3pnsQWtmQqEWHmAkhgDLnqWK3A0BFjDQxoXsjSSUiyxvK6f9eQoJ+C5RZU1kOHguhSLWUENzECborGR0smDIFECqhnAyFsXgyQ9kTAKuSmAfoNYk4ajRNHAr8z7UuClYHBeTMdCFmW4L7gdCSCc747uH+wyATy3krHEcnypDpSLBwjoPzIiWsmfI0T/Ooj/jZ0K0laYCJfiPNnYaXlk1/CFMzDq/Dq1Joh/anHi56c8B3AgZmQWoIv4VNoQfSkf6ArbvjR9y2IcUmkH0AX5aOrxYUAc64g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=F7Zx7niXq8Uxv0dGXqhln9VHcxvy/tzKgtrgeexHazk=; b=T3Yhz42R/4k4iem7x5GS15MzNORVVa/asasL+VwbufErtJ06B+ABOyKVFy+yN1ygjJBT42KR17dJPIkeNQiy9JhcWjfuAgQbHqgHnUiSSFdoh25nKdjCdH5/g5vdosFhvDlH8dnmaG1ZkFurVR4QEyreeB8Nb69vLGTLZLY1l50= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=microsoft.com; Received: from BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) by PH7PR21MB3382.namprd21.prod.outlook.com (2603:10b6:510:1de::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6521.10; Thu, 15 Jun 2023 04:45:47 +0000 Received: from BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682]) by BL0PR2101MB1092.namprd21.prod.outlook.com ([fe80::e673:bfeb:d3c1:7682%3]) with mapi id 15.20.6521.009; Thu, 15 Jun 2023 04:45:47 +0000 From: Dexuan Cui To: bhelgaas@google.com, davem@davemloft.net, decui@microsoft.com, edumazet@google.com, haiyangz@microsoft.com, jakeo@microsoft.com, kuba@kernel.org, kw@linux.com, kys@microsoft.com, leon@kernel.org, linux-pci@vger.kernel.org, lpieralisi@kernel.org, mikelley@microsoft.com, pabeni@redhat.com, robh@kernel.org, saeedm@nvidia.com, wei.liu@kernel.org, longli@microsoft.com, boqun.feng@gmail.com, ssengar@microsoft.com, helgaas@kernel.org Cc: linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, netdev@vger.kernel.org, josete@microsoft.com, simon.horman@corigine.com, stable@vger.kernel.org Subject: [PATCH v4 5/5] PCI: hv: Add a per-bus mutex state_lock Date: Wed, 14 Jun 2023 21:44:51 -0700 Message-Id: <20230615044451.5580-6-decui@microsoft.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230615044451.5580-1-decui@microsoft.com> References: <20230615044451.5580-1-decui@microsoft.com> X-ClientProxiedBy: CYZPR17CA0016.namprd17.prod.outlook.com (2603:10b6:930:8c::25) To BL0PR2101MB1092.namprd21.prod.outlook.com (2603:10b6:207:30::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL0PR2101MB1092:EE_|PH7PR21MB3382:EE_ X-MS-Office365-Filtering-Correlation-Id: 14c06635-3481-46b3-81cf-08db6d5b6306 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c1iIfLsktYnhOlqHlg9+4n8JInyJO99kN1X7kqJHWmJi252YEwgKf2hCokpK4x8oi5Z4lKIwOXiOlBlNOKjyRTHPOyZWWIgnFtep4vOZ8UHuvZTi/ysK2mgXdVejhJ/4cPsxJofSqbIZKgcBdXqX5h7D1CdycyUwZFMb+OyIrPx9ZIQ9NfMNIaSGIbWEfN/hyjUahMdYAAfT7rzkqoNmbJpuEZm5zOKNaDzPvePUzQyb2k1pB5whcyPbrUPvU0F7tskRk1bHVkutnTnQ+rU3HRy6BSY1bvMQy4dDs+IgUxPkkFTXIZolg3FsiHaNLkBb7nMiJOkSg6QfiwGISSw+R03eKQcsKIkCD0e4S/9LqsqWewITizA+eG2I9hJb2u73Xo+PsZJ/xImc+WzpT70MSFWY2vL5blL2UrTugURYxPb7L1ywvTwGfE4itiyMJ4nMIizAK+oISXTg/4gPUnNfcL7laTfINtWWQwvpUfFYgLoMZX9d7MFjKvsijOy//Xy+cpssYNWJF5z7B2NUetang1hv6hZ1MvlvCdYDbW6WUn4ZQBSl/i8c68bL2U83PieYJQtst3jgesmhDj3W0+TN5c01ySWNW/I43qjB5NewOYnT57nlqUN8LW7mOs9zPJAmUpJ4jCgNrFSipMceZnQrVg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR2101MB1092.namprd21.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(366004)(136003)(396003)(451199021)(82950400001)(921005)(478600001)(82960400001)(8676002)(41300700001)(8936002)(10290500003)(66946007)(4326008)(66476007)(316002)(38100700002)(2616005)(66556008)(83380400001)(186003)(6486002)(6666004)(52116002)(6506007)(5660300002)(6512007)(1076003)(86362001)(7416002)(2906002)(36756003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: I8XSpMiT6ij6rpdCFrxem1WwzspJ+f4suPbapzp7/7LgA9WGOg2/6VJuhHG26lpS9UvUpabjQ6C8NhAbo5fkEK8927WMu2TYY4UAipLIr2jMFUD5H4acZLWb1J6bYWONfzSNONSrPmogQMIxiI56bRlEWxfLY21hx0uvpnv4vY3VgRDgurYwVno9V3D44t5Pkzd/gz5uq8ohSX5Me9dhtjqTf3B9MFXw3UtH5D0RuS+XmIH52FTUsJS4HnfSSRCzMoIpXvZXBf5ixR5+fFtldq405uf9lKF63R7/1Ho+pXv4CF8MREcd9zPRhiurkFIgnNr/94huAXLvxI/E9MMqfoxtAhCub5QqiM4NvnX+kZGWNNXZoOOeayyKihpfWrlvDmla8gnbyEyL3ESgGjOa5f3pvYql0sC5Cr71PbbeHq1SSUTz9eYMKzKv1apNAnkSkTSG0qy5BCT8JOfdsnLhyASwbn2qxpVEISVZVRvnplLg2tlj8ROur4ybcnA74vuaIkJveKnLAXcxYitG8J2qTljgYyFJV8gaNnTmsZqEap3y2F44wmWVGGT/iz2e+5mo9oXkfU9n523U/AHRiECi1/s4sqAaexv2Hnni4QwLz0pV9dii6gCukixzGqelk6XB3pM5oN+YZm+y4SdSDDepfr+vjBu691jmLKxmIDqkGIpn8lHgakhwOl+o+LkUZJC33hsvV5hGMQfIMKer/AvIGY4nhPAYAQDfX940R+NyRyEWte8sa0E1US7Yh1plHGgcPAH28itkOtUAU8uK2xmTljUW1+T/NK3lrGh2sThGn0cmehf6kGUR6MjLme41lcFZ3dheiGtaAUgP+j3+8yKYrQlkYw7duNiBZdITPkx/aTrw3W97I4kf7G2q95KakfnvCQUrQme88HAnnvBUL8Xlifxsg0aXudgK/FWIwgdtoMRqj33BcgcfueohAtwDdingE7KjKGqVeXQZd+5wmwG1NF0qQsPg1ZvNAAV9i8nxAQZ0lF064jjjTtXdbsK1RfkLw4o4TAtS/SmITr3g0Rvu9ngwnaiAmNGV1pGe1uUFxI6P6UAoZMxbo+7rojH28oR7aVEQteSZZatION49N1CFaxIL4JbY8q6kPJOn0zyO0O7pfv64b8iQi/5iSBd9LkC5GspMcHq21vpCUWLzSHunUTVNFT9DACFmC3Ud0mkPqCaQwfhQhCW0Xq19/xZ1oaYeVp/ta4cRM/0SXg1RKNG5VZxLA4m0RUUX5+Xw0ulh4nq0jAZ/3RrmcRaCg8XQyOrn7uPS6jFg2hWDAZuwCtB9dTWOYURU3Zkk4p3aFjVsmbhAaGwu7liJ2SS9ftbrEKVj4MnAELji8nq1Kw3je51m4o3H/bf/3Fk5bk0JOQ+5SvLi+podStt9/b4asmU1T8lqpFbG8IErA5iyKEZV9A4+3mFNJXXUhOrbdM8XsfKj4k8f64t/dy9q67vOMPHdb7nC+9NfppNLATP5hRNAva7NQ+qd9eYztgLrjAiFch2u+HyWXh+yhrheSuYoYFVlTz4OL1B2BUf0upgxMEWAJ77dxee0Ce1GoYFeSiwTC16ogX49GZZOuhd72m0VOIUDZIsl7dCY34+5uzKN5bmSSS0XUK7qfAap+a9RwhMJp1v33MG8zt+UWsJ75siyao50K5uL X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14c06635-3481-46b3-81cf-08db6d5b6306 X-MS-Exchange-CrossTenant-AuthSource: BL0PR2101MB1092.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2023 04:45:47.5327 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xtYAgWpq+0tExppT1n26qrdJebYRnHDDVG1X9K9t759ELAgfPqKOtXSmw9Fc1hlaNMUJACrH3gj/aEe8Ub1oAA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR21MB3382 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org In the case of fast device addition/removal, it's possible that hv_eject_device_work() can start to run before create_root_hv_pci_bus() starts to run; as a result, the pci_get_domain_bus_and_slot() in hv_eject_device_work() can return a 'pdev' of NULL, and hv_eject_device_work() can remove the 'hpdev', and immediately send a message PCI_EJECTION_COMPLETE to the host, and the host immediately unassigns the PCI device from the guest; meanwhile, create_root_hv_pci_bus() and the PCI device driver can be probing the dead PCI device and reporting timeout errors. Fix the issue by adding a per-bus mutex 'state_lock' and grabbing the mutex before powering on the PCI bus in hv_pci_enter_d0(): when hv_eject_device_work() starts to run, it's able to find the 'pdev' and call pci_stop_and_remove_bus_device(pdev): if the PCI device driver has loaded, the PCI device driver's probe() function is already called in create_root_hv_pci_bus() -> pci_bus_add_devices(), and now hv_eject_device_work() -> pci_stop_and_remove_bus_device() is able to call the PCI device driver's remove() function and remove the device reliably; if the PCI device driver hasn't loaded yet, the function call hv_eject_device_work() -> pci_stop_and_remove_bus_device() is able to remove the PCI device reliably and the PCI device driver's probe() function won't be called; if the PCI device driver's probe() is already running (e.g., systemd-udev is loading the PCI device driver), it must be holding the per-device lock, and after the probe() finishes and releases the lock, hv_eject_device_work() -> pci_stop_and_remove_bus_device() is able to proceed to remove the device reliably. Fixes: 4daace0d8ce8 ("PCI: hv: Add paravirtual PCI front-end for Microsoft Hyper-V VMs") Signed-off-by: Dexuan Cui Reviewed-by: Michael Kelley Acked-by: Lorenzo Pieralisi Cc: stable@vger.kernel.org --- v2: Removed the "debug code". Fixed the "goto out" in hv_pci_resume() [Michael Kelley] Added Cc:stable v3: Added Michael's Reviewed-by. v4: Added Lorenzo's Acked-by. drivers/pci/controller/pci-hyperv.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index 1a5296fad1c48..2d93d0c4f10db 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -489,7 +489,10 @@ struct hv_pcibus_device { struct fwnode_handle *fwnode; /* Protocol version negotiated with the host */ enum pci_protocol_version_t protocol_version; + + struct mutex state_lock; enum hv_pcibus_state state; + struct hv_device *hdev; resource_size_t low_mmio_space; resource_size_t high_mmio_space; @@ -2605,6 +2608,8 @@ static void pci_devices_present_work(struct work_struct *work) if (!dr) return; + mutex_lock(&hbus->state_lock); + /* First, mark all existing children as reported missing. */ spin_lock_irqsave(&hbus->device_list_lock, flags); list_for_each_entry(hpdev, &hbus->children, list_entry) { @@ -2686,6 +2691,8 @@ static void pci_devices_present_work(struct work_struct *work) break; } + mutex_unlock(&hbus->state_lock); + kfree(dr); } @@ -2834,6 +2841,8 @@ static void hv_eject_device_work(struct work_struct *work) hpdev = container_of(work, struct hv_pci_dev, wrk); hbus = hpdev->hbus; + mutex_lock(&hbus->state_lock); + /* * Ejection can come before or after the PCI bus has been set up, so * attempt to find it and tear down the bus state, if it exists. This @@ -2870,6 +2879,8 @@ static void hv_eject_device_work(struct work_struct *work) put_pcichild(hpdev); put_pcichild(hpdev); /* hpdev has been freed. Do not use it any more. */ + + mutex_unlock(&hbus->state_lock); } /** @@ -3636,6 +3647,7 @@ static int hv_pci_probe(struct hv_device *hdev, return -ENOMEM; hbus->bridge = bridge; + mutex_init(&hbus->state_lock); hbus->state = hv_pcibus_init; hbus->wslot_res_allocated = -1; @@ -3745,9 +3757,11 @@ static int hv_pci_probe(struct hv_device *hdev, if (ret) goto free_irq_domain; + mutex_lock(&hbus->state_lock); + ret = hv_pci_enter_d0(hdev); if (ret) - goto free_irq_domain; + goto release_state_lock; ret = hv_pci_allocate_bridge_windows(hbus); if (ret) @@ -3765,12 +3779,15 @@ static int hv_pci_probe(struct hv_device *hdev, if (ret) goto free_windows; + mutex_unlock(&hbus->state_lock); return 0; free_windows: hv_pci_free_bridge_windows(hbus); exit_d0: (void) hv_pci_bus_exit(hdev, true); +release_state_lock: + mutex_unlock(&hbus->state_lock); free_irq_domain: irq_domain_remove(hbus->irq_domain); free_fwnode: @@ -4020,20 +4037,26 @@ static int hv_pci_resume(struct hv_device *hdev) if (ret) goto out; + mutex_lock(&hbus->state_lock); + ret = hv_pci_enter_d0(hdev); if (ret) - goto out; + goto release_state_lock; ret = hv_send_resources_allocated(hdev); if (ret) - goto out; + goto release_state_lock; prepopulate_bars(hbus); hv_pci_restore_msi_state(hbus); hbus->state = hv_pcibus_installed; + mutex_unlock(&hbus->state_lock); return 0; + +release_state_lock: + mutex_unlock(&hbus->state_lock); out: vmbus_close(hdev->channel); return ret;