From patchwork Thu Aug 8 09:57:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: 15957197901 <15957197901@163.com> X-Patchwork-Id: 1970701 X-Patchwork-Delegate: ktraynor@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=163.com header.i=@163.com header.a=rsa-sha256 header.s=s110527 header.b=BGfxrKsa; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WfxrP1CPRz1yfV for ; Fri, 9 Aug 2024 05:27:17 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 63E3660B65; Thu, 8 Aug 2024 19:27:13 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id n5Mfqkz3kMRN; Thu, 8 Aug 2024 19:27:11 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 50F1960B53 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=163.com header.i=@163.com header.a=rsa-sha256 header.s=s110527 header.b=BGfxrKsa Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 50F1960B53; Thu, 8 Aug 2024 19:27:11 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AC52CC003D; Thu, 8 Aug 2024 19:27:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8A87CC002A for ; Thu, 8 Aug 2024 09:57:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 793BC408F4 for ; Thu, 8 Aug 2024 09:57:28 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id QYEDD05jwNu2 for ; Thu, 8 Aug 2024 09:57:27 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=220.197.31.4; helo=m16.mail.163.com; envelope-from=15957197901@163.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org A7B1D408EF Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=163.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A7B1D408EF Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=163.com header.i=@163.com header.a=rsa-sha256 header.s=s110527 header.b=BGfxrKsa Received: from m16.mail.163.com (m16.mail.163.com [220.197.31.4]) by smtp4.osuosl.org (Postfix) with ESMTP id A7B1D408EF for ; Thu, 8 Aug 2024 09:57:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-ID:MIME-Version; bh=bWzUZ i+iLulZcFg+vrXHIAPZ288y4GkOMmDfkv9Pea8=; b=BGfxrKsavEzv4w6ZMJGGy kmqfIxIJafEKdiMi8mL+bIQebLQpcssxSjYj5uycrccT9yp5/GiMVhYxGxSL/W+j sopk5GNhoTZRUcAXkuKRf4+ygD9bW9m083JILIRjVa5Jd7Wwtx+LssLaf0SJdd8p uJRvjwGXESwHCUkTdH2JNM= Received: from localhost.localdomain (unknown [36.20.109.33]) by gzsmtp3 (Coremail) with SMTP id sigvCgDXXxiClrRmEZrzAA--.28727S2; Thu, 08 Aug 2024 17:57:23 +0800 (CST) From: Xinxin Zhao <15957197901@163.com> To: dev@openvswitch.org Cc: Xinxin Zhao <15957197901@163.com> Date: Thu, 8 Aug 2024 17:57:17 +0800 Message-ID: <20240808095717.75932-1-15957197901@163.com> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 X-CM-TRANSID: sigvCgDXXxiClrRmEZrzAA--.28727S2 X-Coremail-Antispam: 1Uf129KBjvJXoW7Zr1DJr18JrW7Jw43JF1DZFb_yoW8Xw4rpa yDXFy5tFn7tr4Sgr47Za18Ka48CF48ZrZrCrWxGa43X34DWw45Ka9YgFyjqF13C3y0va1v ya1YyryUWrWkA3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x0pRMa09UUUUU= X-Originating-IP: [36.20.109.33] X-CM-SenderInfo: zprvmkyxrzlmmqrbiqqrwthudrp/1tbiYwM1xGV4J7esRQAAs1 X-Mailman-Approved-At: Thu, 08 Aug 2024 19:27:08 +0000 Subject: [ovs-dev] [PATCH] netdev-dpdk: Fixed RCU deadlock when deleting vhostuser port X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When the ovs control thread del vhost-user port and the vhost-event thread process the vhost-user port down concurrently, the main thread may fall into a deadlock. E.g., vhostuser port is created as client. The ovs control thread executes the following process: rte_vhost_driver_unregister->fdset_try_del. At the same time, the vhost-event thread executes the following process: fdset_event_dispatch->vhost_user_read_cb->destroy_device. At this time, vhost-event will wait for rcu scheduling, and the ovs control thread is waiting for pfdentry->busy to be 0. The two threads are waiting for each other and fall into a deadlock. Fixes: afee281 ("netdev-dpdk: Fix dpdk_watchdog failure to quiesce.") Signed-off-by: Xinxin Zhao <15957197901@163.com> Acked-by: Kevin Traynor --- lib/netdev-dpdk.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 02cef6e45..0c02357f5 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1808,7 +1808,16 @@ dpdk_vhost_driver_unregister(struct netdev_dpdk *dev OVS_UNUSED, OVS_EXCLUDED(dpdk_mutex) OVS_EXCLUDED(dev->mutex) { - return rte_vhost_driver_unregister(vhost_id); + int ret; + /* Due to the rcu wait of the vhost-event thread, + * rte_vhost_driver_unregister() may loop endlessly. + * So the unregister action needs to be removed from the rcu_list. + */ + ovsrcu_quiesce_start(); + ret = rte_vhost_driver_unregister(vhost_id); + ovsrcu_quiesce_end(); + + return ret; } static void