From patchwork Tue Sep 19 11:40:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 1836664 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=smartx-com.20230601.gappssmtp.com header.i=@smartx-com.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=2FDAJ/Sj; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RqfwM4YHjz1yny for ; Tue, 19 Sep 2023 21:44:51 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qiZ8M-0007lN-BU; Tue, 19 Sep 2023 07:43:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiZ8K-0007jI-EG for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:12 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qiZ8H-0004F5-Ly for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:12 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-68cbbff84f6so4612244b3a.1 for ; Tue, 19 Sep 2023 04:41:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695123666; x=1695728466; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1eP8kuKM1qKZ8jHbFZXOAaKaPzBH+J+kNlsheIRdgb4=; b=2FDAJ/Sjvj5cC34b2BOMzs+MSk8wJXfzy3liqTv0cKTuODnEKCrHil8btN0zvO1V52 kmdLLuLFdfxpG+uOa+iYPOC/5QGOZ0u4WjeqrrvKYsxQp3SwgaVARNH2WUuiYFbgHX+C 70HCgm088xPXa+Jca771ykN2ukA+oRcSVtramvIJLIlmvUJezj5Pn68UAY2VIW+meKgK 2y6z9fPrbls0iEbR5kbXpasn5Ld+/BsW+ZzQrtOMd9a9+KGC7w4uPnuG4potL+xk+rQs NA7uLpodbyXu62ocY8lROb8W0NdfXgmh8zdZKbdiKT/+buT8RknVTydmWctyiq1nst0E VJ3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695123666; x=1695728466; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1eP8kuKM1qKZ8jHbFZXOAaKaPzBH+J+kNlsheIRdgb4=; b=sY6S80TJDzPGsNCyqyQia19R15ZzsM+tzySH6hrv9GPEZ2O1EDc3cSvl9x7PG6alZu AUkFTyOPIHexEkpRo8GD8iy8Kv79Ue0LY4f3hWroYNHWMK27idppOA6C3mdfW923b9pE JQDNsyZJBXbxgzPA24uM/6s3TJaxpJaBYe8kmmS8A1WuJ3yPRbEx/crhI6TJe+NVy0OO nuZDPDF40uGu0uGx4pTgb6vWraGpztxYRKf5ae7VOrdAa831tV8TFIyUWRDoflS03lj9 lE8ITDTpY6oswQdecAy0xMQmgzVzGWcIH1A2LSuVfjb+Tw4ePQTFSf68B0gIOmZM8gAo Ew+g== X-Gm-Message-State: AOJu0YzwqcALDvDSf/ckeBOic9trGdD6LQiFxoiJHU+1eew3Nkp1XDPi Z/IAduWDx/zOe9s5CstcwxETHQ== X-Google-Smtp-Source: AGHT+IFXhLSk1J4/oFjctJ/Po8nD5YwHwMHiCnwaBvzK3FsmOjQgqT0VcPMdiBDJPHiaO5CfeKMAGg== X-Received: by 2002:a05:6a00:1d29:b0:68e:16ce:f2e3 with SMTP id a41-20020a056a001d2900b0068e16cef2e3mr3023975pfx.0.1695123666044; Tue, 19 Sep 2023 04:41:06 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id z24-20020aa791d8000000b006887be16675sm8487030pfa.205.2023.09.19.04.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 04:41:05 -0700 (PDT) From: Li Feng To: Markus Armbruster , "Michael S. Tsirkin" , Raphael Norwitz , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v5 1/5] vhost-user-common: send get_inflight_fd once Date: Tue, 19 Sep 2023 19:40:22 +0800 Message-ID: <20230919114030.280414-2-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230919114030.280414-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230919114030.280414-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::42d; envelope-from=fengli@smartx.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently the get_inflight_fd will be sent every time the device is started, and the backend will allocate shared memory to save the inflight state. If the backend finds that it receives the second get_inflight_fd, it will release the previous shared memory, which breaks inflight working logic. This patch is a preparation for the following patches. Signed-off-by: Li Feng --- hw/scsi/vhost-scsi-common.c | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/hw/scsi/vhost-scsi-common.c b/hw/scsi/vhost-scsi-common.c index a06f01af26..a61cd0e907 100644 --- a/hw/scsi/vhost-scsi-common.c +++ b/hw/scsi/vhost-scsi-common.c @@ -52,20 +52,28 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) vsc->dev.acked_features = vdev->guest_features; - assert(vsc->inflight == NULL); - vsc->inflight = g_new0(struct vhost_inflight, 1); - ret = vhost_dev_get_inflight(&vsc->dev, - vs->conf.virtqueue_size, - vsc->inflight); + ret = vhost_dev_prepare_inflight(&vsc->dev, vdev); if (ret < 0) { - error_report("Error get inflight: %d", -ret); + error_report("Error setting inflight format: %d", -ret); goto err_guest_notifiers; } - ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); - if (ret < 0) { - error_report("Error set inflight: %d", -ret); - goto err_guest_notifiers; + if (vsc->inflight) { + if (!vsc->inflight->addr) { + ret = vhost_dev_get_inflight(&vsc->dev, + vs->conf.virtqueue_size, + vsc->inflight); + if (ret < 0) { + error_report("Error getting inflight: %d", -ret); + goto err_guest_notifiers; + } + } + + ret = vhost_dev_set_inflight(&vsc->dev, vsc->inflight); + if (ret < 0) { + error_report("Error setting inflight: %d", -ret); + goto err_guest_notifiers; + } } ret = vhost_dev_start(&vsc->dev, vdev, true); @@ -85,9 +93,6 @@ int vhost_scsi_common_start(VHostSCSICommon *vsc) return ret; err_guest_notifiers: - g_free(vsc->inflight); - vsc->inflight = NULL; - k->set_guest_notifiers(qbus->parent, vsc->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&vsc->dev, vdev); @@ -111,12 +116,6 @@ void vhost_scsi_common_stop(VHostSCSICommon *vsc) } assert(ret >= 0); - if (vsc->inflight) { - vhost_dev_free_inflight(vsc->inflight); - g_free(vsc->inflight); - vsc->inflight = NULL; - } - vhost_dev_disable_notifiers(&vsc->dev, vdev); } From patchwork Tue Sep 19 11:40:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Feng X-Patchwork-Id: 1836667 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=smartx-com.20230601.gappssmtp.com header.i=@smartx-com.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=ukKaax96; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rqfwz0NSnz1yny for ; Tue, 19 Sep 2023 21:45:23 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qiZ8W-0007sk-Rw; Tue, 19 Sep 2023 07:43:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qiZ8V-0007rx-G4 for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:23 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qiZ8T-0004Go-S6 for qemu-devel@nongnu.org; Tue, 19 Sep 2023 07:43:23 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-690bd8f89baso875586b3a.2 for ; Tue, 19 Sep 2023 04:41:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=smartx-com.20230601.gappssmtp.com; s=20230601; t=1695123680; x=1695728480; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zEHWKIJ/jlcW2XJm2YoMaGjRhjLSx9EfJkS7TRjaaVY=; b=ukKaax96/pmwiLBVLBAagPrOMEqbJUx2Dck5RENMCUIKcRLcY2F8zKX3/WX18+MAkZ BY7bTwn2KyLGKJ1yPnD0fgT15HKHI1/+hQ6UNFt1fUd7wXvynmNP0kcVCn6v2jYajYgj yaZhzkeZVH0ODZ0qGu/mJeQ9Oue6/1ZuzPlSZP0sc0gnoU4xoM7XpT0UK8WhTrpe8Vj5 aBy1oCWY4ghanw9TxkH6N9oGSdR6AGaskpWiqulBAwO9FtqLiTmLk77TCA+MRqtW6qMa im8v4beSi0N+5+9oM3h4yE2+zm+7GCDvAuWENQKxi2ZBRSrLBTlVG5YFFlcSknwUyBds n0cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695123680; x=1695728480; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zEHWKIJ/jlcW2XJm2YoMaGjRhjLSx9EfJkS7TRjaaVY=; b=S513LsKzgTEXbc0KGaY4n9SkDxID+J77yBKDy/1XMQqoXNY57mpw+bilKupkVHN9H2 3vyLexe1Z5YTLXBVS2wX0VHHUY0Om/uyCQCz/KjhHqgn+DIYn77def6kXi1D16kejy1V MEa3RCZin4Rc6wfck/LUb9jFazRyHgMEceAj1QpXWfChI+LwgHqDICOEcIOOPojMXJui LZAhq0depN5Cu1XAHjt7WHl9jSvx5bBU6s1YifltPTRWFZFQxwefAz4perlqUnQSJOmp D0MU7EhDY036Xu5jlTPOrEKAnlaDijjuzlmDEFw+PXV/088U7DSsSn8iYcIsrC9D3VjU vYyQ== X-Gm-Message-State: AOJu0YwjPxfB6dFkeLzGFl4VcYOlAN4DJj7WA8CbwsSJAZ9TDdBSyKkb q0LRDJ+1GSvxZF5AvnTFrj+kcw== X-Google-Smtp-Source: AGHT+IEVY+j0o0niFXBzQToLnxkV6h5ezezXNBTG5QG3u7D4Qcb5c8wuxlSAtIw0klaTb34NWZEzZA== X-Received: by 2002:a05:6a00:10c4:b0:68f:e810:e86f with SMTP id d4-20020a056a0010c400b0068fe810e86fmr10294630pfu.28.1695123680352; Tue, 19 Sep 2023 04:41:20 -0700 (PDT) Received: from 64217.gitgo.cc ([8.210.91.195]) by smtp.gmail.com with ESMTPSA id z24-20020aa791d8000000b006887be16675sm8487030pfa.205.2023.09.19.04.41.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Sep 2023 04:41:19 -0700 (PDT) From: Li Feng To: Markus Armbruster , Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Paolo Bonzini , Fam Zheng , =?utf-8?q?Alex_Benn=C3=A9e?= , Viresh Kumar , qemu-block@nongnu.org (open list:Block layer core), qemu-devel@nongnu.org (open list:All patches CC here) Cc: Li Feng Subject: [PATCH v5 5/5] vhost-user: fix lost reconnect Date: Tue, 19 Sep 2023 19:40:26 +0800 Message-ID: <20230919114030.280414-6-fengli@smartx.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230919114030.280414-1-fengli@smartx.com> References: <20230721105205.1714449-1-fengli@smartx.com> <20230919114030.280414-1-fengli@smartx.com> MIME-Version: 1.0 Received-SPF: none client-ip=2607:f8b0:4864:20::42d; envelope-from=fengli@smartx.com; helo=mail-pf1-x42d.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When the vhost-user is reconnecting to the backend, and if the vhost-user fails at the get_features in vhost_dev_init(), then the reconnect will fail and it will not be retriggered forever. The reason is: When the vhost-user fails at get_features, the vhost_dev_cleanup will be called immediately. vhost_dev_cleanup calls 'memset(hdev, 0, sizeof(struct vhost_dev))'. The reconnect path is: vhost_user_blk_event vhost_user_async_close(.. vhost_user_blk_disconnect ..) qemu_chr_fe_set_handlers <----- clear the notifier callback schedule vhost_user_async_close_bh The vhost->vdev is null, so the vhost_user_blk_disconnect will not be called, then the event fd callback will not be reinstalled. All vhost-user devices have this issue, including vhost-user-blk/scsi. With this patch, if the vdev->vdev is null, the fd callback will still be reinstalled. Fixes: 71e076a07d ("hw/virtio: generalise CHR_EVENT_CLOSED handling") Signed-off-by: Li Feng --- hw/block/vhost-user-blk.c | 2 +- hw/scsi/vhost-user-scsi.c | 3 ++- hw/virtio/vhost-user-gpio.c | 2 +- hw/virtio/vhost-user.c | 9 +++++++-- include/hw/virtio/vhost-user.h | 3 ++- 5 files changed, 13 insertions(+), 6 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 3c69fa47d5..95c758200d 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -391,7 +391,7 @@ static void vhost_user_blk_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &s->chardev, &s->dev, - vhost_user_blk_disconnect); + vhost_user_blk_disconnect, vhost_user_blk_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 53a62c3170..0effbb4787 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -238,7 +238,8 @@ static void vhost_user_scsi_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &vs->conf.chardev, &vsc->dev, - vhost_user_scsi_disconnect); + vhost_user_scsi_disconnect, + vhost_user_scsi_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user-gpio.c b/hw/virtio/vhost-user-gpio.c index d9979aa5db..04c2cc79f4 100644 --- a/hw/virtio/vhost-user-gpio.c +++ b/hw/virtio/vhost-user-gpio.c @@ -283,7 +283,7 @@ static void vu_gpio_event(void *opaque, QEMUChrEvent event) case CHR_EVENT_CLOSED: /* defer close until later to avoid circular close */ vhost_user_async_close(dev, &gpio->chardev, &gpio->vhost_dev, - vu_gpio_disconnect); + vu_gpio_disconnect, vu_gpio_event); break; case CHR_EVENT_BREAK: case CHR_EVENT_MUX_IN: diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 8dcf049d42..12c4a41f30 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -2643,6 +2643,7 @@ typedef struct { DeviceState *dev; CharBackend *cd; struct vhost_dev *vhost; + IOEventHandler *event_cb; } VhostAsyncCallback; static void vhost_user_async_close_bh(void *opaque) @@ -2657,7 +2658,10 @@ static void vhost_user_async_close_bh(void *opaque) */ if (vhost->vdev) { data->cb(data->dev); - } + } else if (data->event_cb) { + qemu_chr_fe_set_handlers(data->cd, NULL, NULL, data->event_cb, + NULL, data->dev, NULL, true); + } g_free(data); } @@ -2669,7 +2673,8 @@ static void vhost_user_async_close_bh(void *opaque) */ void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb) + vu_async_close_fn cb, + IOEventHandler *event_cb) { if (!runstate_check(RUN_STATE_SHUTDOWN)) { /* diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h index 191216a74f..649e9dd54f 100644 --- a/include/hw/virtio/vhost-user.h +++ b/include/hw/virtio/vhost-user.h @@ -84,6 +84,7 @@ typedef void (*vu_async_close_fn)(DeviceState *cb); void vhost_user_async_close(DeviceState *d, CharBackend *chardev, struct vhost_dev *vhost, - vu_async_close_fn cb); + vu_async_close_fn cb, + IOEventHandler *event_cb); #endif