From patchwork Sat Aug 5 02:33:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: ~hyman X-Patchwork-Id: 1819726 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RLygQ0sLJz1yfp for ; Thu, 10 Aug 2023 17:08:01 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qTzlL-0005Oe-WA; Thu, 10 Aug 2023 03:07:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qTzlJ-0005No-NV for qemu-devel@nongnu.org; Thu, 10 Aug 2023 03:07:13 -0400 Received: from mail-b.sr.ht ([173.195.146.151]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qTzlH-0004oH-TS for qemu-devel@nongnu.org; Thu, 10 Aug 2023 03:07:13 -0400 Authentication-Results: mail-b.sr.ht; dkim=none Received: from git.sr.ht (unknown [173.195.146.142]) by mail-b.sr.ht (Postfix) with ESMTPSA id 1A20B11EF15; Thu, 10 Aug 2023 07:07:10 +0000 (UTC) From: ~hyman Date: Sat, 05 Aug 2023 10:33:21 +0800 Subject: [PATCH QEMU v2 1/3] virtio-scsi-pci: introduce auto-num-queues property MIME-Version: 1.0 Message-ID: <169165122958.4096.6557269987206106594-1@git.sr.ht> X-Mailer: git.sr.ht In-Reply-To: <169165122958.4096.6557269987206106594-0@git.sr.ht> To: qemu-devel Cc: Raphael Norwitz , "Michael S. Tsirkin" , Kevin Wolf , Hanna Reitz , Stefan Hajnoczi , Paolo Bonzini , Fam Zheng , yong.huang@smartx.com Received-SPF: pass client-ip=173.195.146.151; envelope-from=outgoing@sr.ht; helo=mail-b.sr.ht X-Spam_score_int: 15 X-Spam_score: 1.5 X-Spam_bar: + X-Spam_report: (1.5 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_96_XX=3.405, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: ~hyman Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Hyman Huang(黄勇) Commit "6a55882284 virtio-scsi-pci: default num_queues to -smp N" implment sizing the number of virtio-scsi-pci request virtqueues to match the number of vCPUs automatically. Which improves IO preformance remarkably. To enable this feature for the existing VMs, the cloud platform may migrate VMs from the source hypervisor (num_queues is set to 1 by default) to the destination hypervisor (num_queues is set to -smp N) lively. The different num-queues for virtio-scsi-pci devices between the source side and the destination side will result in migration failure due to loading vmstate incorrectly on the destination side. To provide a smooth upgrade solution, introduce the auto-num-queues property for the virtio-scsi-pci device. This allows upper APPs, e.g., libvirt, to recognize the hypervisor's capability of allocating the virtqueues automatically by probing the virtio-scsi-pci.auto-num-queues property. Basing on which, upper APPs can ensure to allocate the same num-queues on the destination side in case of migration failure. Signed-off-by: Hyman Huang(黄勇) --- hw/scsi/vhost-scsi.c | 2 ++ hw/scsi/vhost-user-scsi.c | 2 ++ hw/scsi/virtio-scsi.c | 2 ++ hw/virtio/vhost-scsi-pci.c | 11 +++++++++-- hw/virtio/vhost-user-scsi-pci.c | 11 +++++++++-- hw/virtio/virtio-scsi-pci.c | 11 +++++++++-- include/hw/virtio/virtio-scsi.h | 5 +++++ 7 files changed, 38 insertions(+), 6 deletions(-) diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index 443f67daa4..78a8929c49 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -284,6 +284,8 @@ static Property vhost_scsi_properties[] = { DEFINE_PROP_STRING("vhostfd", VirtIOSCSICommon, conf.vhostfd), DEFINE_PROP_STRING("wwpn", VirtIOSCSICommon, conf.wwpn), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), + DEFINE_PROP_BOOL("auto_num_queues", VirtIOSCSICommon, auto_num_queues, + true), DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index ee99b19e7a..1b837f370a 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -161,6 +161,8 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) static Property vhost_user_scsi_properties[] = { DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), + DEFINE_PROP_BOOL("auto_num_queues", VirtIOSCSICommon, auto_num_queues, + true), DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_size, diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index 45b95ea070..2ec13032aa 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -1279,6 +1279,8 @@ static void virtio_scsi_device_unrealize(DeviceState *dev) } static Property virtio_scsi_properties[] = { + DEFINE_PROP_BOOL("auto_num_queues", VirtIOSCSI, parent_obj.auto_num_queues, + true), DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queues, VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI, diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c index 08980bc23b..927c155278 100644 --- a/hw/virtio/vhost-scsi-pci.c +++ b/hw/virtio/vhost-scsi-pci.c @@ -51,8 +51,15 @@ static void vhost_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { - conf->num_queues = - virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + /* + * Allocate virtqueues automatically only if auto_num_queues + * property set true. + */ + if (dev->vdev.parent_obj.parent_obj.auto_num_queues) + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + else + conf->num_queues = 1; } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pci.c index 75882e3cf9..9c521a7f93 100644 --- a/hw/virtio/vhost-user-scsi-pci.c +++ b/hw/virtio/vhost-user-scsi-pci.c @@ -57,8 +57,15 @@ static void vhost_user_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) VirtIOSCSIConf *conf = &dev->vdev.parent_obj.parent_obj.conf; if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { - conf->num_queues = - virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + /* + * Allocate virtqueues automatically only if auto_num_queues + * property set true. + */ + if (dev->vdev.parent_obj.parent_obj.auto_num_queues) + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + else + conf->num_queues = 1; } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c index e8e3442f38..7551857f90 100644 --- a/hw/virtio/virtio-scsi-pci.c +++ b/hw/virtio/virtio-scsi-pci.c @@ -52,8 +52,15 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) char *bus_name; if (conf->num_queues == VIRTIO_SCSI_AUTO_NUM_QUEUES) { - conf->num_queues = - virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + /* + * Allocate virtqueues automatically only if auto_num_queues + * property set true. + */ + if (dev->vdev.parent_obj.auto_num_queues) + conf->num_queues = + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + else + conf->num_queues = 1; } if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scsi.h index 779568ab5d..2660bbad22 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -72,6 +72,11 @@ struct VirtIOSCSICommon { VirtQueue *ctrl_vq; VirtQueue *event_vq; VirtQueue **cmd_vqs; + /* + * Set to true if virtqueues allow to be allocated to + * match the number of virtual CPUs automatically. + */ + bool auto_num_queues; }; struct VirtIOSCSIReq;