From patchwork Wed Apr 19 11:49:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1770709 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=gOwvYgYa; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q1fr526z6z23tD for ; Wed, 19 Apr 2023 22:15:17 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pp6hN-00029e-IE; Wed, 19 Apr 2023 08:14:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pp6KS-000540-Dz for qemu-devel@nongnu.org; Wed, 19 Apr 2023 07:50:28 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pp6KK-0005yf-PY for qemu-devel@nongnu.org; Wed, 19 Apr 2023 07:50:28 -0400 Received: by mail-pj1-x1033.google.com with SMTP id fw22-20020a17090b129600b00247255b2f40so524763pjb.1 for ; Wed, 19 Apr 2023 04:50:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681905019; x=1684497019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3TCND2PEu+54Kzy/XrvfzrPulI3hsuCCyDLVI94fByY=; b=gOwvYgYamSL3ZtaH7oub6JR9zv7NDUvU6JJSK4YCcEDha+i5zF9gnrowWCNAorXYRW exky3IST+acn4Thllh1Gx4SX9t8eqPqJsMPfFW+sRlWaUA/5spYniaZvM34QcYMyv6gd deqRKajf+xJUsu5Mnn25VEYSXfmo/XPRoLSAFAc9WdcL7hCWv+QnE80bhKyiqo+sWDQd ofzlxulM/ETZjBFbHiGobhk5zH14CU7YuC0wPEP2DCfGch5xsQGOWoUIdliAcCJ1UJq4 O+3vhlMnEBOiV9PsprHg/a/1sU7zwSkUDoUN8Dyyjh/6IxfttvR/Xr143sOt//j4PyE3 BNpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681905019; x=1684497019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3TCND2PEu+54Kzy/XrvfzrPulI3hsuCCyDLVI94fByY=; b=M7WM9MK3XRpQmepjUTzHPgCv6YpSRcv50miGPRH2XRJqQsdlD1uCRniKeANg7My2dl mKj8u6i+vHV3g0mCBWAelLwZFoVmvOiTJN4lzGI+8zxrEPoMy6H0kgmq8ebUkEwk6osQ Mj7qyAg/LxP5FeHUWWWKWzo2/1xfDKso5xWyFSebymH74hsinqc9x/LZs1Iq9uMRPsJ1 WwmGFJ0v9PvJIQk8ICeYDrgtlLcqFMyyXOA+X2r/9oNG6c0Mvjl/9HVdnTHNF86RXCoC DUkI2fAE777ttxOf9qxgaNX0E/WsHezTTN9FV3fpeYZ6PJke5bIG/NB3p29Vy5DyPZTY Jxhg== X-Gm-Message-State: AAQBX9dDBoz+J1znnnFYwiqM355eelgNp/8qyCWyjNwrMssJxC82EV3W 3raCvrMp8VOea2ym5Lqpx+U= X-Google-Smtp-Source: AKy350adNZUIJTJ336DS0dvf8uPjM5/C9asZ8wcYStJyVck05dUAS2EuWgA/3iX/UBTFw/M9ZV7I1A== X-Received: by 2002:a05:6a20:3c8a:b0:d6:7b9b:f89 with SMTP id b10-20020a056a203c8a00b000d67b9b0f89mr4303951pzj.4.1681905019051; Wed, 19 Apr 2023 04:50:19 -0700 (PDT) Received: from localhost ([159.226.94.115]) by smtp.gmail.com with ESMTPSA id v11-20020a62a50b000000b0063afb08afeesm10932962pfm.67.2023.04.19.04.50.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Apr 2023 04:50:18 -0700 (PDT) From: Hawkins Jiawei To: eperezma@redhat.com, jasowang@redhat.com Cc: yin31149@gmail.com, 18801353760@163.com, qemu-devel@nongnu.org Subject: [PATCH 1/2] vdpa: rename vhost_vdpa_net_cvq_add() Date: Wed, 19 Apr 2023 19:49:52 +0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=yin31149@gmail.com; helo=mail-pj1-x1033.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 19 Apr 2023 08:14:05 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org We want to introduce a new version of vhost_vdpa_net_cvq_add() that does not poll immediately after forwarding custom buffers to the device, so that QEMU can send all the SVQ control commands in parallel instead of serialized. Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 99904a0da7..10804c7200 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -590,8 +590,14 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) vhost_vdpa_net_client_stop(nc); } -static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, - size_t in_len) +/** + * vhost_vdpa_net_cvq_add_and_wait() adds SVQ control commands to SVQ, + * kicks the device and polls the device used buffers. + * + * Return the length written by the device. + */ +static ssize_t vhost_vdpa_net_cvq_add_and_wait(VhostVDPAState *s, + size_t out_len, size_t in_len) { /* Buffers for the device */ const struct iovec out = { @@ -636,7 +642,7 @@ static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, memcpy(s->cvq_cmd_out_buffer, &ctrl, sizeof(ctrl)); memcpy(s->cvq_cmd_out_buffer + sizeof(ctrl), data, data_size); - return vhost_vdpa_net_cvq_add(s, sizeof(ctrl) + data_size, + return vhost_vdpa_net_cvq_add_and_wait(s, sizeof(ctrl) + data_size, sizeof(virtio_net_ctrl_ack)); } @@ -753,7 +759,8 @@ static int vhost_vdpa_net_handle_ctrl_avail(VhostShadowVirtqueue *svq, dev_written = sizeof(status); *s->status = VIRTIO_NET_OK; } else { - dev_written = vhost_vdpa_net_cvq_add(s, out.iov_len, sizeof(status)); + dev_written = vhost_vdpa_net_cvq_add_and_wait(s, out.iov_len, + sizeof(status)); if (unlikely(dev_written < 0)) { goto out; } From patchwork Wed Apr 19 11:49:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hawkins Jiawei X-Patchwork-Id: 1770708 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20221208 header.b=CX4lKPvs; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q1fqn3X3bz1ybF for ; Wed, 19 Apr 2023 22:15:00 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pp6hQ-0002AP-53; Wed, 19 Apr 2023 08:14:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pp6Kg-00059k-74 for qemu-devel@nongnu.org; Wed, 19 Apr 2023 07:50:43 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pp6KU-00065I-Ab for qemu-devel@nongnu.org; Wed, 19 Apr 2023 07:50:35 -0400 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1a6f0d8cdfeso15430265ad.2 for ; Wed, 19 Apr 2023 04:50:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681905029; x=1684497029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QAULlSw53JLsLvmavzKWkXLeU05mO+xcLRPRW6r8RE=; b=CX4lKPvsggR/fQ7TbuOyhT/2KQpyzqf8hd5zanHELsqfKoY2LpQ0oxmjSdV1zgjBqu TCh6OGWvKH+eNip1gVjNVTOIr/KFUgxWqP1vLEYw6aIs9yBJnAKTLdhGro7jn3WL3Nd/ oMlLTm+qLS8HRRGNxEwYti+q66hYTBjDBFWBAY9/eI7WtOeTkOFhQOFw6jbh/hvUvk8I HMQwa3BS0KnM+1IPZG8KV+EXr9tJJNKB+1bYxGvJ4vg/PX6zVpTspn7XwdQFg6WQHVMQ yg+9NAc/JJd1yBKuKXK9M8J4Cw4FWH6a14Vb7kcy1hdgDbEvLxp9eTjl98urvqPk3akI L/XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681905029; x=1684497029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QAULlSw53JLsLvmavzKWkXLeU05mO+xcLRPRW6r8RE=; b=TxPMRmd6PzmoDUY5gPRZuyZ1AVay/rEOHRko6IeyCBMZuwBUxQ1rTIKVPfBZLsuzpw gEVa30Js+7EjyNENiUEb4uxPkWKiAOqYw/teXWrt9N0ISR8VBVj+NgapUr3uZ24kDNdc GuZBEkxIr6oCHmK4LfxhrYjb5nk/FXSeChNWehWEuCTpgHAbShP2CrpWNl78eZSzlzy6 BFQC3iEVoEk2zihPrA+A98ADuCTcpIRoKXXErndYTyxFbXj403MG3ckIG1lCE+93+CIz qX2DqXSXLNDQuEykVXsYtUi+DVldPHyiN1EaeBLzy9PoRyCW6qgh/ekKGmzIrmrdlbMI 3ASA== X-Gm-Message-State: AAQBX9euDAwmnkwOxUEeWaCaj45zFJRz8tLrCDSVqmrf+POjVRpF7InD y76ciZbRAYwykbvM9eBus/U= X-Google-Smtp-Source: AKy350Z/fY9HnaNhdVZU29uH6tGmjSXOkTpBmIczdYJHdXLq21ru2rNxBfzL854FEDFSfRcbC00dKw== X-Received: by 2002:a17:903:110e:b0:1a6:6fe3:df91 with SMTP id n14-20020a170903110e00b001a66fe3df91mr5862478plh.50.1681905028771; Wed, 19 Apr 2023 04:50:28 -0700 (PDT) Received: from localhost ([159.226.94.115]) by smtp.gmail.com with ESMTPSA id o2-20020a170902bcc200b001a6a53c3b04sm8922738pls.306.2023.04.19.04.50.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Apr 2023 04:50:28 -0700 (PDT) From: Hawkins Jiawei To: eperezma@redhat.com, jasowang@redhat.com Cc: yin31149@gmail.com, 18801353760@163.com, qemu-devel@nongnu.org Subject: [PATCH 2/2] vdpa: send CVQ state load commands in parallel Date: Wed, 19 Apr 2023 19:49:53 +0800 Message-Id: <3bd9b7f8a7dc2c9b68679139623308e184a6de85.1681732982.git.yin31149@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=yin31149@gmail.com; helo=mail-pl1-x62a.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 19 Apr 2023 08:14:05 -0400 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This patch introduces the vhost_vdpa_net_cvq_add() and refactors the vhost_vdpa_net_load*(), so that QEMU can send CVQ state load commands in parallel. To be more specific, this patch introduces vhost_vdpa_net_cvq_add() to add SVQ control commands to SVQ and kick the device, but does not poll the device used buffers. QEMU will not poll and check the device used buffers in vhost_vdpa_net_load() until all CVQ state load commands have been sent to the device. What's more, in order to avoid buffer overwriting caused by using `svq->cvq_cmd_out_buffer` and `svq->status` as the buffer for all CVQ state load commands when sending CVQ state load commands in parallel, this patch introduces `out_cursor` and `in_cursor` in vhost_vdpa_net_load(), pointing to the available buffer for in descriptor and out descriptor, so that different CVQ state load commands can use their unique buffer. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1578 Signed-off-by: Hawkins Jiawei --- net/vhost-vdpa.c | 137 +++++++++++++++++++++++++++++++++++------------ 1 file changed, 102 insertions(+), 35 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 10804c7200..d1f7113992 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -590,6 +590,44 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) vhost_vdpa_net_client_stop(nc); } +/** + * vhost_vdpa_net_cvq_add() adds SVQ control commands to SVQ, + * kicks the device but does not poll the device used buffers. + * + * Return the length of the device used buffers. + */ +static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, + void **out_cursor, size_t out_len, + void **in_cursor, size_t in_len) +{ + /* Buffers for the device */ + const struct iovec out = { + .iov_base = *out_cursor, + .iov_len = out_len, + }; + const struct iovec in = { + .iov_base = *in_cursor, + .iov_len = in_len, + }; + VhostShadowVirtqueue *svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); + int r; + + r = vhost_svq_add(svq, &out, 1, &in, 1, NULL); + if (unlikely(r != 0)) { + if (unlikely(r == -ENOSPC)) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: No space on device queue\n", + __func__); + } + return r; + } + + /* Update the cursor */ + *out_cursor += out_len; + *in_cursor += in_len; + + return in_len; +} + /** * vhost_vdpa_net_cvq_add_and_wait() adds SVQ control commands to SVQ, * kicks the device and polls the device used buffers. @@ -628,69 +666,64 @@ static ssize_t vhost_vdpa_net_cvq_add_and_wait(VhostVDPAState *s, return vhost_svq_poll(svq); } -static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, uint8_t class, - uint8_t cmd, const void *data, - size_t data_size) +static ssize_t vhost_vdpa_net_load_cmd(VhostVDPAState *s, + void **out_cursor, uint8_t class, uint8_t cmd, + const void *data, size_t data_size, + void **in_cursor) { const struct virtio_net_ctrl_hdr ctrl = { .class = class, .cmd = cmd, }; - assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl)); + assert(sizeof(ctrl) < vhost_vdpa_net_cvq_cmd_page_len() - + (*out_cursor - s->cvq_cmd_out_buffer)); + assert(data_size < vhost_vdpa_net_cvq_cmd_page_len() - sizeof(ctrl) - + (*out_cursor - s->cvq_cmd_out_buffer)); - memcpy(s->cvq_cmd_out_buffer, &ctrl, sizeof(ctrl)); - memcpy(s->cvq_cmd_out_buffer + sizeof(ctrl), data, data_size); + memcpy(*out_cursor, &ctrl, sizeof(ctrl)); + memcpy(*out_cursor + sizeof(ctrl), data, data_size); - return vhost_vdpa_net_cvq_add_and_wait(s, sizeof(ctrl) + data_size, - sizeof(virtio_net_ctrl_ack)); + return vhost_vdpa_net_cvq_add(s, out_cursor, sizeof(ctrl) + data_size, + in_cursor, sizeof(virtio_net_ctrl_ack)); } -static int vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n) +static ssize_t vhost_vdpa_net_load_mac(VhostVDPAState *s, const VirtIONet *n, + void **out_cursor, void **in_cursor) { uint64_t features = n->parent_obj.guest_features; if (features & BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR)) { - ssize_t dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MAC, - VIRTIO_NET_CTRL_MAC_ADDR_SET, - n->mac, sizeof(n->mac)); - if (unlikely(dev_written < 0)) { - return dev_written; - } - - return *s->status != VIRTIO_NET_OK; + return vhost_vdpa_net_load_cmd(s, out_cursor, VIRTIO_NET_CTRL_MAC, + VIRTIO_NET_CTRL_MAC_ADDR_SET, + n->mac, sizeof(n->mac), in_cursor); } return 0; } -static int vhost_vdpa_net_load_mq(VhostVDPAState *s, - const VirtIONet *n) +static ssize_t vhost_vdpa_net_load_mq(VhostVDPAState *s, const VirtIONet *n, + void **out_cursor, void **in_cursor) { struct virtio_net_ctrl_mq mq; uint64_t features = n->parent_obj.guest_features; - ssize_t dev_written; if (!(features & BIT_ULL(VIRTIO_NET_F_MQ))) { return 0; } - mq.virtqueue_pairs = cpu_to_le16(n->curr_queue_pairs); - dev_written = vhost_vdpa_net_load_cmd(s, VIRTIO_NET_CTRL_MQ, - VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, &mq, - sizeof(mq)); - if (unlikely(dev_written < 0)) { - return dev_written; - } - - return *s->status != VIRTIO_NET_OK; + return vhost_vdpa_net_load_cmd(s, out_cursor, VIRTIO_NET_CTRL_MQ, + VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET, + &mq, sizeof(mq), in_cursor); } static int vhost_vdpa_net_load(NetClientState *nc) { VhostVDPAState *s = DO_UPCAST(VhostVDPAState, nc, nc); + VhostShadowVirtqueue *svq; + void *out_cursor, *in_cursor; struct vhost_vdpa *v = &s->vhost_vdpa; const VirtIONet *n; - int r; + ssize_t need_poll_len = 0, r, dev_written; assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_VDPA); @@ -699,16 +732,50 @@ static int vhost_vdpa_net_load(NetClientState *nc) } n = VIRTIO_NET(v->dev->vdev); - r = vhost_vdpa_net_load_mac(s, n); + + need_poll_len = 0; + out_cursor = s->cvq_cmd_out_buffer; + in_cursor = s->status; + + r = vhost_vdpa_net_load_mac(s, n, &out_cursor, &in_cursor); if (unlikely(r < 0)) { - return r; + goto load_err; + } + need_poll_len += r; + + r = vhost_vdpa_net_load_mq(s, n, &out_cursor, &in_cursor); + if (unlikely(r < 0)) { + goto load_err; + } + need_poll_len += r; + +load_err: + /* Poll for all SVQ control commands used buffer from device */ + svq = g_ptr_array_index(s->vhost_vdpa.shadow_vqs, 0); + while (need_poll_len > 0) { + /* + * We can poll here since we've had BQL from the time we sent the + * descriptor. Also, we need to take the answer before SVQ pulls + * by itself, when BQL is released + */ + dev_written = vhost_svq_poll(svq); + need_poll_len -= dev_written; } - r = vhost_vdpa_net_load_mq(s, n); - if (unlikely(r)) { + + /* If code comes from `load_err` label, then we should return directly */ + if (unlikely(r < 0)) { return r; } - return 0; + /* Check the used buffer from device */ + for (virtio_net_ctrl_ack * status = s->status; (void *)status < in_cursor; + ++status) { + if (*status != VIRTIO_NET_OK) { + ++r; + } + } + + return r; } static NetClientInfo net_vhost_vdpa_cvq_info = {