From patchwork Tue Jun 5 19:07:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925603 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hDX6N4Nz9s15 for ; Wed, 6 Jun 2018 05:09:00 +1000 (AEST) Received: from localhost ([::1]:48633 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHKE-0001zT-FF for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:08:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42495) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJX-0001xa-No for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJW-0002yc-An for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:15 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46848 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJW-0002yA-3f for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:14 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 85F87126F88; Tue, 5 Jun 2018 19:08:13 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 491DF2026609; Tue, 5 Jun 2018 19:08:10 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:07:56 -0400 Message-Id: <1528225683-11413-2-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:13 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 1/8] virtio: feature bit, data structure, init for 1.1 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu New feature bit and members for packed ring. Signed-off-by: Wei Xu --- hw/net/vhost_net.c | 2 ++ hw/virtio/virtio.c | 27 ++++++++++++++++++++++++-- include/hw/virtio/virtio.h | 4 +++- include/standard-headers/linux/virtio_config.h | 2 ++ 4 files changed, 32 insertions(+), 3 deletions(-) diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index e037db6..f593086 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -53,6 +53,7 @@ static const int kernel_feature_bits[] = { VIRTIO_F_VERSION_1, VIRTIO_NET_F_MTU, VIRTIO_F_IOMMU_PLATFORM, + VIRTIO_F_RING_PACKED, VHOST_INVALID_FEATURE_BIT }; @@ -78,6 +79,7 @@ static const int user_feature_bits[] = { VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_MTU, VIRTIO_F_IOMMU_PLATFORM, + VIRTIO_F_RING_PACKED, /* This bit implies RARP isn't sent by QEMU out of band */ VIRTIO_NET_F_GUEST_ANNOUNCE, diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 006d3d1..e192a9a 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -39,6 +39,13 @@ typedef struct VRingDesc uint16_t next; } VRingDesc; +typedef struct VRingDescPacked { + uint64_t addr; + uint32_t len; + uint16_t id; + uint16_t flags; +} VRingDescPacked; + typedef struct VRingAvail { uint16_t flags; @@ -62,8 +69,14 @@ typedef struct VRingUsed typedef struct VRingMemoryRegionCaches { struct rcu_head rcu; MemoryRegionCache desc; - MemoryRegionCache avail; - MemoryRegionCache used; + union { + MemoryRegionCache avail; + MemoryRegionCache driver; + }; + union { + MemoryRegionCache used; + MemoryRegionCache device; + }; } VRingMemoryRegionCaches; typedef struct VRing @@ -77,6 +90,11 @@ typedef struct VRing VRingMemoryRegionCaches *caches; } VRing; +typedef struct VRingPackedDescEvent { + uint16_t off_wrap; + uint16_t flags; +} VRingPackedDescEvent ; + struct VirtQueue { VRing vring; @@ -89,6 +107,9 @@ struct VirtQueue uint16_t used_idx; + bool avail_wrap_counter; + bool used_wrap_counter; + /* Last used index value we have signalled on */ uint16_t signalled_used; @@ -1213,6 +1234,8 @@ void virtio_reset(void *opaque) vdev->vq[i].last_avail_idx = 0; vdev->vq[i].shadow_avail_idx = 0; vdev->vq[i].used_idx = 0; + vdev->vq[i].avail_wrap_counter = true; + vdev->vq[i].used_wrap_counter = true; virtio_queue_set_vector(vdev, i, VIRTIO_NO_VECTOR); vdev->vq[i].signalled_used = 0; vdev->vq[i].signalled_used_valid = false; diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 098bdaa..4a7fb21 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -262,7 +262,9 @@ typedef struct VirtIORNGConf VirtIORNGConf; DEFINE_PROP_BIT64("any_layout", _state, _field, \ VIRTIO_F_ANY_LAYOUT, true), \ DEFINE_PROP_BIT64("iommu_platform", _state, _field, \ - VIRTIO_F_IOMMU_PLATFORM, false) + VIRTIO_F_IOMMU_PLATFORM, false), \ + DEFINE_PROP_BIT64("ring_packed", _state, _field, \ + VIRTIO_F_RING_PACKED, false) hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n); hwaddr virtio_queue_get_avail_addr(VirtIODevice *vdev, int n); diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h index b777069..6ee5529 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -71,4 +71,6 @@ * this is for compatibility with legacy systems. */ #define VIRTIO_F_IOMMU_PLATFORM 33 + +#define VIRTIO_F_RING_PACKED 34 #endif /* _LINUX_VIRTIO_CONFIG_H */ From patchwork Tue Jun 5 19:07:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925606 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hHr2XDTz9s1p for ; Wed, 6 Jun 2018 05:11:52 +1000 (AEST) Received: from localhost ([::1]:48652 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHN0-00058k-36 for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:11:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42522) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJa-0001zh-FZ for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJZ-00030c-Fd for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:18 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46852 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJZ-00030B-A5 for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:17 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B559F14F898; Tue, 5 Jun 2018 19:08:16 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 279E0202698A; Tue, 5 Jun 2018 19:08:13 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:07:57 -0400 Message-Id: <1528225683-11413-3-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:16 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:16 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 2/8] virtio: memory cache for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu Mostly reuse memory cache with 1.0 except for the offset calculation. Signed-off-by: Wei Xu Reviewed-by: Paolo Bonzini --- hw/virtio/virtio.c | 29 ++++++++++++++++++++--------- 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index e192a9a..f6c0689 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -150,11 +150,8 @@ static void virtio_init_region_cache(VirtIODevice *vdev, int n) VRingMemoryRegionCaches *old = vq->vring.caches; VRingMemoryRegionCaches *new; hwaddr addr, size; - int event_size; int64_t len; - event_size = virtio_vdev_has_feature(vq->vdev, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0; - addr = vq->vring.desc; if (!addr) { return; @@ -168,7 +165,7 @@ static void virtio_init_region_cache(VirtIODevice *vdev, int n) goto err_desc; } - size = virtio_queue_get_used_size(vdev, n) + event_size; + size = virtio_queue_get_used_size(vdev, n); len = address_space_cache_init(&new->used, vdev->dma_as, vq->vring.used, size, true); if (len < size) { @@ -176,7 +173,7 @@ static void virtio_init_region_cache(VirtIODevice *vdev, int n) goto err_used; } - size = virtio_queue_get_avail_size(vdev, n) + event_size; + size = virtio_queue_get_avail_size(vdev, n); len = address_space_cache_init(&new->avail, vdev->dma_as, vq->vring.avail, size, false); if (len < size) { @@ -2320,14 +2317,28 @@ hwaddr virtio_queue_get_desc_size(VirtIODevice *vdev, int n) hwaddr virtio_queue_get_avail_size(VirtIODevice *vdev, int n) { - return offsetof(VRingAvail, ring) + - sizeof(uint16_t) * vdev->vq[n].vring.num; + int s; + + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + return sizeof(struct VRingPackedDescEvent); + } else { + s = virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0; + return offsetof(VRingAvail, ring) + + sizeof(uint16_t) * vdev->vq[n].vring.num + s; + } } hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) { - return offsetof(VRingUsed, ring) + - sizeof(VRingUsedElem) * vdev->vq[n].vring.num; + int s; + + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + return sizeof(struct VRingPackedDescEvent); + } else { + s = virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX) ? 2 : 0; + return offsetof(VRingUsed, ring) + + sizeof(VRingUsedElem) * vdev->vq[n].vring.num + s; + } } uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) From patchwork Tue Jun 5 19:07:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hLT710Jz9s0W for ; Wed, 6 Jun 2018 05:14:09 +1000 (AEST) Received: from localhost ([::1]:48679 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHPD-0007EO-L7 for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:14:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42561) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJf-00023h-Fk for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJc-00032g-Hw for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:23 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46868 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJc-00032C-BM for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:20 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DA608125933; Tue, 5 Jun 2018 19:08:19 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 522C62026DEF; Tue, 5 Jun 2018 19:08:17 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:07:58 -0400 Message-Id: <1528225683-11413-4-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:19 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:19 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 3/8] virtio: empty check and desc read for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu helper for ring empty check and descriptor read. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 59 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index f6c0689..bd669a2 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -24,6 +24,9 @@ #include "hw/virtio/virtio-access.h" #include "sysemu/dma.h" +#define AVAIL_DESC_PACKED(b) ((b) << 7) +#define USED_DESC_PACKED(b) ((b) << 15) + /* * The alignment to use between consumer and producer parts of vring. * x86 pagesize again. This is the default, used by transports like PCI @@ -357,10 +360,27 @@ int virtio_queue_ready(VirtQueue *vq) return vq->vring.avail != 0; } +static void vring_packed_desc_read(VirtIODevice *vdev, VRingDescPacked *desc, + MemoryRegionCache *cache, int i) +{ + address_space_read_cached(cache, i * sizeof(VRingDescPacked), + desc, sizeof(VRingDescPacked)); + virtio_tswap64s(vdev, &desc->addr); + virtio_tswap32s(vdev, &desc->len); + virtio_tswap16s(vdev, &desc->id); + virtio_tswap16s(vdev, &desc->flags); +} + +static inline bool is_desc_avail(struct VRingDescPacked *desc) +{ + return !!(desc->flags & AVAIL_DESC_PACKED(1)) != + !!(desc->flags & USED_DESC_PACKED(1)); +} + /* Fetch avail_idx from VQ memory only when we really need to know if * guest has added some buffers. * Called within rcu_read_lock(). */ -static int virtio_queue_empty_rcu(VirtQueue *vq) +static int virtio_queue_split_empty_rcu(VirtQueue *vq) { if (unlikely(!vq->vring.avail)) { return 1; @@ -373,7 +393,7 @@ static int virtio_queue_empty_rcu(VirtQueue *vq) return vring_avail_idx(vq) == vq->last_avail_idx; } -int virtio_queue_empty(VirtQueue *vq) +static int virtio_queue_split_empty(VirtQueue *vq) { bool empty; @@ -391,6 +411,42 @@ int virtio_queue_empty(VirtQueue *vq) return empty; } +static int virtio_queue_packed_empty_rcu(VirtQueue *vq) +{ + struct VRingDescPacked desc; + VRingMemoryRegionCaches *cache; + + if (unlikely(!vq->vring.desc)) { + return 1; + } + + cache = vring_get_region_caches(vq); + vring_packed_desc_read(vq->vdev, &desc, &cache->desc, vq->last_avail_idx); + + /* Make sure we see the updated flag */ + smp_mb(); + return !is_desc_avail(&desc); +} + +static int virtio_queue_packed_empty(VirtQueue *vq) +{ + bool empty; + + rcu_read_lock(); + empty = virtio_queue_packed_empty_rcu(vq); + rcu_read_unlock(); + return empty; +} + +int virtio_queue_empty(VirtQueue *vq) +{ + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + return virtio_queue_packed_empty(vq); + } else { + return virtio_queue_split_empty(vq); + } +} + static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem, unsigned int len) { @@ -862,7 +918,7 @@ void *virtqueue_pop(VirtQueue *vq, size_t sz) return NULL; } rcu_read_lock(); - if (virtio_queue_empty_rcu(vq)) { + if (virtio_queue_split_empty_rcu(vq)) { goto done; } /* Needed after virtio_queue_empty(), see comment in From patchwork Tue Jun 5 19:07:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925610 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hN91l76z9s15 for ; Wed, 6 Jun 2018 05:15:37 +1000 (AEST) Received: from localhost ([::1]:48689 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHQc-0008Kd-N4 for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:15:34 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42577) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJg-00024g-Kk for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJf-00034a-ER for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:24 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46872 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJf-00034D-7B for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:23 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B8CB812592A; Tue, 5 Jun 2018 19:08:22 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7BB4D2026609; Tue, 5 Jun 2018 19:08:20 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:07:59 -0400 Message-Id: <1528225683-11413-5-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:22 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:22 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 4/8] virtio: get avail bytes check for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu mostly as same as 1.0 except traversing all desc to feed headcount, need a refactor. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 148 +++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 145 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index bd669a2..cdbb5af 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -650,9 +650,9 @@ static int virtqueue_read_next_desc(VirtIODevice *vdev, VRingDesc *desc, return VIRTQUEUE_READ_DESC_MORE; } -void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, - unsigned int *out_bytes, - unsigned max_in_bytes, unsigned max_out_bytes) +static void virtqueue_split_get_avail_bytes(VirtQueue *vq, + unsigned int *in_bytes, unsigned int *out_bytes, + unsigned max_in_bytes, unsigned max_out_bytes) { VirtIODevice *vdev = vq->vdev; unsigned int max, idx; @@ -775,6 +775,148 @@ err: goto done; } +static void virtqueue_packed_get_avail_bytes(VirtQueue *vq, + unsigned int *in_bytes, unsigned int *out_bytes, + unsigned max_in_bytes, unsigned max_out_bytes) +{ + VirtIODevice *vdev = vq->vdev; + unsigned int max, idx; + unsigned int total_bufs, in_total, out_total; + MemoryRegionCache *desc_cache; + VRingMemoryRegionCaches *caches; + MemoryRegionCache indirect_desc_cache = MEMORY_REGION_CACHE_INVALID; + int64_t len = 0; + VRingDescPacked desc; + + if (unlikely(!vq->vring.desc)) { + if (in_bytes) { + *in_bytes = 0; + } + if (out_bytes) { + *out_bytes = 0; + } + return; + } + + rcu_read_lock(); + idx = vq->last_avail_idx; + total_bufs = in_total = out_total = 0; + + max = vq->vring.num; + caches = vring_get_region_caches(vq); + if (caches->desc.len < max * sizeof(VRingDescPacked)) { + virtio_error(vdev, "Cannot map descriptor ring"); + goto err; + } + + desc_cache = &caches->desc; + vring_packed_desc_read(vdev, &desc, desc_cache, idx); + while (is_desc_avail(&desc)) { + unsigned int num_bufs; + unsigned int i; + + num_bufs = total_bufs; + + if (desc.flags & VRING_DESC_F_INDIRECT) { + if (desc.len % sizeof(VRingDescPacked)) { + virtio_error(vdev, "Invalid size for indirect buffer table"); + goto err; + } + + /* If we've got too many, that implies a descriptor loop. */ + if (num_bufs >= max) { + virtio_error(vdev, "Looped descriptor"); + goto err; + } + + /* loop over the indirect descriptor table */ + len = address_space_cache_init(&indirect_desc_cache, + vdev->dma_as, + desc.addr, desc.len, false); + desc_cache = &indirect_desc_cache; + if (len < desc.len) { + virtio_error(vdev, "Cannot map indirect buffer"); + goto err; + } + + max = desc.len / sizeof(VRingDescPacked); + num_bufs = i = 0; + vring_packed_desc_read(vdev, &desc, desc_cache, i); + } + + do { + /* If we've got too many, that implies a descriptor loop. */ + if (++num_bufs > max) { + virtio_error(vdev, "Looped descriptor"); + goto err; + } + + if (desc.flags & VRING_DESC_F_WRITE) { + in_total += desc.len; + } else { + out_total += desc.len; + } + if (in_total >= max_in_bytes && out_total >= max_out_bytes) { + goto done; + } + + if (desc_cache == &indirect_desc_cache) { + if (++i >= vq->vring.num) { + i -= vq->vring.num; + } + vring_packed_desc_read(vdev, &desc, desc_cache, i); + } else { + if (++idx >= vq->vring.num) { + idx -= vq->vring.num; + } + vring_packed_desc_read(vdev, &desc, desc_cache, idx); + } + /* Make sure we see the flags */ + smp_mb(); + } while (desc.flags & VRING_DESC_F_NEXT); + + if (desc_cache == &indirect_desc_cache) { + address_space_cache_destroy(&indirect_desc_cache); + total_bufs++; + /* We missed one step on for indirect desc */ + idx++; + } else { + total_bufs = num_bufs; + } + + desc_cache = &caches->desc; + vring_packed_desc_read(vdev, &desc, desc_cache, idx % vq->vring.num); + } + +done: + address_space_cache_destroy(&indirect_desc_cache); + if (in_bytes) { + *in_bytes = in_total; + } + if (out_bytes) { + *out_bytes = out_total; + } + rcu_read_unlock(); + return; + +err: + in_total = out_total = 0; + goto done; +} + +void virtqueue_get_avail_bytes(VirtQueue *vq, unsigned int *in_bytes, + unsigned int *out_bytes, + unsigned max_in_bytes, unsigned max_out_bytes) +{ + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + virtqueue_packed_get_avail_bytes(vq, in_bytes, out_bytes, + max_in_bytes, max_out_bytes); + } else { + virtqueue_split_get_avail_bytes(vq, in_bytes, out_bytes, + max_in_bytes, max_out_bytes); + } +} + int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes, unsigned int out_bytes) { From patchwork Tue Jun 5 19:08:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hG16fnCz9s0W for ; Wed, 6 Jun 2018 05:10:17 +1000 (AEST) Received: from localhost ([::1]:48638 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHLT-0003bA-IR for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:10:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42595) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJj-000289-GQ for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJi-00036M-7W for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:27 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:46892 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJi-000361-1T for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:26 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 966A114F899; Tue, 5 Jun 2018 19:08:25 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 597412026609; Tue, 5 Jun 2018 19:08:23 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:08:00 -0400 Message-Id: <1528225683-11413-6-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:25 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Tue, 05 Jun 2018 19:08:25 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 5/8] virtio: queue pop for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 145 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 144 insertions(+), 1 deletion(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index cdbb5af..0160d03 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -1041,7 +1041,7 @@ static void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_nu return elem; } -void *virtqueue_pop(VirtQueue *vq, size_t sz) +static void *virtqueue_split_pop(VirtQueue *vq, size_t sz) { unsigned int i, head, max; VRingMemoryRegionCaches *caches; @@ -1176,6 +1176,149 @@ err_undo_map: goto done; } +static void *virtqueue_packed_pop(VirtQueue *vq, size_t sz) +{ + unsigned int i, head, max; + VRingMemoryRegionCaches *caches; + MemoryRegionCache indirect_desc_cache = MEMORY_REGION_CACHE_INVALID; + MemoryRegionCache *cache; + int64_t len; + VirtIODevice *vdev = vq->vdev; + VirtQueueElement *elem = NULL; + unsigned out_num, in_num, elem_entries; + hwaddr addr[VIRTQUEUE_MAX_SIZE]; + struct iovec iov[VIRTQUEUE_MAX_SIZE]; + VRingDescPacked desc; + + if (unlikely(vdev->broken)) { + return NULL; + } + + rcu_read_lock(); + if (virtio_queue_packed_empty_rcu(vq)) { + goto done; + } + + /* When we start there are none of either input nor output. */ + out_num = in_num = elem_entries = 0; + + max = vq->vring.num; + + if (vq->inuse >= vq->vring.num) { + virtio_error(vdev, "Virtqueue size exceeded"); + goto done; + } + + if (virtio_vdev_has_feature(vdev, VIRTIO_RING_F_EVENT_IDX)) { + /* FIXME: TBD */ + } + + head = vq->last_avail_idx; + i = head; + + caches = vring_get_region_caches(vq); + cache = &caches->desc; + vring_packed_desc_read(vdev, &desc, cache, i); + if (desc.flags & VRING_DESC_F_INDIRECT) { + if (desc.len % sizeof(VRingDescPacked)) { + virtio_error(vdev, "Invalid size for indirect buffer table"); + goto done; + } + + /* loop over the indirect descriptor table */ + len = address_space_cache_init(&indirect_desc_cache, vdev->dma_as, + desc.addr, desc.len, false); + cache = &indirect_desc_cache; + if (len < desc.len) { + virtio_error(vdev, "Cannot map indirect buffer"); + goto done; + } + + max = desc.len / sizeof(VRingDescPacked); + i = 0; + vring_packed_desc_read(vdev, &desc, cache, i); + } + + /* Collect all the descriptors */ + while (1) { + bool map_ok; + + if (desc.flags & VRING_DESC_F_WRITE) { + map_ok = virtqueue_map_desc(vdev, &in_num, addr + out_num, + iov + out_num, + VIRTQUEUE_MAX_SIZE - out_num, true, + desc.addr, desc.len); + } else { + if (in_num) { + virtio_error(vdev, "Incorrect order for descriptors"); + goto err_undo_map; + } + map_ok = virtqueue_map_desc(vdev, &out_num, addr, iov, + VIRTQUEUE_MAX_SIZE, false, + desc.addr, desc.len); + } + if (!map_ok) { + goto err_undo_map; + } + + /* If we've got too many, that implies a descriptor loop. */ + if (++elem_entries > max) { + virtio_error(vdev, "Looped descriptor"); + goto err_undo_map; + } + + if (++i >= vq->vring.num) { + i -= vq->vring.num; + } + + if (desc.flags & VRING_DESC_F_NEXT) { + vring_packed_desc_read(vq->vdev, &desc, cache, i); + } else { + break; + } + } + + /* Now copy what we have collected and mapped */ + elem = virtqueue_alloc_element(sz, out_num, in_num); + for (i = 0; i < out_num; i++) { + elem->out_addr[i] = addr[i]; + elem->out_sg[i] = iov[i]; + } + for (i = 0; i < in_num; i++) { + elem->in_addr[i] = addr[head + out_num + i]; + elem->in_sg[i] = iov[out_num + i]; + } + + vq->last_avail_idx += (cache == &indirect_desc_cache) ? + 1 : out_num + in_num; + if (vq->last_avail_idx >= vq->vring.num) { + vq->last_avail_idx -= vq->vring.num; + vq->avail_wrap_counter = !vq->avail_wrap_counter; + } + vq->inuse++; + + trace_virtqueue_pop(vq, elem, elem->in_num, elem->out_num); +done: + address_space_cache_destroy(&indirect_desc_cache); + rcu_read_unlock(); + + return elem; + +err_undo_map: + virtqueue_undo_map_desc(out_num, in_num, iov); + g_free(elem); + goto done; +} + +void *virtqueue_pop(VirtQueue *vq, size_t sz) +{ + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + return virtqueue_packed_pop(vq, sz); + } else { + return virtqueue_split_pop(vq, sz); + } +} + /* virtqueue_drop_all: * @vq: The #VirtQueue * Drops all queued buffers and indicates them to the guest From patchwork Tue Jun 5 19:08:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925607 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hK438Wrz9s0W for ; Wed, 6 Jun 2018 05:12:56 +1000 (AEST) Received: from localhost ([::1]:48663 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHO2-00066Q-8M for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:12:54 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJq-0002FZ-7q for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJl-00038b-AD for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:34 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:50274 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJl-00038A-3x for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:29 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C7629407572D; Tue, 5 Jun 2018 19:08:28 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 36A8C202698A; Tue, 5 Jun 2018 19:08:25 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:08:01 -0400 Message-Id: <1528225683-11413-7-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 05 Jun 2018 19:08:28 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Tue, 05 Jun 2018 19:08:28 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 6/8] virtio: flush/push for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu Signed-off-by: Wei Xu Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 109 ++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 96 insertions(+), 13 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 0160d03..6f2da83 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -371,6 +371,21 @@ static void vring_packed_desc_read(VirtIODevice *vdev, VRingDescPacked *desc, virtio_tswap16s(vdev, &desc->flags); } +static void vring_packed_desc_write(VirtIODevice *vdev, VRingDescPacked *desc, + MemoryRegionCache *cache, int i) +{ + virtio_tswap64s(vdev, &desc->addr); + virtio_tswap32s(vdev, &desc->len); + virtio_tswap16s(vdev, &desc->id); + virtio_tswap16s(vdev, &desc->flags); + address_space_write_cached(cache, + sizeof(VRingDescPacked) * i, desc, + sizeof(VRingDescPacked)); + address_space_cache_invalidate(cache, + sizeof(VRingDescPacked) * i, + sizeof(VRingDescPacked)); +} + static inline bool is_desc_avail(struct VRingDescPacked *desc) { return !!(desc->flags & AVAIL_DESC_PACKED(1)) != @@ -526,19 +541,11 @@ bool virtqueue_rewind(VirtQueue *vq, unsigned int num) } /* Called within rcu_read_lock(). */ -void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, +static void virtqueue_split_fill(VirtQueue *vq, const VirtQueueElement *elem, unsigned int len, unsigned int idx) { VRingUsedElem uelem; - trace_virtqueue_fill(vq, elem, len, idx); - - virtqueue_unmap_sg(vq, elem, len); - - if (unlikely(vq->vdev->broken)) { - return; - } - if (unlikely(!vq->vring.used)) { return; } @@ -550,16 +557,64 @@ void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, vring_used_write(vq, &uelem, idx); } -/* Called within rcu_read_lock(). */ -void virtqueue_flush(VirtQueue *vq, unsigned int count) +static void virtqueue_packed_fill(VirtQueue *vq, const VirtQueueElement *elem, + unsigned int len, unsigned int idx) { - uint16_t old, new; + uint16_t w, head; + VRingMemoryRegionCaches *caches; + VRingDescPacked desc = { + .addr = 0, + .flags = 0, + }; + + if (unlikely(!vq->vring.desc)) { + return; + } + + caches = vring_get_region_caches(vq); + head = vq->used_idx + idx; + head = head >= vq->vring.num ? (head - vq->vring.num) : head; + vring_packed_desc_read(vq->vdev, &desc, &caches->desc, head); + + w = (desc.flags & AVAIL_DESC_PACKED(1)) >> 7; + desc.flags &= ~(AVAIL_DESC_PACKED(1) | USED_DESC_PACKED(1)); + desc.flags |= AVAIL_DESC_PACKED(w) | USED_DESC_PACKED(w); + if (!(desc.flags & VRING_DESC_F_INDIRECT)) { + if (!(desc.flags & VRING_DESC_F_WRITE)) { + desc.len = 0; + } else { + desc.len = len; + } + } + vring_packed_desc_write(vq->vdev, &desc, &caches->desc, head); + + /* Make sure flags has been updated */ + smp_mb(); +} + +void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem, + unsigned int len, unsigned int idx) +{ + trace_virtqueue_fill(vq, elem, len, idx); + + virtqueue_unmap_sg(vq, elem, len); if (unlikely(vq->vdev->broken)) { - vq->inuse -= count; return; } + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + virtqueue_packed_fill(vq, elem, len, idx); + } else { + virtqueue_split_fill(vq, elem, len, idx); + } +} + +/* Called within rcu_read_lock(). */ +static void virtqueue_split_flush(VirtQueue *vq, unsigned int count) +{ + uint16_t old, new; + if (unlikely(!vq->vring.used)) { return; } @@ -575,6 +630,34 @@ void virtqueue_flush(VirtQueue *vq, unsigned int count) vq->signalled_used_valid = false; } +static void virtqueue_packed_flush(VirtQueue *vq, unsigned int count) +{ + if (unlikely(!vq->vring.desc)) { + return; + } + + vq->inuse -= count; + vq->used_idx += count; + if (vq->used_idx >= vq->vring.num) { + vq->used_idx -= vq->vring.num; + vq->used_wrap_counter = !vq->used_wrap_counter; + } +} + +void virtqueue_flush(VirtQueue *vq, unsigned int count) +{ + if (unlikely(vq->vdev->broken)) { + vq->inuse -= count; + return; + } + + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + virtqueue_packed_flush(vq, count); + } else { + virtqueue_split_flush(vq, count); + } +} + void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem, unsigned int len) { From patchwork Tue Jun 5 19:08:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925605 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hHq15R7z9s1p for ; Wed, 6 Jun 2018 05:11:51 +1000 (AEST) Received: from localhost ([::1]:48653 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHMz-00058r-0F for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:11:49 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42632) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJq-0002Fb-8q for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJo-0003C8-5a for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:34 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:47186 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJn-0003Be-Vq for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:32 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A0E4640201CA; Tue, 5 Jun 2018 19:08:31 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 640CF2026DEF; Tue, 5 Jun 2018 19:08:29 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:08:02 -0400 Message-Id: <1528225683-11413-8-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 05 Jun 2018 19:08:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 05 Jun 2018 19:08:31 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 7/8] virtio: event suppression for packed ring X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu Signed-off-by: Wei Xu Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 115 +++++++++++++++++++++++-- include/standard-headers/linux/virtio_config.h | 13 +++ 2 files changed, 119 insertions(+), 9 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 6f2da83..4543974 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -226,6 +226,24 @@ static void vring_desc_read(VirtIODevice *vdev, VRingDesc *desc, virtio_tswap16s(vdev, &desc->next); } +static void vring_packed_event_read(VirtIODevice *vdev, + MemoryRegionCache *cache, VRingPackedDescEvent *e) +{ + address_space_read_cached(cache, 0, e, sizeof(*e)); + virtio_tswap16s(vdev, &e->off_wrap); + virtio_tswap16s(vdev, &e->flags); +} + +static void vring_packed_event_write(VirtIODevice *vdev, + MemoryRegionCache *cache, VRingPackedDescEvent *e) +{ + virtio_tswap16s(vdev, &e->off_wrap); + virtio_tswap16s(vdev, &e->flags); + address_space_write_cached(cache, 0, e, sizeof(*e)); + address_space_cache_invalidate(cache, 0, sizeof(VRingUsedElem)); +} + + static VRingMemoryRegionCaches *vring_get_region_caches(struct VirtQueue *vq) { VRingMemoryRegionCaches *caches = atomic_rcu_read(&vq->vring.caches); @@ -332,14 +350,8 @@ static inline void vring_set_avail_event(VirtQueue *vq, uint16_t val) address_space_cache_invalidate(&caches->used, pa, sizeof(val)); } -void virtio_queue_set_notification(VirtQueue *vq, int enable) +static void virtio_queue_set_notification_split(VirtQueue *vq, int enable) { - vq->notification = enable; - - if (!vq->vring.desc) { - return; - } - rcu_read_lock(); if (virtio_vdev_has_feature(vq->vdev, VIRTIO_RING_F_EVENT_IDX)) { vring_set_avail_event(vq, vring_avail_idx(vq)); @@ -355,6 +367,38 @@ void virtio_queue_set_notification(VirtQueue *vq, int enable) rcu_read_unlock(); } +static void virtio_queue_set_notification_packed(VirtQueue *vq, int enable) +{ + VRingPackedDescEvent e; + VRingMemoryRegionCaches *caches; + + rcu_read_lock(); + caches = vring_get_region_caches(vq); + vring_packed_event_read(vq->vdev, &caches->device, &e); + if (enable) { + e.flags = RING_EVENT_FLAGS_ENABLE; + } else { + e.flags = RING_EVENT_FLAGS_DISABLE; + } + vring_packed_event_write(vq->vdev, &caches->device, &e); + rcu_read_unlock(); +} + +void virtio_queue_set_notification(VirtQueue *vq, int enable) +{ + vq->notification = enable; + + if (!vq->vring.desc) { + return; + } + + if (virtio_vdev_has_feature(vq->vdev, VIRTIO_F_RING_PACKED)) { + virtio_queue_set_notification_packed(vq, enable); + } else { + virtio_queue_set_notification_split(vq, enable); + } +} + int virtio_queue_ready(VirtQueue *vq) { return vq->vring.avail != 0; @@ -2059,8 +2103,7 @@ static void virtio_set_isr(VirtIODevice *vdev, int value) } } -/* Called within rcu_read_lock(). */ -static bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq) +static bool virtio_split_should_notify(VirtIODevice *vdev, VirtQueue *vq) { uint16_t old, new; bool v; @@ -2083,6 +2126,60 @@ static bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq) return !v || vring_need_event(vring_get_used_event(vq), new, old); } +static bool vring_packed_need_event(VirtQueue *vq, uint16_t off_wrap, + uint16_t new, uint16_t old) +{ + bool wrap = vq->used_wrap_counter; + int off = off_wrap & ~(1 << 15); + + if (new < old) { + new += vq->vring.num; + wrap ^= 1; + } + + if (wrap != off_wrap >> 15) { + off += vq->vring.num; + } + + return vring_need_event(off, new, old); +} + +static bool virtio_packed_should_notify(VirtIODevice *vdev, VirtQueue *vq) +{ + VRingPackedDescEvent e; + uint16_t old, new; + bool v; + VRingMemoryRegionCaches *caches; + + caches = vring_get_region_caches(vq); + vring_packed_event_read(vdev, &caches->driver, &e); + + /* Make sure we see the updated flags */ + smp_mb(); + if (e.flags == RING_EVENT_FLAGS_DISABLE) { + return false; + } else if (e.flags == RING_EVENT_FLAGS_ENABLE) { + return true; + } + + v = vq->signalled_used_valid; + vq->signalled_used_valid = true; + old = vq->signalled_used; + new = vq->signalled_used = vq->used_idx; + + return !v || vring_packed_need_event(vq, e.off_wrap, new, old); +} + +/* Called within rcu_read_lock(). */ +static bool virtio_should_notify(VirtIODevice *vdev, VirtQueue *vq) +{ + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + return virtio_packed_should_notify(vdev, vq); + } else { + return virtio_split_should_notify(vdev, vq); + } +} + void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq) { bool should_notify; diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h index 6ee5529..53e5c83 100644 --- a/include/standard-headers/linux/virtio_config.h +++ b/include/standard-headers/linux/virtio_config.h @@ -73,4 +73,17 @@ #define VIRTIO_F_IOMMU_PLATFORM 33 #define VIRTIO_F_RING_PACKED 34 + +/* Enable events */ +#define RING_EVENT_FLAGS_ENABLE 0x0 +/* Disable events */ +#define RING_EVENT_FLAGS_DISABLE 0x1 +/* + * * Enable events for a specific descriptor + * * (as specified by Descriptor Ring Change Event Offset/Wrap Counter). + * * Only valid if VIRTIO_F_RING_EVENT_IDX has been negotiated. + * */ +#define RING_EVENT_FLAGS_DESC 0x2 +/* The value 0x3 is reserved */ + #endif /* _LINUX_VIRTIO_CONFIG_H */ From patchwork Tue Jun 5 19:08:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Xu X-Patchwork-Id: 925611 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=208.118.235.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 410hQB0Hysz9s0W for ; Wed, 6 Jun 2018 05:17:22 +1000 (AEST) Received: from localhost ([::1]:48699 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHSJ-0001q1-SZ for incoming@patchwork.ozlabs.org; Tue, 05 Jun 2018 15:17:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42650) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fQHJs-0002Hp-U7 for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:37 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fQHJr-0003FN-U2 for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:36 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:56678 helo=mx1.redhat.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fQHJr-0003EO-4z for qemu-devel@nongnu.org; Tue, 05 Jun 2018 15:08:35 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CA8C880401A8; Tue, 5 Jun 2018 19:08:34 +0000 (UTC) Received: from dell-per430-12.lab.eng.pek2.redhat.com (dell-per430-12.lab.eng.pek2.redhat.com [10.73.196.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id 41B032026609; Tue, 5 Jun 2018 19:08:31 +0000 (UTC) From: wexu@redhat.com To: qemu-devel@nongnu.org Date: Tue, 5 Jun 2018 15:08:03 -0400 Message-Id: <1528225683-11413-9-git-send-email-wexu@redhat.com> In-Reply-To: <1528225683-11413-1-git-send-email-wexu@redhat.com> References: <1528225683-11413-1-git-send-email-wexu@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 05 Jun 2018 19:08:34 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Tue, 05 Jun 2018 19:08:34 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'wexu@redhat.com' RCPT:'' X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.187.233.73 Subject: [Qemu-devel] [RFC v2 8/8] virtio: guest driver reload for vhost-net X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jasowang@redhat.com, jfreimann@redhat.com, wexu@redhat.com, tiwei.bie@intel.com, mst@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Wei Xu last_avail, avail_wrap_count, used_idx and used_wrap_count are needed to support vhost-net backend, all these are either 16 or bool variables, since state.num is 64bit wide, so here it is possible to put them to the 'num' without introducing a new case while handling ioctl. Unload/Reload test has been done successfully with a patch in vhost kernel. Signed-off-by: Wei Xu --- hw/virtio/virtio.c | 42 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 4543974..153f6d7 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2862,33 +2862,59 @@ hwaddr virtio_queue_get_used_size(VirtIODevice *vdev, int n) } } -uint16_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) +uint64_t virtio_queue_get_last_avail_idx(VirtIODevice *vdev, int n) { - return vdev->vq[n].last_avail_idx; + uint64_t num; + + num = vdev->vq[n].last_avail_idx; + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + num |= ((uint64_t)vdev->vq[n].avail_wrap_counter) << 16; + num |= ((uint64_t)vdev->vq[n].used_idx) << 32; + num |= ((uint64_t)vdev->vq[n].used_wrap_counter) << 48; + } + + return num; } -void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint16_t idx) +void virtio_queue_set_last_avail_idx(VirtIODevice *vdev, int n, uint64_t num) { - vdev->vq[n].last_avail_idx = idx; - vdev->vq[n].shadow_avail_idx = idx; + vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx = (uint16_t)(num); + + if (virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { + vdev->vq[n].avail_wrap_counter = (uint16_t)(num >> 16); + vdev->vq[n].used_idx = (uint16_t)(num >> 32); + vdev->vq[n].used_wrap_counter = (uint16_t)(num >> 48); + } } void virtio_queue_restore_last_avail_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); - if (vdev->vq[n].vring.desc) { + if (!vdev->vq[n].vring.desc) { + goto out; + } + + if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].last_avail_idx = vring_used_idx(&vdev->vq[n]); - vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; } + vdev->vq[n].shadow_avail_idx = vdev->vq[n].last_avail_idx; + +out: rcu_read_unlock(); } void virtio_queue_update_used_idx(VirtIODevice *vdev, int n) { rcu_read_lock(); - if (vdev->vq[n].vring.desc) { + if (!vdev->vq[n].vring.desc) { + goto out; + } + + if (!virtio_vdev_has_feature(vdev, VIRTIO_F_RING_PACKED)) { vdev->vq[n].used_idx = vring_used_idx(&vdev->vq[n]); } + +out: rcu_read_unlock(); }