From patchwork Thu Mar 1 10:31:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879702 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTlr40Mcz9s15 for ; Thu, 1 Mar 2018 21:52:24 +1100 (AEDT) Received: from localhost ([::1]:55514 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLp0-0006Rx-EK for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:52:22 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52668) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlh-00048Z-Me for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLld-0007eF-5O for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:57 -0500 Received: from mga14.intel.com ([192.55.52.115]:7310) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlc-0007b8-QI for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:53 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:48:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611601" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:50 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:51 +0800 Message-Id: <1519900322-30263-2-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 01/12] memory: rename existing iommu notifier to be iommu mr notifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu IOMMU notifiers before are mostly used for [dev-]IOTLB stuffs. It is not suitable for other kind of notifiers (one example would be the future virt-svm support). Considering that current notifiers are targeted for per memory region, renaming the iommu notifier definitions. This patch has following changes: * all the notifier types from IOMMU_NOTIFIER_* prefix into IOMMU_MR_EVENT_* to better show its usage (for memory regions). * rename IOMMUNotifier to IOMMUMRNotifier * rename iommu_notifier to iommu_mr_notifier Signed-off-by: Peter Xu Signed-off-by: Liu, Yi L --- hw/i386/amd_iommu.c | 6 ++--- hw/i386/intel_iommu.c | 34 +++++++++++++------------- hw/ppc/spapr_iommu.c | 8 +++---- hw/s390x/s390-pci-bus.c | 2 +- hw/vfio/common.c | 10 ++++---- hw/virtio/vhost.c | 10 ++++---- include/exec/memory.h | 55 ++++++++++++++++++++++--------------------- include/hw/i386/intel_iommu.h | 8 +++---- include/hw/vfio/vfio-common.h | 2 +- include/hw/virtio/vhost.h | 4 ++-- memory.c | 37 +++++++++++++++-------------- 11 files changed, 89 insertions(+), 87 deletions(-) diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c index 63d46ff..7bfde37 100644 --- a/hw/i386/amd_iommu.c +++ b/hw/i386/amd_iommu.c @@ -1075,12 +1075,12 @@ static const MemoryRegionOps mmio_mem_ops = { }; static void amdvi_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, - IOMMUNotifierFlag old, - IOMMUNotifierFlag new) + IOMMUMREventFlag old, + IOMMUMREventFlag new) { AMDVIAddressSpace *as = container_of(iommu, AMDVIAddressSpace, iommu); - if (new & IOMMU_NOTIFIER_MAP) { + if (new & IOMMU_MR_EVENT_MAP) { error_report("device %02x.%02x.%x requires iommu notifier which is not " "currently supported", as->bus_num, PCI_SLOT(as->devfn), PCI_FUNC(as->devfn)); diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 2e841cd..9edf392 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -1230,7 +1230,7 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s) static void vtd_iommu_replay_all(IntelIOMMUState *s) { - IntelIOMMUNotifierNode *node; + IntelIOMMUMRNotifierNode *node; QLIST_FOREACH(node, &s->notifiers_list, next) { memory_region_iommu_replay_all(&node->vtd_as->iommu); @@ -1304,7 +1304,7 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, /* * So a device is moving out of (or moving into) a * domain, a replay() suites here to notify all the - * IOMMU_NOTIFIER_MAP registers about this change. + * IOMMU_MR_EVENT_MAP registers about this change. * This won't bring bad even if we have no such * notifier registered - the IOMMU notification * framework will skip MAP notifications if that @@ -1354,7 +1354,7 @@ static void vtd_iotlb_global_invalidate(IntelIOMMUState *s) static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) { - IntelIOMMUNotifierNode *node; + IntelIOMMUMRNotifierNode *node; VTDContextEntry ce; VTDAddressSpace *vtd_as; @@ -1384,7 +1384,7 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, uint16_t domain_id, hwaddr addr, uint8_t am) { - IntelIOMMUNotifierNode *node; + IntelIOMMUMRNotifierNode *node; VTDContextEntry ce; int ret; @@ -2314,21 +2314,21 @@ static IOMMUTLBEntry vtd_iommu_translate(IOMMUMemoryRegion *iommu, hwaddr addr, } static void vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, - IOMMUNotifierFlag old, - IOMMUNotifierFlag new) + IOMMUMREventFlag old, + IOMMUMREventFlag new) { VTDAddressSpace *vtd_as = container_of(iommu, VTDAddressSpace, iommu); IntelIOMMUState *s = vtd_as->iommu_state; - IntelIOMMUNotifierNode *node = NULL; - IntelIOMMUNotifierNode *next_node = NULL; + IntelIOMMUMRNotifierNode *node = NULL; + IntelIOMMUMRNotifierNode *next_node = NULL; - if (!s->caching_mode && new & IOMMU_NOTIFIER_MAP) { + if (!s->caching_mode && new & IOMMU_MR_EVENT_MAP) { error_report("We need to set caching-mode=1 for intel-iommu to enable " "device assignment with IOMMU protection."); exit(1); } - if (old == IOMMU_NOTIFIER_NONE) { + if (old == IOMMU_MR_EVENT_NONE) { node = g_malloc0(sizeof(*node)); node->vtd_as = vtd_as; QLIST_INSERT_HEAD(&s->notifiers_list, node, next); @@ -2338,7 +2338,7 @@ static void vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, /* update notifier node with new flags */ QLIST_FOREACH_SAFE(node, &s->notifiers_list, next, next_node) { if (node->vtd_as == vtd_as) { - if (new == IOMMU_NOTIFIER_NONE) { + if (new == IOMMU_MR_EVENT_NONE) { QLIST_REMOVE(node, next); g_free(node); } @@ -2757,7 +2757,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) } /* Unmap the whole range in the notifier's scope. */ -static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) +static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUMRNotifier *n) { IOMMUTLBEntry entry; hwaddr size; @@ -2813,13 +2813,13 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) static void vtd_address_space_unmap_all(IntelIOMMUState *s) { - IntelIOMMUNotifierNode *node; + IntelIOMMUMRNotifierNode *node; VTDAddressSpace *vtd_as; - IOMMUNotifier *n; + IOMMUMRNotifier *n; QLIST_FOREACH(node, &s->notifiers_list, next) { vtd_as = node->vtd_as; - IOMMU_NOTIFIER_FOREACH(n, &vtd_as->iommu) { + IOMMU_MR_NOTIFIER_FOREACH(n, &vtd_as->iommu) { vtd_address_space_unmap(vtd_as, n); } } @@ -2827,11 +2827,11 @@ static void vtd_address_space_unmap_all(IntelIOMMUState *s) static int vtd_replay_hook(IOMMUTLBEntry *entry, void *private) { - memory_region_notify_one((IOMMUNotifier *)private, entry); + memory_region_notify_one((IOMMUMRNotifier *)private, entry); return 0; } -static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) +static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUMRNotifier *n) { VTDAddressSpace *vtd_as = container_of(iommu_mr, VTDAddressSpace, iommu); IntelIOMMUState *s = vtd_as->iommu_state; diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c index aaa6010..74cddc3 100644 --- a/hw/ppc/spapr_iommu.c +++ b/hw/ppc/spapr_iommu.c @@ -174,14 +174,14 @@ static int spapr_tce_get_attr(IOMMUMemoryRegion *iommu, } static void spapr_tce_notify_flag_changed(IOMMUMemoryRegion *iommu, - IOMMUNotifierFlag old, - IOMMUNotifierFlag new) + IOMMUMREventFlag old, + IOMMUMREventFlag new) { struct sPAPRTCETable *tbl = container_of(iommu, sPAPRTCETable, iommu); - if (old == IOMMU_NOTIFIER_NONE && new != IOMMU_NOTIFIER_NONE) { + if (old == IOMMU_MR_EVENT_NONE && new != IOMMU_MR_EVENT_NONE) { spapr_tce_set_need_vfio(tbl, true); - } else if (old != IOMMU_NOTIFIER_NONE && new == IOMMU_NOTIFIER_NONE) { + } else if (old != IOMMU_MR_EVENT_NONE && new == IOMMU_MR_EVENT_NONE) { spapr_tce_set_need_vfio(tbl, false); } } diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 77a50ca..1bad7ab 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -541,7 +541,7 @@ err: } static void s390_pci_iommu_replay(IOMMUMemoryRegion *iommu, - IOMMUNotifier *notifier) + IOMMUMRNotifier *notifier) { /* It's impossible to plug a pci device on s390x that already has iommu * mappings which need to be replayed, that is due to the "one iommu per diff --git a/hw/vfio/common.c b/hw/vfio/common.c index f895e3c..cbda506 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -346,7 +346,7 @@ static bool vfio_get_vaddr(IOMMUTLBEntry *iotlb, void **vaddr, return true; } -static void vfio_iommu_map_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) +static void vfio_iommu_map_notify(IOMMUMRNotifier *n, IOMMUTLBEntry *iotlb) { VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); VFIOContainer *container = giommu->container; @@ -522,10 +522,10 @@ static void vfio_listener_region_add(MemoryListener *listener, llend = int128_add(int128_make64(section->offset_within_region), section->size); llend = int128_sub(llend, int128_one()); - iommu_notifier_init(&giommu->n, vfio_iommu_map_notify, - IOMMU_NOTIFIER_ALL, - section->offset_within_region, - int128_get64(llend)); + iommu_mr_notifier_init(&giommu->n, vfio_iommu_map_notify, + IOMMU_MR_EVENT_ALL, + section->offset_within_region, + int128_get64(llend)); QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); memory_region_register_iommu_notifier(section->mr, &giommu->n); diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 4a44e6e..49f0a26 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -579,7 +579,7 @@ static void vhost_region_addnop(MemoryListener *listener, vhost_region_add_section(dev, section); } -static void vhost_iommu_unmap_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) +static void vhost_iommu_unmap_notify(IOMMUMRNotifier *n, IOMMUTLBEntry *iotlb) { struct vhost_iommu *iommu = container_of(n, struct vhost_iommu, n); struct vhost_dev *hdev = iommu->hdev; @@ -607,10 +607,10 @@ static void vhost_iommu_region_add(MemoryListener *listener, end = int128_add(int128_make64(section->offset_within_region), section->size); end = int128_sub(end, int128_one()); - iommu_notifier_init(&iommu->n, vhost_iommu_unmap_notify, - IOMMU_NOTIFIER_UNMAP, - section->offset_within_region, - int128_get64(end)); + iommu_mr_notifier_init(&iommu->n, vhost_iommu_unmap_notify, + IOMMU_MR_EVENT_UNMAP, + section->offset_within_region, + int128_get64(end)); iommu->mr = section->mr; iommu->iommu_offset = section->offset_within_address_space - section->offset_within_region; diff --git a/include/exec/memory.h b/include/exec/memory.h index 15e8111..520d409 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -75,36 +75,36 @@ struct IOMMUTLBEntry { }; /* - * Bitmap for different IOMMUNotifier capabilities. Each notifier can + * Bitmap for different IOMMUMRNotifier capabilities. Each notifier can * register with one or multiple IOMMU Notifier capability bit(s). */ typedef enum { - IOMMU_NOTIFIER_NONE = 0, + IOMMU_MR_EVENT_NONE = 0, /* Notify cache invalidations */ - IOMMU_NOTIFIER_UNMAP = 0x1, + IOMMU_MR_EVENT_UNMAP = 0x1, /* Notify entry changes (newly created entries) */ - IOMMU_NOTIFIER_MAP = 0x2, -} IOMMUNotifierFlag; + IOMMU_MR_EVENT_MAP = 0x2, +} IOMMUMREventFlag; -#define IOMMU_NOTIFIER_ALL (IOMMU_NOTIFIER_MAP | IOMMU_NOTIFIER_UNMAP) +#define IOMMU_MR_EVENT_ALL (IOMMU_MR_EVENT_MAP | IOMMU_MR_EVENT_UNMAP) -struct IOMMUNotifier; -typedef void (*IOMMUNotify)(struct IOMMUNotifier *notifier, +struct IOMMUMRNotifier; +typedef void (*IOMMUMRNotify)(struct IOMMUMRNotifier *notifier, IOMMUTLBEntry *data); -struct IOMMUNotifier { - IOMMUNotify notify; - IOMMUNotifierFlag notifier_flags; +struct IOMMUMRNotifier { + IOMMUMRNotify notify; + IOMMUMREventFlag notifier_flags; /* Notify for address space range start <= addr <= end */ hwaddr start; hwaddr end; - QLIST_ENTRY(IOMMUNotifier) node; + QLIST_ENTRY(IOMMUMRNotifier) node; }; -typedef struct IOMMUNotifier IOMMUNotifier; +typedef struct IOMMUMRNotifier IOMMUMRNotifier; -static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, - IOMMUNotifierFlag flags, - hwaddr start, hwaddr end) +static inline void iommu_mr_notifier_init(IOMMUMRNotifier *n, IOMMUMRNotify fn, + IOMMUMREventFlag flags, + hwaddr start, hwaddr end) { n->notify = fn; n->notifier_flags = flags; @@ -210,10 +210,10 @@ typedef struct IOMMUMemoryRegionClass { uint64_t (*get_min_page_size)(IOMMUMemoryRegion *iommu); /* Called when IOMMU Notifier flag changed */ void (*notify_flag_changed)(IOMMUMemoryRegion *iommu, - IOMMUNotifierFlag old_flags, - IOMMUNotifierFlag new_flags); + IOMMUMREventFlag old_flags, + IOMMUMREventFlag new_flags); /* Set this up to provide customized IOMMU replay function */ - void (*replay)(IOMMUMemoryRegion *iommu, IOMMUNotifier *notifier); + void (*replay)(IOMMUMemoryRegion *iommu, IOMMUMRNotifier *notifier); /* Get IOMMU misc attributes */ int (*get_attr)(IOMMUMemoryRegion *iommu, enum IOMMUMemoryRegionAttr, @@ -267,11 +267,11 @@ struct MemoryRegion { struct IOMMUMemoryRegion { MemoryRegion parent_obj; - QLIST_HEAD(, IOMMUNotifier) iommu_notify; - IOMMUNotifierFlag iommu_notify_flags; + QLIST_HEAD(, IOMMUMRNotifier) iommu_notify; + IOMMUMREventFlag iommu_notify_flags; }; -#define IOMMU_NOTIFIER_FOREACH(n, mr) \ +#define IOMMU_MR_NOTIFIER_FOREACH(n, mr) \ QLIST_FOREACH((n), &(mr)->iommu_notify, node) /** @@ -913,7 +913,7 @@ void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, * replaces all old entries for the same virtual I/O address range. * Deleted entries have .@perm == 0. */ -void memory_region_notify_one(IOMMUNotifier *notifier, +void memory_region_notify_one(IOMMUMRNotifier *notifier, IOMMUTLBEntry *entry); /** @@ -921,12 +921,12 @@ void memory_region_notify_one(IOMMUNotifier *notifier, * IOMMU translation entries. * * @mr: the memory region to observe - * @n: the IOMMUNotifier to be added; the notify callback receives a + * @n: the IOMMUMRNotifier to be added; the notify callback receives a * pointer to an #IOMMUTLBEntry as the opaque value; the pointer * ceases to be valid on exit from the notifier. */ void memory_region_register_iommu_notifier(MemoryRegion *mr, - IOMMUNotifier *n); + IOMMUMRNotifier *n); /** * memory_region_iommu_replay: replay existing IOMMU translations to @@ -936,7 +936,8 @@ void memory_region_register_iommu_notifier(MemoryRegion *mr, * @iommu_mr: the memory region to observe * @n: the notifier to which to replay iommu mappings */ -void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n); +void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, + IOMMUMRNotifier *n); /** * memory_region_iommu_replay_all: replay existing IOMMU translations @@ -955,7 +956,7 @@ void memory_region_iommu_replay_all(IOMMUMemoryRegion *iommu_mr); * @n: the notifier to be removed. */ void memory_region_unregister_iommu_notifier(MemoryRegion *mr, - IOMMUNotifier *n); + IOMMUMRNotifier *n); /** * memory_region_iommu_get_attr: return an IOMMU attr if get_attr() is diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 45ec891..1df6fa9 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -67,7 +67,7 @@ typedef union VTD_IR_TableEntry VTD_IR_TableEntry; typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress; typedef struct VTDIrq VTDIrq; typedef struct VTD_MSIMessage VTD_MSIMessage; -typedef struct IntelIOMMUNotifierNode IntelIOMMUNotifierNode; +typedef struct IntelIOMMUMRNotifierNode IntelIOMMUMRNotifierNode; /* Context-Entry */ struct VTDContextEntry { @@ -253,9 +253,9 @@ struct VTD_MSIMessage { /* When IR is enabled, all MSI/MSI-X data bits should be zero */ #define VTD_IR_MSI_DATA (0) -struct IntelIOMMUNotifierNode { +struct IntelIOMMUMRNotifierNode { VTDAddressSpace *vtd_as; - QLIST_ENTRY(IntelIOMMUNotifierNode) next; + QLIST_ENTRY(IntelIOMMUMRNotifierNode) next; }; /* The iommu (DMAR) device state struct */ @@ -295,7 +295,7 @@ struct IntelIOMMUState { GHashTable *vtd_as_by_busptr; /* VTDBus objects indexed by PCIBus* reference */ VTDBus *vtd_as_by_bus_num[VTD_PCI_BUS_MAX]; /* VTDBus objects indexed by bus number */ /* list of registered notifiers */ - QLIST_HEAD(, IntelIOMMUNotifierNode) notifiers_list; + QLIST_HEAD(, IntelIOMMUMRNotifierNode) notifiers_list; /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f3a2ac9..865e3e7 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -97,7 +97,7 @@ typedef struct VFIOGuestIOMMU { VFIOContainer *container; IOMMUMemoryRegion *iommu; hwaddr iommu_offset; - IOMMUNotifier n; + IOMMUMRNotifier n; QLIST_ENTRY(VFIOGuestIOMMU) giommu_next; } VFIOGuestIOMMU; diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index a7f449f..401ce60 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -42,7 +42,7 @@ struct vhost_iommu { struct vhost_dev *hdev; MemoryRegion *mr; hwaddr iommu_offset; - IOMMUNotifier n; + IOMMUMRNotifier n; QLIST_ENTRY(vhost_iommu) iommu_next; }; @@ -80,7 +80,7 @@ struct vhost_dev { struct vhost_log *log; QLIST_ENTRY(vhost_dev) entry; QLIST_HEAD(, vhost_iommu) iommu_list; - IOMMUNotifier n; + IOMMUMRNotifier n; const VhostDevConfigOps *config_ops; }; diff --git a/memory.c b/memory.c index 6515131..6882a78 100644 --- a/memory.c +++ b/memory.c @@ -1702,7 +1702,7 @@ void memory_region_init_iommu(void *_iommu_mr, iommu_mr = IOMMU_MEMORY_REGION(mr); mr->terminates = true; /* then re-forwards */ QLIST_INIT(&iommu_mr->iommu_notify); - iommu_mr->iommu_notify_flags = IOMMU_NOTIFIER_NONE; + iommu_mr->iommu_notify_flags = IOMMU_MR_EVENT_NONE; } static void memory_region_finalize(Object *obj) @@ -1799,12 +1799,12 @@ bool memory_region_is_logging(MemoryRegion *mr, uint8_t client) static void memory_region_update_iommu_notify_flags(IOMMUMemoryRegion *iommu_mr) { - IOMMUNotifierFlag flags = IOMMU_NOTIFIER_NONE; - IOMMUNotifier *iommu_notifier; + IOMMUMREventFlag flags = IOMMU_MR_EVENT_NONE; + IOMMUMRNotifier *iommu_mr_notifier; IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_GET_CLASS(iommu_mr); - IOMMU_NOTIFIER_FOREACH(iommu_notifier, iommu_mr) { - flags |= iommu_notifier->notifier_flags; + IOMMU_MR_NOTIFIER_FOREACH(iommu_mr_notifier, iommu_mr) { + flags |= iommu_mr_notifier->notifier_flags; } if (flags != iommu_mr->iommu_notify_flags && imrc->notify_flag_changed) { @@ -1817,7 +1817,7 @@ static void memory_region_update_iommu_notify_flags(IOMMUMemoryRegion *iommu_mr) } void memory_region_register_iommu_notifier(MemoryRegion *mr, - IOMMUNotifier *n) + IOMMUMRNotifier *n) { IOMMUMemoryRegion *iommu_mr; @@ -1828,7 +1828,7 @@ void memory_region_register_iommu_notifier(MemoryRegion *mr, /* We need to register for at least one bitfield */ iommu_mr = IOMMU_MEMORY_REGION(mr); - assert(n->notifier_flags != IOMMU_NOTIFIER_NONE); + assert(n->notifier_flags != IOMMU_MR_EVENT_NONE); assert(n->start <= n->end); QLIST_INSERT_HEAD(&iommu_mr->iommu_notify, n, node); memory_region_update_iommu_notify_flags(iommu_mr); @@ -1844,7 +1844,8 @@ uint64_t memory_region_iommu_get_min_page_size(IOMMUMemoryRegion *iommu_mr) return TARGET_PAGE_SIZE; } -void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) +void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, + IOMMUMRNotifier *n) { MemoryRegion *mr = MEMORY_REGION(iommu_mr); IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_GET_CLASS(iommu_mr); @@ -1875,15 +1876,15 @@ void memory_region_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) void memory_region_iommu_replay_all(IOMMUMemoryRegion *iommu_mr) { - IOMMUNotifier *notifier; + IOMMUMRNotifier *notifier; - IOMMU_NOTIFIER_FOREACH(notifier, iommu_mr) { + IOMMU_MR_NOTIFIER_FOREACH(notifier, iommu_mr) { memory_region_iommu_replay(iommu_mr, notifier); } } void memory_region_unregister_iommu_notifier(MemoryRegion *mr, - IOMMUNotifier *n) + IOMMUMRNotifier *n) { IOMMUMemoryRegion *iommu_mr; @@ -1896,10 +1897,10 @@ void memory_region_unregister_iommu_notifier(MemoryRegion *mr, memory_region_update_iommu_notify_flags(iommu_mr); } -void memory_region_notify_one(IOMMUNotifier *notifier, +void memory_region_notify_one(IOMMUMRNotifier *notifier, IOMMUTLBEntry *entry) { - IOMMUNotifierFlag request_flags; + IOMMUMREventFlag request_flags; /* * Skip the notification if the notification does not overlap @@ -1911,9 +1912,9 @@ void memory_region_notify_one(IOMMUNotifier *notifier, } if (entry->perm & IOMMU_RW) { - request_flags = IOMMU_NOTIFIER_MAP; + request_flags = IOMMU_MR_EVENT_MAP; } else { - request_flags = IOMMU_NOTIFIER_UNMAP; + request_flags = IOMMU_MR_EVENT_UNMAP; } if (notifier->notifier_flags & request_flags) { @@ -1924,12 +1925,12 @@ void memory_region_notify_one(IOMMUNotifier *notifier, void memory_region_notify_iommu(IOMMUMemoryRegion *iommu_mr, IOMMUTLBEntry entry) { - IOMMUNotifier *iommu_notifier; + IOMMUMRNotifier *iommu_mr_notifier; assert(memory_region_is_iommu(MEMORY_REGION(iommu_mr))); - IOMMU_NOTIFIER_FOREACH(iommu_notifier, iommu_mr) { - memory_region_notify_one(iommu_notifier, &entry); + IOMMU_MR_NOTIFIER_FOREACH(iommu_mr_notifier, iommu_mr) { + memory_region_notify_one(iommu_mr_notifier, &entry); } } From patchwork Thu Mar 1 10:31:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879700 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTlc3TS6z9s15 for ; Thu, 1 Mar 2018 21:52:12 +1100 (AEDT) Received: from localhost ([::1]:55510 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLoo-0006I0-FP for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:52:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52667) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlh-00048Y-Me for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:58 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlf-0007gp-3o for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:57 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLle-0007fT-Qy for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:55 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:48:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611609" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:52 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:52 +0800 Message-Id: <1519900322-30263-3-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 02/12] vfio: rename GuestIOMMU to be GuestIOMMUMR X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch renames GuestIOMMU to GuestIOMMUMR as the existing GuestIOMMU is for MemoryRegion related notifiers. Signed-off-by: Liu, Yi L --- hw/vfio/common.c | 17 +++++++++-------- include/hw/vfio/vfio-common.h | 8 ++++---- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index cbda506..06277d2 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -348,7 +348,7 @@ static bool vfio_get_vaddr(IOMMUTLBEntry *iotlb, void **vaddr, static void vfio_iommu_map_notify(IOMMUMRNotifier *n, IOMMUTLBEntry *iotlb) { - VFIOGuestIOMMU *giommu = container_of(n, VFIOGuestIOMMU, n); + VFIOGuestIOMMUMR *giommu = container_of(n, VFIOGuestIOMMUMR, n); VFIOContainer *container = giommu->container; hwaddr iova = iotlb->iova + giommu->iommu_offset; bool read_only; @@ -504,7 +504,7 @@ static void vfio_listener_region_add(MemoryListener *listener, memory_region_ref(section->mr); if (memory_region_is_iommu(section->mr)) { - VFIOGuestIOMMU *giommu; + VFIOGuestIOMMUMR *giommu; IOMMUMemoryRegion *iommu_mr = IOMMU_MEMORY_REGION(section->mr); trace_vfio_listener_region_add_iommu(iova, end); @@ -526,7 +526,7 @@ static void vfio_listener_region_add(MemoryListener *listener, IOMMU_MR_EVENT_ALL, section->offset_within_region, int128_get64(llend)); - QLIST_INSERT_HEAD(&container->giommu_list, giommu, giommu_next); + QLIST_INSERT_HEAD(&container->giommu_mr_list, giommu, giommu_next); memory_region_register_iommu_notifier(section->mr, &giommu->n); memory_region_iommu_replay(giommu->iommu, &giommu->n); @@ -593,9 +593,9 @@ static void vfio_listener_region_del(MemoryListener *listener, } if (memory_region_is_iommu(section->mr)) { - VFIOGuestIOMMU *giommu; + VFIOGuestIOMMUMR *giommu; - QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { + QLIST_FOREACH(giommu, &container->giommu_mr_list, giommu_next) { if (MEMORY_REGION(giommu->iommu) == section->mr && giommu->n.start == section->offset_within_region) { memory_region_unregister_iommu_notifier(section->mr, @@ -1017,7 +1017,7 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, container = g_malloc0(sizeof(*container)); container->space = space; container->fd = fd; - QLIST_INIT(&container->giommu_list); + QLIST_INIT(&container->giommu_mr_list); QLIST_INIT(&container->hostwin_list); if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) || ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { @@ -1206,11 +1206,12 @@ static void vfio_disconnect_container(VFIOGroup *group) if (QLIST_EMPTY(&container->group_list)) { VFIOAddressSpace *space = container->space; - VFIOGuestIOMMU *giommu, *tmp; + VFIOGuestIOMMUMR *giommu, *tmp; QLIST_REMOVE(container, next); - QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) { + QLIST_FOREACH_SAFE(giommu, &container->giommu_mr_list, + giommu_next, tmp) { memory_region_unregister_iommu_notifier( MEMORY_REGION(giommu->iommu), &giommu->n); QLIST_REMOVE(giommu, giommu_next); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 865e3e7..702a085 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -87,19 +87,19 @@ typedef struct VFIOContainer { * contiguous IOVA window. We may need to generalize that in * future */ - QLIST_HEAD(, VFIOGuestIOMMU) giommu_list; + QLIST_HEAD(, VFIOGuestIOMMUMR) giommu_mr_list; QLIST_HEAD(, VFIOHostDMAWindow) hostwin_list; QLIST_HEAD(, VFIOGroup) group_list; QLIST_ENTRY(VFIOContainer) next; } VFIOContainer; -typedef struct VFIOGuestIOMMU { +typedef struct VFIOGuestIOMMUMR { VFIOContainer *container; IOMMUMemoryRegion *iommu; hwaddr iommu_offset; IOMMUMRNotifier n; - QLIST_ENTRY(VFIOGuestIOMMU) giommu_next; -} VFIOGuestIOMMU; + QLIST_ENTRY(VFIOGuestIOMMUMR) giommu_next; +} VFIOGuestIOMMUMR; typedef struct VFIOHostDMAWindow { hwaddr min_iova; From patchwork Thu Mar 1 10:31:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879697 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsThn60fTz9s15 for ; Thu, 1 Mar 2018 21:49:45 +1100 (AEDT) Received: from localhost ([::1]:55497 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLmR-0004Bc-Ur for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:49:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52677) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLli-00048d-7T for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlg-0007iF-LY for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:58 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlg-0007fT-8o for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:56 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:48:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611614" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:53 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:53 +0800 Message-Id: <1519900322-30263-4-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 03/12] hw/core: introduce IOMMUSVAContext for virt-SVA X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Peter Xu This patch adds IOMMUSVAContext as an abstract for virt-SVA in Qemu. IOMMUSVAContext is per-PASID(Process Address Space Identity). A PASID Tagged AddressSpace should have an IOMMUSVAContext created for it. virt-SVA emulation for emulated SVA capable devices would use IOMMUSVAContext. And for assigned devices, Qemu also needs to propagate guest tlb flush to host through the sva_notifer based on IOMMUSVAContext. This patch proposes to include a sva_notifier list and an IOMMUSVAContextOps in IOMMUSVAContext. * The sva_notifier list would include tlb invalidate nofitifer to propagate guest's iotlb flush to host. * The first callback in IOMMUSVAContextOps would be an address translation callback. For the SVA aware DMAs issued by emulated SVA capable devices, it requires Qemu to emulate data read/write to guest process address space. Qemu needs to do address translation with guest process page table. So the IOMMUSVAContextOps.translate() callback would be helpful for emulating SVA capable devices. Note: to fulfill the IOMMUSVAContext based address translation framework, may duplicate quite a few existing MemoryRegion based translation code in Qemu. As this patchset is mainly to support assigned SVA capable devices. So this patchset hasn't done the duplication. In future, if any requirement for emulating SVA capable device, it would require a separate patchset to fulfill the translation framework. Signed-off-by: Peter Xu Signed-off-by: Liu, Yi L --- hw/core/Makefile.objs | 1 + hw/core/pasid.c | 64 ++++++++++++++++++++++++++++ include/hw/core/pasid.h | 110 ++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 175 insertions(+) create mode 100644 hw/core/pasid.c create mode 100644 include/hw/core/pasid.h diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs index 1240728..01989d2 100644 --- a/hw/core/Makefile.objs +++ b/hw/core/Makefile.objs @@ -6,6 +6,7 @@ common-obj-$(CONFIG_SOFTMMU) += fw-path-provider.o # irq.o needed for qdev GPIO handling: common-obj-y += irq.o common-obj-y += hotplug.o +common-obj-y += pasid.o common-obj-$(CONFIG_SOFTMMU) += nmi.o common-obj-$(CONFIG_EMPTY_SLOT) += empty_slot.o diff --git a/hw/core/pasid.c b/hw/core/pasid.c new file mode 100644 index 0000000..c4b0c5d --- /dev/null +++ b/hw/core/pasid.c @@ -0,0 +1,64 @@ +/* + * QEMU abstract of Shared Virtual Memory logic + * + * Copyright (C) 2018 Red Hat Inc. + * + * Authors: Peter Xu , + * Liu, Yi L + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#include "qemu/osdep.h" +#include "hw/core/pasid.h" + +void iommu_sva_notifier_register(IOMMUSVAContext *sva_ctx, + IOMMUSVANotifier *n, + IOMMUSVANotifyFn fn, + IOMMUSVAEvent event) +{ + n->event = event; + n->sva_notify = fn; + QLIST_INSERT_HEAD(&sva_ctx->sva_notifiers, n, node); + return; +} + +void iommu_sva_notifier_unregister(IOMMUSVAContext *sva_ctx, + IOMMUSVANotifier *notifier) +{ + IOMMUSVANotifier *cur, *next; + + QLIST_FOREACH_SAFE(cur, &sva_ctx->sva_notifiers, node, next) { + if (cur == notifier) { + QLIST_REMOVE(cur, node); + break; + } + } +} + +void iommu_sva_notify(IOMMUSVAContext *sva_ctx, IOMMUSVAEventData *event_data) +{ + IOMMUSVANotifier *cur; + + QLIST_FOREACH(cur, &sva_ctx->sva_notifiers, node) { + if ((cur->event == event_data->event) && cur->sva_notify) { + cur->sva_notify(cur, event_data); + } + } +} + +void iommu_sva_ctx_init(IOMMUSVAContext *sva_ctx) +{ + QLIST_INIT(&sva_ctx->sva_notifiers); +} diff --git a/include/hw/core/pasid.h b/include/hw/core/pasid.h new file mode 100644 index 0000000..4c7dccb --- /dev/null +++ b/include/hw/core/pasid.h @@ -0,0 +1,110 @@ +/* + * QEMU abstraction of Shared Virtual Memory + * + * Copyright (C) 2018 Red Hat Inc. + * + * Authors: Peter Xu , + * Liu, Yi L + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see . + */ + +#ifndef HW_PCI_PASID_H +#define HW_PCI_PASID_H + +#include "qemu/queue.h" +#ifndef CONFIG_USER_ONLY +#include "exec/hwaddr.h" +#endif + +typedef struct IOMMUSVAContext IOMMUSVAContext; + +enum IOMMUSVAEvent { + IOMMU_SVA_EVENT_TLB_INV, +}; +typedef enum IOMMUSVAEvent IOMMUSVAEvent; + +struct IOMMUSVAEventData { + IOMMUSVAEvent event; + uint64_t length; + void *data; +}; +typedef struct IOMMUSVAEventData IOMMUSVAEventData; + +typedef struct IOMMUSVANotifier IOMMUSVANotifier; + +typedef void (*IOMMUSVANotifyFn)(IOMMUSVANotifier *notifier, + IOMMUSVAEventData *event_data); + +typedef struct IOMMUSVATLBEntry IOMMUSVATLBEntry; + +/* See address_space_translate: bit 0 is read, bit 1 is write. */ +typedef enum { + IOMMU_SVA_NONE = 0, + IOMMU_SVA_RO = 1, + IOMMU_SVA_WO = 2, + IOMMU_SVA_RW = 3, +} IOMMUSVAAccessFlags; + +#define IOMMU_SVA_ACCESS_FLAG(r, w) (((r) ? IOMMU_SVA_RO : 0) | \ + ((w) ? IOMMU_SVA_WO : 0)) + +struct IOMMUSVATLBEntry { + AddressSpace *target_as; + hwaddr va; + hwaddr translated_addr; + hwaddr addr_mask; /* 0xfff = 4k translation */ + IOMMUSVAAccessFlags perm; +}; + +typedef struct IOMMUSVAContextOps IOMMUSVAContextOps; +struct IOMMUSVAContextOps { + /* Return a TLB entry that contains a given address. */ + IOMMUSVATLBEntry (*translate)(IOMMUSVAContext *sva_ctx, + hwaddr addr, bool is_write); +}; + +struct IOMMUSVANotifier { + IOMMUSVANotifyFn sva_notify; + /* + * What events we are listening to. Let's allow multiple event + * registrations from beginning. + */ + IOMMUSVAEvent event; + QLIST_ENTRY(IOMMUSVANotifier) node; +}; + +/* + * This stands for an IOMMU unit. Any translation device should have + * this struct inside its own structure to make sure it can leverage + * common IOMMU functionalities. + */ +struct IOMMUSVAContext { + uint32_t pasid; + QLIST_HEAD(, IOMMUSVANotifier) sva_notifiers; + const IOMMUSVAContextOps *sva_ctx_ops; +}; + +void iommu_sva_notifier_register(IOMMUSVAContext *sva_ctx, + IOMMUSVANotifier *n, + IOMMUSVANotifyFn fn, + IOMMUSVAEvent event); +void iommu_sva_notifier_unregister(IOMMUSVAContext *sva_ctx, + IOMMUSVANotifier *notifier); +void iommu_sva_notify(IOMMUSVAContext *sva_ctx, + IOMMUSVAEventData *event_data); + +void iommu_sva_ctx_init(IOMMUSVAContext *sva_ctx); + +#endif From patchwork Thu Mar 1 10:31:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879707 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTqz2vYjz9s15 for ; Thu, 1 Mar 2018 21:55:59 +1100 (AEDT) Received: from localhost ([::1]:55539 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLsT-0001FB-CU for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:55:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52687) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlj-00048p-5H for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLli-0007jJ-AU for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:59 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLli-0007fT-24 for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:48:58 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:48:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611618" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:55 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:54 +0800 Message-Id: <1519900322-30263-5-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 04/12] vfio/pci: add notify framework based on IOMMUSVAContext X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch introduces a notify framework for IOMMUSVAContext.sva_notifiers. Signed-off-by: Liu, Yi L --- hw/vfio/common.c | 1 + include/hw/vfio/vfio-common.h | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 06277d2..1cc96df 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1019,6 +1019,7 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, container->fd = fd; QLIST_INIT(&container->giommu_mr_list); QLIST_INIT(&container->hostwin_list); + QLIST_INIT(&container->gsva_ctx_list); if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) || ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 702a085..4c16b4c 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -29,6 +29,7 @@ #ifdef CONFIG_LINUX #include #endif +#include "hw/core/pasid.h" #define ERR_PREFIX "vfio error: %s: " #define WARN_PREFIX "vfio warning: %s: " @@ -88,6 +89,7 @@ typedef struct VFIOContainer { * future */ QLIST_HEAD(, VFIOGuestIOMMUMR) giommu_mr_list; + QLIST_HEAD(, VFIOGuestIOMMUSVAContext) gsva_ctx_list; QLIST_HEAD(, VFIOHostDMAWindow) hostwin_list; QLIST_HEAD(, VFIOGroup) group_list; QLIST_ENTRY(VFIOContainer) next; @@ -101,6 +103,13 @@ typedef struct VFIOGuestIOMMUMR { QLIST_ENTRY(VFIOGuestIOMMUMR) giommu_next; } VFIOGuestIOMMUMR; +typedef struct VFIOGuestIOMMUSVAContext { + VFIOContainer *container; + IOMMUSVAContext *sva_ctx; + IOMMUSVANotifier n; + QLIST_ENTRY(VFIOGuestIOMMUSVAContext) gsva_ctx_next; +} VFIOGuestIOMMUSVAContext; + typedef struct VFIOHostDMAWindow { hwaddr min_iova; hwaddr max_iova; From patchwork Thu Mar 1 10:31:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879696 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsThm4Plxz9s06 for ; Thu, 1 Mar 2018 21:49:44 +1100 (AEDT) Received: from localhost ([::1]:55498 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLmQ-0004CJ-NF for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:49:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52707) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLll-0004AC-Bw for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:02 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlk-0007mv-A9 for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:01 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlj-0007fT-Tz for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:00 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:48:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611622" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:57 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:55 +0800 Message-Id: <1519900322-30263-6-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 05/12] hw/pci: introduce PCISVAOps to PCIDevice X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch intoduces PCISVAOps for virt-SVA. So far, to setup virt-SVA for assigned SVA capable device, needs to config host translation structures. e.g. for VT-d, needs to set the guest pasid table to host and enable nested translation. Besides, vIOMMU emulator needs to forward guest's cache invalidation to host. On VT-d, it is guest's invalidation to 1st level translation related cache, such invalidation should be forwarded to host. Proposed PCISVAOps are: * sva_bind_guest_pasid_table: set the guest pasid table to host, and enable nested translation in host * sva_register_notifier: register sva_notifier to forward guest's cache invalidation to host * sva_unregister_notifier: unregister sva_notifier The PCISVAOps should be provided by vfio or modules alike. Mainly for assigned SVA capable devices. Take virt-SVA on VT-d as an exmaple: If a guest wants to setup virt-SVA for an assigned SVA capable device, it programs its context entry. vIOMMU emulator captures guest's context entry programming, and figure out the target device. vIOMMU emulator use the pci_device_sva_bind_pasid_table() API to bind the guest pasid table to host. Guest would also program its pasid table. vIOMMU emulator captures guest's pasid entry programming. In Qemu, needs to allocate an AddressSpace to stand for the pasid tagged address space and Qemu also needs to register sva_notifier to forward future cache invalidation request to host. Allocating AddressSpace to stand for the pasid tagged address space is for the emulation of emulated SVA capable devices. Emulated SVA capable devices may issue SVA aware DMAs, Qemu needs to emulate read/write to a pasid tagged AddressSpace. Thus needs an abstraction for such address space in Qemu. Signed-off-by: Liu, Yi L --- hw/pci/pci.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++ include/hw/pci/pci.h | 21 ++++++++++++++++++ 2 files changed, 81 insertions(+) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index e006b6a..157fe21 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2573,6 +2573,66 @@ void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) bus->iommu_opaque = opaque; } +void pci_setup_sva_ops(PCIDevice *dev, PCISVAOps *ops) +{ + if (dev) { + dev->sva_ops = ops; + } + return; +} + +void pci_device_sva_bind_pasid_table(PCIBus *bus, + int32_t devfn, uint64_t addr, uint32_t size) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_bind_pasid_table(bus, devfn, addr, size); + } + return; +} + +void pci_device_sva_register_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_register_notifier(bus, + devfn, + sva_ctx); + } + return; +} + +void pci_device_sva_unregister_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx) +{ + PCIDevice *dev; + + if (!bus) { + return; + } + + dev = bus->devices[devfn]; + if (dev && dev->sva_ops) { + dev->sva_ops->sva_unregister_notifier(bus, + devfn, + sva_ctx); + } + return; +} + static void pci_dev_get_w64(PCIBus *b, PCIDevice *dev, void *opaque) { Range *range = opaque; diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index d8c18c7..32889a4 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -10,6 +10,8 @@ #include "hw/pci/pcie.h" +#include "hw/core/pasid.h" + extern bool pci_available; /* PCI bus */ @@ -262,6 +264,16 @@ struct PCIReqIDCache { }; typedef struct PCIReqIDCache PCIReqIDCache; +typedef struct PCISVAOps PCISVAOps; +struct PCISVAOps { + void (*sva_bind_pasid_table)(PCIBus *bus, int32_t devfn, + uint64_t pasidt_addr, uint32_t size); + void (*sva_register_notifier)(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); + void (*sva_unregister_notifier)(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); +}; + struct PCIDevice { DeviceState qdev; @@ -351,6 +363,7 @@ struct PCIDevice { MSIVectorUseNotifier msix_vector_use_notifier; MSIVectorReleaseNotifier msix_vector_release_notifier; MSIVectorPollNotifier msix_vector_poll_notifier; + PCISVAOps *sva_ops; }; void pci_register_bar(PCIDevice *pci_dev, int region_num, @@ -477,6 +490,14 @@ typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); +void pci_setup_sva_ops(PCIDevice *dev, PCISVAOps *ops); +void pci_device_sva_bind_pasid_table(PCIBus *bus, int32_t devfn, + uint64_t pasidt_addr, uint32_t size); +void pci_device_sva_register_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); +void pci_device_sva_unregister_notifier(PCIBus *bus, int32_t devfn, + IOMMUSVAContext *sva_ctx); + static inline void pci_set_byte(uint8_t *config, uint8_t val) { From patchwork Thu Mar 1 10:31:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879701 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTlq18wFz9s06 for ; Thu, 1 Mar 2018 21:52:23 +1100 (AEDT) Received: from localhost ([::1]:55513 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLoz-0006RT-8D for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:52:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52720) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLln-0004BR-3u for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:07 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlm-0007q1-AG for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:03 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLll-0007fT-Vt for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:02 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611625" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:48:59 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:56 +0800 Message-Id: <1519900322-30263-7-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 06/12] vfio/pci: provide vfio_pci_sva_ops instance X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" VFIO is the bridge for vIOMMU and host IOMMU. Needs to provide API for vIOMMU emulator to set configs to host IOMMU. In this patchset, such API is exposed in hw/pci. Signed-off-by: Liu, Yi L --- hw/vfio/pci.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 033cc8d..a60a4d7 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2775,6 +2775,34 @@ static void vfio_unregister_req_notifier(VFIOPCIDevice *vdev) vdev->req_enabled = false; } +static void vfio_pci_device_sva_bind_pasid_table(PCIBus *bus, + int32_t devfn, uint64_t pasidt_addr, uint32_t size) +{ + /* Propagate the guest pasid table pointer to host IOMMU, and + enable nested translation accordingly. Depends on HW design. + So far, Intel VT-d and AMD IOMMU requires it. */ +} + +static void vfio_pci_device_sva_register_notifier(PCIBus *bus, + int32_t devfn, IOMMUSVAContext *sva_ctx) +{ + /* Register notifier for TLB invalidation propagation + */ +} + +static void vfio_pci_device_sva_unregister_notifier(PCIBus *bus, + int32_t devfn, IOMMUSVAContext *sva_ctx) +{ + /* Unregister notifier for TLB invalidation propagation + */ +} + +static PCISVAOps vfio_pci_sva_ops = { + .sva_bind_pasid_table = vfio_pci_device_sva_bind_pasid_table, + .sva_register_notifier = vfio_pci_device_sva_register_notifier, + .sva_unregister_notifier = vfio_pci_device_sva_unregister_notifier, +}; + static void vfio_realize(PCIDevice *pdev, Error **errp) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); @@ -3019,6 +3047,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_register_req_notifier(vdev); vfio_setup_resetfn_quirk(vdev); + pci_setup_sva_ops(pdev, &vfio_pci_sva_ops); + return; out_teardown: From patchwork Thu Mar 1 10:31:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879706 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTq50CX4z9s15 for ; Thu, 1 Mar 2018 21:55:13 +1100 (AEDT) Received: from localhost ([::1]:55530 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLrj-0000XV-3j for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:55:11 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52744) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLls-0004FD-7V for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLln-0007sD-UD for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:08 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLln-0007fT-Lj for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:03 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611632" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:01 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:57 +0800 Message-Id: <1519900322-30263-8-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 07/12] vfio/pci: register sva notifier X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch shows how sva notifier is registered. And provided an example by registering notify func for tlb flush propagation. Signed-off-by: Liu, Yi L --- hw/vfio/pci.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 2 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index a60a4d7..b7297cc 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2775,6 +2775,26 @@ static void vfio_unregister_req_notifier(VFIOPCIDevice *vdev) vdev->req_enabled = false; } +static VFIOContainer *vfio_get_container_from_busdev(PCIBus *bus, + int32_t devfn) +{ + VFIOGroup *group; + VFIOPCIDevice *vdev_iter; + VFIODevice *vbasedev_iter; + PCIDevice *pdev_iter; + + QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { + vdev_iter = container_of(vbasedev_iter, VFIOPCIDevice, vbasedev); + pdev_iter = &vdev_iter->pdev; + if (pci_get_bus(pdev_iter) == bus && pdev_iter->devfn == devfn) { + return group->container; + } + } + } + return NULL; +} + static void vfio_pci_device_sva_bind_pasid_table(PCIBus *bus, int32_t devfn, uint64_t pasidt_addr, uint32_t size) { @@ -2783,11 +2803,42 @@ static void vfio_pci_device_sva_bind_pasid_table(PCIBus *bus, So far, Intel VT-d and AMD IOMMU requires it. */ } +static void vfio_iommu_sva_tlb_invalidate_notify(IOMMUSVANotifier *n, + IOMMUSVAEventData *event_data) +{ +/* Sample code, would be detailed in coming virt-SVA patchset. + VFIOGuestIOMMUSVAContext *gsva_ctx; + IOMMUSVAContext *sva_ctx; + VFIOContainer *container; + + gsva_ctx = container_of(n, VFIOGuestIOMMUSVAContext, n); + container = gsva_ctx->container; + + TODO: forward to host through VFIO IOCTL +*/ +} + static void vfio_pci_device_sva_register_notifier(PCIBus *bus, int32_t devfn, IOMMUSVAContext *sva_ctx) { - /* Register notifier for TLB invalidation propagation - */ + VFIOContainer *container = vfio_get_container_from_busdev(bus, devfn); + + if (container != NULL) { + VFIOGuestIOMMUSVAContext *gsva_ctx; + gsva_ctx = g_malloc0(sizeof(*gsva_ctx)); + gsva_ctx->sva_ctx = sva_ctx; + gsva_ctx->container = container; + QLIST_INSERT_HEAD(&container->gsva_ctx_list, + gsva_ctx, + gsva_ctx_next); + /* Register vfio_iommu_sva_tlb_invalidate_notify with event flag + IOMMU_SVA_EVENT_TLB_INV */ + iommu_sva_notifier_register(sva_ctx, + &gsva_ctx->n, + vfio_iommu_sva_tlb_invalidate_notify, + IOMMU_SVA_EVENT_TLB_INV); + return; + } } static void vfio_pci_device_sva_unregister_notifier(PCIBus *bus, From patchwork Thu Mar 1 10:31:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879713 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTv05xTLz9ryL for ; Thu, 1 Mar 2018 21:58:36 +1100 (AEDT) Received: from localhost ([::1]:55552 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLv0-0003iH-Su for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:58:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52745) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLls-0004FF-7h for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlp-0007uW-RC for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:08 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlp-0007fT-Do for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:05 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611633" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:03 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:58 +0800 Message-Id: <1519900322-30263-9-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 08/12] hw/pci: introduce pci_device_notify_iommu() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch adds pci_device_notify_iommu() for notify virtual IOMMU emulator when assigned device is added. And adds a new notify_func in PCIBus. vIOMMU emulator provides the instance of this notify_func. Reason: When virtual IOMMU is exposed to guest, vIOMMU emulator needs to programm host IOMMU to setup DMA mapping for assigned devices. This is a per-device operation, to be efficient, vIOMMU emulator needs to record the assigned devices. Example: devices assigned thru vfio, vfio_realize would call pci_device_notify_iommu() to notify vIOMMU emulator to record necessary info for assigned device. Signed-off-by: Liu, Yi L --- hw/alpha/typhoon.c | 2 +- hw/hppa/dino.c | 2 +- hw/i386/amd_iommu.c | 2 +- hw/i386/intel_iommu.c | 22 +++++++++++++++++++++- hw/pci-host/ppce500.c | 2 +- hw/pci-host/prep.c | 2 +- hw/pci-host/sabre.c | 2 +- hw/pci/pci.c | 25 +++++++++++++++++++++++-- hw/ppc/spapr_pci.c | 2 +- hw/s390x/s390-pci-bus.c | 4 ++-- hw/vfio/pci.c | 3 +++ include/hw/pci/pci.h | 12 +++++++++++- include/hw/pci/pci_bus.h | 1 + 13 files changed, 68 insertions(+), 13 deletions(-) diff --git a/hw/alpha/typhoon.c b/hw/alpha/typhoon.c index 6a40869..a7b02cd 100644 --- a/hw/alpha/typhoon.c +++ b/hw/alpha/typhoon.c @@ -894,7 +894,7 @@ PCIBus *typhoon_init(ram_addr_t ram_size, ISABus **isa_bus, "iommu-typhoon", UINT64_MAX); address_space_init(&s->pchip.iommu_as, MEMORY_REGION(&s->pchip.iommu), "pchip0-pci"); - pci_setup_iommu(b, typhoon_pci_dma_iommu, s); + pci_setup_iommu(b, typhoon_pci_dma_iommu, NULL, s); /* Pchip0 PCI special/interrupt acknowledge, 0x801.F800.0000, 64MB. */ memory_region_init_io(&s->pchip.reg_iack, OBJECT(s), &alpha_pci_iack_ops, diff --git a/hw/hppa/dino.c b/hw/hppa/dino.c index 15aefde..7867b46 100644 --- a/hw/hppa/dino.c +++ b/hw/hppa/dino.c @@ -481,7 +481,7 @@ PCIBus *dino_init(MemoryRegion *addr_space, 0xf0000000 + DINO_MEM_CHUNK_SIZE, &s->bm_pci_alias); address_space_init(&s->bm_as, &s->bm, "pci-bm"); - pci_setup_iommu(b, dino_pcihost_set_iommu, s); + pci_setup_iommu(b, dino_pcihost_set_iommu, NULL, s); *p_rtc_irq = qemu_allocate_irq(dino_set_timer_irq, s, 0); *p_ser_irq = qemu_allocate_irq(dino_set_serial_irq, s, 0); diff --git a/hw/i386/amd_iommu.c b/hw/i386/amd_iommu.c index 7bfde37..341a14d 100644 --- a/hw/i386/amd_iommu.c +++ b/hw/i386/amd_iommu.c @@ -1178,7 +1178,7 @@ static void amdvi_realize(DeviceState *dev, Error **err) sysbus_init_mmio(SYS_BUS_DEVICE(s), &s->mmio); sysbus_mmio_map(SYS_BUS_DEVICE(s), 0, AMDVI_BASE_ADDR); - pci_setup_iommu(bus, amdvi_host_dma_iommu, s); + pci_setup_iommu(bus, amdvi_host_dma_iommu, NULL, s); s->devid = object_property_get_int(OBJECT(&s->pci), "addr", err); msi_init(&s->pci.dev, 0, 1, true, false, err); amdvi_init(s); diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 9edf392..2fd0a6d 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3005,6 +3005,26 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) return &vtd_as->as; } +static int vtd_device_notify(PCIBus *bus, + void *opaque, + int devfn, + PCIDevNotifyType type) +{ + IntelIOMMUState *s = opaque; + VTDAddressSpace *vtd_as; + + assert(0 <= devfn && devfn < PCI_DEVFN_MAX); + + vtd_as = vtd_find_add_as(s, bus, devfn); + + if (vtd_as == NULL) { + return -1; + } + + /* TODO: record assigned device in IOMMU Emulator */ + return 0; +} + static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) { X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); @@ -3075,7 +3095,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) g_free, g_free); vtd_init(s); sysbus_mmio_map(SYS_BUS_DEVICE(s), 0, Q35_HOST_BRIDGE_IOMMU_ADDR); - pci_setup_iommu(bus, vtd_host_dma_iommu, dev); + pci_setup_iommu(bus, vtd_host_dma_iommu, vtd_device_notify, dev); /* Pseudo address space under root PCI bus. */ pcms->ioapic_as = vtd_host_dma_iommu(bus, s, Q35_PSEUDO_DEVFN_IOAPIC); } diff --git a/hw/pci-host/ppce500.c b/hw/pci-host/ppce500.c index eb75e08..8175df5 100644 --- a/hw/pci-host/ppce500.c +++ b/hw/pci-host/ppce500.c @@ -469,7 +469,7 @@ static int e500_pcihost_initfn(SysBusDevice *dev) memory_region_init(&s->bm, OBJECT(s), "bm-e500", UINT64_MAX); memory_region_add_subregion(&s->bm, 0x0, &s->busmem); address_space_init(&s->bm_as, &s->bm, "pci-bm"); - pci_setup_iommu(b, e500_pcihost_set_iommu, s); + pci_setup_iommu(b, e500_pcihost_set_iommu, NULL, s); pci_create_simple(b, 0, "e500-host-bridge"); diff --git a/hw/pci-host/prep.c b/hw/pci-host/prep.c index 01f67f9..b4d02cf 100644 --- a/hw/pci-host/prep.c +++ b/hw/pci-host/prep.c @@ -282,7 +282,7 @@ static void raven_pcihost_initfn(Object *obj) memory_region_add_subregion(&s->bm, 0 , &s->bm_pci_memory_alias); memory_region_add_subregion(&s->bm, 0x80000000, &s->bm_ram_alias); address_space_init(&s->bm_as, &s->bm, "raven-bm"); - pci_setup_iommu(&s->pci_bus, raven_pcihost_set_iommu, s); + pci_setup_iommu(&s->pci_bus, raven_pcihost_set_iommu, NULL, s); h->bus = &s->pci_bus; diff --git a/hw/pci-host/sabre.c b/hw/pci-host/sabre.c index e2f4ee4..e119753 100644 --- a/hw/pci-host/sabre.c +++ b/hw/pci-host/sabre.c @@ -399,7 +399,7 @@ static void sabre_realize(DeviceState *dev, Error **errp) /* IOMMU */ memory_region_add_subregion_overlap(&s->sabre_config, 0x200, sysbus_mmio_get_region(SYS_BUS_DEVICE(s->iommu), 0), 1); - pci_setup_iommu(phb->bus, sabre_pci_dma_iommu, s->iommu); + pci_setup_iommu(phb->bus, sabre_pci_dma_iommu, NULL, s->iommu); /* APB secondary busses */ pci_dev = pci_create_multifunction(phb->bus, PCI_DEVFN(1, 0), true, diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 157fe21..0f2db02 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2567,9 +2567,30 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) return &address_space_memory; } -void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) +void pci_device_notify_iommu(PCIDevice *dev, PCIDevNotifyType type) { - bus->iommu_fn = fn; + PCIBus *bus = PCI_BUS(pci_get_bus(dev)); + PCIBus *iommu_bus = bus; + + while (iommu_bus && !iommu_bus->iommu_fn && iommu_bus->parent_dev) { + iommu_bus = PCI_BUS(pci_get_bus(iommu_bus->parent_dev)); + } + if (iommu_bus && iommu_bus->notify_fn) { + iommu_bus->notify_fn(bus, + iommu_bus->iommu_opaque, + dev->devfn, + type); + } + return; +} + +void pci_setup_iommu(PCIBus *bus, + PCIIOMMUFunc iommu_fn, + PCIIOMMUNotifyFunc notify_fn, + void *opaque) +{ + bus->iommu_fn = iommu_fn; + bus->notify_fn = notify_fn; bus->iommu_opaque = opaque; } diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c index 39a1498..2be0a0f 100644 --- a/hw/ppc/spapr_pci.c +++ b/hw/ppc/spapr_pci.c @@ -1687,7 +1687,7 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) memory_region_add_subregion(&sphb->iommu_root, SPAPR_PCI_MSI_WINDOW, &sphb->msiwindow); - pci_setup_iommu(bus, spapr_pci_dma_iommu, sphb); + pci_setup_iommu(bus, spapr_pci_dma_iommu, NULL, sphb); pci_bus_set_route_irq_fn(bus, spapr_route_intx_pin_to_irq); diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 1bad7ab..cc650c4 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -705,7 +705,7 @@ static int s390_pcihost_init(SysBusDevice *dev) s390_pci_set_irq, s390_pci_map_irq, NULL, get_system_memory(), get_system_io(), 0, 64, TYPE_PCI_BUS); - pci_setup_iommu(b, s390_pci_dma_iommu, s); + pci_setup_iommu(b, s390_pci_dma_iommu, NULL, s); bus = BUS(b); qbus_set_hotplug_handler(bus, DEVICE(dev), NULL); @@ -817,7 +817,7 @@ static void s390_pcihost_hot_plug(HotplugHandler *hotplug_dev, PCIDevice *pdev = PCI_DEVICE(dev); pci_bridge_map_irq(pb, dev->id, s390_pci_map_irq); - pci_setup_iommu(&pb->sec_bus, s390_pci_dma_iommu, s); + pci_setup_iommu(&pb->sec_bus, s390_pci_dma_iommu, NULL, s); bus = BUS(&pb->sec_bus); qbus_set_hotplug_handler(bus, DEVICE(s), errp); diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index b7297cc..a9c0898 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3098,6 +3098,8 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_register_req_notifier(vdev); vfio_setup_resetfn_quirk(vdev); + pci_device_notify_iommu(pdev, PCI_NTY_DEV_ADD); + pci_setup_sva_ops(pdev, &vfio_pci_sva_ops); return; @@ -3134,6 +3136,7 @@ static void vfio_exitfn(PCIDevice *pdev) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); + pci_device_notify_iommu(pdev, PCI_NTY_DEV_DEL); vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index 32889a4..964be2b 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -485,10 +485,20 @@ void pci_bus_get_w64_range(PCIBus *bus, Range *range); void pci_device_deassert_intx(PCIDevice *dev); +enum PCIDevNotifyType { + PCI_NTY_DEV_ADD = 0, + PCI_NTY_DEV_DEL, +}; +typedef enum PCIDevNotifyType PCIDevNotifyType; typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); +typedef int (*PCIIOMMUNotifyFunc)(PCIBus *, void *, int, PCIDevNotifyType); AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); -void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); +void pci_device_notify_iommu(PCIDevice *dev, PCIDevNotifyType type); +void pci_setup_iommu(PCIBus *bus, + PCIIOMMUFunc iommu_fn, + PCIIOMMUNotifyFunc notify_fn, + void *opaque); void pci_setup_sva_ops(PCIDevice *dev, PCISVAOps *ops); void pci_device_sva_bind_pasid_table(PCIBus *bus, int32_t devfn, diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h index b7da8f5..54a0c8e 100644 --- a/include/hw/pci/pci_bus.h +++ b/include/hw/pci/pci_bus.h @@ -21,6 +21,7 @@ typedef struct PCIBusClass { struct PCIBus { BusState qbus; PCIIOMMUFunc iommu_fn; + PCIIOMMUNotifyFunc notify_fn; void *iommu_opaque; uint8_t devfn_min; uint32_t slot_reserved_mask; From patchwork Thu Mar 1 10:31:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879705 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTps6pdJz9s06 for ; Thu, 1 Mar 2018 21:55:01 +1100 (AEDT) Received: from localhost ([::1]:55528 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLrX-0000OB-7s for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:54:59 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52749) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLls-0004Fa-Kq for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:09 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlr-0007wd-Id for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:08 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlr-0007fT-6R for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:07 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611638" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:05 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:31:59 +0800 Message-Id: <1519900322-30263-10-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 09/12] intel_iommu: record assigned devices in a list X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch records assigned devices in a list within Intel vIOMMU emulator. The recorded info can be used to filter out affect assigned devices when Qemu captured guest's cache invalidate request. Signed-off-by: Liu, Yi L --- hw/i386/intel_iommu.c | 31 ++++++++++++++++++++++++++----- include/hw/i386/intel_iommu.h | 11 ++++++++++- 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 2fd0a6d..978f47a 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -2685,7 +2685,10 @@ static const MemoryRegionOps vtd_mem_ir_ops = { }, }; -VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) +VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, + PCIBus *bus, + int devfn, + bool allocate) { uintptr_t key = (uintptr_t)bus; VTDBus *vtd_bus = g_hash_table_lookup(s->vtd_as_by_busptr, &key); @@ -2704,7 +2707,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) vtd_dev_as = vtd_bus->dev_as[devfn]; - if (!vtd_dev_as) { + if (!vtd_dev_as && allocate) { snprintf(name, sizeof(name), "intel_iommu_devfn_%d", devfn); vtd_bus->dev_as[devfn] = vtd_dev_as = g_malloc0(sizeof(VTDAddressSpace)); @@ -3001,7 +3004,7 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) assert(0 <= devfn && devfn < PCI_DEVFN_MAX); - vtd_as = vtd_find_add_as(s, bus, devfn); + vtd_as = vtd_find_add_as(s, bus, devfn, true); return &vtd_as->as; } @@ -3012,16 +3015,34 @@ static int vtd_device_notify(PCIBus *bus, { IntelIOMMUState *s = opaque; VTDAddressSpace *vtd_as; + IntelIOMMUAssignedDeviceNode *node = NULL; + IntelIOMMUAssignedDeviceNode *next_node = NULL; assert(0 <= devfn && devfn < PCI_DEVFN_MAX); - vtd_as = vtd_find_add_as(s, bus, devfn); + vtd_as = vtd_find_add_as(s, bus, devfn, false); if (vtd_as == NULL) { return -1; } - /* TODO: record assigned device in IOMMU Emulator */ + if (type == PCI_NTY_DEV_ADD) { + node = g_malloc0(sizeof(*node)); + node->vtd_as = vtd_as; + QLIST_INSERT_HEAD(&s->assigned_device_list, node, next); + return 0; + } + + QLIST_FOREACH_SAFE(node, &s->assigned_device_list, next, next_node) { + if (node->vtd_as == vtd_as) { + if (type == PCI_NTY_DEV_DEL) { + QLIST_REMOVE(node, next); + g_free(node); + } + break; + } + } + return 0; } diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 1df6fa9..0b6dc32 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -68,6 +68,7 @@ typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress; typedef struct VTDIrq VTDIrq; typedef struct VTD_MSIMessage VTD_MSIMessage; typedef struct IntelIOMMUMRNotifierNode IntelIOMMUMRNotifierNode; +typedef struct IntelIOMMUAssignedDeviceNode IntelIOMMUAssignedDeviceNode; /* Context-Entry */ struct VTDContextEntry { @@ -258,6 +259,11 @@ struct IntelIOMMUMRNotifierNode { QLIST_ENTRY(IntelIOMMUMRNotifierNode) next; }; +struct IntelIOMMUAssignedDeviceNode { + VTDAddressSpace *vtd_as; + QLIST_ENTRY(IntelIOMMUAssignedDeviceNode) next; +}; + /* The iommu (DMAR) device state struct */ struct IntelIOMMUState { X86IOMMUState x86_iommu; @@ -296,6 +302,8 @@ struct IntelIOMMUState { VTDBus *vtd_as_by_bus_num[VTD_PCI_BUS_MAX]; /* VTDBus objects indexed by bus number */ /* list of registered notifiers */ QLIST_HEAD(, IntelIOMMUMRNotifierNode) notifiers_list; + /* list of assigned devices */ + QLIST_HEAD(, IntelIOMMUAssignedDeviceNode) assigned_device_list; /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ @@ -310,6 +318,7 @@ struct IntelIOMMUState { /* Find the VTD Address space associated with the given bus pointer, * create a new one if none exists */ -VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn); +VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, + int devfn, bool allocate); #endif From patchwork Thu Mar 1 10:32:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879711 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTsz28NSz9ryL for ; Thu, 1 Mar 2018 21:57:43 +1100 (AEDT) Received: from localhost ([::1]:55546 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLu9-0002sf-AU for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 05:57:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52771) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlu-0004HY-Jg for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlt-0007zC-MP for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:10 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLls-0007fT-W1 for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:09 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611642" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:06 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:32:00 +0800 Message-Id: <1519900322-30263-11-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 10/12] intel_iommu: bind guest pasid table to host X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" For assigned SVA capable devices, needs to bind guest pasid table to host. Intel vIOMMU emulator captures device selective context cache flush, and propagate the guest pasid table pointer to host, in host iommu driver configs the guest pasid table pointer in its translation structure. Signed-off-by: Liu, Yi L --- hw/i386/intel_iommu.c | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 978f47a..d92a66d 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -37,6 +37,10 @@ #include "kvm_i386.h" #include "trace.h" +static bool vtd_device_is_assigned(IntelIOMMUState *s, + PCIBus *bus, + uint16_t devfn); + static void vtd_define_quad(IntelIOMMUState *s, hwaddr addr, uint64_t val, uint64_t wmask, uint64_t w1cmask) { @@ -1255,6 +1259,20 @@ static void vtd_context_global_invalidate(IntelIOMMUState *s) vtd_iommu_replay_all(s); } +static uint64_t vtd_get_pasid_table_from_context(VTDContextEntry *ce) +{ + uint64_t pasidt_addr = ce->hi; + /* TODO: TBD */ + return pasidt_addr; +} + +static uint32_t vtd_get_pasidt_size_from_context(VTDContextEntry *ce) +{ + uint32_t pasidt_size = ce->hi; + /* TODO: TBD */ + return pasidt_size; +} + /* Do a context-cache device-selective invalidation. * @func_mask: FM field after shifting */ @@ -1291,6 +1309,11 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, if (vtd_bus) { devfn = VTD_SID_TO_DEVFN(source_id); for (devfn_it = 0; devfn_it < PCI_DEVFN_MAX; ++devfn_it) { + VTDContextEntry ce; + int ret = 0; + uint64_t pasidt_addr; + uint32_t size; + vtd_as = vtd_bus->dev_as[devfn_it]; if (vtd_as && ((devfn_it & mask) == (devfn & mask))) { trace_vtd_inv_desc_cc_device(bus_n, VTD_PCI_SLOT(devfn_it), @@ -1311,6 +1334,26 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, * happened. */ memory_region_iommu_replay_all(&vtd_as->iommu); + + /* + * If device is SVA capable assigned device, needs + * to bind guest pasid table to host + * + */ + if (!vtd_device_is_assigned(s, vtd_as->bus, devfn_it)) { + continue; + } + + ret = vtd_dev_to_context_entry(s, bus_n, + vtd_as->devfn, &ce); + if (ret) { + continue; + } + + pasidt_addr = vtd_get_pasid_table_from_context(&ce); + size = vtd_get_pasidt_size_from_context(&ce); + pci_device_sva_bind_pasid_table(vtd_as->bus, devfn_it, + pasidt_addr, size); } } } @@ -3046,6 +3089,32 @@ static int vtd_device_notify(PCIBus *bus, return 0; } +static bool vtd_device_is_assigned(IntelIOMMUState *s, + PCIBus *bus, + uint16_t devfn) +{ + VTDAddressSpace *vtd_as; + IntelIOMMUAssignedDeviceNode *node = NULL; + IntelIOMMUAssignedDeviceNode *next_node = NULL; + + vtd_as = vtd_find_add_as(s, bus, devfn, false); + + if (vtd_as == NULL) { + /* + * If vtd_as is NULL, return false for safe + */ + return false; + } + + QLIST_FOREACH_SAFE(node, &s->assigned_device_list, next, next_node) { + if (node->vtd_as == vtd_as) { + return true; + } + } + + return false; +} + static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) { X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); From patchwork Thu Mar 1 10:32:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsTxX3KzXz9ryL for ; Thu, 1 Mar 2018 22:00:48 +1100 (AEDT) Received: from localhost ([::1]:55572 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLx8-0005kW-Fk for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 06:00:46 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52786) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlw-0004JV-9M for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlv-00080P-8m for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:12 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlu-0007fT-Vw for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:11 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611656" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:08 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:32:01 +0800 Message-Id: <1519900322-30263-12-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 11/12] intel_iommu: add framework for PASID AddressSpace management X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch introduces a framework to manage PASID tagged AddressSpace in Intel vIOMMU emulator. PASID tagged AddressSpace is an address sapce which is an abstract of guest process address space in Qemu. The management framework is as below: s->pasid_as_list /|\ \ / | \ \ pasid_as_node ... /|\ \ / | \ \ device ... There is a list to store all the PASID tagged AddressSpace, and each PASID tagged AddressSpace has a device list behind it. This is due to the fact that a PASID tagged AddressSpace can have multiple devices binded. Signed-off-by: Liu, Yi L --- hw/i386/intel_iommu.c | 1 + include/hw/i386/intel_iommu.h | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index d92a66d..b8e8dbb 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3174,6 +3174,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) } QLIST_INIT(&s->notifiers_list); + QLIST_INIT(&s->pasid_as_list); memset(s->vtd_as_by_bus_num, 0, sizeof(s->vtd_as_by_bus_num)); memory_region_init_io(&s->csrmem, OBJECT(s), &vtd_mem_ops, s, "intel_iommu", DMAR_REG_SIZE); diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 0b6dc32..c45dbfe 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -61,6 +61,7 @@ typedef struct VTDContextEntry VTDContextEntry; typedef struct VTDContextCacheEntry VTDContextCacheEntry; typedef struct IntelIOMMUState IntelIOMMUState; typedef struct VTDAddressSpace VTDAddressSpace; +typedef struct VTDPASIDAddressSpace VTDPASIDAddressSpace; typedef struct VTDIOTLBEntry VTDIOTLBEntry; typedef struct VTDBus VTDBus; typedef union VTD_IR_TableEntry VTD_IR_TableEntry; @@ -69,6 +70,8 @@ typedef struct VTDIrq VTDIrq; typedef struct VTD_MSIMessage VTD_MSIMessage; typedef struct IntelIOMMUMRNotifierNode IntelIOMMUMRNotifierNode; typedef struct IntelIOMMUAssignedDeviceNode IntelIOMMUAssignedDeviceNode; +typedef struct IntelPASIDNode IntelPASIDNode; +typedef struct VTDDeviceNode VTDDeviceNode; /* Context-Entry */ struct VTDContextEntry { @@ -84,6 +87,20 @@ struct VTDContextCacheEntry { struct VTDContextEntry context_entry; }; +struct VTDDeviceNode { + PCIBus *bus; + uint8_t devfn; + QLIST_ENTRY(VTDDeviceNode) next; +}; + +struct VTDPASIDAddressSpace { + AddressSpace as; + IOMMUSVAContext sva_ctx; + IntelIOMMUState *iommu_state; + /* list of devices binded to a pasid tagged address space */ + QLIST_HEAD(, VTDDeviceNode) device_list; +}; + struct VTDAddressSpace { PCIBus *bus; uint8_t devfn; @@ -264,6 +281,11 @@ struct IntelIOMMUAssignedDeviceNode { QLIST_ENTRY(IntelIOMMUAssignedDeviceNode) next; }; +struct IntelPASIDNode { + VTDPASIDAddressSpace *pasid_as; + QLIST_ENTRY(IntelPASIDNode) next; +}; + /* The iommu (DMAR) device state struct */ struct IntelIOMMUState { X86IOMMUState x86_iommu; @@ -304,6 +326,8 @@ struct IntelIOMMUState { QLIST_HEAD(, IntelIOMMUMRNotifierNode) notifiers_list; /* list of assigned devices */ QLIST_HEAD(, IntelIOMMUAssignedDeviceNode) assigned_device_list; + /* list of pasid tagged address space */ + QLIST_HEAD(, IntelPASIDNode) pasid_as_list; /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ From patchwork Thu Mar 1 10:32:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liu, Yi L" X-Patchwork-Id: 879720 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=2001:4830:134:3::11; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.intel.com Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zsV0W0ZF0z9s1B for ; Thu, 1 Mar 2018 22:03:23 +1100 (AEDT) Received: from localhost ([::1]:55593 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLzd-0007k2-4T for incoming@patchwork.ozlabs.org; Thu, 01 Mar 2018 06:03:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52798) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erLlx-0004LC-Vi for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erLlw-00082h-Qs for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:14 -0500 Received: from mga14.intel.com ([192.55.52.115]:7314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1erLlw-0007fT-GE for qemu-devel@nongnu.org; Thu, 01 Mar 2018 05:49:12 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Mar 2018 02:49:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.47,408,1515484800"; d="scan'208";a="31611666" Received: from sky-dev.bj.intel.com ([10.238.144.105]) by orsmga003.jf.intel.com with ESMTP; 01 Mar 2018 02:49:10 -0800 From: "Liu, Yi L" To: qemu-devel@nongnu.org, mst@redhat.com, david@gibson.dropbear.id.au Date: Thu, 1 Mar 2018 18:32:02 +0800 Message-Id: <1519900322-30263-13-git-send-email-yi.l.liu@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> References: <1519900322-30263-1-git-send-email-yi.l.liu@linux.intel.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.115 Subject: [Qemu-devel] [PATCH v3 12/12] intel_iommu: bind device to PASID tagged AddressSpace X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.williamson@redhat.com, "Liu, Yi L" , eric.auger.pro@gmail.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This patch shows the idea of how a device is binded to a PASID tagged AddressSpace. when Intel vIOMMU emulator detected a pasid table entry programming from guest. Intel vIOMMU emulator firstly finds a VTDPASIDAddressSpace with the pasid field of pasid cache invalidate request. * If it is to bind a device to a guest process, needs add the device to the device list behind the VTDPASIDAddressSpace. And if the device is assigned device, need to register sva_notfier for future tlb flushing if any mapping changed to the process address space. * If it is to unbind a device from a guest process, then need to remove the device from the device list behind the VTDPASIDAddressSpace. And also needs to unregister the sva_notfier if the device is assigned device. This patch hasn't added the unbind logic. It depends on guest pasid table entry parsing which requires further emulation. Here just want to show the idea for the PASID tagged AddressSpace management framework. Full unregister logic would be included in future virt-SVA patchset. Signed-off-by: Liu, Yi L --- hw/i386/intel_iommu.c | 119 +++++++++++++++++++++++++++++++++++++++++ hw/i386/intel_iommu_internal.h | 10 ++++ 2 files changed, 129 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index b8e8dbb..ed07035 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -1801,6 +1801,118 @@ static bool vtd_process_iotlb_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) return true; } +static VTDPASIDAddressSpace *vtd_get_pasid_as(IntelIOMMUState *s, + uint32_t pasid) +{ + VTDPASIDAddressSpace *vtd_pasid_as = NULL; + IntelPASIDNode *node; + char name[128]; + + QLIST_FOREACH(node, &(s->pasid_as_list), next) { + vtd_pasid_as = node->pasid_as; + if (pasid == vtd_pasid_as->sva_ctx.pasid) { + return vtd_pasid_as; + } + } + + vtd_pasid_as = g_malloc0(sizeof(*vtd_pasid_as)); + vtd_pasid_as->iommu_state = s; + snprintf(name, sizeof(name), "intel_iommu_pasid_%d", pasid); + address_space_init(&vtd_pasid_as->as, NULL, "pasid"); + QLIST_INIT(&vtd_pasid_as->device_list); + + node = g_malloc0(sizeof(*node)); + node->pasid_as = vtd_pasid_as; + QLIST_INSERT_HEAD(&s->pasid_as_list, node, next); + + return vtd_pasid_as; +} + +static void vtd_bind_device_to_pasid_as(VTDPASIDAddressSpace *vtd_pasid_as, + PCIBus *bus, uint8_t devfn) +{ + VTDDeviceNode *node = NULL; + + QLIST_FOREACH(node, &(vtd_pasid_as->device_list), next) { + if (node->bus == bus && node->devfn == devfn) { + return; + } + } + + node = g_malloc0(sizeof(*node)); + node->bus = bus; + node->devfn = devfn; + QLIST_INSERT_HEAD(&(vtd_pasid_as->device_list), node, next); + + pci_device_sva_register_notifier(bus, devfn, &vtd_pasid_as->sva_ctx); + + return; +} + +static bool vtd_process_pc_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) +{ + + IntelIOMMUAssignedDeviceNode *node = NULL; + int ret = 0; + + uint16_t domain_id; + uint32_t pasid; + VTDPASIDAddressSpace *vtd_pasid_as; + + if ((inv_desc->lo & VTD_INV_DESC_PASIDC_RSVD_LO) || + (inv_desc->hi & VTD_INV_DESC_PASIDC_RSVD_HI)) { + return false; + } + + domain_id = VTD_INV_DESC_PASIDC_DID(inv_desc->lo); + + switch (inv_desc->lo & VTD_INV_DESC_PASIDC_G) { + case VTD_INV_DESC_PASIDC_ALL_ALL: + /* TODO: invalidate all pasid related cache */ + break; + + case VTD_INV_DESC_PASIDC_PASID_SI: + pasid = VTD_INV_DESC_PASIDC_PASID(inv_desc->lo); + vtd_pasid_as = vtd_get_pasid_as(s, pasid); + QLIST_FOREACH(node, &(s->assigned_device_list), next) { + VTDAddressSpace *vtd_as = node->vtd_as; + VTDContextEntry ce; + uint16_t did; + uint8_t bus = pci_bus_num(vtd_as->bus); + ret = vtd_dev_to_context_entry(s, bus, + vtd_as->devfn, &ce); + if (ret != 0) { + continue; + } + + did = VTD_CONTEXT_ENTRY_DID(ce.hi); + /* + * If did field equals to the domain_id field of inv_descriptor, + * then the device is affect by this invalidate request, need to + * bind or unbind the device to the pasid tagged address space. + * a) If it is bind, need to add the device to the device list, + * add register tlb flush notifier for it + * b) If it is unbind, need to remove the device from the device + * list, and unregister the tlb flush notifier + * TODO: add unbind logic accordingly, depends on the parsing of + * guest pasid table entry pasrsing, here has no parsing to + * pasid table entry. + * + */ + if (did == domain_id) { + vtd_bind_device_to_pasid_as(vtd_pasid_as, + vtd_as->bus, vtd_as->devfn); + } + } + break; + + default: + return false; + } + + return true; +} + static bool vtd_process_inv_iec_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc) { @@ -1911,6 +2023,13 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s) } break; + case VTD_INV_DESC_PC: + trace_vtd_inv_desc("pc", inv_desc.hi, inv_desc.lo); + if (!vtd_process_pc_desc(s, &inv_desc)) { + return false; + } + break; + case VTD_INV_DESC_IEC: trace_vtd_inv_desc("iec", inv_desc.hi, inv_desc.lo); if (!vtd_process_inv_iec_desc(s, &inv_desc)) { diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index d084099..31d0d53 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -332,6 +332,7 @@ typedef union VTDInvDesc VTDInvDesc; #define VTD_INV_DESC_IEC 0x4 /* Interrupt Entry Cache Invalidate Descriptor */ #define VTD_INV_DESC_WAIT 0x5 /* Invalidation Wait Descriptor */ +#define VTD_INV_DESC_PC 0x7 /* PASID-cache Invalidate Desc */ #define VTD_INV_DESC_NONE 0 /* Not an Invalidate Descriptor */ /* Masks for Invalidation Wait Descriptor*/ @@ -388,6 +389,15 @@ typedef union VTDInvDesc VTDInvDesc; #define VTD_SPTE_LPAGE_L4_RSVD_MASK(aw) \ (0x880ULL | ~(VTD_HAW_MASK(aw) | VTD_SL_IGN_COM)) +#define VTD_INV_DESC_PASIDC_G (3ULL << 4) +#define VTD_INV_DESC_PASIDC_PASID(val) (((val) >> 32) & 0xfffffULL) +#define VTD_INV_DESC_PASIDC_DID(val) (((val) >> 16) & VTD_DOMAIN_ID_MASK) +#define VTD_INV_DESC_PASIDC_RSVD_LO 0xfff000000000ffc0ULL +#define VTD_INV_DESC_PASIDC_RSVD_HI 0xffffffffffffffffULL + +#define VTD_INV_DESC_PASIDC_ALL_ALL (0ULL << 4) +#define VTD_INV_DESC_PASIDC_PASID_SI (1ULL << 4) + /* Information about page-selective IOTLB invalidate */ struct VTDIOTLBPageInvInfo { uint16_t domain_id;