From patchwork Thu Jun 22 21:48:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798710 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=VsOMb8BM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDYj4Ll5z20Wk for ; Fri, 23 Jun 2023 07:50:00 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBA-0003AH-9Z; Thu, 22 Jun 2023 17:49:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSB7-00039a-Ur for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:22 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSB5-0007PY-PQ for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:21 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MK25oo019495; Thu, 22 Jun 2023 21:49:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=KGe7u2IMV2l3K8bxEpI70HCAGNDFcdBWxbYQ0h4LJYw=; b=VsOMb8BMhOQs9bZXLYXRQX5B0QdzcYhYol9MEc2wavnqi33EJ+E1BuGYoTpXqAx0HNLw ibwOoRqJTYyfwrj5t/Ve0GVzFXDVd19TEYg2HDmwdbR4i0sjtIbKnaRN0Gzxa+uGLazr qrvA4PFPD6ItlJx58YybQq2k1h4aESlSF/UMgutWI8bjYYDkN1JNCTm54DCCkgoF0yMj IRW11lVFcmLTB1o9kt81igW8uKR68KHWAvD7LwLX42g66lMGY70xQQ4zs/HMcThUn3Fa fx54j3GmbUHHeSOvcue+ijJI9I1mlGL05rdhHqyLytQ0OOQqT7VP23DaSsofNEmaTCiG oA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r938dtt02-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:15 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKiP1g008398; Thu, 22 Jun 2023 21:49:14 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398ep6y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:14 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76m035791; Thu, 22 Jun 2023 21:49:14 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-2; Thu, 22 Jun 2023 21:49:11 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins , Yi Liu Subject: [PATCH v4 01/15] hw/pci: Add a pci_setup_iommu_ops() helper Date: Thu, 22 Jun 2023 22:48:31 +0100 Message-Id: <20230622214845.3980-2-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=862 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: jnafCakYlNaiLi0NhGgJH6_NQ6hWRtQ9 X-Proofpoint-ORIG-GUID: jnafCakYlNaiLi0NhGgJH6_NQ6hWRtQ9 Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Yi Liu Add a pci_setup_iommu_ops() that uses a newly added structure (PCIIOMMUOps) instead of using PCIIOMMUFunc. The old pci_setup_iommu() that uses PCIIOMMUFunc is still kept for other IOMMUs to get an an address space for a PCI device in vendor specific way. In preparation to expand to supplying vIOMMU attributes, add a alternate helper pci_setup_iommu_ops() to setup the PCI device IOMMU. For now the PCIIOMMUOps just defines the address_space, but it will be extended to have another callback. Signed-off-by: Yi Liu [joao: Massage commit message and subject, and make it a complementary rather than changing every single consumer of pci_setup_iommu()] Signed-off-by: Joao Martins --- v1: https://lore.kernel.org/all/20210302203827.437645-5-yi.l.liu@intel.com/ --- include/hw/pci/pci.h | 7 +++++++ include/hw/pci/pci_bus.h | 1 + hw/pci/pci.c | 26 +++++++++++++++++++++++--- 3 files changed, 31 insertions(+), 3 deletions(-) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index e6d0574a2999..f59aef5a329a 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -368,6 +368,13 @@ typedef AddressSpace *(*PCIIOMMUFunc)(PCIBus *, void *, int); AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque); +typedef struct PCIIOMMUOps PCIIOMMUOps; +struct PCIIOMMUOps { + AddressSpace * (*get_address_space)(PCIBus *bus, + void *opaque, int32_t devfn); +}; +void pci_setup_iommu_ops(PCIBus *bus, const PCIIOMMUOps *iommu_ops, void *opaque); + pcibus_t pci_bar_address(PCIDevice *d, int reg, uint8_t type, pcibus_t size); diff --git a/include/hw/pci/pci_bus.h b/include/hw/pci/pci_bus.h index 56531759578f..fb770b236d69 100644 --- a/include/hw/pci/pci_bus.h +++ b/include/hw/pci/pci_bus.h @@ -35,6 +35,7 @@ struct PCIBus { enum PCIBusFlags flags; PCIIOMMUFunc iommu_fn; void *iommu_opaque; + const PCIIOMMUOps *iommu_ops; uint8_t devfn_min; uint32_t slot_reserved_mask; pci_set_irq_fn set_irq; diff --git a/hw/pci/pci.c b/hw/pci/pci.c index bf38905b7dc0..4e32c09e81d6 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2639,7 +2639,15 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) PCIBus *iommu_bus = bus; uint8_t devfn = dev->devfn; - while (iommu_bus && !iommu_bus->iommu_fn && iommu_bus->parent_dev) { + /* + * get_address_space() callback is mandatory when iommu uses + * pci_setup_iommu_ops(), so needs to ensure its presence in + * the iommu_bus search. + */ + while (iommu_bus && + !(iommu_bus->iommu_fn || + (iommu_bus->iommu_ops && iommu_bus->iommu_ops->get_address_space)) && + iommu_bus->parent_dev) { PCIBus *parent_bus = pci_get_bus(iommu_bus->parent_dev); /* @@ -2678,8 +2686,14 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) iommu_bus = parent_bus; } - if (!pci_bus_bypass_iommu(bus) && iommu_bus && iommu_bus->iommu_fn) { - return iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); + if (!pci_bus_bypass_iommu(bus) && iommu_bus) { + if (iommu_bus->iommu_fn) { + return iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); + } else if (iommu_bus->iommu_ops && + iommu_bus->iommu_ops->get_address_space) { + return iommu_bus->iommu_ops->get_address_space(bus, + iommu_bus->iommu_opaque, devfn); + } } return &address_space_memory; } @@ -2690,6 +2704,12 @@ void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) bus->iommu_opaque = opaque; } +void pci_setup_iommu_ops(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) +{ + bus->iommu_ops = ops; + bus->iommu_opaque = opaque; +} + static void pci_dev_get_w64(PCIBus *b, PCIDevice *dev, void *opaque) { Range *range = opaque; From patchwork Thu Jun 22 21:48:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798720 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=bymh5TDu; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDbd3RhZz20Xt for ; Fri, 23 Jun 2023 07:51:41 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBG-0003Ah-G9; Thu, 22 Jun 2023 17:49:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBD-0003AT-RM for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:28 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBA-0007QU-Bi for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:27 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKYbmR019537; Thu, 22 Jun 2023 21:49:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=2eKyKEiRNiaEmblcCWXkGMWYnkSgA39Uapu6TQfxG+A=; b=bymh5TDuwYZzfS//D8TgI9Ymp6bmACZxC/DBQBmXF0J2Ri2Bt4ANEvvFjAT8BWdt2vpc yDXwm2gxZDKp8ZigdM+n7XpirvA4D9mp6DxijhMWTL2nyzk4yIp4YOskkoIZHwxFsqQ7 ctpYVRUDJpZzejFPtOLCxGBVU3lZYyI4tOYDbNgBlOMXbQ3QhBHbBqm9T2/Fd3cdx/xi 1vZ4gsUebcQqLp+drPr0+mYYs6G6Ap+kNRwMYMNGDNOof/i4HZvaS+4SWsUVWoPP5Glh QecONI5tkQ9+sWBMM4GgI4wBxc20ksO7fWsUXNg7+h0WHzvzgz4MycTEpqbhRJJb1bqA gw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93m3txuf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:20 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKiP1j008398; Thu, 22 Jun 2023 21:49:19 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398ep8p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:19 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76o035791; Thu, 22 Jun 2023 21:49:18 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-3; Thu, 22 Jun 2023 21:49:18 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins , Yi Liu Subject: [PATCH v4 02/15] hw/pci: Refactor pci_device_iommu_address_space() Date: Thu, 22 Jun 2023 22:48:32 +0100 Message-Id: <20230622214845.3980-3-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: CGT5Km1cloUhOCWkN3HadqwwEsAN7gjP X-Proofpoint-ORIG-GUID: CGT5Km1cloUhOCWkN3HadqwwEsAN7gjP Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Yi Liu Refactor pci_device_iommu_address_space() and move the code that fetches the device bus and iommu bus into its own private helper pci_device_get_iommu_bus_devfn(). This is in preparation to introduce pci_device_iommu_get_attr() which will need to use it too. Signed-off-by: Yi Liu [joao: Commit message, and better splitting] Signed-off-by: Joao Martins Reviewed-by: Eric Auger --- Splitted from v1: https://lore.kernel.org/all/20210302203827.437645-6-yi.l.liu@intel.com/ --- hw/pci/pci.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 4e32c09e81d6..90ae92a43d85 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2632,8 +2632,8 @@ static void pci_device_class_base_init(ObjectClass *klass, void *data) assert(conventional || pcie || cxl); } } - -AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +static void pci_device_get_iommu_bus_devfn(PCIDevice *dev, PCIBus **pdevbus, + PCIBus **pbus, uint8_t *pdevfn) { PCIBus *bus = pci_get_bus(dev); PCIBus *iommu_bus = bus; @@ -2686,6 +2686,18 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) iommu_bus = parent_bus; } + + *pdevbus = bus; + *pbus = iommu_bus; + *pdevfn = devfn; +} + +AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +{ + PCIBus *bus, *iommu_bus; + uint8_t devfn; + + pci_device_get_iommu_bus_devfn(dev, &bus, &iommu_bus, &devfn); if (!pci_bus_bypass_iommu(bus) && iommu_bus) { if (iommu_bus->iommu_fn) { return iommu_bus->iommu_fn(bus, iommu_bus->iommu_opaque, devfn); From patchwork Thu Jun 22 21:48:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798709 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=MAUaOIR+; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDYj4pF9z20ZH for ; Fri, 23 Jun 2023 07:50:00 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBJ-0003BU-28; Thu, 22 Jun 2023 17:49:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBH-0003B6-Ti for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:31 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBG-0007Qq-6u for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:31 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MLWLTu007802; Thu, 22 Jun 2023 21:49:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Im49a6fnTBbEY5Y0/C6Ivtgf1RhF+kCnOMpymBmRJlE=; b=MAUaOIR+o5opwNxPYHwqb37yf3TJVL99P/TEDX0qHSY1MUOrG29zmnzdI7QTPgDLjHaL CiwahljE8r0573CHvQ6NjexSO45eHXdiaN3q0N+dbJERXJbQvKcu0IrUxqLIsU9X8Cc9 AX9qo6R/D56BgU1bXLoZZMXZidcplvxmZfNvdKoLwzlPCnVgo1z2J5KN0U3plnDUHcx/ HfYrEpnf51oheOFzQVTOUhap6y5cjbNa2Uf91mCtJNvIJolzz2KQA3+ahxfjifZQ4wVb OfPxN/FMwy4a/R2+MsxCcIQ/3AMbD042GHiKaLYFku7B+QhDoaY7Fqks0KVjeB92FFhH eA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r938dtt09-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:25 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MLYusj008585; Thu, 22 Jun 2023 21:49:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epb2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:24 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76q035791; Thu, 22 Jun 2023 21:49:23 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-4; Thu, 22 Jun 2023 21:49:22 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins , Yi Liu Subject: [PATCH v4 03/15] hw/pci: Introduce pci_device_iommu_get_attr() Date: Thu, 22 Jun 2023 22:48:33 +0100 Message-Id: <20230622214845.3980-4-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: Yu3OEggms4HqIgAQIioV4Yl6XoBIWb5a X-Proofpoint-ORIG-GUID: Yu3OEggms4HqIgAQIioV4Yl6XoBIWb5a Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Yi Liu Introduce pci_device_iommu_get_attr() to get vIOMMU attributes from the PCI device. This is in preparation to ask if vIOMMU has dma translation enabled and also to get IOVA boundaries. Signed-off-by: Yi Liu [joao: Massage commit message; add one more argument in pci_device_get_iommu_bus_devfn(); rename to pci_device_iommu_get_attr() to align with the other already namespaced function. ] Signed-off-by: Joao Martins --- follow-up version from: https://lore.kernel.org/all/20210302203827.437645-6-yi.l.liu@intel.com/ --- include/hw/pci/pci.h | 4 ++++ hw/pci/pci.c | 16 ++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index f59aef5a329a..10c81287b6b3 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -372,8 +372,12 @@ typedef struct PCIIOMMUOps PCIIOMMUOps; struct PCIIOMMUOps { AddressSpace * (*get_address_space)(PCIBus *bus, void *opaque, int32_t devfn); + int (*get_iommu_attr)(PCIBus *bus, void *opaque, int32_t devfn, + enum IOMMUMemoryRegionAttr attr, void *data); }; void pci_setup_iommu_ops(PCIBus *bus, const PCIIOMMUOps *iommu_ops, void *opaque); +int pci_device_iommu_get_attr(PCIDevice *dev, enum IOMMUMemoryRegionAttr attr, + void *data); pcibus_t pci_bar_address(PCIDevice *d, int reg, uint8_t type, pcibus_t size); diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 90ae92a43d85..91ba6f0927a4 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2710,6 +2710,22 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) return &address_space_memory; } +int pci_device_iommu_get_attr(PCIDevice *dev, enum IOMMUMemoryRegionAttr attr, + void *data) +{ + PCIBus *bus, *iommu_bus; + uint8_t devfn; + + pci_device_get_iommu_bus_devfn(dev, &bus, &iommu_bus, &devfn); + if (!pci_bus_bypass_iommu(bus) && iommu_bus && + iommu_bus->iommu_ops && iommu_bus->iommu_ops->get_iommu_attr) { + return iommu_bus->iommu_ops->get_iommu_attr(bus, iommu_bus->iommu_opaque, + devfn, attr, data); + } + + return -ENOENT; +} + void pci_setup_iommu(PCIBus *bus, PCIIOMMUFunc fn, void *opaque) { bus->iommu_fn = fn; From patchwork Thu Jun 22 21:48:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798711 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=0HCeSWev; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZB3q0Dz20Xt for ; Fri, 23 Jun 2023 07:50:26 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBO-0003Bs-Cr; Thu, 22 Jun 2023 17:49:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBL-0003Bf-Nm for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:36 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBK-0007RT-4E for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:35 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MGKEE3020601; Thu, 22 Jun 2023 21:49:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=rWzue+QbNU//80kWBcBr6ZZ15PdalMOP5DaZE9JScdg=; b=0HCeSWev/7mhRtQm8ZkcsTatYH/tIDUZT4tKgTZIbMYKiA6p50L0qeu8j7szXdKzEShT fjUF7whS+2FgO8RgXwxBuejAjwSRm0aB8yUosC1qTO0pxVjBRAcvizWjxwpfxtnAl4eV yLyeFYzTu4dljS5d2rqIMj4esa18+QQ2pswvBWyRtUPb8iG5+t92h3PFJydXrNDEv63D AHOSHv+i98AL0z0FbXFw02LBB+pvEm5DDJ7TwaLwXdTVHTzoi0sK6i9ELWOOWCQXh+Wn SSr7A98qC6EErWdivhIA2YOasiLu9Qhrb0WcGGWOShQFXSmpnI7fvNk/RNSEMFXuOIqi Kw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r94ettxm3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:30 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKIV2s008333; Thu, 22 Jun 2023 21:49:29 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:29 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76s035791; Thu, 22 Jun 2023 21:49:28 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-5; Thu, 22 Jun 2023 21:49:28 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins , Yi Liu Subject: [PATCH v4 04/15] intel-iommu: Switch to pci_setup_iommu_ops() Date: Thu, 22 Jun 2023 22:48:34 +0100 Message-Id: <20230622214845.3980-5-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: 8NiyYnk7YI-CgUdrWTNPM6zF65L-PwD7 X-Proofpoint-ORIG-GUID: 8NiyYnk7YI-CgUdrWTNPM6zF65L-PwD7 Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Yi Liu Use the PCI IOMMU setup function that supply a PCIIOMMUOps argument. This is in preparation to support fetching vIOMMU information via pci_device_get_iommu_attr() which will require switching the driver to pci_setup_iommu_ops(). Signed-off-by: Yi Liu [joao: Split from the original patch] Signed-off-by: Joao Martins --- Splitted from: https://lore.kernel.org/all/20210302203827.437645-5-yi.l.liu@intel.com/#Z2e.:20210302203827.437645-5-yi.l.liu::40intel.com:1hw:i386:intel_iommu.c --- hw/i386/intel_iommu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 94d52f4205d2..1606d1b952d0 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -4032,6 +4032,10 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) return &vtd_as->as; } +static PCIIOMMUOps vtd_iommu_ops = { + .get_address_space = vtd_host_dma_iommu, +}; + static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) { X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); @@ -4155,7 +4159,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) g_free, g_free); vtd_init(s); sysbus_mmio_map(SYS_BUS_DEVICE(s), 0, Q35_HOST_BRIDGE_IOMMU_ADDR); - pci_setup_iommu(bus, vtd_host_dma_iommu, dev); + pci_setup_iommu_ops(bus, &vtd_iommu_ops, dev); /* Pseudo address space under root PCI bus. */ x86ms->ioapic_as = vtd_host_dma_iommu(bus, s, Q35_PSEUDO_DEVFN_IOAPIC); qemu_add_machine_init_done_notifier(&vtd_machine_done_notify); From patchwork Thu Jun 22 21:48:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798717 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=HjIx2mcX; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZn6nmXz20Xt for ; Fri, 23 Jun 2023 07:50:57 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBQ-0003CX-Ki; Thu, 22 Jun 2023 17:49:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBP-0003C8-2O for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:39 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBN-0007S0-Cd for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:38 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MK1l65018598; Thu, 22 Jun 2023 21:49:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=qTwK/JTb+xvlDNF0u9lhU5k4w0ZuGK+yQufGE7YThXM=; b=HjIx2mcXTbs9V/LaxPeZFoS813wrOTESt0iFKVrllx5ETQqZKMTBeHNgH9KdovkXdOHT WfAakN5F3nABmH3NHsJA3z1Gtep655dk+L3y1tDcyDdWOSH/RhaGyE/9XcYysuuli+MO RyHdc04eldNVOx4VOiP9fEveeXgmo6QhqdnCcUveSHlnQmd8yf++zEys3O9l2xHoeeOX AAaqxFdwd3zErkTRe4juIS3dE5ql0u/DhYjszjJYGB4c/8dHAMqUWGW+z6fpvqRj0kAW b5vV/sLpHoqe7dM+aNQIsDfReFBpLrXYrvwyg9EtESnH08HOtxx5xoorvxtp5PUkhoEP rQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r938dtt0d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:34 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJv6AN008427; Thu, 22 Jun 2023 21:49:33 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epf7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:33 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76u035791; Thu, 22 Jun 2023 21:49:33 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-6; Thu, 22 Jun 2023 21:49:32 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 05/15] memory/iommu: Add IOMMU_ATTR_DMA_TRANSLATION attribute Date: Thu, 22 Jun 2023 22:48:35 +0100 Message-Id: <20230622214845.3980-6-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=767 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: UY2x35i-9xBWgr5fR0Smycbk_VRhwKIt X-Proofpoint-ORIG-GUID: UY2x35i-9xBWgr5fR0Smycbk_VRhwKIt Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a new IOMMU attribute IOMMU_ATTR_DMA_TRANSLATION which indicates whether the IOMMU supports DMA Translation. This attribute will be used by VFIO device dirty page tracking so it can restrict the IOVA under tracking to the memory map when vIOMMU is enabled only for interrupt remapping without dma translation enabled. Signed-off-by: Joao Martins --- include/exec/memory.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index 47c2e0221c35..5d6c2ab1f397 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -319,7 +319,8 @@ typedef struct MemoryRegionClass { enum IOMMUMemoryRegionAttr { - IOMMU_ATTR_SPAPR_TCE_FD + IOMMU_ATTR_SPAPR_TCE_FD, + IOMMU_ATTR_DMA_TRANSLATION, }; /* From patchwork Thu Jun 22 21:48:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798716 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=3eBV1M4N; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZj0WHYz20Xt for ; Fri, 23 Jun 2023 07:50:53 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBU-0003EM-Gu; Thu, 22 Jun 2023 17:49:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBS-0003Cg-HX for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:42 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBQ-0007SX-VJ for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:42 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJo5IW021572; Thu, 22 Jun 2023 21:49:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=SoXbQf/Kl+K/f9RW8RzgtC5CWtwRb6pbFr87bE/fgwE=; b=3eBV1M4NaqNrywQAiJb5zYd+ugc8N4DIQN9I2tYVYrMaML1fL9XCjotSazYpVv8Jx+pI nP0njVqoDkoRqKy0hVNx3F1iLpcSeB5pLt1uu6aAwskXtBO3n3D0TkwIfkoLkU+R5qsY HJtP2qDzpe0CRK5ubL5melHSJcuwdFV6HNd/6EBp2ee/tcNKPt0XJX1RFTIiY0ntoSCI DADEEDY/lO0ErglhKzJRJ2+MB3LDhbsrFS3QBoZ2J/myzLB2wCG+wZF6J2vg1xqeUFWL ablw57HVR5WAw3CPAI8uEP8olkoT7liU5B/i8BUozuVByEbk9hb2sKSYoLPAd2pYasWR Kw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93m3txus-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:38 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKQVLL008308; Thu, 22 Jun 2023 21:49:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epgq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:37 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn76w035791; Thu, 22 Jun 2023 21:49:37 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-7; Thu, 22 Jun 2023 21:49:37 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 06/15] intel-iommu: Implement get_attr() method Date: Thu, 22 Jun 2023 22:48:36 +0100 Message-Id: <20230622214845.3980-7-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: MPrgm5I5Q8AIfZDO3JVgW19cWfA7HW0c X-Proofpoint-ORIG-GUID: MPrgm5I5Q8AIfZDO3JVgW19cWfA7HW0c Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Implement IOMMU MR get_attr() method and use the dma_translation property to report the IOMMU_ATTR_DMA_TRANSLATION attribute. Additionally add the necessary get_iommu_attr into the PCIIOMMUOps to support pci_device_iommu_get_attr(). The callback in there acts as a IOMMU-specific address space walker which will call get_attr in the IOMMUMemoryRegion backing the device to fetch the desired attribute. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/i386/intel_iommu.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 1606d1b952d0..ed2a46e008df 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3861,6 +3861,29 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) return; } +static int vtd_iommu_get_attr(IOMMUMemoryRegion *iommu_mr, + enum IOMMUMemoryRegionAttr attr, void *data) +{ + VTDAddressSpace *vtd_as = container_of(iommu_mr, VTDAddressSpace, iommu); + IntelIOMMUState *s = vtd_as->iommu_state; + int ret = 0; + + switch (attr) { + case IOMMU_ATTR_DMA_TRANSLATION: + { + bool *enabled = (bool *)(uintptr_t) data; + + *enabled = s->dma_translation; + break; + } + default: + ret = -EINVAL; + break; + } + + return ret; +} + /* Do the initialization. It will also be called when reset, so pay * attention when adding new initialization stuff. */ @@ -4032,8 +4055,24 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) return &vtd_as->as; } +static int vtd_get_iommu_attr(PCIBus *bus, void *opaque, int32_t devfn, + enum IOMMUMemoryRegionAttr attr, void *data) +{ + IntelIOMMUState *s = opaque; + VTDAddressSpace *vtd_as; + + assert(0 <= devfn && devfn < PCI_DEVFN_MAX); + + vtd_as = vtd_find_add_as(s, bus, devfn, PCI_NO_PASID); + if (!vtd_as) + return -EINVAL; + + return memory_region_iommu_get_attr(&vtd_as->iommu, attr, data); +} + static PCIIOMMUOps vtd_iommu_ops = { .get_address_space = vtd_host_dma_iommu, + .get_iommu_attr = vtd_get_iommu_attr, }; static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) @@ -4197,6 +4236,7 @@ static void vtd_iommu_memory_region_class_init(ObjectClass *klass, imrc->translate = vtd_iommu_translate; imrc->notify_flag_changed = vtd_iommu_notify_flag_changed; imrc->replay = vtd_iommu_replay; + imrc->get_attr = vtd_iommu_get_attr; } static const TypeInfo vtd_iommu_memory_region_info = { From patchwork Thu Jun 22 21:48:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798713 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=t0Nyj1zx; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZK0yG5z20Xt for ; Fri, 23 Jun 2023 07:50:33 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBY-0003Ex-AH; Thu, 22 Jun 2023 17:49:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBW-0003El-SW for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:46 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBV-0007TY-6Z for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:46 -0400 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MIbN5I031279; Thu, 22 Jun 2023 21:49:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=2XsM4/X8GgWe6VN83nLwHow/PhcL0rm5V9PFVUE9tL0=; b=t0Nyj1zx9l7cZHMqvTeHWdO7Mef8jYa/nPbuJFPqDT5gqWwQ7SQ6Z7yPgF/r8hMrcQd7 1cp8D/ncZy6/Jt4g1OcZKlAC4QhHWauCCI8xLRdNBhIGtDqNztQXXbXWnaDM+PkP2Mz9 5ikx1tKgxuTcCnqNhOPl/VnB/Ku8AlGPRYBoN5b+Lmsq1Jvf69wVoiKL4E+cNzedpUm+ VFRgUoCN+nezYExfu2wwsMKa4BBrnxT/eXMGleplA38t0Df+q0PvwsEoxG85jauIGWwd q5mXYOuZ4JmrCWZIDSKphxjgc5jxJiQ6cooOhrp97S+UgiWWhHWJspHJhK7rdf5oyvGZ XA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r94ettxm9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:43 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJgThM008357; Thu, 22 Jun 2023 21:49:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epj8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:41 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn770035791; Thu, 22 Jun 2023 21:49:41 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-8; Thu, 22 Jun 2023 21:49:41 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 07/15] vfio/common: Track whether DMA Translation is enabled on the vIOMMU Date: Thu, 22 Jun 2023 22:48:37 +0100 Message-Id: <20230622214845.3980-8-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: egVIolT5eitF4iXJX4JhxLyUc6xTIC-z X-Proofpoint-ORIG-GUID: egVIolT5eitF4iXJX4JhxLyUc6xTIC-z Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org vfio_get_group() allocates and fills the group/container/space on success which will store the AddressSpace inside the VFIOSpace struct. Use the newly added pci_device_iommu_get_attr() to see if DMA translation is enabled or not. Assume that by default it is enabled. Today, this means only intel-iommu supports it. Signed-off-by: Joao Martins --- include/hw/vfio/vfio-common.h | 1 + hw/vfio/pci.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index eed244f25f34..f41860988d6b 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -70,6 +70,7 @@ typedef struct VFIOMigration { typedef struct VFIOAddressSpace { AddressSpace *as; + bool no_dma_translation; QLIST_HEAD(, VFIOContainer) containers; QLIST_ENTRY(VFIOAddressSpace) list; } VFIOAddressSpace; diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 73874a94de12..8a98e6ffc480 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2900,6 +2900,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) VFIOPCIDevice *vdev = VFIO_PCI(pdev); VFIODevice *vbasedev = &vdev->vbasedev; VFIODevice *vbasedev_iter; + VFIOAddressSpace *space; VFIOGroup *group; char *tmp, *subsys, group_path[PATH_MAX], *group_name; Error *err = NULL; @@ -2907,7 +2908,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) struct stat st; int groupid; int i, ret; - bool is_mdev; + bool is_mdev, dma_translation; char uuid[UUID_FMT_LEN]; char *name; @@ -2961,6 +2962,18 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) goto error; } + space = group->container->space; + + /* + * Support for toggling DMA translation is optional. + * By default, DMA translation is assumed to be enabled i.e. + * space::no_dma_translation is 0. + */ + dma_translation = true; + pci_device_iommu_get_attr(pdev, IOMMU_ATTR_DMA_TRANSLATION, + &dma_translation); + space->no_dma_translation = !dma_translation; + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { if (strcmp(vbasedev_iter->name, vbasedev->name) == 0) { error_setg(errp, "device is already attached"); From patchwork Thu Jun 22 21:48:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798712 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=oQ8UoLeq; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZJ01GDz20Xt for ; Fri, 23 Jun 2023 07:50:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBb-0003Fl-Ss; Thu, 22 Jun 2023 17:49:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBa-0003FW-FA for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:50 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBZ-0007Ua-0f for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:49:50 -0400 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MIIq8O005803; Thu, 22 Jun 2023 21:49:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Po/ra4dh07FrZr1NNp3KpZMu9aR7qsmzh97ErMcsxjQ=; b=oQ8UoLeq+7hFBmk6fgDh6YJpvQnoXvLY0aywHxv1BmfWuAi+KQcnhv/sRDn33gi22WNo R8G/cX4k+hcyw7YGnt7uH5zcAx3k0KzPfn9BlSoPRHQhL/5LhKCVDSD9W+J+IADW3Dl2 pPvIMbz+TOPhrvR8l8KoHjO+b+5hLL9nRktmQhxHbl1tDGsKPTZD4rEFCyHIxma9EtV4 viILI/hCPxPAEVonxdeEdWuvEXg1AooIR37NkIiTqtzjsrBP3RfTjOxRS9OY5kPYq9Ta f8TPuumi3jJixirW3STttzcwYsnN3tLFnpI53lDz1JB4t4TDGme7+aAExJo0hVMxEg8u Sw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r94vcu0q7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:46 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKol2V008185; Thu, 22 Jun 2023 21:49:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epkj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:45 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn772035791; Thu, 22 Jun 2023 21:49:45 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-9; Thu, 22 Jun 2023 21:49:44 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 08/15] vfio/common: Relax vIOMMU detection when DMA translation is off Date: Thu, 22 Jun 2023 22:48:38 +0100 Message-Id: <20230622214845.3980-9-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=882 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: rqWZzmVIHrM4k1sBkTWEexyoA7cpoQBa X-Proofpoint-ORIG-GUID: rqWZzmVIHrM4k1sBkTWEexyoA7cpoQBa Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Relax the vIOMMU migration blocker when the underlying IOMMU reports DMA translation disabled. When it is disabled there will be no DMA mappings via the vIOMMU and the guest can only use it for Interrupt Remapping. The latter is done via vfio_viommu_preset() return value where in addition to validating that the address space is memory, we also check whether the vIOMMU backing the PCI device has DMA translation on. It is assumed to be enabled, if the IOMMU model does not support toggling on/off the dma-translation property. Intel IOMMU right now is the only case supporting, although AMD IOMMU can in theory provide the same functionality. Signed-off-by: Joao Martins --- hw/vfio/common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index fa8fd949b1cf..17c1d882e221 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -419,7 +419,8 @@ static bool vfio_viommu_preset(void) VFIOAddressSpace *space; QLIST_FOREACH(space, &vfio_address_spaces, list) { - if (space->as != &address_space_memory) { + if ((space->as != &address_space_memory) && + !space->no_dma_translation) { return true; } } From patchwork Thu Jun 22 21:48:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798723 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=z/LLDH43; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDcF1Rcyz20Xt for ; Fri, 23 Jun 2023 07:52:13 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBm-0003Nb-5D; Thu, 22 Jun 2023 17:50:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBk-0003Mq-8D for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:00 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBi-0007WQ-MY for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:00 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJu7EF018131; Thu, 22 Jun 2023 21:49:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=tza8bXvQeCVQr3AkHiNg0rEv+YDBN54/6GSbdRIwV8c=; b=z/LLDH43/M0YIro88XcSLjk//eYW/QFTyMQRh0liw1Uxhbobn3XCw4agNsrpn53U9gQ/ gQ+8hAtNnp6efQxdRQeuGu0vfe12kaTOez0NXKiye2OawSzkuRJBeSv1cn1h9nkhA96F UZVhgRQCy2LER+mAmZEyug0dC92ER5ZInT0X/PmJIewRr+bAEX1POea98YmHqGAIDndh d/zlQerAkK/eQHnXrLcp62doVe8Af4fkbJB2M90haV5Bnrt8iaTQfJEno4ynyVwzVusZ iIXWPuEpsxg50q9Ubh4YZVKcZ0lQpVKe0E2OXyvYFriW31fX2RItSV2QQZe6H6AKqjWE gA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93rbtw5q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:53 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKol2Y008185; Thu, 22 Jun 2023 21:49:52 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398eppg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:49:52 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn774035791; Thu, 22 Jun 2023 21:49:52 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-10; Thu, 22 Jun 2023 21:49:49 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 09/15] memory/iommu: Add IOMMU_ATTR_MAX_IOVA attribute Date: Thu, 22 Jun 2023 22:48:39 +0100 Message-Id: <20230622214845.3980-10-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=982 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-ORIG-GUID: 0O4X65kGxE_hq8f_DqxEivt0mAlZpDyh X-Proofpoint-GUID: 0O4X65kGxE_hq8f_DqxEivt0mAlZpDyh Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Add a new IOMMU attribute IOMMU_ATTR_MAX_IOVA which indicates the maximal IOVA that an IOMMU can use. This attribute will be used by VFIO device dirty page tracking so it can track the entire IOVA space when needed (i.e. when vIOMMU is enabled). Signed-off-by: Avihai Horon Signed-off-by: Joao Martins Acked-by: Peter Xu --- include/exec/memory.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 5d6c2ab1f397..742bff82dc77 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -321,6 +321,7 @@ typedef struct MemoryRegionClass { enum IOMMUMemoryRegionAttr { IOMMU_ATTR_SPAPR_TCE_FD, IOMMU_ATTR_DMA_TRANSLATION, + IOMMU_ATTR_MAX_IOVA, }; /* From patchwork Thu Jun 22 21:48:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798722 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=LONTXQeU; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDc742HSz20Xt for ; Fri, 23 Jun 2023 07:52:07 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBs-0003OX-Ft; Thu, 22 Jun 2023 17:50:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBr-0003OO-34 for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:07 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBp-0007kT-GG for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:06 -0400 Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKMFt1004331; Thu, 22 Jun 2023 21:50:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=D8sJBWHFKbzRhdEFK0ztebWVJvFcWgua+sz/1uYPWSg=; b=LONTXQeUr7pGA7lPwfplCXu4FbtxR0cD4RYgjmEiqHg33AvguO80gcWBa2zEA7SzhvDQ pQOivcOjE1YxoH13PMRrIamcAgqclEL5WUHQA+rmwZeASXcBXj5JXwUdtjF2zN4E3b5z WI0c2rKaLeLqlABio1AsHiALxegQiieso8krpGvMFLYEUEcNArPQhMbuDgE+uvi/g0yR v2kUNOUmxuB3BjapZy17FjYSmTipZLP9+u4N81UFar21gVEB1e8yjty3n8EIa2f7YpZ8 plcbTeLBQPTPXWixlAYoMutIF906HjyhFGgKbH/r+jifK4ujN43FQNukEfGLPLtKSDP1 Ig== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93rbtw6b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:00 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJnlAx008353; Thu, 22 Jun 2023 21:50:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398ept7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:00 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn776035791; Thu, 22 Jun 2023 21:49:59 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-11; Thu, 22 Jun 2023 21:49:56 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 10/15] intel-iommu: Implement IOMMU_ATTR_MAX_IOVA get_attr() attribute Date: Thu, 22 Jun 2023 22:48:40 +0100 Message-Id: <20230622214845.3980-11-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-ORIG-GUID: ZXWFStTShDYH87vFRp9vROKOTcwmhV_V X-Proofpoint-GUID: ZXWFStTShDYH87vFRp9vROKOTcwmhV_V Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Implement get_attr() method and use the address width property to report the IOMMU_ATTR_MAX_IOVA attribute. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/i386/intel_iommu.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index ed2a46e008df..989993e303a6 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3876,6 +3876,13 @@ static int vtd_iommu_get_attr(IOMMUMemoryRegion *iommu_mr, *enabled = s->dma_translation; break; } + case IOMMU_ATTR_MAX_IOVA: + { + hwaddr *max_iova = (hwaddr *)(uintptr_t) data; + + *max_iova = MAKE_64BIT_MASK(0, s->aw_bits);; + break; + } default: ret = -EINVAL; break; From patchwork Thu Jun 22 21:48:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798719 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=V3li4+yI; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDbJ0BlXz20ZH for ; Fri, 23 Jun 2023 07:51:24 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSBv-0003Pa-41; Thu, 22 Jun 2023 17:50:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBt-0003Oz-GJ for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:09 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSBr-0007lU-Po for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:09 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MK24Dp019481; Thu, 22 Jun 2023 21:50:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=bQtVazwT3QoQkJyOPrjd5tGv+nILONCMQ+b89mlhz4c=; b=V3li4+yIeP+qLVozSIfmAoUe1oI2Kmp4Gvp/cVhtqZCRDuNEYyZ0yfZEAJQfbcTqosl6 zR4zfVtll92jMBp/YLFWUJLt+2X0e9VXkU7VfpGJmfQnnLzDgKCsXzl+mNEYGSSS6nSg I8O+KYa8O+mptKxLEbtx04At9TS7vbDeLOBauwdJu8yXK/3UbctenGmoiqGgEd4Hnvrf jPdZp8CKRhFwnuxz9BVRMjR7OcWnv72Md9v4r0+8f/K/+TM8nDEV5YrRvMDT1QZQWMQ7 Ue9yfVj1UhZL7vo8foFkb5o8gbcKjbd6q4uLupbKl/4AqxJLMIPii+DWhYJSRs5MQTDo wQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r938dtt0w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:05 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJv6Ag008427; Thu, 22 Jun 2023 21:50:04 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398epwr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:04 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn778035791; Thu, 22 Jun 2023 21:50:04 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-12; Thu, 22 Jun 2023 21:50:03 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 11/15] vfio/common: Move dirty tracking ranges update to helper Date: Thu, 22 Jun 2023 22:48:41 +0100 Message-Id: <20230622214845.3980-12-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: 0u0E5PCU_Cow0xh1xsnImdt7MZzSl9xp X-Proofpoint-ORIG-GUID: 0u0E5PCU_Cow0xh1xsnImdt7MZzSl9xp Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Separate the changes that updates the ranges from the listener, to make it reusable in preparation to expand its use to vIOMMU support. Signed-off-by: Joao Martins --- hw/vfio/common.c | 36 +++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 17c1d882e221..ecfb9afb3fb6 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1413,20 +1413,10 @@ typedef struct VFIODirtyRangesListener { MemoryListener listener; } VFIODirtyRangesListener; -static void vfio_dirty_tracking_update(MemoryListener *listener, - MemoryRegionSection *section) +static void vfio_dirty_tracking_update(hwaddr iova, hwaddr end, + VFIODirtyRanges *range) { - VFIODirtyRangesListener *dirty = container_of(listener, - VFIODirtyRangesListener, - listener); - VFIODirtyRanges *range = &dirty->ranges; - hwaddr iova, end, *min, *max; - - if (!vfio_listener_valid_section(section, "tracking_update") || - !vfio_get_section_iova_range(dirty->container, section, - &iova, &end, NULL)) { - return; - } + hwaddr *min, *max; /* * The address space passed to the dirty tracker is reduced to two ranges: @@ -1450,12 +1440,28 @@ static void vfio_dirty_tracking_update(MemoryListener *listener, } trace_vfio_device_dirty_tracking_update(iova, end, *min, *max); - return; +} + +static void vfio_listener_dirty_tracking_update(MemoryListener *listener, + MemoryRegionSection *section) +{ + VFIODirtyRangesListener *dirty = container_of(listener, + VFIODirtyRangesListener, + listener); + hwaddr iova, end; + + if (!vfio_listener_valid_section(section, "tracking_update") || + !vfio_get_section_iova_range(dirty->container, section, + &iova, &end, NULL)) { + return; + } + + vfio_dirty_tracking_update(iova, end, &dirty->ranges); } static const MemoryListener vfio_dirty_tracking_listener = { .name = "vfio-tracking", - .region_add = vfio_dirty_tracking_update, + .region_add = vfio_listener_dirty_tracking_update, }; static void vfio_dirty_tracking_init(VFIOContainer *container, From patchwork Thu Jun 22 21:48:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798724 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=lu5fCYSm; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDdz5XGFz20Xt for ; Fri, 23 Jun 2023 07:53:43 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSF8-0004GJ-Lz; Thu, 22 Jun 2023 17:53:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSF7-0004G4-78 for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:53:29 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSF5-0000qm-Bc for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:53:28 -0400 Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35ML7LUS015246; Thu, 22 Jun 2023 21:53:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=CtbV7jP++Ydqd322B7U0VwYChw7FX8rCFSPVusynzhM=; b=lu5fCYSmmYZ67QHPxK5gPSHSRRzXk0VgZBgIfq2EE3LcHrvLdCbSMOoUg3wRR0jo9w2U Dv8o3MeFFmpwPKfd55cCqK0y09GkD8sXqzgk2T0cXnIkch0h3xlqUCn5APvj4j7DGxXq EIeFXmkv2tq6JzhJTDDHuDEefClMi2PlED5PccTaApLFj0HNjwj+1QfAwqFX6v2CeDug VH5M1rcH/dKOR5y7tB1T9sTUPBGQ8u7tNxg+hSwT4Xp6M9EQXQaN5d9iwrAdVvt9uNvp HP2ZypM5Zb1Rd77FWYWPx7Pr+b+7JmEJfgAR/wEjocJ2Z4EDnbRvZhZlweyYFkTY7lhx AA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r95cu2upj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:53:20 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKvXxP008371; Thu, 22 Jun 2023 21:50:08 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398eq1v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:08 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn77C035791; Thu, 22 Jun 2023 21:50:08 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-13; Thu, 22 Jun 2023 21:50:08 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 12/15] vfio/common: Support device dirty page tracking with vIOMMU Date: Thu, 22 Jun 2023 22:48:42 +0100 Message-Id: <20230622214845.3980-13-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: jKx3rXMJ6WCQrp1O3Cr69SLkXbIuH8rb X-Proofpoint-ORIG-GUID: jKx3rXMJ6WCQrp1O3Cr69SLkXbIuH8rb Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Currently, device dirty page tracking with vIOMMU is not supported, and a blocker is added and the migration is prevented. When vIOMMU is used, IOVA ranges are DMA mapped/unmapped on the fly as requesting by the vIOMMU. These IOVA ranges can potentially be mapped anywhere in the vIOMMU IOVA space as advertised by the VMM. To support device dirty tracking when vIOMMU enabled instead create the dirty ranges based on the vIOMMU provided limits, which leads to the tracking of the whole IOVA space regardless of what devices use. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- include/hw/vfio/vfio-common.h | 1 + hw/vfio/common.c | 58 +++++++++++++++++++++++++++++------ hw/vfio/pci.c | 7 +++++ 3 files changed, 56 insertions(+), 10 deletions(-) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index f41860988d6b..c4bafad084b4 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -71,6 +71,7 @@ typedef struct VFIOMigration { typedef struct VFIOAddressSpace { AddressSpace *as; bool no_dma_translation; + hwaddr max_iova; QLIST_HEAD(, VFIOContainer) containers; QLIST_ENTRY(VFIOAddressSpace) list; } VFIOAddressSpace; diff --git a/hw/vfio/common.c b/hw/vfio/common.c index ecfb9afb3fb6..85fddef24026 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -428,6 +428,25 @@ static bool vfio_viommu_preset(void) return false; } +static int vfio_viommu_get_max_iova(hwaddr *max_iova) +{ + VFIOAddressSpace *space; + + *max_iova = 0; + + QLIST_FOREACH(space, &vfio_address_spaces, list) { + if (space->as == &address_space_memory) { + continue; + } + + if (*max_iova < space->max_iova) { + *max_iova = space->max_iova; + } + } + + return *max_iova == 0; +} + int vfio_block_giommu_migration(Error **errp) { int ret; @@ -1464,10 +1483,11 @@ static const MemoryListener vfio_dirty_tracking_listener = { .region_add = vfio_listener_dirty_tracking_update, }; -static void vfio_dirty_tracking_init(VFIOContainer *container, +static int vfio_dirty_tracking_init(VFIOContainer *container, VFIODirtyRanges *ranges) { VFIODirtyRangesListener dirty; + int ret; memset(&dirty, 0, sizeof(dirty)); dirty.ranges.min32 = UINT32_MAX; @@ -1475,17 +1495,29 @@ static void vfio_dirty_tracking_init(VFIOContainer *container, dirty.listener = vfio_dirty_tracking_listener; dirty.container = container; - memory_listener_register(&dirty.listener, - container->space->as); + if (vfio_viommu_preset()) { + hwaddr iommu_max_iova; + + ret = vfio_viommu_get_max_iova(&iommu_max_iova); + if (ret) { + return -EINVAL; + } + + vfio_dirty_tracking_update(0, iommu_max_iova, &dirty.ranges); + } else { + memory_listener_register(&dirty.listener, + container->space->as); + /* + * The memory listener is synchronous, and used to calculate the range + * to dirty tracking. Unregister it after we are done as we are not + * interested in any follow-up updates. + */ + memory_listener_unregister(&dirty.listener); + } *ranges = dirty.ranges; - /* - * The memory listener is synchronous, and used to calculate the range - * to dirty tracking. Unregister it after we are done as we are not - * interested in any follow-up updates. - */ - memory_listener_unregister(&dirty.listener); + return 0; } static void vfio_devices_dma_logging_stop(VFIOContainer *container) @@ -1590,7 +1622,13 @@ static int vfio_devices_dma_logging_start(VFIOContainer *container) VFIOGroup *group; int ret = 0; - vfio_dirty_tracking_init(container, &ranges); + ret = vfio_dirty_tracking_init(container, &ranges); + if (ret) { + error_report("Failed to init DMA logging ranges, err %d", + ret); + return -EOPNOTSUPP; + } + feature = vfio_device_feature_dma_logging_start_create(container, &ranges); if (!feature) { diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 8a98e6ffc480..3bda5618c5b5 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -2974,6 +2974,13 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) &dma_translation); space->no_dma_translation = !dma_translation; + /* + * Support for advertised IOMMU address space boundaries is optional. + * By default, it is not advertised i.e. space::max_iova is 0. + */ + pci_device_iommu_get_attr(pdev, IOMMU_ATTR_MAX_IOVA, + &space->max_iova); + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { if (strcmp(vbasedev_iter->name, vbasedev->name) == 0) { error_setg(errp, "device is already attached"); From patchwork Thu Jun 22 21:48:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798721 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=De6JWYhM; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDbg2Hpkz20Xt for ; Fri, 23 Jun 2023 07:51:43 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSCE-0004Ew-9e; Thu, 22 Jun 2023 17:50:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSC8-0003zF-9U for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:24 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSC6-0007vY-JI for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:24 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35ML3wKM002927; Thu, 22 Jun 2023 21:50:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=7yxtD0o2/JPTW1QMXSr2K1uLEa3RYqb7B3xWOE/Esi8=; b=De6JWYhMnjQuqW+aQXMIgsLTPOfwSsSlaqB8xgU7jhF9fnH0Ra6PiGLvawm+cc9RVFSj cb9OutfTiAooXth1lX6Qt09O0ASxGXZkNbCZJucSRZBmy3wbgXS5gVKafH2vlHVLISrF cMacfv0IC2G7pUyFrMkXq2raA/tNlvJqKzBeeymHHaOm5uZb1ra8odtYQBSb5VWivSGn H58XfmJ0iTv/f3qcdhL5qiAwWHXWaq9v1QJykQUVsKEBSXeuOmxJ0d19pndcNYZGEhVY Bpo6cASQhKywkIJG4RHpRSe9HCIVlZS2q1Ln2HzrPqKHR6wNYm13Oe+U6HgWO34z/g71 Gw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r94qaanq6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:18 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKN19w008428; Thu, 22 Jun 2023 21:50:14 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398eq5r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:14 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn77E035791; Thu, 22 Jun 2023 21:50:13 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-14; Thu, 22 Jun 2023 21:50:12 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 13/15] vfio/common: Extract vIOMMU code from vfio_sync_dirty_bitmap() Date: Thu, 22 Jun 2023 22:48:43 +0100 Message-Id: <20230622214845.3980-14-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=877 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: WjT9HdHYQJmMfLHHOvA4x2DWXpTkjzCe X-Proofpoint-ORIG-GUID: WjT9HdHYQJmMfLHHOvA4x2DWXpTkjzCe Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon Extract vIOMMU code from vfio_sync_dirty_bitmap() to a new function and restructure the code. This is done in preparation for optimizing vIOMMU deviice dirty page tracking. No functional changes intended. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 63 +++++++++++++++++++++++++++++------------------- 1 file changed, 38 insertions(+), 25 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 85fddef24026..c530e9d87f21 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1914,37 +1914,50 @@ static int vfio_sync_ram_discard_listener_dirty_bitmap(VFIOContainer *container, &vrdl); } +static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, + MemoryRegionSection *section) +{ + VFIOGuestIOMMU *giommu; + bool found = false; + Int128 llend; + vfio_giommu_dirty_notifier gdn; + int idx; + + QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { + if (MEMORY_REGION(giommu->iommu_mr) == section->mr && + giommu->n.start == section->offset_within_region) { + found = true; + break; + } + } + + if (!found) { + return 0; + } + + gdn.giommu = giommu; + idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, + MEMTXATTRS_UNSPECIFIED); + + llend = int128_add(int128_make64(section->offset_within_region), + section->size); + llend = int128_sub(llend, int128_one()); + + iommu_notifier_init(&gdn.n, vfio_iommu_map_dirty_notify, IOMMU_NOTIFIER_MAP, + section->offset_within_region, int128_get64(llend), + idx); + memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); + + return 0; +} + static int vfio_sync_dirty_bitmap(VFIOContainer *container, MemoryRegionSection *section) { ram_addr_t ram_addr; if (memory_region_is_iommu(section->mr)) { - VFIOGuestIOMMU *giommu; - - QLIST_FOREACH(giommu, &container->giommu_list, giommu_next) { - if (MEMORY_REGION(giommu->iommu_mr) == section->mr && - giommu->n.start == section->offset_within_region) { - Int128 llend; - vfio_giommu_dirty_notifier gdn = { .giommu = giommu }; - int idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, - MEMTXATTRS_UNSPECIFIED); - - llend = int128_add(int128_make64(section->offset_within_region), - section->size); - llend = int128_sub(llend, int128_one()); - - iommu_notifier_init(&gdn.n, - vfio_iommu_map_dirty_notify, - IOMMU_NOTIFIER_MAP, - section->offset_within_region, - int128_get64(llend), - idx); - memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); - break; - } - } - return 0; + return vfio_sync_iommu_dirty_bitmap(container, section); } else if (memory_region_has_ram_discard_manager(section->mr)) { return vfio_sync_ram_discard_listener_dirty_bitmap(container, section); } From patchwork Thu Jun 22 21:48:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=AdcwoVpD; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDZW5DMbz20Xt for ; Fri, 23 Jun 2023 07:50:43 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSC9-00041w-U2; Thu, 22 Jun 2023 17:50:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSC7-0003xk-Ul for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:23 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSC5-0007v2-RH for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:23 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJo4a3021548; Thu, 22 Jun 2023 21:50:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=9kNKx6R9jFh+8uU24jkKwYU0650934+xnHSDNWrDuoo=; b=AdcwoVpDKBcSZMHOY1xne0xmfTsvVUpA8zHs4neSatlVAQQWZgL8jYbkSBE8lORBxlQh xV3763TBzKJM+08K6hHMd0d5c1FUooHo7I520ACb2Ie3pWVLE0fAvhF1Nsxk9kEk0GF9 J8yIq8QqeZRfXXVBvfp3X4pn85GOn9llPmsNBPfBjIeJzSgeW5IRHMwqLfKUY1Qgdnj7 79ztAm1R26bUY9YFY7Pwsssw1DFu1wfo7x7vuvzmkc1G4iBVMUAZXxqRjU9IqIdm4xpd vtyO65VZe58SJPGMjrLEnHSJzgM/Th8PISVGsIi5iGw7Opp0s327UJrqitPdJ75k89sI HA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93m3txvb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:19 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35ML7Sfn008390; Thu, 22 Jun 2023 21:50:18 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398eq96-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:18 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn77G035791; Thu, 22 Jun 2023 21:50:17 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-15; Thu, 22 Jun 2023 21:50:17 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 14/15] vfio/common: Optimize device dirty page tracking with vIOMMU Date: Thu, 22 Jun 2023 22:48:44 +0100 Message-Id: <20230622214845.3980-15-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=781 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: sK5VgQ-L1MaNJgeE2Z4X21Yt6rgcyDcP X-Proofpoint-ORIG-GUID: sK5VgQ-L1MaNJgeE2Z4X21Yt6rgcyDcP Received-SPF: pass client-ip=205.220.177.32; envelope-from=joao.m.martins@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Avihai Horon When vIOMMU is enabled, syncing dirty page bitmaps is done by replaying the vIOMMU mappings and querying the dirty bitmap for each mapping. With device dirty tracking this causes a lot of overhead, since the HW is queried many times (even with small idle guest this can end up with thousands of calls to HW). Optimize this by de-coupling dirty bitmap query from vIOMMU replay. Now a single dirty bitmap is queried per vIOMMU MR section, which is then used for all corresponding vIOMMU mappings within that MR section. Signed-off-by: Avihai Horon Signed-off-by: Joao Martins --- hw/vfio/common.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 72 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index c530e9d87f21..62f91e8e102d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1832,8 +1832,36 @@ out: typedef struct { IOMMUNotifier n; VFIOGuestIOMMU *giommu; + VFIOBitmap vbmap; } vfio_giommu_dirty_notifier; +static int vfio_iommu_set_dirty_bitmap(VFIOContainer *container, + vfio_giommu_dirty_notifier *gdn, + hwaddr iova, hwaddr size, + ram_addr_t ram_addr) +{ + VFIOBitmap *vbmap = &gdn->vbmap; + VFIOBitmap dst_vbmap; + hwaddr start_iova = REAL_HOST_PAGE_ALIGN(gdn->n.start); + hwaddr copy_offset; + int ret; + + ret = vfio_bitmap_alloc(&dst_vbmap, size); + if (ret) { + return -ENOMEM; + } + + copy_offset = (iova - start_iova) / qemu_real_host_page_size(); + bitmap_copy_with_src_offset(dst_vbmap.bitmap, vbmap->bitmap, copy_offset, + dst_vbmap.pages); + + cpu_physical_memory_set_dirty_lebitmap(dst_vbmap.bitmap, ram_addr, + dst_vbmap.pages); + g_free(dst_vbmap.bitmap); + + return 0; +} + static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) { vfio_giommu_dirty_notifier *gdn = container_of(n, @@ -1854,8 +1882,15 @@ static void vfio_iommu_map_dirty_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb) rcu_read_lock(); if (vfio_get_xlat_addr(iotlb, NULL, &translated_addr, NULL)) { - ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1, - translated_addr); + if (gdn->vbmap.bitmap) { + ret = vfio_iommu_set_dirty_bitmap(container, gdn, iova, + iotlb->addr_mask + 1, + translated_addr); + } else { + ret = vfio_get_dirty_bitmap(container, iova, iotlb->addr_mask + 1, + translated_addr); + } + if (ret) { error_report("vfio_iommu_map_dirty_notify(%p, 0x%"HWADDR_PRIx", " "0x%"HWADDR_PRIx") = %d (%s)", @@ -1936,6 +1971,7 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, } gdn.giommu = giommu; + gdn.vbmap.bitmap = NULL; idx = memory_region_iommu_attrs_to_index(giommu->iommu_mr, MEMTXATTRS_UNSPECIFIED); @@ -1943,10 +1979,44 @@ static int vfio_sync_iommu_dirty_bitmap(VFIOContainer *container, section->size); llend = int128_sub(llend, int128_one()); + /* + * Optimize device dirty tracking if the MR section is at least partially + * tracked. Optimization is done by querying a single dirty bitmap for the + * entire range instead of querying dirty bitmap for each vIOMMU mapping. + */ + if (vfio_devices_all_device_dirty_tracking(container)) { + hwaddr start = REAL_HOST_PAGE_ALIGN(section->offset_within_region); + hwaddr end = int128_get64(llend); + hwaddr iommu_max_iova; + hwaddr size; + int ret; + + ret = vfio_viommu_get_max_iova(&iommu_max_iova); + if (ret) { + return -EINVAL; + } + + size = REAL_HOST_PAGE_ALIGN(MIN(iommu_max_iova, end) - start); + + ret = vfio_bitmap_alloc(&gdn.vbmap, size); + if (ret) { + return -ENOMEM; + } + + ret = vfio_devices_query_dirty_bitmap(container, &gdn.vbmap, + start, size); + if (ret) { + g_free(gdn.vbmap.bitmap); + + return ret; + } + } + iommu_notifier_init(&gdn.n, vfio_iommu_map_dirty_notify, IOMMU_NOTIFIER_MAP, section->offset_within_region, int128_get64(llend), idx); memory_region_iommu_replay(giommu->iommu_mr, &gdn.n); + g_free(gdn.vbmap.bitmap); return 0; } From patchwork Thu Jun 22 21:48:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 1798718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=oracle.com header.i=@oracle.com header.a=rsa-sha256 header.s=corp-2023-03-30 header.b=mDD9g1ZJ; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QnDbH4MKYz20Xt for ; Fri, 23 Jun 2023 07:51:23 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qCSCK-0004Sb-HB; Thu, 22 Jun 2023 17:50:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSCD-0004I3-Pn for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:30 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qCSCC-00082c-27 for qemu-devel@nongnu.org; Thu, 22 Jun 2023 17:50:29 -0400 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35MJYYdd019949; Thu, 22 Jun 2023 21:50:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=mMoMFeioJjSbDxrvMabIUSoJaWlEO/hD5COIK4zVbUw=; b=mDD9g1ZJirlQAq3J1DmzuWgp8rFmzu3vrszB81ClUm29nBoZo7SQAoTHdKpsG0rtPwB1 KQQKDdAcB+sceQxQ+XSFYPz5zBsXR9UKqrT1oZJCa7wagpLrdiQPt0tgLy42/+f1cOQ4 mcKmkRYCNd5g2Zpoq+tJkthtCW0N3Sef5ye3vsPCO/FuGvnQSfhVPiyu/Dp1KqMOmuX6 2ZC1v/Hamlzfaeg5rzYYgt24gG8kH+qgly8xj2EX//VpJn+DQl6liYqewoV/SLVBrS6N GhiJecfUHAi4Bf+rcA8jSu7/FgOjnYEMpPNQeCWlEuy8kOIsotRE8166KwFW1WHaNWdS qQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3r93e1jugv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:23 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 35MKIV3H008333; Thu, 22 Jun 2023 21:50:23 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3r9398eqb9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 22 Jun 2023 21:50:22 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 35MLn77I035791; Thu, 22 Jun 2023 21:50:22 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-180-251.vpn.oracle.com [10.175.180.251]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3r9398ep1g-16; Thu, 22 Jun 2023 21:50:22 +0000 From: Joao Martins To: qemu-devel@nongnu.org Cc: Alex Williamson , Cedric Le Goater , Paolo Bonzini , Peter Xu , David Hildenbrand , Philippe Mathieu-Daude , "Michael S. Tsirkin" , Marcel Apfelbaum , Jason Wang , Richard Henderson , Eduardo Habkost , Avihai Horon , Jason Gunthorpe , Joao Martins Subject: [PATCH v4 15/15] vfio/common: Block migration with vIOMMUs without address width limits Date: Thu, 22 Jun 2023 22:48:45 +0100 Message-Id: <20230622214845.3980-16-joao.m.martins@oracle.com> In-Reply-To: <20230622214845.3980-1-joao.m.martins@oracle.com> References: <20230622214845.3980-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-06-22_16,2023-06-22_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=723 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306220186 X-Proofpoint-GUID: xhYTTd1lRk3AP-TeeRjKa7MSHJZSFceh X-Proofpoint-ORIG-GUID: xhYTTd1lRk3AP-TeeRjKa7MSHJZSFceh Received-SPF: pass client-ip=205.220.165.32; envelope-from=joao.m.martins@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Only block the case when the underlying vIOMMU model does not report any address space limits, in addition to DMA translation being off or no vIOMMU present. The limits are needed such that can define the IOVA limits that arm the device dirty tracker. Additionally, reword the migration blocker error message to clarify that we the configured vIOMMU does not support migration, as opposed to implying that just being there blocks migration. Signed-off-by: Joao Martins --- hw/vfio/common.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 62f91e8e102d..c3cc0dd47044 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -449,15 +449,18 @@ static int vfio_viommu_get_max_iova(hwaddr *max_iova) int vfio_block_giommu_migration(Error **errp) { + hwaddr max; int ret; if (giommu_migration_blocker || - !vfio_viommu_preset()) { + !vfio_viommu_preset() || + (vfio_viommu_preset() && !vfio_viommu_get_max_iova(&max))) { return 0; } error_setg(&giommu_migration_blocker, - "Migration is currently not supported with vIOMMU enabled"); + "Migration with vIOMMU is currently not supported " + "without vIOMMU address space boundaries"); ret = migrate_add_blocker(giommu_migration_blocker, errp); if (ret < 0) { error_free(giommu_migration_blocker);