From patchwork Fri Dec 11 09:05:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongji Xie X-Patchwork-Id: 555757 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 827221401AF for ; Sat, 12 Dec 2015 00:24:00 +1100 (AEDT) Received: from localhost ([::1]:47592 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a7Nfy-0002R3-Cb for incoming@patchwork.ozlabs.org; Fri, 11 Dec 2015 08:23:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40429) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a7Jes-0002DP-D6 for qemu-devel@nongnu.org; Fri, 11 Dec 2015 04:06:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1a7Jep-0004jz-6X for qemu-devel@nongnu.org; Fri, 11 Dec 2015 04:06:34 -0500 Received: from e23smtp07.au.ibm.com ([202.81.31.140]:53537) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1a7Jeo-0004jg-LG for qemu-devel@nongnu.org; Fri, 11 Dec 2015 04:06:31 -0500 Received: from localhost by e23smtp07.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 11 Dec 2015 19:06:26 +1000 Received: from d23dlp03.au.ibm.com (202.81.31.214) by e23smtp07.au.ibm.com (202.81.31.204) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Fri, 11 Dec 2015 19:06:24 +1000 X-IBM-Helo: d23dlp03.au.ibm.com X-IBM-MailFrom: xyjxie@linux.vnet.ibm.com X-IBM-RcptTo: qemu-devel@nongnu.org Received: from d23relay10.au.ibm.com (d23relay10.au.ibm.com [9.190.26.77]) by d23dlp03.au.ibm.com (Postfix) with ESMTP id DCA303578052 for ; Fri, 11 Dec 2015 20:06:23 +1100 (EST) Received: from d23av04.au.ibm.com (d23av04.au.ibm.com [9.190.235.139]) by d23relay10.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id tBB96FZv34734196 for ; Fri, 11 Dec 2015 20:06:23 +1100 Received: from d23av04.au.ibm.com (localhost [127.0.0.1]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id tBB95oZu015443 for ; Fri, 11 Dec 2015 20:05:51 +1100 Received: from localhost (commit.cn.ibm.com [9.123.229.41]) by d23av04.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id tBB95nGj014701; Fri, 11 Dec 2015 20:05:50 +1100 From: Yongji Xie To: qemu-devel@nongnu.org, alex.williamson@redhat.com Date: Fri, 11 Dec 2015 17:05:17 +0800 Message-Id: <1449824719-3407-3-git-send-email-xyjxie@linux.vnet.ibm.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1449824719-3407-1-git-send-email-xyjxie@linux.vnet.ibm.com> References: <1449824719-3407-1-git-send-email-xyjxie@linux.vnet.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15121109-0025-0000-0000-0000029197E2 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 202.81.31.140 X-Mailman-Approved-At: Fri, 11 Dec 2015 08:21:11 -0500 Cc: nikunj@linux.vnet.ibm.com, zhong@linux.vnet.ibm.com, aik@ozlabs.ru, paulus@samba.org, mpe@ellerman.id.au, warrier@linux.vnet.ibm.com, Yongji Xie Subject: [Qemu-devel] [RFC PATCH 2/3] vfio/pci: Add support for mmapped sub-page MMIO BARs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The VFIO-PCI ioctl flag VFIO_DEVICE_FLAGS_PCI_PAGE_ALIGNED indicates platform support all PCI MMIO BARs to be page aligned and sub-page(size < PAGE_SIZE) MMIO BARs can be mmapped. But this has an issue for these mmapped sub-page MMIO BARs - KVM would not allow to create memory slot for them via ioctl KVM_SET_USER_MEMORY_REGION because these BARs' size are still less than PAGE_SIZE. So this patch expands these sub-page MMIO BARs to page size in vfio_pci_write_config(), so that the BARs could be passed to KVM ioctl KVM_SET_USER_MEMORY_REGION with a valid size. And we also set the priority of these BARs' memory regions to zero in case of overlap with page unaligned BARs when guest doesn't support all PCI MMIO BARs to be page aligned. Signed-off-by: Yongji Xie --- hw/vfio/pci.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 1fb868c..a7c6171 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -1028,6 +1028,44 @@ static const MemoryRegionOps vfio_vga_ops = { }; /* + * Expand sub-page(size < PAGE_SIZE) MMIO BARs to page size if host support + * all MMIO BARs to be page aligned. And we should set the priority of these + * BARs' memory regions to zero in case of overlap with page unaligned + * BARs when guest doesn't support all MMIO BARs to be page aligned. + */ +static void vfio_expand_bar_to_page_size(PCIDevice *pdev) +{ + VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); + MemoryRegion *mmap_mr; + MemoryRegion *mr; + PCIIORegion *r; + pcibus_t bar_addr; + int i; + + memory_region_transaction_begin(); + for (i = 0; i < PCI_NUM_REGIONS; i++) { + r = &pdev->io_regions[i]; + if (r->size > 0 && r->size < qemu_real_host_page_size) { + bar_addr = r->addr; + if (bar_addr != PCI_BAR_UNMAPPED && + !(bar_addr & ~qemu_real_host_page_mask)) { + mr = &vdev->bars[i].region.mem; + mmap_mr = &vdev->bars[i].region.mmap_mem; + if (memory_region_is_mapped(mr) && vdev->bars[i].region.mmap && + memory_region_size(mr) < qemu_real_host_page_size) { + memory_region_del_subregion(r->address_space, mr); + memory_region_set_size(mr, qemu_real_host_page_size); + memory_region_set_size(mmap_mr, qemu_real_host_page_size); + memory_region_add_subregion_overlap(r->address_space, + bar_addr, mr, 0); + } + } + } + } + memory_region_transaction_commit(); +} + +/* * PCI config space */ uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len) @@ -1112,6 +1150,14 @@ void vfio_pci_write_config(PCIDevice *pdev, } else if (was_enabled && !is_enabled) { vfio_msix_disable(vdev); } + } else if (vdev->vbasedev.flags & VFIO_DEVICE_FLAGS_PCI_PAGE_ALIGNED && + (ranges_overlap(addr, len, PCI_BASE_ADDRESS_0, 24) || + ranges_overlap(addr, l, PCI_ROM_ADDRESS, 4) || + ranges_overlap(addr, l, PCI_ROM_ADDRESS1, 4) || + range_covers_byte(addr, len, PCI_COMMAND))) { + pci_default_write_config(pdev, addr, val, len); + + vfio_expand_bar_to_page_size(pdev); } else { /* Write everything to QEMU to keep emulated bits correct */ pci_default_write_config(pdev, addr, val, len);