From patchwork Mon Nov 11 21:43:00 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 290513 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id 5B0FB2C0085 for ; Tue, 12 Nov 2013 08:43:32 +1100 (EST) Received: from localhost ([::1]:39747 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VfzGb-00015r-GT for incoming@patchwork.ozlabs.org; Mon, 11 Nov 2013 16:43:29 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58703) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VfzGG-00015f-S7 for qemu-devel@nongnu.org; Mon, 11 Nov 2013 16:43:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VfzGA-0003Es-TO for qemu-devel@nongnu.org; Mon, 11 Nov 2013 16:43:08 -0500 Received: from mx1.redhat.com ([209.132.183.28]:57288) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VfzGA-0003Eb-Li for qemu-devel@nongnu.org; Mon, 11 Nov 2013 16:43:02 -0500 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id rABLh1BM024746 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 11 Nov 2013 16:43:01 -0500 Received: from bling.home (ovpn-113-87.phx2.redhat.com [10.3.113.87]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id rABLh1w5007568; Mon, 11 Nov 2013 16:43:01 -0500 To: alex.williamson@redhat.com From: Alex Williamson Date: Mon, 11 Nov 2013 14:43:00 -0700 Message-ID: <20131111214137.4219.53967.stgit@bling.home> User-Agent: StGit/0.16 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: qemu-devel@nongnu.org Subject: [Qemu-devel] [PATCH] vfio-pci: Fix Nvidia MSI ACK through 0x88000 quirk X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When MSI is enabled on Nvidia GeForce cards the driver seems to acknowledge the interrupt by writing a 0xff byte to the MSI capability ID register using the PCI config space mirror at offset 0x88000 from BAR0. Without this, the device will only fire a single interrupt. VFIO handles the PCI capability ID/next registers as virtual w/o write support, so any write through config space is currently dropped. Add a check for this and allow the write through the BAR window. The registers are read-only anyway. Signed-off-by: Alex Williamson --- We might be able to do something with an ioeventfd to make the QEMU component of this asynchronous... tbd. hw/misc/vfio.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index 5a8c305..3f50872 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -1811,6 +1811,34 @@ static void vfio_probe_nvidia_bar5_window_quirk(VFIODevice *vdev, int nr) vdev->host.function); } +static void vfio_nvidia_88000_quirk_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + PCIDevice *pdev = &vdev->pdev; + hwaddr base = quirk->data.address_match & TARGET_PAGE_MASK; + + vfio_generic_quirk_write(opaque, addr, data, size); + + /* + * Nvidia seems to acknowledge MSI interrupts by writing 0xff to the + * MSI capability ID register. Both the ID and next register are + * read-only, so we allow writes covering either of those to real hw. + * NB - only fixed for the 0x88000 MMIO window. + */ + if ((pdev->cap_present & QEMU_PCI_CAP_MSI) && + vfio_range_contained(addr, size, pdev->msi_cap, PCI_MSI_FLAGS)) { + vfio_bar_write(&vdev->bars[quirk->data.bar], addr + base, data, size); + } +} + +static const MemoryRegionOps vfio_nvidia_88000_quirk = { + .read = vfio_generic_quirk_read, + .write = vfio_nvidia_88000_quirk_write, + .endianness = DEVICE_LITTLE_ENDIAN, +}; + /* * Finally, BAR0 itself. We want to redirect any accesses to either * 0x1800 or 0x88000 through the PCI config space access functions. @@ -1837,7 +1865,7 @@ static void vfio_probe_nvidia_bar0_88000_quirk(VFIODevice *vdev, int nr) quirk->data.address_mask = PCIE_CONFIG_SPACE_SIZE - 1; quirk->data.bar = nr; - memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_generic_quirk, + memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_nvidia_88000_quirk, quirk, "vfio-nvidia-bar0-88000-quirk", TARGET_PAGE_ALIGN(quirk->data.address_mask + 1)); memory_region_add_subregion_overlap(&vdev->bars[nr].mem,