From patchwork Sat May 10 23:03:11 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 347741 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id EEB17140094 for ; Sun, 11 May 2014 09:03:48 +1000 (EST) Received: from localhost ([::1]:59138 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WjGJ0-0006QP-Rr for incoming@patchwork.ozlabs.org; Sat, 10 May 2014 19:03:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47605) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WjGIa-00060Z-Rs for qemu-devel@nongnu.org; Sat, 10 May 2014 19:03:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WjGIU-0001Pu-Ji for qemu-devel@nongnu.org; Sat, 10 May 2014 19:03:20 -0400 Received: from mx1.redhat.com ([209.132.183.28]:62579) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WjGIU-0001Pp-Bo for qemu-devel@nongnu.org; Sat, 10 May 2014 19:03:14 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s4AN3CHo004277 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 10 May 2014 19:03:12 -0400 Received: from bling.home (ovpn-113-46.phx2.redhat.com [10.3.113.46]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s4AN3BSH006784; Sat, 10 May 2014 19:03:11 -0400 From: Alex Williamson To: alex.williamson@redhat.com Date: Sat, 10 May 2014 17:03:11 -0600 Message-ID: <20140510230225.17688.6353.stgit@bling.home> User-Agent: StGit/0.17-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org Subject: [Qemu-devel] [PATCH] vfio-pci: Quirk RTL8168 NIC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This device is ridiculous. It has two MMIO BARs, BAR4 and BAR2. BAR4 hosts the MSI-X table, so oviously it would be too easy to access it directly, instead it creates a window register in BAR2 that, among other things, provides access to the MSI-X table. This means MSI-X doesn't work in the guest because the driver actually manages to program the physical table. When interrupt remapping is present, the device MSI will be blocked. The Linux driver doesn't make use of this window, so apparently it's not required to make use of MSI-X. This quirk makes the device work with the Windows driver that does use this window for MSI-X, but I certainly cannot recommend this device for assignment (the Windows 7 driver also constantly pokes PCI config space). Signed-off-by: Alex Williamson --- hw/misc/vfio.c | 145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index 9cf5b84..c3d2f7a 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -1668,6 +1668,150 @@ static void vfio_probe_ati_bar4_window_quirk(VFIODevice *vdev, int nr) vdev->host.function); } +#define PCI_VENDOR_ID_REALTEK 0x10ec + +/* + * RTL8168 devices have a backdoor that can access the MSI-X table. At BAR2 + * offset 0x70 there is a dword data register, offset 0x74 is a dword address + * register. According to the Linux r8169 driver, the MSI-X table is addressed + * when the "type" portion of the address register is set to 0x1. This appears + * to be bits 16:30. Bit 31 is both a write indicator and some sort of + * "address latched" indicator. Bits 12:15 is a mask field, which we're + * going to ignore because we don't really know what it means and the MSI-X + * area always seems to be accessed with a full mask. Bits 0:11 is offset + * within the type. + * + * Example trace: + * + * Read from MSI-X table offset 0 + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x74, 0x1f000, 4) // store read addr + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x74, 4) = 0x8001f000 // latch + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x70, 4) = 0xfee00398 // read data + * + * Write 0xfee00000 to MSI-X table offset 0 + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x70, 0xfee00000, 4) // write data + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x74, 0x8001f000, 4) // do write + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x74, 4) = 0x1f000 // complete + */ + +static uint64_t vfio_rtl8168_window_quirk_read(void *opaque, + hwaddr addr, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + + switch (addr) { + case 4: /* address */ + if (quirk->data.flags) { + DPRINTF("%s fake read(%04x:%02x:%02x.%xd)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + return quirk->data.address_match ^ 0x10000000U; + } + break; + case 0: /* data */ + if (quirk->data.flags) { + uint64_t val; + + DPRINTF("%s MSI-X table read(%04x:%02x:%02x.%xd)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + if (!(vdev->pdev.cap_present & QEMU_PCI_CAP_MSIX)) { + return 0; + } + + io_mem_read(&vdev->pdev.msix_table_mmio, + (hwaddr)(quirk->data.address_match & 0xfff), + &val, size); + return val; + } + } + + DPRINTF("%s direct read(%04x:%02x:%02x.%xd)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + return vfio_bar_read(&vdev->bars[quirk->data.bar], addr + 0x70, size); +} + +static void vfio_rtl8168_window_quirk_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + + switch (addr) { + case 4: /* address */ + if ((data & 0x7fff0000) == 0x10000) { + if (data & 0x10000000U && + vdev->pdev.cap_present & QEMU_PCI_CAP_MSIX) { + + DPRINTF("%s MSI-X table write(%04x:%02x:%02x.%xd)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + io_mem_write(&vdev->pdev.msix_table_mmio, + (hwaddr)(quirk->data.address_match & 0xfff), + data, size); + } + + quirk->data.flags = 1; + quirk->data.address_match = data; + + return; + } + quirk->data.flags = 0; + break; + case 0: /* data */ + quirk->data.address_mask = data; + break; + } + + DPRINTF("%s direct write(%04x:%02x:%02x.%xd)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + vfio_bar_write(&vdev->bars[quirk->data.bar], addr + 0x70, data, size); +} + +static const MemoryRegionOps vfio_rtl8168_window_quirk = { + .read = vfio_rtl8168_window_quirk_read, + .write = vfio_rtl8168_window_quirk_write, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false, + }, + .endianness = DEVICE_LITTLE_ENDIAN, +}; + +static void vfio_probe_rtl8168_bar2_window_quirk(VFIODevice *vdev, int nr) +{ + PCIDevice *pdev = &vdev->pdev; + VFIOQuirk *quirk; + + if (pci_get_word(pdev->config + PCI_VENDOR_ID) != PCI_VENDOR_ID_REALTEK || + pci_get_word(pdev->config + PCI_DEVICE_ID) != 0x8168 || nr != 2) { + return; + } + + quirk = g_malloc0(sizeof(*quirk)); + quirk->vdev = vdev; + quirk->data.bar = nr; + + memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_rtl8168_window_quirk, + quirk, "vfio-rtl8168-window-quirk", 8); + memory_region_add_subregion_overlap(&vdev->bars[nr].mem, + 0x70, &quirk->mem, 1); + + QLIST_INSERT_HEAD(&vdev->bars[nr].quirks, quirk, next); + + DPRINTF("Enabled RTL8168 BAR2 window quirk for device %04x:%02x:%02x.%x\n", + vdev->host.domain, vdev->host.bus, vdev->host.slot, + vdev->host.function); +} /* * Trap the BAR2 MMIO window to config space as well. */ @@ -2071,6 +2215,7 @@ static void vfio_bar_quirk_setup(VFIODevice *vdev, int nr) vfio_probe_nvidia_bar5_window_quirk(vdev, nr); vfio_probe_nvidia_bar0_88000_quirk(vdev, nr); vfio_probe_nvidia_bar0_1800_quirk(vdev, nr); + vfio_probe_rtl8168_bar2_window_quirk(vdev, nr); } static void vfio_bar_quirk_teardown(VFIODevice *vdev, int nr)