From patchwork Mon Jun 2 22:00:21 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 355099 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 1B6D1140092 for ; Tue, 3 Jun 2014 08:40:59 +1000 (EST) Received: from localhost ([::1]:49459 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WraJa-0002sx-UY for incoming@patchwork.ozlabs.org; Mon, 02 Jun 2014 18:02:46 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51099) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WraHO-0007Cd-Mh for qemu-devel@nongnu.org; Mon, 02 Jun 2014 18:00:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WraHH-0000Qg-3J for qemu-devel@nongnu.org; Mon, 02 Jun 2014 18:00:30 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33177) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WraHG-0000Qa-R0 for qemu-devel@nongnu.org; Mon, 02 Jun 2014 18:00:23 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s52M0LGM018558 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 2 Jun 2014 18:00:22 -0400 Received: from bling.home (ovpn-113-35.phx2.redhat.com [10.3.113.35]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s52M0LJr003297; Mon, 2 Jun 2014 18:00:21 -0400 From: Alex Williamson To: qemu-devel@nongnu.org Date: Mon, 02 Jun 2014 16:00:21 -0600 Message-ID: <20140602220021.26111.77277.stgit@bling.home> In-Reply-To: <20140602215946.26111.16417.stgit@bling.home> References: <20140602215946.26111.16417.stgit@bling.home> User-Agent: StGit/0.17-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: alex.williamson@redhat.com Subject: [Qemu-devel] [PULL 1/8] vfio-pci: Quirk RTL8168 NIC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org This device is ridiculous. It has two MMIO BARs, BAR4 and BAR2. BAR4 hosts the MSI-X table, so oviously it would be too easy to access it directly, instead it creates a window register in BAR2 that, among other things, provides access to the MSI-X table. This means MSI-X doesn't work in the guest because the driver actually manages to program the physical table. When interrupt remapping is present, the device MSI will be blocked. The Linux driver doesn't make use of this window, so apparently it's not required to make use of MSI-X. This quirk makes the device work with the Windows driver that does use this window for MSI-X, but I certainly cannot recommend this device for assignment (the Windows 7 driver also constantly pokes PCI config space). Signed-off-by: Alex Williamson --- hw/misc/vfio.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c index 9cf5b84..76715cc 100644 --- a/hw/misc/vfio.c +++ b/hw/misc/vfio.c @@ -1668,6 +1668,149 @@ static void vfio_probe_ati_bar4_window_quirk(VFIODevice *vdev, int nr) vdev->host.function); } +#define PCI_VENDOR_ID_REALTEK 0x10ec + +/* + * RTL8168 devices have a backdoor that can access the MSI-X table. At BAR2 + * offset 0x70 there is a dword data register, offset 0x74 is a dword address + * register. According to the Linux r8169 driver, the MSI-X table is addressed + * when the "type" portion of the address register is set to 0x1. This appears + * to be bits 16:30. Bit 31 is both a write indicator and some sort of + * "address latched" indicator. Bits 12:15 are a mask field, which we can + * ignore because the MSI-X table should always be accessed as a dword (full + * mask). Bits 0:11 is offset within the type. + * + * Example trace: + * + * Read from MSI-X table offset 0 + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x74, 0x1f000, 4) // store read addr + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x74, 4) = 0x8001f000 // latch + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x70, 4) = 0xfee00398 // read data + * + * Write 0xfee00000 to MSI-X table offset 0 + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x70, 0xfee00000, 4) // write data + * vfio: vfio_bar_write(0000:05:00.0:BAR2+0x74, 0x8001f000, 4) // do write + * vfio: vfio_bar_read(0000:05:00.0:BAR2+0x74, 4) = 0x1f000 // complete + */ + +static uint64_t vfio_rtl8168_window_quirk_read(void *opaque, + hwaddr addr, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + + switch (addr) { + case 4: /* address */ + if (quirk->data.flags) { + DPRINTF("%s fake read(%04x:%02x:%02x.%d)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + return quirk->data.address_match ^ 0x10000000U; + } + break; + case 0: /* data */ + if (quirk->data.flags) { + uint64_t val; + + DPRINTF("%s MSI-X table read(%04x:%02x:%02x.%d)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + if (!(vdev->pdev.cap_present & QEMU_PCI_CAP_MSIX)) { + return 0; + } + + io_mem_read(&vdev->pdev.msix_table_mmio, + (hwaddr)(quirk->data.address_match & 0xfff), + &val, size); + return val; + } + } + + DPRINTF("%s direct read(%04x:%02x:%02x.%d)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + return vfio_bar_read(&vdev->bars[quirk->data.bar], addr + 0x70, size); +} + +static void vfio_rtl8168_window_quirk_write(void *opaque, hwaddr addr, + uint64_t data, unsigned size) +{ + VFIOQuirk *quirk = opaque; + VFIODevice *vdev = quirk->vdev; + + switch (addr) { + case 4: /* address */ + if ((data & 0x7fff0000) == 0x10000) { + if (data & 0x10000000U && + vdev->pdev.cap_present & QEMU_PCI_CAP_MSIX) { + + DPRINTF("%s MSI-X table write(%04x:%02x:%02x.%d)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + io_mem_write(&vdev->pdev.msix_table_mmio, + (hwaddr)(quirk->data.address_match & 0xfff), + data, size); + } + + quirk->data.flags = 1; + quirk->data.address_match = data; + + return; + } + quirk->data.flags = 0; + break; + case 0: /* data */ + quirk->data.address_mask = data; + break; + } + + DPRINTF("%s direct write(%04x:%02x:%02x.%d)\n", + memory_region_name(&quirk->mem), vdev->host.domain, + vdev->host.bus, vdev->host.slot, vdev->host.function); + + vfio_bar_write(&vdev->bars[quirk->data.bar], addr + 0x70, data, size); +} + +static const MemoryRegionOps vfio_rtl8168_window_quirk = { + .read = vfio_rtl8168_window_quirk_read, + .write = vfio_rtl8168_window_quirk_write, + .valid = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false, + }, + .endianness = DEVICE_LITTLE_ENDIAN, +}; + +static void vfio_probe_rtl8168_bar2_window_quirk(VFIODevice *vdev, int nr) +{ + PCIDevice *pdev = &vdev->pdev; + VFIOQuirk *quirk; + + if (pci_get_word(pdev->config + PCI_VENDOR_ID) != PCI_VENDOR_ID_REALTEK || + pci_get_word(pdev->config + PCI_DEVICE_ID) != 0x8168 || nr != 2) { + return; + } + + quirk = g_malloc0(sizeof(*quirk)); + quirk->vdev = vdev; + quirk->data.bar = nr; + + memory_region_init_io(&quirk->mem, OBJECT(vdev), &vfio_rtl8168_window_quirk, + quirk, "vfio-rtl8168-window-quirk", 8); + memory_region_add_subregion_overlap(&vdev->bars[nr].mem, + 0x70, &quirk->mem, 1); + + QLIST_INSERT_HEAD(&vdev->bars[nr].quirks, quirk, next); + + DPRINTF("Enabled RTL8168 BAR2 window quirk for device %04x:%02x:%02x.%x\n", + vdev->host.domain, vdev->host.bus, vdev->host.slot, + vdev->host.function); +} /* * Trap the BAR2 MMIO window to config space as well. */ @@ -2071,6 +2214,7 @@ static void vfio_bar_quirk_setup(VFIODevice *vdev, int nr) vfio_probe_nvidia_bar5_window_quirk(vdev, nr); vfio_probe_nvidia_bar0_88000_quirk(vdev, nr); vfio_probe_nvidia_bar0_1800_quirk(vdev, nr); + vfio_probe_rtl8168_bar2_window_quirk(vdev, nr); } static void vfio_bar_quirk_teardown(VFIODevice *vdev, int nr)