From patchwork Thu Feb 26 16:22:05 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 443983 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4270814008F for ; Fri, 27 Feb 2015 03:23:44 +1100 (AEDT) Received: from localhost ([::1]:59750 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR1Dy-0003iX-63 for incoming@patchwork.ozlabs.org; Thu, 26 Feb 2015 11:23:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39258) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR1CX-0000S8-UH for qemu-devel@nongnu.org; Thu, 26 Feb 2015 11:22:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YR1CT-0007N7-KC for qemu-devel@nongnu.org; Thu, 26 Feb 2015 11:22:13 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37122) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YR1CT-0007Mo-5Q for qemu-devel@nongnu.org; Thu, 26 Feb 2015 11:22:09 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1QGM62Y018316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Thu, 26 Feb 2015 11:22:07 -0500 Received: from gimli.home (ovpn-113-210.phx2.redhat.com [10.3.113.210]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1QGM5Kj017177; Thu, 26 Feb 2015 11:22:06 -0500 From: Alex Williamson To: qemu-devel@nongnu.org Date: Thu, 26 Feb 2015 09:22:05 -0700 Message-ID: <20150226161613.24231.139.stgit@gimli.home> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: alex.williamson@redhat.com Subject: [Qemu-devel] [PATCH] vfio-pci: Enable device request notification support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Linux v4.0-rc1 vfio-pci introduced a new virtual interrupt to allow the kernel to request a device from the user. When signaled, QEMU will by default attmempt to hot-unplug the device. This is a one- shot attempt with the expectation that the kernel will continue to poll for the device if it is not returned. Returning the device when requested is the expected standard model of cooperative usage, but we also add an option option to disable this feature. Initially this opt-out is set as an experimental option because we really should honor kernel requests for the device. Signed-off-by: Alex Williamson --- This was posted as an RFC on Feb 4th, now that 4.0-rc1 is out and includes the kernel support for this, I'm reposting as I intend to include it in a pull request prior to the QEMU 2.3 hard freeze. I'm also including only a partial linux-headers update given the timing relative to hard freeze and the number of kvm and virtio changes in 4.0-rc1 that I cannot vouch for. If there are any objections, please let me know. Thanks. hw/vfio/pci.c | 100 ++++++++++++++++++++++++++++++++++++++++++++ linux-headers/linux/vfio.h | 1 2 files changed, 101 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 3c71de3..6b80539 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -153,13 +153,17 @@ typedef struct VFIOPCIDevice { VFIOVGA vga; /* 0xa0000, 0x3b0, 0x3c0 */ PCIHostDeviceAddress host; EventNotifier err_notifier; + EventNotifier req_notifier; uint32_t features; #define VFIO_FEATURE_ENABLE_VGA_BIT 0 #define VFIO_FEATURE_ENABLE_VGA (1 << VFIO_FEATURE_ENABLE_VGA_BIT) +#define VFIO_FEATURE_ENABLE_REQ_BIT 1 +#define VFIO_FEATURE_ENABLE_REQ (1 << VFIO_FEATURE_ENABLE_REQ_BIT) int32_t bootindex; uint8_t pm_cap; bool has_vga; bool pci_aer; + bool req_enabled; bool has_flr; bool has_pm_reset; bool rom_read_failed; @@ -3088,6 +3092,7 @@ static int vfio_populate_device(VFIOPCIDevice *vdev) vdev->has_vga = true; } + irq_info.index = VFIO_PCI_ERR_IRQ_INDEX; ret = ioctl(vdev->vbasedev.fd, VFIO_DEVICE_GET_IRQ_INFO, &irq_info); @@ -3223,6 +3228,97 @@ static void vfio_unregister_err_notifier(VFIOPCIDevice *vdev) event_notifier_cleanup(&vdev->err_notifier); } +static void vfio_req_notifier_handler(void *opaque) +{ + VFIOPCIDevice *vdev = opaque; + + if (!event_notifier_test_and_clear(&vdev->req_notifier)) { + return; + } + + qdev_unplug(&vdev->pdev.qdev, NULL); +} + +static void vfio_register_req_notifier(VFIOPCIDevice *vdev) +{ + struct vfio_irq_info irq_info = { .argsz = sizeof(irq_info), + .index = VFIO_PCI_REQ_IRQ_INDEX }; + int argsz; + struct vfio_irq_set *irq_set; + int32_t *pfd; + + if (!(vdev->features & VFIO_FEATURE_ENABLE_REQ)) { + return; + } + + if (ioctl(vdev->vbasedev.fd, + VFIO_DEVICE_GET_IRQ_INFO, &irq_info) < 0 || irq_info.count < 1) { + return; + } + + if (event_notifier_init(&vdev->req_notifier, 0)) { + error_report("vfio: Unable to init event notifier for device request"); + return; + } + + argsz = sizeof(*irq_set) + sizeof(*pfd); + + irq_set = g_malloc0(argsz); + irq_set->argsz = argsz; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | + VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_REQ_IRQ_INDEX; + irq_set->start = 0; + irq_set->count = 1; + pfd = (int32_t *)&irq_set->data; + + *pfd = event_notifier_get_fd(&vdev->req_notifier); + qemu_set_fd_handler(*pfd, vfio_req_notifier_handler, NULL, vdev); + + if (ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set)) { + error_report("vfio: Failed to set up device request notification"); + qemu_set_fd_handler(*pfd, NULL, NULL, vdev); + event_notifier_cleanup(&vdev->req_notifier); + } else { + vdev->req_enabled = true; + } + + g_free(irq_set); +} + +static void vfio_unregister_req_notifier(VFIOPCIDevice *vdev) +{ + int argsz; + struct vfio_irq_set *irq_set; + int32_t *pfd; + + if (!vdev->req_enabled) { + return; + } + + argsz = sizeof(*irq_set) + sizeof(*pfd); + + irq_set = g_malloc0(argsz); + irq_set->argsz = argsz; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | + VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_REQ_IRQ_INDEX; + irq_set->start = 0; + irq_set->count = 1; + pfd = (int32_t *)&irq_set->data; + *pfd = -1; + + if (ioctl(vdev->vbasedev.fd, VFIO_DEVICE_SET_IRQS, irq_set)) { + error_report("vfio: Failed to de-assign device request fd: %m"); + } + g_free(irq_set); + qemu_set_fd_handler(event_notifier_get_fd(&vdev->req_notifier), + NULL, NULL, vdev); + event_notifier_cleanup(&vdev->req_notifier); + + vdev->req_enabled = false; +} + static int vfio_initfn(PCIDevice *pdev) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); @@ -3370,6 +3466,7 @@ static int vfio_initfn(PCIDevice *pdev) } vfio_register_err_notifier(vdev); + vfio_register_req_notifier(vdev); return 0; @@ -3397,6 +3494,7 @@ static void vfio_exitfn(PCIDevice *pdev) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); + vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); vfio_disable_interrupts(vdev); @@ -3455,6 +3553,8 @@ static Property vfio_pci_dev_properties[] = { intx.mmap_timeout, 1100), DEFINE_PROP_BIT("x-vga", VFIOPCIDevice, features, VFIO_FEATURE_ENABLE_VGA_BIT, false), + DEFINE_PROP_BIT("x-req", VFIOPCIDevice, features, + VFIO_FEATURE_ENABLE_REQ_BIT, true), DEFINE_PROP_INT32("bootindex", VFIOPCIDevice, bootindex, -1), DEFINE_PROP_BOOL("x-mmap", VFIOPCIDevice, vbasedev.allow_mmap, true), /* diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 0f21aa6..95ba870 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -333,6 +333,7 @@ enum { VFIO_PCI_MSI_IRQ_INDEX, VFIO_PCI_MSIX_IRQ_INDEX, VFIO_PCI_ERR_IRQ_INDEX, + VFIO_PCI_REQ_IRQ_INDEX, VFIO_PCI_NUM_IRQS };