From patchwork Tue Feb 10 18:37:28 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 438508 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id CC7B1140192 for ; Wed, 11 Feb 2015 05:38:14 +1100 (AEDT) Received: from localhost ([::1]:41304 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFhN-0001yN-2S for incoming@patchwork.ozlabs.org; Tue, 10 Feb 2015 13:38:13 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39782) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFgj-0000uF-Kp for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YLFgf-0006cI-Ks for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:33 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52916) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YLFgf-0006cE-EN for qemu-devel@nongnu.org; Tue, 10 Feb 2015 13:37:29 -0500 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1AIbSm8020186 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL) for ; Tue, 10 Feb 2015 13:37:28 -0500 Received: from gimli.home (ovpn-113-68.phx2.redhat.com [10.3.113.68]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1AIbS2m022885; Tue, 10 Feb 2015 13:37:28 -0500 From: Alex Williamson To: qemu-devel@nongnu.org Date: Tue, 10 Feb 2015 11:37:28 -0700 Message-ID: <20150210183728.26101.79729.stgit@gimli.home> In-Reply-To: <20150210183106.26101.78658.stgit@gimli.home> References: <20150210183106.26101.78658.stgit@gimli.home> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 209.132.183.28 Cc: alex.williamson@redhat.com Subject: [Qemu-devel] [PULL 3/6] vfio: free dynamically-allocated data in instance_finalize X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Paolo Bonzini In order to enable out-of-BQL address space lookup, destruction of devices needs to be split in two phases. Unrealize is the first phase; once it complete no new accesses will be started, but there may still be pending memory accesses can still be completed. The second part is freeing the device, which only happens once all memory accesses are complete. At this point the reference count has dropped to zero, an RCU grace period must have completed (because the RCU-protected FlatViews hold a reference to the device via memory_region_ref). This is when instance_finalize is called. Freeing data belongs in an instance_finalize callback, because the dynamically allocated memory can still be used after unrealize by the pending memory accesses. This starts the process by creating an instance_finalize callback and freeing most of the dynamically-allocated data in instance_finalize. Because instance_finalize is also called on error paths or also when the device is actually not realized, the common code needs some changes to be ready for this. The error path in vfio_initfn can be simplified too. Signed-off-by: Paolo Bonzini Signed-off-by: Alex Williamson --- hw/vfio/common.c | 5 ++++- hw/vfio/pci.c | 24 ++++++++++++++---------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 6e299b3..b230ef1 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -847,7 +847,7 @@ free_group_exit: void vfio_put_group(VFIOGroup *group) { - if (!QLIST_EMPTY(&group->device_list)) { + if (!group || !QLIST_EMPTY(&group->device_list)) { return; } @@ -902,6 +902,9 @@ int vfio_get_device(VFIOGroup *group, const char *name, void vfio_put_base_device(VFIODevice *vbasedev) { + if (!vbasedev->group) { + return; + } QLIST_REMOVE(vbasedev, next); vbasedev->group = NULL; trace_vfio_put_base_device(vbasedev->fd); diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 23fe9fa..0271c80 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3248,7 +3248,7 @@ static int vfio_initfn(PCIDevice *pdev) ret = vfio_populate_device(vdev); if (ret) { - goto out_put; + return ret; } /* Get a copy of config space */ @@ -3258,7 +3258,7 @@ static int vfio_initfn(PCIDevice *pdev) if (ret < (int)MIN(pci_config_size(&vdev->pdev), vdev->config_size)) { ret = ret < 0 ? -errno : -EFAULT; error_report("vfio: Failed to read device config space"); - goto out_put; + return ret; } /* vfio emulates a lot for us, but some bits need extra love */ @@ -3290,7 +3290,7 @@ static int vfio_initfn(PCIDevice *pdev) ret = vfio_early_setup_msix(vdev); if (ret) { - goto out_put; + return ret; } vfio_map_bars(vdev); @@ -3329,17 +3329,24 @@ out_teardown: pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); vfio_teardown_msi(vdev); vfio_unmap_bars(vdev); -out_put: + return ret; +} + +static void vfio_instance_finalize(Object *obj) +{ + PCIDevice *pci_dev = PCI_DEVICE(obj); + VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pci_dev); + VFIOGroup *group = vdev->vbasedev.group; + g_free(vdev->emulated_config_bits); + g_free(vdev->rom); vfio_put_device(vdev); vfio_put_group(group); - return ret; } static void vfio_exitfn(PCIDevice *pdev) { VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); - VFIOGroup *group = vdev->vbasedev.group; vfio_unregister_err_notifier(vdev); pci_device_set_intx_routing_notifier(&vdev->pdev, NULL); @@ -3349,10 +3356,6 @@ static void vfio_exitfn(PCIDevice *pdev) } vfio_teardown_msi(vdev); vfio_unmap_bars(vdev); - g_free(vdev->emulated_config_bits); - g_free(vdev->rom); - vfio_put_device(vdev); - vfio_put_group(group); } static void vfio_pci_reset(DeviceState *dev) @@ -3440,6 +3443,7 @@ static const TypeInfo vfio_pci_dev_info = { .instance_size = sizeof(VFIOPCIDevice), .class_init = vfio_pci_dev_class_init, .instance_init = vfio_instance_init, + .instance_finalize = vfio_instance_finalize, }; static void register_vfio_pci_dev_type(void)