From patchwork Tue Jul 7 15:49:42 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Graf X-Patchwork-Id: 492364 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 0FBB11402C8 for ; Wed, 8 Jul 2015 01:59:20 +1000 (AEST) Received: from localhost ([::1]:59082 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZCVHC-0005JD-9E for incoming@patchwork.ozlabs.org; Tue, 07 Jul 2015 11:59:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52302) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZCV84-0004t1-26 for qemu-devel@nongnu.org; Tue, 07 Jul 2015 11:49:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZCV7z-0003ov-Dd for qemu-devel@nongnu.org; Tue, 07 Jul 2015 11:49:51 -0400 Received: from cantor2.suse.de ([195.135.220.15]:44473 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZCV7z-0003mc-1D; Tue, 07 Jul 2015 11:49:47 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 65E88ADF1; Tue, 7 Jul 2015 15:49:45 +0000 (UTC) From: Alexander Graf To: qemu-ppc@nongnu.org Date: Tue, 7 Jul 2015 17:49:42 +0200 Message-Id: <1436284182-5063-31-git-send-email-agraf@suse.de> X-Mailer: git-send-email 1.7.12.4 In-Reply-To: <1436284182-5063-1-git-send-email-agraf@suse.de> References: <1436284182-5063-1-git-send-email-agraf@suse.de> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no timestamps) [generic] X-Received-From: 195.135.220.15 Cc: peter.maydell@linaro.org, David Gibson , qemu-devel@nongnu.org, Gavin Shan Subject: [Qemu-devel] [PULL 30/30] sPAPR: Clear stale MSIx table during EEH reset X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Gavin Shan The PCI device MSIx table is cleaned out in hardware after EEH PE reset. However, we still hold the stale MSIx entries in QEMU, which should be cleared accordingly. Otherwise, we will run into another (recursive) EEH error and the PCI devices contained in the PE have to be offlined exceptionally. The patch introduces function spapr_phb_vfio_eeh_pre_reset(), which is called by sPAPR when asserting hot or fundamental reset, to clear stale MSIx table for VFIO PCI devices before EEH PE reset so that MSIx table could be restored properly after EEH PE reset. Signed-off-by: Gavin Shan Signed-off-by: David Gibson Signed-off-by: Alexander Graf --- hw/ppc/spapr_pci_vfio.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/hw/ppc/spapr_pci_vfio.c b/hw/ppc/spapr_pci_vfio.c index 8104074..cca45ed 100644 --- a/hw/ppc/spapr_pci_vfio.c +++ b/hw/ppc/spapr_pci_vfio.c @@ -19,6 +19,7 @@ #include "hw/ppc/spapr.h" #include "hw/pci-host/spapr.h" +#include "hw/pci/msix.h" #include "linux/vfio.h" #include "hw/vfio/vfio.h" @@ -159,6 +160,49 @@ static int spapr_phb_vfio_eeh_get_state(sPAPRPHBState *sphb, int *state) return RTAS_OUT_SUCCESS; } +static void spapr_phb_vfio_eeh_clear_dev_msix(PCIBus *bus, + PCIDevice *pdev, + void *opaque) +{ + /* Check if the device is VFIO PCI device */ + if (!object_dynamic_cast(OBJECT(pdev), "vfio-pci")) { + return; + } + + /* + * The MSIx table will be cleaned out by reset. We need + * disable it so that it can be reenabled properly. Also, + * the cached MSIx table should be cleared as it's not + * reflecting the contents in hardware. + */ + if (msix_enabled(pdev)) { + uint16_t flags; + + flags = pci_host_config_read_common(pdev, + pdev->msix_cap + PCI_MSIX_FLAGS, + pci_config_size(pdev), 2); + flags &= ~PCI_MSIX_FLAGS_ENABLE; + pci_host_config_write_common(pdev, + pdev->msix_cap + PCI_MSIX_FLAGS, + pci_config_size(pdev), flags, 2); + } + + msix_reset(pdev); +} + +static void spapr_phb_vfio_eeh_clear_bus_msix(PCIBus *bus, void *opaque) +{ + pci_for_each_device(bus, pci_bus_num(bus), + spapr_phb_vfio_eeh_clear_dev_msix, NULL); +} + +static void spapr_phb_vfio_eeh_pre_reset(sPAPRPHBState *sphb) +{ + PCIHostState *phb = PCI_HOST_BRIDGE(sphb); + + pci_for_each_bus(phb->bus, spapr_phb_vfio_eeh_clear_bus_msix, NULL); +} + static int spapr_phb_vfio_eeh_reset(sPAPRPHBState *sphb, int option) { sPAPRPHBVFIOState *svphb = SPAPR_PCI_VFIO_HOST_BRIDGE(sphb); @@ -170,9 +214,11 @@ static int spapr_phb_vfio_eeh_reset(sPAPRPHBState *sphb, int option) op.op = VFIO_EEH_PE_RESET_DEACTIVATE; break; case RTAS_SLOT_RESET_HOT: + spapr_phb_vfio_eeh_pre_reset(sphb); op.op = VFIO_EEH_PE_RESET_HOT; break; case RTAS_SLOT_RESET_FUNDAMENTAL: + spapr_phb_vfio_eeh_pre_reset(sphb); op.op = VFIO_EEH_PE_RESET_FUNDAMENTAL; break; default: