From patchwork Wed Mar 22 03:03:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 741848 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vnvh42Swjz9s2s for ; Wed, 22 Mar 2017 14:05:40 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3vnvh41T6XzDqZW for ; Wed, 22 Mar 2017 14:05:40 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3vnvgh45kyzDq7Z for ; Wed, 22 Mar 2017 14:05:20 +1100 (AEDT) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v2M33mol046163 for ; Tue, 21 Mar 2017 23:05:05 -0400 Received: from e23smtp03.au.ibm.com (e23smtp03.au.ibm.com [202.81.31.145]) by mx0a-001b2d01.pphosted.com with ESMTP id 29b9vqh9yv-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 21 Mar 2017 23:05:05 -0400 Received: from localhost by e23smtp03.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 22 Mar 2017 13:05:02 +1000 Received: from d23relay06.au.ibm.com (202.81.31.225) by e23smtp03.au.ibm.com (202.81.31.209) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 22 Mar 2017 13:05:01 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay06.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v2M34r6I50659400 for ; Wed, 22 Mar 2017 14:05:01 +1100 Received: from d23av03.au.ibm.com (localhost [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id v2M34QYD009644 for ; Wed, 22 Mar 2017 14:04:26 +1100 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id v2M34QjV009287; Wed, 22 Mar 2017 14:04:26 +1100 Received: from bran.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) by ozlabs.au.ibm.com (Postfix) with ESMTP id AFC1BA01B6; Wed, 22 Mar 2017 14:04:04 +1100 (AEDT) Received: from gwshan.ozlabs.ibm.com (shangw.ozlabs.ibm.com [10.61.2.199]) by bran.ozlabs.ibm.com (Postfix) with ESMTP id 90C24E3A37; Wed, 22 Mar 2017 14:04:04 +1100 (AEDT) Received: by gwshan.ozlabs.ibm.com (Postfix, from userid 1000) id 77B77AC2732; Wed, 22 Mar 2017 14:04:04 +1100 (AEDT) From: Gavin Shan To: skiboot@lists.ozlabs.org Date: Wed, 22 Mar 2017 14:03:59 +1100 X-Mailer: git-send-email 2.7.4 X-TM-AS-MML: disable x-cbid: 17032203-0008-0000-0000-0000011A0A7F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17032203-0009-0000-0000-00000945217F Message-Id: <1490151841-13124-1-git-send-email-gwshan@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-22_02:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703220025 Subject: [Skiboot] [PATCH 1/3] core/pci: Fix lost NVMe adapter behind PMC 8546 switch X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: markes@us.ibm.com MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The NVMe adapter in below PCI topology is lost. The root cause is the presence bit on its PCI slot is missed, but the PCIe link has been up. The PCI core doesn't probe the adapter behind the slot, leading to lost NVMe adapter in the particular case. PHB3 root port PLX switch 8748 (10b5:8748) PLX swich 9733 (10b5:9733) PMC 8546 swtich (11f8:8546) NVMe adapter (1c58:0023) This fixes the issue by overriding the PCI slot presence bit with PCIe link state bit. Reported-by: Mark E Schreiter Signed-off-by: Gavin Shan --- core/pci.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/core/pci.c b/core/pci.c index ecb94c2..6864e6f 100644 --- a/core/pci.c +++ b/core/pci.c @@ -360,12 +360,19 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) uint16_t bctl; bool was_reset = false; int64_t ecap = 0; + uint32_t lcap = 0; + uint16_t lstat; /* Disable master aborts, clear errors */ pci_cfg_read16(phb, pd->bdfn, PCI_CFG_BRCTL, &bctl); bctl &= ~PCI_CFG_BRCTL_MABORT_REPORT; pci_cfg_write16(phb, pd->bdfn, PCI_CFG_BRCTL, bctl); + if (pci_has_cap(pd, PCI_CFG_CAP_ID_EXP, false)) { + ecap = pci_cap(pd, PCI_CFG_CAP_ID_EXP, false); + pci_cfg_read32(phb, pd->bdfn, ecap+PCICAP_EXP_LCAP, &lcap); + } + /* PCI-E bridge, check the slot state. We don't do that on the * root complex as this is handled separately and not all our * RCs implement the standard register set. @@ -374,7 +381,21 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) pd->dev_type == PCIE_TYPE_SWITCH_DNPORT) { uint16_t slctl, slcap, slsta, lctl; - ecap = pci_cap(pd, PCI_CFG_CAP_ID_EXP, false); + /* + * No need to touch the power supply if the PCIe link has + * been up. Further more, the slot presence bit is lost while + * the PCIe link is up on the specific PCI topology. In that + * case, we need ignore the slot presence bit and go ahead for + * probing. Otherwise, the NVMe adapter won't be probed. + * + * PHB3 root port, PLX switch 8748 (10b5:8748), PLX swich 9733 + * (10b5:9733), PMC 8546 swtich (11f8:8546), NVMe adapter + * (1c58:0023). + */ + pci_cfg_read16(phb, pd->bdfn, ecap+PCICAP_EXP_LSTAT, &lstat); + if ((lcap & PCICAP_EXP_LCAP_DL_ACT_REP) && + (lstat & PCICAP_EXP_LSTAT_DLLL_ACT)) + return true; /* Read the slot status & check for presence detect */ pci_cfg_read16(phb, pd->bdfn, ecap+PCICAP_EXP_SLOTSTAT, &slsta); @@ -430,11 +451,6 @@ static bool pci_enable_bridge(struct phb *phb, struct pci_device *pd) /* PCI-E bridge, wait for link */ if (pd->dev_type == PCIE_TYPE_ROOT_PORT || pd->dev_type == PCIE_TYPE_SWITCH_DNPORT) { - uint32_t lcap; - - /* Read link caps */ - pci_cfg_read32(phb, pd->bdfn, ecap+PCICAP_EXP_LCAP, &lcap); - /* Did link capability say we got reporting ? * * If yes, wait up to 10s, if not, wait 1s if we didn't already