From patchwork Wed Jun 19 12:44:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118682 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPp525g7z9s7h for ; Wed, 19 Jun 2019 22:46:21 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPp30H3jzDqp5 for ; Wed, 19 Jun 2019 22:46:19 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmy34LZzDqk9 for ; Wed, 19 Jun 2019 22:45:22 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYWAU039159 for ; Wed, 19 Jun 2019 08:45:16 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7m3hvaht-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:16 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:14 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:12 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjBkN50659474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:11 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4643711C05B; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 092FD11C04A; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:10 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:44:59 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-4275-0000-0000-00000343B3FC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-4276-0000-0000-00003853DF36 Message-Id: <20190619124510.25182-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 01/12] npu2-hw-procedures: Move some opencapi PHY settings in one-off init X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The PHY_RX_AC_COUPLED and PHY_RX_SPEED_SELECT for opencapi are group settings for the obus. They should be set in the one-off PHY init function at boot and not on the link reset path, as they theoretically impact more than one link. Since we cannot mix link type and/or speed on an optical bus, it has no pratical impact, it just looks cleaner. Also use the OCAPIINF macro for the associated traces. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index 0779ddea..548dae16 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -426,25 +426,6 @@ static uint32_t phy_reset_complete(struct npu2_dev *ndev) { int lane; - if (ndev->type == NPU2_DEV_TYPE_OPENCAPI) { - phy_write(ndev, &NPU2_PHY_RX_AC_COUPLED, 1); - - switch (ndev->link_speed) { - case 20000000000UL: - prlog(PR_INFO, "OCAPI: Link speed set at 20Gb/s\n"); - phy_write(ndev, &NPU2_PHY_RX_SPEED_SELECT, 1); - break; - case 25000000000UL: - case 25781250000UL: - prlog(PR_INFO, "OCAPI: Link speed set at 25.xGb/s\n"); - phy_write(ndev, &NPU2_PHY_RX_SPEED_SELECT, 0); - break; - default: - prlog(PR_CRIT, "OCAPI: Invalid link speed!\n"); - assert(false); - } - } - FOR_EACH_LANE(ndev, lane) { phy_write_lane(ndev, &NPU2_PHY_RX_LANE_ANA_PDWN, lane, 0); phy_write_lane(ndev, &NPU2_PHY_RX_LANE_DIG_PDWN, lane, 0); @@ -1034,6 +1015,22 @@ void npu2_opencapi_phy_init(struct npu2_dev *dev) * Witherspoon it needs to be done in skiboot after device detection. */ phy_write(dev, &NPU2_PHY_RX_RC_ENABLE_AUTO_RECAL, 0x1); + phy_write(dev, &NPU2_PHY_RX_AC_COUPLED, 1); + + switch (dev->link_speed) { + case 20000000000UL: + OCAPIINF(dev, "Link speed set at 20Gb/s\n"); + phy_write(dev, &NPU2_PHY_RX_SPEED_SELECT, 1); + break; + case 25000000000UL: + case 25781250000UL: + OCAPIINF(dev, "Link speed set at 25.xGb/s\n"); + phy_write(dev, &NPU2_PHY_RX_SPEED_SELECT, 0); + break; + default: + OCAPIERR(dev, "Invalid link speed!\n"); + assert(false); + } } void npu2_opencapi_phy_reset(struct npu2_dev *dev) From patchwork Wed Jun 19 12:45:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118684 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPpm3dbVz9sNC for ; Wed, 19 Jun 2019 22:46:56 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPpl44NYzDqmK for ; Wed, 19 Jun 2019 22:46:55 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmy302WzDqk7 for ; Wed, 19 Jun 2019 22:45:21 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYUTh039064 for ; Wed, 19 Jun 2019 08:45:17 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7m3hvajk-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:17 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:15 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:13 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjBDJ55771280 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:11 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AD2AB11C058; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5D1FA11C04C; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:00 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0016-0000-0000-0000028A752D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0017-0000-0000-000032E7CBFB Message-Id: <20190619124510.25182-3-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=824 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 02/12] npu2-opencapi: Make sure the PCI slot has the proper ID X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The PCI slot created for the opencapi PHB didn't have its ID properly defined because it was created before we assign an ID to the PHB. Simply switch the PCI slot creation and PHB registration calls to fix it. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 4c47150f..ada41ddb 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1717,6 +1717,8 @@ static void setup_device(struct npu2_dev *dev) set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, dev->brick_index, 0b00); + pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); + if (npu2_ocapi_training_state != NPU2_TRAIN_DEFAULT) { setup_debug_training_state(dev); } else { @@ -1730,7 +1732,6 @@ static void setup_device(struct npu2_dev *dev) prlog(PR_ERR, "OCAPI: Cannot create PHB slot\n"); } } - pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); return; } From patchwork Wed Jun 19 12:45:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118698 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPtX5byJz9s6w for ; Wed, 19 Jun 2019 22:50:12 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPtX1tm9zDqpg for ; Wed, 19 Jun 2019 22:50:12 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPp12XzgzDqkf for ; Wed, 19 Jun 2019 22:46:17 +1000 (AEST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCY7du029312 for ; Wed, 19 Jun 2019 08:46:14 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7m46ma22-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:46:13 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:16 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:13 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjC5M21430292 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:12 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 02C8611C050; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA2B611C05E; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:11 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:01 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-4275-0000-0000-00000343B3FD X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-4276-0000-0000-00003853DF37 Message-Id: <20190619124510.25182-4-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=882 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 03/12] npu2-hw-procedures: Fix link retraining on reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Link retraining was showing reliability problems due to some opencapi-only settings not being optimized. This patch updates some extra PHY state, as agreed with the PHY team. Though they mostly impact link retraining behavior, they should also be set at boot. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index 548dae16..98e93191 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -68,8 +68,14 @@ static struct npu2_phy_reg NPU2_PHY_RX_PR_FW_OFF = {0x08a, 56, 1}; static struct npu2_phy_reg NPU2_PHY_RX_PR_FW_INERTIA_AMT = {0x08a, 57, 3}; static struct npu2_phy_reg NPU2_PHY_RX_CFG_LTE_MC = {0x000, 60, 4}; static struct npu2_phy_reg NPU2_PHY_RX_A_INTEG_COARSE_GAIN = {0x00a, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_A_CTLE_COARSE = {0x00c, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_A_CTLE_GAIN = {0x00c, 53, 4}; static struct npu2_phy_reg NPU2_PHY_RX_B_INTEG_COARSE_GAIN = {0x026, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_B_CTLE_COARSE = {0x028, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_B_CTLE_GAIN = {0x028, 53, 4}; static struct npu2_phy_reg NPU2_PHY_RX_E_INTEG_COARSE_GAIN = {0x030, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_E_CTLE_COARSE = {0x032, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_E_CTLE_GAIN = {0x032, 53, 4}; /* These registers are per-PHY, not per lane */ static struct npu2_phy_reg NPU2_PHY_RX_SPEED_SELECT = {0x262, 51, 2}; @@ -437,6 +443,16 @@ static uint32_t phy_reset_complete(struct npu2_dev *ndev) phy_write_lane(ndev, &NPU2_PHY_RX_A_INTEG_COARSE_GAIN, lane, 11); phy_write_lane(ndev, &NPU2_PHY_RX_B_INTEG_COARSE_GAIN, lane, 11); phy_write_lane(ndev, &NPU2_PHY_RX_E_INTEG_COARSE_GAIN, lane, 11); + + if (ndev->type == NPU2_DEV_TYPE_OPENCAPI) { + phy_write_lane(ndev, &NPU2_PHY_RX_A_CTLE_GAIN, lane, 0); + phy_write_lane(ndev, &NPU2_PHY_RX_B_CTLE_GAIN, lane, 0); + phy_write_lane(ndev, &NPU2_PHY_RX_E_CTLE_GAIN, lane, 0); + + phy_write_lane(ndev, &NPU2_PHY_RX_A_CTLE_COARSE, lane, 20); + phy_write_lane(ndev, &NPU2_PHY_RX_B_CTLE_COARSE, lane, 20); + phy_write_lane(ndev, &NPU2_PHY_RX_E_CTLE_COARSE, lane, 20); + } } set_iovalid(ndev, true); From patchwork Wed Jun 19 12:45:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118678 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPnD05dPz9s7h for ; Wed, 19 Jun 2019 22:45:36 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPnB51VQzDql8 for ; Wed, 19 Jun 2019 22:45:34 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmy32FWzDqk8 for ; Wed, 19 Jun 2019 22:45:21 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYf9X039885 for ; Wed, 19 Jun 2019 08:45:19 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7m3hvam9-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:18 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:17 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:14 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjCLV51970090 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:12 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5632F11C05B; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1984011C06F; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:02 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0008-0000-0000-000002F51F63 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0009-0000-0000-000022623B08 Message-Id: <20190619124510.25182-5-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=995 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 04/12] npu2-opencapi: Rework link training timeout X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Opencapi link state should be polled for up to 3 seconds. Current code assumes a tight retry loop during fundamental reset at boot, which is not going to be true on link retraining. So update the timeout detection code to use a timebase instead of a simple retry count which could be way too long. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 9 +++++---- include/npu2.h | 2 ++ 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index ada41ddb..5a94c949 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1140,13 +1140,13 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) reg = get_odl_status(chip_id, dev->brick_index); if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == OCAPI_LINK_STATE_TRAINED) { - OCAPIINF(dev, "link trained in %lld ms\n", - OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); + OCAPIINF(dev, "link trained in %ld ms\n", + tb_to_msecs(mftb() - dev->train_start)); check_trained_link(dev, reg); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } - if (slot->retries-- == 0) + if (tb_compare(mftb(), dev->train_timeout) == TB_AAFTERB) return npu2_opencapi_retry_state(slot, reg); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); @@ -1252,7 +1252,8 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) /* Bump lanes - this improves training reliability */ npu2_opencapi_bump_ui_lane(dev); start_training(chip_id, dev); - slot->retries = OCAPI_LINK_TRAINING_TIMEOUT; + dev->train_start = mftb(); + dev->train_timeout = dev->train_start + msecs_to_tb(OCAPI_LINK_TRAINING_TIMEOUT); pci_slot_set_state(slot, OCAPI_SLOT_LINK_START); return slot->ops.poll_link(slot); diff --git a/include/npu2.h b/include/npu2.h index 5b2a436b..57a9cc96 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -160,6 +160,8 @@ struct npu2_dev { uint64_t linux_pe; bool train_need_fence; bool train_fenced; + unsigned long train_start; + unsigned long train_timeout; }; struct npu2 { From patchwork Wed Jun 19 12:45:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118695 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPsR295Pz9s6w for ; Wed, 19 Jun 2019 22:49:15 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPsQ6f0JzDqQc for ; Wed, 19 Jun 2019 22:49:14 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPn03QDzzDqk7 for ; Wed, 19 Jun 2019 22:45:24 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCiTfr082284 for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7natg1fk-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:19 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:17 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:14 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjCUr45875250 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:12 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AA3B011C04C; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6D8B311C05C; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:03 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0028-0000-0000-0000037BAB7C X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0029-0000-0000-0000243BB920 Message-Id: <20190619124510.25182-6-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 05/12] npu2-opencapi: Tweak fundamental reset sequence X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Modify slightly the ordering of a few steps in our init sequence on fundamental reset, so that it can be called from the OS, when the link is already up: - when the card is reset, the link goes down, so we need to fence the brick to prevent errors propagating to the NPU and OS - since fencing and unfencing don't require any delay, let's also fence/unfence during the very first reset at boot. It's useless but doesn't hurt and keep the code simpler. - resetting the PHY must be done a bit later, while fenced and the ODL and DLx in reset Signed-off-by: Frederic Barrat --- hw/npu2-opencapi.c | 48 +++++++++++++++++++++++++--------------------- include/npu2.h | 2 -- 2 files changed, 26 insertions(+), 24 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 5a94c949..f9cb1d26 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1052,6 +1052,28 @@ static int64_t npu2_opencapi_get_presence_state(struct pci_slot __unused *slot, return OPAL_SUCCESS; } +static void fence_brick(struct npu2_dev *dev) +{ + OCAPIDBG(dev, "Fencing brick\n"); + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b11); + /* from 13.2.1, Quiesce Fence State */ + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, + PPC_BIT(dev->brick_index + 6)); +} + +static void unfence_brick(struct npu2_dev *dev) +{ + OCAPIDBG(dev, "Unfencing brick\n"); + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, + PPC_BIT(dev->brick_index)); + + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b10); + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b00); +} + static enum OpalShpcLinkState get_link_width(uint64_t odl_status) { uint64_t tx_lanes, rx_lanes, state; @@ -1166,7 +1188,7 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) return OPAL_HARDWARE; } -static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) +static int64_t npu2_opencapi_creset(struct pci_slot *slot) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -1196,19 +1218,10 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } - if (dev->train_need_fence) { - OCAPIDBG(dev, "Fencing OTL during reset\n"); - set_fence_control(chip_id, dev->npu->xscom_base, - dev->brick_index, 0b11); - npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, - PPC_BIT(dev->brick_index + 6)); - dev->train_fenced = true; - } - dev->train_need_fence = true; slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; - npu2_opencapi_phy_reset(dev); /* fall-through */ case OCAPI_SLOT_FRESET_INIT: + fence_brick(dev); assert_odl_reset(chip_id, dev->brick_index); assert_adapter_reset(dev); pci_slot_set_state(slot, @@ -1217,6 +1230,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); case OCAPI_SLOT_FRESET_ASSERT_DELAY: + npu2_opencapi_phy_reset(dev); deassert_odl_reset(chip_id, dev->brick_index); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_DEASSERT_DELAY); @@ -1234,15 +1248,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(250)); case OCAPI_SLOT_FRESET_DEASSERT_DELAY2: - if (dev->train_fenced) { - OCAPIDBG(dev, "Unfencing OTL after reset\n"); - npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, - PPC_BIT(dev->brick_index)); - set_fence_control(chip_id, dev->npu->xscom_base, - dev->brick_index, 0b00); - dev->train_fenced = false; - } - + unfence_brick(dev); set_init_pattern(chip_id, dev); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT_DELAY); @@ -1705,8 +1711,6 @@ static void setup_device(struct npu2_dev *dev) dev->bdfn = 0; dev->linux_pe = -1; - dev->train_need_fence = false; - dev->train_fenced = false; /* TODO: Procedure 13.1.3.7 - AFU Memory Range BARs */ /* Procedure 13.1.3.8 - AFU MMIO Range BARs */ diff --git a/include/npu2.h b/include/npu2.h index 57a9cc96..6aead6cb 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -158,8 +158,6 @@ struct npu2_dev { /* OpenCAPI */ struct phb phb_ocapi; uint64_t linux_pe; - bool train_need_fence; - bool train_fenced; unsigned long train_start; unsigned long train_timeout; }; From patchwork Wed Jun 19 12:45:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118696 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPsp6PZnz9s6w for ; Wed, 19 Jun 2019 22:49:34 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPsp4PjgzDqxb for ; Wed, 19 Jun 2019 22:49:34 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPn06yVPzDqk9 for ; Wed, 19 Jun 2019 22:45:24 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCY4OG004131 for ; Wed, 19 Jun 2019 08:45:21 -0400 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7m2p496r-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:21 -0400 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:18 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:14 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjDck32309406 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:13 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 08ED611C04C; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C108511C064; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:12 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:04 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0020-0000-0000-0000034B794D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0021-0000-0000-0000219ECCEA Message-Id: <20190619124510.25182-7-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=787 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 06/12] npu2-opencapi: Simplify freset states X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Let's get rid of one transitional state, since there's no need to pause in between releasing the reset signals of the ODL and the adapter. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index f9cb1d26..1328ee93 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -66,8 +66,7 @@ #define OCAPI_SLOT_FRESET_INIT (OCAPI_SLOT_FRESET + 2) #define OCAPI_SLOT_FRESET_ASSERT_DELAY (OCAPI_SLOT_FRESET + 3) #define OCAPI_SLOT_FRESET_DEASSERT_DELAY (OCAPI_SLOT_FRESET + 4) -#define OCAPI_SLOT_FRESET_DEASSERT_DELAY2 (OCAPI_SLOT_FRESET + 5) -#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 6) +#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 5) #define OCAPI_LINK_TRAINING_RETRIES 2 #define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ @@ -1232,22 +1231,13 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) case OCAPI_SLOT_FRESET_ASSERT_DELAY: npu2_opencapi_phy_reset(dev); deassert_odl_reset(chip_id, dev->brick_index); - pci_slot_set_state(slot, - OCAPI_SLOT_FRESET_DEASSERT_DELAY); - /* - * Minimal delay before taking adapter out of - * reset. Could be useless, but doesn't hurt - */ - return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); - - case OCAPI_SLOT_FRESET_DEASSERT_DELAY: deassert_adapter_reset(dev); pci_slot_set_state(slot, - OCAPI_SLOT_FRESET_DEASSERT_DELAY2); + OCAPI_SLOT_FRESET_DEASSERT_DELAY); /* give 250ms to device to be ready */ return pci_slot_set_sm_timeout(slot, msecs_to_tb(250)); - case OCAPI_SLOT_FRESET_DEASSERT_DELAY2: + case OCAPI_SLOT_FRESET_DEASSERT_DELAY: unfence_brick(dev); set_init_pattern(chip_id, dev); pci_slot_set_state(slot, From patchwork Wed Jun 19 12:45:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118692 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPrk17sBz9s6w for ; Wed, 19 Jun 2019 22:48:38 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPrj740tzDqqY for ; Wed, 19 Jun 2019 22:48:37 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmz5VdBzDqk9 for ; Wed, 19 Jun 2019 22:45:23 +1000 (AEST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCY42Z003926 for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7mm727ra-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:19 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:18 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:15 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjD7a60948718 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:13 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6720911C064; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2053511C066; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:05 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0008-0000-0000-000002F51F65 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0009-0000-0000-000022623B09 Message-Id: <20190619124510.25182-8-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=932 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 07/12] npu2-opencapi: Detect PHY reset errors X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" PHY reset can fail! Though past problems are now fixed, let's handle any future failure. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 13 ++++++++++--- hw/npu2-opencapi.c | 5 ++++- include/npu2.h | 2 +- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index 98e93191..e9501f12 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -1049,10 +1049,17 @@ void npu2_opencapi_phy_init(struct npu2_dev *dev) } } -void npu2_opencapi_phy_reset(struct npu2_dev *dev) +int npu2_opencapi_phy_reset(struct npu2_dev *dev) { - run_procedure(dev, 4); /* procedure_phy_reset */ - run_procedure(dev, 6); /* procedure_phy_rx_dccal */ + int rc; + + rc = run_procedure(dev, 4); /* procedure_phy_reset */ + if (rc != PROCEDURE_COMPLETE) + return -1; + rc = run_procedure(dev, 6); /* procedure_phy_rx_dccal */ + if (rc != PROCEDURE_COMPLETE) + return -1; + return 0; } void npu2_opencapi_phy_prbs31(struct npu2_dev *dev) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 1328ee93..153f2c6f 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1200,6 +1200,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); uint32_t chip_id = dev->npu->chip_id; uint8_t presence = 1; + int rc; switch (slot->state) { case OCAPI_SLOT_NORMAL: @@ -1229,7 +1230,9 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); case OCAPI_SLOT_FRESET_ASSERT_DELAY: - npu2_opencapi_phy_reset(dev); + rc = npu2_opencapi_phy_reset(dev); + if (rc) + return OPAL_HARDWARE; deassert_odl_reset(chip_id, dev->brick_index); deassert_adapter_reset(dev); pci_slot_set_state(slot, diff --git a/include/npu2.h b/include/npu2.h index 6aead6cb..4648464b 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -247,7 +247,7 @@ void npu2_clear_link_flag(struct npu2_dev *ndev, uint8_t flag); uint32_t reset_ntl(struct npu2_dev *ndev); extern int nv_zcal_nominal; void npu2_opencapi_phy_init(struct npu2_dev *dev); -void npu2_opencapi_phy_reset(struct npu2_dev *dev); +int npu2_opencapi_phy_reset(struct npu2_dev *dev); void npu2_opencapi_phy_prbs31(struct npu2_dev *dev); void npu2_opencapi_bump_ui_lane(struct npu2_dev *dev); int64_t npu2_freeze_status(struct phb *phb __unused, From patchwork Wed Jun 19 12:45:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118686 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPqD6G2pz9sNR for ; Wed, 19 Jun 2019 22:47:20 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPqD0ZP1zDqr5 for ; Wed, 19 Jun 2019 22:47:20 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmz00dCzDqkG for ; Wed, 19 Jun 2019 22:45:22 +1000 (AEST) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYU0k039076 for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7m3hvan0-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:19 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:17 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:15 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjDB144564556 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:13 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA7B311C066; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7427611C050; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:06 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-4275-0000-0000-00000343B3FE X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-4276-0000-0000-00003853DF38 Message-Id: <20190619124510.25182-9-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 08/12] npu2-opencapi: Improve error reporting to the OS X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" When resetting an opencapi link, the brick will be fenced temporarily. Therefore we can't rely on the fencing state of the brick any more to check for the health of an opencapi PHB, as we could report errors if queried for a PHB state at the same time a link is being reset. Instead, we flag the device as 'broken' when an error interrupt is received, just before raising an event to the OS. When the OS is querying for the state of a PHB, we only have to look at the 'broken' attribute. Note that there's no recovery possible on P9 when an error interrupt is received unexpectedly, as recovery is not supported by hardware. So when a device/link is marked as 'broken', it stays broken. All the OS can do is log the error and notify the drivers. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-common.c | 7 +++++++ hw/npu2-opencapi.c | 21 +++++++++++++++++---- include/npu2.h | 4 ++++ 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/hw/npu2-common.c b/hw/npu2-common.c index f3f2f45a..a2563efc 100644 --- a/hw/npu2-common.c +++ b/hw/npu2-common.c @@ -419,6 +419,13 @@ static void npu2_err_interrupt(struct irq_source *is, uint32_t isn) p->chip_id, irq_name); free(irq_name); show_all_regs(p, brick); + /* + * P9 NPU doesn't support recovering a link going down + * unexpectedly. So we mark the device as broken and + * report it to the OS, so that the error is logged + * and the drivers notified. + */ + npu2_opencapi_set_broken(p, brick); opal_update_pending_evt(OPAL_EVENT_PCI_ERROR, OPAL_EVENT_PCI_ERROR); break; diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 153f2c6f..c11c945f 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1476,14 +1476,12 @@ static int64_t npu2_opencapi_eeh_next_error(struct phb *phb, uint16_t *severity) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(phb); - uint64_t reg; if (!first_frozen_pe || !pci_error_type || !severity) return OPAL_PARAMETER; - reg = npu2_read(dev->npu, NPU2_MISC_FENCE_STATE); - if (reg & PPC_BIT(dev->brick_index)) { - OCAPIERR(dev, "Brick %d fenced!\n", dev->brick_index); + if (dev->flags & NPU2_DEV_BROKEN) { + OCAPIDBG(dev, "Reporting device as broken\n"); *first_frozen_pe = dev->linux_pe; *pci_error_type = OPAL_EEH_PHB_ERROR; *severity = OPAL_EEH_SEV_PHB_DEAD; @@ -1833,6 +1831,21 @@ static const struct phb_ops npu2_opencapi_ops = { .tce_kill = NULL, }; +void npu2_opencapi_set_broken(struct npu2 *npu, int brick) +{ + struct phb *phb; + struct npu2_dev *dev; + + for_each_phb(phb) { + if (phb->phb_type == phb_type_npu_v2_opencapi) { + dev = phb_to_npu2_dev_ocapi(phb); + if (dev->npu == npu && + dev->brick_index == brick) + dev->flags |= NPU2_DEV_BROKEN; + } + } +} + static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t __unused bdfn, uint64_t addr, uint64_t PE_mask) { diff --git a/include/npu2.h b/include/npu2.h index 4648464b..b376d0ee 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -131,6 +131,8 @@ struct npu2_dev_nvlink { const char *slot_label; }; +#define NPU2_DEV_BROKEN 0x1 + struct npu2_dev { enum npu2_dev_type type; uint32_t link_index; @@ -139,6 +141,7 @@ struct npu2_dev { struct dt_node *dt_node; struct npu2_pcie_bar bars[2]; struct npu2 *npu; + long flags; uint32_t bdfn; @@ -256,4 +259,5 @@ int64_t npu2_freeze_status(struct phb *phb __unused, uint16_t *pci_error_type __unused, uint16_t *severity __unused); void npu2_dump_scoms(int chip_id); +void npu2_opencapi_set_broken(struct npu2 *npu, int brick); #endif /* __NPU2_H */ From patchwork Wed Jun 19 12:45:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118694 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPs53rQjz9s6w for ; Wed, 19 Jun 2019 22:48:57 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPs52MZGzDqsd for ; Wed, 19 Jun 2019 22:48:57 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPn03QSYzDqk8 for ; Wed, 19 Jun 2019 22:45:24 +1000 (AEST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYbvv076005 for ; Wed, 19 Jun 2019 08:45:21 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2t7m51c29f-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:21 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:19 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:15 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjEZc48365702 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:14 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0FFAF11C05E; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C779F11C06C; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:13 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:07 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-4275-0000-0000-00000343B3FF X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-4276-0000-0000-00003853DF39 Message-Id: <20190619124510.25182-10-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 09/12] npu2-opencapi: tweak to Oliver's RFC series X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This will obviously need to be adjusted once the foundation code is upstream. Signed-off-by: Frederic Barrat --- core/pci-opal.c | 73 +++++++++++++++++++++++++++++++------------------ 1 file changed, 46 insertions(+), 27 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 75b746fa..9b63f708 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -656,13 +656,41 @@ static int64_t opal_pci_get_power_state(uint64_t id, uint64_t data) } opal_call(OPAL_PCI_GET_POWER_STATE, opal_pci_get_power_state, 2); +static void rescan_slot(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + prerror("Rescanning slot...\n"); + slot->ops.prepare_link_change(slot, true); + if (phb->phb_type != phb_type_npu_v2_opencapi) { + pci_scan_bus(phb, pd->secondary_bus, + pd->subordinate_bus, &pd->children, pd, true); + pci_add_device_nodes(phb, &pd->children, pd->dn, + &phb->lstate, 0); + } else { + pci_scan_bus(phb, 0, 0xff, &phb->devices, NULL, true); + pci_add_device_nodes(phb, &phb->devices, + phb->dt_node, &phb->lstate, 0); + phb->ops->phb_final_fixup(phb); + } +} + +static u32 get_slot_phandle(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + if (pd) + return pd->dn->phandle; + else + return phb->dt_node->phandle; +} + static void link_up_timer(struct timer *t __unused, void *data, uint64_t now __unused) { struct pci_slot *slot = data; - struct phb *phb = slot->phb; - struct pci_device *pd = slot->pd; - struct dt_node *dn = pd->dn; uint8_t link; int64_t rc = OPAL_BUSY; @@ -676,8 +704,9 @@ static void link_up_timer(struct timer *t __unused, void *data, rc = slot->ops.run_sm(slot); if (rc < 0) // error, abort goto done; - if (rc > 0 && slot->retries-- <= 0) // timeout, abort - goto done; + // the slot->retries counter will interfere with our link training attempts. It says as much in comment above (?) + /* if (rc > 0 && slot->retries-- <= 0) // timeout, abort */ + /* goto done; */ if (rc > 0) { // kick the can down the road schedule_timer(t, msecs_to_tb(10)); return; @@ -685,26 +714,19 @@ static void link_up_timer(struct timer *t __unused, void *data, if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) link = 0; - if (link) { - prerror("scanning slot...\n"); - slot->ops.prepare_link_change(slot, true); - - pci_scan_bus(phb, pd->secondary_bus, - pd->subordinate_bus, - &pd->children, pd, true); - pci_add_device_nodes(phb, &pd->children, dn, - &phb->lstate, 0); - } + if (link) + rescan_slot(slot); done: /* * We need to send a completion message back to the kernel, otherwise * it'll sit there forever. */ - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); + // better left out to the state machine? + // pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), slot->power_state, rc <= 0 ? rc : OPAL_BUSY); prerror("sent competition state = %lld from %s\n", rc, __func__); } @@ -714,7 +736,6 @@ static void set_power_timer(struct timer *t, void *data, uint64_t now) struct pci_slot *slot = data; //struct phb *phb = slot->phb; struct pci_device *pd = slot->pd; - struct dt_node *dn = pd->dn; //uint8_t link; struct phb *phb = slot->phb; int64_t rc = OPAL_BUSY; @@ -758,8 +779,7 @@ static void set_power_timer(struct timer *t, void *data, uint64_t now) * In the power up case run through the FRESET handler. This is hella * dumb and we need to do something less stupid in future. */ - pci_slot_set_state(slot, PCI_SLOT_STATE_LINK_START_POLL); - + pci_slot_set_state(slot, PCI_SLOT_STATE_FRESET); // switch to using the link poll timer init_timer(&slot->timer, link_up_timer, slot); link_up_timer(t, data, now); @@ -773,7 +793,7 @@ done: pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), slot->power_state, rc <= 0 ? rc : OPAL_BUSY); prerror("sent competition state = %lld from %s\n", rc, __func__); } @@ -860,13 +880,12 @@ static int64_t opal_pci_set_power_state(uint64_t async_token, /* Otherwise scan the slot so we have the new device in the DT */ if (*state == OPAL_PCI_SLOT_POWER_ON) { - slot->ops.prepare_link_change(slot, true); - pci_scan_bus(phb, pd->secondary_bus, - pd->subordinate_bus, &pd->children, pd, true); - pci_add_device_nodes(phb, &pd->children, pd->dn, - &phb->lstate, 0); + rescan_slot(slot); } else { - pci_remove_bus(phb, &pd->children); + if (phb->phb_type != phb_type_npu_v2_opencapi) + pci_remove_bus(phb, &pd->children); + else + pci_remove_bus(phb, &phb->devices); } phb_unlock(phb); From patchwork Wed Jun 19 12:45:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118691 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPrN73SYz9s6w for ; Wed, 19 Jun 2019 22:48:20 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPrN5pRWzDqs3 for ; Wed, 19 Jun 2019 22:48:20 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmz3qJczDqk8 for ; Wed, 19 Jun 2019 22:45:23 +1000 (AEST) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCY4rp003952 for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7mm727rj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:19 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:18 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:16 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjEhx55443694 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:14 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63FBA11C069; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 26E4211C04A; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:08 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0016-0000-0000-0000028A752E X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0017-0000-0000-000032E7CBFE Message-Id: <20190619124510.25182-11-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 10/12] npu2-opencapi: Activate PCI hotplug on opencapi slot X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Implement the get_power_state() and set_power_state() callbacks for the opencapi PHB and add properties in the device tree to mark the opencapi slot as hot-pluggable. Signed-off-by: Frederic Barrat --- core/pci-slot.c | 3 +++ hw/npu2-opencapi.c | 51 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 51 insertions(+), 3 deletions(-) diff --git a/core/pci-slot.c b/core/pci-slot.c index 8b0cc713..fd7b1168 100644 --- a/core/pci-slot.c +++ b/core/pci-slot.c @@ -32,6 +32,9 @@ static void pci_slot_prepare_link_change(struct pci_slot *slot, bool up) struct pci_device *pd = slot->pd; uint32_t aercap, mask; + if (!pd) + return; + /* * Mask the link down and receiver error before the link becomes * down. Otherwise, unmask the errors when the link is up. diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index c11c945f..c9a8410e 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1047,7 +1047,7 @@ static int64_t npu2_opencapi_get_presence_state(struct pci_slot __unused *slot, * * This may change if we ever support hotplug down the track. */ - *val = true; + *val = OPAL_PCI_SLOT_PRESENT; return OPAL_SUCCESS; } @@ -1105,6 +1105,38 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) return OPAL_SUCCESS; } +static int64_t npu2_opencapi_get_power_state(struct pci_slot *slot, + uint8_t *val) +{ + *val = slot->power_state; + return OPAL_SUCCESS; +} + +static int64_t npu2_opencapi_set_power_state(struct pci_slot *slot, uint8_t val) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + switch (val) { + case PCI_SLOT_POWER_OFF: + OCAPIDBG(dev, "Setting the adapter in reset\n"); + fence_brick(dev); + assert_adapter_reset(dev); + slot->power_state = PCI_SLOT_POWER_OFF; + return OPAL_SUCCESS; + + case PCI_SLOT_POWER_ON: + if (slot->power_state != PCI_SLOT_POWER_OFF) + return OPAL_SUCCESS; + OCAPIDBG(dev, "Reactivating the adapter\n"); + slot->power_state = PCI_SLOT_POWER_ON; + slot->state = OCAPI_SLOT_NORMAL; + return msecs_to_tb(1); + + default: + return OPAL_UNSUPPORTED; + } +} + static void check_trained_link(struct npu2_dev *dev, uint64_t odl_status) { if (get_link_width(odl_status) != OPAL_SHPC_LINK_UP_x8) { @@ -1204,6 +1236,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) switch (slot->state) { case OCAPI_SLOT_NORMAL: + case OCAPI_SLOT_FRESET: case OCAPI_SLOT_FRESET_START: OCAPIDBG(dev, "FRESET starts\n"); @@ -1272,6 +1305,17 @@ static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) return OPAL_UNSUPPORTED; } +static void make_slot_hotpluggable(struct pci_slot *slot, struct phb *phb) +{ + char label[40]; + + slot->pluggable = 1; + pci_slot_add_dt_properties(slot, phb->dt_node); + snprintf(label, sizeof(label), "OPENCAPI-%04x", + (int) PCI_SLOT_PHB_INDEX(slot->id)); + dt_add_property_string(phb->dt_node, "ibm,slot-label", label); +} + static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) { struct pci_slot *slot; @@ -1283,10 +1327,10 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) /* TODO: Figure out other slot functions */ slot->ops.get_presence_state = npu2_opencapi_get_presence_state; slot->ops.get_link_state = npu2_opencapi_get_link_state; - slot->ops.get_power_state = NULL; + slot->ops.get_power_state = npu2_opencapi_get_power_state; slot->ops.get_attention_state = NULL; slot->ops.get_latch_state = NULL; - slot->ops.set_power_state = NULL; + slot->ops.set_power_state = npu2_opencapi_set_power_state; slot->ops.set_attention_state = NULL; slot->ops.poll_link = npu2_opencapi_poll_link; @@ -1294,6 +1338,7 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) slot->ops.freset = npu2_opencapi_freset; slot->ops.hreset = npu2_opencapi_hreset; + make_slot_hotpluggable(slot, phb); return slot; } From patchwork Wed Jun 19 12:45:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118689 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPqv4ldMz9s6w for ; Wed, 19 Jun 2019 22:47:55 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPqv3c8pzDqtR for ; Wed, 19 Jun 2019 22:47:55 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPmz34lFzDqk7 for ; Wed, 19 Jun 2019 22:45:23 +1000 (AEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYMFJ106346 for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7kq9dfxg-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:45:20 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:18 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:16 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjEdd12124322 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:14 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CA7D811C04A; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A2A011C078; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:09 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0028-0000-0000-0000037BAB7E X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0029-0000-0000-0000243BB922 Message-Id: <20190619124510.25182-12-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=757 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 11/12] npu2-opencapi: Handle OPAL_UNMAP_PE operation on set_pe() callback X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" In a hot-unplug scenario, the OS will try to unmap the PE. Skiboot doesn't do anything with the linux PE for opencapi other than being a mailbox, but at least let's be consistent. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index c9a8410e..dd238235 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1497,6 +1497,8 @@ static int64_t npu2_opencapi_set_pe(struct phb *phb, * functions on the device, the OS can define many PEs, we * only keep one, the OS will handle it. */ + if (action == OPAL_UNMAP_PE) + pe_num = -1; dev->linux_pe = pe_num; return OPAL_SUCCESS; } From patchwork Wed Jun 19 12:45:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1118699 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45TPv4652Dz9s6w for ; Wed, 19 Jun 2019 22:50:40 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 45TPv4414szDr1d for ; Wed, 19 Jun 2019 22:50:40 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 45TPp62qCJzDqkL for ; Wed, 19 Jun 2019 22:46:22 +1000 (AEST) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5JCYF8n029890 for ; Wed, 19 Jun 2019 08:46:20 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2t7m46ma43-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 19 Jun 2019 08:46:18 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 19 Jun 2019 13:45:20 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 19 Jun 2019 13:45:16 +0100 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x5JCjFhg45088940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 12:45:15 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 20CD011C04C; Wed, 19 Jun 2019 12:45:15 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D83CF11C05E; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) Received: from pic2.home (unknown [9.145.171.67]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 19 Jun 2019 12:45:14 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com Date: Wed, 19 Jun 2019 14:45:10 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190619124510.25182-1-fbarrat@linux.ibm.com> References: <20190619124510.25182-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19061912-0028-0000-0000-0000037BAB7F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19061912-0029-0000-0000-0000243BB923 Message-Id: <20190619124510.25182-13-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-19_07:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=699 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190104 Subject: [Skiboot] [RFC 12/12] npu2-opencapi: Log an warning when resetting a broken device X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com, groug@kaod.org Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" On P9, the NPU doesn't support recovery if the link goes down unexpectedly. It was not fully verified. We mark the device as broken when we receive an error interrupt from the NPU. However, there's nothing to prevent the OS from trying to reset the device; It may or may not work, it's unsupported territory, so let's log a message to make it clear, as it could help when debugging. We haven't hit any cases where the reset goes badly enough that we'd want to prevent it, so let it go for now. We can revisit later if we have evidence that it's causing more problems than it is worth. Signed-off-by: Frederic Barrat --- hw/npu2-opencapi.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index dd238235..fc9155f1 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1251,6 +1251,10 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } + if (dev->flags & NPU2_DEV_BROKEN) { + OCAPIERR(dev, "Resetting a device which hit a previous error. Device recovery is not supported, so future behavior is undefined\n"); + dev->flags &= ~NPU2_DEV_BROKEN; + } slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; /* fall-through */ case OCAPI_SLOT_FRESET_INIT: