From patchwork Mon Sep 9 12:31:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159698 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RncV2wLxz9s7T for ; Mon, 9 Sep 2019 22:32:42 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RncV1wr4zDqHR for ; Mon, 9 Sep 2019 22:32:42 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk4Kw4zDqNP for ; Mon, 9 Sep 2019 22:32:01 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNoXA101153 for ; Mon, 9 Sep 2019 08:31:57 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwm9fqdsq-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:31:57 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:55 +0100 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:54 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVS9p26542534 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:28 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9ABBFAE05F; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 60295AE058; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:36 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0016-0000-0000-000002A8C552 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0017-0000-0000-0000330946C9 Message-Id: <20190909123151.21944-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 01/16] core/pci: Refactor common paths on slot hotplug X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Refactor code executed to remove or rescan devices when a slot power state changes, synchronously or asynchronously through a timer callback. It will be more useful in a future patch. No functional changes. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- core/pci-opal.c | 43 ++++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 213a7256..d5209600 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -647,11 +647,30 @@ static int64_t opal_pci_get_power_state(uint64_t id, uint64_t data) } opal_call(OPAL_PCI_GET_POWER_STATE, opal_pci_get_power_state, 2); +static void rescan_slot_devices(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + slot->ops.prepare_link_change(slot, true); + pci_scan_bus(phb, pd->secondary_bus, + pd->subordinate_bus, &pd->children, pd, true); + pci_add_device_nodes(phb, &pd->children, pd->dn, + &phb->lstate, 0); +} + +static void remove_slot_devices(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + pci_remove_bus(phb, &pd->children); +} + static void set_power_timer(struct timer *t __unused, void *data, uint64_t now __unused) { struct pci_slot *slot = data; - struct phb *phb = slot->phb; struct pci_device *pd = slot->pd; struct dt_node *dn = pd->dn; uint8_t link; @@ -670,7 +689,7 @@ static void set_power_timer(struct timer *t __unused, void *data, break; case PCI_SLOT_STATE_SPOWER_DONE: if (slot->power_state == OPAL_PCI_SLOT_POWER_OFF) { - pci_remove_bus(phb, &pd->children); + remove_slot_devices(slot); pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, slot->async_token, dn->phandle, @@ -682,12 +701,7 @@ static void set_power_timer(struct timer *t __unused, void *data, if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) link = 0; if (link) { - slot->ops.prepare_link_change(slot, true); - pci_scan_bus(phb, pd->secondary_bus, - pd->subordinate_bus, - &pd->children, pd, true); - pci_add_device_nodes(phb, &pd->children, dn, - &phb->lstate, 0); + rescan_slot_devices(slot); pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, slot->async_token, dn->phandle, @@ -782,15 +796,10 @@ static int64_t opal_pci_set_power_state(uint64_t async_token, init_timer(&slot->timer, set_power_timer, slot); schedule_timer(&slot->timer, msecs_to_tb(10)); } else if (rc == OPAL_SUCCESS) { - if (*state == OPAL_PCI_SLOT_POWER_OFF) { - pci_remove_bus(phb, &pd->children); - } else { - slot->ops.prepare_link_change(slot, true); - pci_scan_bus(phb, pd->secondary_bus, - pd->subordinate_bus, &pd->children, pd, true); - pci_add_device_nodes(phb, &pd->children, pd->dn, - &phb->lstate, 0); - } + if (*state == OPAL_PCI_SLOT_POWER_OFF) + remove_slot_devices(slot); + else + rescan_slot_devices(slot); } phb_unlock(phb); From patchwork Mon Sep 9 12:31:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159717 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnjg3FwZz9s7T for ; Mon, 9 Sep 2019 22:37:11 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnjf6ph3zDqNV for ; Mon, 9 Sep 2019 22:37:10 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbq1QX4zDqHR for ; Mon, 9 Sep 2019 22:32:06 +1000 (AEST) Received: from pps.filterd (m0187473.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CM9Eh106293 for ; Mon, 9 Sep 2019 08:32:04 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uv86uvhrh-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:03 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:56 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:54 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVrO557344188 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:53 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC07DAE055; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A77E4AE053; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:37 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0012-0000-0000-00000348C3E6 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0013-0000-0000-00002183240D Message-Id: <20190909123151.21944-3-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=870 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 02/16] core/pci: Add missing lock in set_power_timer X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" set_power_timer() was not using any lock, though it alters the slot state and devices found under it. So lock the PHB under which the slot is found to avoid concurrent operations. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- core/pci-opal.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/core/pci-opal.c b/core/pci-opal.c index d5209600..175810d0 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -674,7 +674,9 @@ static void set_power_timer(struct timer *t __unused, void *data, struct pci_device *pd = slot->pd; struct dt_node *dn = pd->dn; uint8_t link; + struct phb *phb = slot->phb; + phb_lock(phb); switch (slot->state) { case PCI_SLOT_STATE_SPOWER_START: if (slot->retries-- == 0) { @@ -720,6 +722,7 @@ static void set_power_timer(struct timer *t __unused, void *data, prlog(PR_ERR, "PCI SLOT %016llx: Unexpected state 0x%08x\n", slot->id, slot->state); } + phb_unlock(phb); } static int64_t opal_pci_set_power_state(uint64_t async_token, From patchwork Mon Sep 9 12:31:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159697 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnc16ZC4z9s7T for ; Mon, 9 Sep 2019 22:32:17 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnbz5cRYzDqNV for ; Mon, 9 Sep 2019 22:32:15 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk44FMzDqHR for ; Mon, 9 Sep 2019 22:32:01 +1000 (AEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMrdL106945 for ; Mon, 9 Sep 2019 08:31:59 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwnfwv599-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:31:58 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:57 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:54 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVrni53149886 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:53 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 40649AE056; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 053C0AE059; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:52 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:38 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF37 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E85 Message-Id: <20190909123151.21944-4-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=904 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 03/16] core/pci: Use proper phandle during hotplug for PHB slots X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" PHB slots don't have an associated device (slot->pd = NULL). They were not used by the PCI hotplug framework so far, but with opencapi virtual PHBs, that's changing. With opencapi, devices are directly under the PHB (no root complex or intermediate bridge) and the slot used for hotplug is the PHB slot. This patch uses the proper phandle when replying asynchronously to the OS when using a PHB slot. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- core/pci-opal.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 175810d0..69bd65c6 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -647,6 +647,17 @@ static int64_t opal_pci_get_power_state(uint64_t id, uint64_t data) } opal_call(OPAL_PCI_GET_POWER_STATE, opal_pci_get_power_state, 2); +static u32 get_slot_phandle(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + if (pd) + return pd->dn->phandle; + else + return phb->dt_node->phandle; +} + static void rescan_slot_devices(struct pci_slot *slot) { struct phb *phb = slot->phb; @@ -671,8 +682,6 @@ static void set_power_timer(struct timer *t __unused, void *data, uint64_t now __unused) { struct pci_slot *slot = data; - struct pci_device *pd = slot->pd; - struct dt_node *dn = pd->dn; uint8_t link; struct phb *phb = slot->phb; @@ -682,7 +691,7 @@ static void set_power_timer(struct timer *t __unused, void *data, if (slot->retries-- == 0) { pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), slot->power_state, OPAL_BUSY); } else { schedule_timer(&slot->timer, msecs_to_tb(10)); @@ -694,7 +703,7 @@ static void set_power_timer(struct timer *t __unused, void *data, remove_slot_devices(slot); pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), OPAL_PCI_SLOT_POWER_OFF, OPAL_SUCCESS); break; } @@ -706,12 +715,12 @@ static void set_power_timer(struct timer *t __unused, void *data, rescan_slot_devices(slot); pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), OPAL_PCI_SLOT_POWER_ON, OPAL_SUCCESS); } else if (slot->retries-- == 0) { pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, dn->phandle, + slot->async_token, get_slot_phandle(slot), OPAL_PCI_SLOT_POWER_ON, OPAL_BUSY); } else { schedule_timer(&slot->timer, msecs_to_tb(10)); From patchwork Mon Sep 9 12:31:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159702 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnf15r6dz9sCJ for ; Mon, 9 Sep 2019 22:34:01 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnf11hqLzDqNC for ; Mon, 9 Sep 2019 22:34:01 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk48LHzDqNC for ; Mon, 9 Sep 2019 22:32:01 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNfvA100653 for ; Mon, 9 Sep 2019 08:31:59 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwm9fqdtw-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:31:58 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:57 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:55 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVrIZ51446014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:53 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 81C57AE068; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4E233AE058; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:39 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF36 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E86 Message-Id: <20190909123151.21944-5-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 04/16] core/pci: Train link of PHB slots when hotplugging X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The link of PHB slots must be trained after powering on. This can be done by calling the fundamental reset callback of the slot. We could force a reset for all the slots and have a common path in set_power_state(). But this patch only resets the PHB slot. Some slot implementations do a power cycle during fundamental reset, so calling a reset after powering on would repeat that operation. Signed-off-by: Frederic Barrat Reviewed-by: Andrew Donnellan --- core/pci-opal.c | 130 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 108 insertions(+), 22 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 69bd65c6..6c070030 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -678,11 +678,111 @@ static void remove_slot_devices(struct pci_slot *slot) pci_remove_bus(phb, &pd->children); } +static void link_training_timer(struct timer *t, void *data, + uint64_t now __unused) +{ + struct pci_slot *slot = data; + struct phb *phb = slot->phb; + uint8_t link; + int64_t rc = 0; + + phb_lock(phb); + + rc = slot->ops.run_sm(slot); + if (rc < 0) + goto out; + if (rc > 0) { + schedule_timer(t, rc); + phb_unlock(phb); + return; + } + + if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) + link = 0; + if (!link) { + rc = OPAL_HARDWARE; + goto out; + } + + rescan_slot_devices(slot); +out: + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, get_slot_phandle(slot), + slot->power_state, rc <= 0 ? rc : OPAL_BUSY); + phb_unlock(phb); +} + +static void link_up_timer(struct timer *t, void *data, uint64_t now __unused) +{ + struct pci_slot *slot = data; + struct phb *phb = slot->phb; + uint8_t link; + int64_t rc = OPAL_SUCCESS; + + phb_lock(phb); + if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) + link = 0; + if (!link) { + if (slot->retries-- == 0) { + rc = OPAL_BUSY; + goto out; + } else { + schedule_timer(t, msecs_to_tb(10)); + phb_unlock(phb); + return; + } + } + + rescan_slot_devices(slot); +out: + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, get_slot_phandle(slot), + slot->power_state, rc <= 0 ? rc : OPAL_BUSY); + phb_unlock(phb); +} + +static bool training_needed(struct pci_slot *slot) +{ + struct phb *phb = slot->phb; + struct pci_device *pd = slot->pd; + + if ((pd && !pd->parent) || /* "real" PHB */ + (!pd && phb->phb_type == phb_type_npu_v2_opencapi)) + return true; + return false; +} + +static void wait_for_link_up_and_rescan(struct pci_slot *slot) +{ + int64_t rc; + + /* + * Links for PHB slots need to be retrained by triggering a + * fundamental reset. Other slots also need to be tested for + * readiness + */ + if (training_needed(slot)) { + rc = slot->ops.freset(slot); + if (rc < 0) { + opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, + slot->async_token, + get_slot_phandle(slot), + slot->power_state, + rc <= 0 ? rc : OPAL_BUSY); + return; + } + init_timer(&slot->timer, link_training_timer, slot); + } else { + init_timer(&slot->timer, link_up_timer, slot); + rc = 1; + } + schedule_timer(&slot->timer, rc); +} + static void set_power_timer(struct timer *t __unused, void *data, uint64_t now __unused) { struct pci_slot *slot = data; - uint8_t link; struct phb *phb = slot->phb; phb_lock(phb); @@ -699,9 +799,9 @@ static void set_power_timer(struct timer *t __unused, void *data, break; case PCI_SLOT_STATE_SPOWER_DONE: + pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); if (slot->power_state == OPAL_PCI_SLOT_POWER_OFF) { remove_slot_devices(slot); - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, slot->async_token, get_slot_phandle(slot), OPAL_PCI_SLOT_POWER_OFF, OPAL_SUCCESS); @@ -709,23 +809,7 @@ static void set_power_timer(struct timer *t __unused, void *data, } /* Power on */ - if (slot->ops.get_link_state(slot, &link) != OPAL_SUCCESS) - link = 0; - if (link) { - rescan_slot_devices(slot); - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); - opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, get_slot_phandle(slot), - OPAL_PCI_SLOT_POWER_ON, OPAL_SUCCESS); - } else if (slot->retries-- == 0) { - pci_slot_set_state(slot, PCI_SLOT_STATE_NORMAL); - opal_queue_msg(OPAL_MSG_ASYNC_COMP, NULL, NULL, - slot->async_token, get_slot_phandle(slot), - OPAL_PCI_SLOT_POWER_ON, OPAL_BUSY); - } else { - schedule_timer(&slot->timer, msecs_to_tb(10)); - } - + wait_for_link_up_and_rescan(slot); break; default: prlog(PR_ERR, "PCI SLOT %016llx: Unexpected state 0x%08x\n", @@ -808,10 +892,12 @@ static int64_t opal_pci_set_power_state(uint64_t async_token, init_timer(&slot->timer, set_power_timer, slot); schedule_timer(&slot->timer, msecs_to_tb(10)); } else if (rc == OPAL_SUCCESS) { - if (*state == OPAL_PCI_SLOT_POWER_OFF) + if (*state == OPAL_PCI_SLOT_POWER_OFF) { remove_slot_devices(slot); - else - rescan_slot_devices(slot); + } else { + wait_for_link_up_and_rescan(slot); + rc = OPAL_ASYNC_COMPLETION; + } } phb_unlock(phb); From patchwork Mon Sep 9 12:31:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159700 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RndX1r3cz9s7T for ; Mon, 9 Sep 2019 22:33:36 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RndX0LSvzDqNg for ; Mon, 9 Sep 2019 22:33:36 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk58lrzDqNR for ; Mon, 9 Sep 2019 22:32:02 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMtpD077177 for ; Mon, 9 Sep 2019 08:31:59 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwpmugk77-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:31:58 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:56 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:55 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVrHw43319478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:54 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C7434AE064; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 95F1FAE057; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:40 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF38 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E87 Message-Id: <20190909123151.21944-6-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 05/16] core/pci: Fix scan of devices for opencapi slots X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Opencapi devices are found directly under the PHB and the PHB slot doesn't have an associated PCI device (root complex). So when scanning a PHB, devices are added directly under the PHB, like it's done at boot time. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard --- core/pci-opal.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/core/pci-opal.c b/core/pci-opal.c index 6c070030..23aac84e 100644 --- a/core/pci-opal.c +++ b/core/pci-opal.c @@ -664,10 +664,17 @@ static void rescan_slot_devices(struct pci_slot *slot) struct pci_device *pd = slot->pd; slot->ops.prepare_link_change(slot, true); - pci_scan_bus(phb, pd->secondary_bus, - pd->subordinate_bus, &pd->children, pd, true); - pci_add_device_nodes(phb, &pd->children, pd->dn, - &phb->lstate, 0); + if (phb->phb_type != phb_type_npu_v2_opencapi) { + pci_scan_bus(phb, pd->secondary_bus, + pd->subordinate_bus, &pd->children, pd, true); + pci_add_device_nodes(phb, &pd->children, pd->dn, + &phb->lstate, 0); + } else { + pci_scan_bus(phb, 0, 0xff, &phb->devices, NULL, true); + pci_add_device_nodes(phb, &phb->devices, + phb->dt_node, &phb->lstate, 0); + phb->ops->phb_final_fixup(phb); + } } static void remove_slot_devices(struct pci_slot *slot) @@ -675,7 +682,10 @@ static void remove_slot_devices(struct pci_slot *slot) struct phb *phb = slot->phb; struct pci_device *pd = slot->pd; - pci_remove_bus(phb, &pd->children); + if (phb->phb_type != phb_type_npu_v2_opencapi) + pci_remove_bus(phb, &pd->children); + else + pci_remove_bus(phb, &phb->devices); } static void link_training_timer(struct timer *t, void *data, From patchwork Mon Sep 9 12:31:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159719 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RnkV5q5Vz9sCJ for ; Mon, 9 Sep 2019 22:37:54 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RnkV4GcgzDqNf for ; Mon, 9 Sep 2019 22:37:54 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46RncQ63zfzDqNv for ; Mon, 9 Sep 2019 22:32:38 +1000 (AEST) Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CN478050679 for ; Mon, 9 Sep 2019 08:32:37 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwm8c7paj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:10 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:55 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVsja50462752 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:54 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1AAA5AE057; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DE441AE056; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:53 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:41 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0008-0000-0000-000003132183 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0009-0000-0000-00004A3184E3 Message-Id: <20190909123151.21944-7-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 06/16] npu2-hw-procedures: Move some opencapi PHY settings in one-off init X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The PHY_RX_AC_COUPLED and PHY_RX_SPEED_SELECT for opencapi are group settings for the obus. They should be set in the one-off PHY init function at boot and not on the link reset path, as they theoretically impact more than one link. Since we cannot mix link type and/or speed on an optical bus, it has no pratical impact, it just looks cleaner. Also use the OCAPIINF macro for the associated traces. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index c1ae8f11..6cd780cc 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -418,25 +418,6 @@ static uint32_t phy_reset_complete(struct npu2_dev *ndev) { int lane; - if (ndev->type == NPU2_DEV_TYPE_OPENCAPI) { - phy_write(ndev, &NPU2_PHY_RX_AC_COUPLED, 1); - - switch (ndev->link_speed) { - case 20000000000UL: - prlog(PR_INFO, "OCAPI: Link speed set at 20Gb/s\n"); - phy_write(ndev, &NPU2_PHY_RX_SPEED_SELECT, 1); - break; - case 25000000000UL: - case 25781250000UL: - prlog(PR_INFO, "OCAPI: Link speed set at 25.xGb/s\n"); - phy_write(ndev, &NPU2_PHY_RX_SPEED_SELECT, 0); - break; - default: - prlog(PR_CRIT, "OCAPI: Invalid link speed!\n"); - assert(false); - } - } - FOR_EACH_LANE(ndev, lane) { phy_write_lane(ndev, &NPU2_PHY_RX_LANE_ANA_PDWN, lane, 0); phy_write_lane(ndev, &NPU2_PHY_RX_LANE_DIG_PDWN, lane, 0); @@ -1026,6 +1007,22 @@ void npu2_opencapi_phy_init(struct npu2_dev *dev) * Witherspoon it needs to be done in skiboot after device detection. */ phy_write(dev, &NPU2_PHY_RX_RC_ENABLE_AUTO_RECAL, 0x1); + phy_write(dev, &NPU2_PHY_RX_AC_COUPLED, 1); + + switch (dev->link_speed) { + case 20000000000UL: + OCAPIINF(dev, "Link speed set at 20Gb/s\n"); + phy_write(dev, &NPU2_PHY_RX_SPEED_SELECT, 1); + break; + case 25000000000UL: + case 25781250000UL: + OCAPIINF(dev, "Link speed set at 25.xGb/s\n"); + phy_write(dev, &NPU2_PHY_RX_SPEED_SELECT, 0); + break; + default: + OCAPIERR(dev, "Invalid link speed!\n"); + assert(false); + } } void npu2_opencapi_phy_reset(struct npu2_dev *dev) From patchwork Mon Sep 9 12:31:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159704 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnfq1YNjz9sNF for ; Mon, 9 Sep 2019 22:34:43 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnfp63z1zDqNZ for ; Mon, 9 Sep 2019 22:34:42 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk6znzzDqNT for ; Mon, 9 Sep 2019 22:32:02 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMoSG076839 for ; Mon, 9 Sep 2019 08:32:01 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwpmugk8h-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:00 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:58 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:56 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVskp37879912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:54 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6BFCEAE057; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 31517AE056; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:42 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF3A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E8A Message-Id: <20190909123151.21944-8-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=925 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 07/16] npu2-opencapi: Make sure the PCI slot has the proper ID X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The PCI slot created for the opencapi PHB didn't have its ID properly defined because it was created before we assign an ID to the PHB. Simply switch the PCI slot creation and PHB registration calls to fix it. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 9a391bb0..504c9208 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1704,6 +1704,8 @@ static void setup_device(struct npu2_dev *dev) set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, dev->brick_index, 0b00); + pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); + if (npu2_ocapi_training_state != NPU2_TRAIN_DEFAULT) { setup_debug_training_state(dev); } else { @@ -1717,7 +1719,6 @@ static void setup_device(struct npu2_dev *dev) prlog(PR_ERR, "OCAPI: Cannot create PHB slot\n"); } } - pci_register_phb(&dev->phb_ocapi, OPAL_DYNAMIC_PHB_ID); return; } From patchwork Mon Sep 9 12:31:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159707 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rngp3fsZz9s7T for ; Mon, 9 Sep 2019 22:35:34 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rngp0gMGzDqNf for ; Mon, 9 Sep 2019 22:35:34 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbn1twYzDqNP for ; Mon, 9 Sep 2019 22:32:05 +1000 (AEST) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMf4e104362 for ; Mon, 9 Sep 2019 08:32:03 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwpeh182g-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:02 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:56 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVsxJ23003196 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:54 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B2C79AE05A; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82DEFAE058; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:43 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0016-0000-0000-000002A8C554 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0017-0000-0000-0000330946CA Message-Id: <20190909123151.21944-9-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=973 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 08/16] npu2-hw-procedures: Fix link retraining on reset X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Link retraining was showing reliability problems due to some opencapi-only settings not being optimized. This patch updates some extra PHY state, as agreed with the PHY team. Though they mostly impact link retraining behavior, they should also be set at boot. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index 6cd780cc..ad1627ae 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -60,8 +60,14 @@ static struct npu2_phy_reg NPU2_PHY_RX_PR_FW_OFF = {0x08a, 56, 1}; static struct npu2_phy_reg NPU2_PHY_RX_PR_FW_INERTIA_AMT = {0x08a, 57, 3}; static struct npu2_phy_reg NPU2_PHY_RX_CFG_LTE_MC = {0x000, 60, 4}; static struct npu2_phy_reg NPU2_PHY_RX_A_INTEG_COARSE_GAIN = {0x00a, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_A_CTLE_COARSE = {0x00c, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_A_CTLE_GAIN = {0x00c, 53, 4}; static struct npu2_phy_reg NPU2_PHY_RX_B_INTEG_COARSE_GAIN = {0x026, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_B_CTLE_COARSE = {0x028, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_B_CTLE_GAIN = {0x028, 53, 4}; static struct npu2_phy_reg NPU2_PHY_RX_E_INTEG_COARSE_GAIN = {0x030, 48, 4}; +static struct npu2_phy_reg NPU2_PHY_RX_E_CTLE_COARSE = {0x032, 48, 5}; +static struct npu2_phy_reg NPU2_PHY_RX_E_CTLE_GAIN = {0x032, 53, 4}; /* These registers are per-PHY, not per lane */ static struct npu2_phy_reg NPU2_PHY_RX_SPEED_SELECT = {0x262, 51, 2}; @@ -429,6 +435,16 @@ static uint32_t phy_reset_complete(struct npu2_dev *ndev) phy_write_lane(ndev, &NPU2_PHY_RX_A_INTEG_COARSE_GAIN, lane, 11); phy_write_lane(ndev, &NPU2_PHY_RX_B_INTEG_COARSE_GAIN, lane, 11); phy_write_lane(ndev, &NPU2_PHY_RX_E_INTEG_COARSE_GAIN, lane, 11); + + if (ndev->type == NPU2_DEV_TYPE_OPENCAPI) { + phy_write_lane(ndev, &NPU2_PHY_RX_A_CTLE_GAIN, lane, 0); + phy_write_lane(ndev, &NPU2_PHY_RX_B_CTLE_GAIN, lane, 0); + phy_write_lane(ndev, &NPU2_PHY_RX_E_CTLE_GAIN, lane, 0); + + phy_write_lane(ndev, &NPU2_PHY_RX_A_CTLE_COARSE, lane, 20); + phy_write_lane(ndev, &NPU2_PHY_RX_B_CTLE_COARSE, lane, 20); + phy_write_lane(ndev, &NPU2_PHY_RX_E_CTLE_COARSE, lane, 20); + } } set_iovalid(ndev, true); From patchwork Mon Sep 9 12:31:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnk275dTz9s7T for ; Mon, 9 Sep 2019 22:37:30 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnk25KvTzDqHg for ; Mon, 9 Sep 2019 22:37:30 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnby4rPFzDqHR for ; Mon, 9 Sep 2019 22:32:14 +1000 (AEST) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CR6Zl106252 for ; Mon, 9 Sep 2019 08:32:12 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwm4rykcf-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:09 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:56 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVt3b32178514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:55 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10560AE04D; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C9F33AE051; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:54 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:44 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-4275-0000-0000-000003633701 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-4276-0000-0000-000038758753 Message-Id: <20190909123151.21944-10-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 09/16] npu2-opencapi: Rework link training timeout X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Opencapi link state should be polled for up to 3 seconds. Current code assumes a tight retry loop during fundamental reset at boot, which is not going to be true on link retraining. So update the timeout detection code to use a timebase instead of a simple retry count which could be way too long. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 9 +++++---- include/npu2.h | 2 ++ 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 504c9208..f7be9f09 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1127,13 +1127,13 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) reg = get_odl_status(chip_id, dev->brick_index); if (GETFIELD(OB_ODL_STATUS_TRAINING_STATE_MACHINE, reg) == OCAPI_LINK_STATE_TRAINED) { - OCAPIINF(dev, "link trained in %lld ms\n", - OCAPI_LINK_TRAINING_TIMEOUT - slot->retries); + OCAPIINF(dev, "link trained in %ld ms\n", + tb_to_msecs(mftb() - dev->train_start)); check_trained_link(dev, reg); pci_slot_set_state(slot, OCAPI_SLOT_LINK_TRAINED); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } - if (slot->retries-- == 0) + if (tb_compare(mftb(), dev->train_timeout) == TB_AAFTERB) return npu2_opencapi_retry_state(slot, reg); return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); @@ -1239,7 +1239,8 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) /* Bump lanes - this improves training reliability */ npu2_opencapi_bump_ui_lane(dev); start_training(chip_id, dev); - slot->retries = OCAPI_LINK_TRAINING_TIMEOUT; + dev->train_start = mftb(); + dev->train_timeout = dev->train_start + msecs_to_tb(OCAPI_LINK_TRAINING_TIMEOUT); pci_slot_set_state(slot, OCAPI_SLOT_LINK_START); return slot->ops.poll_link(slot); diff --git a/include/npu2.h b/include/npu2.h index aac7e7a5..d2316dc1 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -147,6 +147,8 @@ struct npu2_dev { uint64_t linux_pe; bool train_need_fence; bool train_fenced; + unsigned long train_start; + unsigned long train_timeout; }; struct npu2 { From patchwork Mon Sep 9 12:31:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159703 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RnfS1nDDz9sNF for ; Mon, 9 Sep 2019 22:34:24 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RnfS0WGtzDqNV for ; Mon, 9 Sep 2019 22:34:24 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbk56xmzDqNQ for ; Mon, 9 Sep 2019 22:32:02 +1000 (AEST) Received: from pps.filterd (m0098396.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMp28076886 for ; Mon, 9 Sep 2019 08:32:00 -0400 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwpmugk86-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:00 -0400 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:58 +0100 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:56 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVtOc36766064 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:55 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4E8DDAE045; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1D9C7AE055; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:45 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0020-0000-0000-00000369C43E X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0021-0000-0000-000021BF453C Message-Id: <20190909123151.21944-11-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 10/16] npu2-opencapi: Tweak fundamental reset sequence X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Modify slightly the ordering of a few steps in our init sequence on fundamental reset, so that it can be called from the OS, when the link is already up: - when the card is reset, the link goes down, so we need to fence the brick to prevent errors propagating to the NPU and OS - since fencing and unfencing don't require any delay, let's also fence/unfence during the very first reset at boot. It's useless but doesn't hurt and keep the code simpler. - resetting the PHY must be done a bit later, while fenced and the ODL and DLx in reset Signed-off-by: Frederic Barrat --- hw/npu2-opencapi.c | 48 +++++++++++++++++++++++++--------------------- include/npu2.h | 2 -- 2 files changed, 26 insertions(+), 24 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index f7be9f09..619d4be8 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1039,6 +1039,28 @@ static int64_t npu2_opencapi_get_presence_state(struct pci_slot __unused *slot, return OPAL_SUCCESS; } +static void fence_brick(struct npu2_dev *dev) +{ + OCAPIDBG(dev, "Fencing brick\n"); + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b11); + /* from 13.2.1, Quiesce Fence State */ + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, + PPC_BIT(dev->brick_index + 6)); +} + +static void unfence_brick(struct npu2_dev *dev) +{ + OCAPIDBG(dev, "Unfencing brick\n"); + npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, + PPC_BIT(dev->brick_index)); + + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b10); + set_fence_control(dev->npu->chip_id, dev->npu->xscom_base, + dev->brick_index, 0b00); +} + static enum OpalShpcLinkState get_link_width(uint64_t odl_status) { uint64_t tx_lanes, rx_lanes, state; @@ -1153,7 +1175,7 @@ static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) return OPAL_HARDWARE; } -static int64_t npu2_opencapi_creset(struct pci_slot *slot __unused) +static int64_t npu2_opencapi_creset(struct pci_slot *slot) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -1183,19 +1205,10 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } - if (dev->train_need_fence) { - OCAPIDBG(dev, "Fencing OTL during reset\n"); - set_fence_control(chip_id, dev->npu->xscom_base, - dev->brick_index, 0b11); - npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, - PPC_BIT(dev->brick_index + 6)); - dev->train_fenced = true; - } - dev->train_need_fence = true; slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; - npu2_opencapi_phy_reset(dev); /* fall-through */ case OCAPI_SLOT_FRESET_INIT: + fence_brick(dev); assert_odl_reset(chip_id, dev->brick_index); assert_adapter_reset(dev); pci_slot_set_state(slot, @@ -1204,6 +1217,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); case OCAPI_SLOT_FRESET_ASSERT_DELAY: + npu2_opencapi_phy_reset(dev); deassert_odl_reset(chip_id, dev->brick_index); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_DEASSERT_DELAY); @@ -1221,15 +1235,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(250)); case OCAPI_SLOT_FRESET_DEASSERT_DELAY2: - if (dev->train_fenced) { - OCAPIDBG(dev, "Unfencing OTL after reset\n"); - npu2_write(dev->npu, NPU2_MISC_FENCE_STATE, - PPC_BIT(dev->brick_index)); - set_fence_control(chip_id, dev->npu->xscom_base, - dev->brick_index, 0b00); - dev->train_fenced = false; - } - + unfence_brick(dev); set_init_pattern(chip_id, dev); pci_slot_set_state(slot, OCAPI_SLOT_FRESET_INIT_DELAY); @@ -1692,8 +1698,6 @@ static void setup_device(struct npu2_dev *dev) dev->bdfn = 0; dev->linux_pe = -1; - dev->train_need_fence = false; - dev->train_fenced = false; /* TODO: Procedure 13.1.3.7 - AFU Memory Range BARs */ /* Procedure 13.1.3.8 - AFU MMIO Range BARs */ diff --git a/include/npu2.h b/include/npu2.h index d2316dc1..6b1063da 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -145,8 +145,6 @@ struct npu2_dev { /* OpenCAPI */ struct phb phb_ocapi; uint64_t linux_pe; - bool train_need_fence; - bool train_fenced; unsigned long train_start; unsigned long train_timeout; }; From patchwork Mon Sep 9 12:31:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rnj02Ldxz9s7T for ; Mon, 9 Sep 2019 22:36:36 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rnhz2slszDqFS for ; Mon, 9 Sep 2019 22:36:35 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbp0ghWzDqFC for ; Mon, 9 Sep 2019 22:32:05 +1000 (AEST) Received: from pps.filterd (m0127361.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMKe4096227 for ; Mon, 9 Sep 2019 08:32:02 -0400 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uv7c2w9d2-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:01 -0400 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06avi18878370.portsmouth.uk.ibm.com (9.149.26.194) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:57 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVtD939911744 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:55 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9F1E7AE056; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 655C9AE04D; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:46 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0012-0000-0000-00000348C3E9 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0013-0000-0000-000021832413 Message-Id: <20190909123151.21944-12-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=911 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 11/16] npu2-opencapi: Simplify freset states X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Let's get rid of one transitional state, since there's no need to pause in between releasing the reset signals of the ODL and the adapter. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 619d4be8..ead6f5fa 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -53,8 +53,7 @@ #define OCAPI_SLOT_FRESET_INIT (OCAPI_SLOT_FRESET + 2) #define OCAPI_SLOT_FRESET_ASSERT_DELAY (OCAPI_SLOT_FRESET + 3) #define OCAPI_SLOT_FRESET_DEASSERT_DELAY (OCAPI_SLOT_FRESET + 4) -#define OCAPI_SLOT_FRESET_DEASSERT_DELAY2 (OCAPI_SLOT_FRESET + 5) -#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 6) +#define OCAPI_SLOT_FRESET_INIT_DELAY (OCAPI_SLOT_FRESET + 5) #define OCAPI_LINK_TRAINING_RETRIES 2 #define OCAPI_LINK_TRAINING_TIMEOUT 3000 /* ms */ @@ -1219,22 +1218,13 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) case OCAPI_SLOT_FRESET_ASSERT_DELAY: npu2_opencapi_phy_reset(dev); deassert_odl_reset(chip_id, dev->brick_index); - pci_slot_set_state(slot, - OCAPI_SLOT_FRESET_DEASSERT_DELAY); - /* - * Minimal delay before taking adapter out of - * reset. Could be useless, but doesn't hurt - */ - return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); - - case OCAPI_SLOT_FRESET_DEASSERT_DELAY: deassert_adapter_reset(dev); pci_slot_set_state(slot, - OCAPI_SLOT_FRESET_DEASSERT_DELAY2); + OCAPI_SLOT_FRESET_DEASSERT_DELAY); /* give 250ms to device to be ready */ return pci_slot_set_sm_timeout(slot, msecs_to_tb(250)); - case OCAPI_SLOT_FRESET_DEASSERT_DELAY2: + case OCAPI_SLOT_FRESET_DEASSERT_DELAY: unfence_brick(dev); set_init_pattern(chip_id, dev); pci_slot_set_state(slot, From patchwork Mon Sep 9 12:31:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159715 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RnjL6wpLz9s7T for ; Mon, 9 Sep 2019 22:36:54 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RnjL5DDLzDqNg for ; Mon, 9 Sep 2019 22:36:54 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbp71HgzDqFC for ; Mon, 9 Sep 2019 22:32:06 +1000 (AEST) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNYZn100427 for ; Mon, 9 Sep 2019 08:32:04 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 2uwm9fqdww-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:04 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:32:00 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:57 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVull54263838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:56 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DD8FCAE058; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ACA6FAE045; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:47 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-4275-0000-0000-000003633702 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-4276-0000-0000-000038758756 Message-Id: <20190909123151.21944-13-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 12/16] npu2-opencapi: Detect PHY reset errors X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" PHY reset can fail! Though past problems are now fixed, let's handle any future failure. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-hw-procedures.c | 13 ++++++++++--- hw/npu2-opencapi.c | 5 ++++- include/npu2.h | 2 +- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/hw/npu2-hw-procedures.c b/hw/npu2-hw-procedures.c index ad1627ae..8379cbbe 100644 --- a/hw/npu2-hw-procedures.c +++ b/hw/npu2-hw-procedures.c @@ -1041,10 +1041,17 @@ void npu2_opencapi_phy_init(struct npu2_dev *dev) } } -void npu2_opencapi_phy_reset(struct npu2_dev *dev) +int npu2_opencapi_phy_reset(struct npu2_dev *dev) { - run_procedure(dev, 4); /* procedure_phy_reset */ - run_procedure(dev, 6); /* procedure_phy_rx_dccal */ + int rc; + + rc = run_procedure(dev, 4); /* procedure_phy_reset */ + if (rc != PROCEDURE_COMPLETE) + return -1; + rc = run_procedure(dev, 6); /* procedure_phy_rx_dccal */ + if (rc != PROCEDURE_COMPLETE) + return -1; + return 0; } void npu2_opencapi_phy_prbs31(struct npu2_dev *dev) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index ead6f5fa..efec162d 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1187,6 +1187,7 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); uint32_t chip_id = dev->npu->chip_id; uint8_t presence = 1; + int rc; switch (slot->state) { case OCAPI_SLOT_NORMAL: @@ -1216,7 +1217,9 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) return pci_slot_set_sm_timeout(slot, msecs_to_tb(5)); case OCAPI_SLOT_FRESET_ASSERT_DELAY: - npu2_opencapi_phy_reset(dev); + rc = npu2_opencapi_phy_reset(dev); + if (rc) + return OPAL_HARDWARE; deassert_odl_reset(chip_id, dev->brick_index); deassert_adapter_reset(dev); pci_slot_set_state(slot, diff --git a/include/npu2.h b/include/npu2.h index 6b1063da..6171cd3c 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -234,7 +234,7 @@ void npu2_clear_link_flag(struct npu2_dev *ndev, uint8_t flag); uint32_t reset_ntl(struct npu2_dev *ndev); extern int nv_zcal_nominal; void npu2_opencapi_phy_init(struct npu2_dev *dev); -void npu2_opencapi_phy_reset(struct npu2_dev *dev); +int npu2_opencapi_phy_reset(struct npu2_dev *dev); void npu2_opencapi_phy_prbs31(struct npu2_dev *dev); void npu2_opencapi_bump_ui_lane(struct npu2_dev *dev); int64_t npu2_freeze_status(struct phb *phb __unused, From patchwork Mon Sep 9 12:31:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159706 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RngS1LzQz9s7T for ; Mon, 9 Sep 2019 22:35:16 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RngR6krbzDqFw for ; Mon, 9 Sep 2019 22:35:15 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbm5npVzDqHR for ; Mon, 9 Sep 2019 22:32:04 +1000 (AEST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMaSJ089384 for ; Mon, 9 Sep 2019 08:32:02 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwn944erj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:02 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:58 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVu3i58720482 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:56 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 364F4AE05D; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 00BEAAE061; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:55 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:48 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0016-0000-0000-000002A8C556 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0017-0000-0000-0000330946CC Message-Id: <20190909123151.21944-14-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 13/16] npu2-opencapi: Improve error reporting to the OS X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" When resetting an opencapi link, the brick will be fenced temporarily. Therefore we can't rely on the fencing state of the brick any more to check for the health of an opencapi PHB, as we could report errors if queried for a PHB state at the same time a link is being reset. Instead, we flag the device as 'broken' when an error interrupt is received, just before raising an event to the OS. When the OS is querying for the state of a PHB, we only have to look at the 'broken' attribute. Note that there's no recovery possible on P9 when an error interrupt is received unexpectedly, as recovery is not supported by hardware. So when a device/link is marked as 'broken', it stays broken. All the OS can do is log the error and notify the drivers. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard --- hw/npu2-common.c | 7 +++++++ hw/npu2-opencapi.c | 21 +++++++++++++++++---- include/npu2.h | 5 +++++ 3 files changed, 29 insertions(+), 4 deletions(-) diff --git a/hw/npu2-common.c b/hw/npu2-common.c index 6d5c35af..51ecd0c8 100644 --- a/hw/npu2-common.c +++ b/hw/npu2-common.c @@ -406,6 +406,13 @@ static void npu2_err_interrupt(struct irq_source *is, uint32_t isn) p->chip_id, irq_name); free(irq_name); show_all_regs(p, brick); + /* + * P9 NPU doesn't support recovering a link going down + * unexpectedly. So we mark the device as broken and + * report it to the OS, so that the error is logged + * and the drivers notified. + */ + npu2_opencapi_set_broken(p, brick); opal_update_pending_evt(OPAL_EVENT_PCI_ERROR, OPAL_EVENT_PCI_ERROR); break; diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index efec162d..c8bc64d1 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1463,14 +1463,12 @@ static int64_t npu2_opencapi_eeh_next_error(struct phb *phb, uint16_t *severity) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(phb); - uint64_t reg; if (!first_frozen_pe || !pci_error_type || !severity) return OPAL_PARAMETER; - reg = npu2_read(dev->npu, NPU2_MISC_FENCE_STATE); - if (reg & PPC_BIT(dev->brick_index)) { - OCAPIERR(dev, "Brick %d fenced!\n", dev->brick_index); + if (dev->flags & NPU2_DEV_BROKEN) { + OCAPIDBG(dev, "Reporting device as broken\n"); *first_frozen_pe = dev->linux_pe; *pci_error_type = OPAL_EEH_PHB_ERROR; *severity = OPAL_EEH_SEV_PHB_DEAD; @@ -1820,6 +1818,21 @@ static const struct phb_ops npu2_opencapi_ops = { .tce_kill = NULL, }; +void npu2_opencapi_set_broken(struct npu2 *npu, int brick) +{ + struct phb *phb; + struct npu2_dev *dev; + + for_each_phb(phb) { + if (phb->phb_type == phb_type_npu_v2_opencapi) { + dev = phb_to_npu2_dev_ocapi(phb); + if (dev->npu == npu && + dev->brick_index == brick) + dev->flags |= NPU2_DEV_BROKEN; + } + } +} + static int64_t opal_npu_spa_setup(uint64_t phb_id, uint32_t __unused bdfn, uint64_t addr, uint64_t PE_mask) { diff --git a/include/npu2.h b/include/npu2.h index 6171cd3c..d2a3430e 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -118,6 +118,8 @@ struct npu2_dev_nvlink { const char *slot_label; }; +#define NPU2_DEV_BROKEN 0x1 + struct npu2_dev { enum npu2_dev_type type; uint32_t link_index; @@ -126,6 +128,7 @@ struct npu2_dev { struct dt_node *dt_node; struct npu2_pcie_bar bars[2]; struct npu2 *npu; + long flags; uint32_t bdfn; @@ -251,4 +254,6 @@ int64_t npu2_map_lpar(struct phb *phb, uint64_t bdf, uint64_t lparid, int64_t npu2_set_relaxed_order(struct phb *phb, uint32_t gcid, int pec, bool enable); +void npu2_opencapi_set_broken(struct npu2 *npu, int brick); + #endif /* __NPU2_H */ From patchwork Mon Sep 9 12:31:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159705 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46Rng839Tnz9s7T for ; Mon, 9 Sep 2019 22:35:00 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46Rng81NgTzDqNW for ; Mon, 9 Sep 2019 22:35:00 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbm5zwTzDqNC for ; Mon, 9 Sep 2019 22:32:04 +1000 (AEST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CMbEQ089416 for ; Mon, 9 Sep 2019 08:32:02 -0400 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwn944er3-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:01 -0400 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:31:59 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:58 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVu3B32374966 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:56 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7C5ABAE056; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4CE60AE058; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:49 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0008-0000-0000-000003132185 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0009-0000-0000-00004A3184E5 Message-Id: <20190909123151.21944-15-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 14/16] npu2-opencapi: Activate PCI hotplug on opencapi slot X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Implement the get_power_state() and set_power_state() callbacks for the opencapi slot and add properties in the device tree to mark the opencapi slot as hot-pluggable. We don't really power off/on the opencapi adapter. The slot at play here is the virtual slot associated to the virtual opencapi PHB. The real PCIe slot where the card is drawing its power from is untouched (skiboot is not even aware which PCIe slot the card is seated on). So the 'fake' power off is fencing the card and set it in reset so that the FPGA image can be updated. The 'fake' power on is not doing much, as the unfencing happens on the subsequent link training. Opencapi slots are named 'OPENCAPI-xxxx' where xxxx is the opal ID of the PHB/slot. This is meant to easily identify the slot used by an AFU device, as the AFU device names are also built around that ID. For example, the device /dev/ocxl/AFP3.0006:00:00.1.0 uses the slot OPENCAPI-0006. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard --- hw/npu2-opencapi.c | 69 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 65 insertions(+), 4 deletions(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index c8bc64d1..af309362 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1032,9 +1032,10 @@ static int64_t npu2_opencapi_get_presence_state(struct pci_slot __unused *slot, * As such we will never be asked to get the presence of a slot that's * empty. * - * This may change if we ever support hotplug down the track. + * This may change if we ever support surprise hotplug down + * the track. */ - *val = true; + *val = OPAL_PCI_SLOT_PRESENT; return OPAL_SUCCESS; } @@ -1092,6 +1093,38 @@ static int64_t npu2_opencapi_get_link_state(struct pci_slot *slot, uint8_t *val) return OPAL_SUCCESS; } +static int64_t npu2_opencapi_get_power_state(struct pci_slot *slot, + uint8_t *val) +{ + *val = slot->power_state; + return OPAL_SUCCESS; +} + +static int64_t npu2_opencapi_set_power_state(struct pci_slot *slot, uint8_t val) +{ + struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); + + switch (val) { + case PCI_SLOT_POWER_OFF: + OCAPIDBG(dev, "Fake power off\n"); + fence_brick(dev); + assert_adapter_reset(dev); + slot->power_state = PCI_SLOT_POWER_OFF; + return OPAL_SUCCESS; + + case PCI_SLOT_POWER_ON: + if (slot->power_state != PCI_SLOT_POWER_OFF) + return OPAL_SUCCESS; + OCAPIDBG(dev, "Fake power on\n"); + slot->power_state = PCI_SLOT_POWER_ON; + slot->state = OCAPI_SLOT_NORMAL; + return OPAL_SUCCESS; + + default: + return OPAL_UNSUPPORTED; + } +} + static void check_trained_link(struct npu2_dev *dev, uint64_t odl_status) { if (get_link_width(odl_status) != OPAL_SHPC_LINK_UP_x8) { @@ -1132,6 +1165,14 @@ static int64_t npu2_opencapi_retry_state(struct pci_slot *slot, return pci_slot_set_sm_timeout(slot, msecs_to_tb(1)); } +static void npu2_opencapi_prepare_link_change(struct pci_slot *slot __unused, + bool up __unused) +{ + /* + * PCI hotplug wants it defined, but we don't need to do anything + */ +} + static int64_t npu2_opencapi_poll_link(struct pci_slot *slot) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(slot->phb); @@ -1259,6 +1300,24 @@ static int64_t npu2_opencapi_hreset(struct pci_slot *slot __unused) return OPAL_UNSUPPORTED; } +static void make_slot_hotpluggable(struct pci_slot *slot, struct phb *phb) +{ + char label[40]; + + /* + * Add a few definitions to the DT so that the linux PCI + * hotplug framework can find the slot and identify it as + * hot-pluggable. + * + * The "ibm,slot-label" property is used by linux as the slot name + */ + slot->pluggable = 1; + pci_slot_add_dt_properties(slot, phb->dt_node); + snprintf(label, sizeof(label), "OPENCAPI-%04x", + (int)PCI_SLOT_PHB_INDEX(slot->id)); + dt_add_property_string(phb->dt_node, "ibm,slot-label", label); +} + static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) { struct pci_slot *slot; @@ -1270,17 +1329,19 @@ static struct pci_slot *npu2_opencapi_slot_create(struct phb *phb) /* TODO: Figure out other slot functions */ slot->ops.get_presence_state = npu2_opencapi_get_presence_state; slot->ops.get_link_state = npu2_opencapi_get_link_state; - slot->ops.get_power_state = NULL; + slot->ops.get_power_state = npu2_opencapi_get_power_state; slot->ops.get_attention_state = NULL; slot->ops.get_latch_state = NULL; - slot->ops.set_power_state = NULL; + slot->ops.set_power_state = npu2_opencapi_set_power_state; slot->ops.set_attention_state = NULL; + slot->ops.prepare_link_change = npu2_opencapi_prepare_link_change; slot->ops.poll_link = npu2_opencapi_poll_link; slot->ops.creset = npu2_opencapi_creset; slot->ops.freset = npu2_opencapi_freset; slot->ops.hreset = npu2_opencapi_hreset; + make_slot_hotpluggable(slot, phb); return slot; } From patchwork Mon Sep 9 12:31:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159709 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RnhY2lSGz9sCJ for ; Mon, 9 Sep 2019 22:36:13 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RnhY098FzDqP7 for ; Mon, 9 Sep 2019 22:36:13 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbn2ZCVzDqNQ for ; Mon, 9 Sep 2019 22:32:05 +1000 (AEST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNfZl121425 for ; Mon, 9 Sep 2019 08:32:02 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwm297yvj-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:02 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:32:00 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:58 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVuau55836856 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:57 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D5C60AE045; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 92D7BAE05A; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:50 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0016-0000-0000-000002A8C558 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0017-0000-0000-0000330946CE Message-Id: <20190909123151.21944-16-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=841 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 15/16] npu2-opencapi: Handle OPAL_UNMAP_PE operation on set_pe() callback X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" In a hot-unplug scenario, the OS will try to unmap the PE. Skiboot doesn't do anything with the linux PE for opencapi other than being a mailbox, but at least let's be consistent. Signed-off-by: Frederic Barrat Reviewed-by: Christophe Lombard Reviewed-by: Andrew Donnellan --- hw/npu2-opencapi.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index af309362..46aeb6d3 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1488,7 +1488,7 @@ static int64_t npu2_opencapi_set_pe(struct phb *phb, uint8_t __unused bcompare, uint8_t __unused dcompare, uint8_t __unused fcompare, - uint8_t __unused action) + uint8_t action) { struct npu2_dev *dev = phb_to_npu2_dev_ocapi(phb); /* @@ -1500,6 +1500,8 @@ static int64_t npu2_opencapi_set_pe(struct phb *phb, * functions on the device, the OS can define many PEs, we * only keep one, the OS will handle it. */ + if (action == OPAL_UNMAP_PE) + pe_num = -1; dev->linux_pe = pe_num; return OPAL_SUCCESS; } From patchwork Mon Sep 9 12:31:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1159708 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46RnhB0y91z9s7T for ; Mon, 9 Sep 2019 22:35:54 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 46RnhB00jDzDqNZ for ; Mon, 9 Sep 2019 22:35:54 +1000 (AEST) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 46Rnbn2kvyzDqNR for ; Mon, 9 Sep 2019 22:32:05 +1000 (AEST) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x89CNcQq121364 for ; Mon, 9 Sep 2019 08:32:03 -0400 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2uwm297yw2-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 09 Sep 2019 08:32:02 -0400 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 9 Sep 2019 13:32:01 +0100 Received: from b06cxnps3075.portsmouth.uk.ibm.com (9.149.109.195) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 9 Sep 2019 13:31:58 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x89CVvpo57999368 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 9 Sep 2019 12:31:57 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2BA1FAE051; Mon, 9 Sep 2019 12:31:57 +0000 (GMT) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E32A6AE04D; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) Received: from pic2.home (unknown [9.145.178.189]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTP; Mon, 9 Sep 2019 12:31:56 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, oohall@gmail.com Date: Mon, 9 Sep 2019 14:31:51 +0200 X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190909123151.21944-1-fbarrat@linux.ibm.com> References: <20190909123151.21944-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19090912-0028-0000-0000-00000399BF3D X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19090912-0029-0000-0000-0000245C1E8F Message-Id: <20190909123151.21944-17-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-09-09_05:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=742 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1909090127 Subject: [Skiboot] [PATCH 16/16] npu2-opencapi: Log a warning when resetting a broken device X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: arbab@linux.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" On P9, the NPU doesn't support recovery if the link goes down unexpectedly. It was not fully verified. We mark the device as broken when we receive an error interrupt from the NPU. However, there's nothing to prevent the OS from trying to reset the device; It may or may not work, it's unsupported territory, so let's log a message to make it clear, as it could help when debugging. We haven't hit any cases where the reset goes badly enough that we'd want to prevent it, so let it go for now. We can revisit later if we have evidence that it's causing more problems than it is worth. Signed-off-by: Frederic Barrat --- hw/npu2-opencapi.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 46aeb6d3..f044fdbf 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1246,6 +1246,10 @@ static int64_t npu2_opencapi_freset(struct pci_slot *slot) OCAPIINF(dev, "no card detected\n"); return OPAL_SUCCESS; } + if (dev->flags & NPU2_DEV_BROKEN) { + OCAPIERR(dev, "Resetting a device which hit a previous error. Device recovery is not supported, so future behavior is undefined\n"); + dev->flags &= ~NPU2_DEV_BROKEN; + } slot->link_retries = OCAPI_LINK_TRAINING_RETRIES; /* fall-through */ case OCAPI_SLOT_FRESET_INIT: