From patchwork Wed Jan 22 15:03:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1227317 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 482pZh4jR8z9sPn for ; Thu, 23 Jan 2020 02:03:56 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 482pZh3s5yzDqLn for ; Thu, 23 Jan 2020 02:03:56 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 482pZW0TrbzDqHk for ; Thu, 23 Jan 2020 02:03:46 +1100 (AEDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00MEtSO8005554 for ; Wed, 22 Jan 2020 10:03:43 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2xp93pmh8u-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 Jan 2020 10:03:43 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 22 Jan 2020 15:03:41 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 22 Jan 2020 15:03:40 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 00MF2kku16974126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Jan 2020 15:02:47 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2BB684C050; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D14124C04A; Wed, 22 Jan 2020 15:03:36 +0000 (GMT) Received: from pic2.home (unknown [9.145.60.113]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 22 Jan 2020 15:03:36 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, arbab@linux.ibm.com, alistair@popple.id.au, oohall@gmail.com Date: Wed, 22 Jan 2020 16:03:33 +0100 X-Mailer: git-send-email 2.21.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 20012215-4275-0000-0000-00000399F2F1 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20012215-4276-0000-0000-000038ADFE5C Message-Id: <20200122150336.2623-1-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-17_05:2020-01-16, 2020-01-17 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 bulkscore=0 lowpriorityscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 impostorscore=0 priorityscore=1501 phishscore=0 adultscore=0 clxscore=1015 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-2001220134 Subject: [Skiboot] [RESEND PATCH v2 1/4] npu2: Rework phb-index assignments for virtual PHBs X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joy_Chu@wistron.com, chhank@tw.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Until now, opencapi PHBs were not using the 'ibm,phb-index' property, as it was thought unnecessary. For nvlink, a phb-index was associated to the npu when parsing hdat data, and the nvlink PHB was reusing the same value. It turns out it helps to have the 'ibm,phb-index' property for opencapi PHBs after all. Otherwise it can lead to wrong results on platforms like mihawk when trying to match entries in the slot table. We end up with an opencapi device inheriting wrong properties in the device tree, because match_slot_phb_entry() default to phb-index 0 if it cannot find the property. Though it doesn't seem to cause any harm, it's wrong and a future patch is expected to start using the slot table for opencapi, so it needs fixing. The twist is that with opencapi, we can have multiple virtual PHBs for a single NPU on P9. There's one PHB per (opencapi) brick. Therefore there's no 1-to-1 mapping between the NPU and PHB index and it no longer makes sense to associate a phb-index to a npu. With this patch, opencapi PHBs created under a NPU use a fixed mapping for their phb-index, based on the brick index. The range of possible values is 7 to 12. Because there can only be one nvlink PHB per NPU, it is always using a phb-index of 7. A side effect is that 2 virtual PHBs on 2 different chips can have the same phb-index, which is similar to what happens for 'real' PCI PHBs, but is different from what was happening on a nvlink-only witherspoon so far. Reviewed-by: Reza Arbab Reviewed-by: Andrew Donnellan Signed-off-by: Frederic Barrat --- Resending series after rebasing to current master, since it was no longer applying cleanly. I've picked up the Reviewed-by's from v2. No code change. Changelog: v2: - add comment summarizing the commit msg in the code (Andrew) - move the axone changes to separate patch hw/npu2-common.c | 1 - hw/npu2-opencapi.c | 2 ++ hw/npu2.c | 2 +- include/npu2.h | 20 +++++++++++++++++++- 4 files changed, 22 insertions(+), 3 deletions(-) diff --git a/hw/npu2-common.c b/hw/npu2-common.c index 51ecd0c8..4e5b104e 100644 --- a/hw/npu2-common.c +++ b/hw/npu2-common.c @@ -537,7 +537,6 @@ static struct npu2 *setup_npu(struct dt_node *dn) npu->index = dt_prop_get_u32(dn, "ibm,npu-index"); npu->chip_id = gcid; npu->xscom_base = dt_get_address(dn, 0, NULL); - npu->phb_index = dt_prop_get_u32(dn, "ibm,phb-index"); init_lock(&npu->i2c_lock); npu->i2c_pin_mode = ~0; // input mode by default diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index 19589c92..133d78b9 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -1734,6 +1734,8 @@ static void setup_device(struct npu2_dev *dev) dt_add_property_strings(dn_phb, "device_type", "pciex"); dt_add_property(dn_phb, "reg", mm_win, sizeof(mm_win)); dt_add_property_cells(dn_phb, "ibm,npu-index", dev->npu->index); + dt_add_property_cells(dn_phb, "ibm,phb-index", + npu2_get_phb_index(dev->brick_index)); dt_add_property_cells(dn_phb, "ibm,chip-id", dev->npu->chip_id); dt_add_property_cells(dn_phb, "ibm,xscom-base", dev->npu->xscom_base); dt_add_property_cells(dn_phb, "ibm,npcq", dev->npu->dt_node->phandle); diff --git a/hw/npu2.c b/hw/npu2.c index aeb7c49e..be442baa 100644 --- a/hw/npu2.c +++ b/hw/npu2.c @@ -1484,7 +1484,7 @@ int npu2_nvlink_init_npu(struct npu2 *npu) "ibm,ioda2-npu2-phb"); dt_add_property_strings(np, "device_type", "pciex"); dt_add_property(np, "reg", reg, sizeof(reg)); - dt_add_property_cells(np, "ibm,phb-index", npu->phb_index); + dt_add_property_cells(np, "ibm,phb-index", npu2_get_phb_index(0)); dt_add_property_cells(np, "ibm,npu-index", npu->index); dt_add_property_cells(np, "ibm,chip-id", npu->chip_id); dt_add_property_cells(np, "ibm,xscom-base", npu->xscom_base); diff --git a/include/npu2.h b/include/npu2.h index d2a3430e..309a7c7f 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -178,7 +178,6 @@ struct npu2 { /* NVLink */ struct phb phb_nvlink; - uint32_t phb_index; /* OCAPI */ uint64_t i2c_port_id_ocapi; @@ -256,4 +255,23 @@ int64_t npu2_set_relaxed_order(struct phb *phb, uint32_t gcid, int pec, void npu2_opencapi_set_broken(struct npu2 *npu, int brick); +#define NPU2_PHB_INDEX_BASE 7 +/* to avoid conflicts with PCI and for historical reasons */ + +static inline int npu2_get_phb_index(unsigned int brick_index) +{ + /* + * There's one virtual PHB per brick with opencapi, so we no + * longer have a 1-to-1 mapping between a NPU and a virtual + * PHB. And we want a static phb-index, as it is needed to use + * a slot table on some platforms. So we associate a per-chip + * phb-index based on the brick index. + * + * nvlink only creates one virtual PHB per chip, so it is + * treated as if using brick 0, which is never used by + * opencapi. + */ + return NPU2_PHB_INDEX_BASE + brick_index; +} + #endif /* __NPU2_H */ From patchwork Wed Jan 22 15:03:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1227318 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 482pbD0stcz9sPn for ; Thu, 23 Jan 2020 02:04:24 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 482pbC6bKGzDqNw for ; Thu, 23 Jan 2020 02:04:23 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 482pZW4jQQzDqJ3 for ; Thu, 23 Jan 2020 02:03:47 +1100 (AEDT) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00MEtILi049576 for ; Wed, 22 Jan 2020 10:03:45 -0500 Received: from e06smtp07.uk.ibm.com (e06smtp07.uk.ibm.com [195.75.94.103]) by mx0a-001b2d01.pphosted.com with ESMTP id 2xp3u78c0x-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 Jan 2020 10:03:44 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 22 Jan 2020 15:03:42 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp07.uk.ibm.com (192.168.101.137) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 22 Jan 2020 15:03:40 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 00MF2lHB33161552 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Jan 2020 15:02:47 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8405B4C046; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 387314C05A; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from pic2.home (unknown [9.145.60.113]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, arbab@linux.ibm.com, alistair@popple.id.au, oohall@gmail.com Date: Wed, 22 Jan 2020 16:03:34 +0100 X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200122150336.2623-1-fbarrat@linux.ibm.com> References: <20200122150336.2623-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 20012215-0028-0000-0000-000003D3715F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20012215-0029-0000-0000-00002497A97E Message-Id: <20200122150336.2623-2-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-17_05:2020-01-16, 2020-01-17 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 bulkscore=0 adultscore=0 mlxlogscore=799 lowpriorityscore=0 mlxscore=0 spamscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 suspectscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-2001220134 Subject: [Skiboot] [RESEND PATCH v2 2/4] npu3: Don't use the device tree to assign the phb-index of the PHB X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joy_Chu@wistron.com, chhank@tw.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" On Axone, there's a 1-to-1 mapping between virtual PHBs and NPUs. We could keep assigning the phb-index of the virtual PHB from the value found in the npu node of the device tree, but to be consistent with P9/npu2 and avoid confusion, this patch assigns the phb-index when the virtual PHB is created, based on the npu index, similarly to what we do on P9. Reviewed-by: Reza Arbab Reviewed-by: Andrew Donnellan Signed-off-by: Frederic Barrat --- Changelog; v2: - new patch: separate the axone bits from the first patch hw/npu3-nvlink.c | 2 +- include/npu3.h | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/hw/npu3-nvlink.c b/hw/npu3-nvlink.c index 96301589..b8856ddc 100644 --- a/hw/npu3-nvlink.c +++ b/hw/npu3-nvlink.c @@ -1478,7 +1478,7 @@ static void npu3_dt_add_props(struct npu3 *npu) "ibm,ioda2-npu3-phb"); dt_add_property_cells(dn, "ibm,phb-index", - dt_prop_get_u32(npu->dt_node, "ibm,phb-index")); + npu3_get_phb_index(npu->index)); dt_add_property_cells(dn, "ibm,phb-diag-data-size", 0); dt_add_property_cells(dn, "ibm,opal-num-pes", NPU3_MAX_PE_NUM); dt_add_property_cells(dn, "ibm,opal-reserved-pe", NPU3_RESERVED_PE_NUM); diff --git a/include/npu3.h b/include/npu3.h index 1c657f94..0fdad4df 100644 --- a/include/npu3.h +++ b/include/npu3.h @@ -177,4 +177,10 @@ int64_t npu3_map_lpar(struct phb *phb, uint64_t bdf, uint64_t lparid, int64_t npu3_set_relaxed_order(struct phb *phb, uint32_t gcid, int pec, bool enable); +#define NPU3_PHB_INDEX_BASE 6 /* immediately after real PHBs */ +static inline int npu3_get_phb_index(unsigned int npu_index) +{ + return NPU3_PHB_INDEX_BASE + npu_index; +} + #endif /* __NPU3_H */ From patchwork Wed Jan 22 15:03:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1227319 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 482pbW6kfBz9sPn for ; Thu, 23 Jan 2020 02:04:39 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 482pbW54bKzDqL9 for ; Thu, 23 Jan 2020 02:04:39 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 482pZY0y5TzDqHk for ; Thu, 23 Jan 2020 02:03:48 +1100 (AEDT) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00MEtInB007814 for ; Wed, 22 Jan 2020 10:03:46 -0500 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0b-001b2d01.pphosted.com with ESMTP id 2xp5yf266f-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 Jan 2020 10:03:45 -0500 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 22 Jan 2020 15:03:43 -0000 Received: from b06avi18626390.portsmouth.uk.ibm.com (9.149.26.192) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 22 Jan 2020 15:03:41 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06avi18626390.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 00MF2l0I48824708 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Jan 2020 15:02:47 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF5554C044; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 91D334C059; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from pic2.home (unknown [9.145.60.113]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, arbab@linux.ibm.com, alistair@popple.id.au, oohall@gmail.com Date: Wed, 22 Jan 2020 16:03:35 +0100 X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200122150336.2623-1-fbarrat@linux.ibm.com> References: <20200122150336.2623-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 20012215-0020-0000-0000-000003A30725 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20012215-0021-0000-0000-000021FA9C4E Message-Id: <20200122150336.2623-3-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-17_05:2020-01-16, 2020-01-17 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 phishscore=0 malwarescore=0 mlxscore=0 suspectscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-2001220134 Subject: [Skiboot] [RESEND PATCH v2 3/4] npu2, npu3: Remove ibm, phb-index property from the NPU dt node X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joy_Chu@wistron.com, chhank@tw.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" The 'ibm,phb-index' property of the NPU node is now useless, as we can have multiple PHBs associated to the same NPU on P9. Let's remove it to avoid confusion. Reviewed-by: Reza Arbab Reviewed-by: Andrew Donnellan Signed-off-by: Frederic Barrat --- Changelog: v2: no change doc/device-tree/opencapi.rst | 1 - hdata/spira.c | 6 ++---- hdata/test/op920.wsp.dts | 2 -- hw/npu3.c | 1 - platforms/astbmc/mihawk.c | 2 -- platforms/astbmc/zaius.c | 2 -- platforms/ibm-fsp/zz.c | 2 -- 7 files changed, 2 insertions(+), 14 deletions(-) diff --git a/doc/device-tree/opencapi.rst b/doc/device-tree/opencapi.rst index 80ff9963..497654ac 100644 --- a/doc/device-tree/opencapi.rst +++ b/doc/device-tree/opencapi.rst @@ -19,7 +19,6 @@ NVLink links are currently unsupported. npu@5011000 { compatible = "ibm,power9-npu"; phandle = <0xe6>; - ibm,phb-index = <0x7>; reg = <0x5011000 0x2c>; ibm,npu-index = <0x0>; ibm,npu-links = <0x2>; /* Number of links wired up to this npu. */ diff --git a/hdata/spira.c b/hdata/spira.c index 77c937b1..7c5918d2 100644 --- a/hdata/spira.c +++ b/hdata/spira.c @@ -1451,7 +1451,7 @@ static void add_stop_levels(void) #define NPU_INDIRECT1 0x800000000c010c3fULL static void add_npu(struct dt_node *xscom, const struct HDIF_array_hdr *links, - int npu_index, int phb_index) + int npu_index) { const struct sppcrd_smp_link *link; struct dt_node *npu; @@ -1472,7 +1472,6 @@ static void add_npu(struct dt_node *xscom, const struct HDIF_array_hdr *links, dt_add_property_cells(npu, "#address-cells", 1); dt_add_property_strings(npu, "compatible", "ibm,power9-npu"); - dt_add_property_cells(npu, "ibm,phb-index", phb_index); dt_add_property_cells(npu, "ibm,npu-index", npu_index); HDIF_iarray_for_each(links, i, link) { @@ -1622,7 +1621,6 @@ static void add_npu(struct dt_node *xscom, const struct HDIF_array_hdr *links, static void add_npus(void) { struct dt_node *xscom; - int phb_index = 7; /* Start counting from 7, for no reason */ int npu_index = 0; /* Only consult HDAT for npu2 */ @@ -1647,7 +1645,7 @@ static void add_npus(void) /* some hostboots will give us an empty array */ if (be32_to_cpu(links->ecnt)) - add_npu(xscom, links, npu_index++, phb_index++); + add_npu(xscom, links, npu_index++); } } diff --git a/hdata/test/op920.wsp.dts b/hdata/test/op920.wsp.dts index e26fa127..f9ead923 100644 --- a/hdata/test/op920.wsp.dts +++ b/hdata/test/op920.wsp.dts @@ -4065,7 +4065,6 @@ #size-cells = <0x0>; #address-cells = <0x1>; compatible = "ibm,power9-npu"; - ibm,phb-index = <0x7>; ibm,npu-index = <0x0>; ibm,npu-links = <0x6>; @@ -4695,7 +4694,6 @@ #size-cells = <0x0>; #address-cells = <0x1>; compatible = "ibm,power9-npu"; - ibm,phb-index = <0x8>; ibm,npu-index = <0x1>; ibm,npu-links = <0x6>; diff --git a/hw/npu3.c b/hw/npu3.c index 4d396b09..65f505a6 100644 --- a/hw/npu3.c +++ b/hw/npu3.c @@ -101,7 +101,6 @@ static void npu3_dt_create_npu(struct dt_node *xscom, uint32_t npu_index) dt_add_property_cells(npu, "reg", npu_base[npu_index], 0x2c); dt_add_property_string(npu, "compatible", "ibm,power9-npu3"); dt_add_property_cells(npu, "ibm,npu-index", npu_index); - dt_add_property_cells(npu, "ibm,phb-index", 7 + npu_index); for (uint32_t i = 0; i < NPU3_LINKS_PER_NPU; i++) npu3_dt_create_link(npu, npu_index, i); diff --git a/platforms/astbmc/mihawk.c b/platforms/astbmc/mihawk.c index 0491887e..6816accd 100644 --- a/platforms/astbmc/mihawk.c +++ b/platforms/astbmc/mihawk.c @@ -184,7 +184,6 @@ static void mihawk_create_npu(void) { struct dt_node *xscom, *npu; int npu_index = 0; - int phb_index = 7; char namebuf[32]; /* Return if there's already an NPU in the device tree */ @@ -198,7 +197,6 @@ static void mihawk_create_npu(void) dt_add_property_cells(npu, "reg", NPU_BASE, NPU_SIZE); dt_add_property_strings(npu, "compatible", "ibm,power9-npu"); dt_add_property_cells(npu, "ibm,npu-index", npu_index++); - dt_add_property_cells(npu, "ibm,phb-index", phb_index++); dt_add_property_cells(npu, "ibm,npu-links", 2); create_link(npu, 1, 2); create_link(npu, 2, 3); diff --git a/platforms/astbmc/zaius.c b/platforms/astbmc/zaius.c index ab355888..36dee632 100644 --- a/platforms/astbmc/zaius.c +++ b/platforms/astbmc/zaius.c @@ -157,7 +157,6 @@ static void zaius_create_npu(void) { struct dt_node *xscom, *npu; int npu_index = 0; - int phb_index = 7; char namebuf[32]; /* Abort if there's already an NPU in the device tree */ @@ -171,7 +170,6 @@ static void zaius_create_npu(void) dt_add_property_cells(npu, "reg", NPU_BASE, NPU_SIZE); dt_add_property_strings(npu, "compatible", "ibm,power9-npu"); dt_add_property_cells(npu, "ibm,npu-index", npu_index++); - dt_add_property_cells(npu, "ibm,phb-index", phb_index++); dt_add_property_cells(npu, "ibm,npu-links", 2); create_link(npu, 1, 2); create_link(npu, 2, 3); diff --git a/platforms/ibm-fsp/zz.c b/platforms/ibm-fsp/zz.c index 2bde315c..92f4fcb2 100644 --- a/platforms/ibm-fsp/zz.c +++ b/platforms/ibm-fsp/zz.c @@ -65,7 +65,6 @@ static void add_opencapi_dt_nodes(void) { struct dt_node *npu, *xscom; int npu_index = 0; - int phb_index = 7; /* * In an ideal world, we should get all the NPU links @@ -115,7 +114,6 @@ static void add_opencapi_dt_nodes(void) dt_add_property_cells(npu, "reg", NPU_BASE, NPU_SIZE); dt_add_property_strings(npu, "compatible", "ibm,power9-npu"); dt_add_property_cells(npu, "ibm,npu-index", npu_index++); - dt_add_property_cells(npu, "ibm,phb-index", phb_index++); dt_add_property_cells(npu, "ibm,npu-links", 2); create_link(npu, 1, 2); From patchwork Wed Jan 22 15:03:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frederic Barrat X-Patchwork-Id: 1227320 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 482pbt2CyKz9sPn for ; Thu, 23 Jan 2020 02:04:58 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 482pbt03rhzDqST for ; Thu, 23 Jan 2020 02:04:58 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=fbarrat@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 482pZY360kzDqJ3 for ; Thu, 23 Jan 2020 02:03:48 +1100 (AEDT) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00MEtNiq025453 for ; Wed, 22 Jan 2020 10:03:46 -0500 Received: from e06smtp02.uk.ibm.com (e06smtp02.uk.ibm.com [195.75.94.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 2xp2jfv0w3-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 22 Jan 2020 10:03:45 -0500 Received: from localhost by e06smtp02.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 22 Jan 2020 15:03:43 -0000 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp02.uk.ibm.com (192.168.101.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 22 Jan 2020 15:03:41 -0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 00MF3cPB57081940 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 22 Jan 2020 15:03:38 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 458184C04E; Wed, 22 Jan 2020 15:03:38 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC1F34C063; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) Received: from pic2.home (unknown [9.145.60.113]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 22 Jan 2020 15:03:37 +0000 (GMT) From: Frederic Barrat To: skiboot@lists.ozlabs.org, andrew.donnellan@au1.ibm.com, clombard@linux.ibm.com, arbab@linux.ibm.com, alistair@popple.id.au, oohall@gmail.com Date: Wed, 22 Jan 2020 16:03:36 +0100 X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200122150336.2623-1-fbarrat@linux.ibm.com> References: <20200122150336.2623-1-fbarrat@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 20012215-0008-0000-0000-0000034BBC8A X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20012215-0009-0000-0000-00004A6C25B7 Message-Id: <20200122150336.2623-4-fbarrat@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-17_05:2020-01-16, 2020-01-17 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 clxscore=1015 mlxlogscore=999 mlxscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 phishscore=0 adultscore=0 suspectscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-2001220134 Subject: [Skiboot] [RESEND PATCH v2 4/4] npu3: Register virtual PHBs with static IDs X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joy_Chu@wistron.com, chhank@tw.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Assigning opal IDs to virtual PHBs dynamically may lead to userland seeing the PCI domain ID for an adapter vary when adding or removing another adapter (GPU or opencapi). This patch switches to using static opal IDs for virtual PHBs, based on their ibm,phb-index property, which was made static by a previous patch. Note that the PCI domain IDs will increase on the second chip (or more, if we had more) because we now reserve 16 IDs per chip for PHBs. This affects Axone only. We don't change anything on P9 and npu2, to avoid altering how domain IDs have been shown on already GA'd platforms. Reviewed-by: Reza Arbab Reviewed-by: Andrew Donnellan Signed-off-by: Frederic Barrat --- Changelog: v2: new patch: use static opal IDs for virtual PHBs on axone hw/npu3-nvlink.c | 3 ++- include/npu3.h | 6 ++++++ include/phb4.h | 7 ++++++- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/hw/npu3-nvlink.c b/hw/npu3-nvlink.c index b8856ddc..4eb704b1 100644 --- a/hw/npu3-nvlink.c +++ b/hw/npu3-nvlink.c @@ -904,7 +904,8 @@ static void npu3_create_phb(struct npu3 *npu) assert(phb->dt_node); list_head_init(&phb->virt_devices); - pci_register_phb(phb, OPAL_DYNAMIC_PHB_ID); + pci_register_phb(phb, npu3_get_opal_id(npu->chip_id, + npu3_get_phb_index(npu->index))); npu3_create_phb_slot(npu); npu3_ioda_reset(phb, true); } diff --git a/include/npu3.h b/include/npu3.h index 0fdad4df..dda60ae1 100644 --- a/include/npu3.h +++ b/include/npu3.h @@ -20,6 +20,7 @@ #include #include #include +#include enum npu3_dev_type { NPU3_DEV_TYPE_UNKNOWN = 0, @@ -183,4 +184,9 @@ static inline int npu3_get_phb_index(unsigned int npu_index) return NPU3_PHB_INDEX_BASE + npu_index; } +static inline int npu3_get_opal_id(unsigned int chip_id, unsigned int index) +{ + return phb4_get_opal_id(chip_id, index); +} + #endif /* __NPU3_H */ diff --git a/include/phb4.h b/include/phb4.h index ca701a31..3dd0a5cf 100644 --- a/include/phb4.h +++ b/include/phb4.h @@ -246,10 +246,15 @@ static inline void phb4_set_err_pending(struct phb4 *p, bool pending) } #define PHB4_PER_CHIP 6 /* Max 6 PHBs per chip on p9 */ +#define PHB4_MAX_PHBS_PER_CHIP_P9 PHB4_PER_CHIP +#define PHB4_MAX_PHBS_PER_CHIP_P9P 0x10 /* extra for virt PHBs */ static inline int phb4_get_opal_id(unsigned int chip_id, unsigned int index) { - return chip_id * PHB4_PER_CHIP + index; + if (PVR_TYPE(mfspr(SPR_PVR)) == PVR_TYPE_P9) + return chip_id * PHB4_MAX_PHBS_PER_CHIP_P9 + index; + else + return chip_id * PHB4_MAX_PHBS_PER_CHIP_P9P + index; } #endif /* __PHB4_H */