From patchwork Wed Dec 12 06:58:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Donnellan X-Patchwork-Id: 1011594 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 43F73K4QD0z9s5c for ; Wed, 12 Dec 2018 17:59:41 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=au1.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 43F73K21x7zDqqD for ; Wed, 12 Dec 2018 17:59:41 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=au1.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=au1.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=andrew.donnellan@au1.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=au1.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 43F73B4Sf3zDqcG for ; Wed, 12 Dec 2018 17:59:34 +1100 (AEDT) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id wBC6wTiD023018 for ; Wed, 12 Dec 2018 01:59:32 -0500 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2pawgng390-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 12 Dec 2018 01:59:31 -0500 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 12 Dec 2018 06:59:29 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 12 Dec 2018 06:59:27 -0000 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id wBC6xQL961341876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 12 Dec 2018 06:59:26 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 99ED552052; Wed, 12 Dec 2018 06:59:26 +0000 (GMT) Received: from ozlabs.au.ibm.com (unknown [9.192.253.14]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id 009C452059; Wed, 12 Dec 2018 06:59:26 +0000 (GMT) Received: from intelligence.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 89DBAA026A; Wed, 12 Dec 2018 17:59:24 +1100 (AEDT) From: Andrew Donnellan To: skiboot@lists.ozlabs.org Date: Wed, 12 Dec 2018 17:58:45 +1100 X-Mailer: git-send-email 2.11.0 In-Reply-To: References: In-Reply-To: References: X-TM-AS-GCONF: 00 x-cbid: 18121206-4275-0000-0000-000002EF53C8 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18121206-4276-0000-0000-000037FC7450 Message-Id: <542ba2cf95249cd4d20c798e79a095d3058d9a38.1544597914.git-series.andrew.donnellan@au1.ibm.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-12-12_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1812120061 Subject: [Skiboot] [PATCH 02/13] hw/npu2: Merge implementations of BAR accessor functions X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: alistair@popple.id.au, arbab@linux.ibm.com, fbarrat@linux.vnet.ibm.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" Move npu2_{get,read,write}_bar() to common code. Get rid of the OpenCAPI write_bar() function. Fix the OpenCAPI code to use the NVLink-style BAR accessor functions and struct npu2_bar. While we're there, fix a trivial bug in npu2_read_bar() when reading PHY BARs - the global MMIO BAR is stack 0, not stack 2. I don't think we ever read that BAR, so this has never been a problem. Signed-off-by: Andrew Donnellan Reviewed-by: Frederic Barrat --- hw/npu2-common.c | 77 ++++++++++++++++++++++++++++++++- hw/npu2-opencapi.c | 109 ++++++++++++++++------------------------------ hw/npu2.c | 78 +--------------------------------- include/npu2.h | 4 ++- 4 files changed, 120 insertions(+), 148 deletions(-) diff --git a/hw/npu2-common.c b/hw/npu2-common.c index 6e6b12f0d1ae..3446acb45bea 100644 --- a/hw/npu2-common.c +++ b/hw/npu2-common.c @@ -22,6 +22,7 @@ #include #include #include +#include /* * We use the indirect method because it uses the same addresses as @@ -97,6 +98,82 @@ void npu2_write_mask_4b(struct npu2 *p, uint64_t reg, uint32_t val, uint32_t mas (uint64_t)new_val << 32); } +void npu2_get_bar(uint32_t gcid, struct npu2_bar *bar) +{ + phys_map_get(gcid, bar->type, bar->index, &bar->base, &bar->size); +} + +void npu2_read_bar(struct npu2 *p, struct npu2_bar *bar) +{ + uint64_t reg, val; + + reg = NPU2_REG_OFFSET(0, NPU2_BLOCK_SM_0, bar->reg); + val = npu2_read(p, reg); + + switch (NPU2_REG(bar->reg)) { + case NPU2_PHY_BAR: + bar->base = GETFIELD(NPU2_PHY_BAR_ADDR, val) << 21; + bar->enabled = GETFIELD(NPU2_PHY_BAR_ENABLE, val); + + if (NPU2_REG_STACK(reg) == NPU2_STACK_STCK_0) + /* This is the global MMIO BAR */ + bar->size = 0x1000000; + else + bar->size = 0x200000; + break; + case NPU2_NTL0_BAR: + case NPU2_NTL1_BAR: + bar->base = GETFIELD(NPU2_NTL_BAR_ADDR, val) << 16; + bar->enabled = GETFIELD(NPU2_NTL_BAR_ENABLE, val); + bar->size = 0x10000 << GETFIELD(NPU2_NTL_BAR_SIZE, val); + break; + case NPU2_GENID_BAR: + bar->base = GETFIELD(NPU2_GENID_BAR_ADDR, val) << 16; + bar->enabled = GETFIELD(NPU2_GENID_BAR_ENABLE, val); + bar->size = 0x20000; + break; + default: + bar->base = 0ul; + bar->enabled = false; + bar->size = 0; + break; + } +} + +void npu2_write_bar(struct npu2 *p, struct npu2_bar *bar, uint32_t gcid, + uint32_t scom) +{ + uint64_t reg, val; + int block; + + switch (NPU2_REG(bar->reg)) { + case NPU2_PHY_BAR: + val = SETFIELD(NPU2_PHY_BAR_ADDR, 0ul, bar->base >> 21); + val = SETFIELD(NPU2_PHY_BAR_ENABLE, val, bar->enabled); + break; + case NPU2_NTL0_BAR: + case NPU2_NTL1_BAR: + val = SETFIELD(NPU2_NTL_BAR_ADDR, 0ul, bar->base >> 16); + val = SETFIELD(NPU2_NTL_BAR_ENABLE, val, bar->enabled); + val = SETFIELD(NPU2_NTL_BAR_SIZE, val, ilog2(bar->size >> 16)); + break; + case NPU2_GENID_BAR: + val = SETFIELD(NPU2_GENID_BAR_ADDR, 0ul, bar->base >> 16); + val = SETFIELD(NPU2_GENID_BAR_ENABLE, val, bar->enabled); + break; + default: + val = 0ul; + } + + for (block = NPU2_BLOCK_SM_0; block <= NPU2_BLOCK_SM_3; block++) { + reg = NPU2_REG_OFFSET(0, block, bar->reg); + if (p) + npu2_write(p, reg, val); + else + npu2_scom_write(gcid, scom, reg, NPU2_MISC_DA_LEN_8B, val); + } +} + static bool _i2c_presence_detect(struct npu2_dev *dev) { uint8_t state, data; diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c index b160a41741b9..88e4eb1974a2 100644 --- a/hw/npu2-opencapi.c +++ b/hw/npu2-opencapi.c @@ -730,63 +730,29 @@ static void address_translation_config(uint32_t gcid, uint32_t scom_base, } } -/* TODO: Merge this with NVLink implementation - we don't use the npu2_bar - * wrapper for the PHY BARs yet */ -static void write_bar(uint32_t gcid, uint32_t scom_base, uint64_t reg, - uint64_t addr, uint64_t size) -{ - uint64_t val; - int block; - switch (NPU2_REG(reg)) { - case NPU2_PHY_BAR: - val = SETFIELD(NPU2_PHY_BAR_ADDR, 0ul, addr >> 21); - val = SETFIELD(NPU2_PHY_BAR_ENABLE, val, 1); - break; - case NPU2_NTL0_BAR: - case NPU2_NTL1_BAR: - val = SETFIELD(NPU2_NTL_BAR_ADDR, 0ul, addr >> 16); - val = SETFIELD(NPU2_NTL_BAR_SIZE, val, ilog2(size >> 16)); - val = SETFIELD(NPU2_NTL_BAR_ENABLE, val, 1); - break; - case NPU2_GENID_BAR: - val = SETFIELD(NPU2_GENID_BAR_ADDR, 0ul, addr >> 16); - val = SETFIELD(NPU2_GENID_BAR_ENABLE, val, 1); - break; - default: - val = 0ul; - } - - for (block = NPU2_BLOCK_SM_0; block <= NPU2_BLOCK_SM_3; block++) { - npu2_scom_write(gcid, scom_base, NPU2_REG_OFFSET(0, block, reg), - NPU2_MISC_DA_LEN_8B, val); - prlog(PR_DEBUG, "OCAPI: Setting BAR %llx to %llx\n", - NPU2_REG_OFFSET(0, block, reg), val); - } -} - static void setup_global_mmio_bar(uint32_t gcid, uint32_t scom_base, uint64_t reg[]) { - uint64_t addr, size; - - prlog(PR_DEBUG, "OCAPI: patching up PHY0 bar, %s\n", __func__); - phys_map_get(gcid, NPU_PHY, 0, &addr, &size); - write_bar(gcid, scom_base, - NPU2_REG_OFFSET(NPU2_STACK_STCK_2, 0, NPU2_PHY_BAR), - addr, size); - prlog(PR_DEBUG, "OCAPI: patching up PHY1 bar, %s\n", __func__); - phys_map_get(gcid, NPU_PHY, 1, &addr, &size); - write_bar(gcid, scom_base, - NPU2_REG_OFFSET(NPU2_STACK_STCK_1, 0, NPU2_PHY_BAR), - addr, size); - - prlog(PR_DEBUG, "OCAPI: setup global mmio, %s\n", __func__); - phys_map_get(gcid, NPU_REGS, 0, &addr, &size); - write_bar(gcid, scom_base, - NPU2_REG_OFFSET(NPU2_STACK_STCK_0, 0, NPU2_PHY_BAR), - addr, size); - reg[0] = addr; - reg[1] = size; + struct npu2_bar *bar; + struct npu2_bar phy_bars[] = { + { .type = NPU_PHY, .index = 0, + .reg = NPU2_REG_OFFSET(NPU2_STACK_STCK_2, 0, NPU2_PHY_BAR), + .enabled = true }, + { .type = NPU_PHY, .index = 1, + .reg = NPU2_REG_OFFSET(NPU2_STACK_STCK_1, 0, NPU2_PHY_BAR), + .enabled = true }, + { .type = NPU_REGS, .index = 0, + .reg = NPU2_REG_OFFSET(NPU2_STACK_STCK_0, 0, NPU2_PHY_BAR), + .enabled = true }, + }; + + for (int i = 0; i < ARRAY_SIZE(phy_bars); i++) { + bar = &phy_bars[i]; + npu2_get_bar(gcid, bar); + npu2_write_bar(NULL, bar, gcid, scom_base); + } + reg[0] = phy_bars[2].base; + reg[1] = phy_bars[2].size; } /* Procedure 13.1.3.8 - AFU MMIO Range BARs */ @@ -803,15 +769,18 @@ static void setup_afu_mmio_bars(uint32_t gcid, uint32_t scom_base, prlog(PR_DEBUG, "OCAPI: %s: Setup AFU MMIO BARs\n", __func__); phys_map_get(gcid, NPU_OCAPI_MMIO, dev->brick_index, &addr, &size); - - prlog(PR_DEBUG, "OCAPI: AFU MMIO set to %llx, size %llx\n", addr, size); - write_bar(gcid, scom_base, NPU2_REG_OFFSET(stack, 0, offset), addr, - size); - dev->bars[0].base = addr; - dev->bars[0].size = size; - - reg = SETFIELD(NPU2_CQ_CTL_MISC_MMIOPA_ADDR, 0ull, addr >> 16); - reg = SETFIELD(NPU2_CQ_CTL_MISC_MMIOPA_SIZE, reg, ilog2(size >> 16)); + dev->bars[0].type = NPU_OCAPI_MMIO; + dev->bars[0].index = dev->brick_index; + dev->bars[0].reg = NPU2_REG_OFFSET(stack, 0, offset); + dev->bars[0].enabled = true; + npu2_get_bar(gcid, &dev->bars[0]); + + prlog(PR_DEBUG, "OCAPI: AFU MMIO set to %llx, size %llx\n", + dev->bars[0].base, dev->bars[0].size); + npu2_write_bar(NULL, &dev->bars[0], gcid, scom_base); + + reg = SETFIELD(NPU2_CQ_CTL_MISC_MMIOPA_ADDR, 0ull, dev->bars[0].base >> 16); + reg = SETFIELD(NPU2_CQ_CTL_MISC_MMIOPA_SIZE, reg, ilog2(dev->bars[0].size >> 16)); prlog(PR_DEBUG, "OCAPI: PA translation %llx\n", reg); npu2_scom_write(gcid, scom_base, NPU2_REG_OFFSET(stack, NPU2_BLOCK_CTL, @@ -825,15 +794,15 @@ static void setup_afu_config_bars(uint32_t gcid, uint32_t scom_base, { uint64_t stack = index_to_stack(dev->brick_index); int stack_num = stack - NPU2_STACK_STCK_0; - uint64_t addr, size; prlog(PR_DEBUG, "OCAPI: %s: Setup AFU Config BARs\n", __func__); - phys_map_get(gcid, NPU_GENID, stack_num, &addr, &size); - prlog(PR_DEBUG, "OCAPI: Assigning GENID BAR: %016llx\n", addr); - write_bar(gcid, scom_base, NPU2_REG_OFFSET(stack, 0, NPU2_GENID_BAR), - addr, size); - dev->bars[1].base = addr; - dev->bars[1].size = size; + dev->bars[1].type = NPU_GENID; + dev->bars[1].index = stack_num; + dev->bars[1].reg = NPU2_REG_OFFSET(stack, 0, NPU2_GENID_BAR); + dev->bars[1].enabled = true; + npu2_get_bar(gcid, &dev->bars[1]); + prlog(PR_DEBUG, "OCAPI: Assigning GENID BAR: %016llx\n", dev->bars[1].base); + npu2_write_bar(NULL, &dev->bars[1], gcid, scom_base); } static void otl_enabletx(uint32_t gcid, uint32_t scom_base, diff --git a/hw/npu2.c b/hw/npu2.c index ba5575cf3097..6b0880682427 100644 --- a/hw/npu2.c +++ b/hw/npu2.c @@ -106,84 +106,6 @@ static struct npu2_dev *npu2_bdf_to_dev(struct npu2 *p, return NULL; } -static inline void npu2_get_bar(uint32_t gcid, struct npu2_bar *bar) -{ - phys_map_get(gcid, bar->type, bar->index, &bar->base, &bar->size); -} - -static void npu2_read_bar(struct npu2 *p, struct npu2_bar *bar) -{ - uint64_t reg, val; - - reg = NPU2_REG_OFFSET(0, NPU2_BLOCK_SM_0, bar->reg); - val = npu2_read(p, reg); - - switch (NPU2_REG(bar->reg)) { - case NPU2_PHY_BAR: - bar->base = GETFIELD(NPU2_PHY_BAR_ADDR, val) << 21; - bar->enabled = GETFIELD(NPU2_PHY_BAR_ENABLE, val); - - if (NPU2_REG_STACK(reg) == NPU2_STACK_STCK_2) - /* This is the global MMIO BAR */ - bar->size = 0x1000000; - else - bar->size = 0x200000; - break; - case NPU2_NTL0_BAR: - case NPU2_NTL1_BAR: - bar->base = GETFIELD(NPU2_NTL_BAR_ADDR, val) << 16; - bar->enabled = GETFIELD(NPU2_NTL_BAR_ENABLE, val); - bar->size = 0x10000 << GETFIELD(NPU2_NTL_BAR_SIZE, val); - break; - case NPU2_GENID_BAR: - bar->base = GETFIELD(NPU2_GENID_BAR_ADDR, val) << 16; - bar->enabled = GETFIELD(NPU2_GENID_BAR_ENABLE, val); - bar->size = 0x20000; - break; - default: - bar->base = 0ul; - bar->enabled = false; - bar->size = 0; - break; - } -} - -static void npu2_write_bar(struct npu2 *p, - struct npu2_bar *bar, - uint32_t gcid, - uint32_t scom) -{ - uint64_t reg, val; - int block; - - switch (NPU2_REG(bar->reg)) { - case NPU2_PHY_BAR: - val = SETFIELD(NPU2_PHY_BAR_ADDR, 0ul, bar->base >> 21); - val = SETFIELD(NPU2_PHY_BAR_ENABLE, val, bar->enabled); - break; - case NPU2_NTL0_BAR: - case NPU2_NTL1_BAR: - val = SETFIELD(NPU2_NTL_BAR_ADDR, 0ul, bar->base >> 16); - val = SETFIELD(NPU2_NTL_BAR_ENABLE, val, bar->enabled); - val = SETFIELD(NPU2_NTL_BAR_SIZE, val, 1); - break; - case NPU2_GENID_BAR: - val = SETFIELD(NPU2_GENID_BAR_ADDR, 0ul, bar->base >> 16); - val = SETFIELD(NPU2_GENID_BAR_ENABLE, val, bar->enabled); - break; - default: - val = 0ul; - } - - for (block = NPU2_BLOCK_SM_0; block <= NPU2_BLOCK_SM_3; block++) { - reg = NPU2_REG_OFFSET(0, block, bar->reg); - if (p) - npu2_write(p, reg, val); - else - npu2_scom_write(gcid, scom, reg, NPU2_MISC_DA_LEN_8B, val); - } -} - /* Trap for PCI command (0x4) to enable or disable device's BARs */ static int64_t npu2_cfg_write_cmd(void *dev, struct pci_cfg_reg_filter *pcrf __unused, diff --git a/include/npu2.h b/include/npu2.h index da5a5597feb0..c5c5b43843c3 100644 --- a/include/npu2.h +++ b/include/npu2.h @@ -210,6 +210,10 @@ void npu2_write(struct npu2 *p, uint64_t reg, uint64_t val); uint64_t npu2_read(struct npu2 *p, uint64_t reg); void npu2_write_mask(struct npu2 *p, uint64_t reg, uint64_t val, uint64_t mask); void npu2_write_mask_4b(struct npu2 *p, uint64_t reg, uint32_t val, uint32_t mask); +void npu2_get_bar(uint32_t gcid, struct npu2_bar *bar); +void npu2_read_bar(struct npu2 *p, struct npu2_bar *bar); +void npu2_write_bar(struct npu2 *p, struct npu2_bar *bar, uint32_t gcid, + uint32_t scom); int64_t npu2_dev_procedure(void *dev, struct pci_cfg_reg_filter *pcrf, uint32_t offset, uint32_t len, uint32_t *data, bool write);