From patchwork Wed Jul 26 05:59:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812964 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=TtmpGeU7; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jt70dwBz1yZv for ; Wed, 26 Jul 2023 16:00:14 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZ9-0003Wd-LH; Wed, 26 Jul 2023 06:00:07 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZ7-0003WH-RS for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:05 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 4B43F3F16F for ; Wed, 26 Jul 2023 06:00:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351205; bh=E4RfIg6J3pCXVsZfqcn/35Ptbo3JMqtfnuZYYuz3rP4=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TtmpGeU7sFhY7T4W0JH+a5374/8tr9tW8L/Ls1kvSsNZEFa8qoOGdOD6LO3VBgWPY ie9pysK+R9iv1lShi5g67BB3T7ben5Of6xdaVlffgHswdJ5pkfkzc1gdXG/iCp5ISJ CQ1raZxaxV5EA/wQxqJbNLmqkXoppidhoIDjW0aeFklqUCFWNSG+1K64tlVqKeH8qc LDiFATql79OnzEsL2C+C3/QzgiEcIQkRnb5yK5ED2k+S4kchq6f/KG8eE6+AMZEIYH iMOO4dxpLUy1I5A7JRYls6PqkFZ4W7/LU/ahas0uxs5Zj4Km7au1oz3DnUEzCcN0op tyYqTsOxsn/8A== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L] [PATCH 1/6] EDAC/skx_common: Enable EDAC support for the "near" memory Date: Wed, 26 Jul 2023 13:59:12 +0800 Message-Id: <20230726055917.3291633-2-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 The current {skx,i10nm}_edac miss the EDAC support to decode errors from the 1st level memory (the fast "near" memory as cache) of the 2-level memory system. Introduce a helper function skx_error_in_mem() to check whether errors are from memory at the beginning of skx_mce_check_error(). As long as the errors are from memory (either the 1-level memory system or the 2-level memory system), decode the errors. Reported-and-tested-by: Youquan Song Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/all/20230113032802.41752-1-qiuxu.zhuo@intel.com (cherry picked from commit 6e8746cb735166eaf3ceb086b31bb0431f5e3532) Signed-off-by: Kai-Heng Feng --- drivers/edac/skx_common.c | 18 ++++++++++++------ drivers/edac/skx_common.h | 24 ++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/drivers/edac/skx_common.c b/drivers/edac/skx_common.c index f0f8e98f6efb..5fdcb6d5b96b 100644 --- a/drivers/edac/skx_common.c +++ b/drivers/edac/skx_common.c @@ -632,12 +632,18 @@ static bool skx_error_in_1st_level_mem(const struct mce *m) if (!skx_mem_cfg_2lm) return false; - errcode = GET_BITFIELD(m->status, 0, 15); + errcode = GET_BITFIELD(m->status, 0, 15) & MCACOD_MEM_ERR_MASK; - if ((errcode & 0xef80) != 0x280) - return false; + return errcode == MCACOD_EXT_MEM_ERR; +} - return true; +static bool skx_error_in_mem(const struct mce *m) +{ + u32 errcode; + + errcode = GET_BITFIELD(m->status, 0, 15) & MCACOD_MEM_ERR_MASK; + + return (errcode == MCACOD_MEM_CTL_ERR || errcode == MCACOD_EXT_MEM_ERR); } int skx_mce_check_error(struct notifier_block *nb, unsigned long val, @@ -651,8 +657,8 @@ int skx_mce_check_error(struct notifier_block *nb, unsigned long val, if (mce->kflags & MCE_HANDLED_CEC) return NOTIFY_DONE; - /* ignore unless this is memory related with an address */ - if ((mce->status & 0xefff) >> 7 != 1 || !(mce->status & MCI_STATUS_ADDRV)) + /* Ignore unless this is memory related with an address */ + if (!skx_error_in_mem(mce) || !(mce->status & MCI_STATUS_ADDRV)) return NOTIFY_DONE; memset(&res, 0, sizeof(res)); diff --git a/drivers/edac/skx_common.h b/drivers/edac/skx_common.h index 0cbadd3d2cd3..312032657264 100644 --- a/drivers/edac/skx_common.h +++ b/drivers/edac/skx_common.h @@ -56,6 +56,30 @@ #define MCI_MISC_ECC_MODE(m) (((m) >> 59) & 15) #define MCI_MISC_ECC_DDRT 8 /* read from DDRT */ +/* + * According to Intel Architecture spec vol 3B, + * Table 15-10 "IA32_MCi_Status [15:0] Compound Error Code Encoding" + * memory errors should fit one of these masks: + * 000f 0000 1mmm cccc (binary) + * 000f 0010 1mmm cccc (binary) [RAM used as cache] + * where: + * f = Correction Report Filtering Bit. If 1, subsequent errors + * won't be shown + * mmm = error type + * cccc = channel + */ +#define MCACOD_MEM_ERR_MASK 0xef80 +/* + * Errors from either the memory of the 1-level memory system or the + * 2nd level memory (the slow "far" memory) of the 2-level memory system. + */ +#define MCACOD_MEM_CTL_ERR 0x80 +/* + * Errors from the 1st level memory (the fast "near" memory as cache) + * of the 2-level memory system. + */ +#define MCACOD_EXT_MEM_ERR 0x280 + /* * Each cpu socket contains some pci devices that provide global * information, and also some that are local to each of the two From patchwork Wed Jul 26 05:59:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812966 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=obn3RMHN; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jt831P1z1ydq for ; Wed, 26 Jul 2023 16:00:16 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZB-0003YN-Tj; Wed, 26 Jul 2023 06:00:09 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZ9-0003Wi-SE for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:07 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 4CA7F3F16F for ; Wed, 26 Jul 2023 06:00:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351207; bh=lCzR4uv4rxaKL/BiEdrZe1ryWMoB2YCa4Q/C8vCrXMQ=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=obn3RMHNDfBz56OdrGP+0FjnJLaa/EFVAaLHsf5s8AkafrTufg4Zmurz/YndxdpIP ikfIYUo57Iy6vxQsnr8C1fW24XORu9DMpe0G076Oa3FT1NwR4yumhNuyEur80GqN6L 42rk3IYCkLDIkk1+jwtqUy8eC0c0rF7bF6eqtJwttigW0j0bg3Db5e1PZhWRE/noOk P4/alSgQ8fj364h7MJ3TpM/8IJbJqvkLCdp4EERJ+dq/ON9fUD2IKhox/dbSBjR4XQ DOtjiCUsDrZd/pXm3MaA9Yo9YTvYsdXcLTDsMOYWAc9af4UeTwBFIHtlRsuFONoKLU G2ptOOoIexrrw== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L] [PATCH 2/6] EDAC/skx_common: Delete duplicated and unreachable code Date: Wed, 26 Jul 2023 13:59:13 +0800 Message-Id: <20230726055917.3291633-3-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 skx_mce_check_error() returns early if the error isn't from memory. So when skx_mce_output_error() is invoked from skx_mce_check_error(), it doesn't need to re-check whether the error is from memory. Delete the duplicated and unreachable code from skx_mce_output_error(). Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/all/20230113032802.41752-1-qiuxu.zhuo@intel.com (cherry picked from commit d2415e2e5330fc11f4c688fa518751bdc90259f5) Signed-off-by: Kai-Heng Feng --- drivers/edac/skx_common.c | 58 ++++++++++++++------------------------- 1 file changed, 21 insertions(+), 37 deletions(-) diff --git a/drivers/edac/skx_common.c b/drivers/edac/skx_common.c index 5fdcb6d5b96b..fd8186fe005c 100644 --- a/drivers/edac/skx_common.c +++ b/drivers/edac/skx_common.c @@ -560,44 +560,28 @@ static void skx_mce_output_error(struct mem_ctl_info *mci, tp_event = HW_EVENT_ERR_CORRECTED; } - /* - * According to Intel Architecture spec vol 3B, - * Table 15-10 "IA32_MCi_Status [15:0] Compound Error Code Encoding" - * memory errors should fit one of these masks: - * 000f 0000 1mmm cccc (binary) - * 000f 0010 1mmm cccc (binary) [RAM used as cache] - * where: - * f = Correction Report Filtering Bit. If 1, subsequent errors - * won't be shown - * mmm = error type - * cccc = channel - * If the mask doesn't match, report an error to the parsing logic - */ - if (!((errcode & 0xef80) == 0x80 || (errcode & 0xef80) == 0x280)) { - optype = "Can't parse: it is not a mem"; - } else { - switch (optypenum) { - case 0: - optype = "generic undef request error"; - break; - case 1: - optype = "memory read error"; - break; - case 2: - optype = "memory write error"; - break; - case 3: - optype = "addr/cmd error"; - break; - case 4: - optype = "memory scrubbing error"; - scrub_err = true; - break; - default: - optype = "reserved"; - break; - } + switch (optypenum) { + case 0: + optype = "generic undef request error"; + break; + case 1: + optype = "memory read error"; + break; + case 2: + optype = "memory write error"; + break; + case 3: + optype = "addr/cmd error"; + break; + case 4: + optype = "memory scrubbing error"; + scrub_err = true; + break; + default: + optype = "reserved"; + break; } + if (res->decoded_by_adxl) { len = snprintf(skx_msg, MSG_SIZE, "%s%s err_code:0x%04x:0x%04x %s", overflow ? " OVERFLOW" : "", From patchwork Wed Jul 26 05:59:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812967 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=kt3dxL8K; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jt9431qz1yZv for ; Wed, 26 Jul 2023 16:00:17 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZE-0003aU-4Z; Wed, 26 Jul 2023 06:00:12 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZC-0003Z1-Uf for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:10 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 5867A3F16F for ; Wed, 26 Jul 2023 06:00:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351210; bh=O8zDOj9iVBc5SHt0oTENjY6mgGo4j1NzWK/FjV+MZqk=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kt3dxL8Kydfwp8NjOOct6oU/cpyXNR9xe87ZBk14+gKEhRt6tLS0VcsYZvJqaHgQo XAc/imy6VdW/LDLkUqgW3BoXH0/qVGAzb3GZcQs3UUC1Ul1Fc8spL2AY9WnA4N0teT vldFGQAgscMgpG1zI3LJZu31NMZ1jfmEI23nj1ykFv5L64KlKvoDeP7MHxEOCTqPMJ G5Iq/p+E7z3SvsREIpf4bGYRuHwIsx9C38dcukeFeWR57AUE1n2ljeyJlc50bm+4sm N1TEw4BS6IzjL5JWFBWRifyIU79ZCpf/U0EN2HHEhbqcw8TgJLcmquWiblnaWZGgG2 4v39y4W4hLXAw== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L] [PATCH 3/6] EDAC/i10nm: Add Intel Emerald Rapids server support Date: Wed, 26 Jul 2023 13:59:14 +0800 Message-Id: <20230726055917.3291633-4-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 The Emerald Rapids CPU model uses similar memory controller registers as Sapphire Rapids server. Add Emerald Rapids CPU model number ID for EDAC support. Tested-by: Li Zhang Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/all/20230113032802.41752-1-qiuxu.zhuo@intel.com (cherry picked from commit e4b2bc6616e21f4a7ce4e7452f716e3db8fe66b6) Signed-off-by: Kai-Heng Feng --- drivers/edac/i10nm_base.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c index 65aeea53e2df..e11726f7fe36 100644 --- a/drivers/edac/i10nm_base.c +++ b/drivers/edac/i10nm_base.c @@ -637,6 +637,7 @@ static const struct x86_cpu_id i10nm_cpuids[] = { X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ICELAKE_X, X86_STEPPINGS(0x4, 0xf), &i10nm_cfg1), X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ICELAKE_D, X86_STEPPINGS(0x0, 0xf), &i10nm_cfg1), X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(SAPPHIRERAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg), + X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(EMERALDRAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg), {} }; MODULE_DEVICE_TABLE(x86cpu, i10nm_cpuids); From patchwork Wed Jul 26 05:59:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812968 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=UiWMDonD; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jtL3pH9z1yZv for ; Wed, 26 Jul 2023 16:00:26 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZL-0003kv-OM; Wed, 26 Jul 2023 06:00:19 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZI-0003cl-N9 for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:16 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 5A9363F16F for ; Wed, 26 Jul 2023 06:00:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351213; bh=lyP1aeNsvuzm4PxYP/jGb5VZWYuP8SoOf3E4zSNTgyQ=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UiWMDonDreIDCk5dQbdQtiPyIh+YFtvlmurdS5VQXvBwo/OFgKNMvbIgP2F6wSYOT Pcg7r6wBTM6LXEGcV2HrCbum8nkXK32ugtEalr/BD2b+/K2eXxWRS5rVd3FNgy0iIu llIjOs5nlAeK+BJ3Sy5rOB9SYAT03upJHpuon4CvsnfVgFdHHPdx5y/CoXMfX5Vqlx 8LM+Rn6hY51ya/s0PuNOjSz2hPQRTEjubtv596+2xtWqipo6TUZmvyrpLRRuLEDPoy KL4tZyvafJImWe+iqM8FnWAKVddST4yjOzkifzCbaZNPTnfsiIo5Tj2FqSKoHwjAlA y33hs+6YDxKKQ== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L] [PATCH 4/6] EDAC/i10nm: Make more configurations CPU model specific Date: Wed, 26 Jul 2023 13:59:15 +0800 Message-Id: <20230726055917.3291633-5-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 The numbers of memory controllers per socket, channels per memory controller, DIMMs per channel and the triples of bus/device/function of PCI devices used in i10nm_edac can be CPU model specific. Add new fields to the structure res_config for above numbers and triples to make them CPU model specific. Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/all/20230113032802.41752-1-qiuxu.zhuo@intel.com (cherry picked from commit dd7814b78539416c6e561eeaa0951b3e88ac799e) Signed-off-by: Kai-Heng Feng --- drivers/edac/i10nm_base.c | 131 ++++++++++++++++++++++++++------------ drivers/edac/skx_common.h | 32 +++++++++- 2 files changed, 121 insertions(+), 42 deletions(-) diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c index e11726f7fe36..5682d64cceb6 100644 --- a/drivers/edac/i10nm_base.c +++ b/drivers/edac/i10nm_base.c @@ -148,35 +148,47 @@ static void __enable_retry_rd_err_log(struct skx_imc *imc, int chan, bool enable static void enable_retry_rd_err_log(bool enable) { + int i, j, imc_num, chan_num; struct skx_imc *imc; struct skx_dev *d; - int i, j; edac_dbg(2, "\n"); - list_for_each_entry(d, i10nm_edac_list, list) - for (i = 0; i < I10NM_NUM_IMC; i++) { + list_for_each_entry(d, i10nm_edac_list, list) { + imc_num = res_cfg->ddr_imc_num; + chan_num = res_cfg->ddr_chan_num; + + for (i = 0; i < imc_num; i++) { imc = &d->imc[i]; if (!imc->mbase) continue; - for (j = 0; j < I10NM_NUM_CHANNELS; j++) { - if (imc->hbm_mc) { - __enable_retry_rd_err_log(imc, j, enable, - res_cfg->offsets_scrub_hbm0, - res_cfg->offsets_demand_hbm0, - NULL); - __enable_retry_rd_err_log(imc, j, enable, - res_cfg->offsets_scrub_hbm1, - res_cfg->offsets_demand_hbm1, - NULL); - } else { - __enable_retry_rd_err_log(imc, j, enable, - res_cfg->offsets_scrub, - res_cfg->offsets_demand, - res_cfg->offsets_demand2); - } + for (j = 0; j < chan_num; j++) + __enable_retry_rd_err_log(imc, j, enable, + res_cfg->offsets_scrub, + res_cfg->offsets_demand, + res_cfg->offsets_demand2); + } + + imc_num += res_cfg->hbm_imc_num; + chan_num = res_cfg->hbm_chan_num; + + for (; i < imc_num; i++) { + imc = &d->imc[i]; + if (!imc->mbase || !imc->hbm_mc) + continue; + + for (j = 0; j < chan_num; j++) { + __enable_retry_rd_err_log(imc, j, enable, + res_cfg->offsets_scrub_hbm0, + res_cfg->offsets_demand_hbm0, + NULL); + __enable_retry_rd_err_log(imc, j, enable, + res_cfg->offsets_scrub_hbm1, + res_cfg->offsets_demand_hbm1, + NULL); } + } } } @@ -318,9 +330,9 @@ static bool i10nm_check_2lm(struct res_config *cfg) int i; list_for_each_entry(d, i10nm_edac_list, list) { - d->sad_all = pci_get_dev_wrapper(d->seg, d->bus[1], - PCI_SLOT(cfg->sad_all_devfn), - PCI_FUNC(cfg->sad_all_devfn)); + d->sad_all = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->sad_all_bdf.bus], + res_cfg->sad_all_bdf.dev, + res_cfg->sad_all_bdf.fun); if (!d->sad_all) continue; @@ -444,11 +456,15 @@ static int i10nm_get_ddr_munits(void) u64 base; list_for_each_entry(d, i10nm_edac_list, list) { - d->util_all = pci_get_dev_wrapper(d->seg, d->bus[1], 29, 1); + d->util_all = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->util_all_bdf.bus], + res_cfg->util_all_bdf.dev, + res_cfg->util_all_bdf.fun); if (!d->util_all) return -ENODEV; - d->uracu = pci_get_dev_wrapper(d->seg, d->bus[0], 0, 1); + d->uracu = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->uracu_bdf.bus], + res_cfg->uracu_bdf.dev, + res_cfg->uracu_bdf.fun); if (!d->uracu) return -ENODEV; @@ -461,9 +477,10 @@ static int i10nm_get_ddr_munits(void) edac_dbg(2, "socket%d mmio base 0x%llx (reg 0x%x)\n", j++, base, reg); - for (i = 0; i < I10NM_NUM_DDR_IMC; i++) { - mdev = pci_get_dev_wrapper(d->seg, d->bus[0], - 12 + i, 0); + for (i = 0; i < res_cfg->ddr_imc_num; i++) { + mdev = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->ddr_mdev_bdf.bus], + res_cfg->ddr_mdev_bdf.dev + i, + res_cfg->ddr_mdev_bdf.fun); if (i == 0 && !mdev) { i10nm_printk(KERN_ERR, "No IMC found\n"); return -ENODEV; @@ -519,7 +536,9 @@ static int i10nm_get_hbm_munits(void) u64 base; list_for_each_entry(d, i10nm_edac_list, list) { - d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[1], 30, 3); + d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->pcu_cr3_bdf.bus], + res_cfg->pcu_cr3_bdf.dev, + res_cfg->pcu_cr3_bdf.fun); if (!d->pcu_cr3) return -ENODEV; @@ -540,11 +559,13 @@ static int i10nm_get_hbm_munits(void) } base += I10NM_GET_HBM_IMC_MMIO_OFFSET(reg); - lmc = I10NM_NUM_DDR_IMC; + lmc = res_cfg->ddr_imc_num; + + for (i = 0; i < res_cfg->hbm_imc_num; i++) { + mdev = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->hbm_mdev_bdf.bus], + res_cfg->hbm_mdev_bdf.dev + i / 4, + res_cfg->hbm_mdev_bdf.fun + i % 4); - for (i = 0; i < I10NM_NUM_HBM_IMC; i++) { - mdev = pci_get_dev_wrapper(d->seg, d->bus[0], - 12 + i / 4, 1 + i % 4); if (i == 0 && !mdev) { i10nm_printk(KERN_ERR, "No hbm mc found\n"); return -ENODEV; @@ -594,8 +615,16 @@ static struct res_config i10nm_cfg0 = { .type = I10NM, .decs_did = 0x3452, .busno_cfg_offset = 0xcc, + .ddr_imc_num = 4, + .ddr_chan_num = 2, + .ddr_dimm_num = 2, .ddr_chan_mmio_sz = 0x4000, - .sad_all_devfn = PCI_DEVFN(29, 0), + .sad_all_bdf = {1, 29, 0}, + .pcu_cr3_bdf = {1, 30, 3}, + .util_all_bdf = {1, 29, 1}, + .uracu_bdf = {0, 0, 1}, + .ddr_mdev_bdf = {0, 12, 0}, + .hbm_mdev_bdf = {0, 12, 1}, .sad_all_offset = 0x108, .offsets_scrub = offsets_scrub_icx, .offsets_demand = offsets_demand_icx, @@ -605,8 +634,16 @@ static struct res_config i10nm_cfg1 = { .type = I10NM, .decs_did = 0x3452, .busno_cfg_offset = 0xd0, + .ddr_imc_num = 4, + .ddr_chan_num = 2, + .ddr_dimm_num = 2, .ddr_chan_mmio_sz = 0x4000, - .sad_all_devfn = PCI_DEVFN(29, 0), + .sad_all_bdf = {1, 29, 0}, + .pcu_cr3_bdf = {1, 30, 3}, + .util_all_bdf = {1, 29, 1}, + .uracu_bdf = {0, 0, 1}, + .ddr_mdev_bdf = {0, 12, 0}, + .hbm_mdev_bdf = {0, 12, 1}, .sad_all_offset = 0x108, .offsets_scrub = offsets_scrub_icx, .offsets_demand = offsets_demand_icx, @@ -616,10 +653,21 @@ static struct res_config spr_cfg = { .type = SPR, .decs_did = 0x3252, .busno_cfg_offset = 0xd0, + .ddr_imc_num = 4, + .ddr_chan_num = 2, + .ddr_dimm_num = 2, + .hbm_imc_num = 16, + .hbm_chan_num = 2, + .hbm_dimm_num = 1, .ddr_chan_mmio_sz = 0x8000, .hbm_chan_mmio_sz = 0x4000, .support_ddr5 = true, - .sad_all_devfn = PCI_DEVFN(10, 0), + .sad_all_bdf = {1, 10, 0}, + .pcu_cr3_bdf = {1, 30, 3}, + .util_all_bdf = {1, 29, 1}, + .uracu_bdf = {0, 0, 1}, + .ddr_mdev_bdf = {0, 12, 0}, + .hbm_mdev_bdf = {0, 12, 1}, .sad_all_offset = 0x300, .offsets_scrub = offsets_scrub_spr, .offsets_scrub_hbm0 = offsets_scrub_spr_hbm0, @@ -753,6 +801,7 @@ static int __init i10nm_init(void) struct skx_dev *d; int rc, i, off[3] = {0xd0, 0xc8, 0xcc}; u64 tolm, tohm; + int imc_num; edac_dbg(2, "\n"); @@ -793,6 +842,8 @@ static int __init i10nm_init(void) if (i10nm_get_hbm_munits() && rc) goto fail; + imc_num = res_cfg->ddr_imc_num + res_cfg->hbm_imc_num; + list_for_each_entry(d, i10nm_edac_list, list) { rc = skx_get_src_id(d, 0xf8, &src_id); if (rc < 0) @@ -803,7 +854,7 @@ static int __init i10nm_init(void) goto fail; edac_dbg(2, "src_id = %d node_id = %d\n", src_id, node_id); - for (i = 0; i < I10NM_NUM_IMC; i++) { + for (i = 0; i < imc_num; i++) { if (!d->imc[i].mdev) continue; @@ -813,12 +864,12 @@ static int __init i10nm_init(void) d->imc[i].node_id = node_id; if (d->imc[i].hbm_mc) { d->imc[i].chan_mmio_sz = cfg->hbm_chan_mmio_sz; - d->imc[i].num_channels = I10NM_NUM_HBM_CHANNELS; - d->imc[i].num_dimms = I10NM_NUM_HBM_DIMMS; + d->imc[i].num_channels = cfg->hbm_chan_num; + d->imc[i].num_dimms = cfg->hbm_dimm_num; } else { d->imc[i].chan_mmio_sz = cfg->ddr_chan_mmio_sz; - d->imc[i].num_channels = I10NM_NUM_DDR_CHANNELS; - d->imc[i].num_dimms = I10NM_NUM_DDR_DIMMS; + d->imc[i].num_channels = cfg->ddr_chan_num; + d->imc[i].num_dimms = cfg->ddr_dimm_num; } rc = skx_register_mci(&d->imc[i], d->imc[i].mdev, diff --git a/drivers/edac/skx_common.h b/drivers/edac/skx_common.h index 312032657264..982e1bcb1edf 100644 --- a/drivers/edac/skx_common.h +++ b/drivers/edac/skx_common.h @@ -173,19 +173,47 @@ struct decoded_addr { bool decoded_by_adxl; }; +struct pci_bdf { + u32 bus : 8; + u32 dev : 5; + u32 fun : 3; +}; + struct res_config { enum type type; /* Configuration agent device ID */ unsigned int decs_did; /* Default bus number configuration register offset */ int busno_cfg_offset; + /* DDR memory controllers per socket */ + int ddr_imc_num; + /* DDR channels per DDR memory controller */ + int ddr_chan_num; + /* DDR DIMMs per DDR memory channel */ + int ddr_dimm_num; /* Per DDR channel memory-mapped I/O size */ int ddr_chan_mmio_sz; + /* HBM memory controllers per socket */ + int hbm_imc_num; + /* HBM channels per HBM memory controller */ + int hbm_chan_num; + /* HBM DIMMs per HBM memory channel */ + int hbm_dimm_num; /* Per HBM channel memory-mapped I/O size */ int hbm_chan_mmio_sz; bool support_ddr5; - /* SAD device number and function number */ - unsigned int sad_all_devfn; + /* SAD device BDF */ + struct pci_bdf sad_all_bdf; + /* PCU device BDF */ + struct pci_bdf pcu_cr3_bdf; + /* UTIL device BDF */ + struct pci_bdf util_all_bdf; + /* URACU device BDF */ + struct pci_bdf uracu_bdf; + /* DDR mdev device BDF */ + struct pci_bdf ddr_mdev_bdf; + /* HBM mdev device BDF */ + struct pci_bdf hbm_mdev_bdf; int sad_all_offset; /* Offsets of retry_rd_err_log registers */ u32 *offsets_scrub; From patchwork Wed Jul 26 05:59:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812969 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=Cxwbxddp; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jtL4mdNz1ydm for ; Wed, 26 Jul 2023 16:00:26 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZN-0003mY-4l; Wed, 26 Jul 2023 06:00:21 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZJ-0003iB-BQ for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:17 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 16F2A42A9B for ; Wed, 26 Jul 2023 06:00:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351215; bh=vtiAvGXQzBP3eaIQuU0u7AeH5tPvCaeQKuhJLmUM2iU=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CxwbxddpqIe93lg85ikFlx5DZBBHalEVMYXnvf7bqbQVQkK8TnA+c/7J+jMfyLG2O +srwDP1p4p7xGwqcORqG7IibSQgT4UsGjRoaVu8/DveFbfuwqiON8TvVmObsJUQi02 LaMzQiYeQ8tbbaQ3QrJbnzDAx6aE24WXiAHrO183zvei8tZJbZ9SS+utHfJEmJFiEr DIJKpoB4h+nQ3Dwww1+Le8fW0FksxfkClFzgDkdHGufGKYVcy2z5Ju7388QHR00ucn BTofyVdCm7R6itwUl59BaHbI+/5w62uPSFx4rCs/k0EIXqBtLMuKyBtiJq4yJOPbWl py4MI6hcsi6Mw== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L] [PATCH 5/6] EDAC/i10nm: Add Intel Granite Rapids server support Date: Wed, 26 Jul 2023 13:59:16 +0800 Message-Id: <20230726055917.3291633-6-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 The Granite Rapids CPU model uses similar memory controller registers as Sapphire Rapids server but with some different configurations: - Various memory controller numbers for different Granite Rapids CPUs. So detect the number of present memory controllers at run time. - Different MMIO offsets of memory controllers. - Different triples of bus/dev/fun of some PCI devices used in i10nm_edac. Add above configurations and Granite Rapids CPU model ID for EDAC support. [Tony: Fixed 2 typos s/strcture/structure/] Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/all/20230113032802.41752-1-qiuxu.zhuo@intel.com (cherry picked from commit ba987eaaabf99b462cdfed86274e3455d5126349) Signed-off-by: Kai-Heng Feng --- drivers/edac/i10nm_base.c | 237 ++++++++++++++++++++++++++++++++++---- drivers/edac/skx_common.h | 5 +- 2 files changed, 217 insertions(+), 25 deletions(-) diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c index 5682d64cceb6..46034310b78e 100644 --- a/drivers/edac/i10nm_base.c +++ b/drivers/edac/i10nm_base.c @@ -13,7 +13,7 @@ #include "edac_module.h" #include "skx_common.h" -#define I10NM_REVISION "v0.0.5" +#define I10NM_REVISION "v0.0.6" #define EDAC_MOD_STR "i10nm_edac" /* Debug macros */ @@ -22,25 +22,34 @@ #define I10NM_GET_SCK_BAR(d, reg) \ pci_read_config_dword((d)->uracu, 0xd0, &(reg)) -#define I10NM_GET_IMC_BAR(d, i, reg) \ - pci_read_config_dword((d)->uracu, 0xd8 + (i) * 4, &(reg)) +#define I10NM_GET_IMC_BAR(d, i, reg) \ + pci_read_config_dword((d)->uracu, \ + (res_cfg->type == GNR ? 0xd4 : 0xd8) + (i) * 4, &(reg)) #define I10NM_GET_SAD(d, offset, i, reg)\ - pci_read_config_dword((d)->sad_all, (offset) + (i) * 8, &(reg)) + pci_read_config_dword((d)->sad_all, (offset) + (i) * \ + (res_cfg->type == GNR ? 12 : 8), &(reg)) #define I10NM_GET_HBM_IMC_BAR(d, reg) \ pci_read_config_dword((d)->uracu, 0xd4, &(reg)) #define I10NM_GET_CAPID3_CFG(d, reg) \ - pci_read_config_dword((d)->pcu_cr3, 0x90, &(reg)) + pci_read_config_dword((d)->pcu_cr3, \ + res_cfg->type == GNR ? 0x290 : 0x90, &(reg)) +#define I10NM_GET_CAPID5_CFG(d, reg) \ + pci_read_config_dword((d)->pcu_cr3, \ + res_cfg->type == GNR ? 0x298 : 0x98, &(reg)) #define I10NM_GET_DIMMMTR(m, i, j) \ - readl((m)->mbase + ((m)->hbm_mc ? 0x80c : 0x2080c) + \ + readl((m)->mbase + ((m)->hbm_mc ? 0x80c : \ + (res_cfg->type == GNR ? 0xc0c : 0x2080c)) + \ (i) * (m)->chan_mmio_sz + (j) * 4) #define I10NM_GET_MCDDRTCFG(m, i) \ readl((m)->mbase + ((m)->hbm_mc ? 0x970 : 0x20970) + \ (i) * (m)->chan_mmio_sz) #define I10NM_GET_MCMTR(m, i) \ - readl((m)->mbase + ((m)->hbm_mc ? 0xef8 : 0x20ef8) + \ + readl((m)->mbase + ((m)->hbm_mc ? 0xef8 : \ + (res_cfg->type == GNR ? 0xaf8 : 0x20ef8)) + \ (i) * (m)->chan_mmio_sz) #define I10NM_GET_AMAP(m, i) \ - readl((m)->mbase + ((m)->hbm_mc ? 0x814 : 0x20814) + \ + readl((m)->mbase + ((m)->hbm_mc ? 0x814 : \ + (res_cfg->type == GNR ? 0xc14 : 0x20814)) + \ (i) * (m)->chan_mmio_sz) #define I10NM_GET_REG32(m, i, offset) \ readl((m)->mbase + (i) * (m)->chan_mmio_sz + (offset)) @@ -56,7 +65,10 @@ #define I10NM_GET_HBM_IMC_MMIO_OFFSET(reg) \ ((GET_BITFIELD(reg, 0, 10) << 12) + 0x140000) +#define I10NM_GNR_IMC_MMIO_OFFSET 0x24c000 +#define I10NM_GNR_IMC_MMIO_SIZE 0x4000 #define I10NM_HBM_IMC_MMIO_SIZE 0x9000 +#define I10NM_DDR_IMC_CH_CNT(reg) GET_BITFIELD(reg, 21, 24) #define I10NM_IS_HBM_PRESENT(reg) GET_BITFIELD(reg, 27, 30) #define I10NM_IS_HBM_IMC(reg) GET_BITFIELD(reg, 29, 29) @@ -323,6 +335,79 @@ static struct pci_dev *pci_get_dev_wrapper(int dom, unsigned int bus, return pdev; } +/** + * i10nm_get_imc_num() - Get the number of present DDR memory controllers. + * + * @cfg : The pointer to the structure of EDAC resource configurations. + * + * For Granite Rapids CPUs, the number of present DDR memory controllers read + * at runtime overwrites the value statically configured in @cfg->ddr_imc_num. + * For other CPUs, the number of present DDR memory controllers is statically + * configured in @cfg->ddr_imc_num. + * + * RETURNS : 0 on success, < 0 on failure. + */ +static int i10nm_get_imc_num(struct res_config *cfg) +{ + int n, imc_num, chan_num = 0; + struct skx_dev *d; + u32 reg; + + list_for_each_entry(d, i10nm_edac_list, list) { + d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->pcu_cr3_bdf.bus], + res_cfg->pcu_cr3_bdf.dev, + res_cfg->pcu_cr3_bdf.fun); + if (!d->pcu_cr3) + continue; + + if (I10NM_GET_CAPID5_CFG(d, reg)) + continue; + + n = I10NM_DDR_IMC_CH_CNT(reg); + + if (!chan_num) { + chan_num = n; + edac_dbg(2, "Get DDR CH number: %d\n", chan_num); + } else if (chan_num != n) { + i10nm_printk(KERN_NOTICE, "Get DDR CH numbers: %d, %d\n", chan_num, n); + } + } + + switch (cfg->type) { + case GNR: + /* + * One channel per DDR memory controller for Granite Rapids CPUs. + */ + imc_num = chan_num; + + if (!imc_num) { + i10nm_printk(KERN_ERR, "Invalid DDR MC number\n"); + return -ENODEV; + } + + if (imc_num > I10NM_NUM_DDR_IMC) { + i10nm_printk(KERN_ERR, "Need to make I10NM_NUM_DDR_IMC >= %d\n", imc_num); + return -EINVAL; + } + + if (cfg->ddr_imc_num != imc_num) { + /* + * Store the number of present DDR memory controllers. + */ + cfg->ddr_imc_num = imc_num; + edac_dbg(2, "Set DDR MC number: %d", imc_num); + } + + return 0; + default: + /* + * For other CPUs, the number of present DDR memory controllers + * is statically pre-configured in cfg->ddr_imc_num. + */ + return 0; + } +} + static bool i10nm_check_2lm(struct res_config *cfg) { struct skx_dev *d; @@ -445,6 +530,98 @@ static bool i10nm_mc_decode(struct decoded_addr *res) return true; } +/** + * get_gnr_mdev() - Get the PCI device of the @logical_idx-th DDR memory controller. + * + * @d : The pointer to the structure of CPU socket EDAC device. + * @logical_idx : The logical index of the present memory controller (0 ~ max present MC# - 1). + * @physical_idx : To store the corresponding physical index of @logical_idx. + * + * RETURNS : The PCI device of the @logical_idx-th DDR memory controller, NULL on failure. + */ +static struct pci_dev *get_gnr_mdev(struct skx_dev *d, int logical_idx, int *physical_idx) +{ +#define GNR_MAX_IMC_PCI_CNT 28 + + struct pci_dev *mdev; + int i, logical = 0; + + /* + * Detect present memory controllers from { PCI device: 8-5, function 7-1 } + */ + for (i = 0; i < GNR_MAX_IMC_PCI_CNT; i++) { + mdev = pci_get_dev_wrapper(d->seg, + d->bus[res_cfg->ddr_mdev_bdf.bus], + res_cfg->ddr_mdev_bdf.dev + i / 7, + res_cfg->ddr_mdev_bdf.fun + i % 7); + + if (mdev) { + if (logical == logical_idx) { + *physical_idx = i; + return mdev; + } + + pci_dev_put(mdev); + logical++; + } + } + + return NULL; +} + +/** + * get_ddr_munit() - Get the resource of the i-th DDR memory controller. + * + * @d : The pointer to the structure of CPU socket EDAC device. + * @i : The index of the CPU socket relative DDR memory controller. + * @offset : To store the MMIO offset of the i-th DDR memory controller. + * @size : To store the MMIO size of the i-th DDR memory controller. + * + * RETURNS : The PCI device of the i-th DDR memory controller, NULL on failure. + */ +static struct pci_dev *get_ddr_munit(struct skx_dev *d, int i, u32 *offset, unsigned long *size) +{ + struct pci_dev *mdev; + int physical_idx; + u32 reg; + + switch (res_cfg->type) { + case GNR: + if (I10NM_GET_IMC_BAR(d, 0, reg)) { + i10nm_printk(KERN_ERR, "Failed to get mc0 bar\n"); + return NULL; + } + + mdev = get_gnr_mdev(d, i, &physical_idx); + if (!mdev) + return NULL; + + *offset = I10NM_GET_IMC_MMIO_OFFSET(reg) + + I10NM_GNR_IMC_MMIO_OFFSET + + physical_idx * I10NM_GNR_IMC_MMIO_SIZE; + *size = I10NM_GNR_IMC_MMIO_SIZE; + + break; + default: + if (I10NM_GET_IMC_BAR(d, i, reg)) { + i10nm_printk(KERN_ERR, "Failed to get mc%d bar\n", i); + return NULL; + } + + mdev = pci_get_dev_wrapper(d->seg, + d->bus[res_cfg->ddr_mdev_bdf.bus], + res_cfg->ddr_mdev_bdf.dev + i, + res_cfg->ddr_mdev_bdf.fun); + if (!mdev) + return NULL; + + *offset = I10NM_GET_IMC_MMIO_OFFSET(reg); + *size = I10NM_GET_IMC_MMIO_SIZE(reg); + } + + return mdev; +} + static int i10nm_get_ddr_munits(void) { struct pci_dev *mdev; @@ -478,9 +655,8 @@ static int i10nm_get_ddr_munits(void) j++, base, reg); for (i = 0; i < res_cfg->ddr_imc_num; i++) { - mdev = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->ddr_mdev_bdf.bus], - res_cfg->ddr_mdev_bdf.dev + i, - res_cfg->ddr_mdev_bdf.fun); + mdev = get_ddr_munit(d, i, &off, &size); + if (i == 0 && !mdev) { i10nm_printk(KERN_ERR, "No IMC found\n"); return -ENODEV; @@ -490,13 +666,6 @@ static int i10nm_get_ddr_munits(void) d->imc[i].mdev = mdev; - if (I10NM_GET_IMC_BAR(d, i, reg)) { - i10nm_printk(KERN_ERR, "Failed to get mc bar\n"); - return -ENODEV; - } - - off = I10NM_GET_IMC_MMIO_OFFSET(reg); - size = I10NM_GET_IMC_MMIO_SIZE(reg); edac_dbg(2, "mc%d mmio base 0x%llx size 0x%lx (reg 0x%x)\n", i, base + off, size, reg); @@ -536,9 +705,6 @@ static int i10nm_get_hbm_munits(void) u64 base; list_for_each_entry(d, i10nm_edac_list, list) { - d->pcu_cr3 = pci_get_dev_wrapper(d->seg, d->bus[res_cfg->pcu_cr3_bdf.bus], - res_cfg->pcu_cr3_bdf.dev, - res_cfg->pcu_cr3_bdf.fun); if (!d->pcu_cr3) return -ENODEV; @@ -678,6 +844,23 @@ static struct res_config spr_cfg = { .offsets_demand_hbm1 = offsets_demand_spr_hbm1, }; +static struct res_config gnr_cfg = { + .type = GNR, + .decs_did = 0x3252, + .busno_cfg_offset = 0xd0, + .ddr_imc_num = 12, + .ddr_chan_num = 1, + .ddr_dimm_num = 2, + .ddr_chan_mmio_sz = 0x4000, + .support_ddr5 = true, + .sad_all_bdf = {0, 13, 0}, + .pcu_cr3_bdf = {0, 5, 0}, + .util_all_bdf = {0, 13, 1}, + .uracu_bdf = {0, 0, 1}, + .ddr_mdev_bdf = {0, 5, 1}, + .sad_all_offset = 0x300, +}; + static const struct x86_cpu_id i10nm_cpuids[] = { X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ATOM_TREMONT_D, X86_STEPPINGS(0x0, 0x3), &i10nm_cfg0), X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ATOM_TREMONT_D, X86_STEPPINGS(0x4, 0xf), &i10nm_cfg1), @@ -686,6 +869,7 @@ static const struct x86_cpu_id i10nm_cpuids[] = { X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(ICELAKE_D, X86_STEPPINGS(0x0, 0xf), &i10nm_cfg1), X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(SAPPHIRERAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg), X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(EMERALDRAPIDS_X, X86_STEPPINGS(0x0, 0xf), &spr_cfg), + X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS(GRANITERAPIDS_X, X86_STEPPINGS(0x0, 0xf), &gnr_cfg), {} }; MODULE_DEVICE_TABLE(x86cpu, i10nm_cpuids); @@ -705,7 +889,7 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci, { struct skx_pvt *pvt = mci->pvt_info; struct skx_imc *imc = pvt->imc; - u32 mtr, amap, mcddrtcfg; + u32 mtr, amap, mcddrtcfg = 0; struct dimm_info *dimm; int i, j, ndimms; @@ -715,7 +899,10 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci, ndimms = 0; amap = I10NM_GET_AMAP(imc, i); - mcddrtcfg = I10NM_GET_MCDDRTCFG(imc, i); + + if (res_cfg->type != GNR) + mcddrtcfg = I10NM_GET_MCDDRTCFG(imc, i); + for (j = 0; j < imc->num_dimms; j++) { dimm = edac_get_dimm(mci, i, j, 0); mtr = I10NM_GET_DIMMMTR(imc, i, j); @@ -834,6 +1021,10 @@ static int __init i10nm_init(void) return -ENODEV; } + rc = i10nm_get_imc_num(cfg); + if (rc < 0) + goto fail; + mem_cfg_2lm = i10nm_check_2lm(cfg); skx_set_mem_cfg(mem_cfg_2lm); diff --git a/drivers/edac/skx_common.h b/drivers/edac/skx_common.h index 982e1bcb1edf..b6d3607dffe2 100644 --- a/drivers/edac/skx_common.h +++ b/drivers/edac/skx_common.h @@ -33,7 +33,7 @@ #define SKX_NUM_CHANNELS 3 /* Channels per memory controller */ #define SKX_NUM_DIMMS 2 /* Max DIMMS per channel */ -#define I10NM_NUM_DDR_IMC 4 +#define I10NM_NUM_DDR_IMC 12 #define I10NM_NUM_DDR_CHANNELS 2 #define I10NM_NUM_DDR_DIMMS 2 @@ -129,7 +129,8 @@ struct skx_pvt { enum type { SKX, I10NM, - SPR + SPR, + GNR }; enum { From patchwork Wed Jul 26 05:59:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kai-Heng Feng X-Patchwork-Id: 1812970 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=FpnXUhpJ; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R9jtR1tMSz1yZv for ; Wed, 26 Jul 2023 16:00:31 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qOXZR-0003uN-Rr; Wed, 26 Jul 2023 06:00:25 +0000 Received: from smtp-relay-canonical-1.internal ([10.131.114.174] helo=smtp-relay-canonical-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qOXZL-0003ki-KB for kernel-team@lists.ubuntu.com; Wed, 26 Jul 2023 06:00:19 +0000 Received: from localhost.localdomain (unknown [10.101.196.174]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-1.canonical.com (Postfix) with ESMTPSA id 2F04642A9B for ; Wed, 26 Jul 2023 06:00:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1690351219; bh=GaV4mvus0YD8Nqt/4gAtEtfdxq/JwNngg2lly4JtNps=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FpnXUhpJO8jRXxnkpFzWZFTZWeLBUtqq7JUhxmvslMFN9xtX73Y4qM8XyvUao9tgX e7AsdgRj0ceLgEkfDUA2iPpO7fTItEr3EZahXysc/IJCFBcTKdCTANK6t7L2ah189f xQPugOJSgUq9QzoeWUV8ngdJZ7I87FwknKw9+VB80q69CG1sAFduccSnWj7RdTa9sI u2i3937Fh8/1DyeqWEw1StMszb8bKq6lVLtdyStkCU+dyxzLs2MRd4pWQzYxerrgN3 ANpLvnHGKt9coAzB7X91SLRxhpO0bPGNLa3lx0L9KeJotu4diHFVZJy9KFLsxOP3dA N3sJ/lzA/aNiA== From: Kai-Heng Feng To: kernel-team@lists.ubuntu.com Subject: [L/Unstable] [PATCH 6/6] EDAC/i10nm: Skip the absent memory controllers Date: Wed, 26 Jul 2023 13:59:17 +0800 Message-Id: <20230726055917.3291633-7-kai.heng.feng@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230726055917.3291633-1-kai.heng.feng@canonical.com> References: <20230726055917.3291633-1-kai.heng.feng@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Qiuxu Zhuo BugLink: https://bugs.launchpad.net/bugs/2028746 Some Sapphire Rapids workstations' absent memory controllers still appear as PCIe devices that fool the i10nm_edac driver and result in "shift exponent -66 is negative" call traces from skx_get_dimm_info(). Skip the absent memory controllers to avoid the call traces. Reported-by: Kai-Heng Feng Closes: https://lore.kernel.org/linux-edac/CAAd53p41Ku1m1rapeqb1xtD+kKuk+BaUW=dumuoF0ZO3GhFjFA@mail.gmail.com/T/#m5de16dce60a8c836ec235868c7c16e3fefad0cc2 Tested-by: Kai-Heng Feng Reported-by: Koba Ko Closes: https://lore.kernel.org/linux-edac/SA1PR11MB71305B71CCCC3D9305835202892AA@SA1PR11MB7130.namprd11.prod.outlook.com/T/#t Tested-by: Koba Ko Fixes: d4dc89d069aa ("EDAC, i10nm: Add a driver for Intel 10nm server processors") Signed-off-by: Qiuxu Zhuo Signed-off-by: Tony Luck Link: https://lore.kernel.org/r/20230710013232.59712-1-qiuxu.zhuo@intel.com (cherry picked from commit c545f5e412250555bd4e717d062b117f20bab418 linux-next) Signed-off-by: Kai-Heng Feng --- drivers/edac/i10nm_base.c | 54 +++++++++++++++++++++++++++++++++++---- 1 file changed, 49 insertions(+), 5 deletions(-) diff --git a/drivers/edac/i10nm_base.c b/drivers/edac/i10nm_base.c index 46034310b78e..7ed98b85aa90 100644 --- a/drivers/edac/i10nm_base.c +++ b/drivers/edac/i10nm_base.c @@ -622,13 +622,49 @@ static struct pci_dev *get_ddr_munit(struct skx_dev *d, int i, u32 *offset, unsi return mdev; } +/** + * i10nm_imc_absent() - Check whether the memory controller @imc is absent + * + * @imc : The pointer to the structure of memory controller EDAC device. + * + * RETURNS : true if the memory controller EDAC device is absent, false otherwise. + */ +static bool i10nm_imc_absent(struct skx_imc *imc) +{ + u32 mcmtr; + int i; + + switch (res_cfg->type) { + case SPR: + for (i = 0; i < res_cfg->ddr_chan_num; i++) { + mcmtr = I10NM_GET_MCMTR(imc, i); + edac_dbg(1, "ch%d mcmtr reg %x\n", i, mcmtr); + if (mcmtr != ~0) + return false; + } + + /* + * Some workstations' absent memory controllers still + * appear as PCIe devices, misleading the EDAC driver. + * By observing that the MMIO registers of these absent + * memory controllers consistently hold the value of ~0. + * + * We identify a memory controller as absent by checking + * if its MMIO register "mcmtr" == ~0 in all its channels. + */ + return true; + default: + return false; + } +} + static int i10nm_get_ddr_munits(void) { struct pci_dev *mdev; void __iomem *mbase; unsigned long size; struct skx_dev *d; - int i, j = 0; + int i, lmc, j = 0; u32 reg, off; u64 base; @@ -654,7 +690,7 @@ static int i10nm_get_ddr_munits(void) edac_dbg(2, "socket%d mmio base 0x%llx (reg 0x%x)\n", j++, base, reg); - for (i = 0; i < res_cfg->ddr_imc_num; i++) { + for (lmc = 0, i = 0; i < res_cfg->ddr_imc_num; i++) { mdev = get_ddr_munit(d, i, &off, &size); if (i == 0 && !mdev) { @@ -664,8 +700,6 @@ static int i10nm_get_ddr_munits(void) if (!mdev) continue; - d->imc[i].mdev = mdev; - edac_dbg(2, "mc%d mmio base 0x%llx size 0x%lx (reg 0x%x)\n", i, base + off, size, reg); @@ -676,7 +710,17 @@ static int i10nm_get_ddr_munits(void) return -ENODEV; } - d->imc[i].mbase = mbase; + d->imc[lmc].mbase = mbase; + if (i10nm_imc_absent(&d->imc[lmc])) { + pci_dev_put(mdev); + iounmap(mbase); + d->imc[lmc].mbase = NULL; + edac_dbg(2, "Skip absent mc%d\n", i); + continue; + } else { + d->imc[lmc].mdev = mdev; + lmc++; + } } }