From patchwork Fri May 31 21:38:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942315 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=qjNzGb37; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-pci+bounces-8143-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc2k0DXBz20PW for ; Sat, 1 Jun 2024 07:39:26 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id AA5952885AC for ; Fri, 31 May 2024 21:39:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4C05F1DDD1; Fri, 31 May 2024 21:39:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="qjNzGb37" X-Original-To: linux-pci@vger.kernel.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2062.outbound.protection.outlook.com [40.107.94.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 991A178C9C; Fri, 31 May 2024 21:39:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191550; cv=fail; b=alkNuFLeke4iNnETjiY1CPuMfMw6WmcXdILgj2Ei2CUiFBbjsPVRAjqwuMErw+k7g37U/a9w/4DxqGNexcvAVo+JOmf0EdbG7SWvm2O+OQE4jxkhb7WFLO4G4n++OSpMUSS/qnAnbLa0j9Cv60zqbl2r8ps5XqinJPz0vVsJiQg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191550; c=relaxed/simple; bh=x4iJYIjZuoFqT+tWZfr25SAwOXOd8EuwGhhWsOBO3Ic=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ya3febT7AZdc3do+Df4y+/8P7laTZGw7gdtHgq66qwNHbzKjSPpOMJjW3FfK9160YJhYKjbqLQdAEBLTmNu6uzZgN9aovS9h+mjH2J1nu5yw7IQB9iKrfNMPXrUdPQhsl2n1L4ndhU2XVQ524adODieV2zey6QdS0rIMmbEWuoQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=qjNzGb37; arc=fail smtp.client-ip=40.107.94.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UKLIUsvifyP1XYwwO7uxHIhfg5eaeIyvBrf+ISYaa+Hnqe1xwT04Ylom4Qugt7vNciypb6reG9AWJNWH3qz3jhwDYaRnMWrLUbak4QbvpTgiaFAddN2zfIRfXWp1lBKq/NMOemjeR4udI0hxPfIOzDL6gPlQPj8FDr4W+eHMriMpIx4thsX8vjial0iy/VGy2EzkC45wGTgTVvdIh90/71MwEanplLLRXmmChbIOAzkc7Ca3QGYyejBs/w4jGEEvusO/0qZgVestaTI2Ki/C0AWeDdp+vKXvDxLhevPbF8c5VG5fW6J8b1THrZMPhL82MzLv+BbMXSvl6vLU6YMHBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HvLzinwUQqz7Xakud+KTfFCb38kjnyCVKHafbku1Xo8=; b=Jw4kKSxUZVn6GQ1KMTBr7csh4pJ3dkfni3HyKd5pGK0lDO88NG7XxRgREcX7o7zU95IzwyEQ5WVhXRms2oGmbNaCeCTcGQ01VabP9Eq+y6FnbjzZtuR0rRU9xLm+Ql6SCzHotxV8B4THWewF9nEviYvt1Z6dkRNSgz4XD7MkiYZLm4tcsOXytmHXC4Qu7gu+7txPaBQmKUmRtNles8BA2AF1wbnkSIIsrgBOLccMinfYbYz0K4RY8Dp/aJ1C/oAKhSpI8cs5t2zhxidgX3NoXpGMgEUnPsmao674w1mZ85GGnt8wMCWmyqJI5blcL75zfeyrDSRy/qXs+IdGJ/Pa3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HvLzinwUQqz7Xakud+KTfFCb38kjnyCVKHafbku1Xo8=; b=qjNzGb37cOr06bbQNfMqoCffuRqJuSqj5HR3dXXlzEM7noJWRVAm9pL4bZThzpj+OtHDN1rC5edqj4D6Kmn6/X5/dIWCyitpxgep/Tbu2n9hHgZlJ35E0ypY2ajoE9EzfRo6igebqZg+QlVraD10qveed8cqkwcK/WKCAfwMszY= Received: from PH8PR20CA0021.namprd20.prod.outlook.com (2603:10b6:510:23c::26) by PH8PR12MB8431.namprd12.prod.outlook.com (2603:10b6:510:25a::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.29; Fri, 31 May 2024 21:39:05 +0000 Received: from CY4PEPF0000EE34.namprd05.prod.outlook.com (2603:10b6:510:23c:cafe::76) by PH8PR20CA0021.outlook.office365.com (2603:10b6:510:23c::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.25 via Frontend Transport; Fri, 31 May 2024 21:39:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE34.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:39:04 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:39:03 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 1/9] PCI: Introduce PCIe TPH support framework Date: Fri, 31 May 2024 16:38:33 -0500 Message-ID: <20240531213841.3246055-2-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE34:EE_|PH8PR12MB8431:EE_ X-MS-Office365-Filtering-Correlation-Id: 31e8a3cc-48f3-4111-cf9a-08dc81ba180d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|7416005|82310400017|376005|36860700004; X-Microsoft-Antispam-Message-Info: UQZSw/Rb9sj7JtGnyWuR2j9D474sYzPaM2LB+ukww+Xo679y2a2SNEA4yHDHADqVFdogukd82OPM4cIip/gEQMvqWuoSuDahMvqm5d1ftU9+GCa7JyZU47XZB5uW5V9PKY3KOiWYJ6FaaYaHt5I5hoOov/0giSqcvfQ7H9ecLJac9AmRj+0/d9GvDDejdRGZRrLN4w01vqOu95PWL0jx/QpHX0lnyBGydBh9nizfUKh3zAhsFRyu/dQP5EWrDoquW71kPuCdQZ98KKQWV/dY5jlnZhT9891gt24W4IvCmqdOkAgx0xqxQV7aNcuR9rapF4PmPjHemj30H0Sxgj2cmS+cUJEZEzyXC0y2dt5DXT4PWrAuO4GfC4B7nOzgudTBH+LSWCVb+O7oF7U3zKRRgd8IE8vhO2sxvR0VMtT2Q+D7iMILYsml/wgQWPTXfmuklVaLTCO1Dendr1UXyX/Ore+38RTGK4kL6CHwS9zYb2UPlwWyv9Ov4IOYUfgAqRv8nxwjuXn+j33QGMYkqYUAbwZT3srRJrBOBd3ki5TlEwogCxiHC7dJIK8pHf1mpI6T3ycZCkfCdKgbWnyeZiMDcsdWi+L0yPli4j9smKx65lqb7gDHzUeHWw1xXJ88U39S7T2kDDANu0MaUE4NhtqhHmpo0GBxPweHCFmI4C8bKcnkPiM03/vFescM2Lp9RbeRlK8aI3qPc5Q2qadpAu4aYmSOWRMbrLakJ0sa2CSHEOdTjmtuv6QgfRghYH6oupp8n/PG9HqZurmUohHe4X3cQRwc/ivnYPYjmpgNBZ3wROFZLVNE8vRTFifSIm/buxPZwxhpjKNrTTGIypW+LLIMwLMIvbZEu6Ln01xl8AwTE4euTLH0i2LLTMzzz+FUKoIlQkiq1/2W+agh9NkUo9sRcT+cSMSNg7RClxzddLEAPaTq3JKVbZTKF/1MeYJtxhGHHAV6D19gHFTJ3Ge5V5SDg0dIcbjZKSSF07oOot7hPyGiqw0uo9A5gWBPr3IcwSpVwfEf8imT5hL8V2nI2+Twn3X+jYtETrl7MSJgRRhAe3Q3VKLNBZ5Tawtpg2Nd09pnKAe5bL+z+9JJDiBglFYXNVyKFCp1B5wFFPbRdp2rtaxU/IKiN3YbLqk5uQn6361WPfuXOmiSf9K6nfieaDmSci/Ht22DADXN3CO0H9tmf57uzq9Wb6hnuPw79xtCZgdmewqTwf/HKETmI81055SVl9zGeeJuvA6KbFtZrsmmkYIYCEiCd5pnSFBV0z4lms6oi+aU3p3jy1mN9sjgPTr78k5BzLxgYsr2mjBFr8p46L/JNDVFMRO4Cxf6pzFDj+FyecT4yVMPTpSziENIUqRolwgWfPAzqEcd7FDff+hqeThdAc89mwlp2EaihGwUe0/x X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(7416005)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:39:04.6132 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 31e8a3cc-48f3-4111-cf9a-08dc81ba180d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE34.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB8431 This patch implements the framework for PCIe TPH support. It introduces tph.c source file, along with CONFIG_PCIE_TPH, to Linux PCIe subsystem. A new member, named tph_cap, is also introduced in pci_dev to cache TPH capability offset. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek Reviewed-by: Jonathan Cameron --- drivers/pci/pci.h | 6 ++++++ drivers/pci/pcie/Kconfig | 10 ++++++++++ drivers/pci/pcie/Makefile | 1 + drivers/pci/pcie/tph.c | 28 ++++++++++++++++++++++++++++ drivers/pci/probe.c | 1 + include/linux/pci.h | 4 ++++ 6 files changed, 50 insertions(+) create mode 100644 drivers/pci/pcie/tph.c diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index fd44565c4756..b371b5b45f86 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -506,6 +506,12 @@ static inline int pci_iov_bus_range(struct pci_bus *bus) #endif /* CONFIG_PCI_IOV */ +#ifdef CONFIG_PCIE_TPH +void pcie_tph_init(struct pci_dev *dev); +#else +static inline void pcie_tph_init(struct pci_dev *dev) {} +#endif + #ifdef CONFIG_PCIE_PTM void pci_ptm_init(struct pci_dev *dev); void pci_save_ptm_state(struct pci_dev *dev); diff --git a/drivers/pci/pcie/Kconfig b/drivers/pci/pcie/Kconfig index 17919b99fa66..d22857325b3e 100644 --- a/drivers/pci/pcie/Kconfig +++ b/drivers/pci/pcie/Kconfig @@ -155,3 +155,13 @@ config PCIE_EDR the PCI Firmware Specification r3.2. Enable this if you want to support hybrid DPC model which uses both firmware and OS to implement DPC. + +config PCIE_TPH + bool "TLP Processing Hints" + default n + help + This option adds support for PCIE TLP Processing Hints (TPH). + TPH allows endpoint devices to provide optimization hints, such as + desired caching behavior, for requests that target memory space. + These hints, called steering tags, can empower the system hardware + to optimize the utilization of platform resources. diff --git a/drivers/pci/pcie/Makefile b/drivers/pci/pcie/Makefile index 6461aa93fe76..3542b42ea0b9 100644 --- a/drivers/pci/pcie/Makefile +++ b/drivers/pci/pcie/Makefile @@ -13,3 +13,4 @@ obj-$(CONFIG_PCIE_PME) += pme.o obj-$(CONFIG_PCIE_DPC) += dpc.o obj-$(CONFIG_PCIE_PTM) += ptm.o obj-$(CONFIG_PCIE_EDR) += edr.o +obj-$(CONFIG_PCIE_TPH) += tph.o diff --git a/drivers/pci/pcie/tph.c b/drivers/pci/pcie/tph.c new file mode 100644 index 000000000000..5f0cc06b74bb --- /dev/null +++ b/drivers/pci/pcie/tph.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * TPH (TLP Processing Hints) support + * + * Copyright (C) 2024 Advanced Micro Devices, Inc. + * Eric Van Tassell + * Wei Huang + */ + +#define pr_fmt(fmt) "TPH: " fmt +#define dev_fmt pr_fmt + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../pci.h" + +void pcie_tph_init(struct pci_dev *dev) +{ + dev->tph_cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_TPH); +} + diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 15168881ec94..1f1ae55a5f83 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2484,6 +2484,7 @@ static void pci_init_capabilities(struct pci_dev *dev) pci_dpc_init(dev); /* Downstream Port Containment */ pci_rcec_init(dev); /* Root Complex Event Collector */ pci_doe_init(dev); /* Data Object Exchange */ + pcie_tph_init(dev); /* TLP Processing Hints */ pcie_report_downtraining(dev); pci_init_reset_methods(dev); diff --git a/include/linux/pci.h b/include/linux/pci.h index 5bece7fd11f8..d75a88ec5136 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -530,6 +530,10 @@ struct pci_dev { /* These methods index pci_reset_fn_methods[] */ u8 reset_methods[PCI_NUM_RESET_METHODS]; /* In priority order */ + +#ifdef CONFIG_PCIE_TPH + u16 tph_cap; /* TPH capability offset */ +#endif }; static inline struct pci_dev *pci_physfn(struct pci_dev *dev) From patchwork Fri May 31 21:38:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942316 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=Ul4xMccU; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-pci+bounces-8144-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc350zPFz20PW for ; Sat, 1 Jun 2024 07:39:45 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 60E4328890F for ; Fri, 31 May 2024 21:39:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7ADD478C9C; Fri, 31 May 2024 21:39:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Ul4xMccU" X-Original-To: linux-pci@vger.kernel.org Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2043.outbound.protection.outlook.com [40.107.93.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4DB37FBA3; Fri, 31 May 2024 21:39:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191566; cv=fail; b=cgkbl5DgK7/iNgKO47qybSSK7WZ2LPlp0eU5FSp40Ye4L8TAwplHQ2Ius4r63oxZyauT4Q4oqcM8wCjdy4GjDcPrs/6WFhWAMWM4t+GZoiZZpG7pODbNNpjOLP1JNlPuv+Q07agadV2r8W9lAifgOX/WXtQanEA4cx90EHCTgFA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191566; c=relaxed/simple; bh=peZkMUNMc7CZ8IhQMFMcjFi9mhA+RU7+T60cSPEDw7c=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=poG9YeptOIiSzIRJ1RyQ2VFcN3P8pjED2yJnLr6siQFCz1J870r4kFLrWMNDE6VBLPlj3BzKy+fRjuedOoQx2c+raeaAPwy7tiVr80kFAWJuyWeaik16w4WLxcOCqdJuYSybovrGU8x+TivUG2bihAZE+JgdlwmjW0gfzL17+4E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Ul4xMccU; arc=fail smtp.client-ip=40.107.93.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=APYaqn9rPf1VeL/rtYh5p1xi3Zs+1ITUidv5obOlmX01vnlU07siGrSr5pccoTMOUuDvyhiByyVuu77ZO8abPmzQLuayZNH5rZUep1obZlSehVPuwPcw5OGMnuLLCLYd/f6jWb2fgU3ycstPCVcgsdAtiMAD+ljSU4EEJTezEOCS27O4JaYOv3sSdWEa2XyCe9plwMHMuk9IE9fmHCpo18AxV/f75aTqAZryA65WaM2fiSCDPqmMdke6qPsg47xcPYu14a4L5cGZRBY5h0Ch5JriedrvUJZQsS0xcJdcInOcuDDkQntCNtDK+sZ/kXN/5j7e8eyEfv3iH5C/0q8Umg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w6gRbxu50tgqllwlngZrnZIaLKZPG/fwRTXHfOd8cbA=; b=g7hTTfVtuIDsF9b25JIrvt9mT+Mac+g7D+8NlQLR+keigc2xyctxjjUDLGeFPtY1LdMQUsIy1i55yUIqqG99UWx8bKdIidjhb+Oxw/j9wA0RimjOl1RNykhGb1RfqIAEUH2RitwGJGz8hI1wkudpQaFACxjIE7fpAjZGEGYHtpvYGV33v8kp4PLTRl5YRVVwo55ugWm6MOlYecJtpUYK1QCQkKKR1jAAz3DovA4A6jC3PnmariXZ8NZt24E82iCOsjc5PoRM7lnVTfpzBDpU0mVYhqIaDgGfOWTDiQrvg/CmjoZ+1WHu2odwrBIZL2fSk9w+ijUkeZvyQ1W/cbrUfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w6gRbxu50tgqllwlngZrnZIaLKZPG/fwRTXHfOd8cbA=; b=Ul4xMccUNY641M7BbTfrpD4qvp3NVXKZKsCSo0gscO+ZQgMOE5M06K83ze67HcBtKe8niUcSSme7G8hHt9Jz2gSZU8aJ4v+klLDT0uDWIrQn9NhdVL+sz3pJIQXUGxVuF5nM/tNwWh8WDd77mZg1+Mo7jEovpHJXQapAT7h8N1k= Received: from PH8PR15CA0020.namprd15.prod.outlook.com (2603:10b6:510:2d2::20) by SJ0PR12MB8137.namprd12.prod.outlook.com (2603:10b6:a03:4e5::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.24; Fri, 31 May 2024 21:39:21 +0000 Received: from CY4PEPF0000EE31.namprd05.prod.outlook.com (2603:10b6:510:2d2:cafe::a7) by PH8PR15CA0020.outlook.office365.com (2603:10b6:510:2d2::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.25 via Frontend Transport; Fri, 31 May 2024 21:39:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE31.mail.protection.outlook.com (10.167.242.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:39:20 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:39:19 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 2/9] PCI: Add TPH related register definition Date: Fri, 31 May 2024 16:38:34 -0500 Message-ID: <20240531213841.3246055-3-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE31:EE_|SJ0PR12MB8137:EE_ X-MS-Office365-Filtering-Correlation-Id: 821a0dc3-80fb-45fa-205d-08dc81ba21ce X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|36860700004|1800799015|7416005|82310400017; X-Microsoft-Antispam-Message-Info: aTgNKeQpQjlZQDLdMjJwV1qsUAV5U3PgDVIUqX5Ykpogb5A3IcqkJDphyQzKMHO+HsQWhIyohgBJMEetZDiU736GU4ps3Vzns4wvLkJx6SjKIensvRi4Qn80sfruhltetcVscjkOHJx+zJsnj29Ts6Nzh60jt9MwvVbWtRWW136920BEh2Z0+W05GI2zo2sASXrEC6AD6U7qgzDyTm04FsyFt9UHW/i9z26mR1MPK3oLkRPdLwCaJMwJhasUWAr7pCETgmMOxb+LU9tv9GItsF1LWtXurUVBSrEHmYsvSinXpiNfiIsCX0cV7M5V9sddv/htIxjVdu7cW+kMZ3AcT76xqpsnfMOohGemQIdrsrd7mUrD7iKrYP1IouSDiIWE4mGaoAfA4qyrIeiOwBYcRZiKKrq0MuQ7W7JyOL/iwn0yC+95jrSxWjYX38Ht+BhpHx2e5w1Rn7BmAqGowa+LdZvXRnB/eyzR48ejGbA/QWNyToEWue93Y1kgdZFkPPslkmJ+cV0OgxyZUVDOYbu8xIG3V6qfFH1Rn+Eu5POt7q9UFNsJhgvbghynIidzGk7jSnjIahpfOf10Irtf4bM8wjefr7hNUbn6jrKeL/J0BDV/P+H2h7Iv6WQZx5zj6XxModJRReEI8lHYDyCoc2w06fd/IbTd5kaunUR6eHIUH9iaHPpspcVZ+f1SyGjSgt+wpB5QoglQbqMR6n0VR4fZt8I4BGtDU76lmkK0ziQKcBlWEE0/hmnPPHBwRCn5tnrqo08SopzOwNd4QTTXTnV/mE5awKbP17SDhPlmsQHMXe6g3Ov65bCdXbiy3wA4q7aE20DbXah0bCpj/jYl+Q1XzqzpAlVctxzH6fv7ZrfObibEHY52fFdC7LKMj6/cDBMlD0eHA0LJsJYnisqW1CI122gW796f+AW0igYUnTq1r5n0fBLACJxLR1HY/I14Uau1RzleMOmftKENNMMDzqkVjI7EB5X2v2vYoKyul0oa0riH3add2ZsHvD84usoYgtfAkpi0P+vx3NS7bo0r2QqXXIguwpWX4FP1zPovGcZRzWnCNGXzxq9KINkf12nK9Gigvr9PK+DuLGoyIs4wSw0l1ehNNY3XJHEIvZLmBd49sYDb8QL/N/RjPSloSh/+OcJsyfUzPKlj05CiW9i8zZBJ7to5B2VfHCBlKN5JhUcLuVfaPYRHr7ueZ4ss19TUXRYlk29+8gO4A7qFj/qil3qTL5lQFHfdO9XNKHyn/iBOel8mXFbGSgpLg0eIdByrJZanxryrC25mYRz+J+HcrNCdIk7xJrlr9xMTqWTQPJlK05n8y/OBW/qMsTuhrQRaVlwrgr5VA5ZKJOY3CYBiL2stIHW8TDrACWxxNdGu8ep8W+Ti/qivKEcWfN0CHzfcqU+k X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(7416005)(82310400017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:39:20.9731 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 821a0dc3-80fb-45fa-205d-08dc81ba21ce X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE31.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB8137 Linux has some basic, but incomplete, definition for the TPH Requester capability registers. Also the control registers of TPH Requester and the TPH Completer are missing. This patch adds all required definitions to support TPH enablement. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- drivers/vfio/pci/vfio_pci_config.c | 7 +++--- include/uapi/linux/pci_regs.h | 35 ++++++++++++++++++++++++++---- 2 files changed, 35 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c index 97422aafaa7b..de622cdfc2a4 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -1434,14 +1434,15 @@ static int vfio_ext_cap_len(struct vfio_pci_core_device *vdev, u16 ecap, u16 epo if (ret) return pcibios_err_to_errno(ret); - if ((dword & PCI_TPH_CAP_LOC_MASK) == PCI_TPH_LOC_CAP) { + if (((dword & PCI_TPH_CAP_LOC_MASK) >> PCI_TPH_CAP_LOC_SHIFT) + == PCI_TPH_LOC_CAP) { int sts; sts = dword & PCI_TPH_CAP_ST_MASK; sts >>= PCI_TPH_CAP_ST_SHIFT; - return PCI_TPH_BASE_SIZEOF + (sts * 2) + 2; + return PCI_TPH_ST_TABLE + (sts * 2) + 2; } - return PCI_TPH_BASE_SIZEOF; + return PCI_TPH_ST_TABLE; case PCI_EXT_CAP_ID_DVSEC: ret = pci_read_config_dword(pdev, epos + PCI_DVSEC_HEADER1, &dword); if (ret) diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index 94c00996e633..ae1cf048b04a 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -657,6 +657,7 @@ #define PCI_EXP_DEVCAP2_ATOMIC_COMP64 0x00000100 /* 64b AtomicOp completion */ #define PCI_EXP_DEVCAP2_ATOMIC_COMP128 0x00000200 /* 128b AtomicOp completion */ #define PCI_EXP_DEVCAP2_LTR 0x00000800 /* Latency tolerance reporting */ +#define PCI_EXP_DEVCAP2_TPH_COMP 0x00003000 /* TPH completer support */ #define PCI_EXP_DEVCAP2_OBFF_MASK 0x000c0000 /* OBFF support mechanism */ #define PCI_EXP_DEVCAP2_OBFF_MSG 0x00040000 /* New message signaling */ #define PCI_EXP_DEVCAP2_OBFF_WAKE 0x00080000 /* Re-use WAKE# for OBFF */ @@ -1020,15 +1021,41 @@ #define PCI_DPA_CAP_SUBSTATE_MASK 0x1F /* # substates - 1 */ #define PCI_DPA_BASE_SIZEOF 16 /* size with 0 substates */ +/* TPH Completer Support */ +#define PCI_EXP_DEVCAP2_TPH_COMP_SHIFT 12 +#define PCI_EXP_DEVCAP2_TPH_COMP_NONE 0x0 /* None */ +#define PCI_EXP_DEVCAP2_TPH_COMP_TPH_ONLY 0x1 /* TPH only */ +#define PCI_EXP_DEVCAP2_TPH_COMP_TPH_AND_EXT 0x3 /* TPH and Extended TPH */ + /* TPH Requester */ #define PCI_TPH_CAP 4 /* capability register */ +#define PCI_TPH_CAP_NO_ST 0x1 /* no ST mode supported */ +#define PCI_TPH_CAP_NO_ST_SHIFT 0x0 /* no ST mode supported shift */ +#define PCI_TPH_CAP_INT_VEC 0x2 /* interrupt vector mode supported */ +#define PCI_TPH_CAP_INT_VEC_SHIFT 0x1 /* interrupt vector mode supported shift */ +#define PCI_TPH_CAP_DS 0x4 /* device specific mode supported */ +#define PCI_TPH_CAP_DS_SHIFT 0x4 /* device specific mode supported shift */ #define PCI_TPH_CAP_LOC_MASK 0x600 /* location mask */ -#define PCI_TPH_LOC_NONE 0x000 /* no location */ -#define PCI_TPH_LOC_CAP 0x200 /* in capability */ -#define PCI_TPH_LOC_MSIX 0x400 /* in MSI-X */ +#define PCI_TPH_CAP_LOC_SHIFT 9 /* location shift */ +#define PCI_TPH_LOC_NONE 0x0 /* no ST Table */ +#define PCI_TPH_LOC_CAP 0x1 /* ST Table in extended capability */ +#define PCI_TPH_LOC_MSIX 0x2 /* ST table in MSI-X table */ #define PCI_TPH_CAP_ST_MASK 0x07FF0000 /* ST table mask */ #define PCI_TPH_CAP_ST_SHIFT 16 /* ST table shift */ -#define PCI_TPH_BASE_SIZEOF 0xc /* size with no ST table */ + +#define PCI_TPH_CTRL 0x8 /* control register */ +#define PCI_TPH_CTRL_MODE_SEL_MASK 0x7 /* ST Model Select mask */ +#define PCI_TPH_CTRL_MODE_SEL_SHIFT 0x0 /* ST Model Select shift */ +#define PCI_TPH_NO_ST_MODE 0x0 /* No ST Mode */ +#define PCI_TPH_INT_VEC_MODE 0x1 /* Interrupt Vector Mode */ +#define PCI_TPH_DEV_SPEC_MODE 0x2 /* Device Specific Mode */ +#define PCI_TPH_CTRL_REQ_EN_MASK 0x300 /* TPH Requester mask */ +#define PCI_TPH_CTRL_REQ_EN_SHIFT 8 /* TPH Requester shift */ +#define PCI_TPH_REQ_DISABLE 0x0 /* No TPH request allowed */ +#define PCI_TPH_REQ_TPH_ONLY 0x1 /* 8-bit TPH tags allowed */ +#define PCI_TPH_REQ_EXT_TPH 0x3 /* 16-bit TPH tags allowed */ + +#define PCI_TPH_ST_TABLE 0xc /* base of ST table */ /* Downstream Port Containment */ #define PCI_EXP_DPC_CAP 0x04 /* DPC Capability */ From patchwork Fri May 31 21:38:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942317 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=K+kaaRrt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-pci+bounces-8145-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc3V3jRmz20PW for ; Sat, 1 Jun 2024 07:40:06 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 418941C23DD6 for ; Fri, 31 May 2024 21:40:04 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 190837EEF5; Fri, 31 May 2024 21:39:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="K+kaaRrt" X-Original-To: linux-pci@vger.kernel.org Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2051.outbound.protection.outlook.com [40.107.102.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 487E382486; Fri, 31 May 2024 21:39:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191583; cv=fail; b=DQgo51SdNdjc51kttZJwgWGZLo2bgP9HG1xzQWo8ZSKZq5T2Q71etVVPS3rS6zDXX/HP6BZEJaxO3B20bjg/juFIlOtqs9SBxrBCqZq5aUNJxJdOu4qkbclgLVh6xQSRuYvf6ExivuPmi06AABsfxWCBjNfOoPFvUHt0pH23VL0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191583; c=relaxed/simple; bh=GrxYnukooNDUViI+nJNy8Janil6rg2fBr24u3k7GREk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WVHVTmZvZytajXpgUR2YhOkosDBEBtlKHlw0zRV1cpFhtdQY4axiCLcaixzGX+bk4VCq2xXy9lIg4Du4xooCzpfvglphAd3xJkTZmD6Cic2PsFJmYd6HGn+M6nxz6wy+3eFKWgZqr63/ocKYjAFtxtL4nxGnxmWBalJOPumYTjE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=K+kaaRrt; arc=fail smtp.client-ip=40.107.102.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DcvlsYLHNtgn7XJSMOxKK2hU1YfnFhU2G9QRJC4jrz0I/qfaANZiORxQmD4WcfNGi8XZpayGY1ngv0Vsh8u/bXL3AV8llYWwKasTvUSuzery9VtArMpmXWZ+8NwjSHyOnRDTHP/asJAc3LmMTzHRZMMwxDZf2ntKNE89XMRo68zizGFXn45v9uoOQErTFd9b3uV7ZHqHZrcs7aAs0g0+/ku5PVjS5b41k6LlYTTv1tOvB4jbcrIKRKeGUUOhMD1uKlgo9bJRpBnV8xoZ7zc+8uF5wfxmuxnAkCGxzjVvBjhDuYRxnYL2SH1tWe2TahUO2ljWCZOU5e2ydy1q2qh2Ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0BhhnLBD+BxpmDMXE4whS2a4wCEfWBpWo+i2ezdk+mw=; b=QN6KsT4sg7tNwS9+wSWKoCOi9JzEXyjjcT6943C04ZsUQn5OWqZ78pivVzNX0gVOtiUvajT4AxnSZNljFP4mWpb1Mc9HquQ1G96pBOF0AibcX75sEFLV1unT1kcGwrP5afAdj2PYrjM5pD1C8UST4EweQLXlgbMq5o0LQYJLvnxvuj44gV9Eq2DKRP1aNG0pGRSfrPsukSEEENlJIFZ7ei+WXEI73tVL6LWwqjdfFujvZ6WmvaUC5BBzfLuRuFlTPGISnIpjKNv0tWlwO51YVZCwmwOvQf8k/RTzlliPBaUKBbfJzFpY7lZnIQ7cPLc+9Hoto4mCRvTYYnPGYJQGAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0BhhnLBD+BxpmDMXE4whS2a4wCEfWBpWo+i2ezdk+mw=; b=K+kaaRrtQbegcmUidBBcHw4raHi2DQo5a0oS2G063RUcXUJPYgVjt7UxbbVtINGKJtpPat6m5p2HOT+5tw2d/yNqi6kaIUrGOt5bflcSntAtsx6/MJ8NdquVBoXcUrTKEJzGdyPJMT66tMzZ/3aSnbulhfE8yCjLnaWaikCWZGE= Received: from DS7PR05CA0105.namprd05.prod.outlook.com (2603:10b6:8:56::19) by SN7PR12MB8101.namprd12.prod.outlook.com (2603:10b6:806:321::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.22; Fri, 31 May 2024 21:39:38 +0000 Received: from CY4PEPF0000EE32.namprd05.prod.outlook.com (2603:10b6:8:56:cafe::d) by DS7PR05CA0105.outlook.office365.com (2603:10b6:8:56::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7656.10 via Frontend Transport; Fri, 31 May 2024 21:39:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE32.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:39:38 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:39:35 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 3/9] PCI/TPH: Implement a command line option to disable TPH Date: Fri, 31 May 2024 16:38:35 -0500 Message-ID: <20240531213841.3246055-4-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE32:EE_|SN7PR12MB8101:EE_ X-MS-Office365-Filtering-Correlation-Id: a0fee1b1-69af-4884-b4a8-08dc81ba2c23 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|7416005|82310400017|36860700004; X-Microsoft-Antispam-Message-Info: Kpy49RzKg9fD2zJau67l+CfLyQqFr7VkMwV4EeCpjjTRWVWKYTH+pklHF2cI152pqeWWuG4mntglxyR1i7L/6PTUdDvyQteSaRKA7g4dfWwn+TfIgCxQRI+RKOVo/82ep35KZCKHSVJ8VdM6q5iynnZwYpDfrQ137rbOZaBflHFkwYHIGw7K0r4rm76Dp3gExGx59wkABRwGHjrp+SxqMa86klVPozNi+W7qQl9LhKxHZxnOAt+7bjlh9jQXyGH319LzrGYwqI9slajiZ3W0p3pIdPZpGQsyGkoIbMTNL/dYtJIgbYnMoWyNSFR0KnNHB/z5texvjpzc1p/YtCZq7f/oceZeAiXQctSgdEU/Gud1mkw2V35pVTkcLAq30vF3t7EnPUKhjDRRFpjgAaEArTdigedWb5K0S0YoTs+PRjFKqyxcmrZ5Zj4O57COqMCo8M2FLxm/vUmxNA+q2lHzkmErbvbs0BdqvHDd0wMbRuedJSFNrwXQkFtGY3sgVVEmZWC/RCau/QHrqi7QVvr0XICqoWMskvFYhgpKSlpwKc8E85gU4qjtf+Qlaj9HYjTv1j42XeoeGhpG6yqAQNK5/rXM9gtn+PG6DLKI6ya/R2SsAUUdtmRlgT2n7WVv8bpL/o9ECTxt6nQcnmCH4KB9X33pkMImWSKAnglzaL4K7kulAVykcFZ/fOBEm4dz+6AMwcb9RF7UBuRJ0wSRRQbmefoaaN1YrnUYJ6RPwQx274xpHeKk6j8WKcCE689hlc9vw5soY5cKLJ+A9h1Ekuz4xs7SaEbyLMEySuM10TWTtw6Kz8Xw/UD7M41LdI8tk4Ktow6eiVMpcBcuntZukvRiRVv+NL6Yik4VnlaClUEklLOZ8h8WakT1wOFOYADUr/QKQTPVff4d6UQdmZpu4YCiPsEgXSm7DVDCcrGGT9yAG8gQ7fNChPbt+8oK3CFnRkAdDBxYgBusz72P6QeohqU6Ewk8iOIT+5+QEeU7Vj0+RbijnU+Jc6RclrIhO50bc1vpB26B+qAxzccXjhz7eCjMc6ugcelVVK87jnOonIg97sb6RqPaAB1QkGkhyR0NKTGQl3XfReIgTa1o8zHLXWo02wTNmKzlr5J9rQN8Wz5nyJ6EtA0Kd4CzcfiU2qQ/4TL0XSDUZSNwwMZVuRXLJ0aG6rTDlZP5p2BqNJmEJ2w0sQ78QnNrgZv6NgUhUaXffFpNy+bdrGrqkITGEqT6vho/tCuvDWUfS2uTFCFXGjZRU8ii5kMQnerDAAUakS2oUbKPQp//WP9Lo7Mo2EmRI4Cz1cxN/NAMJlFPaABL/dKz50jfh4i4MR3rSUZBWtw0CZ9tOsC3ZpE8smz/HaNGvQjocNMjLXnz60U3xsPNgWWHmdsKYBgQ2jdy6uun8Bdy9FB7 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(7416005)(82310400017)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:39:38.3268 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a0fee1b1-69af-4884-b4a8-08dc81ba2c23 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE32.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8101 Provide a kernel option, with related helper functions, to completely disable TPH so that no TPH headers are generated. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- .../admin-guide/kernel-parameters.txt | 1 + drivers/pci/pci-driver.c | 7 ++++- drivers/pci/pci.c | 12 ++++++++ drivers/pci/pcie/tph.c | 30 +++++++++++++++++++ include/linux/pci-tph.h | 19 ++++++++++++ include/linux/pci.h | 1 + 6 files changed, 69 insertions(+), 1 deletion(-) create mode 100644 include/linux/pci-tph.h diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 500cfa776225..fedcc69e35c1 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4623,6 +4623,7 @@ nomio [S390] Do not use MIO instructions. norid [S390] ignore the RID field and force use of one PCI domain per PCI function + notph [PCIE] Do not use PCIe TPH pcie_aspm= [PCIE] Forcibly enable or ignore PCIe Active State Power Management. diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index af2996d0d17f..9722d070c0ca 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "pci.h" #include "pcie/portdrv.h" @@ -322,8 +323,12 @@ static long local_pci_probe(void *_ddi) pm_runtime_get_sync(dev); pci_dev->driver = pci_drv; rc = pci_drv->probe(pci_dev, ddi->id); - if (!rc) + if (!rc) { + if (pci_tph_disabled()) + pcie_tph_disable(pci_dev); + return rc; + } if (rc < 0) { pci_dev->driver = NULL; pm_runtime_put_sync(dev); diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 59e0949fb079..31c443504ce9 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -157,6 +157,9 @@ static bool pcie_ari_disabled; /* If set, the PCIe ATS capability will not be used. */ static bool pcie_ats_disabled; +/* If set, the PCIe TPH capability will not be used. */ +static bool pcie_tph_disabled; + /* If set, the PCI config space of each device is printed during boot. */ bool pci_early_dump; @@ -166,6 +169,12 @@ bool pci_ats_disabled(void) } EXPORT_SYMBOL_GPL(pci_ats_disabled); +bool pci_tph_disabled(void) +{ + return pcie_tph_disabled; +} +EXPORT_SYMBOL_GPL(pci_tph_disabled); + /* Disable bridge_d3 for all PCIe ports */ static bool pci_bridge_d3_disable; /* Force bridge_d3 for all PCIe ports */ @@ -6806,6 +6815,9 @@ static int __init pci_setup(char *str) pci_no_domains(); } else if (!strncmp(str, "noari", 5)) { pcie_ari_disabled = true; + } else if (!strcmp(str, "notph")) { + pr_info("PCIe: TPH is disabled\n"); + pcie_tph_disabled = true; } else if (!strncmp(str, "cbiosize=", 9)) { pci_cardbus_io_size = memparse(str + 9, &str); } else if (!strncmp(str, "cbmemsize=", 10)) { diff --git a/drivers/pci/pcie/tph.c b/drivers/pci/pcie/tph.c index 5f0cc06b74bb..5dc533b89a33 100644 --- a/drivers/pci/pcie/tph.c +++ b/drivers/pci/pcie/tph.c @@ -16,11 +16,41 @@ #include #include #include +#include #include #include #include "../pci.h" +static int tph_set_reg_field_u32(struct pci_dev *dev, u8 offset, u32 mask, + u8 shift, u32 field) +{ + u32 reg_val; + int ret; + + if (!dev->tph_cap) + return -EINVAL; + + ret = pci_read_config_dword(dev, dev->tph_cap + offset, ®_val); + if (ret) + return ret; + + reg_val &= ~mask; + reg_val |= (field << shift) & mask; + + ret = pci_write_config_dword(dev, dev->tph_cap + offset, reg_val); + + return ret; +} + +int pcie_tph_disable(struct pci_dev *dev) +{ + return tph_set_reg_field_u32(dev, PCI_TPH_CTRL, + PCI_TPH_CTRL_REQ_EN_MASK, + PCI_TPH_CTRL_REQ_EN_SHIFT, + PCI_TPH_REQ_DISABLE); +} + void pcie_tph_init(struct pci_dev *dev) { dev->tph_cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_TPH); diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h new file mode 100644 index 000000000000..e187d7e89e8c --- /dev/null +++ b/include/linux/pci-tph.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * TPH (TLP Processing Hints) + * + * Copyright (C) 2024 Advanced Micro Devices, Inc. + * Eric Van Tassell + * Wei Huang + */ +#ifndef LINUX_PCI_TPH_H +#define LINUX_PCI_TPH_H + +#ifdef CONFIG_PCIE_TPH +int pcie_tph_disable(struct pci_dev *dev); +#else +static inline int pcie_tph_disable(struct pci_dev *dev) +{ return -EOPNOTSUPP; } +#endif + +#endif /* LINUX_PCI_TPH_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index d75a88ec5136..d88ebe87815a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1841,6 +1841,7 @@ static inline bool pci_aer_available(void) { return false; } #endif bool pci_ats_disabled(void); +bool pci_tph_disabled(void); #ifdef CONFIG_PCIE_PTM int pci_enable_ptm(struct pci_dev *dev, u8 *granularity); From patchwork Fri May 31 21:38:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942318 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=Zvi+4+jt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-pci+bounces-8146-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc3s6FDZz20Pb for ; Sat, 1 Jun 2024 07:40:25 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 0043A1C23FD2 for ; Fri, 31 May 2024 21:40:24 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 53A9E1DDD1; Fri, 31 May 2024 21:39:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Zvi+4+jt" X-Original-To: linux-pci@vger.kernel.org Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2051.outbound.protection.outlook.com [40.107.102.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98A4D8249E; Fri, 31 May 2024 21:39:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.102.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191598; cv=fail; b=jtbDSd4k2kt8/7QRwfUuTbSI271B3qv0ZHCwRUIci00zHIORBbiN9c+I5bqtaahVDkT1/LgVPyEOBMaD8G7Wsxf4yEad0OnM4/x++oUZFp9egLS96jB2/it0OQ1CAfS5LpSvHIUBvrqNffFlxbdUOhe6oxLO/O7+NRjMieQxvNU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191598; c=relaxed/simple; bh=ze5WfOe+D4ySVTHAEZS7qudi74rejujs48vqCgwz1aI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pGeAwhayI/DEuNsMed4erdRF86D2zHWAymcYIREYfi+JDKJoURvGsRIUqBflNJ+zESjEurSGrf00TUEXxX78em1cjeIn/r8tte8gXkc1vqH2REWkPYMiyXsboQvYh5LNnezbpjfBxi7nAlwq2mGwQk/ihRuli0Cz71YyJx5jSX8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Zvi+4+jt; arc=fail smtp.client-ip=40.107.102.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HqpZK1loVVgxrwTCDdGuVHDLuE3hxIq9H5ehzHcdtZwYpOyfIVvZCHWNnHCO88IRnww3bjIXrcVoTbkvNmnY2Swy5iIUZYPnkRIodo7znyGU7od1fJrPrtRhtj28ePekPJcPHU1kYidpYHIeBTtEbxiLdn+1anzebDf04DiOBtycaBRrWTe5fvCZNv/fW3AQSo7VH1I3Rb377pZIbAhIhjbP/feqM6STz7Zut9xRty1s/Umlngkib/UEFRXX/2eV9HdwdIJThBTy+zqbYzy/Id3wT5hPfR2mvW+2NxWVudQAPiDbcdlfT7dI1kARSv/HX8bBiyiB3WazUvXqklLzsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=TOzRGb8DxJNJFzmxTVXHq+t764c0uTSA2LqkTrWP360=; b=Vk7338I2xATuYl8WMRDdavEo/yOimgabQ5yVd85fDcSsPSx2F+z3rN5Q++2lUk19i/jVnK0kXySj42vdWJjZ8Okjk2gsiiluoysJs9EdBzJBw4XIdwO8nc1Ohmhfaar+qZFlKWiVG7cW/y9ypy3nFaPxZx1B74U0qtn399SMw1+fKOUQRsaoIT1Xpb6jSJlTc/BeOyOw0gKpv6f2zMLJaI/+gFG79gYjrFpXGjRoBEXi+vTFR8+icZCBK3yAUl2hL+kWTTvvMebvBfjFYTToCcPKEUrlbhcrLZYpYMFUcJIqHol144pjRgEF/bgW+dsW+5KA3DllJtiJEE2dXW4tKw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TOzRGb8DxJNJFzmxTVXHq+t764c0uTSA2LqkTrWP360=; b=Zvi+4+jtKV9hyEDql2DY4ka6w9Zj3533HU6jZpY1o7sawMbnqzzQHUTUPYuoGomQNqCb7MxqryNAuhMNB546EBtlyWlgiKzBsT64o1goUDfCsvpsyIJlEy3GeWSI0ISTL0Tr6JUODETXnN4LGGJSIvzOFn8wagGIl89gFcGGbM8= Received: from CY8PR11CA0036.namprd11.prod.outlook.com (2603:10b6:930:4a::22) by SN7PR12MB7370.namprd12.prod.outlook.com (2603:10b6:806:299::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.17; Fri, 31 May 2024 21:39:54 +0000 Received: from CY4PEPF0000EE33.namprd05.prod.outlook.com (2603:10b6:930:4a:cafe::3c) by CY8PR11CA0036.outlook.office365.com (2603:10b6:930:4a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22 via Frontend Transport; Fri, 31 May 2024 21:39:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE33.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:39:53 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:39:52 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 4/9] PCI/TPH: Implement a command line option to force No ST Mode Date: Fri, 31 May 2024 16:38:36 -0500 Message-ID: <20240531213841.3246055-5-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE33:EE_|SN7PR12MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: c257b5a7-cd46-446b-8575-08dc81ba3548 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|7416005|1800799015|82310400017|36860700004; X-Microsoft-Antispam-Message-Info: b7GJdae/2AMIeGdOE/YfdUUWdtgecdgt6+zuFBfRLkHUhPpBzlmZGnP9ikcUBpAzFqNhQTVWbBZNVm32f9WOyzWNEGo2zE9K5NXV/O2tpT/Sa0l+S1L7b0T5ttNaemC75PpmnuFQF9FQUhjGa69jaYoSuzwJPfUhYT2ppZRAErvA0y8o+SJrMFUOOQ8s32spEg0i4X0c5JfO2/deHwVz+UQE2Da61tkgIwgFoTolsv1DApeWmes60T4Zt4yCkyj4MLORMGxF5sGQnV/xWwoyunl9EJbm27rJWNwjscnCsxAjeCGzPbheghDF3bTn6RNztMT9rXxonVBC02TPmZLee36MQDFMFWvaj1eEKdOLg3aUXF/yDv7ubj508WhrOJJeMmvS3ROLlmmkdUKOc8L5cDPGks1rj8jk8bFbTs79uVrZmpLAKyA/HB2YgeEeh+IYUr8+f1kLLxsPgfnUK3gbaibla/9wNOmyVz9qUCiRfWA/9Mc+2p+apAJaVi0e2T4CgeOZkOAKIzPw3vk95DDpLlfnEyPzDfjT1FRWDNebE871xHtVA12f4YBLom6xjuMk8b53TZ6uuhfcUja5/4him/db7mdIo9cWYA0emyk2go7OG6x7GclVfu4UuzxZlPafsSH2R6MWHjB5JQSsl1FYhtjTPplSBPM6mGw21mvk9RetiW1SZcV9FQ/+TV+lGRvwpo1cNvLo+hgHS4GEZEz04lYMGEO1tsjZ0MD5iMaV359Q9HSuAe5ID36yOIB2Fia7ViLEKwKOVJUd9nHpb7OYU8Xq72DEkP0cWj2OS4x/71S0rrguTO0v4uN5w6WY0FcaUR6p4BWJoqonyyXo4NH6CdgTn6VieTXlaa5e1TG6ufRuz998sqI/T46bEWR5FAcU10GxGpO7jKbJtU4PzJo2MP2SAVJxyHLHGGtETYvDSGvx0fSJD/l0GxGCrbVC3FkgpeNB2kfmzYUAOtgjX71BJ76ku3wMB+VZmg/WOV3pP8XsYWU4aCFVkvBxxY0tIgWrbTaweXAckeK/nXDMOYiinfSwGnQyUF3091yiSngV/bsvff33FTGz8Xnu73jd1sAcBzYCxIx6/+QMhQzJwwX4nkig96E/nahv9RfYp1u+d46WcezE3JkfnG8tv/7j/t/VF9eFsj9FxzIpSR0mDQQSSiXXdZF58xfhccwRD9FfSDfeJaC0RNshiOdPM/F8T+DgcrrE4xs3lozdBF6YsNnaxlNyKbhL5trnKkDsX6utEdrb2S9MKm4h7p47dYwRB0fX72eIyDVqbTZk1U+9OsUrA8J5yJbBC4SQEo2t0n0ejze73Bb95ClNhh1oTOeRB0kFyhPokVCdGeO9fx8RnVQImb09wWJHiGWPy5zd+8N5ADVSU2dIohvoGDABymKVr51m X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(7416005)(1800799015)(82310400017)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:39:53.6519 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c257b5a7-cd46-446b-8575-08dc81ba3548 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE33.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7370 When "No ST mode" is enabled, end-point devices can generate TPH headers but with all steering tags treated as zero. A steering tag of zero is interpreted as "using the default policy" by the root complex. This is essential to quantify the benefit of steering tags for some given workloads. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- .../admin-guide/kernel-parameters.txt | 1 + drivers/pci/pci-driver.c | 7 ++++++- drivers/pci/pci.c | 12 +++++++++++ drivers/pci/pcie/tph.c | 21 +++++++++++++++++++ include/linux/pci-tph.h | 3 +++ include/linux/pci.h | 1 + 6 files changed, 44 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index fedcc69e35c1..e97a4a239563 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4624,6 +4624,7 @@ norid [S390] ignore the RID field and force use of one PCI domain per PCI function notph [PCIE] Do not use PCIe TPH + nostmode [PCIE] Force TPH to use No ST Mode pcie_aspm= [PCIE] Forcibly enable or ignore PCIe Active State Power Management. diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 9722d070c0ca..aa98843d9884 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -324,8 +324,13 @@ static long local_pci_probe(void *_ddi) pci_dev->driver = pci_drv; rc = pci_drv->probe(pci_dev, ddi->id); if (!rc) { - if (pci_tph_disabled()) + if (pci_tph_disabled()) { pcie_tph_disable(pci_dev); + return rc; + } + + if (pci_tph_nostmode()) + tph_set_dev_nostmode(pci_dev); return rc; } diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 31c443504ce9..f3558a551bf2 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -160,6 +160,9 @@ static bool pcie_ats_disabled; /* If set, the PCIe TPH capability will not be used. */ static bool pcie_tph_disabled; +/* If TPH is enabled, "No ST Mode" will be enforced. */ +static bool pcie_tph_nostmode; + /* If set, the PCI config space of each device is printed during boot. */ bool pci_early_dump; @@ -175,6 +178,12 @@ bool pci_tph_disabled(void) } EXPORT_SYMBOL_GPL(pci_tph_disabled); +bool pci_tph_nostmode(void) +{ + return pcie_tph_nostmode; +} +EXPORT_SYMBOL_GPL(pci_tph_nostmode); + /* Disable bridge_d3 for all PCIe ports */ static bool pci_bridge_d3_disable; /* Force bridge_d3 for all PCIe ports */ @@ -6818,6 +6827,9 @@ static int __init pci_setup(char *str) } else if (!strcmp(str, "notph")) { pr_info("PCIe: TPH is disabled\n"); pcie_tph_disabled = true; + } else if (!strcmp(str, "nostmode")) { + pr_info("PCIe: TPH No ST Mode is enabled\n"); + pcie_tph_nostmode = true; } else if (!strncmp(str, "cbiosize=", 9)) { pci_cardbus_io_size = memparse(str + 9, &str); } else if (!strncmp(str, "cbmemsize=", 10)) { diff --git a/drivers/pci/pcie/tph.c b/drivers/pci/pcie/tph.c index 5dc533b89a33..d5f7309fdf52 100644 --- a/drivers/pci/pcie/tph.c +++ b/drivers/pci/pcie/tph.c @@ -43,6 +43,27 @@ static int tph_set_reg_field_u32(struct pci_dev *dev, u8 offset, u32 mask, return ret; } +int tph_set_dev_nostmode(struct pci_dev *dev) +{ + int ret; + + /* set ST Mode Select to "No ST Mode" */ + ret = tph_set_reg_field_u32(dev, PCI_TPH_CTRL, + PCI_TPH_CTRL_MODE_SEL_MASK, + PCI_TPH_CTRL_MODE_SEL_SHIFT, + PCI_TPH_NO_ST_MODE); + if (ret) + return ret; + + /* set "TPH Requester Enable" to "TPH only" */ + ret = tph_set_reg_field_u32(dev, PCI_TPH_CTRL, + PCI_TPH_CTRL_REQ_EN_MASK, + PCI_TPH_CTRL_REQ_EN_SHIFT, + PCI_TPH_REQ_TPH_ONLY); + + return ret; +} + int pcie_tph_disable(struct pci_dev *dev) { return tph_set_reg_field_u32(dev, PCI_TPH_CTRL, diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h index e187d7e89e8c..95269afc8b7d 100644 --- a/include/linux/pci-tph.h +++ b/include/linux/pci-tph.h @@ -11,9 +11,12 @@ #ifdef CONFIG_PCIE_TPH int pcie_tph_disable(struct pci_dev *dev); +int tph_set_dev_nostmode(struct pci_dev *dev); #else static inline int pcie_tph_disable(struct pci_dev *dev) { return -EOPNOTSUPP; } +static inline int tph_set_dev_nostmode(struct pci_dev *dev) +{ return -EOPNOTSUPP; } #endif #endif /* LINUX_PCI_TPH_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index d88ebe87815a..5f520624d133 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1842,6 +1842,7 @@ static inline bool pci_aer_available(void) { return false; } bool pci_ats_disabled(void); bool pci_tph_disabled(void); +bool pci_tph_nostmode(void); #ifdef CONFIG_PCIE_PTM int pci_enable_ptm(struct pci_dev *dev, u8 *granularity); From patchwork Fri May 31 21:38:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942319 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=u3U0tzpD; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-pci+bounces-8147-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc4m5Lv8z20Pb for ; Sat, 1 Jun 2024 07:41:12 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id EB44A288AFE for ; Fri, 31 May 2024 21:41:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E184A78C9C; Fri, 31 May 2024 21:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="u3U0tzpD" X-Original-To: linux-pci@vger.kernel.org Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2056.outbound.protection.outlook.com [40.107.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC0A412F59C; Fri, 31 May 2024 21:40:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.56 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191621; cv=fail; b=XYGrTTBWVU+vMcu3JzVrTJOJLEUTA9N5EZaCmKcUVHLUJamYWYCV+Ejcv2FIU5baxMm/PY3fjMPg058Llcr8C6vW3nzhH8LJojoQssRvcbe60l4XgWzFft7krVRIxAWN6hSHdrL0LkklGlfHKt+lwj7DrAcmNUCrBCqyxub0DJE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191621; c=relaxed/simple; bh=fN1/SnMOUSpE6+Yk9dJGV+MassN9C624pORQd/UNf0E=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HriFZil2cSDW+gHE9zR7k+gh+JFgMz0UoGcDD/3frpArYEJ6/eYamuOhNioixMzHEEDk5tDhzPmEhpfnABnXEVlVJbCKJvleGQ0fXyKchrNKGG55Fsj11SxL8opJOqT9uIBKvO1enHdkXXl7V6cOeX9xI2ctQTXsxhyEdcy1FbA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=u3U0tzpD; arc=fail smtp.client-ip=40.107.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oStHU+trlBhnJoaZOfuq47k1Vk83mOpaP0FcE69eBEgL6mtzSc1YRklVmKlDLkBteKMKPalw23xhiMLn86uzKs2TdAodVI+vFC7eJ5wReFtN6ySwgn4U4nE5QOjQLfo/nipsjakpjeckTcrJSeRsKGDFWrMH4u5M/YLJdeO4MRg5O6YJJs31fSutjDWYuu+U1H1oKiFdnDuJXplt5ywn4yz8yvFlAWZMlFPE2W/L/lvdOY7TaCmcFQBTgWYzwTVPu5hEIhhpbAcTl4sxgbAmABdJH8ubzwCBvQG621fNoe9TrP0ViFuM5Dy1rMDySQFurDhJ8XYZfC2kuzMGpsFNkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZnTAkJwl0BgHNN/haqCAdMLfzMbgPq6y+j7SX5sJJww=; b=f2Jb0iT9owlh2Hx07gvUpQ9vWluPqkHdM5jaEvOgBFa/A0hKuavdSW+3qyBqat9lB8lg9GLiXU/1/MNNBdWFp9POONwtylCQmUFJyZlA8U6v0VVWpplykac/obj+1lKG2itofENU1DRUW490l7rXZXS5i6MPXlRuf4feSOyD7CqcAhb11kvV7cIvvMbayXRFFU3dc/rOFC/c8pZO4R7U31c0X+jTKfqjaFWdEqUmrkaFcmHRr1YMoz1zmF5wfsE0LTt7IqgQ/auHOfvEJ0AvM+5bDO1VRI3HieBDkMORBhhn+GzNoLEkVNjBCvHb/oq0hcwfR3Tq7IF32esgaCUM2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=temperror (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=temperror action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZnTAkJwl0BgHNN/haqCAdMLfzMbgPq6y+j7SX5sJJww=; b=u3U0tzpDNLf6SCkrou4wY9TLIO6odYShzR0eGuK34uHw0B3dc+7wwvtdYGXMgUo1vx9ijiAMyCY4zhvp/21GaPotzWmLPfsf6w1fv/0Z4lEZLZjGsFuNNphaTJwuNB9cHCwMiySFiG1yr9d9xnysAQe/NTv+0qzY7fN/p2kbQ9g= Received: from BY3PR05CA0044.namprd05.prod.outlook.com (2603:10b6:a03:39b::19) by IA0PR12MB8301.namprd12.prod.outlook.com (2603:10b6:208:40b::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.21; Fri, 31 May 2024 21:40:14 +0000 Received: from CO1PEPF000075EE.namprd03.prod.outlook.com (2603:10b6:a03:39b:cafe::32) by BY3PR05CA0044.outlook.office365.com (2603:10b6:a03:39b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7656.9 via Frontend Transport; Fri, 31 May 2024 21:40:13 +0000 X-MS-Exchange-Authentication-Results: spf=temperror (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=temperror action=none header.from=amd.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of amd.com: DNS Timeout) Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000075EE.mail.protection.outlook.com (10.167.249.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:40:12 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:40:08 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 5/9] PCI/TPH: Introduce API functions to manage steering tags Date: Fri, 31 May 2024 16:38:37 -0500 Message-ID: <20240531213841.3246055-6-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000075EE:EE_|IA0PR12MB8301:EE_ X-MS-Office365-Filtering-Correlation-Id: 72bb4b88-2e90-4217-4a33-08dc81ba40d1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|7416005|1800799015|376005|82310400017; X-Microsoft-Antispam-Message-Info: Rgs2vUBqQTOLlPGNwK15T/tdmty4mjZtKUlEun/B7K6Z2su+csD6vKegeTQrvvvVcxLZzETd4TuNASvyavgj4Gf5QpAKtnReyaG7GavBH3ig2wQgUJl+nBm1pBwl75B0yj1u1LObpDhC0hXZS7o7Uw02V1IG2/PwbUakilhh43mwlaAWpL8yPgDgXy6BtOXTr6/iTUUbYy4OBYLCTTUPcI0z7qklMkarfXQLD9ncV/UPlk25zV40ElSZmV/ebuLego7llZ/TrxhaGBUcxzZm2BhjiHaKHTdaqKaTsq4hNBSpHsUSxJTmEQSe+ssSQiuqcQJ1rjG29mlr869+VdXI1psiXtM7u86mHp/kls6PPMumffVHW9nWQfe8L+YL3uQ3Ab5Z6f0JVtErtmuIcNdsHxjbOsI5x6mnZbD5Ga+WcBaT2AQn/hbkLFTNKsvAzlsmxA527SINCf8NLAm7M8GzN2vEqzRAvPBLesJtGgvvKMlrMT+zwfeEJCJD5r3FBMr82VYdK5JclnZivBBJ6G3a5cV8SZanA4eD+8RXEW7zfP85H2MWIi70n0EqDubr4dfRXsVUKsZbt8Tw84o7lYGgofyWdfbS3NUjwXNJ/hQlkHDDlp0lVkBWVO3IfmC10ssAtkX6OPos92Ty36OLo/vgQZdj3DUHc3VJ7FNdphpDi84YL3NlSs7FIxe3LBuE216kjalbJW1fmh214pOJFI98tLCvEO7kOP2Yd1XZm+u+66V1oiVSMMD6h0aOj1ulbb4w4JOpbvveoyXeNyhfwmp47rRl/+mR/sOnuC4E6G1a26SY9cGmkRFk83mdUKQMLI72BZmlZBM2tavm2QQIvM6rCBhciPPatRYROBSGjQOu7UJSDkU9VKxx49oUn97iv1U6UcqSA9ynwlLIMeFiAIO9ZHtn/WKZdGDb/TpgQ7GxPKkOWSr/nivAz9jWiRwhkw0VBtj3/zqxPUIbuzeTKR0hgEyVTUmjQ+36uLuPe1tu6/kqwg43IEW8LuG1a99aGxQxAHARoPA4aCXOUHAuOq0dL0HLajBwUN2ZWYu/ANLVySQnJiXEReHnGRmo0KFPaTeV1I38INcrou1bsjz7hGIYJ0zhCVKHdEeIVJ4bo+08bxrAJH+S1UeiJILS49Za9mg6Tb8Rpg4iNZ5AsK5P4Hhs5wRQ7Ac5+znV1fGGjqUSHbOMCsAyXETavKPintRmgQXttXPckmo1bnWOX6TdjzKg/7TG5gsD1whl6t10knqvLAWjIP51LzaS5FGwZufT7Y+E8AJarcJjgwbW9Dt6P/ww71qSAic0dR7nKB5HOuNOUEyL/AW9q8MvSTm+b4KdtBv/sRZzMC8a/xmXZCOEhu8pJqC0KYA1NnhlvOruFBlYxFVRtpRygG0LxdjyypjlSvho X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(7416005)(1800799015)(376005)(82310400017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:40:12.9407 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 72bb4b88-2e90-4217-4a33-08dc81ba40d1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000075EE.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8301 This patch introduces three API functions, pcie_tph_intr_vec_supported(), pcie_tph_get_st() and pcie_tph_set_st(), for a driver to query, retrieve or configure device's steering tags. There are two possible locations for steering tag table and the code automatically figure out the right location to set the tags if pcie_tph_set_st() is called. Note the tag value is always zero currently and will be extended in the follow-up patches. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- drivers/pci/pcie/tph.c | 402 ++++++++++++++++++++++++++++++++++++++++ include/linux/pci-tph.h | 22 +++ 2 files changed, 424 insertions(+) diff --git a/drivers/pci/pcie/tph.c b/drivers/pci/pcie/tph.c index d5f7309fdf52..320b99c60365 100644 --- a/drivers/pci/pcie/tph.c +++ b/drivers/pci/pcie/tph.c @@ -43,6 +43,336 @@ static int tph_set_reg_field_u32(struct pci_dev *dev, u8 offset, u32 mask, return ret; } +static int tph_get_reg_field_u32(struct pci_dev *dev, u8 offset, u32 mask, + u8 shift, u32 *field) +{ + u32 reg_val; + int ret; + + if (!dev->tph_cap) + return -EINVAL; + + ret = pci_read_config_dword(dev, dev->tph_cap + offset, ®_val); + if (ret) + return ret; + + *field = (reg_val & mask) >> shift; + + return 0; +} + +static int tph_get_table_size(struct pci_dev *dev, u16 *size_out) +{ + int ret; + u32 tmp; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CAP, + PCI_TPH_CAP_ST_MASK, + PCI_TPH_CAP_ST_SHIFT, &tmp); + + if (ret) + return ret; + + *size_out = (u16)tmp; + + return 0; +} + +/* + * For a given device, return a pointer to the MSI table entry at msi_index. + */ +static void __iomem *tph_msix_table_entry(struct pci_dev *dev, + u16 msi_index) +{ + void __iomem *entry; + u16 tbl_sz; + int ret; + + ret = tph_get_table_size(dev, &tbl_sz); + if (ret || msi_index > tbl_sz) + return NULL; + + entry = dev->msix_base + msi_index * PCI_MSIX_ENTRY_SIZE; + + return entry; +} + +/* + * For a given device, return a pointer to the vector control register at + * offset 0xc of MSI table entry at msi_index. + */ +static void __iomem *tph_msix_vector_control(struct pci_dev *dev, + u16 msi_index) +{ + void __iomem *vec_ctrl_addr = tph_msix_table_entry(dev, msi_index); + + if (vec_ctrl_addr) + vec_ctrl_addr += PCI_MSIX_ENTRY_VECTOR_CTRL; + + return vec_ctrl_addr; +} + +/* + * Translate from MSI-X interrupt index to struct msi_desc * + */ +static struct msi_desc *tph_msix_index_to_desc(struct pci_dev *dev, int index) +{ + struct msi_desc *entry; + + msi_lock_descs(&dev->dev); + msi_for_each_desc(entry, &dev->dev, MSI_DESC_ASSOCIATED) { + if (entry->msi_index == index) + return entry; + } + msi_unlock_descs(&dev->dev); + + return NULL; +} + +static bool tph_int_vec_mode_supported(struct pci_dev *dev) +{ + u32 mode = 0; + int ret; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CAP, + PCI_TPH_CAP_INT_VEC, + PCI_TPH_CAP_INT_VEC_SHIFT, &mode); + if (ret) + return false; + + return !!mode; +} + +static int tph_get_table_location(struct pci_dev *dev, u8 *loc_out) +{ + u32 loc; + int ret; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CAP, PCI_TPH_CAP_LOC_MASK, + PCI_TPH_CAP_LOC_SHIFT, &loc); + if (ret) + return ret; + + *loc_out = (u8)loc; + + return 0; +} + +static bool msix_nr_in_bounds(struct pci_dev *dev, int msix_nr) +{ + u16 tbl_sz; + + if (tph_get_table_size(dev, &tbl_sz)) + return false; + + return msix_nr <= tbl_sz; +} + +/* Return root port capability - 0 means none */ +static int get_root_port_completer_cap(struct pci_dev *dev) +{ + struct pci_dev *rp; + int ret; + int val; + + rp = pcie_find_root_port(dev); + if (!rp) { + pr_err("cannot find root port of %s\n", dev_name(&dev->dev)); + return 0; + } + + ret = pcie_capability_read_dword(rp, PCI_EXP_DEVCAP2, &val); + if (ret) { + pr_err("cannot read device capabilities 2 of %s\n", + dev_name(&dev->dev)); + return 0; + } + + val &= PCI_EXP_DEVCAP2_TPH_COMP; + + return val >> PCI_EXP_DEVCAP2_TPH_COMP_SHIFT; +} + +/* + * TPH device needs to be below a rootport with the TPH Completer and + * the completer must offer a compatible level of completer support to that + * requested by the device driver. + */ +static bool completer_support_ok(struct pci_dev *dev, u8 req) +{ + int rp_cap; + + rp_cap = get_root_port_completer_cap(dev); + + if (req > rp_cap) { + pr_err("root port lacks proper TPH completer capability\n"); + return false; + } + + return true; +} + +/* + * The PCI Specification version 5.0 requires the "No ST Mode" mode + * be supported by any compatible device. + */ +static bool no_st_mode_supported(struct pci_dev *dev) +{ + bool no_st; + int ret; + u32 tmp; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CAP, PCI_TPH_CAP_NO_ST, + PCI_TPH_CAP_NO_ST_SHIFT, &tmp); + if (ret) + return false; + + no_st = !!tmp; + + if (!no_st) { + pr_err("TPH devices must support no ST mode\n"); + return false; + } + + return true; +} + +static int tph_write_ctrl_reg(struct pci_dev *dev, u32 value) +{ + int ret; + + ret = tph_set_reg_field_u32(dev, PCI_TPH_CTRL, ~0L, 0, value); + + if (ret) + goto err_out; + + return 0; + +err_out: + /* minimizing possible harm by disabling TPH */ + pcie_tph_disable(dev); + return ret; +} + +/* Update the ST Mode Select field of the TPH Control Register */ +static int tph_set_ctrl_reg_mode_sel(struct pci_dev *dev, u8 st_mode) +{ + int ret; + u32 ctrl_reg; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CTRL, ~0L, 0, &ctrl_reg); + if (ret) + return ret; + + /* clear the mode select and enable fields */ + ctrl_reg &= ~(PCI_TPH_CTRL_MODE_SEL_MASK); + ctrl_reg |= ((u32)(st_mode << PCI_TPH_CTRL_MODE_SEL_SHIFT) & + PCI_TPH_CTRL_MODE_SEL_MASK); + + ret = tph_write_ctrl_reg(dev, ctrl_reg); + if (ret) + return ret; + + return 0; +} + +/* Write the steering tag to MSI-X vector control register */ +static void tph_write_tag_to_msix(struct pci_dev *dev, int msix_nr, u16 tag) +{ + u32 val; + void __iomem *vec_ctrl; + struct msi_desc *msi_desc; + + msi_desc = tph_msix_index_to_desc(dev, msix_nr); + if (!msi_desc) { + pr_err("MSI-X descriptor for #%d not found\n", msix_nr); + return; + } + + vec_ctrl = tph_msix_vector_control(dev, msi_desc->msi_index); + + val = readl(vec_ctrl); + val &= 0xffff; + val |= (tag << 16); + writel(val, vec_ctrl); + + /* read back to flush the update */ + val = readl(vec_ctrl); + msi_unlock_descs(&dev->dev); +} + +/* Update the TPH Requester Enable field of the TPH Control Register */ +static int tph_set_ctrl_reg_en(struct pci_dev *dev, u8 req_type) +{ + int ret; + u32 ctrl_reg; + + ret = tph_get_reg_field_u32(dev, PCI_TPH_CTRL, ~0L, 0, + &ctrl_reg); + if (ret) + return ret; + + /* clear the mode select and enable fields and set new values*/ + ctrl_reg &= ~(PCI_TPH_CTRL_REQ_EN_MASK); + ctrl_reg |= (((u32)req_type << PCI_TPH_CTRL_REQ_EN_SHIFT) & + PCI_TPH_CTRL_REQ_EN_MASK); + + ret = tph_write_ctrl_reg(dev, ctrl_reg); + if (ret) + return ret; + + return 0; +} + +static bool pcie_tph_write_st(struct pci_dev *dev, unsigned int msix_nr, + u8 req_type, u16 tag) +{ + int offset; + u8 loc; + int ret; + + /* setting ST isn't needed - not an error, just return true */ + if (!dev->tph_cap || pci_tph_disabled() || pci_tph_nostmode() || + !dev->msix_enabled || !tph_int_vec_mode_supported(dev)) + return true; + + /* setting ST is incorrect in the following cases - return error */ + if (!no_st_mode_supported(dev) || !msix_nr_in_bounds(dev, msix_nr) || + !completer_support_ok(dev, req_type)) + return false; + + /* + * disable TPH before updating the tag to avoid potential instability + * as cautioned about in the "ST Table Programming" of PCI-E spec + */ + pcie_tph_disable(dev); + + ret = tph_get_table_location(dev, &loc); + if (ret) + return false; + + switch (loc) { + case PCI_TPH_LOC_MSIX: + tph_write_tag_to_msix(dev, msix_nr, tag); + break; + case PCI_TPH_LOC_CAP: + offset = dev->tph_cap + PCI_TPH_ST_TABLE + + msix_nr * sizeof(u16); + pci_write_config_word(dev, offset, tag); + break; + default: + pr_err("unable to write steering tag for device %s\n", + dev_name(&dev->dev)); + return false; + } + + /* select interrupt vector mode */ + tph_set_ctrl_reg_mode_sel(dev, PCI_TPH_INT_VEC_MODE); + tph_set_ctrl_reg_en(dev, req_type); + + return true; +} + int tph_set_dev_nostmode(struct pci_dev *dev) { int ret; @@ -77,3 +407,75 @@ void pcie_tph_init(struct pci_dev *dev) dev->tph_cap = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_TPH); } +/** + * pcie_tph_intr_vec_supported() - Check if interrupt vector mode supported for dev + * @dev: pci device + * + * Return: + * true : intr vector mode supported + * false: intr vector mode not supported + */ +bool pcie_tph_intr_vec_supported(struct pci_dev *dev) +{ + if (!dev->tph_cap || pci_tph_disabled() || !dev->msix_enabled || + !tph_int_vec_mode_supported(dev)) + return false; + + return true; +} +EXPORT_SYMBOL(pcie_tph_intr_vec_supported); + +/** + * pcie_tph_get_st() - Retrieve steering tag for a specific CPU + * @dev: pci device + * @cpu: the acpi cpu_uid. + * @mem_type: memory type (vram, nvram) + * @req_type: request type (disable, tph, extended tph) + * @tag: steering tag return value + * + * Return: + * true : success + * false: failed + */ +bool pcie_tph_get_st(struct pci_dev *dev, unsigned int cpu, + enum tph_mem_type mem_type, u8 req_type, + u16 *tag) +{ + *tag = 0; + + return true; +} +EXPORT_SYMBOL(pcie_tph_get_st); + +/** + * pcie_tph_set_st() - Set steering tag in ST table entry + * @dev: pci device + * @msix_nr: ordinal number of msix interrupt. + * @cpu: the acpi cpu_uid. + * @mem_type: memory type (vram, nvram) + * @req_type: request type (disable, tph, extended tph) + * + * Return: + * true : success + * false: failed + */ +bool pcie_tph_set_st(struct pci_dev *dev, unsigned int msix_nr, + unsigned int cpu, enum tph_mem_type mem_type, + u8 req_type) +{ + u16 tag; + bool ret = true; + + ret = pcie_tph_get_st(dev, cpu, mem_type, req_type, &tag); + + if (!ret) + return false; + + pr_debug("%s: writing tag %d for msi-x intr %d (cpu: %d)\n", + __func__, tag, msix_nr, cpu); + + ret = pcie_tph_write_st(dev, msix_nr, req_type, tag); + + return ret; +} +EXPORT_SYMBOL(pcie_tph_set_st); diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h index 95269afc8b7d..4fbd1e2fd98c 100644 --- a/include/linux/pci-tph.h +++ b/include/linux/pci-tph.h @@ -9,14 +9,36 @@ #ifndef LINUX_PCI_TPH_H #define LINUX_PCI_TPH_H +enum tph_mem_type { + TPH_MEM_TYPE_VM, /* volatile memory type */ + TPH_MEM_TYPE_PM /* persistent memory type */ +}; + #ifdef CONFIG_PCIE_TPH int pcie_tph_disable(struct pci_dev *dev); int tph_set_dev_nostmode(struct pci_dev *dev); +bool pcie_tph_intr_vec_supported(struct pci_dev *dev); +bool pcie_tph_get_st(struct pci_dev *dev, unsigned int cpu, + enum tph_mem_type tag_type, u8 req_enable, + u16 *tag); +bool pcie_tph_set_st(struct pci_dev *dev, unsigned int msix_nr, + unsigned int cpu, enum tph_mem_type tag_type, + u8 req_enable); #else static inline int pcie_tph_disable(struct pci_dev *dev) { return -EOPNOTSUPP; } static inline int tph_set_dev_nostmode(struct pci_dev *dev) { return -EOPNOTSUPP; } +static inline bool pcie_tph_intr_vec_supported(struct pci_dev *dev) +{ return false; } +static inline bool pcie_tph_get_st(struct pci_dev *dev, unsigned int cpu, + enum tph_mem_type tag_type, u8 req_enable, + u16 *tag) +{ return false; } +static inline bool pcie_tph_set_st(struct pci_dev *dev, unsigned int msix_nr, + unsigned int cpu, enum tph_mem_type tag_type, + u8 req_enable) +{ return true; } #endif #endif /* LINUX_PCI_TPH_H */ From patchwork Fri May 31 21:38:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942320 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=hBOnZnT9; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-pci+bounces-8148-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc592mB3z20Pb for ; Sat, 1 Jun 2024 07:41:33 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 650FA1C23E96 for ; Fri, 31 May 2024 21:41:31 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 055E680626; Fri, 31 May 2024 21:40:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hBOnZnT9" X-Original-To: linux-pci@vger.kernel.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2089.outbound.protection.outlook.com [40.107.94.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42A3880043; Fri, 31 May 2024 21:40:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191633; cv=fail; b=CKMy+t7TZYoG0ra2kvYTiHIblyEneGrKggdA2UZYQkt1jy4cToD7yjvtjjnipmHM4t6LG6MMvBiGJ3FKHsA4ddFJNjC/yf7leD3HYoe7XH3022VjPNgygzSHpW2Nso8Rgb4ptnK9Zb1H8Z0l0caDOWQf6EWm7KvIOeGsbNzKzwY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191633; c=relaxed/simple; bh=DukWMzIob+iLaT8VORBQPoAb3+xYvQnMu0pPzp0UjUM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EzSQqsrHfk9VC0/BvAtRfmaQn68p2YJUPTSPwLBg+2mPOs63ahMdIm9eo8awd3iPyNTUeNyL4Ogtgzz286d3CWGHbB3Zz1K068rjFoXbydr4NGERZO9QtMgY23hWTRvNJ7W3rXNhz6UfZFoJI6gu6w/gM5r5zN29GW63B1kVCmc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hBOnZnT9; arc=fail smtp.client-ip=40.107.94.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ieulYGWmmOTp0bQE5mhmTFLHK7TOHJ9uSymUiyuSMStiQfHIMhd7iqa2b9UHsMTxQmvokoYVEwuDalL34G6JO0nmrwoXz6LG0jFXKifqpPpsbTeB5y/hUccSWzGTxjMfTLA8Cqklksj8VhXsnaLBcqY4tlwMZ3pynr60mOfqUQY/42D2o1ilNHTbJTHRT3Ju+Qw5klkNkaWjgO9sYfYNadOal7q0Z43UNaB0b6hzXH4uzGZdbZ1/8ibAUNd9H/QWnaNXpgmLII9ILBCaswLg2vLgpHz90CbQpxrd96zmgLgghFGFAwAl/mDD/f4qEYtCqqY8VZIh/HYoYqgp0QXwZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Qw6JZNUpyHIwJR2o+d1zz9gOLAtHgXaxRHZt2MGu6BQ=; b=U93cFhoxjGVNgI3TQsWgcPqO2qSTpZxc+woWbbS6tCw3JL+K8id5ktT1JiVuJGV/Ltx/Mb4pBEp1k5Zmd+TDfLFUcitV0/IzMJV+whW4s2hXhfnVdenSHvZPst2+eO+VFm/TdaP/kAyPRTkD3CAAiYEHWQLHUZ8FSvL46TWskpr1gSlDtKGi+2IFBZD6dr/ri7Bfjfgkdo9OHu8C2mv5RHHvAa+X6714T6GJmSFC+QsqsQBoz3EAem4d8r0JBkkk5FSLm0YNkU9mTsujRsamkfXFMQJO6AV/gIjt2Epga6VG5Lxfxov2w4RxLHwdspsYllCM0FpXIu4Kix8AXVfaJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qw6JZNUpyHIwJR2o+d1zz9gOLAtHgXaxRHZt2MGu6BQ=; b=hBOnZnT9jWqstM6nnUo28hgLm5/fffVWW6vS7WkvB5DE8m51PEwLtsdu9tH6y5NWVA+/ga482uHK8PuCKLycP7Zywe3ZrnYBgZ9ZUmuPM4AZTK1pXBqLAXPvrYR2pQFR7Vy1DC/VM7JCrNISbt6lVcZazPX+vauIePskaUHoHHM= Received: from CY5PR13CA0030.namprd13.prod.outlook.com (2603:10b6:930::18) by CH3PR12MB9341.namprd12.prod.outlook.com (2603:10b6:610:1cd::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Fri, 31 May 2024 21:40:27 +0000 Received: from CY4PEPF0000EE30.namprd05.prod.outlook.com (2603:10b6:930:0:cafe::8a) by CY5PR13CA0030.outlook.office365.com (2603:10b6:930::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7656.10 via Frontend Transport; Fri, 31 May 2024 21:40:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE30.mail.protection.outlook.com (10.167.242.36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:40:26 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:40:25 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 6/9] PCI/TPH: Retrieve steering tag from ACPI _DSM Date: Fri, 31 May 2024 16:38:38 -0500 Message-ID: <20240531213841.3246055-7-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE30:EE_|CH3PR12MB9341:EE_ X-MS-Office365-Filtering-Correlation-Id: 7152c140-1b3e-48fe-7fe1-08dc81ba48e9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|36860700004|7416005|376005|1800799015; X-Microsoft-Antispam-Message-Info: cNtJ1aYrsM/uneCIgbyBdtFNgfbcN4M6ESeUVYHWto+uTYYjduiLfd1bSY/PzBkA0rGW5AfH9xbPA6QB+TUQRxn3ERu1/Hsn1pcJk9JquEAgB57OXW1K8ZweXWvDM6SM5dHxRIZhKIoVWDeDoRfUbKYAiIa3vOEWvD6gUagnDvo9nIz8R4V3tvIHZOVtLX0IfVd7wL/nb/rDN+hg5Erbz5hodRa5hIQCF2mfVCGPhi/82Xwad7zyO+DZeDafAMQtaeCCM9vWvBxR13Q/OWhn9wJIkkAJ7iBaaPR8za6rSr7UsUlwgwZuxT39OHMmVhoHhxQF97X25rLBV1UABktgxZh5FtcFobaW5pJNGmhBrH0oIwDBrSyWopAobNvDnHLhIRD2Y0V1PVC9xNYCawgRmQGhmok4vhw18+eUP737b55CAVBSm1VRyFddN1R1C6huDbiSBINb1ThR4Iru2eKgnvBhul6OC7eOaTJCtlRnA7mKXhuhWkE1Rec7y2oLI4lejWKgtNM5UklcmW6wfFOM9XxNmxBq09XZosWiCdfFZcHy79OkgIOZ1O6BzBamocbZORNjERxqR/J8H4RErxXPaM045LJqLSUd6FsDaByJ4lNd3qB0M9s8DjRrryJ9Da2jOzyFziBruCR7ajVyDVLb41pgLyr8CK/xhPgGSKcNsBloOpGO0hhnrel2I0HlmY6bxW4B4uFlgvv76vi9RKIgTP4n5Cnj0TdOGQrpNlTJ4uWLvVLpG/YkG3dVYJgCNXCAv8+5EC30AaN4domX1mgfpMkrOp3acKz4teZxf4QoCZKvADO6SB/sjfGDEG4+f8ucOToQpNBmsTxk9TlVsXzMn2MExG/24mouzLBUzZnK31ZRW+TSXuHIezO+F9jbu6ZdQcrp+/tDkeNAIh+WiH4xdskqSF8mNMEPznt2P95LfBdjO2tP+68xRgLZzd0WFb9rMBDIs3C+CNxPRVv6TyvqpdqJXT6Vzd9/Mmoftp7t1Linhja/5/nXcSCkYFHrJsAf70udRI9Xjd1mN6fEwjlQTLaiiL6T/RJcyhoEasWjbTf8JTzyf6JwALPfob5OrAmwrKWFNhj3AzX05SCVqFz2mWbY0M9sAC766qkrHjG31zyOaI/eo2bMB42Sf5/T2bOnwC71+vekTyGWzBcLVkQUsSnGDRwhMoKa8oXiJS4nQN8zsF+IUoxJANg5kc3RTFh3L9UOX/o6RojP1T19kxIrNCNPmRO2IlSMC7oNNiIdSiWW/WBuUB7AHQv6cXyKfsChVYmAWDor1PuVOiy0PAgWsywQyNMrbWw7BPtt9bed/CTFBk3N50oXjlpyBUrh5hTQ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(7416005)(376005)(1800799015);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:40:26.5836 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7152c140-1b3e-48fe-7fe1-08dc81ba48e9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE30.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9341 According to PCI SIG ECN, calling the _DSM firmware method for a given CPU_UID returns the steering tags for different types of memory (volatile, non-volatile). These tags are supposed to be used in ST table entry for optimal results. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- drivers/pci/pcie/tph.c | 103 +++++++++++++++++++++++++++++++++++++++- include/linux/pci-tph.h | 34 +++++++++++++ 2 files changed, 136 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pcie/tph.c b/drivers/pci/pcie/tph.c index 320b99c60365..425935a14b62 100644 --- a/drivers/pci/pcie/tph.c +++ b/drivers/pci/pcie/tph.c @@ -158,6 +158,98 @@ static int tph_get_table_location(struct pci_dev *dev, u8 *loc_out) return 0; } +static u16 tph_extract_tag(enum tph_mem_type mem_type, u8 req_type, + union st_info *st_tag) +{ + switch (req_type) { + case PCI_TPH_REQ_TPH_ONLY: /* 8 bit tags */ + switch (mem_type) { + case TPH_MEM_TYPE_VM: + if (st_tag->vm_st_valid) + return st_tag->vm_st; + break; + case TPH_MEM_TYPE_PM: + if (st_tag->pm_st_valid) + return st_tag->pm_st; + break; + } + break; + case PCI_TPH_REQ_EXT_TPH: /* 16 bit tags */ + switch (mem_type) { + case TPH_MEM_TYPE_VM: + if (st_tag->vm_xst_valid) + return st_tag->vm_xst; + break; + case TPH_MEM_TYPE_PM: + if (st_tag->pm_xst_valid) + return st_tag->pm_xst; + break; + } + break; + default: + pr_err("invalid steering tag in ACPI _DSM\n"); + return 0; + } + + return 0; +} + +#define MIN_ST_DSM_REV 7 +#define ST_DSM_FUNC_INDEX 0xf +static bool invoke_dsm(acpi_handle handle, u32 cpu_uid, u8 ph, + u8 target_type, bool cache_ref_valid, + u64 cache_ref, union st_info *st_out) +{ + union acpi_object in_obj, in_buf[3], *out_obj; + + in_buf[0].integer.type = ACPI_TYPE_INTEGER; + in_buf[0].integer.value = 0; /* 0 => processor cache steering tags */ + + in_buf[1].integer.type = ACPI_TYPE_INTEGER; + in_buf[1].integer.value = cpu_uid; + + in_buf[2].integer.type = ACPI_TYPE_INTEGER; + in_buf[2].integer.value = ph & 3; + in_buf[2].integer.value |= (target_type & 1) << 2; + in_buf[2].integer.value |= (cache_ref_valid & 1) << 3; + in_buf[2].integer.value |= (cache_ref << 32); + + in_obj.type = ACPI_TYPE_PACKAGE; + in_obj.package.count = ARRAY_SIZE(in_buf); + in_obj.package.elements = in_buf; + + out_obj = acpi_evaluate_dsm(handle, &pci_acpi_dsm_guid, MIN_ST_DSM_REV, + ST_DSM_FUNC_INDEX, &in_obj); + + if (!out_obj) + return false; + + if (out_obj->type != ACPI_TYPE_BUFFER) { + pr_err("invalid return type %d from TPH _DSM\n", + out_obj->type); + ACPI_FREE(out_obj); + return false; + } + + st_out->value = *((u64 *)(out_obj->buffer.pointer)); + + ACPI_FREE(out_obj); + + return true; +} + +static acpi_handle root_complex_acpi_handle(struct pci_dev *dev) +{ + struct pci_dev *root_port; + + root_port = pcie_find_root_port(dev); + + if (!root_port || !root_port->bus || !root_port->bus->bridge) + return NULL; + + return ACPI_HANDLE(root_port->bus->bridge); +} + static bool msix_nr_in_bounds(struct pci_dev *dev, int msix_nr) { u16 tbl_sz; @@ -441,7 +533,16 @@ bool pcie_tph_get_st(struct pci_dev *dev, unsigned int cpu, enum tph_mem_type mem_type, u8 req_type, u16 *tag) { - *tag = 0; + union st_info info; + + if (!invoke_dsm(root_complex_acpi_handle(dev), cpu, 0, 0, false, 0, + &info)) { + *tag = 0; + return false; + } + + *tag = tph_extract_tag(mem_type, req_type, &info); + pr_debug("%s: cpu=%d tag=%d\n", __func__, cpu, *tag); return true; } diff --git a/include/linux/pci-tph.h b/include/linux/pci-tph.h index 4fbd1e2fd98c..79533c6254c2 100644 --- a/include/linux/pci-tph.h +++ b/include/linux/pci-tph.h @@ -14,6 +14,40 @@ enum tph_mem_type { TPH_MEM_TYPE_PM /* persistent memory type */ }; +/* + * The st_info struct defines the steering tag returned by the firmware _DSM + * method defined in PCI SIG ECN. The specification is available at: + * https://members.pcisig.com/wg/PCI-SIG/document/15470. + + * @vm_st_valid: 8 bit tag for volatile memory is valid + * @vm_xst_valid: 16 bit tag for volatile memory is valid + * @vm_ignore: 1 => was and will be ignored, 0 => ph should be supplied + * @vm_st: 8 bit steering tag for volatile mem + * @vm_xst: 16 bit steering tag for volatile mem + * @pm_st_valid: 8 bit tag for persistent memory is valid + * @pm_xst_valid: 16 bit tag for persistent memory is valid + * @pm_ignore: 1 => was and will be ignore, 0 => ph should be supplied + * @pm_st: 8 bit steering tag for persistent mem + * @pm_xst: 16 bit steering tag for persistent mem + */ +union st_info { + struct { + u64 vm_st_valid:1, + vm_xst_valid:1, + vm_ph_ignore:1, + rsvd1:5, + vm_st:8, + vm_xst:16, + pm_st_valid:1, + pm_xst_valid:1, + pm_ph_ignore:1, + rsvd2:5, + pm_st:8, + pm_xst:16; + }; + u64 value; +}; + #ifdef CONFIG_PCIE_TPH int pcie_tph_disable(struct pci_dev *dev); int tph_set_dev_nostmode(struct pci_dev *dev); From patchwork Fri May 31 21:38:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942321 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=SQdh/sPW; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-pci+bounces-8149-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc5Y3B7vz20Pb for ; Sat, 1 Jun 2024 07:41:53 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 076DE289ECF for ; Fri, 31 May 2024 21:41:52 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 64DB1823DC; Fri, 31 May 2024 21:40:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="SQdh/sPW" X-Original-To: linux-pci@vger.kernel.org Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2080.outbound.protection.outlook.com [40.107.100.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4E42208D4; Fri, 31 May 2024 21:40:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191646; cv=fail; b=bEKnshZe4ltE2fQCW/kxoqX5eQHZhUW1TAJye8KhLJwaQtKxkQm+uMflUWKp4hr4/UD+aHyx9xazALGVsNI/4/vxcHgWer9NQQ8U+y+8GKrWsTvgvzupuO3TRjEoh8TFb4sx9Rbw2BTjArn/9BWAgauTzRAskuq/iqkDgeHXBYc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191646; c=relaxed/simple; bh=3PN6My46xsk5M0E5ymn6jaJxzJz6OEGRK1s+KV31RCk=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mj4m15FF0luKweQ6VhCodq9yhxjpojcX8aYV774vKkZDRfosuk3xxyRLFpdgMUXtDVx5GR2hvE8Rt0z4qAUVdZi09YNhOMRaVO1sQze34v0vXzrgFdBX6w9rMxikyWndnnF0ecIt1VMemsZGpai9RtoVFS6NAbk5rtAwNhWv5xc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=SQdh/sPW; arc=fail smtp.client-ip=40.107.100.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VQUlXRnDt+sUUR8Mlcmtj/ix4VK8uxrVAFcJzaC4AXV4z2S0buHktdQrbVbRKikkjiQ13/hdg0mHRtI+1AJH8qeuEok/wqxQe6ePA8M96TlwbOemcxwlhodRduPdoenmKZmQpUQI8SfdQCMA03PQo9nNRduNCs5dDAwg/JmTk4UhlsCgnRdf/fOqK6JHqa8IFbhsyc3Le5e7pH3NX4sNbPkDelsBsduiODU9TSx5GZXzzfJcdCfEWeEGuOZRyK4qhospjKFaFROLnHgSMlgD7VO0A6a9wVBV9ob9iWW2RYQtjJv1IJBOkWwtUYa4/gPMlWXY5au+jhnXJBqZR9UQtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/yDcA/qlR2gvxpmAheEqE4dr2AMvLFvdV8bstO6LBBc=; b=EJI96UNwxvtfrs2NjLX8OgbUrUqsO4mOUACL3JQV8Kq6GEpRVQ6bmQ0LxEmVjL+304eQBsYbGDZxQr4KHEleqVjwigKhbqC7tty3aB6+CPVbppwrmuQq+huzmkHPM2N7XrvfQ4Vkwk9uF7KjM9aNn+rtt36vF8+lIXPP7Cx/5nYIK8nafR3hFNlYPxckO5Hi/y2aqJdhkRQ2IU3dk8kvSFsZfkTki+i80qZ7cJ+bLicBoMur3MA697V4Dr0q+myBxMtZIMQrTatS44/4vJFu1jmLvkYc2Kciy7gU78Z639BDpnvhjKiV3IYSpL2/wLkoSHGhbjI8dx7JEEp2Vgavmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/yDcA/qlR2gvxpmAheEqE4dr2AMvLFvdV8bstO6LBBc=; b=SQdh/sPWWX2kFF9rFtJZATVgb7Sll9vAiKblvZapK1YEeInHXCpWWMuWOei7qHWrzCCdnvWTiqlGhWcdeXH4tFn/jBNquoAI+T5acRJQ+Y8Toq6Dr8X4zsEz8jwbeqbh2Cs5SCQDtTPPWoKl4PCtrNod4xYZKfuLP4CGYsA7PjQ= Received: from BL1PR13CA0297.namprd13.prod.outlook.com (2603:10b6:208:2bc::32) by SN7PR12MB7912.namprd12.prod.outlook.com (2603:10b6:806:341::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Fri, 31 May 2024 21:40:41 +0000 Received: from BL6PEPF0001AB4B.namprd04.prod.outlook.com (2603:10b6:208:2bc:cafe::12) by BL1PR13CA0297.outlook.office365.com (2603:10b6:208:2bc::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7656.10 via Frontend Transport; Fri, 31 May 2024 21:40:41 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4B.mail.protection.outlook.com (10.167.242.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:40:41 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:40:39 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 7/9] PCI/TPH: Add TPH documentation Date: Fri, 31 May 2024 16:38:39 -0500 Message-ID: <20240531213841.3246055-8-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4B:EE_|SN7PR12MB7912:EE_ X-MS-Office365-Filtering-Correlation-Id: a3dd5628-651f-4842-47b1-08dc81ba51b7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005|7416005; X-Microsoft-Antispam-Message-Info: ViZuwbrxnM2qEptqEe1cToZJ8DTF6ga3fm3sF6UtUsQdIrUhzoOD6aY7zy4wE0z7AYdkPj48/JMjVYJMcuIygm5SpI8+giuv2KOtg498qiRi5NK/2rAZAXG1J5DJf13W2urYiBfKewaO14/Qy4rWHe5xk369LEbRVYDl8rS+yhUoBTOgp6NMQZjV2ngJmrhsAevTGdMfr2OIGM37bB7Q3mysUewrxaqfnmAUV24/TxeoALPonUS6HGjQNcMzMKK296KGphdnd7tKUYchNKMxQmarBFZZh1yzWbTAM6gLClH+0dwtU6fQFuuxyeXYuJy5hZ+rSRhFf84nhc9jSqUAgIH2Q97FWW/pTFwDS41WI6bG1bR6jFE+/MODJxHeW2x8MFOHI7EAp1BX5zco4CjjOziCvuy4esNlP3yziJS+86cf9zjcUhyPZPHgM3WQXwvvAQIW4UkPUG4MuvSRYZjhIPb1Z6uQ0/B4P4RpERxl6C2ZSTbx564yksuu0RyzWLhHSJ28lA4Pa/rhegPnK+Y1eBSLu4XaPo8ragqE9ABh1HT7rpg994wJOOMxt8Ry/Vxj9xd0JKIoaEMQDRNDBBfzL5wif2qI4Ah94/DEHqLVHYve77jcEdGgckain3sKxlTXmUyoEnyF6Po6w5LCKMg3/LiFIIMXJ9qcPspWAthgXFpjWJB1xx53T3ZqxFty/moPlZluMD54+sVEyhP8xI3O56xLdjTAIPWBvaO1W6WHFiYfNNnRxFWSs9oYweouXe99ft7w3mlLsiBqVWj2PRqx9GE5b5WThnbwDH68cKbKAjbK1zoJW5yaKeInUFPBnByE4mC0lpNdytsWoBKRpYTEvw8Extq+iMN5+rcHU9CRsgUJEyUSA+080JO3us+3GOsUExS4U166z+2EjF51Q6zGivz1ixvESoTYCphbrf7Gd2IAZQNj/xmdKrbLpDt1wlSXawC8eFuGI3IZI5JDBn84XMm/SPXDlA0LHUyqxVVzQeGwdJw1vdCG9kX/p1iLj98zBhglBPBKAF+zmhCn/tICXmGTu1nJdN2DS5b3C2cxHRxv8ozxnaUjD5nOnvFOFkuHPjTIisResbCHLpbMBEJdeJHTyjXpyaNsUCVNMlDKtlxVLCh7OSkseXUJB+4g0GEnhZIvhnL+8GTbQKHHPhD2IAYzB/Sq0Cu/unx+3uM/5sQyVcaeGv4eC2VknQI+XOLZ72JVr4IuABMsjH1eVK7u9gKWJDvB7AFeYLbCEfTXEXU9Ni8gzmTTUXdbX28qjMLkfum90IVAz+PZY9bxVc+/lSGW3qsiuXLLIe4uEyU5kcjQSVCkuPWixF9K/xX7bguKsduzN0NNGkH+eIdV/eLq6ZKhbE403XhbMpFbl6tBoWbjJt2Y7kLIzBn/uePc3vRb X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005)(7416005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:40:41.4008 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a3dd5628-651f-4842-47b1-08dc81ba51b7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4B.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7912 Provide a document for TPH feature, including the description of kernel options and driver API interface. Co-developed-by: Eric Van Tassell Signed-off-by: Eric Van Tassell Signed-off-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek Reviewed-by: Jonathan Cameron --- Documentation/PCI/index.rst | 1 + Documentation/PCI/tph.rst | 57 ++++++++++++++++++++++++++++ Documentation/driver-api/pci/pci.rst | 3 ++ 3 files changed, 61 insertions(+) create mode 100644 Documentation/PCI/tph.rst diff --git a/Documentation/PCI/index.rst b/Documentation/PCI/index.rst index e73f84aebde3..5e7c4e6e726b 100644 --- a/Documentation/PCI/index.rst +++ b/Documentation/PCI/index.rst @@ -18,3 +18,4 @@ PCI Bus Subsystem pcieaer-howto endpoint/index boot-interrupts + tph diff --git a/Documentation/PCI/tph.rst b/Documentation/PCI/tph.rst new file mode 100644 index 000000000000..ea9c8313f3e4 --- /dev/null +++ b/Documentation/PCI/tph.rst @@ -0,0 +1,57 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=========== +TPH Support +=========== + + +:Copyright: 2024 Advanced Micro Devices, Inc. +:Authors: - Eric van Tassell + - Wei Huang + +Overview +======== +TPH (TLP Processing Hints) is a PCIe feature that allows endpoint devices +to provide optimization hints, such as desired caching behavior, for +requests that target memory space. These hints, in a format called steering +tags, are provided in the requester's TLP headers and can empower the system +hardware, including the Root Complex, to optimize the utilization of platform +resources for the requests. + +User Guide +========== + +Kernel Options +-------------- +There are two kernel command line options available to control TPH feature + + * "notph": TPH will be disabled for all endpoint devices. + * "nostmode": TPH will be enabled but the ST Mode will be forced to "No ST Mode". + +Device Driver API +----------------- +In brief, an endpoint device driver using the TPH interface to configure +Interrupt Vector Mode will call pcie_tph_set_st() when setting up MSI-X +interrupts as shown below: + +.. code-block:: c + + for (i = 0, j = 0; i < nr_rings; i++) { + ... + rc = request_irq(irq->vector, irq->handler, flags, irq->name, NULL); + ... + if (!pcie_tph_set_st(pdev, i, cpumask_first(irq->cpu_mask), + TPH_MEM_TYPE_VM, PCI_TPH_REQ_TPH_ONLY)) + pr_err("Error in configuring steering tag\n"); + ... + } + +If a device only supports TPH vendor specific mode, its driver can call +pcie_tph_get_st() to retrieve the steering tag for a specific CPU and uses +the tag to control TPH behavior. + +.. kernel-doc:: drivers/pci/pcie/tph.c + :export: + +.. kernel-doc:: drivers/pci/pcie/tph.c + :identifiers: pcie_tph_set_st diff --git a/Documentation/driver-api/pci/pci.rst b/Documentation/driver-api/pci/pci.rst index aa40b1cc243b..3d896b2cf16e 100644 --- a/Documentation/driver-api/pci/pci.rst +++ b/Documentation/driver-api/pci/pci.rst @@ -46,6 +46,9 @@ PCI Support Library .. kernel-doc:: drivers/pci/pci-sysfs.c :internal: +.. kernel-doc:: drivers/pci/pcie/tph.c + :export: + PCI Hotplug Support Library --------------------------- From patchwork Fri May 31 21:38:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942322 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=QrBrVAeo; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-pci+bounces-8150-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc5z6WJGz20Pb for ; Sat, 1 Jun 2024 07:42:15 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id E7DA11F26F32 for ; Fri, 31 May 2024 21:42:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6494F7EF10; Fri, 31 May 2024 21:41:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="QrBrVAeo" X-Original-To: linux-pci@vger.kernel.org Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2052.outbound.protection.outlook.com [40.107.100.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACFE6132103; Fri, 31 May 2024 21:40:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.100.52 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191661; cv=fail; b=TeevGTNTpvFxIHlw7eSyKkl/VirfuBqNXkXM7efpxNlUHwAwjIJHoA+CBudBNgLrW8c4O7zGFcYFqZHNlyQmtpjruk71VTt3oCG1AkP1qOJM/a95hD41vBLb7CNrgjYB0djfu5tcc4+7JzDpRwHjOIQe67gJHLve572Ey6nQ7m8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191661; c=relaxed/simple; bh=gVAPO8A9qbDZ3d3KKdX8upTYxJ7EQYzYoBu8myehXHQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=M6Ff1/NZsHs08FiKEWFY1rVTCDYBkcJ6tnZV0K5n/a2BRjbKg7q6ybdRDHwVM/siV7x6c/fXZfHKupwS5Hmw0AFbDgzMeeWoi6bmQioqbXgg+DrQ9pBgPjd5anPheSonnLF5vCTkcArluItSFJ3XQ7tt7Klw878ZXxPLCrJ23pM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=QrBrVAeo; arc=fail smtp.client-ip=40.107.100.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iIM7k8qzwkswhMxD6aYaewq7+1GF9t0o8ln4vpLw6ZGkwRNFOw8wCrYWp/HN2H0ye6S7m4ZtwMudQuQNmloqvodXKx3PZ/R12cTn0ey+Rr43v/hcI4t/iR4j/zL+R8CWGU23xhq+sMYyqgfsayYXlcdgFouw0jfno0SyTjMkjpyjWhrZp8ci8K7KI3yux1vdEZeE629CVem7WmRSUiq8iRQs9M7wBhCUvAg8L/x2D/15rQTmw1U0ySM8ipW6qIhmsBp7zL3dqMs1GtVHDfpyS0q5w5EFDBjq1wwBDEookq5jyI2y2zMUts2RRki61FtdIpIeeWc7Vzp/CuZmmosp0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vFreZlJ6kEVOKJTpADeeDAOTkBq0d7DEUnWEIsazckk=; b=MTVTHDlwHZJNJfS0b7dpFTsLbnWqrRvVWlnApAjevIusUarsr5FvpCoPstUDT++McXgQFYcLRZ5aGtWkKn2qa5Kc5EH8HrUYztxYrlOrHS/77MvSogJXNhfGN76Pnx9paWz9z4Uq96yVRpWFDNWIjPpGG4Lt9g43ntu3mQ8t6XBQTGIVJyC+wTcTwvKsImQbKmbP1rQ6yJTNoAwXXWDwjd6cvxD3qx8KsXNhCK2yL+usIso/j5D/gE6RR8QW5lBbxRpY+PAGttZrLYFl16XW1/EY8ViTDFkv2DBUgCG4pw/zYqNwJq9Nz4mOxkzMCP7wJy8qAzCvXmKwMHbDJ5k5fg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vFreZlJ6kEVOKJTpADeeDAOTkBq0d7DEUnWEIsazckk=; b=QrBrVAeoZvkEhsFQF4oddW2fHnHSmuXdXaj59QgTbgwG7DH8+TqgNFKG0DH3XNpvnyOKJdTwSmNd6xsHdFakmfTY3BwU8ljnPEkfcSjEWt/bLz2ecOkxLwwTq0HbVez4Pdhw+lIRRVDAbpQ6tWpSjC37o/0aSlKlGiZtmWn5VqQ= Received: from BN9PR03CA0203.namprd03.prod.outlook.com (2603:10b6:408:f9::28) by CY8PR12MB7564.namprd12.prod.outlook.com (2603:10b6:930:97::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22; Fri, 31 May 2024 21:40:57 +0000 Received: from BL6PEPF0001AB50.namprd04.prod.outlook.com (2603:10b6:408:f9:cafe::62) by BN9PR03CA0203.outlook.office365.com (2603:10b6:408:f9::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.22 via Frontend Transport; Fri, 31 May 2024 21:40:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB50.mail.protection.outlook.com (10.167.242.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:40:56 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:40:55 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 8/9] bnxt_en: Add TPH support in BNXT driver Date: Fri, 31 May 2024 16:38:40 -0500 Message-ID: <20240531213841.3246055-9-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB50:EE_|CY8PR12MB7564:EE_ X-MS-Office365-Filtering-Correlation-Id: 031bbb94-7464-43e2-d3b9-08dc81ba5aee X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005|7416005; X-Microsoft-Antispam-Message-Info: gyP2XUUPs0SsYrbFVR8FPdi4SQuysLyEP3ysDDZ015zC6dXHK0joaALxXggxTwTTz3Ri9+sIvPwXC8OqTfOKtWK5k4m/F3YBumsqqTwmrbqcMkkpEjIu8bv4W3ti+9VzzpaD6UwyPGg0D9F2xwUC5dgD3wU00lqevj1oXs7DdfPhOMp1MsmLjfRH/Etm/Mt9VT1Az71GEeaf7r6zue8w4TSqyMjo3vBFNbFC6Hpal8lTGWHCcCsTBCJfyx99TbqP8zos0JmJjqihOPVKG7CnUKXluGg6NGg6GUgnvWqOcDr5yM4KnULVgV53AXqjtMcXmVhoydcFNXcKMW2z6gSxl9gmO1Y5CR28KVWfxj8PmXkR+aoQAEH1LKcw6h3kwynygGBo3q+7u3BTbxGPSS7Q/l99DPH0uRNCI1NiviQHXjS3mD2dic8RPcCxJstB1uXS/DYy1TrA+RdXnewH7kTp1nzCYU27JGtJLjJpfd9hdfBQk1IDh08crLTAnhcmr7pkUD6P/TM8PneQ8w2OFt1idqpmEfWMyAq42h1zkAcLWLL703s8iQzA20ffI3PGM8F3CEcu99wzhJCWep8j23LDytWJkvNHdg3GXAHzJgE3WaMt6jVEvKPxfBJb3Ol5LFofAfl7ZxYPxwQ5PGpxCmhEXJ5L5dcZr+MTGPzLb6GsQYutfydkvQYUoBa6csBEE2T7gZhnNnXeVGc48MsoRCq8up5bRChZQx3oi6i+BwnEgHhdR2x9mPdvADJAPmboy/beZwiyqN2mHfXHnJgoJJa9ODkL9zyoOqQif5y+MZqZe5L+F0T2fjuVoXUPNGzAb4vH8Kr5DG0rPCQ0zG0mRQUo7RCmyKOYUXvaj0xdmDeJKjntvqHrNOrHdo/9nVQ/4jidw/iqHw+XIGCVa5U5DR4aEOYEN5+1Itvi9HWQyqVuJBTwqhX6BbWx66De4ngNYxo84xkHkeK/kq/ep1sBNE0Ssa8plhnilXq8UMa4x1eTtg6qiHJnrOx6B7FwyJBvJlLmQAjDjeBq6AMOCRmxBeoJv/g7P3B3t4ec8hnzZInzTjzNGzVf1IfmyqU6APbEY00onaa5A/MRjSKmkAm/dBsPzu/6ZQUj3rNG2nXj8aJkpaXF379HMHYMVxP5ahLbA7530IT8zPRoninEfpc79MgkwdXTqQXr3tiNAywjgffkVn8SaAAbeG3Y/Te6DM6YXZvq/KBJq9YHHyi6UjWbGFCiZCPRPx5tXafif4TkSNbabDsckILzjQ5N2j3zsrY2GljemEoVFfAEAFediPI5plus/DaB6sy+ej5fHWOKBvMM1dyYqT/gjZzzqH5M/NMeI559Yr7iN74+2qX6E5GrMjOcW74ymiSFGHp7s6iXYvCJHf8iNYgx0pNKvmSDu26lI3Cs X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005)(7416005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:40:56.8800 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 031bbb94-7464-43e2-d3b9-08dc81ba5aee X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB50.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7564 From: Manoj Panicker As a usage example, this patch implements TPH support in Broadcom BNXT device driver by invoking pcie_tph_set_st() function when interrupt affinity is changed. Signed-off-by: Manoj Panicker Reviewed-by: Wei Huang Reviewed-by: Ajit Khaparde Reviewed-by: Somnath Kotur Reviewed-by: Andy Gospodarek --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 54 +++++++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt.h | 4 ++ 2 files changed, 58 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index c437ca1c0fd3..2207dac8ce18 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -55,6 +55,7 @@ #include #include #include +#include #include "bnxt_hsi.h" #include "bnxt.h" @@ -10683,6 +10684,8 @@ static void bnxt_free_irq(struct bnxt *bp) free_cpumask_var(irq->cpu_mask); irq->have_cpumask = 0; } + if (pcie_tph_intr_vec_supported(bp->pdev)) + irq_set_affinity_notifier(irq->vector, NULL); free_irq(irq->vector, bp->bnapi[i]); } @@ -10690,6 +10693,45 @@ static void bnxt_free_irq(struct bnxt *bp) } } +static void bnxt_rtnl_lock_sp(struct bnxt *bp); +static void bnxt_rtnl_unlock_sp(struct bnxt *bp); +static void __bnxt_irq_affinity_notify(struct irq_affinity_notify *notify, + const cpumask_t *mask) +{ + struct bnxt_irq *irq; + + irq = container_of(notify, struct bnxt_irq, affinity_notify); + cpumask_copy(irq->cpu_mask, mask); + + if (!pcie_tph_set_st(irq->bp->pdev, irq->msix_nr, + cpumask_first(irq->cpu_mask), + TPH_MEM_TYPE_VM, PCI_TPH_REQ_TPH_ONLY)) + netdev_dbg(irq->bp->dev, "error in setting steering tag\n"); + + if (netif_running(irq->bp->dev)) { + rtnl_lock(); + bnxt_close_nic(irq->bp, false, false); + bnxt_open_nic(irq->bp, false, false); + rtnl_unlock(); + } +} + +static void __bnxt_irq_affinity_release(struct kref __always_unused *ref) +{ +} + +static inline void bnxt_register_affinity_notifier(struct bnxt_irq *irq) +{ + struct irq_affinity_notify *notify; + + notify = &irq->affinity_notify; + notify->irq = irq->vector; + notify->notify = __bnxt_irq_affinity_notify; + notify->release = __bnxt_irq_affinity_release; + + irq_set_affinity_notifier(irq->vector, notify); +} + static int bnxt_request_irq(struct bnxt *bp) { int i, j, rc = 0; @@ -10735,6 +10777,7 @@ static int bnxt_request_irq(struct bnxt *bp) int numa_node = dev_to_node(&bp->pdev->dev); irq->have_cpumask = 1; + irq->msix_nr = map_idx; cpumask_set_cpu(cpumask_local_spread(i, numa_node), irq->cpu_mask); rc = irq_set_affinity_hint(irq->vector, irq->cpu_mask); @@ -10744,6 +10787,17 @@ static int bnxt_request_irq(struct bnxt *bp) irq->vector); break; } + + if (pcie_tph_intr_vec_supported(bp->pdev)) { + irq->bp = bp; + bnxt_register_affinity_notifier(irq); + + /* first setup */ + if (!pcie_tph_set_st(bp->pdev, i, + cpumask_first(irq->cpu_mask), + TPH_MEM_TYPE_VM, PCI_TPH_REQ_TPH_ONLY)) + netdev_dbg(bp->dev, "error in setting steering tag\n"); + } } } return rc; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 656ab81c0272..4a841e8ccfb7 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1195,6 +1195,10 @@ struct bnxt_irq { u8 have_cpumask:1; char name[IFNAMSIZ + 2]; cpumask_var_t cpu_mask; + + int msix_nr; + struct bnxt *bp; + struct irq_affinity_notify affinity_notify; }; #define HWRM_RING_ALLOC_TX 0x1 From patchwork Fri May 31 21:38:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 1942323 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=amd.com header.i=@amd.com header.a=rsa-sha256 header.s=selector1 header.b=jdgrGhZC; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45d1:ec00::1; helo=ny.mirrors.kernel.org; envelope-from=linux-pci+bounces-8151-incoming=patchwork.ozlabs.org@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org [IPv6:2604:1380:45d1:ec00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Vrc6L3ZGQz20Pb for ; Sat, 1 Jun 2024 07:42:34 +1000 (AEST) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 4B2AA1C24B27 for ; Fri, 31 May 2024 21:42:32 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 69DFF81AA5; Fri, 31 May 2024 21:41:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="jdgrGhZC" X-Original-To: linux-pci@vger.kernel.org Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2049.outbound.protection.outlook.com [40.107.212.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 994DC78C9C; Fri, 31 May 2024 21:41:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.49 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191679; cv=fail; b=c7aZ2BEnZkZHyMgwgywTmwbnhYe6IIYN9kqnpRNudVStxkHLgoB0zt4huLyIsHCzynlDPfE50DapbQavcirWiu680H0OavM4+iKm9SaR5sGTizWGMfejHjLiChmeXt7dlFqKIiLH6yE1r76Ymxwkok69yBrkrespuA9RCLp4mpI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717191679; c=relaxed/simple; bh=btENwbSnpFzoHh1T01k2JDYx10vsDoRjrXVQQ9ca9+I=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ITt5xkwluWbyxWV8bzIYZCkbtu0PmhBc95RQF6eEhhWUF7F52IwFJpI8fADJYN/1VZIKnV6/OLVN2DfTvbvvuv4ZLplZF/zM+n557Zx4gGujaEUmPUmCUxzoQzWMdi1ChaJb6g/8thCyFs86y1wWbiNBsd61WRU1F2jqbFlzMRY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=jdgrGhZC; arc=fail smtp.client-ip=40.107.212.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fDz5xJQYuzQ/AwMZ2+Q7R/hiKazsdXV63vZuFPEmIHU6UlYIZ10q9AqSEHxG8EV0/iY/dY7KKpPr/F/WIpQHhgiIsVmjbm+bod2E/PL3S6ptKVl+Ll4WCBdIduLvEzzKFNtizOSX5bUEdegsoIMUQAqgW5hz9U2X9OMtv3JAfHe+20lGTkcJNaaIZkaKXWDJu+F8GRMubULn0hAABDd7zCR9WXxGrI28+z2GuYaaOgurEZ/8wUFEZShXC1jcJh7AoBPpjMR/SFniYXjQ7rQ+ZW0sPouji+21ENcvUXff5YrIa7yeHL1oedgt+AgIO8S15pIC3AE+cArEw/KKJkr8xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/FNRNjmADYCA1uNaWioZ9Zr+Zk5zuPBkHnmWIsj3W+8=; b=noRfpcZsvQfRwZ7cVHVLT9yiPB6X82Ooxr1zzRNsBFsSZ4cAGImTv1L9jGjW4tseXC1/U1zUVkLNefEmtqR1Jl5EgdAwe9VaJm6v8bHF/d40aWrHrElcna2FYInUTO+uY87ODbL99lga55RRMrsk6gJQLjjjM4Ip3/l+e0R7i7l/O3ZFAL9R9ORtqBEDNmywOe3iEntxSAJwLp5c6KhHqz2gWTcHkpN95ieoBDR+6a4oRyQn3xuT4jdNG8oCEs+yYBYbdaqXO7NIyMEhbz8qcNN9qN1meSkbHWq93UYeM8YA9KHfVyH+jCwWwCQTUd62EE3Sb7cKUBUuW/nULHPTsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/FNRNjmADYCA1uNaWioZ9Zr+Zk5zuPBkHnmWIsj3W+8=; b=jdgrGhZCXFPUdV9P3GHxoASqU6YC+9aTXWvqZd6j85q1yUqKT0F5FBJR+6PwNcj+NQYSX77NM3W5xSw2j+ICv0XNvZk2WRN6wDlThpH3/zt6jZWrucUuWeAzuhe+8HZXE1D/01m7YxsU4sXJY+7Sfz+XZP2YKLK19rFNbW2pXGE= Received: from BN9PR03CA0171.namprd03.prod.outlook.com (2603:10b6:408:f4::26) by PH8PR12MB6988.namprd12.prod.outlook.com (2603:10b6:510:1bf::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Fri, 31 May 2024 21:41:13 +0000 Received: from BL6PEPF0001AB4E.namprd04.prod.outlook.com (2603:10b6:408:f4:cafe::86) by BN9PR03CA0171.outlook.office365.com (2603:10b6:408:f4::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.31 via Frontend Transport; Fri, 31 May 2024 21:41:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL6PEPF0001AB4E.mail.protection.outlook.com (10.167.242.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7633.15 via Frontend Transport; Fri, 31 May 2024 21:41:13 +0000 Received: from weiserver.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Fri, 31 May 2024 16:41:12 -0500 From: Wei Huang To: , , , CC: , , , , , , , , , , , , , , , , , Subject: [PATCH V2 9/9] bnxt_en: Pass NQ ID to the FW when allocating RX/RX AGG rings Date: Fri, 31 May 2024 16:38:41 -0500 Message-ID: <20240531213841.3246055-10-wei.huang2@amd.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240531213841.3246055-1-wei.huang2@amd.com> References: <20240531213841.3246055-1-wei.huang2@amd.com> Precedence: bulk X-Mailing-List: linux-pci@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4E:EE_|PH8PR12MB6988:EE_ X-MS-Office365-Filtering-Correlation-Id: 8510eab1-3843-4900-ec3b-08dc81ba64b5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|7416005|1800799015|82310400017|376005; X-Microsoft-Antispam-Message-Info: 5KPmR4vkT0zaOwqXL0T/DSky+3OcjymRMYvLddz+rrwjWDLDUnqetuGFCjEirTNwFs0iiIWjWtlQOhVefWlWJ5W8+5QnuazvmcoH9tzqhZAMNGLLILm1Omy1u08AjY0yKZS1LPxkUd5vLY5D1VuJKUPCgo7hmMZCnTgpEn4HbN/H4/BVZk9LJ5kR9QxOIHPupI/I9SRIT9V3Hwi7hcafobk5gtV2jEbwmPVmyKTsd8OX1AwgOT+vQKhTxQI2mhKI86vX8grhRZhqYW0X0VZe96oJIUmkMypt34zEzFB1CrXPKjH7dJrUHWy7rr93nL6bo6RU7/MhMIlSrWKWIeg0S0OQBcsn1Mr+jUCNONwJhSMATRLOxXi1ZC+2mNKgz1Ift9EUmt4X/wwxeHE2+EZ70gjMvT3Xo+5lzXATXmK4opQ/QyX5bCJEiPix834wSUgswwaW+8yFqco3fxjaCvdqIaqVQzMOQ6flxwEdh0LDmgCjJ722ytqiyhx0n2EAxeN43CkbyW/LDQWz+34frwsT8Ws8LhT0TLJdok/J6mbMBNUxCguKdTtgBUXBHh/LokMRwbCXNH9OdSdXdRMJ1+8KBuTovec3uQmZfSaVZyTRdWWInuLDgIueegHVgEcqNlAaeq0VVLx7wDeXTFNQpx4lq671/cdub6X+wDtJHz86wyRL+fEpsEt+rtYkY4ZcH1pZG/UuXbn57eOdMKWbMi9En4xzRs6FzmqbRy1q3ZKrCxqb2fsmyGSHBoEIZKw/8yCeCjj13vVBIeg701xDyOGuYfj1wDJ1brzD2V9va5H3NHasb0MaaagvYXyEw2iwtZg+qJImTwGrXEsc4j9BYjkurAti4to0AD50g4haAqRvovbK0x+03BVNUBelCIxfNO7vjiso3yGN99xWvJuWjmfM6/HVQxiBrtCYC/rLgUSmkOMw8Cbav+xE+Oh6m1HJZYREAjcrRgUNDzp3/EIsFtuHE/P4/CV6ZF+Z8owfFm6Rr1tHLIe8hUDkjEgSuen1F6JK8YWoD67YtF0xzywDUDRpONjnENAlHpcCufca7u0nXRVO7zrKu1yfL7Jrvj5UuK/Y3ExJZfEjLh8a6llJVHeQKr99qbYrB4qogS01vttAPfyCU5hpAtJmbF8VQJ/pBIZ10AjS76okgG2MsTGFw676zPlps/S0PaG+ZtHSVRqRRrZTnUdC2W2wfk4B4amzmAD/kniv69/VpfZRdAGFG3zbd92/5aavvnJum669aIRthdVugAFdhGfAmaCUoID60v2YqQNwD65zlqTokiMyg1lgJVjKxcsC9AdWVMMnKy0kzpFal/kYNOHaPkwBv1LsRD5WFl/dY92xCxgVTuLBDv6O77VxV+ygPscCQ5C9jAtJWb9v1UNnWPl1gwqfv1eBykpM X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(7416005)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 21:41:13.2843 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8510eab1-3843-4900-ec3b-08dc81ba64b5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4E.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6988 From: Michael Chan Newer firmware can use the NQ ring ID associated with each RX/RX AGG ring to enable PCIe steering tag. Older firmware will just ignore the information. Signed-off-by: Michael Chan Signed-off-by: Andy Gospodarek Reviewed-by: Hongguang Gao Reviewed-by: Ajit Khaparde --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 2207dac8ce18..308b4747d041 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -6699,10 +6699,12 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp, /* Association of rx ring with stats context */ grp_info = &bp->grp_info[ring->grp_idx]; + req->nq_ring_id = cpu_to_le16(grp_info->cp_fw_ring_id); req->rx_buf_size = cpu_to_le16(bp->rx_buf_use_size); req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); req->enables |= cpu_to_le32( - RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); + RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID | + RING_ALLOC_REQ_ENABLES_NQ_RING_ID_VALID); if (NET_IP_ALIGN == 2) flags = RING_ALLOC_REQ_FLAGS_RX_SOP_PAD; req->flags = cpu_to_le16(flags); @@ -6714,11 +6716,13 @@ static int hwrm_ring_alloc_send_msg(struct bnxt *bp, /* Association of agg ring with rx ring */ grp_info = &bp->grp_info[ring->grp_idx]; req->rx_ring_id = cpu_to_le16(grp_info->rx_fw_ring_id); + req->nq_ring_id = cpu_to_le16(grp_info->cp_fw_ring_id); req->rx_buf_size = cpu_to_le16(BNXT_RX_PAGE_SIZE); req->stat_ctx_id = cpu_to_le32(grp_info->fw_stats_ctx); req->enables |= cpu_to_le32( RING_ALLOC_REQ_ENABLES_RX_RING_ID_VALID | - RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID); + RING_ALLOC_REQ_ENABLES_RX_BUF_SIZE_VALID | + RING_ALLOC_REQ_ENABLES_NQ_RING_ID_VALID); } else { req->ring_type = RING_ALLOC_REQ_RING_TYPE_RX; }