From patchwork Thu Aug 8 07:20:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970404 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=ef+fuBxD; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdjr0vSbz1yfV for ; Thu, 8 Aug 2024 17:20:32 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 9261E80C53; Thu, 8 Aug 2024 07:20:30 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Tpf5GJ7Ozc4B; Thu, 8 Aug 2024 07:20:29 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 955BC8142B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101629; bh=0eM/MSAybzdG43x8zln4KkRpSlbf4jE3U30r45cWqnQ=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=ef+fuBxDX7rSBT7NipTVrRlsmyCEFyKCLtWz22ZjlWyGwPfUEQfsIAnQB5q49q1vi 2S49TBZCkdZcSgP1xxVEdmHvmhTxRp9ZU8fsSsLPDXVzPshArU7/lptNcL8LzLAgCg g0617Ku3f3s7h8gu3cGzxeSnyF3D4TIZ5YOMP0SLTxuQliDm5R/ifkEk/nOMkA/MOr c7rjqucwlGJVF9701+nCOu5LI/7WduUqqx9tQyRN3u8NEXyeJBlDvjp049D3B1XVe1 G27MsVp63qh7D9CUf0/77pogcq9Mp4vCkmJAnIGe7km2b9NvkK2E1qYsXC3AHNrPMx jh7dKQLEIu7Xg== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 955BC8142B; Thu, 8 Aug 2024 07:20:29 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id A45671BF369 for ; Thu, 8 Aug 2024 07:20:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 992B760772 for ; Thu, 8 Aug 2024 07:20:23 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id GDrkXDgCn017 for ; Thu, 8 Aug 2024 07:20:23 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org BFE82607F3 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BFE82607F3 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id BFE82607F3 for ; Thu, 8 Aug 2024 07:20:22 +0000 (UTC) X-CSE-ConnectionGUID: ajmEEgjqSAqsAGvs5K7fag== X-CSE-MsgGUID: tCK4vscGQZeVBrL/8BTQpQ== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361338" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361338" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:22 -0700 X-CSE-ConnectionGUID: XLpM+XxkRR2LtDAwpT0TYg== X-CSE-MsgGUID: GgCcubMuSLanyRL3Ext6RA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073455" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:19 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:09 +0200 Message-ID: <20240808072016.10321-2-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101623; x=1754637623; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xptcCGYuG32q+AtCv8x+/YYVF/62bjFzUKScHMnQ76o=; b=UDknW9hvjxnq6Lc5//2jIKZQEDI9MhL++zPLvfD9rQXtg1vEPN3wihp3 IqZAqAlXhZ8nij/RldHVGWAJIBzdHyyWVzzNxrvggfAspWAtIB3DnxaxR 5p6tXGnGWvlPcRZGYXiG/7669CDLC4+hT8sCI9LrepcsfJnce9y4vdWl/ zeP+dK29Uy20zWlKPAVFAlkOEr1nmuNTED4bTcHfLtG8YkkJLAB/sQ8Lf pg2K/B3VyObWv1Lx2W1HlCPpkER9YdguxrUt6lobt7FsNWE6UiUJautdz kXVmKVOZtEg1zd2K4DR2Qpr2KSVViKcIJH823pY3BfjXRu3S8VqnWSFgc A==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=UDknW9hv Subject: [Intel-wired-lan] [iwl-next v3 1/8] ice: devlink PF MSI-X max and min parameter X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Use generic devlink PF MSI-X parameter to allow user to change MSI-X range. Reviewed-by: Wojciech Drewek Signed-off-by: Michal Swiatkowski --- .../net/ethernet/intel/ice/devlink/devlink.c | 56 ++++++++++++++++++- drivers/net/ethernet/intel/ice/ice.h | 8 +++ drivers/net/ethernet/intel/ice/ice_irq.c | 14 ++++- 3 files changed, 76 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.c b/drivers/net/ethernet/intel/ice/devlink/devlink.c index 29a5f822cb8b..bdc22ea13e0f 100644 --- a/drivers/net/ethernet/intel/ice/devlink/devlink.c +++ b/drivers/net/ethernet/intel/ice/devlink/devlink.c @@ -1518,6 +1518,32 @@ static int ice_devlink_local_fwd_validate(struct devlink *devlink, u32 id, return 0; } +static int +ice_devlink_msix_max_pf_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + if (val.vu16 > ICE_MAX_MSIX) { + NL_SET_ERR_MSG_MOD(extack, "PF max MSI-X is too high"); + return -EINVAL; + } + + return 0; +} + +static int +ice_devlink_msix_min_pf_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + if (val.vu16 <= ICE_MIN_MSIX) { + NL_SET_ERR_MSG_MOD(extack, "PF min MSI-X is too low"); + return -EINVAL; + } + + return 0; +} + enum ice_param_id { ICE_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, ICE_DEVLINK_PARAM_ID_TX_SCHED_LAYERS, @@ -1535,6 +1561,15 @@ static const struct devlink_param ice_dvl_rdma_params[] = { ice_devlink_enable_iw_validate), }; +static const struct devlink_param ice_dvl_msix_params[] = { + DEVLINK_PARAM_GENERIC(MSIX_VEC_PER_PF_MAX, + BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, ice_devlink_msix_max_pf_validate), + DEVLINK_PARAM_GENERIC(MSIX_VEC_PER_PF_MIN, + BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, ice_devlink_msix_min_pf_validate), +}; + static const struct devlink_param ice_dvl_sched_params[] = { DEVLINK_PARAM_DRIVER(ICE_DEVLINK_PARAM_ID_TX_SCHED_LAYERS, "tx_scheduling_layers", @@ -1637,6 +1672,7 @@ void ice_devlink_unregister(struct ice_pf *pf) int ice_devlink_register_params(struct ice_pf *pf) { struct devlink *devlink = priv_to_devlink(pf); + union devlink_param_value value; struct ice_hw *hw = &pf->hw; int status; @@ -1645,11 +1681,27 @@ int ice_devlink_register_params(struct ice_pf *pf) if (status) return status; + status = devl_params_register(devlink, ice_dvl_msix_params, + ARRAY_SIZE(ice_dvl_msix_params)); + if (status) + return status; + if (hw->func_caps.common_cap.tx_sched_topo_comp_mode_en) status = devl_params_register(devlink, ice_dvl_sched_params, ARRAY_SIZE(ice_dvl_sched_params)); + if (status) + return status; - return status; + value.vu16 = pf->msix.max; + devl_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MAX, + value); + value.vu16 = pf->msix.min; + devl_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MIN, + value); + + return 0; } void ice_devlink_unregister_params(struct ice_pf *pf) @@ -1659,6 +1711,8 @@ void ice_devlink_unregister_params(struct ice_pf *pf) devl_params_unregister(devlink, ice_dvl_rdma_params, ARRAY_SIZE(ice_dvl_rdma_params)); + devl_params_unregister(devlink, ice_dvl_msix_params, + ARRAY_SIZE(ice_dvl_msix_params)); if (hw->func_caps.common_cap.tx_sched_topo_comp_mode_en) devl_params_unregister(devlink, ice_dvl_sched_params, diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index d6f80da30dec..a67456057c77 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -95,6 +95,7 @@ #define ICE_MIN_LAN_TXRX_MSIX 1 #define ICE_MIN_LAN_OICR_MSIX 1 #define ICE_MIN_MSIX (ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_LAN_OICR_MSIX) +#define ICE_MAX_MSIX 256 #define ICE_FDIR_MSIX 2 #define ICE_RDMA_NUM_AEQ_MSIX 4 #define ICE_MIN_RDMA_MSIX 2 @@ -545,6 +546,12 @@ struct ice_agg_node { u8 valid; }; +struct ice_pf_msix { + u16 cur; + u16 min; + u16 max; +}; + struct ice_pf { struct pci_dev *pdev; struct ice_adapter *adapter; @@ -615,6 +622,7 @@ struct ice_pf { struct msi_map ll_ts_irq; /* LL_TS interrupt MSIX vector */ u16 max_pf_txqs; /* Total Tx queues PF wide */ u16 max_pf_rxqs; /* Total Rx queues PF wide */ + struct ice_pf_msix msix; u16 num_lan_msix; /* Total MSIX vectors for base driver */ u16 num_lan_tx; /* num LAN Tx queues setup */ u16 num_lan_rx; /* num LAN Rx queues setup */ diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c index ad82ff7d1995..4e559fd6e49f 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.c +++ b/drivers/net/ethernet/intel/ice/ice_irq.c @@ -252,7 +252,19 @@ void ice_clear_interrupt_scheme(struct ice_pf *pf) int ice_init_interrupt_scheme(struct ice_pf *pf) { int total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors; - int vectors, max_vectors; + union devlink_param_value value; + int vectors, max_vectors, err; + + /* load default PF MSI-X range */ + err = devl_param_driverinit_value_get(priv_to_devlink(pf), + DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MIN, + &value); + pf->msix.min = err ? ICE_MIN_MSIX : value.vu16; + + err = devl_param_driverinit_value_get(priv_to_devlink(pf), + DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MAX, + &value); + pf->msix.max = err ? total_vectors / 2 : value.vu16; vectors = ice_ena_msix_range(pf); From patchwork Thu Aug 8 07:20:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970405 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=jhbFfd/F; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdjv01j0z1ydk for ; Thu, 8 Aug 2024 17:20:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id DA25E81521; Thu, 8 Aug 2024 07:20:32 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 4k1zWNLGW5yw; Thu, 8 Aug 2024 07:20:30 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AB03A8142B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101630; bh=2M0KT6dFUmPlMKIDVahDI6YBsO4ih2oRkh80lE0jFHA=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=jhbFfd/F+xlSVYwG3WshXAql2CE1w7W1q+30pp5l1dx7syYrnojOl313OxA95y9jI /QpONvAzBU96zd9PBZJ8jy0Skl00TN2P+zoEc6/FjzZw5d1qYJBpD4yvB0HRxqlobb VVZP6BVd/SZq+RjVimM412Ag2SOHjdKRCkNaMd5lqiCqwlWmbWflvn9QA6Prgvni1c g8ClbQ79iSjc1rKiQdG3HJ7Dg1+tD1aEu7KarV+0HAOJEe/asmAT7PBGKPDZzxkGlD izCEDlagK5e1kTwSJwzO2xN8WXMC4f6VnKP/VPW68fCDxWbyXB713dFVr1034xcqZs E2GN1l6kGmhoQ== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id AB03A8142B; Thu, 8 Aug 2024 07:20:30 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id B12421BF369 for ; Thu, 8 Aug 2024 07:20:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9ED93607F3 for ; Thu, 8 Aug 2024 07:20:26 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id HL_3n0XWfmve for ; Thu, 8 Aug 2024 07:20:25 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org B267A60772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B267A60772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id B267A60772 for ; Thu, 8 Aug 2024 07:20:25 +0000 (UTC) X-CSE-ConnectionGUID: p4FV3MdGS1uq1dHLJrDjUw== X-CSE-MsgGUID: AXqxIyEqRayZzJ+7gwRQeQ== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361346" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361346" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:25 -0700 X-CSE-ConnectionGUID: fiO8PlElRv+hZS+VE7Fo6Q== X-CSE-MsgGUID: awvjiorsRfSaZ9AeNQ88/A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073467" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:22 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:10 +0200 Message-ID: <20240808072016.10321-3-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101626; x=1754637626; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A4XtbszZiM7QqSIYGR3X4epx10Nl7udz3xpHFv+d9Bo=; b=cbrUCCx+gZ107UWq3HVH2yf5AxEaee12HGYWVC4zXS1lRYGpjq4nKlhF 5mjWePRD0YFwrPOXxnzK6Mxsd0IXPgqfw82cqp1sgy2YI2FxvWYnw4b2s lAjXWYRLJOmy6Oqv9Wts8VQvYsI3DfnwqAbP7d6pjfzGYAEx3lu9bqt+g IAcrWj8X0jR17/kqPY3wYBikjsN1+xM3IP20BKI3K48MKLHpRiL6Zxx8+ Lwv/K7UPrL+9EmPLfxgH0IGO5OQkpg8JmbmHXlxWll212nRdhvAq0uUZ/ a+VjEzgqBBYS41lbCYL4G+KWPIJyPcMQHPOmneCXeohV/SBVZuBKX7yeg Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=cbrUCCx+ Subject: [Intel-wired-lan] [iwl-next v3 2/8] ice: remove splitting MSI-X between features X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" With dynamic approach to alloc MSI-X there is no sense to statically split MSI-X between PF features. Reviewed-by: Wojciech Drewek Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_irq.c | 168 ++--------------------- 1 file changed, 10 insertions(+), 158 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c index 4e559fd6e49f..4579df21701b 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.c +++ b/drivers/net/ethernet/intel/ice/ice_irq.c @@ -84,157 +84,6 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only) return entry; } -/** - * ice_reduce_msix_usage - Reduce usage of MSI-X vectors - * @pf: board private structure - * @v_remain: number of remaining MSI-X vectors to be distributed - * - * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled. - * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of - * remaining vectors. - */ -static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain) -{ - int v_rdma; - - if (!ice_is_rdma_ena(pf)) { - pf->num_lan_msix = v_remain; - return; - } - - /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */ - v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1; - - if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) { - dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n"); - clear_bit(ICE_FLAG_RDMA_ENA, pf->flags); - - pf->num_rdma_msix = 0; - pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX; - } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) || - (v_remain - v_rdma < v_rdma)) { - /* Support minimum RDMA and give remaining vectors to LAN MSIX - */ - pf->num_rdma_msix = ICE_MIN_RDMA_MSIX; - pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX; - } else { - /* Split remaining MSIX with RDMA after accounting for AEQ MSIX - */ - pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 + - ICE_RDMA_NUM_AEQ_MSIX; - pf->num_lan_msix = v_remain - pf->num_rdma_msix; - } -} - -/** - * ice_ena_msix_range - Request a range of MSIX vectors from the OS - * @pf: board private structure - * - * Compute the number of MSIX vectors wanted and request from the OS. Adjust - * device usage if there are not enough vectors. Return the number of vectors - * reserved or negative on failure. - */ -static int ice_ena_msix_range(struct ice_pf *pf) -{ - int num_cpus, hw_num_msix, v_other, v_wanted, v_actual; - struct device *dev = ice_pf_to_dev(pf); - int err; - - hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors; - num_cpus = num_online_cpus(); - - /* LAN miscellaneous handler */ - v_other = ICE_MIN_LAN_OICR_MSIX; - - /* Flow Director */ - if (test_bit(ICE_FLAG_FD_ENA, pf->flags)) - v_other += ICE_FDIR_MSIX; - - /* switchdev */ - v_other += ICE_ESWITCH_MSIX; - - v_wanted = v_other; - - /* LAN traffic */ - pf->num_lan_msix = num_cpus; - v_wanted += pf->num_lan_msix; - - /* RDMA auxiliary driver */ - if (ice_is_rdma_ena(pf)) { - pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX; - v_wanted += pf->num_rdma_msix; - } - - if (v_wanted > hw_num_msix) { - int v_remain; - - dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n", - v_wanted, hw_num_msix); - - if (hw_num_msix < ICE_MIN_MSIX) { - err = -ERANGE; - goto exit_err; - } - - v_remain = hw_num_msix - v_other; - if (v_remain < ICE_MIN_LAN_TXRX_MSIX) { - v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX; - v_remain = ICE_MIN_LAN_TXRX_MSIX; - } - - ice_reduce_msix_usage(pf, v_remain); - v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other; - - dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n", - pf->num_lan_msix); - if (ice_is_rdma_ena(pf)) - dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n", - pf->num_rdma_msix); - } - - /* actually reserve the vectors */ - v_actual = pci_alloc_irq_vectors(pf->pdev, ICE_MIN_MSIX, v_wanted, - PCI_IRQ_MSIX); - if (v_actual < 0) { - dev_err(dev, "unable to reserve MSI-X vectors\n"); - err = v_actual; - goto exit_err; - } - - if (v_actual < v_wanted) { - dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n", - v_wanted, v_actual); - - if (v_actual < ICE_MIN_MSIX) { - /* error if we can't get minimum vectors */ - pci_free_irq_vectors(pf->pdev); - err = -ERANGE; - goto exit_err; - } else { - int v_remain = v_actual - v_other; - - if (v_remain < ICE_MIN_LAN_TXRX_MSIX) - v_remain = ICE_MIN_LAN_TXRX_MSIX; - - ice_reduce_msix_usage(pf, v_remain); - - dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n", - pf->num_lan_msix); - - if (ice_is_rdma_ena(pf)) - dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n", - pf->num_rdma_msix); - } - } - - return v_actual; - -exit_err: - pf->num_rdma_msix = 0; - pf->num_lan_msix = 0; - return err; -} - /** * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme * @pf: board private structure @@ -266,15 +115,18 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) &value); pf->msix.max = err ? total_vectors / 2 : value.vu16; - vectors = ice_ena_msix_range(pf); - - if (vectors < 0) - return -ENOMEM; - - if (pci_msix_can_alloc_dyn(pf->pdev)) + if (pci_msix_can_alloc_dyn(pf->pdev)) { + vectors = pf->msix.min; max_vectors = total_vectors; - else + } else { + vectors = pf->msix.max; max_vectors = vectors; + } + + vectors = pci_alloc_irq_vectors(pf->pdev, pf->msix.min, vectors, + PCI_IRQ_MSIX); + if (vectors < pf->msix.min) + return -ENOMEM; ice_init_irq_tracker(pf, max_vectors, vectors); From patchwork Thu Aug 8 07:20:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970406 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=5Qasaxop; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdjx3Z7nz1ydk for ; Thu, 8 Aug 2024 17:20:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B20038144E; Thu, 8 Aug 2024 07:20:33 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id RelQCSjUBK4a; Thu, 8 Aug 2024 07:20:32 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8998581491 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101632; bh=XZwKG2vfqYr92GJuipYUMh4qQfSYI74c/Tsj65PcdLE=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=5Qasaxop61Q0HMc7kt+l6Nu7ITCOne7r+xb5Cu2CHdHr4wot/8DU3oUJqs+HLka3a tUVOshjzENjTucb2mkj3cDP6S2A0x6vd/wXO8sdLAAVS1uXgyRp2qZcNJ8ZMbe0Yz7 qQwt5YmoC8k2RabyaPaaPBQWb5j5m8OSSuuWHAs2biD6GsrXMtchnP+PAp8r37wMjC 60Iy/sqMDHT8OiJFjf1UikXSFDPak7h0lMnIIRwYYS1EaqYt04E9CJBienpRPflItg TP3pxLgMfJCtZQc+btjsgOcdkMG5BYEW2er4KJQC3bnLyZiO7Zv6iWtjy0y9PwDNsy B5jfBMZ2QaXvQ== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 8998581491; Thu, 8 Aug 2024 07:20:32 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id CDD761BF369 for ; Thu, 8 Aug 2024 07:20:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BBE34607F3 for ; Thu, 8 Aug 2024 07:20:29 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id QrnrR0eVuI3T for ; Thu, 8 Aug 2024 07:20:28 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org A4F0760772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A4F0760772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id A4F0760772 for ; Thu, 8 Aug 2024 07:20:28 +0000 (UTC) X-CSE-ConnectionGUID: K8y7577YT3aa1UlW/JP8qg== X-CSE-MsgGUID: 34N9tCECR+izsBY+VRr8qQ== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361352" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361352" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:28 -0700 X-CSE-ConnectionGUID: 9VLFfEYjRuOeMCZIvIZYPQ== X-CSE-MsgGUID: ghml+QrhQ+SCl2XTR2f6Kw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073478" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:25 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:11 +0200 Message-ID: <20240808072016.10321-4-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101629; x=1754637629; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=e7m24AGRhdmcoya9lNTF3zRk0X+iOHiyA6NoDCvS1Ck=; b=YmGYTmfC45TLTdNvYpvdXGHDzWzAN6gKNS9w6MDLq1F8j6aHh9kw8dBF V22GVnRvDxHeGV5M9kHWdyybc4JGDQp7o0jX+DvQf8ktAAF/059DUbenG 1dAQBlG1z6Ts1/C83gUDbCQNXT4i9HSEjSmrMCRymJPSNOM84NQKvIS2j wPpKye+IKsJ+cp4FCezRd9G5OfbuKNCN+FVdL+zRtcGA9eU3tHmKVrGxf p/+CC1qTS0OJl5dWSG2FXmx9b662L+ByDff60zs7oD1cHoWezbP2GfvsU +tW135SGliv0hSTfur1BARrxsHfJMkezSv3TB9gB/5AdodeMVDJvajOnc Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=YmGYTmfC Subject: [Intel-wired-lan] [iwl-next v3 3/8] ice: get rid of num_lan_msix field X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Remove the field to allow having more queues than MSI-X on VSI. As default the number will be the same, but if there won't be more MSI-X available VSI can run with at least one MSI-X. Reviewed-by: Wojciech Drewek Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice.h | 1 - drivers/net/ethernet/intel/ice/ice_base.c | 10 +++----- drivers/net/ethernet/intel/ice/ice_ethtool.c | 8 +++--- drivers/net/ethernet/intel/ice/ice_irq.c | 11 +++------ drivers/net/ethernet/intel/ice/ice_lib.c | 26 +++++++++++--------- 5 files changed, 27 insertions(+), 29 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index a67456057c77..eecf171bd0b6 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -623,7 +623,6 @@ struct ice_pf { u16 max_pf_txqs; /* Total Tx queues PF wide */ u16 max_pf_rxqs; /* Total Rx queues PF wide */ struct ice_pf_msix msix; - u16 num_lan_msix; /* Total MSIX vectors for base driver */ u16 num_lan_tx; /* num LAN Tx queues setup */ u16 num_lan_rx; /* num LAN Rx queues setup */ u16 next_vsi; /* Next free slot in pf->vsi[] - 0-based! */ diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c index 1f4792da72da..c8165915c685 100644 --- a/drivers/net/ethernet/intel/ice/ice_base.c +++ b/drivers/net/ethernet/intel/ice/ice_base.c @@ -774,13 +774,11 @@ int ice_vsi_alloc_q_vectors(struct ice_vsi *vsi) return 0; err_out: - while (v_idx--) - ice_free_q_vector(vsi, v_idx); - dev_err(dev, "Failed to allocate %d q_vector for VSI %d, ret=%d\n", - vsi->num_q_vectors, vsi->vsi_num, err); - vsi->num_q_vectors = 0; - return err; + dev_info(dev, "Failed to allocate %d q_vectors for VSI %d, new value %d", + vsi->num_q_vectors, vsi->vsi_num, v_idx); + vsi->num_q_vectors = v_idx; + return v_idx ? 0 : err; } /** diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c index b877002d549b..1a0876209c3f 100644 --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c @@ -3813,8 +3813,8 @@ ice_get_ts_info(struct net_device *dev, struct kernel_ethtool_ts_info *info) */ static int ice_get_max_txq(struct ice_pf *pf) { - return min3(pf->num_lan_msix, (u16)num_online_cpus(), - (u16)pf->hw.func_caps.common_cap.num_txq); + return min_t(u16, num_online_cpus(), + pf->hw.func_caps.common_cap.num_txq); } /** @@ -3823,8 +3823,8 @@ static int ice_get_max_txq(struct ice_pf *pf) */ static int ice_get_max_rxq(struct ice_pf *pf) { - return min3(pf->num_lan_msix, (u16)num_online_cpus(), - (u16)pf->hw.func_caps.common_cap.num_rxq); + return min_t(u16, num_online_cpus(), + pf->hw.func_caps.common_cap.num_rxq); } /** diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c index 4579df21701b..99b82662434b 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.c +++ b/drivers/net/ethernet/intel/ice/ice_irq.c @@ -102,7 +102,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) { int total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors; union devlink_param_value value; - int vectors, max_vectors, err; + int vectors, err; /* load default PF MSI-X range */ err = devl_param_driverinit_value_get(priv_to_devlink(pf), @@ -115,20 +115,17 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) &value); pf->msix.max = err ? total_vectors / 2 : value.vu16; - if (pci_msix_can_alloc_dyn(pf->pdev)) { + if (pci_msix_can_alloc_dyn(pf->pdev)) vectors = pf->msix.min; - max_vectors = total_vectors; - } else { + else vectors = pf->msix.max; - max_vectors = vectors; - } vectors = pci_alloc_irq_vectors(pf->pdev, pf->msix.min, vectors, PCI_IRQ_MSIX); if (vectors < pf->msix.min) return -ENOMEM; - ice_init_irq_tracker(pf, max_vectors, vectors); + ice_init_irq_tracker(pf, pf->msix.max, vectors); return 0; } diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index d9c34d28c9cd..5fb33a215242 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -157,6 +157,16 @@ static void ice_vsi_set_num_desc(struct ice_vsi *vsi) } } +static u16 ice_get_rxq_count(struct ice_pf *pf) +{ + return min_t(u16, ice_get_avail_rxq_count(pf), num_online_cpus()); +} + +static u16 ice_get_txq_count(struct ice_pf *pf) +{ + return min_t(u16, ice_get_avail_txq_count(pf), num_online_cpus()); +} + /** * ice_vsi_set_num_qs - Set number of queues, descriptors and vectors for a VSI * @vsi: the VSI being configured @@ -178,9 +188,7 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi) vsi->alloc_txq = vsi->req_txq; vsi->num_txq = vsi->req_txq; } else { - vsi->alloc_txq = min3(pf->num_lan_msix, - ice_get_avail_txq_count(pf), - (u16)num_online_cpus()); + vsi->alloc_txq = ice_get_txq_count(pf); } pf->num_lan_tx = vsi->alloc_txq; @@ -193,17 +201,14 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi) vsi->alloc_rxq = vsi->req_rxq; vsi->num_rxq = vsi->req_rxq; } else { - vsi->alloc_rxq = min3(pf->num_lan_msix, - ice_get_avail_rxq_count(pf), - (u16)num_online_cpus()); + vsi->alloc_rxq = ice_get_rxq_count(pf); } } pf->num_lan_rx = vsi->alloc_rxq; - vsi->num_q_vectors = min_t(int, pf->num_lan_msix, - max_t(int, vsi->alloc_rxq, - vsi->alloc_txq)); + vsi->num_q_vectors = max_t(int, vsi->alloc_rxq, + vsi->alloc_txq); break; case ICE_VSI_SF: vsi->alloc_txq = 1; @@ -1173,12 +1178,11 @@ static void ice_set_rss_vsi_ctx(struct ice_vsi_ctx *ctxt, struct ice_vsi *vsi) static void ice_chnl_vsi_setup_q_map(struct ice_vsi *vsi, struct ice_vsi_ctx *ctxt) { - struct ice_pf *pf = vsi->back; u16 qcount, qmap; u8 offset = 0; int pow; - qcount = min_t(int, vsi->num_rxq, pf->num_lan_msix); + qcount = vsi->num_rxq; pow = order_base_2(qcount); qmap = FIELD_PREP(ICE_AQ_VSI_TC_Q_OFFSET_M, offset); From patchwork Thu Aug 8 07:20:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970407 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=hm6d68sS; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdjz4P6pz1ydk for ; Thu, 8 Aug 2024 17:20:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 06B36817B5; Thu, 8 Aug 2024 07:20:36 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 3BoxVvVqSShf; Thu, 8 Aug 2024 07:20:35 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 194A280C2E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101635; bh=8yrPzc+5xQugS8oueX1Cxu64XkUERctkvzq0+oYmUxY=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=hm6d68sS6wQ88XcV6phZwZQa14EDhbSfslyAxo+xzWgFjST2osdsQUaO8YVNw6wO1 phgECupz6WLhK4smyyqVwuIAJEsBUcWj3LQzhB0NC+GJbol3qXLTh2BY4eQosfCyTt 0ukb5W7G8b7KyHNm/elTU/LKb7Fol/X3j4tcqA3uj6gTuAZrdHKRtY45II3PhZH1wr 8VWRELBK6mFRVeFQynIa4vlXrQnBaXXuJoG0Zav0wdyf5SWobBOXn8fj5/wHavlHlN 0U8QTQA+N5WZLhdKDqU3wL66dB0/YtP5fdY60ca8eWcWm7fK3+mjhTwJ/2WYzO8U1r HiAEm+sQKpUTw== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 194A280C2E; Thu, 8 Aug 2024 07:20:35 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 3541C1BF369 for ; Thu, 8 Aug 2024 07:20:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 234CB607F3 for ; Thu, 8 Aug 2024 07:20:33 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id RJkzkBMLH4fS for ; Thu, 8 Aug 2024 07:20:32 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 174E160772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 174E160772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id 174E160772 for ; Thu, 8 Aug 2024 07:20:32 +0000 (UTC) X-CSE-ConnectionGUID: /6DpuE4AReqMQHe+vlyuvg== X-CSE-MsgGUID: s8Sc1I63RPSN8egfhcikHA== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361361" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361361" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:32 -0700 X-CSE-ConnectionGUID: RSvGtUeOSeaC72+vAY9kqw== X-CSE-MsgGUID: NvXsgF4fQp2tT0zO0eEtqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073488" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:28 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:12 +0200 Message-ID: <20240808072016.10321-5-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101632; x=1754637632; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y2mAZun2kjCClkZTVShWLtw9ohP/asJRYt0zl7PSfJk=; b=I9h/4wYplNVy07E9Dre+9nCcr1Q5wd5w0GbIO7ltXIC20nA0xuDCgIcF MgEgRO9UTaXlwydmjqaR0ziBmRpGExPe345BbvaFUTddwqDlH7zF6+bkB JZTJ+RUUJa5fhnpk9Q7lWJiuEV2FE3uFPEkyYiK/tWF6lOkH2Crzwzrap uJr1a9IMBRXNLwn+fi0bX5kPb394Q/kLnssetg6hrn1+OFUxXjCNOIPPi te4yzQYQLFDS2dZ5NeE3r9jlpp8H1+JHtRv23YYnpK9Bf/fZR5zlf+kun D5iDMi9djiRwDdq806ti2mURmfFycCLDmGsWbqWWhrzid+zDOSBMTqQ/E g==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=I9h/4wYp Subject: [Intel-wired-lan] [iwl-next v3 4/8] ice, irdma: move interrupts code to irdma X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Move responsibility of MSI-X requesting for RDMA feature from ice driver to irdma driver. It is done to allow simple fallback when there is not enough MSI-X available. Change amount of MSI-X used for control from 4 to 1, as it isn't needed to have more than one MSI-X for this purpose. Signed-off-by: Michal Swiatkowski --- drivers/infiniband/hw/irdma/hw.c | 2 - drivers/infiniband/hw/irdma/main.c | 46 ++++++++++++++++- drivers/infiniband/hw/irdma/main.h | 3 ++ drivers/net/ethernet/intel/ice/ice.h | 2 - drivers/net/ethernet/intel/ice/ice_idc.c | 64 ++++++------------------ include/linux/net/intel/iidc.h | 2 + 6 files changed, 63 insertions(+), 56 deletions(-) diff --git a/drivers/infiniband/hw/irdma/hw.c b/drivers/infiniband/hw/irdma/hw.c index ad50b77282f8..69ce1862eabe 100644 --- a/drivers/infiniband/hw/irdma/hw.c +++ b/drivers/infiniband/hw/irdma/hw.c @@ -498,8 +498,6 @@ static int irdma_save_msix_info(struct irdma_pci_f *rf) iw_qvlist->num_vectors = rf->msix_count; if (rf->msix_count <= num_online_cpus()) rf->msix_shared = true; - else if (rf->msix_count > num_online_cpus() + 1) - rf->msix_count = num_online_cpus() + 1; pmsix = rf->msix_entries; for (i = 0, ceq_idx = 0; i < rf->msix_count; i++, iw_qvinfo++) { diff --git a/drivers/infiniband/hw/irdma/main.c b/drivers/infiniband/hw/irdma/main.c index 3f13200ff71b..1ee8969595d3 100644 --- a/drivers/infiniband/hw/irdma/main.c +++ b/drivers/infiniband/hw/irdma/main.c @@ -206,6 +206,43 @@ static void irdma_lan_unregister_qset(struct irdma_sc_vsi *vsi, ibdev_dbg(&iwdev->ibdev, "WS: LAN free_res for rdma qset failed.\n"); } +static int irdma_init_interrupts(struct irdma_pci_f *rf, struct ice_pf *pf) +{ + int i; + + rf->msix_count = num_online_cpus() + IRDMA_NUM_AEQ_MSIX; + rf->msix_entries = kcalloc(rf->msix_count, sizeof(*rf->msix_entries), + GFP_KERNEL); + if (!rf->msix_entries) + return -ENOMEM; + + for (i = 0; i < rf->msix_count; i++) + if (ice_alloc_rdma_qvector(pf, &rf->msix_entries[i])) + break; + + if (i < IRDMA_MIN_MSIX) { + for (; i > 0; i--) + ice_free_rdma_qvector(pf, &rf->msix_entries[i]); + + kfree(rf->msix_entries); + return -ENOMEM; + } + + rf->msix_count = i; + + return 0; +} + +static void irdma_deinit_interrupts(struct irdma_pci_f *rf, struct ice_pf *pf) +{ + int i; + + for (i = 0; i < rf->msix_count; i++) + ice_free_rdma_qvector(pf, &rf->msix_entries[i]); + + kfree(rf->msix_entries); +} + static void irdma_remove(struct auxiliary_device *aux_dev) { struct iidc_auxiliary_dev *iidc_adev = container_of(aux_dev, @@ -216,6 +253,7 @@ static void irdma_remove(struct auxiliary_device *aux_dev) irdma_ib_unregister_device(iwdev); ice_rdma_update_vsi_filter(pf, iwdev->vsi_num, false); + irdma_deinit_interrupts(iwdev->rf, pf); pr_debug("INIT: Gen2 PF[%d] device remove success\n", PCI_FUNC(pf->pdev->devfn)); } @@ -230,9 +268,7 @@ static void irdma_fill_device_info(struct irdma_device *iwdev, struct ice_pf *pf rf->gen_ops.unregister_qset = irdma_lan_unregister_qset; rf->hw.hw_addr = pf->hw.hw_addr; rf->pcidev = pf->pdev; - rf->msix_count = pf->num_rdma_msix; rf->pf_id = pf->hw.pf_id; - rf->msix_entries = &pf->msix_entries[pf->rdma_base_vector]; rf->default_vsi.vsi_idx = vsi->vsi_num; rf->protocol_used = pf->rdma_mode & IIDC_RDMA_PROTOCOL_ROCEV2 ? IRDMA_ROCE_PROTOCOL_ONLY : IRDMA_IWARP_PROTOCOL_ONLY; @@ -281,6 +317,10 @@ static int irdma_probe(struct auxiliary_device *aux_dev, const struct auxiliary_ irdma_fill_device_info(iwdev, pf, vsi); rf = iwdev->rf; + err = irdma_init_interrupts(rf, pf); + if (err) + goto err_init_interrupts; + err = irdma_ctrl_init_hw(rf); if (err) goto err_ctrl_init; @@ -311,6 +351,8 @@ static int irdma_probe(struct auxiliary_device *aux_dev, const struct auxiliary_ err_rt_init: irdma_ctrl_deinit_hw(rf); err_ctrl_init: + irdma_deinit_interrupts(rf, pf); +err_init_interrupts: kfree(iwdev->rf); ib_dealloc_device(&iwdev->ibdev); diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h index 9f0ed6e84471..ef9a9b79d711 100644 --- a/drivers/infiniband/hw/irdma/main.h +++ b/drivers/infiniband/hw/irdma/main.h @@ -117,6 +117,9 @@ extern struct auxiliary_driver i40iw_auxiliary_drv; #define IRDMA_IRQ_NAME_STR_LEN (64) +#define IRDMA_NUM_AEQ_MSIX 1 +#define IRDMA_MIN_MSIX 2 + enum init_completion_state { INVALID_STATE = 0, INITIAL_STATE, diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index eecf171bd0b6..b4ecf71d702b 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -97,8 +97,6 @@ #define ICE_MIN_MSIX (ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_LAN_OICR_MSIX) #define ICE_MAX_MSIX 256 #define ICE_FDIR_MSIX 2 -#define ICE_RDMA_NUM_AEQ_MSIX 4 -#define ICE_MIN_RDMA_MSIX 2 #define ICE_ESWITCH_MSIX 1 #define ICE_NO_VSI 0xffff #define ICE_VSI_MAP_CONTIG 0 diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c index 145b27f2a4ce..bab3e81cad5d 100644 --- a/drivers/net/ethernet/intel/ice/ice_idc.c +++ b/drivers/net/ethernet/intel/ice/ice_idc.c @@ -228,61 +228,34 @@ void ice_get_qos_params(struct ice_pf *pf, struct iidc_qos_params *qos) } EXPORT_SYMBOL_GPL(ice_get_qos_params); -/** - * ice_alloc_rdma_qvectors - Allocate vector resources for RDMA driver - * @pf: board private structure to initialize - */ -static int ice_alloc_rdma_qvectors(struct ice_pf *pf) +int ice_alloc_rdma_qvector(struct ice_pf *pf, struct msix_entry *entry) { - if (ice_is_rdma_ena(pf)) { - int i; - - pf->msix_entries = kcalloc(pf->num_rdma_msix, - sizeof(*pf->msix_entries), - GFP_KERNEL); - if (!pf->msix_entries) - return -ENOMEM; + struct msi_map map = ice_alloc_irq(pf, true); - /* RDMA is the only user of pf->msix_entries array */ - pf->rdma_base_vector = 0; - - for (i = 0; i < pf->num_rdma_msix; i++) { - struct msix_entry *entry = &pf->msix_entries[i]; - struct msi_map map; + if (map.index < 0) + return -ENOMEM; - map = ice_alloc_irq(pf, false); - if (map.index < 0) - break; + entry->entry = map.index; + entry->vector = map.virq; - entry->entry = map.index; - entry->vector = map.virq; - } - } return 0; } +EXPORT_SYMBOL_GPL(ice_alloc_rdma_qvector); /** * ice_free_rdma_qvector - free vector resources reserved for RDMA driver * @pf: board private structure to initialize + * @entry: MSI-X entry to be removed */ -static void ice_free_rdma_qvector(struct ice_pf *pf) +void ice_free_rdma_qvector(struct ice_pf *pf, struct msix_entry *entry) { - int i; - - if (!pf->msix_entries) - return; - - for (i = 0; i < pf->num_rdma_msix; i++) { - struct msi_map map; + struct msi_map map; - map.index = pf->msix_entries[i].entry; - map.virq = pf->msix_entries[i].vector; - ice_free_irq(pf, map); - } - - kfree(pf->msix_entries); - pf->msix_entries = NULL; + map.index = entry->entry; + map.virq = entry->vector; + ice_free_irq(pf, map); } +EXPORT_SYMBOL_GPL(ice_free_rdma_qvector); /** * ice_adev_release - function to be mapped to AUX dev's release op @@ -382,12 +355,6 @@ int ice_init_rdma(struct ice_pf *pf) return -ENOMEM; } - /* Reserve vector resources */ - ret = ice_alloc_rdma_qvectors(pf); - if (ret < 0) { - dev_err(dev, "failed to reserve vectors for RDMA\n"); - goto err_reserve_rdma_qvector; - } pf->rdma_mode |= IIDC_RDMA_PROTOCOL_ROCEV2; ret = ice_plug_aux_dev(pf); if (ret) @@ -395,8 +362,6 @@ int ice_init_rdma(struct ice_pf *pf) return 0; err_plug_aux_dev: - ice_free_rdma_qvector(pf); -err_reserve_rdma_qvector: pf->adev = NULL; xa_erase(&ice_aux_id, pf->aux_idx); return ret; @@ -412,6 +377,5 @@ void ice_deinit_rdma(struct ice_pf *pf) return; ice_unplug_aux_dev(pf); - ice_free_rdma_qvector(pf); xa_erase(&ice_aux_id, pf->aux_idx); } diff --git a/include/linux/net/intel/iidc.h b/include/linux/net/intel/iidc.h index 1c1332e4df26..13274c3def66 100644 --- a/include/linux/net/intel/iidc.h +++ b/include/linux/net/intel/iidc.h @@ -78,6 +78,8 @@ int ice_del_rdma_qset(struct ice_pf *pf, struct iidc_rdma_qset_params *qset); int ice_rdma_request_reset(struct ice_pf *pf, enum iidc_reset_type reset_type); int ice_rdma_update_vsi_filter(struct ice_pf *pf, u16 vsi_id, bool enable); void ice_get_qos_params(struct ice_pf *pf, struct iidc_qos_params *qos); +int ice_alloc_rdma_qvector(struct ice_pf *pf, struct msix_entry *entry); +void ice_free_rdma_qvector(struct ice_pf *pf, struct msix_entry *entry); /* Structure representing auxiliary driver tailored information about the core * PCI dev, each auxiliary driver using the IIDC interface will have an From patchwork Thu Aug 8 07:20:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970408 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=rlcI/qo1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdk248DMz1ydk for ; Thu, 8 Aug 2024 17:20:42 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0E3B581491; Thu, 8 Aug 2024 07:20:39 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Ncn85Aanbo85; Thu, 8 Aug 2024 07:20:37 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 455F180C24 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101637; bh=XOVSJmLIn7G2VVkuXAKd7W1EdB1lj3Can+wko35kiRM=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=rlcI/qo1a0E/+UJOAWsTL8x8op6fjyH7Smnai+m4ylXhdOOsz7DajVwJklD+9ZRmn 7N438neHcEMObTIQUOWvz9hndsYhSDwLJ6eVRlj0HAhmZocH8K6ixtOrn92Vtwk+No gPv5VxyZrO0Dv+X9RER3xKgirYJJ9cIstcnR1EYSz+F42sZ4LhQOzktdcKftGCeYwS 7qpcotrbEf+e7M+YxbE2cEWgjzvGZ7dnsPubRtZCfzAe8EJyZQUEpUneAxLRjQAGCe 1QsGsOGxr39PC2uTiNI31CENyYFAdy4+cR3wVWF2NT7MvRuPkrckEP0cen2aOvM0ex TO7vK/93KIeJQ== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 455F180C24; Thu, 8 Aug 2024 07:20:37 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 8D95E1BF369 for ; Thu, 8 Aug 2024 07:20:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8586F607F3 for ; Thu, 8 Aug 2024 07:20:35 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 94zLGravC6Dj for ; Thu, 8 Aug 2024 07:20:34 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 97DE760772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 97DE760772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id 97DE760772 for ; Thu, 8 Aug 2024 07:20:34 +0000 (UTC) X-CSE-ConnectionGUID: p4EwS6IJQTSRy8hnJGmG1w== X-CSE-MsgGUID: T4P+XoyFQT2VnekwnEfnLg== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361365" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361365" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:34 -0700 X-CSE-ConnectionGUID: OIhqJ5ESSTigx1bdzFhe5w== X-CSE-MsgGUID: aXPMeFdiSSqcvkpKsySDxw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073500" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:31 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:13 +0200 Message-ID: <20240808072016.10321-6-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101634; x=1754637634; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xs4ffIfWO6q7Ek0DJDpYk7H+AE3ciC+EIAZDVbZ2zDg=; b=Ell4XDoAYI8ZdrgEfZeQ2ia1UrdZDIa7PlPYQBr16ZYglSHBLvwDPV6K H0gA+cCam3i3EvwcujbvEClcUlIgPA6Qzq39SzOEDE/o1hQ3ET1QuSusY 8AsCNPdCR3buvjPgMXE9ZszingBpdq8xQ8X1ZjyHu0mzTdyQBClLnOGaC Hb6x9ewJbGARKmzpcqplccJnyNeb6u3DM9wqWEuQmhbr8xl09ea0VYfTD TlbWxQDA15h6YqKwih5vmDIHii5KzxDCRmOlrBaYdXGd52nGaDhdEiFfj ZIkGBBk2Q+olY1We4uJWqt3wXfSCqCQImfBHbXmEYQeM1E5H2BfuihqfG Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Ell4XDoA Subject: [Intel-wired-lan] [iwl-next v3 5/8] ice: treat dyn_allowed only as suggestion X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" It can be needed to have some MSI-X allocated as static and rest as dynamic. For example on PF VSI. We want to always have minimum one MSI-X on it, because of that it is allocated as a static one, rest can be dynamic if it is supported. Change the ice_get_irq_res() to allow using static entries if they are free even if caller wants dynamic one. Adjust limit values to the new approach. Min and max in limit means the values that are valid, so decrease max and num_static by one. Set vsi::irq_dyn_alloc if dynamic allocation is supported. Reviewed-by: Wojciech Drewek Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_irq.c | 25 ++++++++++++------------ drivers/net/ethernet/intel/ice/ice_lib.c | 2 ++ 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c index 99b82662434b..10caacaae804 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.c +++ b/drivers/net/ethernet/intel/ice/ice_irq.c @@ -45,7 +45,7 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index) /** * ice_get_irq_res - get an interrupt resource * @pf: board private structure - * @dyn_only: force entry to be dynamically allocated + * @dyn_allowed: allow entry to be dynamically allocated * * Allocate new irq entry in the free slot of the tracker. Since xarray * is used, always allocate new entry at the lowest possible index. Set @@ -53,11 +53,12 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index) * * Returns allocated irq entry or NULL on failure. */ -static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only) +static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, + bool dyn_allowed) { - struct xa_limit limit = { .max = pf->irq_tracker.num_entries, + struct xa_limit limit = { .max = pf->irq_tracker.num_entries - 1, .min = 0 }; - unsigned int num_static = pf->irq_tracker.num_static; + unsigned int num_static = pf->irq_tracker.num_static - 1; struct ice_irq_entry *entry; unsigned int index; int ret; @@ -66,9 +67,9 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only) if (!entry) return NULL; - /* skip preallocated entries if the caller says so */ - if (dyn_only) - limit.min = num_static; + /* only already allocated if the caller says so */ + if (!dyn_allowed) + limit.max = num_static; ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit, GFP_KERNEL); @@ -78,7 +79,7 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only) entry = NULL; } else { entry->index = index; - entry->dynamic = index >= num_static; + entry->dynamic = index > num_static; } return entry; @@ -133,7 +134,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) /** * ice_alloc_irq - Allocate new interrupt vector * @pf: board private structure - * @dyn_only: force dynamic allocation of the interrupt + * @dyn_allowed: allow dynamic allocation of the interrupt * * Allocate new interrupt vector for a given owner id. * return struct msi_map with interrupt details and track @@ -146,20 +147,20 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) * interrupt will be allocated with pci_msix_alloc_irq_at. * * Some callers may only support dynamically allocated interrupts. - * This is indicated with dyn_only flag. + * This is indicated with dyn_allowed flag. * * On failure, return map with negative .index. The caller * is expected to check returned map index. * */ -struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only) +struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_allowed) { int sriov_base_vector = pf->sriov_base_vector; struct msi_map map = { .index = -ENOENT }; struct device *dev = ice_pf_to_dev(pf); struct ice_irq_entry *entry; - entry = ice_get_irq_res(pf, dyn_only); + entry = ice_get_irq_res(pf, dyn_allowed); if (!entry) return map; diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 5fb33a215242..fcde08334880 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -572,6 +572,8 @@ ice_vsi_alloc_def(struct ice_vsi *vsi, struct ice_channel *ch) return -ENOMEM; } + vsi->irq_dyn_alloc = pci_msix_can_alloc_dyn(vsi->back->pdev); + switch (vsi->type) { case ICE_VSI_PF: case ICE_VSI_SF: From patchwork Thu Aug 8 07:20:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970409 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=wC3Ozaub; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdk43GQJz1ydk for ; Thu, 8 Aug 2024 17:20:44 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D70ED815AD; Thu, 8 Aug 2024 07:20:40 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id r3EDbe57PWQH; Thu, 8 Aug 2024 07:20:39 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 688FB80C53 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101639; bh=IJZHzakzHJ8Bo3VBgC0WZWgDQX8L0cUZUABA5hJZMe8=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=wC3Ozaubx18wCqDrjfIxgWRR02nQE6rLiHrX2dkXd1tDcSP5F3GMsSgFuFGhMjVqj n5TrkYzHVs2mzRPK6ZC/WBEXHmWI42Mdje8CW0IpEqzOIDETBNchj68Za3eYdpX0yq EXwdIouLn+Evht/S13KAsDK/aWVu2xK068jjSqdauRnpycYeGs7lg4u3ugMIlxWJST ZwrZi0S+0qIvXxwp6Y9g6mik2DN1OWCTHgGPHjXjObZNyvo8mG2Tdr/FHBQ7Sk0yJh tQ2vjp+aHcXOk4BpttN3hlbxi3Qu7zVLmVQLoLCPzo1b2M13pt18iguJrdFNKYcQfc DBBEt/Q2uRCyA== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 688FB80C53; Thu, 8 Aug 2024 07:20:39 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 665FE1BF369 for ; Thu, 8 Aug 2024 07:20:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 5E775607F3 for ; Thu, 8 Aug 2024 07:20:38 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id tPxftLv2pbkE for ; Thu, 8 Aug 2024 07:20:37 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 8153B60772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8153B60772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id 8153B60772 for ; Thu, 8 Aug 2024 07:20:37 +0000 (UTC) X-CSE-ConnectionGUID: 3XJv+d1MRva8h6wg8P+ZQA== X-CSE-MsgGUID: ADOtZ31ITZmk6mZxc19xmw== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361373" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361373" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:37 -0700 X-CSE-ConnectionGUID: 4DV3NfKWTuaYiPKo9QHjfg== X-CSE-MsgGUID: UrMGCU2kQgekRARRK9FvzQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073506" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:34 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:14 +0200 Message-ID: <20240808072016.10321-7-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101637; x=1754637637; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WK8qdsTxDcmqkJblWSy4esjLmghvrfMZPQnaAtP1zd4=; b=VU/tyy5JJBE/cwyKdHKadJaeNcZEqH3cQvMT9CgEVveCsg3bU4KjTjPc VBk6DLrhQybXIm4kPMYLWb1pC2bze5lmuHJRMgv0PBm5JHxsIikTFB0/P wgqc8DmKu6CeZYi+S567PMQAzGFW5quWavp9DIKEronaE9M3T4uwSjhKM RcoMHSyEzylrVf/6JiPxEWMHH5FvGtOTNoMF773LbZ1R0MrkTEtWdiXWS 7deqqPHFi0UeJJvAtHjlXb4Q5HDogCrjVTrbkdhCq6Gj4qeH1AoANtfBm nlPrQMyNsR2vRKIG7r6o+mskw9q0mJk2ZZzLt6CM4HG6FbXgGAJlclXV4 g==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=VU/tyy5J Subject: [Intel-wired-lan] [iwl-next v3 6/8] ice: enable_rdma devlink param X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Implement enable_rdma devlink parameter to allow user to turn RDMA feature on and off. It is useful when there is no enough interrupts and user doesn't need RDMA feature. Reviewed-by: Jan Sokolowski Reviewed-by: Przemek Kitszel Signed-off-by: Michal Swiatkowski --- .../net/ethernet/intel/ice/devlink/devlink.c | 19 +++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_lib.c | 8 +++++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/ice/devlink/devlink.c b/drivers/net/ethernet/intel/ice/devlink/devlink.c index bdc22ea13e0f..530d443a8e14 100644 --- a/drivers/net/ethernet/intel/ice/devlink/devlink.c +++ b/drivers/net/ethernet/intel/ice/devlink/devlink.c @@ -1544,6 +1544,19 @@ ice_devlink_msix_min_pf_validate(struct devlink *devlink, u32 id, return 0; } +static int ice_devlink_enable_rdma_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + struct ice_pf *pf = devlink_priv(devlink); + bool new_state = val.vbool; + + if (new_state && !test_bit(ICE_FLAG_RDMA_ENA, pf->flags)) + return -EOPNOTSUPP; + + return 0; +} + enum ice_param_id { ICE_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX, ICE_DEVLINK_PARAM_ID_TX_SCHED_LAYERS, @@ -1559,6 +1572,8 @@ static const struct devlink_param ice_dvl_rdma_params[] = { ice_devlink_enable_iw_get, ice_devlink_enable_iw_set, ice_devlink_enable_iw_validate), + DEVLINK_PARAM_GENERIC(ENABLE_RDMA, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, ice_devlink_enable_rdma_validate), }; static const struct devlink_param ice_dvl_msix_params[] = { @@ -1700,6 +1715,10 @@ int ice_devlink_register_params(struct ice_pf *pf) devl_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_MSIX_VEC_PER_PF_MIN, value); + value.vbool = test_bit(ICE_FLAG_RDMA_ENA, pf->flags); + devl_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + value); return 0; } diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index fcde08334880..93a7b875f8aa 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -834,7 +834,13 @@ bool ice_is_safe_mode(struct ice_pf *pf) */ bool ice_is_rdma_ena(struct ice_pf *pf) { - return test_bit(ICE_FLAG_RDMA_ENA, pf->flags); + union devlink_param_value value; + int err; + + err = devl_param_driverinit_value_get(priv_to_devlink(pf), + DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + &value); + return err ? test_bit(ICE_FLAG_RDMA_ENA, pf->flags) : value.vbool; } /** From patchwork Thu Aug 8 07:20:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970410 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=u+5Bc/oM; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdk72LYCz1ydk for ; Thu, 8 Aug 2024 17:20:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id AF334817A9; Thu, 8 Aug 2024 07:20:45 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 8lAD6a0jQkc3; Thu, 8 Aug 2024 07:20:43 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4D18B81496 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101643; bh=rY3/asp6QhbjCJj93ZYAxdmEqospVeOw61n6GpIvVmo=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=u+5Bc/oMvNu9Etv3LKiEYWzzX9f7s19JbtA/Gajiu6687GF+YLc8E2eoi2Fb4L4St 5gwyrn1+g7k9b1kAozJU1kmplWrkERGSQ9sZHtjvKhPxjw1n/Oul5djB+EawA483wf K3bLtbOpWDQurHFpv7cLmKOro5qBGpiz9U7esNVS5PlEdBjWrR/Eilw9PWt5tu6aYZ CgwBmlim9La9DJKfFfba/bu8FIW9mehZWoC0jpWQzXqi71QYroQ4l+dyhtLoT8bhaA 2hxrdbwMs14+E1Z5a+F9VZdNZQhv9nKBvgasPhPGfEj2UIfdarEiBmhyxp40WNEy7j KzDAxmIF/nKZA== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 4D18B81496; Thu, 8 Aug 2024 07:20:43 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 3BD251BF369 for ; Thu, 8 Aug 2024 07:20:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 26D4E607F3 for ; Thu, 8 Aug 2024 07:20:42 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id lPM17kcAWve9 for ; Thu, 8 Aug 2024 07:20:40 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org B09B760772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B09B760772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id B09B760772 for ; Thu, 8 Aug 2024 07:20:40 +0000 (UTC) X-CSE-ConnectionGUID: MNoJ8+6qTIGLKvc6qQbzcQ== X-CSE-MsgGUID: LUnB1zDTSq2UNDY2o8G3yg== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361385" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361385" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:40 -0700 X-CSE-ConnectionGUID: xLCa2vZTTEWVqYEJ4NvyUg== X-CSE-MsgGUID: s/KteXOCQaK2DEEOgkgx/A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073512" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:37 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:15 +0200 Message-ID: <20240808072016.10321-8-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101641; x=1754637641; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2iAur2PGepsejhYkr3ZsYF5aT0CyObt1pbXN87ylCTc=; b=ADQocDczewQPUt44t8HeHGPVIsNMEh5YoJVD42GA69S+Gy7Pp+Sq+RF1 1kQrH/JOtdTHMznS1m48c26Hfyk6/doaPpp2fgfBQEFja7UQUAll6JhvG bmNYSuLV+tOFj6IQ3P8w/JFV2oIEv4xPPEijskOD1rrumV66xXoNjcfdi YG9zqkRtTflfVvvuMJvigEEkW0JvkOkbuZtAZSthvIqpA4DA6kSgGIV3V kTSwr4FNTVVEJwTADCDPBweBTT5SGJVuxI+fsmmeDzzZbADwLQP1OReEj Rk8ygB8kD5Mqn4UnHkJgN+h2zTSRw9v2XOSKe0G0twHCzh23i0kBQqw5r A==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=ADQocDcz Subject: [Intel-wired-lan] [iwl-next v3 7/8] ice: simplify VF MSI-X managing X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" After implementing pf->msix.max field, base vector for other use cases (like VFs) can be fixed. This simplify code when changing MSI-X amount on particular VF, because there is no need to move a base vector. A fixed base vector allows to reserve vectors from the beginning instead of from the end, which is also simpler in code. Store total and rest value in the same struct as max and min for PF. Move tracking vectors from ice_sriov.c to ice_irq.c as it can be also use for other none PF use cases (SIOV). Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice.h | 10 +- drivers/net/ethernet/intel/ice/ice_irq.c | 75 +++++++--- drivers/net/ethernet/intel/ice/ice_irq.h | 13 +- drivers/net/ethernet/intel/ice/ice_sriov.c | 153 ++------------------- 4 files changed, 78 insertions(+), 173 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index b4ecf71d702b..8a6c7e350534 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -548,6 +548,8 @@ struct ice_pf_msix { u16 cur; u16 min; u16 max; + u16 total; + u16 rest; }; struct ice_pf { @@ -564,13 +566,7 @@ struct ice_pf { /* OS reserved IRQ details */ struct msix_entry *msix_entries; struct ice_irq_tracker irq_tracker; - /* First MSIX vector used by SR-IOV VFs. Calculated by subtracting the - * number of MSIX vectors needed for all SR-IOV VFs from the number of - * MSIX vectors allowed on this PF. - */ - u16 sriov_base_vector; - unsigned long *sriov_irq_bm; /* bitmap to track irq usage */ - u16 sriov_irq_size; /* size of the irq_bm bitmap */ + struct ice_virt_irq_tracker virt_irq_tracker; u16 ctrl_vsi_idx; /* control VSI index in pf->vsi array */ diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c index 10caacaae804..61977876c943 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.c +++ b/drivers/net/ethernet/intel/ice/ice_irq.c @@ -20,6 +20,19 @@ ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors, xa_init_flags(&pf->irq_tracker.entries, XA_FLAGS_ALLOC); } +static int +ice_init_virt_irq_tracker(struct ice_pf *pf, u16 base, u16 num_entries) +{ + pf->virt_irq_tracker.bm = bitmap_zalloc(num_entries, GFP_KERNEL); + if (!pf->virt_irq_tracker.bm) + return -ENOMEM; + + pf->virt_irq_tracker.num_entries = num_entries; + pf->virt_irq_tracker.base = base; + + return 0; +} + /** * ice_deinit_irq_tracker - free xarray tracker * @pf: board private structure @@ -29,6 +42,11 @@ static void ice_deinit_irq_tracker(struct ice_pf *pf) xa_destroy(&pf->irq_tracker.entries); } +static void ice_deinit_virt_irq_tracker(struct ice_pf *pf) +{ + bitmap_free(pf->virt_irq_tracker.bm); +} + /** * ice_free_irq_res - free a block of resources * @pf: board private structure @@ -93,6 +111,7 @@ void ice_clear_interrupt_scheme(struct ice_pf *pf) { pci_free_irq_vectors(pf->pdev); ice_deinit_irq_tracker(pf); + ice_deinit_virt_irq_tracker(pf); } /** @@ -116,6 +135,9 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) &value); pf->msix.max = err ? total_vectors / 2 : value.vu16; + pf->msix.total = total_vectors; + pf->msix.rest = total_vectors - pf->msix.max; + if (pci_msix_can_alloc_dyn(pf->pdev)) vectors = pf->msix.min; else @@ -128,7 +150,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) ice_init_irq_tracker(pf, pf->msix.max, vectors); - return 0; + return ice_init_virt_irq_tracker(pf, pf->msix.max, pf->msix.rest); } /** @@ -155,7 +177,6 @@ int ice_init_interrupt_scheme(struct ice_pf *pf) */ struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_allowed) { - int sriov_base_vector = pf->sriov_base_vector; struct msi_map map = { .index = -ENOENT }; struct device *dev = ice_pf_to_dev(pf); struct ice_irq_entry *entry; @@ -164,10 +185,6 @@ struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_allowed) if (!entry) return map; - /* fail if we're about to violate SRIOV vectors space */ - if (sriov_base_vector && entry->index >= sriov_base_vector) - goto exit_free_res; - if (pci_msix_can_alloc_dyn(pf->pdev) && entry->dynamic) { map = pci_msix_alloc_irq_at(pf->pdev, entry->index, NULL); if (map.index < 0) @@ -215,26 +232,40 @@ void ice_free_irq(struct ice_pf *pf, struct msi_map map) } /** - * ice_get_max_used_msix_vector - Get the max used interrupt vector - * @pf: board private structure + * ice_virt_get_irqs - get irqs for SR-IOV usacase + * @pf: pointer to PF structure + * @needed: number of irqs to get * - * Return index of maximum used interrupt vectors with respect to the - * beginning of the MSIX table. Take into account that some interrupts - * may have been dynamically allocated after MSIX was initially enabled. + * This returns the first MSI-X vector index in PF space that is used by this + * VF. This index is used when accessing PF relative registers such as + * GLINT_VECT2FUNC and GLINT_DYN_CTL. + * This will always be the OICR index in the AVF driver so any functionality + * using vf->first_vector_idx for queue configuration_id: id of VF which will + * use this irqs */ -int ice_get_max_used_msix_vector(struct ice_pf *pf) +int ice_virt_get_irqs(struct ice_pf *pf, u16 needed) { - unsigned long start, index, max_idx; - void *entry; + int res = bitmap_find_next_zero_area(pf->virt_irq_tracker.bm, + pf->virt_irq_tracker.num_entries, + 0, needed, 0); - /* Treat all preallocated interrupts as used */ - start = pf->irq_tracker.num_static; - max_idx = start - 1; + if (res >= pf->virt_irq_tracker.num_entries) + return -ENOENT; - xa_for_each_start(&pf->irq_tracker.entries, index, entry, start) { - if (index > max_idx) - max_idx = index; - } + bitmap_set(pf->virt_irq_tracker.bm, res, needed); - return max_idx; + /* conversion from number in bitmap to global irq index */ + return res + pf->virt_irq_tracker.base; +} + +/** + * ice_virt_free_irqs - free irqs used by the VF + * @pf: pointer to PF structure + * @index: first index to be free + * @irqs: number of irqs to free + */ +void ice_virt_free_irqs(struct ice_pf *pf, u16 index, u16 irqs) +{ + bitmap_clear(pf->virt_irq_tracker.bm, index - pf->virt_irq_tracker.base, + irqs); } diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h index f35efc08575e..d5e0fdd9b535 100644 --- a/drivers/net/ethernet/intel/ice/ice_irq.h +++ b/drivers/net/ethernet/intel/ice/ice_irq.h @@ -15,11 +15,22 @@ struct ice_irq_tracker { u16 num_static; /* preallocated entries */ }; +struct ice_virt_irq_tracker { + unsigned long *bm; /* bitmap to track irq usage */ + u16 num_entries; + /* First MSIX vector used by SR-IOV VFs. Calculated by subtracting the + * number of MSIX vectors needed for all SR-IOV VFs from the number of + * MSIX vectors allowed on this PF. + */ + u16 base; +}; + int ice_init_interrupt_scheme(struct ice_pf *pf); void ice_clear_interrupt_scheme(struct ice_pf *pf); struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only); void ice_free_irq(struct ice_pf *pf, struct msi_map map); -int ice_get_max_used_msix_vector(struct ice_pf *pf); +int ice_virt_get_irqs(struct ice_pf *pf, u16 needed); +void ice_virt_free_irqs(struct ice_pf *pf, u16 index, u16 irqs); #endif diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c index e34fe2516ccc..865421b8ed83 100644 --- a/drivers/net/ethernet/intel/ice/ice_sriov.c +++ b/drivers/net/ethernet/intel/ice/ice_sriov.c @@ -122,27 +122,6 @@ static void ice_dis_vf_mappings(struct ice_vf *vf) dev_err(dev, "Scattered mode for VF Rx queues is not yet implemented\n"); } -/** - * ice_sriov_free_msix_res - Reset/free any used MSIX resources - * @pf: pointer to the PF structure - * - * Since no MSIX entries are taken from the pf->irq_tracker then just clear - * the pf->sriov_base_vector. - * - * Returns 0 on success, and -EINVAL on error. - */ -static int ice_sriov_free_msix_res(struct ice_pf *pf) -{ - if (!pf) - return -EINVAL; - - bitmap_free(pf->sriov_irq_bm); - pf->sriov_irq_size = 0; - pf->sriov_base_vector = 0; - - return 0; -} - /** * ice_free_vfs - Free all VFs * @pf: pointer to the PF structure @@ -177,6 +156,7 @@ void ice_free_vfs(struct ice_pf *pf) ice_eswitch_detach_vf(pf, vf); ice_dis_vf_qs(vf); + ice_virt_free_irqs(pf, vf->first_vector_idx, vf->num_msix); if (test_bit(ICE_VF_STATE_INIT, vf->vf_states)) { /* disable VF qp mappings and set VF disable state */ @@ -199,9 +179,6 @@ void ice_free_vfs(struct ice_pf *pf) mutex_unlock(&vf->cfg_lock); } - if (ice_sriov_free_msix_res(pf)) - dev_err(dev, "Failed to free MSIX resources used by SR-IOV\n"); - vfs->num_qps_per = 0; ice_free_vf_entries(pf); @@ -370,40 +347,6 @@ void ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector) q_vector->reg_idx = vf->first_vector_idx + q_vector->vf_reg_idx; } -/** - * ice_sriov_set_msix_res - Set any used MSIX resources - * @pf: pointer to PF structure - * @num_msix_needed: number of MSIX vectors needed for all SR-IOV VFs - * - * This function allows SR-IOV resources to be taken from the end of the PF's - * allowed HW MSIX vectors so that the irq_tracker will not be affected. We - * just set the pf->sriov_base_vector and return success. - * - * If there are not enough resources available, return an error. This should - * always be caught by ice_set_per_vf_res(). - * - * Return 0 on success, and -EINVAL when there are not enough MSIX vectors - * in the PF's space available for SR-IOV. - */ -static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed) -{ - u16 total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors; - int vectors_used = ice_get_max_used_msix_vector(pf); - int sriov_base_vector; - - sriov_base_vector = total_vectors - num_msix_needed; - - /* make sure we only grab irq_tracker entries from the list end and - * that we have enough available MSIX vectors - */ - if (sriov_base_vector < vectors_used) - return -EINVAL; - - pf->sriov_base_vector = sriov_base_vector; - - return 0; -} - /** * ice_set_per_vf_res - check if vectors and queues are available * @pf: pointer to the PF structure @@ -428,11 +371,9 @@ static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed) */ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs) { - int vectors_used = ice_get_max_used_msix_vector(pf); u16 num_msix_per_vf, num_txq, num_rxq, avail_qs; int msix_avail_per_vf, msix_avail_for_sriov; struct device *dev = ice_pf_to_dev(pf); - int err; lockdep_assert_held(&pf->vfs.table_lock); @@ -440,8 +381,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs) return -EINVAL; /* determine MSI-X resources per VF */ - msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors - - vectors_used; + msix_avail_for_sriov = pf->virt_irq_tracker.num_entries; msix_avail_per_vf = msix_avail_for_sriov / num_vfs; if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) { num_msix_per_vf = ICE_NUM_VF_MSIX_MED; @@ -480,13 +420,6 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs) return -ENOSPC; } - err = ice_sriov_set_msix_res(pf, num_msix_per_vf * num_vfs); - if (err) { - dev_err(dev, "Unable to set MSI-X resources for %d VFs, err %d\n", - num_vfs, err); - return err; - } - /* only allow equal Tx/Rx queue count (i.e. queue pairs) */ pf->vfs.num_qps_per = min_t(int, num_txq, num_rxq); pf->vfs.num_msix_per = num_msix_per_vf; @@ -496,52 +429,6 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs) return 0; } -/** - * ice_sriov_get_irqs - get irqs for SR-IOV usacase - * @pf: pointer to PF structure - * @needed: number of irqs to get - * - * This returns the first MSI-X vector index in PF space that is used by this - * VF. This index is used when accessing PF relative registers such as - * GLINT_VECT2FUNC and GLINT_DYN_CTL. - * This will always be the OICR index in the AVF driver so any functionality - * using vf->first_vector_idx for queue configuration_id: id of VF which will - * use this irqs - * - * Only SRIOV specific vectors are tracked in sriov_irq_bm. SRIOV vectors are - * allocated from the end of global irq index. First bit in sriov_irq_bm means - * last irq index etc. It simplifies extension of SRIOV vectors. - * They will be always located from sriov_base_vector to the last irq - * index. While increasing/decreasing sriov_base_vector can be moved. - */ -static int ice_sriov_get_irqs(struct ice_pf *pf, u16 needed) -{ - int res = bitmap_find_next_zero_area(pf->sriov_irq_bm, - pf->sriov_irq_size, 0, needed, 0); - /* conversion from number in bitmap to global irq index */ - int index = pf->sriov_irq_size - res - needed; - - if (res >= pf->sriov_irq_size || index < pf->sriov_base_vector) - return -ENOENT; - - bitmap_set(pf->sriov_irq_bm, res, needed); - return index; -} - -/** - * ice_sriov_free_irqs - free irqs used by the VF - * @pf: pointer to PF structure - * @vf: pointer to VF structure - */ -static void ice_sriov_free_irqs(struct ice_pf *pf, struct ice_vf *vf) -{ - /* Move back from first vector index to first index in bitmap */ - int bm_i = pf->sriov_irq_size - vf->first_vector_idx - vf->num_msix; - - bitmap_clear(pf->sriov_irq_bm, bm_i, vf->num_msix); - vf->first_vector_idx = 0; -} - /** * ice_init_vf_vsi_res - initialize/setup VF VSI resources * @vf: VF to initialize/setup the VSI for @@ -555,7 +442,7 @@ static int ice_init_vf_vsi_res(struct ice_vf *vf) struct ice_vsi *vsi; int err; - vf->first_vector_idx = ice_sriov_get_irqs(pf, vf->num_msix); + vf->first_vector_idx = ice_virt_get_irqs(pf, vf->num_msix); if (vf->first_vector_idx < 0) return -ENOMEM; @@ -855,16 +742,10 @@ static int ice_create_vf_entries(struct ice_pf *pf, u16 num_vfs) */ static int ice_ena_vfs(struct ice_pf *pf, u16 num_vfs) { - int total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors; struct device *dev = ice_pf_to_dev(pf); struct ice_hw *hw = &pf->hw; int ret; - pf->sriov_irq_bm = bitmap_zalloc(total_vectors, GFP_KERNEL); - if (!pf->sriov_irq_bm) - return -ENOMEM; - pf->sriov_irq_size = total_vectors; - /* Disable global interrupt 0 so we don't try to handle the VFLR. */ wr32(hw, GLINT_DYN_CTL(pf->oicr_irq.index), ICE_ITR_NONE << GLINT_DYN_CTL_ITR_INDX_S); @@ -917,7 +798,6 @@ static int ice_ena_vfs(struct ice_pf *pf, u16 num_vfs) /* rearm interrupts here */ ice_irq_dynamic_ena(hw, NULL, NULL); clear_bit(ICE_OICR_INTR_DIS, pf->state); - bitmap_free(pf->sriov_irq_bm); return ret; } @@ -991,16 +871,7 @@ u32 ice_sriov_get_vf_total_msix(struct pci_dev *pdev) { struct ice_pf *pf = pci_get_drvdata(pdev); - return pf->sriov_irq_size - ice_get_max_used_msix_vector(pf); -} - -static int ice_sriov_move_base_vector(struct ice_pf *pf, int move) -{ - if (pf->sriov_base_vector - move < ice_get_max_used_msix_vector(pf)) - return -ENOMEM; - - pf->sriov_base_vector -= move; - return 0; + return pf->virt_irq_tracker.num_entries; } static void ice_sriov_remap_vectors(struct ice_pf *pf, u16 restricted_id) @@ -1019,7 +890,8 @@ static void ice_sriov_remap_vectors(struct ice_pf *pf, u16 restricted_id) continue; ice_dis_vf_mappings(tmp_vf); - ice_sriov_free_irqs(pf, tmp_vf); + ice_virt_free_irqs(pf, tmp_vf->first_vector_idx, + tmp_vf->num_msix); vf_ids[to_remap] = tmp_vf->vf_id; to_remap += 1; @@ -1031,7 +903,7 @@ static void ice_sriov_remap_vectors(struct ice_pf *pf, u16 restricted_id) continue; tmp_vf->first_vector_idx = - ice_sriov_get_irqs(pf, tmp_vf->num_msix); + ice_virt_get_irqs(pf, tmp_vf->num_msix); /* there is no need to rebuild VSI as we are only changing the * vector indexes not amount of MSI-X or queues */ @@ -1102,20 +974,15 @@ int ice_sriov_set_msix_vec_count(struct pci_dev *vf_dev, int msix_vec_count) prev_msix = vf->num_msix; prev_queues = vf->num_vf_qs; - if (ice_sriov_move_base_vector(pf, msix_vec_count - prev_msix)) { - ice_put_vf(vf); - return -ENOSPC; - } - ice_dis_vf_mappings(vf); - ice_sriov_free_irqs(pf, vf); + ice_virt_free_irqs(pf, vf->first_vector_idx, vf->num_msix); /* Remap all VFs beside the one is now configured */ ice_sriov_remap_vectors(pf, vf->vf_id); vf->num_msix = msix_vec_count; vf->num_vf_qs = queues; - vf->first_vector_idx = ice_sriov_get_irqs(pf, vf->num_msix); + vf->first_vector_idx = ice_virt_get_irqs(pf, vf->num_msix); if (vf->first_vector_idx < 0) goto unroll; @@ -1141,7 +1008,7 @@ int ice_sriov_set_msix_vec_count(struct pci_dev *vf_dev, int msix_vec_count) vf->num_msix = prev_msix; vf->num_vf_qs = prev_queues; - vf->first_vector_idx = ice_sriov_get_irqs(pf, vf->num_msix); + vf->first_vector_idx = ice_virt_get_irqs(pf, vf->num_msix); if (vf->first_vector_idx < 0) return -EINVAL; From patchwork Thu Aug 8 07:20:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1970411 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=dB6S9yuN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wfdk94qybz1ydk for ; Thu, 8 Aug 2024 17:20:49 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 527AF81521; Thu, 8 Aug 2024 07:20:47 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id oCxhjWCLVedI; Thu, 8 Aug 2024 07:20:46 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.166.34; helo=ash.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3A35F81871 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1723101646; bh=kjSrqHVsw50ial18cYrPWHYHgVqUmeTGjn5pAcNByTA=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=dB6S9yuNas5LEUCiKlrcPEwCtKLVJum6W6+Uy3lmZqzULlFtwdVjPUakFgEnvjX/w NM/mOTN6r+EPCB/X18SdA0fHGZPjq/0boEIMd74uEWu5JM6c7nzHwNi69IZPAK4cJt SvVoDsV1wNbcRTg6JD9IDwEdpJpAnO3yFqi98nOgiYdlSkEQgAVnknRWKDnEvkFNZw UjCRikKBy2wza7jR+goACh6xWRG5tjbFiguny/jgS6aLeaQbttOBuV7ocLF4arjvt7 DT63MpL/StUk5MHVK3IhtJ/UO/aNUMbVjqRqowK18D+dFtOacJ030g0HnYEfbEKNLj OWE239+gAhOZg== Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 3A35F81871; Thu, 8 Aug 2024 07:20:46 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 162AE1BF369 for ; Thu, 8 Aug 2024 07:20:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 00635607F3 for ; Thu, 8 Aug 2024 07:20:44 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id ednNsyXj6ucq for ; Thu, 8 Aug 2024 07:20:43 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=198.175.65.16; helo=mgamail.intel.com; envelope-from=michal.swiatkowski@linux.intel.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 49A6F60772 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 49A6F60772 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by smtp3.osuosl.org (Postfix) with ESMTPS id 49A6F60772 for ; Thu, 8 Aug 2024 07:20:43 +0000 (UTC) X-CSE-ConnectionGUID: SBO1Lk68Ra2NjTWr5Qlkuw== X-CSE-MsgGUID: 21/jKGQWTwqdXF+0f7Co8g== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="21361393" X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="21361393" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2024 00:20:43 -0700 X-CSE-ConnectionGUID: DTJwXOX6T2GOc/cy3vn76w== X-CSE-MsgGUID: S8RDVoyyRFW1oE2VY44O8w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,272,1716274800"; d="scan'208";a="57073518" Received: from gk3153-dr2-r750-36946.igk.intel.com ([10.102.20.192]) by fmviesa008.fm.intel.com with ESMTP; 08 Aug 2024 00:20:40 -0700 From: Michal Swiatkowski To: intel-wired-lan@lists.osuosl.org Date: Thu, 8 Aug 2024 09:20:16 +0200 Message-ID: <20240808072016.10321-9-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> References: <20240808072016.10321-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723101643; x=1754637643; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a2QTM4YmmefzVNxyTiU8f/nPHpRT+z0Aq+zZodKDD3A=; b=Fmu5rTyZb6CXgw8R9kRvOX5Ps943vD97RVAWq9cyiVtJCD6nBqMhfXsq vjgoKuIlf9177Fm50Ytn3+NKL8YCx6s/lM4Dj3By49ggmRs18ch//kVCE BxBYXoEakMaKWRjw8MwWxl/4uJuntYXE7frB4T5TpPMmOOERVALFExvma TMvUkEUVgzLBkoUYUZiJU4FdW/iJgK/4M2msOqI0oiIuQPGcPrWG83Msw uNlDIESmS2RmoG9GLjiWy8ulre7JJnwlM9FYyK2pO+6hg6J7cFs0SowtP UaOUo0w/1JzNN6akpdosQqlBqNBX6qMNny6VPwzB4eXzdOoKChq5sOAdn w==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dmarc=none (p=none dis=none) header.from=linux.intel.com X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Fmu5rTyZ Subject: [Intel-wired-lan] [iwl-next v3 8/8] ice: init flow director before RDMA X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wojciech.drewek@intel.com, marcin.szycik@intel.com, netdev@vger.kernel.org, konrad.knitter@intel.com, pawel.chmielewski@intel.com, nex.sw.ncis.nat.hpm.dev@intel.com, pio.raczynski@gmail.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, jiri@resnulli.us, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Flow director needs only one MSI-X. Load it before RDMA to save MSI-X for it. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 8f02b33adad1..2a3c6c29cccb 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -5185,11 +5185,12 @@ int ice_load(struct ice_pf *pf) ice_napi_add(vsi); + ice_init_features(pf); + err = ice_init_rdma(pf); if (err) goto err_init_rdma; - ice_init_features(pf); ice_service_task_restart(pf); clear_bit(ICE_DOWN, pf->state); @@ -5197,6 +5198,7 @@ int ice_load(struct ice_pf *pf) return 0; err_init_rdma: + ice_deinit_features(pf); ice_tc_indir_block_unregister(vsi); err_tc_indir_block_register: ice_unregister_netdev(vsi); @@ -5220,8 +5222,8 @@ void ice_unload(struct ice_pf *pf) devl_assert_locked(priv_to_devlink(pf)); - ice_deinit_features(pf); ice_deinit_rdma(pf); + ice_deinit_features(pf); ice_tc_indir_block_unregister(vsi); ice_unregister_netdev(vsi); ice_devlink_destroy_pf_port(pf);