From patchwork Mon Nov 14 12:57:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703581 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=SP6mQNHx; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qTY3nkvz1yhv for ; Tue, 15 Nov 2022 00:12:01 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 98354408AD; Mon, 14 Nov 2022 13:11:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 98354408AD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431519; bh=47IxZYaVRBnKq1LoMssIGBlcurITiVdxEsFKOJxQBg4=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=SP6mQNHxX7kkEBDaqeeqRphY2WczxnS9Q4zwIJIlW/xFiZMzdvO7JewcpVX9Ct0m1 Q/JYk20rDoA96NVzbIgYCJWgLFvwKS+d0Se2zbMBT+R2WlpH+pVssz39amEqvOmK6K T1BdUjuilaHxIGLT5GZjHYBt0Cth8mR+3swJi7h50QP9vcOerq2omIQWDf0nH6YamS Sc20U4pwlTW4Fu27oWhgnIl8VAKJET303jf6KpkaHaX+zqPUe0jT8bLX3xA5J9ILym cdS80ar41Dc/KGnasyCyryj/8RxqGDGNojlkIXuXLzD+Po/ic60bVlD+x2LaqbshUD CcQrcDvLaD2+A== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5YJ6Zu5mAjUr; Mon, 14 Nov 2022 13:11:58 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 38A954098B; Mon, 14 Nov 2022 13:11:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 38A954098B X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id A09401BF379 for ; Mon, 14 Nov 2022 13:11:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7BFCA8186B for ; Mon, 14 Nov 2022 13:11:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7BFCA8186B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OxDGdSxUsnNJ for ; Mon, 14 Nov 2022 13:11:55 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org BB17E817FF Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by smtp1.osuosl.org (Postfix) with ESMTPS id BB17E817FF for ; Mon, 14 Nov 2022 13:11:55 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="309591017" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="309591017" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:11:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305842" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305842" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:11:50 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:43 +0100 Message-Id: <20221114125755.13659-2-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431515; x=1699967515; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7Ce8aRJrwaIeECv3XGlXlz0W0+lnMTqEOS/2LgKL3XE=; b=TqAA9tW1ttO6qhRCXoV3NWjH5pCwfpKHdkpuhvBy68K2WbTWwRUYQHWw E6lj1kw9QZH+uzUGM1s2Yvx/QFXEsAV64QnnbDy6m8XxAjV6NBrhItyXz FVPFpT3AxKjMRjzs+V04uUPFHkcV6/pv/AqIGJ+ppjwOa3Xj9IKwHDi6G yd8zBY7NR2VJ2H1PtQY+SLpFiMG6vx/se8TU9rYnBrXpBUS49UPpe7VHq grP9QjGP6gEfA0/WeXl2m0csLruyqYIL+Q+GbPDNKHI2TaAU5Fm3TR/tj sKHITB06EJtM5fZgCrROHvyLHHcAJuPwoGVeLjxhwR6nCKDicmIjbItHz w==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=TqAA9tW1 Subject: [Intel-wired-lan] [PATCH net-next 01/13] ice: move RDMA init to ice_idc.c X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Simplify probe flow by moving all RDMA related code to ice_init_rdma(). Unroll irq allocation if RDMA initialization fails. Implement ice_deinit_rdma() and use it in remove flow. Signed-off-by: Michal Swiatkowski Acked-by: Dave Ertman --- drivers/net/ethernet/intel/ice/ice.h | 1 + drivers/net/ethernet/intel/ice/ice_idc.c | 52 ++++++++++++++++++++++- drivers/net/ethernet/intel/ice/ice_main.c | 29 +++---------- 3 files changed, 57 insertions(+), 25 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index f88ee051e71c..c0079e88cda7 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -900,6 +900,7 @@ void ice_print_link_msg(struct ice_vsi *vsi, bool isup); int ice_plug_aux_dev(struct ice_pf *pf); void ice_unplug_aux_dev(struct ice_pf *pf); int ice_init_rdma(struct ice_pf *pf); +void ice_deinit_rdma(struct ice_pf *pf); const char *ice_aq_str(enum ice_aq_err aq_err); bool ice_is_wol_supported(struct ice_hw *hw); void ice_fdir_del_all_fltrs(struct ice_vsi *vsi); diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c index 895c32bcc8b5..579d2a433ea1 100644 --- a/drivers/net/ethernet/intel/ice/ice_idc.c +++ b/drivers/net/ethernet/intel/ice/ice_idc.c @@ -6,6 +6,8 @@ #include "ice_lib.h" #include "ice_dcb_lib.h" +static DEFINE_IDA(ice_aux_ida); + /** * ice_get_auxiliary_drv - retrieve iidc_auxiliary_drv struct * @pf: pointer to PF struct @@ -245,6 +247,17 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf) return 0; } +/** + * ice_free_rdma_qvector - free vector resources reserved for RDMA driver + * @pf: board private structure to initialize + */ +static void ice_free_rdma_qvector(struct ice_pf *pf) +{ + pf->num_avail_sw_msix -= pf->num_rdma_msix; + ice_free_res(pf->irq_tracker, pf->rdma_base_vector, + ICE_RES_RDMA_VEC_ID); +} + /** * ice_adev_release - function to be mapped to AUX dev's release op * @dev: pointer to device to free @@ -331,12 +344,47 @@ int ice_init_rdma(struct ice_pf *pf) struct device *dev = &pf->pdev->dev; int ret; + if (!ice_is_rdma_ena(pf)) { + dev_warn(dev, "RDMA is not supported on this device\n"); + return 0; + } + + pf->aux_idx = ida_alloc(&ice_aux_ida, GFP_KERNEL); + if (pf->aux_idx < 0) { + dev_err(dev, "Failed to allocate device ID for AUX driver\n"); + return -ENOMEM; + } + /* Reserve vector resources */ ret = ice_reserve_rdma_qvector(pf); if (ret < 0) { dev_err(dev, "failed to reserve vectors for RDMA\n"); - return ret; + goto err_reserve_rdma_qvector; } pf->rdma_mode |= IIDC_RDMA_PROTOCOL_ROCEV2; - return ice_plug_aux_dev(pf); + ret = ice_plug_aux_dev(pf); + if (ret) + goto err_plug_aux_dev; + return 0; + +err_plug_aux_dev: + ice_free_rdma_qvector(pf); +err_reserve_rdma_qvector: + pf->adev = NULL; + ida_free(&ice_aux_ida, pf->aux_idx); + return ret; +} + +/** + * ice_deinit_rdma - deinitialize RDMA on PF + * @pf: ptr to ice_pf + */ +void ice_deinit_rdma(struct ice_pf *pf) +{ + if (!ice_is_rdma_ena(pf)) + return; + + ice_unplug_aux_dev(pf); + ice_free_rdma_qvector(pf); + ida_free(&ice_aux_ida, pf->aux_idx); } diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index a9fc89aebebe..d5bd56a213ae 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -44,7 +44,6 @@ MODULE_PARM_DESC(debug, "netif level (0=none,...,16=all), hw debug_mask (0x8XXXX MODULE_PARM_DESC(debug, "netif level (0=none,...,16=all)"); #endif /* !CONFIG_DYNAMIC_DEBUG */ -static DEFINE_IDA(ice_aux_ida); DEFINE_STATIC_KEY_FALSE(ice_xdp_locking_key); EXPORT_SYMBOL(ice_xdp_locking_key); @@ -4904,30 +4903,16 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) /* ready to go, so clear down state bit */ clear_bit(ICE_DOWN, pf->state); - if (ice_is_rdma_ena(pf)) { - pf->aux_idx = ida_alloc(&ice_aux_ida, GFP_KERNEL); - if (pf->aux_idx < 0) { - dev_err(dev, "Failed to allocate device ID for AUX driver\n"); - err = -ENOMEM; - goto err_devlink_reg_param; - } - - err = ice_init_rdma(pf); - if (err) { - dev_err(dev, "Failed to initialize RDMA: %d\n", err); - err = -EIO; - goto err_init_aux_unroll; - } - } else { - dev_warn(dev, "RDMA is not supported on this device\n"); + err = ice_init_rdma(pf); + if (err) { + dev_err(dev, "Failed to initialize RDMA: %d\n", err); + err = -EIO; + goto err_devlink_reg_param; } ice_devlink_register(pf); return 0; -err_init_aux_unroll: - pf->adev = NULL; - ida_free(&ice_aux_ida, pf->aux_idx); err_devlink_reg_param: ice_devlink_unregister_params(pf); err_netdev_reg: @@ -5040,9 +5025,7 @@ static void ice_remove(struct pci_dev *pdev) ice_service_task_stop(pf); ice_aq_cancel_waiting_tasks(pf); - ice_unplug_aux_dev(pf); - if (pf->aux_idx >= 0) - ida_free(&ice_aux_ida, pf->aux_idx); + ice_deinit_rdma(pf); ice_devlink_unregister_params(pf); set_bit(ICE_DOWN, pf->state); From patchwork Mon Nov 14 12:57:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703582 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=Hl0HDoP/; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qTv16DRz1yhv for ; Tue, 15 Nov 2022 00:12:19 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4C266817D3; Mon, 14 Nov 2022 13:12:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4C266817D3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431537; bh=5q8Jx0RCvetMATsWuZMwio6ec79x1f62yKZFQPsIjAA=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=Hl0HDoP/OJnjH/OAyHbwsYxZ/yxPriM1bOtBmBDZB30z7QMhV7HUKGzV8tMfobUUX OH7ly9uvS2PyhRGhxDah2HucNTvkRfXI3hl7/jlYu5vf3J+JV4xACHKiZBV5JuNmJG +WemhyBH/JxQWYCoO8Zd+nM4iFmeYR9U1xd2lLI2TOy4H9ApzRzqVbg8Wnjo5B5gxA TWOVKQRWWNsWrxNyF9nghAV2OlQWwiOo7Uw4dJvKH56lbC4Xw0QAN4r8GVoaZTVsly uuonVobZspPupyxW3lX6Cgk9jE7YXOgcL8Ug4Puo1Xq9/TaeIM4/mRkL3qOOMwYwDN ESVRIy4/xQCUQ== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a3IpgOMfaz6M; Mon, 14 Nov 2022 13:12:14 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 351DF817FF; Mon, 14 Nov 2022 13:12:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 351DF817FF X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 43D051BF379 for ; Mon, 14 Nov 2022 13:12:13 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2CAB360DF4 for ; Mon, 14 Nov 2022 13:12:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2CAB360DF4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id biHlCQ2jpvpM for ; Mon, 14 Nov 2022 13:12:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0519E60DCA Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 0519E60DCA for ; Mon, 14 Nov 2022 13:12:08 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="310672421" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="310672421" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305911" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305911" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:11:55 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:44 +0100 Message-Id: <20221114125755.13659-3-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431529; x=1699967529; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RGhxY1WcWjeK9Ck3s1XWaiH0nTPWhb8gwqUkEEZ9TeY=; b=NDJr5x2i3ROJGuhY0zcDUbQ3y5bnBM+KEa0SNhRevaa8RVE8rU/p05hP c4exPqwN+Qpo56vjHNZr9d4+v/XTZgO37uh1McnMAlf9WfSb1YQ2s6/bF pokMOQ4TKrN6yJKDCfkakUehbiyWXgAwzgp3fwkex3EO1P2+y/bxUVWsV l3yWgsFYJ6t7aqDe1BjLAhLTxCGPqSngHpNVvw+6JDjtdbcKHHwDjISQN XGzusdUga5Y87kp+DANbSHTsSc+XHKbtmmI1jPoY6uQjgIQInnqoxpV6L Lhsz3TrZCjPp4Zl4UMNcnq27QT2Iga+d5xSEvXWVHE8QRaQ7bG1G/1LDD Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NDJr5x2i Subject: [Intel-wired-lan] [PATCH net-next 02/13] ice: alloc id for RDMA using xa_array X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Use xa_array instead of deprecated ida to alloc id for RDMA aux driver. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_idc.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c index 579d2a433ea1..e6bc2285071e 100644 --- a/drivers/net/ethernet/intel/ice/ice_idc.c +++ b/drivers/net/ethernet/intel/ice/ice_idc.c @@ -6,7 +6,7 @@ #include "ice_lib.h" #include "ice_dcb_lib.h" -static DEFINE_IDA(ice_aux_ida); +static DEFINE_XARRAY_ALLOC1(ice_aux_id); /** * ice_get_auxiliary_drv - retrieve iidc_auxiliary_drv struct @@ -349,8 +349,9 @@ int ice_init_rdma(struct ice_pf *pf) return 0; } - pf->aux_idx = ida_alloc(&ice_aux_ida, GFP_KERNEL); - if (pf->aux_idx < 0) { + ret = xa_alloc(&ice_aux_id, &pf->aux_idx, NULL, XA_LIMIT(1, INT_MAX), + GFP_KERNEL); + if (ret) { dev_err(dev, "Failed to allocate device ID for AUX driver\n"); return -ENOMEM; } @@ -371,7 +372,7 @@ int ice_init_rdma(struct ice_pf *pf) ice_free_rdma_qvector(pf); err_reserve_rdma_qvector: pf->adev = NULL; - ida_free(&ice_aux_ida, pf->aux_idx); + xa_erase(&ice_aux_id, pf->aux_idx); return ret; } @@ -386,5 +387,5 @@ void ice_deinit_rdma(struct ice_pf *pf) ice_unplug_aux_dev(pf); ice_free_rdma_qvector(pf); - ida_free(&ice_aux_ida, pf->aux_idx); + xa_erase(&ice_aux_id, pf->aux_idx); } From patchwork Mon Nov 14 12:57:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703583 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=jF08Da89; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qV02hgfz1yhv for ; Tue, 15 Nov 2022 00:12:24 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 8C55681911; Mon, 14 Nov 2022 13:12:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8C55681911 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431542; bh=TUyrs16X/YXzRGMLDs9T0v6PoXM7kj6AR+kJRVHjLpE=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=jF08Da89huIp8FlaP1sVi23QuqYP/Dhu2vjSqXY8sf+W530RZqe7aAPSA88l1xb43 fEPcob+APxJ2c4t453dTqC1zDC1pRUzZdlhh0j61mdfRmYiD4fymja5FGhW5yi4i1d U6gDWEEhX5piLfRHd+ylF7ClwCslmcOYaU4Tv4gjQ3bS35sOzUvaiGpMoJo3pwOrXm OQWxAoAXK0vctQX0jU5xMBkmaVVh/eWBPdFi2Zc3my3sJZ53omAPjodpy16z84jE+a 25J582oGyT6G9UZ3jaJTJkJO2Lv2Ey7YqNbcgiQkSVsIeV9UJJJax+Bjge23mjEz6C icMjS/bqMUbvA== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7uz2mSPjI4QW; Mon, 14 Nov 2022 13:12:18 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id DB65581886; Mon, 14 Nov 2022 13:12:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DB65581886 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id AC3F21BF379 for ; Mon, 14 Nov 2022 13:12:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8808960DCA for ; Mon, 14 Nov 2022 13:12:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8808960DCA X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IBQIttW5MNJW for ; Mon, 14 Nov 2022 13:12:10 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6BD9C60DDF Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6BD9C60DDF for ; Mon, 14 Nov 2022 13:12:10 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="310672424" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="310672424" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305917" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305917" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:01 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:45 +0100 Message-Id: <20221114125755.13659-4-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431530; x=1699967530; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0I4KNigSToVod2GIqxEI96OLu9b4/Oo3Es5Hq22nUBo=; b=i5aKH8POIamuTQ8enLHYfCBOXkgZcd6gEUBu0ls2J3Br00ShWvz9R+h6 H9ltiBun/dBV6WunbZXR1J4i9U1L50SzsG8t9RBk63TXxn4FiqX2PjMwv +VRgiGmiK6iHW6IW7KAv+rK9hNRz2YO/pKVimswnRdRwgOoJTOILyJfqL kVOSf+rVey2odHJLfR2qtRs6s0B+zNnXJ+EWpJwdm6GE8aKIA5s38jyRq 56sI3IkXpMmGyVDNuB1vmT1OEnOcO8XJnMg9Fll6HZNapwcX+eUxHIKYe +rC+cuIbj6Eyy84e75C9e/uuuaZ+uoFkEUXbYxH8P4pUAKFutT/oefTzT Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=i5aKH8PO Subject: [Intel-wired-lan] [PATCH net-next 03/13] ice: cleanup in VSI config/deconfig code X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Do few small cleanups: 1) Rename the function to reflect that it doesn't configure all things related to VSI. ice_vsi_cfg_lan() better fits to what function is doing. ice_vsi_cfg() can be use to name function that will configure whole VSI. 2) Remove unused ethtype field from VSI. There is no need to set ethtype here, because it is never used. 3) Remove unnecessary check for ICE_VSI_CHNL. There is check for ICE_VSI_CHNL in ice_vsi_get_qs, so there is no need to check it before calling the function. 4) Simplify ice_vsi_alloc() call. There is no need to check the type of VSI before calling ice_vsi_alloc(). For ICE_VSI_CHNL vf is always NULL (ice_vsi_setup() is called with vf=NULL). For ICE_VSI_VF or ICE_VSI_CTRL ch is always NULL and for other VSI types ch and vf are always NULL. 5) Remove unnecessary call to ice_vsi_dis_irq(). ice_vsi_dis_irq() will be called in ice_vsi_close() flow (ice_vsi_close() -> ice_vsi_down() -> ice_vsi_dis_irq()). Remove unnecessary call. 6) Don't remove specific filters in release. All hw filters are removed in ice_fltr_remove_alli(), which is always called in VSI release flow. There is no need to remove only ethertype filters before calling ice_fltr_remove_all(). 7) Rename ice_vsi_clear() to ice_vsi_free(). As ice_vsi_clear() only free memory allocated in ice_vsi_alloc() rename it to ice_vsi_free() which better shows what function is doing. 8) Free coalesce param in rebuild. There is potential memory leak if configuration of VSI lan fails. Free coalesce to avoid it. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice.h | 3 +- drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +- drivers/net/ethernet/intel/ice/ice_lib.c | 51 +++++++------------- drivers/net/ethernet/intel/ice/ice_lib.h | 2 +- drivers/net/ethernet/intel/ice/ice_main.c | 12 ++--- 5 files changed, 26 insertions(+), 44 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index c0079e88cda7..5c5b188fb3e9 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -347,7 +347,6 @@ struct ice_vsi { struct ice_vf *vf; /* VF associated with this VSI */ - u16 ethtype; /* Ethernet protocol for pause frame */ u16 num_gfltr; u16 num_bfltr; @@ -882,7 +881,7 @@ ice_fetch_u64_stats_per_ring(struct u64_stats_sync *syncp, int ice_up(struct ice_vsi *vsi); int ice_down(struct ice_vsi *vsi); int ice_down_up(struct ice_vsi *vsi); -int ice_vsi_cfg(struct ice_vsi *vsi); +int ice_vsi_cfg_lan(struct ice_vsi *vsi); struct ice_vsi *ice_lb_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi); int ice_vsi_determine_xdp_res(struct ice_vsi *vsi); int ice_prepare_xdp_rings(struct ice_vsi *vsi, struct bpf_prog *prog); diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c index f71a7521c7bd..1dfce48cfdc2 100644 --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c @@ -656,7 +656,7 @@ static int ice_lbtest_prepare_rings(struct ice_vsi *vsi) if (status) goto err_setup_rx_ring; - status = ice_vsi_cfg(vsi); + status = ice_vsi_cfg_lan(vsi); if (status) goto err_setup_rx_ring; diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 7276badfa19e..42203a0efcac 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -348,7 +348,7 @@ static void ice_vsi_free_arrays(struct ice_vsi *vsi) } /** - * ice_vsi_clear - clean up and deallocate the provided VSI + * ice_vsi_free - clean up and deallocate the provided VSI * @vsi: pointer to VSI being cleared * * This deallocates the VSI's queue resources, removes it from the PF's @@ -356,7 +356,7 @@ static void ice_vsi_free_arrays(struct ice_vsi *vsi) * * Returns 0 on success, negative on failure */ -int ice_vsi_clear(struct ice_vsi *vsi) +int ice_vsi_free(struct ice_vsi *vsi) { struct ice_pf *pf = NULL; struct device *dev; @@ -2516,12 +2516,7 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, struct ice_vsi *vsi; int ret, i; - if (vsi_type == ICE_VSI_CHNL) - vsi = ice_vsi_alloc(pf, vsi_type, ch, NULL); - else if (vsi_type == ICE_VSI_VF || vsi_type == ICE_VSI_CTRL) - vsi = ice_vsi_alloc(pf, vsi_type, NULL, vf); - else - vsi = ice_vsi_alloc(pf, vsi_type, NULL, NULL); + vsi = ice_vsi_alloc(pf, vsi_type, ch, vf); if (!vsi) { dev_err(dev, "could not allocate VSI\n"); @@ -2530,17 +2525,13 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, vsi->port_info = pi; vsi->vsw = pf->first_sw; - if (vsi->type == ICE_VSI_PF) - vsi->ethtype = ETH_P_PAUSE; ice_alloc_fd_res(vsi); - if (vsi_type != ICE_VSI_CHNL) { - if (ice_vsi_get_qs(vsi)) { - dev_err(dev, "Failed to allocate queues. vsi->idx = %d\n", - vsi->idx); - goto unroll_vsi_alloc; - } + if (ice_vsi_get_qs(vsi)) { + dev_err(dev, "Failed to allocate queues. vsi->idx = %d\n", + vsi->idx); + goto unroll_vsi_alloc; } /* set RSS capabilities */ @@ -2692,7 +2683,7 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, unroll_vsi_alloc: if (vsi_type == ICE_VSI_VF) ice_enable_lag(pf->lag); - ice_vsi_clear(vsi); + ice_vsi_free(vsi); return NULL; } @@ -3016,9 +3007,6 @@ int ice_vsi_release(struct ice_vsi *vsi) if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) ice_rss_clean(vsi); - /* Disable VSI and free resources */ - if (vsi->type != ICE_VSI_LB) - ice_vsi_dis_irq(vsi); ice_vsi_close(vsi); /* SR-IOV determines needed MSIX resources all at once instead of per @@ -3034,18 +3022,12 @@ int ice_vsi_release(struct ice_vsi *vsi) pf->num_avail_sw_msix += vsi->num_q_vectors; } - if (!ice_is_safe_mode(pf)) { - if (vsi->type == ICE_VSI_PF) { - ice_fltr_remove_eth(vsi, ETH_P_PAUSE, ICE_FLTR_TX, - ICE_DROP_PACKET); - ice_cfg_sw_lldp(vsi, true, false); - /* The Rx rule will only exist to remove if the LLDP FW - * engine is currently stopped - */ - if (!test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) - ice_cfg_sw_lldp(vsi, false, false); - } - } + /* The Rx rule will only exist to remove if the LLDP FW + * engine is currently stopped + */ + if (!ice_is_safe_mode(pf) && vsi->type == ICE_VSI_PF && + !test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) + ice_cfg_sw_lldp(vsi, false, false); if (ice_is_vsi_dflt_vsi(vsi)) ice_clear_dflt_vsi(vsi); @@ -3085,7 +3067,7 @@ int ice_vsi_release(struct ice_vsi *vsi) * for ex: during rmmod. */ if (!ice_is_reset_in_progress(pf->state)) - ice_vsi_clear(vsi); + ice_vsi_free(vsi); return 0; } @@ -3384,6 +3366,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) ret = -EIO; goto err_vectors; } else { + kfree(coalesce); return ice_schedule_reset(pf, ICE_RESET_PFR); } } @@ -3402,7 +3385,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) vsi->netdev = NULL; } err_vsi: - ice_vsi_clear(vsi); + ice_vsi_free(vsi); set_bit(ICE_RESET_FAILED, pf->state); kfree(coalesce); return ret; diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h index dcdf69a693e9..6203114b805c 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_lib.h @@ -42,7 +42,7 @@ void ice_cfg_sw_lldp(struct ice_vsi *vsi, bool tx, bool create); int ice_set_link(struct ice_vsi *vsi, bool ena); void ice_vsi_delete(struct ice_vsi *vsi); -int ice_vsi_clear(struct ice_vsi *vsi); +int ice_vsi_free(struct ice_vsi *vsi); int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index d5bd56a213ae..d39ceb3a3b72 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -6091,12 +6091,12 @@ static int ice_vsi_vlan_setup(struct ice_vsi *vsi) } /** - * ice_vsi_cfg - Setup the VSI + * ice_vsi_cfg_lan - Setup the VSI lan related config * @vsi: the VSI being configured * * Return 0 on success and negative value on error */ -int ice_vsi_cfg(struct ice_vsi *vsi) +int ice_vsi_cfg_lan(struct ice_vsi *vsi) { int err; @@ -6314,7 +6314,7 @@ int ice_up(struct ice_vsi *vsi) { int err; - err = ice_vsi_cfg(vsi); + err = ice_vsi_cfg_lan(vsi); if (!err) err = ice_up_complete(vsi); @@ -6855,7 +6855,7 @@ int ice_vsi_open_ctrl(struct ice_vsi *vsi) if (err) goto err_setup_rx; - err = ice_vsi_cfg(vsi); + err = ice_vsi_cfg_lan(vsi); if (err) goto err_setup_rx; @@ -6909,7 +6909,7 @@ int ice_vsi_open(struct ice_vsi *vsi) if (err) goto err_setup_rx; - err = ice_vsi_cfg(vsi); + err = ice_vsi_cfg_lan(vsi); if (err) goto err_setup_rx; @@ -8343,7 +8343,7 @@ static void ice_remove_q_channels(struct ice_vsi *vsi, bool rem_fltr) ice_vsi_delete(ch->ch_vsi); /* Delete VSI from PF and HW VSI arrays */ - ice_vsi_clear(ch->ch_vsi); + ice_vsi_free(ch->ch_vsi); /* free the channel */ kfree(ch); From patchwork Mon Nov 14 12:57:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703585 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=FQQRm/a7; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qV86Cxdz1yhv for ; Tue, 15 Nov 2022 00:12:32 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2D461415C7; Mon, 14 Nov 2022 13:12:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2D461415C7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431551; bh=HwDrpJbWZIC6irCpkiAJvcmX1uZ5rIwuytHOYojLsn8=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=FQQRm/a7uQiVKz23RKlNrKA8xRZAEElmbUgigfreD/ERmI5Io6VwybV5NGGwcXjHI WK9I6glYzJW7KzsJiz2NjJH0iUbxT5/G0Q8E0mNFc9tCvO1gU8cJ47d952PlV9bbJ2 JUD+fYFCa8uunh1dhdlJvjKzRtovJp6Pxvxv+4bYcJMTZ4UAXDKYNp0isaXIe1i8u1 Cn0sAjD2HVsRmgIGgtHB8m1b5qMyPkWWl1joPcPP4UuZlD6KRgrAhYUiUCH1EF78NF Q1yXry14+zrx0CuFPBKRm2YM18ysonmet4+jwSGZC9oiWLIXUaHVmWeof265BH3aKs d56o7K+m1Ym5g== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7AKoGbWFHAeb; Mon, 14 Nov 2022 13:12:26 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id D79204098B; Mon, 14 Nov 2022 13:12:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D79204098B X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 76BF21BF379 for ; Mon, 14 Nov 2022 13:12:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 523D260DDF for ; Mon, 14 Nov 2022 13:12:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 523D260DDF X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HYArwgfeER-0 for ; Mon, 14 Nov 2022 13:12:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 88AA360DCA Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id 88AA360DCA for ; Mon, 14 Nov 2022 13:12:18 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="310672436" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="310672436" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305939" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305939" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:06 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:46 +0100 Message-Id: <20221114125755.13659-5-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431538; x=1699967538; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qVmbCiixoxgd5wqB6Vt3p8AnCwMjCYDT/JqiFV+0Qi4=; b=kDeQGl6f8p1dSDNV0+JpnMgFStPhBBGCubqmCBzrLJpw7sSeExFTKI+7 hzk7kzGYkFwas86yC3JSK2w1N6MUX2GJkXwngx3lfD334bso+VzJanpJ6 GGPZzfpvf7Ff+xkh4B0TF7DQjeRWHBg8d75Uw8PKCfCimA/8CJFI6sc4Y q4yrxOItK6H7UFXMJVZefV+FIDsxUJbn0p/DYuU9+AisIazvNq6WJkr6w MbtVdTRseEIe11BUfUyqgE0Wk2Uj2395jFECXWBQwxqYtAiURZ1dpiiCl pWbjuU751lyQD2gRQIXjgzcp1QUMhNavV+8ubOsL1xdp1Dq2mYpxiY+rc Q==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=kDeQGl6f Subject: [Intel-wired-lan] [PATCH net-next 04/13] ice: split ice_vsi_setup into smaller functions X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Main goal is to reuse the same functions in VSI config and rebuild paths. To do this split ice_vsi_setup into smaller pieces and reuse it during rebuild. ice_vsi_alloc() should only alloc memory, not set the default values for VSI. Move setting defaults to separate function. This will allow config of already allocated VSI, for example in reload path. The path is mostly moving code around without introducing new functionality. Functions ice_vsi_cfg() and ice_vsi_decfg() were added, but they are using code that already exist. Use flag to pass information about VSI initialization during rebuild instead of using boolean value. Signed-off-by: Michal Swiatkowski Co-developed-by: Jacob Keller Signed-off-by: Jacob Keller --- drivers/net/ethernet/intel/ice/ice_lib.c | 648 +++++++++----------- drivers/net/ethernet/intel/ice/ice_lib.h | 7 +- drivers/net/ethernet/intel/ice/ice_main.c | 12 +- drivers/net/ethernet/intel/ice/ice_vf_lib.c | 2 +- 4 files changed, 301 insertions(+), 368 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 42203a0efcac..298bb901e4e7 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -447,9 +447,55 @@ static irqreturn_t ice_eswitch_msix_clean_rings(int __always_unused irq, void *d return IRQ_HANDLED; } +/** + * ice_vsi_alloc_def - set default values for already allocated VSI + * @vsi_type: type of VSI + * @ch: ptr to channel + * @vf: VF for ICE_VSI_VF and ICE_VSI_CTRL + */ +static int +ice_vsi_alloc_def(struct ice_vsi *vsi, struct ice_vf *vf, + struct ice_channel *ch) +{ + if (vsi->type != ICE_VSI_CHNL) { + ice_vsi_set_num_qs(vsi, vf); + if (ice_vsi_alloc_arrays(vsi)) + return -ENOMEM; + } + + switch (vsi->type) { + case ICE_VSI_SWITCHDEV_CTRL: + /* Setup eswitch MSIX irq handler for VSI */ + vsi->irq_handler = ice_eswitch_msix_clean_rings; + break; + case ICE_VSI_PF: + /* Setup default MSIX irq handler for VSI */ + vsi->irq_handler = ice_msix_clean_rings; + break; + case ICE_VSI_CTRL: + /* Setup ctrl VSI MSIX irq handler */ + vsi->irq_handler = ice_msix_clean_ctrl_vsi; + break; + case ICE_VSI_CHNL: + if (!ch) + return -EINVAL; + + vsi->num_rxq = ch->num_rxq; + vsi->num_txq = ch->num_txq; + vsi->next_base_q = ch->base_q; + break; + default: + ice_vsi_free_arrays(vsi); + return -EINVAL; + } + + return 0; +} + /** * ice_vsi_alloc - Allocates the next available struct VSI in the PF * @pf: board private structure + * @pi: pointer to the port_info instance * @vsi_type: type of VSI * @ch: ptr to channel * @vf: VF for ICE_VSI_VF and ICE_VSI_CTRL @@ -461,8 +507,9 @@ static irqreturn_t ice_eswitch_msix_clean_rings(int __always_unused irq, void *d * returns a pointer to a VSI on success, NULL on failure. */ static struct ice_vsi * -ice_vsi_alloc(struct ice_pf *pf, enum ice_vsi_type vsi_type, - struct ice_channel *ch, struct ice_vf *vf) +ice_vsi_alloc(struct ice_pf *pf, struct ice_port_info *pi, + enum ice_vsi_type vsi_type, struct ice_channel *ch, + struct ice_vf *vf) { struct device *dev = ice_pf_to_dev(pf); struct ice_vsi *vsi = NULL; @@ -488,61 +535,11 @@ ice_vsi_alloc(struct ice_pf *pf, enum ice_vsi_type vsi_type, vsi->type = vsi_type; vsi->back = pf; + vsi->port_info = pi; + /* For VSIs which don't have a connected VF, this will be NULL */ + vsi->vf = vf; set_bit(ICE_VSI_DOWN, vsi->state); - if (vsi_type == ICE_VSI_VF) - ice_vsi_set_num_qs(vsi, vf); - else if (vsi_type != ICE_VSI_CHNL) - ice_vsi_set_num_qs(vsi, NULL); - - switch (vsi->type) { - case ICE_VSI_SWITCHDEV_CTRL: - if (ice_vsi_alloc_arrays(vsi)) - goto err_rings; - - /* Setup eswitch MSIX irq handler for VSI */ - vsi->irq_handler = ice_eswitch_msix_clean_rings; - break; - case ICE_VSI_PF: - if (ice_vsi_alloc_arrays(vsi)) - goto err_rings; - - /* Setup default MSIX irq handler for VSI */ - vsi->irq_handler = ice_msix_clean_rings; - break; - case ICE_VSI_CTRL: - if (ice_vsi_alloc_arrays(vsi)) - goto err_rings; - - /* Setup ctrl VSI MSIX irq handler */ - vsi->irq_handler = ice_msix_clean_ctrl_vsi; - - /* For the PF control VSI this is NULL, for the VF control VSI - * this will be the first VF to allocate it. - */ - vsi->vf = vf; - break; - case ICE_VSI_VF: - if (ice_vsi_alloc_arrays(vsi)) - goto err_rings; - vsi->vf = vf; - break; - case ICE_VSI_CHNL: - if (!ch) - goto err_rings; - vsi->num_rxq = ch->num_rxq; - vsi->num_txq = ch->num_txq; - vsi->next_base_q = ch->base_q; - break; - case ICE_VSI_LB: - if (ice_vsi_alloc_arrays(vsi)) - goto err_rings; - break; - default: - dev_warn(dev, "Unknown VSI type %d\n", vsi->type); - goto unlock_pf; - } - if (vsi->type == ICE_VSI_CTRL && !vf) { /* Use the last VSI slot as the index for PF control VSI */ vsi->idx = pf->num_alloc_vsi - 1; @@ -560,11 +557,7 @@ ice_vsi_alloc(struct ice_pf *pf, enum ice_vsi_type vsi_type, if (vsi->type == ICE_VSI_CTRL && vf) vf->ctrl_vsi_idx = vsi->idx; - goto unlock_pf; -err_rings: - devm_kfree(dev, vsi); - vsi = NULL; unlock_pf: mutex_unlock(&pf->sw_mutex); return vsi; @@ -1129,12 +1122,12 @@ ice_chnl_vsi_setup_q_map(struct ice_vsi *vsi, struct ice_vsi_ctx *ctxt) /** * ice_vsi_init - Create and initialize a VSI * @vsi: the VSI being configured - * @init_vsi: is this call creating a VSI + * @init_vsi: flag, tell if VSI need to be initialized * * This initializes a VSI context depending on the VSI type to be added and * passes it down to the add_vsi aq command to create a new VSI. */ -static int ice_vsi_init(struct ice_vsi *vsi, bool init_vsi) +static int ice_vsi_init(struct ice_vsi *vsi, int init_vsi) { struct ice_pf *pf = vsi->back; struct ice_hw *hw = &pf->hw; @@ -1196,7 +1189,7 @@ static int ice_vsi_init(struct ice_vsi *vsi, bool init_vsi) /* if updating VSI context, make sure to set valid_section: * to indicate which section of VSI context being updated */ - if (!init_vsi) + if (!(init_vsi & ICE_VSI_FLAG_INIT)) ctxt->info.valid_sections |= cpu_to_le16(ICE_AQ_VSI_PROP_Q_OPT_VALID); } @@ -1209,7 +1202,8 @@ static int ice_vsi_init(struct ice_vsi *vsi, bool init_vsi) if (ret) goto out; - if (!init_vsi) /* means VSI being updated */ + if (!(init_vsi & ICE_VSI_FLAG_INIT)) + /* means VSI being updated */ /* must to indicate which section of VSI context are * being modified */ @@ -1224,7 +1218,7 @@ static int ice_vsi_init(struct ice_vsi *vsi, bool init_vsi) cpu_to_le16(ICE_AQ_VSI_PROP_SECURITY_VALID); } - if (init_vsi) { + if (init_vsi & ICE_VSI_FLAG_INIT) { ret = ice_add_vsi(hw, vsi->idx, ctxt, NULL); if (ret) { dev_err(dev, "Add VSI failed, err %d\n", ret); @@ -2493,39 +2487,89 @@ static void ice_set_agg_vsi(struct ice_vsi *vsi) } /** - * ice_vsi_setup - Set up a VSI by a given type - * @pf: board private structure - * @pi: pointer to the port_info instance - * @vsi_type: VSI type - * @vf: pointer to VF to which this VSI connects. This field is used primarily - * for the ICE_VSI_VF type. Other VSI types should pass NULL. - * @ch: ptr to channel - * - * This allocates the sw VSI structure and its queue resources. + * ice_free_vf_ctrl_res - Free the VF control VSI resource + * @pf: pointer to PF structure + * @vsi: the VSI to free resources for * - * Returns pointer to the successfully allocated and configured VSI sw struct on - * success, NULL on failure. + * Check if the VF control VSI resource is still in use. If no VF is using it + * any more, release the VSI resource. Otherwise, leave it to be cleaned up + * once no other VF uses it. */ -struct ice_vsi * -ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, - enum ice_vsi_type vsi_type, struct ice_vf *vf, - struct ice_channel *ch) +static void ice_free_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi) +{ + struct ice_vf *vf; + unsigned int bkt; + + rcu_read_lock(); + ice_for_each_vf_rcu(pf, bkt, vf) { + if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) { + rcu_read_unlock(); + return; + } + } + rcu_read_unlock(); + + /* No other VFs left that have control VSI. It is now safe to reclaim + * SW interrupts back to the common pool. + */ + ice_free_res(pf->irq_tracker, vsi->base_vector, + ICE_RES_VF_CTRL_VEC_ID); + pf->num_avail_sw_msix += vsi->num_q_vectors; +} + +static int ice_vsi_cfg_tc_lan(struct ice_pf *pf, struct ice_vsi *vsi) { u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 }; struct device *dev = ice_pf_to_dev(pf); - struct ice_vsi *vsi; int ret, i; - vsi = ice_vsi_alloc(pf, vsi_type, ch, vf); + /* configure VSI nodes based on number of queues and TC's */ + ice_for_each_traffic_class(i) { + if (!(vsi->tc_cfg.ena_tc & BIT(i))) + continue; - if (!vsi) { - dev_err(dev, "could not allocate VSI\n"); - return NULL; + if (vsi->type == ICE_VSI_CHNL) { + if (!vsi->alloc_txq && vsi->num_txq) + max_txqs[i] = vsi->num_txq; + else + max_txqs[i] = pf->num_lan_tx; + } else { + max_txqs[i] = vsi->alloc_txq; + } } - vsi->port_info = pi; + dev_dbg(dev, "vsi->tc_cfg.ena_tc = %d\n", vsi->tc_cfg.ena_tc); + ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc, + max_txqs); + if (ret) { + dev_err(dev, "VSI %d failed lan queue config, error %d\n", + vsi->vsi_num, ret); + return ret; + } + + return 0; +} + +/** + * ice_vsi_cfg_def - configure default VSI based on the type + * @vsi: pointer to VSI + * @vf: pointer to VF to which this VSI connects. This field is used primarily + * for the ICE_VSI_VF type. Other VSI types should pass NULL. + * @ch: ptr to channel + */ +static int +ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) +{ + struct device *dev = ice_pf_to_dev(vsi->back); + struct ice_pf *pf = vsi->back; + int ret; + vsi->vsw = pf->first_sw; + ret = ice_vsi_alloc_def(vsi, vf, ch); + if (ret) + return ret; + ice_alloc_fd_res(vsi); if (ice_vsi_get_qs(vsi)) { @@ -2568,6 +2612,14 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, goto unroll_vector_base; ice_vsi_map_rings_to_vectors(vsi); + if (ice_is_xdp_ena_vsi(vsi)) { + ret = ice_vsi_determine_xdp_res(vsi); + if (ret) + goto unroll_vector_base; + ret = ice_prepare_xdp_rings(vsi, vsi->xdp_prog); + if (ret) + goto unroll_vector_base; + } /* ICE_VSI_CTRL does not need RSS so skip RSS processing */ if (vsi->type != ICE_VSI_CTRL) @@ -2624,29 +2676,135 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, goto unroll_vsi_init; } - /* configure VSI nodes based on number of queues and TC's */ - ice_for_each_traffic_class(i) { - if (!(vsi->tc_cfg.ena_tc & BIT(i))) - continue; + return 0; - if (vsi->type == ICE_VSI_CHNL) { - if (!vsi->alloc_txq && vsi->num_txq) - max_txqs[i] = vsi->num_txq; - else - max_txqs[i] = pf->num_lan_tx; - } else { - max_txqs[i] = vsi->alloc_txq; - } +unroll_vector_base: + /* reclaim SW interrupts back to the common pool */ + ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx); + pf->num_avail_sw_msix += vsi->num_q_vectors; +unroll_alloc_q_vector: + ice_vsi_free_q_vectors(vsi); +unroll_vsi_init: + ice_vsi_delete(vsi); +unroll_get_qs: + ice_vsi_put_qs(vsi); +unroll_vsi_alloc: + ice_vsi_free_arrays(vsi); + return ret; +} + +/** + * ice_vsi_cfg - configure VSI and tc on it + * @vsi: pointer to VSI + * @vf: pointer to VF to which this VSI connects. This field is used primarily + * for the ICE_VSI_VF type. Other VSI types should pass NULL. + * @ch: ptr to channel + */ +int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) +{ + int ret; + + ret = ice_vsi_cfg_def(vsi, vf, ch); + if (ret) + return ret; + + ret = ice_vsi_cfg_tc_lan(vsi->back, vsi); + if (ret) + ice_vsi_decfg(vsi); + + return ret; +} + +/** + * ice_vsi_decfg - remove all VSI configuration + * @vsi: pointer to VSI + */ +void ice_vsi_decfg(struct ice_vsi *vsi) +{ + struct ice_pf *pf = vsi->back; + int err; + + /* The Rx rule will only exist to remove if the LLDP FW + * engine is currently stopped + */ + if (!ice_is_safe_mode(pf) && vsi->type == ICE_VSI_PF && + !test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) + ice_cfg_sw_lldp(vsi, false, false); + + ice_fltr_remove_all(vsi); + ice_rm_vsi_lan_cfg(vsi->port_info, vsi->idx); + err = ice_rm_vsi_rdma_cfg(vsi->port_info, vsi->idx); + if (err) + dev_err(ice_pf_to_dev(pf), "Failed to remove RDMA scheduler config for VSI %u, err %d\n", + vsi->vsi_num, err); + + if (ice_is_xdp_ena_vsi(vsi)) + /* return value check can be skipped here, it always returns + * 0 if reset is in progress + */ + ice_destroy_xdp_rings(vsi); + + ice_vsi_clear_rings(vsi); + ice_vsi_free_q_vectors(vsi); + ice_vsi_delete(vsi); + ice_vsi_put_qs(vsi); + ice_vsi_free_arrays(vsi); + + /* SR-IOV determines needed MSIX resources all at once instead of per + * VSI since when VFs are spawned we know how many VFs there are and how + * many interrupts each VF needs. SR-IOV MSIX resources are also + * cleared in the same manner. + */ + if (vsi->type == ICE_VSI_CTRL && vsi->vf) { + ice_free_vf_ctrl_res(pf, vsi); + } else if (vsi->type != ICE_VSI_VF) { + /* reclaim SW interrupts back to the common pool */ + ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx); + pf->num_avail_sw_msix += vsi->num_q_vectors; + vsi->base_vector = 0; } - dev_dbg(dev, "vsi->tc_cfg.ena_tc = %d\n", vsi->tc_cfg.ena_tc); - ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc, - max_txqs); - if (ret) { - dev_err(dev, "VSI %d failed lan queue config, error %d\n", - vsi->vsi_num, ret); - goto unroll_clear_rings; + if (vsi->type == ICE_VSI_VF && + vsi->agg_node && vsi->agg_node->valid) + vsi->agg_node->num_vsis--; + if (vsi->agg_node) { + vsi->agg_node->valid = false; + vsi->agg_node->agg_id = 0; } +} + +/** + * ice_vsi_setup - Set up a VSI by a given type + * @pf: board private structure + * @pi: pointer to the port_info instance + * @vsi_type: VSI type + * @vf: pointer to VF to which this VSI connects. This field is used primarily + * for the ICE_VSI_VF type. Other VSI types should pass NULL. + * @ch: ptr to channel + * + * This allocates the sw VSI structure and its queue resources. + * + * Returns pointer to the successfully allocated and configured VSI sw struct on + * success, NULL on failure. + */ +struct ice_vsi * +ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, + enum ice_vsi_type vsi_type, struct ice_vf *vf, + struct ice_channel *ch) +{ + struct device *dev = ice_pf_to_dev(pf); + struct ice_vsi *vsi; + int ret; + + vsi = ice_vsi_alloc(pf, pi, vsi_type, ch, vf); + if (!vsi) { + dev_err(dev, "could not allocate VSI\n"); + return NULL; + } + + ret = ice_vsi_cfg(vsi, vf, ch); + if (ret) + goto err_vsi_cfg; /* Add switch rule to drop all Tx Flow Control Frames, of look up * type ETHERTYPE from VSIs, and restrict malicious VF from sending @@ -2657,30 +2815,18 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, * be dropped so that VFs cannot send LLDP packets to reconfig DCB * settings in the HW. */ - if (!ice_is_safe_mode(pf)) - if (vsi->type == ICE_VSI_PF) { - ice_fltr_add_eth(vsi, ETH_P_PAUSE, ICE_FLTR_TX, - ICE_DROP_PACKET); - ice_cfg_sw_lldp(vsi, true, true); - } + if (!ice_is_safe_mode(pf) && vsi->type == ICE_VSI_PF) { + ice_fltr_add_eth(vsi, ETH_P_PAUSE, ICE_FLTR_TX, + ICE_DROP_PACKET); + ice_cfg_sw_lldp(vsi, true, true); + } if (!vsi->agg_node) ice_set_agg_vsi(vsi); + return vsi; -unroll_clear_rings: - ice_vsi_clear_rings(vsi); -unroll_vector_base: - /* reclaim SW interrupts back to the common pool */ - ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx); - pf->num_avail_sw_msix += vsi->num_q_vectors; -unroll_alloc_q_vector: - ice_vsi_free_q_vectors(vsi); -unroll_vsi_init: - ice_vsi_delete(vsi); -unroll_get_qs: - ice_vsi_put_qs(vsi); -unroll_vsi_alloc: +err_vsi_cfg: if (vsi_type == ICE_VSI_VF) ice_enable_lag(pf->lag); ice_vsi_free(vsi); @@ -2946,37 +3092,6 @@ void ice_napi_del(struct ice_vsi *vsi) netif_napi_del(&vsi->q_vectors[v_idx]->napi); } -/** - * ice_free_vf_ctrl_res - Free the VF control VSI resource - * @pf: pointer to PF structure - * @vsi: the VSI to free resources for - * - * Check if the VF control VSI resource is still in use. If no VF is using it - * any more, release the VSI resource. Otherwise, leave it to be cleaned up - * once no other VF uses it. - */ -static void ice_free_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi) -{ - struct ice_vf *vf; - unsigned int bkt; - - rcu_read_lock(); - ice_for_each_vf_rcu(pf, bkt, vf) { - if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) { - rcu_read_unlock(); - return; - } - } - rcu_read_unlock(); - - /* No other VFs left that have control VSI. It is now safe to reclaim - * SW interrupts back to the common pool. - */ - ice_free_res(pf->irq_tracker, vsi->base_vector, - ICE_RES_VF_CTRL_VEC_ID); - pf->num_avail_sw_msix += vsi->num_q_vectors; -} - /** * ice_vsi_release - Delete a VSI and free its resources * @vsi: the VSI being removed @@ -2986,7 +3101,6 @@ static void ice_free_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi) int ice_vsi_release(struct ice_vsi *vsi) { struct ice_pf *pf; - int err; if (!vsi->back) return -ENODEV; @@ -3004,41 +3118,14 @@ int ice_vsi_release(struct ice_vsi *vsi) clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); } + if (vsi->type == ICE_VSI_PF) + ice_devlink_destroy_pf_port(pf); + if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) ice_rss_clean(vsi); ice_vsi_close(vsi); - - /* SR-IOV determines needed MSIX resources all at once instead of per - * VSI since when VFs are spawned we know how many VFs there are and how - * many interrupts each VF needs. SR-IOV MSIX resources are also - * cleared in the same manner. - */ - if (vsi->type == ICE_VSI_CTRL && vsi->vf) { - ice_free_vf_ctrl_res(pf, vsi); - } else if (vsi->type != ICE_VSI_VF) { - /* reclaim SW interrupts back to the common pool */ - ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx); - pf->num_avail_sw_msix += vsi->num_q_vectors; - } - - /* The Rx rule will only exist to remove if the LLDP FW - * engine is currently stopped - */ - if (!ice_is_safe_mode(pf) && vsi->type == ICE_VSI_PF && - !test_bit(ICE_FLAG_FW_LLDP_AGENT, pf->flags)) - ice_cfg_sw_lldp(vsi, false, false); - - if (ice_is_vsi_dflt_vsi(vsi)) - ice_clear_dflt_vsi(vsi); - ice_fltr_remove_all(vsi); - ice_rm_vsi_lan_cfg(vsi->port_info, vsi->idx); - err = ice_rm_vsi_rdma_cfg(vsi->port_info, vsi->idx); - if (err) - dev_err(ice_pf_to_dev(vsi->back), "Failed to remove RDMA scheduler config for VSI %u, err %d\n", - vsi->vsi_num, err); - ice_vsi_delete(vsi); - ice_vsi_free_q_vectors(vsi); + ice_vsi_decfg(vsi); if (vsi->netdev) { if (test_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state)) { @@ -3052,16 +3139,6 @@ int ice_vsi_release(struct ice_vsi *vsi) } } - if (vsi->type == ICE_VSI_PF) - ice_devlink_destroy_pf_port(pf); - - if (vsi->type == ICE_VSI_VF && - vsi->agg_node && vsi->agg_node->valid) - vsi->agg_node->num_vsis--; - ice_vsi_clear_rings(vsi); - - ice_vsi_put_qs(vsi); - /* retain SW VSI data structure since it is needed to unregister and * free VSI netdev when PF is not in reset recovery pending state,\ * for ex: during rmmod. @@ -3189,29 +3266,24 @@ ice_vsi_rebuild_set_coalesce(struct ice_vsi *vsi, /** * ice_vsi_rebuild - Rebuild VSI after reset * @vsi: VSI to be rebuild - * @init_vsi: is this an initialization or a reconfigure of the VSI + * @init_vsi: flag, tell if VSI need to be initialized * * Returns 0 on success and negative value on failure */ -int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) +int ice_vsi_rebuild(struct ice_vsi *vsi, int init_vsi) { - u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 }; struct ice_coalesce_stored *coalesce; - int prev_num_q_vectors = 0; - enum ice_vsi_type vtype; + int prev_num_q_vectors; struct ice_pf *pf; - int ret, i; + int ret; if (!vsi) return -EINVAL; pf = vsi->back; - vtype = vsi->type; - if (WARN_ON(vtype == ICE_VSI_VF && !vsi->vf)) + if (WARN_ON(vsi->type == ICE_VSI_VF && !vsi->vf)) return -EINVAL; - ice_vsi_init_vlan_ops(vsi); - coalesce = kcalloc(vsi->num_q_vectors, sizeof(struct ice_coalesce_stored), GFP_KERNEL); if (!coalesce) @@ -3219,174 +3291,30 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi) prev_num_q_vectors = ice_vsi_rebuild_get_coalesce(vsi, coalesce); - ice_rm_vsi_lan_cfg(vsi->port_info, vsi->idx); - ret = ice_rm_vsi_rdma_cfg(vsi->port_info, vsi->idx); + ice_vsi_decfg(vsi); + ret = ice_vsi_cfg_def(vsi, vsi->vf, vsi->ch); if (ret) - dev_err(ice_pf_to_dev(vsi->back), "Failed to remove RDMA scheduler config for VSI %u, err %d\n", - vsi->vsi_num, ret); - ice_vsi_free_q_vectors(vsi); - - /* SR-IOV determines needed MSIX resources all at once instead of per - * VSI since when VFs are spawned we know how many VFs there are and how - * many interrupts each VF needs. SR-IOV MSIX resources are also - * cleared in the same manner. - */ - if (vtype != ICE_VSI_VF) { - /* reclaim SW interrupts back to the common pool */ - ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx); - pf->num_avail_sw_msix += vsi->num_q_vectors; - vsi->base_vector = 0; - } - - if (ice_is_xdp_ena_vsi(vsi)) - /* return value check can be skipped here, it always returns - * 0 if reset is in progress - */ - ice_destroy_xdp_rings(vsi); - ice_vsi_put_qs(vsi); - ice_vsi_clear_rings(vsi); - ice_vsi_free_arrays(vsi); - if (vtype == ICE_VSI_VF) - ice_vsi_set_num_qs(vsi, vsi->vf); - else - ice_vsi_set_num_qs(vsi, NULL); - - ret = ice_vsi_alloc_arrays(vsi); - if (ret < 0) - goto err_vsi; - - ice_vsi_get_qs(vsi); - - ice_alloc_fd_res(vsi); - ice_vsi_set_tc_cfg(vsi); - - /* Initialize VSI struct elements and create VSI in FW */ - ret = ice_vsi_init(vsi, init_vsi); - if (ret < 0) - goto err_vsi; - - switch (vtype) { - case ICE_VSI_CTRL: - case ICE_VSI_SWITCHDEV_CTRL: - case ICE_VSI_PF: - ret = ice_vsi_alloc_q_vectors(vsi); - if (ret) - goto err_rings; - - ret = ice_vsi_setup_vector_base(vsi); - if (ret) - goto err_vectors; - - ret = ice_vsi_set_q_vectors_reg_idx(vsi); - if (ret) - goto err_vectors; - - ret = ice_vsi_alloc_rings(vsi); - if (ret) - goto err_vectors; - - ice_vsi_map_rings_to_vectors(vsi); - if (ice_is_xdp_ena_vsi(vsi)) { - ret = ice_vsi_determine_xdp_res(vsi); - if (ret) - goto err_vectors; - ret = ice_prepare_xdp_rings(vsi, vsi->xdp_prog); - if (ret) - goto err_vectors; - } - /* ICE_VSI_CTRL does not need RSS so skip RSS processing */ - if (vtype != ICE_VSI_CTRL) - /* Do not exit if configuring RSS had an issue, at - * least receive traffic on first queue. Hence no - * need to capture return value - */ - if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) - ice_vsi_cfg_rss_lut_key(vsi); - - /* disable or enable CRC stripping */ - if (vsi->netdev) - ice_vsi_cfg_crc_strip(vsi, !!(vsi->netdev->features & - NETIF_F_RXFCS)); - - break; - case ICE_VSI_VF: - ret = ice_vsi_alloc_q_vectors(vsi); - if (ret) - goto err_rings; - - ret = ice_vsi_set_q_vectors_reg_idx(vsi); - if (ret) - goto err_vectors; - - ret = ice_vsi_alloc_rings(vsi); - if (ret) - goto err_vectors; - - break; - case ICE_VSI_CHNL: - if (test_bit(ICE_FLAG_RSS_ENA, pf->flags)) { - ice_vsi_cfg_rss_lut_key(vsi); - ice_vsi_set_rss_flow_fld(vsi); - } - break; - default: - break; - } - - /* configure VSI nodes based on number of queues and TC's */ - for (i = 0; i < vsi->tc_cfg.numtc; i++) { - /* configure VSI nodes based on number of queues and TC's. - * ADQ creates VSIs for each TC/Channel but doesn't - * allocate queues instead it reconfigures the PF queues - * as per the TC command. So max_txqs should point to the - * PF Tx queues. - */ - if (vtype == ICE_VSI_CHNL) - max_txqs[i] = pf->num_lan_tx; - else - max_txqs[i] = vsi->alloc_txq; - - if (ice_is_xdp_ena_vsi(vsi)) - max_txqs[i] += vsi->num_xdp_txq; - } - - if (test_bit(ICE_FLAG_TC_MQPRIO, pf->flags)) - /* If MQPRIO is set, means channel code path, hence for main - * VSI's, use TC as 1 - */ - ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, 1, max_txqs); - else - ret = ice_cfg_vsi_lan(vsi->port_info, vsi->idx, - vsi->tc_cfg.ena_tc, max_txqs); + goto err_vsi_cfg; + ret = ice_vsi_cfg_tc_lan(pf, vsi); if (ret) { - dev_err(ice_pf_to_dev(pf), "VSI %d failed lan queue config, error %d\n", - vsi->vsi_num, ret); - if (init_vsi) { + if (init_vsi & ICE_VSI_FLAG_INIT) { ret = -EIO; - goto err_vectors; + goto err_vsi_cfg_tc_lan; } else { kfree(coalesce); return ice_schedule_reset(pf, ICE_RESET_PFR); } } + ice_vsi_rebuild_set_coalesce(vsi, coalesce, prev_num_q_vectors); kfree(coalesce); return 0; -err_vectors: - ice_vsi_free_q_vectors(vsi); -err_rings: - if (vsi->netdev) { - vsi->current_netdev_flags = 0; - unregister_netdev(vsi->netdev); - free_netdev(vsi->netdev); - vsi->netdev = NULL; - } -err_vsi: - ice_vsi_free(vsi); - set_bit(ICE_RESET_FAILED, pf->state); +err_vsi_cfg_tc_lan: + ice_vsi_decfg(vsi); +err_vsi_cfg: kfree(coalesce); return ret; } diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h index 6203114b805c..ad4d5314ca76 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_lib.h @@ -61,8 +61,11 @@ int ice_vsi_release(struct ice_vsi *vsi); void ice_vsi_close(struct ice_vsi *vsi); +int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, + struct ice_channel *ch); int ice_ena_vsi(struct ice_vsi *vsi, bool locked); +void ice_vsi_decfg(struct ice_vsi *vsi); void ice_dis_vsi(struct ice_vsi *vsi, bool locked); int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id); @@ -70,7 +73,9 @@ int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id); int ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id); -int ice_vsi_rebuild(struct ice_vsi *vsi, bool init_vsi); +#define ICE_VSI_FLAG_INIT BIT(0) +#define ICE_VSI_FLAG_NO_INIT 0 +int ice_vsi_rebuild(struct ice_vsi *vsi, int init_vsi); bool ice_is_reset_in_progress(unsigned long *state); int ice_wait_for_reset(struct ice_pf *pf, unsigned long timeout); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index d39ceb3a3b72..c71f60c3bb15 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -4204,13 +4204,13 @@ int ice_vsi_recfg_qs(struct ice_vsi *vsi, int new_rx, int new_tx) /* set for the next time the netdev is started */ if (!netif_running(vsi->netdev)) { - ice_vsi_rebuild(vsi, false); + ice_vsi_rebuild(vsi, ICE_VSI_FLAG_NO_INIT); dev_dbg(ice_pf_to_dev(pf), "Link is down, queue count change happens when link is brought up\n"); goto done; } ice_vsi_close(vsi); - ice_vsi_rebuild(vsi, false); + ice_vsi_rebuild(vsi, ICE_VSI_FLAG_NO_INIT); ice_pf_dcb_recfg(pf); ice_vsi_open(vsi); done: @@ -6994,7 +6994,7 @@ static int ice_vsi_rebuild_by_type(struct ice_pf *pf, enum ice_vsi_type type) continue; /* rebuild the VSI */ - err = ice_vsi_rebuild(vsi, true); + err = ice_vsi_rebuild(vsi, ICE_VSI_FLAG_INIT); if (err) { dev_err(dev, "rebuild VSI failed, err %d, VSI index %d, type %s\n", err, vsi->idx, ice_vsi_type_str(type)); @@ -8403,7 +8403,7 @@ static int ice_rebuild_channels(struct ice_pf *pf) type = vsi->type; /* rebuild ADQ VSI */ - err = ice_vsi_rebuild(vsi, true); + err = ice_vsi_rebuild(vsi, ICE_VSI_FLAG_INIT); if (err) { dev_err(dev, "VSI (type:%s) at index %d rebuild failed, err %d\n", ice_vsi_type_str(type), vsi->idx, err); @@ -8629,14 +8629,14 @@ static int ice_setup_tc_mqprio_qdisc(struct net_device *netdev, void *type_data) cur_rxq = vsi->num_rxq; /* proceed with rebuild main VSI using correct number of queues */ - ret = ice_vsi_rebuild(vsi, false); + ret = ice_vsi_rebuild(vsi, ICE_VSI_FLAG_NO_INIT); if (ret) { /* fallback to current number of queues */ dev_info(dev, "Rebuild failed with new queues, try with current number of queues\n"); vsi->req_txq = cur_txq; vsi->req_rxq = cur_rxq; clear_bit(ICE_RESET_FAILED, pf->state); - if (ice_vsi_rebuild(vsi, false)) { + if (ice_vsi_rebuild(vsi, ICE_VSI_FLAG_NO_INIT)) { dev_err(dev, "Rebuild of main VSI failed again\n"); return ret; } diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c index 1c51778db951..149642765874 100644 --- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c @@ -256,7 +256,7 @@ static int ice_vf_rebuild_vsi(struct ice_vf *vf) if (WARN_ON(!vsi)) return -EINVAL; - if (ice_vsi_rebuild(vsi, true)) { + if (ice_vsi_rebuild(vsi, ICE_VSI_FLAG_INIT)) { dev_err(ice_pf_to_dev(pf), "failed to rebuild VF %d VSI\n", vf->vf_id); return -EIO; From patchwork Mon Nov 14 12:57:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703584 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=58W2+Hkf; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qV248wMz1yhv for ; Tue, 15 Nov 2022 00:12:26 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 8CF4260DCA; Mon, 14 Nov 2022 13:12:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 8CF4260DCA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431544; bh=2/pTC5JnUr4YsinjZha0lVuWa/SoqmhGlO7ZiOrRxuA=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=58W2+HkfOKN+G0pRyoXKYX6Tki+kYGOjYi4uPXTJSkH36e9f6jERlf5PVVJKuMGHG YUTBGjDgI0kaOc36g52eIyL/4OfTzGTi5oxkQfIDnIUqMM6ocuTTyisAR/+Lz67Jkp 3XDTtperInLy3sEc2R26XPzjZ7tWJ6z0a9ZEk7LCl9SAueQipBOQvyV6qYv6Kg1R8n indwLV16NSCIqTbszv+JBLo7mef7yWwBIIHWRhtQbtIW0zo14W00uG6qvQp/WM98LN zwLBL4rJH14uhtuw3kdmdWe1jB9WJ7jR1s1wTcpBTIghtdLEAV2qinsxEiUKq7Tpzq mVAJilPulXxBg== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1qz8KAl--NQ9; Mon, 14 Nov 2022 13:12:22 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id A404C60DDF; Mon, 14 Nov 2022 13:12:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A404C60DDF X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id E2F751BF379 for ; Mon, 14 Nov 2022 13:12:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BDACC60DF4 for ; Mon, 14 Nov 2022 13:12:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BDACC60DF4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ewl2yISlNC8Y for ; Mon, 14 Nov 2022 13:12:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B266060DDF Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by smtp3.osuosl.org (Postfix) with ESMTPS id B266060DDF for ; Mon, 14 Nov 2022 13:12:18 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="310672449" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="310672449" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305957" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305957" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:12 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:47 +0100 Message-Id: <20221114125755.13659-6-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431538; x=1699967538; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=W8DRebUemNpzTl8h23j1odQJiwJ/RQ2puj1jmmHtHpY=; b=Uwyc0T+BPW5CbXyBGZECjweMSY8apXPfBJN/jeh6tFN12tW/yVM4idqI M7/NWf5Uk4nsMlaF2BsApx+MQF6dju8fD1hmgUxxgWoc0KKmXUFDoWBFr oGk83ZZ/G3e3Ep56g0RwfoNdMb5okQoDc+E6dkUZ/NIu3CVxsq75MQt1H SASDqLeDvvb7LgKnPIAFhlnH7PGIIhkOqyT1DLXfZnODkyLu3MkHMmRB3 r3vaBiBaIHJbE6iXxC1jIqZTwxo33ZjEEWGKnlYDLDFbSDD5GvnH1dWYQ xHDFTgMY2Hg2XAUfdqBcxDix2hj5x1fox3zv4JEEwG47+NPWS4+4d5Vyx w==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=Uwyc0T+B Subject: [Intel-wired-lan] [PATCH net-next 05/13] ice: stop hard coding the ICE_VSI_CTRL location X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" From: Jacob Keller When allocating the ICE_VSI_CTRL, the allocated struct ice_vsi pointer is stored into the PF's pf->vsi array at a fixed location. This was historically done on the basis that it could provide an O(1) lookup for the special control VSI. Since we store the ctrl_vsi_idx, we already have O(1) lookup regardless of where in the array we store this VSI. Simplify the logic in ice_vsi_alloc by using the same method of storing the control VSI as other types of VSIs. Signed-off-by: Jacob Keller Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_lib.c | 34 +++++++++++------------- 1 file changed, 15 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 298bb901e4e7..dc672af10c25 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -379,10 +379,7 @@ int ice_vsi_free(struct ice_vsi *vsi) /* updates the PF for this cleared VSI */ pf->vsi[vsi->idx] = NULL; - if (vsi->idx < pf->next_vsi && vsi->type != ICE_VSI_CTRL) - pf->next_vsi = vsi->idx; - if (vsi->idx < pf->next_vsi && vsi->type == ICE_VSI_CTRL && vsi->vf) - pf->next_vsi = vsi->idx; + pf->next_vsi = vsi->idx; ice_vsi_free_arrays(vsi); mutex_unlock(&pf->sw_mutex); @@ -540,23 +537,22 @@ ice_vsi_alloc(struct ice_pf *pf, struct ice_port_info *pi, vsi->vf = vf; set_bit(ICE_VSI_DOWN, vsi->state); - if (vsi->type == ICE_VSI_CTRL && !vf) { - /* Use the last VSI slot as the index for PF control VSI */ - vsi->idx = pf->num_alloc_vsi - 1; - pf->ctrl_vsi_idx = vsi->idx; - pf->vsi[vsi->idx] = vsi; - } else { - /* fill slot and make note of the index */ - vsi->idx = pf->next_vsi; - pf->vsi[pf->next_vsi] = vsi; + /* fill slot and make note of the index */ + vsi->idx = pf->next_vsi; + pf->vsi[pf->next_vsi] = vsi; - /* prepare pf->next_vsi for next use */ - pf->next_vsi = ice_get_free_slot(pf->vsi, pf->num_alloc_vsi, - pf->next_vsi); - } + /* prepare pf->next_vsi for next use */ + pf->next_vsi = ice_get_free_slot(pf->vsi, pf->num_alloc_vsi, + pf->next_vsi); - if (vsi->type == ICE_VSI_CTRL && vf) - vf->ctrl_vsi_idx = vsi->idx; + if (vsi->type == ICE_VSI_CTRL) { + if (vf) { + vf->ctrl_vsi_idx = vsi->idx; + } else { + WARN_ON(pf->ctrl_vsi_idx != ICE_NO_VSI); + pf->ctrl_vsi_idx = vsi->idx; + } + } unlock_pf: mutex_unlock(&pf->sw_mutex); From patchwork Mon Nov 14 12:57:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703586 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=U4UA2BdJ; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVC187qz1yhv for ; Tue, 15 Nov 2022 00:12:35 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4B9D2817D3; Mon, 14 Nov 2022 13:12:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4B9D2817D3 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431553; bh=wwBAaZuz6+RHsEZmJpx1e+VHEVRggeD3IqJKe9Yu3Dk=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=U4UA2BdJ5aC2lm0LTqRJIeSeVlNd1xpWfGkGg6tbVzGlivtebIXlJ1GX0c9WU6nZA UtxTA8X3n3J8IdTpTM/h3DKhAupcxghupCChkhDxoSAxmoazOozVvonTYKyy2ippLW hNU95O9dWoh1ZX997Lv5IWMMpWxiqyvYN+IofRKu3ycc3g/Qiqk5+VLBQ2Ny2z22Jk wRF59Rw91ZjuMoPIumUKchpubuqkKy7Cwn63GRpqKIv6X0IprnP0gzlcNRVGOIAcc+ XhfLl3MMYRXUcc4zfrwVkWlRI/1PFnwfMuMj/iKHx6N6nZg4E9GyFnUZaoyHOC8+Ys ng2+oOXhTjJjw== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MosyHRp_7UrH; Mon, 14 Nov 2022 13:12:30 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 6497F81826; Mon, 14 Nov 2022 13:12:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6497F81826 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 314C51BF379 for ; Mon, 14 Nov 2022 13:12:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 0C4D981826 for ; Mon, 14 Nov 2022 13:12:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0C4D981826 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MPaF3CMDWbKL for ; Mon, 14 Nov 2022 13:12:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 10802817D3 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id 10802817D3 for ; Mon, 14 Nov 2022 13:12:22 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766716" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766716" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616305992" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616305992" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:17 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:48 +0100 Message-Id: <20221114125755.13659-7-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431543; x=1699967543; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=H23wPFre84Eiqe2JaFMTuy4YXxIJ6b2eUozWj3eO1FU=; b=lZV5OqOZBeE9WmNxCT+MQE9m3OLwFDM2NBImBr+oRmyAg04IKOuTSPc3 DO5Rcz+9ltw1L4S8XtjJ5DCbcZl32/Cgt07kxs3q9JDGyG3twBF4R90ty Iik59oY3WR9IYmjBc9ENPrZw1e+H6+z2L8MKQ3Uip0YcB4x04AdDoSKue UptlXQY0/pyb4s/ReHk3N5unafJe5Tb/MTlb4UrhdZt2Ie8UTo/1cQe9o 8VzsqOPvS9+2+B+a9+/V6t/bbttqqGkXkKN7AsMD/g9t0fN3wVKU0i+Q1 naUdm3TIra+O/V0aLcWJwNK153PHwu4av/YTSC2Zsg93c/LOjoFMm1YMc Q==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=lZV5OqOZ Subject: [Intel-wired-lan] [PATCH net-next 06/13] ice: split probe into smaller functions X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Part of code from probe can be reused in reload flow. Move this code to separate function. Create unroll functions for each part of initialization, like: ice_init_dev() and ice_deinit_dev(). It simplifies unrolling and can be used in remove flow. Avoid freeing port info as it could be reused in reload path. Will be freed in remove path since is allocated via devm_kzalloc(). Signed-off-by: Michal Swiatkowski Reviewed-by: Przemek Kitszel --- drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 11 +- drivers/net/ethernet/intel/ice/ice_main.c | 896 ++++++++++++-------- 3 files changed, 562 insertions(+), 347 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 5c5b188fb3e9..62219f995cf2 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -924,6 +924,8 @@ int ice_open(struct net_device *netdev); int ice_open_internal(struct net_device *netdev); int ice_stop(struct net_device *netdev); void ice_service_task_schedule(struct ice_pf *pf); +int ice_load(struct ice_pf *pf); +void ice_unload(struct ice_pf *pf); /** * ice_set_rdma_cap - enable RDMA support diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c index 039342a0ed15..9c48e6cf5a0e 100644 --- a/drivers/net/ethernet/intel/ice/ice_common.c +++ b/drivers/net/ethernet/intel/ice/ice_common.c @@ -1088,8 +1088,10 @@ int ice_init_hw(struct ice_hw *hw) if (status) goto err_unroll_cqinit; - hw->port_info = devm_kzalloc(ice_hw_to_dev(hw), - sizeof(*hw->port_info), GFP_KERNEL); + if (!hw->port_info) + hw->port_info = devm_kzalloc(ice_hw_to_dev(hw), + sizeof(*hw->port_info), + GFP_KERNEL); if (!hw->port_info) { status = -ENOMEM; goto err_unroll_cqinit; @@ -1214,11 +1216,6 @@ void ice_deinit_hw(struct ice_hw *hw) ice_free_hw_tbls(hw); mutex_destroy(&hw->tnl_lock); - if (hw->port_info) { - devm_kfree(ice_hw_to_dev(hw), hw->port_info); - hw->port_info = NULL; - } - /* Attempt to disable FW logging before shutting down control queues */ ice_cfg_fw_log(hw, false); ice_destroy_all_ctrlq(hw); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index c71f60c3bb15..bf98334fadbc 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -3405,53 +3405,6 @@ static void ice_set_netdev_features(struct net_device *netdev) netdev->hw_features |= NETIF_F_RXFCS; } -/** - * ice_cfg_netdev - Allocate, configure and register a netdev - * @vsi: the VSI associated with the new netdev - * - * Returns 0 on success, negative value on failure - */ -static int ice_cfg_netdev(struct ice_vsi *vsi) -{ - struct ice_netdev_priv *np; - struct net_device *netdev; - u8 mac_addr[ETH_ALEN]; - - netdev = alloc_etherdev_mqs(sizeof(*np), vsi->alloc_txq, - vsi->alloc_rxq); - if (!netdev) - return -ENOMEM; - - set_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); - vsi->netdev = netdev; - np = netdev_priv(netdev); - np->vsi = vsi; - - ice_set_netdev_features(netdev); - - ice_set_ops(netdev); - - if (vsi->type == ICE_VSI_PF) { - SET_NETDEV_DEV(netdev, ice_pf_to_dev(vsi->back)); - ether_addr_copy(mac_addr, vsi->port_info->mac.perm_addr); - eth_hw_addr_set(netdev, mac_addr); - ether_addr_copy(netdev->perm_addr, mac_addr); - } - - netdev->priv_flags |= IFF_UNICAST_FLT; - - /* Setup netdev TC information */ - ice_vsi_cfg_netdev_tc(vsi, vsi->tc_cfg.ena_tc); - - /* setup watchdog timeout value to be 5 second */ - netdev->watchdog_timeo = 5 * HZ; - - netdev->min_mtu = ETH_MIN_MTU; - netdev->max_mtu = ICE_MAX_MTU; - - return 0; -} - /** * ice_fill_rss_lut - Fill the RSS lookup table with default values * @lut: Lookup table @@ -3704,78 +3657,6 @@ static int ice_tc_indir_block_register(struct ice_vsi *vsi) return flow_indr_dev_register(ice_indr_setup_tc_cb, np); } -/** - * ice_setup_pf_sw - Setup the HW switch on startup or after reset - * @pf: board private structure - * - * Returns 0 on success, negative value on failure - */ -static int ice_setup_pf_sw(struct ice_pf *pf) -{ - struct device *dev = ice_pf_to_dev(pf); - bool dvm = ice_is_dvm_ena(&pf->hw); - struct ice_vsi *vsi; - int status; - - if (ice_is_reset_in_progress(pf->state)) - return -EBUSY; - - status = ice_aq_set_port_params(pf->hw.port_info, dvm, NULL); - if (status) - return -EIO; - - vsi = ice_pf_vsi_setup(pf, pf->hw.port_info); - if (!vsi) - return -ENOMEM; - - /* init channel list */ - INIT_LIST_HEAD(&vsi->ch_list); - - status = ice_cfg_netdev(vsi); - if (status) - goto unroll_vsi_setup; - /* netdev has to be configured before setting frame size */ - ice_vsi_cfg_frame_size(vsi); - - /* init indirect block notifications */ - status = ice_tc_indir_block_register(vsi); - if (status) { - dev_err(dev, "Failed to register netdev notifier\n"); - goto unroll_cfg_netdev; - } - - /* Setup DCB netlink interface */ - ice_dcbnl_setup(vsi); - - /* registering the NAPI handler requires both the queues and - * netdev to be created, which are done in ice_pf_vsi_setup() - * and ice_cfg_netdev() respectively - */ - ice_napi_add(vsi); - - status = ice_init_mac_fltr(pf); - if (status) - goto unroll_napi_add; - - return 0; - -unroll_napi_add: - ice_tc_indir_block_unregister(vsi); -unroll_cfg_netdev: - if (vsi) { - ice_napi_del(vsi); - if (vsi->netdev) { - clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); - free_netdev(vsi->netdev); - vsi->netdev = NULL; - } - } - -unroll_vsi_setup: - ice_vsi_release(vsi); - return status; -} - /** * ice_get_avail_q_count - Get count of queues in use * @pf_qmap: bitmap to get queue use count from @@ -4473,6 +4354,21 @@ static int ice_init_fdir(struct ice_pf *pf) return err; } +static void ice_deinit_fdir(struct ice_pf *pf) +{ + struct ice_vsi *vsi = ice_get_ctrl_vsi(pf); + + if (!vsi) + return; + + ice_vsi_manage_fdir(vsi, false); + ice_vsi_release(vsi); + if (pf->ctrl_vsi_idx != ICE_NO_VSI) { + pf->vsi[pf->ctrl_vsi_idx] = NULL; + pf->ctrl_vsi_idx = ICE_NO_VSI; + } +} + /** * ice_get_opt_fw_name - return optional firmware file name or NULL * @pf: pointer to the PF instance @@ -4573,14 +4469,13 @@ static void ice_print_wake_reason(struct ice_pf *pf) /** * ice_register_netdev - register netdev and devlink port - * @pf: pointer to the PF struct + * @vsi: pointer to the VSI struct */ -static int ice_register_netdev(struct ice_pf *pf) +static int ice_register_netdev(struct ice_vsi *vsi) { - struct ice_vsi *vsi; - int err = 0; + struct ice_pf *pf = vsi->back; + int err; - vsi = ice_get_main_vsi(pf); if (!vsi || !vsi->netdev) return -EIO; @@ -4598,168 +4493,208 @@ static int ice_register_netdev(struct ice_pf *pf) netif_tx_stop_all_queues(vsi->netdev); return 0; + err_register_netdev: ice_devlink_destroy_pf_port(pf); err_devlink_create: - free_netdev(vsi->netdev); - vsi->netdev = NULL; - clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); + unregister_netdev(vsi->netdev); + clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); return err; } +static void ice_unregister_netdev(struct ice_vsi *vsi) +{ + if (!vsi || !vsi->netdev) + return; + + unregister_netdev(vsi->netdev); + ice_devlink_destroy_pf_port(vsi->back); + clear_bit(ICE_VSI_NETDEV_REGISTERED, vsi->state); +} + /** - * ice_probe - Device initialization routine - * @pdev: PCI device information struct - * @ent: entry in ice_pci_tbl + * ice_cfg_netdev - Allocate, configure and register a netdev + * @vsi: the VSI associated with the new netdev * - * Returns 0 on success, negative on failure + * Returns 0 on success, negative value on failure */ -static int -ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) +static int ice_cfg_netdev(struct ice_vsi *vsi) { - struct device *dev = &pdev->dev; - struct ice_pf *pf; - struct ice_hw *hw; - int i, err; + struct ice_netdev_priv *np; + struct net_device *netdev; + u8 mac_addr[ETH_ALEN]; - if (pdev->is_virtfn) { - dev_err(dev, "can't probe a virtual function\n"); - return -EINVAL; - } + netdev = alloc_etherdev_mqs(sizeof(*np), vsi->alloc_txq, + vsi->alloc_rxq); + if (!netdev) + return -ENOMEM; - /* this driver uses devres, see - * Documentation/driver-api/driver-model/devres.rst - */ - err = pcim_enable_device(pdev); - if (err) - return err; + set_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); + vsi->netdev = netdev; + np = netdev_priv(netdev); + np->vsi = vsi; - err = pcim_iomap_regions(pdev, BIT(ICE_BAR0), dev_driver_string(dev)); - if (err) { - dev_err(dev, "BAR0 I/O map error %d\n", err); - return err; + ice_set_netdev_features(netdev); + ice_set_ops(netdev); + + if (vsi->type == ICE_VSI_PF) { + SET_NETDEV_DEV(netdev, ice_pf_to_dev(vsi->back)); + ether_addr_copy(mac_addr, vsi->port_info->mac.perm_addr); + eth_hw_addr_set(netdev, mac_addr); } - pf = ice_allocate_pf(dev); - if (!pf) - return -ENOMEM; + netdev->priv_flags |= IFF_UNICAST_FLT; - /* initialize Auxiliary index to invalid value */ - pf->aux_idx = -1; + /* Setup netdev TC information */ + ice_vsi_cfg_netdev_tc(vsi, vsi->tc_cfg.ena_tc); - /* set up for high or low DMA */ - err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)); - if (err) { - dev_err(dev, "DMA configuration failed: 0x%x\n", err); - return err; - } + netdev->max_mtu = ICE_MAX_MTU; - pci_enable_pcie_error_reporting(pdev); - pci_set_master(pdev); + return 0; +} - pf->pdev = pdev; - pci_set_drvdata(pdev, pf); - set_bit(ICE_DOWN, pf->state); - /* Disable service task until DOWN bit is cleared */ - set_bit(ICE_SERVICE_DIS, pf->state); +static void ice_decfg_netdev(struct ice_vsi *vsi) +{ + clear_bit(ICE_VSI_NETDEV_ALLOCD, vsi->state); + free_netdev(vsi->netdev); + vsi->netdev = NULL; +} - hw = &pf->hw; - hw->hw_addr = pcim_iomap_table(pdev)[ICE_BAR0]; - pci_save_state(pdev); +static int ice_start_eth(struct ice_vsi *vsi) +{ + int err; - hw->back = pf; - hw->vendor_id = pdev->vendor; - hw->device_id = pdev->device; - pci_read_config_byte(pdev, PCI_REVISION_ID, &hw->revision_id); - hw->subsystem_vendor_id = pdev->subsystem_vendor; - hw->subsystem_device_id = pdev->subsystem_device; - hw->bus.device = PCI_SLOT(pdev->devfn); - hw->bus.func = PCI_FUNC(pdev->devfn); - ice_set_ctrlq_len(hw); + err = ice_init_mac_fltr(vsi->back); + if (err) + return err; - pf->msg_enable = netif_msg_init(debug, ICE_DFLT_NETIF_M); + rtnl_lock(); + err = ice_vsi_open(vsi); + rtnl_unlock(); -#ifndef CONFIG_DYNAMIC_DEBUG - if (debug < -1) - hw->debug_mask = debug; -#endif + return err; +} - err = ice_init_hw(hw); - if (err) { - dev_err(dev, "ice_init_hw failed: %d\n", err); - err = -EIO; - goto err_exit_unroll; - } +static int ice_init_eth(struct ice_pf *pf) +{ + struct ice_vsi *vsi = ice_get_main_vsi(pf); + struct device *dev = ice_pf_to_dev(pf); + int err; - ice_init_feature_support(pf); + if (!vsi) + return -EINVAL; - ice_request_fw(pf); + /* init channel list */ + INIT_LIST_HEAD(&vsi->ch_list); - /* if ice_request_fw fails, ICE_FLAG_ADV_FEATURES bit won't be - * set in pf->state, which will cause ice_is_safe_mode to return - * true - */ - if (ice_is_safe_mode(pf)) { - /* we already got function/device capabilities but these don't - * reflect what the driver needs to do in safe mode. Instead of - * adding conditional logic everywhere to ignore these - * device/function capabilities, override them. - */ - ice_set_safe_mode_caps(hw); - } + err = ice_cfg_netdev(vsi); + if (err) + return err; + /* Setup DCB netlink interface */ + ice_dcbnl_setup(vsi); - err = ice_init_pf(pf); + err = ice_set_cpu_rx_rmap(vsi); if (err) { - dev_err(dev, "ice_init_pf failed: %d\n", err); - goto err_init_pf_unroll; + dev_err(dev, "Failed to set CPU Rx map VSI %d error %d\n", + vsi->vsi_num, err); + goto err_set_cpu_rx_rmap; } + err = ice_init_mac_fltr(pf); + if (err) + goto err_init_mac_fltr; - ice_devlink_init_regions(pf); + err = ice_register_netdev(vsi); + if (err) + goto err_register_netdev; - pf->hw.udp_tunnel_nic.set_port = ice_udp_tunnel_set_port; - pf->hw.udp_tunnel_nic.unset_port = ice_udp_tunnel_unset_port; + err = ice_tc_indir_block_register(vsi); + if (err) + goto err_tc_indir_block_register; + + ice_napi_add(vsi); + + return 0; + +err_tc_indir_block_register: + ice_unregister_netdev(vsi); +err_register_netdev: +err_init_mac_fltr: + ice_free_cpu_rx_rmap(vsi); +err_set_cpu_rx_rmap: + ice_decfg_netdev(vsi); + return err; +} + +static void ice_deinit_eth(struct ice_pf *pf) +{ + struct ice_vsi *vsi = ice_get_main_vsi(pf); + + if (!vsi) + return; + + ice_vsi_close(vsi); + ice_unregister_netdev(vsi); + ice_free_cpu_rx_rmap(vsi); + ice_tc_indir_block_unregister(vsi); + ice_decfg_netdev(vsi); +} + +static int ice_init_dev(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + int err; + + err = ice_init_hw(&pf->hw); + if (err) { + dev_err(dev, "ice_init_hw failed: %d\n", err); + return err; + } + + ice_init_feature_support(pf); + + ice_request_fw(pf); + + /* if ice_request_fw fails, ICE_FLAG_ADV_FEATURES bit won't be + * set in pf->state, which will cause ice_is_safe_mode to return + * true + */ + if (ice_is_safe_mode(pf)) { + /* we already got function/device capabilities but these don't + * reflect what the driver needs to do in safe mode. Instead of + * adding conditional logic everywhere to ignore these + * device/function capabilities, override them. + */ + ice_set_safe_mode_caps(&pf->hw); + } + + err = ice_init_pf(pf); + if (err) { + dev_err(dev, "ice_init_pf failed: %d\n", err); + goto err_init_pf; + } + + pf->hw.udp_tunnel_nic.set_port = ice_udp_tunnel_set_port; + pf->hw.udp_tunnel_nic.unset_port = ice_udp_tunnel_unset_port; pf->hw.udp_tunnel_nic.flags = UDP_TUNNEL_NIC_INFO_MAY_SLEEP; pf->hw.udp_tunnel_nic.shared = &pf->hw.udp_tunnel_shared; - i = 0; if (pf->hw.tnl.valid_count[TNL_VXLAN]) { - pf->hw.udp_tunnel_nic.tables[i].n_entries = + pf->hw.udp_tunnel_nic.tables[0].n_entries = pf->hw.tnl.valid_count[TNL_VXLAN]; - pf->hw.udp_tunnel_nic.tables[i].tunnel_types = + pf->hw.udp_tunnel_nic.tables[0].tunnel_types = UDP_TUNNEL_TYPE_VXLAN; - i++; } if (pf->hw.tnl.valid_count[TNL_GENEVE]) { - pf->hw.udp_tunnel_nic.tables[i].n_entries = + pf->hw.udp_tunnel_nic.tables[1].n_entries = pf->hw.tnl.valid_count[TNL_GENEVE]; - pf->hw.udp_tunnel_nic.tables[i].tunnel_types = + pf->hw.udp_tunnel_nic.tables[1].tunnel_types = UDP_TUNNEL_TYPE_GENEVE; - i++; - } - - pf->num_alloc_vsi = hw->func_caps.guar_num_vsi; - if (!pf->num_alloc_vsi) { - err = -EIO; - goto err_init_pf_unroll; - } - if (pf->num_alloc_vsi > UDP_TUNNEL_NIC_MAX_SHARING_DEVICES) { - dev_warn(&pf->pdev->dev, - "limiting the VSI count due to UDP tunnel limitation %d > %d\n", - pf->num_alloc_vsi, UDP_TUNNEL_NIC_MAX_SHARING_DEVICES); - pf->num_alloc_vsi = UDP_TUNNEL_NIC_MAX_SHARING_DEVICES; - } - - pf->vsi = devm_kcalloc(dev, pf->num_alloc_vsi, sizeof(*pf->vsi), - GFP_KERNEL); - if (!pf->vsi) { - err = -ENOMEM; - goto err_init_pf_unroll; } err = ice_init_interrupt_scheme(pf); if (err) { dev_err(dev, "ice_init_interrupt_scheme failed: %d\n", err); err = -EIO; - goto err_init_vsi_unroll; + goto err_init_interrupt_scheme; } /* In case of MSIX we are going to setup the misc vector right here @@ -4770,49 +4705,94 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) err = ice_req_irq_msix_misc(pf); if (err) { dev_err(dev, "setup of misc vector failed: %d\n", err); - goto err_init_interrupt_unroll; + goto err_req_irq_msix_misc; } - /* create switch struct for the switch element created by FW on boot */ - pf->first_sw = devm_kzalloc(dev, sizeof(*pf->first_sw), GFP_KERNEL); - if (!pf->first_sw) { - err = -ENOMEM; - goto err_msix_misc_unroll; - } + return 0; - if (hw->evb_veb) - pf->first_sw->bridge_mode = BRIDGE_MODE_VEB; - else - pf->first_sw->bridge_mode = BRIDGE_MODE_VEPA; +err_req_irq_msix_misc: + ice_clear_interrupt_scheme(pf); +err_init_interrupt_scheme: + ice_deinit_pf(pf); +err_init_pf: + ice_deinit_hw(&pf->hw); + return err; +} - pf->first_sw->pf = pf; +static void ice_deinit_dev(struct ice_pf *pf) +{ + ice_free_irq_msix_misc(pf); + ice_clear_interrupt_scheme(pf); + ice_deinit_pf(pf); + ice_deinit_hw(&pf->hw); +} - /* record the sw_id available for later use */ - pf->first_sw->sw_id = hw->port_info->sw_id; +static void ice_init_features(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); - err = ice_setup_pf_sw(pf); - if (err) { - dev_err(dev, "probe failed due to setup PF switch: %d\n", err); - goto err_alloc_sw_unroll; - } + if (ice_is_safe_mode(pf)) + return; - clear_bit(ICE_SERVICE_DIS, pf->state); + /* initialize DDP driven features */ + if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags)) + ice_ptp_init(pf); - /* tell the firmware we are up */ - err = ice_send_version(pf); - if (err) { - dev_err(dev, "probe failed sending driver version %s. error: %d\n", - UTS_RELEASE, err); - goto err_send_version_unroll; + if (ice_is_feature_supported(pf, ICE_F_GNSS)) + ice_gnss_init(pf); + + /* Note: Flow director init failure is non-fatal to load */ + if (ice_init_fdir(pf)) + dev_err(dev, "could not initialize flow director\n"); + + /* Note: DCB init failure is non-fatal to load */ + if (ice_init_pf_dcb(pf, false)) { + clear_bit(ICE_FLAG_DCB_CAPABLE, pf->flags); + clear_bit(ICE_FLAG_DCB_ENA, pf->flags); + } else { + ice_cfg_lldp_mib_change(&pf->hw, true); } - /* since everything is good, start the service timer */ - mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); + if (ice_init_lag(pf)) + dev_warn(dev, "Failed to init link aggregation support\n"); +} + +static void ice_deinit_features(struct ice_pf *pf) +{ + ice_deinit_lag(pf); + if (test_bit(ICE_FLAG_DCB_CAPABLE, pf->flags)) + ice_cfg_lldp_mib_change(&pf->hw, false); + ice_deinit_fdir(pf); + if (ice_is_feature_supported(pf, ICE_F_GNSS)) + ice_gnss_exit(pf); + if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags)) + ice_ptp_release(pf); +} + +static void ice_init_wakeup(struct ice_pf *pf) +{ + /* Save wakeup reason register for later use */ + pf->wakeup_reason = rd32(&pf->hw, PFPM_WUS); + + /* check for a power management event */ + ice_print_wake_reason(pf); + + /* clear wake status, all bits */ + wr32(&pf->hw, PFPM_WUS, U32_MAX); + + /* Disable WoL at init, wait for user to enable */ + device_set_wakeup_enable(ice_pf_to_dev(pf), false); +} + +static int ice_init_link(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + int err; err = ice_init_link_events(pf->hw.port_info); if (err) { dev_err(dev, "ice_init_link_events failed: %d\n", err); - goto err_send_version_unroll; + return err; } /* not a fatal error if this fails */ @@ -4848,91 +4828,336 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) set_bit(ICE_FLAG_NO_MEDIA, pf->flags); } - ice_verify_cacheline_size(pf); + return err; +} - /* Save wakeup reason register for later use */ - pf->wakeup_reason = rd32(hw, PFPM_WUS); +static int ice_init_pf_sw(struct ice_pf *pf) +{ + bool dvm = ice_is_dvm_ena(&pf->hw); + struct ice_vsi *vsi; + int err; - /* check for a power management event */ - ice_print_wake_reason(pf); + /* create switch struct for the switch element created by FW on boot */ + pf->first_sw = kzalloc(sizeof(*pf->first_sw), GFP_KERNEL); + if (!pf->first_sw) + return -ENOMEM; - /* clear wake status, all bits */ - wr32(hw, PFPM_WUS, U32_MAX); + if (pf->hw.evb_veb) + pf->first_sw->bridge_mode = BRIDGE_MODE_VEB; + else + pf->first_sw->bridge_mode = BRIDGE_MODE_VEPA; - /* Disable WoL at init, wait for user to enable */ - device_set_wakeup_enable(dev, false); + pf->first_sw->pf = pf; - if (ice_is_safe_mode(pf)) { - ice_set_safe_mode_vlan_cfg(pf); - goto probe_done; + /* record the sw_id available for later use */ + pf->first_sw->sw_id = pf->hw.port_info->sw_id; + + err = ice_aq_set_port_params(pf->hw.port_info, dvm, NULL); + if (err) + goto err_aq_set_port_params; + + vsi = ice_pf_vsi_setup(pf, pf->hw.port_info); + if (!vsi) { + err = -ENOMEM; + goto err_pf_vsi_setup; } - /* initialize DDP driven features */ - if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags)) - ice_ptp_init(pf); + return 0; - if (ice_is_feature_supported(pf, ICE_F_GNSS)) - ice_gnss_init(pf); +err_pf_vsi_setup: +err_aq_set_port_params: + kfree(pf->first_sw); + return err; +} - /* Note: Flow director init failure is non-fatal to load */ - if (ice_init_fdir(pf)) - dev_err(dev, "could not initialize flow director\n"); +static void ice_deinit_pf_sw(struct ice_pf *pf) +{ + struct ice_vsi *vsi = ice_get_main_vsi(pf); - /* Note: DCB init failure is non-fatal to load */ - if (ice_init_pf_dcb(pf, false)) { - clear_bit(ICE_FLAG_DCB_CAPABLE, pf->flags); - clear_bit(ICE_FLAG_DCB_ENA, pf->flags); - } else { - ice_cfg_lldp_mib_change(&pf->hw, true); + if (!vsi) + return; + + ice_vsi_release(vsi); + kfree(pf->first_sw); +} + +static int ice_alloc_vsis(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + + pf->num_alloc_vsi = pf->hw.func_caps.guar_num_vsi; + if (!pf->num_alloc_vsi) + return -EIO; + + if (pf->num_alloc_vsi > UDP_TUNNEL_NIC_MAX_SHARING_DEVICES) { + dev_warn(dev, + "limiting the VSI count due to UDP tunnel limitation %d > %d\n", + pf->num_alloc_vsi, UDP_TUNNEL_NIC_MAX_SHARING_DEVICES); + pf->num_alloc_vsi = UDP_TUNNEL_NIC_MAX_SHARING_DEVICES; } - if (ice_init_lag(pf)) - dev_warn(dev, "Failed to init link aggregation support\n"); + pf->vsi = devm_kcalloc(dev, pf->num_alloc_vsi, sizeof(*pf->vsi), + GFP_KERNEL); + if (!pf->vsi) + return -ENOMEM; - /* print PCI link speed and width */ - pcie_print_link_status(pf->pdev); + return 0; +} -probe_done: - err = ice_register_netdev(pf); - if (err) - goto err_netdev_reg; +static void ice_dealloc_vsis(struct ice_pf *pf) +{ + pf->num_alloc_vsi = 0; + devm_kfree(ice_pf_to_dev(pf), pf->vsi); + pf->vsi = NULL; +} + +static int ice_init_devlink(struct ice_pf *pf) +{ + int err; err = ice_devlink_register_params(pf); if (err) - goto err_netdev_reg; + return err; + + ice_devlink_init_regions(pf); + ice_devlink_register(pf); + + return 0; +} + +static void ice_deinit_devlink(struct ice_pf *pf) +{ + ice_devlink_unregister(pf); + ice_devlink_destroy_regions(pf); + ice_devlink_unregister_params(pf); +} + +static int ice_init(struct ice_pf *pf) +{ + int err; + + err = ice_init_dev(pf); + if (err) + return err; + + err = ice_alloc_vsis(pf); + if (err) + goto err_alloc_vsis; + + err = ice_init_pf_sw(pf); + if (err) + goto err_init_pf_sw; + + ice_init_wakeup(pf); + + err = ice_init_link(pf); + if (err) + goto err_init_link; + + err = ice_send_version(pf); + if (err) + goto err_init_link; + + ice_verify_cacheline_size(pf); + + if (ice_is_safe_mode(pf)) + ice_set_safe_mode_vlan_cfg(pf); + else + /* print PCI link speed and width */ + pcie_print_link_status(pf->pdev); /* ready to go, so clear down state bit */ clear_bit(ICE_DOWN, pf->state); + clear_bit(ICE_SERVICE_DIS, pf->state); + + /* since everything is good, start the service timer */ + mod_timer(&pf->serv_tmr, round_jiffies(jiffies + pf->serv_tmr_period)); + + return 0; + +err_init_link: + ice_deinit_pf_sw(pf); +err_init_pf_sw: + ice_dealloc_vsis(pf); +err_alloc_vsis: + ice_deinit_dev(pf); + return err; +} + +static void ice_deinit(struct ice_pf *pf) +{ + set_bit(ICE_SERVICE_DIS, pf->state); + set_bit(ICE_DOWN, pf->state); + + ice_deinit_dev(pf); + ice_dealloc_vsis(pf); + ice_deinit_pf_sw(pf); +} + +/** + * ice_load - load pf by init hw and starting VSI + * @pf: pointer to the pf instance + */ +int ice_load(struct ice_pf *pf) +{ + struct ice_vsi *vsi; + int err; + + err = ice_reset(&pf->hw, ICE_RESET_PFR); + if (err) + return err; + + err = ice_init_dev(pf); + if (err) + return err; + + vsi = ice_get_main_vsi(pf); + err = ice_vsi_cfg(vsi, NULL, NULL); + if (err) + goto err_vsi_cfg; + + err = ice_start_eth(ice_get_main_vsi(pf)); + if (err) + goto err_start_eth; + err = ice_init_rdma(pf); + if (err) + goto err_init_rdma; + + ice_init_features(pf); + ice_service_task_restart(pf); + + clear_bit(ICE_DOWN, pf->state); + + return 0; + +err_init_rdma: + ice_vsi_close(ice_get_main_vsi(pf)); +err_start_eth: + ice_vsi_decfg(ice_get_main_vsi(pf)); +err_vsi_cfg: + ice_deinit_dev(pf); + return err; +} + +/** + * ice_unload - unload pf by stopping VSI and deinit hw + * @pf: pointer to the pf instance + */ +void ice_unload(struct ice_pf *pf) +{ + ice_deinit_features(pf); + ice_deinit_rdma(pf); + ice_vsi_close(ice_get_main_vsi(pf)); + ice_vsi_decfg(ice_get_main_vsi(pf)); + ice_deinit_dev(pf); +} + +/** + * ice_probe - Device initialization routine + * @pdev: PCI device information struct + * @ent: entry in ice_pci_tbl + * + * Returns 0 on success, negative on failure + */ +static int +ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent) +{ + struct device *dev = &pdev->dev; + struct ice_pf *pf; + struct ice_hw *hw; + int err; + + if (pdev->is_virtfn) { + dev_err(dev, "can't probe a virtual function\n"); + return -EINVAL; + } + + /* this driver uses devres, see + * Documentation/driver-api/driver-model/devres.rst + */ + err = pcim_enable_device(pdev); + if (err) + return err; + + err = pcim_iomap_regions(pdev, BIT(ICE_BAR0), dev_driver_string(dev)); if (err) { - dev_err(dev, "Failed to initialize RDMA: %d\n", err); - err = -EIO; - goto err_devlink_reg_param; + dev_err(dev, "BAR0 I/O map error %d\n", err); + return err; } - ice_devlink_register(pf); - return 0; + pf = ice_allocate_pf(dev); + if (!pf) + return -ENOMEM; -err_devlink_reg_param: - ice_devlink_unregister_params(pf); -err_netdev_reg: -err_send_version_unroll: - ice_vsi_release_all(pf); -err_alloc_sw_unroll: - set_bit(ICE_SERVICE_DIS, pf->state); + /* initialize Auxiliary index to invalid value */ + pf->aux_idx = -1; + + /* set up for high or low DMA */ + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64)); + if (err) { + dev_err(dev, "DMA configuration failed: 0x%x\n", err); + return err; + } + + pci_enable_pcie_error_reporting(pdev); + pci_set_master(pdev); + + pf->pdev = pdev; + pci_set_drvdata(pdev, pf); set_bit(ICE_DOWN, pf->state); - devm_kfree(dev, pf->first_sw); -err_msix_misc_unroll: - ice_free_irq_msix_misc(pf); -err_init_interrupt_unroll: - ice_clear_interrupt_scheme(pf); -err_init_vsi_unroll: - devm_kfree(dev, pf->vsi); -err_init_pf_unroll: - ice_deinit_pf(pf); - ice_devlink_destroy_regions(pf); - ice_deinit_hw(hw); -err_exit_unroll: + /* Disable service task until DOWN bit is cleared */ + set_bit(ICE_SERVICE_DIS, pf->state); + + hw = &pf->hw; + hw->hw_addr = pcim_iomap_table(pdev)[ICE_BAR0]; + pci_save_state(pdev); + + hw->back = pf; + hw->port_info = NULL; + hw->vendor_id = pdev->vendor; + hw->device_id = pdev->device; + pci_read_config_byte(pdev, PCI_REVISION_ID, &hw->revision_id); + hw->subsystem_vendor_id = pdev->subsystem_vendor; + hw->subsystem_device_id = pdev->subsystem_device; + hw->bus.device = PCI_SLOT(pdev->devfn); + hw->bus.func = PCI_FUNC(pdev->devfn); + ice_set_ctrlq_len(hw); + + pf->msg_enable = netif_msg_init(debug, ICE_DFLT_NETIF_M); + +#ifndef CONFIG_DYNAMIC_DEBUG + if (debug < -1) + hw->debug_mask = debug; +#endif + + err = ice_init(pf); + if (err) + goto err_init; + + err = ice_init_eth(pf); + if (err) + goto err_init_eth; + + err = ice_init_rdma(pf); + if (err) + goto err_init_rdma; + + err = ice_init_devlink(pf); + if (err) + goto err_init_devlink; + + ice_init_features(pf); + + return 0; + +err_init_devlink: + ice_deinit_rdma(pf); +err_init_rdma: + ice_deinit_eth(pf); +err_init_eth: + ice_deinit(pf); +err_init: pci_disable_pcie_error_reporting(pdev); pci_disable_device(pdev); return err; @@ -5008,7 +5233,7 @@ static void ice_remove(struct pci_dev *pdev) struct ice_pf *pf = pci_get_drvdata(pdev); int i; - ice_devlink_unregister(pf); + ice_deinit_devlink(pf); for (i = 0; i < ICE_MAX_RESET_WAIT; i++) { if (!ice_is_reset_in_progress(pf->state)) break; @@ -5025,30 +5250,22 @@ static void ice_remove(struct pci_dev *pdev) ice_service_task_stop(pf); ice_aq_cancel_waiting_tasks(pf); - ice_deinit_rdma(pf); - ice_devlink_unregister_params(pf); set_bit(ICE_DOWN, pf->state); - ice_deinit_lag(pf); - if (test_bit(ICE_FLAG_PTP_SUPPORTED, pf->flags)) - ice_ptp_release(pf); - if (ice_is_feature_supported(pf, ICE_F_GNSS)) - ice_gnss_exit(pf); + ice_deinit_features(pf); + ice_deinit_rdma(pf); if (!ice_is_safe_mode(pf)) ice_remove_arfs(pf); ice_setup_mc_magic_wake(pf); ice_vsi_release_all(pf); mutex_destroy(&(&pf->hw)->fdir_fltr_lock); ice_set_wake(pf); - ice_free_irq_msix_misc(pf); + ice_deinit_dev(pf); ice_for_each_vsi(pf, i) { if (!pf->vsi[i]) continue; ice_vsi_free_q_vectors(pf->vsi[i]); } - ice_deinit_pf(pf); - ice_devlink_destroy_regions(pf); - ice_deinit_hw(&pf->hw); /* Issue a PFR as part of the prescribed driver unload flow. Do not * do it via ice_schedule_reset() since there is no need to rebuild @@ -5056,7 +5273,6 @@ static void ice_remove(struct pci_dev *pdev) */ ice_reset(&pf->hw, ICE_RESET_PFR); pci_wait_for_pending_transaction(pdev); - ice_clear_interrupt_scheme(pf); pci_disable_pcie_error_reporting(pdev); pci_disable_device(pdev); } From patchwork Mon Nov 14 12:57:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703587 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=2mthQD8H; dkim-atps=neutral Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVG1N4Vz1yhv for ; Tue, 15 Nov 2022 00:12:38 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5A824415C7; Mon, 14 Nov 2022 13:12:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5A824415C7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431556; bh=ypR3RkunOmlQrQb4xaMqRRxvFKN1+pTQHcFCrP3Ei5E=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=2mthQD8HiH/RBSA7wLAiKNJDzlPk5EVqti6WWXyfrj2hCL2mfysgX4s13ZWf2lkgJ k8/2I6SGDfSCzcPMR8VyO/xWQHmjokNqAK1nlReQeAs5f+exB12ACkDM1aaj6ZhTsg JqcGfQCvqii0L3JTqU7MxYIFhYzFqtGhUpFcrT/8wGX/lmmtAFvynqaj+ObBAcPQCT hsCPXcQLJkpLj3HoVIs8c0FTkBs2QiaRFjTb/i56Hnc98zixBOJH4zwegLJm5y7/bJ fQtf/+7+Yb8w4UoLBuYpF2ZZ3U7Hm3fA1tChYV/UB5v5sEAfir/Nr/eBJKZFcbitsE Pyz2wk5sCAstQ== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4GtVqO_PVyWo; Mon, 14 Nov 2022 13:12:35 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp4.osuosl.org (Postfix) with ESMTP id 370A6415D7; Mon, 14 Nov 2022 13:12:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 370A6415D7 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 89B171BF379 for ; Mon, 14 Nov 2022 13:12:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 6463E81826 for ; Mon, 14 Nov 2022 13:12:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6463E81826 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3KNVPgZEKdO4 for ; Mon, 14 Nov 2022 13:12:27 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3B4AB817D3 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id 3B4AB817D3 for ; Mon, 14 Nov 2022 13:12:27 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766728" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766728" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306013" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306013" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:22 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:49 +0100 Message-Id: <20221114125755.13659-8-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431547; x=1699967547; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=unS16nVWy3isgLs85nEntpNArvPSZg+8ZBDHAbDTZtc=; b=NkCU/8CdQD/DFnNQCnf4mLUMz11a4P5b5Sb8cwnkbiW1wYCva+aPexYC NzFJXvcF8UreOmN9vnIOrAKrPgpu5osOAeYXITFUmtnPwpKTDBd4zGQOx UMUASHnD3C233uXiu86u0FsGf2cM50JsiJFfC+Kb4+AyIkYDw+DKefi8e f5xNWN07cDSnchOETMp9zjfRuwIvY4eBq2QGwUtd7uWPqREu4TnGs7+YT ePFVNOGN5jiSQfZxwxifjgqgCcExO+jLrwfWwv4XoYm55vqOK2wxEojXt 8BPMW3kumB/wmsGE54lR4YryBuWcfcYHdkmOtkYReqRGfqfuui9GcsYTI w==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=NkCU/8Cd Subject: [Intel-wired-lan] [PATCH net-next 07/13] ice: sync netdev filters after clearing VSI X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" In driver reload path the netdev isn't removed, but VSI is. Remove filters on netdev right after removing them on VSI. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_fltr.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/net/ethernet/intel/ice/ice_fltr.c b/drivers/net/ethernet/intel/ice/ice_fltr.c index 40e678cfb507..aff7a141c30d 100644 --- a/drivers/net/ethernet/intel/ice/ice_fltr.c +++ b/drivers/net/ethernet/intel/ice/ice_fltr.c @@ -208,6 +208,11 @@ static int ice_fltr_remove_eth_list(struct ice_vsi *vsi, struct list_head *list) void ice_fltr_remove_all(struct ice_vsi *vsi) { ice_remove_vsi_fltr(&vsi->back->hw, vsi->idx); + /* sync netdev filters if exist */ + if (vsi->netdev) { + __dev_uc_unsync(vsi->netdev, NULL); + __dev_mc_unsync(vsi->netdev, NULL); + } } /** From patchwork Mon Nov 14 12:57:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703588 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=a1SFlCvm; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVK6tlLz1yhv for ; Tue, 15 Nov 2022 00:12:41 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 281EB8186B; Mon, 14 Nov 2022 13:12:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 281EB8186B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431560; bh=XJpx1CrGSAFYHNK7lkabGs/rWMN/EZNCB3rBNqlVuSo=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=a1SFlCvmjkIT9OtH9i9N9z4qNZdfAh6gzNhQGiPsbsslGwF90mkhuQ3LeJCcci4i8 HyCN8ChQActR7kPqKPpCaCjkDpL+AguJhMirlKVK4wNF3iUUJ0PJ/E+wLL29KYU1Cm 2B80HqhzizWBdZsO4A6mVgh4NMLmwb3E7MGXL14bo8fquJrdYW/gZGjkEW2HVCEhU+ DrdyTVI5rQjrSDAJqgWwaLjX0zptqJyvAiPfNCrxodX6F04n2o92zA/w/EqOWx9uvW zTEZuCMsznBusAoUmHr/HtNhnJqCVL7agG5cBNZdvNMGyEHdwKBd1uK1nXD/vMBboY DC6K4rEnp8l5g== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EgqKYzc7iDRD; Mon, 14 Nov 2022 13:12:39 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 0E3BC81886; Mon, 14 Nov 2022 13:12:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0E3BC81886 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id E15C81BF379 for ; Mon, 14 Nov 2022 13:12:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C829681987 for ; Mon, 14 Nov 2022 13:12:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C829681987 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3laogJvAWmJk for ; Mon, 14 Nov 2022 13:12:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 37B7E817FF Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id 37B7E817FF for ; Mon, 14 Nov 2022 13:12:32 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766737" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766737" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306029" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306029" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:27 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:50 +0100 Message-Id: <20221114125755.13659-9-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431552; x=1699967552; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BoNdE6LmPMBapYPb9imTZwp+5J8dOM4lJJ0qwSGKJOQ=; b=WBnw9U0nuktbbwQ56eMb//Wprfypnx79JSNhAfF7aLH9MyBQtcgpTyhU o+JNSB0moVouaZJUFnoPFi3kCv1hgoDEwUIar1qpSyRNkWCeqCwALlwCb GRCq2lmhxie1awLlElZIK664lcfELX1DKWUdRBtDu3x3dA0jbWuhqqTAL yIDD7rNTfzltvC7PIrzdL7Kvpit1kLUlc6MfVhuNVahhI86YHxZF/2a8I ktoe1A10c56w31CyHnhM46I4TDPxy84AW8WvKckZd0h2vyGML0t9Jvw6k 1IbAINM6zG1Y1/s3jZ+BMHssmzcaR5Y5rBC1bd0Q7M8DnL7osSsu3z2AZ Q==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=WBnw9U0n Subject: [Intel-wired-lan] [PATCH net-next 08/13] ice: move VSI delete outside deconfig X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" In deconfig VSI shouldn't be deleted from hw. Rewrite VSI delete function to reflect that sometimes it is only needed to remove VSI from hw without freeing the memory: ice_vsi_delete() -> delete from HW and free memory ice_vsi_delete_from_hw() -> delete only from HW Value returned from ice_vsi_free() is never used. Change return type to void. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_lib.c | 28 +++++++++++------------ drivers/net/ethernet/intel/ice/ice_lib.h | 1 - drivers/net/ethernet/intel/ice/ice_main.c | 5 +--- 3 files changed, 14 insertions(+), 20 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index dc672af10c25..32da2da74056 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -282,10 +282,10 @@ static int ice_get_free_slot(void *array, int size, int curr) } /** - * ice_vsi_delete - delete a VSI from the switch + * ice_vsi_delete_from_hw - delete a VSI from the switch * @vsi: pointer to VSI being removed */ -void ice_vsi_delete(struct ice_vsi *vsi) +static void ice_vsi_delete_from_hw(struct ice_vsi *vsi) { struct ice_pf *pf = vsi->back; struct ice_vsi_ctx *ctxt; @@ -353,26 +353,21 @@ static void ice_vsi_free_arrays(struct ice_vsi *vsi) * * This deallocates the VSI's queue resources, removes it from the PF's * VSI array if necessary, and deallocates the VSI - * - * Returns 0 on success, negative on failure */ -int ice_vsi_free(struct ice_vsi *vsi) +static void ice_vsi_free(struct ice_vsi *vsi) { struct ice_pf *pf = NULL; struct device *dev; - if (!vsi) - return 0; - - if (!vsi->back) - return -EINVAL; + if (!vsi || !vsi->back) + return; pf = vsi->back; dev = ice_pf_to_dev(pf); if (!pf->vsi[vsi->idx] || pf->vsi[vsi->idx] != vsi) { dev_dbg(dev, "vsi does not exist at pf->vsi[%d]\n", vsi->idx); - return -EINVAL; + return; } mutex_lock(&pf->sw_mutex); @@ -384,8 +379,12 @@ int ice_vsi_free(struct ice_vsi *vsi) ice_vsi_free_arrays(vsi); mutex_unlock(&pf->sw_mutex); devm_kfree(dev, vsi); +} - return 0; +void ice_vsi_delete(struct ice_vsi *vsi) +{ + ice_vsi_delete_from_hw(vsi); + ice_vsi_free(vsi); } /** @@ -2681,7 +2680,7 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) unroll_alloc_q_vector: ice_vsi_free_q_vectors(vsi); unroll_vsi_init: - ice_vsi_delete(vsi); + ice_vsi_delete_from_hw(vsi); unroll_get_qs: ice_vsi_put_qs(vsi); unroll_vsi_alloc: @@ -2742,7 +2741,6 @@ void ice_vsi_decfg(struct ice_vsi *vsi) ice_vsi_clear_rings(vsi); ice_vsi_free_q_vectors(vsi); - ice_vsi_delete(vsi); ice_vsi_put_qs(vsi); ice_vsi_free_arrays(vsi); @@ -3140,7 +3138,7 @@ int ice_vsi_release(struct ice_vsi *vsi) * for ex: during rmmod. */ if (!ice_is_reset_in_progress(pf->state)) - ice_vsi_free(vsi); + ice_vsi_delete(vsi); return 0; } diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h index ad4d5314ca76..8905f8721a76 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_lib.h @@ -42,7 +42,6 @@ void ice_cfg_sw_lldp(struct ice_vsi *vsi, bool tx, bool create); int ice_set_link(struct ice_vsi *vsi, bool ena); void ice_vsi_delete(struct ice_vsi *vsi); -int ice_vsi_free(struct ice_vsi *vsi); int ice_vsi_cfg_tc(struct ice_vsi *vsi, u8 ena_tc); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index bf98334fadbc..49d29eb61c17 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -8555,12 +8555,9 @@ static void ice_remove_q_channels(struct ice_vsi *vsi, bool rem_fltr) /* clear the VSI from scheduler tree */ ice_rm_vsi_lan_cfg(ch->ch_vsi->port_info, ch->ch_vsi->idx); - /* Delete VSI from FW */ + /* Delete VSI from FW, PF and HW VSI arrays */ ice_vsi_delete(ch->ch_vsi); - /* Delete VSI from PF and HW VSI arrays */ - ice_vsi_free(ch->ch_vsi); - /* free the channel */ kfree(ch); } From patchwork Mon Nov 14 12:57:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703589 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=4gxPxT7N; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVQ0g1Kz1yhv for ; Tue, 15 Nov 2022 00:12:46 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id EC6B160DCA; Mon, 14 Nov 2022 13:12:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org EC6B160DCA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431564; bh=aiREGnDvupZ31Sa/D8nSORpiWROSY16VrB4JkKExxYo=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=4gxPxT7NqYn8Q9chVcJas8dG5U38A40rCSiUrgaVz5UPSJrbOeC6uhd9gwyPi/oHz dubvRrCfAOwm0iNcNeSz3QKAjBnGyFy60kqUifVnxwICv6yF069sDi8e8lopNCoaK+ u15WUn18QProUeGpaK0PvgoXGIgszI5/edplSCeQB7ZPbON5XlcdhC/SudLVzjFgJS p/fB2gi2NrBSSYZy8AOfcvrBJ1z2/bJ6mNue38Wu0PzzqCc2XFg2ynWhzNxgDwChfF YsQ/K2BX5N2dgLQGo/CtS6ZQscX7bz9DtCB1zUTWCok70uOCWEpLEo4iuSZHTQvhtL fcuOBwLEzFbWw== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dWKbx5k0rwcO; Mon, 14 Nov 2022 13:12:43 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id D348760DDF; Mon, 14 Nov 2022 13:12:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D348760DDF X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 127B11BF379 for ; Mon, 14 Nov 2022 13:12:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E244181886 for ; Mon, 14 Nov 2022 13:12:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E244181886 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q74U5Yr0o8eu for ; Mon, 14 Nov 2022 13:12:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 297E48186B Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id 297E48186B for ; Mon, 14 Nov 2022 13:12:37 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766756" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766756" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306044" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306044" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:32 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:51 +0100 Message-Id: <20221114125755.13659-10-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431557; x=1699967557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rgAPs7log2UWG8/xCvTZeDeVszLluOPvI71EOX9QBj8=; b=TH30E5rjjS0uSvp75uM0O9IPquN97Aifz6i28+moZ3GFYenuhh0Mn9eb Et7lKj9shHNmd9xOf52Po5ZNI15QKDfjwaRxg7Hhc09SR0uj6VwWF4I1o hHSAIoLdQdsBGIK3Co+w0Tr+Z2ZaDnJC76XNs5YupcuSK/P4Wh/2pqldZ 3s4WKrmvceBNLiBzxhAnE/q/rP+eGDH+HR5YY/wtb234dGKJgbyXQpgax BdUePaiZ5fDccov9pdd2/qyY+cfZDovUjtRibkoIdgYMfgasz5aVwG0gO DdJIR5Ac5vFjdoecGU2dHfAeJx7At2QXM/RMUowBFzzLCcFNYDGTTrKhD Q==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=TH30E5rj Subject: [Intel-wired-lan] [PATCH net-next 09/13] ice: update VSI instead of init in some case X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" ice_vsi_cfg() is called from different contexts: 1) VSI exsist in HW, but it is reconfigured, because of changing queues for example -> update instead of init should be used 2) VSI doesn't exsist, because rest has happened -> init command should be sent To support both cases pass boolean value which will store information what type of command has to be sent to HW. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_lib.c | 16 ++++++++++------ drivers/net/ethernet/intel/ice/ice_lib.h | 4 ++-- drivers/net/ethernet/intel/ice/ice_main.c | 2 +- 3 files changed, 13 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index 32da2da74056..bc04f2b9f8a2 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -2551,9 +2551,11 @@ static int ice_vsi_cfg_tc_lan(struct ice_pf *pf, struct ice_vsi *vsi) * @vf: pointer to VF to which this VSI connects. This field is used primarily * for the ICE_VSI_VF type. Other VSI types should pass NULL. * @ch: ptr to channel + * @init_vsi: is this an initialization or a reconfigure of the VSI */ static int -ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) +ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch, + int init_vsi) { struct device *dev = ice_pf_to_dev(vsi->back); struct ice_pf *pf = vsi->back; @@ -2580,7 +2582,7 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) ice_vsi_set_tc_cfg(vsi); /* create the VSI */ - ret = ice_vsi_init(vsi, true); + ret = ice_vsi_init(vsi, init_vsi); if (ret) goto unroll_get_qs; @@ -2694,12 +2696,14 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) * @vf: pointer to VF to which this VSI connects. This field is used primarily * for the ICE_VSI_VF type. Other VSI types should pass NULL. * @ch: ptr to channel + * @init_vsi: is this an initialization or a reconfigure of the VSI */ -int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch) +int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, struct ice_channel *ch, + int init_vsi) { int ret; - ret = ice_vsi_cfg_def(vsi, vf, ch); + ret = ice_vsi_cfg_def(vsi, vf, ch, init_vsi); if (ret) return ret; @@ -2796,7 +2800,7 @@ ice_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi, return NULL; } - ret = ice_vsi_cfg(vsi, vf, ch); + ret = ice_vsi_cfg(vsi, vf, ch, ICE_VSI_FLAG_INIT); if (ret) goto err_vsi_cfg; @@ -3286,7 +3290,7 @@ int ice_vsi_rebuild(struct ice_vsi *vsi, int init_vsi) prev_num_q_vectors = ice_vsi_rebuild_get_coalesce(vsi, coalesce); ice_vsi_decfg(vsi); - ret = ice_vsi_cfg_def(vsi, vsi->vf, vsi->ch); + ret = ice_vsi_cfg_def(vsi, vsi->vf, vsi->ch, init_vsi); if (ret) goto err_vsi_cfg; diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h index 8905f8721a76..b76f05e1f8a3 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.h +++ b/drivers/net/ethernet/intel/ice/ice_lib.h @@ -60,8 +60,6 @@ int ice_vsi_release(struct ice_vsi *vsi); void ice_vsi_close(struct ice_vsi *vsi); -int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, - struct ice_channel *ch); int ice_ena_vsi(struct ice_vsi *vsi, bool locked); void ice_vsi_decfg(struct ice_vsi *vsi); @@ -75,6 +73,8 @@ ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id); #define ICE_VSI_FLAG_INIT BIT(0) #define ICE_VSI_FLAG_NO_INIT 0 int ice_vsi_rebuild(struct ice_vsi *vsi, int init_vsi); +int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vf *vf, + struct ice_channel *ch, int init_vsi); bool ice_is_reset_in_progress(unsigned long *state); int ice_wait_for_reset(struct ice_pf *pf, unsigned long timeout); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 49d29eb61c17..9766032e95ec 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -5012,7 +5012,7 @@ int ice_load(struct ice_pf *pf) return err; vsi = ice_get_main_vsi(pf); - err = ice_vsi_cfg(vsi, NULL, NULL); + err = ice_vsi_cfg(vsi, NULL, NULL, ICE_VSI_FLAG_INIT); if (err) goto err_vsi_cfg; From patchwork Mon Nov 14 12:57:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703590 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=gvsxvJE4; dkim-atps=neutral Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVT2j3xz1yhv for ; Tue, 15 Nov 2022 00:12:49 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A38B281926; Mon, 14 Nov 2022 13:12:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A38B281926 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431567; bh=2wW+V6vTT5nkWD0W+6nA1vE1LlLEX7FK3cJqY1L3Gls=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=gvsxvJE4ID2XMtcC2KySW8N9glMy60V6CFbhkNby0OvDlH69MXYw5tKODTBWTZ55B AMwAJLWH++H+sq7CfnBVHvjBJEBl5G7mW0TZO38pg+VJOr7tFxazS4hbnkAqHZLKZq OEDbee+IsDkVKmyG/j1mRdW/jSeTRtAQiZhOk4iAlzly2ztvwxRqxKq+flHtzg68t+ Lv67N/HeTJfbUwHLB9TYCyPuFZkzKgeArLpFUEJUq23yK0wFrf8u3PYyWyHmrJozw+ sq9p+bYujJNREvGl/4YyNEtv+73dhYveX1CsDv6THUP4qkHmdkY4oDZl9P57NC019N miHjZUxEgA8PQ== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BqbTZaieuQJo; Mon, 14 Nov 2022 13:12:46 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 9B4C38190B; Mon, 14 Nov 2022 13:12:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9B4C38190B X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 221C31BF379 for ; Mon, 14 Nov 2022 13:12:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C2E4B818D4 for ; Mon, 14 Nov 2022 13:12:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C2E4B818D4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g_xzdwavlKrC for ; Mon, 14 Nov 2022 13:12:42 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E95A3817FF Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id E95A3817FF for ; Mon, 14 Nov 2022 13:12:41 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766772" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766772" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306051" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306051" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:37 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:52 +0100 Message-Id: <20221114125755.13659-11-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431561; x=1699967561; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/CSUny8U47MerrLf+/3eVAxaY7Q88qyef3AfZpl/+Sc=; b=l4iv71jszU8lg9p+33fviqSMHGVHgK3r5CVZ8x1gBMIY792OqbGmFzHQ +7ZxZeWOJFL9AQ7MlL4DqSmT5U2WVuHYR7aYKwPICfdJb1wB/Lril0CM5 yN0YWICOSJWWtbgT0qRSDddgiGUr30DnxziSxTQz/5e3zmLsduEDMWuZQ 6NCKG0YOCvWhAc6CLIVn/7O5Dpjv5CW7D7B9+lygNWx4oIcA5fb0opvd4 l5L5M0eUTccwX8PfXhUVmf4Ob4uF+muAHdbwi/7aQL5YwZttGg5hpF7g3 08O9ssUrlVwa9i9E59gTljzRk6FOmYDEEzM9WyRiCJyztcuEmk08q5ImK Q==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=l4iv71js Subject: [Intel-wired-lan] [PATCH net-next 10/13] ice: implement devlink reinit action X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Call ice_unload() and ice_load() in driver reinit flow. Block reinit when switchdev, ADQ or SRIOV is active. In reload path we don't want to rebuild all features. Ask user to remove them instead of quitely removing it in reload path. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice_devlink.c | 103 +++++++++++++++---- 1 file changed, 81 insertions(+), 22 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c index 455489e9457d..8f73c4008a56 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.c +++ b/drivers/net/ethernet/intel/ice/ice_devlink.c @@ -376,10 +376,7 @@ static int ice_devlink_info_get(struct devlink *devlink, /** * ice_devlink_reload_empr_start - Start EMP reset to activate new firmware - * @devlink: pointer to the devlink instance to reload - * @netns_change: if true, the network namespace is changing - * @action: the action to perform. Must be DEVLINK_RELOAD_ACTION_FW_ACTIVATE - * @limit: limits on what reload should do, such as not resetting + * @pf: pointer to the pf instance * @extack: netlink extended ACK structure * * Allow user to activate new Embedded Management Processor firmware by @@ -392,12 +389,9 @@ static int ice_devlink_info_get(struct devlink *devlink, * any source. */ static int -ice_devlink_reload_empr_start(struct devlink *devlink, bool netns_change, - enum devlink_reload_action action, - enum devlink_reload_limit limit, +ice_devlink_reload_empr_start(struct ice_pf *pf, struct netlink_ext_ack *extack) { - struct ice_pf *pf = devlink_priv(devlink); struct device *dev = ice_pf_to_dev(pf); struct ice_hw *hw = &pf->hw; u8 pending; @@ -435,12 +429,52 @@ ice_devlink_reload_empr_start(struct devlink *devlink, bool netns_change, return 0; } +/** + * ice_devlink_reload_down - prepare for reload + * @devlink: pointer to the devlink instance to reload + * @netns_change: if true, the network namespace is changing + * @action: the action to perform + * @limit: limits on what reload should do, such as not resetting + * @extack: netlink extended ACK structure + */ +static int +ice_devlink_reload_down(struct devlink *devlink, bool netns_change, + enum devlink_reload_action action, + enum devlink_reload_limit limit, + struct netlink_ext_ack *extack) +{ + struct ice_pf *pf = devlink_priv(devlink); + + switch (action) { + case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: + if (ice_is_eswitch_mode_switchdev(pf)) { + NL_SET_ERR_MSG_MOD(extack, + "Go to legacy mode before doing reinit\n"); + return -EOPNOTSUPP; + } + if (ice_is_adq_active(pf)) { + NL_SET_ERR_MSG_MOD(extack, + "Turn off ADQ before doing reinit\n"); + return -EOPNOTSUPP; + } + if (ice_has_vfs(pf)) { + NL_SET_ERR_MSG_MOD(extack, + "Remove all VFs before doing reinit\n"); + return -EOPNOTSUPP; + } + ice_unload(pf); + return 0; + case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: + return ice_devlink_reload_empr_start(pf, extack); + default: + WARN_ON(1); + return -EOPNOTSUPP; + } +} + /** * ice_devlink_reload_empr_finish - Wait for EMP reset to finish - * @devlink: pointer to the devlink instance reloading - * @action: the action requested - * @limit: limits imposed by userspace, such as not resetting - * @actions_performed: on return, indicate what actions actually performed + * @pf: pointer to the pf instance * @extack: netlink extended ACK structure * * Wait for driver to finish rebuilding after EMP reset is completed. This @@ -448,17 +482,11 @@ ice_devlink_reload_empr_start(struct devlink *devlink, bool netns_change, * for the driver's rebuild to complete. */ static int -ice_devlink_reload_empr_finish(struct devlink *devlink, - enum devlink_reload_action action, - enum devlink_reload_limit limit, - u32 *actions_performed, +ice_devlink_reload_empr_finish(struct ice_pf *pf, struct netlink_ext_ack *extack) { - struct ice_pf *pf = devlink_priv(devlink); int err; - *actions_performed = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE); - err = ice_wait_for_reset(pf, 60 * HZ); if (err) { NL_SET_ERR_MSG_MOD(extack, "Device still resetting after 1 minute"); @@ -713,12 +741,43 @@ ice_devlink_port_unsplit(struct devlink *devlink, struct devlink_port *port, return ice_devlink_port_split(devlink, port, 1, extack); } +/** + * ice_devlink_reload_up - do reload up after reinit + * @devlink: pointer to the devlink instance reloading + * @action: the action requested + * @limit: limits imposed by userspace, such as not resetting + * @actions_performed: on return, indicate what actions actually performed + * @extack: netlink extended ACK structure + */ +static int +ice_devlink_reload_up(struct devlink *devlink, + enum devlink_reload_action action, + enum devlink_reload_limit limit, + u32 *actions_performed, + struct netlink_ext_ack *extack) +{ + struct ice_pf *pf = devlink_priv(devlink); + + switch (action) { + case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: + *actions_performed = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + return ice_load(pf); + case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: + *actions_performed = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE); + return ice_devlink_reload_empr_finish(pf, extack); + default: + WARN_ON(1); + return -EOPNOTSUPP; + } +} + static const struct devlink_ops ice_devlink_ops = { .supported_flash_update_params = DEVLINK_SUPPORT_FLASH_UPDATE_OVERWRITE_MASK, - .reload_actions = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), + .reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) | + BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE), /* The ice driver currently does not support driver reinit */ - .reload_down = ice_devlink_reload_empr_start, - .reload_up = ice_devlink_reload_empr_finish, + .reload_down = ice_devlink_reload_down, + .reload_up = ice_devlink_reload_up, .port_split = ice_devlink_port_split, .port_unsplit = ice_devlink_port_unsplit, .eswitch_mode_get = ice_eswitch_mode_get, From patchwork Mon Nov 14 12:57:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703591 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=EX+TQFKS; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVZ0d2Gz1yhv for ; Tue, 15 Nov 2022 00:12:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A6EC540412; Mon, 14 Nov 2022 13:12:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A6EC540412 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431571; bh=WyRJw4tCthgGrNqpsHQ3vseIMy2nMfGu85avytoZROI=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=EX+TQFKSRYRfKZ3I9DmOVpdmNPTJRaP0/tGKcCR3/P4OAVhGbzMNXSqGtsIHvUC3b t4LBn0GR6bnaAlzYuZb5Kil48IqVJ8RLVddFijiPJdfEXzHL/XLlZ6hQNXo4//OLDE tLs8odyTF22LdLtzyGYBv03PU6HGyp/uPsFws/E03wDV5O2fP77dpsTc85O/HIJLK/ YPMBStwfHH6epGEaKBrO6IVqrw+Bsjrc3nDnjps/Z6NbLPswmA8Z3cw4yqYFeGdKSS rubb8LLAnd4eLCR4XlkZ4YnJlsy6P+1GBSgVIzt4UQvZz89b1J4G+r3EFNv/kJDEgn AIMsyU5dNFYlA== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RPUNWcnywy98; Mon, 14 Nov 2022 13:12:51 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp2.osuosl.org (Postfix) with ESMTP id A618040A07; Mon, 14 Nov 2022 13:12:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A618040A07 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id A2FF11BF379 for ; Mon, 14 Nov 2022 13:12:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7C95381987 for ; Mon, 14 Nov 2022 13:12:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7C95381987 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HAJiboCzDZpB for ; Mon, 14 Nov 2022 13:12:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org BF61681926 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by smtp1.osuosl.org (Postfix) with ESMTPS id BF61681926 for ; Mon, 14 Nov 2022 13:12:46 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="313766784" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="313766784" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306070" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306070" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:41 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:53 +0100 Message-Id: <20221114125755.13659-12-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431566; x=1699967566; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KpfNIjp3GXxfULG/7bGrVA214jUNUf6R3CPFkLj7qaQ=; b=nfAE+IPdGYGz9vHxRFFrGcFNdkharr5gr0tL3oS8Wa6TILwuTlAuBWE9 wOVk+8jT453CV6m5JdQtrJr4xR3KEzjIx197f6B6G82kjwKk2E5bInkgX 8LEFNsbu7E510Bd8fpfeN3zn1zQ7iJ9hFsYfJCair+9qgjy6OTiujG6jP WXx60gK2vepW+PoQnlCRhPO/B5v0ezU2olW+rnZpxq4H0SyVT6mrcOdjq a6l56vLk7rzeIukE3gKjA38LLJoHQGPE2zebK+YWGiEnb9alK7K+CzkYR ij6LM91Gc2O2GJJYm0/GVOooDQUA6vXvwbpB2LMljjc0xWIf/UPKmvR8f Q==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=nfAE+IPd Subject: [Intel-wired-lan] [PATCH net-next 11/13] ice: introduce eswitch capable flag X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" The flag is used to check if hardware support eswitch mode. Block going to switchdev if the flag is unset. It can be also use to turn the eswitch feature off to save MSI-X interrupt. Signed-off-by: Michal Swiatkowski --- drivers/net/ethernet/intel/ice/ice.h | 1 + drivers/net/ethernet/intel/ice/ice_eswitch.c | 6 ++++++ drivers/net/ethernet/intel/ice/ice_main.c | 5 ++++- 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index 62219f995cf2..df1f6d85cd43 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -501,6 +501,7 @@ enum ice_pf_flags { ICE_FLAG_PLUG_AUX_DEV, ICE_FLAG_MTU_CHANGED, ICE_FLAG_GNSS, /* GNSS successfully initialized */ + ICE_FLAG_ESWITCH_CAPABLE, /* switchdev mode can be supported */ ICE_PF_FLAGS_NBITS /* must be last */ }; diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c index f9f15acae90a..8532d5c47bad 100644 --- a/drivers/net/ethernet/intel/ice/ice_eswitch.c +++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c @@ -554,6 +554,12 @@ ice_eswitch_mode_set(struct devlink *devlink, u16 mode, return -EOPNOTSUPP; } + if (!test_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags)) { + dev_info(ice_pf_to_dev(pf), "There is no support for switchdev in hardware"); + NL_SET_ERR_MSG_MOD(extack, "There is no support for switchdev in hardware"); + return -EOPNOTSUPP; + } + switch (mode) { case DEVLINK_ESWITCH_MODE_LEGACY: dev_info(ice_pf_to_dev(pf), "PF %d changed eswitch mode to legacy", diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 9766032e95ec..0dfc427e623a 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -3739,8 +3739,10 @@ static void ice_set_pf_caps(struct ice_pf *pf) if (func_caps->common_cap.dcb) set_bit(ICE_FLAG_DCB_CAPABLE, pf->flags); clear_bit(ICE_FLAG_SRIOV_CAPABLE, pf->flags); + clear_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags); if (func_caps->common_cap.sr_iov_1_1) { set_bit(ICE_FLAG_SRIOV_CAPABLE, pf->flags); + set_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags); pf->vfs.num_supported = min_t(int, func_caps->num_allocd_vfs, ICE_MAX_SRIOV_VFS); } @@ -3881,7 +3883,8 @@ static int ice_ena_msix_range(struct ice_pf *pf) v_other += ICE_FDIR_MSIX; /* switchdev */ - v_other += ICE_ESWITCH_MSIX; + if (test_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags)) + v_other += ICE_ESWITCH_MSIX; v_wanted = v_other; From patchwork Mon Nov 14 12:57:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703592 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=dFik2U6h; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVg0hdpz1yhv for ; Tue, 15 Nov 2022 00:12:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2C7BB4019D; Mon, 14 Nov 2022 13:12:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2C7BB4019D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431577; bh=ob8Al1n50F+VSkZtfGq/j2mUk7QCoLtXDiZHuUa4RXQ=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=dFik2U6hyxvHHeH9egBJsrNEU84GTqo9H+zIXsxPCCShW5ESF8sJOJbf+r+UarOfX ETHVmGth+Zn0/8YRGC2PeGbCE3HzwXcYIjUKUa/ukuDMqLhcKLZLW5g64jjdVO+XeX 3RgQjfFWrYig2kXTiim+jVOlUkX87LwefCpx2UDTQYmO+OUc8ace9WkutJoBiWHwCy Efo9Cds9VsxTFPkNCj9RSn/dZFNfQUmnyquyqh1iZEexLEy0iNTdvj8E3dDltD3Y4x IXAUu4EVgkyzsEM/98hjX8yu032U+bceqx5mRpwGreh3kocR2Wv213VD5YAYaO2Qxy drEaSnusMzLDg== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7znEq7FT7MYo; Mon, 14 Nov 2022 13:12:55 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp2.osuosl.org (Postfix) with ESMTP id 8278840A63; Mon, 14 Nov 2022 13:12:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8278840A63 X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id E22961BF379 for ; Mon, 14 Nov 2022 13:12:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CAB8D60DF4 for ; Mon, 14 Nov 2022 13:12:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CAB8D60DF4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xpY9CnOwrprT for ; Mon, 14 Nov 2022 13:12:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6567A60DDF Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6567A60DDF for ; Mon, 14 Nov 2022 13:12:52 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="376229337" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="376229337" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:12:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306084" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306084" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:46 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:54 +0100 Message-Id: <20221114125755.13659-13-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431572; x=1699967572; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5CXANIjA6BWHHNb13wm9L8q6AtXb5fYM6qjRVPqs3A4=; b=bqTR0lysmGVlk5TXSBpU+SQ823eEtuuifHTDVPyKc7Ra0sfWEfLEqfqu hBTb5tsXp7ddZ/NVW+b+mC2B6p8ur5KKnG8+S1dRFdtMIdGT/y3nC3jSv m+YMBGyv9l+q760mMEmzga74+b3fhj4cQZVM3NiibjDXUYbVhcuSexhxG ghu8wzuGjhs7j0uJcc6V5guP3recSrA9L/Rbuwiti3XRe1Ary4jumR5oa BXWzSf/RhIoP5aTvD487k/G+568ot4G2Gkektr2tCsoLiNI7phT8R5Pjx xU3nes+cIswsuv28y/B3v7Nd5DgxAHu/5pKAVs6dGQlHa+4BT5+D86mjq g==; X-Mailman-Original-Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=bqTR0lys Subject: [Intel-wired-lan] [PATCH net-next 12/13] ice, irdma: prepare reservation of MSI-X to reload X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Move MSI-X number for LAN and RDMA to structure to have it in one place. Use req_msix to store how many MSI-X for each feature user requested. Structure msix is used to store the current number of MSI-X. The MSI-X number needs to be adjust if kernel doesn't support that many MSI-X or if hardware doesn't support it. Rewrite MSI-X adjustment function to use it in both cases. Use the same algorithm like previously. First allocate minimum MSI-X for each feature than if there is enough MSI-X distribute it equally between each one. Signed-off-by: Michal Swiatkowski Co-developed-by: Piotr Raczynski Signed-off-by: Piotr Raczynski --- drivers/infiniband/hw/irdma/main.c | 2 +- drivers/net/ethernet/intel/ice/ice.h | 16 +- drivers/net/ethernet/intel/ice/ice_ethtool.c | 4 +- drivers/net/ethernet/intel/ice/ice_idc.c | 6 +- drivers/net/ethernet/intel/ice/ice_lib.c | 8 +- drivers/net/ethernet/intel/ice/ice_main.c | 284 +++++++++++-------- 6 files changed, 189 insertions(+), 131 deletions(-) diff --git a/drivers/infiniband/hw/irdma/main.c b/drivers/infiniband/hw/irdma/main.c index 514453777e07..0a70b27b93a3 100644 --- a/drivers/infiniband/hw/irdma/main.c +++ b/drivers/infiniband/hw/irdma/main.c @@ -230,7 +230,7 @@ static void irdma_fill_device_info(struct irdma_device *iwdev, struct ice_pf *pf rf->gen_ops.unregister_qset = irdma_lan_unregister_qset; rf->hw.hw_addr = pf->hw.hw_addr; rf->pcidev = pf->pdev; - rf->msix_count = pf->num_rdma_msix; + rf->msix_count = pf->msix.rdma; rf->pf_id = pf->hw.pf_id; rf->msix_entries = &pf->msix_entries[pf->rdma_base_vector]; rf->default_vsi.vsi_idx = vsi->vsi_num; diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h index df1f6d85cd43..ef9ea43470bf 100644 --- a/drivers/net/ethernet/intel/ice/ice.h +++ b/drivers/net/ethernet/intel/ice/ice.h @@ -518,6 +518,14 @@ struct ice_agg_node { u8 valid; }; +struct ice_msix { + u16 all; /* All reserved MSI-X */ + u16 misc; /* MSI-X reserved for misc */ + u16 eth; /* MSI-X reserved for eth */ + u16 rdma; /* MSI-X reserved for RDMA */ + u16 vf; /* MSI-X reserved for VF */ +}; + struct ice_pf { struct pci_dev *pdev; @@ -531,6 +539,10 @@ struct ice_pf { /* OS reserved IRQ details */ struct msix_entry *msix_entries; struct ice_res_tracker *irq_tracker; + /* Store MSI-X requested by user via devlink */ + struct ice_msix req_msix; + /* Store currently used MSI-X */ + struct ice_msix msix; /* First MSIX vector used by SR-IOV VFs. Calculated by subtracting the * number of MSIX vectors needed for all SR-IOV VFs from the number of * MSIX vectors allowed on this PF. @@ -561,7 +573,6 @@ struct ice_pf { struct tty_driver *ice_gnss_tty_driver; struct tty_port *gnss_tty_port[ICE_GNSS_TTY_MINOR_DEVICES]; struct gnss_serial *gnss_serial[ICE_GNSS_TTY_MINOR_DEVICES]; - u16 num_rdma_msix; /* Total MSIX vectors for RDMA driver */ u16 rdma_base_vector; /* spinlock to protect the AdminQ wait list */ @@ -578,7 +589,6 @@ struct ice_pf { u16 num_avail_sw_msix; /* remaining MSIX SW vectors left unclaimed */ u16 max_pf_txqs; /* Total Tx queues PF wide */ u16 max_pf_rxqs; /* Total Rx queues PF wide */ - u16 num_lan_msix; /* Total MSIX vectors for base driver */ u16 num_lan_tx; /* num LAN Tx queues setup */ u16 num_lan_rx; /* num LAN Rx queues setup */ u16 next_vsi; /* Next free slot in pf->vsi[] - 0-based! */ @@ -934,7 +944,7 @@ void ice_unload(struct ice_pf *pf); */ static inline void ice_set_rdma_cap(struct ice_pf *pf) { - if (pf->hw.func_caps.common_cap.rdma && pf->num_rdma_msix) { + if (pf->hw.func_caps.common_cap.rdma && pf->msix.rdma) { set_bit(ICE_FLAG_RDMA_ENA, pf->flags); set_bit(ICE_FLAG_PLUG_AUX_DEV, pf->flags); } diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c index 1dfce48cfdc2..65f71da6bc70 100644 --- a/drivers/net/ethernet/intel/ice/ice_ethtool.c +++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c @@ -3513,7 +3513,7 @@ ice_get_ts_info(struct net_device *dev, struct ethtool_ts_info *info) */ static int ice_get_max_txq(struct ice_pf *pf) { - return min3(pf->num_lan_msix, (u16)num_online_cpus(), + return min3(pf->msix.eth, (u16)num_online_cpus(), (u16)pf->hw.func_caps.common_cap.num_txq); } @@ -3523,7 +3523,7 @@ static int ice_get_max_txq(struct ice_pf *pf) */ static int ice_get_max_rxq(struct ice_pf *pf) { - return min3(pf->num_lan_msix, (u16)num_online_cpus(), + return min3(pf->msix.eth, (u16)num_online_cpus(), (u16)pf->hw.func_caps.common_cap.num_rxq); } diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c index e6bc2285071e..2534a3460596 100644 --- a/drivers/net/ethernet/intel/ice/ice_idc.c +++ b/drivers/net/ethernet/intel/ice/ice_idc.c @@ -237,11 +237,11 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf) if (ice_is_rdma_ena(pf)) { int index; - index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix, + index = ice_get_res(pf, pf->irq_tracker, pf->msix.rdma, ICE_RES_RDMA_VEC_ID); if (index < 0) return index; - pf->num_avail_sw_msix -= pf->num_rdma_msix; + pf->num_avail_sw_msix -= pf->msix.rdma; pf->rdma_base_vector = (u16)index; } return 0; @@ -253,7 +253,7 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf) */ static void ice_free_rdma_qvector(struct ice_pf *pf) { - pf->num_avail_sw_msix -= pf->num_rdma_msix; + pf->num_avail_sw_msix -= pf->msix.rdma; ice_free_res(pf->irq_tracker, pf->rdma_base_vector, ICE_RES_RDMA_VEC_ID); } diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index bc04f2b9f8a2..11b0b3cca76c 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -184,7 +184,7 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi, struct ice_vf *vf) vsi->alloc_txq = vsi->req_txq; vsi->num_txq = vsi->req_txq; } else { - vsi->alloc_txq = min3(pf->num_lan_msix, + vsi->alloc_txq = min3(pf->msix.eth, ice_get_avail_txq_count(pf), (u16)num_online_cpus()); } @@ -199,7 +199,7 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi, struct ice_vf *vf) vsi->alloc_rxq = vsi->req_rxq; vsi->num_rxq = vsi->req_rxq; } else { - vsi->alloc_rxq = min3(pf->num_lan_msix, + vsi->alloc_rxq = min3(pf->msix.eth, ice_get_avail_rxq_count(pf), (u16)num_online_cpus()); } @@ -207,7 +207,7 @@ static void ice_vsi_set_num_qs(struct ice_vsi *vsi, struct ice_vf *vf) pf->num_lan_rx = vsi->alloc_rxq; - vsi->num_q_vectors = min_t(int, pf->num_lan_msix, + vsi->num_q_vectors = min_t(int, pf->msix.eth, max_t(int, vsi->alloc_rxq, vsi->alloc_txq)); break; @@ -1100,7 +1100,7 @@ ice_chnl_vsi_setup_q_map(struct ice_vsi *vsi, struct ice_vsi_ctx *ctxt) u8 offset = 0; int pow; - qcount = min_t(int, vsi->num_rxq, pf->num_lan_msix); + qcount = min_t(int, vsi->num_rxq, pf->msix.eth); pow = order_base_2(qcount); qmap = ((offset << ICE_AQ_VSI_TC_Q_OFFSET_S) & diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index 0dfc427e623a..a4c283bc8da0 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -3818,119 +3818,193 @@ static int ice_init_pf(struct ice_pf *pf) } /** - * ice_reduce_msix_usage - Reduce usage of MSI-X vectors + * ice_distribute_msix - distribute amount of MSI-X across all features * @pf: board private structure - * @v_remain: number of remaining MSI-X vectors to be distributed + * @msix: structure that stores MSI-X amount for all features + * @num_msix: amount of MSI-X to distribute + * @req_lan: amount of requested LAN MSI-X + * @req_rdma: amount of requested RDMA MSI-X * - * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled. - * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of - * remaining vectors. + * The function is trying to distribute MSI-X across all supported features. + * As a side effect the function can turn off not critical features if the + * number of msix is too low. + * + * Function returns -ERANGE if the amount of msix is too low to support + * basic driver operations. It is critical error. + * + * In rest cases function returns number of used MSI-X in total, + * + * First try to distribute minimum amount of MSI-X across all features. If + * there still are remaing MSI-X distribute it equally. + * + * Function can be called to distribute MSI-X amount received from hardware + * or from kernel. To support both cases MSI-X for VF isn't set here. This + * should be set outside this function based on returned value. */ -static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain) +static int +ice_distribute_msix(struct ice_pf *pf, struct ice_msix *msix, int num_msix, + int req_lan, int req_rdma) { - int v_rdma; + struct device *dev = ice_pf_to_dev(pf); + int rem_msix = num_msix; + int additional_msix; - if (!ice_is_rdma_ena(pf)) { - pf->num_lan_msix = v_remain; - return; + msix->all = num_msix; + msix->eth = 0; + msix->rdma = 0; + msix->misc = 0; + + if (num_msix < ICE_MIN_MSIX) { + dev_warn(dev, "not enough MSI-X vectors. wanted = %d, available = %d\n", + ICE_MIN_MSIX, num_msix); + return -ERANGE; } - /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */ - v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1; + msix->misc = ICE_MIN_LAN_OICR_MSIX; + /* Minimum LAN */ + msix->eth = ICE_MIN_LAN_TXRX_MSIX; + req_lan -= ICE_MIN_LAN_TXRX_MSIX; + rem_msix = rem_msix - msix->misc - msix->eth; + additional_msix = req_lan; - if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) { - dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n"); - clear_bit(ICE_FLAG_RDMA_ENA, pf->flags); + /* MSI-X distribution for misc */ + if (test_bit(ICE_FLAG_FD_ENA, pf->flags)) { + if (rem_msix >= ICE_FDIR_MSIX) { + msix->misc += ICE_FDIR_MSIX; + rem_msix -= ICE_FDIR_MSIX; + } else { + dev_warn(dev, "not enough MSI-X vectors. wanted = %d, available = %d\n", + ICE_FDIR_MSIX, rem_msix); + dev_warn(dev, "turning flow director off\n"); + clear_bit(ICE_FLAG_FD_ENA, pf->flags); + goto exit; + } + } + + if (test_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags)) { + if (rem_msix >= ICE_ESWITCH_MSIX) { + msix->misc += ICE_ESWITCH_MSIX; + rem_msix -= ICE_ESWITCH_MSIX; + } else { + dev_warn(dev, "not enough MSI-X vectors. wanted = %d, available = %d\n", + ICE_ESWITCH_MSIX, rem_msix); + dev_warn(dev, "turning eswitch off\n"); + clear_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags); + goto exit; + } + } - pf->num_rdma_msix = 0; - pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX; - } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) || - (v_remain - v_rdma < v_rdma)) { - /* Support minimum RDMA and give remaining vectors to LAN MSIX */ - pf->num_rdma_msix = ICE_MIN_RDMA_MSIX; - pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX; + /* Minimum RDMA + * Minimum MSI-X for RDMA is ICE_MIN_RDMA_MSIX, but it is better to + * have ICE_RDMA_NUM_AEQ_MSIX + 1 for queue. Instead of starting + * with ICE_MIN_RDMA_MSIX and going to distribution for LAN, try + * to start with ICE_RDMA_NUM_AEQ_MSIX + 1 if there are MSI-X for it + */ + if (ice_is_rdma_ena(pf)) { + if (rem_msix >= ICE_RDMA_NUM_AEQ_MSIX + 1 && + req_rdma >= ICE_RDMA_NUM_AEQ_MSIX + 1) { + msix->rdma = ICE_RDMA_NUM_AEQ_MSIX + 1; + rem_msix -= msix->rdma; + req_rdma -= msix->rdma; + additional_msix += req_rdma; + } else if (rem_msix >= ICE_MIN_RDMA_MSIX) { + msix->rdma = ICE_MIN_RDMA_MSIX; + rem_msix -= ICE_MIN_RDMA_MSIX; + req_rdma -= ICE_MIN_RDMA_MSIX; + additional_msix += req_rdma; + } else { + dev_warn(dev, "not enough MSI-X vectors. wanted = %d, available = %d\n", + ICE_MIN_RDMA_MSIX, rem_msix); + dev_warn(dev, "turning RDMA off\n"); + clear_bit(ICE_FLAG_RDMA_ENA, pf->flags); + goto exit; + } } else { - /* Split remaining MSIX with RDMA after accounting for AEQ MSIX - */ - pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 + - ICE_RDMA_NUM_AEQ_MSIX; - pf->num_lan_msix = v_remain - pf->num_rdma_msix; + req_rdma = 0; } -} -/** - * ice_ena_msix_range - Request a range of MSIX vectors from the OS - * @pf: board private structure - * - * Compute the number of MSIX vectors wanted and request from the OS. Adjust - * device usage if there are not enough vectors. Return the number of vectors - * reserved or negative on failure. - */ -static int ice_ena_msix_range(struct ice_pf *pf) -{ - int num_cpus, hw_num_msix, v_other, v_wanted, v_actual; - struct device *dev = ice_pf_to_dev(pf); - int err, i; + /* Check if there is enough additional MSI-X for all used features */ + if (rem_msix >= additional_msix) { + /* Best case scenario */ + msix->eth += req_lan; + rem_msix -= req_lan; - hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors; - num_cpus = num_online_cpus(); + msix->rdma += req_rdma; + rem_msix -= req_rdma; + } else if (rem_msix) { + int factor = DIV_ROUND_UP(additional_msix, rem_msix); + int rdma = req_rdma / factor; - /* LAN miscellaneous handler */ - v_other = ICE_MIN_LAN_OICR_MSIX; + dev_warn(dev, "not enough MSI-X vectors. requested = %d, obtained = %d\n", + additional_msix, rem_msix); - /* Flow Director */ - if (test_bit(ICE_FLAG_FD_ENA, pf->flags)) - v_other += ICE_FDIR_MSIX; + msix->rdma += rdma; + rem_msix -= rdma; - /* switchdev */ - if (test_bit(ICE_FLAG_ESWITCH_CAPABLE, pf->flags)) - v_other += ICE_ESWITCH_MSIX; + msix->eth += rem_msix; + rem_msix = 0; - v_wanted = v_other; + dev_notice(dev, "enable %d MSI-X vectors for LAN and %d for RDMA\n", + msix->eth, msix->rdma); + } - /* LAN traffic */ - pf->num_lan_msix = num_cpus; - v_wanted += pf->num_lan_msix; + /* It is fine if rem_msix isn't 0 here, as it can be used for VF. + * It isn't done here as this function also cover MSI-X distribution + * for MSI-X amount returned from kernel + */ - /* RDMA auxiliary driver */ - if (ice_is_rdma_ena(pf)) { - pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX; - v_wanted += pf->num_rdma_msix; - } +exit: + /* Return number of used MSI-X instead of remaing */ + return num_msix - rem_msix; +} - if (v_wanted > hw_num_msix) { - int v_remain; +/** + * ice_set_def_msix - set default value for MSI-X based on hardware + * @pf: board private structure + * + * Function will return -ERANGE when there is critical error and + * minimum amount of MSI-X can't be used. In success returns 0. + */ +static int +ice_set_def_msix(struct ice_pf *pf) +{ + int hw_msix = pf->hw.func_caps.common_cap.num_msix_vectors; + int num_cpus = num_present_cpus(); + int used_msix; - dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n", - v_wanted, hw_num_msix); + /* set default only if it wasn't previously set in devlink API */ + if (pf->req_msix.eth) + return 0; - if (hw_num_msix < ICE_MIN_MSIX) { - err = -ERANGE; - goto exit_err; - } + used_msix = ice_distribute_msix(pf, &pf->req_msix, hw_msix, num_cpus, + num_cpus + ICE_RDMA_NUM_AEQ_MSIX); - v_remain = hw_num_msix - v_other; - if (v_remain < ICE_MIN_LAN_TXRX_MSIX) { - v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX; - v_remain = ICE_MIN_LAN_TXRX_MSIX; - } + if (used_msix < 0) + return -ERANGE; - ice_reduce_msix_usage(pf, v_remain); - v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other; + pf->req_msix.vf = hw_msix - used_msix; - dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n", - pf->num_lan_msix); - if (ice_is_rdma_ena(pf)) - dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n", - pf->num_rdma_msix); - } + return 0; +} +/** + * ice_ena_msix_range - Request a range of MSIX vectors from the OS + * @pf: board private structure + * + * Compute the number of MSIX vectors wanted and request from the OS. Adjust + * device usage if there are not enough vectors. Return the number of vectors + * reserved or negative on failure. + */ +static int ice_ena_msix_range(struct ice_pf *pf) +{ + struct device *dev = ice_pf_to_dev(pf); + int v_wanted, v_actual, err, i; + + v_wanted = pf->req_msix.eth + pf->req_msix.rdma + pf->req_msix.misc; pf->msix_entries = devm_kcalloc(dev, v_wanted, sizeof(*pf->msix_entries), GFP_KERNEL); - if (!pf->msix_entries) { - err = -ENOMEM; - goto exit_err; - } + if (!pf->msix_entries) + return -ENOMEM; for (i = 0; i < v_wanted; i++) pf->msix_entries[i].entry = i; @@ -3938,46 +4012,17 @@ static int ice_ena_msix_range(struct ice_pf *pf) /* actually reserve the vectors */ v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries, ICE_MIN_MSIX, v_wanted); - if (v_actual < 0) { - dev_err(dev, "unable to reserve MSI-X vectors\n"); - err = v_actual; - goto msix_err; - } - if (v_actual < v_wanted) { - dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n", - v_wanted, v_actual); - - if (v_actual < ICE_MIN_MSIX) { - /* error if we can't get minimum vectors */ - pci_disable_msix(pf->pdev); - err = -ERANGE; - goto msix_err; - } else { - int v_remain = v_actual - v_other; - - if (v_remain < ICE_MIN_LAN_TXRX_MSIX) - v_remain = ICE_MIN_LAN_TXRX_MSIX; - - ice_reduce_msix_usage(pf, v_remain); - - dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n", - pf->num_lan_msix); - - if (ice_is_rdma_ena(pf)) - dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n", - pf->num_rdma_msix); - } - } + v_actual = ice_distribute_msix(pf, &pf->msix, v_actual, + pf->req_msix.eth, pf->req_msix.rdma); + if (v_actual < 0) + goto msix_err; return v_actual; msix_err: devm_kfree(dev, pf->msix_entries); -exit_err: - pf->num_rdma_msix = 0; - pf->num_lan_msix = 0; return err; } @@ -4014,6 +4059,9 @@ static int ice_init_interrupt_scheme(struct ice_pf *pf) { int vectors; + if (ice_set_def_msix(pf)) + return -ERANGE; + vectors = ice_ena_msix_range(pf); if (vectors < 0) From patchwork Mon Nov 14 12:57:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Swiatkowski X-Patchwork-Id: 1703593 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=osuosl.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=osuosl.org header.i=@osuosl.org header.a=rsa-sha256 header.s=default header.b=7STF/tQr; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4N9qVt59nYz1yhv for ; Tue, 15 Nov 2022 00:13:09 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0A73C60DCA; Mon, 14 Nov 2022 13:13:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0A73C60DCA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1668431588; bh=L4jJM2MVZd3le8jSuIE9gPJMtmrEh9jxkkg6knt1OYI=; h=From:To:Date:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=7STF/tQryBkIKmIC4RFp83zhe6DWaTg8swByj6jZm4Lm4Vfga6wE6Nrry3g6CfnHu VdsnuKfdBW3Ic4JJaD5hHPMlbBFQxBSoHyoA41xWw2Hw5vouIzMNbtosUpvZtXvyBa U7DHTSQo+EksYHbkB/ilDFW3BDw+f7Ov/Ud5SQG4Img+YmLOH2SE57q6Ycc16nolr+ B+lUREQR5e4l0dcOl1BGEdijAhO6pmiQjQFMLJZzNF81dswwF3vpCkhve5dX/Kn/B/ jOOUvLzAIHShRCTlJ2XNlaRASSJW+ySB3YrqlBdbERnCD/imAFT7o3t4H6SO59zIHU npm6+/cl9WBaQ== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2R3RHgL6yWlj; Mon, 14 Nov 2022 13:13:07 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id B9C7A60DDF; Mon, 14 Nov 2022 13:13:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B9C7A60DDF X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 404EB1BF379 for ; Mon, 14 Nov 2022 13:13:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1BC66818D4 for ; Mon, 14 Nov 2022 13:13:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1BC66818D4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MRZPmRXxivDG for ; Mon, 14 Nov 2022 13:13:04 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 02AFF81886 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by smtp1.osuosl.org (Postfix) with ESMTPS id 02AFF81886 for ; Mon, 14 Nov 2022 13:13:03 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="311968103" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="311968103" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Nov 2022 05:13:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10531"; a="616306114" X-IronPort-AV: E=Sophos;i="5.96,161,1665471600"; d="scan'208";a="616306114" Received: from wasp.igk.intel.com ([10.102.20.192]) by orsmga006.jf.intel.com with ESMTP; 14 Nov 2022 05:12:51 -0800 From: Michal Swiatkowski To: netdev@vger.kernel.org, davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com Date: Mon, 14 Nov 2022 13:57:55 +0100 Message-Id: <20221114125755.13659-14-michal.swiatkowski@linux.intel.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> References: <20221114125755.13659-1-michal.swiatkowski@linux.intel.com> MIME-Version: 1.0 X-Mailman-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668431583; x=1699967583; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dMZroapyEHbAR0dzHzMmRn1QLPwonaNJAXsRz1v3Etc=; b=TUNr5v5cwvQ4zJN4NPZ+7BES/Tf+LO8LCDsvz97o6JD9Yn90IpDaabPa fhGPa4TNzGjlAreEuIrGs2nJ/ql+tq+1ZBSrQIBq57QtYyVG8fOvxdLXf QH7sJ1LQ/+CsTQWf7MvJ2crYGlSpsW3xlIs0TDQoKBvjM6Y1Gf1UkAsFC t9kGSI9Eruko2eclIzxCI3GCs3G53UiAyDIy9kEWqLiNTwY3WRoi4QGy2 Wunrc7dkFeGYdWACjkSQfFzYDdZWD0n3DkPuUC55jumf7im3xeAo1PkOB yi8LK2yktQTH74HNdHPh4r3NH4FzRUNKf4jU8pStliwOzYLyGNssCZofI w==; X-Mailman-Original-Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=TUNr5v5c Subject: [Intel-wired-lan] [PATCH net-next 13/13] devlink, ice: add MSIX vectors as devlink resource X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mustafa.ismail@intel.com, leszek.kaliszczuk@intel.com, Michal Kubiak , jiri@nvidia.com, intel-wired-lan@lists.osuosl.org, shiraz.saleem@intel.com, przemyslaw.kitszel@intel.com Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" From: Michal Kubiak Implement devlink resource to control how many MSI-X vectors are used for eth, VF and RDMA. Show misc MSI-X as read only. This is first approach to control the mix of resources managed by ice driver. This commit registers number of available MSI-X as devlink resource and also add specific resources for eth, vf and RDMA. Also, make those resources generic. $ devlink resource show pci/0000:31:00.0 name msix size 1024 occ 172 unit entry dpipe_tables none resources: name msix_misc size 4 unit entry dpipe_tables none name msix_eth size 92 occ 92 unit entry size_min 1 size_max 128 size_gran 1 dpipe_tables none name msix_vf size 128 occ 0 unit entry size_min 0 size_max 1020 size_gran 1 dpipe_tables none name msix_rdma size 76 occ 76 unit entry size_min 0 size_max 132 size_gran 1 dpipe_tables none example commands: $ devlink resource set pci/0000:31:00.0 path msix/msix_eth size 16 $ devlink resource set pci/0000:31:00.0 path msix/msix_vf size 512 $ devlink dev reload pci/0000:31:00.0 Signed-off-by: Michal Kubiak Co-developed-by: Piotr Raczynski Signed-off-by: Piotr Raczynski Co-developed-by: Michal Swiatkowski Signed-off-by: Michal Swiatkowski --- .../networking/devlink/devlink-resource.rst | 10 ++ drivers/net/ethernet/intel/ice/ice_devlink.c | 160 ++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_devlink.h | 2 + drivers/net/ethernet/intel/ice/ice_main.c | 5 + drivers/net/ethernet/intel/ice/ice_sriov.c | 3 +- include/net/devlink.h | 14 ++ 6 files changed, 192 insertions(+), 2 deletions(-) diff --git a/Documentation/networking/devlink/devlink-resource.rst b/Documentation/networking/devlink/devlink-resource.rst index 3d5ae51e65a2..dcd3d124ebfb 100644 --- a/Documentation/networking/devlink/devlink-resource.rst +++ b/Documentation/networking/devlink/devlink-resource.rst @@ -36,6 +36,16 @@ device drivers and their description must be added to the following table: - Description * - ``physical_ports`` - A limited capacity of physical ports that the switch ASIC can support + * - ``msix`` + - Amount of MSIX interrupt vectors that the device can support + * - ``msix_misc`` + - Amount of MSIX interrupt vectors that can be used for miscellaneous purposes + * - ``msix_eth`` + - Amount of MSIX interrupt vectors that can be used by ethernet + * - ``msix_vf`` + - Amount of MSIX interrupt vectors that can be used by virtual functions + * - ``msix_rdma`` + - Amount of MSIX interrupt vectors that can be used by RDMA example usage ------------- diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c index 8f73c4008a56..00f01cf84812 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.c +++ b/drivers/net/ethernet/intel/ice/ice_devlink.c @@ -496,6 +496,27 @@ ice_devlink_reload_empr_finish(struct ice_pf *pf, return 0; } +static void ice_devlink_read_resources_size(struct ice_pf *pf) +{ + struct devlink *devlink = priv_to_devlink(pf); + u64 size_new; + + devl_resource_size_get(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_ETH, + &size_new); + pf->req_msix.eth = size_new; + + devl_resource_size_get(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_VF, + &size_new); + pf->msix.vf = size_new; + + devl_resource_size_get(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_RDMA, + &size_new); + pf->req_msix.rdma = size_new; +} + /** * ice_devlink_port_opt_speed_str - convert speed to a string * @speed: speed value @@ -761,6 +782,7 @@ ice_devlink_reload_up(struct devlink *devlink, switch (action) { case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: *actions_performed = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT); + ice_devlink_read_resources_size(pf); return ice_load(pf); case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: *actions_performed = BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE); @@ -955,6 +977,144 @@ void ice_devlink_unregister(struct ice_pf *pf) devlink_unregister(priv_to_devlink(pf)); } +static u64 ice_devlink_res_msix_pf_occ_get(void *priv) +{ + struct ice_pf *pf = priv; + + return pf->msix.eth; +} + +static u64 ice_devlink_res_msix_vf_occ_get(void *priv) +{ + struct ice_pf *pf = priv; + + return pf->vfs.num_msix_per * ice_get_num_vfs(pf); +} + +static u64 ice_devlink_res_msix_rdma_occ_get(void *priv) +{ + struct ice_pf *pf = priv; + + return pf->msix.rdma; +} + +static u64 ice_devlink_res_msix_occ_get(void *priv) +{ + struct ice_pf *pf = priv; + + return ice_devlink_res_msix_pf_occ_get(priv) + + ice_devlink_res_msix_vf_occ_get(priv) + + ice_devlink_res_msix_rdma_occ_get(priv) + + pf->msix.misc; +} + +int ice_devlink_register_resources(struct ice_pf *pf) +{ + struct devlink *devlink = priv_to_devlink(pf); + struct devlink_resource_size_params params; + struct device *dev = ice_pf_to_dev(pf); + struct ice_msix *req = &pf->req_msix; + int err, max_pf_msix, max_rdma_msix; + const char *res_name; + + max_pf_msix = min_t(int, num_online_cpus(), ICE_MAX_LG_RSS_QS); + max_rdma_msix = max_pf_msix + ICE_RDMA_NUM_AEQ_MSIX; + + devlink_resource_size_params_init(¶ms, req->all, req->all, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + res_name = DEVLINK_RESOURCE_GENERIC_NAME_MSIX; + err = devlink_resource_register(devlink, res_name, req->all, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + DEVLINK_RESOURCE_ID_PARENT_TOP, + ¶ms); + if (err) + goto res_create_err; + + devlink_resource_size_params_init(¶ms, req->misc, req->misc, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + res_name = DEVLINK_RESOURCE_GENERIC_NAME_MSIX_MISC; + err = devlink_resource_register(devlink, res_name, req->misc, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_MISC, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + ¶ms); + if (err) + goto res_create_err; + + devlink_resource_size_params_init(¶ms, ICE_MIN_LAN_TXRX_MSIX, + max_pf_msix, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + res_name = DEVLINK_RESOURCE_GENERIC_NAME_MSIX_ETH; + err = devlink_resource_register(devlink, res_name, req->eth, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_ETH, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + ¶ms); + if (err) + goto res_create_err; + + devlink_resource_size_params_init(¶ms, 0, req->all - req->misc, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + + res_name = DEVLINK_RESOURCE_GENERIC_NAME_MSIX_VF; + err = devlink_resource_register(devlink, res_name, req->vf, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_VF, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + ¶ms); + if (err) + goto res_create_err; + + devlink_resource_size_params_init(¶ms, ICE_MIN_RDMA_MSIX, + max_rdma_msix, 1, + DEVLINK_RESOURCE_UNIT_ENTRY); + + res_name = DEVLINK_RESOURCE_GENERIC_NAME_MSIX_RDMA; + err = devlink_resource_register(devlink, res_name, req->rdma, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_RDMA, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + ¶ms); + if (err) + goto res_create_err; + + devlink_resource_occ_get_register(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX, + ice_devlink_res_msix_occ_get, pf); + + devlink_resource_occ_get_register(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_ETH, + ice_devlink_res_msix_pf_occ_get, pf); + + devlink_resource_occ_get_register(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_VF, + ice_devlink_res_msix_vf_occ_get, pf); + + devlink_resource_occ_get_register(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_RDMA, + ice_devlink_res_msix_rdma_occ_get, + pf); + return 0; + +res_create_err: + dev_err(dev, "Failed to register devlink resource: %s error: %pe\n", + res_name, ERR_PTR(err)); + devlink_resources_unregister(devlink); + + return err; +} + +void ice_devlink_unregister_resources(struct ice_pf *pf) +{ + struct devlink *devlink = priv_to_devlink(pf); + + devlink_resource_occ_get_unregister(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX); + devlink_resource_occ_get_unregister(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_ETH); + devlink_resource_occ_get_unregister(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_VF); + devlink_resource_occ_get_unregister(devlink, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_RDMA); + devlink_resources_unregister(devlink); +} + /** * ice_devlink_set_switch_id - Set unique switch id based on pci dsn * @pf: the PF to create a devlink port for diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.h b/drivers/net/ethernet/intel/ice/ice_devlink.h index fe006d9946f8..35233ea087fc 100644 --- a/drivers/net/ethernet/intel/ice/ice_devlink.h +++ b/drivers/net/ethernet/intel/ice/ice_devlink.h @@ -10,6 +10,8 @@ void ice_devlink_register(struct ice_pf *pf); void ice_devlink_unregister(struct ice_pf *pf); int ice_devlink_register_params(struct ice_pf *pf); void ice_devlink_unregister_params(struct ice_pf *pf); +int ice_devlink_register_resources(struct ice_pf *pf); +void ice_devlink_unregister_resources(struct ice_pf *pf); int ice_devlink_create_pf_port(struct ice_pf *pf); void ice_devlink_destroy_pf_port(struct ice_pf *pf); int ice_devlink_create_vf_port(struct ice_vf *vf); diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index a4c283bc8da0..ba6f93e7e640 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -4970,6 +4970,10 @@ static int ice_init_devlink(struct ice_pf *pf) if (err) return err; + err = ice_devlink_register_resources(pf); + if (err) + return err; + ice_devlink_init_regions(pf); ice_devlink_register(pf); @@ -4980,6 +4984,7 @@ static void ice_deinit_devlink(struct ice_pf *pf) { ice_devlink_unregister(pf); ice_devlink_destroy_regions(pf); + ice_devlink_unregister_resources(pf); ice_devlink_unregister_params(pf); } diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c index 3ba1408c56a9..d42df5983494 100644 --- a/drivers/net/ethernet/intel/ice/ice_sriov.c +++ b/drivers/net/ethernet/intel/ice/ice_sriov.c @@ -518,8 +518,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs) return -ENOSPC; /* determine MSI-X resources per VF */ - msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors - - pf->irq_tracker->num_entries; + msix_avail_for_sriov = pf->msix.vf; msix_avail_per_vf = msix_avail_for_sriov / num_vfs; if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) { num_msix_per_vf = ICE_NUM_VF_MSIX_MED; diff --git a/include/net/devlink.h b/include/net/devlink.h index 611a23a3deb2..a041c50f72c6 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -409,7 +409,21 @@ typedef u64 devlink_resource_occ_get_t(void *priv); #define DEVLINK_RESOURCE_ID_PARENT_TOP 0 +enum devlink_msix_resource_id { + /* generic resource for MSIX */ + DEVLINK_RESOURCE_GENERIC_ID_MSIX = 1, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_MISC, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_ETH, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_VF, + DEVLINK_RESOURCE_GENERIC_ID_MSIX_RDMA, +}; + #define DEVLINK_RESOURCE_GENERIC_NAME_PORTS "physical_ports" +#define DEVLINK_RESOURCE_GENERIC_NAME_MSIX "msix" +#define DEVLINK_RESOURCE_GENERIC_NAME_MSIX_MISC "msix_misc" +#define DEVLINK_RESOURCE_GENERIC_NAME_MSIX_ETH "msix_eth" +#define DEVLINK_RESOURCE_GENERIC_NAME_MSIX_VF "msix_vf" +#define DEVLINK_RESOURCE_GENERIC_NAME_MSIX_RDMA "msix_rdma" #define __DEVLINK_PARAM_MAX_STRING_VALUE 32 enum devlink_param_type {