From patchwork Mon Jul 27 11:02:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336862 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcSx09RCz9sPf for ; Mon, 27 Jul 2020 21:06:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728639AbgG0LGw (ORCPT ); Mon, 27 Jul 2020 07:06:52 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40685 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728527AbgG0LGt (ORCPT ); Mon, 27 Jul 2020 07:06:49 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:11 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6BI4022173; Mon, 27 Jul 2020 14:06:11 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Ban002384; Mon, 27 Jul 2020 14:06:11 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6BIk002383; Mon, 27 Jul 2020 14:06:11 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 01/13] devlink: Add reload level option to devlink reload command Date: Mon, 27 Jul 2020 14:02:21 +0300 Message-Id: <1595847753-2234-2-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add devlink reload level to allow the user to request a specific reload level. The level parameter is optional, if not specified then driver's default reload level is used (backward compatible). Reload levels supported are: driver: driver entities re-instantiation only. fw_reset: firmware reset and driver entities re-instantiation. fw_live_patch: firmware live patching only. Signed-off-by: Moshe Shemesh --- drivers/net/ethernet/mellanox/mlx4/main.c | 6 ++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 6 ++- drivers/net/ethernet/mellanox/mlxsw/core.c | 6 ++- drivers/net/netdevsim/dev.c | 6 ++- include/net/devlink.h | 6 ++- include/uapi/linux/devlink.h | 19 +++++++ net/core/devlink.c | 52 +++++++++++++++++-- 7 files changed, 86 insertions(+), 15 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c index 954c22c79f6b..57d9d4381cb0 100644 --- a/drivers/net/ethernet/mellanox/mlx4/main.c +++ b/drivers/net/ethernet/mellanox/mlx4/main.c @@ -3935,7 +3935,7 @@ static int mlx4_restart_one_up(struct pci_dev *pdev, bool reload, struct devlink *devlink); static int mlx4_devlink_reload_down(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack) + enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlx4_priv *priv = devlink_priv(devlink); struct mlx4_dev *dev = &priv->dev; @@ -3951,7 +3951,7 @@ static int mlx4_devlink_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int mlx4_devlink_reload_up(struct devlink *devlink, +static int mlx4_devlink_reload_up(struct devlink *devlink, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlx4_priv *priv = devlink_priv(devlink); @@ -3969,6 +3969,8 @@ static int mlx4_devlink_reload_up(struct devlink *devlink, static const struct devlink_ops mlx4_devlink_ops = { .port_type_set = mlx4_devlink_port_type_set, + .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER), + .default_reload_level = DEVLINK_RELOAD_LEVEL_DRIVER, .reload_down = mlx4_devlink_reload_down, .reload_up = mlx4_devlink_reload_up, }; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index c709e9a385f6..5424e31a0f45 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -89,7 +89,7 @@ mlx5_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, } static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack) + enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlx5_core_dev *dev = devlink_priv(devlink); @@ -97,7 +97,7 @@ static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int mlx5_devlink_reload_up(struct devlink *devlink, +static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlx5_core_dev *dev = devlink_priv(devlink); @@ -118,6 +118,8 @@ static const struct devlink_ops mlx5_devlink_ops = { #endif .flash_update = mlx5_devlink_flash_update, .info_get = mlx5_devlink_info_get, + .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER), + .default_reload_level = DEVLINK_RELOAD_LEVEL_DRIVER, .reload_down = mlx5_devlink_reload_down, .reload_up = mlx5_devlink_reload_up, }; diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c index b01f8f2fab63..360d749eb042 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/core.c +++ b/drivers/net/ethernet/mellanox/mlxsw/core.c @@ -1113,7 +1113,7 @@ mlxsw_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, static int mlxsw_devlink_core_bus_device_reload_down(struct devlink *devlink, - bool netns_change, + bool netns_change, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlxsw_core *mlxsw_core = devlink_priv(devlink); @@ -1126,7 +1126,7 @@ mlxsw_devlink_core_bus_device_reload_down(struct devlink *devlink, } static int -mlxsw_devlink_core_bus_device_reload_up(struct devlink *devlink, +mlxsw_devlink_core_bus_device_reload_up(struct devlink *devlink, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlxsw_core *mlxsw_core = devlink_priv(devlink); @@ -1266,6 +1266,8 @@ mlxsw_devlink_trap_policer_counter_get(struct devlink *devlink, } static const struct devlink_ops mlxsw_devlink_ops = { + .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_FW_RESET), + .default_reload_level = DEVLINK_RELOAD_LEVEL_FW_RESET, .reload_down = mlxsw_devlink_core_bus_device_reload_down, .reload_up = mlxsw_devlink_core_bus_device_reload_up, .port_type_set = mlxsw_devlink_port_type_set, diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c index ce719c830a77..680482f687f4 100644 --- a/drivers/net/netdevsim/dev.c +++ b/drivers/net/netdevsim/dev.c @@ -697,7 +697,7 @@ static int nsim_dev_reload_create(struct nsim_dev *nsim_dev, static void nsim_dev_reload_destroy(struct nsim_dev *nsim_dev); static int nsim_dev_reload_down(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack) + enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct nsim_dev *nsim_dev = devlink_priv(devlink); @@ -713,7 +713,7 @@ static int nsim_dev_reload_down(struct devlink *devlink, bool netns_change, return 0; } -static int nsim_dev_reload_up(struct devlink *devlink, +static int nsim_dev_reload_up(struct devlink *devlink, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct nsim_dev *nsim_dev = devlink_priv(devlink); @@ -873,6 +873,8 @@ nsim_dev_devlink_trap_policer_counter_get(struct devlink *devlink, } static const struct devlink_ops nsim_dev_devlink_ops = { + .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER), + .default_reload_level = DEVLINK_RELOAD_LEVEL_DRIVER, .reload_down = nsim_dev_reload_down, .reload_up = nsim_dev_reload_up, .info_get = nsim_dev_info_get, diff --git a/include/net/devlink.h b/include/net/devlink.h index 19d990c8edcc..b291cd8d6be6 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -985,9 +985,11 @@ enum devlink_trap_group_generic_id { } struct devlink_ops { + unsigned long supported_reload_levels; + enum devlink_reload_level default_reload_level; int (*reload_down)(struct devlink *devlink, bool netns_change, - struct netlink_ext_ack *extack); - int (*reload_up)(struct devlink *devlink, + enum devlink_reload_level level, struct netlink_ext_ack *extack); + int (*reload_up)(struct devlink *devlink, enum devlink_reload_level level, struct netlink_ext_ack *extack); int (*port_type_set)(struct devlink_port *devlink_port, enum devlink_port_type port_type); diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index cfef4245ea5a..fa5f66db5012 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -272,6 +272,23 @@ enum { DEVLINK_ATTR_TRAP_METADATA_TYPE_FA_COOKIE, }; +/** + * enum devlink_reload_level - Reload level. + * @DEVLINK_RELOAD_LEVEL_DRIVER: Driver entities re-instantiation only. + * @DEVLINK_RELOAD_LEVEL_FW_RESET: FW reset and driver entities re-instantiation. + * @DEVLINK_RELOAD_LEVEL_FW_LIVE_PATCH: FW live patching only. + */ +enum devlink_reload_level { + DEVLINK_RELOAD_LEVEL_UNSPEC, + DEVLINK_RELOAD_LEVEL_DRIVER, + DEVLINK_RELOAD_LEVEL_FW_RESET, + DEVLINK_RELOAD_LEVEL_FW_LIVE_PATCH, + + /* Add new reload levels above */ + __DEVLINK_RELOAD_LEVEL_MAX, + DEVLINK_RELOAD_LEVEL_MAX = __DEVLINK_RELOAD_LEVEL_MAX - 1 +}; + enum devlink_attr { /* don't change the order or add anything between, this is ABI! */ DEVLINK_ATTR_UNSPEC, @@ -458,6 +475,8 @@ enum devlink_attr { DEVLINK_ATTR_PORT_LANES, /* u32 */ DEVLINK_ATTR_PORT_SPLITTABLE, /* u8 */ + DEVLINK_ATTR_RELOAD_LEVEL, /* u8 */ + /* add new attributes above here, update the policy in devlink.c */ __DEVLINK_ATTR_MAX, diff --git a/net/core/devlink.c b/net/core/devlink.c index 0ca89196a367..31b367a1612d 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -462,6 +462,12 @@ static int devlink_nl_put_handle(struct sk_buff *msg, struct devlink *devlink) return 0; } +static bool +devlink_reload_level_is_supported(struct devlink *devlink, enum devlink_reload_level level) +{ + return test_bit(level, &devlink->ops->supported_reload_levels); +} + static int devlink_nl_fill(struct sk_buff *msg, struct devlink *devlink, enum devlink_command cmd, u32 portid, u32 seq, int flags) @@ -2958,21 +2964,21 @@ bool devlink_is_reload_failed(const struct devlink *devlink) EXPORT_SYMBOL_GPL(devlink_is_reload_failed); static int devlink_reload(struct devlink *devlink, struct net *dest_net, - struct netlink_ext_ack *extack) + enum devlink_reload_level level, struct netlink_ext_ack *extack) { int err; if (!devlink->reload_enabled) return -EOPNOTSUPP; - err = devlink->ops->reload_down(devlink, !!dest_net, extack); + err = devlink->ops->reload_down(devlink, !!dest_net, level, extack); if (err) return err; if (dest_net && !net_eq(dest_net, devlink_net(devlink))) devlink_reload_netns_change(devlink, dest_net); - err = devlink->ops->reload_up(devlink, extack); + err = devlink->ops->reload_up(devlink, level, extack); devlink_reload_failed_set(devlink, !!err); return err; } @@ -2980,6 +2986,7 @@ static int devlink_reload(struct devlink *devlink, struct net *dest_net, static int devlink_nl_cmd_reload(struct sk_buff *skb, struct genl_info *info) { struct devlink *devlink = info->user_ptr[0]; + enum devlink_reload_level level; struct net *dest_net = NULL; int err; @@ -3000,7 +3007,20 @@ static int devlink_nl_cmd_reload(struct sk_buff *skb, struct genl_info *info) return PTR_ERR(dest_net); } - err = devlink_reload(devlink, dest_net, info->extack); + if (info->attrs[DEVLINK_ATTR_RELOAD_LEVEL]) + level = nla_get_u8(info->attrs[DEVLINK_ATTR_RELOAD_LEVEL]); + else + level = devlink->ops->default_reload_level; + + if (level == DEVLINK_RELOAD_LEVEL_UNSPEC || level > DEVLINK_RELOAD_LEVEL_MAX) { + NL_SET_ERR_MSG_MOD(info->extack, "Invalid reload level"); + return -EINVAL; + } else if (!devlink_reload_level_is_supported(devlink, level)) { + NL_SET_ERR_MSG_MOD(info->extack, "Requested reload level is not supported"); + return -EOPNOTSUPP; + } + + err = devlink_reload(devlink, dest_net, level, info->extack); if (dest_net) put_net(dest_net); @@ -7026,6 +7046,7 @@ static const struct nla_policy devlink_nl_policy[DEVLINK_ATTR_MAX + 1] = { [DEVLINK_ATTR_TRAP_POLICER_RATE] = { .type = NLA_U64 }, [DEVLINK_ATTR_TRAP_POLICER_BURST] = { .type = NLA_U64 }, [DEVLINK_ATTR_PORT_FUNCTION] = { .type = NLA_NESTED }, + [DEVLINK_ATTR_RELOAD_LEVEL] = { .type = NLA_U8 }, }; static const struct genl_ops devlink_nl_ops[] = { @@ -7351,6 +7372,21 @@ static struct genl_family devlink_nl_family __ro_after_init = { .n_mcgrps = ARRAY_SIZE(devlink_nl_mcgrps), }; +static int devlink_reload_levels_verify(struct devlink *devlink) +{ + const struct devlink_ops *ops; + + if (!devlink_reload_supported(devlink)) + return 0; + + ops = devlink->ops; + if (WARN_ON(ops->supported_reload_levels >= BIT(__DEVLINK_RELOAD_LEVEL_MAX) || + ops->supported_reload_levels & BIT(DEVLINK_RELOAD_LEVEL_UNSPEC) || + !(ops->supported_reload_levels & BIT(ops->default_reload_level)))) + return -EINVAL; + return 0; +} + /** * devlink_alloc - Allocate new devlink instance resources * @@ -7371,6 +7407,11 @@ struct devlink *devlink_alloc(const struct devlink_ops *ops, size_t priv_size) if (!devlink) return NULL; devlink->ops = ops; + if (devlink_reload_levels_verify(devlink)) { + kfree(devlink); + return NULL; + } + xa_init_flags(&devlink->snapshot_ids, XA_FLAGS_ALLOC); __devlink_net_set(devlink, &init_net); INIT_LIST_HEAD(&devlink->port_list); @@ -9599,7 +9640,8 @@ static void __net_exit devlink_pernet_pre_exit(struct net *net) if (net_eq(devlink_net(devlink), net)) { if (WARN_ON(!devlink_reload_supported(devlink))) continue; - err = devlink_reload(devlink, &init_net, NULL); + err = devlink_reload(devlink, &init_net, + devlink->ops->default_reload_level, NULL); if (err && err != -EOPNOTSUPP) pr_warn("Failed to reload devlink instance into init_net\n"); } From patchwork Mon Jul 27 11:02:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336869 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTt6LYJz9sR4 for ; Mon, 27 Jul 2020 21:07:46 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728718AbgG0LHp (ORCPT ); Mon, 27 Jul 2020 07:07:45 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40684 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726701AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6Bn5022176; Mon, 27 Jul 2020 14:06:11 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6BIK002386; Mon, 27 Jul 2020 14:06:11 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6B3W002385; Mon, 27 Jul 2020 14:06:11 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 02/13] devlink: Add reload levels data to dev get Date: Mon, 27 Jul 2020 14:02:22 +0300 Message-Id: <1595847753-2234-3-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Expose devlink reload supported levels and driver's default level to the user through devlink dev get command. Examples: $ devlink dev show pci/0000:82:00.0: reload_levels_info: default_level driver supported_levels: driver fw_reset fw_live_patch pci/0000:82:00.1: reload_levels_info: default_level driver supported_levels: driver fw_reset fw_live_patch $ devlink dev show -jp { "dev": { "pci/0000:82:00.0": { "reload_levels_info": { "default_level": "driver", "supported_levels": [ "driver","fw_reset","fw_live_patch" ] } }, "pci/0000:82:00.1": { "reload_levels_info": { "default_level": "driver", "supported_levels": [ "driver","fw_reset","fw_live_patch" ] } } } } Signed-off-by: Moshe Shemesh --- include/uapi/linux/devlink.h | 3 +++ net/core/devlink.c | 38 +++++++++++++++++++++++++++++++----- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h index fa5f66db5012..249e921ff106 100644 --- a/include/uapi/linux/devlink.h +++ b/include/uapi/linux/devlink.h @@ -476,6 +476,9 @@ enum devlink_attr { DEVLINK_ATTR_PORT_SPLITTABLE, /* u8 */ DEVLINK_ATTR_RELOAD_LEVEL, /* u8 */ + DEVLINK_ATTR_RELOAD_DEFAULT_LEVEL, /* u8 */ + DEVLINK_ATTR_RELOAD_SUPPORTED_LEVELS, /* nested */ + DEVLINK_ATTR_RELOAD_LEVELS_INFO, /* nested */ /* add new attributes above here, update the policy in devlink.c */ diff --git a/net/core/devlink.c b/net/core/devlink.c index 31b367a1612d..f1812fc620d4 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -462,6 +462,11 @@ static int devlink_nl_put_handle(struct sk_buff *msg, struct devlink *devlink) return 0; } +static bool devlink_reload_supported(struct devlink *devlink) +{ + return devlink->ops->reload_down && devlink->ops->reload_up; +} + static bool devlink_reload_level_is_supported(struct devlink *devlink, enum devlink_reload_level level) { @@ -472,7 +477,9 @@ static int devlink_nl_fill(struct sk_buff *msg, struct devlink *devlink, enum devlink_command cmd, u32 portid, u32 seq, int flags) { + struct nlattr *reload_levels_info, *supported_levels; void *hdr; + int i; hdr = genlmsg_put(msg, portid, seq, &devlink_nl_family, flags, cmd); if (!hdr) @@ -483,9 +490,35 @@ static int devlink_nl_fill(struct sk_buff *msg, struct devlink *devlink, if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_FAILED, devlink->reload_failed)) goto nla_put_failure; + if (devlink_reload_supported(devlink)) { + reload_levels_info = nla_nest_start(msg, DEVLINK_ATTR_RELOAD_LEVELS_INFO); + if (!reload_levels_info) + goto nla_put_failure; + if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_DEFAULT_LEVEL, + devlink->ops->default_reload_level)) + goto reload_levels_info_nest_cancel; + + supported_levels = nla_nest_start(msg, DEVLINK_ATTR_RELOAD_SUPPORTED_LEVELS); + if (!supported_levels) + goto reload_levels_info_nest_cancel; + + for (i = 0; i <= DEVLINK_RELOAD_LEVEL_MAX; i++) { + if (!devlink_reload_level_is_supported(devlink, i)) + continue; + if (nla_put_u8(msg, DEVLINK_ATTR_RELOAD_LEVEL, i)) + goto supported_levels_nest_cancel; + } + nla_nest_end(msg, supported_levels); + nla_nest_end(msg, reload_levels_info); + } + genlmsg_end(msg, hdr); return 0; +supported_levels_nest_cancel: + nla_nest_cancel(msg, supported_levels); +reload_levels_info_nest_cancel: + nla_nest_cancel(msg, reload_levels_info); nla_put_failure: genlmsg_cancel(msg, hdr); return -EMSGSIZE; @@ -2943,11 +2976,6 @@ static void devlink_reload_netns_change(struct devlink *devlink, DEVLINK_CMD_PARAM_NEW); } -static bool devlink_reload_supported(const struct devlink *devlink) -{ - return devlink->ops->reload_down && devlink->ops->reload_up; -} - static void devlink_reload_failed_set(struct devlink *devlink, bool reload_failed) { From patchwork Mon Jul 27 11:02:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336868 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTp0jC4z9sRR for ; Mon, 27 Jul 2020 21:07:42 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728378AbgG0LHb (ORCPT ); Mon, 27 Jul 2020 07:07:31 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40696 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728495AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6Cwo022180; Mon, 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Cms002388; Mon, 27 Jul 2020 14:06:12 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6Cht002387; Mon, 27 Jul 2020 14:06:12 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 03/13] net/mlx5: Add functions to set/query MFRL register Date: Mon, 27 Jul 2020 14:02:23 +0300 Message-Id: <1595847753-2234-4-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add functions to query and set the MFRL reset options supported by firmware. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- .../ethernet/mellanox/mlx5/core/fw_reset.c | 46 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/fw_reset.h | 13 ++++++ 3 files changed, 60 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 10e6886c96ba..4d45a2f6fed6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -16,7 +16,7 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \ transobj.o vport.o sriov.o fs_cmd.o fs_core.o pci_irq.o \ fs_counters.o rl.o lag.o dev.o events.o wq.o lib/gid.o \ lib/devcom.o lib/pci_vsc.o lib/dm.o diag/fs_tracepoint.o \ - diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o + diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o fw_reset.o # # Netdev basic diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c new file mode 100644 index 000000000000..76d2cece29ac --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ + +#include "fw_reset.h" + +static int mlx5_reg_mfrl_set(struct mlx5_core_dev *dev, u8 reset_level, + u8 reset_type_sel, u8 sync_resp, bool sync_start) +{ + u32 out[MLX5_ST_SZ_DW(mfrl_reg)] = {}; + u32 in[MLX5_ST_SZ_DW(mfrl_reg)] = {}; + + MLX5_SET(mfrl_reg, in, reset_level, reset_level); + MLX5_SET(mfrl_reg, in, rst_type_sel, reset_type_sel); + MLX5_SET(mfrl_reg, in, pci_sync_for_fw_update_resp, sync_resp); + MLX5_SET(mfrl_reg, in, pci_sync_for_fw_update_start, sync_start); + + return mlx5_core_access_reg(dev, in, sizeof(in), out, sizeof(out), MLX5_REG_MFRL, 0, 1); +} + +int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_type) +{ + u32 out[MLX5_ST_SZ_DW(mfrl_reg)] = {}; + u32 in[MLX5_ST_SZ_DW(mfrl_reg)] = {}; + int err; + + err = mlx5_core_access_reg(dev, in, sizeof(in), out, sizeof(out), MLX5_REG_MFRL, 0, 0); + if (err) + return err; + + if (reset_level) + *reset_level = MLX5_GET(mfrl_reg, out, reset_level); + if (reset_type) + *reset_type = MLX5_GET(mfrl_reg, out, reset_type); + + return 0; +} + +int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel) +{ + return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, reset_type_sel, 0, true); +} + +int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev) +{ + return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL0, 0, 0, false); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h new file mode 100644 index 000000000000..1bbd95182ca6 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ + +#ifndef __MLX5_FW_RESET_H +#define __MLX5_FW_RESET_H + +#include "mlx5_core.h" + +int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_type); +int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel); +int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev); + +#endif From patchwork Mon Jul 27 11:02:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336855 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcSC0cKpz9sPf for ; Mon, 27 Jul 2020 21:06:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728560AbgG0LGS (ORCPT ); Mon, 27 Jul 2020 07:06:18 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40699 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728534AbgG0LGQ (ORCPT ); Mon, 27 Jul 2020 07:06:16 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6CNb022190; Mon, 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6CtK002390; Mon, 27 Jul 2020 14:06:12 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6CKl002389; Mon, 27 Jul 2020 14:06:12 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 04/13] net/mlx5: Set cap for pci sync for fw update event Date: Mon, 27 Jul 2020 14:02:24 +0300 Message-Id: <1595847753-2234-5-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Set capability to notify the firmware that this host driver is capable of handling pci sync for firmware update events. Signed-off-by: Moshe Shemesh --- drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index e32d46c33701..5fb23cfecd71 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -548,6 +548,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx) if (MLX5_CAP_GEN_MAX(dev, dct)) MLX5_SET(cmd_hca_cap, set_hca_cap, dct, 1); + if (MLX5_CAP_GEN_MAX(dev, pci_sync_for_fw_update_event)) + MLX5_SET(cmd_hca_cap, set_hca_cap, pci_sync_for_fw_update_event, 1); + if (MLX5_CAP_GEN_MAX(dev, num_vhca_ports)) MLX5_SET(cmd_hca_cap, set_hca_cap, From patchwork Mon Jul 27 11:02:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336865 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTW56KWz9sRR for ; Mon, 27 Jul 2020 21:07:27 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728129AbgG0LHZ (ORCPT ); Mon, 27 Jul 2020 07:07:25 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40691 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728541AbgG0LHV (ORCPT ); Mon, 27 Jul 2020 07:07:21 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6C8J022230; Mon, 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6CEx002392; Mon, 27 Jul 2020 14:06:12 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6CrE002391; Mon, 27 Jul 2020 14:06:12 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 05/13] net/mlx5: Handle sync reset request event Date: Mon, 27 Jul 2020 14:02:25 +0300 Message-Id: <1595847753-2234-6-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Once the driver gets sync_reset_request from firmware it prepares for the coming reset and sends acknowledge. After getting this event the driver expects device reset, either it will trigger PCI reset on sync_reset_now event or such PCI reset will be triggered by another PF of the same device. So it moves to reset requested mode and if it gets PCI reset triggered by the other PF it does silent recovery without reporting it as an error. Signed-off-by: Moshe Shemesh --- .../ethernet/mellanox/mlx5/core/fw_reset.c | 88 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/fw_reset.h | 3 + .../net/ethernet/mellanox/mlx5/core/health.c | 74 +++++++++++++--- .../net/ethernet/mellanox/mlx5/core/main.c | 10 +++ include/linux/mlx5/driver.h | 11 +++ 5 files changed, 173 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 76d2cece29ac..13f83fc1783f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -3,6 +3,13 @@ #include "fw_reset.h" +struct mlx5_fw_reset { + struct mlx5_core_dev *dev; + struct mlx5_nb nb; + struct workqueue_struct *wq; + struct work_struct reset_request_work; +}; + static int mlx5_reg_mfrl_set(struct mlx5_core_dev *dev, u8 reset_level, u8 reset_type_sel, u8 sync_resp, bool sync_start) { @@ -44,3 +51,84 @@ int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev) { return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL0, 0, 0, false); } + +static int mlx5_fw_set_reset_sync_ack(struct mlx5_core_dev *dev) +{ + return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, 0, 1, false); +} + +static void mlx5_sync_reset_request_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + reset_request_work); + struct mlx5_core_dev *dev = fw_reset->dev; + + mlx5_health_set_reset_requested_mode(dev); + mlx5_reload_health_poll_timer(dev); + if (mlx5_fw_set_reset_sync_ack(dev)) + mlx5_core_warn(dev, "PCI Sync FW Update Reset Ack Failed.\n"); + else + mlx5_core_warn(dev, "PCI Sync FW Update Reset Ack. Device reset is expected.\n"); +} + +static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct mlx5_eqe *eqe) +{ + struct mlx5_eqe_sync_fw_update *sync_fw_update_eqe; + u8 sync_event_rst_type; + + sync_fw_update_eqe = &eqe->data.sync_fw_update; + sync_event_rst_type = sync_fw_update_eqe->sync_rst_state & SYNC_RST_STATE_MASK; + switch (sync_event_rst_type) { + case MLX5_SYNC_RST_STATE_RESET_REQUEST: + queue_work(fw_reset->wq, &fw_reset->reset_request_work); + break; + } +} + +static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long action, void *data) +{ + struct mlx5_fw_reset *fw_reset = mlx5_nb_cof(nb, struct mlx5_fw_reset, nb); + struct mlx5_eqe *eqe = data; + + switch (eqe->sub_type) { + case MLX5_GENERAL_SUBTYPE_PCI_SYNC_FOR_FW_UPDATE_EVENT: + mlx5_sync_reset_events_handle(fw_reset, eqe); + break; + default: + return NOTIFY_DONE; + } + + return NOTIFY_OK; +} + +int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) +{ + struct mlx5_fw_reset *fw_reset = kzalloc(sizeof(*fw_reset), GFP_KERNEL); + + if (!fw_reset) + return -ENOMEM; + fw_reset->wq = create_singlethread_workqueue("mlx5_fw_reset_events"); + if (!fw_reset->wq) { + kfree(fw_reset); + return -ENOMEM; + } + + fw_reset->dev = dev; + dev->priv.fw_reset = fw_reset; + + INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); + + MLX5_NB_INIT(&fw_reset->nb, fw_reset_event_notifier, GENERAL_EVENT); + mlx5_eq_notifier_register(dev, &fw_reset->nb); + + return 0; +} + +void mlx5_fw_reset_events_cleanup(struct mlx5_core_dev *dev) +{ + struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset; + + mlx5_eq_notifier_unregister(dev, &fw_reset->nb); + destroy_workqueue(fw_reset->wq); + kvfree(dev->priv.fw_reset); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h index 1bbd95182ca6..278f538ea92a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h @@ -10,4 +10,7 @@ int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_ty int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel); int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev); +int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev); +void mlx5_fw_reset_events_cleanup(struct mlx5_core_dev *dev); + #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index b31f769d2df9..371134789375 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -47,6 +47,8 @@ enum { MAX_MISSES = 3, }; +#define MLX5_HEALTH_RR_POLL_INTERVAL (HZ / 10) + enum { MLX5_HEALTH_SYNDR_FW_ERR = 0x1, MLX5_HEALTH_SYNDR_IRISC_ERR = 0x7, @@ -222,6 +224,7 @@ void mlx5_enter_error_state(struct mlx5_core_dev *dev, bool force) #define MLX5_FW_RESET_WAIT_MS 1000 void mlx5_error_sw_reset(struct mlx5_core_dev *dev) { + struct mlx5_core_health *health = &dev->priv.health; unsigned long end, delay_ms = MLX5_FW_RESET_WAIT_MS; int lock = -EBUSY; @@ -229,7 +232,8 @@ void mlx5_error_sw_reset(struct mlx5_core_dev *dev) if (dev->state != MLX5_DEVICE_STATE_INTERNAL_ERROR) goto unlock; - mlx5_core_err(dev, "start\n"); + if (!test_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags)) + mlx5_core_err(dev, "start\n"); if (check_fatal_sensors(dev) == MLX5_SENSOR_FW_SYND_RFR) { /* Get cr-dump and reset FW semaphore */ @@ -262,7 +266,8 @@ void mlx5_error_sw_reset(struct mlx5_core_dev *dev) if (!lock) lock_sem_sw_reset(dev, false); - mlx5_core_err(dev, "end\n"); + if (!test_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags)) + mlx5_core_err(dev, "end\n"); unlock: mutex_unlock(&dev->intf_state_mutex); @@ -310,10 +315,15 @@ static void mlx5_handle_bad_state(struct mlx5_core_dev *dev) #define MLX5_RECOVERY_WAIT_MSECS 60000 static int mlx5_health_try_recover(struct mlx5_core_dev *dev) { + struct mlx5_core_health *health = &dev->priv.health; unsigned long end; - mlx5_core_warn(dev, "handling bad device here\n"); - mlx5_handle_bad_state(dev); + if (!test_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags)) { + mlx5_core_warn(dev, "handling bad device here\n"); + mlx5_handle_bad_state(dev); + } else { + mlx5_disable_device(dev); + } end = jiffies + msecs_to_jiffies(MLX5_RECOVERY_WAIT_MSECS); while (sensor_pci_not_working(dev)) { if (time_after(jiffies, end)) { @@ -324,7 +334,8 @@ static int mlx5_health_try_recover(struct mlx5_core_dev *dev) msleep(100); } - mlx5_core_err(dev, "starting health recovery flow\n"); + if (!test_and_clear_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags)) + mlx5_core_err(dev, "starting health recovery flow\n"); mlx5_recover_device(dev); if (!test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state) || check_fatal_sensors(dev)) { @@ -609,7 +620,8 @@ static void mlx5_fw_fatal_reporter_err_work(struct work_struct *work) dev = container_of(priv, struct mlx5_core_dev, priv); mlx5_enter_error_state(dev, false); - if (IS_ERR_OR_NULL(health->fw_fatal_reporter)) { + if (test_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags) || + IS_ERR_OR_NULL(health->fw_fatal_reporter)) { if (mlx5_health_try_recover(dev)) mlx5_core_err(dev, "health recovery failed\n"); return; @@ -660,13 +672,17 @@ static void mlx5_fw_reporters_destroy(struct mlx5_core_dev *dev) devlink_health_reporter_destroy(health->fw_fatal_reporter); } -static unsigned long get_next_poll_jiffies(void) +static unsigned long get_next_poll_jiffies(bool reset_requested) { unsigned long next; - get_random_bytes(&next, sizeof(next)); - next %= HZ; - next += jiffies + MLX5_HEALTH_POLL_INTERVAL; + if (reset_requested) { + next = jiffies + MLX5_HEALTH_RR_POLL_INTERVAL; + } else { + get_random_bytes(&next, sizeof(next)); + next %= HZ; + next += jiffies + MLX5_HEALTH_POLL_INTERVAL; + } return next; } @@ -699,9 +715,16 @@ static void poll_health(struct timer_list *t) fatal_error = check_fatal_sensors(dev); if (fatal_error && !health->fatal_error) { - mlx5_core_err(dev, "Fatal error %u detected\n", fatal_error); + if (!test_and_clear_bit(MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, + &health->reset_flags)) { + mlx5_core_err(dev, "Fatal error %u detected\n", + fatal_error); + print_health_info(dev); + } else { + mlx5_core_warn(dev, "Got Device Reset\n"); + set_bit(MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, &health->reset_flags); + } dev->priv.health.fatal_error = fatal_error; - print_health_info(dev); mlx5_trigger_health_work(dev); goto out; } @@ -725,7 +748,9 @@ static void poll_health(struct timer_list *t) queue_work(health->wq, &health->report_work); out: - mod_timer(&health->timer, get_next_poll_jiffies()); + mod_timer(&health->timer, + get_next_poll_jiffies(test_bit(MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, + &health->reset_flags))); } void mlx5_start_health_poll(struct mlx5_core_dev *dev) @@ -756,6 +781,15 @@ void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health) del_timer_sync(&health->timer); } +void mlx5_reload_health_poll_timer(struct mlx5_core_dev *dev) +{ + struct mlx5_core_health *health = &dev->priv.health; + + mod_timer(&health->timer, + get_next_poll_jiffies(test_bit(MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, + &health->reset_flags))); +} + void mlx5_drain_health_wq(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; @@ -768,6 +802,20 @@ void mlx5_drain_health_wq(struct mlx5_core_dev *dev) cancel_work_sync(&health->fatal_report_work); } +void mlx5_health_set_reset_requested_mode(struct mlx5_core_dev *dev) +{ + struct mlx5_core_health *health = &dev->priv.health; + + set_bit(MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, &health->reset_flags); +} + +void mlx5_health_clear_reset_requested_mode(struct mlx5_core_dev *dev) +{ + struct mlx5_core_health *health = &dev->priv.health; + + clear_bit(MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, &health->reset_flags); +} + void mlx5_health_flush(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 5fb23cfecd71..8aa4d5aec105 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -57,6 +57,7 @@ #include "lib/mpfs.h" #include "eswitch.h" #include "devlink.h" +#include "fw_reset.h" #include "lib/mlx5.h" #include "fpga/core.h" #include "fpga/ipsec.h" @@ -832,6 +833,12 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) goto err_eq_cleanup; } + err = mlx5_fw_reset_events_init(dev); + if (err) { + mlx5_core_err(dev, "failed to initialize fw reset events\n"); + goto err_events_cleanup; + } + mlx5_cq_debugfs_init(dev); mlx5_init_reserved_gids(dev); @@ -893,6 +900,8 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) mlx5_geneve_destroy(dev->geneve); mlx5_vxlan_destroy(dev->vxlan); mlx5_cq_debugfs_cleanup(dev); + mlx5_fw_reset_events_cleanup(dev); +err_events_cleanup: mlx5_events_cleanup(dev); err_eq_cleanup: mlx5_eq_table_cleanup(dev); @@ -920,6 +929,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev) mlx5_cleanup_clock(dev); mlx5_cleanup_reserved_gids(dev); mlx5_cq_debugfs_cleanup(dev); + mlx5_fw_reset_events_cleanup(dev); mlx5_events_cleanup(dev); mlx5_eq_table_cleanup(dev); mlx5_irq_table_cleanup(dev); diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 6a97ad601991..27b6086f3095 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -423,6 +423,11 @@ struct mlx5_sq_bfreg { unsigned int offset; }; +enum { + MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, + MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, +}; + struct mlx5_core_health { struct health_buffer __iomem *health; __be32 __iomem *health_counter; @@ -432,6 +437,7 @@ struct mlx5_core_health { u8 synd; u32 fatal_error; u32 crdump_size; + unsigned long reset_flags; /* wq spinlock to synchronize draining */ spinlock_t wq_lock; struct workqueue_struct *wq; @@ -501,6 +507,7 @@ struct mlx5_mpfs; struct mlx5_eswitch; struct mlx5_lag; struct mlx5_devcom; +struct mlx5_fw_reset; struct mlx5_eq_table; struct mlx5_irq_table; @@ -578,6 +585,7 @@ struct mlx5_priv { struct mlx5_core_sriov sriov; struct mlx5_lag *lag; struct mlx5_devcom *devcom; + struct mlx5_fw_reset *fw_reset; struct mlx5_core_roce roce; struct mlx5_fc_stats fc_stats; struct mlx5_rl_table rl_table; @@ -942,8 +950,11 @@ void mlx5_health_cleanup(struct mlx5_core_dev *dev); int mlx5_health_init(struct mlx5_core_dev *dev); void mlx5_start_health_poll(struct mlx5_core_dev *dev); void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health); +void mlx5_reload_health_poll_timer(struct mlx5_core_dev *dev); void mlx5_drain_health_wq(struct mlx5_core_dev *dev); void mlx5_trigger_health_work(struct mlx5_core_dev *dev); +void mlx5_health_set_reset_requested_mode(struct mlx5_core_dev *dev); +void mlx5_health_clear_reset_requested_mode(struct mlx5_core_dev *dev); int mlx5_buf_alloc(struct mlx5_core_dev *dev, int size, struct mlx5_frag_buf *buf); void mlx5_buf_free(struct mlx5_core_dev *dev, struct mlx5_frag_buf *buf); From patchwork Mon Jul 27 11:02:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336859 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcSg75Y9z9sR4 for ; Mon, 27 Jul 2020 21:06:43 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727987AbgG0LGn (ORCPT ); Mon, 27 Jul 2020 07:06:43 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40689 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728537AbgG0LGT (ORCPT ); Mon, 27 Jul 2020 07:06:19 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6CLr022234; Mon, 27 Jul 2020 14:06:12 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Clg002394; Mon, 27 Jul 2020 14:06:12 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6CYt002393; Mon, 27 Jul 2020 14:06:12 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 06/13] net/mlx5: Handle sync reset now event Date: Mon, 27 Jul 2020 14:02:26 +0300 Message-Id: <1595847753-2234-7-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On sync_reset_now event the driver does reload and PCI link toggle to activate firmware upgrade reset. When the firmware sends this event it syncs the event on all PFs, so all PFs will do PCI link toggle at once. To do PCI link toggle, the driver ensures that no other device ID under the same bridge by checking that all the PF functions under the same PCI bridge have same device ID. If no other device it uses PCI bridge link control to turn link down and up. Signed-off-by: Moshe Shemesh --- .../ethernet/mellanox/mlx5/core/fw_reset.c | 126 ++++++++++++++++++ 1 file changed, 126 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 13f83fc1783f..87f2b24f4aaa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -8,6 +8,9 @@ struct mlx5_fw_reset { struct mlx5_nb nb; struct workqueue_struct *wq; struct work_struct reset_request_work; + struct work_struct reset_now_work; + struct completion done; + int ret; }; static int mlx5_reg_mfrl_set(struct mlx5_core_dev *dev, u8 reset_level, @@ -71,6 +74,124 @@ static void mlx5_sync_reset_request_event(struct work_struct *work) mlx5_core_warn(dev, "PCI Sync FW Update Reset Ack. Device reset is expected.\n"); } +#define MLX5_PCI_LINK_UP_TIMEOUT 2000 + +static int mlx5_pci_link_toggle(struct mlx5_core_dev *dev) +{ + struct pci_bus *bridge_bus = dev->pdev->bus; + struct pci_dev *bridge = bridge_bus->self; + u16 reg16, dev_id, sdev_id; + unsigned long timeout; + struct pci_dev *sdev; + int cap, err; + u32 reg32; + + /* Check that all functions under the pci bridge are PFs of + * this device otherwise fail this function. + */ + err = pci_read_config_word(dev->pdev, PCI_DEVICE_ID, &dev_id); + if (err) + return err; + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + err = pci_read_config_word(sdev, PCI_DEVICE_ID, &sdev_id); + if (err) + return err; + if (sdev_id != dev_id) + return -EPERM; + } + + cap = pci_find_capability(bridge, PCI_CAP_ID_EXP); + if (!cap) + return -EOPNOTSUPP; + + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + pci_save_state(sdev); + pci_cfg_access_lock(sdev); + } + /* PCI link toggle */ + err = pci_read_config_word(bridge, cap + PCI_EXP_LNKCTL, ®16); + if (err) + return err; + reg16 |= PCI_EXP_LNKCTL_LD; + err = pci_write_config_word(bridge, cap + PCI_EXP_LNKCTL, reg16); + if (err) + return err; + msleep(500); + reg16 &= ~PCI_EXP_LNKCTL_LD; + err = pci_write_config_word(bridge, cap + PCI_EXP_LNKCTL, reg16); + if (err) + return err; + + /* Check link */ + err = pci_read_config_dword(bridge, cap + PCI_EXP_LNKCAP, ®32); + if (err) + return err; + if (!(reg32 & PCI_EXP_LNKCAP_DLLLARC)) { + mlx5_core_warn(dev, "No PCI link reporting capability (0x%08x)\n", reg32); + msleep(1000); + goto restore; + } + + timeout = jiffies + msecs_to_jiffies(MLX5_PCI_LINK_UP_TIMEOUT); + do { + err = pci_read_config_word(bridge, cap + PCI_EXP_LNKSTA, ®16); + if (err) + return err; + if (reg16 & PCI_EXP_LNKSTA_DLLLA) + break; + msleep(20); + } while (!time_after(jiffies, timeout)); + + if (reg16 & PCI_EXP_LNKSTA_DLLLA) { + mlx5_core_info(dev, "PCI Link up\n"); + } else { + mlx5_core_err(dev, "PCI link not ready (0x%04x) after %d ms\n", + reg16, MLX5_PCI_LINK_UP_TIMEOUT); + err = -ETIMEDOUT; + } + +restore: + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + pci_cfg_access_unlock(sdev); + pci_restore_state(sdev); + } + + return err; +} + +static void mlx5_sync_reset_now_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + reset_now_work); + struct mlx5_core_dev *dev = fw_reset->dev; + int err; + + mlx5_health_clear_reset_requested_mode(dev); + mlx5_stop_health_poll(dev, true); + + mlx5_core_warn(dev, "Sync Reset now. Device is going to reset.\n"); + + err = mlx5_cmd_fast_teardown_hca(dev); + if (err) { + mlx5_core_warn(dev, "Fast teardown failed, no reset done, err %d\n", err); + goto done; + } + + err = mlx5_pci_link_toggle(dev); + if (err) { + mlx5_core_warn(dev, "mlx5_pci_link_toggle failed, no reset done, err %d\n", err); + goto done; + } + + mlx5_enter_error_state(dev, true); + mlx5_unload_one(dev, false); +done: + if (err) + mlx5_start_health_poll(dev); + fw_reset->ret = err; + complete(&fw_reset->done); +} + static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct mlx5_eqe *eqe) { struct mlx5_eqe_sync_fw_update *sync_fw_update_eqe; @@ -82,6 +203,9 @@ static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct case MLX5_SYNC_RST_STATE_RESET_REQUEST: queue_work(fw_reset->wq, &fw_reset->reset_request_work); break; + case MLX5_SYNC_RST_STATE_RESET_NOW: + queue_work(fw_reset->wq, &fw_reset->reset_now_work); + break; } } @@ -117,10 +241,12 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) dev->priv.fw_reset = fw_reset; INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); + INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); MLX5_NB_INIT(&fw_reset->nb, fw_reset_event_notifier, GENERAL_EVENT); mlx5_eq_notifier_register(dev, &fw_reset->nb); + init_completion(&fw_reset->done); return 0; } From patchwork Mon Jul 27 11:02:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336871 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcV34g2Pz9sRN for ; Mon, 27 Jul 2020 21:07:55 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728725AbgG0LHx (ORCPT ); Mon, 27 Jul 2020 07:07:53 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40694 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728539AbgG0LGQ (ORCPT ); Mon, 27 Jul 2020 07:06:16 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6Dg9022237; Mon, 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Dfs002396; Mon, 27 Jul 2020 14:06:13 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6DUe002395; Mon, 27 Jul 2020 14:06:13 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 07/13] net/mlx5: Handle sync reset abort event Date: Mon, 27 Jul 2020 14:02:27 +0300 Message-Id: <1595847753-2234-8-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If firmware sends sync_reset_abort to driver the driver should clear the reset requested mode as reset is not expected any more. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/fw_reset.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 87f2b24f4aaa..e665080e9a4e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -9,6 +9,7 @@ struct mlx5_fw_reset { struct workqueue_struct *wq; struct work_struct reset_request_work; struct work_struct reset_now_work; + struct work_struct reset_abort_work; struct completion done; int ret; }; @@ -192,6 +193,16 @@ static void mlx5_sync_reset_now_event(struct work_struct *work) complete(&fw_reset->done); } +static void mlx5_sync_reset_abort_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + reset_abort_work); + struct mlx5_core_dev *dev = fw_reset->dev; + + mlx5_health_clear_reset_requested_mode(dev); + mlx5_core_warn(dev, "PCI Sync FW Update Reset Aborted.\n"); +} + static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct mlx5_eqe *eqe) { struct mlx5_eqe_sync_fw_update *sync_fw_update_eqe; @@ -206,6 +217,9 @@ static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct case MLX5_SYNC_RST_STATE_RESET_NOW: queue_work(fw_reset->wq, &fw_reset->reset_now_work); break; + case MLX5_SYNC_RST_STATE_RESET_ABORT: + queue_work(fw_reset->wq, &fw_reset->reset_abort_work); + break; } } @@ -242,6 +256,7 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); + INIT_WORK(&fw_reset->reset_abort_work, mlx5_sync_reset_abort_event); MLX5_NB_INIT(&fw_reset->nb, fw_reset_event_notifier, GENERAL_EVENT); mlx5_eq_notifier_register(dev, &fw_reset->nb); From patchwork Mon Jul 27 11:02:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336872 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcV8557Kz9sRR for ; Mon, 27 Jul 2020 21:08:00 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728719AbgG0LHw (ORCPT ); Mon, 27 Jul 2020 07:07:52 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40700 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728531AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6DVL022240; Mon, 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6DZ8002398; Mon, 27 Jul 2020 14:06:13 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6Dpb002397; Mon, 27 Jul 2020 14:06:13 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 08/13] net/mlx5: Add support for devlink reload level fw reset Date: Mon, 27 Jul 2020 14:02:28 +0300 Message-Id: <1595847753-2234-9-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for devlink reload level fw_reset which does firmware reset and driver entities re-instantiation. Once this reload command is executed the driver initiates fw sync reset flow, where the firmware synchronizes all PFs on coming reset and driver's entities re-instantiation. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/devlink.c | 53 +++++++++++++++++-- .../ethernet/mellanox/mlx5/core/fw_reset.c | 14 +++++ .../ethernet/mellanox/mlx5/core/fw_reset.h | 1 + 3 files changed, 64 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 5424e31a0f45..905d55cab4c3 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -4,6 +4,7 @@ #include #include "mlx5_core.h" +#include "fw_reset.h" #include "fs_core.h" #include "eswitch.h" @@ -88,13 +89,48 @@ mlx5_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, return 0; } +static int mlx5_devlink_trigger_fw_reset(struct devlink *devlink, struct netlink_ext_ack *extack) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + u8 reset_level, reset_type, net_port_alive; + int err; + + err = mlx5_reg_mfrl_query(dev, &reset_level, &reset_type); + if (err) + return err; + if (!(reset_level & MLX5_MFRL_REG_RESET_LEVEL3)) { + NL_SET_ERR_MSG_MOD(extack, "FW reset requires reboot"); + return -EINVAL; + } + + net_port_alive = !!(reset_type & MLX5_MFRL_REG_RESET_TYPE_NET_PORT_ALIVE); + err = mlx5_fw_set_reset_sync(dev, net_port_alive); + if (err) + goto out; + + err = mlx5_fw_wait_fw_reset_done(dev); +out: + if (err) + NL_SET_ERR_MSG_MOD(extack, "FW reset command failed"); + return err; +} + static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, enum devlink_reload_level level, struct netlink_ext_ack *extack) { struct mlx5_core_dev *dev = devlink_priv(devlink); - mlx5_unload_one(dev, false); - return 0; + switch (level) { + case DEVLINK_RELOAD_LEVEL_DRIVER: + mlx5_unload_one(dev, false); + return 0; + case DEVLINK_RELOAD_LEVEL_FW_RESET: + return mlx5_devlink_trigger_fw_reset(devlink, extack); + default: + /* Unsupported level should not get to this function */ + WARN_ON(1); + return -EOPNOTSUPP; + } } static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_level level, @@ -102,7 +138,15 @@ static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_l { struct mlx5_core_dev *dev = devlink_priv(devlink); - return mlx5_load_one(dev, false); + switch (level) { + case DEVLINK_RELOAD_LEVEL_DRIVER: + case DEVLINK_RELOAD_LEVEL_FW_RESET: + return mlx5_load_one(dev, false); + default: + /* Unsupported level should not get to this function */ + WARN_ON(1); + return -EOPNOTSUPP; + } } static const struct devlink_ops mlx5_devlink_ops = { @@ -118,7 +162,8 @@ static const struct devlink_ops mlx5_devlink_ops = { #endif .flash_update = mlx5_devlink_flash_update, .info_get = mlx5_devlink_info_get, - .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER), + .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER) | + BIT(DEVLINK_RELOAD_LEVEL_FW_RESET), .default_reload_level = DEVLINK_RELOAD_LEVEL_DRIVER, .reload_down = mlx5_devlink_reload_down, .reload_up = mlx5_devlink_reload_up, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index e665080e9a4e..f95df226b915 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -239,6 +239,20 @@ static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long acti return NOTIFY_OK; } +#define MLX5_FW_RESET_TIMEOUT_MSEC 5000 +int mlx5_fw_wait_fw_reset_done(struct mlx5_core_dev *dev) +{ + unsigned long timeout = msecs_to_jiffies(MLX5_FW_RESET_TIMEOUT_MSEC); + struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset; + + if (!wait_for_completion_timeout(&fw_reset->done, timeout)) { + mlx5_core_warn(dev, "FW sync reset timeout after %d seconds\n", + MLX5_FW_RESET_TIMEOUT_MSEC / 1000); + return -ETIMEDOUT; + } + return fw_reset->ret; +} + int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) { struct mlx5_fw_reset *fw_reset = kzalloc(sizeof(*fw_reset), GFP_KERNEL); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h index 278f538ea92a..d7ee951a2258 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.h @@ -10,6 +10,7 @@ int mlx5_reg_mfrl_query(struct mlx5_core_dev *dev, u8 *reset_level, u8 *reset_ty int mlx5_fw_set_reset_sync(struct mlx5_core_dev *dev, u8 reset_type_sel); int mlx5_fw_set_live_patch(struct mlx5_core_dev *dev); +int mlx5_fw_wait_fw_reset_done(struct mlx5_core_dev *dev); int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev); void mlx5_fw_reset_events_cleanup(struct mlx5_core_dev *dev); From patchwork Mon Jul 27 11:02:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336866 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTX6wJbz9sR4 for ; Mon, 27 Jul 2020 21:07:28 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728662AbgG0LHM (ORCPT ); Mon, 27 Jul 2020 07:07:12 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40706 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728536AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6Dwf022243; Mon, 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6DJB002400; Mon, 27 Jul 2020 14:06:13 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6DeK002399; Mon, 27 Jul 2020 14:06:13 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 09/13] devlink: Add enable_remote_dev_reset generic parameter Date: Mon, 27 Jul 2020 14:02:29 +0300 Message-Id: <1595847753-2234-10-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The enable_remote_dev_reset devlink param flags that the host admin allows device resets that can be initiated by other hosts. This parameter is useful for setups where a device is shared by different hosts, such as multi-host setup. Once the user set this parameter to false, the driver should NACK any attempt to reset the device while the driver is loaded. Signed-off-by: Moshe Shemesh --- Documentation/networking/devlink/devlink-params.rst | 6 ++++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 15 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index d075fd090b3d..54c9f107c4b0 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -108,3 +108,9 @@ own name. * - ``region_snapshot_enable`` - Boolean - Enable capture of ``devlink-region`` snapshots. + * - ``enable_remote_dev_reset`` + - Boolean + - Enable device reset by remote host. When cleared, the device driver + will NACK any attempt of other host to reset the device. This parameter + is useful for setups where a device is shared by different hosts, such + as multi-host setup. diff --git a/include/net/devlink.h b/include/net/devlink.h index b291cd8d6be6..125e7bb9bb82 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -420,6 +420,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_FW_LOAD_POLICY, DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE, DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE, + DEVLINK_PARAM_GENERIC_ID_ENABLE_REMOTE_DEV_RESET, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -457,6 +458,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_ENABLE_ROCE_NAME "enable_roce" #define DEVLINK_PARAM_GENERIC_ENABLE_ROCE_TYPE DEVLINK_PARAM_TYPE_BOOL +#define DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_NAME "enable_remote_dev_reset" +#define DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_TYPE DEVLINK_PARAM_TYPE_BOOL + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index f1812fc620d4..4d7b0f2a6f7b 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -3230,6 +3230,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_ENABLE_ROCE_NAME, .type = DEVLINK_PARAM_GENERIC_ENABLE_ROCE_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_REMOTE_DEV_RESET, + .name = DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_NAME, + .type = DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) From patchwork Mon Jul 27 11:02:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336870 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTy26bNz9sPf for ; Mon, 27 Jul 2020 21:07:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728710AbgG0LHo (ORCPT ); Mon, 27 Jul 2020 07:07:44 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40708 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728542AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6DXB022246; Mon, 27 Jul 2020 14:06:13 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6D2L002402; Mon, 27 Jul 2020 14:06:13 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6Dh9002401; Mon, 27 Jul 2020 14:06:13 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 10/13] net/mlx5: Add devlink param enable_remote_dev_reset support Date: Mon, 27 Jul 2020 14:02:30 +0300 Message-Id: <1595847753-2234-11-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The enable_remote_dev_reset devlink param flags that the host admin allows resets by other hosts. In case it is cleared mlx5 host PF driver will send NACK on pci sync for firmware update reset request and the command will fail. By default enable_remote_dev_reset parameter is true, so pci sync for firmware update reset is enabled. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/devlink.c | 29 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/fw_reset.c | 12 ++++++++ include/linux/mlx5/driver.h | 1 + 3 files changed, 42 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 905d55cab4c3..a81204b3dd7c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -275,6 +275,32 @@ static int mlx5_devlink_large_group_num_validate(struct devlink *devlink, u32 id } #endif +static int mlx5_devlink_enable_remote_dev_reset_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + struct mlx5_core_health *health; + + health = &dev->priv.health; + if (ctx->val.vbool) + clear_bit(MLX5_HEALTH_RESET_FLAGS_NACK_RESET_REQUEST, &health->reset_flags); + else + set_bit(MLX5_HEALTH_RESET_FLAGS_NACK_RESET_REQUEST, &health->reset_flags); + return 0; +} + +static int mlx5_devlink_enable_remote_dev_reset_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + struct mlx5_core_health *health; + + health = &dev->priv.health; + ctx->val.vbool = !test_bit(MLX5_HEALTH_RESET_FLAGS_NACK_RESET_REQUEST, + &health->reset_flags); + return 0; +} + static const struct devlink_param mlx5_devlink_params[] = { DEVLINK_PARAM_DRIVER(MLX5_DEVLINK_PARAM_ID_FLOW_STEERING_MODE, "flow_steering_mode", DEVLINK_PARAM_TYPE_STRING, @@ -290,6 +316,9 @@ static const struct devlink_param mlx5_devlink_params[] = { NULL, NULL, mlx5_devlink_large_group_num_validate), #endif + DEVLINK_PARAM_GENERIC(ENABLE_REMOTE_DEV_RESET, BIT(DEVLINK_PARAM_CMODE_RUNTIME), + mlx5_devlink_enable_remote_dev_reset_get, + mlx5_devlink_enable_remote_dev_reset_set, NULL), }; static void mlx5_devlink_set_params_init_values(struct devlink *devlink) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index f95df226b915..45fdecbadf52 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -61,12 +61,24 @@ static int mlx5_fw_set_reset_sync_ack(struct mlx5_core_dev *dev) return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, 0, 1, false); } +static int mlx5_fw_set_reset_sync_nack(struct mlx5_core_dev *dev) +{ + return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, 0, 2, false); +} + static void mlx5_sync_reset_request_event(struct work_struct *work) { struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, reset_request_work); struct mlx5_core_dev *dev = fw_reset->dev; + int err; + if (test_bit(MLX5_HEALTH_RESET_FLAGS_NACK_RESET_REQUEST, &dev->priv.health.reset_flags)) { + err = mlx5_fw_set_reset_sync_nack(dev); + mlx5_core_warn(dev, "PCI Sync FW Update Reset Nack %s", + err ? "Failed" : "Sent"); + return; + } mlx5_health_set_reset_requested_mode(dev); mlx5_reload_health_poll_timer(dev); if (mlx5_fw_set_reset_sync_ack(dev)) diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 27b6086f3095..4999dff9dc8c 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -426,6 +426,7 @@ struct mlx5_sq_bfreg { enum { MLX5_HEALTH_RESET_FLAGS_RESET_REQUESTED, MLX5_HEALTH_RESET_FLAGS_SILENT_RECOVERY, + MLX5_HEALTH_RESET_FLAGS_NACK_RESET_REQUEST, }; struct mlx5_core_health { From patchwork Mon Jul 27 11:02:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336867 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTd601wz9sRR for ; Mon, 27 Jul 2020 21:07:33 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728699AbgG0LHc (ORCPT ); Mon, 27 Jul 2020 07:07:32 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40713 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728548AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6Exr022250; Mon, 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Err002404; Mon, 27 Jul 2020 14:06:14 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6DNL002403; Mon, 27 Jul 2020 14:06:13 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 11/13] net/mlx5: Add support for fw live patch event Date: Mon, 27 Jul 2020 14:02:31 +0300 Message-Id: <1595847753-2234-12-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Firmware live patch event notifies the driver that the firmware was just updated using live patch. In such case the driver should not reload or re-initiate entities, part to updating the firmware version and re-initiate the firmware tracer which can be updated by live patch with new strings database to help debugging an issue. Signed-off-by: Moshe Shemesh --- .../mellanox/mlx5/core/diag/fw_tracer.c | 31 +++++++++++++++++++ .../mellanox/mlx5/core/diag/fw_tracer.h | 1 + .../ethernet/mellanox/mlx5/core/fw_reset.c | 27 ++++++++++++++++ include/linux/mlx5/device.h | 1 + 4 files changed, 60 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c index ad3594c4afcb..08dae045d185 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c @@ -1064,6 +1064,37 @@ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer) kvfree(tracer); } +int mlx5_fw_tracer_recreate_strings_db(struct mlx5_fw_tracer *tracer) +{ + struct mlx5_core_dev *dev; + int err; + + if (IS_ERR_OR_NULL(tracer)) + return -EINVAL; + + cancel_work_sync(&tracer->read_fw_strings_work); + mlx5_fw_tracer_clean_ready_list(tracer); + mlx5_fw_tracer_clean_print_hash(tracer); + mlx5_fw_tracer_clean_saved_traces_array(tracer); + mlx5_fw_tracer_free_strings_db(tracer); + + dev = tracer->dev; + err = mlx5_query_mtrc_caps(tracer); + if (err) { + mlx5_core_dbg(dev, "FWTracer: Failed to query capabilities %d\n", err); + return err; + } + + err = mlx5_fw_tracer_allocate_strings_db(tracer); + if (err) { + mlx5_core_warn(dev, "FWTracer: Allocate strings DB failed %d\n", err); + return err; + } + mlx5_fw_tracer_init_saved_traces_array(tracer); + + return 0; +} + static int fw_tracer_event(struct notifier_block *nb, unsigned long action, void *data) { struct mlx5_fw_tracer *tracer = mlx5_nb_cof(nb, struct mlx5_fw_tracer, nb); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h index 40601fba80ba..1a755098aeeb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.h @@ -191,5 +191,6 @@ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer); int mlx5_fw_tracer_trigger_core_dump_general(struct mlx5_core_dev *dev); int mlx5_fw_tracer_get_saved_traces_objects(struct mlx5_fw_tracer *tracer, struct devlink_fmsg *fmsg); +int mlx5_fw_tracer_recreate_strings_db(struct mlx5_fw_tracer *tracer); #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index 45fdecbadf52..87465a5e5577 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -2,11 +2,13 @@ /* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ #include "fw_reset.h" +#include "diag/fw_tracer.h" struct mlx5_fw_reset { struct mlx5_core_dev *dev; struct mlx5_nb nb; struct workqueue_struct *wq; + struct work_struct fw_live_patch_work; struct work_struct reset_request_work; struct work_struct reset_now_work; struct work_struct reset_abort_work; @@ -66,6 +68,27 @@ static int mlx5_fw_set_reset_sync_nack(struct mlx5_core_dev *dev) return mlx5_reg_mfrl_set(dev, MLX5_MFRL_REG_RESET_LEVEL3, 0, 2, false); } +static void mlx5_fw_live_patch_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + fw_live_patch_work); + struct mlx5_core_dev *dev = fw_reset->dev; + struct mlx5_fw_tracer *tracer; + + mlx5_core_info(dev, "Live patch updated firmware version: %d.%d.%d\n", fw_rev_maj(dev), + fw_rev_min(dev), fw_rev_sub(dev)); + + tracer = dev->tracer; + if (IS_ERR_OR_NULL(tracer)) + return; + + mlx5_fw_tracer_cleanup(tracer); + if (mlx5_fw_tracer_recreate_strings_db(tracer)) + mlx5_core_err(dev, "Failed to recreate FW tracer strings DB\n"); + if (mlx5_fw_tracer_init(tracer)) + mlx5_core_err(dev, "Failed to re-initialize FW tracer\n"); +} + static void mlx5_sync_reset_request_event(struct work_struct *work) { struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, @@ -241,6 +264,9 @@ static int fw_reset_event_notifier(struct notifier_block *nb, unsigned long acti struct mlx5_eqe *eqe = data; switch (eqe->sub_type) { + case MLX5_GENERAL_SUBTYPE_FW_LIVE_PATCH_EVENT: + queue_work(fw_reset->wq, &fw_reset->fw_live_patch_work); + break; case MLX5_GENERAL_SUBTYPE_PCI_SYNC_FOR_FW_UPDATE_EVENT: mlx5_sync_reset_events_handle(fw_reset, eqe); break; @@ -280,6 +306,7 @@ int mlx5_fw_reset_events_init(struct mlx5_core_dev *dev) fw_reset->dev = dev; dev->priv.fw_reset = fw_reset; + INIT_WORK(&fw_reset->fw_live_patch_work, mlx5_fw_live_patch_event); INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); INIT_WORK(&fw_reset->reset_abort_work, mlx5_sync_reset_abort_event); diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index 57db125e5802..58e63bb718a2 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -364,6 +364,7 @@ enum { enum { MLX5_GENERAL_SUBTYPE_DELAY_DROP_TIMEOUT = 0x1, MLX5_GENERAL_SUBTYPE_PCI_POWER_CHANGE_EVENT = 0x5, + MLX5_GENERAL_SUBTYPE_FW_LIVE_PATCH_EVENT = 0x7, MLX5_GENERAL_SUBTYPE_PCI_SYNC_FOR_FW_UPDATE_EVENT = 0x8, }; From patchwork Mon Jul 27 11:02:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336864 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTM5VjKz9sRR for ; Mon, 27 Jul 2020 21:07:19 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728672AbgG0LHN (ORCPT ); Mon, 27 Jul 2020 07:07:13 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40718 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728547AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6EmX022254; Mon, 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6EuV002406; Mon, 27 Jul 2020 14:06:14 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6Ex2002405; Mon, 27 Jul 2020 14:06:14 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 12/13] net/mlx5: Add support for devlink reload level live patch Date: Mon, 27 Jul 2020 14:02:32 +0300 Message-Id: <1595847753-2234-13-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for devlink reload level fw_live_patch which does live patching to firmware. The driver checks if the firmware is capable of handling the pending firmware changes as a live patch. If it is then it triggers fw_live_patch flow. Signed-off-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/devlink.c | 30 ++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index a81204b3dd7c..804e1e7b25cc 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -115,6 +115,29 @@ static int mlx5_devlink_trigger_fw_reset(struct devlink *devlink, struct netlink return err; } +static int mlx5_devlink_trigger_fw_live_patch(struct devlink *devlink, + struct netlink_ext_ack *extack) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + u8 reset_level; + int err; + + err = mlx5_reg_mfrl_query(dev, &reset_level, NULL); + if (err) + return err; + if (!(reset_level & MLX5_MFRL_REG_RESET_LEVEL0)) { + NL_SET_ERR_MSG_MOD(extack, + "FW upgrade to the stored FW can't be done by FW live patching"); + return -EINVAL; + } + + err = mlx5_fw_set_live_patch(dev); + if (err) + return err; + + return 0; +} + static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, enum devlink_reload_level level, struct netlink_ext_ack *extack) { @@ -126,6 +149,8 @@ static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, return 0; case DEVLINK_RELOAD_LEVEL_FW_RESET: return mlx5_devlink_trigger_fw_reset(devlink, extack); + case DEVLINK_RELOAD_LEVEL_FW_LIVE_PATCH: + return mlx5_devlink_trigger_fw_live_patch(devlink, extack); default: /* Unsupported level should not get to this function */ WARN_ON(1); @@ -142,6 +167,8 @@ static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_l case DEVLINK_RELOAD_LEVEL_DRIVER: case DEVLINK_RELOAD_LEVEL_FW_RESET: return mlx5_load_one(dev, false); + case DEVLINK_RELOAD_LEVEL_FW_LIVE_PATCH: + return 0; default: /* Unsupported level should not get to this function */ WARN_ON(1); @@ -163,7 +190,8 @@ static const struct devlink_ops mlx5_devlink_ops = { .flash_update = mlx5_devlink_flash_update, .info_get = mlx5_devlink_info_get, .supported_reload_levels = BIT(DEVLINK_RELOAD_LEVEL_DRIVER) | - BIT(DEVLINK_RELOAD_LEVEL_FW_RESET), + BIT(DEVLINK_RELOAD_LEVEL_FW_RESET) | + BIT(DEVLINK_RELOAD_LEVEL_FW_LIVE_PATCH), .default_reload_level = DEVLINK_RELOAD_LEVEL_DRIVER, .reload_down = mlx5_devlink_reload_down, .reload_up = mlx5_devlink_reload_up, From patchwork Mon Jul 27 11:02:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 1336863 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BFcTK1dhBz9sRR for ; Mon, 27 Jul 2020 21:07:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728681AbgG0LHO (ORCPT ); Mon, 27 Jul 2020 07:07:14 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:40719 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728546AbgG0LGR (ORCPT ); Mon, 27 Jul 2020 07:06:17 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (dev-l-vrt-135.mtl.labs.mlnx [10.234.135.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 06RB6E8t022257; Mon, 27 Jul 2020 14:06:14 +0300 Received: from dev-l-vrt-135.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Debian-10) with ESMTP id 06RB6Ef6002408; Mon, 27 Jul 2020 14:06:14 +0300 Received: (from moshe@localhost) by dev-l-vrt-135.mtl.labs.mlnx (8.15.2/8.15.2/Submit) id 06RB6EJV002407; Mon, 27 Jul 2020 14:06:14 +0300 From: Moshe Shemesh To: "David S. Miller" , Jiri Pirko , Vasundhara Volam Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next RFC 13/13] devlink: Add Documentation/networking/devlink/devlink-reload.rst Date: Mon, 27 Jul 2020 14:02:33 +0300 Message-Id: <1595847753-2234-14-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1595847753-2234-1-git-send-email-moshe@mellanox.com> References: <1595847753-2234-1-git-send-email-moshe@mellanox.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add devlink reload rst documentation file. Update index file to include it. Signed-off-by: Moshe Shemesh --- .../networking/devlink/devlink-reload.rst | 56 +++++++++++++++++++ Documentation/networking/devlink/index.rst | 1 + 2 files changed, 57 insertions(+) create mode 100644 Documentation/networking/devlink/devlink-reload.rst diff --git a/Documentation/networking/devlink/devlink-reload.rst b/Documentation/networking/devlink/devlink-reload.rst new file mode 100644 index 000000000000..092574229bab --- /dev/null +++ b/Documentation/networking/devlink/devlink-reload.rst @@ -0,0 +1,56 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============== +Devlink Reload +============== + +``devlink-reload`` provides mechanism to reload either driver or firmware or both, +depends on reload level selected. +The driver reload is used to re-initiate driver's entities, including applying +new values to ``devlink`` entities which are used during driver load, such as +``devlink-params`` in configuration mode ``driverinit`` or ``devlink-resources``. +The firmware reload is used either to reset the firmware or upgrade the firmware +if new firmware is already stored and waiting to be activated. +Some driver's may support fw_live_patch which will do firmware upgrade, +applying changes without the need for reset. + +Reload levels +============= + +Reload may be set in different reload levels. + +.. list-table:: Possible reload levels + :widths: 5 90 + + * - Name + - Description + * - ``driver`` + - Driver entities re-instantiation only. + * - ``fw_reset`` + - Firmware reset and driver entities re-instantiation. Can be used for + firmware upgrade if new firmware is stored and driver supports such + firmware upgrade. + * - ``fw_live_patch`` + - Firmware live patch, applies firmware changes without reset. + +Change namespace +================ + +All devlink instances are created in init_net and stay there for a +lifetime. Allow user to be able to move devlink instances into +namespaces during devlink reload operation. That ensures proper +re-instantiation of driver objects, including netdevices. + +example usage +------------- + +.. code:: shell + + $ devlink dev reload help + $ devlink dev reload DEV [ netns { PID | NAME | ID } ] [ level { driver | fw_reset | fw_live_patch } ] + + # Run reload command with driver's default level: + $ devlink dev reload pci/0000:82:00.0 + + # Run reload command with fw_reset level: + $ devlink dev reload pci/0000:82:00.0 level fw_reset diff --git a/Documentation/networking/devlink/index.rst b/Documentation/networking/devlink/index.rst index 7684ae5c4a4a..d82874760ae2 100644 --- a/Documentation/networking/devlink/index.rst +++ b/Documentation/networking/devlink/index.rst @@ -20,6 +20,7 @@ general. devlink-params devlink-region devlink-resource + devlink-reload devlink-trap Driver-specific documentation