Message ID | 1523529499-3326-1-git-send-email-talatb@mellanox.com |
---|---|
State | New |
Headers | show |
Series | net/mlx5e: Set EQE based as default TX interrupt moderation mode | expand |
On 04/12/18 12:38, Talat Batheesh wrote: > From: Tal Gilboa <talgi@mellanox.com> > > BugLink: http://bugs.launchpad.net/bugs/1763325 > > The default TX moderation mode was mistakenly set to CQE based. The > intention was to add a control ability in order to improve some specific > use-cases. In general, we prefer to use EQE based moderation as it gives > much better numbers for the common cases. > > CQE based causes a degradation in the common case since it resets the > moderation timer on CQE generation. This causes an issue when TSO is > well utilized (large TSO sessions). The timer is set to 16us so traffic > of ~64KB TSO sessions per second would mean timer reset (CQE per TSO > session -> long time between CQEs). In this case we quickly reach the > tcp_limit_output_bytes (256KB by default) and cause a halt in TX traffic. > > By setting EQE based moderation we make sure timer would expire after > 16us regardless of the packet rate. > This fixes an up to 40% packet rate and up to 23% bandwidth degradtions. > > Fixes: 0088cbbc4b66 ("net/mlx5e: Enable CQE based moderation on TX CQ") > Signed-off-by: Tal Gilboa <talgi@mellanox.com> > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com> > Signed-off-by: David S. Miller <davem@davemloft.net> > (cherry picked from commit 48bfc39791b8b4a25f165e711f18b9c1617cefbc) > Signed-off-by: Talat Batheesh <talatb@mellanox.com> > > Conflicts: > drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > Was conflict sine the mlx5e_build_nic_params changes over 4.16 kernel. > --- > drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index 0d352d4..33faac2 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -4075,7 +4075,7 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, > struct mlx5e_params *params, > u16 max_channels) > { > - u8 cq_period_mode = 0; > + u8 rx_cq_period_mode; > u32 link_speed = 0; > u32 pci_bw = 0; > > @@ -4111,12 +4111,12 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, > params->lro_timeout = mlx5e_choose_lro_timeout(mdev, MLX5E_DEFAULT_LRO_TIMEOUT); > > /* CQ moderation params */ > - cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? > + rx_cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? > MLX5_CQ_PERIOD_MODE_START_FROM_CQE : > MLX5_CQ_PERIOD_MODE_START_FROM_EQE; > params->rx_am_enabled = MLX5_CAP_GEN(mdev, cq_moderation); > - mlx5e_set_rx_cq_mode_params(params, cq_period_mode); > - mlx5e_set_tx_cq_mode_params(params, cq_period_mode); > + mlx5e_set_rx_cq_mode_params(params, rx_cq_period_mode); > + mlx5e_set_tx_cq_mode_params(params, MLX5_CQ_PERIOD_MODE_START_FROM_EQE); > > /* TX inline */ > params->tx_max_inline = mlx5e_get_max_inline_cap(mdev); > Note: this patch is targeted for BIONIC. Kleber
On Thu, Apr 12, 2018 at 01:38:19PM +0300, Talat Batheesh wrote: > From: Tal Gilboa <talgi@mellanox.com> > > BugLink: http://bugs.launchpad.net/bugs/1763325 > > The default TX moderation mode was mistakenly set to CQE based. The > intention was to add a control ability in order to improve some specific > use-cases. In general, we prefer to use EQE based moderation as it gives > much better numbers for the common cases. > > CQE based causes a degradation in the common case since it resets the > moderation timer on CQE generation. This causes an issue when TSO is > well utilized (large TSO sessions). The timer is set to 16us so traffic > of ~64KB TSO sessions per second would mean timer reset (CQE per TSO > session -> long time between CQEs). In this case we quickly reach the > tcp_limit_output_bytes (256KB by default) and cause a halt in TX traffic. > > By setting EQE based moderation we make sure timer would expire after > 16us regardless of the packet rate. > This fixes an up to 40% packet rate and up to 23% bandwidth degradtions. > > Fixes: 0088cbbc4b66 ("net/mlx5e: Enable CQE based moderation on TX CQ") > Signed-off-by: Tal Gilboa <talgi@mellanox.com> > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com> > Signed-off-by: David S. Miller <davem@davemloft.net> > (cherry picked from commit 48bfc39791b8b4a25f165e711f18b9c1617cefbc) > Signed-off-by: Talat Batheesh <talatb@mellanox.com> > > Conflicts: > drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > Was conflict sine the mlx5e_build_nic_params changes over 4.16 kernel. This patch has already been applied to bionic throught v4.15.17 stable update. Thanks!
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 0d352d4..33faac2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4075,7 +4075,7 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, struct mlx5e_params *params, u16 max_channels) { - u8 cq_period_mode = 0; + u8 rx_cq_period_mode; u32 link_speed = 0; u32 pci_bw = 0; @@ -4111,12 +4111,12 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, params->lro_timeout = mlx5e_choose_lro_timeout(mdev, MLX5E_DEFAULT_LRO_TIMEOUT); /* CQ moderation params */ - cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? + rx_cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? MLX5_CQ_PERIOD_MODE_START_FROM_CQE : MLX5_CQ_PERIOD_MODE_START_FROM_EQE; params->rx_am_enabled = MLX5_CAP_GEN(mdev, cq_moderation); - mlx5e_set_rx_cq_mode_params(params, cq_period_mode); - mlx5e_set_tx_cq_mode_params(params, cq_period_mode); + mlx5e_set_rx_cq_mode_params(params, rx_cq_period_mode); + mlx5e_set_tx_cq_mode_params(params, MLX5_CQ_PERIOD_MODE_START_FROM_EQE); /* TX inline */ params->tx_max_inline = mlx5e_get_max_inline_cap(mdev);