From patchwork Thu Apr 12 10:38:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Talat Batheesh X-Patchwork-Id: 897647 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=mellanox.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=Mellanox.com header.i=@Mellanox.com header.b="MOYOUFSx"; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) by ozlabs.org (Postfix) with ESMTP id 40MLQl0jrHz9s2b; Thu, 12 Apr 2018 22:52:15 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1f6bhw-00052R-BH; Thu, 12 Apr 2018 12:52:08 +0000 Received: from mail-ve1eur01on0047.outbound.protection.outlook.com ([104.47.1.47] helo=EUR01-VE1-obe.outbound.protection.outlook.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1f6Zcg-0000YQ-Lx for kernel-team@lists.ubuntu.com; Thu, 12 Apr 2018 10:38:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DzRnJfOJRj2tgFMywUUefm041P0VYMUW118Tl/MX6Hs=; b=MOYOUFSxGdWPcbbmJabwye4rb+rNrRTHzJD4RUWbHq2Yoa4jXLxxsfpf2e/Ud0VdDlLd6qPjXUMHla0x2/cPSrsGnkQETV0jEXKo0irJ3iHQbAL7kpnHSv84voOinS3Ixfn87Ex+assKvSaUkqTnfbaQmMyGSXAh6/IGViq5XCg= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=talatb@mellanox.com; Received: from dev-h-vrt-015.mth.labs.mlnx (193.47.165.251) by VI1PR0501MB2094.eurprd05.prod.outlook.com (2603:10a6:800:37::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.653.12; Thu, 12 Apr 2018 10:38:32 +0000 From: Talat Batheesh To: kernel-team@lists.ubuntu.com Subject: [PATCH] net/mlx5e: Set EQE based as default TX interrupt moderation mode Date: Thu, 12 Apr 2018 13:38:19 +0300 Message-Id: <1523529499-3326-1-git-send-email-talatb@mellanox.com> X-Mailer: git-send-email 2.7.5 MIME-Version: 1.0 X-Originating-IP: [193.47.165.251] X-ClientProxiedBy: AM3PR04CA0135.eurprd04.prod.outlook.com (2603:10a6:207::19) To VI1PR0501MB2094.eurprd05.prod.outlook.com (2603:10a6:800:37::10) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(2017052603328)(7153060)(7193020); SRVR:VI1PR0501MB2094; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2094; 3:StPRqZSsbrVqBWBNnWAaifoXuNsn6mDYka3USdb/8nYkybTeOFlq/P0DzYRsbB7dzekMUOK+2SXjbJXhGVYTp8J649XZfqISUzPFPumsuvBnt/3e0BWJ271cIualys7An3IJ8AknfB7GKkGGG3gkG18ZTEsQybOajfqI3CHo0vGq33oX2pca/V2tr5htGs6ypjjTJkfVA2DvdhMQiTx5hFI+SJeRDLT7Z4YLn+8l3uBaNyM+wIG7B8ZNYEikFp5H; 25:reuufj/cEAYXNOkHBRTRgXjcxSN8ToBR3XjIPOdk/sQV44kNQh3xW8s4lrvyP7LTt+WLHvY5GjsHTP8lkwdB/c9fP/qYcZGRtbh7JBnek7Zp1re+6SluniFHvHaZp3Q0Nw7BdniMnBpaXO6c7pgUichKXca8fUbDnTJ7UgKxvbvyxvvtN6IUAV74H5+L5Y72CeUbHvK7f9/DHRBUiPrPPZKbzFK+ddSmRzlPXPk/0wX9dX61x6M1Ov9I5Tnt0uHRwzpneDni3RXsIFe57SugljlI0bTwkno93ZBLMYmNwtkp4Ie8ZhLdEho4fZ28zgGT2KZoxlH0psouznpFIDzKrw==; 31:ddrwDJtfFVUJvhqRTpLgp68mw9VjmIsBUpR/vi/PLYJpIE79gweV/N1pnxQA8wSB961T7/8fR9gQKs+59TYFdowc7VwETBTj9U03X2L+e36RYwZjzvyYjNsqTDjUB8xNbBg3XtIR8YM3xMFsAZfA7bhpFe2mLbOK+agChzKu5dwJomKVyhmY5AXohoBUn9l4388Ok4RTKoqS5w0cfpQuY75oGWxmx83qCF4cRtumQ7I= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2094: X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2094; 20:z5qotctlokvH585Om/Ah1bki2G4Wv++jg2upHmR1x4NgnNNEspJl0kh6thF6sVXLsENBQt0O+ziPpMh4LXcWl2XiY7bwNTXqV7GIS7NBlZyQoLVdK5wkd4TlQDXbUOle9ZypRBlwklMuh6fZ9Fzs8X42XX6mMThWBaJChimX+L74ga5Uvqk8p99WnQoyLuYsDO0BcDIRNbVbJkSCferMbKsyNvk1q3GdfaXijHcWzlFN+0N4BCToqZBzz0SeNkcr+U/2RXlbH3H1Mxg5EvetDA5HECi+qGR77Wljq8YbA4aKP+5et9+I113N/wa5q7vB58brojeXcD4gXN31c0gPcNIvV2Gf4LyJANPjY1Ve3Ccw8CiG/8m6+xJi+vOk+L4yrc0462rWtDlu8L/K360HAbhYVyrNwVAkRLvRHmKm8GbgoOFOYbslSGHBueXx2SunbE3eSbEBdLCVbCE1cuotLg0n62TTi9VgS12oxo90/18J+SM1RkZK0n387N4ZsMZj; 4:c/OwDf+JXQkcOsn2LjziizBfYxBBdYrHdsLCUezETgS0d5rCXbWVO6HRO6n8SRhgEktjHXk2FsqD2f7VKakx/gxfcN/IxuWuo1Y2Xltl0+2kQRfEjAEUumtu2LDFJYBDe0pJ6y2QSeArtyxCQ7T7eys7v9BVZrAyrtzJirJQZQicg1uSkZ4ZMbIJPVp72DTGNyxUvLhIZzqts4wIpdzcqtnkwK/tqTOqjVOaqgB92hLvnqpizobfxz085z6bE167a8Z7epeTVlym2Ao7Zt2RvZ2/xxvawYoNhICo1ga15G4ihbNw5NtwkXEbvxSPrlteWjbkAlGY85LfeIl8Vuy8Q9ljQZQy9izKM/6HL80UQiA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(211254476889703)(24339691489775); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231221)(944501327)(52105095)(10201501046)(3002001)(6055026)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(6072148)(201708071742011); SRVR:VI1PR0501MB2094; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2094; X-Forefront-PRVS: 06400060E1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(376002)(396003)(39860400002)(346002)(39380400002)(199004)(189003)(25786009)(6306002)(7736002)(59450400001)(26005)(16526019)(478600001)(50226002)(16586007)(186003)(305945005)(106356001)(386003)(4326008)(2361001)(1720100001)(316002)(6512007)(1857600001)(36756003)(107886003)(5660300001)(2351001)(3846002)(52116002)(8936002)(48376002)(50466002)(68736007)(2616005)(6116002)(51416003)(2906002)(81166006)(86362001)(956004)(6486002)(6916009)(6506007)(6666003)(53936002)(66066001)(486006)(105586002)(47776003)(81156014)(966005)(476003)(97736004)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2094; H:dev-h-vrt-015.mth.labs.mlnx; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0501MB2094; 23:AoMQB1DBTsvyqLBIghAEIoA/MTlmWwjDIw1eKdb?= Hv0BZj62vuoZ0u2GqdD/wxrhgs8kXOE7Pg9Yj44HGzDAOm3nPxJ5wKdGDzQu4panBmWLGqf98iasa4VLI4ccK1d+xUAZNxLnveqY3HIne2zRXT0+ysKo282xS9Vp8CmF/xxkCvYrw7MowL/pUnwQZr/VVpGj4GXXZHD3Ypyd982RdWKA1oHy0BWTuLrtLgUFUvC4zDU7OXvgBkLsCUbYI8AfI2R1VUNA29ohbT2PUfUPp54CWyNi2AyS90KJd+Bz7ShEksjS2M1bZB1jew6YzP4M9+ClofZCeFLrqTs7QC64fwinbedVmpKzG4Mkx1SP3a9hMgspoqomPiWfA9uL+LDz4foFdMj+sCiWRkCkflQUhsnbBxvC9MsJoLbtyGBeV/ypvLOkBRS7JZ+IxHZSlPgy9vcJbbSHHOPFNu/Ubq1v8xDp4sYUto0/wm3NJ+16aqMbL7eIYQVRCD9N5c2GwmBsjNAyB++dA00kdCz9Je7QTXNldgrjdj7JDmmZbf/Hpn5r+r/Y++BPQcHaG/dFmhCZJx2hk25OU9OkeKfS92NQWXInkKnrO0H2cEc3rfH93nc3NJAXllEPb9s5rbfb2N2CHPw7NineLR9ydmWkMu63CrA4mOxno/DOXmGiVfiZ1W8DbvPerfUMbynRGpILzGBYVm04SoE/ndLp/ZFVgaJvFmQYQTdi9+pKlfCI/CeeeG9NmwPxWxVezqFWaDM2DW5N450Df6T+BqJ+qi7vSKKy4M8OnovhWObDDjiXD+vSSW5PJZ5zrogTJTUlsQ5DRskNdz8B/PBHXI6Gtxvxu5hxwcJNndN5wiO91h5y4vXowIl+7lhCorjPxIT2pzDi9SEelg2goA95xrdzNrHxEIjJO8v0qAbqVSFo2IHsHPvEGGgckJpRWXx3KVees5I/fFXxOdJ3lwmaw5KSc7QrFwS3CjTmkGM9eW9wqCl0NKLLQRxexbgXdZuC5FfHZdVQtXOe54Fl40FhF5ySAiQKdS6rkHDzEU0jnz+fRQw/BtRwa5NOuyT3PPRuTml9gzNq9hJZ+gJ3Kr/Wcfu9jXF9f7e9RaxREIsWwjJqKT6umBeJGCwKNluOqazW9XGcuKjj+ubr90ubFyZa9QgboocErQcZBeZYMFo6WUInXGgej6velwyfktJQp4jRYB0e7laua+Fnpw+3EUo97tD650OVIUAClLt42l/vbBUx3KRMxdv4QIn1AhKw6tlw8x3tSf2Y1D4aN/qFWK30ebr0HN9jwgPkdGg== X-Microsoft-Antispam-Message-Info: hAbDh9eVYbjOYUG2Pp8HWldDTHE7uAH4t0i+CLbcghwM1Cn3twUookC6RKnMiaWZWUOrYxdqlX++dsEgKtu7qqY2mb4/cNRySmJfsOdBsk7HFq0LFmDih4qk5I4HvTTB+3dl7c3CsnH7FwkBI49ZDZzLLe9cw25aQErOSX1C7F8Yc9KFfsB/niQuD/HuVAu/ X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2094; 6:TPr56QAaXP3nhCiiett7eL0JRKAX9QS0K6MdHgASe5O5ncS0q41Bu8IBNEuomSdtAX2O7ambYaKoED+rtpkQTlNwEChdGiXHb+1jm/bBugc/xrYynhdxw/H5QVkaQYik177BKcjZA33OeBaOipmMVKAUSs+/176zei/eC8A8/y1A1+o862tlFaMzvJGNK/jxe81Vmf8+IwdrzH7jzE7xox5Uss4z4YzocPMyKEPm3kxm6AAqysU+nKvHSsq6OuFCR8STcU7GLNY4Gwp0T/zHmd+mmIP42Jc+RvYqi9X1UlEL1Sc5RGuiizCC4Dnekk/QfHYjV4Lb4hzJXfRGFziIpL3pj3eFAErB7mBSI+kL21lU8iTaIoJeQRff+FUMNGCfPNIwApx52VLM5t4Wp30I4hBL7aEzym8xWTVQwuz4KD42/Vl76ixQDXn5agpYpfR8g8MOFzv+KZQGTOuxbgKEVA==; 5:gkP2JK3a1SnFUeQ1Rpwb6u2u/Zh3ChsyHBLcVxS1pNsKnesCTQQWLYjpCcicUySCiUGduxClzPfKBmGPB+ACyaMVNtNf25lLQCUmyl2Wnwq10GOqU5igpNxt5DHlaij1piXcXvMcrsYDM7tuhLgV40SlNtAMYSh8eZDnAVlw04s=; 24:6Qu/A//LcIw8LpF2ZTdf5Afny91te/lc4P8+ukSFQKlR6dwl6dESd0ezIcAHZvO68/KpgPC/02k8d74xD1VEpUIHz9cjNjgO9HL87k9dyqE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2094; 7:Smz0C73H6ezumVwd/+oJPCZRgM83KuU4aYW8E3Ll4qLJJnoQcOaHsYH8is98EeiZO0cJkRJH0nSlRLdsBJMQPxH+zLC2zDg4Y2iBoySbAgRyeJRjiv6TQ/xaht75/M+KjBugPJnoRNpokVJNfzrBW2kf/aDX4/HQHFooba3PNe2JcJsfE2T+JsYV0laMDCaFcYDg5h3UcBEIwTn7RTOxVb9Ts6+cN4o6jug7tC0U0xjcQJYy63Fr0yvDK21J4QdX X-MS-Office365-Filtering-Correlation-Id: 2487d97f-4aec-4b03-aa2a-08d5a06189e6 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2018 10:38:32.3815 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2487d97f-4aec-4b03-aa2a-08d5a06189e6 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2094 X-Mailman-Approved-At: Thu, 12 Apr 2018 12:52:07 +0000 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: talgi@mellanox.com, noas@mellanox.com Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Tal Gilboa BugLink: http://bugs.launchpad.net/bugs/1763325 The default TX moderation mode was mistakenly set to CQE based. The intention was to add a control ability in order to improve some specific use-cases. In general, we prefer to use EQE based moderation as it gives much better numbers for the common cases. CQE based causes a degradation in the common case since it resets the moderation timer on CQE generation. This causes an issue when TSO is well utilized (large TSO sessions). The timer is set to 16us so traffic of ~64KB TSO sessions per second would mean timer reset (CQE per TSO session -> long time between CQEs). In this case we quickly reach the tcp_limit_output_bytes (256KB by default) and cause a halt in TX traffic. By setting EQE based moderation we make sure timer would expire after 16us regardless of the packet rate. This fixes an up to 40% packet rate and up to 23% bandwidth degradtions. Fixes: 0088cbbc4b66 ("net/mlx5e: Enable CQE based moderation on TX CQ") Signed-off-by: Tal Gilboa Signed-off-by: Saeed Mahameed Signed-off-by: David S. Miller (cherry picked from commit 48bfc39791b8b4a25f165e711f18b9c1617cefbc) Signed-off-by: Talat Batheesh Conflicts: drivers/net/ethernet/mellanox/mlx5/core/en_main.c Was conflict sine the mlx5e_build_nic_params changes over 4.16 kernel. --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 0d352d4..33faac2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4075,7 +4075,7 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, struct mlx5e_params *params, u16 max_channels) { - u8 cq_period_mode = 0; + u8 rx_cq_period_mode; u32 link_speed = 0; u32 pci_bw = 0; @@ -4111,12 +4111,12 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev, params->lro_timeout = mlx5e_choose_lro_timeout(mdev, MLX5E_DEFAULT_LRO_TIMEOUT); /* CQ moderation params */ - cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? + rx_cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ? MLX5_CQ_PERIOD_MODE_START_FROM_CQE : MLX5_CQ_PERIOD_MODE_START_FROM_EQE; params->rx_am_enabled = MLX5_CAP_GEN(mdev, cq_moderation); - mlx5e_set_rx_cq_mode_params(params, cq_period_mode); - mlx5e_set_tx_cq_mode_params(params, cq_period_mode); + mlx5e_set_rx_cq_mode_params(params, rx_cq_period_mode); + mlx5e_set_tx_cq_mode_params(params, MLX5_CQ_PERIOD_MODE_START_FROM_EQE); /* TX inline */ params->tx_max_inline = mlx5e_get_max_inline_cap(mdev);