From patchwork Tue Aug 22 17:31:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Madalin Bucur X-Patchwork-Id: 804606 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3xcHtk5gzgz9s76 for ; Wed, 23 Aug 2017 03:42:06 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3xcHtk4MZMzDqR9 for ; Wed, 23 Aug 2017 03:42:06 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0064.outbound.protection.outlook.com [104.47.38.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3xcHfK3s6SzDrCf for ; Wed, 23 Aug 2017 03:31:21 +1000 (AEST) Received: from MWHPR03CA0004.namprd03.prod.outlook.com (10.175.133.142) by CY4PR03MB3320.namprd03.prod.outlook.com (10.171.247.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1362.18; Tue, 22 Aug 2017 17:31:17 +0000 Received: from BY2FFO11OLC005.protection.gbl (2a01:111:f400:7c0c::125) by MWHPR03CA0004.outlook.office365.com (2603:10b6:300:117::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1385.8 via Frontend Transport; Tue, 22 Aug 2017 17:31:16 +0000 Authentication-Results: spf=fail (sender IP is 192.88.168.50) smtp.mailfrom=nxp.com; vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=fail action=none header.from=nxp.com; Received-SPF: Fail (protection.outlook.com: domain of nxp.com does not designate 192.88.168.50 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.168.50; helo=tx30smr01.am.freescale.net; Received: from tx30smr01.am.freescale.net (192.88.168.50) by BY2FFO11OLC005.mail.protection.outlook.com (10.1.14.145) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1341.15 via Frontend Transport; Tue, 22 Aug 2017 17:31:16 +0000 Received: from fsr-fed2164-101.ea.freescale.net (fsr-fed2164-101.ea.freescale.net [10.171.73.197]) by tx30smr01.am.freescale.net (8.14.3/8.14.0) with ESMTP id v7MHV6Ks013594; Tue, 22 Aug 2017 10:31:14 -0700 From: Madalin Bucur To: , Subject: [PATCH v2 5/6] Documentation: networking: add RSS information Date: Tue, 22 Aug 2017 20:31:05 +0300 Message-ID: <1503423066-15420-6-git-send-email-madalin.bucur@nxp.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1503423066-15420-1-git-send-email-madalin.bucur@nxp.com> References: <1503423066-15420-1-git-send-email-madalin.bucur@nxp.com> X-EOPAttributedMessage: 0 X-Matching-Connectors: 131478966769469279; (91ab9b29-cfa4-454e-5278-08d120cd25b8); () X-Forefront-Antispam-Report: CIP:192.88.168.50; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(336005)(39380400002)(39860400002)(2980300002)(1109001)(1110001)(339900001)(199003)(189002)(8676002)(36756003)(86362001)(81166006)(77096006)(626005)(85426001)(305945005)(8936002)(4326008)(43066003)(53936002)(2950100002)(189998001)(97736004)(54906002)(6666003)(50986999)(76176999)(498600001)(48376002)(47776003)(5660300001)(356003)(33646002)(3450700001)(104016004)(50466002)(2906002)(68736007)(81156014)(5003940100001)(106466001)(105606002)(50226002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR03MB3320; H:tx30smr01.am.freescale.net; FPR:; SPF:Fail; PTR:InfoDomainNonexistent; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BY2FFO11OLC005; 1:YrzEv45kIkK4vgOz4WZPplZpKaDig5jSFlde7a+oCv2jw6N/PA9esAyhJpCYayyC5vma+/Jke2sfW5RasmqdaHOFcJubzjnrAvEhC1bDs1k1qy4lLv7UOKAO+cCAHDzL MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 121dc4cb-2db9-4310-5aff-08d4e983981c X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(2017052603171)(201703131430075)(201703131517081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:CY4PR03MB3320; X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB3320; 3:dyjzAxFaqVSTOtH7h7wHQjlduWIfVhXfXwQjTRNfdmByi3dco0ppuCLjVFDrFKQlsZgiHweXC28J4JGqWbCgjlyf3yU76ds2ZA0rB8woyMeEwMwHyX6OHozKluWMuDupmtgFnRDxlEEqkMy3Alv4gXUKctUNCfiYzXl/vgjRhjyYcTBpSR1QfuG163MCtb8bfLQImSaG6LyZBoI4ffF1PepIXopdq/HRchVrDnXE2YwVQY1exVqLHkM6CS5N6e5+/1wZ0WPWohAPZ+35CBOaVQOuJR8AmpSnXviGPREHNIZ+MOMhf9vJusYcXs9atPVTmqg4S/I9JsaUfXnVSd80FcgnzCgKUreu6tI5kZXoaOw=; 25:H95Wy+X3+fOVVBAnFPrWpCSn1uDoFThc8bAsSuEgDsbW5YrkxHm4ftDPHDmhrF12ziFfhXYSk0Qm2ovwON+G6RQh9sxAS1/Rb0VQw7LJG6tSiTU9Xchn9pGQHtEdt5D1Y9ozFPnFBdCJuAGU0v4iHH0G7iMeLQd7YvA+pyIW7zaa5LkLYfk3DqoIxHUGF8YKaSRVtru9eF6uFsTrdaQVel9tvqo3qnhGulk7QsQ9bP3lD8w7PhsfN2iyE0u8TMtTq1EEgj4eEGpLsCJhIAQOcSV4roWFLXSuNOXK3Wjtyomqds0ZYhMgtTO/0JCgX4DYyVBtgI130LT/v5oUaA8XUg== X-MS-TrafficTypeDiagnostic: CY4PR03MB3320: X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB3320; 31:dBRi04LA5zX4+jwYdlBbzzZuDUtS2XeKmbCexLobAcPV8Dd4tJGFu0wiGstPCKT6c4guOHAiiq7ADNupin1d/C3smYkAHyqdAJg+YZRMS48U8N5R+SlZiavEqhbk3kHa03EtUUEkGd+g9IRcHX7hlpc1/IpHLawfL75cHBsEIaxccnvcMulsziMTRmBp0d2Pg0zePxh51kWEMLFgOFYaQvn2hVj3JmM1Y497lSqKihM=; 4:EqORSMgDuK+D4ccQKn5evp3ur6Muy/WUwwUvenFHbYLYSwuyTrbFKN3vzJgBlmxF2lewhHOXMiunCjOBx7TYHFUMNdXM286rhYnNq4G316fNMTZQD54TQ33QfnbJkh4svO9e8AMT6aFty33WWR4cmtHpKBopPVNxiwXXg0/dzrILVwuZ4POuadpVyOhD6mucbqkdo9Otbpj1zhQLGoctH9JyUDtmRE9P+xxsoVzAoBcyASlj4Ag43vBrWGSUMzmdtjd5zSo0FurIN245BWm6wcRMovWkFBZTxUQA4zpiPQzJZ1tQLjPa4wqAYZvN9u4N5iZnvCk7xsvSLQ9rMVdEooTC7PPn9O8qM1g97x4XAs65AGfsC4hDaM35zNCtljLW X-Exchange-Antispam-Report-Test: UriScan:(185117386973197)(275809806118684)(211171220733660); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6095135)(601004)(2401047)(8121501046)(5005006)(13018025)(13016025)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(3002001)(6055026)(6096035)(20161123561025)(20161123559100)(20161123556025)(20161123565025)(20161123563025)(201703131430075)(201703131433075)(201703131441075)(201703131448075)(201703161259150)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:CY4PR03MB3320; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(400006)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:CY4PR03MB3320; X-Forefront-PRVS: 04073E895A X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR03MB3320; 23:HTJCg60ZXr5jeuJMi0fr1UJh+ilgnsttrWqG38It1?= =?us-ascii?Q?Xo5sg+mDCWUIegcwc8f2c+C8v9YxVzj8JNFKdML5n1EWAVgrtbOqi4zkakxK?= =?us-ascii?Q?I9u08Imz3q9yT14uN5mhrSnkOVAGuc6JScmJrgJ5iO8NyWL2SM3iucph16hu?= =?us-ascii?Q?+5mGoMBUSqIOhT6/GzQ6j+ly3AwKkWiJMOoSUBZTWZrQrMRym4VFwTEXHM93?= =?us-ascii?Q?o+fW0iq+2a3Y8WFjNoKt7a1DgbV1Cb6KsWQ4dhU7qffyuepy93PTc6rCwCRu?= =?us-ascii?Q?5xaIqtF6+hvV61bJZLAmEaecp7xe59DjxYz1L4YWs3qWNuxfFgSF+f9l/Z7z?= =?us-ascii?Q?M8FjI0FL9EEqEzfRknWBpzQobgg+lgKq1cNPhbscUgFDJ9TvE2YLw6RkjxPt?= =?us-ascii?Q?CXUR2/lbRbVht1sk1Wq4X+yUyqBsGah/5C2VCLJMxV22eMJa8CiUIoOlIcQ6?= =?us-ascii?Q?ML/XVR4quJZJR4KynJ2o7iTg/SAHGIgYukcFeZ80FBr7+Ye4n9Fzrh3DHCSG?= =?us-ascii?Q?XcsZXkmeMh5sOd69+LM8mUilEvVRVSFMdidt7eE+wfAU4BBPW2lH6Rk6dKGj?= =?us-ascii?Q?Dw6rwYGiU8QQ44QikpBNIQHzOi3ReNkzaoJiOzdUlmPf4wRAsbrJZzmAJIYt?= =?us-ascii?Q?5g17YVMxtU/77KpfKD/xZ4Bat697UIFqUGB+dmuyI15cACnH/ZmVcPCPN4OY?= =?us-ascii?Q?nOELplqaMzvE+ebYH2vLUIqFAvV8Zzmk4uWNvzCBjdlW5lHFXq0yo3p8aXfJ?= =?us-ascii?Q?Y2R8tGnMDG2s2d52kFq0TfX3JV4A1HAX7tl1s+0UiRTmYIliBmzVH7kCxS41?= =?us-ascii?Q?T2zNUqbkw/IwR7dx+4PFArFMXBtmKS5uhUIZyBzLngaX9Q4INPaGhSIZEbsv?= =?us-ascii?Q?l/cV5CcPXCJhvQh43+4+wwTQEyWi/wjdm50+vRxdlza+iLNbu11fumIdJleg?= =?us-ascii?Q?eoJWHyMX41R89W8iJeJ++f1KperF9EFTMeHhLDqh1URutsrz/P1RJYOxHDcT?= =?us-ascii?Q?j9Qx83X2Oj0bp19VgV+a/hFROlbLzMOI6BBWDnvmL7RsqFhaeosJ3r8OLnUD?= =?us-ascii?Q?jbpqQugef0mST2Q3A3zvNtqKPzNXeo1ay6oM0JwDB8oF/ZZDQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR03MB3320; 6:nh2udTPG8ghhzUIX5Iy8KmRZIXh+AooPwSg3uEpyJRrY9r/p4gYIXWizxd0ALSdI4krkHNspGyEGMJG7l+yHzV0gEIrouwAYH9l1naFn51f49kB89FZRT4Vwf3Pvx9fOr5FURVI1BzC1kWgqOG9JBlXUUa4JObVfHKf/lFNIkdsO0opca4RxwqoqzuTV5iSehwB74yPATPZfND/Ra0nDULXXWNyMqtmaQMdNXMWrXSHJ/yD2go7QDgxUlv2nhh0D6Lmm6v5B8sDSh4fSQPZkxjqCeQa8Z19Y/LjwaxnziSkOMV9JSxudVqbp5TDehMbC9NTmYL5602gb2IK/dJDEaw==; 5:ZjiQ+vv5kzanQkGsnCjxiR0E21Ji8FNc6e/wkpZbNBOSFcQOvrpx7BPNq3L6XlL4i2O8Z914pE03ErxfjGl9oHkxHCh5tHMbXXsAM234MXIj4qDK3+q6nfpjDz1M3jhd/fz8sxPoT70fiit4yzT7XA==; 24:XXtwUMudN5Fsxj4q8+h73GLDIS/tsY8zdyKBkTsRUsqGw1X54uhtALs/VnEueDSj228de1dW+zsTP370aWxcOZHEFkS6rh+AAf/ORsbbElI=; 7:FVYnmefdo/drhc0U1p8TXZnDlL55zpmYyFN+pZrkAOhgR0YO0B3B5FcmRDYFNVMFmOFEwyn/IRkoKSrXMFnC65e2z6E5iZiZNcIZXNJ3t4+qmsI9+Kyc9pLY18TNK7TEPnQxIDrdl9NLOWgtqwLNmbc7iOTkS/YGGPQDLl1vOHojNDsKCcqaxdTTY794TBTu8CLg9aWsaY+el9jmbsuCUlNhaSrL3OsgJ5bWKXGk+SQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Aug 2017 17:31:16.6193 (UTC) X-MS-Exchange-CrossTenant-Id: 5afe0b00-7697-4969-b663-5eab37d5f47e X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=5afe0b00-7697-4969-b663-5eab37d5f47e; Ip=[192.88.168.50]; Helo=[tx30smr01.am.freescale.net] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR03MB3320 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: madalin.bucur@nxp.com Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Signed-off-by: Madalin Bucur --- Documentation/networking/dpaa.txt | 68 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/Documentation/networking/dpaa.txt b/Documentation/networking/dpaa.txt index 76e016d..f88194f 100644 --- a/Documentation/networking/dpaa.txt +++ b/Documentation/networking/dpaa.txt @@ -13,6 +13,7 @@ Contents - Configuring DPAA Ethernet in your kernel - DPAA Ethernet Frame Processing - DPAA Ethernet Features + - DPAA IRQ Affinity and Receive Side Scaling - Debugging DPAA Ethernet Overview @@ -147,7 +148,10 @@ gradually. The driver has Rx and Tx checksum offloading for UDP and TCP. Currently the Rx checksum offload feature is enabled by default and cannot be controlled through -ethtool. +ethtool. Also, rx-flow-hash and rx-hashing was added. The addition of RSS +provides a big performance boost for the forwarding scenarios, allowing +different traffic flows received by one interface to be processed by different +CPUs in parallel. The driver has support for multiple prioritized Tx traffic classes. Priorities range from 0 (lowest) to 3 (highest). These are mapped to HW workqueues with @@ -166,6 +170,68 @@ classes as follows: tc qdisc add dev root handle 1: \ mqprio num_tc 4 map 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 hw 1 +DPAA IRQ Affinity and Receive Side Scaling +========================================== + +Traffic coming on the DPAA Rx queues or on the DPAA Tx confirmation +queues is seen by the CPU as ingress traffic on a certain portal. +The DPAA QMan portal interrupts are affined each to a certain CPU. +The same portal interrupt services all the QMan portal consumers. + +By default the DPAA Ethernet driver enables RSS, making use of the +DPAA FMan Parser and Keygen blocks to distribute traffic on 128 +hardware frame queues using a hash on IP v4/v6 source and destination +and L4 source and destination ports, in present in the received frame. +When RSS is disabled, all traffic received by a certain interface is +received on the default Rx frame queue. The default DPAA Rx frame +queues are configured to put the received traffic into a pool channel +that allows any available CPU portal to dequeue the ingress traffic. +The default frame queues have the HOLDACTIVE option set, ensuring that +traffic bursts from a certain queue are serviced by the same CPU. +This ensures a very low rate of frame reordering. A drawback of this +is that only one CPU at a time can service the traffic received by a +certain interface when RSS is not enabled. + +To implement RSS, the DPAA Ethernet driver allocates an extra set of +128 Rx frame queues that are configured to dedicated channels, in a +round-robin manner. The mapping of the frame queues to CPUs is now +hardcoded, there is no indirection table to move traffic for a certain +FQ (hash result) to another CPU. The ingress traffic arriving on one +of these frame queues will arrive at the same portal and will always +be processed by the same CPU. This ensures intra-flow order preservation +and workload distribution for multiple traffic flows. + +RSS can be turned off for a certain interface using ethtool, i.e. + + # ethtool -N fm1-mac9 rx-flow-hash tcp4 "" + +To turn it back on, one needs to set rx-flow-hash for tcp4/6 or udp4/6: + + # ethtool -N fm1-mac9 rx-flow-hash udp4 sfdn + +There is no independent control for individual protocols, any command +run for one of tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 is +going to control the rx-flow-hashing for all protocols on that interface. + +Besides using the FMan Keygen computed hash for spreading traffic on the +128 Rx FQs, the DPAA Ethernet driver also sets the skb hash value when +the NETIF_F_RXHASH feature is on (active by default). This can be turned +on or off through ethtool, i.e.: + + # ethtool -K fm1-mac9 rx-hashing off + # ethtool -k fm1-mac9 | grep hash + receive-hashing: off + # ethtool -K fm1-mac9 rx-hashing on + Actual changes: + receive-hashing: on + # ethtool -k fm1-mac9 | grep hash + receive-hashing: on + +Please note that Rx hashing depends upon the rx-flow-hashing being on +for that interface - turning off rx-flow-hashing will also disable the +rx-hashing (without ethtool reporting it as off as that depends on the +NETIF_F_RXHASH feature flag). + Debugging =========