From patchwork Wed Jan 16 16:51:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ioana Radulescu X-Patchwork-Id: 1026039 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=nxp.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=nxp.com header.i=@nxp.com header.b="nzkKK+qj"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 43ftY96kdbz9sCX for ; Thu, 17 Jan 2019 03:52:29 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731232AbfAPQw2 (ORCPT ); Wed, 16 Jan 2019 11:52:28 -0500 Received: from mail-eopbgr00065.outbound.protection.outlook.com ([40.107.0.65]:28576 "EHLO EUR02-AM5-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727337AbfAPQw1 (ORCPT ); Wed, 16 Jan 2019 11:52:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EgOJtAhgTX8GoqXZ1RsuFdNP2Ur6I9sCGCF1TO3aRAw=; b=nzkKK+qjAgTH6dENLo5qQ32z66cKZKj7h0ybg6oqGh8pspPqJYGpAJCOvZP+aI1uPVNXeQo896ilCvOIsWcES3sj6vHGVY7INMAG96eY83OAgADzKwox0xw8Tt3a3TfsaoJFBFJqleq3hqDxvpYZsL0XTduptJtWtn25/vIRKB8= Received: from AM6PR04MB5000.eurprd04.prod.outlook.com (20.177.33.219) by AM6PR04MB4949.eurprd04.prod.outlook.com (20.177.35.208) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.18; Wed, 16 Jan 2019 16:51:44 +0000 Received: from AM6PR04MB5000.eurprd04.prod.outlook.com ([fe80::a56b:316f:9d27:fbd4]) by AM6PR04MB5000.eurprd04.prod.outlook.com ([fe80::a56b:316f:9d27:fbd4%5]) with mapi id 15.20.1516.019; Wed, 16 Jan 2019 16:51:44 +0000 From: Ioana Ciocoi Radulescu To: "netdev@vger.kernel.org" , "davem@davemloft.net" CC: Ioana Ciornei Subject: [PATCH net-next] dpaa2-eth: Fix ndo_stop routine Thread-Topic: [PATCH net-next] dpaa2-eth: Fix ndo_stop routine Thread-Index: AQHUrbvDQKqd+Rn7eUyUIXksV3FtYw== Date: Wed, 16 Jan 2019 16:51:44 +0000 Message-ID: <1547657499-32122-1-git-send-email-ruxandra.radulescu@nxp.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: VI1P194CA0013.EURP194.PROD.OUTLOOK.COM (2603:10a6:800:be::23) To AM6PR04MB5000.eurprd04.prod.outlook.com (2603:10a6:20b:7::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=ruxandra.radulescu@nxp.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.7.4 x-originating-ip: [86.34.165.90] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR04MB4949; 6:WTfZIVTzA8PrpMwRqSAxhLwmWUSFYw0trX23LO1nvfUuZ14GbI9a283lv5xXHX1lSe7M4GO/FKle8y081F4Fi38FIZtHsuSu2bLJ98K0JE0krLHoKcputp2Nf/3KMujTc7fkUAR1ZfiALyLDM1jY+HUzv3okPHV8ZNjHwyvoe7ieJLqWxeyECoaRK6YDp3Pfyi37P/Z4Lq8hDNLP7cbj4r978aZlbEU3bVmsYNtL5AzKwtTn8U92WU+jhKDtUGdt7j/SLVnD7uzNcfvP5f1VDpQ/9UnWa71ysdNmQulaUksWE6kKApOcb6nDMQ5yl6+bNOauxpdC9WAkP8uErpCFVWKXKQG9q605BE760wz4q3Rl1xZStAjRJIXm5rilCA+2+EE933G2Px533vCD9jTJdwNpBgK2U7soNWWdLmRV0DhpI1JTos6hji+/cSJ5wwSrW66M8H6QaF8fE/CJiAt7Rg==; 5:A7920vjhDrufLx0TLYVpHHiTZuiGEkAJ3UyKr/NsqcmQmBDgB14iwCUxHpVdkoxKzW6IchAIqmkSg+aw2vswx1AVbSX3GFBKmaszumUfq+cFPkCrW5+ca9dS0mH3z7HZbuOq3uw5Bc9Ghed548UMDkVyPdfHrj6YZZgvrj3T0rQmoBJ27jDdUaKFLF4ZSB86lHckfxwMgNjyiuTNB9jqKw==; 7:O/SidjyeRALcvZkL66Rho/hAOlVMffnMnsVr3CuX6CZu1r5VRPJpbssM+lz1gaY1S1b+385c0vUgCGgk8JBvM1HD6sHyzj9DaJFRMnBTqe70wFG9C5UcVNixidHQBsKk8V31GujmcscEuDUjXHXbhQ== x-ms-office365-filtering-correlation-id: d86c49eb-8c95-4a3d-5849-08d67bd2e568 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM6PR04MB4949; x-ms-traffictypediagnostic: AM6PR04MB4949: x-microsoft-antispam-prvs: x-forefront-prvs: 091949432C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(136003)(39860400002)(396003)(346002)(189003)(199004)(25786009)(8936002)(4326008)(486006)(7736002)(81166006)(81156014)(68736007)(97736004)(5660300001)(105586002)(305945005)(8676002)(106356001)(36756003)(50226002)(6512007)(53936002)(3846002)(86362001)(2906002)(2501003)(386003)(71190400001)(52116002)(478600001)(26005)(14454004)(476003)(99286004)(6116002)(6506007)(2616005)(102836004)(14444005)(256004)(316002)(186003)(66066001)(71200400001)(110136005)(6486002)(6436002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR04MB4949; H:AM6PR04MB5000.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: r2+qsbvAs+rkJPb2QD/PCq/cellZV9RAFu4aaDLtF65iUi0doCZdYcVTbTledPvMbdzsy6boV/BjIHyyzkBlTsHIvXx5GFOzZp60R/h+l1aIw9LDmXvAvAlZuiof6WdoDVENXtrqr61nigal2UpQmsUnEBt+q1+UhT9SFTOAc0B6YlxygTPE5Dq1p5PxhLof+BvkCkS//S5fKoal2JJbZY6eBNvATINZ/gtUAMOb5qrRSaiEciRJkp2PchVx8Kplp2xdsgKvILNkocnSaSMljXscXmiWQhy4cY4/1csoKYo8VGy+1J9R4d9yr2pF23zo4Q4Kp7FqeOrLq31sJD2Izl3lF4ZH3RODlGQ4vzQSdg55PYur8B0OBGw1lKXVPK+DB9DTFMhxTFrRrvw3daWHUVKUCoH7vHbz4JP3qVgvVQE= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: d86c49eb-8c95-4a3d-5849-08d67bd2e568 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jan 2019 16:51:43.8961 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB4949 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In the current implementation, on interface down we disabled NAPI and then manually drained any remaining ingress frames. This could lead to a situation when, under heavy traffic, the data availability notification for some of the channels would not get rearmed correctly. Change the implementation such that we let all remaining ingress frames be processed as usual and only disable NAPI once the hardware queues are empty. We also add a wait on the Tx side, to allow hardware time to process all in-flight Tx frames before issueing the disable command. Signed-off-by: Ioana Radulescu --- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 68 ++++++++++++------------ 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 1ca9a18..ecf9218 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -1243,34 +1243,36 @@ static int dpaa2_eth_open(struct net_device *net_dev) return err; } -/* The DPIO store must be empty when we call this, - * at the end of every NAPI cycle. - */ -static u32 drain_channel(struct dpaa2_eth_channel *ch) +/* Total number of in-flight frames on ingress queues */ +static u32 ingress_fq_count(struct dpaa2_eth_priv *priv) { - u32 drained = 0, total = 0; + struct dpaa2_eth_fq *fq; + u32 fcnt = 0, bcnt = 0, total = 0; + int i, err; - do { - pull_channel(ch); - drained = consume_frames(ch, NULL); - total += drained; - } while (drained); + for (i = 0; i < priv->num_fqs; i++) { + fq = &priv->fq[i]; + err = dpaa2_io_query_fq_count(NULL, fq->fqid, &fcnt, &bcnt); + if (err) { + netdev_warn(priv->net_dev, "query_fq_count failed"); + break; + } + total += fcnt; + } return total; } -static u32 drain_ingress_frames(struct dpaa2_eth_priv *priv) +static void wait_for_fq_empty(struct dpaa2_eth_priv *priv) { - struct dpaa2_eth_channel *ch; - int i; - u32 drained = 0; - - for (i = 0; i < priv->num_channels; i++) { - ch = priv->channel[i]; - drained += drain_channel(ch); - } + int retries = 10; + u32 pending; - return drained; + do { + pending = ingress_fq_count(priv); + if (pending) + msleep(100); + } while (pending && --retries); } static int dpaa2_eth_stop(struct net_device *net_dev) @@ -1278,14 +1280,22 @@ static int dpaa2_eth_stop(struct net_device *net_dev) struct dpaa2_eth_priv *priv = netdev_priv(net_dev); int dpni_enabled = 0; int retries = 10; - u32 drained; netif_tx_stop_all_queues(net_dev); netif_carrier_off(net_dev); - /* Loop while dpni_disable() attempts to drain the egress FQs - * and confirm them back to us. + /* On dpni_disable(), the MC firmware will: + * - stop MAC Rx and wait for all Rx frames to be enqueued to software + * - cut off WRIOP dequeues from egress FQs and wait until transmission + * of all in flight Tx frames is finished (and corresponding Tx conf + * frames are enqueued back to software) + * + * Before calling dpni_disable(), we wait for all Tx frames to arrive + * on WRIOP. After it finishes, wait until all remaining frames on Rx + * and Tx conf queues are consumed on NAPI poll. */ + msleep(500); + do { dpni_disable(priv->mc_io, 0, priv->mc_token); dpni_is_enabled(priv->mc_io, 0, priv->mc_token, &dpni_enabled); @@ -1300,19 +1310,9 @@ static int dpaa2_eth_stop(struct net_device *net_dev) */ } - /* Wait for NAPI to complete on every core and disable it. - * In particular, this will also prevent NAPI from being rescheduled if - * a new CDAN is serviced, effectively discarding the CDAN. We therefore - * don't even need to disarm the channels, except perhaps for the case - * of a huge coalescing value. - */ + wait_for_fq_empty(priv); disable_ch_napi(priv); - /* Manually drain the Rx and TxConf queues */ - drained = drain_ingress_frames(priv); - if (drained) - netdev_dbg(net_dev, "Drained %d frames.\n", drained); - /* Empty the buffer pool */ drain_pool(priv);