From patchwork Wed Mar 14 23:17:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Manlunas, Felix" X-Patchwork-Id: 886075 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cavium.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com header.i=@CAVIUMNETWORKS.onmicrosoft.com header.b="YTdquAMo"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 401nh72PGJz9sVD for ; Thu, 15 Mar 2018 10:17:59 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751362AbeCNXRx (ORCPT ); Wed, 14 Mar 2018 19:17:53 -0400 Received: from mail-co1nam03on0052.outbound.protection.outlook.com ([104.47.40.52]:56915 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751131AbeCNXRv (ORCPT ); Wed, 14 Mar 2018 19:17:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=227MzNc8QvY2zZMe+P/sIuL3GzXOMcU9C2B+K/CYb5o=; b=YTdquAMoKoBCwYbhfECociz/lT3nhpUyXqkc/ubyAekNnPUtHhsB97sd1gSidL1FwqGwrVbDI50wg9ZExy7f2hdFUYSTUIbsS5PiDcYUeDkkmfjRF5yBGjz4YGFhYTsqpvMSKZ30ZC+wdpP7j923wwlkcLR0hbxAeTkpQhqEJ2g= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Felix.Manlunas@cavium.com; Received: from localhost (50.233.148.156) by DM6PR07MB4217.namprd07.prod.outlook.com (2603:10b6:5:bd::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.588.14; Wed, 14 Mar 2018 23:17:49 +0000 Date: Wed, 14 Mar 2018 16:17:47 -0700 From: Felix Manlunas To: davem@davemloft.net Cc: netdev@vger.kernel.org, raghu.vatsavayi@cavium.com, derek.chickles@cavium.com, satananda.burla@cavium.com, felix.manlunas@cavium.com Subject: [PATCH net-next] liquidio: Allow RX descriptors to be consumed before disabling NAPI. Message-ID: <20180314231747.GA1263@felix-thinkpad.cavium.com> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: BYAPR01CA0013.prod.exchangelabs.com (2603:10b6:a02:80::26) To DM6PR07MB4217.namprd07.prod.outlook.com (2603:10b6:5:bd::26) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cdc1b77f-637a-4d61-01f8-08d58a01ce17 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DM6PR07MB4217; X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4217; 3:R622JysvbDBfZBP8DkKA3N4z2dUvHHT8ykJ7rlmOYL2l4Z5Ximk4L+k3D9HeQ6Ju7XbUNLRXIp+IrYUUESRGyX49X+WkEQPjkR15SZ2uBrqOqmANGq9GXsuEvN/5QwfMuYla2k/Z21n/tkX1zTvMKaCISrUXzcrA5bhYB6TgIywnZlXPi0NmPdZCCaZL1MqFUuGRo6swh4lb4azlOqylBqlW5nzTUSqhRxlKb6RKCiMec+glFl8b4USHxTzMaRnN; 25:Kt1kkwS2z0k1V5+fkZ6gAW6Gx3SSO6ojSrACXfwOiuho3KVE0yASgWhOdZC04fQMkaqt2l7ITlRogo5TYeCwy7CS7t9Z/MNho5rTqjjjUOO6TPDABBFHagnHKL8gtwVru3ppJTu3McioDuFNQvdX3dBq/A9wrl2j7fvlEmwAyTyZDhYFhnfcyYPlafirEJ8oQOEodapPTqkT6uHs5qOBpiNyaUekpEk0R0dWeb1mmRA4z/Tml0DxJa4brOOPRtNQt4Cj0Qf2n8vxBLoJ+J5ykowJZ1T4PClv1S3DLU0Sdc39/v5zTmKZETVtZluGvoRzEOO9UDpEK+VA5gFK+wx0ew==; 31:w9CEkPrzqQmi1kZ8xmxmPeQMU6W1xCVzzNZD5gucNGAYvGCTSvOAUGYCmn7OFGGzFieqkTqDbMsqCiYkuO4dB3Ps3AO8Ji1Joj19enytKJnfLsnaovAbR9qiuS7Rp9LckN9D+r0P3YA+c722OhEeol1cZeD9x28PxCLDKxtvMp+/4giGKjVxC8Ubyh2/v6HvpnlsiZ1zKUvBGqlJLl5GyfpxUJx6PqiCAMh/vnvwowI= X-MS-TrafficTypeDiagnostic: DM6PR07MB4217: X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4217; 20:V5BU45g1eAj2holVIYsniV8k8DHI9gPUm5ftwsfLNoUVvRTiYNZAFnCSLasALNsJOxtKU4WM1qlgG45gVcHS3DaA8oIqXF/WlYGoqNdpVIIsyZGYUHSquD8gao6ywr8M20AOolYcykCI+TzNK63/BMhadWm8b72sDe7pZPsA5Ao1V/Y8S12LLb4cjA0Bay8f1AHkDqxW6GSPKl6uHNk0e9B1jS+F+Fks6r6J+F0fHey4zz+Hy6m4TONMVlSAAC1sJxAUh3R9De3r289oahrOI9QSjr+kKMxWeZV4UdmGAbdRB1F+I999moNQ+yy+fZYCQ7It+SHSefD/AzAU6af6ea2+2ag6sNJ1UcETD+zl+EvG5w02dve2IF++7eMDhbVclT+VZfeFk6u8EA0C+KRxIGLaLJUriTzkwCNommcFpYaeNAyIjTifeE0V8Wu8ZYl+n3WNh/A3DueGEGfBIhwoyV7gQwKB+O5RB56yA1KaD5ZXieKxw4krV/aWdH2st5m8; 4:f0m9Gq/0LKKVuOcRea0QQndzeWgE9/YkTS35kPltN1OxpKZzmsd5xJRrvCfr1pE63OOnpE+k/h3gB5lUIZbCv/qRBnpjLPO14jhzQOLU16hmY2zvu/52RuWHXcmSjsdWQrywIJ8a7W4ODJcv1o0+ltn/fmRe1jH7DRG/fbyMMpJQyMZXNNE3yvs4LyDahbsx+SFkTNDv+u/QEBrCKBcwb/QndPl3xtn+ICEL75ljQ4rk6RaP3TMt7vIZgLjRssnQpsGZnmR8w7PPD+MmQUyERA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3231221)(944501244)(52105095)(93006095)(93001095)(3002001)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123564045)(20161123560045)(20161123562045)(6072148)(201708071742011); SRVR:DM6PR07MB4217; BCL:0; PCL:0; RULEID:; SRVR:DM6PR07MB4217; X-Forefront-PRVS: 0611A21987 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(376002)(346002)(396003)(39380400002)(39860400002)(366004)(199004)(189003)(16526019)(26005)(6486002)(386003)(52116002)(81156014)(72206003)(8936002)(186003)(97736004)(68736007)(6496006)(33656002)(47776003)(8676002)(478600001)(81166006)(66066001)(23726003)(105586002)(76506005)(86362001)(16586007)(25786009)(1076002)(2906002)(53936002)(107886003)(305945005)(316002)(50466002)(59450400001)(106356001)(6916009)(5660300001)(2361001)(6116002)(2351001)(4326008)(58126008)(3846002)(7736002)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR07MB4217; H:localhost; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4217; 23:rs5L8gmWje7y/pB+j41HYrKbOTCZFxERvmqwoFseNgNuouwXeRaTImoUpfDCIQFKCdOzccg1cTwPAOFxm47Lo1l6RPoN9WlIIZ1oC8HJ5OS6UmTiRcnUHw2SV6U9p//sMXJZCAtjKwkbMbQ3+2j9BTTS+NWRNGY4WeT5dH7JvrcNAuJNBTdN7Ln59hmaa2hEPm+UgX3s+UA8v0vrfwrhP6brpTsbPQki6nXisBvkwftpuv65V2HV3uslLSUbisa8FO2bzim8n4al4JvwT6pQSaHy7ExxSzTk54o+h9JDfWeGZljXOuIFCSbs6B4y0F3cHsj/9jkwcwZEgqVsli1zy/ODTF4ItVIaBMqbuLPqG4cGbdBk03zb9LzOiz3T5KyYP+dGYBktc/04Aa5M5wuihVwjdMSsymc3F07OLUGu8wvTVoDRIWhno+sNKQoWgCOANQ6ekVr+KbszXKQ71cN85hW63p5cRW4GAikzflKwTp00AKBLFjVxyOD36a5JuW1nA206BALrMvt5lv3ixrrcuq1b+7+NSkAC58oR7MQeiCHSQvQH69YQoD5VnDcR6ag9HG13XxLcB0wQHXOFBHwjR+5kHTX52Hj1ppQAvZP7mC57Jj6mEaexdf1ckSH54FxEeHyjywdvZ0wnoQ+2JcG9WF/duJlrex2v9kNk7sR4T9cCl2Q/NNMLHg71YntC6ckBE+kO2ATXSNYVQ8tYrr+dGTaPjMJArx6ML1GlH2Iy+GR9h0yz8IbWC76uGWiFyTuegncZx4QiwvnCkeoaEROPAuwCui0xNv2kczvNMnhWm6oRLbb4D8AtanrkuCKndLfOEISjhcqRq0ZZb0LlPJN5Dj4erklgv4OXHo/g15J9QMyDgHAI0/ai+YEIiCE37WJUkfAS29HuFr8J19G2q9I2OGTS35zdPLRe82F3ep/v5ISPYtJoEwJdDtKnU8BWu2MJgiwrgSFcg2wbKwx7vISq9Zn+HNih06l87qhChRDEXb5RhkBQPdocopUmGGdbAAqZZdd4xU+ozwrZA4kI7BNmmNKTNS+gba8SwH7GRJ2m5uuaMsZYq0dgjWjELvY7pFHymHAWOpMHoHtB4TzOx335TNal4sY5TzQWLRgnjobXbnmSzP7EOvtpx26rTi0UnBhrBzS7twe8W4AhcMIyWVe9K43IPGADr/TqqCIIIbf/s7A= X-Microsoft-Antispam-Message-Info: A9MG8CqwNY6bfmZumGv8fCZp+eZWZmA7T7ID+XRw0G65XO+08OfDN1bN5Y9Wg2jImR2cUKnGT1aPcUZKIyXFFRrjxyGYa/YID1QnSuSlyZrDJcT6in/z9JUkri2QXRL18LTReg1lzusTXUG89pZ/VQ1yE7gLbWPuSdzEZpY8svK7nXnOPM8kkDz2A15K10NX X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4217; 6:IigHvi7c/U8zVXnvUEqp5fnCf75lU9JGxUFI3fiq+iicyMiYZ+JWcOPVMurTx75HW1vdHnnWx3zjFJDq/bldrkI0rFUgV1n+1Gw+Hvv7hE2Wemp69Zx7bWkXYdNK4ZLFRQX6zHGEOQzhCVICrX8BczJ8PkRRr3+yIQSTFjEDvXbr+p4A0VEMttMXoslFskNMisLwkegZYM1RRTdgl0yxh6s6ycckJ1XigiRPzz/5WJf32WNjMuT8EFNxdpJZN75aWu4/BoC/ATbxQV+LA6578zC3mnT3/mtmWl5ulphyEzEmw1cOBZJwVI6ziwgQdsOgdri6S6dFxlcuLs9vLjSDcIHoFoiKBWazhur2pIgjVzM=; 5:pox617+iu7Kx2l7lJTJ2eNFHnO05gCX32HQdPvbhMAw6BTbrKI05jEP+fRNiAv4BDw9OKgC0x390z/XqdHpAQL7BdMTksFU85pdOS/0sKQ4ysMk9LvgRIlsCnwb55OJS6eF7xy2DIpfHgSu/kKS05JlD7RlAAQyBk9KEZMQSH5s=; 24:X4fspX60Y68ICdq+ABrK1Mnqqxc30cv5XlaTpYk9vrVxxcxXtOpXvhVQpUY8CGufAokWl9Id9+mFUB7ICy37MuACwUf3YBXQeoBqPJMxnCo=; 7:6Z/cGRxF9GKwMK57/MFDwQDaQYic3zJt6Pob4b6pnlsbrnYsiUu/nRZSxFaIdbKdSf3fSqUX3rY2zA+oGD9jOsbEHWU9zWRHsALyLmyg2hT9dM9UD7mlpVF48GHV9PdNE9S4qviQvF0r6J5Uj63phqESwqEoyOgww40PFhMytivFlY5pDQROILlhddy1wT6GqFDH3/uboT+QDZF3AnuC70juvwl9LHxKbbCAp6godm/F57EeW87kQ+2X/x3gFwEy SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2018 23:17:49.8914 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cdc1b77f-637a-4d61-01f8-08d58a01ce17 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR07MB4217 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Raghu Vatsavayi Signed-off-by: Raghu Vatsavayi Acked-by: Derek Chickles Signed-off-by: Felix Manlunas --- drivers/net/ethernet/cavium/liquidio/lio_core.c | 23 ++++++++++++++++++++ drivers/net/ethernet/cavium/liquidio/lio_main.c | 25 +++++++++++++--------- drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 23 ++++++++++++-------- .../net/ethernet/cavium/liquidio/octeon_network.h | 1 + 4 files changed, 53 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_core.c b/drivers/net/ethernet/cavium/liquidio/lio_core.c index e7b6eb8..8e4716a 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_core.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_core.c @@ -1157,3 +1157,26 @@ int liquidio_change_mtu(struct net_device *netdev, int new_mtu) octeon_free_soft_command(oct, sc); return 0; } + +int lio_wait_for_clean_oq(struct octeon_device *oct) +{ + int retry = 100, pending_pkts = 0; + int idx; + + do { + pending_pkts = 0; + + for (idx = 0; idx < MAX_OCTEON_OUTPUT_QUEUES(oct); idx++) { + if (!(oct->io_qmask.oq & BIT_ULL(idx))) + continue; + pending_pkts += + atomic_read(&oct->droq[idx]->pkts_pending); + } + + if (pending_pkts > 0) + schedule_timeout_uninterruptible(1); + + } while (retry-- && pending_pkts); + + return pending_pkts; +} diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index 140085b..345fe6e 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -2240,16 +2240,6 @@ static int liquidio_stop(struct net_device *netdev) struct octeon_device *oct = lio->oct_dev; struct napi_struct *napi, *n; - if (oct->props[lio->ifidx].napi_enabled) { - list_for_each_entry_safe(napi, n, &netdev->napi_list, dev_list) - napi_disable(napi); - - oct->props[lio->ifidx].napi_enabled = 0; - - if (OCTEON_CN23XX_PF(oct)) - oct->droq[0]->ops.poll_mode = 0; - } - ifstate_reset(lio, LIO_IFSTATE_RUNNING); netif_tx_disable(netdev); @@ -2275,6 +2265,21 @@ static int liquidio_stop(struct net_device *netdev) lio->ptp_clock = NULL; } + /* Wait for any pending Rx descriptors */ + if (lio_wait_for_clean_oq(oct)) + netif_info(lio, rx_err, lio->netdev, + "Proceeding with stop interface after partial RX desc processing\n"); + + if (oct->props[lio->ifidx].napi_enabled == 1) { + list_for_each_entry_safe(napi, n, &netdev->napi_list, dev_list) + napi_disable(napi); + + oct->props[lio->ifidx].napi_enabled = 0; + + if (OCTEON_CN23XX_PF(oct)) + oct->droq[0]->ops.poll_mode = 0; + } + dev_info(&oct->pci_dev->dev, "%s interface is stopped\n", netdev->name); return 0; diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c index 3342d64..66655df 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c @@ -1281,15 +1281,6 @@ static int liquidio_stop(struct net_device *netdev) /* tell Octeon to stop forwarding packets to host */ send_rx_ctrl_cmd(lio, 0); - if (oct->props[lio->ifidx].napi_enabled) { - list_for_each_entry_safe(napi, n, &netdev->napi_list, dev_list) - napi_disable(napi); - - oct->props[lio->ifidx].napi_enabled = 0; - - oct->droq[0]->ops.poll_mode = 0; - } - netif_info(lio, ifdown, lio->netdev, "Stopping interface!\n"); /* Inform that netif carrier is down */ lio->intf_open = 0; @@ -1302,6 +1293,20 @@ static int liquidio_stop(struct net_device *netdev) txqs_stop(netdev); + /* Wait for any pending Rx descriptors */ + if (lio_wait_for_clean_oq(oct)) + netif_info(lio, rx_err, lio->netdev, + "Proceeding with stop interface after partial RX desc processing\n"); + + if (oct->props[lio->ifidx].napi_enabled == 1) { + list_for_each_entry_safe(napi, n, &netdev->napi_list, dev_list) + napi_disable(napi); + + oct->props[lio->ifidx].napi_enabled = 0; + + oct->droq[0]->ops.poll_mode = 0; + } + dev_info(&oct->pci_dev->dev, "%s interface is stopped\n", netdev->name); return 0; diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h index 76803a5..b7ee2d3 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h +++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h @@ -190,6 +190,7 @@ irqreturn_t liquidio_msix_intr_handler(int irq __attribute__((unused)), int octeon_setup_interrupt(struct octeon_device *oct, u32 num_ioqs); +int lio_wait_for_clean_oq(struct octeon_device *oct); /** * \brief Register ethtool operations * @param netdev pointer to network device