From patchwork Wed Apr 25 18:23:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Manlunas, Felix" X-Patchwork-Id: 904659 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=cavium.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=CAVIUMNETWORKS.onmicrosoft.com header.i=@CAVIUMNETWORKS.onmicrosoft.com header.b="EjCRVX1n"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40WT9k4vHGz9s0v for ; Thu, 26 Apr 2018 04:24:10 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756298AbeDYSYJ (ORCPT ); Wed, 25 Apr 2018 14:24:09 -0400 Received: from mail-co1nam03on0081.outbound.protection.outlook.com ([104.47.40.81]:16160 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756183AbeDYSXz (ORCPT ); Wed, 25 Apr 2018 14:23:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JJqvsrW2Yqv7LvGMcayWzT6TcJBpo2ukrglquWbkF9Q=; b=EjCRVX1nqVZy8r/WGEHWLmgAUhEEpmMEYYS1XHQWBR394ikhzmn8luM3aupEGv3mp6cZtC/eedSrF8X1S/hgiXqTX1mNE/bMv5nYr9vIM1CEmH2ohCaB5dPBISAuEHFuKU1rxJJSPiGnDUfaMDIqEVyL6a9YGqAu0mneTFr/r2Y= Received: from localhost (50.233.148.156) by DM6PR07MB4218.namprd07.prod.outlook.com (2603:10b6:5:bd::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.715.18; Wed, 25 Apr 2018 18:23:53 +0000 Date: Wed, 25 Apr 2018 11:23:50 -0700 From: Felix Manlunas To: davem@davemloft.net Cc: netdev@vger.kernel.org, raghu.vatsavayi@cavium.com, derek.chickles@cavium.com, satananda.burla@cavium.com, felix.manlunas@cavium.com, intiyaz.basha@cavium.com Subject: [PATCH net-next 6/6] liquidio: enhanced ethtool --set-channels feature Message-ID: <20180425182350.GA13911@felix-thinkpad.cavium.com> References: <20180425182301.GA13840@felix-thinkpad.cavium.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180425182301.GA13840@felix-thinkpad.cavium.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CO2PR07CA0081.namprd07.prod.outlook.com (2603:10b6:100::49) To DM6PR07MB4218.namprd07.prod.outlook.com (2603:10b6:5:bd::27) X-MS-PublicTrafficType: Email X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(2017052603328)(7153060)(7193020); SRVR:DM6PR07MB4218; X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4218; 3:Ej48VGOoteoi4gq5hTkR4ebaOz3TChSxG5dInfk1v8xHiIN7S6eH78PJENo6SXshT3R0Cf/j0fem59KlN98eepGC6Zg+x6J4nY5WUuLxS4kFW/YGNffNS0cxi0E0gYi0gghQmFS2GhQlMFSLaK5KlJJGHT3Z/w2/YIeKbVtQdaE6Rhf38Akt4YIm/4HIrmNsRcaaVIxkkV221hUdIne8aVH6t/J0SrN/dHC8qQ4UQ9AGHAwubvaq9lTD6no1qLRU; 25:oPikceF+z0AUMtufgaXS+8p035WO2x27sYAX20OrBs8bvWlxJaEeyd6Bqr+lLwneicv/PJnsvjkGEq9FyQZ3YYUxa4fClNOYVsp1zBBPAMPZ+ZBNOnRlPsVS86xFae9+pVcr1t5D1VkSrn6nw5z3l+ZQo/YmBRMjOCgOqqG+53UOk4vHFF1cw/nVShoEahA0mbejvGd/sFgXmsj5tVrj+3sXXm6nsThvHwV8YE++h30DsAOZPS0cC58cjU6Mk4FioLSo4lhJdeGmH56eQaujIAzvTS2ItxW/Bi/9RuIjBpx4fHVlhgVJseyRaojyVtW/2jK7Juz2fhreDGPpDgr+qQ==; 31:3kOuTyH0jy4GTPKzH3MS1rcWc+x0KqSm86uU2ybCpwYFNZv4zrfD3opM8MIjvnUMr3kgV9Oh2UZ+ByG+ffGw2JKyAoJSWnlmZvW018aCRkB5+KCr7jZlSekFjOq2g1uUycQCBt8R93Puz1U1NZSKjF5yfULela1ueDwjRHLLy0OQFow9GD2GdqWbRgskZmz0jt9LD0L3On6KsiCL0nBA5jdR1lz4StIV61j6NEFBM98= X-MS-TrafficTypeDiagnostic: DM6PR07MB4218: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Felix.Manlunas@cavium.com; X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4218; 20:NmnRjWIUdOJ6RjWGPCk6Aw6/roQIoYZHJVgY6rD3a9eG5WCZ6j2sBd7Yqho/LC4QgJDww1XPtAzD4yLSKtrAKCaiqzpxmzIQnYyNcUJheykP6AMbAY1uRx3BayrdZZzkznN7C+gMg+BL3/KLPQEYFuCrDJVPmlZMeQ9HMsa10JwHgpHp0pMCfNqJ2bJhdRO+BCxHX27vCxlld9sd2f8bz2cnFhWFqg/FQDK5FqfMJo1qjsOrpe/h46xq0t1bdAMgjeQfwx8b4pBd6N1cvmQG08aAecmYWqJZxVvxT/u09Ra/aMl4ndia5axOg7QBm2i5yPoWzgYv9xezIq7lelmH14cN/oerrm8zyoH138E2JTOIE5JLt33iGWNXAk9bXeUifs3F0Q3jLtpqt1uRi3isb9rJcyvyahZ12x50O1nmYvIaQKWRtumKXd5yTvkDKcLmEvt6oH90IJocOfTr0WfMqx3mhIaBidH1kiWsByjtkLQHqJJkz96NtEDrG2JOzgtM; 4:Vt3tllIMNUaHwSNHZ9w0UhdVjiQZUqB7zTGJ3SqtwKjFCDflQGiq04u09mxiYeTIBm2+QyMz9xGTLtNYSS4Gl+wOF964Lf+tgyDwFgJc/mjrdJ6T7g6x8iJ7D7eL1mJWe7MDU1QgNFLTzRM+sGn2Q6B4OSZ7ZhV6AaYA+UDt9P6pZ68HzsxIjeq9fezySDgEqNQyUUmLC7mINAsLrITn9AKH738ZNB2H1L1Zr75y7b3jlgexA414z1k+ZfNNzqwy7egwPVwyFO9yUFRWkEjbwg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231232)(944501410)(52105095)(93006095)(93001095)(3002001)(10201501046)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123564045)(20161123558120)(6072148)(201708071742011); SRVR:DM6PR07MB4218; BCL:0; PCL:0; RULEID:; SRVR:DM6PR07MB4218; X-Forefront-PRVS: 06530126A4 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6069001)(396003)(39380400002)(366004)(39850400004)(346002)(376002)(189003)(199004)(66066001)(97736004)(50466002)(4326008)(76176011)(25786009)(107886003)(6496006)(52116002)(11346002)(86362001)(486006)(476003)(47776003)(44832011)(23726003)(3846002)(1076002)(575784001)(305945005)(6116002)(956004)(7736002)(446003)(81166006)(81156014)(53936002)(72206003)(2351001)(2361001)(8936002)(16586007)(316002)(5660300001)(8676002)(33656002)(76506005)(105586002)(68736007)(26005)(386003)(59450400001)(6486002)(58126008)(478600001)(6916009)(2906002)(106356001)(186003)(16526019)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR07MB4218; H:localhost; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4218; 23:s0QJGKY7xep8pBtm3zKoyOeaFfm1z9dJPF5pDDej1kCxPS4JHJNoL4gfOBN+SyT8DxYRRK2r9ciFiQbx8WsP656jEUX08xWlga2hyX1PKsBbN8zw9HahhEdxuYbUjyORXB1RgXXUTZMsfD0KnGeL/dkl7JCvIr8mirDQRF31ToSoWdZNgHr4wnmAfwrIlVwL0caIAew2uBoQQpYpjTHM7Fsw5iG0gBPzLePqfUGSvVihCfzXIXfCycq7adExDSdpNWN3zmcAo+tMyxxNLdxs3+bMMJD7F+gtmkFHpEriNwKEq4evAycdIX7xzAbf5zPn8fbjIURD9n2Sh6XGIXgpYdmTiKKe+a54rnc3YV6om3SuciWwk91dCP5lJKqILsIx4Rx3zRPYg4aZWjPl9iDvMLqe77o/UJD4xcyjGdrZ4deD6lIYekXC0rw9mhZAZeS5b6D2oKeKZWy55A16HpZ7fduz0z1abfOB1XOGpjKK0X7ZPd6i+1o6FGsQQSCHYEK13mTDEca+9Lv48FzIPqsDMRvMhJjquhfcHrW9MNw+HDZTsyUEcAvoLLUNmZBk/zZ7TPCDJcVi5QU90zb6PkvqIyDHIWVa7TcyG+ZuMl+ms+Um7xbz3mNNNAkKqdVZiSgZUAs8ZaVzgmNHJITwPHYCB8bnMuEERvvqTbFUs4C8ffPh8CaOnMpGQIth3r++AGwoE4LM/+6ziTIUMQkt3TXMkiefCbSa/0pTrKI106kutxU2h5rLqiAM3wfcklRRx5yCTXLQqV6pkE9zdohxRQoiantTCPHkvNK+J88VXoun3UF5LTflhBtB8je4KgGh0MNEPADgQ48uspzd+XGqSXG32VS7dMb5PsXMwRsF6eP6kKufCnGjW3rtU5eEAtZqXqAJu5GeZdQK6vmg6Kr8YEKbHs75YLhOdd3KHAX4rZtETS5L3xy+giEkcN0sTVYmYilq2gM/ROzOmYUez/Bx5wOXhbiIUTIDC0dHP99NIkwgJDye/ytUQBEDGPMm7yqw16SS4Pgc1x3O8EeH6nx7db7yxc8R1QsdVKK6NFPDtlV+wviJuVIa6bZjJKXexrVQVmKx+vMfsD+3P11POc0NjTr/RNetWk4QHlK5l30Eahg4Dgr78AUOSgivHvAf0ryHDucYqSWY4qAVcS5dojagpcuzGoF6FHmfuoWPu+n2rupWW9ddT4DfRPD7LqSaOi404/T6pmpr0KRwT91rGDCDLMJWTnT2m4tmVT2IBwdWWN6JN7khYTCT55xDHVvx1er4BJMJBI5p03QanPuhDiZYGz8IlQ29iR1XNQEpPiOMAqd3eXFURiks3aNNBRVNDqwqqKqL X-Microsoft-Antispam-Message-Info: B1HJ7eOWFiimPnrWX9hBbgdf7QvxhLfPeVexYyi2R4wC/futE4E9kUaVzu7x8Sl18rWEqi4KbBdWkVXzqCQbwypiUMITZXXAubLlBXlqw6VQ3hqee22WSIaOsWyX4FuLEHzb9VjyGOI0g+bpNHZAC147Oh2Gi2PhPuUdo+r4UVwQqXNb7tVQ1kCg2q3+cB52 X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4218; 6:WRh3ePxYxU+VsHzZLZ6T48nYD2YblrZ5yGxtE2ywajBjBhvNk0GxDLrY8sWnrlKNONOe2G/qPFLHz5KFLVECJFdLLulgciCj5rj3aykHO+uZADhD2LTQkDAjSgrPfe+XfYf2jQttJ/GEejyYAfxDIlWwGEVMHHo31ZycAyQHD26bzHwuukSMnd9HWvU0xTf5BsaNBW0JWwyYQE6vV/1DRYb3KLY7ozXmZ085/Y/XG6LAMwGtFBZJNDwS8ULKZ5FfmJcpU3/WdSdpHfOihpXOhzQJvgE5I87pnQ2xcOhXodepH9W+wabMjCOaTro7b2xwwX25ktFvHrDC4b4z7LZo+tckzrgPwbxWO1TdAbK7nN39sSjkdMAG/u253gkIUHg6Y9ABef6sBrc75vh8CWbuEokDceR/Ar/XjdVT/DmNjdRqmajvOzuYQ4L4dl3mG3kYylxyO3rp5cApWpQinDCjNg==; 5:g+Tp891gOwzsIASCy1/xN1BBnIb7muSxVC1MmubtXpXuDcBkLe3CWJ0qVQpX7Vg2amYB9miBj4TOC4gtrZY2pIdsxxCZW8JS2nwftZOfrwX9KrJwGOHiqTRI0ulWGqyypGWp20H7zsqp1SpQbpTyOFwFsx1qwJGiVjx01dcnYC8=; 24:MJ5lTT9+mLCSSR7fc81WmJXJWv0YH8i7jVEyhTnil3WLuI2lwBLNRQ7MQ0pPIk059Ydc3lnEnJGZHONDB7igP70OWdZjY3NyJ5BSwTSqa1Y= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM6PR07MB4218; 7:RWWZ8oV+6nFlYsJ54zmtgNs1p2q/FRVT/D3/HsEHuXmO1SIvTUz5pM4pxMT+ZsXkpDAZ4CEEcdlAwmnEVIZgBXI0qVSaLI+TrFBDjeKjshVwt81bTu3uOTIzaiWGsJBYNF7BQB44luRvmhOw+xR4idITq9fKOL9s+7MgMKSghFL7npJMQpnEXjhR7H7CoeEmOaFNEkPCfqmZHoKdn2qKt3c93VtZCK9ON1jfj3WgBpkaSGy9ZcOoG9PGOSJMcPMy X-MS-Office365-Filtering-Correlation-Id: d6110d1d-ffb2-469b-789e-08d5aad9b32c X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2018 18:23:53.2048 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d6110d1d-ffb2-469b-789e-08d5aad9b32c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR07MB4218 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Intiyaz Basha Enhancing driver to accept max supported queues for ethtool --set-channels Signed-off-by: Intiyaz Basha Acked-by: Derek Chickles Signed-off-by: Felix Manlunas --- .../ethernet/cavium/liquidio/cn23xx_pf_device.c | 6 +- .../ethernet/cavium/liquidio/cn23xx_pf_device.h | 2 + drivers/net/ethernet/cavium/liquidio/lio_core.c | 4 +- drivers/net/ethernet/cavium/liquidio/lio_ethtool.c | 263 +++++++++++++++++++-- drivers/net/ethernet/cavium/liquidio/lio_main.c | 64 +++-- drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 8 +- .../net/ethernet/cavium/liquidio/liquidio_common.h | 1 + .../net/ethernet/cavium/liquidio/octeon_device.c | 12 +- .../net/ethernet/cavium/liquidio/octeon_device.h | 2 +- .../net/ethernet/cavium/liquidio/octeon_network.h | 12 +- 10 files changed, 316 insertions(+), 58 deletions(-) diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c index bc9861c..929d485 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.c @@ -1245,7 +1245,7 @@ static void cn23xx_setup_reg_address(struct octeon_device *oct) CN23XX_SLI_MAC_PF_INT_ENB64(oct->pcie_port, oct->pf_num); } -static int cn23xx_sriov_config(struct octeon_device *oct) +int cn23xx_sriov_config(struct octeon_device *oct) { struct octeon_cn23xx_pf *cn23xx = (struct octeon_cn23xx_pf *)oct->chip; u32 max_rings, total_rings, max_vfs, rings_per_vf; @@ -1269,8 +1269,8 @@ static int cn23xx_sriov_config(struct octeon_device *oct) break; } - if (max_rings <= num_present_cpus()) - num_pf_rings = 1; + if (oct->sriov_info.num_pf_rings) + num_pf_rings = oct->sriov_info.num_pf_rings; else num_pf_rings = num_present_cpus(); diff --git a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h index 63b3de4..e6f31d0 100644 --- a/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h +++ b/drivers/net/ethernet/cavium/liquidio/cn23xx_pf_device.h @@ -61,6 +61,8 @@ u32 cn23xx_pf_get_oq_ticks(struct octeon_device *oct, u32 time_intr_in_us); void cn23xx_dump_pf_initialized_regs(struct octeon_device *oct); +int cn23xx_sriov_config(struct octeon_device *oct); + int cn23xx_fw_loaded(struct octeon_device *oct); void cn23xx_tell_vf_its_macaddr_changed(struct octeon_device *oct, int vfidx, diff --git a/drivers/net/ethernet/cavium/liquidio/lio_core.c b/drivers/net/ethernet/cavium/liquidio/lio_core.c index b4f9275..7e24b51 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_core.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_core.c @@ -78,7 +78,7 @@ void lio_delete_glists(struct lio *lio) if (!lio->glist) return; - for (i = 0; i < lio->linfo.num_txpciq; i++) { + for (i = 0; i < lio->oct_dev->num_iqs; i++) { do { g = (struct octnic_gather *) lio_list_delete_head(&lio->glist[i]); @@ -1036,8 +1036,8 @@ int octeon_setup_interrupt(struct octeon_device *oct, u32 num_ioqs) int num_ioq_vectors; int irqret, err; - oct->num_msix_irqs = num_ioqs; if (oct->msix_on) { + oct->num_msix_irqs = num_ioqs; if (OCTEON_CN23XX_PF(oct)) { num_interrupts = MAX_IOQ_INTERRUPTS_PER_PF + 1; diff --git a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c index 9926a12..7ca246e 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_ethtool.c @@ -353,7 +353,14 @@ lio_ethtool_get_channels(struct net_device *dev, rx_count = CFG_GET_NUM_RXQS_NIC_IF(conf6x, lio->ifidx); tx_count = CFG_GET_NUM_TXQS_NIC_IF(conf6x, lio->ifidx); } else if (OCTEON_CN23XX_PF(oct)) { - max_combined = lio->linfo.num_txpciq; + if (oct->sriov_info.sriov_enabled) { + max_combined = lio->linfo.num_txpciq; + } else { + struct octeon_config *conf23_pf = + CHIP_CONF(oct, cn23xx_pf); + + max_combined = CFG_GET_IQ_MAX_Q(conf23_pf); + } combined_count = oct->num_iqs; } else if (OCTEON_CN23XX_VF(oct)) { u64 reg_val = 0ULL; @@ -417,9 +424,15 @@ lio_irq_reallocate_irqs(struct octeon_device *oct, uint32_t num_ioqs) kfree(oct->irq_name_storage); oct->irq_name_storage = NULL; + + if (octeon_allocate_ioq_vector(oct, num_ioqs)) { + dev_err(&oct->pci_dev->dev, "OCTEON: ioq vector allocation failed\n"); + return -1; + } + if (octeon_setup_interrupt(oct, num_ioqs)) { dev_info(&oct->pci_dev->dev, "Setup interrupt failed\n"); - return 1; + return -1; } /* Enable Octeon device interrupts */ @@ -449,7 +462,16 @@ lio_ethtool_set_channels(struct net_device *dev, combined_count = channel->combined_count; if (OCTEON_CN23XX_PF(oct)) { - max_combined = channel->max_combined; + if (oct->sriov_info.sriov_enabled) { + max_combined = lio->linfo.num_txpciq; + } else { + struct octeon_config *conf23_pf = + CHIP_CONF(oct, + cn23xx_pf); + + max_combined = + CFG_GET_IQ_MAX_Q(conf23_pf); + } } else if (OCTEON_CN23XX_VF(oct)) { u64 reg_val = 0ULL; u64 ctrl = CN23XX_VF_SLI_IQ_PKT_CONTROL64(0); @@ -477,7 +499,6 @@ lio_ethtool_set_channels(struct net_device *dev, if (lio_reset_queues(dev, combined_count)) return -EINVAL; - lio_irq_reallocate_irqs(oct, combined_count); if (stopped) dev->netdev_ops->ndo_open(dev); @@ -816,12 +837,120 @@ lio_ethtool_get_ringparam(struct net_device *netdev, ering->rx_jumbo_max_pending = 0; } +int lio_23xx_reconfigure_queue_count(struct lio *lio) +{ + struct octeon_device *oct = lio->oct_dev; + struct liquidio_if_cfg_context *ctx; + u32 resp_size, ctx_size, data_size; + struct liquidio_if_cfg_resp *resp; + struct octeon_soft_command *sc; + union oct_nic_if_cfg if_cfg; + struct lio_version *vdata; + u32 ifidx_or_pfnum; + int retval; + int j; + + resp_size = sizeof(struct liquidio_if_cfg_resp); + ctx_size = sizeof(struct liquidio_if_cfg_context); + data_size = sizeof(struct lio_version); + sc = (struct octeon_soft_command *) + octeon_alloc_soft_command(oct, data_size, + resp_size, ctx_size); + if (!sc) { + dev_err(&oct->pci_dev->dev, "%s: Failed to allocate soft command\n", + __func__); + return -1; + } + + resp = (struct liquidio_if_cfg_resp *)sc->virtrptr; + ctx = (struct liquidio_if_cfg_context *)sc->ctxptr; + vdata = (struct lio_version *)sc->virtdptr; + + vdata->major = cpu_to_be16(LIQUIDIO_BASE_MAJOR_VERSION); + vdata->minor = cpu_to_be16(LIQUIDIO_BASE_MINOR_VERSION); + vdata->micro = cpu_to_be16(LIQUIDIO_BASE_MICRO_VERSION); + + ifidx_or_pfnum = oct->pf_num; + WRITE_ONCE(ctx->cond, 0); + ctx->octeon_id = lio_get_device_id(oct); + init_waitqueue_head(&ctx->wc); + + if_cfg.u64 = 0; + if_cfg.s.num_iqueues = oct->sriov_info.num_pf_rings; + if_cfg.s.num_oqueues = oct->sriov_info.num_pf_rings; + if_cfg.s.base_queue = oct->sriov_info.pf_srn; + if_cfg.s.gmx_port_id = oct->pf_num; + + sc->iq_no = 0; + octeon_prepare_soft_command(oct, sc, OPCODE_NIC, + OPCODE_NIC_QCOUNT_UPDATE, 0, + if_cfg.u64, 0); + sc->callback = lio_if_cfg_callback; + sc->callback_arg = sc; + sc->wait_time = LIO_IFCFG_WAIT_TIME; + + retval = octeon_send_soft_command(oct, sc); + if (retval == IQ_SEND_FAILED) { + dev_err(&oct->pci_dev->dev, + "iq/oq config failed status: %x\n", + retval); + goto qcount_update_fail; + } + + if (sleep_cond(&ctx->wc, &ctx->cond) == -EINTR) { + dev_err(&oct->pci_dev->dev, "Wait interrupted\n"); + return -1; + } + + retval = resp->status; + if (retval) { + dev_err(&oct->pci_dev->dev, "iq/oq config failed\n"); + goto qcount_update_fail; + } + + octeon_swap_8B_data((u64 *)(&resp->cfg_info), + (sizeof(struct liquidio_if_cfg_info)) >> 3); + + lio->ifidx = ifidx_or_pfnum; + lio->linfo.num_rxpciq = hweight64(resp->cfg_info.iqmask); + lio->linfo.num_txpciq = hweight64(resp->cfg_info.iqmask); + for (j = 0; j < lio->linfo.num_rxpciq; j++) { + lio->linfo.rxpciq[j].u64 = + resp->cfg_info.linfo.rxpciq[j].u64; + } + + for (j = 0; j < lio->linfo.num_txpciq; j++) { + lio->linfo.txpciq[j].u64 = + resp->cfg_info.linfo.txpciq[j].u64; + } + + lio->linfo.hw_addr = resp->cfg_info.linfo.hw_addr; + lio->linfo.gmxport = resp->cfg_info.linfo.gmxport; + lio->linfo.link.u64 = resp->cfg_info.linfo.link.u64; + lio->txq = lio->linfo.txpciq[0].s.q_no; + lio->rxq = lio->linfo.rxpciq[0].s.q_no; + + octeon_free_soft_command(oct, sc); + dev_info(&oct->pci_dev->dev, "Queue count updated to %d\n", + lio->linfo.num_rxpciq); + + return 0; + +qcount_update_fail: + octeon_free_soft_command(oct, sc); + + return -1; +} + static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) { struct lio *lio = GET_LIO(netdev); struct octeon_device *oct = lio->oct_dev; + int i, queue_count_update = 0; struct napi_struct *napi, *n; - int i, update = 0; + int ret; + + schedule_timeout_uninterruptible(msecs_to_jiffies(100)); if (wait_for_pending_requests(oct)) dev_err(&oct->pci_dev->dev, "There were pending requests\n"); @@ -830,7 +959,7 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) dev_err(&oct->pci_dev->dev, "IQ had pending instructions\n"); if (octeon_set_io_queues_off(oct)) { - dev_err(&oct->pci_dev->dev, "setting io queues off failed\n"); + dev_err(&oct->pci_dev->dev, "Setting io queues off failed\n"); return -1; } @@ -843,9 +972,40 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) netif_napi_del(napi); if (num_qs != oct->num_iqs) { - netif_set_real_num_rx_queues(netdev, num_qs); - netif_set_real_num_tx_queues(netdev, num_qs); - update = 1; + ret = netif_set_real_num_rx_queues(netdev, num_qs); + if (ret) { + dev_err(&oct->pci_dev->dev, + "Setting real number rx failed\n"); + return ret; + } + + ret = netif_set_real_num_tx_queues(netdev, num_qs); + if (ret) { + dev_err(&oct->pci_dev->dev, + "Setting real number tx failed\n"); + return ret; + } + + /* The value of queue_count_update decides whether it is the + * queue count or the descriptor count that is being + * re-configured. + */ + queue_count_update = 1; + } + + /* Re-configuration of queues can happen in two scenarios, SRIOV enabled + * and SRIOV disabled. Few things like recreating queue zero, resetting + * glists and IRQs are required for both. For the latter, some more + * steps like updating sriov_info for the octeon device need to be done. + */ + if (queue_count_update) { + lio_delete_glists(lio); + + /* Delete mbox for PF which is SRIOV disabled because sriov_info + * will be now changed. + */ + if ((OCTEON_CN23XX_PF(oct)) && !oct->sriov_info.sriov_enabled) + oct->fn_list.free_mbox(oct); } for (i = 0; i < MAX_OCTEON_OUTPUT_QUEUES(oct); i++) { @@ -860,24 +1020,91 @@ static int lio_reset_queues(struct net_device *netdev, uint32_t num_qs) octeon_delete_instr_queue(oct, i); } + if (queue_count_update) { + /* For PF re-configure sriov related information */ + if ((OCTEON_CN23XX_PF(oct)) && + !oct->sriov_info.sriov_enabled) { + oct->sriov_info.num_pf_rings = num_qs; + if (cn23xx_sriov_config(oct)) { + dev_err(&oct->pci_dev->dev, + "Queue reset aborted: SRIOV config failed\n"); + return -1; + } + + num_qs = oct->sriov_info.num_pf_rings; + } + } + if (oct->fn_list.setup_device_regs(oct)) { dev_err(&oct->pci_dev->dev, "Failed to configure device registers\n"); return -1; } - if (liquidio_setup_io_queues(oct, 0, num_qs, num_qs)) { - dev_err(&oct->pci_dev->dev, "IO queues initialization failed\n"); - return -1; + /* The following are needed in case of queue count re-configuration and + * not for descriptor count re-configuration. + */ + if (queue_count_update) { + if (octeon_setup_instr_queues(oct)) + return -1; + + if (octeon_setup_output_queues(oct)) + return -1; + + /* Recreating mbox for PF that is SRIOV disabled */ + if (OCTEON_CN23XX_PF(oct) && !oct->sriov_info.sriov_enabled) { + if (oct->fn_list.setup_mbox(oct)) { + dev_err(&oct->pci_dev->dev, "Mailbox setup failed\n"); + return -1; + } + } + + /* Deleting and recreating IRQs whether the interface is SRIOV + * enabled or disabled. + */ + if (lio_irq_reallocate_irqs(oct, num_qs)) { + dev_err(&oct->pci_dev->dev, "IRQs could not be allocated\n"); + return -1; + } + + /* Enable the input and output queues for this Octeon device */ + if (oct->fn_list.enable_io_queues(oct)) { + dev_err(&oct->pci_dev->dev, "Failed to enable input/output queues\n"); + return -1; + } + + for (i = 0; i < oct->num_oqs; i++) + writel(oct->droq[i]->max_count, + oct->droq[i]->pkts_credit_reg); + + /* Informing firmware about the new queue count. It is required + * for firmware to allocate more number of queues than those at + * load time. + */ + if (OCTEON_CN23XX_PF(oct) && !oct->sriov_info.sriov_enabled) { + if (lio_23xx_reconfigure_queue_count(lio)) + return -1; + } } - /* Enable the input and output queues for this Octeon device */ - if (oct->fn_list.enable_io_queues(oct)) { - dev_err(&oct->pci_dev->dev, "Failed to enable input/output queues"); + /* Once firmware is aware of the new value, queues can be recreated */ + if (liquidio_setup_io_queues(oct, 0, num_qs, num_qs)) { + dev_err(&oct->pci_dev->dev, "I/O queues creation failed\n"); return -1; } - if (update && lio_send_queue_count_update(netdev, num_qs)) - return -1; + if (queue_count_update) { + if (lio_setup_glists(oct, lio, num_qs)) { + dev_err(&oct->pci_dev->dev, "Gather list allocation failed\n"); + return -1; + } + + /* Send firmware the information about new number of queues + * if the interface is a VF or a PF that is SRIOV enabled. + */ + if (oct->sriov_info.sriov_enabled || OCTEON_CN23XX_VF(oct)) + if (lio_send_queue_count_update(netdev, num_qs)) + return -1; + } return 0; } @@ -922,7 +1149,7 @@ static int lio_ethtool_set_ringparam(struct net_device *netdev, CFG_SET_NUM_RX_DESCS_NIC_IF(octeon_get_conf(oct), lio->ifidx, rx_count); - if (lio_reset_queues(netdev, lio->linfo.num_txpciq)) + if (lio_reset_queues(netdev, oct->num_iqs)) goto err_lio_reset_queues; if (stopped) diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c index f414cd7..dc801b1 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c @@ -497,7 +497,7 @@ static void liquidio_deinit_pci(void) */ static inline int check_txq_status(struct lio *lio) { - int numqs = lio->netdev->num_tx_queues; + int numqs = lio->netdev->real_num_tx_queues; int ret_val = 0; int q, iq; @@ -1521,7 +1521,7 @@ static void free_netsgbuf(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); spin_unlock(&lio->glist_lock[iq]); @@ -1564,7 +1564,7 @@ static void free_netsgbuf_with_resp(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -1851,11 +1851,6 @@ static int liquidio_open(struct net_device *netdev) ifstate_set(lio, LIO_IFSTATE_RUNNING); - /* Ready for link status updates */ - lio->intf_open = 1; - - netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n"); - if (OCTEON_CN23XX_PF(oct)) { if (!oct->msix_on) if (setup_tx_poll_fn(netdev)) @@ -1865,7 +1860,12 @@ static int liquidio_open(struct net_device *netdev) return -1; } - start_txqs(netdev); + netif_tx_start_all_queues(netdev); + + /* Ready for link status updates */ + lio->intf_open = 1; + + netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n"); /* tell Octeon to start forwarding packets to host */ send_rx_ctrl_cmd(lio, 1); @@ -1888,11 +1888,15 @@ static int liquidio_stop(struct net_device *netdev) ifstate_reset(lio, LIO_IFSTATE_RUNNING); - netif_tx_disable(netdev); + /* Stop any link updates */ + lio->intf_open = 0; + + stop_txqs(netdev); /* Inform that netif carrier is down */ netif_carrier_off(netdev); - lio->intf_open = 0; + netif_tx_disable(netdev); + lio->linfo.link.s.link_up = 0; lio->link_changes++; @@ -2312,7 +2316,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) lio = GET_LIO(netdev); oct = lio->oct_dev; - q_idx = skb_iq(lio, skb); + q_idx = skb_iq(oct, skb); tag = q_idx; iq_no = lio->linfo.txpciq[q_idx].s.q_no; @@ -3278,6 +3282,7 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) struct liquidio_if_cfg_resp *resp; struct octdev_props *props; int retval, num_iqueues, num_oqueues; + int max_num_queues = 0; union oct_nic_if_cfg if_cfg; unsigned int base_queue; unsigned int gmx_port_id; @@ -3360,7 +3365,7 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) sc->callback = lio_if_cfg_callback; sc->callback_arg = sc; - sc->wait_time = 3000; + sc->wait_time = LIO_IFCFG_WAIT_TIME; retval = octeon_send_soft_command(octeon_dev, sc); if (retval == IQ_SEND_FAILED) { @@ -3414,11 +3419,20 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) resp->cfg_info.oqmask); goto setup_nic_dev_fail; } + + if (OCTEON_CN6XXX(octeon_dev)) { + max_num_queues = CFG_GET_IQ_MAX_Q(CHIP_CONF(octeon_dev, + cn6xxx)); + } else if (OCTEON_CN23XX_PF(octeon_dev)) { + max_num_queues = CFG_GET_IQ_MAX_Q(CHIP_CONF(octeon_dev, + cn23xx_pf)); + } + dev_dbg(&octeon_dev->pci_dev->dev, - "interface %d, iqmask %016llx, oqmask %016llx, numiqueues %d, numoqueues %d\n", + "interface %d, iqmask %016llx, oqmask %016llx, numiqueues %d, numoqueues %d max_num_queues: %d\n", i, resp->cfg_info.iqmask, resp->cfg_info.oqmask, - num_iqueues, num_oqueues); - netdev = alloc_etherdev_mq(LIO_SIZE, num_iqueues); + num_iqueues, num_oqueues, max_num_queues); + netdev = alloc_etherdev_mq(LIO_SIZE, max_num_queues); if (!netdev) { dev_err(&octeon_dev->pci_dev->dev, "Device allocation failed\n"); @@ -3433,6 +3447,20 @@ static int setup_nic_devices(struct octeon_device *octeon_dev) netdev->netdev_ops = &lionetdevops; SWITCHDEV_SET_OPS(netdev, &lio_pf_switchdev_ops); + retval = netif_set_real_num_rx_queues(netdev, num_oqueues); + if (retval) { + dev_err(&octeon_dev->pci_dev->dev, + "setting real number rx failed\n"); + goto setup_nic_dev_fail; + } + + retval = netif_set_real_num_tx_queues(netdev, num_iqueues); + if (retval) { + dev_err(&octeon_dev->pci_dev->dev, + "setting real number tx failed\n"); + goto setup_nic_dev_fail; + } + lio = GET_LIO(netdev); memset(lio, 0, sizeof(struct lio)); @@ -4053,7 +4081,9 @@ static int octeon_device_init(struct octeon_device *octeon_dev) } atomic_set(&octeon_dev->status, OCT_DEV_MBOX_SETUP_DONE); - if (octeon_allocate_ioq_vector(octeon_dev)) { + if (octeon_allocate_ioq_vector + (octeon_dev, + octeon_dev->sriov_info.num_pf_rings)) { dev_err(&octeon_dev->pci_dev->dev, "OCTEON: ioq vector allocation failed\n"); return 1; } diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c index 246752a..4b5ba02 100644 --- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c +++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c @@ -849,7 +849,7 @@ static void free_netsgbuf(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -893,7 +893,7 @@ static void free_netsgbuf_with_resp(void *buf) i++; } - iq = skb_iq(lio, skb); + iq = skb_iq(lio->oct_dev, skb); spin_lock(&lio->glist_lock[iq]); list_add_tail(&g->list, &lio->glist[iq]); @@ -1392,7 +1392,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev) lio = GET_LIO(netdev); oct = lio->oct_dev; - q_idx = skb_iq(lio, skb); + q_idx = skb_iq(lio->oct_dev, skb); tag = q_idx; iq_no = lio->linfo.txpciq[q_idx].s.q_no; @@ -2324,7 +2324,7 @@ static int octeon_device_init(struct octeon_device *oct) } atomic_set(&oct->status, OCT_DEV_MBOX_SETUP_DONE); - if (octeon_allocate_ioq_vector(oct)) { + if (octeon_allocate_ioq_vector(oct, oct->sriov_info.rings_per_vf)) { dev_err(&oct->pci_dev->dev, "ioq vector allocation failed\n"); return 1; } diff --git a/drivers/net/ethernet/cavium/liquidio/liquidio_common.h b/drivers/net/ethernet/cavium/liquidio/liquidio_common.h index 34a94da..ba854f1 100644 --- a/drivers/net/ethernet/cavium/liquidio/liquidio_common.h +++ b/drivers/net/ethernet/cavium/liquidio/liquidio_common.h @@ -84,6 +84,7 @@ enum octeon_tag_type { #define OPCODE_NIC_IF_CFG 0x09 #define OPCODE_NIC_VF_DRV_NOTICE 0x0A #define OPCODE_NIC_INTRMOD_PARAMS 0x0B +#define OPCODE_NIC_QCOUNT_UPDATE 0x12 #define OPCODE_NIC_SET_TRUSTED_VF 0x13 #define OPCODE_NIC_SYNC_OCTEON_TIME 0x14 #define VF_DRV_LOADED 1 diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_device.c b/drivers/net/ethernet/cavium/liquidio/octeon_device.c index f38abf6..f878a55 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_device.c +++ b/drivers/net/ethernet/cavium/liquidio/octeon_device.c @@ -824,23 +824,18 @@ int octeon_deregister_device(struct octeon_device *oct) } int -octeon_allocate_ioq_vector(struct octeon_device *oct) +octeon_allocate_ioq_vector(struct octeon_device *oct, u32 num_ioqs) { - int i, num_ioqs = 0; struct octeon_ioq_vector *ioq_vector; int cpu_num; int size; - - if (OCTEON_CN23XX_PF(oct)) - num_ioqs = oct->sriov_info.num_pf_rings; - else if (OCTEON_CN23XX_VF(oct)) - num_ioqs = oct->sriov_info.rings_per_vf; + int i; size = sizeof(struct octeon_ioq_vector) * num_ioqs; oct->ioq_vector = vzalloc(size); if (!oct->ioq_vector) - return 1; + return -1; for (i = 0; i < num_ioqs; i++) { ioq_vector = &oct->ioq_vector[i]; ioq_vector->oct_dev = oct; @@ -856,6 +851,7 @@ octeon_allocate_ioq_vector(struct octeon_device *oct) else ioq_vector->ioq_num = i; } + return 0; } diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_device.h b/drivers/net/ethernet/cavium/liquidio/octeon_device.h index 91937cc..9430c0a 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_device.h +++ b/drivers/net/ethernet/cavium/liquidio/octeon_device.h @@ -867,7 +867,7 @@ void *oct_get_config_info(struct octeon_device *oct, u16 card_type); struct octeon_config *octeon_get_conf(struct octeon_device *oct); void octeon_free_ioq_vector(struct octeon_device *oct); -int octeon_allocate_ioq_vector(struct octeon_device *oct); +int octeon_allocate_ioq_vector(struct octeon_device *oct, u32 num_ioqs); void lio_enable_irq(struct octeon_droq *droq, struct octeon_instr_queue *iq); /* LiquidIO driver pivate flags */ diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h index 8894889..f3bf635 100644 --- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h +++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h @@ -47,6 +47,8 @@ struct liquidio_if_cfg_resp { u64 status; }; +#define LIO_IFCFG_WAIT_TIME 3000 /* In milli seconds */ + /* Structure of a node in list of gather components maintained by * NIC driver for each network device. */ @@ -544,7 +546,7 @@ static inline void stop_txqs(struct net_device *netdev) { int i; - for (i = 0; i < netdev->num_tx_queues; i++) + for (i = 0; i < netdev->real_num_tx_queues; i++) netif_stop_subqueue(netdev, i); } @@ -557,7 +559,7 @@ static inline void wake_txqs(struct net_device *netdev) struct lio *lio = GET_LIO(netdev); int i, qno; - for (i = 0; i < netdev->num_tx_queues; i++) { + for (i = 0; i < netdev->real_num_tx_queues; i++) { qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs].s.q_no; if (__netif_subqueue_stopped(netdev, i)) { @@ -578,14 +580,14 @@ static inline void start_txqs(struct net_device *netdev) int i; if (lio->linfo.link.s.link_up) { - for (i = 0; i < netdev->num_tx_queues; i++) + for (i = 0; i < netdev->real_num_tx_queues; i++) netif_start_subqueue(netdev, i); } } -static inline int skb_iq(struct lio *lio, struct sk_buff *skb) +static inline int skb_iq(struct octeon_device *oct, struct sk_buff *skb) { - return skb->queue_mapping % lio->linfo.num_txpciq; + return skb->queue_mapping % oct->num_iqs; } /**