From patchwork Wed Sep 2 14:35:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Edward Cree X-Patchwork-Id: 1355825 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=solarflare.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BhRMv41Fpz9sTM for ; Thu, 3 Sep 2020 00:36:43 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727997AbgIBOgW (ORCPT ); Wed, 2 Sep 2020 10:36:22 -0400 Received: from dispatch1-us1.ppe-hosted.com ([148.163.129.52]:51398 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726618AbgIBOgL (ORCPT ); Wed, 2 Sep 2020 10:36:11 -0400 Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.60]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 71B2B6009C; Wed, 2 Sep 2020 14:36:10 +0000 (UTC) Received: from us4-mdac16-17.ut7.mdlocal (unknown [10.7.65.241]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 6F6DB2009A; Wed, 2 Sep 2020 14:36:10 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.174]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id F1B8F1C0054; Wed, 2 Sep 2020 14:36:09 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id ABF5B1C0080; Wed, 2 Sep 2020 14:36:09 +0000 (UTC) Received: from [10.17.20.203] (10.17.20.203) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Sep 2020 15:36:04 +0100 From: Edward Cree Subject: [PATCH net-next 1/5] sfc: add and use efx_tx_send_pending in tx.c To: , CC: References: Message-ID: <1edd44e5-a73a-149f-fe0c-96969627d211@solarflare.com> Date: Wed, 2 Sep 2020 15:35:53 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Originating-IP: [10.17.20.203] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25640.003 X-TM-AS-Result: No-3.555900-8.000000-10 X-TMASE-MatchedRID: 7k7c1meM2Cj6mS2x8PRvpsZkWeHTuv2uAKbvziCwm7j2u2oLJUFmGMiT Wug2C4DNl1M7KT9/aqAnsasxq4/76mJZXQNDzktSGjzBgnFZvQ4TcSAXT4xW4LJlZWzEYNxAbnj BcAybL/ynQFe+ESmb0wbYbps2GRg0V3LZOzkjrY+iAZ3zAhQYggeCHewokHM/e7ijHq7g9oadNC 251TPFyrxzOVuPy6ov5GzdUnKKAE5WmVSbj8vwrNUcSZdVkdtz+LljbN4c70MTLunq7JxcvUFhN Q5NCwwg0d9L2XyFbWnUPEKgrYXxxo9oUcx9VMLgOX/V8P8ail1yZ8zcONpAscRB0bsfrpPIqxB3 2o9eGckrmQHTr2S7jieB9cuRcVaonvPRL2YHViS8+X6w8/WcVlbjK6gV4EVIg3CLcpYSGvmnWRG wVBanyVBxQmiUr9slxlOi1Fh8+TXpsag+4i8C+Hzm6hivSaZZop2lf3StGhISt1bcvKF7ZNkrC8 mOIB+LwL6SxPpr1/I= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--3.555900-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25640.003 X-MDID: 1599057370-n0UwdVncKUAo Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Instead of using efx_tx_queue_partner(), which relies on the assumption that tx_queues_per_channel is 2, efx_tx_send_pending() iterates over txqs with efx_for_each_channel_tx_queue(). Signed-off-by: Edward Cree --- drivers/net/ethernet/sfc/tx.c | 59 ++++++++++++++++++----------------- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/sfc/tx.c b/drivers/net/ethernet/sfc/tx.c index 727201d5eb24..71eb99db5439 100644 --- a/drivers/net/ethernet/sfc/tx.c +++ b/drivers/net/ethernet/sfc/tx.c @@ -268,6 +268,19 @@ static int efx_enqueue_skb_pio(struct efx_tx_queue *tx_queue, } #endif /* EFX_USE_PIO */ +/* Send any pending traffic for a channel. xmit_more is shared across all + * queues for a channel, so we must check all of them. + */ +static void efx_tx_send_pending(struct efx_channel *channel) +{ + struct efx_tx_queue *q; + + efx_for_each_channel_tx_queue(q, channel) { + if (q->xmit_more_available) + efx_nic_push_buffers(q); + } +} + /* * Add a socket buffer to a TX queue * @@ -336,21 +349,11 @@ netdev_tx_t __efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb efx_tx_maybe_stop_queue(tx_queue); - /* Pass off to hardware */ - if (__netdev_tx_sent_queue(tx_queue->core_txq, skb_len, xmit_more)) { - struct efx_tx_queue *txq2 = efx_tx_queue_partner(tx_queue); - - /* There could be packets left on the partner queue if - * xmit_more was set. If we do not push those they - * could be left for a long time and cause a netdev watchdog. - */ - if (txq2->xmit_more_available) - efx_nic_push_buffers(txq2); + tx_queue->xmit_more_available = true; - efx_nic_push_buffers(tx_queue); - } else { - tx_queue->xmit_more_available = xmit_more; - } + /* Pass off to hardware */ + if (__netdev_tx_sent_queue(tx_queue->core_txq, skb_len, xmit_more)) + efx_tx_send_pending(tx_queue->channel); if (segments) { tx_queue->tso_bursts++; @@ -371,14 +374,8 @@ netdev_tx_t __efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb * on this queue or a partner queue then we need to push here to get the * previous packets out. */ - if (!xmit_more) { - struct efx_tx_queue *txq2 = efx_tx_queue_partner(tx_queue); - - if (txq2->xmit_more_available) - efx_nic_push_buffers(txq2); - - efx_nic_push_buffers(tx_queue); - } + if (!xmit_more) + efx_tx_send_pending(tx_queue->channel); return NETDEV_TX_OK; } @@ -489,18 +486,24 @@ netdev_tx_t efx_hard_start_xmit(struct sk_buff *skb, EFX_WARN_ON_PARANOID(!netif_device_present(net_dev)); - /* PTP "event" packet */ - if (unlikely(efx_xmit_with_hwtstamp(skb)) && - unlikely(efx_ptp_is_ptp_tx(efx, skb))) { - return efx_ptp_tx(efx, skb); - } - index = skb_get_queue_mapping(skb); type = skb->ip_summed == CHECKSUM_PARTIAL ? EFX_TXQ_TYPE_OFFLOAD : 0; if (index >= efx->n_tx_channels) { index -= efx->n_tx_channels; type |= EFX_TXQ_TYPE_HIGHPRI; } + + /* PTP "event" packet */ + if (unlikely(efx_xmit_with_hwtstamp(skb)) && + unlikely(efx_ptp_is_ptp_tx(efx, skb))) { + /* There may be existing transmits on the channel that are + * waiting for this packet to trigger the doorbell write. + * We need to send the packets at this point. + */ + efx_tx_send_pending(efx_get_tx_channel(efx, index)); + return efx_ptp_tx(efx, skb); + } + tx_queue = efx_get_tx_queue(efx, index, type); return __efx_enqueue_skb(tx_queue, skb); From patchwork Wed Sep 2 14:36:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Edward Cree X-Patchwork-Id: 1355826 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=solarflare.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BhRN95ndDz9sTM for ; Thu, 3 Sep 2020 00:36:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728104AbgIBOgz (ORCPT ); Wed, 2 Sep 2020 10:36:55 -0400 Received: from dispatch1-us1.ppe-hosted.com ([148.163.129.52]:56702 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726285AbgIBOgo (ORCPT ); Wed, 2 Sep 2020 10:36:44 -0400 Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.62]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 54E0B60155; Wed, 2 Sep 2020 14:36:44 +0000 (UTC) Received: from us4-mdac16-21.ut7.mdlocal (unknown [10.7.65.245]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id B0F9F800B4; Wed, 2 Sep 2020 14:36:43 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.175]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 2DB8E2800A7; Wed, 2 Sep 2020 14:36:42 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 678817000AC; Wed, 2 Sep 2020 14:36:42 +0000 (UTC) Received: from [10.17.20.203] (10.17.20.203) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Sep 2020 15:36:26 +0100 From: Edward Cree Subject: [PATCH net-next 2/5] sfc: use tx_queue->old_read_count in EF100 TX path To: , CC: References: Message-ID: <6c1deaca-102d-40bb-5ffe-9507f01a8788@solarflare.com> Date: Wed, 2 Sep 2020 15:36:22 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Originating-IP: [10.17.20.203] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25640.003 X-TM-AS-Result: No-4.229900-8.000000-10 X-TMASE-MatchedRID: B4VcMIpPJ800hZmSa+ztRMzSKGx9g8xheouvej40T4gd0WOKRkwsh1ym Rv3NQjsE4aBLis6ititw5T4Iaj538mJZXQNDzktSMiMrbc70Pfdc1jwHBugxQIxRWJphhsrcrzM i7F2pFcpny03aprSl5E4r6tWjD8M+YAgEin+2ERDiHyvyXeXh5sHWhOU1PTVYNCjbDif9OIzSJJ cbp1Y+W8Q1xvPMAvfZDvP/tkcd5GOMQmg2+WN4O095wQijrwBLfS0Ip2eEHny+qryzYw2E8Jkw8 KdMzN86KrauXd3MZDUYmhvFbntAL6TRb8tZ3yWI26UtHqFTik1hKMIvwkzgVs1EcebW768Tg5qe iqlH6shWa9YJ+3+CBf2iSrkndGlHB+RsQUnQHf3jCHw9x1eKskODDY5/BuEsoxrk6Q2mIeSJ+wv 7oJjhGclmajRS8yWxQwymtxuJ6y0= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--4.229900-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25640.003 X-MDID: 1599057403-2gRDV7kY_GZZ Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org As in the Siena/EF10 case, it minimises cacheline ping-pong between the TX and completion paths. Signed-off-by: Edward Cree --- drivers/net/ethernet/sfc/ef100_tx.c | 8 ++++++-- drivers/net/ethernet/sfc/net_driver.h | 14 ++++++++++++++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/sfc/ef100_tx.c b/drivers/net/ethernet/sfc/ef100_tx.c index a09546e43408..47d67b1e379b 100644 --- a/drivers/net/ethernet/sfc/ef100_tx.c +++ b/drivers/net/ethernet/sfc/ef100_tx.c @@ -359,15 +359,19 @@ int ef100_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb) goto err; ef100_tx_make_descriptors(tx_queue, skb, segments); - fill_level = efx_channel_tx_fill_level(tx_queue->channel); + fill_level = efx_channel_tx_old_fill_level(tx_queue->channel); if (fill_level > efx->txq_stop_thresh) { + struct efx_tx_queue *txq2; + netif_tx_stop_queue(tx_queue->core_txq); /* Re-read after a memory barrier in case we've raced with * the completion path. Otherwise there's a danger we'll never * restart the queue if all completions have just happened. */ smp_mb(); - fill_level = efx_channel_tx_fill_level(tx_queue->channel); + efx_for_each_channel_tx_queue(txq2, tx_queue->channel) + txq2->old_read_count = READ_ONCE(txq2->read_count); + fill_level = efx_channel_tx_old_fill_level(tx_queue->channel); if (fill_level < efx->txq_stop_thresh) netif_tx_start_queue(tx_queue->core_txq); } diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 338ebb0402be..2358d35bffb5 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -1692,6 +1692,20 @@ efx_channel_tx_fill_level(struct efx_channel *channel) return fill_level; } +/* Conservative approximation of efx_channel_tx_fill_level using cached value */ +static inline unsigned int +efx_channel_tx_old_fill_level(struct efx_channel *channel) +{ + struct efx_tx_queue *tx_queue; + unsigned int fill_level = 0; + + efx_for_each_channel_tx_queue(tx_queue, channel) + fill_level = max(fill_level, + tx_queue->insert_count - tx_queue->old_read_count); + + return fill_level; +} + /* Get all supported features. * If a feature is not fixed, it is present in hw_features. * If a feature is fixed, it does not present in hw_features, but From patchwork Wed Sep 2 14:36:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Edward Cree X-Patchwork-Id: 1355827 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=solarflare.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BhRP11m5rz9sPB for ; Thu, 3 Sep 2020 00:37:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728101AbgIBOhh (ORCPT ); Wed, 2 Sep 2020 10:37:37 -0400 Received: from dispatch1-us1.ppe-hosted.com ([148.163.129.52]:37170 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728073AbgIBOhV (ORCPT ); Wed, 2 Sep 2020 10:37:21 -0400 Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.64]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 2742D600E2; Wed, 2 Sep 2020 14:37:21 +0000 (UTC) Received: from us4-mdac16-53.ut7.mdlocal (unknown [10.7.66.24]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 24ADE2009B; Wed, 2 Sep 2020 14:37:21 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.66.40]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 9EA77220068; Wed, 2 Sep 2020 14:37:20 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 576F0BC0068; Wed, 2 Sep 2020 14:37:20 +0000 (UTC) Received: from [10.17.20.203] (10.17.20.203) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Sep 2020 15:37:01 +0100 From: Edward Cree Subject: [PATCH net-next 3/5] sfc: use efx_channel_tx_[old_]fill_level() in Siena/EF10 TX datapath To: , CC: References: Message-ID: <727d78d7-aa74-1920-a5fa-796881afb923@solarflare.com> Date: Wed, 2 Sep 2020 15:36:46 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Originating-IP: [10.17.20.203] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25640.003 X-TM-AS-Result: No-6.967300-8.000000-10 X-TMASE-MatchedRID: oOTqEOEfBbscAZrHLi86M8ZkWeHTuv2uAKbvziCwm7jBSouFRjh0Andd uDm7lgGn8XVI39JCRnSjfNAVYAJRAr+Q0YdVmuyWnFVnNmvv47tLXPA26IG0hN9RlPzeVuQQ8b1 mXTqetmzERx+bb7v/+VLil+9DIcvM1x7q6N2snafHmyDJSEsI2z39vrXxKlsd1l38M6aWfEgU45 rKCfVRMXChf2aWJ/fe1rs6dM2A4+ROi++CZQcQcodlc1JaOB1TD6NL4tGmXzXc9j0JdwtdWaPFj JEFr+olA9Mriq0CDAg9wJeM2pSaRd934/rDAK3zUpXqgxV1N6nTkz4ESBI6DHBM7r9Rch7Ub3mg 1ALZ3OlGCULcizy/4b72rZBe1NOXftwZ3X11IV0= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--6.967300-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25640.003 X-MDID: 1599057441-6ICPZX4MDH2E Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Instead of open-coding the calculation with efx_tx_queue_partner(), use the functions that iterate over numbers of queues other than 2 with efx_for_each_channel_tx_queue(). Signed-off-by: Edward Cree --- drivers/net/ethernet/sfc/net_driver.h | 4 ---- drivers/net/ethernet/sfc/tx.c | 14 ++++++-------- drivers/net/ethernet/sfc/tx_common.c | 5 +---- 3 files changed, 7 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 2358d35bffb5..08c4858fc604 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -1681,10 +1681,6 @@ efx_channel_tx_fill_level(struct efx_channel *channel) struct efx_tx_queue *tx_queue; unsigned int fill_level = 0; - /* This function is currently only used by EF100, which maybe - * could do something simpler and just compute the fill level - * of the single TXQ that's really in use. - */ efx_for_each_channel_tx_queue(tx_queue, channel) fill_level = max(fill_level, tx_queue->insert_count - tx_queue->read_count); diff --git a/drivers/net/ethernet/sfc/tx.c b/drivers/net/ethernet/sfc/tx.c index 71eb99db5439..713b129f9b83 100644 --- a/drivers/net/ethernet/sfc/tx.c +++ b/drivers/net/ethernet/sfc/tx.c @@ -59,13 +59,12 @@ u8 *efx_tx_get_copy_buffer_limited(struct efx_tx_queue *tx_queue, static void efx_tx_maybe_stop_queue(struct efx_tx_queue *txq1) { - /* We need to consider both queues that the net core sees as one */ - struct efx_tx_queue *txq2 = efx_tx_queue_partner(txq1); + /* We need to consider all queues that the net core sees as one */ struct efx_nic *efx = txq1->efx; + struct efx_tx_queue *txq2; unsigned int fill_level; - fill_level = max(txq1->insert_count - txq1->old_read_count, - txq2->insert_count - txq2->old_read_count); + fill_level = efx_channel_tx_old_fill_level(txq1->channel); if (likely(fill_level < efx->txq_stop_thresh)) return; @@ -85,11 +84,10 @@ static void efx_tx_maybe_stop_queue(struct efx_tx_queue *txq1) */ netif_tx_stop_queue(txq1->core_txq); smp_mb(); - txq1->old_read_count = READ_ONCE(txq1->read_count); - txq2->old_read_count = READ_ONCE(txq2->read_count); + efx_for_each_channel_tx_queue(txq2, txq1->channel) + txq2->old_read_count = READ_ONCE(txq2->read_count); - fill_level = max(txq1->insert_count - txq1->old_read_count, - txq2->insert_count - txq2->old_read_count); + fill_level = efx_channel_tx_old_fill_level(txq1->channel); EFX_WARN_ON_ONCE_PARANOID(fill_level >= efx->txq_entries); if (likely(fill_level < efx->txq_stop_thresh)) { smp_mb(); diff --git a/drivers/net/ethernet/sfc/tx_common.c b/drivers/net/ethernet/sfc/tx_common.c index 793e234819a8..02f7c772d261 100644 --- a/drivers/net/ethernet/sfc/tx_common.c +++ b/drivers/net/ethernet/sfc/tx_common.c @@ -242,7 +242,6 @@ void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index) { unsigned int fill_level, pkts_compl = 0, bytes_compl = 0; struct efx_nic *efx = tx_queue->efx; - struct efx_tx_queue *txq2; EFX_WARN_ON_ONCE_PARANOID(index > tx_queue->ptr_mask); @@ -261,9 +260,7 @@ void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index) if (unlikely(netif_tx_queue_stopped(tx_queue->core_txq)) && likely(efx->port_enabled) && likely(netif_device_present(efx->net_dev))) { - txq2 = efx_tx_queue_partner(tx_queue); - fill_level = max(tx_queue->insert_count - tx_queue->read_count, - txq2->insert_count - txq2->read_count); + fill_level = efx_channel_tx_fill_level(tx_queue->channel); if (fill_level <= efx->txq_wake_thresh) netif_tx_wake_queue(tx_queue->core_txq); } From patchwork Wed Sep 2 14:37:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Edward Cree X-Patchwork-Id: 1355828 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=solarflare.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BhRP61SlGz9sPB for ; Thu, 3 Sep 2020 00:37:46 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728104AbgIBOhn (ORCPT ); Wed, 2 Sep 2020 10:37:43 -0400 Received: from dispatch1-us1.ppe-hosted.com ([148.163.129.52]:54896 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727997AbgIBOha (ORCPT ); Wed, 2 Sep 2020 10:37:30 -0400 Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.62]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 339116011B; Wed, 2 Sep 2020 14:37:30 +0000 (UTC) Received: from us4-mdac16-73.ut7.mdlocal (unknown [10.7.64.192]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 2E97B80094; Wed, 2 Sep 2020 14:37:30 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.176]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 9789728007B; Wed, 2 Sep 2020 14:37:29 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 359DB6800A9; Wed, 2 Sep 2020 14:37:29 +0000 (UTC) Received: from [10.17.20.203] (10.17.20.203) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Sep 2020 15:37:19 +0100 From: Edward Cree Subject: [PATCH net-next 4/5] sfc: rewrite efx_tx_may_pio To: , CC: References: Message-ID: <606f9dec-c0fd-7cf3-0a35-9d3bfcb39605@solarflare.com> Date: Wed, 2 Sep 2020 15:37:09 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Originating-IP: [10.17.20.203] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25640.003 X-TM-AS-Result: No-11.129200-8.000000-10 X-TMASE-MatchedRID: +0eKDNvbD445s4xK97Sm/8ZkWeHTuv2uDvc/j9oMIgWgJ1CKZ3CoSNAY WUo4HSIk8XVI39JCRnSjfNAVYAJRAq0iin8P0KjVPwKTD1v8YV5MkOX0UoduuZ+4ziUPq4Lxg6c H7aDJatZCY6mLc3Q3xdTWDiU7V0XtRGYk457itShI3TL4xxtmW8WLeiJW8Z+pvn1ivDdpAIfr7f jWKbC0RQcgrPmtZZiYqNhuFB/lXK/+0PrwrzfHMRYZoKAPfQ6CAKbvziCwm7jAJMh4mAwEG5AaN p2uvngxrpYdrzWjH/rBpVoBHc91GqF2E6v/FJwhlFSf//R6rYp9LQinZ4QefL6qvLNjDYTwmTDw p0zM3zoqtq5d3cxkNdpTQHOytsvtmBo9Xcygo9k1CbbnZgICbNgrdVYONZ7QpNo28YLPHb4= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--11.129200-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25640.003 X-MDID: 1599057450-XRR3eJ3ipNeg Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use efx_for_each_channel_tx_queue() rather than efx_tx_queue_partner(). Make some related simplifications of efx_nic_tx_is_empty() to remove entry points that aren't used. Signed-off-by: Edward Cree --- drivers/net/ethernet/sfc/nic_common.h | 30 ++------------------------- drivers/net/ethernet/sfc/tx.c | 26 ++++++++++++++++++++++- 2 files changed, 27 insertions(+), 29 deletions(-) diff --git a/drivers/net/ethernet/sfc/nic_common.h b/drivers/net/ethernet/sfc/nic_common.h index 974107354087..3f88c6444fa1 100644 --- a/drivers/net/ethernet/sfc/nic_common.h +++ b/drivers/net/ethernet/sfc/nic_common.h @@ -65,8 +65,7 @@ efx_tx_desc(struct efx_tx_queue *tx_queue, unsigned int index) /* Report whether this TX queue would be empty for the given write_count. * May return false negative. */ -static inline bool __efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue, - unsigned int write_count) +static inline bool efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue, unsigned int write_count) { unsigned int empty_read_count = READ_ONCE(tx_queue->empty_read_count); @@ -76,17 +75,6 @@ static inline bool __efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue, return ((empty_read_count ^ write_count) & ~EFX_EMPTY_COUNT_VALID) == 0; } -/* Report whether the NIC considers this TX queue empty, using - * packet_write_count (the write count recorded for the last completable - * doorbell push). May return false negative. EF10 only, which is OK - * because only EF10 supports PIO. - */ -static inline bool efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue) -{ - EFX_WARN_ON_ONCE_PARANOID(!tx_queue->efx->type->option_descriptors); - return __efx_nic_tx_is_empty(tx_queue, tx_queue->packet_write_count); -} - /* Get partner of a TX queue, seen as part of the same net core queue */ /* XXX is this a thing on EF100? */ static inline struct efx_tx_queue *efx_tx_queue_partner(struct efx_tx_queue *tx_queue) @@ -97,20 +85,6 @@ static inline struct efx_tx_queue *efx_tx_queue_partner(struct efx_tx_queue *tx_ return tx_queue + EFX_TXQ_TYPE_OFFLOAD; } -/* Decide whether we can use TX PIO, ie. write packet data directly into - * a buffer on the device. This can reduce latency at the expense of - * throughput, so we only do this if both hardware and software TX rings - * are empty. This also ensures that only one packet at a time can be - * using the PIO buffer. - */ -static inline bool efx_nic_may_tx_pio(struct efx_tx_queue *tx_queue) -{ - struct efx_tx_queue *partner = efx_tx_queue_partner(tx_queue); - - return tx_queue->piobuf && efx_nic_tx_is_empty(tx_queue) && - efx_nic_tx_is_empty(partner); -} - int efx_enqueue_skb_tso(struct efx_tx_queue *tx_queue, struct sk_buff *skb, bool *data_mapped); @@ -125,7 +99,7 @@ int efx_enqueue_skb_tso(struct efx_tx_queue *tx_queue, struct sk_buff *skb, static inline bool efx_nic_may_push_tx_desc(struct efx_tx_queue *tx_queue, unsigned int write_count) { - bool was_empty = __efx_nic_tx_is_empty(tx_queue, write_count); + bool was_empty = efx_nic_tx_is_empty(tx_queue, write_count); tx_queue->empty_read_count = 0; return was_empty && tx_queue->write_count - write_count == 1; diff --git a/drivers/net/ethernet/sfc/tx.c b/drivers/net/ethernet/sfc/tx.c index 713b129f9b83..09955b5c9d1e 100644 --- a/drivers/net/ethernet/sfc/tx.c +++ b/drivers/net/ethernet/sfc/tx.c @@ -264,6 +264,30 @@ static int efx_enqueue_skb_pio(struct efx_tx_queue *tx_queue, ++tx_queue->insert_count; return 0; } + +/* Decide whether we can use TX PIO, ie. write packet data directly into + * a buffer on the device. This can reduce latency at the expense of + * throughput, so we only do this if both hardware and software TX rings + * are empty, including all queues for the channel. This also ensures that + * only one packet at a time can be using the PIO buffer. If the xmit_more + * flag is set then we don't use this - there'll be another packet along + * shortly and we want to hold off the doorbell. + */ +static bool efx_tx_may_pio(struct efx_tx_queue *tx_queue) +{ + struct efx_channel *channel = tx_queue->channel; + + if (!tx_queue->piobuf) + return false; + + EFX_WARN_ON_ONCE_PARANOID(!channel->efx->type->option_descriptors); + + efx_for_each_channel_tx_queue(tx_queue, channel) + if (!efx_nic_tx_is_empty(tx_queue, tx_queue->packet_write_count)) + return false; + + return true; +} #endif /* EFX_USE_PIO */ /* Send any pending traffic for a channel. xmit_more is shared across all @@ -326,7 +350,7 @@ netdev_tx_t __efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb goto err; #ifdef EFX_USE_PIO } else if (skb_len <= efx_piobuf_size && !xmit_more && - efx_nic_may_tx_pio(tx_queue)) { + efx_tx_may_pio(tx_queue)) { /* Use PIO for short packets with an empty queue. */ if (efx_enqueue_skb_pio(tx_queue, skb)) goto err; From patchwork Wed Sep 2 14:37:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Edward Cree X-Patchwork-Id: 1355832 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=solarflare.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BhRQW061Mz9sPB for ; Thu, 3 Sep 2020 00:38:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727991AbgIBOi5 (ORCPT ); Wed, 2 Sep 2020 10:38:57 -0400 Received: from dispatch1-us1.ppe-hosted.com ([148.163.129.52]:43898 "EHLO dispatch1-us1.ppe-hosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727842AbgIBOhv (ORCPT ); Wed, 2 Sep 2020 10:37:51 -0400 Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.64]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id B9BBE60121; Wed, 2 Sep 2020 14:37:50 +0000 (UTC) Received: from us4-mdac16-60.ut7.mdlocal (unknown [10.7.66.51]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id B816B2009B; Wed, 2 Sep 2020 14:37:50 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.66.37]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 2C89922006F; Wed, 2 Sep 2020 14:37:50 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id D5B42B40081; Wed, 2 Sep 2020 14:37:49 +0000 (UTC) Received: from [10.17.20.203] (10.17.20.203) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 2 Sep 2020 15:37:31 +0100 From: Edward Cree Subject: [PATCH net-next 5/5] sfc: remove efx_tx_queue_partner To: , CC: References: Message-ID: Date: Wed, 2 Sep 2020 15:37:27 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB X-Originating-IP: [10.17.20.203] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25640.003 X-TM-AS-Result: No-1.764400-8.000000-10 X-TMASE-MatchedRID: QifGWEak2+NkukQunGNAEaiUivh0j2Pv6VTG9cZxEjJwGpdgNQ0JrEAc 6DyoS2rIj6kCfX0Edc7U1g4lO1dF7RXQZ3CP5OysMiMrbc70Pfcr9gVlOIN/6p+4ziUPq4LxwDS YCfpWO53i8zVgXoAltkWL4rBlm20vjaPj0W1qn0SujVRFkkVsm06jPCV3lQD12/8kgG7wqjZK/e jkZlThALKLwL/neQSOexBjPccZ1CTrTIWQzj4gcuWy4yH95JvdaY0lwVEsJaM7eLqX+4w4O5BEc rkRxYJ4UjKnO1KVKKwSkbDwum07zqq0MV8nSMBvkLxsYTGf9c0= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--1.764400-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25640.003 X-MDID: 1599057470-oH0x40BGajk5 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org All users of this function are now gone. Signed-off-by: Edward Cree --- drivers/net/ethernet/sfc/nic_common.h | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/drivers/net/ethernet/sfc/nic_common.h b/drivers/net/ethernet/sfc/nic_common.h index 3f88c6444fa1..82271f0b8627 100644 --- a/drivers/net/ethernet/sfc/nic_common.h +++ b/drivers/net/ethernet/sfc/nic_common.h @@ -75,16 +75,6 @@ static inline bool efx_nic_tx_is_empty(struct efx_tx_queue *tx_queue, unsigned i return ((empty_read_count ^ write_count) & ~EFX_EMPTY_COUNT_VALID) == 0; } -/* Get partner of a TX queue, seen as part of the same net core queue */ -/* XXX is this a thing on EF100? */ -static inline struct efx_tx_queue *efx_tx_queue_partner(struct efx_tx_queue *tx_queue) -{ - if (tx_queue->label & EFX_TXQ_TYPE_OFFLOAD) - return tx_queue - EFX_TXQ_TYPE_OFFLOAD; - else - return tx_queue + EFX_TXQ_TYPE_OFFLOAD; -} - int efx_enqueue_skb_tso(struct efx_tx_queue *tx_queue, struct sk_buff *skb, bool *data_mapped);