From patchwork Thu Mar 15 15:08:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Abeni X-Patchwork-Id: 886284 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 402BnZ3DbCz9sVj for ; Fri, 16 Mar 2018 02:09:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E39E48A1DA; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S1IAXfDm1jD5; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 3F3EA8A1ED; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 972E71C0A7F for ; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 941CA8A1EB for ; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gPRdVQVjyX0n for ; Thu, 15 Mar 2018 15:09:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by hemlock.osuosl.org (Postfix) with ESMTPS id B88DC8A1E6 for ; Thu, 15 Mar 2018 15:09:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B85FC4068023; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.32.181.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0258C1FB2; Thu, 15 Mar 2018 15:08:59 +0000 (UTC) From: Paolo Abeni To: netdev@vger.kernel.org Date: Thu, 15 Mar 2018 16:08:12 +0100 Message-Id: <384ee099d617f3d3786a618b11cc10616923ec45.1521124830.git.pabeni@redhat.com> In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Mar 2018 15:09:00 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Mar 2018 15:09:00 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'pabeni@redhat.com' RCPT:'' Subject: [Intel-wired-lan] [RFC PATCH 2/2] ixgbe: setup XPS via netif_set_xps() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eric Dumazet , intel-wired-lan@lists.osuosl.org, "David S. Miller" MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Before this commit, ixgbe with the default setting lacks XPS mapping for CPUs id greater than the number of tx queues. As a consequence the xmit path for such CPUs experience a relevant cost in __netdev_pick_tx, mainly due to skb_tx_hash(), as reported by the perf tool: 7.55%--netdev_pick_tx | --6.92%--__netdev_pick_tx | --6.35%--__skb_tx_hash | --5.94%--__skb_get_hash | --3.22%--__skb_flow_dissect in the following scenario: ethtool -L em1 combined 1 taskset 2 netperf -H 192.168.1.1 -t UDP_STREAM -- -m 1 MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.101.1 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 1 10.00 11497225 0 9.20 After this commit the perf tool reports: 0.85%--__netdev_pick_tx and netperf reports: MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.101.1 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 1 10.00 12736058 0 10.19 roughly +10% in xmit tput. Signed-off-by: Paolo Abeni --- drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 +- drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c | 1 + drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 13 +++---------- 3 files changed, 5 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h index c1e3a0039ea5..04aaecce81d2 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h @@ -293,7 +293,6 @@ enum ixgbe_ring_state_t { __IXGBE_RX_CSUM_UDP_ZERO_ERR, __IXGBE_RX_FCOE, __IXGBE_TX_FDIR_INIT_DONE, - __IXGBE_TX_XPS_INIT_DONE, __IXGBE_TX_DETECT_HANG, __IXGBE_HANG_CHECK_ARMED, __IXGBE_TX_XDP_RING, @@ -827,6 +826,7 @@ enum ixgbe_state_t { __IXGBE_PTP_RUNNING, __IXGBE_PTP_TX_IN_PROGRESS, __IXGBE_RESET_REQUESTED, + __IXGBE_TX_XPS_INIT_DONE, }; struct ixgbe_cb { diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c index c0e6ab42e0e1..da4e6416e8eb 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c @@ -3224,6 +3224,7 @@ static int ixgbe_set_channels(struct net_device *dev, #endif /* use setup TC to update any traffic class queue mapping */ + clear_bit(__IXGBE_TX_XPS_INIT_DONE, &adapter->state); return ixgbe_setup_tc(dev, adapter->hw_tcs); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 85369423452d..5bd45fc737fa 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -3566,16 +3566,6 @@ void ixgbe_configure_tx_ring(struct ixgbe_adapter *adapter, ring->atr_sample_rate = 0; } - /* initialize XPS */ - if (!test_and_set_bit(__IXGBE_TX_XPS_INIT_DONE, &ring->state)) { - struct ixgbe_q_vector *q_vector = ring->q_vector; - - if (q_vector) - netif_set_xps_queue(ring->netdev, - &q_vector->affinity_mask, - ring->queue_index); - } - clear_bit(__IXGBE_HANG_CHECK_ARMED, &ring->state); /* reinitialize tx_buffer_info */ @@ -5626,6 +5616,9 @@ static void ixgbe_up_complete(struct ixgbe_adapter *adapter) int err; u32 ctrl_ext; + if (!test_and_set_bit(__IXGBE_TX_XPS_INIT_DONE, &adapter->state)) + netif_set_xps(adapter->netdev); + ixgbe_get_hw_control(adapter); ixgbe_setup_gpie(adapter);