From patchwork Thu Mar 15 15:08:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Abeni X-Patchwork-Id: 886282 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 402BnX4f7wz9sTy for ; Fri, 16 Mar 2018 02:09:04 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 79B3C87C81; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oaM2IFF7MbNS; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 0089187B74; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id E5E5F1C0A7F for ; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id E2D9A87B74 for ; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TyuRc06wTzO1 for ; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 25DAD87B45 for ; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A84440201A5; Thu, 15 Mar 2018 15:08:59 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.32.181.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5D432C1233; Thu, 15 Mar 2018 15:08:58 +0000 (UTC) From: Paolo Abeni To: netdev@vger.kernel.org Date: Thu, 15 Mar 2018 16:08:11 +0100 Message-Id: <1add7d2952a1b86129c4a7c1f46cf382d6107cb7.1521124830.git.pabeni@redhat.com> In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 15 Mar 2018 15:08:59 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 15 Mar 2018 15:08:59 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'pabeni@redhat.com' RCPT:'' Subject: [Intel-wired-lan] [RFC PATCH 1/2] net: introduce netif_set_xps() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eric Dumazet , intel-wired-lan@lists.osuosl.org, "David S. Miller" MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" netif_set_xps() configures XPS on the given netdevice so that XPS mapping exists for each online CPU. Also, factor out an unlocked version of netif_set_xps_queue() to allow configuring all the netdev queues acquiring the xps lock only once. Netdevice can leverage such helper replacing all the per queue call to netif_set_xps_queue() with a single netif_set_xps(). Signed-off-by: Paolo Abeni --- include/linux/netdevice.h | 6 +++++ net/core/dev.c | 58 +++++++++++++++++++++++++++++++++++------------ 2 files changed, 50 insertions(+), 14 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 5fbb9f1da7fd..95727ccf0865 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3193,6 +3193,7 @@ static inline void netif_wake_subqueue(struct net_device *dev, u16 queue_index) #ifdef CONFIG_XPS int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, u16 index); +int netif_set_xps(struct net_device *dev); #else static inline int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, @@ -3200,6 +3201,11 @@ static inline int netif_set_xps_queue(struct net_device *dev, { return 0; } + +int netif_set_xps(struct net_device *dev) +{ + return 0; +} #endif u16 __skb_tx_hash(const struct net_device *dev, struct sk_buff *skb, diff --git a/net/core/dev.c b/net/core/dev.c index 12a9aad0b057..5a8d3d9ef9b4 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -2177,8 +2177,8 @@ static struct xps_map *expand_xps_map(struct xps_map *map, return new_map; } -int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, - u16 index) +int __netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, + u16 index) { struct xps_dev_maps *dev_maps, *new_dev_maps = NULL; int i, cpu, tci, numa_node_id = -2; @@ -2197,18 +2197,14 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, if (maps_sz < L1_CACHE_BYTES) maps_sz = L1_CACHE_BYTES; - mutex_lock(&xps_map_mutex); - dev_maps = xmap_dereference(dev->xps_maps); /* allocate memory for queue storage */ for_each_cpu_and(cpu, cpu_online_mask, mask) { if (!new_dev_maps) new_dev_maps = kzalloc(maps_sz, GFP_KERNEL); - if (!new_dev_maps) { - mutex_unlock(&xps_map_mutex); + if (!new_dev_maps) return -ENOMEM; - } tci = cpu * num_tc + tc; map = dev_maps ? xmap_dereference(dev_maps->cpu_map[tci]) : @@ -2295,7 +2291,7 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, NUMA_NO_NODE); if (!dev_maps) - goto out_no_maps; + return 0; /* removes queue from unused CPUs */ for_each_possible_cpu(cpu) { @@ -2312,11 +2308,8 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, RCU_INIT_POINTER(dev->xps_maps, NULL); kfree_rcu(dev_maps, rcu); } - -out_no_maps: - mutex_unlock(&xps_map_mutex); - return 0; + error: /* remove any maps that we added */ for_each_possible_cpu(cpu) { @@ -2330,13 +2323,50 @@ int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, } } - mutex_unlock(&xps_map_mutex); - kfree(new_dev_maps); return -ENOMEM; } + +int netif_set_xps_queue(struct net_device *dev, const struct cpumask *mask, + u16 index) +{ + int ret; + + mutex_lock(&xps_map_mutex); + ret = __netif_set_xps_queue(dev, mask, index); + mutex_unlock(&xps_map_mutex); + return ret; +} EXPORT_SYMBOL(netif_set_xps_queue); +int netif_set_xps(struct net_device *dev) +{ + cpumask_var_t queuemask; + int cpu, queue, err = 0; + + if (!alloc_cpumask_var(&queuemask, GFP_KERNEL)) + return -ENOMEM; + + mutex_lock(&xps_map_mutex); + for (queue = 0; queue < dev->real_num_tx_queues; ++queue) { + cpumask_clear(queuemask); + for (cpu = queue; cpu < nr_cpu_ids; + cpu += dev->real_num_tx_queues) + cpumask_set_cpu(cpu, queuemask); + + err = __netif_set_xps_queue(dev, queuemask, queue); + if (err) + goto out; + } + +out: + mutex_unlock(&xps_map_mutex); + + free_cpumask_var(queuemask); + return err; +} +EXPORT_SYMBOL(netif_set_xps); + #endif void netdev_reset_tc(struct net_device *dev) { From patchwork Thu Mar 15 15:08:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Abeni X-Patchwork-Id: 886284 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 402BnZ3DbCz9sVj for ; Fri, 16 Mar 2018 02:09:06 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E39E48A1DA; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S1IAXfDm1jD5; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 3F3EA8A1ED; Thu, 15 Mar 2018 15:09:04 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 972E71C0A7F for ; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 941CA8A1EB for ; Thu, 15 Mar 2018 15:09:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gPRdVQVjyX0n for ; Thu, 15 Mar 2018 15:09:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by hemlock.osuosl.org (Postfix) with ESMTPS id B88DC8A1E6 for ; Thu, 15 Mar 2018 15:09:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B85FC4068023; Thu, 15 Mar 2018 15:09:00 +0000 (UTC) Received: from localhost.localdomain.com (unknown [10.32.181.48]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0258C1FB2; Thu, 15 Mar 2018 15:08:59 +0000 (UTC) From: Paolo Abeni To: netdev@vger.kernel.org Date: Thu, 15 Mar 2018 16:08:12 +0100 Message-Id: <384ee099d617f3d3786a618b11cc10616923ec45.1521124830.git.pabeni@redhat.com> In-Reply-To: References: X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Mar 2018 15:09:00 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 15 Mar 2018 15:09:00 +0000 (UTC) for IP:'10.11.54.5' DOMAIN:'int-mx05.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'pabeni@redhat.com' RCPT:'' Subject: [Intel-wired-lan] [RFC PATCH 2/2] ixgbe: setup XPS via netif_set_xps() X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.24 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eric Dumazet , intel-wired-lan@lists.osuosl.org, "David S. Miller" MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" Before this commit, ixgbe with the default setting lacks XPS mapping for CPUs id greater than the number of tx queues. As a consequence the xmit path for such CPUs experience a relevant cost in __netdev_pick_tx, mainly due to skb_tx_hash(), as reported by the perf tool: 7.55%--netdev_pick_tx | --6.92%--__netdev_pick_tx | --6.35%--__skb_tx_hash | --5.94%--__skb_get_hash | --3.22%--__skb_flow_dissect in the following scenario: ethtool -L em1 combined 1 taskset 2 netperf -H 192.168.1.1 -t UDP_STREAM -- -m 1 MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.101.1 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 1 10.00 11497225 0 9.20 After this commit the perf tool reports: 0.85%--__netdev_pick_tx and netperf reports: MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.101.1 () port 0 AF_INET Socket Message Elapsed Messages Size Size Time Okay Errors Throughput bytes bytes secs # # 10^6bits/sec 212992 1 10.00 12736058 0 10.19 roughly +10% in xmit tput. Signed-off-by: Paolo Abeni --- drivers/net/ethernet/intel/ixgbe/ixgbe.h | 2 +- drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c | 1 + drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 13 +++---------- 3 files changed, 5 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h index c1e3a0039ea5..04aaecce81d2 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h @@ -293,7 +293,6 @@ enum ixgbe_ring_state_t { __IXGBE_RX_CSUM_UDP_ZERO_ERR, __IXGBE_RX_FCOE, __IXGBE_TX_FDIR_INIT_DONE, - __IXGBE_TX_XPS_INIT_DONE, __IXGBE_TX_DETECT_HANG, __IXGBE_HANG_CHECK_ARMED, __IXGBE_TX_XDP_RING, @@ -827,6 +826,7 @@ enum ixgbe_state_t { __IXGBE_PTP_RUNNING, __IXGBE_PTP_TX_IN_PROGRESS, __IXGBE_RESET_REQUESTED, + __IXGBE_TX_XPS_INIT_DONE, }; struct ixgbe_cb { diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c index c0e6ab42e0e1..da4e6416e8eb 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c @@ -3224,6 +3224,7 @@ static int ixgbe_set_channels(struct net_device *dev, #endif /* use setup TC to update any traffic class queue mapping */ + clear_bit(__IXGBE_TX_XPS_INIT_DONE, &adapter->state); return ixgbe_setup_tc(dev, adapter->hw_tcs); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 85369423452d..5bd45fc737fa 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -3566,16 +3566,6 @@ void ixgbe_configure_tx_ring(struct ixgbe_adapter *adapter, ring->atr_sample_rate = 0; } - /* initialize XPS */ - if (!test_and_set_bit(__IXGBE_TX_XPS_INIT_DONE, &ring->state)) { - struct ixgbe_q_vector *q_vector = ring->q_vector; - - if (q_vector) - netif_set_xps_queue(ring->netdev, - &q_vector->affinity_mask, - ring->queue_index); - } - clear_bit(__IXGBE_HANG_CHECK_ARMED, &ring->state); /* reinitialize tx_buffer_info */ @@ -5626,6 +5616,9 @@ static void ixgbe_up_complete(struct ixgbe_adapter *adapter) int err; u32 ctrl_ext; + if (!test_and_set_bit(__IXGBE_TX_XPS_INIT_DONE, &adapter->state)) + netif_set_xps(adapter->netdev); + ixgbe_get_hw_control(adapter); ixgbe_setup_gpie(adapter);