From patchwork Fri Jul 8 02:15:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brenden Blanco X-Patchwork-Id: 646277 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3rlyll0SyWz9sBg for ; Fri, 8 Jul 2016 12:16:19 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=plumgrid-com.20150623.gappssmtp.com header.i=@plumgrid-com.20150623.gappssmtp.com header.b=NIJETW7Y; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754026AbcGHCQO (ORCPT ); Thu, 7 Jul 2016 22:16:14 -0400 Received: from mail-pf0-f171.google.com ([209.85.192.171]:35764 "EHLO mail-pf0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753993AbcGHCPy (ORCPT ); Thu, 7 Jul 2016 22:15:54 -0400 Received: by mail-pf0-f171.google.com with SMTP id c2so11321981pfa.2 for ; Thu, 07 Jul 2016 19:15:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plumgrid-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=R8hNSl+flaDKwdxEPHdy8b09Lp6vi0YKyOyqczLZX10=; b=NIJETW7YaoWeG9t7PIVkSG94GteSUmxX/7ZKx+X+FUCQyymRuMAfIye4plpPGtKwKr 8tpTrm9oM4PKeIf/oLi3JlfStp4NO56hiB0yzbykslng9HgRe6HF/1MjQKPbhtTuxETa H9w9lkDW2r3c62gLKsXONSwp7gWRakCeI4wrMyEooXbid0VNoEZqMPhv9F3KjC2uH8aI XHGLukDtLci5Ny3jTM8abCfSv5V7wBd0qf8NSD/KwR9ucDTmrxGqK07q1MMh02iCuunF ayJhpp6GsAzmg5ZefGSx+0qJ7LwOYBca2uilTUeKtrEttxSQ7CJPf9+SxqXe4AbZB0gj Y/jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=R8hNSl+flaDKwdxEPHdy8b09Lp6vi0YKyOyqczLZX10=; b=XX2fkVUsTqq0bEwaAOXaom6EPrVv84cik3DVqGEiRJrcTq39qCPkVG7GDaSH1uZiNi hMZw6aRwcs8+ma8Wo9zTkGRfn4HJxKNljghusq/ij1bN3Vgzz6YV7ITCwJRlrQ4sjo9g q1u9Cq3kH5gFHfH0fDuasuVVNE7jHRV6p/lQe8n9m2eQ0cZN92FbffdzCI88r2uqf1iO t99Vv9vVDW1AnMDvhGraCaWGnwvp82xYf7xDl0j70f75C31ljd/5HU0MTx41ZlR9fLbl TKapcRxunYdbQiiDVvf/9Qf9NKgDadhFTcgpJeGsTlYc7/puVCCBe4Axdu3dSwuBOpWW 5DcQ== X-Gm-Message-State: ALyK8tLBGyZKeJYsb5uBteA2h5JeD9S0o2QhQW5DfgS2nLYf6L1ccdjJLsqh4kPCzqhhkReg X-Received: by 10.98.54.198 with SMTP id d189mr5660207pfa.39.1467944148746; Thu, 07 Jul 2016 19:15:48 -0700 (PDT) Received: from iovisor-test1.plumgrid.com ([12.97.19.201]) by smtp.gmail.com with ESMTPSA id jh3sm401497pac.14.2016.07.07.19.15.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 07 Jul 2016 19:15:48 -0700 (PDT) From: Brenden Blanco To: davem@davemloft.net, netdev@vger.kernel.org Cc: Brenden Blanco , Martin KaFai Lau , Jesper Dangaard Brouer , Ari Saha , Alexei Starovoitov , Or Gerlitz , john.fastabend@gmail.com, hannes@stressinduktion.org, Thomas Graf , Tom Herbert , Daniel Borkmann Subject: [PATCH v6 12/12] net/mlx4_en: add prefetch in xdp rx path Date: Thu, 7 Jul 2016 19:15:24 -0700 Message-Id: <1467944124-14891-13-git-send-email-bblanco@plumgrid.com> X-Mailer: git-send-email 2.8.2 In-Reply-To: <1467944124-14891-1-git-send-email-bblanco@plumgrid.com> References: <1467944124-14891-1-git-send-email-bblanco@plumgrid.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org XDP programs read and/or write packet data very early, and cache miss is seen to be a bottleneck. Add prefetch logic in the xdp case 3 packets in the future. Throughput improved from 10Mpps to 12.5Mpps. LLC misses as reported by perf stat reduced from ~14% to ~7%. Prefetch values of 0 through 5 were compared with >3 showing dimishing returns. Before: 21.94% ksoftirqd/0 [mlx4_en] [k] 0x000000000001d6e4 12.96% ksoftirqd/0 [mlx4_en] [k] mlx4_en_process_rx_cq 12.28% ksoftirqd/0 [mlx4_en] [k] mlx4_en_xmit_frame 11.93% ksoftirqd/0 [mlx4_en] [k] mlx4_en_poll_tx_cq 4.77% ksoftirqd/0 [mlx4_en] [k] mlx4_en_prepare_rx_desc 3.13% ksoftirqd/0 [mlx4_en] [k] mlx4_en_free_tx_desc.isra.30 2.68% ksoftirqd/0 [kernel.vmlinux] [k] bpf_map_lookup_elem 2.22% ksoftirqd/0 [kernel.vmlinux] [k] percpu_array_map_lookup_elem 2.02% ksoftirqd/0 [mlx4_core] [k] mlx4_eq_int 1.92% ksoftirqd/0 [mlx4_en] [k] mlx4_en_rx_recycle After: 20.70% ksoftirqd/0 [mlx4_en] [k] mlx4_en_xmit_frame 18.14% ksoftirqd/0 [mlx4_en] [k] mlx4_en_process_rx_cq 16.30% ksoftirqd/0 [mlx4_en] [k] mlx4_en_poll_tx_cq 6.49% ksoftirqd/0 [mlx4_en] [k] mlx4_en_prepare_rx_desc 4.06% ksoftirqd/0 [mlx4_en] [k] mlx4_en_free_tx_desc.isra.30 2.76% ksoftirqd/0 [mlx4_en] [k] mlx4_en_rx_recycle 2.37% ksoftirqd/0 [mlx4_core] [k] mlx4_eq_int 1.44% ksoftirqd/0 [kernel.vmlinux] [k] bpf_map_lookup_elem 1.43% swapper [kernel.vmlinux] [k] intel_idle 1.20% ksoftirqd/0 [kernel.vmlinux] [k] percpu_array_map_lookup_elem 1.19% ksoftirqd/0 [mlx4_core] [k] 0x0000000000049eb8 Signed-off-by: Brenden Blanco --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index 41c76fe..65e93f7 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -881,10 +881,17 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud * read bytes but not past the end of the frag. */ if (prog) { + struct mlx4_en_rx_alloc *pref; struct xdp_buff xdp; + int pref_index; dma_addr_t dma; u32 act; + pref_index = (index + 3) & ring->size_mask; + pref = ring->rx_info + + (pref_index << priv->log_rx_info); + prefetch(page_address(pref->page) + pref->page_offset); + dma = be64_to_cpu(rx_desc->data[0].addr); dma_sync_single_for_cpu(priv->ddev, dma, priv->frag_info[0].frag_size,