From patchwork Thu Aug 15 03:46:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Samudrala, Sridhar" X-Patchwork-Id: 1147391 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=osuosl.org (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=intel-wired-lan-bounces@osuosl.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 468C6y6tjhz9sNm for ; Thu, 15 Aug 2019 13:46:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 4751186938; Thu, 15 Aug 2019 03:46:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OLEQl-OXUN9R; Thu, 15 Aug 2019 03:46:31 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 9242586955; Thu, 15 Aug 2019 03:46:30 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 108261BF3BD for ; Thu, 15 Aug 2019 03:46:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id BD65287729 for ; Thu, 15 Aug 2019 03:46:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IJ2w9DFFHxLw for ; Thu, 15 Aug 2019 03:46:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by whitealder.osuosl.org (Postfix) with ESMTPS id D25A387726 for ; Thu, 15 Aug 2019 03:46:24 +0000 (UTC) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Aug 2019 20:46:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,387,1559545200"; d="scan'208";a="352124060" Received: from arch-p28.jf.intel.com ([10.166.187.31]) by orsmga005.jf.intel.com with ESMTP; 14 Aug 2019 20:46:23 -0700 From: Sridhar Samudrala To: magnus.karlsson@intel.com, bjorn.topel@intel.com, netdev@vger.kernel.org, bpf@vger.kernel.org, sridhar.samudrala@intel.com, intel-wired-lan@lists.osuosl.org, maciej.fijalkowski@intel.com, tom.herbert@intel.com Date: Wed, 14 Aug 2019 20:46:22 -0700 Message-Id: <1565840783-8269-5-git-send-email-sridhar.samudrala@intel.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1565840783-8269-1-git-send-email-sridhar.samudrala@intel.com> References: <1565840783-8269-1-git-send-email-sridhar.samudrala@intel.com> Subject: [Intel-wired-lan] [PATCH bpf-next 4/5] ixgbe: Enable XDP_SKIP_BPF option for AF_XDP sockets X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" This patch skips calling BPF program in the receive path if the queue is associated with UMEM that is not shared and bound to an AF_XDP socket that has enabled skip bpf during bind() call. Signed-off-by: Sridhar Samudrala --- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 20 +++++++++++++++++-- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 16 +++++++++++++-- 2 files changed, 32 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index dc7b128c780e..594792860cdd 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2197,6 +2197,7 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, int err, result = IXGBE_XDP_PASS; struct bpf_prog *xdp_prog; struct xdp_frame *xdpf; + struct xdp_umem *umem; u32 act; rcu_read_lock(); @@ -2207,6 +2208,13 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, prefetchw(xdp->data_hard_start); /* xdp_frame write */ + umem = xdp_get_umem_from_qid(rx_ring->netdev, rx_ring->queue_index); + if (xsk_umem_skip_bpf(umem)) { + err = xsk_umem_rcv(umem, xdp); + result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; + goto xdp_out; + } + act = bpf_prog_run_xdp(xdp_prog, xdp); switch (act) { case XDP_PASS: @@ -2400,8 +2408,16 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, total_rx_packets++; } - if (xdp_xmit & IXGBE_XDP_REDIR) - xdp_do_flush_map(); + if (xdp_xmit & IXGBE_XDP_REDIR) { + struct xdp_umem *umem; + + umem = xdp_get_umem_from_qid(rx_ring->netdev, + rx_ring->queue_index); + if (xsk_umem_skip_bpf(umem)) + xsk_umem_flush(umem); + else + xdp_do_flush_map(); + } if (xdp_xmit & IXGBE_XDP_TX) { struct ixgbe_ring *ring = adapter->xdp_ring[smp_processor_id()]; diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index 6b609553329f..9ea8a769d7a8 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -148,6 +148,12 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; + if (xsk_umem_skip_bpf(rx_ring->xsk_umem)) { + err = xsk_umem_rcv(rx_ring->xsk_umem, xdp); + result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; + return result; + } + rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); act = bpf_prog_run_xdp(xdp_prog, xdp); @@ -527,8 +533,14 @@ int ixgbe_clean_rx_irq_zc(struct ixgbe_q_vector *q_vector, ixgbe_rx_skb(q_vector, skb); } - if (xdp_xmit & IXGBE_XDP_REDIR) - xdp_do_flush_map(); + if (xdp_xmit & IXGBE_XDP_REDIR) { + struct xdp_umem *umem = rx_ring->xsk_umem; + + if (xsk_umem_skip_bpf(umem)) + xsk_umem_flush(umem); + else + xdp_do_flush_map(); + } if (xdp_xmit & IXGBE_XDP_TX) { struct ixgbe_ring *ring = adapter->xdp_ring[smp_processor_id()];