From patchwork Thu Mar 2 00:55:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 734427 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vYYl50k2dz9s7q for ; Thu, 2 Mar 2017 11:55:29 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="dKDvRXa1"; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id A7CC08A2EF; Thu, 2 Mar 2017 00:55:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q1VSW0qFAoGU; Thu, 2 Mar 2017 00:55:27 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 041608A2E3; Thu, 2 Mar 2017 00:55:27 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id BF6561C03F0 for ; Thu, 2 Mar 2017 00:55:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id BA846882F8 for ; Thu, 2 Mar 2017 00:55:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JqjTG21fZSrY for ; Thu, 2 Mar 2017 00:55:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f194.google.com (mail-pf0-f194.google.com [209.85.192.194]) by fraxinus.osuosl.org (Postfix) with ESMTPS id DFE9B882DF for ; Thu, 2 Mar 2017 00:55:24 +0000 (UTC) Received: by mail-pf0-f194.google.com with SMTP id b5so2592306pfa.0 for ; Wed, 01 Mar 2017 16:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:subject:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=E3LxScfuHZoJconm5ODfGuhTIdsLoIlGrgiIZIgXMY4=; b=dKDvRXa1F9bt/yGk8LeCDNupXZZ1jmgLMEC/5JLsZfNEGOctsWZpGP3Y1G374dLteQ 9OmRSp8wh1mebeTpHCgFrxjdG6bYLQQUPJPy/e4+mCAIDBMvNRHj4WLeHFcibd6UDNk+ odBsJRTohQh47mo1flDvGMDXQqhMuvUBmQdImkczIqDmBc80S1j2Y4Se4PJEVteQMbv0 jVME9RobmwbKjU0RgSHi+mNIxzY/gVShAXCoK9oBEk1SohoLJNpXIgCZdJBl0hU2Xl2E xR/OjhssaJsc1RTnpDgdY99xX99QHlYGNHXUo0vHxJmK3+aG3K66fc0VR0nbo2sLWEwW 4FTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=E3LxScfuHZoJconm5ODfGuhTIdsLoIlGrgiIZIgXMY4=; b=sHUG5sqJd0nVEDVuhDJBt9SOkWU3ATFD/3jC2VBOCFMNPiPYb+h/JmMHva4Dy0VFlF FhSmyZ05gA/VARlmATVrOJ/GezD3s9rx4XKIiyWg2sJtezbznYHm08CJwK4faEywldcd TmioKa9QwrLxlBco/AWkoxZav2+168ceB7mjT7Q+VWVts859qSatvN04EQ7Y+quBTy9H zFBDQtDnxAsRXtuAQuFOylRbNoo1EQgzWq09ZVjKfTFfmPaZHEIuoAO5tGPv70ajXVSk wpvDrYIDMlyKIFTPZkP02hMQaGDoaXrsFOvabW/Wzwl+o7AFqyhkG/enDqrsyy5/EINe r0hw== X-Gm-Message-State: AMke39nO5dbHk/oibeu8+YlQw2OLBJ76vQz8KUd9u14VeRnwK5HKCYwuDwNufUXK6eIlow== X-Received: by 10.99.125.23 with SMTP id y23mr5946611pgc.79.1488416124514; Wed, 01 Mar 2017 16:55:24 -0800 (PST) Received: from [127.0.1.1] ([72.168.144.124]) by smtp.gmail.com with ESMTPSA id 129sm12839099pgj.59.2017.03.01.16.55.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Mar 2017 16:55:23 -0800 (PST) From: John Fastabend X-Google-Original-From: John Fastabend To: alexander.duyck@gmail.com Date: Wed, 01 Mar 2017 16:55:00 -0800 Message-ID: <20170302005500.30505.88418.stgit@john-Precision-Tower-5810> In-Reply-To: <20170302005417.30505.14047.stgit@john-Precision-Tower-5810> References: <20170302005417.30505.14047.stgit@john-Precision-Tower-5810> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Cc: daniel@iogearbox.net, intel-wired-lan@lists.osuosl.org, bjorn.topel@intel.com, alexei.starovoitov@gmail.com, magnus.karlsson@intel.com Subject: [Intel-wired-lan] [net-next PATCH v2 3/3] ixgbe: xdp support for adjust head X-BeenThere: intel-wired-lan@lists.osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-wired-lan-bounces@lists.osuosl.org Sender: "Intel-wired-lan" Add adjust_head support for XDP however at the moment we are only adding IXGBE_SKB_PAD bytes of headroom to align with driver paths. The infrastructure is is such that a follow on patch can extend headroom up to 196B without changing RX path. Signed-off-by: John Fastabend --- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 38 +++++++++++++++++-------- 1 file changed, 26 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index e754fe0..c8bf64e 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2026,6 +2026,7 @@ static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer) static void ixgbe_add_rx_frag(struct ixgbe_ring *rx_ring, struct ixgbe_rx_buffer *rx_buffer, struct sk_buff *skb, + unsigned int headroom, unsigned int size) { #if (PAGE_SIZE < 8192) @@ -2036,7 +2037,8 @@ static void ixgbe_add_rx_frag(struct ixgbe_ring *rx_ring, SKB_DATA_ALIGN(size); #endif skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_buffer->page, - rx_buffer->page_offset, size, truesize); + rx_buffer->page_offset - (IXGBE_SKB_PAD - headroom), + size, truesize); #if (PAGE_SIZE < 8192) rx_buffer->page_offset ^= truesize; #else @@ -2109,6 +2111,7 @@ static void ixgbe_put_rx_buffer(struct ixgbe_ring *rx_ring, static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, struct ixgbe_rx_buffer *rx_buffer, union ixgbe_adv_rx_desc *rx_desc, + unsigned int headroom, unsigned int size) { void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; @@ -2117,6 +2120,7 @@ static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, #else unsigned int truesize = SKB_DATA_ALIGN(size); #endif + unsigned int off_page; struct sk_buff *skb; /* prefetch first cache line of first page */ @@ -2130,12 +2134,14 @@ static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, if (unlikely(!skb)) return NULL; + off_page = IXGBE_SKB_PAD - headroom; + if (size > IXGBE_RX_HDR_SIZE) { if (!ixgbe_test_staterr(rx_desc, IXGBE_RXD_STAT_EOP)) IXGBE_CB(skb)->dma = rx_buffer->dma; skb_add_rx_frag(skb, 0, rx_buffer->page, - rx_buffer->page_offset, + rx_buffer->page_offset - off_page, size, truesize); #if (PAGE_SIZE < 8192) rx_buffer->page_offset ^= truesize; @@ -2143,7 +2149,8 @@ static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, rx_buffer->page_offset += truesize; #endif } else { - memcpy(__skb_put(skb, size), va, ALIGN(size, sizeof(long))); + memcpy(__skb_put(skb, size), va - off_page, + ALIGN(size, sizeof(long))); rx_buffer->pagecnt_bias++; } @@ -2153,6 +2160,7 @@ static struct sk_buff *ixgbe_construct_skb(struct ixgbe_ring *rx_ring, static struct sk_buff *ixgbe_build_skb(struct ixgbe_ring *rx_ring, struct ixgbe_rx_buffer *rx_buffer, union ixgbe_adv_rx_desc *rx_desc, + unsigned int headroom, unsigned int size) { void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; @@ -2176,7 +2184,7 @@ static struct sk_buff *ixgbe_build_skb(struct ixgbe_ring *rx_ring, return NULL; /* update pointers within the skb to store the data */ - skb_reserve(skb, IXGBE_SKB_PAD); + skb_reserve(skb, headroom); __skb_put(skb, size); /* record DMA address if this is the start of a chain of buffers */ @@ -2203,7 +2211,8 @@ static int ixgbe_xmit_xdp_ring(struct ixgbe_adapter *adapter, static int ixgbe_run_xdp(struct ixgbe_adapter *adapter, struct ixgbe_ring *rx_ring, struct ixgbe_rx_buffer *rx_buffer, - unsigned int size) + unsigned int *headroom, + unsigned int *size) { int result = IXGBE_XDP_PASS; struct bpf_prog *xdp_prog; @@ -2218,14 +2227,16 @@ static int ixgbe_run_xdp(struct ixgbe_adapter *adapter, goto xdp_out; addr = page_address(rx_buffer->page) + rx_buffer->page_offset; - xdp.data_hard_start = addr; + xdp.data_hard_start = addr - *headroom; xdp.data = addr; - xdp.data_end = addr + size; + xdp.data_end = addr + *size; act = bpf_prog_run_xdp(xdp_prog, &xdp); switch (act) { case XDP_PASS: - break; + *headroom = xdp.data - xdp.data_hard_start; + *size = xdp.data_end - xdp.data; + return IXGBE_XDP_PASS; case XDP_TX: result = ixgbe_xmit_xdp_ring(adapter, &xdp); if (result == IXGBE_XDP_TX) @@ -2274,6 +2285,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, u16 cleaned_count = ixgbe_desc_unused(rx_ring); while (likely(total_rx_packets < budget)) { + unsigned int headroom = ixgbe_rx_offset(rx_ring); union ixgbe_adv_rx_desc *rx_desc; struct ixgbe_rx_buffer *rx_buffer; struct sk_buff *skb; @@ -2299,7 +2311,8 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, rx_buffer = ixgbe_get_rx_buffer(rx_ring, rx_desc, &skb, size); - consumed = ixgbe_run_xdp(adapter, rx_ring, rx_buffer, size); + consumed = ixgbe_run_xdp(adapter, rx_ring, rx_buffer, + &headroom, &size); if (consumed) { ixgbe_put_rx_buffer(rx_ring, rx_buffer, skb); @@ -2312,13 +2325,14 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, /* retrieve a buffer from the ring */ if (skb) - ixgbe_add_rx_frag(rx_ring, rx_buffer, skb, size); + ixgbe_add_rx_frag(rx_ring, rx_buffer, skb, + headroom, size); else if (ring_uses_build_skb(rx_ring)) skb = ixgbe_build_skb(rx_ring, rx_buffer, - rx_desc, size); + rx_desc, headroom, size); else skb = ixgbe_construct_skb(rx_ring, rx_buffer, - rx_desc, size); + rx_desc, headroom, size); /* exit if we failed to retrieve a buffer */ if (!skb) {