From patchwork Tue Aug 25 11:35:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1351024 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=KxPN3LdU; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbRsZ4xPRz9sTN for ; Tue, 25 Aug 2020 21:41:38 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730089AbgHYLlI (ORCPT ); Tue, 25 Aug 2020 07:41:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729363AbgHYLga (ORCPT ); Tue, 25 Aug 2020 07:36:30 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51C47C061755; Tue, 25 Aug 2020 04:36:16 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id u20so7142236pfn.0; Tue, 25 Aug 2020 04:36:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MKByQ46pgwwug2FS04MHEvNcYmmcyxdFTwu8Aog6KoM=; b=KxPN3LdURiJQJJHI+7W95jH6ria7e92JnvDmYZ7sJfxiC4VpG4107WK+DCGB8aH2Yb W0rzC1IL3n7ra0sbNPycNgX0fPf3XmQZhGterI4gFWkEvPn0txoDpmCeuWo54CogFORG kwGLMlHDdpS2vJqtiCmIHqP8RqkCh1NB/Q7O4jxcpIzopQAw5x+Ohdjdnq1BuAajGHcg 3ao4QtB8ptWDYRQKqTao2pULQiiig48U0PaQ4s0jm3LcE4g8YYwRsaUGHBXMEtydSUTc 6AGWG9jx08aI2ffMNyDlVi0TjDz4/Hzx+VsP9hGI4m6pzfdN2My74O548l4jiuAMoQxt 6szA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MKByQ46pgwwug2FS04MHEvNcYmmcyxdFTwu8Aog6KoM=; b=onmViFSTUAF2/M5vWw+xBMLiXvA4jGuajCAhEYCt/DX7e+6SwsB3F8AB/lg1yS/S58 j+8Kmil6gjIMfBKgIREJNItWf/gTpp8ArmZurbXsOPxuZrjjQAKSlrOosTJvk+0ZEtLy wwel3nccUA6mJ7u9795/sXsLu8YowpuksexL9UImgheb/s5LfxH3niwoWBoeSHUX9vGm qxivXPsxgPRwnbNQRVWtHYHSgQugiNf0wrmS9JnhdUK6plXVYTr3b6V8fn9OsQU1UjBM 9r4zHui/N8YYvOonQaHIWrqI6wEVIZI9ibPFp/F8J31p3Es+PLJot7dJZ7l8WieKTX+1 9jWQ== X-Gm-Message-State: AOAM530QXGrtRymxZkBPEiYPfG+x/GOX0hQ7Bvs+ApHrWCXo48GhSbka E7/HqpvlcteIEd3ycfO+Jz0= X-Google-Smtp-Source: ABdhPJy089I37yUmgoghRv6ll7QM6Tz7TRYt0VT7Dn/UuGtC67TNorNbHf/g6lUG6Bo5pMugRHcQkQ== X-Received: by 2002:a17:902:780f:: with SMTP id p15mr7737669pll.56.1598355375887; Tue, 25 Aug 2020 04:36:15 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id e7sm12699937pgn.64.2020.08.25.04.36.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 04:36:15 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, netdev@vger.kernel.org, bpf@vger.kernel.org, kuba@kernel.org Subject: [PATCH net-next v2 1/3] i40e, xsk: remove HW descriptor prefetch in AF_XDP path Date: Tue, 25 Aug 2020 13:35:54 +0200 Message-Id: <20200825113556.18342-2-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825113556.18342-1-bjorn.topel@gmail.com> References: <20200825113556.18342-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Björn Töpel The software prefetching of HW descriptors has a negative impact on the performance. Therefore, it is now removed. Performance for the rx_drop benchmark increased with 2%. Signed-off-by: Björn Töpel Tested-by: Aaron Brown --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 13 +++++++++++++ drivers/net/ethernet/intel/i40e/i40e_txrx_common.h | 13 ------------- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 12 ++++++++++++ 3 files changed, 25 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index 3e5c566ceb01..e1a76fc05b8d 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -2299,6 +2299,19 @@ void i40e_finalize_xdp_rx(struct i40e_ring *rx_ring, unsigned int xdp_res) } } +/** + * i40e_inc_ntc: Advance the next_to_clean index + * @rx_ring: Rx ring + **/ +static void i40e_inc_ntc(struct i40e_ring *rx_ring) +{ + u32 ntc = rx_ring->next_to_clean + 1; + + ntc = (ntc < rx_ring->count) ? ntc : 0; + rx_ring->next_to_clean = ntc; + prefetch(I40E_RX_DESC(rx_ring, ntc)); +} + /** * i40e_clean_rx_irq - Clean completed descriptors from Rx ring - bounce buf * @rx_ring: rx descriptor ring to transact packets on diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h b/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h index 667c4dc4b39f..1397dd3c1c57 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx_common.h @@ -99,19 +99,6 @@ static inline bool i40e_rx_is_programming_status(u64 qword1) return qword1 & I40E_RXD_QW1_LENGTH_SPH_MASK; } -/** - * i40e_inc_ntc: Advance the next_to_clean index - * @rx_ring: Rx ring - **/ -static inline void i40e_inc_ntc(struct i40e_ring *rx_ring) -{ - u32 ntc = rx_ring->next_to_clean + 1; - - ntc = (ntc < rx_ring->count) ? ntc : 0; - rx_ring->next_to_clean = ntc; - prefetch(I40E_RX_DESC(rx_ring, ntc)); -} - void i40e_xsk_clean_rx_ring(struct i40e_ring *rx_ring); void i40e_xsk_clean_tx_ring(struct i40e_ring *tx_ring); bool i40e_xsk_any_rx_ring_enabled(struct i40e_vsi *vsi); diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 8ce57b507a21..1f2dd591dbf1 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -253,6 +253,18 @@ static struct sk_buff *i40e_construct_skb_zc(struct i40e_ring *rx_ring, return skb; } +/** + * i40e_inc_ntc: Advance the next_to_clean index + * @rx_ring: Rx ring + **/ +static void i40e_inc_ntc(struct i40e_ring *rx_ring) +{ + u32 ntc = rx_ring->next_to_clean + 1; + + ntc = (ntc < rx_ring->count) ? ntc : 0; + rx_ring->next_to_clean = ntc; +} + /** * i40e_clean_rx_irq_zc - Consumes Rx packets from the hardware ring * @rx_ring: Rx ring From patchwork Tue Aug 25 11:35:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1351021 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=CgrO8+VW; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbRrq6vvSz9sTN for ; Tue, 25 Aug 2020 21:40:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730032AbgHYLk4 (ORCPT ); Tue, 25 Aug 2020 07:40:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730022AbgHYLga (ORCPT ); Tue, 25 Aug 2020 07:36:30 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A2DBC061757; Tue, 25 Aug 2020 04:36:19 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id ep8so1098481pjb.3; Tue, 25 Aug 2020 04:36:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rhY88JdRNjLHY9J/WjwwpO1RSozTLyNiKj+QUN0VOyY=; b=CgrO8+VWYvlAsqmozCU+JIoxKWxkdIiuXyG7ARljMwhuDvYjcCx/fHEe6M5j5Q/0TQ /m9T9bSr38rkVrwb9jaPpEy1+8/i1dRVkjZzfuFBnHcARFEhVJhWQO9Oax9KuzvVOqhH jl1LLPp/KRDK0kYWLxwUnEqS2ghXAqG2ii/8QTKOXiYHHu+zpPqnR8//B9DXbtyAaEiZ mgr5JfoMdXFZqCDBxfovvactrjG+/zLBAZWjWO9A9/ie39W4T4HoRU1cjbQwI3eyGhnM Pax+nuL93mrASW6KqwSTBciGVa3MCML/euErqtyHkdOwCsDfNcQ00KDWenMRX1mH2QJ4 OhtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rhY88JdRNjLHY9J/WjwwpO1RSozTLyNiKj+QUN0VOyY=; b=BX46+GaynbOJX05jVJM5Ma9j2Y6sUlTUaGHeomoqAA7QFSayERkL4P6fC1c3DfaNxY ILW0Cs+92tr16o0IE0geVJh7W2N34MeoZ6vCgi8SntawpyuEkD2X6oqVsR1EuO8USkqe CHQTw102wAeCEKaZbwsL5qDeqTJY5YokCeGmIv+EksfcJo7LjktfTRliSJ/bbKFd2C6m ml+EclRLUWr2dfL+q1RodgZRYlgYtdmaM63LRBk1QjU4s6OygnabPMviAay2HE0dGqv4 GiWyNTe+oT0IuUkTOpmnIMYg6LZMWJcVtM5wNf5XhqFyxM2Twlq6eYkqqh8ZZFfC/mVt bVjw== X-Gm-Message-State: AOAM531x5zP9FcPSe9EeTo/B0vG1PgU5MK019acoh/jq59JaPvPZ0MXx /Bk8p63wr0WXkZIU6kWvCVY= X-Google-Smtp-Source: ABdhPJyCC+GCfoM0vWSgxtdii4EKkhlDYmcWbY8C2oOAZu3rGXIWPvsLl8/0umYTv10ZskgrJF9XwA== X-Received: by 2002:a17:90a:c781:: with SMTP id gn1mr1217955pjb.151.1598355378935; Tue, 25 Aug 2020 04:36:18 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id e7sm12699937pgn.64.2020.08.25.04.36.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 04:36:18 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, netdev@vger.kernel.org, bpf@vger.kernel.org, kuba@kernel.org Subject: [PATCH net-next v2 2/3] i40e: use 16B HW descriptors instead of 32B Date: Tue, 25 Aug 2020 13:35:55 +0200 Message-Id: <20200825113556.18342-3-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825113556.18342-1-bjorn.topel@gmail.com> References: <20200825113556.18342-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: Björn Töpel The i40e NIC supports two flavors of HW descriptors, 16 and 32 byte. The latter has, obviously, room for more offloading information. However, the only fields of the 32B HW descriptor that is being used by the driver, is also available in the 16B descriptor. In other words; Reading and writing 32 bytes instead of 16 byte is a waste of bus bandwidth. This commit starts using 16 byte descriptors instead of 32 byte descriptors. For AF_XDP the rx_drop benchmark was improved by 2%. Signed-off-by: Björn Töpel Tested-by: Aaron Brown --- drivers/net/ethernet/intel/i40e/i40e.h | 2 +- drivers/net/ethernet/intel/i40e/i40e_debugfs.c | 10 ++++------ drivers/net/ethernet/intel/i40e/i40e_main.c | 4 ++-- drivers/net/ethernet/intel/i40e/i40e_trace.h | 6 +++--- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 6 +++--- drivers/net/ethernet/intel/i40e/i40e_txrx.h | 2 +- drivers/net/ethernet/intel/i40e/i40e_type.h | 5 ++++- 7 files changed, 18 insertions(+), 17 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e.h b/drivers/net/ethernet/intel/i40e/i40e.h index a7e212d1caa2..ada0e93c38f0 100644 --- a/drivers/net/ethernet/intel/i40e/i40e.h +++ b/drivers/net/ethernet/intel/i40e/i40e.h @@ -90,7 +90,7 @@ #define I40E_OEM_RELEASE_MASK 0x0000ffff #define I40E_RX_DESC(R, i) \ - (&(((union i40e_32byte_rx_desc *)((R)->desc))[i])) + (&(((union i40e_rx_desc *)((R)->desc))[i])) #define I40E_TX_DESC(R, i) \ (&(((struct i40e_tx_desc *)((R)->desc))[i])) #define I40E_TX_CTXTDESC(R, i) \ diff --git a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c index d3ad2e3aa838..d7c13ca9be7d 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_debugfs.c +++ b/drivers/net/ethernet/intel/i40e/i40e_debugfs.c @@ -604,10 +604,9 @@ static void i40e_dbg_dump_desc(int cnt, int vsi_seid, int ring_id, int desc_n, } else { rxd = I40E_RX_DESC(ring, i); dev_info(&pf->pdev->dev, - " d[%03x] = 0x%016llx 0x%016llx 0x%016llx 0x%016llx\n", + " d[%03x] = 0x%016llx 0x%016llx\n", i, rxd->read.pkt_addr, - rxd->read.hdr_addr, - rxd->read.rsvd1, rxd->read.rsvd2); + rxd->read.hdr_addr); } } } else if (cnt == 3) { @@ -625,10 +624,9 @@ static void i40e_dbg_dump_desc(int cnt, int vsi_seid, int ring_id, int desc_n, } else { rxd = I40E_RX_DESC(ring, desc_n); dev_info(&pf->pdev->dev, - "vsi = %02i rx ring = %02i d[%03x] = 0x%016llx 0x%016llx 0x%016llx 0x%016llx\n", + "vsi = %02i rx ring = %02i d[%03x] = 0x%016llx 0x%016llx\n", vsi_seid, ring_id, desc_n, - rxd->read.pkt_addr, rxd->read.hdr_addr, - rxd->read.rsvd1, rxd->read.rsvd2); + rxd->read.pkt_addr, rxd->read.hdr_addr); } } else { dev_info(&pf->pdev->dev, "dump desc rx/tx/xdp []\n"); diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 2e433fdbf2c3..e9516dea6d32 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -3320,8 +3320,8 @@ static int i40e_configure_rx_ring(struct i40e_ring *ring) rx_ctx.base = (ring->dma / 128); rx_ctx.qlen = ring->count; - /* use 32 byte descriptors */ - rx_ctx.dsize = 1; + /* use 16 byte descriptors */ + rx_ctx.dsize = 0; /* descriptor type is always zero * rx_ctx.dtype = 0; diff --git a/drivers/net/ethernet/intel/i40e/i40e_trace.h b/drivers/net/ethernet/intel/i40e/i40e_trace.h index 424f02077e2e..983f8b98b275 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_trace.h +++ b/drivers/net/ethernet/intel/i40e/i40e_trace.h @@ -112,7 +112,7 @@ DECLARE_EVENT_CLASS( i40e_rx_template, TP_PROTO(struct i40e_ring *ring, - union i40e_32byte_rx_desc *desc, + union i40e_16byte_rx_desc *desc, struct sk_buff *skb), TP_ARGS(ring, desc, skb), @@ -140,7 +140,7 @@ DECLARE_EVENT_CLASS( DEFINE_EVENT( i40e_rx_template, i40e_clean_rx_irq, TP_PROTO(struct i40e_ring *ring, - union i40e_32byte_rx_desc *desc, + union i40e_16byte_rx_desc *desc, struct sk_buff *skb), TP_ARGS(ring, desc, skb)); @@ -148,7 +148,7 @@ DEFINE_EVENT( DEFINE_EVENT( i40e_rx_template, i40e_clean_rx_irq_rx, TP_PROTO(struct i40e_ring *ring, - union i40e_32byte_rx_desc *desc, + union i40e_16byte_rx_desc *desc, struct sk_buff *skb), TP_ARGS(ring, desc, skb)); diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index e1a76fc05b8d..0db656ef5b9a 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -533,11 +533,11 @@ static void i40e_fd_handle_status(struct i40e_ring *rx_ring, u64 qword0_raw, { struct i40e_pf *pf = rx_ring->vsi->back; struct pci_dev *pdev = pf->pdev; - struct i40e_32b_rx_wb_qw0 *qw0; + struct i40e_16b_rx_wb_qw0 *qw0; u32 fcnt_prog, fcnt_avail; u32 error; - qw0 = (struct i40e_32b_rx_wb_qw0 *)&qword0_raw; + qw0 = (struct i40e_16b_rx_wb_qw0 *)&qword0_raw; error = (qword1 & I40E_RX_PROG_STATUS_DESC_QW1_ERROR_MASK) >> I40E_RX_PROG_STATUS_DESC_QW1_ERROR_SHIFT; @@ -1418,7 +1418,7 @@ int i40e_setup_rx_descriptors(struct i40e_ring *rx_ring) u64_stats_init(&rx_ring->syncp); /* Round up to nearest 4K */ - rx_ring->size = rx_ring->count * sizeof(union i40e_32byte_rx_desc); + rx_ring->size = rx_ring->count * sizeof(union i40e_rx_desc); rx_ring->size = ALIGN(rx_ring->size, 4096); rx_ring->desc = dma_alloc_coherent(dev, rx_ring->size, &rx_ring->dma, GFP_KERNEL); diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.h b/drivers/net/ethernet/intel/i40e/i40e_txrx.h index 4036893d6825..0eacd5f21e9d 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.h +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.h @@ -110,7 +110,7 @@ enum i40e_dyn_idx_t { */ #define I40E_RX_HDR_SIZE I40E_RXBUFFER_256 #define I40E_PACKET_HDR_PAD (ETH_HLEN + ETH_FCS_LEN + (VLAN_HLEN * 2)) -#define i40e_rx_desc i40e_32byte_rx_desc +#define i40e_rx_desc i40e_16byte_rx_desc #define I40E_RX_DMA_ATTR \ (DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_WEAK_ORDERING) diff --git a/drivers/net/ethernet/intel/i40e/i40e_type.h b/drivers/net/ethernet/intel/i40e/i40e_type.h index 52410d609ba1..97d29df65f9e 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_type.h +++ b/drivers/net/ethernet/intel/i40e/i40e_type.h @@ -628,7 +628,7 @@ union i40e_16byte_rx_desc { __le64 hdr_addr; /* Header buffer address */ } read; struct { - struct { + struct i40e_16b_rx_wb_qw0 { struct { union { __le16 mirroring_status; @@ -647,6 +647,9 @@ union i40e_16byte_rx_desc { __le64 status_error_len; } qword1; } wb; /* writeback */ + struct { + u64 qword[2]; + } raw; }; union i40e_32byte_rx_desc { From patchwork Tue Aug 25 11:35:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1351020 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=kJ/OjMY/; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbRrm4ZCPz9sTN for ; Tue, 25 Aug 2020 21:40:56 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725893AbgHYLkx (ORCPT ); Tue, 25 Aug 2020 07:40:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730027AbgHYLga (ORCPT ); Tue, 25 Aug 2020 07:36:30 -0400 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95E5FC0613ED; Tue, 25 Aug 2020 04:36:22 -0700 (PDT) Received: by mail-pj1-x1041.google.com with SMTP id ds1so1116246pjb.1; Tue, 25 Aug 2020 04:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0D1IRpXS9A/gM5G1vVZT30mGYc3nb9zLCzTnVKaIp0Q=; b=kJ/OjMY/0zKjOsjwXbWe80OaUcrCWI2VaXvSlTeKsqE/pX4FYryY+C6pLo7Zshe01D JiL4Yu1reJroYMY0XzsVs52vmfe/eX+AB9pdeJAe7OHpJoZcvD8BWuCbQfA2bKVF3tB6 mvN1ay89Qqv9RcqMHYEYJoPHZQpStIptLmEzOS3hepeAaBRMp+Pib3AODqBnTPDgGEIK AKKJpdXv12RdvewPtJrVhPey/YpwcdPbPz4C8gtk1fPnxeQgs4gAHA8H5Ll16OaR9RwW 40GQVAaZw0g/Or1vIDEjlbTdK39oAuBHSrrgZvZ+sAWp+hyiTDUtR1K7O8ZQGrs1Sla7 7kdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0D1IRpXS9A/gM5G1vVZT30mGYc3nb9zLCzTnVKaIp0Q=; b=Moq2VDWSX2eOCnqhHaZL3ODXk/HVGKQRCcuK0nX1X7Ss3KBe98YT4soTEcVi0IHbIi RUv2tMx08FG/Xd5xmHFWvgxlnghm5qVWkVOwr0+ovv7zaUi6SBXSxYT9oEs9ymTxI860 mqcmxU8551RqXBsr6G03OwP7KXXzyqeso0WpUih9Xxsuw1W/FZoIxOzKUuDMKam7p1ww muGLjZ6pWg/1CmKQ8NcnZvZC+6+8W4UgnFuOuWvwlIiotPENuvwaPeOKtwAv3V2ZYS95 NiOozBKbOJrAfp6DFEXhqv6DoFzAHQmjMjzFb8LJlZ9iL3g/X3PHBfNdiI8rikJbLn9u zzkg== X-Gm-Message-State: AOAM5315sPbA9I6hltsn935W6HP2t2/JqYtc4c5XYfx5KOxuyWqtv2HN AjYFrj6VuwwZEQcCRKLptFg= X-Google-Smtp-Source: ABdhPJxHHZOwCMiCYZq1sRRMNmhU/TdDDnu697pELj/7wX2ML19Au0dKJA6pClE6hVfKwDNZu9wyTA== X-Received: by 2002:a17:90a:9203:: with SMTP id m3mr1266792pjo.148.1598355382229; Tue, 25 Aug 2020 04:36:22 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id e7sm12699937pgn.64.2020.08.25.04.36.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 04:36:21 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, netdev@vger.kernel.org, bpf@vger.kernel.org, kuba@kernel.org Subject: [PATCH net-next v2 3/3] i40e, xsk: move buffer allocation out of the Rx processing loop Date: Tue, 25 Aug 2020 13:35:56 +0200 Message-Id: <20200825113556.18342-4-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825113556.18342-1-bjorn.topel@gmail.com> References: <20200825113556.18342-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: Björn Töpel Instead of checking in each iteration of the Rx packet processing loop, move the allocation out of the loop and do it once for each napi activation. For AF_XDP the rx_drop benchmark was improved by 6%. Signed-off-by: Björn Töpel Tested-by: Aaron Brown --- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 1f2dd591dbf1..ae40592c31f9 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -277,8 +277,8 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) unsigned int total_rx_bytes = 0, total_rx_packets = 0; u16 cleaned_count = I40E_DESC_UNUSED(rx_ring); unsigned int xdp_res, xdp_xmit = 0; - bool failure = false; struct sk_buff *skb; + bool failure; while (likely(total_rx_packets < (unsigned int)budget)) { union i40e_rx_desc *rx_desc; @@ -286,13 +286,6 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) unsigned int size; u64 qword; - if (cleaned_count >= I40E_RX_BUFFER_WRITE) { - failure = failure || - !i40e_alloc_rx_buffers_zc(rx_ring, - cleaned_count); - cleaned_count = 0; - } - rx_desc = I40E_RX_DESC(rx_ring, rx_ring->next_to_clean); qword = le64_to_cpu(rx_desc->wb.qword1.status_error_len); @@ -367,6 +360,9 @@ int i40e_clean_rx_irq_zc(struct i40e_ring *rx_ring, int budget) napi_gro_receive(&rx_ring->q_vector->napi, skb); } + if (cleaned_count >= I40E_RX_BUFFER_WRITE) + failure = !i40e_alloc_rx_buffers_zc(rx_ring, cleaned_count); + i40e_finalize_xdp_rx(rx_ring, xdp_xmit); i40e_update_rx_stats(rx_ring, total_rx_bytes, total_rx_packets);