From patchwork Tue Aug 25 09:16:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1350916 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=dKNfoIko; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbNfl6d7bz9sSP for ; Tue, 25 Aug 2020 19:17:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729357AbgHYJQ6 (ORCPT ); Tue, 25 Aug 2020 05:16:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728717AbgHYJQv (ORCPT ); Tue, 25 Aug 2020 05:16:51 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35D5EC061574 for ; Tue, 25 Aug 2020 02:16:51 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id o5so6392181pgb.2 for ; Tue, 25 Aug 2020 02:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b4uiR7mjiUEO8q/J67JSbBTgAuzIsuSUAuOtPzxxrK0=; b=dKNfoIkozlHWaHujT1mzqUPk6evaug7/VktNscdnpKS8wZ/CBMC4y5tJMkqWEPP2j+ 6wxF+ohJeaTMM9zH352iYNj3pSsHGRXa2WujPLqbL09to6lJMpB9bNlKhhC6J7BDQk8k Zt7FeRz+2LtfI0T8UnGbtoX4D+a5tWBcuLtcsWol+qbi+KzAD49u64bFvKPk5yFZg40T VVBMEX9MAmPOeY3g5Fs06i7L/ldJAJviDFmpXAMM78bcUDi1CyVRv58F56juIhS15tr5 jVy2ITGI8zWoao23JxAFaatZkbv4Qgej+jjP49hfX0MclsWpPrytaegs9uTA0AogMnPw EnLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b4uiR7mjiUEO8q/J67JSbBTgAuzIsuSUAuOtPzxxrK0=; b=E69Cz6czF8/5RJ0CrHsNJsy/6v3eU+KmMFSBpTNqBtE0sNhaQgdEeidHTW1TZEZwSP SsWYrNdp8FwFkGighpswhMasJqfYrjVO6UQmSa9N/U6XGNPonudJPmtI5odhOoKfP7pc hxuflFHRq5wXy0FIi2Kb/QW/BTwkGj9pvvE9cXQr/XBWz8tKp+WLksutdI1WX6ILVzpC I5UlvsPbylgjQVIscyVnJG4Q1/wjvSZhCzHDiTexU3P92/0fpGbClgHqiePX1k4bocuZ W1O1MZkKqukNRvNqtcCFfTMyieJLLNx7HQRe0RlEXps/k6xa+9VFWOhjnspF2dZ9vhEl pRWw== X-Gm-Message-State: AOAM530tksy/Kk8u3ihrt0bqWBk4Ucnhe2ffQHnIMP3lgWxysZU5VoyQ GvVITauRV5KSc4zuHB3QaqY= X-Google-Smtp-Source: ABdhPJzHBBNilV6r/snjUSKwBGOSJHXBrAS2V/2nBmJVpI8OLkFynozFebZswPvi4RWt2aDmbn9w+A== X-Received: by 2002:a17:902:6bc6:: with SMTP id m6mr6899554plt.302.1598347010745; Tue, 25 Aug 2020 02:16:50 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.45]) by smtp.gmail.com with ESMTPSA id 2sm2121857pjg.32.2020.08.25.02.16.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 02:16:50 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, magnus.karlsson@gmail.com, netdev@vger.kernel.org, maciej.fijalkowski@intel.com, piotr.raczynski@intel.com, maciej.machnikowski@intel.com, lirongqing@baidu.com Subject: [PATCH net 1/3] i40e: avoid premature Rx buffer reuse Date: Tue, 25 Aug 2020 11:16:27 +0200 Message-Id: <20200825091629.12949-2-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825091629.12949-1-bjorn.topel@gmail.com> References: <20200825091629.12949-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Björn Töpel The page recycle code, incorrectly, relied on that a page fragment could not be freed inside xdp_do_redirect(). This assumption leads to that page fragments that are used by the stack/XDP redirect can be reused and overwritten. To avoid this, store the page count prior invoking xdp_do_redirect(). Longer explanation: Intel NICs have a recycle mechanism. The main idea is that a page is split into two parts. One part is owned by the driver, one part might be owned by someone else, such as the stack. t0: Page is allocated, and put on the Rx ring +--------------- used by NIC ->| upper buffer (rx_buffer) +--------------- | lower buffer +--------------- page count == USHRT_MAX rx_buffer->pagecnt_bias == USHRT_MAX t1: Buffer is received, and passed to the stack (e.g.) +--------------- | upper buff (skb) +--------------- used by NIC ->| lower buffer (rx_buffer) +--------------- page count == USHRT_MAX rx_buffer->pagecnt_bias == USHRT_MAX - 1 t2: Buffer is received, and redirected +--------------- | upper buff (skb) +--------------- used by NIC ->| lower buffer (rx_buffer) +--------------- Now, prior calling xdp_do_redirect(): page count == USHRT_MAX rx_buffer->pagecnt_bias == USHRT_MAX - 2 This means that buffer *cannot* be flipped/reused, because the skb is still using it. The problem arises when xdp_do_redirect() actually frees the segment. Then we get: page count == USHRT_MAX - 1 rx_buffer->pagecnt_bias == USHRT_MAX - 2 From a recycle perspective, the buffer can be flipped and reused, which means that the skb data area is passed to the Rx HW ring! To work around this, the page count is stored prior calling xdp_do_redirect(). Note that this is not optimal, since the NIC could actually reuse the "lower buffer" again. However, then we need to track whether XDP_REDIRECT consumed the buffer or not. Fixes: d9314c474d4f ("i40e: add support for XDP_REDIRECT") Reported-by: Li RongQing Signed-off-by: Björn Töpel --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 28 +++++++++++++++------ 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index 3e5c566ceb01..5e446dc39190 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -1873,7 +1873,8 @@ static inline bool i40e_page_is_reusable(struct page *page) * * In either case, if the page is reusable its refcount is increased. **/ -static bool i40e_can_reuse_rx_page(struct i40e_rx_buffer *rx_buffer) +static bool i40e_can_reuse_rx_page(struct i40e_rx_buffer *rx_buffer, + int rx_buffer_pgcnt) { unsigned int pagecnt_bias = rx_buffer->pagecnt_bias; struct page *page = rx_buffer->page; @@ -1884,7 +1885,7 @@ static bool i40e_can_reuse_rx_page(struct i40e_rx_buffer *rx_buffer) #if (PAGE_SIZE < 8192) /* if we are only owner of page we can reuse it */ - if (unlikely((page_count(page) - pagecnt_bias) > 1)) + if (unlikely((rx_buffer_pgcnt - pagecnt_bias) > 1)) return false; #else #define I40E_LAST_OFFSET \ @@ -1939,6 +1940,15 @@ static void i40e_add_rx_frag(struct i40e_ring *rx_ring, #endif } +static int i40e_rx_buffer_page_count(struct i40e_rx_buffer *rx_buffer) +{ +#if (PAGE_SIZE < 8192) + return page_count(rx_buffer->page); +#else + return 0; +#endif +} + /** * i40e_get_rx_buffer - Fetch Rx buffer and synchronize data for use * @rx_ring: rx descriptor ring to transact packets on @@ -1948,11 +1958,13 @@ static void i40e_add_rx_frag(struct i40e_ring *rx_ring, * for use by the CPU. */ static struct i40e_rx_buffer *i40e_get_rx_buffer(struct i40e_ring *rx_ring, - const unsigned int size) + const unsigned int size, + int *rx_buffer_pgcnt) { struct i40e_rx_buffer *rx_buffer; rx_buffer = i40e_rx_bi(rx_ring, rx_ring->next_to_clean); + *rx_buffer_pgcnt = i40e_rx_buffer_page_count(rx_buffer); prefetchw(rx_buffer->page); /* we are reusing so sync this buffer for CPU use */ @@ -2112,9 +2124,10 @@ static struct sk_buff *i40e_build_skb(struct i40e_ring *rx_ring, * either recycle the buffer or unmap it and free the associated resources. */ static void i40e_put_rx_buffer(struct i40e_ring *rx_ring, - struct i40e_rx_buffer *rx_buffer) + struct i40e_rx_buffer *rx_buffer, + int rx_buffer_pgcnt) { - if (i40e_can_reuse_rx_page(rx_buffer)) { + if (i40e_can_reuse_rx_page(rx_buffer, rx_buffer_pgcnt)) { /* hand second half of page back to the ring */ i40e_reuse_rx_page(rx_ring, rx_buffer); } else { @@ -2319,6 +2332,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget) unsigned int xdp_xmit = 0; bool failure = false; struct xdp_buff xdp; + int rx_buffer_pgcnt; #if (PAGE_SIZE < 8192) xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, 0); @@ -2370,7 +2384,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget) break; i40e_trace(clean_rx_irq, rx_ring, rx_desc, skb); - rx_buffer = i40e_get_rx_buffer(rx_ring, size); + rx_buffer = i40e_get_rx_buffer(rx_ring, size, &rx_buffer_pgcnt); /* retrieve a buffer from the ring */ if (!skb) { @@ -2413,7 +2427,7 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget) break; } - i40e_put_rx_buffer(rx_ring, rx_buffer); + i40e_put_rx_buffer(rx_ring, rx_buffer, rx_buffer_pgcnt); cleaned_count++; if (i40e_is_non_eop(rx_ring, rx_desc, skb)) From patchwork Tue Aug 25 09:16:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1350913 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=DAg8CGn/; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbNfj1Cb4z9sSP for ; Tue, 25 Aug 2020 19:17:01 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729369AbgHYJQ7 (ORCPT ); Tue, 25 Aug 2020 05:16:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729351AbgHYJQz (ORCPT ); Tue, 25 Aug 2020 05:16:55 -0400 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53CD7C061755 for ; Tue, 25 Aug 2020 02:16:55 -0700 (PDT) Received: by mail-pj1-x1041.google.com with SMTP id mt12so913036pjb.4 for ; Tue, 25 Aug 2020 02:16:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R8XTi8qvu+toNhdfEJW+FeTXeAE1uYK8ZjIgOFsPChs=; b=DAg8CGn//Li0QGdpYMY1ZV0Evl5CbSIXuCdVKC2uYBYv1XAmKF8hAKNiXisfUdSmxH qmFSr0WKJZBcppTQ6WH7ezGvpq5pto1Veo9UqRk6gHDLSv06imc5NplC/uQB8EE5TX5y nlaC8MRyl91cKGxhWDv7hLGk35Zqit5P2L+X4uqDGkg5lMSu1bqqkcZiweJkNz2jucgY h4MrruWencTIr1ndxxocusN04+1yF0HEJXoVdmzQUDB/YrSbq4oR11aQeVDGFJPAsj/b rZmW/NODRsBSk5CU/0qmYOEbL8Y//tM1f7OIVxRBb7G8/Hw9wyOwSpPL7hlal43kdgQw M31Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R8XTi8qvu+toNhdfEJW+FeTXeAE1uYK8ZjIgOFsPChs=; b=kd2ZOXHhprrnoh5s4Q4Px2J/RZhXVVmqafzwsF564y9kIWqxSkx7YkrTpPH+FmfZIy BBp1+dUPfOxm3/WRhUd8wyZI5oGPPVd/p0+YSh4CA90+QPKbzh/RUqIzxKx/08b/TQ3D qtg9gu07N6ThcPs+m2gOZsU7JUtD81wP9I6f5YfUA+TGUvVCr8peBChXsElJ6CCpY7Oj u7OAZcw2UbQGC6ilCdY6WSMxoMocitWEPibyCufZ/GpNI4kKUT2PFJvBrYUsaxlhrBxm 3AP9JnPWb9TLFSYvoc2tTqupZHVLlY7agxKTt52qJg7KxXqk58P6s3Yy8ZfC8V3tROE+ Z/Tw== X-Gm-Message-State: AOAM533QfCs69yCvohWXkBeTHLOVdSk3uDO/SbbOJ2MTL+SxPmznPYDI tmEvA7AwGiQQ2FDoZ6/8DU4+lWr+aozlbQ== X-Google-Smtp-Source: ABdhPJx6RmfZMrj9BxYXpc2JaWUs+lxm8tEAqhpdGCByfEK/MApwWSZPVNerqpnfQ4kTyhiTw5vLFw== X-Received: by 2002:a17:90a:d808:: with SMTP id a8mr835295pjv.127.1598347014929; Tue, 25 Aug 2020 02:16:54 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.45]) by smtp.gmail.com with ESMTPSA id 2sm2121857pjg.32.2020.08.25.02.16.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 02:16:53 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, magnus.karlsson@gmail.com, netdev@vger.kernel.org, maciej.fijalkowski@intel.com, piotr.raczynski@intel.com, maciej.machnikowski@intel.com, lirongqing@baidu.com Subject: [PATCH net 2/3] ixgbe: avoid premature Rx buffer reuse Date: Tue, 25 Aug 2020 11:16:28 +0200 Message-Id: <20200825091629.12949-3-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825091629.12949-1-bjorn.topel@gmail.com> References: <20200825091629.12949-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Björn Töpel The page recycle code, incorrectly, relied on that a page fragment could not be freed inside xdp_do_redirect(). This assumption leads to that page fragments that are used by the stack/XDP redirect can be reused and overwritten. To avoid this, store the page count prior invoking xdp_do_redirect(). Fixes: 6453073987ba ("ixgbe: add initial support for xdp redirect") Signed-off-by: Björn Töpel --- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 28 ++++++++++++++----- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 2f8a4cfc5fa1..fb5c311d72b6 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -1945,7 +1945,8 @@ static inline bool ixgbe_page_is_reserved(struct page *page) return (page_to_nid(page) != numa_mem_id()) || page_is_pfmemalloc(page); } -static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer) +static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer, + int rx_buffer_pgcnt) { unsigned int pagecnt_bias = rx_buffer->pagecnt_bias; struct page *page = rx_buffer->page; @@ -1956,7 +1957,7 @@ static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer) #if (PAGE_SIZE < 8192) /* if we are only owner of page we can reuse it */ - if (unlikely((page_ref_count(page) - pagecnt_bias) > 1)) + if (unlikely((rx_buffer_pgcnt - pagecnt_bias) > 1)) return false; #else /* The last offset is a bit aggressive in that we assume the @@ -2018,14 +2019,25 @@ static void ixgbe_add_rx_frag(struct ixgbe_ring *rx_ring, #endif } +static int ixgbe_rx_buffer_page_count(struct ixgbe_rx_buffer *rx_buffer) +{ +#if (PAGE_SIZE < 8192) + return page_count(rx_buffer->page); +#else + return 0; +#endif +} + static struct ixgbe_rx_buffer *ixgbe_get_rx_buffer(struct ixgbe_ring *rx_ring, union ixgbe_adv_rx_desc *rx_desc, struct sk_buff **skb, - const unsigned int size) + const unsigned int size, + int *rx_buffer_pgcnt) { struct ixgbe_rx_buffer *rx_buffer; rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean]; + *rx_buffer_pgcnt = ixgbe_rx_buffer_page_count(rx_buffer); prefetchw(rx_buffer->page); *skb = rx_buffer->skb; @@ -2055,9 +2067,10 @@ static struct ixgbe_rx_buffer *ixgbe_get_rx_buffer(struct ixgbe_ring *rx_ring, static void ixgbe_put_rx_buffer(struct ixgbe_ring *rx_ring, struct ixgbe_rx_buffer *rx_buffer, - struct sk_buff *skb) + struct sk_buff *skb, + int rx_buffer_pgcnt) { - if (ixgbe_can_reuse_rx_page(rx_buffer)) { + if (ixgbe_can_reuse_rx_page(rx_buffer, rx_buffer_pgcnt)) { /* hand second half of page back to the ring */ ixgbe_reuse_rx_page(rx_ring, rx_buffer); } else { @@ -2296,6 +2309,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, u16 cleaned_count = ixgbe_desc_unused(rx_ring); unsigned int xdp_xmit = 0; struct xdp_buff xdp; + int rx_buffer_pgcnt; xdp.rxq = &rx_ring->xdp_rxq; @@ -2327,7 +2341,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, */ dma_rmb(); - rx_buffer = ixgbe_get_rx_buffer(rx_ring, rx_desc, &skb, size); + rx_buffer = ixgbe_get_rx_buffer(rx_ring, rx_desc, &skb, size, &rx_buffer_pgcnt); /* retrieve a buffer from the ring */ if (!skb) { @@ -2372,7 +2386,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector, break; } - ixgbe_put_rx_buffer(rx_ring, rx_buffer, skb); + ixgbe_put_rx_buffer(rx_ring, rx_buffer, skb, rx_buffer_pgcnt); cleaned_count++; /* place incomplete frames back on ring for completion */ From patchwork Tue Aug 25 09:16:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= X-Patchwork-Id: 1350915 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=kI1hBVAr; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BbNfk4pWkz9sSP for ; Tue, 25 Aug 2020 19:17:02 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729375AbgHYJRB (ORCPT ); Tue, 25 Aug 2020 05:17:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729371AbgHYJQ7 (ORCPT ); Tue, 25 Aug 2020 05:16:59 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7773DC061574 for ; Tue, 25 Aug 2020 02:16:59 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id g33so6391214pgb.4 for ; Tue, 25 Aug 2020 02:16:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lSOHb2BtMg2wPWXugkRPn8o1h+t2kwdV3CyIiFwfvr8=; b=kI1hBVArsOgYGkWu3YDpNcrIjZ/zjWZtkKwTeplIQvAyLW/VjhSFoU9tuza+6o5QXP dI6gDDlaXbn6rJm+wbWxLMrV3aa9CIyWhzaBCSchlx/X9wgzBOoiHgH9tQRF8B6USaDM VIVz1J17MuptUXI0o30kK9GT5sb6V6JI/z/+syvNUPptvxDNNE3Gh64TE7uBp3KvO3b5 p/o4JM+fHsoL5hP+nB74Az+C2X8Pv49kAIQuRn+SAfvsQdxT+zuRxoZXncHfDTDrTteG GFOtFSpYF5MkLkS3rsVWzs+vjQe61+GtGRcFzxeVCI5stQSNFVm2JsK/no96Mfu9tuS0 2wlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lSOHb2BtMg2wPWXugkRPn8o1h+t2kwdV3CyIiFwfvr8=; b=KCJNXlaiWNQ8SaTciFkb8xpSQKXGjv+P2tWIdNNu19cc2AhPwByiZRl8tndKP3cij5 joanmFwEORniN7h0deTfcf9dPiA7DmorrBUl1wsr0pezAOPmiMTDEnuWbFoQQeptR6gi EOUybCwldEh4FqLKS9VLQT8+zHQquOfC+dCiX5ej2eYPXdiamkJOmrtHZsj5hmDRV/s8 W/8R025vWwc/LY22cN2zLxgUmoXawzYIJYEPAoR26inoWMw8ZwZzo8x3FPGlJGoleAx2 r4sLcAdHqSsZp25p1Rbsb4TCS8/pbCu0nzt9UAEcgTb2tJjhjXTVrKv70gLbxTbmFhyP z9Tg== X-Gm-Message-State: AOAM531hMEkKlkSKBmKEWa/xzDAi/YjpSTF4muujvUHRpyICicpIax8B Srw/E6EvFfYXh5jMv4TxFn8= X-Google-Smtp-Source: ABdhPJwWItfwTBa/7GlcEVgEAzPrBO0irVfimfNXGk8E7ihltw5bz80TrhDJLF7RooYRBcUVe1I4SA== X-Received: by 2002:a63:2944:: with SMTP id p65mr6384832pgp.271.1598347019028; Tue, 25 Aug 2020 02:16:59 -0700 (PDT) Received: from btopel-mobl.ger.intel.com ([192.55.55.45]) by smtp.gmail.com with ESMTPSA id 2sm2121857pjg.32.2020.08.25.02.16.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 02:16:58 -0700 (PDT) From: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= To: jeffrey.t.kirsher@intel.com, intel-wired-lan@lists.osuosl.org Cc: =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , magnus.karlsson@intel.com, magnus.karlsson@gmail.com, netdev@vger.kernel.org, maciej.fijalkowski@intel.com, piotr.raczynski@intel.com, maciej.machnikowski@intel.com, lirongqing@baidu.com Subject: [PATCH net 3/3] ice: avoid premature Rx buffer reuse Date: Tue, 25 Aug 2020 11:16:29 +0200 Message-Id: <20200825091629.12949-4-bjorn.topel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200825091629.12949-1-bjorn.topel@gmail.com> References: <20200825091629.12949-1-bjorn.topel@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Björn Töpel The page recycle code, incorrectly, relied on that a page fragment could not be freed inside xdp_do_redirect(). This assumption leads to that page fragments that are used by the stack/XDP redirect can be reused and overwritten. To avoid this, store the page count prior invoking xdp_do_redirect(). Fixes: efc2214b6047 ("ice: Add support for XDP") Signed-off-by: Björn Töpel --- drivers/net/ethernet/intel/ice/ice_txrx.c | 31 ++++++++++++++++------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index 9d0d6b0025cf..03a88c8f17b7 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -768,7 +768,8 @@ ice_rx_buf_adjust_pg_offset(struct ice_rx_buf *rx_buf, unsigned int size) * pointing to; otherwise, the DMA mapping needs to be destroyed and * page freed */ -static bool ice_can_reuse_rx_page(struct ice_rx_buf *rx_buf) +static bool ice_can_reuse_rx_page(struct ice_rx_buf *rx_buf, + int rx_buf_pgcnt) { unsigned int pagecnt_bias = rx_buf->pagecnt_bias; struct page *page = rx_buf->page; @@ -779,7 +780,7 @@ static bool ice_can_reuse_rx_page(struct ice_rx_buf *rx_buf) #if (PAGE_SIZE < 8192) /* if we are only owner of page we can reuse it */ - if (unlikely((page_count(page) - pagecnt_bias) > 1)) + if (unlikely((rx_buf_pgcnt - pagecnt_bias) > 1)) return false; #else #define ICE_LAST_OFFSET \ @@ -859,6 +860,15 @@ ice_reuse_rx_page(struct ice_ring *rx_ring, struct ice_rx_buf *old_buf) new_buf->pagecnt_bias = old_buf->pagecnt_bias; } +static int ice_rx_buf_page_count(struct ice_rx_buf *rx_buf) +{ +#if (PAGE_SIZE < 8192) + return page_count(rx_buf->page); +#else + return 0; +#endif +} + /** * ice_get_rx_buf - Fetch Rx buffer and synchronize data for use * @rx_ring: Rx descriptor ring to transact packets on @@ -870,11 +880,13 @@ ice_reuse_rx_page(struct ice_ring *rx_ring, struct ice_rx_buf *old_buf) */ static struct ice_rx_buf * ice_get_rx_buf(struct ice_ring *rx_ring, struct sk_buff **skb, - const unsigned int size) + const unsigned int size, + int *rx_buf_pgcnt) { struct ice_rx_buf *rx_buf; rx_buf = &rx_ring->rx_buf[rx_ring->next_to_clean]; + *rx_buf_pgcnt = ice_rx_buf_page_count(rx_buf); prefetchw(rx_buf->page); *skb = rx_buf->skb; @@ -1017,7 +1029,7 @@ ice_construct_skb(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf, * of the rx_buf. It will either recycle the buffer or unmap it and free * the associated resources. */ -static void ice_put_rx_buf(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf) +static void ice_put_rx_buf(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf, int rx_buf_pgcnt) { u16 ntc = rx_ring->next_to_clean + 1; @@ -1028,7 +1040,7 @@ static void ice_put_rx_buf(struct ice_ring *rx_ring, struct ice_rx_buf *rx_buf) if (!rx_buf) return; - if (ice_can_reuse_rx_page(rx_buf)) { + if (ice_can_reuse_rx_page(rx_buf, rx_buf_pgcnt)) { /* hand second half of page back to the ring */ ice_reuse_rx_page(rx_ring, rx_buf); } else { @@ -1088,6 +1100,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) unsigned int xdp_res, xdp_xmit = 0; struct bpf_prog *xdp_prog = NULL; struct xdp_buff xdp; + int rx_buf_pgcnt; bool failure; xdp.rxq = &rx_ring->xdp_rxq; @@ -1125,7 +1138,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) dma_rmb(); if (rx_desc->wb.rxdid == FDIR_DESC_RXDID || !rx_ring->netdev) { - ice_put_rx_buf(rx_ring, NULL); + ice_put_rx_buf(rx_ring, NULL, 0); cleaned_count++; continue; } @@ -1134,7 +1147,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) ICE_RX_FLX_DESC_PKT_LEN_M; /* retrieve a buffer from the ring */ - rx_buf = ice_get_rx_buf(rx_ring, &skb, size); + rx_buf = ice_get_rx_buf(rx_ring, &skb, size, &rx_buf_pgcnt); if (!size) { xdp.data = NULL; @@ -1174,7 +1187,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) total_rx_pkts++; cleaned_count++; - ice_put_rx_buf(rx_ring, rx_buf); + ice_put_rx_buf(rx_ring, rx_buf, rx_buf_pgcnt); continue; construct_skb: if (skb) { @@ -1193,7 +1206,7 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) break; } - ice_put_rx_buf(rx_ring, rx_buf); + ice_put_rx_buf(rx_ring, rx_buf, rx_buf_pgcnt); cleaned_count++; /* skip if it is NOP desc */