From patchwork Fri Jan 15 22:33:15 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Hay X-Patchwork-Id: 568511 X-Patchwork-Delegate: jeffrey.t.kirsher@intel.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ozlabs.org (Postfix) with ESMTP id 6C1B3140BA4 for ; Sat, 16 Jan 2016 09:33:36 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D538191A9C; Fri, 15 Jan 2016 22:33:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jtQg+LlyOk5O; Fri, 15 Jan 2016 22:33:30 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id EEAB291A73; Fri, 15 Jan 2016 22:33:25 +0000 (UTC) X-Original-To: intel-wired-lan@lists.osuosl.org Delivered-To: intel-wired-lan@lists.osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id 420D01C2189 for ; Fri, 15 Jan 2016 22:33:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 3E60393643 for ; Fri, 15 Jan 2016 22:33:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BbfMpsm1A7tF for ; Fri, 15 Jan 2016 22:33:23 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by hemlock.osuosl.org (Postfix) with ESMTP id E1A8093648 for ; Fri, 15 Jan 2016 22:33:23 +0000 (UTC) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP; 15 Jan 2016 14:33:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,302,1449561600"; d="scan'208";a="894132183" Received: from jahay1-mobl2.amr.corp.intel.com (HELO localhost.localdomain.localdomain) ([134.134.176.81]) by fmsmga002.fm.intel.com with ESMTP; 15 Jan 2016 14:33:22 -0800 From: Joshua Hay To: intel-wired-lan@lists.osuosl.org Date: Fri, 15 Jan 2016 14:33:15 -0800 Message-Id: <1452897202-15204-9-git-send-email-joshua.a.hay@intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1452897202-15204-1-git-send-email-joshua.a.hay@intel.com> References: <1452897202-15204-1-git-send-email-joshua.a.hay@intel.com> Subject: [Intel-wired-lan] [next PATCH S28 08/15] i40e/i40evf: avoid atomics X-BeenThere: intel-wired-lan@lists.osuosl.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-wired-lan-bounces@lists.osuosl.org Sender: "Intel-wired-lan" From: Mitch Williams In the case where we have a page fully used by receive data, we need to release the page fully to the stack. Instead of calling get_page (which increments the page count) followed by free_page (which decrements the page count), just donate our reference to the stack. Although this donation is not tax deductable, it does allow us to avoid two very expensive atomic operations that reverse each other. Signed-off-by: Mitch Williams Change-ID: If70739792d5748995fc175ec92ac2171ed4ad8fc Tested-by: Andrew Bowers --- Testing Hints: make sure receive traffic is stable under heavy load with packet split enabled, and that no data corruption is seen. drivers/net/ethernet/intel/i40e/i40e_txrx.c | 21 +++++++++++++-------- drivers/net/ethernet/intel/i40evf/i40e_txrx.c | 21 +++++++++++++-------- 2 files changed, 26 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index 60e993e..cd9bb7f 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -1663,28 +1663,33 @@ static int i40e_clean_rx_irq_ps(struct i40e_ring *rx_ring, const int budget) rx_bi->page_offset + copysize, rx_packet_len, I40E_RXBUFFER_2048); - get_page(rx_bi->page); - /* switch to the other half-page here; the allocation - * code programs the right addr into HW. If we haven't - * used this half-page, the address won't be changed, - * and HW can just use it next time through. - */ - rx_bi->page_offset ^= PAGE_SIZE / 2; /* If the page count is more than 2, then both halves * of the page are used and we need to free it. Do it * here instead of in the alloc code. Otherwise one * of the half-pages might be released between now and * then, and we wouldn't know which one to use. + * Don't call get_page and free_page since those are + * both expensive atomic operations that just change + * the refcount in opposite directions. Just give the + * page to the stack; he can have our refcount. */ if (page_count(rx_bi->page) > 2) { dma_unmap_page(rx_ring->dev, rx_bi->page_dma, PAGE_SIZE, DMA_FROM_DEVICE); - __free_page(rx_bi->page); rx_bi->page = NULL; rx_bi->page_dma = 0; rx_ring->rx_stats.realloc_count++; + } else { + get_page(rx_bi->page); + /* switch to the other half-page here; the + * allocation code programs the right addr + * into HW. If we haven't used this half-page, + * the address won't be changed, and HW can + * just use it next time through. + */ + rx_bi->page_offset ^= PAGE_SIZE / 2; } } diff --git a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c index c1ea819..ecd2df2 100644 --- a/drivers/net/ethernet/intel/i40evf/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40evf/i40e_txrx.c @@ -1126,28 +1126,33 @@ static int i40e_clean_rx_irq_ps(struct i40e_ring *rx_ring, const int budget) rx_bi->page_offset + copysize, rx_packet_len, I40E_RXBUFFER_2048); - get_page(rx_bi->page); - /* switch to the other half-page here; the allocation - * code programs the right addr into HW. If we haven't - * used this half-page, the address won't be changed, - * and HW can just use it next time through. - */ - rx_bi->page_offset ^= PAGE_SIZE / 2; /* If the page count is more than 2, then both halves * of the page are used and we need to free it. Do it * here instead of in the alloc code. Otherwise one * of the half-pages might be released between now and * then, and we wouldn't know which one to use. + * Don't call get_page and free_page since those are + * both expensive atomic operations that just change + * the refcount in opposite directions. Just give the + * page to the stack; he can have our refcount. */ if (page_count(rx_bi->page) > 2) { dma_unmap_page(rx_ring->dev, rx_bi->page_dma, PAGE_SIZE, DMA_FROM_DEVICE); - __free_page(rx_bi->page); rx_bi->page = NULL; rx_bi->page_dma = 0; rx_ring->rx_stats.realloc_count++; + } else { + get_page(rx_bi->page); + /* switch to the other half-page here; the + * allocation code programs the right addr + * into HW. If we haven't used this half-page, + * the address won't be changed, and HW can + * just use it next time through. + */ + rx_bi->page_offset ^= PAGE_SIZE / 2; } }