From patchwork Thu Aug 20 17:11:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 1348527 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=p+PeHxk3; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BXWRB1klJz9sRK for ; Fri, 21 Aug 2020 03:12:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730532AbgHTRMA (ORCPT ); Thu, 20 Aug 2020 13:12:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43734 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730522AbgHTRL0 (ORCPT ); Thu, 20 Aug 2020 13:11:26 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4CF3C061386 for ; Thu, 20 Aug 2020 10:11:25 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id u189so2969477ybg.17 for ; Thu, 20 Aug 2020 10:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Ugou+I4rzj+9uQFCuzQnBcn1RiOI69RVVmOwp83QRW0=; b=p+PeHxk33ogmlP3/o0eJgU6tzzESrQYmNZzWZ0XdCnuNfjKboWxXWZNF+A3/PYq26Z fQEmPfossD2U3n154N7g+jaAosT0BoTNCY5Q3cQk33Tx7CIf6YZV+YPujZgctFYvGTof N+8JT2IoKYEDbpilRFySGWUZ3iZWXAaPgzAbras1xvtjvRwj4OZhTTU2n/DE8U4GagK/ JMu45UeNnuYEhXWdkZNLkfZwl6xIW1mJeSQA/4nJ6AEbmcOgt69dgosSXZg0b7pqJEOI XEdcmiW9dcbTOtyR8yJkrCqBEqNHjov+/HJlQm7HuKcaURqTVFmBvnRC1nQRZbQwUAzw Uhcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Ugou+I4rzj+9uQFCuzQnBcn1RiOI69RVVmOwp83QRW0=; b=k36n4tVH3uSoXaLMjqd1R1m+fl8FQJ9r9wTlNRhT3I40NaYgt9VSfk4Qo2MZGERI2v Bh6rGcamtU5Yp3Pmg3mZZwa+5MkdL93elxLUAz6Gd6ep43Cli78xnKfzCUjj0ElNcehA rFQNcLCv4ZTCswimU1QJ8qJxW5R3+q6oXbtA6UjkNUy/EoAup4S0p6AMRO1vvVMv3YVo H1wSTujQSYK2jBHMOjCzCEMowB+274K5GTi4KcCuv2isf4fpyrpKjgWpyaVc2DIFfbzF +I6+PGdK7azELCI3r1Q04XLvyz0ddP/fhApHnSIMUNzcaHJHTckPA+EeUBUrpy5or2AH XAHA== X-Gm-Message-State: AOAM532X3q1S4Amhg784m/x7BMwhA4neuj+adwiaA8sKZPOum88NGwjJ uIJnENTPdSAJZwYqevVxEToZdJVaBS98yw== X-Google-Smtp-Source: ABdhPJypOk5W8XY+5MFI60/xJPaMgdgI7Y1Va5AQCiU/aTEAroRxrkavqKfpJ9QBhY0lPvmb1LwKi5TaJp8m4g== X-Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:7220:84ff:fe09:1424]) (user=edumazet job=sendgmr) by 2002:a25:846:: with SMTP id 67mr6122655ybi.474.1597943485019; Thu, 20 Aug 2020 10:11:25 -0700 (PDT) Date: Thu, 20 Aug 2020 10:11:16 -0700 In-Reply-To: <20200820171118.1822853-1-edumazet@google.com> Message-Id: <20200820171118.1822853-2-edumazet@google.com> Mime-Version: 1.0 References: <20200820171118.1822853-1-edumazet@google.com> X-Mailer: git-send-email 2.28.0.297.g1956fa8f8d-goog Subject: [PATCH net-next 1/3] selftests: net: tcp_mmap: use madvise(MADV_DONTNEED) From: Eric Dumazet To: "David S . Miller" Cc: netdev , Eric Dumazet , Eric Dumazet , Soheil Hassas Yeganeh , Arjun Roy Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When TCP_ZEROCOPY_RECEIVE operation has been added, I made the mistake of automatically un-mapping prior content before mapping new pages. This has the unfortunate effect of adding potentially long MMU operations (like TLB flushes) while socket lock is held. Using madvise(MADV_DONTNEED) right after pages has been used has two benefits : 1) This releases pages sooner, allowing pages to be recycled if they were part of a page pool in a NIC driver. 2) No more long unmap operations while preventing immediate processing of incoming packets. The cost of the added system call is small enough. Arjun will submit a kernel patch allowing to opt out from the unmap attempt in tcp_zerocopy_receive() Signed-off-by: Eric Dumazet Cc: Arjun Roy Cc: Soheil Hassas Yeganeh --- tools/testing/selftests/net/tcp_mmap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/testing/selftests/net/tcp_mmap.c b/tools/testing/selftests/net/tcp_mmap.c index a61b7b3da5496285876b0e16b18a3060850b0803..59ec0b59f7b76ff75685bd96901d8237e0665b2b 100644 --- a/tools/testing/selftests/net/tcp_mmap.c +++ b/tools/testing/selftests/net/tcp_mmap.c @@ -179,6 +179,10 @@ void *child_thread(void *arg) total_mmap += zc.length; if (xflg) hash_zone(addr, zc.length); + /* It is more efficient to unmap the pages right now, + * instead of doing this in next TCP_ZEROCOPY_RECEIVE. + */ + madvise(addr, zc.length, MADV_DONTNEED); total += zc.length; } if (zc.recv_skip_hint) { From patchwork Thu Aug 20 17:11:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 1348529 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=hBOeh3XM; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BXWS35w4pz9sRN for ; Fri, 21 Aug 2020 03:12:51 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730503AbgHTRMc (ORCPT ); Thu, 20 Aug 2020 13:12:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43740 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730523AbgHTRL3 (ORCPT ); Thu, 20 Aug 2020 13:11:29 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 485C1C061387 for ; Thu, 20 Aug 2020 10:11:28 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id a73so1616835pfa.10 for ; Thu, 20 Aug 2020 10:11:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=VQfBKsvDWx+HzFQd4Qe36rkEa5O6q5wzeJ4kSs1eZGM=; b=hBOeh3XMAs16M16NFXVYBnfpzGtiCG/irlj0yVBHy4h9MFN74LPP94HEpnJSw46xs0 0/DVKAxaD16HMEhNDTdEm359yPuFJB1suezF2kYC5aS2/xy4GfRcDQYtseeBn+2Zw1Mg vq4VUPrvxxchDoMh1vSIbQiyTwKRlqkboeFiXtCjL9srfZoenKQNtdd7czsPtl/nfQQK SZq1MBy6NGdlrqDnJ9toVSV/jXvikmDm+A7pLPNDkOgh2WTNhqABn70HN9+2qhy96Mik IxUqnsI/uzPuIdhN/1UK8MF5I7S2n06Kp8ipoiMWDyq15Ct0Ii/rIYCo/eeJLIildyil quXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VQfBKsvDWx+HzFQd4Qe36rkEa5O6q5wzeJ4kSs1eZGM=; b=hW5434lXkLBMhmzy2Ue+tZKHrGqJkyyD66aKN8S2vbWjXP6Vos/6lIK7oy1EL10hLe 40ikUP+ZkTwrv9XbaN64IYU51Ddnca2v9xTnMr7df0HtlN4PyyfSttFzhCFqlT7/FwFN WVzB6FgXn6Ik93sQAu5/m41WRoG5DQVbSbWyTXFSQlec2EdOE7JF9VjHyk77KxWJ9y9q VU/BJBt4CyrwJ65J6quonQF2q3lxNTxD510SDZHB5l+kofqq6mxRKrXu38G2ZqY24jwc 0ezV87rEHQW0pqgCvY5AfkSAn7Tx89NiDPJnnvu/v88hWqI3U04oialya4gFzYcm3kvb zFVw== X-Gm-Message-State: AOAM532ksxf7O4SJm7ezLahRDBif523FFcF5nvnE64QSponnt11IKjgR LVLJr8CIt9TKRxo8EFcj+zh0uQK1l3En0g== X-Google-Smtp-Source: ABdhPJwVhZ0C0P+499YBfdv/2KLJ/qiOsRSUUoEGXVAObJ0RrJR1zp7NSauvUKr1j6egdWsfWxvg5kPcqiFjrA== X-Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:7220:84ff:fe09:1424]) (user=edumazet job=sendgmr) by 2002:a17:90b:194d:: with SMTP id nk13mr3062339pjb.220.1597943487612; Thu, 20 Aug 2020 10:11:27 -0700 (PDT) Date: Thu, 20 Aug 2020 10:11:17 -0700 In-Reply-To: <20200820171118.1822853-1-edumazet@google.com> Message-Id: <20200820171118.1822853-3-edumazet@google.com> Mime-Version: 1.0 References: <20200820171118.1822853-1-edumazet@google.com> X-Mailer: git-send-email 2.28.0.297.g1956fa8f8d-goog Subject: [PATCH net-next 2/3] selftests: net: tcp_mmap: Use huge pages in send path From: Eric Dumazet To: "David S . Miller" Cc: netdev , Eric Dumazet , Eric Dumazet , Soheil Hassas Yeganeh , Arjun Roy Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org There are significant gains using huge pages when available, as shown in [1]. This patch adds mmap_large_buffer() and uses it in client side (tx path of this reference tool) Following patch will use the feature for server side. [1] https://patchwork.ozlabs.org/project/netdev/patch/20200820154359.1806305-1-edumazet@google.com/ Signed-off-by: Eric Dumazet Cc: Arjun Roy Cc: Soheil Hassas Yeganeh --- tools/testing/selftests/net/tcp_mmap.c | 29 +++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/net/tcp_mmap.c b/tools/testing/selftests/net/tcp_mmap.c index 59ec0b59f7b76ff75685bd96901d8237e0665b2b..ca2618f3e7a12ab6863665f465dea2e8d469131b 100644 --- a/tools/testing/selftests/net/tcp_mmap.c +++ b/tools/testing/selftests/net/tcp_mmap.c @@ -123,6 +123,28 @@ void hash_zone(void *zone, unsigned int length) #define ALIGN_UP(x, align_to) (((x) + ((align_to)-1)) & ~((align_to)-1)) #define ALIGN_PTR_UP(p, ptr_align_to) ((typeof(p))ALIGN_UP((unsigned long)(p), ptr_align_to)) + +static void *mmap_large_buffer(size_t need, size_t *allocated) +{ + void *buffer; + size_t sz; + + /* Attempt to use huge pages if possible. */ + sz = ALIGN_UP(need, map_align); + buffer = mmap(NULL, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB, -1, 0); + + if (buffer == (void *)-1) { + sz = need; + buffer = mmap(NULL, sz, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (buffer != (void *)-1) + fprintf(stderr, "MAP_HUGETLB attempt failed, look at /sys/kernel/mm/hugepages for optimal performance\n"); + } + *allocated = sz; + return buffer; +} + void *child_thread(void *arg) { unsigned long total_mmap = 0, total = 0; @@ -351,6 +373,7 @@ int main(int argc, char *argv[]) uint64_t total = 0; char *host = NULL; int fd, c, on = 1; + size_t buffer_sz; char *buffer; int sflg = 0; int mss = 0; @@ -441,8 +464,8 @@ int main(int argc, char *argv[]) } do_accept(fdlisten); } - buffer = mmap(NULL, chunk_size, PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + + buffer = mmap_large_buffer(chunk_size, &buffer_sz); if (buffer == (char *)-1) { perror("mmap"); exit(1); @@ -488,6 +511,6 @@ int main(int argc, char *argv[]) total += wr; } close(fd); - munmap(buffer, chunk_size); + munmap(buffer, buffer_sz); return 0; } From patchwork Thu Aug 20 17:11:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Dumazet X-Patchwork-Id: 1348528 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=GuCkuujF; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BXWRP5cb1z9sRK for ; Fri, 21 Aug 2020 03:12:17 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727917AbgHTRMK (ORCPT ); Thu, 20 Aug 2020 13:12:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730524AbgHTRLb (ORCPT ); Thu, 20 Aug 2020 13:11:31 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 168B5C061388 for ; Thu, 20 Aug 2020 10:11:31 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id x18so1626023qkb.16 for ; Thu, 20 Aug 2020 10:11:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=TprXnGjijq3y+NFkKvqy7Kb/WDKfn8pQZD2ICR8FoaY=; b=GuCkuujFfNH+K8er5uA7hFGdirYdlcT+ZrOsxgmzcW1yAsmR385cvBQ4zWjwBhi0Pq L5w0MjEMlEFt90o/LF9xxZZr4nbQbPP1R4xLnjlz4dib9fkwmky2eTXwyeHjXrzmdBc5 pG+CP+2koOQPTeeIbMHIrfK4nV4yYce3TBfRW8gYHvseHFU7lN12RDRPbfpBMKIcG1YA N9INuR4zAiS1Zt2bwSpvXC2x+RjzrrDCsosyobyHZ1O8PWvSj7yuNRqMIyBSiBTbGqaR tds5En+RKhGVpGDEGdbgE9sgsxP8muvs3U5JtCJBQYUAtmyNyPGJ/gm/1z3ljB06ViGO UIzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TprXnGjijq3y+NFkKvqy7Kb/WDKfn8pQZD2ICR8FoaY=; b=XWS2mLo+1nMs46jthGHzKcuV3blUWWzvRGmH7wsTOYSTWM92tVaR0URu2SMYGLEp/3 yra2ZZ6TLzvp229goBGKh/DdVs76oqVtkvUgHuhar7GA2HL4Pj1abJogmErpqhTiUOex qnrFEwx9TnftYZnb1+Tcj2v0qBXSJTi0VM8pzAU5Psjm3k4ugQ7b3HdyOgpzf/8Z1GYL Q9YTWO20oc4oJugBBJhsAhUhvSt5gpX98g7Mh3IY2JgM7r19vffR2xCb5VTwP0QLQHrP 96tAgCWSYhFNSuWh8fgXEWXI800iwRkoVKIY1QSPNxck4k8SBBRZ/5Qc0FXCVkQErYrg dgmQ== X-Gm-Message-State: AOAM532SOeY/DekU6oRdyoamO+3Lobdv/KnclRypW+Ywosdq5VWOcXVd a9u1wdTEqktEw19JFqmuPVDF3O7kqIG2wg== X-Google-Smtp-Source: ABdhPJwrb05+UGEqWEcmVcTfH+GSWx90kgx6LdI5LX7v7yBACkmTBY+6kM/fWPbtJoDnFt/8oH8U6JJk6IEENw== X-Received: from edumazet1.svl.corp.google.com ([2620:15c:2c4:201:7220:84ff:fe09:1424]) (user=edumazet job=sendgmr) by 2002:ad4:4c0a:: with SMTP id bz10mr3809138qvb.78.1597943490225; Thu, 20 Aug 2020 10:11:30 -0700 (PDT) Date: Thu, 20 Aug 2020 10:11:18 -0700 In-Reply-To: <20200820171118.1822853-1-edumazet@google.com> Message-Id: <20200820171118.1822853-4-edumazet@google.com> Mime-Version: 1.0 References: <20200820171118.1822853-1-edumazet@google.com> X-Mailer: git-send-email 2.28.0.297.g1956fa8f8d-goog Subject: [PATCH net-next 3/3] selftests: net: tcp_mmap: Use huge pages in receive path From: Eric Dumazet To: "David S . Miller" Cc: netdev , Eric Dumazet , Eric Dumazet , Soheil Hassas Yeganeh , Arjun Roy Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org One down side of using TCP rx zerocopy is one extra TLB miss per page after the mapping operation. While if the application is using hugepages, the non zerocopy recvmsg() will not have to pay these TLB costs. This patch allows server side to use huge pages for the non zero copy case, to allow fair comparisons when both solutions use optimal conditions. Signed-off-by: Eric Dumazet Cc: Arjun Roy Cc: Soheil Hassas Yeganeh --- tools/testing/selftests/net/tcp_mmap.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/net/tcp_mmap.c b/tools/testing/selftests/net/tcp_mmap.c index ca2618f3e7a12ab6863665f465dea2e8d469131b..00f837c9bc6c4549c19dfc27aa2d08c454ea169e 100644 --- a/tools/testing/selftests/net/tcp_mmap.c +++ b/tools/testing/selftests/net/tcp_mmap.c @@ -157,6 +157,7 @@ void *child_thread(void *arg) void *addr = NULL; double throughput; struct rusage ru; + size_t buffer_sz; int lu, fd; fd = (int)(unsigned long)arg; @@ -164,9 +165,9 @@ void *child_thread(void *arg) gettimeofday(&t0, NULL); fcntl(fd, F_SETFL, O_NDELAY); - buffer = malloc(chunk_size); - if (!buffer) { - perror("malloc"); + buffer = mmap_large_buffer(chunk_size, &buffer_sz); + if (buffer == (void *)-1) { + perror("mmap"); goto error; } if (zflg) { @@ -256,7 +257,7 @@ void *child_thread(void *arg) ru.ru_nvcsw); } error: - free(buffer); + munmap(buffer, buffer_sz); close(fd); if (zflg) munmap(raddr, chunk_size + map_align);