From patchwork Thu Sep 10 08:31:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Magnus Karlsson X-Patchwork-Id: 1361327 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Z2AL50nn; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BnBvn5bCKz9sTN for ; Thu, 10 Sep 2020 18:32:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730466AbgIJIcK (ORCPT ); Thu, 10 Sep 2020 04:32:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730522AbgIJIbX (ORCPT ); Thu, 10 Sep 2020 04:31:23 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47C71C061573; Thu, 10 Sep 2020 01:31:23 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id j7so563707plk.11; Thu, 10 Sep 2020 01:31:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7qt3/l6N2AZLqaqvzhij+GOrBrkyjnxL/cc0fShcUdk=; b=Z2AL50nn/FtLTyGy4R8nWtAT01imkaha6rroHgeaFmDYp/+B6tp/hZawK4cCI4+I0R 9EakqKXdEGBc5cv41G+qLBZ3lvIFomJD9axUcrcofb+EutCM4Lx6jDLBJmw+wZya6UJm LCWRE3N7qWRerpDiv9ykg5l0i+eeusSZfQSqJ1wz6mZGT0ORJCNhUSuJi4+ObPoRw9Pu gwFIXwPr8XKE157hRXFKWJ28wLCiCRQKccgKd2v3R11/rDQD2cj3Ef5itZUKUQHNpgcb Nn3HzUdhrKZ0OERL1bBen+sUSEy2y44q4lem9RmzHRhFYzRIWGUH1k9AY4TjXtMGlIth TV/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7qt3/l6N2AZLqaqvzhij+GOrBrkyjnxL/cc0fShcUdk=; b=irJKYMW3cO+HWN51WRZLZx5XLIAFX5gfA1DVkJp7RZSpuNXbXhm/6bGLAVS01EA+Lk UksYdzlD1b1faqsiCp10ZjdGiQ69NnTNncV9NTi3I3PF7clBd3dnZ/08c3VlHesIUV2j QXpalrwBaRhHqtaf4ylOloowbqY0k8IPuPgpzE1J+qi099Fy4X3flBfR945aHUWOjh4D WBXMPBYK9V0JcYLmhDEZXXtWEbaDxSx45dMxgPESk9pgJvpkjQ5dr8ohOHV80GhIhQG1 mOFUMrkiN6SB2Sc5m04tSBNCbVA1Uwkq9zwQalseeDSAER8ZUgSw0BcJ+xIYjXmGytw3 eWZg== X-Gm-Message-State: AOAM530gGSF7scOsm1m0iPjBVY0bkxKJjsVoVAJ6USkOSEpYVem0IVp5 MZGequ9S4GPukEbrKEe60Ls= X-Google-Smtp-Source: ABdhPJz297VamGf6+zuAL4AKKDvvUPi8m0yGDADgStOcR9lFJOGM4ZSI5pEZ4G3Adqkqzr+vENjKLg== X-Received: by 2002:a17:90a:4d47:: with SMTP id l7mr4297027pjh.30.1599726682871; Thu, 10 Sep 2020 01:31:22 -0700 (PDT) Received: from VM.ger.corp.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id c7sm5183438pfj.100.2020.09.10.01.31.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Sep 2020 01:31:22 -0700 (PDT) From: Magnus Karlsson To: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com Cc: bpf@vger.kernel.org Subject: [PATCH bpf-next 1/3] samples/bpf: fix one packet sending in xdpsock Date: Thu, 10 Sep 2020 10:31:04 +0200 Message-Id: <1599726666-8431-2-git-send-email-magnus.karlsson@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> References: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Magnus Karlsson Fix the sending of a single packet (or small burst) in xdpsock when executing in copy mode. Currently, the l2fwd application in xdpsock only transmits the packets after a batch of them has been received, which might be confusing if you only send one packet and expect that it is returned pronto. Fix this by calling sendto() more often and add a comment in the code that states that this can be optimized if needed. Signed-off-by: Magnus Karlsson Reported-by: Tirthendu Sarkar --- samples/bpf/xdpsock_user.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index 4cead34..b6175cb 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -897,6 +897,14 @@ static inline void complete_tx_l2fwd(struct xsk_socket_info *xsk, if (!xsk->outstanding_tx) return; + /* In copy mode, Tx is driven by a syscall so we need to use e.g. sendto() to + * really send the packets. In zero-copy mode we do not have to do this, since Tx + * is driven by the NAPI loop. So as an optimization, we do not have to call + * sendto() all the time in zero-copy mode for l2fwd. + */ + if (opt_xdp_bind_flags & XDP_COPY) + kick_tx(xsk); + ndescs = (xsk->outstanding_tx > opt_batch_size) ? opt_batch_size : xsk->outstanding_tx; From patchwork Thu Sep 10 08:31:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Magnus Karlsson X-Patchwork-Id: 1361326 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=Av7QYZFW; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BnBvg4Lzvz9sTN for ; Thu, 10 Sep 2020 18:32:15 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730240AbgIJIcO (ORCPT ); Thu, 10 Sep 2020 04:32:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730533AbgIJIb3 (ORCPT ); Thu, 10 Sep 2020 04:31:29 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21552C061757; Thu, 10 Sep 2020 01:31:26 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id kk9so2688012pjb.2; Thu, 10 Sep 2020 01:31:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DR5bYLJNAjyREK9wIlSLcrbAvH4kMtjbkkaPEja+364=; b=Av7QYZFWs/sh8ixaUfmwP+XDBgCJ97sokGpFvz4pfe9ZD/9MxvUCA+1wFiYKJLHHZX 42lGau2X9JXTvKOWvLIHlcMxviPOir054xQ6dl2N3/kYVrmMf0j9T+ZuFWzLSo/Y5tfB 6hhjiJB7zbeI2bn7Kpwtyje4T0MTJ9sdhLsL2ENHSNfNWuOvdlHFM4KRB7lohS9oKZQo sSfGthuNUV0ZaxLV7X3znDPrYblt+ta9gWzEA2eP+qzezFosrbAN0y6H2axvRCgkl9CS OX5Jj0t2pEcJSNddPNh3zEcaVbYnwLnJqzLMduHKvutl/19oD4aIoZ0Jkrm3HOD29IT/ p3EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DR5bYLJNAjyREK9wIlSLcrbAvH4kMtjbkkaPEja+364=; b=H1AkXB1aQI8t44hUOe4JGenk2FGSWPeEB+TFEB0wqc76/qC+7ZLYlplLK86ebPzdwA y7frQwAsPAAZZLRL6j/uGZkijNOIkgvAVZDuSz/MZW5iwgEH0mv3ulgIEnMwFT4ai8Ux c1VHRAtP/DlBRfV1IJy56jlIeyZnK4Se5kadp/vhoKJlk13F99ZJLfp4T+f3M7Axh1eO cJGG1VHgLjt21La2coku5WdseXKUeQ1x2Pd47Vd8afF6rnNjifE6CfEICy62sA6LrUKY D8WfJzVb5ykLdGc7H0AMyrlb4hJxVkNpnli+JFuWFU38JxCmidjAJ1qn8hcv3gIIoD/3 m8fQ== X-Gm-Message-State: AOAM533tq5XrrBItnhFyNIBElqYQO5UyRknlebS+2kC+GBHwiB95GC/Y 5/Rd80T2zxhmYg8Pp5TxRS4= X-Google-Smtp-Source: ABdhPJzqEoRjHVftj4VeNsfwfqHt6vWtZQdsxb6T42tZaB8OWKUt22czG8j0QsqDLgCJU/Rn3PD/nA== X-Received: by 2002:a17:90a:644b:: with SMTP id y11mr4438798pjm.13.1599726685702; Thu, 10 Sep 2020 01:31:25 -0700 (PDT) Received: from VM.ger.corp.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id c7sm5183438pfj.100.2020.09.10.01.31.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Sep 2020 01:31:25 -0700 (PDT) From: Magnus Karlsson To: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com Cc: bpf@vger.kernel.org Subject: [PATCH bpf-next 2/3] samples/bpf: fix possible deadlock in xdpsock Date: Thu, 10 Sep 2020 10:31:05 +0200 Message-Id: <1599726666-8431-3-git-send-email-magnus.karlsson@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> References: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Magnus Karlsson Fix a possible deadlock in the l2fwd application in xdpsock that can occur when there is no space in the Tx ring. There are two ways to get the kernel to consume entries in the Tx ring: calling sendto() to make it send packets and freeing entries from the completion ring, as the kernel will not send a packet if there is no space for it to add a completion entry in the completion ring. The Tx loop in l2fwd only used to call sendto(). This patches adds cleaning the completion ring in that loop. Signed-off-by: Magnus Karlsson --- samples/bpf/xdpsock_user.c | 1 + 1 file changed, 1 insertion(+) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index b6175cb..b60bf4e 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -1125,6 +1125,7 @@ static void l2fwd(struct xsk_socket_info *xsk, struct pollfd *fds) while (ret != rcvd) { if (ret < 0) exit_with_error(-ret); + complete_tx_l2fwd(xsk, fds); if (xsk_ring_prod__needs_wakeup(&xsk->tx)) kick_tx(xsk); ret = xsk_ring_prod__reserve(&xsk->tx, rcvd, &idx_tx); From patchwork Thu Sep 10 08:31:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Magnus Karlsson X-Patchwork-Id: 1361331 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=UHcE0TK6; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BnBvx3t8Zz9sTN for ; Thu, 10 Sep 2020 18:32:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730553AbgIJIc2 (ORCPT ); Thu, 10 Sep 2020 04:32:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730538AbgIJIb3 (ORCPT ); Thu, 10 Sep 2020 04:31:29 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7369C0613ED; Thu, 10 Sep 2020 01:31:28 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id o16so2670813pjr.2; Thu, 10 Sep 2020 01:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=sSNYJQoq7VIn+vtERBylFPcQxQX6LuWyosQXlSFU3cQ=; b=UHcE0TK6XW0LUH+yA9/uVjm3cxevVJQZX3lUOg1MBIyA9raAg0xLnbcs8J3VQDy5xs 3pdHdQuPgGTt+7dXNFyL3pgB5Oin7dT3RhoPtZXZtoQiePs3LaoEDedgQgI7NDcMB1Pc gY0cndgg9+n6pGnoJ84OdmJwt55+iWcfV83TcsVsTujfdSE69tlFJcdDH/bmYP3SATLo 3uPKm4MoW6LqvVCDRUw8KeKYr7TuuMPBMPF1RwaszfS2rh2GLcKSAY59iNmq3FnCJdF1 SCKhIhHUcYyn/6Z0zSUIZ9EWRmakE9kQIiRVKMvzib2IofPZ/NB5DkNhlRzR4hoR8zQ7 xk3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=sSNYJQoq7VIn+vtERBylFPcQxQX6LuWyosQXlSFU3cQ=; b=LBntxdedNatSoTO3Qtoz4ZuuCdCAZcs3yOoMsB+t3VANwkyokSLciIbB2sxBX0zTiv pL3ZIMskOy0MnCgPy+keV3/SSq3Nw5olEjml1WjJhGJfY36Pi44VMcksvpzLVslY4PN3 YtM5iEcjEWVMY4TJ+jen4dJOKT1y1FL2DzosjAALoaQB51ws9vCaVmjEnIBhLcXviMC6 8G1DZyju5/JdDR5oxqqqVGXZyNg66Pqgws6cKuyWsfRipmHVOS3puNDuSA5whsZdg2Wg INHEuqEx3kZmwWi949O0oMXFr2A5HIgFzGayHCuoPLRnVBWBWSzTC5WK35zGWWdTZP/2 RQsw== X-Gm-Message-State: AOAM533BHUrT+ZP9VFtclDekiZEPwXxZtexqULBUwU/IiQ7yiC2VZf3R EnlvYwnCKKlrPGRKZGHCCM0= X-Google-Smtp-Source: ABdhPJxXAfEgM9tV8AlLz+vaUk2lc/oSuiwnQ6x9FyFJ93oagdyTA1EpdzmQ7jg/MS4Pcgc61E2aaQ== X-Received: by 2002:a17:902:9303:: with SMTP id bc3mr4616620plb.170.1599726688316; Thu, 10 Sep 2020 01:31:28 -0700 (PDT) Received: from VM.ger.corp.intel.com ([192.55.54.40]) by smtp.gmail.com with ESMTPSA id c7sm5183438pfj.100.2020.09.10.01.31.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Sep 2020 01:31:27 -0700 (PDT) From: Magnus Karlsson To: magnus.karlsson@intel.com, bjorn.topel@intel.com, ast@kernel.org, daniel@iogearbox.net, netdev@vger.kernel.org, jonathan.lemon@gmail.com Cc: bpf@vger.kernel.org Subject: [PATCH bpf-next 3/3] samples/bpf: add quiet option to xdpsock Date: Thu, 10 Sep 2020 10:31:06 +0200 Message-Id: <1599726666-8431-4-git-send-email-magnus.karlsson@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> References: <1599726666-8431-1-git-send-email-magnus.karlsson@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Magnus Karlsson Add a quiet option (-Q) that disables the statistics print outs of xdpsock. This is good to have when measuring 0% loss rate performance as it will be quite terrible if the application uses printfs. Signed-off-by: Magnus Karlsson --- samples/bpf/xdpsock_user.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/samples/bpf/xdpsock_user.c b/samples/bpf/xdpsock_user.c index b60bf4e..b220173 100644 --- a/samples/bpf/xdpsock_user.c +++ b/samples/bpf/xdpsock_user.c @@ -78,6 +78,7 @@ static int opt_pkt_count; static u16 opt_pkt_size = MIN_PKT_SIZE; static u32 opt_pkt_fill_pattern = 0x12345678; static bool opt_extra_stats; +static bool opt_quiet; static int opt_poll; static int opt_interval = 1; static u32 opt_xdp_bind_flags = XDP_USE_NEED_WAKEUP; @@ -718,6 +719,7 @@ static struct option long_options[] = { {"tx-pkt-size", required_argument, 0, 's'}, {"tx-pkt-pattern", required_argument, 0, 'P'}, {"extra-stats", no_argument, 0, 'x'}, + {"quiet", no_argument, 0, 'Q'}, {0, 0, 0, 0} }; @@ -753,6 +755,7 @@ static void usage(const char *prog) " Min size: %d, Max size %d.\n" " -P, --tx-pkt-pattern=nPacket fill pattern. Default: 0x%x\n" " -x, --extra-stats Display extra statistics.\n" + " -Q, --quiet Do not display any stats.\n" "\n"; fprintf(stderr, str, prog, XSK_UMEM__DEFAULT_FRAME_SIZE, opt_batch_size, MIN_PKT_SIZE, MIN_PKT_SIZE, @@ -768,7 +771,7 @@ static void parse_command_line(int argc, char **argv) opterr = 0; for (;;) { - c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:x", + c = getopt_long(argc, argv, "Frtli:q:pSNn:czf:muMd:b:C:s:P:xQ", long_options, &option_index); if (c == -1) break; @@ -852,6 +855,9 @@ static void parse_command_line(int argc, char **argv) case 'x': opt_extra_stats = 1; break; + case 'Q': + opt_quiet = 1; + break; default: usage(basename(argv[0])); } @@ -1286,9 +1292,11 @@ int main(int argc, char **argv) setlocale(LC_ALL, ""); - ret = pthread_create(&pt, NULL, poller, NULL); - if (ret) - exit_with_error(ret); + if (!opt_quiet) { + ret = pthread_create(&pt, NULL, poller, NULL); + if (ret) + exit_with_error(ret); + } prev_time = get_nsecs(); start_time = prev_time; @@ -1302,7 +1310,8 @@ int main(int argc, char **argv) benchmark_done = true; - pthread_join(pt, NULL); + if (!opt_quiet) + pthread_join(pt, NULL); xdpsock_cleanup();