From patchwork Thu Jul 26 14:40:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 949757 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="lh8eQi2H"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 41bvt10pHwz9rxx for ; Fri, 27 Jul 2018 00:41:13 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731230AbeGZP6V (ORCPT ); Thu, 26 Jul 2018 11:58:21 -0400 Received: from mail-pg1-f193.google.com ([209.85.215.193]:40443 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729385AbeGZP6V (ORCPT ); Thu, 26 Jul 2018 11:58:21 -0400 Received: by mail-pg1-f193.google.com with SMTP id x5-v6so1297142pgp.7 for ; Thu, 26 Jul 2018 07:41:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ReKgz+Ex2WfpgItxlZNClL0L5GsAfUBRmfmJ5Joag1s=; b=lh8eQi2H1aV3Ls6cKwg6XjRjWxTz89Ij+8DFhO6CZkLdctMGMgKjS6sfk67UVw/yNX kzfY9qHI2CMiu2LqPQF0dU8S2saNsYERO1CTX1D5u6cNqlUlh+lpdXXJLrtKO2iNU1tU Ou5UhVqjdTWeb06Dl9GBLzPkWiASzDZqyAoyqdYzBSKy7l6VyJjj19avK0F2GsVyTLBb bYC3A2IGl2Yoy4D4f6FVr8rHdWzxYPGPhQSbLyPiAZ1ztHFuGfVtTEteQzKsWqCFNN8x Dt3REzwE+5Mb/xbmfH2SoMJKDviBQD8sxmuz9gc174VgYH4GM4nKwz+3uolRl9edWu5J 3A9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ReKgz+Ex2WfpgItxlZNClL0L5GsAfUBRmfmJ5Joag1s=; b=jQYww0EAidgiVGfvjrDWAavfIvjiRD8nx77vSsq/txqLRMcZjy/7z3+YMk/hdwHHsb oUKsNae4V4ovZRPQ9qkquHWDa7PpJ58slbFgDR8yvQKKfA/Mgdti01oLdw2uotVx/GwT u2DtCZDuzoRBG0Oa2HANVive/Yj7d6Obl33Ecsy2jb1DBCXtdTKNMHJtKSutS//Smnnb E+H7+kYOSYFDipDtv5lRjGd6os+tJqcIDgco01AufWnh5WCs/WujKHTNnr+YOFpnLbxA AIY1hD/ltwax0APIIbdvOEKvQ8hYA5dxGZTmG0+MB/qZQZNOXhmvOSCs/h9EQrFYyw0h bcHw== X-Gm-Message-State: AOUpUlE0NKaLZ2o8VTOKUq3rMmkUbf/gaTKRCrGv2wxDuXX1VvXJnNGs Ma/AdtNsWTPTNJdKi6QvzzjYkrqS X-Google-Smtp-Source: AAOMgpeupvSaZq4Pk/0l3vLlQ22JAUmtV7uWAB7EACshM7BW4vQJ3eiUOqaEfoecftVfNx1yHAVFUg== X-Received: by 2002:a65:4107:: with SMTP id w7-v6mr2171681pgp.302.1532616070140; Thu, 26 Jul 2018 07:41:10 -0700 (PDT) Received: from localhost.localdomain (i153-145-22-9.s42.a013.ap.plala.or.jp. [153.145.22.9]) by smtp.gmail.com with ESMTPSA id p3-v6sm2649982pfo.130.2018.07.26.07.41.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 26 Jul 2018 07:41:09 -0700 (PDT) From: Toshiaki Makita To: netdev@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann Cc: Toshiaki Makita , Jesper Dangaard Brouer , Jakub Kicinski Subject: [PATCH v5 bpf-next 4/9] veth: Handle xdp_frames in xdp napi ring Date: Thu, 26 Jul 2018 23:40:27 +0900 Message-Id: <20180726144032.2116-5-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180726144032.2116-1-toshiaki.makita1@gmail.com> References: <20180726144032.2116-1-toshiaki.makita1@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Toshiaki Makita This is preparation for XDP TX and ndo_xdp_xmit. This allows napi handler to handle xdp_frames through xdp ring as well as sk_buff. v3: - Revert v2 change around rings and use a flag to differentiate skb and xdp_frame, since bulk skb xmit makes little performance difference for now. v2: - Use another ring instead of using flag to differentiate skb and xdp_frame. This approach makes bulk skb transmit possible in veth_xmit later. - Clear xdp_frame feilds in skb->head. - Implement adjust_tail. Signed-off-by: Toshiaki Makita Acked-by: John Fastabend --- drivers/net/veth.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 82 insertions(+), 5 deletions(-) diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 1b4006d3df32..ef22d991f678 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -22,12 +22,12 @@ #include #include #include -#include #include #define DRV_NAME "veth" #define DRV_VERSION "1.0" +#define VETH_XDP_FLAG BIT(0) #define VETH_RING_SIZE 256 #define VETH_XDP_HEADROOM (XDP_PACKET_HEADROOM + NET_IP_ALIGN) @@ -115,6 +115,24 @@ static const struct ethtool_ops veth_ethtool_ops = { /* general routines */ +static bool veth_is_xdp_frame(void *ptr) +{ + return (unsigned long)ptr & VETH_XDP_FLAG; +} + +static void *veth_ptr_to_xdp(void *ptr) +{ + return (void *)((unsigned long)ptr & ~VETH_XDP_FLAG); +} + +static void veth_ptr_free(void *ptr) +{ + if (veth_is_xdp_frame(ptr)) + xdp_return_frame(veth_ptr_to_xdp(ptr)); + else + kfree_skb(ptr); +} + static void __veth_xdp_flush(struct veth_priv *priv) { /* Write ptr_ring before reading rx_notify_masked */ @@ -249,6 +267,61 @@ static struct sk_buff *veth_build_skb(void *head, int headroom, int len, return skb; } +static struct sk_buff *veth_xdp_rcv_one(struct veth_priv *priv, + struct xdp_frame *frame) +{ + int len = frame->len, delta = 0; + struct bpf_prog *xdp_prog; + unsigned int headroom; + struct sk_buff *skb; + + rcu_read_lock(); + xdp_prog = rcu_dereference(priv->xdp_prog); + if (likely(xdp_prog)) { + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = frame->data - frame->headroom; + xdp.data = frame->data; + xdp.data_end = frame->data + frame->len; + xdp.data_meta = frame->data - frame->metasize; + xdp.rxq = &priv->xdp_rxq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + + switch (act) { + case XDP_PASS: + delta = frame->data - xdp.data; + len = xdp.data_end - xdp.data; + break; + default: + bpf_warn_invalid_xdp_action(act); + case XDP_ABORTED: + trace_xdp_exception(priv->dev, xdp_prog, act); + case XDP_DROP: + goto err_xdp; + } + } + rcu_read_unlock(); + + headroom = frame->data - delta - (void *)frame; + skb = veth_build_skb(frame, headroom, len, 0); + if (!skb) { + xdp_return_frame(frame); + goto err; + } + + memset(frame, 0, sizeof(*frame)); + skb->protocol = eth_type_trans(skb, priv->dev); +err: + return skb; +err_xdp: + rcu_read_unlock(); + xdp_return_frame(frame); + + return NULL; +} + static struct sk_buff *veth_xdp_rcv_skb(struct veth_priv *priv, struct sk_buff *skb) { @@ -358,12 +431,16 @@ static int veth_xdp_rcv(struct veth_priv *priv, int budget) int i, done = 0; for (i = 0; i < budget; i++) { - struct sk_buff *skb = __ptr_ring_consume(&priv->xdp_ring); + void *ptr = __ptr_ring_consume(&priv->xdp_ring); + struct sk_buff *skb; - if (!skb) + if (!ptr) break; - skb = veth_xdp_rcv_skb(priv, skb); + if (veth_is_xdp_frame(ptr)) + skb = veth_xdp_rcv_one(priv, veth_ptr_to_xdp(ptr)); + else + skb = veth_xdp_rcv_skb(priv, ptr); if (skb) napi_gro_receive(&priv->xdp_napi, skb); @@ -416,7 +493,7 @@ static void veth_napi_del(struct net_device *dev) napi_disable(&priv->xdp_napi); netif_napi_del(&priv->xdp_napi); priv->rx_notify_masked = false; - ptr_ring_cleanup(&priv->xdp_ring, __skb_array_destroy_skb); + ptr_ring_cleanup(&priv->xdp_ring, veth_ptr_free); } static int veth_enable_xdp(struct net_device *dev)