From patchwork Tue Apr 24 14:39:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Toshiaki Makita X-Patchwork-Id: 903541 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="gOzFE6al"; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 40VmGv4Y50z9ry1 for ; Wed, 25 Apr 2018 00:41:11 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752749AbeDXOlI (ORCPT ); Tue, 24 Apr 2018 10:41:08 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:40390 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754080AbeDXOjt (ORCPT ); Tue, 24 Apr 2018 10:39:49 -0400 Received: by mail-pg0-f65.google.com with SMTP id e9so11026548pgr.7 for ; Tue, 24 Apr 2018 07:39:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Nddq7PL2Ewq21g7hrQdsAx48Qo0uTMzk8Kl8YjGJTNU=; b=gOzFE6alWIoz7+cQzSBOg6Y+TS8KeC8iLQEqoqFlSqhV5hZcyakyRkXlds6RCKXHOM V8Fzy3lM6C9X/Itui7UV9vUqWlWVjED9SDdU299UkVDH8PDiPeL15J7X+xf8Lj5+zjfb HyK/gMtTQ8eYNFmt/HMOt4OiECpK7Pf6xBflCl/U2ZtwOwmCuWyaoF6wPoy8jKmUqR9N stGdK70Q9TbACJ0B34r3iOAXVMyilLpw3HvZiSVjIHL8nW0Pfwul1jQxbbiRoBeJoodb UPTg/v5GKB+buNLA75olAR65xFdFyRjaN8rIUMky/JEUyxP2e6h2x9OPa+JXumQm36FW jJtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Nddq7PL2Ewq21g7hrQdsAx48Qo0uTMzk8Kl8YjGJTNU=; b=JaZzT5yJHYiBFO2Yf+9bVg6ytcA+ZSmZZz2a6bI5ieDSxnYRvz+0dALY5LVtC2iqSq KG3nnIpFPanh8pLsPahJ1KgeRBFd+jzHMGSA1/EReVR9Xtm7o56uoHZBrAUbBcTHFXuX kK6BOKuJrou6pyzEw47MGdZC4lOhnWh98NRNr4DugnDw1rbvkvoIkZ+s/2Vqv3fiS3L/ uxUS/napKPMZ9DZNJHUkCtZjsbWTu3VS4xo8M+Sv+Tu3EDxv3FO96SR7IRmHbSuEACgh GuRPdCYFL5jBlurB20mdua4j1PWRX+P9nPjTKUCdFgRph6vmXi6TYo/wgGXjJD5uFjNs XRXA== X-Gm-Message-State: ALQs6tDiGDkdhT+8rzgewOdLticcT64hkgI1oNpF7anxKF3KoFPf0YNF +yvbGcFUX85KsqDJ/TDx6rv/3ksltJw= X-Google-Smtp-Source: AIpwx4+K/sigS3NGKb1ZWSy3AdDAlWSag50NygiFi/r/zDrzwnYR/ydyQ4bXYO43DXW2Xsk2M+XDUQ== X-Received: by 10.98.102.79 with SMTP id a76mr24097211pfc.162.1524580788953; Tue, 24 Apr 2018 07:39:48 -0700 (PDT) Received: from localhost.localdomain (i121-115-166-6.s42.a013.ap.plala.or.jp. [121.115.166.6]) by smtp.gmail.com with ESMTPSA id o64sm28179970pfb.62.2018.04.24.07.39.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 24 Apr 2018 07:39:48 -0700 (PDT) From: Toshiaki Makita To: netdev@vger.kernel.org Cc: Toshiaki Makita Subject: [PATCH RFC 5/9] veth: Handle xdp_frame in xdp napi ring Date: Tue, 24 Apr 2018 23:39:19 +0900 Message-Id: <20180424143923.26519-6-toshiaki.makita1@gmail.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180424143923.26519-1-toshiaki.makita1@gmail.com> References: <20180424143923.26519-1-toshiaki.makita1@gmail.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Toshiaki Makita This is preparation for XDP TX and ndo_xdp_xmit. Now the napi ring accepts both skb and xdp_frame. When xdp_frame is enqueued, skb will not be allocated until XDP program on veth returns PASS. This will speedup the XDP processing when ndo_xdp_xmit is implemented and xdp_frame is enqueued by the peer device. Signed-off-by: Toshiaki Makita --- drivers/net/veth.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 2 deletions(-) diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 452771f31c30..89c91c1c9935 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -27,6 +27,7 @@ #define DRV_NAME "veth" #define DRV_VERSION "1.0" +#define VETH_XDP_FLAG 0x1UL #define VETH_RING_SIZE 256 #define VETH_XDP_HEADROOM (XDP_PACKET_HEADROOM + NET_IP_ALIGN) @@ -48,6 +49,16 @@ struct veth_priv { struct xdp_rxq_info xdp_rxq; }; +static bool veth_is_xdp_frame(void *ptr) +{ + return (unsigned long)ptr & VETH_XDP_FLAG; +} + +static void *veth_ptr_to_xdp(void *ptr) +{ + return (void *)((unsigned long)ptr & ~VETH_XDP_FLAG); +} + /* * ethtool interface */ @@ -117,7 +128,14 @@ static void veth_ptr_free(void *ptr) { if (!ptr) return; - dev_kfree_skb_any(ptr); + + if (veth_is_xdp_frame(ptr)) { + struct xdp_frame *frame = veth_ptr_to_xdp(ptr); + + xdp_return_frame(frame); + } else { + dev_kfree_skb_any(ptr); + } } static void veth_xdp_flush(struct veth_priv *priv) @@ -263,6 +281,60 @@ static struct sk_buff *veth_build_skb(void *head, int headroom, int len, return skb; } +static struct sk_buff *veth_xdp_rcv_one(struct veth_priv *priv, + struct xdp_frame *frame) +{ + struct bpf_prog *xdp_prog; + unsigned int headroom; + struct sk_buff *skb; + int len, delta = 0; + + rcu_read_lock(); + xdp_prog = rcu_dereference(priv->xdp_prog); + if (xdp_prog) { + struct xdp_buff xdp; + u32 act; + + xdp.data_hard_start = frame->data - frame->headroom; + xdp.data = frame->data; + xdp.data_end = frame->data + frame->len; + xdp.data_meta = frame->data - frame->metasize; + xdp.rxq = &priv->xdp_rxq; + + act = bpf_prog_run_xdp(xdp_prog, &xdp); + + switch (act) { + case XDP_PASS: + delta = frame->data - xdp.data; + break; + default: + bpf_warn_invalid_xdp_action(act); + case XDP_ABORTED: + trace_xdp_exception(priv->dev, xdp_prog, act); + case XDP_DROP: + goto err_xdp; + } + } + rcu_read_unlock(); + + headroom = frame->data - delta - (void *)frame; + len = frame->len + delta; + skb = veth_build_skb(frame, headroom, len, 0); + if (!skb) { + xdp_return_frame(frame); + goto err; + } + + skb->protocol = eth_type_trans(skb, priv->dev); +err: + return skb; +err_xdp: + rcu_read_unlock(); + xdp_return_frame(frame); + + return NULL; +} + static struct sk_buff *veth_xdp_rcv_skb(struct veth_priv *priv, struct sk_buff *skb) { @@ -372,7 +444,10 @@ static int veth_xdp_rcv(struct veth_priv *priv, int budget) if (!ptr) break; - skb = veth_xdp_rcv_skb(priv, ptr); + if (veth_is_xdp_frame(ptr)) + skb = veth_xdp_rcv_one(priv, veth_ptr_to_xdp(ptr)); + else + skb = veth_xdp_rcv_skb(priv, ptr); if (skb) napi_gro_receive(&priv->xdp_napi, skb);