From patchwork Sat Apr 2 01:21:57 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brenden Blanco X-Patchwork-Id: 605375 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3qcL8L0rHJz9sDG for ; Sat, 2 Apr 2016 12:22:26 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=plumgrid-com.20150623.gappssmtp.com header.i=@plumgrid-com.20150623.gappssmtp.com header.b=fDonN+uY; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932585AbcDBBWV (ORCPT ); Fri, 1 Apr 2016 21:22:21 -0400 Received: from mail-pf0-f170.google.com ([209.85.192.170]:36436 "EHLO mail-pf0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756792AbcDBBWQ (ORCPT ); Fri, 1 Apr 2016 21:22:16 -0400 Received: by mail-pf0-f170.google.com with SMTP id e128so81003032pfe.3 for ; Fri, 01 Apr 2016 18:22:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plumgrid-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1Dpr2hSXAu//mr8/yxbSvy2osGWKwDmIxdlCzJR9Ao4=; b=fDonN+uYwjILGVwHpIRRpOyTT8co+hhMEBgZP2FNhgZKXFxxlJafHFLroRbrnyfsGb EirpI5JtUnf9Nuy9Fbyop7CanKYnY20N5sREPWd/8yHkUyur4lW4sUK1IXiLbuPt5dXV 0l1TZ3NCNOHi6hehqogBh8bhb0oTiosXdpPCcACiNqg8pZBXlqlJOho7/aIO0yTk9/jm c9qFBXo6t3ggNVR/NHACHh5owYkKujSrqT8wtNVwHuI7Qs4CftGJTyX9Q1ZAfBrvA+G9 ErYb+j2Fw25WGqHQqNkAtUfupJ+PEH7fXA1JjPgOPkHWa+rGxjENzf1J8pBsJiMAr3gP RUqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1Dpr2hSXAu//mr8/yxbSvy2osGWKwDmIxdlCzJR9Ao4=; b=eNHDvXUOovOyhJC350SU1xycSKnfbjX7xbvmMAHswCHgzVpkrdrjxmv/Ocagz4IDX9 X4BjgMCA6jRCqXLPEkey5vryi1wQnkAo8BqlAu/kcDOWz1GePJJoOkHqMMs6B4eCdwW7 +LyxTo6nlhUNYTMadafhEE5knmlw+HQGY9A7iTBGkmAW2nkFo8szfwx0cYm+I3fFXzal J6LTQCrpC24gVc20o/aoejvXihKvtjf6ChGQejNY1AXY4SpqxewCJl1SJ0MB4bydsljd rWx5t1Tf/8lBzSlRy7dsnYrP8xX4bed+zFDZ+FG9D2Q1wKKi62j8u947+ErGTd7iGgRC jI9Q== X-Gm-Message-State: AD7BkJIm6KWxXpiSTDZc7VfqHuZyp8c6cvoGcxxFcDWuQrHAdhgU1mJxd8B1fAqcOdb5FMkM X-Received: by 10.98.79.205 with SMTP id f74mr2784106pfj.68.1459560135168; Fri, 01 Apr 2016 18:22:15 -0700 (PDT) Received: from iovisor-test1.plumgrid.com ([12.97.19.201]) by smtp.gmail.com with ESMTPSA id kl14sm24742480pab.23.2016.04.01.18.22.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 01 Apr 2016 18:22:14 -0700 (PDT) From: Brenden Blanco To: davem@davemloft.net Cc: Brenden Blanco , netdev@vger.kernel.org, tom@herbertland.com, alexei.starovoitov@gmail.com, gerlitz@mellanox.com, daniel@iogearbox.net, john.fastabend@gmail.com, brouer@redhat.com Subject: [RFC PATCH 4/5] mlx4: add support for fast rx drop bpf program Date: Fri, 1 Apr 2016 18:21:57 -0700 Message-Id: <1459560118-5582-5-git-send-email-bblanco@plumgrid.com> X-Mailer: git-send-email 2.8.0 In-Reply-To: <1459560118-5582-1-git-send-email-bblanco@plumgrid.com> References: <1459560118-5582-1-git-send-email-bblanco@plumgrid.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add support for the BPF_PROG_TYPE_PHYS_DEV hook in mlx4 driver. Since bpf programs require a skb context to navigate the packet, build a percpu fake skb with the minimal fields. This avoids the costly allocation for packets that end up being dropped. Since mlx4 is so far the only user of this pseudo skb, the build function is defined locally. Signed-off-by: Brenden Blanco --- drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 61 ++++++++++++++++++++++++++ drivers/net/ethernet/mellanox/mlx4/en_rx.c | 18 ++++++++ drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 2 + 3 files changed, 81 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index b4b258c..89ca787 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -31,6 +31,7 @@ * */ +#include #include #include #include @@ -1966,6 +1967,9 @@ void mlx4_en_free_resources(struct mlx4_en_priv *priv) mlx4_en_destroy_cq(priv, &priv->rx_cq[i]); } + if (priv->prog) + bpf_prog_put(priv->prog); + } int mlx4_en_alloc_resources(struct mlx4_en_priv *priv) @@ -2456,6 +2460,61 @@ static int mlx4_en_set_tx_maxrate(struct net_device *dev, int queue_index, u32 m return err; } +static DEFINE_PER_CPU(struct sk_buff, percpu_pseudo_skb); + +static void build_pseudo_skb_for_bpf(struct sk_buff *skb, void *data, + unsigned int length) +{ + /* data_len is intentionally not set here so that skb_is_nonlinear() + * returns false + */ + + skb->len = length; + skb->head = data; + skb->data = data; +} + +int mlx4_call_bpf(struct bpf_prog *prog, void *data, unsigned int length) +{ + struct sk_buff *skb = this_cpu_ptr(&percpu_pseudo_skb); + int ret; + + build_pseudo_skb_for_bpf(skb, data, length); + + rcu_read_lock(); + ret = BPF_PROG_RUN(prog, (void *)skb); + rcu_read_unlock(); + + return ret; +} + +static int mlx4_bpf_set(struct net_device *dev, int fd) +{ + struct mlx4_en_priv *priv = netdev_priv(dev); + struct bpf_prog *prog = NULL, *old_prog; + + if (fd >= 0) { + prog = bpf_prog_get(fd); + if (IS_ERR(prog)) + return PTR_ERR(prog); + + if (prog->type != BPF_PROG_TYPE_PHYS_DEV) { + bpf_prog_put(prog); + return -EINVAL; + } + } + + old_prog = xchg(&priv->prog, prog); + if (old_prog) { + synchronize_net(); + bpf_prog_put(old_prog); + } + + priv->dev->bpf_valid = !!prog; + + return 0; +} + static const struct net_device_ops mlx4_netdev_ops = { .ndo_open = mlx4_en_open, .ndo_stop = mlx4_en_close, @@ -2486,6 +2545,7 @@ static const struct net_device_ops mlx4_netdev_ops = { .ndo_features_check = mlx4_en_features_check, #endif .ndo_set_tx_maxrate = mlx4_en_set_tx_maxrate, + .ndo_bpf_set = mlx4_bpf_set, }; static const struct net_device_ops mlx4_netdev_ops_master = { @@ -2524,6 +2584,7 @@ static const struct net_device_ops mlx4_netdev_ops_master = { .ndo_features_check = mlx4_en_features_check, #endif .ndo_set_tx_maxrate = mlx4_en_set_tx_maxrate, + .ndo_bpf_set = mlx4_bpf_set, }; struct mlx4_en_bond { diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index 86bcfe5..03fe005 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -748,6 +748,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud struct mlx4_en_rx_ring *ring = priv->rx_ring[cq->ring]; struct mlx4_en_rx_alloc *frags; struct mlx4_en_rx_desc *rx_desc; + struct bpf_prog *prog; struct sk_buff *skb; int index; int nr; @@ -764,6 +765,8 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud if (budget <= 0) return polled; + prog = READ_ONCE(priv->prog); + /* We assume a 1:1 mapping between CQEs and Rx descriptors, so Rx * descriptor offset can be deduced from the CQE index instead of * reading 'cqe->index' */ @@ -840,6 +843,21 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud l2_tunnel = (dev->hw_enc_features & NETIF_F_RXCSUM) && (cqe->vlan_my_qpn & cpu_to_be32(MLX4_CQE_L2_TUNNEL)); + /* A bpf program gets first chance to drop the packet. It may + * read bytes but not past the end of the frag. A non-zero + * return indicates packet should be dropped. + */ + if (prog) { + struct ethhdr *ethh; + + ethh = (struct ethhdr *)(page_address(frags[0].page) + + frags[0].page_offset); + if (mlx4_call_bpf(prog, ethh, length)) { + priv->stats.rx_dropped++; + goto next; + } + } + if (likely(dev->features & NETIF_F_RXCSUM)) { if (cqe->status & cpu_to_be16(MLX4_CQE_STATUS_TCP | MLX4_CQE_STATUS_UDP)) { diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h index d12ab6a..3d0fc89 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -568,6 +568,7 @@ struct mlx4_en_priv { struct hlist_head mac_hash[MLX4_EN_MAC_HASH_SIZE]; struct hwtstamp_config hwtstamp_config; u32 counter_index; + struct bpf_prog *prog; #ifdef CONFIG_MLX4_EN_DCB struct ieee_ets ets; @@ -682,6 +683,7 @@ int mlx4_en_create_drop_qp(struct mlx4_en_priv *priv); void mlx4_en_destroy_drop_qp(struct mlx4_en_priv *priv); int mlx4_en_free_tx_buf(struct net_device *dev, struct mlx4_en_tx_ring *ring); void mlx4_en_rx_irq(struct mlx4_cq *mcq); +int mlx4_call_bpf(struct bpf_prog *prog, void *data, unsigned int length); int mlx4_SET_MCAST_FLTR(struct mlx4_dev *dev, u8 port, u64 mac, u64 clear, u8 mode); int mlx4_SET_VLAN_FLTR(struct mlx4_dev *dev, struct mlx4_en_priv *priv);