From patchwork Mon Jul 11 21:29:48 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brenden Blanco X-Patchwork-Id: 647098 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3rpJD649Mtz9sXx for ; Tue, 12 Jul 2016 07:30:30 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=plumgrid-com.20150623.gappssmtp.com header.i=@plumgrid-com.20150623.gappssmtp.com header.b=mr1GmLQK; dkim-atps=neutral Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753162AbcGKVaN (ORCPT ); Mon, 11 Jul 2016 17:30:13 -0400 Received: from mail-pa0-f45.google.com ([209.85.220.45]:33223 "EHLO mail-pa0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752219AbcGKVaK (ORCPT ); Mon, 11 Jul 2016 17:30:10 -0400 Received: by mail-pa0-f45.google.com with SMTP id ks6so13356025pab.0 for ; Mon, 11 Jul 2016 14:30:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plumgrid-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5/gbThSxA0lN65Dvz0C2kF6qvFuNv1Sk4L+C9ua9WKc=; b=mr1GmLQK+87EPdAJz3PdeObrmZueAhgoZ2+E7TgOCmQs8XFs50l0i4Wevxl0CKTx48 S5WyXJ64Qx9Ezh60w4cZDgnQq+TQTXXO70MiNVzvCcfSg9GmJOxI2NILB5QyGOq6S6Ad CsIHUjVYbhgFzQA8R1HwbZaIKXEVkuFimHFEHMW3AjCYdAqHdZdeVbp6clcCzH1RcRly 3cCyA99c8UUEJJpvAx4QZB5XwrF5CmRfWlDpSWRotko7RcPCGV8sZkH6BxF4nTqK5tay 5hvVuotm4ijOPdB2PtWW9dhrbYYSC55zOXcMIg0TORGQmPOl/Ogxmz0IIiHUmQXoCdHg TePw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5/gbThSxA0lN65Dvz0C2kF6qvFuNv1Sk4L+C9ua9WKc=; b=m3P8twRH+SBafh6zaWSlKl/9UJwQjZlIYPbnZZoy0uKbb6AO8BxhqXQrZ5f3NDgfT9 f3CxBN4Tym0SVbjbhDhPeAWNC9LlyftKHy/4bLijukKDAxfe43FvpPrddMOGt0dyJ/cS oy30ZEXKwqanRv602uWPUc2TAxvQcLNVINPbYwuHOKuJqgxq683/n2sNlUhJ9FrCMSzQ z2byaxbXXr6RTGfuvthFam2IVPlelKoMke73aAQ/RozH5Js2FBqlPrTBcYzgS/enA/hv ytdulMFv9KbKO6lH/SPky4MBZTTW6F4SxCojPGEaqIgVMEvHfB1Sy++QiCj/IlI5fU3Z tgpg== X-Gm-Message-State: ALyK8tJvOGPHBGBcmvalg5oWBUtYFMb0TFT2y9vvnyXRLzxqZH7sy+5hj8ABQXaeNmlHAzLo X-Received: by 10.66.144.229 with SMTP id sp5mr38254893pab.97.1468272609067; Mon, 11 Jul 2016 14:30:09 -0700 (PDT) Received: from iovisor-test1.plumgrid.com ([12.97.19.201]) by smtp.gmail.com with ESMTPSA id w63sm6722150pfi.9.2016.07.11.14.30.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 11 Jul 2016 14:30:08 -0700 (PDT) From: Brenden Blanco To: davem@davemloft.net, netdev@vger.kernel.org Cc: Brenden Blanco , Jamal Hadi Salim , Saeed Mahameed , Martin KaFai Lau , Jesper Dangaard Brouer , Ari Saha , Alexei Starovoitov , Or Gerlitz , john.fastabend@gmail.com, hannes@stressinduktion.org, Thomas Graf , Tom Herbert , Daniel Borkmann Subject: [PATCH v7 01/11] bpf: add XDP prog type for early driver filter Date: Mon, 11 Jul 2016 14:29:48 -0700 Message-Id: <1468272598-21390-2-git-send-email-bblanco@plumgrid.com> X-Mailer: git-send-email 2.8.2 In-Reply-To: <1468272598-21390-1-git-send-email-bblanco@plumgrid.com> References: <1468272598-21390-1-git-send-email-bblanco@plumgrid.com> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a new bpf prog type that is intended to run in early stages of the packet rx path. Only minimal packet metadata will be available, hence a new context type, struct xdp_md, is exposed to userspace. So far only expose the packet start and end pointers, and only in read mode. An XDP program must return one of the well known enum values, all other return codes are reserved for future use. Unfortunately, this restriction is hard to enforce at verification time, so take the approach of warning at runtime when such programs are encountered. Out of bounds return codes should alias to XDP_ABORTED. Signed-off-by: Brenden Blanco --- include/linux/filter.h | 18 ++++++++++ include/uapi/linux/bpf.h | 20 +++++++++++ kernel/bpf/verifier.c | 1 + net/core/filter.c | 91 ++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 130 insertions(+) diff --git a/include/linux/filter.h b/include/linux/filter.h index 6fc31ef..522dbc9 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -368,6 +368,11 @@ struct bpf_skb_data_end { void *data_end; }; +struct xdp_buff { + void *data; + void *data_end; +}; + /* compute the linear packet data range [data, data_end) which * will be accessed by cls_bpf and act_bpf programs */ @@ -429,6 +434,18 @@ static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog, return BPF_PROG_RUN(prog, skb); } +static inline u32 bpf_prog_run_xdp(const struct bpf_prog *prog, + struct xdp_buff *xdp) +{ + u32 ret; + + rcu_read_lock(); + ret = BPF_PROG_RUN(prog, (void *)xdp); + rcu_read_unlock(); + + return ret; +} + static inline unsigned int bpf_prog_size(unsigned int proglen) { return max(sizeof(struct bpf_prog), @@ -509,6 +526,7 @@ bool bpf_helper_changes_skb_data(void *func); struct bpf_prog *bpf_patch_insn_single(struct bpf_prog *prog, u32 off, const struct bpf_insn *patch, u32 len); +void bpf_warn_invalid_xdp_action(int act); #ifdef CONFIG_BPF_JIT extern int bpf_jit_enable; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 262a7e8..4282d44 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -94,6 +94,7 @@ enum bpf_prog_type { BPF_PROG_TYPE_SCHED_CLS, BPF_PROG_TYPE_SCHED_ACT, BPF_PROG_TYPE_TRACEPOINT, + BPF_PROG_TYPE_XDP, }; #define BPF_PSEUDO_MAP_FD 1 @@ -437,4 +438,23 @@ struct bpf_tunnel_key { __u32 tunnel_label; }; +/* User return codes for XDP prog type. + * A valid XDP program must return one of these defined values. All other + * return codes are reserved for future use. Unknown return codes will result + * in packet drop. + */ +enum xdp_action { + XDP_ABORTED = 0, + XDP_DROP, + XDP_PASS, +}; + +/* user accessible metadata for XDP packet hook + * new fields must be added to the end of this structure + */ +struct xdp_md { + __u32 data; + __u32 data_end; +}; + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e206c21..a8d67d0 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -713,6 +713,7 @@ static int check_ptr_alignment(struct verifier_env *env, struct reg_state *reg, switch (env->prog->type) { case BPF_PROG_TYPE_SCHED_CLS: case BPF_PROG_TYPE_SCHED_ACT: + case BPF_PROG_TYPE_XDP: break; default: verbose("verifier is misconfigured\n"); diff --git a/net/core/filter.c b/net/core/filter.c index 10c4a2f..3c993ac 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2369,6 +2369,12 @@ tc_cls_act_func_proto(enum bpf_func_id func_id) } } +static const struct bpf_func_proto * +xdp_func_proto(enum bpf_func_id func_id) +{ + return sk_filter_func_proto(func_id); +} + static bool __is_valid_access(int off, int size, enum bpf_access_type type) { if (off < 0 || off >= sizeof(struct __sk_buff)) @@ -2436,6 +2442,56 @@ static bool tc_cls_act_is_valid_access(int off, int size, return __is_valid_access(off, size, type); } +static bool __is_valid_xdp_access(int off, int size, + enum bpf_access_type type) +{ + if (off < 0 || off >= sizeof(struct xdp_md)) + return false; + if (off % size != 0) + return false; + if (size != 4) + return false; + + return true; +} + +static bool xdp_is_valid_access(int off, int size, + enum bpf_access_type type, + enum bpf_reg_type *reg_type) +{ + if (type == BPF_WRITE) + return false; + + switch (off) { + case offsetof(struct xdp_md, data): + *reg_type = PTR_TO_PACKET; + break; + case offsetof(struct xdp_md, data_end): + *reg_type = PTR_TO_PACKET_END; + break; + } + + return __is_valid_xdp_access(off, size, type); +} + +void bpf_warn_invalid_xdp_action(int act) +{ + WARN_ONCE(1, "\n" + "*****************************************************\n" + "** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE **\n" + "** **\n" + "** XDP program returned unknown value %-10u **\n" + "** **\n" + "** XDP programs must return a well-known return **\n" + "** value. Invalid return values will result in **\n" + "** packet loss. **\n" + "** **\n" + "** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE **\n" + "*****************************************************\n", + act); +} +EXPORT_SYMBOL_GPL(bpf_warn_invalid_xdp_action); + static u32 bpf_net_convert_ctx_access(enum bpf_access_type type, int dst_reg, int src_reg, int ctx_off, struct bpf_insn *insn_buf, @@ -2587,6 +2643,29 @@ static u32 bpf_net_convert_ctx_access(enum bpf_access_type type, int dst_reg, return insn - insn_buf; } +static u32 xdp_convert_ctx_access(enum bpf_access_type type, int dst_reg, + int src_reg, int ctx_off, + struct bpf_insn *insn_buf, + struct bpf_prog *prog) +{ + struct bpf_insn *insn = insn_buf; + + switch (ctx_off) { + case offsetof(struct xdp_md, data): + *insn++ = BPF_LDX_MEM(bytes_to_bpf_size(FIELD_SIZEOF(struct xdp_buff, data)), + dst_reg, src_reg, + offsetof(struct xdp_buff, data)); + break; + case offsetof(struct xdp_md, data_end): + *insn++ = BPF_LDX_MEM(bytes_to_bpf_size(FIELD_SIZEOF(struct xdp_buff, data_end)), + dst_reg, src_reg, + offsetof(struct xdp_buff, data_end)); + break; + } + + return insn - insn_buf; +} + static const struct bpf_verifier_ops sk_filter_ops = { .get_func_proto = sk_filter_func_proto, .is_valid_access = sk_filter_is_valid_access, @@ -2599,6 +2678,12 @@ static const struct bpf_verifier_ops tc_cls_act_ops = { .convert_ctx_access = bpf_net_convert_ctx_access, }; +static const struct bpf_verifier_ops xdp_ops = { + .get_func_proto = xdp_func_proto, + .is_valid_access = xdp_is_valid_access, + .convert_ctx_access = xdp_convert_ctx_access, +}; + static struct bpf_prog_type_list sk_filter_type __read_mostly = { .ops = &sk_filter_ops, .type = BPF_PROG_TYPE_SOCKET_FILTER, @@ -2614,11 +2699,17 @@ static struct bpf_prog_type_list sched_act_type __read_mostly = { .type = BPF_PROG_TYPE_SCHED_ACT, }; +static struct bpf_prog_type_list xdp_type __read_mostly = { + .ops = &xdp_ops, + .type = BPF_PROG_TYPE_XDP, +}; + static int __init register_sk_filter_ops(void) { bpf_register_prog_type(&sk_filter_type); bpf_register_prog_type(&sched_cls_type); bpf_register_prog_type(&sched_act_type); + bpf_register_prog_type(&xdp_type); return 0; }