From patchwork Wed Aug 26 13:19:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hangbin Liu X-Patchwork-Id: 1351942 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=E2+zyS8J; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bc61S1YLFz9sTR for ; Wed, 26 Aug 2020 23:20:44 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730186AbgHZNU3 (ORCPT ); Wed, 26 Aug 2020 09:20:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730174AbgHZNUX (ORCPT ); Wed, 26 Aug 2020 09:20:23 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D6643C061574; Wed, 26 Aug 2020 06:20:22 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id 17so964938pfw.9; Wed, 26 Aug 2020 06:20:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R1taOkAwFV1KVpE9MBDA2IKivPG7n34ZgstgOj1vxxA=; b=E2+zyS8Jv9b1Mh+Q5XLjwCFWW3IU2EKXsv4wUA4e886fza3UalhVDCQIvm1FmJMLl5 Hsa6Yu6RmKCpMnD26sMu/CFRM8yc0ab2hgdGXn+KLTa5dpy5eH+ZqNfNm8W4MLVDMRA3 dFda5ntxcppD69u4lCUeyLJ5QeDfJP1nskStvGZws8PDruI0a0tDsO5V0k4AYkHucjGV uQOaZo/9qnYp6vZeAouMsqXNzvhjf5bH62iPWudqMGO0gikF4IOtfonn9K0nGLTh8NKK gKBkEideZh+bVPlzljO4+7P7p95yosqmVml/xQ2HH0xWjWzBoHP8bkgdKuRICFk59Lz8 6kdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=R1taOkAwFV1KVpE9MBDA2IKivPG7n34ZgstgOj1vxxA=; b=Dfjt6pYOlQRPipjajpoqzgSstO+w4P0Y8q/tyyRm6s6zpgXQdlHU/6alZriL9dfp9t TY936PM6kCcMyXoVVdZ+TuYRKGOReuKHqNNXSZkOFp3or1JIjAxXt6K8SBZYiHVWxZ2r BLs7lnZuQwcnQZ/PA/OzG6kWq/hHPLELSIEWTDRS2kulvVovzh8/mnbpcrX/8GaHjbwz ekdkXvWGW/yZYvf8Y/zXRIc01aAKdoW7CPPJAw5UxWN9W2x8qS+b8p0XVehcTTpE/qIu m/iwCBSdihYYl3EQ3x/dMBP89+QQ1t+8HpehwGCeguOX+UVWo00Jc5bEUh/wJly0SGXt 0JqA== X-Gm-Message-State: AOAM530erw/QdioOC3YvtbJQdA/qcVabT+8otbh50IGCpVC8ehurqZKX 23H9i+22kta0bupLLuzwkV8Tqd9MdhB57Arr X-Google-Smtp-Source: ABdhPJzFR1y0C3Ug7v+8XdiMztCgCCmmlR8I4lHkSrEdnSSwljoOgMj29hoFTLyyyw3PYX86I/yJhA== X-Received: by 2002:a63:af01:: with SMTP id w1mr10712309pge.23.1598448022170; Wed, 26 Aug 2020 06:20:22 -0700 (PDT) Received: from dhcp-12-153.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id s129sm3131794pfb.39.2020.08.26.06.20.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2020 06:20:21 -0700 (PDT) From: Hangbin Liu To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jiri Benc , Jesper Dangaard Brouer , Eelco Chaudron , ast@kernel.org, Daniel Borkmann , Lorenzo Bianconi , David Ahern , Andrii Nakryiko B , Hangbin Liu Subject: [PATCHv9 bpf-next 1/5] bpf: add a new bpf argument type ARG_CONST_MAP_PTR_OR_NULL Date: Wed, 26 Aug 2020 21:19:58 +0800 Message-Id: <20200826132002.2808380-2-liuhangbin@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200826132002.2808380-1-liuhangbin@gmail.com> References: <20200715130816.2124232-1-liuhangbin@gmail.com> <20200826132002.2808380-1-liuhangbin@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a new bpf argument type ARG_CONST_MAP_PTR_OR_NULL which could be used when we want to allow NULL pointer for map parameter. The bpf helper need to take care and check if the map is NULL when use this type. Signed-off-by: Hangbin Liu --- v9: merge the patch from [1] in to this series. v1-v8: no this patch [1] https://lore.kernel.org/bpf/20200715070001.2048207-1-liuhangbin@gmail.com/ --- include/linux/bpf.h | 2 ++ kernel/bpf/verifier.c | 23 ++++++++++++++++------- 2 files changed, 18 insertions(+), 7 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index a6131d95e31e..cb40a1281ea2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -276,6 +276,7 @@ enum bpf_arg_type { ARG_PTR_TO_ALLOC_MEM, /* pointer to dynamically allocated memory */ ARG_PTR_TO_ALLOC_MEM_OR_NULL, /* pointer to dynamically allocated memory or NULL */ ARG_CONST_ALLOC_SIZE_OR_ZERO, /* number of allocated bytes requested */ + ARG_CONST_MAP_PTR_OR_NULL, /* const argument used as pointer to bpf_map or NULL */ }; /* type of values returned from helper functions */ @@ -369,6 +370,7 @@ enum bpf_reg_type { PTR_TO_RDONLY_BUF_OR_NULL, /* reg points to a readonly buffer or NULL */ PTR_TO_RDWR_BUF, /* reg points to a read/write buffer */ PTR_TO_RDWR_BUF_OR_NULL, /* reg points to a read/write buffer or NULL */ + CONST_PTR_TO_MAP_OR_NULL, /* reg points to struct bpf_map or NULL */ }; /* The information passed from prog-specific *_is_valid_access diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7e5908b83ec7..53a84335a8fd 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -411,7 +411,8 @@ static bool reg_type_may_be_null(enum bpf_reg_type type) type == PTR_TO_BTF_ID_OR_NULL || type == PTR_TO_MEM_OR_NULL || type == PTR_TO_RDONLY_BUF_OR_NULL || - type == PTR_TO_RDWR_BUF_OR_NULL; + type == PTR_TO_RDWR_BUF_OR_NULL || + type == CONST_PTR_TO_MAP_OR_NULL; } static bool reg_may_point_to_spin_lock(const struct bpf_reg_state *reg) @@ -427,7 +428,8 @@ static bool reg_type_may_be_refcounted_or_null(enum bpf_reg_type type) type == PTR_TO_TCP_SOCK || type == PTR_TO_TCP_SOCK_OR_NULL || type == PTR_TO_MEM || - type == PTR_TO_MEM_OR_NULL; + type == PTR_TO_MEM_OR_NULL || + type == CONST_PTR_TO_MAP_OR_NULL; } static bool arg_type_may_be_refcounted(enum bpf_arg_type type) @@ -509,6 +511,7 @@ static const char * const reg_type_str[] = { [PTR_TO_RDONLY_BUF_OR_NULL] = "rdonly_buf_or_null", [PTR_TO_RDWR_BUF] = "rdwr_buf", [PTR_TO_RDWR_BUF_OR_NULL] = "rdwr_buf_or_null", + [CONST_PTR_TO_MAP_OR_NULL] = "map_ptr_or_null", }; static char slot_type_char[] = { @@ -3957,9 +3960,13 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, expected_type = SCALAR_VALUE; if (type != expected_type) goto err_type; - } else if (arg_type == ARG_CONST_MAP_PTR) { + } else if (arg_type == ARG_CONST_MAP_PTR || + arg_type == ARG_CONST_MAP_PTR_OR_NULL) { expected_type = CONST_PTR_TO_MAP; - if (type != expected_type) + if (register_is_null(reg) && + arg_type == ARG_CONST_MAP_PTR_OR_NULL) + /* final test in check_stack_boundary() */; + else if (type != expected_type) goto err_type; } else if (arg_type == ARG_PTR_TO_CTX || arg_type == ARG_PTR_TO_CTX_OR_NULL) { @@ -4076,9 +4083,9 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return -EFAULT; } - if (arg_type == ARG_CONST_MAP_PTR) { - /* bpf_map_xxx(map_ptr) call: remember that map_ptr */ - meta->map_ptr = reg->map_ptr; + if (arg_type == ARG_CONST_MAP_PTR || + arg_type == ARG_CONST_MAP_PTR_OR_NULL) { + meta->map_ptr = register_is_null(reg) ? NULL : reg->map_ptr; } else if (arg_type == ARG_PTR_TO_MAP_KEY) { /* bpf_map_xxx(..., map_ptr, ..., key) call: * check that [key, key + map->key_size) are within @@ -6977,6 +6984,8 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state, reg->type = PTR_TO_RDONLY_BUF; } else if (reg->type == PTR_TO_RDWR_BUF_OR_NULL) { reg->type = PTR_TO_RDWR_BUF; + } else if (reg->type == CONST_PTR_TO_MAP_OR_NULL) { + reg->type = CONST_PTR_TO_MAP; } if (is_null) { /* We don't need id and ref_obj_id from this point From patchwork Wed Aug 26 13:19:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hangbin Liu X-Patchwork-Id: 1351938 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=PeMVk6iv; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bc61F0dbbz9sPB for ; Wed, 26 Aug 2020 23:20:33 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730243AbgHZNUa (ORCPT ); Wed, 26 Aug 2020 09:20:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730237AbgHZNU2 (ORCPT ); Wed, 26 Aug 2020 09:20:28 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91288C061756; Wed, 26 Aug 2020 06:20:27 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id t9so965793pfq.8; Wed, 26 Aug 2020 06:20:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3GQzu7xhU5dcPYrgwyDBuE7wfh4Wf22UF6zvfx+fHs4=; b=PeMVk6ivQX4SAZItUoPXdudLrkLjGNY6qik3HlXSazhAstX9Z29uCa9w2NF0766fjk atOcd206qf2i+Bjz5XBNvzHZa5IZcWfty7Ym/WbEw+MPhmLJdm5m+ku6b9zYmqZT4nq9 CzQ7dkeFCZ6Q88ZbS/obM019r/xTXyeLDDqe/xj6BEBZLwqCIqTfM4KhpJt53vPDyyaq s6KTVBlYKl1MpklPCk8I5IppaF8I+TdiaMLpnhaZwPwzE14XdfqdVrYWGcMU7FCacl6y xBATEhwXurK3QHdlzuIk9gR5MFSNw+YB7BQ3hGFdZYnilrmNhq6qiAWEUWz863fNeoX4 yHIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3GQzu7xhU5dcPYrgwyDBuE7wfh4Wf22UF6zvfx+fHs4=; b=kV1QVY6DrdGRieFaYsZ9muQkkXlZNN/P/y6slHLO348Yr5SuCOE12bbGKNu+tWuzW4 EpoPPIjvcO0QCQ7kwWUaJX/jarXzWzuKeFCoDmCw5zjr0ikqqehAXtcLgJsN/95opU3z cjDOS8u2dO2RfSehKSCjVk64oem49SBgU0vCX4BPVsZ+PQldEONMqdoysscq7TLohBb6 5YIIetODKJ+X2ZSupB8fejlAIYuJ2Myf8f+rxR6viNXS8qDU6g9PAgHdW3W/02oBb/Bs WX55bY9GxlTi4YuAwvjYWX6LQ+1C2h63hayeKB8Rjlevp2LI0lVbvUkurs7Dgtod2YCe 7+NQ== X-Gm-Message-State: AOAM532ufjJZIzwu/Xu6H6AWd5zBsZjtt/ZrJrbikbl28aMaFxy25Rm4 zvfunI5Npo9vjqSNy/551DpavH1zIcDE6JJu X-Google-Smtp-Source: ABdhPJwdGEuLRSOJf0ldFeV2v8PaNydcFFqSPCX5QrKmWfLdrwGNsfoojjqqZosUxtoTunEG1YWzJA== X-Received: by 2002:a63:1e65:: with SMTP id p37mr10406535pgm.58.1598448026330; Wed, 26 Aug 2020 06:20:26 -0700 (PDT) Received: from dhcp-12-153.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id s129sm3131794pfb.39.2020.08.26.06.20.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2020 06:20:25 -0700 (PDT) From: Hangbin Liu To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jiri Benc , Jesper Dangaard Brouer , Eelco Chaudron , ast@kernel.org, Daniel Borkmann , Lorenzo Bianconi , David Ahern , Andrii Nakryiko B , Hangbin Liu Subject: [PATCHv9 bpf-next 2/5] xdp: add a new helper for dev map multicast support Date: Wed, 26 Aug 2020 21:19:59 +0800 Message-Id: <20200826132002.2808380-3-liuhangbin@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200826132002.2808380-1-liuhangbin@gmail.com> References: <20200715130816.2124232-1-liuhangbin@gmail.com> <20200826132002.2808380-1-liuhangbin@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org This patch is for xdp multicast support. which has been discussed before[0], The goal is to be able to implement an OVS-like data plane in XDP, i.e., a software switch that can forward XDP frames to multiple ports. To achieve this, an application needs to specify a group of interfaces to forward a packet to. It is also common to want to exclude one or more physical interfaces from the forwarding operation - e.g., to forward a packet to all interfaces in the multicast group except the interface it arrived on. While this could be done simply by adding more groups, this quickly leads to a combinatorial explosion in the number of groups an application has to maintain. To avoid the combinatorial explosion, we propose to include the ability to specify an "exclude group" as part of the forwarding operation. This needs to be a group (instead of just a single port index), because a physical interface can be part of a logical grouping, such as a bond device. Thus, the logical forwarding operation becomes a "set difference" operation, i.e. "forward to all ports in group A that are not also in group B". This series implements such an operation using device maps to represent the groups. This means that the XDP program specifies two device maps, one containing the list of netdevs to redirect to, and the other containing the exclude list. To achieve this, I re-implement a new helper bpf_redirect_map_multi() to accept two maps, the forwarding map and exclude map. The forwarding map could be DEVMAP or DEVMAP_HASH, but the exclude map *must* be DEVMAP_HASH to get better performace. If user don't want to use exclude map and just want simply stop redirecting back to ingress device, they can use flag BPF_F_EXCLUDE_INGRESS. As both bpf_xdp_redirect_map() and this new helpers are using struct bpf_redirect_info, I add a new ex_map and set tgt_value to NULL in the new helper to make a difference with bpf_xdp_redirect_map(). Also I keep the the general data path in net/core/filter.c, the native data path in kernel/bpf/devmap.c so we can use direct calls to get better performace. [0] https://xdp-project.net/#Handling-multicast Signed-off-by: Hangbin Liu --- v9: a) Use ARG_CONST_MAP_PTR_OR_NULL for helper arg2 v8: a) Update function dev_in_exclude_map(): - remove duplicate ex_map map_type check in - lookup the element in dev map by obj dev index directly instead of looping all the map v7: a) Fix helper flag check b) Limit the *ex_map* to use DEVMAP_HASH only and update function dev_in_exclude_map() to get better performance. v6: converted helper return types from int to long v5: a) Check devmap_get_next_key() return value. b) Pass through flags to __bpf_tx_xdp_map() instead of bool value. c) In function dev_map_enqueue_multi(), consume xdpf for the last obj instead of the first on. d) Update helper description and code comments to explain that we use NULL target value to distinguish multicast and unicast forwarding. e) Update memory model, memory id and frame_sz in xdpf_clone(). v4: Fix bpf_xdp_redirect_map_multi_proto arg2_type typo v3: Based on Toke's suggestion, do the following update a) Update bpf_redirect_map_multi() description in bpf.h. b) Fix exclude_ifindex checking order in dev_in_exclude_map(). c) Fix one more xdpf clone in dev_map_enqueue_multi(). d) Go find next one in dev_map_enqueue_multi() if the interface is not able to forward instead of abort the whole loop. e) Remove READ_ONCE/WRITE_ONCE for ex_map. v2: Add new syscall bpf_xdp_redirect_map_multi() which could accept include/exclude maps directly. --- include/linux/bpf.h | 20 ++++++ include/linux/filter.h | 1 + include/net/xdp.h | 1 + include/uapi/linux/bpf.h | 27 +++++++ kernel/bpf/devmap.c | 124 +++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 6 ++ net/core/filter.c | 112 +++++++++++++++++++++++++++-- net/core/xdp.c | 29 ++++++++ tools/include/uapi/linux/bpf.h | 27 +++++++ 9 files changed, 342 insertions(+), 5 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index cb40a1281ea2..3aed07836df0 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1324,6 +1324,11 @@ int dev_xdp_enqueue(struct net_device *dev, struct xdp_buff *xdp, struct net_device *dev_rx); int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp, struct net_device *dev_rx); +bool dev_in_exclude_map(struct bpf_dtab_netdev *obj, struct bpf_map *map, + int exclude_ifindex); +int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx, + struct bpf_map *map, struct bpf_map *ex_map, + u32 flags); int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb, struct bpf_prog *xdp_prog); bool dev_map_can_have_prog(struct bpf_map *map); @@ -1499,6 +1504,21 @@ int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp, return 0; } +static inline +bool dev_in_exclude_map(struct bpf_dtab_netdev *obj, struct bpf_map *map, + int exclude_ifindex) +{ + return false; +} + +static inline +int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx, + struct bpf_map *map, struct bpf_map *ex_map, + u32 flags) +{ + return 0; +} + struct sk_buff; static inline int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, diff --git a/include/linux/filter.h b/include/linux/filter.h index 995625950cc1..583dbd4c8dce 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -612,6 +612,7 @@ struct bpf_redirect_info { u32 tgt_index; void *tgt_value; struct bpf_map *map; + struct bpf_map *ex_map; u32 kern_flags; }; diff --git a/include/net/xdp.h b/include/net/xdp.h index 3814fb631d52..8453d477bb22 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -132,6 +132,7 @@ void xdp_warn(const char *msg, const char *func, const int line); #define XDP_WARN(msg) xdp_warn(msg, __func__, __LINE__) struct xdp_frame *xdp_convert_zc_to_xdp_frame(struct xdp_buff *xdp); +struct xdp_frame *xdpf_clone(struct xdp_frame *xdpf); static inline void xdp_convert_frame_to_buff(struct xdp_frame *frame, struct xdp_buff *xdp) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 0388bc0200b0..bc31ea0b715b 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -3561,6 +3561,27 @@ union bpf_attr { * On success, the strictly positive length of the string, * including the trailing NUL character. On error, a negative * value. + * + * long bpf_redirect_map_multi(struct bpf_map *map, struct bpf_map *ex_map, u64 flags) + * Description + * This is a multicast implementation for XDP redirect. It will + * redirect the packet to ALL the interfaces in *map*, but + * exclude the interfaces in *ex_map*. + * + * The frowarding *map* could be either BPF_MAP_TYPE_DEVMAP or + * BPF_MAP_TYPE_DEVMAP_HASH. But the *ex_map* must be + * BPF_MAP_TYPE_DEVMAP_HASH to get better performance. + * + * Currently the *flags* only supports *BPF_F_EXCLUDE_INGRESS*, + * which additionally excludes the current ingress device. + * + * See also bpf_redirect_map() as a unicast implementation, + * which supports redirecting packet to a specific ifindex + * in the map. As both helpers use struct bpf_redirect_info + * to store the redirect info, we will use a a NULL tgt_value + * to distinguish multicast and unicast redirecting. + * Return + * **XDP_REDIRECT** on success, or **XDP_ABORTED** on error. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -3711,6 +3732,7 @@ union bpf_attr { FN(inode_storage_get), \ FN(inode_storage_delete), \ FN(d_path), \ + FN(redirect_map_multi), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper @@ -3882,6 +3904,11 @@ enum bpf_lwt_encap_mode { BPF_LWT_ENCAP_IP, }; +/* BPF_FUNC_redirect_map_multi flags. */ +enum { + BPF_F_EXCLUDE_INGRESS = (1ULL << 0), +}; + #define __bpf_md_ptr(type, name) \ union { \ type name; \ diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index 10abb06065bb..1e6695f38f78 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -512,6 +512,130 @@ int dev_map_enqueue(struct bpf_dtab_netdev *dst, struct xdp_buff *xdp, return __xdp_enqueue(dev, xdp, dev_rx); } +/* Use direct call in fast path instead of map->ops->map_get_next_key() */ +static int devmap_get_next_key(struct bpf_map *map, void *key, void *next_key) +{ + + switch (map->map_type) { + case BPF_MAP_TYPE_DEVMAP: + return dev_map_get_next_key(map, key, next_key); + case BPF_MAP_TYPE_DEVMAP_HASH: + return dev_map_hash_get_next_key(map, key, next_key); + default: + break; + } + + return -ENOENT; +} + +bool dev_in_exclude_map(struct bpf_dtab_netdev *obj, struct bpf_map *map, + int exclude_ifindex) +{ + if (obj->dev->ifindex == exclude_ifindex) + return true; + + if (!map) + return false; + + return __dev_map_hash_lookup_elem(map, obj->dev->ifindex) != NULL; +} + +static struct bpf_dtab_netdev *devmap_get_next_obj(struct xdp_buff *xdp, struct bpf_map *map, + struct bpf_map *ex_map, u32 *key, + u32 *next_key, int ex_ifindex) +{ + struct bpf_dtab_netdev *obj; + struct net_device *dev; + u32 *tmp_key = key; + int err; + + err = devmap_get_next_key(map, tmp_key, next_key); + if (err) + return NULL; + + for (;;) { + switch (map->map_type) { + case BPF_MAP_TYPE_DEVMAP: + obj = __dev_map_lookup_elem(map, *next_key); + break; + case BPF_MAP_TYPE_DEVMAP_HASH: + obj = __dev_map_hash_lookup_elem(map, *next_key); + break; + default: + break; + } + + if (!obj || dev_in_exclude_map(obj, ex_map, ex_ifindex)) + goto find_next; + + dev = obj->dev; + + if (!dev->netdev_ops->ndo_xdp_xmit) + goto find_next; + + err = xdp_ok_fwd_dev(dev, xdp->data_end - xdp->data); + if (unlikely(err)) + goto find_next; + + return obj; + +find_next: + tmp_key = next_key; + err = devmap_get_next_key(map, tmp_key, next_key); + if (err) + break; + } + + return NULL; +} + +int dev_map_enqueue_multi(struct xdp_buff *xdp, struct net_device *dev_rx, + struct bpf_map *map, struct bpf_map *ex_map, + u32 flags) +{ + struct bpf_dtab_netdev *obj = NULL, *next_obj = NULL; + struct xdp_frame *xdpf, *nxdpf; + bool last_one = false; + int ex_ifindex; + u32 key, next_key; + + ex_ifindex = flags & BPF_F_EXCLUDE_INGRESS ? dev_rx->ifindex : 0; + + /* Find first available obj */ + obj = devmap_get_next_obj(xdp, map, ex_map, NULL, &key, ex_ifindex); + if (!obj) + return 0; + + xdpf = xdp_convert_buff_to_frame(xdp); + if (unlikely(!xdpf)) + return -EOVERFLOW; + + for (;;) { + /* Check if we still have one more available obj */ + next_obj = devmap_get_next_obj(xdp, map, ex_map, &key, + &next_key, ex_ifindex); + if (!next_obj) + last_one = true; + + if (last_one) { + bq_enqueue(obj->dev, xdpf, dev_rx); + return 0; + } + + nxdpf = xdpf_clone(xdpf); + if (unlikely(!nxdpf)) { + xdp_return_frame_rx_napi(xdpf); + return -ENOMEM; + } + + bq_enqueue(obj->dev, nxdpf, dev_rx); + + /* Deal with next obj */ + obj = next_obj; + key = next_key; + } +} + int dev_map_generic_redirect(struct bpf_dtab_netdev *dst, struct sk_buff *skb, struct bpf_prog *xdp_prog) { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 53a84335a8fd..39685a7aa157 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4267,6 +4267,7 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, case BPF_MAP_TYPE_DEVMAP: case BPF_MAP_TYPE_DEVMAP_HASH: if (func_id != BPF_FUNC_redirect_map && + func_id != BPF_FUNC_redirect_map_multi && func_id != BPF_FUNC_map_lookup_elem) goto error; break; @@ -4366,6 +4367,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, map->map_type != BPF_MAP_TYPE_XSKMAP) goto error; break; + case BPF_FUNC_redirect_map_multi: + if (map->map_type != BPF_MAP_TYPE_DEVMAP && + map->map_type != BPF_MAP_TYPE_DEVMAP_HASH) + goto error; + break; case BPF_FUNC_sk_redirect_map: case BPF_FUNC_msg_redirect_map: case BPF_FUNC_sock_map_update: diff --git a/net/core/filter.c b/net/core/filter.c index 47eef9a0be6a..aa10ae72809b 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3539,12 +3539,19 @@ static const struct bpf_func_proto bpf_xdp_adjust_meta_proto = { }; static int __bpf_tx_xdp_map(struct net_device *dev_rx, void *fwd, - struct bpf_map *map, struct xdp_buff *xdp) + struct bpf_map *map, struct xdp_buff *xdp, + struct bpf_map *ex_map, u32 flags) { switch (map->map_type) { case BPF_MAP_TYPE_DEVMAP: case BPF_MAP_TYPE_DEVMAP_HASH: - return dev_map_enqueue(fwd, xdp, dev_rx); + /* We use a NULL fwd value to distinguish multicast + * and unicast forwarding + */ + if (fwd) + return dev_map_enqueue(fwd, xdp, dev_rx); + else + return dev_map_enqueue_multi(xdp, dev_rx, map, ex_map, flags); case BPF_MAP_TYPE_CPUMAP: return cpu_map_enqueue(fwd, xdp, dev_rx); case BPF_MAP_TYPE_XSKMAP: @@ -3601,12 +3608,14 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, { struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); struct bpf_map *map = READ_ONCE(ri->map); + struct bpf_map *ex_map = ri->ex_map; u32 index = ri->tgt_index; void *fwd = ri->tgt_value; int err; ri->tgt_index = 0; ri->tgt_value = NULL; + ri->ex_map = NULL; WRITE_ONCE(ri->map, NULL); if (unlikely(!map)) { @@ -3618,7 +3627,7 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, err = dev_xdp_enqueue(fwd, xdp, dev); } else { - err = __bpf_tx_xdp_map(dev, fwd, map, xdp); + err = __bpf_tx_xdp_map(dev, fwd, map, xdp, ex_map, ri->flags); } if (unlikely(err)) @@ -3632,6 +3641,55 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, } EXPORT_SYMBOL_GPL(xdp_do_redirect); +static int dev_map_redirect_multi(struct net_device *dev, struct sk_buff *skb, + struct bpf_prog *xdp_prog, + struct bpf_map *map, struct bpf_map *ex_map, + u32 flags) + +{ + struct bpf_dtab_netdev *dst; + struct sk_buff *nskb; + bool exclude_ingress; + u32 key, next_key; + void *fwd; + int err; + + /* Get first key from forward map */ + err = map->ops->map_get_next_key(map, NULL, &key); + if (err) + return err; + + exclude_ingress = !!(flags & BPF_F_EXCLUDE_INGRESS); + + for (;;) { + fwd = __xdp_map_lookup_elem(map, key); + if (fwd) { + dst = (struct bpf_dtab_netdev *)fwd; + if (dev_in_exclude_map(dst, ex_map, + exclude_ingress ? dev->ifindex : 0)) + goto find_next; + + nskb = skb_clone(skb, GFP_ATOMIC); + if (!nskb) + return -ENOMEM; + + /* Try forword next one no mater the current forward + * succeed or not */ + dev_map_generic_redirect(dst, nskb, xdp_prog); + } + +find_next: + err = map->ops->map_get_next_key(map, &key, &next_key); + if (err) + break; + + key = next_key; + } + + consume_skb(skb); + return 0; +} + static int xdp_do_generic_redirect_map(struct net_device *dev, struct sk_buff *skb, struct xdp_buff *xdp, @@ -3639,19 +3697,30 @@ static int xdp_do_generic_redirect_map(struct net_device *dev, struct bpf_map *map) { struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); + struct bpf_map *ex_map = ri->ex_map; u32 index = ri->tgt_index; void *fwd = ri->tgt_value; int err = 0; ri->tgt_index = 0; ri->tgt_value = NULL; + ri->ex_map = NULL; WRITE_ONCE(ri->map, NULL); if (map->map_type == BPF_MAP_TYPE_DEVMAP || map->map_type == BPF_MAP_TYPE_DEVMAP_HASH) { - struct bpf_dtab_netdev *dst = fwd; + /* We use a NULL fwd value to distinguish multicast + * and unicast forwarding + */ + if (fwd) { + struct bpf_dtab_netdev *dst = fwd; + + err = dev_map_generic_redirect(dst, skb, xdp_prog); + } else { + err = dev_map_redirect_multi(dev, skb, xdp_prog, map, + ex_map, ri->flags); + } - err = dev_map_generic_redirect(dst, skb, xdp_prog); if (unlikely(err)) goto err; } else if (map->map_type == BPF_MAP_TYPE_XSKMAP) { @@ -3765,6 +3834,37 @@ static const struct bpf_func_proto bpf_xdp_redirect_map_proto = { .arg3_type = ARG_ANYTHING, }; +BPF_CALL_3(bpf_xdp_redirect_map_multi, struct bpf_map *, map, + struct bpf_map *, ex_map, u64, flags) +{ + struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); + + /* Limit ex_map type to DEVMAP_HASH to get better performance */ + if (unlikely(!map || + (ex_map && ex_map->map_type != BPF_MAP_TYPE_DEVMAP_HASH) || + flags & ~BPF_F_EXCLUDE_INGRESS)) + return XDP_ABORTED; + + ri->tgt_index = 0; + /* Set the tgt_value to NULL to distinguish with bpf_xdp_redirect_map */ + ri->tgt_value = NULL; + ri->flags = flags; + ri->ex_map = ex_map; + + WRITE_ONCE(ri->map, map); + + return XDP_REDIRECT; +} + +static const struct bpf_func_proto bpf_xdp_redirect_map_multi_proto = { + .func = bpf_xdp_redirect_map_multi, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_CONST_MAP_PTR, + .arg2_type = ARG_CONST_MAP_PTR_OR_NULL, + .arg3_type = ARG_ANYTHING, +}; + static unsigned long bpf_skb_copy(void *dst_buff, const void *skb, unsigned long off, unsigned long len) { @@ -6833,6 +6933,8 @@ xdp_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_xdp_redirect_proto; case BPF_FUNC_redirect_map: return &bpf_xdp_redirect_map_proto; + case BPF_FUNC_redirect_map_multi: + return &bpf_xdp_redirect_map_multi_proto; case BPF_FUNC_xdp_adjust_tail: return &bpf_xdp_adjust_tail_proto; case BPF_FUNC_fib_lookup: diff --git a/net/core/xdp.c b/net/core/xdp.c index 48aba933a5a8..9fd3e89768c4 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -467,3 +467,32 @@ void xdp_warn(const char *msg, const char *func, const int line) WARN(1, "XDP_WARN: %s(line:%d): %s\n", func, line, msg); }; EXPORT_SYMBOL_GPL(xdp_warn); + +struct xdp_frame *xdpf_clone(struct xdp_frame *xdpf) +{ + unsigned int headroom, totalsize; + struct xdp_frame *nxdpf; + struct page *page; + void *addr; + + headroom = xdpf->headroom + sizeof(*xdpf); + totalsize = headroom + xdpf->len; + + if (unlikely(totalsize > PAGE_SIZE)) + return NULL; + page = dev_alloc_page(); + if (!page) + return NULL; + addr = page_to_virt(page); + + memcpy(addr, xdpf, totalsize); + + nxdpf = addr; + nxdpf->data = addr + headroom; + nxdpf->frame_sz = PAGE_SIZE; + nxdpf->mem.type = MEM_TYPE_PAGE_ORDER0; + nxdpf->mem.id = 0; + + return nxdpf; +} +EXPORT_SYMBOL_GPL(xdpf_clone); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 0388bc0200b0..bc31ea0b715b 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -3561,6 +3561,27 @@ union bpf_attr { * On success, the strictly positive length of the string, * including the trailing NUL character. On error, a negative * value. + * + * long bpf_redirect_map_multi(struct bpf_map *map, struct bpf_map *ex_map, u64 flags) + * Description + * This is a multicast implementation for XDP redirect. It will + * redirect the packet to ALL the interfaces in *map*, but + * exclude the interfaces in *ex_map*. + * + * The frowarding *map* could be either BPF_MAP_TYPE_DEVMAP or + * BPF_MAP_TYPE_DEVMAP_HASH. But the *ex_map* must be + * BPF_MAP_TYPE_DEVMAP_HASH to get better performance. + * + * Currently the *flags* only supports *BPF_F_EXCLUDE_INGRESS*, + * which additionally excludes the current ingress device. + * + * See also bpf_redirect_map() as a unicast implementation, + * which supports redirecting packet to a specific ifindex + * in the map. As both helpers use struct bpf_redirect_info + * to store the redirect info, we will use a a NULL tgt_value + * to distinguish multicast and unicast redirecting. + * Return + * **XDP_REDIRECT** on success, or **XDP_ABORTED** on error. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -3711,6 +3732,7 @@ union bpf_attr { FN(inode_storage_get), \ FN(inode_storage_delete), \ FN(d_path), \ + FN(redirect_map_multi), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper @@ -3882,6 +3904,11 @@ enum bpf_lwt_encap_mode { BPF_LWT_ENCAP_IP, }; +/* BPF_FUNC_redirect_map_multi flags. */ +enum { + BPF_F_EXCLUDE_INGRESS = (1ULL << 0), +}; + #define __bpf_md_ptr(type, name) \ union { \ type name; \ From patchwork Wed Aug 26 13:20:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hangbin Liu X-Patchwork-Id: 1351940 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=KAR4IBKB; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bc61Q3WfKz9sTQ for ; Wed, 26 Aug 2020 23:20:42 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730254AbgHZNUl (ORCPT ); Wed, 26 Aug 2020 09:20:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730248AbgHZNUc (ORCPT ); Wed, 26 Aug 2020 09:20:32 -0400 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B83F8C061574; Wed, 26 Aug 2020 06:20:32 -0700 (PDT) Received: by mail-pg1-x532.google.com with SMTP id w186so1013074pgb.8; Wed, 26 Aug 2020 06:20:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uhBqgyl2R4ybVhAEljTLnh8gFHPl7M0Dc/ZnWPWBd14=; b=KAR4IBKBC6HqfcdozsZw0TclW67y3+Dmm+TX6QasG3ZeA4jaq8LpvYQh8aT67jlAa8 6a/dEXqGPSAKl25Rja+JaBTDH8zian8RAVoucYRq4eXMKAqLpphgRTPkAlI4yF89DZCK K5ptPbBTUuXACD+a060MkneK0IY3+CllPE0wpKAFu15A+W9j7PVXhZDb4pbxzinHxOsG yYDWUlnw+0/raTOYxk6RBp8t/ZgP02pS2SMMIZ5tRdgj9rlVb7cScoGVhHdjs4jgx0cX Dy5ux6Lr2y2TPKn4Y14nOkHa3hy+v2h/QwM6ZUXxgotKLaSFT46N+vONjTkdFLr1t/NT 4l1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uhBqgyl2R4ybVhAEljTLnh8gFHPl7M0Dc/ZnWPWBd14=; b=flRdL5iJhTmveuR0naU48o0o2fUCaxIKvmp6T49xc3dBJzdCdHx3cdeTa9+PBhE95/ p4yVGZ/CYirQjft4CIhabUkquD2ZgZ4ltO3Lrgcu3h2tWXUb330P4UX20cFxfO56cmUy wfsisEmftc+QIIs+1DuNVZdAKEOxTgjWiCaD3RBRo2XK9qJXIk1SUoNnH9MlliFZn2j7 a687MtB3HvP3Yv/D55UlFUrfTPKLTYZFDpMCSy1OPem6JKSpLcb3Obv7SZzT7W/h7Yxa B4iJdDgSi1kY2KxucBDlaJQrIOjnWQbwwJyyl6D+XqEd7F2og/f7h6sw9YNjhRhQrH8T 07Sg== X-Gm-Message-State: AOAM530ZtS4nN06phI2VLKIEiI3v/sIlOPkpV/nre8ObgrQayphHT6Sm DjpRlC4iENWeWNYYNI36nwB5UR3GpEERJ1Xu X-Google-Smtp-Source: ABdhPJzfyuxYnm/GdhSBgrXuFL4bimXcfyJRXat3yjxl1DULJxaYCOKxg5V+QVHh2nVN0Iwhs/AK2Q== X-Received: by 2002:aa7:83cf:: with SMTP id j15mr11991940pfn.251.1598448030541; Wed, 26 Aug 2020 06:20:30 -0700 (PDT) Received: from dhcp-12-153.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id s129sm3131794pfb.39.2020.08.26.06.20.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2020 06:20:30 -0700 (PDT) From: Hangbin Liu To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jiri Benc , Jesper Dangaard Brouer , Eelco Chaudron , ast@kernel.org, Daniel Borkmann , Lorenzo Bianconi , David Ahern , Andrii Nakryiko B , Hangbin Liu Subject: [PATCHv9 bpf-next 3/5] sample/bpf: add xdp_redirect_map_multicast test Date: Wed, 26 Aug 2020 21:20:00 +0800 Message-Id: <20200826132002.2808380-4-liuhangbin@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200826132002.2808380-1-liuhangbin@gmail.com> References: <20200715130816.2124232-1-liuhangbin@gmail.com> <20200826132002.2808380-1-liuhangbin@gmail.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This is a sample for xdp multicast. In the sample we could forward all packets between given interfaces. Signed-off-by: Hangbin Liu --- v9: use NULL directly for arg2 and redefine the maps with btf format v6-v8: no update v5: add a null_map as we have strict the arg2 to ARG_CONST_MAP_PTR. Move the testing part to bpf selftest in next patch. v4: no update. v3: add rxcnt map to show the packet transmit speed. v2: no update. --- samples/bpf/Makefile | 3 + samples/bpf/xdp_redirect_map_multi_kern.c | 43 ++++++ samples/bpf/xdp_redirect_map_multi_user.c | 166 ++++++++++++++++++++++ 3 files changed, 212 insertions(+) create mode 100644 samples/bpf/xdp_redirect_map_multi_kern.c create mode 100644 samples/bpf/xdp_redirect_map_multi_user.c diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile index a6d3646b3818..177cab805768 100644 --- a/samples/bpf/Makefile +++ b/samples/bpf/Makefile @@ -41,6 +41,7 @@ tprogs-y += test_map_in_map tprogs-y += per_socket_stats_example tprogs-y += xdp_redirect tprogs-y += xdp_redirect_map +tprogs-y += xdp_redirect_map_multi tprogs-y += xdp_redirect_cpu tprogs-y += xdp_monitor tprogs-y += xdp_rxq_info @@ -97,6 +98,7 @@ test_map_in_map-objs := test_map_in_map_user.o per_socket_stats_example-objs := cookie_uid_helper_example.o xdp_redirect-objs := xdp_redirect_user.o xdp_redirect_map-objs := xdp_redirect_map_user.o +xdp_redirect_map_multi-objs := xdp_redirect_map_multi_user.o xdp_redirect_cpu-objs := bpf_load.o xdp_redirect_cpu_user.o xdp_monitor-objs := bpf_load.o xdp_monitor_user.o xdp_rxq_info-objs := xdp_rxq_info_user.o @@ -156,6 +158,7 @@ always-y += tcp_tos_reflect_kern.o always-y += tcp_dumpstats_kern.o always-y += xdp_redirect_kern.o always-y += xdp_redirect_map_kern.o +always-y += xdp_redirect_map_multi_kern.o always-y += xdp_redirect_cpu_kern.o always-y += xdp_monitor_kern.o always-y += xdp_rxq_info_kern.o diff --git a/samples/bpf/xdp_redirect_map_multi_kern.c b/samples/bpf/xdp_redirect_map_multi_kern.c new file mode 100644 index 000000000000..db58d56cef89 --- /dev/null +++ b/samples/bpf/xdp_redirect_map_multi_kern.c @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * modify it under the terms of version 2 of the GNU General Public + * License as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + */ +#define KBUILD_MODNAME "foo" +#include +#include + +struct { + __uint(type, BPF_MAP_TYPE_DEVMAP_HASH); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, 256); +} forward_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); + __type(key, u32); + __type(value, long); + __uint(max_entries, 1); +} rxcnt SEC(".maps"); + +SEC("xdp_redirect_map_multi") +int xdp_redirect_map_multi_prog(struct xdp_md *ctx) +{ + long *value; + u32 key = 0; + + /* count packet in global counter */ + value = bpf_map_lookup_elem(&rxcnt, &key); + if (value) + *value += 1; + + return bpf_redirect_map_multi(&forward_map, NULL, BPF_F_EXCLUDE_INGRESS); +} + +char _license[] SEC("license") = "GPL"; diff --git a/samples/bpf/xdp_redirect_map_multi_user.c b/samples/bpf/xdp_redirect_map_multi_user.c new file mode 100644 index 000000000000..49f44c91b672 --- /dev/null +++ b/samples/bpf/xdp_redirect_map_multi_user.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "bpf_util.h" +#include +#include + +#define MAX_IFACE_NUM 32 + +static __u32 xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; +static int ifaces[MAX_IFACE_NUM] = {}; +static int rxcnt; + +static void int_exit(int sig) +{ + __u32 prog_id = 0; + int i; + + for (i = 0; ifaces[i] > 0; i++) { + if (bpf_get_link_xdp_id(ifaces[i], &prog_id, xdp_flags)) { + printf("bpf_get_link_xdp_id failed\n"); + exit(1); + } + if (prog_id) + bpf_set_link_xdp_fd(ifaces[i], -1, xdp_flags); + } + + exit(0); +} + +static void poll_stats(int interval) +{ + unsigned int nr_cpus = bpf_num_possible_cpus(); + __u64 values[nr_cpus], prev[nr_cpus]; + + memset(prev, 0, sizeof(prev)); + + while (1) { + __u64 sum = 0; + __u32 key = 0; + int i; + + sleep(interval); + assert(bpf_map_lookup_elem(rxcnt, &key, values) == 0); + for (i = 0; i < nr_cpus; i++) + sum += (values[i] - prev[i]); + if (sum) + printf("Forwarding %10llu pkt/s\n", sum / interval); + memcpy(prev, values, sizeof(values)); + } +} + +static void usage(const char *prog) +{ + fprintf(stderr, + "usage: %s [OPTS] ...\n" + "OPTS:\n" + " -S use skb-mode\n" + " -N enforce native mode\n" + " -F force loading prog\n", + prog); +} + +int main(int argc, char **argv) +{ + struct bpf_prog_load_attr prog_load_attr = { + .prog_type = BPF_PROG_TYPE_XDP, + }; + int prog_fd, forward_map; + int i, ret, opt, ifindex; + char ifname[IF_NAMESIZE]; + struct bpf_object *obj; + char filename[256]; + + while ((opt = getopt(argc, argv, "SNF")) != -1) { + switch (opt) { + case 'S': + xdp_flags |= XDP_FLAGS_SKB_MODE; + break; + case 'N': + /* default, set below */ + break; + case 'F': + xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; + break; + default: + usage(basename(argv[0])); + return 1; + } + } + + if (!(xdp_flags & XDP_FLAGS_SKB_MODE)) + xdp_flags |= XDP_FLAGS_DRV_MODE; + + if (optind == argc) { + printf("usage: %s ...\n", argv[0]); + return 1; + } + + printf("Get interfaces"); + for (i = 0; i < MAX_IFACE_NUM && argv[optind + i]; i++) { + ifaces[i] = if_nametoindex(argv[optind + i]); + if (!ifaces[i]) + ifaces[i] = strtoul(argv[optind + i], NULL, 0); + if (!if_indextoname(ifaces[i], ifname)) { + perror("Invalid interface name or i"); + return 1; + } + printf(" %d", ifaces[i]); + } + printf("\n"); + + snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]); + prog_load_attr.file = filename; + + if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd)) + return 1; + + forward_map = bpf_object__find_map_fd_by_name(obj, "forward_map"); + rxcnt = bpf_object__find_map_fd_by_name(obj, "rxcnt"); + + if (forward_map < 0 || rxcnt < 0) { + printf("bpf_object__find_map_fd_by_name failed\n"); + return 1; + } + + signal(SIGINT, int_exit); + signal(SIGTERM, int_exit); + + /* Init forward multicast groups and exclude group */ + for (i = 0; ifaces[i] > 0; i++) { + ifindex = ifaces[i]; + + /* Add all the interfaces to group all */ + ret = bpf_map_update_elem(forward_map, &ifindex, &ifindex, 0); + if (ret) { + perror("bpf_map_update_elem"); + goto err_out; + } + + /* bind prog_fd to each interface */ + ret = bpf_set_link_xdp_fd(ifindex, prog_fd, xdp_flags); + if (ret) { + printf("Set xdp fd failed on %d\n", ifindex); + goto err_out; + } + + } + + poll_stats(2); + + return 0; + +err_out: + return 1; +} From patchwork Wed Aug 26 13:20:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hangbin Liu X-Patchwork-Id: 1351944 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=uQpNFA+H; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bc61X6f1Dz9sTQ for ; Wed, 26 Aug 2020 23:20:48 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730247AbgHZNUs (ORCPT ); Wed, 26 Aug 2020 09:20:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730227AbgHZNUg (ORCPT ); Wed, 26 Aug 2020 09:20:36 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30346C061574; Wed, 26 Aug 2020 06:20:36 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id n3so1092176pjq.1; Wed, 26 Aug 2020 06:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pOK5dq4CJDPPIJtzc7/q9xQGuIdlneXWVPNpaEGfNGY=; b=uQpNFA+H2efG57sMj5zvPPgSlTNclR2Gvw7UnXAj8joi8Bzr6jhzRPtEuNvDdZi+G4 KWFeVk4kRb3B723z+TeWmnYKZcskuSJTeyV3yHLIdjtFDl6Zrhk7ud/Z95UD77qdu4sr yToSljmj2B0VN24FP1B+ZhND5JhFx0H5PMtPfV9r7VY94YVWIb4QdLvlNJfAAwowqdtj r3nWOlrT0PpSUIAwAZUfDW3RpuPbbjtm5p+2O4yU2Wf3N1+ke22+GhcFjqxViXYnwfHh EOEEhKR7uOj339dnLmqv94glN8nyAx27j6mD/ELS8ppvkT6PCOJo7BFn//LPAMkfzwoK UrRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pOK5dq4CJDPPIJtzc7/q9xQGuIdlneXWVPNpaEGfNGY=; b=UQ8de3ourxsTyrSiNWeEdTUNyeL1pBQgdSc0IQcy5+vGELwmZ8QaZVSiBNH3/tCBPM RFzogQZiMmig+PT6HjAOjQUj+O6OBagtFMV0KpPFkhrTXoT9hjstfho5hSGy3KjU/xly 6ANTXrEO1cSjVdtTI9Z6O1frv1HG5umq7qoWgWSS8aIVem9PSlwZmIn9heYFt5mx9lnZ FLF9rF8rtH657puSe87V4rVlt5bYQYAa48+PflY79+JfuH+omyboLNG91wYg3Nn1ua5N mSx1ZOI9Bj2w6QS/UIE2k3cDmjVPEx6thRc2y6vGxDMJmGqHNWYSWP6KFC4oa/WMBwjP Ybzg== X-Gm-Message-State: AOAM531qlht8m2OS8fEl1Y1u0n+OOzKz7i05YgE5zka7v0evbHxIncUc mCUV7ZJJZNI9pSzeOCKgoCPGP2pQMYMoO60D X-Google-Smtp-Source: ABdhPJzq51M37UV8M0yuvdArKq48i0v+Z5/rZ3eV48QFBctc/mEL6Ufa+xm1DSwYYc8q14beAv85wQ== X-Received: by 2002:a17:902:5996:: with SMTP id p22mr7756067pli.162.1598448035038; Wed, 26 Aug 2020 06:20:35 -0700 (PDT) Received: from dhcp-12-153.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id s129sm3131794pfb.39.2020.08.26.06.20.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2020 06:20:34 -0700 (PDT) From: Hangbin Liu To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jiri Benc , Jesper Dangaard Brouer , Eelco Chaudron , ast@kernel.org, Daniel Borkmann , Lorenzo Bianconi , David Ahern , Andrii Nakryiko B , Hangbin Liu Subject: [PATCHv9 bpf-next 4/5] selftests/bpf: add xdp_redirect_multi test Date: Wed, 26 Aug 2020 21:20:01 +0800 Message-Id: <20200826132002.2808380-5-liuhangbin@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200826132002.2808380-1-liuhangbin@gmail.com> References: <20200715130816.2124232-1-liuhangbin@gmail.com> <20200826132002.2808380-1-liuhangbin@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Add a bpf selftest for new helper xdp_redirect_map_multi(). In this test we have 3 forward groups and 1 exclude group. The test will redirect each interface's packets to all the interfaces in the forward group, and exclude the interface in exclude map. We will also test both DEVMAP and DEVMAP_HASH with xdp generic and drv. For more test details, you can find it in the test script. Here is the test result. ]# ./test_xdp_redirect_multi.sh Pass: xdpgeneric arp ns1-2 Pass: xdpgeneric arp ns1-3 Pass: xdpgeneric arp ns1-4 Pass: xdpgeneric ping ns1-2 Pass: xdpgeneric ping ns1-3 Pass: xdpgeneric ping ns1-4 Pass: xdpgeneric ping6 ns2-1 Pass: xdpgeneric ping6 ns2-3 Pass: xdpgeneric ping6 ns2-4 Pass: xdpdrv arp ns1-2 Pass: xdpdrv arp ns1-3 Pass: xdpdrv arp ns1-4 Pass: xdpdrv ping ns1-2 Pass: xdpdrv ping ns1-3 Pass: xdpdrv ping ns1-4 Pass: xdpdrv ping6 ns2-1 Pass: xdpdrv ping6 ns2-3 Pass: xdpdrv ping6 ns2-4 Summary: PASS 18, FAIL 0 Signed-off-by: Hangbin Liu --- v9: use NULL directly for arg2 and redefine the maps with btf format --- tools/testing/selftests/bpf/Makefile | 4 +- .../bpf/progs/xdp_redirect_multi_kern.c | 77 ++++++++ .../selftests/bpf/test_xdp_redirect_multi.sh | 164 +++++++++++++++++ .../selftests/bpf/xdp_redirect_multi.c | 173 ++++++++++++++++++ 4 files changed, 417 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/xdp_redirect_multi_kern.c create mode 100755 tools/testing/selftests/bpf/test_xdp_redirect_multi.sh create mode 100644 tools/testing/selftests/bpf/xdp_redirect_multi.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 09657d0afb5c..ae44bd708d0e 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -51,6 +51,7 @@ TEST_FILES = test_lwt_ip_encap.o \ # Order correspond to 'make run_tests' order TEST_PROGS := test_kmod.sh \ test_xdp_redirect.sh \ + test_xdp_redirect_multi.sh \ test_xdp_meta.sh \ test_xdp_veth.sh \ test_offload.py \ @@ -79,7 +80,8 @@ TEST_PROGS_EXTENDED := with_addr.sh \ # Compile but not part of 'make run_tests' TEST_GEN_PROGS_EXTENDED = test_sock_addr test_skb_cgroup_id_user \ flow_dissector_load test_flow_dissector test_tcp_check_syncookie_user \ - test_lirc_mode2_user xdping test_cpp runqslower bench + test_lirc_mode2_user xdping test_cpp runqslower bench \ + xdp_redirect_multi TEST_CUSTOM_PROGS = urandom_read diff --git a/tools/testing/selftests/bpf/progs/xdp_redirect_multi_kern.c b/tools/testing/selftests/bpf/progs/xdp_redirect_multi_kern.c new file mode 100644 index 000000000000..30c703efcb28 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/xdp_redirect_multi_kern.c @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * modify it under the terms of version 2 of the GNU General Public + * License as published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + */ +#define KBUILD_MODNAME "foo" +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +struct { + __uint(type, BPF_MAP_TYPE_DEVMAP); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, 128); +} forward_map_v4 SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_DEVMAP_HASH); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, 128); +} forward_map_v6 SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_DEVMAP_HASH); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, 128); +} forward_map_all SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_DEVMAP_HASH); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(int)); + __uint(max_entries, 128); +} exclude_map SEC(".maps"); + +SEC("xdp_redirect_map_multi") +int xdp_redirect_map_multi_prog(struct xdp_md *ctx) +{ + void *data_end = (void *)(long)ctx->data_end; + void *data = (void *)(long)ctx->data; + struct ethhdr *eth = data; + __u16 h_proto; + __u64 nh_off; + + nh_off = sizeof(*eth); + if (data + nh_off > data_end) + return XDP_DROP; + + h_proto = eth->h_proto; + + if (h_proto == bpf_htons(ETH_P_IP)) + return bpf_redirect_map_multi(&forward_map_v4, &exclude_map, + BPF_F_EXCLUDE_INGRESS); + else if (h_proto == bpf_htons(ETH_P_IPV6)) + return bpf_redirect_map_multi(&forward_map_v6, &exclude_map, + BPF_F_EXCLUDE_INGRESS); + else + return bpf_redirect_map_multi(&forward_map_all, NULL, + BPF_F_EXCLUDE_INGRESS); +} + +char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/test_xdp_redirect_multi.sh b/tools/testing/selftests/bpf/test_xdp_redirect_multi.sh new file mode 100755 index 000000000000..c1395d8634c5 --- /dev/null +++ b/tools/testing/selftests/bpf/test_xdp_redirect_multi.sh @@ -0,0 +1,164 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Test topology: +# - - - - - - - - - - - - - - - - - - - - - - - - - +# | veth1 veth2 veth3 veth4 | ... init net +# - -| - - - - - - | - - - - - - | - - - - - - | - - +# --------- --------- --------- --------- +# | veth0 | | veth0 | | veth0 | | veth0 | ... +# --------- --------- --------- --------- +# ns1 ns2 ns3 ns4 +# +# Forward multicast groups: +# Forward group all has interfaces: veth1, veth2, veth3, veth4, ... (All traffic except IPv4, IPv6) +# Forward group v4 has interfaces: veth1, veth3, veth4, ... (For IPv4 traffic only) +# Forward group v6 has interfaces: veth2, veth3, veth4, ... (For IPv6 traffic only) +# Exclude Groups: +# Exclude group: veth3 (assume ns3 is in black list) +# +# Test modules: +# XDP modes: generic, native +# map types: group v4 use DEVMAP, others use DEVMAP_HASH +# +# Test cases: +# ARP(we didn't block ARP for ns3): +# ns1 -> gw: ns2, ns3, ns4 should receive the arp request +# IPv4: +# ns1 -> ns2 (fail), ns1 -> ns3 (fail), ns1 -> ns4 (pass) +# IPv6 +# ns2 -> ns1 (fail), ns2 -> ns3 (fail), ns2 -> ns4 (pass) +# + + +# netns numbers +NUM=4 +IFACES="" +DRV_MODE="xdpgeneric xdpdrv" +PASS=0 +FAIL=0 + +test_pass() +{ + echo "Pass: $@" + PASS=$((PASS + 1)) +} + +test_fail() +{ + echo "fail: $@" + FAIL=$((FAIL + 1)) +} + +clean_up() +{ + for i in $(seq $NUM); do + ip link del veth$i 2> /dev/null + ip netns del ns$i 2> /dev/null + done + rm -f xdp_redirect_*.log arp_ns*.log +} + +# Kselftest framework requirement - SKIP code is 4. +check_env() +{ + ip link set dev lo xdpgeneric off &>/dev/null + if [ $? -ne 0 ];then + echo "selftests: [SKIP] Could not run test without the ip xdpgeneric support" + exit 4 + fi + + which tcpdump &>/dev/null + if [ $? -ne 0 ];then + echo "selftests: [SKIP] Could not run test without tcpdump" + exit 4 + fi +} + +setup_ns() +{ + local mode=$1 + IFACES="" + + for i in $(seq $NUM); do + ip netns add ns$i + ip link add veth$i type veth peer name veth0 netns ns$i + ip link set veth$i up + ip -n ns$i link set veth0 up + + ip -n ns$i addr add 192.0.2.$i/24 dev veth0 + ip -n ns$i addr add 2001:db8::$i/64 dev veth0 + ip -n ns$i link set veth0 $mode obj \ + xdp_dummy.o sec xdp_dummy &> /dev/null || \ + { test_fail "Unable to load dummy xdp" && exit 1; } + IFACES="$IFACES veth$i" + done +} + +do_ping_tests() +{ + local mode=$1 + + # arp test + ip netns exec ns2 tcpdump -i veth0 -nn -l -e &> arp_ns1-2_${mode}.log & + ip netns exec ns3 tcpdump -i veth0 -nn -l -e &> arp_ns1-3_${mode}.log & + ip netns exec ns4 tcpdump -i veth0 -nn -l -e &> arp_ns1-4_${mode}.log & + ip netns exec ns1 ping 192.0.2.254 -c 4 &> /dev/null + sleep 2 + pkill -9 tcpdump + grep -q "Request who-has 192.0.2.254 tell 192.0.2.1" arp_ns1-2_${mode}.log && \ + test_pass "$mode arp ns1-2" || test_fail "$mode arp ns1-2" + grep -q "Request who-has 192.0.2.254 tell 192.0.2.1" arp_ns1-3_${mode}.log && \ + test_pass "$mode arp ns1-3" || test_fail "$mode arp ns1-3" + grep -q "Request who-has 192.0.2.254 tell 192.0.2.1" arp_ns1-4_${mode}.log && \ + test_pass "$mode arp ns1-4" || test_fail "$mode arp ns1-4" + + # ping test + ip netns exec ns1 ping 192.0.2.2 -c 4 &> /dev/null && \ + test_fail "$mode ping ns1-2" || test_pass "$mode ping ns1-2" + ip netns exec ns1 ping 192.0.2.3 -c 4 &> /dev/null && \ + test_fail "$mode ping ns1-3" || test_pass "$mode ping ns1-3" + ip netns exec ns1 ping 192.0.2.4 -c 4 &> /dev/null && \ + test_pass "$mode ping ns1-4" || test_fail "$mode ping ns1-4" + + # ping6 test + ip netns exec ns2 ping6 2001:db8::1 -c 4 &> /dev/null && \ + test_fail "$mode ping6 ns2-1" || test_pass "$mode ping6 ns2-1" + ip netns exec ns2 ping6 2001:db8::3 -c 4 &> /dev/null && \ + test_fail "$mode ping6 ns2-3" || test_pass "$mode ping6 ns2-3" + ip netns exec ns2 ping6 2001:db8::4 -c 4 &> /dev/null && \ + test_pass "$mode ping6 ns2-4" || test_fail "$mode ping6 ns2-4" +} + +do_tests() +{ + local mode=$1 + local drv_p + + [ ${mode} == "xdpdrv" ] && drv_p="-N" || drv_p="-S" + + # run `ulimit -l unlimited` if you got errors like + # libbpf: Error in bpf_object__probe_global_data():Operation not permitted(1). + ./xdp_redirect_multi $drv_p $IFACES &> xdp_redirect_${mode}.log & + xdp_pid=$! + sleep 10 + + do_ping_tests $mode + + kill $xdp_pid +} + +trap clean_up 0 2 3 6 9 + +check_env + +for mode in ${DRV_MODE}; do + setup_ns $mode + do_tests $mode + sleep 10 + clean_up + sleep 5 +done + +echo "Summary: PASS $PASS, FAIL $FAIL" +[ $FAIL -eq 0 ] && exit 0 || exit 1 diff --git a/tools/testing/selftests/bpf/xdp_redirect_multi.c b/tools/testing/selftests/bpf/xdp_redirect_multi.c new file mode 100644 index 000000000000..5626005cb679 --- /dev/null +++ b/tools/testing/selftests/bpf/xdp_redirect_multi.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "bpf_util.h" +#include +#include + +#define MAX_IFACE_NUM 32 + +static __u32 xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST; +static int ifaces[MAX_IFACE_NUM] = {}; + +static void int_exit(int sig) +{ + __u32 prog_id = 0; + int i; + + for (i = 0; ifaces[i] > 0; i++) { + if (bpf_get_link_xdp_id(ifaces[i], &prog_id, xdp_flags)) { + printf("bpf_get_link_xdp_id failed\n"); + exit(1); + } + if (prog_id) + bpf_set_link_xdp_fd(ifaces[i], -1, xdp_flags); + } + + exit(0); +} + +static void usage(const char *prog) +{ + fprintf(stderr, + "usage: %s [OPTS] ...\n" + "OPTS:\n" + " -S use skb-mode\n" + " -N enforce native mode\n" + " -F force loading prog\n", + prog); +} + +int main(int argc, char **argv) +{ + int prog_fd, group_all, group_v4, group_v6, exclude; + struct bpf_prog_load_attr prog_load_attr = { + .prog_type = BPF_PROG_TYPE_XDP, + }; + int i, ret, opt, ifindex; + char ifname[IF_NAMESIZE]; + struct bpf_object *obj; + char filename[256]; + + while ((opt = getopt(argc, argv, "SNF")) != -1) { + switch (opt) { + case 'S': + xdp_flags |= XDP_FLAGS_SKB_MODE; + break; + case 'N': + /* default, set below */ + break; + case 'F': + xdp_flags &= ~XDP_FLAGS_UPDATE_IF_NOEXIST; + break; + default: + usage(basename(argv[0])); + return 1; + } + } + + if (!(xdp_flags & XDP_FLAGS_SKB_MODE)) + xdp_flags |= XDP_FLAGS_DRV_MODE; + + if (optind == argc) { + printf("usage: %s ...\n", argv[0]); + return 1; + } + + printf("Get interfaces"); + for (i = 0; i < MAX_IFACE_NUM && argv[optind + i]; i++) { + ifaces[i] = if_nametoindex(argv[optind + i]); + if (!ifaces[i]) + ifaces[i] = strtoul(argv[optind + i], NULL, 0); + if (!if_indextoname(ifaces[i], ifname)) { + perror("Invalid interface name or i"); + return 1; + } + printf(" %d", ifaces[i]); + } + printf("\n"); + + snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]); + prog_load_attr.file = filename; + + if (bpf_prog_load_xattr(&prog_load_attr, &obj, &prog_fd)) + return 1; + + group_all = bpf_object__find_map_fd_by_name(obj, "forward_map_all"); + group_v4 = bpf_object__find_map_fd_by_name(obj, "forward_map_v4"); + group_v6 = bpf_object__find_map_fd_by_name(obj, "forward_map_v6"); + exclude = bpf_object__find_map_fd_by_name(obj, "exclude_map"); + + if (group_all < 0 || group_v4 < 0 || group_v6 < 0 || exclude < 0) { + printf("bpf_object__find_map_fd_by_name failed\n"); + return 1; + } + + signal(SIGINT, int_exit); + signal(SIGTERM, int_exit); + + /* Init forward multicast groups and exclude group */ + for (i = 0; ifaces[i] > 0; i++) { + ifindex = ifaces[i]; + + /* Add all the interfaces to group all */ + ret = bpf_map_update_elem(group_all, &ifindex, &ifindex, 0); + if (ret) { + perror("bpf_map_update_elem"); + goto err_out; + } + + /* For testing: remove the 1st interfaces from group v6 */ + if (i != 0) { + ret = bpf_map_update_elem(group_v6, &ifindex, &ifindex, 0); + if (ret) { + perror("bpf_map_update_elem"); + goto err_out; + } + } + + /* For testing: remove the 2nd interfaces from group v4 */ + if (i != 1) { + ret = bpf_map_update_elem(group_v4, &ifindex, &ifindex, 0); + if (ret) { + perror("bpf_map_update_elem"); + goto err_out; + } + } + + /* For testing: add the 3rd interfaces to exclude map */ + if (i == 2) { + ret = bpf_map_update_elem(exclude, &ifindex, &ifindex, 0); + if (ret) { + perror("bpf_map_update_elem"); + goto err_out; + } + } + + /* bind prog_fd to each interface */ + ret = bpf_set_link_xdp_fd(ifindex, prog_fd, xdp_flags); + if (ret) { + printf("Set xdp fd failed on %d\n", ifindex); + goto err_out; + } + + } + + /* sleep some time for testing */ + sleep(999); + + return 0; + +err_out: + return 1; +} From patchwork Wed Aug 26 13:20:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hangbin Liu X-Patchwork-Id: 1351946 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=nd8crma7; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bc61Y2YCbz9sT6 for ; Wed, 26 Aug 2020 23:20:49 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730257AbgHZNUs (ORCPT ); Wed, 26 Aug 2020 09:20:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730252AbgHZNUj (ORCPT ); Wed, 26 Aug 2020 09:20:39 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DCEE4C061756; Wed, 26 Aug 2020 06:20:39 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id s2so266215pjr.4; Wed, 26 Aug 2020 06:20:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YsZt/qgPuo31p7ki4/qp/orJWLUccxrYtZtCufXmmGM=; b=nd8crma76C9c+OzNQNWF/Hs0ItGfua4eRi6vBbJEFYLPOZaWXz0RVNtzLJrXIxLo6n ydvhtlvcFVD5/GiFQjVG/AR+d7vbhTP1/TzP4/dwt8OknNwbAtv2ah/kR3ts/F2TCTA1 limRyoqEnqYtQPs0vdK2IKeS56w2p2EiA+4m6SebFSktgLiJ/Rmu6xJ2xRTr4dlJT0ne 913nj5yza2gMJqxD6jUshu7QJLQAL09LIES3UvmEs+6TQMxSLmNDMJQ42mv3nPT/GsO6 x97aaGu0IiwJ0fNiWt/Cmk1kpw3Ziw0s+Z/5uh10q4Fz1TtdbvknUeM9OM0oxeTMCbZS pWwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YsZt/qgPuo31p7ki4/qp/orJWLUccxrYtZtCufXmmGM=; b=KVYrFQ00zNEELhMQuEZB8gWLeEBHSJs5mq6+p3LvD6+a3lu2xJnI6ry6xIinCD23Ol RJNtzTDPvhT6RzgjNNlrZpScNiiVbEiOUQsXvYbYWcdBC/tWw04MBpdOmuIj71vsDq3U siOCpjtB7CSkqk7+z0Ftz3CzUYbQsCi8+EZ3rqhR9mhKDPW+2TMGeXzk46ojQDnGQUp9 Sgsq8u2OJgV3SgDhBp8I7fE8gUvXISLIe/3OaKl3DbfQevAuQe8+iPk7FYKJFDhAsXKI cQ59hMZ9Cw6S1ai5WFXLmDj5F7vPxpgnO225KUdws55CP5rWVWNDmeVPX9PrsovClguO +utw== X-Gm-Message-State: AOAM531ojplYf+gkph6vyqj9/n6yggP3hCN87Ufu8ykytF2nIxCeYwfu 7ikoQIRFrLtY7aoBoqhsyy1OoEPHihHQQs/B X-Google-Smtp-Source: ABdhPJwIsEA/FKpvulKJkH73nYf91yvmAqW8UKzAPuJo1PKB14zmEcg+LBES1+YYJIlP2XinPTzsYw== X-Received: by 2002:a17:90a:c791:: with SMTP id gn17mr5784051pjb.44.1598448039168; Wed, 26 Aug 2020 06:20:39 -0700 (PDT) Received: from dhcp-12-153.nay.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id s129sm3131794pfb.39.2020.08.26.06.20.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Aug 2020 06:20:38 -0700 (PDT) From: Hangbin Liu To: bpf@vger.kernel.org Cc: netdev@vger.kernel.org, =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= , Jiri Benc , Jesper Dangaard Brouer , Eelco Chaudron , ast@kernel.org, Daniel Borkmann , Lorenzo Bianconi , David Ahern , Andrii Nakryiko B , Hangbin Liu Subject: [PATCHv9 bpf-next 5/5] selftests/bpf: Add verifier tests for bpf arg ARG_CONST_MAP_PTR_OR_NULL Date: Wed, 26 Aug 2020 21:20:02 +0800 Message-Id: <20200826132002.2808380-6-liuhangbin@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200826132002.2808380-1-liuhangbin@gmail.com> References: <20200715130816.2124232-1-liuhangbin@gmail.com> <20200826132002.2808380-1-liuhangbin@gmail.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Use helper bpf_redirect_map() and bpf_redirect_map_multi() to test bpf arg ARG_CONST_MAP_PTR and ARG_CONST_MAP_PTR_OR_NULL. Make sure the map arg could be verified correctly when it is NULL or valid map pointer. Add devmap and devmap_hash in struct bpf_test due to bpf_redirect_{map, map_multi} limit. Test result: ]# ./test_verifier 702 705 #702/p ARG_CONST_MAP_PTR: null pointer OK #703/p ARG_CONST_MAP_PTR: valid map pointer OK #704/p ARG_CONST_MAP_PTR_OR_NULL: null pointer for ex_map OK #705/p ARG_CONST_MAP_PTR_OR_NULL: valid map pointer for ex_map OK Summary: 4 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Hangbin Liu --- v1-v9: no update, new added --- tools/testing/selftests/bpf/test_verifier.c | 22 +++++- .../testing/selftests/bpf/verifier/map_ptr.c | 70 +++++++++++++++++++ 2 files changed, 91 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/test_verifier.c b/tools/testing/selftests/bpf/test_verifier.c index 9be395d9dc64..f89a13e60692 100644 --- a/tools/testing/selftests/bpf/test_verifier.c +++ b/tools/testing/selftests/bpf/test_verifier.c @@ -50,7 +50,7 @@ #define MAX_INSNS BPF_MAXINSNS #define MAX_TEST_INSNS 1000000 #define MAX_FIXUPS 8 -#define MAX_NR_MAPS 20 +#define MAX_NR_MAPS 22 #define MAX_TEST_RUNS 8 #define POINTER_VALUE 0xcafe4all #define TEST_DATA_LEN 64 @@ -87,6 +87,8 @@ struct bpf_test { int fixup_sk_storage_map[MAX_FIXUPS]; int fixup_map_event_output[MAX_FIXUPS]; int fixup_map_reuseport_array[MAX_FIXUPS]; + int fixup_map_devmap[MAX_FIXUPS]; + int fixup_map_devmap_hash[MAX_FIXUPS]; const char *errstr; const char *errstr_unpriv; uint32_t insn_processed; @@ -640,6 +642,8 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, int *fixup_sk_storage_map = test->fixup_sk_storage_map; int *fixup_map_event_output = test->fixup_map_event_output; int *fixup_map_reuseport_array = test->fixup_map_reuseport_array; + int *fixup_map_devmap = test->fixup_map_devmap; + int *fixup_map_devmap_hash = test->fixup_map_devmap_hash; if (test->fill_helper) { test->fill_insns = calloc(MAX_TEST_INSNS, sizeof(struct bpf_insn)); @@ -817,6 +821,22 @@ static void do_test_fixup(struct bpf_test *test, enum bpf_prog_type prog_type, fixup_map_reuseport_array++; } while (*fixup_map_reuseport_array); } + if (*fixup_map_devmap) { + map_fds[20] = __create_map(BPF_MAP_TYPE_DEVMAP, + sizeof(u32), sizeof(u32), 1, 0); + do { + prog[*fixup_map_devmap].imm = map_fds[20]; + fixup_map_devmap++; + } while (*fixup_map_devmap); + } + if (*fixup_map_devmap_hash) { + map_fds[21] = __create_map(BPF_MAP_TYPE_DEVMAP_HASH, + sizeof(u32), sizeof(u32), 1, 0); + do { + prog[*fixup_map_devmap_hash].imm = map_fds[21]; + fixup_map_devmap_hash++; + } while (*fixup_map_devmap_hash); + } } struct libcap { diff --git a/tools/testing/selftests/bpf/verifier/map_ptr.c b/tools/testing/selftests/bpf/verifier/map_ptr.c index b52209db8250..51df7d8784dc 100644 --- a/tools/testing/selftests/bpf/verifier/map_ptr.c +++ b/tools/testing/selftests/bpf/verifier/map_ptr.c @@ -60,3 +60,73 @@ .result = ACCEPT, .retval = 1, }, +{ + "ARG_CONST_MAP_PTR: null pointer", + .insns = { + /* bpf_redirect_map arg1 (map) */ + BPF_MOV64_IMM(BPF_REG_1, 0), + /* bpf_redirect_map arg2 (ifindex) */ + BPF_MOV64_IMM(BPF_REG_2, 0), + /* bpf_redirect_map arg3 (flags) */ + BPF_MOV64_IMM(BPF_REG_3, 0), + BPF_EMIT_CALL(BPF_FUNC_redirect_map), + BPF_EXIT_INSN(), + }, + .result = REJECT, + .prog_type = BPF_PROG_TYPE_XDP, + .errstr = "R1 type=inv expected=map_ptr", +}, +{ + "ARG_CONST_MAP_PTR: valid map pointer", + .insns = { + BPF_MOV64_IMM(BPF_REG_1, 0), + /* bpf_redirect_map arg1 (map) */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + /* bpf_redirect_map arg2 (ifindex) */ + BPF_MOV64_IMM(BPF_REG_2, 0), + /* bpf_redirect_map arg3 (flags) */ + BPF_MOV64_IMM(BPF_REG_3, 0), + BPF_EMIT_CALL(BPF_FUNC_redirect_map), + BPF_EXIT_INSN(), + }, + .fixup_map_devmap = { 1 }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_XDP, +}, +{ + "ARG_CONST_MAP_PTR_OR_NULL: null pointer for ex_map", + .insns = { + BPF_MOV64_IMM(BPF_REG_1, 0), + /* bpf_redirect_map_multi arg1 (in_map) */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + /* bpf_redirect_map_multi arg2 (ex_map) */ + BPF_MOV64_IMM(BPF_REG_2, 0), + /* bpf_redirect_map_multi arg3 (flags) */ + BPF_MOV64_IMM(BPF_REG_3, 0), + BPF_EMIT_CALL(BPF_FUNC_redirect_map_multi), + BPF_EXIT_INSN(), + }, + .fixup_map_devmap = { 1 }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_XDP, + .retval = 4, +}, +{ + "ARG_CONST_MAP_PTR_OR_NULL: valid map pointer for ex_map", + .insns = { + BPF_MOV64_IMM(BPF_REG_1, 0), + /* bpf_redirect_map_multi arg1 (in_map) */ + BPF_LD_MAP_FD(BPF_REG_1, 0), + /* bpf_redirect_map_multi arg2 (ex_map) */ + BPF_LD_MAP_FD(BPF_REG_2, 1), + /* bpf_redirect_map_multi arg3 (flags) */ + BPF_MOV64_IMM(BPF_REG_3, 0), + BPF_EMIT_CALL(BPF_FUNC_redirect_map_multi), + BPF_EXIT_INSN(), + }, + .fixup_map_devmap = { 1 }, + .fixup_map_devmap_hash = { 3 }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_XDP, + .retval = 4, +},