From patchwork Wed Aug 19 09:24:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347638 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=otdNTkSB; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjPc1WMbz9sPf for ; Wed, 19 Aug 2020 19:37:56 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727115AbgHSJhx (ORCPT ); Wed, 19 Aug 2020 05:37:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727042AbgHSJhr (ORCPT ); Wed, 19 Aug 2020 05:37:47 -0400 Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com [IPv6:2a00:1450:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40F2CC061342 for ; Wed, 19 Aug 2020 02:37:47 -0700 (PDT) Received: by mail-wr1-x444.google.com with SMTP id r2so20826877wrs.8 for ; Wed, 19 Aug 2020 02:37:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sAJPVYcsrbGx3N5LhQWWl+joMPHyHeZRVA3ccro29ww=; b=otdNTkSBdCH8NouPaZoJ/IoaSoOX4VJpc/vHghqMUvq3A0/XmtaP151GhG7DgdwGxr bnqWSyLJhzE9CdKrGzMvOqz/zt4s8DH5HbjiQVOn20o+5n82MPltEHrPuhi0HJTIHpcc gRhDcO7Nc4DoUZuH7GZtQRWPYjb47f9aAzMv4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sAJPVYcsrbGx3N5LhQWWl+joMPHyHeZRVA3ccro29ww=; b=hr+lII6prHjvDJJwiqH0SplyS4iPdqe4WdyoRW6Rnv4dISWj7CrNvaKLOCL8JuWIP1 q7tvYLyZ4+mSVeGLDEsbE8egL/CHnonaftX+/gjWpdy0NHjkj+1vIxgKjyTgO7aWY+NI XTMSdGuFxnxWEJmXeJMnfk73GN0bWyBV9EKYkfgYidaqXjVubxPQiTezT3aZSQG1kFTP 3zvpPtZNhvg8GNPg2gOE+NUCCiRykrAoKTdUHUQi7cJ5Md0E4QoV4c+8fXhY2eItrg73 iYP95/fP2puvyZP7POkZkj589ihRliBxeM53jqfp5XmVFnzYAglBJYjNiBctVA35qitR kqzQ== X-Gm-Message-State: AOAM532Uc6yoNwm04u7apxAIlUDTFPKpUBymDhcsQ5L9a/89WqDtEQ2Q LOszeD+XkrAub6a+KXsFZJiSwQ== X-Google-Smtp-Source: ABdhPJwH3BKtvXWM6LtGEcfQiLJDAcZ6ofcvTHAtBCdHgYGX0o6naRXxaXP3o7K6v2ojtD/Vd/n67w== X-Received: by 2002:a05:6000:150:: with SMTP id r16mr23638240wrx.63.1597829865824; Wed, 19 Aug 2020 02:37:45 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:45 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Daniel Borkmann , Lorenz Bauer , "David S. Miller" , Jakub Kicinski , Eric Dumazet , Alexey Kuznetsov , Hideaki YOSHIFUJI , Alexei Starovoitov Cc: kernel-team@cloudflare.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 1/6] net: sk_msg: simplify sk_psock initialization Date: Wed, 19 Aug 2020 10:24:31 +0100 Message-Id: <20200819092436.58232-2-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Initializing psock->sk_proto and other saved callbacks is only done in sk_psock_update_proto, after sk_psock_init has returned. The logic for this is difficult to follow, and needlessly complex. Instead, initialize psock->sk_proto whenever we allocate a new psock. Additionally, assert the following invariants: * The SK has no ULP: ULP does it's own finagling of sk->sk_prot * sk_user_data is unused: we need it to store sk_psock Protect our access to sk_user_data with sk_callback_lock, which is what other users like reuseport arrays, etc. do. The result is that an sk_psock is always fully initialized, and that psock->sk_proto is always the "original" struct proto. The latter allows us to use psock->sk_proto when initializing IPv6 TCP / UDP callbacks for sockmap. Signed-off-by: Lorenz Bauer Acked-by: John Fastabend --- include/linux/skmsg.h | 17 ----------------- net/core/skmsg.c | 34 ++++++++++++++++++++++++++++------ net/core/sock_map.c | 14 ++++---------- net/ipv4/tcp_bpf.c | 13 +++++-------- net/ipv4/udp_bpf.c | 9 ++++----- 5 files changed, 41 insertions(+), 46 deletions(-) diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h index 1e9ed840b9fc..3119928fc103 100644 --- a/include/linux/skmsg.h +++ b/include/linux/skmsg.h @@ -340,23 +340,6 @@ static inline void sk_psock_update_proto(struct sock *sk, struct sk_psock *psock, struct proto *ops) { - /* Initialize saved callbacks and original proto only once, since this - * function may be called multiple times for a psock, e.g. when - * psock->progs.msg_parser is updated. - * - * Since we've not installed the new proto, psock is not yet in use and - * we can initialize it without synchronization. - */ - if (!psock->sk_proto) { - struct proto *orig = READ_ONCE(sk->sk_prot); - - psock->saved_unhash = orig->unhash; - psock->saved_close = orig->close; - psock->saved_write_space = sk->sk_write_space; - - psock->sk_proto = orig; - } - /* Pairs with lockless read in sk_clone_lock() */ WRITE_ONCE(sk->sk_prot, ops); } diff --git a/net/core/skmsg.c b/net/core/skmsg.c index 6a32a1fd34f8..1c81caf9630f 100644 --- a/net/core/skmsg.c +++ b/net/core/skmsg.c @@ -494,14 +494,34 @@ static void sk_psock_backlog(struct work_struct *work) struct sk_psock *sk_psock_init(struct sock *sk, int node) { - struct sk_psock *psock = kzalloc_node(sizeof(*psock), - GFP_ATOMIC | __GFP_NOWARN, - node); - if (!psock) - return NULL; + struct sk_psock *psock; + struct proto *prot; + write_lock_bh(&sk->sk_callback_lock); + + if (inet_csk_has_ulp(sk)) { + psock = ERR_PTR(-EINVAL); + goto out; + } + + if (sk->sk_user_data) { + psock = ERR_PTR(-EBUSY); + goto out; + } + + psock = kzalloc_node(sizeof(*psock), GFP_ATOMIC | __GFP_NOWARN, node); + if (!psock) { + psock = ERR_PTR(-ENOMEM); + goto out; + } + + prot = READ_ONCE(sk->sk_prot); psock->sk = sk; - psock->eval = __SK_NONE; + psock->eval = __SK_NONE; + psock->sk_proto = prot; + psock->saved_unhash = prot->unhash; + psock->saved_close = prot->close; + psock->saved_write_space = sk->sk_write_space; INIT_LIST_HEAD(&psock->link); spin_lock_init(&psock->link_lock); @@ -516,6 +536,8 @@ struct sk_psock *sk_psock_init(struct sock *sk, int node) rcu_assign_sk_user_data_nocopy(sk, psock); sock_hold(sk); +out: + write_unlock_bh(&sk->sk_callback_lock); return psock; } EXPORT_SYMBOL_GPL(sk_psock_init); diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 119f52a99dc1..abe4bac40db9 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -184,8 +184,6 @@ static int sock_map_init_proto(struct sock *sk, struct sk_psock *psock) { struct proto *prot; - sock_owned_by_me(sk); - switch (sk->sk_type) { case SOCK_STREAM: prot = tcp_bpf_get_proto(sk, psock); @@ -272,8 +270,8 @@ static int sock_map_link(struct bpf_map *map, struct sk_psock_progs *progs, } } else { psock = sk_psock_init(sk, map->numa_node); - if (!psock) { - ret = -ENOMEM; + if (IS_ERR(psock)) { + ret = PTR_ERR(psock); goto out_progs; } } @@ -322,8 +320,8 @@ static int sock_map_link_no_progs(struct bpf_map *map, struct sock *sk) if (!psock) { psock = sk_psock_init(sk, map->numa_node); - if (!psock) - return -ENOMEM; + if (IS_ERR(psock)) + return PTR_ERR(psock); } ret = sock_map_init_proto(sk, psock); @@ -478,8 +476,6 @@ static int sock_map_update_common(struct bpf_map *map, u32 idx, return -EINVAL; if (unlikely(idx >= map->max_entries)) return -E2BIG; - if (inet_csk_has_ulp(sk)) - return -EINVAL; link = sk_psock_init_link(); if (!link) @@ -855,8 +851,6 @@ static int sock_hash_update_common(struct bpf_map *map, void *key, WARN_ON_ONCE(!rcu_read_lock_held()); if (unlikely(flags > BPF_EXIST)) return -EINVAL; - if (inet_csk_has_ulp(sk)) - return -EINVAL; link = sk_psock_init_link(); if (!link) diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index 7aa68f4aae6c..37f4cb2bba5c 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -567,10 +567,9 @@ static void tcp_bpf_rebuild_protos(struct proto prot[TCP_BPF_NUM_CFGS], prot[TCP_BPF_TX].sendpage = tcp_bpf_sendpage; } -static void tcp_bpf_check_v6_needs_rebuild(struct sock *sk, struct proto *ops) +static void tcp_bpf_check_v6_needs_rebuild(struct proto *ops) { - if (sk->sk_family == AF_INET6 && - unlikely(ops != smp_load_acquire(&tcpv6_prot_saved))) { + if (unlikely(ops != smp_load_acquire(&tcpv6_prot_saved))) { spin_lock_bh(&tcpv6_prot_lock); if (likely(ops != tcpv6_prot_saved)) { tcp_bpf_rebuild_protos(tcp_bpf_prots[TCP_BPF_IPV6], ops); @@ -603,13 +602,11 @@ struct proto *tcp_bpf_get_proto(struct sock *sk, struct sk_psock *psock) int family = sk->sk_family == AF_INET6 ? TCP_BPF_IPV6 : TCP_BPF_IPV4; int config = psock->progs.msg_parser ? TCP_BPF_TX : TCP_BPF_BASE; - if (!psock->sk_proto) { - struct proto *ops = READ_ONCE(sk->sk_prot); - - if (tcp_bpf_assert_proto_ops(ops)) + if (sk->sk_family == AF_INET6) { + if (tcp_bpf_assert_proto_ops(psock->sk_proto)) return ERR_PTR(-EINVAL); - tcp_bpf_check_v6_needs_rebuild(sk, ops); + tcp_bpf_check_v6_needs_rebuild(psock->sk_proto); } return &tcp_bpf_prots[family][config]; diff --git a/net/ipv4/udp_bpf.c b/net/ipv4/udp_bpf.c index eddd973e6575..7a94791efc1a 100644 --- a/net/ipv4/udp_bpf.c +++ b/net/ipv4/udp_bpf.c @@ -22,10 +22,9 @@ static void udp_bpf_rebuild_protos(struct proto *prot, const struct proto *base) prot->close = sock_map_close; } -static void udp_bpf_check_v6_needs_rebuild(struct sock *sk, struct proto *ops) +static void udp_bpf_check_v6_needs_rebuild(struct proto *ops) { - if (sk->sk_family == AF_INET6 && - unlikely(ops != smp_load_acquire(&udpv6_prot_saved))) { + if (unlikely(ops != smp_load_acquire(&udpv6_prot_saved))) { spin_lock_bh(&udpv6_prot_lock); if (likely(ops != udpv6_prot_saved)) { udp_bpf_rebuild_protos(&udp_bpf_prots[UDP_BPF_IPV6], ops); @@ -46,8 +45,8 @@ struct proto *udp_bpf_get_proto(struct sock *sk, struct sk_psock *psock) { int family = sk->sk_family == AF_INET ? UDP_BPF_IPV4 : UDP_BPF_IPV6; - if (!psock->sk_proto) - udp_bpf_check_v6_needs_rebuild(sk, READ_ONCE(sk->sk_prot)); + if (sk->sk_family == AF_INET6) + udp_bpf_check_v6_needs_rebuild(psock->sk_proto); return &udp_bpf_prots[family]; } From patchwork Wed Aug 19 09:24:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347643 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=Ik1xBFOY; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjQ81ZLGz9sTW for ; Wed, 19 Aug 2020 19:38:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727840AbgHSJiW (ORCPT ); Wed, 19 Aug 2020 05:38:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726876AbgHSJhu (ORCPT ); Wed, 19 Aug 2020 05:37:50 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6D35C061344 for ; Wed, 19 Aug 2020 02:37:48 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id c15so20782892wrs.11 for ; Wed, 19 Aug 2020 02:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WiWRWCNZNwwliRoiTcSOlw+lhqkkpwfiYRA5mQGxIvs=; b=Ik1xBFOYB1oLNrhbKRJfScNWGubdlq191L2B4NjYIPulQZJcQPCjvxJfSPRW4zp0GK HPn5WDA1HimM6dkKxefxcxZncDhCNvZeh4drUEtFPScHtgaCGPslxL5MYxGcPi1H24uN 3IL4QxUHMPqQPMZtocn1D1Y4izV7fpIxZaiPs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WiWRWCNZNwwliRoiTcSOlw+lhqkkpwfiYRA5mQGxIvs=; b=rjIEjlHzLsvl8wnfuT8ef7v+LsPLcLtygAp7W/nqHI29qEclzSQVA7D/iTpTQOedX2 1uiLVHt90P3fFHxLmzsnsUNMBX2ZTYmVtqBIaLIabDhBf8kwsrhriaPmCAj1QB9thQGU miGZW89Gyng9AsX2QTqAVN3VvEjGTMZJewnWvNQfEorACSrNJrU42MR6+M9DoHDzBdBb P3uO0ApBv7mNqEDmdY6ObJCeFC+KKjL0/OoS252+zucipAJo84LDr7BSUNg0Kvn4Iv+V zo9T1yAmXLzQGnov0DtnnI8tT56I5TQle/RTJxwMXpe0TTb86dwCdpAEMPWCz4e0/1nA 2gVw== X-Gm-Message-State: AOAM533IA8t41GZkxQqKnagrD1F47aYLNI5ldb77DAkB0wqJdfoB7EQz oxLhsTNdofTY9APZWGGImuqJiA== X-Google-Smtp-Source: ABdhPJySI28r+GBXPnpyhbIrRBb8xxRx5WXhP//f4XWwnNBnnNvfO5NxPeNkEKOgTPqNsb1zjAbBRg== X-Received: by 2002:a5d:6381:: with SMTP id p1mr23604730wru.112.1597829867403; Wed, 19 Aug 2020 02:37:47 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:46 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Daniel Borkmann , Lorenz Bauer , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov Cc: kernel-team@cloudflare.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 2/6] bpf: sockmap: merge sockmap and sockhash update functions Date: Wed, 19 Aug 2020 10:24:32 +0100 Message-Id: <20200819092436.58232-3-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Merge the two very similar functions sock_map_update_elem and sock_hash_update_elem into one. Signed-off-by: Lorenz Bauer Acked-by: Yonghong Song Acked-by: John Fastabend --- net/core/sock_map.c | 53 ++++++++------------------------------------- 1 file changed, 9 insertions(+), 44 deletions(-) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index abe4bac40db9..f464a0ebc871 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -559,10 +559,12 @@ static bool sock_map_sk_state_allowed(const struct sock *sk) return false; } -static int sock_map_update_elem(struct bpf_map *map, void *key, - void *value, u64 flags) +static int sock_hash_update_common(struct bpf_map *map, void *key, + struct sock *sk, u64 flags); + +int sock_map_update_elem(struct bpf_map *map, void *key, + void *value, u64 flags) { - u32 idx = *(u32 *)key; struct socket *sock; struct sock *sk; int ret; @@ -591,8 +593,10 @@ static int sock_map_update_elem(struct bpf_map *map, void *key, sock_map_sk_acquire(sk); if (!sock_map_sk_state_allowed(sk)) ret = -EOPNOTSUPP; + else if (map->map_type == BPF_MAP_TYPE_SOCKMAP) + ret = sock_map_update_common(map, *(u32 *)key, sk, flags); else - ret = sock_map_update_common(map, idx, sk, flags); + ret = sock_hash_update_common(map, key, sk, flags); sock_map_sk_release(sk); out: fput(sock->file); @@ -909,45 +913,6 @@ static int sock_hash_update_common(struct bpf_map *map, void *key, return ret; } -static int sock_hash_update_elem(struct bpf_map *map, void *key, - void *value, u64 flags) -{ - struct socket *sock; - struct sock *sk; - int ret; - u64 ufd; - - if (map->value_size == sizeof(u64)) - ufd = *(u64 *)value; - else - ufd = *(u32 *)value; - if (ufd > S32_MAX) - return -EINVAL; - - sock = sockfd_lookup(ufd, &ret); - if (!sock) - return ret; - sk = sock->sk; - if (!sk) { - ret = -EINVAL; - goto out; - } - if (!sock_map_sk_is_suitable(sk)) { - ret = -EOPNOTSUPP; - goto out; - } - - sock_map_sk_acquire(sk); - if (!sock_map_sk_state_allowed(sk)) - ret = -EOPNOTSUPP; - else - ret = sock_hash_update_common(map, key, sk, flags); - sock_map_sk_release(sk); -out: - fput(sock->file); - return ret; -} - static int sock_hash_get_next_key(struct bpf_map *map, void *key, void *key_next) { @@ -1216,7 +1181,7 @@ const struct bpf_map_ops sock_hash_ops = { .map_alloc = sock_hash_alloc, .map_free = sock_hash_free, .map_get_next_key = sock_hash_get_next_key, - .map_update_elem = sock_hash_update_elem, + .map_update_elem = sock_map_update_elem, .map_delete_elem = sock_hash_delete_elem, .map_lookup_elem = sock_hash_lookup, .map_lookup_elem_sys_only = sock_hash_lookup_sys, From patchwork Wed Aug 19 09:24:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347644 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=S2mA7NIz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjQF2vBmz9sPf for ; Wed, 19 Aug 2020 19:38:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727069AbgHSJiV (ORCPT ); Wed, 19 Aug 2020 05:38:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727074AbgHSJhw (ORCPT ); Wed, 19 Aug 2020 05:37:52 -0400 Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54C7BC061347 for ; Wed, 19 Aug 2020 02:37:50 -0700 (PDT) Received: by mail-wm1-x344.google.com with SMTP id g8so1403087wmk.3 for ; Wed, 19 Aug 2020 02:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gkrehb+NY6EpSKSWw1O3tHpluUDTjNcCIKKF3vkOvVk=; b=S2mA7NIzqZi2WbjNSenaPFYnG7D8TnMSSBbnZwpsfy0dPWN2nmJ4rf0GcxhmwlInyF i+j8y/QAZ8poyhKF6kgfQFa9tfujr7DPl1twJCmzHkuNlmw9i0Kj5JR0j5nYGrZv1HEk ZtWInXBNLA7TyA/6YZWg+Cz+iwuP/8L1GmpwI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gkrehb+NY6EpSKSWw1O3tHpluUDTjNcCIKKF3vkOvVk=; b=e/oycyOWCb/FDZN5J1C8NfYpW71Rf5CefInFn/NpjmfDwNcH0iMa7y0ybyPFtjSQWV EssCprQQUw3r/nL1vKZ1U86JO09e2n3D8tSwZvYoM5X3vu94TCDQHlLzZQEdOfRR3qQS Zdm+VNbZxZTf3V9szFOKw0xIVlrBZPlObrd1DGQGro4pS8vaMc47b3ppbKpmOWcNX4TA MOdPoUJmOBwoCPdAwTf+25CTJjR4f2wQEejsQMuiAPTimCUyBFOs3bZ7ANHaP3JDFrLJ faN58Cz8k1gU0/7ZeBLcwVdUrkA4VHUbcxxGDZixE3okvL89BIN/5kAf+jBLPa6RyHSq +JVQ== X-Gm-Message-State: AOAM533WQ2gU0Qmt2sANsjtS6t8qY8AcY5enTWv8Dryg30i8XqEoWQRB W4EjaQ8WrGCgxO6vgBcrVgJ7Jw== X-Google-Smtp-Source: ABdhPJz2KAAKA5XiONSTOgNjMoq0Ddnp/ynNK45lDn4chmtOLuEPVbwthqYKle4pW6HCBNFADw1hBg== X-Received: by 2002:a1c:24d5:: with SMTP id k204mr3914788wmk.159.1597829869028; Wed, 19 Aug 2020 02:37:49 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:48 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Alexei Starovoitov , Daniel Borkmann , Lorenz Bauer , "David S. Miller" , Jakub Kicinski Cc: kernel-team@cloudflare.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 3/6] bpf: sockmap: call sock_map_update_elem directly Date: Wed, 19 Aug 2020 10:24:33 +0100 Message-Id: <20200819092436.58232-4-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Don't go via map->ops to call sock_map_update_elem, since we know what function to call in bpf_map_update_value. Since check_map_func_compatibility doesn't allow calling BPF_FUNC_map_update_elem from BPF context, we can remove ops->map_update_elem and rename the function to sock_map_update_elem_sys. Signed-off-by: Lorenz Bauer Acked-by: Yonghong Song Acked-by: John Fastabend --- include/linux/bpf.h | 7 +++++++ kernel/bpf/syscall.c | 5 +++-- net/core/sock_map.c | 6 ++---- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index cef4ef0d2b4e..cf3416d1b8c2 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1635,6 +1635,7 @@ int sock_map_prog_update(struct bpf_map *map, struct bpf_prog *prog, struct bpf_prog *old, u32 which); int sock_map_get_from_fd(const union bpf_attr *attr, struct bpf_prog *prog); int sock_map_prog_detach(const union bpf_attr *attr, enum bpf_prog_type ptype); +int sock_map_update_elem_sys(struct bpf_map *map, void *key, void *value, u64 flags); void sock_map_unhash(struct sock *sk); void sock_map_close(struct sock *sk, long timeout); #else @@ -1656,6 +1657,12 @@ static inline int sock_map_prog_detach(const union bpf_attr *attr, { return -EOPNOTSUPP; } + +static inline int sock_map_update_elem_sys(struct bpf_map *map, void *key, void *value, + u64 flags) +{ + return -EOPNOTSUPP; +} #endif /* CONFIG_BPF_STREAM_PARSER */ #if defined(CONFIG_INET) && defined(CONFIG_BPF_SYSCALL) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 2f343ce15747..5867cf615a3c 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -157,10 +157,11 @@ static int bpf_map_update_value(struct bpf_map *map, struct fd f, void *key, if (bpf_map_is_dev_bound(map)) { return bpf_map_offload_update_elem(map, key, value, flags); } else if (map->map_type == BPF_MAP_TYPE_CPUMAP || - map->map_type == BPF_MAP_TYPE_SOCKHASH || - map->map_type == BPF_MAP_TYPE_SOCKMAP || map->map_type == BPF_MAP_TYPE_STRUCT_OPS) { return map->ops->map_update_elem(map, key, value, flags); + } else if (map->map_type == BPF_MAP_TYPE_SOCKHASH || + map->map_type == BPF_MAP_TYPE_SOCKMAP) { + return sock_map_update_elem_sys(map, key, value, flags); } else if (IS_FD_PROG_ARRAY(map)) { return bpf_fd_array_map_update_elem(map, f.file, key, value, flags); diff --git a/net/core/sock_map.c b/net/core/sock_map.c index f464a0ebc871..018367fb889f 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -562,8 +562,8 @@ static bool sock_map_sk_state_allowed(const struct sock *sk) static int sock_hash_update_common(struct bpf_map *map, void *key, struct sock *sk, u64 flags); -int sock_map_update_elem(struct bpf_map *map, void *key, - void *value, u64 flags) +int sock_map_update_elem_sys(struct bpf_map *map, void *key, + void *value, u64 flags) { struct socket *sock; struct sock *sk; @@ -687,7 +687,6 @@ const struct bpf_map_ops sock_map_ops = { .map_free = sock_map_free, .map_get_next_key = sock_map_get_next_key, .map_lookup_elem_sys_only = sock_map_lookup_sys, - .map_update_elem = sock_map_update_elem, .map_delete_elem = sock_map_delete_elem, .map_lookup_elem = sock_map_lookup, .map_release_uref = sock_map_release_progs, @@ -1181,7 +1180,6 @@ const struct bpf_map_ops sock_hash_ops = { .map_alloc = sock_hash_alloc, .map_free = sock_hash_free, .map_get_next_key = sock_hash_get_next_key, - .map_update_elem = sock_map_update_elem, .map_delete_elem = sock_hash_delete_elem, .map_lookup_elem = sock_hash_lookup, .map_lookup_elem_sys_only = sock_hash_lookup_sys, From patchwork Wed Aug 19 09:24:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347640 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=EqDQKOzO; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjPr0dcxz9sTW for ; Wed, 19 Aug 2020 19:38:08 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727064AbgHSJiG (ORCPT ); Wed, 19 Aug 2020 05:38:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727125AbgHSJhz (ORCPT ); Wed, 19 Aug 2020 05:37:55 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF940C06134A for ; Wed, 19 Aug 2020 02:37:51 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id 9so1391194wmj.5 for ; Wed, 19 Aug 2020 02:37:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5kro16pmJ95iM/u9rOMoUVfxNBy72CdD/uqKbTNgUOo=; b=EqDQKOzOCBOt39rsYZLeQz04NqKOPVw3IJCRpDYkOHzJIFsi0Mvn/Z/Tf5HDiRArdc rpm9TbA1aGG+iChFutrOuLrkfuaGlguDEsGu58cMjk42Y7zu0c5Y3ykOwOu8S4/8sllW GErA935YNJs+JFIu542hS5oSXbhcUNs1E/8zQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5kro16pmJ95iM/u9rOMoUVfxNBy72CdD/uqKbTNgUOo=; b=pfnBE3Igmh15ddnwjJ6N81SmyamUwAMsKY/jmCQrFxFU9z8PiX0WwaqAO6XL1N4+eI Yxl6n3iPo5GFMfPwgESI6uiuGd3q+WU1uXRWu5dZh+s//BMlUIMhW6aewZ3twS0tS46k 6mseOb2QPvmBwr0zG02PYlS9Q9gh2nIoZtj1BH6u5Cz7Duig/Qo8lnNrjp8QvUz1ly2r Q7ORoQBd1hcKGsmqH2MvXPlZFsLrgSyyQMo90lqcyHNQA3lRDpUonF44T9T52aj760At llhGWqIdwOR7LkIJ5EqOv2jtQW/w4YD6B540u4n648ZA4hCFEVTLcvih7AGsb9OkYz9o inmA== X-Gm-Message-State: AOAM530FkU1iFaSgsBsN1tp8XLR3itMab9iC/CDK+LbNTCe2og3WQFFr OASuxz7NRBQtAkuBdx3S2kLSbQ== X-Google-Smtp-Source: ABdhPJyFZQOMNl3FbJ8gIJxc6cc1wD74uK1824MgJXDkM5LPxqm7GuxvxcjADiCDF5pk8Th2+EU2IA== X-Received: by 2002:a1c:6689:: with SMTP id a131mr3892270wmc.157.1597829870465; Wed, 19 Aug 2020 02:37:50 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:49 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Alexei Starovoitov , Daniel Borkmann Cc: kernel-team@cloudflare.com, Lorenz Bauer , netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 4/6] bpf: override the meaning of ARG_PTR_TO_MAP_VALUE for sockmap and sockhash Date: Wed, 19 Aug 2020 10:24:34 +0100 Message-Id: <20200819092436.58232-5-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org The verifier assumes that map values are simple blobs of memory, and therefore treats ARG_PTR_TO_MAP_VALUE, etc. as such. However, there are map types where this isn't true. For example, sockmap and sockhash store sockets. In general this isn't a big problem: we can just write helpers that explicitly requests PTR_TO_SOCKET instead of ARG_PTR_TO_MAP_VALUE. The one exception are the standard map helpers like map_update_elem, map_lookup_elem, etc. Here it would be nice we could overload the function prototype for different kinds of maps. Unfortunately, this isn't entirely straight forward: We only know the type of the map once we have resolved meta->map_ptr in check_func_arg. This means we can't swap out the prototype in check_helper_call until we're half way through the function. Instead, modify check_func_arg to treat ARG_PTR_TO_MAP_VALUE* to mean "the native type for the map" instead of "pointer to memory" for sockmap and sockhash. This means we don't have to modify the function prototype at all Signed-off-by: Lorenz Bauer --- kernel/bpf/verifier.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index b6ccfce3bf4c..47f9b94bb9d4 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3872,6 +3872,38 @@ static int int_ptr_type_to_size(enum bpf_arg_type type) return -EINVAL; } +static int override_map_arg_type(struct bpf_verifier_env *env, + const struct bpf_call_arg_meta *meta, + enum bpf_arg_type *arg_type) +{ + if (!meta->map_ptr) { + /* kernel subsystem misconfigured verifier */ + verbose(env, "invalid map_ptr to access map->type\n"); + return -EACCES; + } + + switch (meta->map_ptr->map_type) { + case BPF_MAP_TYPE_SOCKMAP: + case BPF_MAP_TYPE_SOCKHASH: + switch (*arg_type) { + case ARG_PTR_TO_MAP_VALUE: + *arg_type = ARG_PTR_TO_SOCKET; + break; + case ARG_PTR_TO_MAP_VALUE_OR_NULL: + *arg_type = ARG_PTR_TO_SOCKET_OR_NULL; + break; + default: + verbose(env, "invalid arg_type for sockmap/sockhash\n"); + return -EINVAL; + } + break; + + default: + break; + } + return 0; +} + static int check_func_arg(struct bpf_verifier_env *env, u32 arg, struct bpf_call_arg_meta *meta, const struct bpf_func_proto *fn) @@ -3904,6 +3936,14 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return -EACCES; } + if (arg_type == ARG_PTR_TO_MAP_VALUE || + arg_type == ARG_PTR_TO_UNINIT_MAP_VALUE || + arg_type == ARG_PTR_TO_MAP_VALUE_OR_NULL) { + err = override_map_arg_type(env, meta, &arg_type); + if (err) + return err; + } + if (arg_type == ARG_PTR_TO_MAP_KEY || arg_type == ARG_PTR_TO_MAP_VALUE || arg_type == ARG_PTR_TO_UNINIT_MAP_VALUE || From patchwork Wed Aug 19 09:24:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347642 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=Rfhvso3N; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjQ25Y6Mz9sTW for ; Wed, 19 Aug 2020 19:38:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727113AbgHSJiP (ORCPT ); Wed, 19 Aug 2020 05:38:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727772AbgHSJhz (ORCPT ); Wed, 19 Aug 2020 05:37:55 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79C18C061757 for ; Wed, 19 Aug 2020 02:37:53 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id x5so1402704wmi.2 for ; Wed, 19 Aug 2020 02:37:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ue3HR4wod0zNE7CrQUQ6Ehp/OR8t+fJlFQGIBrnJpsQ=; b=Rfhvso3NOUQY38ai/WyXwxP2YDeNCMIDRXmyKlgbLBMHkJMlOQqpezKRmyPSc8yE5l AByj7j2lupGigIB1Sbv2NCH5gmhw9szstkADVs3B8XOOH4xvAL/1WJbu6Y1Q492zCM3C Kga7QxNepaIpc5TzhWJpd+NO3oPe+g4sBseQo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ue3HR4wod0zNE7CrQUQ6Ehp/OR8t+fJlFQGIBrnJpsQ=; b=pGPfaJPEZDxe6yJ7ju2hctie28nVuOlhaSafl6w8gpaoy+fZCeeOx6kt0heiI75+NY MDE01hIuygj9D7Pd+e/ZFybgufNaOdTj3c7bvWvUrhcKhTUkOT/n1nsZEH+sHcNpHq8W o7ihdqujyfGWwvEwLAv6zp3sAGWPhlTakvo4SQkpwAhzC0Cn0G2rBD4TFcU41HzGhUw2 IkTjR+qIEeiNQxWNeNjRJO9ezhVnNG4kagDurdGj//5ZqjODJhf1SzQ7XLDdMUnpoKUZ 62d5h+ZF+2WqYpOII7cI2R7p35rNHymgbm5KPlvSJHf3xo3Hc7f3n86dqJ6TVpQ++SQ3 PhpA== X-Gm-Message-State: AOAM530Q33TNIWAsWCkDUWXiC74jIG8ISuyt2h3QsKD2yeu7GMvEH8lz RVuRbwcpLbK+fmj8JkXg0xMniA== X-Google-Smtp-Source: ABdhPJyfOTr3NfoG4pa7vgJMAO71CXb8trGz1d/9B8xOJxYcpPa2cnYahVhVGBZH7ZVo9P+QPxqnTA== X-Received: by 2002:a1c:4e10:: with SMTP id g16mr3787777wmh.146.1597829872125; Wed, 19 Aug 2020 02:37:52 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:51 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Alexei Starovoitov , Daniel Borkmann , Lorenz Bauer , "David S. Miller" , Jakub Kicinski Cc: kernel-team@cloudflare.com, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH bpf-next 5/6] bpf: sockmap: allow update from BPF Date: Wed, 19 Aug 2020 10:24:35 +0100 Message-Id: <20200819092436.58232-6-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Allow calling bpf_map_update_elem on sockmap and sockhash from a BPF context. The synchronization required for this is a bit fiddly: we need to prevent the socket from changing it's state while we add it to the sockmap, since we rely on getting a callback via sk_prot->unhash. However, we can't just lock_sock like in sock_map_sk_acquire because that might sleep. So instead we disable softirq processing and use bh_lock_sock to prevent further modification. Signed-off-by: Lorenz Bauer Acked-by: Yonghong Song --- kernel/bpf/verifier.c | 6 ++++-- net/core/sock_map.c | 24 ++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 47f9b94bb9d4..421fccf18dea 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -4254,7 +4254,8 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, func_id != BPF_FUNC_map_delete_elem && func_id != BPF_FUNC_msg_redirect_map && func_id != BPF_FUNC_sk_select_reuseport && - func_id != BPF_FUNC_map_lookup_elem) + func_id != BPF_FUNC_map_lookup_elem && + func_id != BPF_FUNC_map_update_elem) goto error; break; case BPF_MAP_TYPE_SOCKHASH: @@ -4263,7 +4264,8 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, func_id != BPF_FUNC_map_delete_elem && func_id != BPF_FUNC_msg_redirect_hash && func_id != BPF_FUNC_sk_select_reuseport && - func_id != BPF_FUNC_map_lookup_elem) + func_id != BPF_FUNC_map_lookup_elem && + func_id != BPF_FUNC_map_update_elem) goto error; break; case BPF_MAP_TYPE_REUSEPORT_SOCKARRAY: diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 018367fb889f..b2c886c34566 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -603,6 +603,28 @@ int sock_map_update_elem_sys(struct bpf_map *map, void *key, return ret; } +static int sock_map_update_elem(struct bpf_map *map, void *key, + void *value, u64 flags) +{ + struct sock *sk = (struct sock *)value; + int ret; + + if (!sock_map_sk_is_suitable(sk)) + return -EOPNOTSUPP; + + local_bh_disable(); + bh_lock_sock(sk); + if (!sock_map_sk_state_allowed(sk)) + ret = -EOPNOTSUPP; + else if (map->map_type == BPF_MAP_TYPE_SOCKMAP) + ret = sock_map_update_common(map, *(u32 *)key, sk, flags); + else + ret = sock_hash_update_common(map, key, sk, flags); + bh_unlock_sock(sk); + local_bh_enable(); + return ret; +} + BPF_CALL_4(bpf_sock_map_update, struct bpf_sock_ops_kern *, sops, struct bpf_map *, map, void *, key, u64, flags) { @@ -687,6 +709,7 @@ const struct bpf_map_ops sock_map_ops = { .map_free = sock_map_free, .map_get_next_key = sock_map_get_next_key, .map_lookup_elem_sys_only = sock_map_lookup_sys, + .map_update_elem = sock_map_update_elem, .map_delete_elem = sock_map_delete_elem, .map_lookup_elem = sock_map_lookup, .map_release_uref = sock_map_release_progs, @@ -1180,6 +1203,7 @@ const struct bpf_map_ops sock_hash_ops = { .map_alloc = sock_hash_alloc, .map_free = sock_hash_free, .map_get_next_key = sock_hash_get_next_key, + .map_update_elem = sock_map_update_elem, .map_delete_elem = sock_hash_delete_elem, .map_lookup_elem = sock_hash_lookup, .map_lookup_elem_sys_only = sock_hash_lookup_sys, From patchwork Wed Aug 19 09:24:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1347648 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=rBXwlDqm; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BWjQQ0xdBz9sPf for ; Wed, 19 Aug 2020 19:38:38 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727898AbgHSJib (ORCPT ); Wed, 19 Aug 2020 05:38:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727817AbgHSJh6 (ORCPT ); Wed, 19 Aug 2020 05:37:58 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B133C061344 for ; Wed, 19 Aug 2020 02:37:55 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id r15so10897013wrp.13 for ; Wed, 19 Aug 2020 02:37:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FDpi2QIDOUNs+oTRdf9VTGQOJTmQUxGd6EEehSUSOqc=; b=rBXwlDqmQW2mi1eDwy4yCmDy+M/G0KgX4nrRLieMNgsDNPMtj48MeMupJpAkXcSfmW UE6FXBfkK6LLbyyWSoydJcxOERfic38uXyWnijiDTbbfTCAqZ48Hex2oF7ewCoviuE08 57gSGtLv9G0fKd7/xffCssHOyLaPVJTw9/o00= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FDpi2QIDOUNs+oTRdf9VTGQOJTmQUxGd6EEehSUSOqc=; b=h+Ir4YVs3vRMQtdZyd4WCpfWQjF/OCPSpeerT5sXcBLoqcHinJe9g7EV4jIa9s18Tv R5FT3PuZ/JCe1pI5wLMStLyNtClYPs0sE046WEnHB4KBQeJMZ8xTM8/ISJZEsvtduMwu zEBKS7f0n8yOas0F1SOAXN006qM/Or0oJ91O2hFs6cgQbi51wd+VbLDIIfwrgq9yB4kG Bh8RjCK/SvPYWYGk9VgsRzNlfJ37aF6KOJCBbTwpXeirbIo8C2+mGyAPuLQNQlxUBIie NudxpCluyBTzxIw27c9B3V1GpEnJPfCQXVMzHEuSce0xlq5hYOGnOgrQnVrK+ojaVWf/ 1HmA== X-Gm-Message-State: AOAM530N6ueilX7Op4lbzUHuOC5vN6HBHE/ztiJI1nc4OoZ1na4pgB5E HeRrz00i7VOW8B1nVLC5PknauQ== X-Google-Smtp-Source: ABdhPJzJ5o5wV5OdLa3eRVHCRMu61AKEaX+ywMO2j7xpboPjL0u2x/kdvZgCLds8X1cMmb4o0Mr4Sg== X-Received: by 2002:adf:9526:: with SMTP id 35mr25432425wrs.326.1597829874068; Wed, 19 Aug 2020 02:37:54 -0700 (PDT) Received: from antares.lan (c.d.0.4.4.2.3.3.e.9.1.6.6.d.0.6.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:60d6:619e:3324:40dc]) by smtp.gmail.com with ESMTPSA id 3sm4204565wms.36.2020.08.19.02.37.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2020 02:37:53 -0700 (PDT) From: Lorenz Bauer To: jakub@cloudflare.com, john.fastabend@gmail.com, Shuah Khan , Alexei Starovoitov , Daniel Borkmann Cc: kernel-team@cloudflare.com, Lorenz Bauer , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH bpf-next 6/6] selftests: bpf: test sockmap update from BPF Date: Wed, 19 Aug 2020 10:24:36 +0100 Message-Id: <20200819092436.58232-7-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200819092436.58232-1-lmb@cloudflare.com> References: <20200819092436.58232-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a test which copies a socket from a sockmap into another sockmap or sockhash. This excercises bpf_map_update_elem support from BPF context. Compare the socket cookies from source and destination to ensure that the copy succeeded. Signed-off-by: Lorenz Bauer --- .../selftests/bpf/prog_tests/sockmap_basic.c | 76 +++++++++++++++++++ .../selftests/bpf/progs/test_sockmap_copy.c | 48 ++++++++++++ 2 files changed, 124 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/test_sockmap_copy.c diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c index 96e7b7f84c65..d30cabc00e9e 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c +++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c @@ -4,6 +4,7 @@ #include "test_progs.h" #include "test_skmsg_load_helpers.skel.h" +#include "test_sockmap_copy.skel.h" #define TCP_REPAIR 19 /* TCP sock is under repair right now */ @@ -101,6 +102,77 @@ static void test_skmsg_helpers(enum bpf_map_type map_type) test_skmsg_load_helpers__destroy(skel); } +static void test_sockmap_copy(enum bpf_map_type map_type) +{ + struct bpf_prog_test_run_attr attr; + struct test_sockmap_copy *skel; + __u64 src_cookie, dst_cookie; + int err, prog, s, src, dst; + const __u32 zero = 0; + char dummy[14] = {0}; + + s = connected_socket_v4(); + if (CHECK_FAIL(s == -1)) + return; + + skel = test_sockmap_copy__open_and_load(); + if (CHECK_FAIL(!skel)) { + close(s); + perror("test_sockmap_copy__open_and_load"); + return; + } + + prog = bpf_program__fd(skel->progs.copy_sock_map); + src = bpf_map__fd(skel->maps.src); + if (map_type == BPF_MAP_TYPE_SOCKMAP) + dst = bpf_map__fd(skel->maps.dst_sock_map); + else + dst = bpf_map__fd(skel->maps.dst_sock_hash); + + err = bpf_map_update_elem(src, &zero, &s, BPF_NOEXIST); + if (CHECK_FAIL(err)) { + perror("bpf_map_update"); + goto out; + } + + err = bpf_map_lookup_elem(src, &zero, &src_cookie); + if (CHECK_FAIL(err)) { + perror("bpf_map_lookup_elem(src)"); + goto out; + } + + attr = (struct bpf_prog_test_run_attr){ + .prog_fd = prog, + .repeat = 1, + .data_in = dummy, + .data_size_in = sizeof(dummy), + }; + + err = bpf_prog_test_run_xattr(&attr); + if (err) { + test__fail(); + perror("bpf_prog_test_run"); + goto out; + } else if (!attr.retval) { + PRINT_FAIL("bpf_prog_test_run: program returned %u\n", + attr.retval); + goto out; + } + + err = bpf_map_lookup_elem(dst, &zero, &dst_cookie); + if (CHECK_FAIL(err)) { + perror("bpf_map_lookup_elem(dst)"); + goto out; + } + + if (dst_cookie != src_cookie) + PRINT_FAIL("cookie %llu != %llu\n", dst_cookie, src_cookie); + +out: + close(s); + test_sockmap_copy__destroy(skel); +} + void test_sockmap_basic(void) { if (test__start_subtest("sockmap create_update_free")) @@ -111,4 +183,8 @@ void test_sockmap_basic(void) test_skmsg_helpers(BPF_MAP_TYPE_SOCKMAP); if (test__start_subtest("sockhash sk_msg load helpers")) test_skmsg_helpers(BPF_MAP_TYPE_SOCKHASH); + if (test__start_subtest("sockmap copy")) + test_sockmap_copy(BPF_MAP_TYPE_SOCKMAP); + if (test__start_subtest("sockhash copy")) + test_sockmap_copy(BPF_MAP_TYPE_SOCKHASH); } diff --git a/tools/testing/selftests/bpf/progs/test_sockmap_copy.c b/tools/testing/selftests/bpf/progs/test_sockmap_copy.c new file mode 100644 index 000000000000..9d0c9f28cab2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_sockmap_copy.c @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2020 Cloudflare +#include "vmlinux.h" +#include + +struct { + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} src SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} dst_sock_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKHASH); + __uint(max_entries, 1); + __type(key, __u32); + __type(value, __u64); +} dst_sock_hash SEC(".maps"); + +SEC("classifier/copy_sock_map") +int copy_sock_map(void *ctx) +{ + struct bpf_sock *sk; + bool failed = false; + __u32 key = 0; + + sk = bpf_map_lookup_elem(&src, &key); + if (!sk) + return SK_DROP; + + if (bpf_map_update_elem(&dst_sock_map, &key, sk, 0)) + failed = true; + + if (bpf_map_update_elem(&dst_sock_hash, &key, sk, 0)) + failed = true; + + bpf_sk_release(sk); + return failed ? SK_DROP : SK_PASS; +} + +char _license[] SEC("license") = "GPL";