From patchwork Mon Sep 7 14:46:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359028 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=P6W93AJI; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlY4p2GJnz9sR4 for ; Tue, 8 Sep 2020 02:04:26 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730344AbgIGQEJ (ORCPT ); Mon, 7 Sep 2020 12:04:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729997AbgIGOsa (ORCPT ); Mon, 7 Sep 2020 10:48:30 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9B4BC061756 for ; Mon, 7 Sep 2020 07:48:29 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id a9so14468483wmm.2 for ; Mon, 07 Sep 2020 07:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lmVqWjcSMQaUhOrqAhhgjsf1XhmhvdxSOSZRzY4PrPY=; b=P6W93AJI2TuCIV+txKN1dFEZnN675MBF6B8HGrQ67CRDnKXcGg24L5XlpfUy3WOVVb 8Fw3irXJJdqGEhYppUujcCew1+WIK79AaOcUg67Yq1IJRIN5hhYQtoJhnkSU0Fz8Ttbe NBLh+HUGKH8OgIabYtk9RdrCSF0OdnedEw/EU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lmVqWjcSMQaUhOrqAhhgjsf1XhmhvdxSOSZRzY4PrPY=; b=cpcS/ekOp5QcJmtYs6+pwPDyFpTQO/GgFxBnSCOAHgAeupbvFYRnck7omuaGZ0aMau nhOeGbbDMq8Kew8DlJL39LYuj0TD6W9NTOSpsBSqJF9Yt2WMEQhxx7tJkD1wJ+OZS//W 6E0g0Gea8+Zq07xxwS5PTsKkiJ0oSU/fQerXjE1yVqI2/mD59DNkvgfKQBwr0fHxEBLc PVr4FwmLbApm0kiJchBiCfMwU9AIJpq6+OkbSJdTLmxjpFaArtj0ZlEPLIKO3pTTEJkl dEM+ZraEFbjYdPUN1R0yEfMEV35406BJBwkOEfO6pF1z5m39x1IWgj6OTtaL7SMVlLfy UBkA== X-Gm-Message-State: AOAM532kG4+juihIhXt0bVgh8BXIcdJtzoVO79RUZfMf8514UgHdTk4u IF4+JhoWSsxmJ9LhvH59DAcutA== X-Google-Smtp-Source: ABdhPJxKoIxaPLnvtZgRAKldcf+5DA+XYcxlgEZjcrxY4f5XFhOWlkutUeH3mS+IUTlmeMKyIkfRLA== X-Received: by 2002:a1c:1b93:: with SMTP id b141mr20423690wmb.166.1599490108258; Mon, 07 Sep 2020 07:48:28 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:27 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 1/7] bpf: Allow passing BTF pointers as PTR_TO_SOCK_COMMON Date: Mon, 7 Sep 2020 15:46:55 +0100 Message-Id: <20200907144701.44867-2-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Tracing programs can derive struct sock pointers from a variety of sources, e.g. a bpf_iter for sk_storage maps receives one as part of the context. It's desirable to be able to pass these to functions that expect PTR_TO_SOCK_COMMON. For example, it enables us to insert such a socket into a sockmap via map_elem_update. Note that we can't use struct sock* in cases where a function expects PTR_TO_SOCKET: not all struct sock* that a tracing program may derive are indeed for a full socket, code must check the socket state instead. Teach the verifier that a PTR_TO_BTF_ID for a struct sock is equivalent to PTR_TO_SOCK_COMMON. There is one hazard here: bpf_sk_release also takes a PTR_TO_SOCK_COMMON, but expects it to be refcounted. Since this isn't the case for pointers derived from BTF we must prevent them from being passed to the function. Luckily, we can simply check that the ref_obj_id is not zero in release_reference, and return an error otherwise. Signed-off-by: Lorenz Bauer --- kernel/bpf/verifier.c | 61 +++++++++++++++++++++++++------------------ 1 file changed, 36 insertions(+), 25 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index b4e9c56b8b32..f1f45ce42d60 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3908,6 +3908,9 @@ static int resolve_map_arg_type(struct bpf_verifier_env *env, return 0; } +BTF_ID_LIST(btf_sock_common_ids) +BTF_ID(struct, sock) + static int check_func_arg(struct bpf_verifier_env *env, u32 arg, struct bpf_call_arg_meta *meta, const struct bpf_func_proto *fn) @@ -3984,7 +3987,8 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, } else if (arg_type == ARG_PTR_TO_SOCK_COMMON) { expected_type = PTR_TO_SOCK_COMMON; /* Any sk pointer can be ARG_PTR_TO_SOCK_COMMON */ - if (!type_is_sk_pointer(type)) + if (!type_is_sk_pointer(type) && + type != PTR_TO_BTF_ID) goto err_type; if (reg->ref_obj_id) { if (meta->ref_obj_id) { @@ -3995,6 +3999,7 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, } meta->ref_obj_id = reg->ref_obj_id; } + meta->btf_id = btf_sock_common_ids[0]; } else if (arg_type == ARG_PTR_TO_SOCKET || arg_type == ARG_PTR_TO_SOCKET_OR_NULL) { expected_type = PTR_TO_SOCKET; @@ -4004,33 +4009,9 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, goto err_type; } } else if (arg_type == ARG_PTR_TO_BTF_ID) { - bool ids_match = false; - expected_type = PTR_TO_BTF_ID; if (type != expected_type) goto err_type; - if (!fn->check_btf_id) { - if (reg->btf_id != meta->btf_id) { - ids_match = btf_struct_ids_match(&env->log, reg->off, reg->btf_id, - meta->btf_id); - if (!ids_match) { - verbose(env, "Helper has type %s got %s in R%d\n", - kernel_type_name(meta->btf_id), - kernel_type_name(reg->btf_id), regno); - return -EACCES; - } - } - } else if (!fn->check_btf_id(reg->btf_id, arg)) { - verbose(env, "Helper does not support %s in R%d\n", - kernel_type_name(reg->btf_id), regno); - - return -EACCES; - } - if ((reg->off && !ids_match) || !tnum_is_const(reg->var_off) || reg->var_off.value) { - verbose(env, "R%d is a pointer to in-kernel struct with non-zero offset\n", - regno); - return -EACCES; - } } else if (arg_type == ARG_PTR_TO_SPIN_LOCK) { if (meta->func_id == BPF_FUNC_spin_lock) { if (process_spin_lock(env, regno, true)) @@ -4085,6 +4066,33 @@ static int check_func_arg(struct bpf_verifier_env *env, u32 arg, return -EFAULT; } + if (type == PTR_TO_BTF_ID) { + bool ids_match = false; + + if (!fn->check_btf_id) { + if (reg->btf_id != meta->btf_id) { + ids_match = btf_struct_ids_match(&env->log, reg->off, reg->btf_id, + meta->btf_id); + if (!ids_match) { + verbose(env, "Helper has type %s got %s in R%d\n", + kernel_type_name(meta->btf_id), + kernel_type_name(reg->btf_id), regno); + return -EACCES; + } + } + } else if (!fn->check_btf_id(reg->btf_id, arg)) { + verbose(env, "Helper does not support %s in R%d\n", + kernel_type_name(reg->btf_id), regno); + + return -EACCES; + } + if ((reg->off && !ids_match) || !tnum_is_const(reg->var_off) || reg->var_off.value) { + verbose(env, "R%d is a pointer to in-kernel struct with non-zero offset\n", + regno); + return -EACCES; + } + } + if (arg_type == ARG_CONST_MAP_PTR) { /* bpf_map_xxx(map_ptr) call: remember that map_ptr */ meta->map_ptr = reg->map_ptr; @@ -4561,6 +4569,9 @@ static int release_reference(struct bpf_verifier_env *env, int err; int i; + if (!ref_obj_id) + return -EINVAL; + err = release_reference_state(cur_func(env), ref_obj_id); if (err) return err; From patchwork Mon Sep 7 14:46:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359026 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=BHf/AKJz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlY4F1Ywvz9sRK for ; Tue, 8 Sep 2020 02:03:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730173AbgIGP76 (ORCPT ); Mon, 7 Sep 2020 11:59:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729998AbgIGOsb (ORCPT ); Mon, 7 Sep 2020 10:48:31 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD830C061757 for ; Mon, 7 Sep 2020 07:48:30 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id l9so14464775wme.3 for ; Mon, 07 Sep 2020 07:48:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Vfp8n6DVc0oODsUGu6ZVYaW1CH61INrkZIfFkv2gVnw=; b=BHf/AKJz1829Lg83WzSwfGXrEifbEg5SkqK29SEqyBsM5Srjhu2Qr2LlgkAE8LIR69 s2EJvfw4c9dytNVFjs3qGABTKo59V214wYl/v/+rzcb14fVirkF55wY2xTZZVXN3J1/0 ilJVzNQD3LTU+4SNdZLmM94zCKQr/RrJWh7uE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Vfp8n6DVc0oODsUGu6ZVYaW1CH61INrkZIfFkv2gVnw=; b=jHOMox0t+qK5UkJkrG0/H2Yk6QDFAKVZD/ZqoCFryEjQCzHjjEXyRhi6YbQ2tQ4CUP aWE+DD493G4BkS+AL7OCwq9XaUqYuUu/V4IaLFnK7aDiHOCzuDhclEw3CgzcJnPgGKsb Iw/Dbohx5CaOuQtwV2R70/dhOoRVHniHzXWmt4NFaX3KJCLhVotbzc2DejxkhdfLVhOM yQGkcvUHtqUUYq20LS7ebAQX8Tc/SbLXfF1LT/1zsgyCNP8EnwvurRL0LAptTb8x+vzc Li+eYo9F6tUhTXXJbgxD/1BparesoUhk+goDJO9NHWIfNlKUw97O3DCitRbxiMpdy97n zwKA== X-Gm-Message-State: AOAM530KXOPOG2Yj66nnPjODwwrVxBJ3orYEnf2e13Vo81xBbyz3gW98 AXCK25xWIF53yQleHQOKrNamQsjk1H+4SQ== X-Google-Smtp-Source: ABdhPJwre4LBfpd379RDlo7a3r1LMQ6ygrSI4EHa7LhG7VQXhF3YxWzMUWSUp7eSA1+71JwZprszuQ== X-Received: by 2002:a7b:c2aa:: with SMTP id c10mr20937949wmk.86.1599490109486; Mon, 07 Sep 2020 07:48:29 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:28 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 2/7] net: sockmap: Remove unnecessary sk_fullsock checks Date: Mon, 7 Sep 2020 15:46:56 +0100 Message-Id: <20200907144701.44867-3-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org The lookup paths for sockmap and sockhash currently include a check that returns NULL if the socket we just found is not a full socket. However, this check is not necessary. On insertion we ensure that we have a full socket (caveat around sock_ops), so request sockets are not a problem. Time-wait sockets are allocated separate from the original socket and then fed into the hashdance. They don't affect the sockets already stored in the sockmap. Suggested-by: Jakub Sitnicki Signed-off-by: Lorenz Bauer --- net/core/sock_map.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 078386d7d9a2..82494810d0ee 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -382,7 +382,7 @@ static void *sock_map_lookup(struct bpf_map *map, void *key) struct sock *sk; sk = __sock_map_lookup_elem(map, *(u32 *)key); - if (!sk || !sk_fullsock(sk)) + if (!sk) return NULL; if (sk_is_refcounted(sk) && !refcount_inc_not_zero(&sk->sk_refcnt)) return NULL; @@ -1110,7 +1110,7 @@ static void *sock_hash_lookup(struct bpf_map *map, void *key) struct sock *sk; sk = __sock_hash_lookup_elem(map, key); - if (!sk || !sk_fullsock(sk)) + if (!sk) return NULL; if (sk_is_refcounted(sk) && !refcount_inc_not_zero(&sk->sk_refcnt)) return NULL; From patchwork Mon Sep 7 14:46:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1358951 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=i90G2QHx; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlWQf4wjlz9sSn for ; Tue, 8 Sep 2020 00:49:46 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729991AbgIGOtg (ORCPT ); Mon, 7 Sep 2020 10:49:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730002AbgIGOtE (ORCPT ); Mon, 7 Sep 2020 10:49:04 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6617EC0613ED for ; Mon, 7 Sep 2020 07:48:32 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id e11so12476852wme.0 for ; Mon, 07 Sep 2020 07:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zZ9QbxDdNDSOlAy4MSZ177vTpuQ3OmxNZoFmXtn1kwg=; b=i90G2QHxORHJrgeppgvVTOZ5IhPXK5MQVFK79Vm8xHqvx92GA7lZDz+i9Yo0447ZTJ n3yoc/3XcOAeXHSuUol6LBazmosmS84JO0IoZZHb157i0zHkfWX52f0+sRVzrMvmXVw6 EpoExm4foGSUMsPR4nyF3fhJVhbV0KHbqlyOM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zZ9QbxDdNDSOlAy4MSZ177vTpuQ3OmxNZoFmXtn1kwg=; b=pNSpJ2BvU0gD+UXML0X32r/cI6uD7ND80lHbPPZt9xymBAGrF5TZ3RyCIDEDb3pzjJ Hv7A9wK0dZ+ZkcvkeXo3ae2wvsIgI4LtKiMl0Us8FshhBNXKHrF1x81i/CEWtDuIpI0h T9zutDeS2FOaPCvYmGto/lc6cD6m/pdxP2Nr+dQs3o72JIuKq9EHp6EcWdTSAaszEUl1 y1HJSFp6/WECLk9ac+ULx3yG4OIvIEfpIfWZqVeeSRPgNO0C6m6CjRQyIgu297LNeW8W kiqX0T+5536RH0iAsWT4GWRoQ3Ybsvt47u1PrAVIu+jXN6bXWsyieI5x8cVjB/5NhL7I gaZQ== X-Gm-Message-State: AOAM532N2WShh/SqH9LMPlq8DobnCj5JvCI8n1VYQ/l9CFvI4eZ0mEXr wo6Dy/AkqhKSAHJa5ouLqBNDBA== X-Google-Smtp-Source: ABdhPJyVGphwa9PlvkdD2BdlyP2qUNAhYKI8i3++XoYdetcOLlsAagLR9ReBMWk9xqj2pIiI5OhpaA== X-Received: by 2002:a1c:a5ca:: with SMTP id o193mr20949363wme.106.1599490110985; Mon, 07 Sep 2020 07:48:30 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:30 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 3/7] net: Allow iterating sockmap and sockhash Date: Mon, 7 Sep 2020 15:46:57 +0100 Message-Id: <20200907144701.44867-4-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Add bpf_iter support for sockmap / sockhash, based on the bpf_sk_storage and hashtable implementation. sockmap and sockhash share the same iteration context: a pointer to an arbitrary key and a pointer to a socket. Both pointers may be NULL, and so BPF has to perform a NULL check before accessing them. Technically it's not possible for sockhash iteration to yield a NULL socket, but we ignore this to be able to use a single iteration point. Iteration will visit all keys that remain unmodified during the lifetime of the iterator. It may or may not visit newly added ones. Switch from using rcu_dereference_raw to plain rcu_dereference, so we gain another guard rail if CONFIG_PROVE_RCU is enabled. Signed-off-by: Lorenz Bauer --- net/core/sock_map.c | 280 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 278 insertions(+), 2 deletions(-) diff --git a/net/core/sock_map.c b/net/core/sock_map.c index 82494810d0ee..e1f05e3fa1d0 100644 --- a/net/core/sock_map.c +++ b/net/core/sock_map.c @@ -2,6 +2,7 @@ /* Copyright (c) 2017 - 2018 Covalent IO, Inc. http://covalent.io */ #include +#include #include #include #include @@ -703,6 +704,109 @@ const struct bpf_func_proto bpf_msg_redirect_map_proto = { .arg4_type = ARG_ANYTHING, }; +struct sock_map_seq_info { + struct bpf_map *map; + struct sock *sk; + u32 index; +}; + +struct bpf_iter__sockmap { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct bpf_map *, map); + __bpf_md_ptr(void *, key); + __bpf_md_ptr(struct sock *, sk); +}; + +DEFINE_BPF_ITER_FUNC(sockmap, struct bpf_iter_meta *meta, + struct bpf_map *map, void *key, + struct sock *sk) + +static void *sock_map_seq_lookup_elem(struct sock_map_seq_info *info) +{ + if (unlikely(info->index >= info->map->max_entries)) + return NULL; + + info->sk = __sock_map_lookup_elem(info->map, info->index); + + /* can't return sk directly, since that might be NULL */ + return info; +} + +static void *sock_map_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct sock_map_seq_info *info = seq->private; + + if (*pos == 0) + ++*pos; + + /* pairs with sock_map_seq_stop */ + rcu_read_lock(); + return sock_map_seq_lookup_elem(info); +} + +static void *sock_map_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct sock_map_seq_info *info = seq->private; + + ++*pos; + ++info->index; + + return sock_map_seq_lookup_elem(info); +} + +static int sock_map_seq_show(struct seq_file *seq, void *v) +{ + struct sock_map_seq_info *info = seq->private; + struct bpf_iter__sockmap ctx = {}; + struct bpf_iter_meta meta; + struct bpf_prog *prog; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, !v); + if (!prog) + return 0; + + ctx.meta = &meta; + ctx.map = info->map; + if (v) { + ctx.key = &info->index; + ctx.sk = info->sk; + } + + return bpf_iter_run_prog(prog, &ctx); +} + +static void sock_map_seq_stop(struct seq_file *seq, void *v) +{ + if (!v) + (void)sock_map_seq_show(seq, NULL); + + /* pairs with sock_map_seq_start */ + rcu_read_unlock(); +} + +static const struct seq_operations sock_map_seq_ops = { + .start = sock_map_seq_start, + .next = sock_map_seq_next, + .stop = sock_map_seq_stop, + .show = sock_map_seq_show, +}; + +static int sock_map_init_seq_private(void *priv_data, + struct bpf_iter_aux_info *aux) +{ + struct sock_map_seq_info *info = priv_data; + + info->map = aux->map; + return 0; +} + +static const struct bpf_iter_seq_info sock_map_iter_seq_info = { + .seq_ops = &sock_map_seq_ops, + .init_seq_private = sock_map_init_seq_private, + .seq_priv_size = sizeof(struct sock_map_seq_info), +}; + static int sock_map_btf_id; const struct bpf_map_ops sock_map_ops = { .map_meta_equal = bpf_map_meta_equal, @@ -717,6 +821,7 @@ const struct bpf_map_ops sock_map_ops = { .map_check_btf = map_check_no_btf, .map_btf_name = "bpf_stab", .map_btf_id = &sock_map_btf_id, + .iter_seq_info = &sock_map_iter_seq_info, }; struct bpf_shtab_elem { @@ -953,7 +1058,7 @@ static int sock_hash_get_next_key(struct bpf_map *map, void *key, if (!elem) goto find_first_elem; - elem_next = hlist_entry_safe(rcu_dereference_raw(hlist_next_rcu(&elem->node)), + elem_next = hlist_entry_safe(rcu_dereference(hlist_next_rcu(&elem->node)), struct bpf_shtab_elem, node); if (elem_next) { memcpy(key_next, elem_next->key, key_size); @@ -965,7 +1070,7 @@ static int sock_hash_get_next_key(struct bpf_map *map, void *key, find_first_elem: for (; i < htab->buckets_num; i++) { head = &sock_hash_select_bucket(htab, i)->head; - elem_next = hlist_entry_safe(rcu_dereference_raw(hlist_first_rcu(head)), + elem_next = hlist_entry_safe(rcu_dereference(hlist_first_rcu(head)), struct bpf_shtab_elem, node); if (elem_next) { memcpy(key_next, elem_next->key, key_size); @@ -1199,6 +1304,117 @@ const struct bpf_func_proto bpf_msg_redirect_hash_proto = { .arg4_type = ARG_ANYTHING, }; +struct sock_hash_seq_info { + struct bpf_map *map; + struct bpf_shtab *htab; + u32 bucket_id; +}; + +static void *sock_hash_seq_find_next(struct sock_hash_seq_info *info, + struct bpf_shtab_elem *prev_elem) +{ + const struct bpf_shtab *htab = info->htab; + struct bpf_shtab_bucket *bucket; + struct bpf_shtab_elem *elem; + struct hlist_node *node; + + /* try to find next elem in the same bucket */ + if (prev_elem) { + node = rcu_dereference(hlist_next_rcu(&prev_elem->node)); + elem = hlist_entry_safe(node, struct bpf_shtab_elem, node); + if (elem) + return elem; + + /* no more elements, continue in the next bucket */ + info->bucket_id++; + } + + for (; info->bucket_id < htab->buckets_num; info->bucket_id++) { + bucket = &htab->buckets[info->bucket_id]; + node = rcu_dereference(hlist_first_rcu(&bucket->head)); + elem = hlist_entry_safe(node, struct bpf_shtab_elem, node); + if (elem) + return elem; + } + + return NULL; +} + +static void *sock_hash_seq_start(struct seq_file *seq, loff_t *pos) +{ + struct sock_hash_seq_info *info = seq->private; + + if (*pos == 0) + ++*pos; + + /* pairs with sock_hash_seq_stop */ + rcu_read_lock(); + return sock_hash_seq_find_next(info, NULL); +} + +static void *sock_hash_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct sock_hash_seq_info *info = seq->private; + + ++*pos; + return sock_hash_seq_find_next(info, v); +} + +static int sock_hash_seq_show(struct seq_file *seq, void *v) +{ + struct sock_hash_seq_info *info = seq->private; + struct bpf_iter__sockmap ctx = {}; + struct bpf_shtab_elem *elem = v; + struct bpf_iter_meta meta; + struct bpf_prog *prog; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, !elem); + if (!prog) + return 0; + + ctx.meta = &meta; + ctx.map = info->map; + if (elem) { + ctx.key = elem->key; + ctx.sk = elem->sk; + } + + return bpf_iter_run_prog(prog, &ctx); +} + +static void sock_hash_seq_stop(struct seq_file *seq, void *v) +{ + if (!v) + (void)sock_hash_seq_show(seq, NULL); + + /* pairs with sock_hash_seq_start */ + rcu_read_unlock(); +} + +static const struct seq_operations sock_hash_seq_ops = { + .start = sock_hash_seq_start, + .next = sock_hash_seq_next, + .stop = sock_hash_seq_stop, + .show = sock_hash_seq_show, +}; + +static int sock_hash_init_seq_private(void *priv_data, + struct bpf_iter_aux_info *aux) +{ + struct sock_hash_seq_info *info = priv_data; + + info->map = aux->map; + info->htab = container_of(aux->map, struct bpf_shtab, map); + return 0; +} + +static const struct bpf_iter_seq_info sock_hash_iter_seq_info = { + .seq_ops = &sock_hash_seq_ops, + .init_seq_private = sock_hash_init_seq_private, + .seq_priv_size = sizeof(struct sock_hash_seq_info), +}; + static int sock_hash_map_btf_id; const struct bpf_map_ops sock_hash_ops = { .map_meta_equal = bpf_map_meta_equal, @@ -1213,6 +1429,7 @@ const struct bpf_map_ops sock_hash_ops = { .map_check_btf = map_check_no_btf, .map_btf_name = "bpf_shtab", .map_btf_id = &sock_hash_map_btf_id, + .iter_seq_info = &sock_hash_iter_seq_info, }; static struct sk_psock_progs *sock_map_progs(struct bpf_map *map) @@ -1323,3 +1540,62 @@ void sock_map_close(struct sock *sk, long timeout) release_sock(sk); saved_close(sk, timeout); } + +static int sock_map_iter_attach_target(struct bpf_prog *prog, + union bpf_iter_link_info *linfo, + struct bpf_iter_aux_info *aux) +{ + struct bpf_map *map; + int err = -EINVAL; + + if (!linfo->map.map_fd) + return -EBADF; + + map = bpf_map_get_with_uref(linfo->map.map_fd); + if (IS_ERR(map)) + return PTR_ERR(map); + + if (map->map_type != BPF_MAP_TYPE_SOCKMAP && + map->map_type != BPF_MAP_TYPE_SOCKHASH) + goto put_map; + + if (prog->aux->max_rdonly_access > map->key_size) { + err = -EACCES; + goto put_map; + } + + aux->map = map; + return 0; + +put_map: + bpf_map_put_with_uref(map); + return err; +} + +static void sock_map_iter_detach_target(struct bpf_iter_aux_info *aux) +{ + bpf_map_put_with_uref(aux->map); +} + +static struct bpf_iter_reg sock_map_iter_reg = { + .target = "sockmap", + .attach_target = sock_map_iter_attach_target, + .detach_target = sock_map_iter_detach_target, + .show_fdinfo = bpf_iter_map_show_fdinfo, + .fill_link_info = bpf_iter_map_fill_link_info, + .ctx_arg_info_size = 2, + .ctx_arg_info = { + { offsetof(struct bpf_iter__sockmap, key), + PTR_TO_RDONLY_BUF_OR_NULL }, + { offsetof(struct bpf_iter__sockmap, sk), + PTR_TO_BTF_ID_OR_NULL }, + }, +}; + +static int __init bpf_sockmap_iter_init(void) +{ + sock_map_iter_reg.ctx_arg_info[1].btf_id = + btf_sock_ids[BTF_SOCK_TYPE_SOCK]; + return bpf_iter_reg_target(&sock_map_iter_reg); +} +late_initcall(bpf_sockmap_iter_init); From patchwork Mon Sep 7 14:46:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359019 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=n7eDaMD4; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlXzg4xbFz9sRK for ; Tue, 8 Sep 2020 01:59:59 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730004AbgIGP7t (ORCPT ); Mon, 7 Sep 2020 11:59:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730001AbgIGOtE (ORCPT ); Mon, 7 Sep 2020 10:49:04 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D2F13C061786 for ; Mon, 7 Sep 2020 07:48:33 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id c19so12471758wmd.1 for ; Mon, 07 Sep 2020 07:48:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=drF67q2mBxuUFYjet2Np2C1FhA/wmc+gwqkjVPZbrp8=; b=n7eDaMD4bQpckFEp/Dx9wUpU9C/Kvsfi0sLn+uNHd2uuSra/jsl5ivhFcv7IIZEAME OvyPWG30DvZHJS+nyPFMxsIeCU2nAlGJmQQuReWx1054qQug7ZGDe/7ETgX8E8zAEwFI 6QjL2ediZ0i4/NFCutpGoqtY0N6Na0+8sDcUU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=drF67q2mBxuUFYjet2Np2C1FhA/wmc+gwqkjVPZbrp8=; b=qHEslcl4q+8UbRG0vO2GBu25uXcvSepfp5MQcTiz4p4ZwwhFOebKA5HGsLarrmEIBT 1K0AnluoY6Q3rwVIsvs73KWvSfhhQmk8gW48InfbbgBkIkmknnaGQITXTLcXGNJ4wbEI hWpHIfMu6nyKuTaZwwrj9T7UR3AHZzNm0nLzGEq+/3ZFAF6Gpe03gWKbX0aYGPLImHZ/ woKQUyaZuwBqOQUZQ1GbeNOUk1xanBiCuPKXp8wPkYWKwmG6kN9Xy3XxLnrWPLHcme3C h7kfiCaEPL7hmKQ4UD6idQ93drIwBAN/O7f3M09SaQixoayix6p+st2MJqCYJQhhkCfX 3K7g== X-Gm-Message-State: AOAM531PSXdPYucUuLhyiB1Ca0Nalk9vOoDK7zpGSLA8+bkt8yhb2X6v Hws1w7msuuIRxr34Erl25sv72g== X-Google-Smtp-Source: ABdhPJxwaOkr2N7ZS9Zi2d9KUuSUFSPWMKMf/DnAfi0Qke+iqNaB//wvdCTMLiJiyzxRojFza/H/eA== X-Received: by 2002:a05:600c:2053:: with SMTP id p19mr20892423wmg.50.1599490112563; Mon, 07 Sep 2020 07:48:32 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:31 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 4/7] bpf: sockmap: accept sock_common pointer when updating Date: Mon, 7 Sep 2020 15:46:58 +0100 Message-Id: <20200907144701.44867-5-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org The sockmap update path already ensures that only full sockets are inserted (see sock_map_sk_state_allowed). Allow BPF to pass pointers to sock_common to map_update_elem(sockmap). This allows calling the helper with struct sock pointer derived from the sockmap iterator context. Signed-off-by: Lorenz Bauer --- kernel/bpf/verifier.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f1f45ce42d60..a4c398e05673 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3895,7 +3895,7 @@ static int resolve_map_arg_type(struct bpf_verifier_env *env, case BPF_MAP_TYPE_SOCKMAP: case BPF_MAP_TYPE_SOCKHASH: if (*arg_type == ARG_PTR_TO_MAP_VALUE) { - *arg_type = ARG_PTR_TO_SOCKET; + *arg_type = ARG_PTR_TO_SOCK_COMMON; } else { verbose(env, "invalid arg_type for sockmap/sockhash\n"); return -EINVAL; From patchwork Mon Sep 7 14:46:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359018 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=K7AOBaa7; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlXzf6WQbz9sR4 for ; Tue, 8 Sep 2020 01:59:58 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729820AbgIGP7Y (ORCPT ); Mon, 7 Sep 2020 11:59:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730004AbgIGOtE (ORCPT ); Mon, 7 Sep 2020 10:49:04 -0400 Received: from mail-wr1-x443.google.com (mail-wr1-x443.google.com [IPv6:2a00:1450:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C189C061795 for ; Mon, 7 Sep 2020 07:48:35 -0700 (PDT) Received: by mail-wr1-x443.google.com with SMTP id z1so16128438wrt.3 for ; Mon, 07 Sep 2020 07:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HvozQHes9NCAIlJe5t1kqMTRgWB1n5u3io5910Bjnj0=; b=K7AOBaa7VY6wI1kAgnnIzKu1DIQp6xhuGkyutmjvIJJLKmK/K/YuLF0I9Tq/jcRc7N ZhwL+lvG4LRBabGQEO+BSYybYHtRQXL2fy9mQAKNXuoA3PtPOTiEcDQTUteHU+yzojGU MqJOCEDf2GmC8M2Pk3lMSQdQmza5jLZnTIA5E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HvozQHes9NCAIlJe5t1kqMTRgWB1n5u3io5910Bjnj0=; b=tWtEEoyB84eC1rDttcpoKHlix5o668RVqPW9DHneuoNSqFBWIlu1Mge9sAhxf7uIE7 mowgszyfy5MCJBdDQK4bsFu+R5FEAevPtVl1582/6zkbPEwnO6APsUQWO6rwGBxfPksf Wf55dto2XiFuJGxBy2r0GNSEEepFhXYBlh/QdKAtDC61sbj8RGupGAyOXT18kY3ToqBJ ZQT1eyPxzRyF0kB6uzGU5cQ92XGB7MJc+HPd1mKX/fY4znZLsYkQL03somJ7Vx7vVzgo bvMqAgfSuQ4Et3KyP6mIPemN9yA2D58AEIYGa/VOGru0nbbFsw9K2+OLd5CK/fc0Lpl4 5mxA== X-Gm-Message-State: AOAM531q8Y6kqgdJ+/uZyIqIzZv+ugIB5uIzDZlLVxzlihMbN3K+UpBI 2aLUkMhqKLDodoh5lIv3PVmiZA== X-Google-Smtp-Source: ABdhPJyCjY14U1/W2QJqCOt4VjB1cImP2r2c6RLICktK3GL3/wpWtK8rhp0QH3A2nt230h+vIJqBZw== X-Received: by 2002:adf:dcc3:: with SMTP id x3mr21495196wrm.120.1599490113997; Mon, 07 Sep 2020 07:48:33 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:33 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 5/7] selftests: bpf: Ensure that BTF sockets cannot be released Date: Mon, 7 Sep 2020 15:46:59 +0100 Message-Id: <20200907144701.44867-6-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Being able to pass a BTF struct sock* to bpf_sk_release would screw up reference counting, and must therefore be prevented. Add a test which ensures that this property holds. Signed-off-by: Lorenz Bauer --- .../bpf/prog_tests/reference_tracking.c | 20 ++++++++++++++++++- .../bpf/progs/test_sk_ref_track_invalid.c | 20 +++++++++++++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_sk_ref_track_invalid.c diff --git a/tools/testing/selftests/bpf/prog_tests/reference_tracking.c b/tools/testing/selftests/bpf/prog_tests/reference_tracking.c index ac1ee10cffd8..3f19c8a16bb4 100644 --- a/tools/testing/selftests/bpf/prog_tests/reference_tracking.c +++ b/tools/testing/selftests/bpf/prog_tests/reference_tracking.c @@ -1,7 +1,8 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include "test_sk_ref_track_invalid.skel.h" -void test_reference_tracking(void) +static void test_sk_lookup(void) { const char *file = "test_sk_lookup_kern.o"; const char *obj_name = "ref_track"; @@ -50,3 +51,20 @@ void test_reference_tracking(void) cleanup: bpf_object__close(obj); } + +static void test_sk_release_invalid(void) +{ + struct test_sk_ref_track_invalid *skel; + int duration = 0; + + skel = test_sk_ref_track_invalid__open_and_load(); + if (CHECK(skel, "open_and_load", "verifier accepted sk_release of BTF struct sock*\n")) + test_sk_ref_track_invalid__destroy(skel); +} + +void test_reference_tracking(void) +{ + test_sk_lookup(); + if (test__start_subtest("invalid sk_release")) + test_sk_release_invalid(); +} diff --git a/tools/testing/selftests/bpf/progs/test_sk_ref_track_invalid.c b/tools/testing/selftests/bpf/progs/test_sk_ref_track_invalid.c new file mode 100644 index 000000000000..9017d92a807b --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_sk_ref_track_invalid.c @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2020 Cloudflare + +#include "bpf_iter.h" +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +SEC("iter/bpf_sk_storage_map") +int dump_bpf_sk_storage_map(struct bpf_iter__bpf_sk_storage_map *ctx) +{ + struct sock *sk = ctx->sk; + + if (sk) + bpf_sk_release((struct bpf_sock *)sk); + + return 0; +} From patchwork Mon Sep 7 14:47:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359016 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=ZafEmUnF; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlXyq1v6Pz9sSn for ; Tue, 8 Sep 2020 01:59:15 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730287AbgIGP7K (ORCPT ); Mon, 7 Sep 2020 11:59:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730006AbgIGOtE (ORCPT ); Mon, 7 Sep 2020 10:49:04 -0400 Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5A0FC061796 for ; Mon, 7 Sep 2020 07:48:36 -0700 (PDT) Received: by mail-wr1-x442.google.com with SMTP id g4so16097939wrs.5 for ; Mon, 07 Sep 2020 07:48:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ycv2LQ1rQbHAkz0iq7QhpQk+4LCrq+rjRQEPfWaxh9M=; b=ZafEmUnFRg4uyNOqzQLXfM069d6h4Xm6A7CJw40OZxGaKgkJSunroD7xrXj5inD6ps LNEhw/jGD9uyoVcb7iE5F0ZT8pUXi5U74wmgAFEMiTz5O1LZxmaAjZRDY4Nv+SrNKOF2 IQILXaoOuEg5pO5dQI87twTaLNgEN1EHB92w4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ycv2LQ1rQbHAkz0iq7QhpQk+4LCrq+rjRQEPfWaxh9M=; b=Dq6CCP0YNFDk4gSiY6Yf3h1lIZ5HtJ7KW9TNwknUJIN7ba23ZmvkOP57anOa49k722 OYExitzXsdRjBFPZN8Dx4Anb36afrWxmZTb3y0M4uoOm57S5bl2A7y9phhSFSCuYMt2m LaTxaZM0Oqj+TN57NbDk/6HLtWI6/lnMvOMZb5IX29ycPDLVs1NmfPXHEmeQR9bvKEZD OKL3xG7vw0aP3JF9LmzykL+HeLBptPNcBK/wexpRU09GV2K/tIksEDYehCmTW042dIoE B8NCL4OBropCYJPxrkPjGNu5SXNC3Qz70Nkbl3mDOo3dnLp0aB5ORzFUkjFZu8rER5X8 42Pg== X-Gm-Message-State: AOAM530uGndZ0e3GAWRO4x+2LcpKugIj6b7k815m5MQcMW+aah2TtpwH 6o2vElbdJWV9pGZkmz2hwQ1npw== X-Google-Smtp-Source: ABdhPJz7brgUlOjNxN4cWCGqg1+jTxo8V0+1m1qzCX1Kg+tazAvndOAOcBDEv/REQGoYphafJ7FKfA== X-Received: by 2002:adf:e58b:: with SMTP id l11mr23454916wrm.210.1599490115380; Mon, 07 Sep 2020 07:48:35 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:34 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 6/7] selftests: bpf: Add helper to compare socket cookies Date: Mon, 7 Sep 2020 15:47:00 +0100 Message-Id: <20200907144701.44867-7-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org We compare socket cookies to ensure that insertion into a sockmap worked. Pull this out into a helper function for use in other tests. Signed-off-by: Lorenz Bauer --- .../selftests/bpf/prog_tests/sockmap_basic.c | 50 +++++++++++++------ 1 file changed, 36 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c index 0b79d78b98db..0fe2a737fc8e 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c +++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c @@ -47,6 +47,37 @@ static int connected_socket_v4(void) return -1; } +static void compare_cookies(struct bpf_map *src, struct bpf_map *dst) +{ + __u32 i, max_entries = bpf_map__max_entries(src); + int err, duration, src_fd, dst_fd; + + src_fd = bpf_map__fd(src); + dst_fd = bpf_map__fd(dst); + + for (i = 0; i < max_entries; i++) { + __u64 src_cookie, dst_cookie; + + err = bpf_map_lookup_elem(src_fd, &i, &src_cookie); + if (err && errno == ENOENT) { + err = bpf_map_lookup_elem(dst_fd, &i, &dst_cookie); + CHECK(!err, "map_lookup_elem(dst)", "element %u not deleted\n", i); + CHECK(err && errno != ENOENT, "map_lookup_elem(dst)", "%s\n", + strerror(errno)); + continue; + } + if (CHECK(err, "lookup_elem(src)", "%s\n", strerror(errno))) + continue; + + err = bpf_map_lookup_elem(dst_fd, &i, &dst_cookie); + if (CHECK(err, "lookup_elem(dst)", "%s\n", strerror(errno))) + continue; + + CHECK(dst_cookie != src_cookie, "cookie mismatch", + "%llu != %llu (pos %u)\n", dst_cookie, src_cookie, i); + } +} + /* Create a map, populate it with one socket, and free the map. */ static void test_sockmap_create_update_free(enum bpf_map_type map_type) { @@ -106,9 +137,9 @@ static void test_skmsg_helpers(enum bpf_map_type map_type) static void test_sockmap_update(enum bpf_map_type map_type) { struct bpf_prog_test_run_attr tattr; - int err, prog, src, dst, duration = 0; + int err, prog, src, duration = 0; struct test_sockmap_update *skel; - __u64 src_cookie, dst_cookie; + struct bpf_map *dst_map; const __u32 zero = 0; char dummy[14] = {0}; __s64 sk; @@ -124,18 +155,14 @@ static void test_sockmap_update(enum bpf_map_type map_type) prog = bpf_program__fd(skel->progs.copy_sock_map); src = bpf_map__fd(skel->maps.src); if (map_type == BPF_MAP_TYPE_SOCKMAP) - dst = bpf_map__fd(skel->maps.dst_sock_map); + dst_map = skel->maps.dst_sock_map; else - dst = bpf_map__fd(skel->maps.dst_sock_hash); + dst_map = skel->maps.dst_sock_hash; err = bpf_map_update_elem(src, &zero, &sk, BPF_NOEXIST); if (CHECK(err, "update_elem(src)", "errno=%u\n", errno)) goto out; - err = bpf_map_lookup_elem(src, &zero, &src_cookie); - if (CHECK(err, "lookup_elem(src, cookie)", "errno=%u\n", errno)) - goto out; - tattr = (struct bpf_prog_test_run_attr){ .prog_fd = prog, .repeat = 1, @@ -148,12 +175,7 @@ static void test_sockmap_update(enum bpf_map_type map_type) "errno=%u retval=%u\n", errno, tattr.retval)) goto out; - err = bpf_map_lookup_elem(dst, &zero, &dst_cookie); - if (CHECK(err, "lookup_elem(dst, cookie)", "errno=%u\n", errno)) - goto out; - - CHECK(dst_cookie != src_cookie, "cookie mismatch", "%llu != %llu\n", - dst_cookie, src_cookie); + compare_cookies(skel->maps.src, dst_map); out: test_sockmap_update__destroy(skel); From patchwork Mon Sep 7 14:47:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenz Bauer X-Patchwork-Id: 1359017 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=cloudflare.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=cloudflare.com header.i=@cloudflare.com header.a=rsa-sha256 header.s=google header.b=PvmMw1oN; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BlXyt4Q6Sz9sSn for ; Tue, 8 Sep 2020 01:59:18 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730006AbgIGP7M (ORCPT ); Mon, 7 Sep 2020 11:59:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730005AbgIGOtE (ORCPT ); Mon, 7 Sep 2020 10:49:04 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 27DB5C061797 for ; Mon, 7 Sep 2020 07:48:38 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id z9so14475111wmk.1 for ; Mon, 07 Sep 2020 07:48:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CG6pfa8grmhbhWsnM+Yrdzr0cEQrHTB9JZN/vMnBbo8=; b=PvmMw1oNa+Rcn1KwdNg+gTs+dVGEzQ6Auz202r+OnwdXH11O8R6dl/0+mKrCtu0O2J PP8pJjUbk7u2F8p9tIsvvvY+W7rOzBU9XIUiGJm5kxWpJGYrK+wPixVMD7HA28pQ5pWj OaWx7su0XoUjUoqTIApF8ZfAo682hBUtLd4+0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CG6pfa8grmhbhWsnM+Yrdzr0cEQrHTB9JZN/vMnBbo8=; b=dgLEvejAQaGoje+10gqUNdJDcx4tWaCZ4ffUymeXixqHSuIc6NKzxj375LQ6hNQAfn rR06JkseM8T3c7gOlh4kiEGksIVD9cg12l9BPJq46eMYS3QXpaBJCLAIr8ObtMncs8Xi O7O2csTTZ+5H5Ty7S9UUS4L/5aOdjEvsSKIJk9aRnajB390KmscLVcg4YnWAb0H85R2r TOfSihj6cSmN01dCbDGAGvlA0W8rbDzfI/OPu9JQpsXohxQ8ZxiPP6bFyFAtvNCeS7jV OGiegjR4fpXV7DQO9u8NPQhFBUSehlvLEHSq2hPZAzfvOeyqDfcSM6o9dOrWyhz27Tm3 yzSA== X-Gm-Message-State: AOAM53209qgqvWRDOK722MhEfZOchsQmIPSUFmf3rWKbOHXf0OxaFSnm jZMnKltyWWUkvp6WI5uWC+YPDg== X-Google-Smtp-Source: ABdhPJz/xsWSmRnD42infgl/U5iXNuFuQGbUi/KUIQmJLcgEuUgsY6v6LjsMaNGl9gycY9Tmey1iMQ== X-Received: by 2002:a7b:c768:: with SMTP id x8mr22635774wmk.189.1599490116808; Mon, 07 Sep 2020 07:48:36 -0700 (PDT) Received: from antares.lan (2.e.3.8.e.0.6.b.6.2.5.e.8.e.4.b.f.f.6.2.a.5.a.7.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:7a5a:26ff:b4e8:e526:b60e:83e2]) by smtp.gmail.com with ESMTPSA id 59sm8816834wro.82.2020.09.07.07.48.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Sep 2020 07:48:36 -0700 (PDT) From: Lorenz Bauer To: ast@kernel.org, yhs@fb.com, daniel@iogearbox.net, jakub@cloudflare.com, john.fastabend@gmail.com, kafai@fb.com Cc: bpf@vger.kernel.org, kernel-team@cloudflare.com, Lorenz Bauer Subject: [PATCH bpf-next v4 7/7] selftests: bpf: Test copying a sockmap via bpf_iter Date: Mon, 7 Sep 2020 15:47:01 +0100 Message-Id: <20200907144701.44867-8-lmb@cloudflare.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200907144701.44867-1-lmb@cloudflare.com> References: <20200907144701.44867-1-lmb@cloudflare.com> MIME-Version: 1.0 Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Add a test that exercises a basic sockmap / sockhash copy using bpf_iter. Signed-off-by: Lorenz Bauer --- .../selftests/bpf/prog_tests/sockmap_basic.c | 88 +++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 9 ++ .../selftests/bpf/progs/bpf_iter_sockmap.c | 57 ++++++++++++ .../selftests/bpf/progs/bpf_iter_sockmap.h | 3 + 4 files changed, 157 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_sockmap.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_sockmap.h diff --git a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c index 0fe2a737fc8e..088903bdf10c 100644 --- a/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c +++ b/tools/testing/selftests/bpf/prog_tests/sockmap_basic.c @@ -6,6 +6,9 @@ #include "test_skmsg_load_helpers.skel.h" #include "test_sockmap_update.skel.h" #include "test_sockmap_invalid_update.skel.h" +#include "bpf_iter_sockmap.skel.h" + +#include "progs/bpf_iter_sockmap.h" #define TCP_REPAIR 19 /* TCP sock is under repair right now */ @@ -193,6 +196,87 @@ static void test_sockmap_invalid_update(void) test_sockmap_invalid_update__destroy(skel); } +static void test_sockmap_copy(enum bpf_map_type map_type) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + int err, len, src_fd, iter_fd, duration; + union bpf_iter_link_info linfo = {0}; + __s64 sock_fd[SOCKMAP_MAX_ENTRIES]; + __u32 i, num_sockets, max_elems; + struct bpf_iter_sockmap *skel; + struct bpf_map *src, *dst; + struct bpf_link *link; + char buf[64]; + + skel = bpf_iter_sockmap__open_and_load(); + if (CHECK(!skel, "bpf_iter_sockmap__open_and_load", "skeleton open_and_load failed\n")) + return; + + for (i = 0; i < ARRAY_SIZE(sock_fd); i++) + sock_fd[i] = -1; + + /* Make sure we have at least one "empty" entry to test iteration of + * an empty slot in an array. + */ + num_sockets = ARRAY_SIZE(sock_fd) - 1; + + if (map_type == BPF_MAP_TYPE_SOCKMAP) { + src = skel->maps.sockmap; + max_elems = bpf_map__max_entries(src); + } else { + src = skel->maps.sockhash; + max_elems = num_sockets; + } + + dst = skel->maps.dst; + src_fd = bpf_map__fd(src); + + for (i = 0; i < num_sockets; i++) { + sock_fd[i] = connected_socket_v4(); + if (CHECK(sock_fd[i] == -1, "connected_socket_v4", "cannot connect\n")) + goto out; + + err = bpf_map_update_elem(src_fd, &i, &sock_fd[i], BPF_NOEXIST); + if (CHECK(err, "map_update", "failed: %s\n", strerror(errno))) + goto out; + } + + linfo.map.map_fd = src_fd; + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + link = bpf_program__attach_iter(skel->progs.copy_sockmap, &opts); + if (CHECK(IS_ERR(link), "attach_iter", "attach_iter failed\n")) + goto out; + + iter_fd = bpf_iter_create(bpf_link__fd(link)); + if (CHECK(iter_fd < 0, "create_iter", "create_iter failed\n")) + goto free_link; + + /* do some tests */ + while ((len = read(iter_fd, buf, sizeof(buf))) > 0) + ; + if (CHECK(len < 0, "read", "failed: %s\n", strerror(errno))) + goto close_iter; + + /* test results */ + if (CHECK(skel->bss->elems != max_elems, "elems", "got %u expected %u\n", + skel->bss->elems, max_elems)) + goto close_iter; + + compare_cookies(src, dst); + +close_iter: + close(iter_fd); +free_link: + bpf_link__destroy(link); +out: + for (i = 0; i < num_sockets; i++) { + if (sock_fd[i] >= 0) + close(sock_fd[i]); + } + bpf_iter_sockmap__destroy(skel); +} + void test_sockmap_basic(void) { if (test__start_subtest("sockmap create_update_free")) @@ -209,4 +293,8 @@ void test_sockmap_basic(void) test_sockmap_update(BPF_MAP_TYPE_SOCKHASH); if (test__start_subtest("sockmap update in unsafe context")) test_sockmap_invalid_update(); + if (test__start_subtest("sockmap copy")) + test_sockmap_copy(BPF_MAP_TYPE_SOCKMAP); + if (test__start_subtest("sockhash copy")) + test_sockmap_copy(BPF_MAP_TYPE_SOCKHASH); } diff --git a/tools/testing/selftests/bpf/progs/bpf_iter.h b/tools/testing/selftests/bpf/progs/bpf_iter.h index c196280df90d..df682af75510 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter.h +++ b/tools/testing/selftests/bpf/progs/bpf_iter.h @@ -13,6 +13,7 @@ #define udp6_sock udp6_sock___not_used #define bpf_iter__bpf_map_elem bpf_iter__bpf_map_elem___not_used #define bpf_iter__bpf_sk_storage_map bpf_iter__bpf_sk_storage_map___not_used +#define bpf_iter__sockmap bpf_iter__sockmap___not_used #include "vmlinux.h" #undef bpf_iter_meta #undef bpf_iter__bpf_map @@ -26,6 +27,7 @@ #undef udp6_sock #undef bpf_iter__bpf_map_elem #undef bpf_iter__bpf_sk_storage_map +#undef bpf_iter__sockmap struct bpf_iter_meta { struct seq_file *seq; @@ -96,3 +98,10 @@ struct bpf_iter__bpf_sk_storage_map { struct sock *sk; void *value; }; + +struct bpf_iter__sockmap { + struct bpf_iter_meta *meta; + struct bpf_map *map; + void *key; + struct sock *sk; +}; diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.c b/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.c new file mode 100644 index 000000000000..d8d107c80b7b --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.c @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2020 Cloudflare */ +#include "bpf_iter.h" +#include "bpf_tracing_net.h" +#include "bpf_iter_sockmap.h" +#include +#include +#include + +char _license[] SEC("license") = "GPL"; + +struct { + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __uint(max_entries, SOCKMAP_MAX_ENTRIES); + __type(key, __u32); + __type(value, __u64); +} sockmap SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKHASH); + __uint(max_entries, SOCKMAP_MAX_ENTRIES); + __type(key, __u32); + __type(value, __u64); +} sockhash SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_SOCKHASH); + __uint(max_entries, SOCKMAP_MAX_ENTRIES); + __type(key, __u32); + __type(value, __u64); +} dst SEC(".maps"); + +__u32 elems = 0; + +SEC("iter/sockmap") +int copy_sockmap(struct bpf_iter__sockmap *ctx) +{ + struct sock *sk = ctx->sk; + __u32 tmp, *key = ctx->key; + int ret; + + if (!key) + return 0; + + elems++; + + /* We need a temporary buffer on the stack, since the verifier doesn't + * let us use the pointer from the context as an argument to the helper. + */ + tmp = *key; + + if (sk) + return bpf_map_update_elem(&dst, &tmp, sk, 0) != 0; + + ret = bpf_map_delete_elem(&dst, &tmp); + return ret && ret != -ENOENT; +} diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.h b/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.h new file mode 100644 index 000000000000..f98ad727ac06 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/bpf_iter_sockmap.h @@ -0,0 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define SOCKMAP_MAX_ENTRIES (64)