From patchwork Fri Oct 6 09:42:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1844327 X-Patchwork-Delegate: pablo@netfilter.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ERc8V+W7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S23RL3NxGz1yq9 for ; Fri, 6 Oct 2023 20:44:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231446AbjJFJoJ (ORCPT ); Fri, 6 Oct 2023 05:44:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231193AbjJFJoI (ORCPT ); Fri, 6 Oct 2023 05:44:08 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A476D9F for ; Fri, 6 Oct 2023 02:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696585400; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=2KVyFvcShHYWZo53v4H0HBxXa/ymfkcRVqevp9GQKOE=; b=ERc8V+W7O4Lhn57R8NXVccYhjeJRpfAyJm3FwgUdErpKxpTsDJft4+qjbjvf1KX6bDT0XA EKHV7uiufzEn1d71nd/B7tX404ucYlPAnzr5SGt6zteNPQNGn9rbWosBOJv9USmUAnCXWU TdVC7zTpLv9DpiU/+7wYRbOsoJKZ494= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-61-3l-J3DX5PEefLMLXyRuPYw-1; Fri, 06 Oct 2023 05:43:18 -0400 X-MC-Unique: 3l-J3DX5PEefLMLXyRuPYw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8D8743C176ED for ; Fri, 6 Oct 2023 09:43:18 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.194.252]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 98A07215670B; Fri, 6 Oct 2023 09:43:17 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [nft PATCH 1/3] tests/shell: mount all of "/var/run" in "test-wrapper.sh" Date: Fri, 6 Oct 2023 11:42:18 +0200 Message-ID: <20231006094226.711628-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org After reboot, "/var/run/netns" does not exist before we run the first `ip netns add` command. Previously, "test-wrapper.sh" would mount a tmpfs on that directory, but that fails, if the directory doesn't exist. You will notice this, by deleting /var/run/netns (which only root can delete or create, and which is wiped on reboot). Instead, mount all of "/var/run". Then we can also create /var/run/netns directory. This means, any other content from /var/run is hidden too. That's probably desirable, because it means we don't depend on stuff that happens to be there. If we would require other content in /var/run, then the test runner needs to be aware of the requirement and ensure it's present. But best is just to not require anything. It's only iproute2 which insists on /var/run/netns. Signed-off-by: Thomas Haller --- tests/shell/helpers/test-wrapper.sh | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/tests/shell/helpers/test-wrapper.sh b/tests/shell/helpers/test-wrapper.sh index e10360c9b266..13b918f8b8e1 100755 --- a/tests/shell/helpers/test-wrapper.sh +++ b/tests/shell/helpers/test-wrapper.sh @@ -23,11 +23,11 @@ START_TIME="$(cut -d ' ' -f1 /proc/uptime)" export TMPDIR="$NFT_TEST_TESTTMPDIR" -CLEANUP_UMOUNT_RUN_NETNS=n +CLEANUP_UMOUNT_VAR_RUN=n cleanup() { - if [ "$CLEANUP_UMOUNT_RUN_NETNS" = y ] ; then - umount "/var/run/netns" || : + if [ "$CLEANUP_UMOUNT_VAR_RUN" = y ] ; then + umount "/var/run" &>/dev/null || : fi } @@ -38,16 +38,20 @@ printf '%s\n' "$TEST" > "$NFT_TEST_TESTTMPDIR/name" read tainted_before < /proc/sys/kernel/tainted if [ "$NFT_TEST_HAS_UNSHARED_MOUNT" = y ] ; then - # We have a private mount namespace. We will mount /run/netns as a tmpfs, - # this is useful because `ip netns add` wants to add files there. + # We have a private mount namespace. We will mount /var/run/ as a tmpfs. # - # When running as rootless, this is necessary to get such tests to - # pass. When running rootful, it's still useful to not touch the - # "real" /var/run/netns of the system. - mkdir -p /var/run/netns - if mount -t tmpfs --make-private "/var/run/netns" ; then - CLEANUP_UMOUNT_RUN_NETNS=y + # The main purpose is so that we can create /var/run/netns, which is + # required for `ip netns add` to work. When running as rootless, this + # is necessary to get such tests to pass. When running rootful, it's + # still useful to not touch the "real" /var/run/netns of the system. + # + # Note that this also hides everything that might reside in /var/run. + # That is desirable, as tests should not depend on content there (or if + # they do, we need to explicitly handle it as appropriate). + if mount -t tmpfs --make-private "/var/run" ; then + CLEANUP_UMOUNT_VAR_RUN=y fi + mkdir -p /var/run/netns fi TEST_TAGS_PARSED=0 From patchwork Fri Oct 6 09:42:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1844329 X-Patchwork-Delegate: pablo@netfilter.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BALumjq3; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S23RM2nxNz20Vp for ; Fri, 6 Oct 2023 20:44:15 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231451AbjJFJoM (ORCPT ); Fri, 6 Oct 2023 05:44:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231475AbjJFJoK (ORCPT ); Fri, 6 Oct 2023 05:44:10 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD326AD for ; Fri, 6 Oct 2023 02:43:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696585401; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V0t+ETjYOtGPta+fgEV7Z64IPA9YgMP2S1S9KfLNjOI=; b=BALumjq3T/WORs2674EZpNXsI1FEuizCnbzC4zqmqU/Zo0eRnlIL2KvjlcmuRO6Ku/EDaU X2+ESHJF3eNSO9o2Zgmyq2tihaK6XvJtXRO1X4xJaLnuskDIn0+73LgWfwEyyGj0Xo7tOd r/mgsl1VKDHWSic/DtD9BFYxtyXcPv8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-556-DLyEpoocMnimzQf1o2RC5w-1; Fri, 06 Oct 2023 05:43:20 -0400 X-MC-Unique: DLyEpoocMnimzQf1o2RC5w-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0523085A5A8 for ; Fri, 6 Oct 2023 09:43:20 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.194.252]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1E4FD2156711; Fri, 6 Oct 2023 09:43:18 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [nft PATCH 2/3] tests/shell: preserve result directory with NFT_TEST_FAIL_ON_SKIP Date: Fri, 6 Oct 2023 11:42:19 +0200 Message-ID: <20231006094226.711628-2-thaller@redhat.com> In-Reply-To: <20231006094226.711628-1-thaller@redhat.com> References: <20231006094226.711628-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org On a successful run, the result directory will be deleted (unless run with "-k|--keep-logs" option or NFT_TEST_KEEP_LOGS=y). With NFT_TEST_FAIL_ON_SKIP=y, when there are no failures but skipped tests, also preserve the result. Signed-off-by: Thomas Haller --- tests/shell/run-tests.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh index 4ff0b55ad7b1..7672b2fe5074 100755 --- a/tests/shell/run-tests.sh +++ b/tests/shell/run-tests.sh @@ -850,7 +850,12 @@ echo "" kmemleak_found=0 check_kmemleak_force -if [ "$failed" -gt 0 ] || [ "$NFT_TEST_FAIL_ON_SKIP" = y -a "$skipped" -gt 0 ] ; then +failed_total="$failed" +if [ "$NFT_TEST_FAIL_ON_SKIP" = y ] ; then + failed_total="$((failed_total + skipped))" +fi + +if [ "$failed_total" -gt 0 ] ; then RR="$RED" elif [ "$skipped" -gt 0 ] ; then RR="$YELLOW" @@ -875,7 +880,7 @@ END_TIME="$(cut -d ' ' -f1 /proc/uptime)" WALL_TIME="$(awk -v start="$START_TIME" -v end="$END_TIME" "BEGIN { print(end - start) }")" printf "%s\n" "$WALL_TIME" "$START_TIME" "$END_TIME" > "$NFT_TEST_TMPDIR/times" -if [ "$failed" -gt 0 -o "$NFT_TEST_KEEP_LOGS" = y ] ; then +if [ "$failed_total" -gt 0 -o "$NFT_TEST_KEEP_LOGS" = y ] ; then msg_info "check the temp directory \"$NFT_TEST_TMPDIR\" (\"$NFT_TEST_LATEST\")" msg_info " ls -lad \"$NFT_TEST_LATEST\"/*/*" msg_info " grep -R ^ \"$NFT_TEST_LATEST\"/" From patchwork Fri Oct 6 09:42:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1844328 X-Patchwork-Delegate: pablo@netfilter.org Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=f4oQrMpn; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S23RL6g5bz1yqH for ; Fri, 6 Oct 2023 20:44:14 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231193AbjJFJoL (ORCPT ); Fri, 6 Oct 2023 05:44:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35532 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231488AbjJFJoK (ORCPT ); Fri, 6 Oct 2023 05:44:10 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48FE5BE for ; Fri, 6 Oct 2023 02:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1696585403; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ppKJ8BojkAk7QeOEmq+k75zKV1DUB0JgxK9nS9unI50=; b=f4oQrMpnjbde4H3irTJK8bN0THLFliKewxdg0zW6aGuT187iCcCVrT1vGBzlLOdWTwLFAF RFZk+fHQiemW9w+zLzozswsS5LRhLKp/rnY60ytZf9LVaK/stsiBk51ByeH+nXEiSwFUQx LprOZll7nWHrG3/9Dtm1qY3Oyz1P75Y= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-562-4HooiBEFOumiW7bbGYr-5A-1; Fri, 06 Oct 2023 05:43:22 -0400 X-MC-Unique: 4HooiBEFOumiW7bbGYr-5A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DC0EB1DD35D3 for ; Fri, 6 Oct 2023 09:43:21 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.194.252]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 937B2215670B; Fri, 6 Oct 2023 09:43:20 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [nft PATCH 3/3] tests/shell: add "-S|--setup-host" option to set sysctl for rootless tests Date: Fri, 6 Oct 2023 11:42:20 +0200 Message-ID: <20231006094226.711628-3-thaller@redhat.com> In-Reply-To: <20231006094226.711628-1-thaller@redhat.com> References: <20231006094226.711628-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org Most tests can run just fine without root. A few of them will fail if /proc/sys/net/core/{wmem_max,rmem_max} is too small (as it is by default on the host). The easy workaround is to bump those limits once. This has to be repeated after each reboot. Doing that manually (every time) is cumbersome. Add a "--setup-host" option for that. Usage: $ sudo ./tests/shell/run-tests.sh -S Setting up host for running as rootless (requires root). echo 4096000 > /proc/sys/net/core/rmem_max (previous value 100000) echo 4096000 > /proc/sys/net/core/wmem_max (previous value 100000) Signed-off-by: Thomas Haller --- tests/shell/run-tests.sh | 46 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/tests/shell/run-tests.sh b/tests/shell/run-tests.sh index 7672b2fe5074..22105c2e90e2 100755 --- a/tests/shell/run-tests.sh +++ b/tests/shell/run-tests.sh @@ -166,6 +166,9 @@ usage() { echo " -s|--sequential : Sets NFT_TEST_JOBS=0, which also enables global cleanups." echo " Also sets NFT_TEST_SHUFFLE_TESTS=n if left unspecified." echo " -Q|--quick : Sets NFT_TEST_SKIP_slow=y." + echo " -S|--setup-host : Modify the host to run as rootless. Otherwise, some tests will be" + echo " skipped. Basically, this bumps /proc/sys/net/core/{wmem_max,rmem_max}." + echo " Must run as root and this option must be specified alone." echo " -- : Separate options from tests." echo " [TESTS...] : Other options are treated as test names," echo " that is, executables that are run by the runner." @@ -302,10 +305,25 @@ export NFT_TEST_RANDOM_SEED TESTS=() +SETUP_HOST= +SETUP_HOST_OTHER= + +ARGV_ORIG=( "$@" ) + while [ $# -gt 0 ] ; do A="$1" shift case "$A" in + -S|--setup-host) + ;; + *) + SETUP_HOST_OTHER=y + ;; + esac + case "$A" in + -S|--setup-host) + SETUP_HOST="$A" + ;; -v) VERBOSE=y ;; @@ -353,6 +371,34 @@ while [ $# -gt 0 ] ; do esac done +sysctl_bump() { + local sysctl="$1" + local val="$2" + local cur; + + cur="$(cat "$sysctl" 2>/dev/null)" || : + if [ -n "$cur" -a "$cur" -ge "$val" ] ; then + echo "# Skip: echo $val > $sysctl (current value $cur)" + return 0 + fi + echo " echo $val > $sysctl (previous value $cur)" + echo "$val" > "$sysctl" +} + +setup_host() { + echo "Setting up host for running as rootless (requires root)." + sysctl_bump /proc/sys/net/core/rmem_max $((4000*1024)) || return $? + sysctl_bump /proc/sys/net/core/wmem_max $((4000*1024)) || return $? +} + +if [ -n "$SETUP_HOST" ] ; then + if [ "$SETUP_HOST_OTHER" = y ] ; then + msg_error "The $SETUP_HOST option must be specified alone." + fi + setup_host + exit $? +fi + find_tests() { find "$1" -type f -executable | sort }