From patchwork Tue Oct 17 08:49:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1849898 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TaC96wpp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S8nmq1918z20cb for ; Tue, 17 Oct 2023 19:52:43 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234693AbjJQIwj (ORCPT ); Tue, 17 Oct 2023 04:52:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234700AbjJQIwh (ORCPT ); Tue, 17 Oct 2023 04:52:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60A0512F for ; Tue, 17 Oct 2023 01:51:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697532706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MWAhkw0rJ86pIH8jacqS5LVB537+a/TJxkJm3B5Qy2s=; b=TaC96wpp8hP/faCpfGdyf6hsOHaxAvEee2mnvG+OIjyeNLwXnoxbrVz36bXVrvKDNo+FMq Wf+8KMBVBCD3t4UWBuWpY+m16c1j5YTn2vQpOKBnLc6uXo1Ejmd1EVtFAoCHNPSHkXq1eR B7NsNTJ0+Qoxi+/rN8jV1VJezQXMIpo= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-548-t8T7S_I8PUCsVzpBDqJU7A-1; Tue, 17 Oct 2023 04:51:45 -0400 X-MC-Unique: t8T7S_I8PUCsVzpBDqJU7A-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 05D2C1C0783A for ; Tue, 17 Oct 2023 08:51:45 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.93]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 5C290492BEF; Tue, 17 Oct 2023 08:51:44 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [PATCH nft v2 1/3] tests/shell: add "tests/shell/helpers/eval-exit-code" Date: Tue, 17 Oct 2023 10:49:06 +0200 Message-ID: <20231017085133.1203402-2-thaller@redhat.com> In-Reply-To: <20231017085133.1203402-1-thaller@redhat.com> References: <20231017085133.1203402-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org This script is used to compare the running kernel version (`ulimit -r`) against a list of versions, to find out whether a test should be skipped or not. If kernel has a bug, a test might not pass. In that case, the test should exit with error code 77 as we also want to pass the test suite on distro kernels. However, if we know that the bug was fixed in a certain kernel version, then we the failure to be fatal and noticeable. Example usage: if ! _check_some_condition ; then echo "The condition to check for kernel bug https://git.kernel.org/XYZ failed" "$NFT_TEST_BASEDIR/helpers/eval-exit-code" kernel 6.5.6 6.6 exit $? fi Note that "eval-exit-code" will always exit with a non-zero exit code. It will also print a message about the comparison, which ends up in the test output. Signed-off-by: Thomas Haller --- tests/shell/helpers/eval-exit-code | 116 +++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100755 tests/shell/helpers/eval-exit-code diff --git a/tests/shell/helpers/eval-exit-code b/tests/shell/helpers/eval-exit-code new file mode 100755 index 000000000000..d007c0fc0fd6 --- /dev/null +++ b/tests/shell/helpers/eval-exit-code @@ -0,0 +1,116 @@ +#!/bin/bash -e + +die() { + printf "%s\n" "$*" + exit 2 +} + +usage() { + echo "usage: $0 [MODE] [ARGS...]" + echo "" + echo "Evaluates whether to exit a test with skip or failure reason." + echo "" + echo "The use case is for example a kernel bug, which prevents a test from passing. We" + echo "know that certain newer kernel versions have the fix, and we require the test to" + echo "pass there. When an assertion fails, The script will determine that the test should" + echo "have passed and exit with code \"1\". When running against an older kernel, the failure" + echo "is expected and the script exits with \"77\" to indicate a skip." + echo "" + echo "The script always either exits with:" + echo " 1: the check determined that we have a hard failure (a message is printed)" + echo " 77: the check determined that we skip (a message is printed)" + echo " 2: an error in the script happend (invalid arguments?)" + echo "" + echo "MODE can be one of:" + echo " \"kernel\": compares \`ulimit -r\` against the arguments. The arguments are" + echo " kernel versions for which we expect to support the feature and when called" + echo " on such a kernel, the script returns \"1\" to indicate a hard failure. Against" + echo " older kernels, \"77\" is returned. Multiple kernel version can be provided for" + echo " example \`$0 kernel 6.5.6 6.6\`." + echo "" + echo "USAGE:" + echo " if ! _check_some_condition ; then" + echo " echo \"The condition to check for kernel bug https://git.kernel.org/XYZ failed\"" + echo " \"\$NFT_TEST_BASEDIR/helpers/eval-exit-code\" kernel 6.5.6 6.6" + echo " exit \$?" + echo " fi" + exit 2 +} + +[ "$#" -eq 0 ] && usage + +_kernel_check_skip() { + local kversion="$1" + local compare="$2" + + if [ "$kversion" = "$compare" ] ; then + return 1 + fi + if [[ "$kversion" == "$compare"[.-]* ]] ; then + return 1 + fi + + local a1="$(printf '%s\n' 0 "$kversion" "$compare" 100000)" + local a2="$(printf '%s' "$a1" | sort -V)" + + if [ "$a1" != "$a2" ] ; then + return 1 + fi + return 0 +} + +_PRINT_ARG="???" + +eval_kernel() { + local compare + local kversion + + if [ -n "$_EVAL_EXIT_CODE_UNAME" ] ; then + # Only for testing. + kversion="$_EVAL_EXIT_CODE_UNAME" + else + kversion="$(uname -r)" || die "uname -r failed" + fi + + [ $# -ge 1 ] || die "Operation kernel expects one or more kernel versions to be compared with >= against $kversion" + + _PRINT_ARG="$kversion" + + local all_skip=1 + for compare; do + _kernel_check_skip "$kversion" "$compare" || all_skip=0 + done + if [ "$all_skip" -eq 1 ] ; then + return 77 + fi + return 1 +} + +run_op() { + local mode="$1" + shift + + rc=2 + "eval_$mode" "$@" || rc=$? + if [ "$rc" -eq 77 ] ; then + echo "Checking \"$mode\" ($@) against $_PRINT_ARG indicates to skip the test" + else + echo "Checking \"$mode\" ($@) against $_PRINT_ARG indicates a hard failure" + fi + exit "$rc" +} + +mode="$1" +shift + +case "$mode" in + kernel) + run_op "$mode" "$@" + ;; + -h|--help) + usage + ;; + *) + die "Invalid mode \"$mode\". Try $0 --help" + ;; +esac From patchwork Tue Oct 17 08:49:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1849899 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=duRDXRMF; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S8nmq4QNvz20Pd for ; Tue, 17 Oct 2023 19:52:43 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234620AbjJQIwk (ORCPT ); Tue, 17 Oct 2023 04:52:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234752AbjJQIwi (ORCPT ); Tue, 17 Oct 2023 04:52:38 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8AC4F5 for ; Tue, 17 Oct 2023 01:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697532707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VShFsPTZvZdCOAHWPPleUMqHQz29TIaDQQHW+ItmcqQ=; b=duRDXRMFHkAmCVBh7/Pqv2YjLRGieFZPmAWf13vlBtxyJP8WqrPcSPCQJM6WlnemkIT3ag FcIhP2rXcdaZpu+Xx8Rp0sKcF2XGlbYL06/dtsMN022mpdts+YMeYT7pjCa5ymIGuXUOq0 vONZgFnSZMQiPj4lLikfIN6txqpX/fk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-657-2xAjM3D1OE6sfoJOnL0XwQ-1; Tue, 17 Oct 2023 04:51:46 -0400 X-MC-Unique: 2xAjM3D1OE6sfoJOnL0XwQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C227D81D9E2 for ; Tue, 17 Oct 2023 08:51:45 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.93]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 41097492BF0; Tue, 17 Oct 2023 08:51:45 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [PATCH nft v2 2/3] tests/shell: skip "table_onoff" test on older kernels Date: Tue, 17 Oct 2023 10:49:07 +0200 Message-ID: <20231017085133.1203402-3-thaller@redhat.com> In-Reply-To: <20231017085133.1203402-1-thaller@redhat.com> References: <20231017085133.1203402-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org The "table_onoff" test can only pass with certain (recent) kernels. Conditionally exit with status 77, if "eval-exit-code" determines that we don't have a suitable kernel version. In this case, we can find the fixes in: v6.6 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c9bd26513b3a11b3adb3c2ed8a31a01a87173ff1 v6.5.6 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5e5754e9e77ce400d70ff3c30fea466c8dfe9a9f v6.1.56 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c4b0facd5c20ceae3d07018a3417f06302fa9cd1 v5.15.135 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0dcc9b4097d860d9af52db5366a8755c13468d13 Fixes: bcca2d67656f ('tests: add test for dormant on/off/on bug') Signed-off-by: Thomas Haller --- tests/shell/testcases/transactions/table_onoff | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/shell/testcases/transactions/table_onoff b/tests/shell/testcases/transactions/table_onoff index 831d4614c1f2..0e70ad2cc3f4 100755 --- a/tests/shell/testcases/transactions/table_onoff +++ b/tests/shell/testcases/transactions/table_onoff @@ -11,7 +11,9 @@ delete table ip t EOF if [ $? -eq 0 ]; then - exit 1 + echo "Command to re-awaken a dormant table did not fail. Lacking https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c9bd26513b3a11b3adb3c2ed8a31a01a87173ff1 ?" + "$NFT_TEST_BASEDIR/helpers/eval-exit-code" kernel 6.6 6.5.6 6.1.56 5.15.135 + exit $? fi set -e From patchwork Tue Oct 17 08:49:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Haller X-Patchwork-Id: 1849900 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Nzec84gE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=netfilter-devel-owner@vger.kernel.org; receiver=patchwork.ozlabs.org) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by legolas.ozlabs.org (Postfix) with ESMTP id 4S8nmr0Z8yz20d4 for ; Tue, 17 Oct 2023 19:52:44 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234700AbjJQIwk (ORCPT ); Tue, 17 Oct 2023 04:52:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234758AbjJQIwi (ORCPT ); Tue, 17 Oct 2023 04:52:38 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B2A4FC for ; Tue, 17 Oct 2023 01:51:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697532708; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PMUAV9soztNnC3XA+QFgpSl/liCfrpysk6E27v4lHLg=; b=Nzec84gEeMPI7rd6e8wLcw3MNABjxYXJPhE0sKtmQC+InlvDPsASTtc9hsXs9jv9O5BCwZ C1X/MexI7U3fv9fNks6FkbnAGPG17bkNsm2yH8XiRhy1iFVekNTOWMOVAD7yb9m9O7OxM7 dxsHAENk3vP2IJLFAlJDbSwaNs0c8gQ= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-152-k1VolVd0N32P6FBD-MxoZw-1; Tue, 17 Oct 2023 04:51:46 -0400 X-MC-Unique: k1VolVd0N32P6FBD-MxoZw-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 893C73827983 for ; Tue, 17 Oct 2023 08:51:46 +0000 (UTC) Received: from localhost.localdomain (unknown [10.39.192.93]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 09AB3492BEE; Tue, 17 Oct 2023 08:51:45 +0000 (UTC) From: Thomas Haller To: NetFilter Cc: Thomas Haller Subject: [PATCH nft v2 3/3] tests/shell: skip "vlan_8021ad_tag" test on older kernels Date: Tue, 17 Oct 2023 10:49:08 +0200 Message-ID: <20231017085133.1203402-4-thaller@redhat.com> In-Reply-To: <20231017085133.1203402-1-thaller@redhat.com> References: <20231017085133.1203402-1-thaller@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: netfilter-devel@vger.kernel.org The "vlan_8021ad_tag" test can only pass with certain (recent) kernels. Conditionally exit with status 77, if "eval-exit-code" determines that we don't have a suitable kernel version. In this case, we can find the fixes in: v6.6 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=af84f9e447a65b4b9f79e7e5d69e19039b431c56 v6.5.7 : https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3f6a5636a96687381b329649950f21258bae380d Fixes: 74cf3d16d8e9 ('tests: shell: add vlan match test case') Signed-off-by: Thomas Haller --- tests/shell/testcases/packetpath/vlan_8021ad_tag | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tests/shell/testcases/packetpath/vlan_8021ad_tag b/tests/shell/testcases/packetpath/vlan_8021ad_tag index 379a5710c1cb..6d908fe330df 100755 --- a/tests/shell/testcases/packetpath/vlan_8021ad_tag +++ b/tests/shell/testcases/packetpath/vlan_8021ad_tag @@ -47,4 +47,10 @@ EOF ip netns exec "$ns1" ping -c 1 10.1.1.2 ip netns exec "$ns2" $NFT list ruleset -ip netns exec "$ns2" $NFT list chain netdev t c | grep 'counter packets 1 bytes 84' +OUT="$(ip netns exec "$ns2" $NFT list chain netdev t c)" + +if ! printf "%s" "$OUT" | grep -q 'counter packets 1 bytes 84' ; then + echo "Filter did not match. Missing https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=af84f9e447a65b4b9f79e7e5d69e19039b431c56 ?" + "$NFT_TEST_BASEDIR/helpers/eval-exit-code" kernel 6.6 6.5.7 + exit $? +fi