From patchwork Mon Oct 7 11:50:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Huth X-Patchwork-Id: 1993578 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=djtZ2Vus; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XMcwD23htz1xsn for ; Mon, 7 Oct 2024 22:52:44 +1100 (AEDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sxmGT-0006qs-EK; Mon, 07 Oct 2024 07:51:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sxmGL-0006mK-4D for qemu-devel@nongnu.org; Mon, 07 Oct 2024 07:50:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sxmGJ-00058f-Li for qemu-devel@nongnu.org; Mon, 07 Oct 2024 07:50:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728301850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+WoMzCiXM4pA8Lpp1pnLdwd/6UlAnSuE1EoMVwCxQHQ=; b=djtZ2Vus1kQt7t9ByseSGn9KMFLWj/5Jl8JGLmcWj5kl4mmifkeMuMlsyS0HjEle4pce3X E3BW5j7TOYREf48YRLVumo7KMt3E6t1bjJmgT4zWVK3HnrwBYdVY50/8bR7O7ijYVlJFEg rSKnTEgzjN29ip+0uhnnj3Bk6Jdat1k= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-451-aqM-ThftNFCpCQtfp0r5zA-1; Mon, 07 Oct 2024 07:50:47 -0400 X-MC-Unique: aqM-ThftNFCpCQtfp0r5zA-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1EAEA19560B5; Mon, 7 Oct 2024 11:50:46 +0000 (UTC) Received: from thuth-p1g4.redhat.com (unknown [10.39.194.46]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7AFB219560AE; Mon, 7 Oct 2024 11:50:43 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org Cc: Peter Maydell , Michael Tokarev Subject: [PULL 05/12] gitlab-ci/build-oss-fuzz: print FAILED marker in case the test failed and run all tests Date: Mon, 7 Oct 2024 13:50:20 +0200 Message-ID: <20241007115027.243425-6-thuth@redhat.com> In-Reply-To: <20241007115027.243425-1-thuth@redhat.com> References: <20241007115027.243425-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 Received-SPF: pass client-ip=170.10.129.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.153, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org From: Michael Tokarev currently, if an oss-fuzz fails, the script does just `exit 1` without any additional output, and looking at the build log in the gitlab ci it is not clear what actually failed, without looking at build-oss-fuzz script and seeing this `exit 1`. Print easily recognizable error message about test failure, so it becomes obvious what exactly has failed. While at it, continue running other tests even in case of failure, and exit non-zero if at least one test failed. Signed-off-by: Michael Tokarev Reviewed-by: Thomas Huth Message-ID: <20241003121656.1173612-1-mjt@tls.msk.ru> Signed-off-by: Thomas Huth --- .gitlab-ci.d/buildtest.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 669c980c4b..34d3f4e9ab 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -628,12 +628,15 @@ build-oss-fuzz: - CC="clang" CXX="clang++" CFLAGS="-fsanitize=address" ./scripts/oss-fuzz/build.sh - export ASAN_OPTIONS="fast_unwind_on_malloc=0" + - failures=0 - for fuzzer in $(find ./build-oss-fuzz/DEST_DIR/ -executable -type f | grep -v slirp); do grep "LLVMFuzzerTestOneInput" ${fuzzer} > /dev/null 2>&1 || continue ; echo Testing ${fuzzer} ... ; - "${fuzzer}" -runs=1 -seed=1 || exit 1 ; + "${fuzzer}" -runs=1 -seed=1 || { echo "FAILED:"" ${fuzzer} exit code is $?"; failures=$(($failures+1)); }; done + - echo "Number of failures:"" $failures" + - test $failures = 0 build-tci: extends: .native_build_job_template