From patchwork Mon Jul 29 16:02:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Schwinge X-Patchwork-Id: 1966115 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=baylibre-com.20230601.gappssmtp.com header.i=@baylibre-com.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=gIi91QFL; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WXjnG6Zjvz1ybX for ; Tue, 30 Jul 2024 02:02:58 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2349D385840F for ; Mon, 29 Jul 2024 16:02:57 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by sourceware.org (Postfix) with ESMTPS id C7483385B529 for ; Mon, 29 Jul 2024 16:02:19 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org C7483385B529 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=baylibre.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=baylibre.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org C7483385B529 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::434 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1722268943; cv=none; b=eScBEyQH+bSqPX/OjxbKUCriLbWAhET2ZuZgH3cZHBe4trQ98jBwjyInrcFm8/Z0s4UWrN0xvc11TWCn5MQ9SqMaJYXCujvH+QUCDrymzhaT0a8CqPRBnrfDcrZkH0YrBbeHEYi1EbKlFMQWU04Rc7BgNmNSzUBB2EdvprfspXk= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1722268943; c=relaxed/simple; bh=U3P0t+Iwnpmom6Q1CyPRYaakdAW9eF6z33WftyEcdlU=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=jQEDN2+cHylq/xCqi0DtKZoRHqre7d1H3woiP2Bqz7k9lvOitpAyjTrnrRmhitYIbOekO+stjKVRa//WPhKCIatScjcDe+9rYy5UiOqt9aardoNJmCd0MOpJd5vdRVMbbdcaaBBJo2Q4SidIE+tEBDuItF7FX9c4pla8JlvrhtU= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-368663d7f80so1552003f8f.3 for ; Mon, 29 Jul 2024 09:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=baylibre-com.20230601.gappssmtp.com; s=20230601; t=1722268938; x=1722873738; darn=gcc.gnu.org; h=mime-version:message-id:date:user-agent:references:in-reply-to :subject:to:from:from:to:cc:subject:date:message-id:reply-to; bh=3w7JBP6iGw5mAopAc+Eg0D7slud6coyGNwpRd4t4/wM=; b=gIi91QFL8FB7sRQm2qLCIA9ZsHugBrEmr/TAbsynq8IVnYZZLIqiKvklJ0AhlQcGux Pid5FasrYMbPZSdQ5u2hslqv9byX5bwPT6q2yqpElVOipACJCd1um3crvRxvWpYWmQ43 hjvdTFsoQFdNB8UAOdkXo79+eq9uBMSgHLXvYwOslmyclI45rMnSdDViNFsbVaZPQLtW m9aHgc9uUHQnQ4mFORtB+NVYjVjuG1r0RdstEin/zmpow3LLqRgLi30nYlYwK4WUBsHG YFQH4RLsqVvgnk+BWQJ5tdyEKVwCChakO8RPJSWaZNkd3Lk5gDOjVAUTKvfHvAAlEnI1 n6bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722268938; x=1722873738; h=mime-version:message-id:date:user-agent:references:in-reply-to :subject:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=3w7JBP6iGw5mAopAc+Eg0D7slud6coyGNwpRd4t4/wM=; b=Fs6XxzR3B/ePllGPkiGoVz+CAUL/EHmwMKTfZ+oQPXsrV9AbQPNckZQzzjRt5aQB9X qxnbVj3Xvg8VD63bUDWMJ934/VCHJZ528EZBetu9vHyolafNqfhN9XTUQRGxN+EUKsqZ Nlr4Yfd6LEEvTo3yDobQfKtUsGmgng/Ft1Qn0C6vo8H+9aRI7ASF4z9hCtOw5lOsmxdI oTqz9x6HKjhWOJkL+wSHjSTXh34f9AmdjSV/YJro2G/TIjDW98kLerDai4cthdvrUFe2 f6CZB/E8v5kEDJ1VyfS8idgr9omFYCVayxLM+TrdgBQAu8OR0Bu6AIb4e/fNH3lsCEn8 KlMg== X-Forwarded-Encrypted: i=1; AJvYcCWzI7WQ7gKBpFsAGED4exFbFVMloDZLM4U9gANMNNq0s+ODKFRE2oXZHlUmFP/Nb4p5rx+LOj3zRzxBz/ic5EX0B9UrCbTqLQ== X-Gm-Message-State: AOJu0YwEkL9o42d3Mrp3f23XxZEwDLH2DVLLPZrEvXLlJAZ1Oiyz1Tmz mMxtel3Q/ZDNVsOz9pGcBK8//5G8gd7FutyvGa5e2OQSA4JOBcOVLzQ8MsGOm/Q= X-Google-Smtp-Source: AGHT+IG5klgJAk76VLKghq4JpzNBp1mEP+KSH1V1tr09ucPE6lohtPiOJti2hHarFfPzC1qf7PBC5Q== X-Received: by 2002:a5d:52c8:0:b0:367:95d2:4ec0 with SMTP id ffacd0b85a97d-36b5d0dddf7mr4517634f8f.62.1722268938102; Mon, 29 Jul 2024 09:02:18 -0700 (PDT) Received: from euler.schwinge.ddns.net (p200300c8b733b9005e8fc6f38b6af531.dip0.t-ipconnect.de. [2003:c8:b733:b900:5e8f:c6f3:8b6a:f531]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-36b367e072asm12504804f8f.42.2024.07.29.09.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jul 2024 09:02:17 -0700 (PDT) From: Thomas Schwinge To: Jonathan Wakely , libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org Subject: Polish libstdc++ 'dg-final' action 'file-io-diff' (was: [PATCH 4/8] libstdc++: Add file-io-diff to replace @diff@ markup in I/O tests) In-Reply-To: <20240722163010.3508334-4-jwakely@redhat.com> References: <20240722163010.3508334-1-jwakely@redhat.com> <20240722163010.3508334-4-jwakely@redhat.com> User-Agent: Notmuch/0.30+8~g47a4bad (https://notmuchmail.org) Emacs/29.4 (x86_64-pc-linux-gnu) Date: Mon, 29 Jul 2024 18:02:15 +0200 Message-ID: <87cymwqj94.fsf@euler.schwinge.ddns.net> MIME-Version: 1.0 X-Spam-Status: No, score=-10.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_DBL_SPAM autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Hi Jonathan! On 2024-07-22T17:28:38+0100, Jonathan Wakely wrote: > This adds a new dg-final action to compare two files after a test has > run, [...] Nice! > --- a/libstdc++-v3/testsuite/lib/libstdc++.exp > +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp > +# Compare output file written by test to expected result. > +# With two arguments the comparison is done via 'diff arg1 arg2'. > +# With one argument the comparison is done via 'diff arg1.tst arg1.txt'. > +proc file-io-diff { args } { > + set nargs [llength $args] > + if { $nargs < 1 } { > + error "too few arguments to file-io-diff" > + } > + if { $nargs > 2 } { > + error "too many arguments to file-io-diff" > + } > + if { $nargs == 1 } { > + set file1 [lindex $args 0] > + set file2 "${file1}.txt" > + append file1 ".tst" > + } else { > + set file1 [lindex $args 0] > + set file2 [lindex $args 1] > + } > + > + spawn -noecho diff -u $file1 $file2 > + expect { > + -re ".+" { > + set msg "files differ\n" > + append msg $expect_out(0,string) > + fail $msg > + exp_continue > + } > + } > + return > +} Via "deficient" GCN and nvptx target testing of my WIP C++ enablement trees, I ran into a minor nuisance here; OK to push the attached "Polish libstdc++ 'dg-final' action 'file-io-diff'"? On a proper target, powerpc64le GNU/Linux, with: $ echo yo >> libstdc++-v3/testsuite/data/istream_extractor_other-2.tst ... injected just for demonstration purposes (triggering for '27_io/basic_istream/extractors_other/char/2.cc' one instance of 'FAIL: files differ'), my patch changes 'libstdc++.sum' as follows: --- build-gcc/powerpc64le-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.sum 2024-07-29 17:11:42.333879749 +0200 +++ build-gcc/powerpc64le-unknown-linux-gnu/libstdc++-v3/testsuite/libstdc++.sum 2024-07-29 15:46:27.528214632 +0200 @@ -1,2 +1,2 @@ -Test run by tschwinge on Mon Jul 29 16:53:34 2024 +Test run by tschwinge on Mon Jul 29 15:28:20 2024 Native configuration is powerpc64le-unknown-linux-gnu @@ -11075,2 +11075,3 @@ PASS: 27_io/basic_filebuf/close/12790-1.cc -std=gnu++17 execution test +PASS: 27_io/basic_filebuf/close/char/1.cc -std=gnu++17 file-io-diff filebuf_members-1 PASS: 27_io/basic_filebuf/close/char/1.cc -std=gnu++17 (test for excess errors) @@ -11722,2 +11723,4 @@ PASS: 27_io/basic_istream/extractors_other/char/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/extractors_other/char/2.cc -std=gnu++17 file-io-diff istream_extractor_other-1 +FAIL: 27_io/basic_istream/extractors_other/char/2.cc -std=gnu++17 file-io-diff istream_extractor_other-2 PASS: 27_io/basic_istream/extractors_other/char/2.cc -std=gnu++17 (test for excess errors) @@ -11748,2 +11751,4 @@ PASS: 27_io/basic_istream/extractors_other/wchar_t/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/extractors_other/wchar_t/2.cc -std=gnu++17 file-io-diff wistream_extractor_other-1 +PASS: 27_io/basic_istream/extractors_other/wchar_t/2.cc -std=gnu++17 file-io-diff wistream_extractor_other-2 PASS: 27_io/basic_istream/extractors_other/wchar_t/2.cc -std=gnu++17 (test for excess errors) @@ -11772,2 +11777,3 @@ PASS: 27_io/basic_istream/get/char/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/get/char/2.cc -std=gnu++17 file-io-diff istream_unformatted-1 PASS: 27_io/basic_istream/get/char/2.cc -std=gnu++17 (test for excess errors) @@ -11779,2 +11785,3 @@ PASS: 27_io/basic_istream/get/wchar_t/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/get/wchar_t/2.cc -std=gnu++17 file-io-diff istream_unformatted-1 PASS: 27_io/basic_istream/get/wchar_t/2.cc -std=gnu++17 (test for excess errors) @@ -11812,2 +11819,3 @@ PASS: 27_io/basic_istream/ignore/char/2.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/ignore/char/3.cc -std=gnu++17 file-io-diff istream_unformatted-1 PASS: 27_io/basic_istream/ignore/char/3.cc -std=gnu++17 (test for excess errors) @@ -11828,2 +11836,3 @@ PASS: 27_io/basic_istream/ignore/wchar_t/2.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/ignore/wchar_t/3.cc -std=gnu++17 file-io-diff istream_unformatted-1 PASS: 27_io/basic_istream/ignore/wchar_t/3.cc -std=gnu++17 (test for excess errors) @@ -11844,2 +11853,3 @@ PASS: 27_io/basic_istream/peek/char/12296.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/peek/char/6414.cc -std=gnu++17 file-io-diff istream_seeks-1 PASS: 27_io/basic_istream/peek/char/6414.cc -std=gnu++17 (test for excess errors) @@ -11850,2 +11860,3 @@ PASS: 27_io/basic_istream/peek/wchar_t/12296.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/peek/wchar_t/6414.cc -std=gnu++17 file-io-diff wistream_seeks-1 PASS: 27_io/basic_istream/peek/wchar_t/6414.cc -std=gnu++17 (test for excess errors) @@ -11893,2 +11904,4 @@ PASS: 27_io/basic_istream/seekg/char/exceptions_badbit_throw.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/seekg/char/fstream.cc -std=gnu++17 file-io-diff istream_seeks-1 +PASS: 27_io/basic_istream/seekg/char/fstream.cc -std=gnu++17 file-io-diff istream_seeks-2 PASS: 27_io/basic_istream/seekg/char/fstream.cc -std=gnu++17 (test for excess errors) @@ -11907,2 +11920,4 @@ PASS: 27_io/basic_istream/seekg/wchar_t/exceptions_badbit_throw.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/seekg/wchar_t/fstream.cc -std=gnu++17 file-io-diff wistream_seeks-1 +PASS: 27_io/basic_istream/seekg/wchar_t/fstream.cc -std=gnu++17 file-io-diff wistream_seeks-2 PASS: 27_io/basic_istream/seekg/wchar_t/fstream.cc -std=gnu++17 (test for excess errors) @@ -11941,2 +11956,4 @@ PASS: 27_io/basic_istream/tellg/char/exceptions_badbit_throw.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/tellg/char/fstream.cc -std=gnu++17 file-io-diff istream_seeks-1 +PASS: 27_io/basic_istream/tellg/char/fstream.cc -std=gnu++17 file-io-diff istream_seeks-2 PASS: 27_io/basic_istream/tellg/char/fstream.cc -std=gnu++17 (test for excess errors) @@ -11955,2 +11972,4 @@ PASS: 27_io/basic_istream/tellg/wchar_t/exceptions_badbit_throw.cc -std=gnu++17 execution test +PASS: 27_io/basic_istream/tellg/wchar_t/fstream.cc -std=gnu++17 file-io-diff wistream_seeks-1 +PASS: 27_io/basic_istream/tellg/wchar_t/fstream.cc -std=gnu++17 file-io-diff wistream_seeks-2 PASS: 27_io/basic_istream/tellg/wchar_t/fstream.cc -std=gnu++17 (test for excess errors) @@ -12026,2 +12045,3 @@ PASS: 27_io/basic_ofstream/native_handle/wchar_t/1.cc -std=gnu++26 execution test +PASS: 27_io/basic_ofstream/open/char/1.cc -std=gnu++17 file-io-diff ofstream_members-1 PASS: 27_io/basic_ofstream/open/char/1.cc -std=gnu++17 (test for excess errors) @@ -12221,4 +12241,7 @@ PASS: 27_io/basic_ostream/inserters_character/wchar_t/deleted.cc -std=gnu++26 (test for excess errors) +PASS: 27_io/basic_ostream/inserters_other/char/1.cc -std=gnu++17 file-io-diff ostream_inserter_other-1 +PASS: 27_io/basic_ostream/inserters_other/char/1.cc -std=gnu++17 file-io-diff ostream_inserter_other-2 PASS: 27_io/basic_ostream/inserters_other/char/1.cc -std=gnu++17 (test for excess errors) PASS: 27_io/basic_ostream/inserters_other/char/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_ostream/inserters_other/char/2.cc -std=gnu++17 file-io-diff ostream_inserter_other_in ostream_inserter_other_out PASS: 27_io/basic_ostream/inserters_other/char/2.cc -std=gnu++17 (test for excess errors) @@ -12255,4 +12278,7 @@ PASS: 27_io/basic_ostream/inserters_other/char/volatile_ptr.cc -std=gnu++26 execution test +PASS: 27_io/basic_ostream/inserters_other/wchar_t/1.cc -std=gnu++17 file-io-diff wostream_inserter_other-1 +PASS: 27_io/basic_ostream/inserters_other/wchar_t/1.cc -std=gnu++17 file-io-diff wostream_inserter_other-2 PASS: 27_io/basic_ostream/inserters_other/wchar_t/1.cc -std=gnu++17 (test for excess errors) PASS: 27_io/basic_ostream/inserters_other/wchar_t/1.cc -std=gnu++17 execution test +PASS: 27_io/basic_ostream/inserters_other/wchar_t/2.cc -std=gnu++17 file-io-diff wostream_inserter_other_in.txt wostream_inserter_other_out.txt PASS: 27_io/basic_ostream/inserters_other/wchar_t/2.cc -std=gnu++17 (test for excess errors) @@ -13103,2 +13129,3 @@ PASS: 27_io/ios_base/storage/68197.cc -std=gnu++17 execution test +PASS: 27_io/ios_base/sync_with_stdio/1.cc -std=gnu++17 file-io-diff ios_base_members_static-1 PASS: 27_io/ios_base/sync_with_stdio/1.cc -std=gnu++17 (test for excess errors) @@ -16601,3 +16628,2 @@ PASS: ext/vstring/types/23767.cc -std=gnu++17 (test for excess errors) -FAIL: files differ PASS: special_functions/01_assoc_laguerre/check_nan.cc -std=gnu++17 (test for excess errors) @@ -19501,3 +19527,3 @@ -# of expected passes 18615 +# of expected passes 18641 # of unexpected failures 1 Note several instances of 'PASS: [...] file-io-diff [...]' appear, and the unspecific 'FAIL: files differ' (near the end) turn into specific: FAIL: 27_io/basic_istream/extractors_other/char/2.cc -std=gnu++17 file-io-diff istream_extractor_other-2 (Again, that FAIL's injected for demonstration purposes only.) The '*.log' and '*.sum' files also look as expected. Grüße Thomas From 5309eeb8f74a690711474679b059e52d4f6ab8ca Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Mon, 29 Jul 2024 13:32:36 +0200 Subject: [PATCH] Polish libstdc++ 'dg-final' action 'file-io-diff' Follow-up to recent commit 515da03a838db05443ebcc4c543a405bed764188 "libstdc++: Add file-io-diff to replace @diff@ markup in I/O tests". Currently, if a 'dg-final' action 'file-io-diff' passes, we print nothing (should: 'PASS: [...]'), but if it fails, we just print: 'FAIL: files differ', for example ('*.log' file): [...] FAIL: 27_io/basic_ostream/inserters_other/wchar_t/2.cc -std=gnu++17 (test for excess errors) [...] UNRESOLVED: 27_io/basic_ostream/inserters_other/wchar_t/2.cc -std=gnu++17 compilation failed to produce executable diff: wostream_inserter_other_in.txt: No such file or directory diff: wostream_inserter_other_out.txt: No such file or directory FAIL: files differ diff: wostream_inserter_other_in.txt: No such file or directory diff: wostream_inserter_other_out.txt: No such file or directory When later the '*.sum' files get sorted, these 'FAIL: files differ' instances aren't grouped anymore with the other test cases' results, but they appear en bloc, lexically sorted between ('e[...]' and 's[...]'), for example: [...] PASS: ext/vstring/types/23767.cc -std=gnu++17 (test for excess errors) FAIL: files differ FAIL: files differ FAIL: files differ PASS: special_functions/01_assoc_laguerre/check_nan.cc -std=gnu++17 (test for excess errors) [...] Also, we shouldn't emit the actual 'diff' into the '*.sum' file, but just into the '*.log* file, and there's no need for 'spawn'/'expect', as we're not matching any specific messages. libstdc++-v3/ * testsuite/lib/libstdc++.exp (file-io-diff): Polish. --- libstdc++-v3/testsuite/lib/libstdc++.exp | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/libstdc++-v3/testsuite/lib/libstdc++.exp b/libstdc++-v3/testsuite/lib/libstdc++.exp index c11e752ecfb..18331c80bc2 100644 --- a/libstdc++-v3/testsuite/lib/libstdc++.exp +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp @@ -1671,6 +1671,8 @@ if { [info exists env(GCC_RUNTEST_PARALLELIZE_DIR)] \ } +# Utility functions, invoked via dg-final. + # Compare output file written by test to expected result. # With two arguments the comparison is done via 'diff arg1 arg2'. # With one argument the comparison is done via 'diff arg1.tst arg1.txt'. @@ -1682,6 +1684,10 @@ proc file-io-diff { args } { if { $nargs > 2 } { error "too many arguments to file-io-diff" } + + set testcase [testname-for-summary] + set description "$testcase file-io-diff $args" + if { $nargs == 1 } { set file1 [lindex $args 0] set file2 "${file1}.txt" @@ -1691,14 +1697,11 @@ proc file-io-diff { args } { set file2 [lindex $args 1] } - spawn -noecho diff -u $file1 $file2 - expect { - -re ".+" { - set msg "files differ\n" - append msg $expect_out(0,string) - fail $msg - exp_continue - } + if { [catch { exec diff -u $file1 $file2 } msg] } { + fail $description + verbose -log "'diff':\n$msg" + } else { + pass $description } return } -- 2.34.1