From patchwork Fri Jul 15 07:22:11 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Schwinge X-Patchwork-Id: 648692 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3rrPD93DnBz9sBf for ; Fri, 15 Jul 2016 17:22:48 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b=F58zgZ/E; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; q=dns; s=default; b=INiCwMn95hTgqeUk LfN5ia68MzgBainmxHNFGJhg90bDAW91WjnH6FlwOYA73TN4U8P15mSFr5Z+dTmD IEf5Qbsyd+jCW4ADuowI6l3CgJ5Lt8ztAGF20dX5/zsSXcQ5lZS3LyIvSvepfW3o Wr9XLYaLO8FR1TD1f8BHD7tKcyM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=default; bh=4AGdYE12FKrTncJvzsDcfs byjs0=; b=F58zgZ/Ehtk01PZX0B+u6qaMWiJY5axMKFlUSoPC2oqJmJuGwjZBHP 7s/cz87o2NBivug0Gd96XoCmXazBRKM+Pc15k4+nO3TWmVm4NB3dN7MDJrScIPER scQeG7papEMp8kvKMASakZ00jvKEw0JeRFzJ++FofdHKGyI3eeKmw= Received: (qmail 53398 invoked by alias); 15 Jul 2016 07:22:40 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 53384 invoked by uid 89); 15 Jul 2016 07:22:37 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.4 required=5.0 tests=AWL, BAYES_00, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=no version=3.3.2 spammy=disclaimer, fd, DISCLAIMER, UD:gmane.org X-HELO: relay1.mentorg.com Received: from relay1.mentorg.com (HELO relay1.mentorg.com) (192.94.38.131) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with (AES256-GCM-SHA384 encrypted) ESMTPS; Fri, 15 Jul 2016 07:22:27 +0000 Received: from svr-orw-fem-02x.mgc.mentorg.com ([147.34.96.206] helo=SVR-ORW-FEM-02.mgc.mentorg.com) by relay1.mentorg.com with esmtp id 1bNxS3-0003I2-TA from Thomas_Schwinge@mentor.com ; Fri, 15 Jul 2016 00:22:23 -0700 Received: from tftp-cs (147.34.91.1) by svr-orw-fem-02.mgc.mentorg.com (147.34.96.168) with Microsoft SMTP Server id 14.3.224.2; Fri, 15 Jul 2016 00:22:23 -0700 Received: by tftp-cs (Postfix, from userid 49978) id 06FCFC2222; Fri, 15 Jul 2016 00:22:22 -0700 (PDT) From: Thomas Schwinge To: Martin =?utf-8?Q?Li=C5=A1ka?= CC: Jakub Jelinek , Jeff Law , , Joseph Myers Subject: Re: [RFC, v2] Test coverage for --param boundary values In-Reply-To: <69e2f7ea-7f29-6a9a-fafe-7247d1bcf81f@suse.cz> References: <69e2f7ea-7f29-6a9a-fafe-7247d1bcf81f@suse.cz> User-Agent: Notmuch/0.9-125-g4686d11 (http://notmuchmail.org) Emacs/24.5.1 (i586-pc-linux-gnu) Date: Fri, 15 Jul 2016 09:22:11 +0200 Message-ID: <8760s7e57g.fsf@kepler.schwinge.homeip.net> MIME-Version: 1.0 Hi! On Fri, 8 Jul 2016 14:47:46 +0200, Martin Liška wrote: > From f84ce7be4a998089541fb4512e19f54a4ec25cf6 Mon Sep 17 00:00:00 2001 > From: marxin > Date: Fri, 8 Jul 2016 10:59:24 +0200 > Subject: [PATCH] Add tests that test boundary values of params This became r238249. Yay for pushing testing boundaries; next is some fuzzy testing for GCC? ;-D > gcc/ChangeLog: > > 2016-07-08 Martin Liska > > * Makefile.in: Append rule for params-options.h. > * params-options.h: New file. > > gcc/testsuite/ChangeLog: > > 2016-07-08 Martin Liska > > * gcc.dg/params/blocksort-part.c: New test. > * gcc.dg/params/params.exp: New file. :-/ (GNU-style ChangeLogs are just so useful... not.) > --- /dev/null > +++ b/gcc/testsuite/gcc.dg/params/blocksort-part.c > @@ -0,0 +1,706 @@ > + > +/*-------------------------------------------------------------*/ > +/*--- Block sorting machinery ---*/ > +/*--- blocksort.c ---*/ > +/*-------------------------------------------------------------*/ > + > +/* ------------------------------------------------------------------ > + This file is part of bzip2/libbzip2, a program and library for > + lossless, block-sorting data compression. > + > + bzip2/libbzip2 version 1.0.6 of 6 September 2010 > + Copyright (C) 1996-2010 Julian Seward > + > + Please read the WARNING, DISCLAIMER and PATENTS sections in the > + README file. > + > + This program is released under the terms of the license contained > + in the file LICENSE. > +[...] Are there any issues with including that file without the accompanying/referenced README and LICENSE files? > --- /dev/null > +++ b/gcc/testsuite/gcc.dg/params/params.exp > @@ -0,0 +1,64 @@ > +[...] > +# GCC testsuite that uses the `dg.exp' driver. > + > +# Load support procs. > +load_lib gcc-dg.exp > + > +# Initialize `dg'. > +dg-init > + > +proc param_run_test { param_name param_value } { > + global srcdir > + global subdir > + > + dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value" > +} > + > +set fd [open "$objdir/../../params.options" r] (I do understand what you're doing there, but) is it kosher to refer to a file from GCC's build tree inside the test suite? I thought the idea was that you could run the testsuite without the build tree being available -- as much that's possible. As far as I remember there are a few exception to this rule already, so maybe adding one more is not much of a problem. ;-) (I do know that for our internal testing we have a list of files that need to be preserved from the GCC build directory for later testing without the build directory being available; so I'll add this file to the list; Joseph CCed in case he has some additional comments after returning from his vacations.) shows one instance of the problem, that I could quickly find: [...] ERROR: tcl error sourcing /home/jenkins/workspace/BuildToolchainThunder_elf_test_upstream/toolchain/src/gcc/testsuite/gcc.dg/params/params.exp. ERROR: couldn't open "/home/jenkins/workspace/BuildToolchainThunder_elf_test_upstream/toolchain/testresults/aarch64-thunderx-elf/../../params.options": no such file or directory [...] > +set text [read $fd] > +close $fd > + > +# Main loop. > +foreach params [split $text "\n"] { > + set parts [split $params "="] > + set name [string trim [lindex $parts 0] '"'] > + set values [split [lindex $parts 1] ","] > + if { [llength $values] == 3 } { > + set default [lindex $values 0] > + set min [lindex $values 1] > + set max [lindex $values 2] > + set int_max "INT_MAX" > + > + if { $min != -1 } { > + param_run_test $name $min > + } > + if { $max != $min && $max > 0 && $max != $int_max } { > + param_run_test $name $max > + } > + } > + if { [llength $values] == 5 } { > + foreach v $values { > + param_run_test $name $v > + } > + } > +} This produces ambiguous test result lines: Running /scratch/tschwing/nvidiak20-2/gcc/trunk-light/source-gcc/gcc/testsuite/gcc.dg/params/params.exp ... PASS: gcc.dg/params/blocksort-part.c (test for excess errors) PASS: gcc.dg/params/blocksort-part.c (test for excess errors) PASS: gcc.dg/params/blocksort-part.c (test for excess errors) PASS: gcc.dg/params/blocksort-part.c (test for excess errors) [many more] OK to fix this as follows: PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=0 (test for excess errors) PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=50 (test for excess errors) PASS: gcc.dg/params/blocksort-part.c -O3 --param inline-min-speedup=0 (test for excess errors) PASS: gcc.dg/params/blocksort-part.c -O3 --param max-inline-insns-single=0 (test for excess errors) [...] commit ae350ce9af062d7d2c05225280596b1bc982728b Author: Thomas Schwinge Date: Fri Jul 15 09:13:19 2016 +0200 Disambiguate gcc.dg/params/params.exp test results gcc/testsuite/ * gcc.dg/params/params.exp (param_run_test): Swap flags and default-extra-flags arguments when calling dg-runtest. --- gcc/testsuite/gcc.dg/params/params.exp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Grüße Thomas diff --git gcc/testsuite/gcc.dg/params/params.exp gcc/testsuite/gcc.dg/params/params.exp index f2f397e..dd82a5d 100644 --- gcc/testsuite/gcc.dg/params/params.exp +++ gcc/testsuite/gcc.dg/params/params.exp @@ -28,7 +28,7 @@ proc param_run_test { param_name param_value } { global srcdir global subdir - dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value" + dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" "" } set fd [open "$objdir/../../params.options" r]