From patchwork Wed Jun 26 00:31:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Malcolm X-Patchwork-Id: 1952314 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=gEEdfX+1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W82hd1Z6Xz20XB for ; Wed, 26 Jun 2024 10:32:14 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 10D113870C22 for ; Wed, 26 Jun 2024 00:32:12 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4F8F63870C14 for ; Wed, 26 Jun 2024 00:31:45 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4F8F63870C14 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 4F8F63870C14 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719361907; cv=none; b=D4ajoAqh469HhhJYTF5Rf9e0NNPKASRliJTarbIfrzTiAKgCf1UAmD3E9HfQL/ZQMvd0MgwXMBFytDgH7EIzraw41TzSrnJ3H+APawqBrXQ7ZmeOA69iVs/p76QSn++Rkv6js7YhuJATZT0q43Zkgfzd3/kOiE+K9gEYjhXVQOg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1719361907; c=relaxed/simple; bh=grYoUCoP9gPXdj1stTwoEJvrNopvvNg0IBFe0Ahaxtk=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=PPNpzVfjM9QlM3uQcdq1oiRFvihfIE978+Tut0Lz+04Sq5l4kbkA2if/ZUeWqjE1FK+5qKFF9aA/y98X6lT1M0MIxT2GZv237OVkytT7I/LjjwFXse0BsWGbD9gSIerhVRgm+DMT2fphB0WFX4qK3pUcuu24t3AfmH3+/L0ds2g= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719361905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bxdbCIokjhux4t0kfWZ+HWWZPUzUZPao8ujKc3RDnGY=; b=gEEdfX+1N/M6XdKWLRBcSlkczsHxQpEU2ru8TcaBrmRIiSUHV17JgEX3UamcSsS96yInmd i3agiZvP3lKeLkzuXrwOMgIt5RGatVpVjUcXRAj1uCuSkNUAaPO3cYmXxKMnIGP8kKIp7V etXTcJ/9tGmwFtB9lnIEbcr0aiuPFyw= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-316-z_bycPkGMpKLzz1GY_H8Ag-1; Tue, 25 Jun 2024 20:31:41 -0400 X-MC-Unique: z_bycPkGMpKLzz1GY_H8Ag-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id EF9581956080; Wed, 26 Jun 2024 00:31:37 +0000 (UTC) Received: from t14s.localdomain.com (unknown [10.22.16.8]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 807B9300021A; Wed, 26 Jun 2024 00:31:36 +0000 (UTC) From: David Malcolm To: gcc-patches@gcc.gnu.org, Dimitar Dimitrov Cc: David Malcolm Subject: [pushed] testsuite: use check-jsonschema for validating .sarif files [PR109360] Date: Tue, 25 Jun 2024 20:31:31 -0400 Message-Id: <20240626003131.1428584-1-dmalcolm@redhat.com> In-Reply-To: <5ced30f68e42da0bf2f975a17ba3ca32c92654d5.camel@redhat.com> References: <5ced30f68e42da0bf2f975a17ba3ca32c92654d5.camel@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-10.4 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, KAM_SHORT, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, RCVD_IN_SBL_CSS, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org As reported here: https://gcc.gnu.org/pipermail/gcc-patches/2024-June/655434.html the schema validation I added for generated .sarif files in r15-1541-ga84fe222029ff2 used the "jsonschema" command line tool, which has been deprecated by more recent versions of the Python 3 "jsonschema" module. This patch updates the validation to use the more recent "check-jsonschema" command line tool, from the Python 3 "check-jsonschema" module, fixing the testsuite FAILs due to the deprecation message. As an added bonus, the output on validation failures is *much* nicer, e.g. if I undo r15-1540-g9f4fdc3acebcf6, the error messages begin like this: verify-sarif-file: res: Schema validation errors were encountered. diagnostic-format-sarif-file-bad-utf8-pr109098-1.c.sarif::$.runs[0].results[0].locations[0].physicalLocation.region.startColumn: 0 is less than the minimum of 1 diagnostic-format-sarif-file-bad-utf8-pr109098-1.c.sarif::$.runs[0].results[0].relatedLocations[0].physicalLocation.region.startColumn: 0 is less than the minimum of 1 diagnostic-format-sarif-file-bad-utf8-pr109098-1.c.sarif::$.runs[0].results[0].relatedLocations[1].physicalLocation.region.startColumn: 0 is less than the minimum of 1 diagnostic-format-sarif-file-bad-utf8-pr109098-1.c.sarif::$.runs[0].results[0].relatedLocations[2].physicalLocation.region.startColumn: 0 is less than the minimum of 1 child process exited abnormally FAIL: c-c++-common/diagnostic-format-sarif-file-bad-utf8-pr109098-1.c -Wc++-compat (test .sarif output against SARIF schema) Tested with Python 3.8 with check_jsonschema 0.28.6 Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu. Successful run of analyzer integration tests on x86_64-pc-linux-gnu. Pushed to trunk as r15-1633-g17967907102099. gcc/ChangeLog: PR testsuite/109360 * doc/install.texi (Python3 modules): Update SARIF validation requirement to use check-jsonschema rather than jsonschema. gcc/testsuite/ChangeLog: PR testsuite/109360 * lib/scansarif.exp (verify-sarif-file): Use check-jsonschema rather than jsonschema, updating the invocation accordingly. * lib/target-supports.exp (check_effective_target_jsonschema): Convert to... (check_effective_target_check_jsonschema): ...this. Signed-off-by: David Malcolm --- gcc/doc/install.texi | 7 ++++--- gcc/testsuite/lib/scansarif.exp | 8 ++++---- gcc/testsuite/lib/target-supports.exp | 6 +++--- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/gcc/doc/install.texi b/gcc/doc/install.texi index 0c7691651466..b54569925837 100644 --- a/gcc/doc/install.texi +++ b/gcc/doc/install.texi @@ -461,9 +461,10 @@ is shown below: @code{gcov}, @code{gzip}, @code{json}, @code{os} and @code{pytest}. @item SARIF testsuite -Tests of SARIF output will use the @code{jsonschema} program from the -@code{jsonschema} module (if available) to validate generated .sarif files. -If this tool is not found, the validation parts of those tests are skipped. +Tests of SARIF output will use the @code{check-jsonschema} program from +the @code{check-jsonschema} module (if available) to validate generated +.sarif files. If this tool is not found, the validation parts of those +tests are skipped. @item c++ cxx api generation @code{csv}, @code{os}, @code{sys} and @code{time}. diff --git a/gcc/testsuite/lib/scansarif.exp b/gcc/testsuite/lib/scansarif.exp index 3eb38b8102e9..cc0890ef5d8b 100644 --- a/gcc/testsuite/lib/scansarif.exp +++ b/gcc/testsuite/lib/scansarif.exp @@ -57,7 +57,7 @@ proc scan-sarif-file-not { args } { # Assuming python3 is available, use verify-sarif-file.py to check # that the .sarif file is UTF-8 encoded and is parseable as JSON. # -# Assuming "jsonschema" is available, use it to verify that the .sarif +# Assuming "check-jsonschema" is available, use it to verify that the .sarif # file complies with the SARIF schema. proc verify-sarif-file { args } { @@ -86,8 +86,8 @@ proc verify-sarif-file { args } { # Verify that the file complies with the SARIF schema. # Check that jsonschema is installed. - if { ![check_effective_target_jsonschema] } { - unsupported "$testcase verify-sarif-file: jsonschema is missing" + if { ![check_effective_target_check_jsonschema] } { + unsupported "$testcase verify-sarif-file: check-jsonschema is missing" return } @@ -95,7 +95,7 @@ proc verify-sarif-file { args } { verbose "schema_file: $schema_file" 2 set what "$testcase (test .sarif output against SARIF schema)" - if [catch {exec jsonschema --instance $output_file $schema_file} res ] { + if [catch {exec check-jsonschema --schemafile $schema_file $output_file} res ] { verbose "verify-sarif-file: res: $res" 2 fail "$what" return diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp index ed30cd18ad69..828c46fb15e5 100644 --- a/gcc/testsuite/lib/target-supports.exp +++ b/gcc/testsuite/lib/target-supports.exp @@ -13506,10 +13506,10 @@ proc check_effective_target_heap_trampoline {} { return 0 } -# Return 1 if jsonschema is available. +# Return 1 if check-jsonschema is available. -proc check_effective_target_jsonschema { } { - set result [remote_exec host "jsonschema --version"] +proc check_effective_target_check_jsonschema { } { + set result [remote_exec host "check-jsonschema --version"] set status [lindex $result 0] if { $status == 0 } then { return 1;