From patchwork Thu Aug 15 18:11:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Malcolm X-Patchwork-Id: 1972870 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=f+a5XUUA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WlCs80BnVz1yXl for ; Fri, 16 Aug 2024 04:12:42 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9F75D385C6C6 for ; Thu, 15 Aug 2024 18:12:40 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTP id 161AF3858D29 for ; Thu, 15 Aug 2024 18:12:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 161AF3858D29 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 161AF3858D29 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1723745525; cv=none; b=ijde7IH/pUOfudXPJy2iz5xsqUQmet1TZJOh5sUZF80aMa6nYLd59mSIF3/Vz8b8rqDHCUWxHLuN5i5eeLNjjMSGJHCCJswn1QO43fqf6UpjA0nW3etLT8D1VkmMQ0CEEPMhTCrttWe5t+H6TP0Strs6j5EP5z9+QIUFaunGRTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1723745525; c=relaxed/simple; bh=ipA5l3VR7sJ39VEBOyITAC/p41X+qLwV7JPsnl5OaJc=; h=DKIM-Signature:From:To:Subject:Date:Message-Id:MIME-Version; b=DLfZ1spe+c4MTTqOb7INBWy9/6YgMcoqC63f6itOoiOe9p0M2BPhVYBIV0VCmBEjELW50BXrDt+hm6Bxy1k7ZfVnKDGRDBhLtSv6MpiHkCRSuiXKB8WNccgXquAQ+WyuekpmUyLdWFIoN5Azey7A1/EhuN/u9RhYsU4+mYdCIX8= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723745522; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=q04dxcxy8VBpw+JlpVvR6myZOLWuJI4oe/ormri1Yi4=; b=f+a5XUUApoWR3x7mKGT3K5n71VC6ti1h2OXrtA0eyecIID+PMPRrMqeFY8lE6H05AOb7bi /gZgjUd2R+hDJ7vRB52CXVUac9mk1qqgbU3Eg/NmQMpgOHxq0ctWyw87RzeATO102cW0ME BOUBKNqgI2dD8wzqWKYf2x934fGEwqc= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-237-o97QPu4rO4iimBVmdsASzQ-1; Thu, 15 Aug 2024 14:12:00 -0400 X-MC-Unique: o97QPu4rO4iimBVmdsASzQ-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 02F0C1955F3B for ; Thu, 15 Aug 2024 18:12:00 +0000 (UTC) Received: from t14s.localdomain.com (unknown [10.22.16.43]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 127CA1955D47; Thu, 15 Aug 2024 18:11:58 +0000 (UTC) From: David Malcolm To: gcc-patches@gcc.gnu.org Cc: David Malcolm Subject: [PATCH 0/7] v3 of libdiagnostics Date: Thu, 15 Aug 2024 14:11:49 -0400 Message-Id: <20240815181156.1815075-1-dmalcolm@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, LIKELY_SPAM_BODY, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Here's v3 of my patch kit for "libdiagnostics", which makes GCC's diagnostics subsystem available as a shared library; see: https://gcc.gnu.org/wiki/libdiagnostics New in v3: * it bootstraps and pass regression tests * I added an opt-in configure flag: --enable-libdiagnostics, which must be enabled to build it (along with --enable-host-shared) * a new "sarif-replay" command-line tool that takes .sarif files and replays the diagnostics within them as if they were GCC diagnostics, in GCC's textual format (i.e. GCC as a SARIF *consumer*, as well as producer). This is implemented on top of libdiagnostics hence I've been "eating my own dogfood" * support for execution paths in libdiagnostics API * lots of fixes Patch 1 has libdiagnostic.h, the public header file Patch 2 has the implementation Patch 3 has the C++ wrapper API I added in v2 Patch 4 has a refactoring of gcc-dg.exp I needed for patch 5. Patch 5 has the testsuite for libdiagnostics itself Patch 6 implements JSON parsing support Patch 7 implements the sarif-replay command-line tool, and its testsuite, exercising various valid, invalid, and malformed input files. Successfully bootstrapped & regrtested on x86_64-pc-linux-gnu, both with and without --enable-libdiagnostics. With --enable-libdiagnostics the patch kit has this effect: # of .sum files: 20->22 (+2) FAIL: 110 PASS: 617481->617679 (+198) 100.03% XFAIL: 4512 XPASS: 13 UNTESTED: 136 UNSUPPORTED: 8058 where the two new .sum files are: BUILD/gcc/testsuite/libdiagnostics/libdiagnostics.sum: PASS: 132 tests BUILD/gcc/testsuite/sarif-replay/sarif-replay.sum: PASS: 66 tests OK for trunk? David Malcolm (7): libdiagnostics v3: header libdiagnostics v3: implementation libdiagnostics v3: add C++ wrapper API testsuite: move dg-test cleanup code from gcc-dg.exp to its own file libdiagnostics v3: test suite json: add json parsing support libdiagnostics: add a "sarif-replay" command-line tool [PR96032] configure | 42 + configure.ac | 35 + contrib/regenerate-sarif-spec-index.py | 60 + gcc/Makefile.in | 191 +- gcc/configure | 26 +- gcc/configure.ac | 16 + gcc/diagnostic-event-id.h | 6 + gcc/doc/install.texi | 13 + gcc/json-parsing.cc | 2394 +++++++++++++++++ gcc/json-parsing.h | 113 + gcc/json.cc | 2 +- gcc/json.h | 122 +- gcc/libdiagnostics++.h | 595 ++++ gcc/libdiagnostics.cc | 1652 ++++++++++++ gcc/libdiagnostics.h | 691 +++++ gcc/libdiagnostics.map | 72 + gcc/libsarifreplay.cc | 1747 ++++++++++++ gcc/libsarifreplay.h | 59 + gcc/sarif-replay.cc | 239 ++ gcc/sarif-spec-urls.def | 496 ++++ gcc/selftest-run-tests.cc | 1 + gcc/selftest.h | 1 + gcc/testsuite/lib/dg-test-cleanup.exp | 116 + gcc/testsuite/lib/gcc-dg.exp | 106 +- gcc/testsuite/lib/sarif-replay-dg.exp | 90 + gcc/testsuite/lib/sarif-replay.exp | 204 ++ .../libdiagnostics.dg/libdiagnostics.exp | 296 ++ gcc/testsuite/libdiagnostics.dg/sarif.py | 23 + gcc/testsuite/libdiagnostics.dg/test-dump.c | 69 + .../libdiagnostics.dg/test-error-c.py | 54 + .../test-error-with-note-c.py | 50 + .../libdiagnostics.dg/test-error-with-note.c | 74 + .../libdiagnostics.dg/test-error-with-note.cc | 55 + gcc/testsuite/libdiagnostics.dg/test-error.c | 59 + gcc/testsuite/libdiagnostics.dg/test-error.cc | 47 + .../libdiagnostics.dg/test-fix-it-hint-c.py | 46 + .../libdiagnostics.dg/test-fix-it-hint.c | 81 + .../libdiagnostics.dg/test-fix-it-hint.cc | 74 + .../libdiagnostics.dg/test-helpers++.h | 28 + .../libdiagnostics.dg/test-helpers.h | 72 + .../libdiagnostics.dg/test-labelled-ranges.c | 69 + .../libdiagnostics.dg/test-labelled-ranges.cc | 64 + .../libdiagnostics.dg/test-labelled-ranges.py | 48 + .../test-logical-location-c.py | 37 + .../libdiagnostics.dg/test-logical-location.c | 79 + .../libdiagnostics.dg/test-metadata-c.py | 45 + .../libdiagnostics.dg/test-metadata.c | 61 + .../test-multiple-lines-c.py | 83 + .../libdiagnostics.dg/test-multiple-lines.c | 76 + .../libdiagnostics.dg/test-no-column-c.py | 35 + .../libdiagnostics.dg/test-no-column.c | 52 + .../test-no-diagnostics-c.py | 42 + .../libdiagnostics.dg/test-no-diagnostics.c | 25 + .../test-note-with-fix-it-hint-c.py | 54 + .../test-note-with-fix-it-hint.c | 69 + .../test-text-sink-options.c | 59 + .../libdiagnostics.dg/test-warning-c.py | 54 + .../test-warning-with-path-c.py | 108 + .../test-warning-with-path.c | 125 + .../libdiagnostics.dg/test-warning.c | 65 + .../test-write-sarif-to-file-c.py | 55 + .../test-write-sarif-to-file.c | 55 + .../test-write-text-to-file.c | 47 + .../2.1.0-invalid/3.1-not-an-object.sarif | 6 + .../3.11.11-malformed-placeholder.sarif | 15 + ...1-missing-arguments-for-placeholders.sarif | 14 + ...ot-enough-arguments-for-placeholders.sarif | 14 + .../2.1.0-invalid/3.13.2-no-version.sarif | 6 + .../3.13.2-version-not-a-string.sarif | 6 + .../2.1.0-invalid/3.13.4-bad-runs.sarif | 7 + .../2.1.0-invalid/3.13.4-no-runs.sarif | 6 + .../3.13.4-non-object-in-runs.sarif | 7 + .../2.1.0-invalid/3.27.10-bad-level.sarif | 25 + .../2.1.0-unhandled/3.27.10-none-level.sarif | 25 + .../2.1.0-valid/error-with-note.sarif | 34 + .../2.1.0-valid/escaped-braces.sarif | 17 + .../2.1.0-valid/null-runs.sarif | 2 + .../2.1.0-valid/signal-1.c.sarif | 193 ++ .../2.1.0-valid/spec-example-1.sarif | 15 + .../2.1.0-valid/spec-example-2.sarif | 73 + .../2.1.0-valid/spec-example-3.sarif | 65 + .../2.1.0-valid/spec-example-4.sarif | 766 ++++++ .../2.1.0-valid/tutorial-example.sarif | 117 + gcc/testsuite/sarif-replay.dg/dg.exp | 46 + .../malformed-json/array-missing-comma.sarif | 6 + .../array-with-trailing-comma.sarif | 6 + .../malformed-json/bad-token.sarif | 6 + .../malformed-json/object-missing-comma.sarif | 7 + .../object-with-trailing-comma.sarif | 6 + libatomic/testsuite/lib/libatomic.exp | 1 + libgomp/testsuite/lib/libgomp.exp | 1 + libitm/testsuite/lib/libitm.exp | 1 + libphobos/testsuite/lib/libphobos-dg.exp | 1 + libstdc++-v3/testsuite/lib/libstdc++.exp | 1 + libvtv/testsuite/lib/libvtv.exp | 1 + 95 files changed, 12912 insertions(+), 108 deletions(-) create mode 100644 contrib/regenerate-sarif-spec-index.py create mode 100644 gcc/json-parsing.cc create mode 100644 gcc/json-parsing.h create mode 100644 gcc/libdiagnostics++.h create mode 100644 gcc/libdiagnostics.cc create mode 100644 gcc/libdiagnostics.h create mode 100644 gcc/libdiagnostics.map create mode 100644 gcc/libsarifreplay.cc create mode 100644 gcc/libsarifreplay.h create mode 100644 gcc/sarif-replay.cc create mode 100644 gcc/sarif-spec-urls.def create mode 100644 gcc/testsuite/lib/dg-test-cleanup.exp create mode 100644 gcc/testsuite/lib/sarif-replay-dg.exp create mode 100644 gcc/testsuite/lib/sarif-replay.exp create mode 100644 gcc/testsuite/libdiagnostics.dg/libdiagnostics.exp create mode 100644 gcc/testsuite/libdiagnostics.dg/sarif.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-dump.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-with-note-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-with-note.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error-with-note.cc create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-error.cc create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it-hint-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it-hint.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-fix-it-hint.cc create mode 100644 gcc/testsuite/libdiagnostics.dg/test-helpers++.h create mode 100644 gcc/testsuite/libdiagnostics.dg/test-helpers.h create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled-ranges.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled-ranges.cc create mode 100644 gcc/testsuite/libdiagnostics.dg/test-labelled-ranges.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical-location-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-logical-location.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-metadata.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple-lines-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-multiple-lines.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no-column-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no-column.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no-diagnostics-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-no-diagnostics.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with-fix-it-hint-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-note-with-fix-it-hint.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-text-sink-options.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning-with-path-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning-with-path.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-warning.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-sarif-to-file-c.py create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-sarif-to-file.c create mode 100644 gcc/testsuite/libdiagnostics.dg/test-write-text-to-file.c create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.1-not-an-object.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.11.11-malformed-placeholder.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.11.11-missing-arguments-for-placeholders.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.11.11-not-enough-arguments-for-placeholders.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.13.2-no-version.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.13.2-version-not-a-string.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.13.4-bad-runs.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.13.4-no-runs.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.13.4-non-object-in-runs.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-invalid/3.27.10-bad-level.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-unhandled/3.27.10-none-level.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/error-with-note.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/escaped-braces.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/null-runs.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/signal-1.c.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec-example-1.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec-example-2.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec-example-3.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/spec-example-4.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/2.1.0-valid/tutorial-example.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/dg.exp create mode 100644 gcc/testsuite/sarif-replay.dg/malformed-json/array-missing-comma.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/malformed-json/array-with-trailing-comma.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/malformed-json/bad-token.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/malformed-json/object-missing-comma.sarif create mode 100644 gcc/testsuite/sarif-replay.dg/malformed-json/object-with-trailing-comma.sarif