From patchwork Tue Oct 20 15:01:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1384991 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=EzPcNvAs; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CFxfV5MPtz9sRK for ; Wed, 21 Oct 2020 02:01:36 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 61D713945C38; Tue, 20 Oct 2020 15:01:33 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qv1-xf32.google.com (mail-qv1-xf32.google.com [IPv6:2607:f8b0:4864:20::f32]) by sourceware.org (Postfix) with ESMTPS id D9DF83851C25 for ; Tue, 20 Oct 2020 15:01:30 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org D9DF83851C25 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qv1-xf32.google.com with SMTP id w5so977447qvn.12 for ; Tue, 20 Oct 2020 08:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:references:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=RJhNE4IRN/+mRHJ3LeSKgkG8Z2Wp+jCKN+t3fdrJ28Y=; b=EzPcNvAszYpgNAQLLDZOb6gbxP1aplfBSDOEvhl8g+jb+V8igwacsRvWqonM0KwVFv kjXIJwkGM3uzcvU4uHXo3DI3yz9rMTWMH2KJci0Py3BaIbQy28vzxWrsefVJ10QL0OqJ Qx4Gf2t3LsLkJHKVPmNHYWijSG4NJpvFA4zHrsUh3RTvk8RW1pRMQyE6FMM58ueRLhRa eK2iKQxAis1r27WOn2Ivhkvu9NIEQcQVSe+iBllJladkwT2FTvd/7XP3ooTdR9eSEOD7 SrVlYyrjoYknxt3w3jOHsjbdwTnxPlnHL9QEtY7NKydq27U8YTzDs9/kmm2uI1nQ/P+D ZoCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:references:message-id :date:user-agent:mime-version:in-reply-to:content-language; bh=RJhNE4IRN/+mRHJ3LeSKgkG8Z2Wp+jCKN+t3fdrJ28Y=; b=OP6zryUHKPWwx2LcsUlhShCGM9ObJQrd9sPkavC3Xfe2EcMqAV8kcEQxRjxPdCRKVJ dpMT7BepQbURoAIt63lyuFuGM5pXlxWbzEWd/8y21nvsZFqFS+fcsfyw/MqhGkc5CN6Z bnHtuIgtV0LceQiDYSX6DhhcuuZTdmB4RI6uIo+s0JQJ6II0nFqVjau9KpMrjcRXxpmn MApsih2Zuz75UlsZSL/uIdr00PpmRr8jYi3Ocytt2qPMywlVamWH6op+/SMjmmUpodwG iJbjiXDkWrxc5vkaseclW2wJa93Kj1DUpz8+KqPBcZUVggBe7rBj45HjAXln6n+ZYsnn 5/PA== X-Gm-Message-State: AOAM532IGnhQPQ3GPswVsKgySg5dAxIK266rjJoMcTNjYHy2tOHi2hOT SF2EmafEr3QOYmvqGGFwAX0= X-Google-Smtp-Source: ABdhPJyfNnTjxIbHWvQspPVfN3ly0zHqBHgR/whpkD3vON9IJOJgWM/egyRHgWNtYC9KMzQLDNaLjw== X-Received: by 2002:ad4:5387:: with SMTP id i7mr3614472qvv.43.1603206090219; Tue, 20 Oct 2020 08:01:30 -0700 (PDT) Received: from ?IPv6:2620:10d:c0a8:1102:9962:2eab:92de:6d69? ([2620:10d:c091:480::1:6d18]) by smtp.googlemail.com with ESMTPSA id h9sm791516qth.78.2020.10.20.08.01.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 20 Oct 2020 08:01:28 -0700 (PDT) Subject: preprocessor: Further fix for EOF in macro args [PR97471] From: Nathan Sidwell To: GCC Patches References: <22291356-353e-fe73-600c-ac985df58df2@acm.org> Message-ID: <3715fa3f-5003-1a52-efff-1dfdf355f9a6@acm.org> Date: Tue, 20 Oct 2020 11:01:27 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <22291356-353e-fe73-600c-ac985df58df2@acm.org> Content-Language: en-US X-Spam-Status: No, score=-11.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" My previous attempt at fixing this was incorrect. The problem occurs earlier in that _cpp_lex_direct processes the unwinding EOF needs in collect_args mode. This patch changes it not to do that, in the same way as directive parsing works. Also collect_args shouldn't push_back such fake EOFs, and neither should funlike_invocation_p. libcpp/ * lex.c (_cpp_lex_direct): Do not complete EOF processing when parsing_args. * macro.c (collect_args): Do not unwind fake EOF. (funlike_invocation_p): Do not unwind fake EOF. (cpp_context): Replace abort with gcc_assert. gcc/testsuite/ * gcc.dg/cpp/endif.c: Move to ... * c-c++-common/cpp/endif.c: ... here. * gcc.dg/cpp/endif.h: Move to ... * c-c++-common/cpp/endif.h: ... here. * c-c++-common/cpp/eof-2.c: Adjust diagnostic. * c-c++-common/cpp/eof-3.c: Adjust diagnostic. pushing to trunk diff --git c/gcc/testsuite/gcc.dg/cpp/endif.c w/gcc/testsuite/c-c++-common/cpp/endif.c similarity index 100% rename from gcc/testsuite/gcc.dg/cpp/endif.c rename to gcc/testsuite/c-c++-common/cpp/endif.c diff --git c/gcc/testsuite/gcc.dg/cpp/endif.h w/gcc/testsuite/c-c++-common/cpp/endif.h similarity index 100% rename from gcc/testsuite/gcc.dg/cpp/endif.h rename to gcc/testsuite/c-c++-common/cpp/endif.h diff --git c/gcc/testsuite/c-c++-common/cpp/eof-2.c w/gcc/testsuite/c-c++-common/cpp/eof-2.c index 3a4af7f6850..9cc4fed6564 100644 --- c/gcc/testsuite/c-c++-common/cpp/eof-2.c +++ w/gcc/testsuite/c-c++-common/cpp/eof-2.c @@ -5,4 +5,4 @@ #define f(x) x #include "eof-2.h" - /* { dg-regexp {[^\n]*eof-2.h:4: error: unterminated argument list invoking macro "f"\n} } */ + /* { dg-regexp {[^\n]*eof-2.h:4:21: error: unterminated argument list invoking macro "f"\n} } */ diff --git c/gcc/testsuite/c-c++-common/cpp/eof-3.c w/gcc/testsuite/c-c++-common/cpp/eof-3.c index 316918e3a6c..e309a548c0c 100644 --- c/gcc/testsuite/c-c++-common/cpp/eof-3.c +++ w/gcc/testsuite/c-c++-common/cpp/eof-3.c @@ -3,6 +3,6 @@ /* { dg-do preprocess } */ /* { dg-additional-options "-include $srcdir/c-c++-common/cpp/eof-2.h" } */ - /* { dg-regexp {[^\n]*eof-2.h:4: error: unterminated argument list invoking macro "f"\n} } */ + /* { dg-regexp {[^\n]*eof-2.h:4:21: error: unterminated argument list invoking macro "f"\n} } */ token ) diff --git c/libcpp/lex.c w/libcpp/lex.c index 2fe77d162a2..fb222924c8c 100644 --- c/libcpp/lex.c +++ w/libcpp/lex.c @@ -2768,7 +2768,10 @@ _cpp_lex_direct (cpp_reader *pfile) if (!_cpp_get_fresh_line (pfile)) { result->type = CPP_EOF; - if (!pfile->state.in_directive) + /* Not a real EOF in a directive or arg parsing -- we refuse + to advance to the next file now, and will once we're out + of those modes. */ + if (!pfile->state.in_directive && !pfile->state.parsing_args) { /* Tell the compiler the line number of the EOF token. */ result->src_loc = pfile->line_table->highest_line; diff --git c/libcpp/macro.c w/libcpp/macro.c index 9cb3b10a9a0..0874028b211 100644 --- c/libcpp/macro.c +++ w/libcpp/macro.c @@ -1259,13 +1259,10 @@ collect_args (cpp_reader *pfile, const cpp_hashnode *node, if (token->type == CPP_EOF) { - /* We still need the CPP_EOF to end directives, to end - pre-expansion of a macro argument, and at the end of the main - file. We do not want it at the end of a -include'd (forced) - header file. */ - if (pfile->state.in_directive - || !pfile->line_table->depth - || pfile->context->prev) + /* Unless the EOF is marking the end of an argument, it's a fake + one from the end of a file that _cpp_clean_line will not have + advanced past. */ + if (token == &pfile->endarg) _cpp_backup_tokens (pfile, 1); cpp_error (pfile, CPP_DL_ERROR, "unterminated argument list invoking macro \"%s\"", @@ -1328,13 +1325,19 @@ funlike_invocation_p (cpp_reader *pfile, cpp_hashnode *node, pfile->state.parsing_args = 2; return collect_args (pfile, node, pragma_buff, num_args); } - - /* Back up. We may have skipped padding, in which case backing - up more than one token when expanding macros is in general - too difficult. We re-insert it in its own context. */ - _cpp_backup_tokens (pfile, 1); - if (padding) - _cpp_push_token_context (pfile, NULL, padding, 1); + + /* Back up. A CPP_EOF is either an EOF from an argument we're + expanding, or a fake one from lex_direct. We want to backup the + former, but not the latter. We may have skipped padding, in + which case backing up more than one token when expanding macros + is in general too difficult. We re-insert it in its own + context. */ + if (token->type != CPP_EOF || token == &pfile->endarg) + { + _cpp_backup_tokens (pfile, 1); + if (padding) + _cpp_push_token_context (pfile, NULL, padding, 1); + } return NULL; } @@ -2638,8 +2641,7 @@ _cpp_pop_context (cpp_reader *pfile) cpp_context *context = pfile->context; /* We should not be popping the base context. */ - if (context == &pfile->base_context) - abort (); + gcc_assert (context != &pfile->base_context); if (context->c.macro) {