From patchwork Tue Nov 17 12:46:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1401532 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=2620:52:3:1:0:246e:9693:128c; helo=sourceware.org; envelope-from=gcc-patches-bounces@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=G2c6flIM; dkim-atps=neutral Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Cb5Kh6RRPz9sRK for ; Tue, 17 Nov 2020 23:46:32 +1100 (AEDT) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 9EC6739724A2; Tue, 17 Nov 2020 12:46:30 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-qk1-x72e.google.com (mail-qk1-x72e.google.com [IPv6:2607:f8b0:4864:20::72e]) by sourceware.org (Postfix) with ESMTPS id 22475385EC4E for ; Tue, 17 Nov 2020 12:46:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 22475385EC4E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=acm.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nathanmsidwell@gmail.com Received: by mail-qk1-x72e.google.com with SMTP id n132so20162820qke.1 for ; Tue, 17 Nov 2020 04:46:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:to:from:subject:message-id:date:user-agent:mime-version :content-language; bh=biYKoEMN0h6DKLNxfLhTv7+3YmSp8R8y99M4ta+LM0Y=; b=G2c6flIMuHPqEujsiqZjKGB4AtWLvfmZIao0SwvFywj2A2IXnVl0qPW91j/O3exag8 FSJVJScH3iXcnFfBcE5TgVdkRZR4U3NLoMM1+5q82UDgxlOrJrrCKuhuIu0NyG0Yn4ql Y1/mCx+FDDPye1vG2r+wfVJN97tlStcfD5gINrtt97u2/fdO100A43Nyg9rKWBQrh2i6 Ent2l7GhD650YUjYU9dK5Uj0/YKXz9HL0eO72gVpEytvZOz641DfhfYl5NggzZOsmouf +P+8JawZ5EzI3VPbLS3sSn/bGjqzJV68yHKbf35G8MFnpc+b+LVvTiXoFwSjNEKp1De8 krpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:to:from:subject:message-id:date :user-agent:mime-version:content-language; bh=biYKoEMN0h6DKLNxfLhTv7+3YmSp8R8y99M4ta+LM0Y=; b=js0pkoPkNc9EaZ4gWB+BSW350lzACNjCR3qtZGY4CMRZKJ+aHdzTAbapsJ4TXIhXBG XTDZin7TYk9w9gp4GaAF2JEScbU5bvNA6YTjer1ivWTyNAILiDEzv1wJOoR1yZCbaC0G Tnhe8v5QUVdadoX3nrAf8bhSlR343gyFFcjoa3M2PY5gRKhNFgQHCW5vvpBjbcthQYyT 7jEq/saWcY/+dJklC4IBAMLqu2MTlzVMUeCJ5UvigrSI29+JEBe6KovpArXdrncJM3Fu eo0CRjh+mGaoHF2uQu8zYEOdSNrTOEK+/ww+rltJ3Iy2YaLqsR9/Pt1BENsJ6CRrnn/O nK6w== X-Gm-Message-State: AOAM533PTf155CUL86YlSJzSHFSeUb7lyQ1QoTLblHsQuloTvoxQuS+6 WR6xTjlv5vuvNSesmXK0PXo= X-Google-Smtp-Source: ABdhPJwy2FrSvL0BEIfAVy+Ble8eucTy4cMF+FRtTY3c3/CGZeDt7OmuGhEutqFV916sMAEFVz1Drg== X-Received: by 2002:a37:6cd:: with SMTP id 196mr19263942qkg.228.1605617187548; Tue, 17 Nov 2020 04:46:27 -0800 (PST) Received: from ?IPv6:2620:10d:c0a8:1102:81bf:d41:8bba:9858? ([2620:10d:c091:480::1:113e]) by smtp.googlemail.com with ESMTPSA id b64sm2335460qkg.19.2020.11.17.04.46.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Nov 2020 04:46:26 -0800 (PST) To: GCC Patches From: Nathan Sidwell Subject: c-family: token streamer Message-ID: Date: Tue, 17 Nov 2020 07:46:21 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 Content-Language: en-US X-Spam-Status: No, score=-11.3 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces@gcc.gnu.org Sender: "Gcc-patches" This is broken out of modules patch 01-langhooks.diff, I realized that this part is independent, and removes some duplicated code -- migrated to the token_streamer class. gcc/c-family/ * c-ppoutput.c (scan_translation_unit): Use token_streamer, remove code duplicating that functionality. pushing to trunk diff --git i/gcc/c-family/c-ppoutput.c w/gcc/c-family/c-ppoutput.c index 44c6f30e06b..517de15d97c 100644 --- i/gcc/c-family/c-ppoutput.c +++ w/gcc/c-family/c-ppoutput.c @@ -304,120 +304,18 @@ token_streamer::stream (cpp_reader *pfile, const cpp_token *token, static void scan_translation_unit (cpp_reader *pfile) { - bool avoid_paste = false; - bool do_line_adjustments - = cpp_get_options (parse_in)->lang != CLK_ASM - && !flag_no_line_commands; - bool in_pragma = false; - bool line_marker_emitted = false; + token_streamer streamer (pfile); print.source = NULL; for (;;) { - location_t loc; - const cpp_token *token = cpp_get_token_with_location (pfile, &loc); - - if (token->type == CPP_PADDING) - { - avoid_paste = true; - if (print.source == NULL - || (!(print.source->flags & PREV_WHITE) - && token->val.source == NULL)) - print.source = token->val.source; - continue; - } + location_t spelling_loc; + const cpp_token *token + = cpp_get_token_with_location (pfile, &spelling_loc); + streamer.stream (pfile, token, spelling_loc); if (token->type == CPP_EOF) break; - - /* Subtle logic to output a space if and only if necessary. */ - if (avoid_paste) - { - int src_line = LOCATION_LINE (loc); - - if (print.source == NULL) - print.source = token; - - if (src_line != print.src_line - && do_line_adjustments - && !in_pragma) - { - line_marker_emitted = do_line_change (pfile, token, loc, false); - putc (' ', print.outf); - print.printed = true; - } - else if (print.source->flags & PREV_WHITE - || (print.prev - && cpp_avoid_paste (pfile, print.prev, token)) - || (print.prev == NULL && token->type == CPP_HASH)) - { - putc (' ', print.outf); - print.printed = true; - } - } - else if (token->flags & PREV_WHITE) - { - int src_line = LOCATION_LINE (loc); - - if (src_line != print.src_line - && do_line_adjustments - && !in_pragma) - line_marker_emitted = do_line_change (pfile, token, loc, false); - putc (' ', print.outf); - print.printed = true; - } - - avoid_paste = false; - print.source = NULL; - print.prev = token; - if (token->type == CPP_PRAGMA) - { - const char *space; - const char *name; - - line_marker_emitted = maybe_print_line (token->src_loc); - fputs ("#pragma ", print.outf); - c_pp_lookup_pragma (token->val.pragma, &space, &name); - if (space) - fprintf (print.outf, "%s %s", space, name); - else - fprintf (print.outf, "%s", name); - print.printed = true; - in_pragma = true; - } - else if (token->type == CPP_PRAGMA_EOL) - { - maybe_print_line (token->src_loc); - in_pragma = false; - } - else - { - if (cpp_get_options (parse_in)->debug) - linemap_dump_location (line_table, token->src_loc, print.outf); - - if (do_line_adjustments - && !in_pragma - && !line_marker_emitted - && print.prev_was_system_token != !!in_system_header_at (loc) - && !is_location_from_builtin_token (loc)) - /* The system-ness of this token is different from the one - of the previous token. Let's emit a line change to - mark the new system-ness before we emit the token. */ - { - do_line_change (pfile, token, loc, false); - print.prev_was_system_token = !!in_system_header_at (loc); - } - cpp_output_token (token, print.outf); - line_marker_emitted = false; - print.printed = true; - } - - /* CPP_COMMENT tokens and raw-string literal tokens can - have embedded new-line characters. Rather than enumerating - all the possible token types just check if token uses - val.str union member. */ - if (cpp_token_val_index (token) == CPP_TOKEN_FLD_STR) - account_for_newlines (token->val.str.text, token->val.str.len); } }