From patchwork Tue Jul 5 15:10:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nathan Sidwell X-Patchwork-Id: 1652559 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.a=rsa-sha256 header.s=default header.b=W1todIEk; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=sourceware.org; envelope-from=gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LcmMp5mNpz9ryY for ; Wed, 6 Jul 2022 01:11:02 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id CD57338582B4 for ; Tue, 5 Jul 2022 15:10:59 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org CD57338582B4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gcc.gnu.org; s=default; t=1657033859; bh=5x4d0Xm/FyTEGHT3BCZya2PGB5+7lHh0gRHn0hkSqRc=; h=Date:To:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=W1todIEkn1QAbqH7zF4VWNZk5s387WQF33ZSSEApPodaJ+8jBIkTcrDHPghF/U71P C7Go7QJN8zAwZReLMammuquexSFAy/b4vRzt29Kru3UWxDgPocj0xbg2KrbXyFxEvI C7xnopzik67cT7qE1TsCAf4wt5K8L4Vy+Ysvpaxc= X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by sourceware.org (Postfix) with ESMTPS id D987A3858C2F for ; Tue, 5 Jul 2022 15:10:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D987A3858C2F Received: by mail-pl1-x631.google.com with SMTP id b2so11207866plx.7 for ; Tue, 05 Jul 2022 08:10:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:message-id:date:mime-version:user-agent :content-language:to:from:subject; bh=5x4d0Xm/FyTEGHT3BCZya2PGB5+7lHh0gRHn0hkSqRc=; b=vyO1awmkjnZZFHZL3A0ZgdDJekdke8evBv5UGst7ic6KjBpElOMXB+TmSg/g6Js9ES +TKwShJd6jMUYbBk60PAVdB5jSW+KdQxbA82iL4+5w1BpY1lEsowZOS8U5oIap7pWw1K gpkNJPxIQR8EzMeEslAklt46tIbPhS6xCMMPxgMOlM0lMdiwGow4LzwR3vKBNAqSwBB/ HWw2SPrR7oNWIC/Y7KpGHvNXGNKLXL0JOyK9V0Zo3yt83tcNRNJwL4YzmuRaiSTXYsSI kUIWHP6d8PWqMKDnM0I2B+3aDbFIhWPILtn4r29UrrUulebhC/GSjyZ19pHPwrkOYV7O PKJg== X-Gm-Message-State: AJIora9Yu6D3aOuboMKLic5z6oKvlS3ZwH1O9naOG/bWV4oH4RRarb9v stAQOUCEhoqvWIUryFjoLk8kqVp8jRc= X-Google-Smtp-Source: AGRyM1tE+zeyv4IuAlEvTxcvLq7UIJUKpaYOw0dAx8lr3SvRrlCw6etAFMzQ3kiTmaf7VdVJnqdG6g== X-Received: by 2002:a17:902:e5cb:b0:16a:7321:c3a1 with SMTP id u11-20020a170902e5cb00b0016a7321c3a1mr41771104plf.62.1657033833403; Tue, 05 Jul 2022 08:10:33 -0700 (PDT) Received: from ?IPV6:2620:10d:c085:21c8::1488? ([2620:10d:c090:400::5:f3c8]) by smtp.googlemail.com with ESMTPSA id d11-20020a170903230b00b0016a5384071bsm23724176plh.1.2022.07.05.08.10.31 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Jul 2022 08:10:32 -0700 (PDT) Message-ID: Date: Tue, 5 Jul 2022 11:10:27 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Content-Language: en-US To: GCC Patches Subject: c++: Prune ordinary locations X-Spam-Status: No, score=-3038.9 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, GIT_PATCH_0, HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Nathan Sidwell via Gcc-patches From: Nathan Sidwell Reply-To: Nathan Sidwell Errors-To: gcc-patches-bounces+incoming=patchwork.ozlabs.org@gcc.gnu.org Sender: "Gcc-patches" Like macro locations, we only need to emit ordinary location information for locations emitted into the CMI. This adds a hash table noting which ordinary lines are needed. These are then sorted and (sufficiently) adjacent lines are coalesced to a single map. There is a tradeoff here, allowing greater separation reduces the number of line maps, but increases the number of locations. It appears allowing 2 or 3 intervening lines is the sweet spot, and this patch chooses 2. Compiling a hello-world #includeing in it's GMF gives a reduction in number of locations of 5 fold, but an increase in number of maps about 4 fold. Examining one of the xtreme-header tests we halve the number of locations and increase the number of maps by 9 fold. Module interfaces that emit no entities (or macros, if a header-unit), will now have no location tables. nathan From 47794da8d8ea61ea8f6a0e21d3c1731a56d0cff3 Mon Sep 17 00:00:00 2001 From: Nathan Sidwell Date: Fri, 24 Jun 2022 05:57:42 -0700 Subject: [PATCH] c++: Prune ordinary locations Like macro locations, we only need to emit ordinary location information for locations emitted into the CMI. This adds a hash table noting which ordinary lines are needed. These are then sorted and (sufficiently) adjacent lines are coalesced to a single map. There is a tradeoff here, allowing greater separation reduces the number of line maps, but increases the number of locations. It appears allowing 2 or 3 intervening lines is the sweet spot, and this patch chooses 2. Compiling a hello-world #includeing in it's GMF gives a reduction in number of locations of 5 fold, but an increase in number of maps about 4 fold. Examining one of the xtreme-header tests we halve the number of locations and increase the number of maps by 9 fold. Module interfaces that emit no entities (or macros, if a header-unit), will now have no location tables. gcc/cp/ * module.cc (struct ord_loc_info, ord_loc_traits): New. (ord_loc_tabke, ord_loc_remap): New globals. (struct location_map_info): Delete. (struct module_state_config): Rename ordinary_loc_align to loc_range_bits. (module_for_ordinary_loc): Adjust. (module_state::note_location): Note ordinary locations, return bool. (module_state::write_location): Adjust ordinary location streaming. (module_state::read_location): Likewise. (module_state::write_init_maps): Allocate ord_loc_table. (module_state::write_prepare_maps): Reimplement ordinary map preparation. (module_state::read_prepare_maps): Adjust. (module_state::write_ordinary_maps): Reimplement. (module_state::write_macro_maps): Adjust. (module_state::read_ordinary_maps): Reimplement. (module_state::write_macros): Adjust. (module_state::write_config): Adjust. (module_state::read_config): Adjust. (module_state::write_begin): Adjust. (module_state::read_initial): Adjust. gcc/testsuite/ * g++.dg/modules/loc-prune-1.C: Adjust. * g++.dg/modules/loc-prune-4.C: New. * g++.dg/modules/pr98718_a.C: Adjust. * g++.dg/modules/pr98718_b.C: Adjust. * g++.dg/modules/pr99072.H: Adjust. --- gcc/cp/module.cc | 625 ++++++++++++--------- gcc/testsuite/g++.dg/modules/loc-prune-1.C | 2 +- gcc/testsuite/g++.dg/modules/loc-prune-4.C | 22 + gcc/testsuite/g++.dg/modules/pr98718_a.C | 2 +- gcc/testsuite/g++.dg/modules/pr98718_b.C | 2 +- gcc/testsuite/g++.dg/modules/pr99072.H | 4 +- 6 files changed, 373 insertions(+), 284 deletions(-) create mode 100644 gcc/testsuite/g++.dg/modules/loc-prune-4.C diff --git a/gcc/cp/module.cc b/gcc/cp/module.cc index 238a5eb74d2..f27f4d091e5 100644 --- a/gcc/cp/module.cc +++ b/gcc/cp/module.cc @@ -3240,6 +3240,66 @@ public: static loc_spans spans; +/* Information about ordinary locations we stream out. */ +struct ord_loc_info +{ + const line_map_ordinary *src; // line map we're based on + unsigned offset; // offset to this line + unsigned span; // number of locs we span + unsigned remap; // serialization + + static int compare (const void *a_, const void *b_) + { + auto *a = static_cast (a_); + auto *b = static_cast (b_); + + if (a->src != b->src) + return a->src < b->src ? -1 : +1; + + // Ensure no overlap + gcc_checking_assert (a->offset + a->span <= b->offset + || b->offset + b->span <= a->offset); + + gcc_checking_assert (a->offset != b->offset); + return a->offset < b->offset ? -1 : +1; + } +}; +struct ord_loc_traits +{ + typedef ord_loc_info value_type; + typedef value_type compare_type; + + static const bool empty_zero_p = false; + + static hashval_t hash (const value_type &v) + { + auto h = pointer_hash::hash (v.src); + return iterative_hash_hashval_t (v.offset, h); + } + static bool equal (const value_type &v, const compare_type p) + { + return v.src == p.src && v.offset == p.offset; + } + + static void mark_empty (value_type &v) + { + v.src = nullptr; + } + static bool is_empty (value_type &v) + { + return !v.src; + } + + static bool is_deleted (value_type &) { return false; } + static void mark_deleted (value_type &) { gcc_unreachable (); } + + static void remove (value_type &) {} +}; +/* Table keyed by ord_loc_info, used for noting. */ +static hash_table *ord_loc_table; +/* Sorted vector, used for writing. */ +static vec *ord_loc_remap; + /* Information about macro locations we stream out. */ struct macro_loc_info { @@ -3401,15 +3461,7 @@ void slurping::release_macros () elf_in::release (from, macro_tbl); } -/* Information about location maps used during writing. */ - -struct location_map_info { - range_t num_maps; - - unsigned max_range; -}; - -/* Flage for extensions that end up being streamed. */ +/* Flags for extensions that end up being streamed. */ enum streamed_extensions { SE_OPENMP = 1 << 0, @@ -3652,14 +3704,13 @@ class GTY((chain_next ("%h.parent"), for_user)) module_state { private: void write_init_maps (); - location_map_info write_prepare_maps (module_state_config *); + range_t write_prepare_maps (module_state_config *, bool); bool read_prepare_maps (const module_state_config *); - void write_ordinary_maps (elf_out *to, location_map_info &, - module_state_config *, bool, unsigned *crc_ptr); - bool read_ordinary_maps (); - void write_macro_maps (elf_out *to, location_map_info &, - module_state_config *, unsigned *crc_ptr); + void write_ordinary_maps (elf_out *to, range_t &, + bool, unsigned *crc_ptr); + bool read_ordinary_maps (unsigned, unsigned); + void write_macro_maps (elf_out *to, range_t &, unsigned *crc_ptr); bool read_macro_maps (unsigned); private: @@ -3678,7 +3729,7 @@ class GTY((chain_next ("%h.parent"), for_user)) module_state { static cpp_macro *deferred_macro (cpp_reader *, location_t, cpp_hashnode *); public: - static void note_location (location_t); + static bool note_location (location_t); static void write_location (bytes_out &, location_t); location_t read_location (bytes_in &) const; @@ -14444,14 +14495,14 @@ struct module_state_config { unsigned num_entities; unsigned ordinary_locs; unsigned macro_locs; - unsigned ordinary_loc_align; + unsigned loc_range_bits; unsigned active_init; public: module_state_config () :dialect_str (get_dialect ()), num_imports (0), num_partitions (0), num_entities (0), - ordinary_locs (0), macro_locs (0), ordinary_loc_align (0), + ordinary_locs (0), macro_locs (0), loc_range_bits (0), active_init (0) { } @@ -15562,7 +15613,7 @@ module_for_ordinary_loc (location_t loc) module_state *probe = (*ool)[pos + half]; if (loc < probe->ordinary_locs.first) len = half; - else if (loc < probe->ordinary_locs.second) + else if (loc < probe->ordinary_locs.first + probe->ordinary_locs.second) return probe; else { @@ -15589,7 +15640,7 @@ module_for_macro_loc (location_t loc) pos += half + 1; len = len - (half + 1); } - else if (loc >= (probe->macro_locs.first + probe->macro_locs.second)) + else if (loc >= probe->macro_locs.first + probe->macro_locs.second) len = half; else return probe; @@ -15614,10 +15665,11 @@ module_state::imported_from () const /* Note that LOC will need writing. This allows us to prune locations that are not needed. */ -void +bool module_state::note_location (location_t loc) { - if (!macro_loc_table) + bool added = false; + if (!macro_loc_table && !ord_loc_table) ; else if (loc < RESERVED_LOCATION_COUNT) ; @@ -15654,17 +15706,33 @@ module_state::note_location (location_t loc) tloc = mac_map->macro_locations[ix]; note_location (tloc); } + added = true; } } } else if (IS_ORDINARY_LOC (loc)) { - /* This is where we should note we use this location. See comment - about write_ordinary_maps. */ + if (spans.ordinary (loc)) + { + const line_map *map = linemap_lookup (line_table, loc); + const line_map_ordinary *ord_map = linemap_check_ordinary (map); + ord_loc_info lkup; + lkup.src = ord_map; + lkup.span = 1 << ord_map->m_column_and_range_bits; + lkup.offset = (loc - MAP_START_LOCATION (ord_map)) & ~(lkup.span - 1); + lkup.remap = 0; + ord_loc_info *slot = (ord_loc_table->find_slot_with_hash + (lkup, ord_loc_traits::hash (lkup), INSERT)); + if (!slot->src) + { + *slot = lkup; + added = true; + } + } } else gcc_unreachable (); - return; + return added; } /* If we're not streaming, record that we need location LOC. @@ -15745,16 +15813,37 @@ module_state::write_location (bytes_out &sec, location_t loc) } else if (IS_ORDINARY_LOC (loc)) { - if (const loc_spans::span *span = spans.ordinary (loc)) + const ord_loc_info *info = nullptr; + unsigned offset = 0; + if (unsigned hwm = ord_loc_remap->length ()) { - unsigned off = loc; + info = ord_loc_remap->begin (); + while (hwm != 1) + { + unsigned mid = hwm / 2; + if (MAP_START_LOCATION (info[mid].src) + info[mid].offset <= loc) + { + info += mid; + hwm -= mid; + } + else + hwm = mid; + } + offset = loc - MAP_START_LOCATION (info->src) - info->offset; + if (offset > info->span) + info = nullptr; + } - off += span->ordinary_delta; + gcc_checking_assert (bool (info) == bool (spans.ordinary (loc))); + + if (info) + { + offset += info->remap; sec.u (LK_ORDINARY); - sec.u (off); + sec.u (offset); dump (dumper::LOCATION) - && dump ("Ordinary location %u output %u", loc, off); + && dump ("Ordinary location %u output %u", loc, offset); } else if (const module_state *import = module_for_ordinary_loc (loc)) { @@ -15809,7 +15898,7 @@ module_state::read_location (bytes_in &sec) const { unsigned off = sec.u (); - if (macro_locs.first) + if (macro_locs.second) { if (off < macro_locs.second) locus = off + macro_locs.first; @@ -15828,15 +15917,10 @@ module_state::read_location (bytes_in &sec) const unsigned off = sec.u (); if (ordinary_locs.second) { - location_t adjusted = off; - - adjusted += slurp->loc_deltas.first; - if (adjusted >= ordinary_locs.second) + if (off < ordinary_locs.second) + locus = off + ordinary_locs.first; + else sec.set_overrun (); - else if (adjusted >= ordinary_locs.first) - locus = adjusted; - else if (adjusted < spans.main_start ()) - locus = off; } else locus = loc; @@ -15870,7 +15954,7 @@ module_state::read_location (bytes_in &sec) const { if (kind == LK_IMPORT_MACRO) { - if (!import->macro_locs.first) + if (!import->macro_locs.second) locus = import->loc; else if (off < import->macro_locs.second) locus = off + import->macro_locs.first; @@ -15881,8 +15965,7 @@ module_state::read_location (bytes_in &sec) const { if (!import->ordinary_locs.second) locus = import->loc; - else if (off < (import->ordinary_locs.second - - import->ordinary_locs.first)) + else if (off < import->ordinary_locs.second) locus = import->ordinary_locs.first + off; else sec.set_overrun (); @@ -15895,26 +15978,21 @@ module_state::read_location (bytes_in &sec) const return locus; } -/* Prepare the span adjustments. */ - -// FIXME:QOI I do not prune the unreachable locations. Modules with -// textually-large GMFs could well cause us to run out of locations. -// Regular single-file modules could also be affected. We should -// determine which locations we need to represent, so that we do not -// grab more locations than necessary. An example is in -// write_macro_maps where we work around macro expansions that are not -// covering any locations -- the macro expands to nothing. Perhaps we -// should decompose locations so that we can have a more graceful -// degradation upon running out? +/* Allocate hash tables to record needed locations. */ void module_state::write_init_maps () { macro_loc_table = new hash_table (EXPERIMENT (1, 400)); + ord_loc_table = new hash_table (EXPERIMENT (1, 400)); } -location_map_info -module_state::write_prepare_maps (module_state_config *cfg) +/* Prepare the span adjustments. We prune unneeded locations -- at + this point every needed location must have been seen by + note_location. */ + +range_t +module_state::write_prepare_maps (module_state_config *cfg, bool has_partitions) { dump () && dump ("Preparing locations"); dump.indent (); @@ -15925,82 +16003,98 @@ module_state::write_prepare_maps (module_state_config *cfg) spans[loc_spans::SPAN_RESERVED].macro.first, spans[loc_spans::SPAN_RESERVED].macro.second); - location_map_info info; + range_t info {0, 0}; - info.num_maps.first = info.num_maps.second = 0; + // Sort the noted lines. + vec_alloc (ord_loc_remap, ord_loc_table->size ()); + for (auto iter = ord_loc_table->begin (), end = ord_loc_table->end (); + iter != end; ++iter) + ord_loc_remap->quick_push (*iter); + ord_loc_remap->qsort (&ord_loc_info::compare); - /* Figure the alignment of ordinary location spans. */ - unsigned max_range = 0; - for (unsigned ix = loc_spans::SPAN_FIRST; ix != spans.length (); ix++) + // Note included-from maps. + bool added = false; + const line_map_ordinary *current = nullptr; + for (auto iter = ord_loc_remap->begin (), end = ord_loc_remap->end (); + iter != end; ++iter) + if (iter->src != current) + { + current = iter->src; + for (auto probe = current; + auto from = linemap_included_from (probe); + probe = linemap_check_ordinary (linemap_lookup (line_table, from))) + { + if (has_partitions) + { + // Partition locations need to elide their module map + // entry. + probe + = linemap_check_ordinary (linemap_lookup (line_table, from)); + if (MAP_MODULE_P (probe)) + from = linemap_included_from (probe); + } + + if (!note_location (from)) + break; + added = true; + } + } + if (added) { - loc_spans::span &span = spans[ix]; + // Reconstruct the line array as we added items to the hash table. + vec_free (ord_loc_remap); + vec_alloc (ord_loc_remap, ord_loc_table->size ()); + for (auto iter = ord_loc_table->begin (), end = ord_loc_table->end (); + iter != end; ++iter) + ord_loc_remap->quick_push (*iter); + ord_loc_remap->qsort (&ord_loc_info::compare); + } + delete ord_loc_table; + ord_loc_table = nullptr; - if (span.ordinary.first != span.ordinary.second) + // Merge (sufficiently) adjacent spans, and calculate remapping. + constexpr unsigned adjacency = 2; // Allow 2 missing lines. + auto begin = ord_loc_remap->begin (), end = ord_loc_remap->end (); + auto dst = begin; + unsigned offset = 0, range_bits = 0; + ord_loc_info *base = nullptr; + for (auto iter = begin; iter != end; ++iter) + { + if (base && iter->src == base->src) { - line_map_ordinary const *omap - = linemap_check_ordinary (linemap_lookup (line_table, - span.ordinary.first)); - - /* We should exactly match up. */ - gcc_checking_assert (MAP_START_LOCATION (omap) == span.ordinary.first); - - line_map_ordinary const *fmap = omap; - for (; MAP_START_LOCATION (omap) < span.ordinary.second; omap++) + if (base->offset + base->span + + ((adjacency << base->src->m_column_and_range_bits) + // If there are few c&r bits, allow further separation. + | (adjacency << 4)) + >= iter->offset) { - /* We should never find a module linemap in an interval. */ - gcc_checking_assert (!MAP_MODULE_P (omap)); - - if (max_range < omap->m_range_bits) - max_range = omap->m_range_bits; + // Merge. + offset -= base->span; + base->span = iter->offset + iter->span - base->offset; + offset += base->span; + continue; } - - info.num_maps.first += omap - fmap; } + else if (range_bits < iter->src->m_range_bits) + range_bits = iter->src->m_range_bits; + + offset += ((1u << iter->src->m_range_bits) - 1); + offset &= ~((1u << iter->src->m_range_bits) - 1); + iter->remap = offset; + offset += iter->span; + base = dst; + *dst++ = *iter; } + ord_loc_remap->truncate (dst - begin); - /* Adjust the maps. Ordinary ones ascend, and we must maintain - alignment. Macro ones descend, but are unaligned. */ - location_t ord_off = spans[loc_spans::SPAN_FIRST].ordinary.first; - location_t range_mask = (1u << max_range) - 1; - - dump () && dump ("Ordinary maps range bits:%u, preserve:%x, zero:%u", - max_range, ord_off & range_mask, ord_off & ~range_mask); - - for (unsigned ix = loc_spans::SPAN_FIRST; ix != spans.length (); ix++) - { - loc_spans::span &span = spans[ix]; - - line_map_ordinary const *omap - = linemap_check_ordinary (linemap_lookup (line_table, - span.ordinary.first)); - location_t base = MAP_START_LOCATION (omap); - - /* Preserve the low MAX_RANGE bits of base by incrementing ORD_OFF. */ - unsigned low_bits = base & range_mask; - if ((ord_off & range_mask) > low_bits) - low_bits += range_mask + 1; - ord_off = (ord_off & ~range_mask) + low_bits; - span.ordinary_delta = ord_off - base; - - for (; MAP_START_LOCATION (omap) < span.ordinary.second; omap++) - { - location_t start_loc = MAP_START_LOCATION (omap); - unsigned to = start_loc + span.ordinary_delta; - location_t end_loc = MAP_START_LOCATION (omap + 1); - - dump () && dump ("Ordinary span:%u [%u,%u):%u->%d(%u)", - ix, start_loc, - end_loc, end_loc - start_loc, - span.ordinary_delta, to); - - /* There should be no change in the low order bits. */ - gcc_checking_assert (((start_loc ^ to) & range_mask) == 0); - } - - /* The ending serialized value. */ - ord_off = span.ordinary.second + span.ordinary_delta; - } + info.first = ord_loc_remap->length (); + cfg->ordinary_locs = offset; + cfg->loc_range_bits = range_bits; + dump () && dump ("Ordinary maps:%u locs:%u range_bits:%u", + info.first, cfg->ordinary_locs, + cfg->loc_range_bits); + // Remap the macro locations. vec_alloc (macro_loc_remap, macro_loc_table->size ()); for (auto iter = macro_loc_table->begin (), end = macro_loc_table->end (); iter != end; ++iter) @@ -16009,7 +16103,7 @@ module_state::write_prepare_maps (module_state_config *cfg) macro_loc_table = nullptr; macro_loc_remap->qsort (¯o_loc_info::compare); - unsigned offset = 0; + offset = 0; for (auto iter = macro_loc_remap->begin (), end = macro_loc_remap->end (); iter != end; ++iter) { @@ -16017,16 +16111,15 @@ module_state::write_prepare_maps (module_state_config *cfg) iter->remap = offset; offset += mac->n_tokens; } - info.num_maps.second = macro_loc_remap->length (); + info.second = macro_loc_remap->length (); cfg->macro_locs = offset; - dump () && dump ("Ordinary:%u maps hwm:%u macro:%u maps %u locs", - info.num_maps.first, ord_off, - info.num_maps.second, cfg->macro_locs); + dump () && dump ("Macro maps:%u locs:%u", info.second, cfg->macro_locs); dump.outdent (); - info.max_range = max_range; + // If we have no ordinary locs, we must also have no macro locs. + gcc_checking_assert (cfg->ordinary_locs || !cfg->macro_locs); return info; } @@ -16035,8 +16128,6 @@ bool module_state::read_prepare_maps (const module_state_config *cfg) { location_t ordinary = line_table->highest_location + 1; - ordinary = ((ordinary + (1u << cfg->ordinary_loc_align)) - & ~((1u << cfg->ordinary_loc_align) - 1)); ordinary += cfg->ordinary_locs; location_t macro = LINEMAPS_MACRO_LOWEST_LOCATION (line_table); @@ -16061,13 +16152,12 @@ module_state::read_prepare_maps (const module_state_config *cfg) return false; } -/* Write the location maps. This also determines the shifts for the - location spans. */ +/* Write & read the location maps. Not called if there are no + locations. */ void -module_state::write_ordinary_maps (elf_out *to, location_map_info &info, - module_state_config *cfg, bool has_partitions, - unsigned *crc_p) +module_state::write_ordinary_maps (elf_out *to, range_t &info, + bool has_partitions, unsigned *crc_p) { dump () && dump ("Writing ordinary location maps"); dump.indent (); @@ -16076,45 +16166,36 @@ module_state::write_ordinary_maps (elf_out *to, location_map_info &info, filenames.create (20); /* Determine the unique filenames. */ - // FIXME:QOI We should find the set of filenames when working out - // which locations we actually need. See write_prepare_maps. - for (unsigned ix = loc_spans::SPAN_FIRST; ix != spans.length (); ix++) - { - loc_spans::span &span = spans[ix]; - line_map_ordinary const *omap - = linemap_check_ordinary (linemap_lookup (line_table, - span.ordinary.first)); + const line_map_ordinary *current = nullptr; + for (auto iter = ord_loc_remap->begin (), end = ord_loc_remap->end (); + iter != end; ++iter) + if (iter->src != current) + { + current = iter->src; + const char *fname = ORDINARY_MAP_FILE_NAME (iter->src); - /* We should exactly match up. */ - gcc_checking_assert (MAP_START_LOCATION (omap) == span.ordinary.first); + /* We should never find a module linemap in an interval. */ + gcc_checking_assert (!MAP_MODULE_P (iter->src)); - for (; MAP_START_LOCATION (omap) < span.ordinary.second; omap++) - { - const char *fname = ORDINARY_MAP_FILE_NAME (omap); - - /* We should never find a module linemap in an interval. */ - gcc_checking_assert (!MAP_MODULE_P (omap)); - - /* We expect very few filenames, so just an array. - (Not true when headers are still in play :() */ - for (unsigned jx = filenames.length (); jx--;) - { - const char *name = filenames[jx]; - if (0 == strcmp (name, fname)) - { - /* Reset the linemap's name, because for things like - preprocessed input we could have multiple - instances of the same name, and we'd rather not - percolate that. */ - const_cast (omap)->to_file = name; - fname = NULL; - break; - } - } - if (fname) - filenames.safe_push (fname); - } - } + /* We expect very few filenames, so just an array. + (Not true when headers are still in play :() */ + for (unsigned jx = filenames.length (); jx--;) + { + const char *name = filenames[jx]; + if (0 == strcmp (name, fname)) + { + /* Reset the linemap's name, because for things like + preprocessed input we could have multiple instances + of the same name, and we'd rather not percolate + that. */ + const_cast (iter->src)->to_file = name; + fname = NULL; + break; + } + } + if (fname) + filenames.safe_push (fname); + } bytes_out sec (to); sec.begin (); @@ -16130,56 +16211,45 @@ module_state::write_ordinary_maps (elf_out *to, location_map_info &info, sec.str (fname); } - location_t offset = spans[loc_spans::SPAN_FIRST].ordinary.first; - location_t range_mask = (1u << info.max_range) - 1; - - dump () && dump ("Ordinary maps:%u, range bits:%u, preserve:%x, zero:%u", - info.num_maps.first, info.max_range, offset & range_mask, - offset & ~range_mask); - sec.u (info.num_maps.first); /* Num maps. */ - sec.u (info.max_range); /* Maximum range bits */ - sec.u (offset & range_mask); /* Bits to preserve. */ - sec.u (offset & ~range_mask); - - for (unsigned ix = loc_spans::SPAN_FIRST; ix != spans.length (); ix++) + sec.u (info.first); /* Num maps. */ + const ord_loc_info *base = nullptr; + for (auto iter = ord_loc_remap->begin (), end = ord_loc_remap->end (); + iter != end; ++iter) { - loc_spans::span &span = spans[ix]; - line_map_ordinary const *omap - = linemap_check_ordinary (linemap_lookup (line_table, - span.ordinary.first)); - for (; MAP_START_LOCATION (omap) < span.ordinary.second; omap++) + dump (dumper::LOCATION) + && dump ("Span:%u ordinary [%u+%u,+%u)->[%u,+%u)", + iter - ord_loc_remap->begin (), + MAP_START_LOCATION (iter->src), iter->offset, iter->span, + iter->remap, iter->span); + + if (!base || iter->src != base->src) + base = iter; + sec.u (iter->offset - base->offset); + if (base == iter) { - location_t start_loc = MAP_START_LOCATION (omap); - unsigned to = start_loc + span.ordinary_delta; + sec.u (iter->src->sysp); + sec.u (iter->src->m_range_bits); + sec.u (iter->src->m_column_and_range_bits - iter->src->m_range_bits); - dump (dumper::LOCATION) - && dump ("Span:%u ordinary [%u,%u)->%u", ix, start_loc, - MAP_START_LOCATION (omap + 1), to); - - /* There should be no change in the low order bits. */ - gcc_checking_assert (((start_loc ^ to) & range_mask) == 0); - sec.u (to); - - /* Making accessors just for here, seems excessive. */ - sec.u (omap->reason); - sec.u (omap->sysp); - sec.u (omap->m_range_bits); - sec.u (omap->m_column_and_range_bits - omap->m_range_bits); - - const char *fname = ORDINARY_MAP_FILE_NAME (omap); + const char *fname = ORDINARY_MAP_FILE_NAME (iter->src); for (unsigned ix = 0; ix != filenames.length (); ix++) if (filenames[ix] == fname) { sec.u (ix); break; } - sec.u (ORDINARY_MAP_STARTING_LINE_NUMBER (omap)); - + unsigned line = ORDINARY_MAP_STARTING_LINE_NUMBER (iter->src); + line += iter->offset >> iter->src->m_column_and_range_bits; + sec.u (line); + } + sec.u (iter->remap); + if (base == iter) + { /* Write the included from location, which means reading it while reading in the ordinary maps. So we'd better not be getting ahead of ourselves. */ - location_t from = linemap_included_from (omap); - gcc_checking_assert (from < MAP_START_LOCATION (omap)); + location_t from = linemap_included_from (iter->src); + gcc_checking_assert (from < MAP_START_LOCATION (iter->src)); if (from != UNKNOWN_LOCATION && has_partitions) { /* A partition's span will have a from pointing at a @@ -16191,15 +16261,7 @@ module_state::write_ordinary_maps (elf_out *to, location_map_info &info, } write_location (sec, from); } - /* The ending serialized value. */ - offset = MAP_START_LOCATION (omap) + span.ordinary_delta; } - dump () && dump ("Ordinary location hwm:%u", offset); - sec.u (offset); - - // Record number of locations and alignment. - cfg->ordinary_loc_align = info.max_range; - cfg->ordinary_locs = offset; filenames.release (); @@ -16208,8 +16270,7 @@ module_state::write_ordinary_maps (elf_out *to, location_map_info &info, } void -module_state::write_macro_maps (elf_out *to, location_map_info &info, - module_state_config *, unsigned *crc_p) +module_state::write_macro_maps (elf_out *to, range_t &info, unsigned *crc_p) { dump () && dump ("Writing macro location maps"); dump.indent (); @@ -16217,8 +16278,8 @@ module_state::write_macro_maps (elf_out *to, location_map_info &info, bytes_out sec (to); sec.begin (); - dump () && dump ("Macro maps:%u", info.num_maps.second); - sec.u (info.num_maps.second); + dump () && dump ("Macro maps:%u", info.second); + sec.u (info.second); unsigned macro_num = 0; for (auto iter = macro_loc_remap->end (), begin = macro_loc_remap->begin (); @@ -16258,14 +16319,14 @@ module_state::write_macro_maps (elf_out *to, location_map_info &info, iter->remap); macro_num++; } - gcc_assert (macro_num == info.num_maps.second); + gcc_assert (macro_num == info.second); sec.end (to, to->name (MOD_SNAME_PFX ".mlm"), crc_p); dump.outdent (); } bool -module_state::read_ordinary_maps () +module_state::read_ordinary_maps (unsigned num_ord_locs, unsigned range_bits) { bytes_in sec; @@ -16291,70 +16352,62 @@ module_state::read_ordinary_maps () filenames.quick_push (fname); } - unsigned num_ordinary = sec.u (); - unsigned max_range = sec.u (); - unsigned low_bits = sec.u (); - location_t zero = sec.u (); - location_t range_mask = (1u << max_range) - 1; - - dump () && dump ("Ordinary maps:%u, range bits:%u, preserve:%x, zero:%u", - num_ordinary, max_range, low_bits, zero); + unsigned num_ordinary = sec.u (); + dump () && dump ("Ordinary maps:%u, range_bits:%u", num_ordinary, range_bits); location_t offset = line_table->highest_location + 1; - /* Ensure offset doesn't go backwards at the start. */ - if ((offset & range_mask) > low_bits) - offset += range_mask + 1; - offset = (offset & ~range_mask); - - bool propagated = spans.maybe_propagate (this, offset + low_bits); + offset += ((1u << range_bits) - 1); + offset &= ~((1u << range_bits) - 1); + ordinary_locs.first = offset; + bool propagated = spans.maybe_propagate (this, offset); line_map_ordinary *maps = static_cast (line_map_new_raw (line_table, false, num_ordinary)); - location_t lwm = offset; - slurp->loc_deltas.first = offset - zero; - ordinary_locs.first = zero + low_bits + slurp->loc_deltas.first; - dump () && dump ("Ordinary loc delta %d", slurp->loc_deltas.first); - + const line_map_ordinary *base = nullptr; for (unsigned ix = 0; ix != num_ordinary && !sec.get_overrun (); ix++) { line_map_ordinary *map = &maps[ix]; - unsigned hwm = sec.u (); - /* Record the current HWM so that the below read_location is - ok. */ - ordinary_locs.second = hwm + slurp->loc_deltas.first; - map->start_location = hwm + (offset - zero); - if (map->start_location < lwm) - sec.set_overrun (); - lwm = map->start_location; - dump (dumper::LOCATION) && dump ("Map:%u %u->%u", ix, hwm, lwm); - map->reason = lc_reason (sec.u ()); - map->sysp = sec.u (); - map->m_range_bits = sec.u (); - map->m_column_and_range_bits = map->m_range_bits + sec.u (); - - unsigned fnum = sec.u (); - map->to_file = (fnum < filenames.length () ? filenames[fnum] : ""); - map->to_line = sec.u (); - - /* Root the outermost map at our location. */ - location_t from = read_location (sec); - map->included_from = from != UNKNOWN_LOCATION ? from : loc; + unsigned offset = sec.u (); + if (!offset) + { + map->reason = LC_RENAME; + map->sysp = sec.u (); + map->m_range_bits = sec.u (); + map->m_column_and_range_bits = sec.u () + map->m_range_bits; + unsigned fnum = sec.u (); + map->to_file = (fnum < filenames.length () ? filenames[fnum] : ""); + map->to_line = sec.u (); + base = map; + } + else + { + *map = *base; + map->to_line += offset >> map->m_column_and_range_bits; + } + unsigned remap = sec.u (); + map->start_location = remap + ordinary_locs.first; + if (base == map) + { + /* Root the outermost map at our location. */ + ordinary_locs.second = remap; + location_t from = read_location (sec); + map->included_from = from != UNKNOWN_LOCATION ? from : loc; + } } - location_t hwm = sec.u (); - ordinary_locs.second = hwm + slurp->loc_deltas.first; - + ordinary_locs.second = num_ord_locs; /* highest_location is the one handed out, not the next one to hand out. */ - line_table->highest_location = ordinary_locs.second - 1; + line_table->highest_location = ordinary_locs.first + ordinary_locs.second - 1; if (line_table->highest_location >= LINE_MAP_MAX_LOCATION_WITH_COLS) /* We shouldn't run out of locations, as we checked before starting. */ sec.set_overrun (); - dump () && dump ("Ordinary location hwm:%u", ordinary_locs.second); + dump () && dump ("Ordinary location [%u,+%u)", + ordinary_locs.first, ordinary_locs.second); if (propagated) spans.close (); @@ -16988,6 +17041,10 @@ module_state::write_macros (elf_out *to, vec *macros, if (mac.def) write_define (sec, mac.def); } + if (count) + // We may have ended on a tokenless macro with a very short + // location, that will cause problems reading its bit flags. + sec.u (0); sec.end (to, to->name (MOD_SNAME_PFX ".def"), crc_p); if (count) @@ -17453,7 +17510,7 @@ module_state::write_config (elf_out *to, module_state_config &config, cfg.u (config.ordinary_locs); cfg.u (config.macro_locs); - cfg.u (config.ordinary_loc_align); + cfg.u (config.loc_range_bits); cfg.u (config.active_init); @@ -17639,7 +17696,7 @@ module_state::read_config (module_state_config &config) config.ordinary_locs = cfg.u (); config.macro_locs = cfg.u (); - config.ordinary_loc_align = cfg.u (); + config.loc_range_bits = cfg.u (); config.active_init = cfg.u (); @@ -17656,7 +17713,7 @@ ool_cmp (const void *a_, const void *b_) auto *b = *static_cast (b_); if (a == b) return 0; - else if (a->ordinary_locs.first < b->ordinary_locs.second) + else if (a->ordinary_locs.first < b->ordinary_locs.first) return -1; else return +1; @@ -17690,6 +17747,7 @@ module_state::write_begin (elf_out *to, cpp_reader *reader, bitmap partitions = NULL; if (!is_header () && !is_partition ()) partitions = BITMAP_GGC_ALLOC (); + write_init_maps (); unsigned mod_hwm = 1; for (unsigned ix = 1; ix != modules->length (); ix++) @@ -17727,14 +17785,15 @@ module_state::write_begin (elf_out *to, cpp_reader *reader, gcc_checking_assert (!slot->is_lazy ()); } } + + if (imp->is_direct () && (imp->remap || imp->is_partition ())) + note_location (imp->imported_from ()); } if (partitions && bitmap_empty_p (partitions)) /* No partitions present. */ partitions = nullptr; - write_init_maps (); - /* Find the set of decls we must write out. */ depset::hash table (DECL_NAMESPACE_BINDINGS (global_namespace)->size () * 8); /* Add the specializations before the writables, so that we can @@ -17785,11 +17844,10 @@ module_state::write_begin (elf_out *to, cpp_reader *reader, if (is_header ()) macros = prepare_macros (reader); - location_map_info map_info = write_prepare_maps (&config); - unsigned counts[MSC_HWM]; - config.num_imports = mod_hwm; config.num_partitions = modules->length () - mod_hwm; + auto map_info = write_prepare_maps (&config, bool (config.num_partitions)); + unsigned counts[MSC_HWM]; memset (counts, 0, sizeof (counts)); /* depset::cluster is the cluster number, @@ -17926,8 +17984,10 @@ module_state::write_begin (elf_out *to, cpp_reader *reader, write_partitions (to, config.num_partitions, &crc); /* Write the line maps. */ - write_ordinary_maps (to, map_info, &config, config.num_partitions, &crc); - write_macro_maps (to, map_info, &config, &crc); + if (config.ordinary_locs) + write_ordinary_maps (to, map_info, bool (config.num_partitions), &crc); + if (config.macro_locs) + write_macro_maps (to, map_info, &crc); if (is_header ()) { @@ -17947,6 +18007,7 @@ module_state::write_begin (elf_out *to, cpp_reader *reader, sccs.release (); vec_free (macro_loc_remap); + vec_free (ord_loc_remap); vec_free (ool); // FIXME:QOI: Have a command line switch to control more detailed @@ -17990,7 +18051,9 @@ module_state::read_initial (cpp_reader *reader) bool have_locs = ok && read_prepare_maps (&config); /* Ordinary maps before the imports. */ - if (have_locs && !read_ordinary_maps ()) + if (!(have_locs && config.ordinary_locs)) + ordinary_locs.first = line_table->highest_location + 1; + else if (!read_ordinary_maps (config.ordinary_locs, config.loc_range_bits)) ok = false; /* Allocate the REMAP vector. */ @@ -18017,7 +18080,7 @@ module_state::read_initial (cpp_reader *reader) { /* Allocate space in the entities array now -- that array must be - monotionically in step with the modules array. */ + monotonically in step with the modules array. */ entity_lwm = vec_safe_length (entity_ary); entity_num = config.num_entities; gcc_checking_assert (modules->length () == 1 @@ -18048,7 +18111,9 @@ module_state::read_initial (cpp_reader *reader) gcc_assert (!from ()->is_frozen ()); /* Macro maps after the imports. */ - if (ok && have_locs && !read_macro_maps (config.macro_locs)) + if (!(ok && have_locs && config.macro_locs)) + macro_locs.first = LINEMAPS_MACRO_LOWEST_LOCATION (line_table); + else if (!read_macro_maps (config.macro_locs)) ok = false; /* Note whether there's an active initializer. */ diff --git a/gcc/testsuite/g++.dg/modules/loc-prune-1.C b/gcc/testsuite/g++.dg/modules/loc-prune-1.C index 6978e496f1b..55d46e41523 100644 --- a/gcc/testsuite/g++.dg/modules/loc-prune-1.C +++ b/gcc/testsuite/g++.dg/modules/loc-prune-1.C @@ -14,6 +14,6 @@ int foo (int = YES) // { dg-final { scan-lang-dump { Macro maps:1} module } } // { dg-final { scan-lang-dump { Macro:0 YES 1/1.2 locations } module } } -// { dg-final { scan-lang-dump { Ordinary:[0-9]* maps hwm:[0-9]* macro:1 maps 1 locs} module } } +// { dg-final { scan-lang-dump { Macro maps:1 locs:1} module } } // { dg-final { scan-lang-dump-not {Macro:. NOT } module } } // { dg-final { scan-lang-dump-not {Macro:. AGAIN_NO } module } } diff --git a/gcc/testsuite/g++.dg/modules/loc-prune-4.C b/gcc/testsuite/g++.dg/modules/loc-prune-4.C new file mode 100644 index 00000000000..765c378e51e --- /dev/null +++ b/gcc/testsuite/g++.dg/modules/loc-prune-4.C @@ -0,0 +1,22 @@ +// { dg-additional-options {-Wno-pedantic -fmodules-ts -fdump-lang-module-lineno} } + +# 4 "unused" 1 +# 5 "" 2 + +export module foo; + +int foo (int) // separate +{ + + + return 0; +} + +int bar (int); // merge lines +int baz (int); + + +// { dg-final { scan-lang-dump {Ordinary maps:2 locs:12288 range_bits:5} module } } +// { dg-final { scan-lang-dump { 1 source file names\n Source file...=[^\n]*loc-prune-4.C\n} module } } +// { dg-final { scan-lang-dump { Span:0 ordinary \[2.....\+12288,\+4096\)->\[0,\+4096\)} module } } +// { dg-final { scan-lang-dump { Span:1 ordinary \[2.....\+40960,\+8192\)->\[4096,\+8192\)} module } } diff --git a/gcc/testsuite/g++.dg/modules/pr98718_a.C b/gcc/testsuite/g++.dg/modules/pr98718_a.C index ebd95ea1f85..01242d9e676 100644 --- a/gcc/testsuite/g++.dg/modules/pr98718_a.C +++ b/gcc/testsuite/g++.dg/modules/pr98718_a.C @@ -14,5 +14,5 @@ namespace std _GLIBCXX_VISIBILITY(default) export module hello:format; // { dg-module-cmi hello:format } -// { dg-final { scan-lang-dump { Ordinary:4 maps hwm:[0-9]* macro:0 maps 0 locs} module } } +// { dg-final { scan-lang-dump { Macro maps:0 locs:0} module } } // { dg-final { scan-lang-dump-not { Macro:. _GLIBCXX_VISIBILITY} module } } diff --git a/gcc/testsuite/g++.dg/modules/pr98718_b.C b/gcc/testsuite/g++.dg/modules/pr98718_b.C index 6cb4698bad5..d2a1c53f93e 100644 --- a/gcc/testsuite/g++.dg/modules/pr98718_b.C +++ b/gcc/testsuite/g++.dg/modules/pr98718_b.C @@ -14,5 +14,5 @@ export module hello; export import :format; // { dg-module-cmi hello } -// { dg-final { scan-lang-dump { Ordinary:8 maps hwm:[0-9]* macro:0 maps 0 locs} module } } +// { dg-final { scan-lang-dump { Macro maps:0 locs:0} module } } // { dg-final { scan-lang-dump-not { Macro:. _GLIBCXX_VISIBILITY} module } } diff --git a/gcc/testsuite/g++.dg/modules/pr99072.H b/gcc/testsuite/g++.dg/modules/pr99072.H index eda0c07f9e2..3be69964f39 100644 --- a/gcc/testsuite/g++.dg/modules/pr99072.H +++ b/gcc/testsuite/g++.dg/modules/pr99072.H @@ -7,4 +7,6 @@ # 1 "REALNAME" // { dg-additional-options {-fmodule-header -fpreprocessed -fdump-lang-module-lineno} } -// { dg-final { scan-lang-dump { 4 source file names\n Source file\[0\]=REALNAME\n Source file\[1\]=\n Source file\[2\]=\n Source file\[3\]=/usr/include/stdc-predef.h\n} module } } +// All locations are pruned. +// { dg-final { scan-lang-dump-not {Writing ordinary location maps} module } } +// { dg-final { scan-lang-dump-not { Span:. ordinary } module } } -- 2.30.2