From patchwork Fri Jul 7 18:49:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 1805040 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sourceware.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.a=rsa-sha256 header.s=default header.b=Kbgf4DC6; dkim-atps=neutral Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QyMwH2Y4Zz20WT for ; Sat, 8 Jul 2023 04:52:47 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5D311388450A for ; Fri, 7 Jul 2023 18:52:45 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 5D311388450A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1688755965; bh=Hn1FGL8Y4sPlfe3tMG2IXTj1HwXKdRtZYNUwP4/bxdc=; h=To:Subject:In-Reply-To:References:Date:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Kbgf4DC6C7daw+aOdjHhvcNMK1f/lKTKFUCkQQyXl0v3FgUoqNxaXhhlggK+M12Wi rQ/7qbdIQnklUcgVFJZvKnpuQhAAtNsKuQ6+60pg1N2TOYrjIn4DM1vGBrd1jVmcrO DKjzLMTRlXk4i1Pz3+d+HEA89eU63TemMSTdyyv4= X-Original-To: libc-alpha@sourceware.org Delivered-To: libc-alpha@sourceware.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4F6A1384816C for ; Fri, 7 Jul 2023 18:49:31 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 4F6A1384816C Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-522-AOOvhT7LPSejhcTtpXzCtg-1; Fri, 07 Jul 2023 14:49:29 -0400 X-MC-Unique: AOOvhT7LPSejhcTtpXzCtg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C4E1E101A529 for ; Fri, 7 Jul 2023 18:49:28 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.2.16.31]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ED9D840C206F for ; Fri, 7 Jul 2023 18:49:27 +0000 (UTC) To: libc-alpha@sourceware.org Subject: [PATCH v2 25/32] elf: Move most of the _dl_find_object data to the protected heap In-Reply-To: Message-ID: <925bdafbfcec97f229531a1e865527f3e9d405e6.1688741159.git.fweimer@redhat.com> References: X-From-Line: 925bdafbfcec97f229531a1e865527f3e9d405e6 Mon Sep 17 00:00:00 2001 Date: Fri, 07 Jul 2023 20:49:26 +0200 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Spam-Status: No, score=-10.6 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Florian Weimer via Libc-alpha From: Florian Weimer Reply-To: Florian Weimer Errors-To: libc-alpha-bounces+incoming=patchwork.ozlabs.org@sourceware.org Sender: "Libc-alpha" The heap is mostly read-only by design, so allocation padding is no longer required. The protected heap is not visible to malloc, so it's not necessary to deallocate the allocations during __libc_freeres anymore. --- elf/dl-find_object.c | 94 ++++++++----------------------------------- elf/dl-find_object.h | 3 -- elf/dl-libc_freeres.c | 2 - 3 files changed, 16 insertions(+), 83 deletions(-) diff --git a/elf/dl-find_object.c b/elf/dl-find_object.c index f81351b0ef..82f493d817 100644 --- a/elf/dl-find_object.c +++ b/elf/dl-find_object.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -91,8 +92,9 @@ static struct dl_find_object_internal *_dlfo_nodelete_mappings to avoid data races. The memory allocations are never deallocated, but slots used for - objects that have been dlclose'd can be reused by dlopen. The - memory can live in the regular C malloc heap. + objects that have been dlclose'd can be reused by dlopen. + Allocations come from the protected memory heap. This makes it + harder to inject DWARF data. The segments are populated from the start of the list, with the mappings with the highest address. Only if this segment is full, @@ -111,9 +113,6 @@ struct dlfo_mappings_segment initialization; read in the TM region. */ struct dlfo_mappings_segment *previous; - /* Used by __libc_freeres to deallocate malloc'ed memory. */ - void *to_free; - /* Count of array elements in use and allocated. */ size_t size; /* Read in the TM region. */ size_t allocated; @@ -154,44 +153,15 @@ _dlfo_mappings_segment_count_allocated (struct dlfo_mappings_segment *seg) /* This is essentially an arbitrary value. dlopen allocates plenty of memory anyway, so over-allocated a bit does not hurt. Not having - many small-ish segments helps to avoid many small binary searches. - Not using a power of 2 means that we do not waste an extra page - just for the malloc header if a mapped allocation is used in the - glibc allocator. */ -enum { dlfo_mappings_initial_segment_size = 63 }; - -/* Allocate an empty segment. This used for the first ever - allocation. */ -static struct dlfo_mappings_segment * -_dlfo_mappings_segment_allocate_unpadded (size_t size) -{ - if (size < dlfo_mappings_initial_segment_size) - size = dlfo_mappings_initial_segment_size; - /* No overflow checks here because the size is a mapping count, and - struct link_map_private is larger than what we allocate here. */ - enum - { - element_size = sizeof ((struct dlfo_mappings_segment) {}.objects[0]) - }; - size_t to_allocate = (sizeof (struct dlfo_mappings_segment) - + size * element_size); - struct dlfo_mappings_segment *result = malloc (to_allocate); - if (result != NULL) - { - result->previous = NULL; - result->to_free = NULL; /* Minimal malloc memory cannot be freed. */ - result->size = 0; - result->allocated = size; - } - return result; -} + many small-ish segments helps to avoid many small binary searches. */ +enum { dlfo_mappings_initial_segment_size = 64 }; /* Allocate an empty segment that is at least SIZE large. PREVIOUS points to the chain of previously allocated segments and can be NULL. */ static struct dlfo_mappings_segment * _dlfo_mappings_segment_allocate (size_t size, - struct dlfo_mappings_segment * previous) + struct dlfo_mappings_segment *previous) { /* Exponential sizing policies, so that lookup approximates a binary search. */ @@ -200,11 +170,10 @@ _dlfo_mappings_segment_allocate (size_t size, if (previous == NULL) minimum_growth = dlfo_mappings_initial_segment_size; else - minimum_growth = 2* previous->allocated; + minimum_growth = 2 * previous->allocated; if (size < minimum_growth) size = minimum_growth; } - enum { cache_line_size_estimate = 128 }; /* No overflow checks here because the size is a mapping count, and struct link_map_private is larger than what we allocate here. */ enum @@ -212,28 +181,13 @@ _dlfo_mappings_segment_allocate (size_t size, element_size = sizeof ((struct dlfo_mappings_segment) {}.objects[0]) }; size_t to_allocate = (sizeof (struct dlfo_mappings_segment) - + size * element_size - + 2 * cache_line_size_estimate); - char *ptr = malloc (to_allocate); - if (ptr == NULL) + + size * element_size); + struct dlfo_mappings_segment *result = _dl_protmem_allocate (to_allocate); + if (result == NULL) return NULL; - char *original_ptr = ptr; - /* Start and end at a (conservative) 128-byte cache line boundary. - Do not use memalign for compatibility with partially interposing - malloc implementations. */ - char *end = PTR_ALIGN_DOWN (ptr + to_allocate, cache_line_size_estimate); - ptr = PTR_ALIGN_UP (ptr, cache_line_size_estimate); - struct dlfo_mappings_segment *result - = (struct dlfo_mappings_segment *) ptr; result->previous = previous; - result->to_free = original_ptr; result->size = 0; - /* We may have obtained slightly more space if malloc happened - to provide an over-aligned pointer. */ - result->allocated = (((uintptr_t) (end - ptr) - - sizeof (struct dlfo_mappings_segment)) - / element_size); - assert (result->allocated >= size); + result->allocated = size; return result; } @@ -577,11 +531,12 @@ _dl_find_object_init (void) /* Allocate the data structures. */ size_t loaded_size = _dlfo_process_initial (); - _dlfo_nodelete_mappings = malloc (_dlfo_nodelete_mappings_size - * sizeof (*_dlfo_nodelete_mappings)); + _dlfo_nodelete_mappings + = _dl_protmem_allocate (_dlfo_nodelete_mappings_size + * sizeof (*_dlfo_nodelete_mappings)); if (loaded_size > 0) _dlfo_loaded_mappings[0] - = _dlfo_mappings_segment_allocate_unpadded (loaded_size); + = _dlfo_mappings_segment_allocate (loaded_size, NULL); if (_dlfo_nodelete_mappings == NULL || (loaded_size > 0 && _dlfo_loaded_mappings[0] == NULL)) _dl_fatal_printf ("\ @@ -838,20 +793,3 @@ _dl_find_object_dlclose (struct link_map_private *map) return; } } - -void -_dl_find_object_freeres (void) -{ - for (int idx = 0; idx < 2; ++idx) - { - for (struct dlfo_mappings_segment *seg = _dlfo_loaded_mappings[idx]; - seg != NULL; ) - { - struct dlfo_mappings_segment *previous = seg->previous; - free (seg->to_free); - seg = previous; - } - /* Stop searching in shared objects. */ - _dlfo_loaded_mappings[idx] = 0; - } -} diff --git a/elf/dl-find_object.h b/elf/dl-find_object.h index edcc0a7755..54601e7d00 100644 --- a/elf/dl-find_object.h +++ b/elf/dl-find_object.h @@ -135,7 +135,4 @@ bool _dl_find_object_update (struct link_map_private *new_l) attribute_hidden; data structures. Needs to be protected by loader write lock. */ void _dl_find_object_dlclose (struct link_map_private *l) attribute_hidden; -/* Called from __libc_freeres to deallocate malloc'ed memory. */ -void _dl_find_object_freeres (void) attribute_hidden; - #endif /* _DL_FIND_OBJECT_H */ diff --git a/elf/dl-libc_freeres.c b/elf/dl-libc_freeres.c index 88c0e444b8..066629639c 100644 --- a/elf/dl-libc_freeres.c +++ b/elf/dl-libc_freeres.c @@ -128,6 +128,4 @@ __rtld_libc_freeres (void) void *scope_free_list = GL(dl_scope_free_list); GL(dl_scope_free_list) = NULL; free (scope_free_list); - - _dl_find_object_freeres (); }