From patchwork Fri May 10 18:03:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wilco Dijkstra X-Patchwork-Id: 1098260 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=sourceware.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=libc-alpha-return-101932-incoming=patchwork.ozlabs.org@sourceware.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=sourceware.org header.i=@sourceware.org header.b="IR35O3/r"; dkim=pass (1024-bit key; unprotected) header.d=armh.onmicrosoft.com header.i=@armh.onmicrosoft.com header.b="fm2AWIXY"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 450ylC6xxRz9s4V for ; Sat, 11 May 2019 04:04:07 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id :content-type:content-transfer-encoding:mime-version; q=dns; s= default; b=P3lRlZVcJT49j3rTwiPTQ2w4YncNatkYAsbfNGxLnv6iK4cRvd5k1 XGhYMgQSGFHmeSSfroor6bkbWIK8qOG43b3XeKvkvWaxz+8jyA8aV30l72TdMWFD Ly0F+vHZyPlGZ4v83/ervzOrogcyHSJN/+f4gnfDAJummncj5Skdzw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id :content-type:content-transfer-encoding:mime-version; s=default; bh=r/TvAPYll7Vi/sWCl9kfxk56oJM=; b=IR35O3/rQkhs1JKjRYSg8B+YfcEw s6FYGu3Gw015U982BU1Uiv4xMXg/Az5wr9/XRqLZngFpuQvEr9A+HOGPFaTkYjHD Ny1P2EP51UhFFKnqrCIAJngDQ6CpqZwRfC/NVLN+EipY/dtJGrcCQF2nMbGFtcy7 0Zf3eNTnx7KtXrQ= Received: (qmail 16336 invoked by alias); 10 May 2019 18:04:00 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 16328 invoked by uid 89); 10 May 2019 18:04:00 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-16.9 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS autolearn=ham version=3.3.1 spammy=H*Ad:U*dj, 1897 X-HELO: EUR01-VE1-obe.outbound.protection.outlook.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zo3K/aE8f6+XAOhGhdMf44Yx3kQDebPDuFi5RQ9whOs=; b=fm2AWIXYSKQxay3M5OmCQWgOfJiujqmCtqZnrIbaTjht+Vo7DMHJmvzJcdL04u/rlAIxa9ssNhVQi0DmqY7VuRjTxY0fFOoYM7oyrbm3umLEv1L/eJHc27LpZyoVdoZRj4W+N6hIZl6Sw/13gWRd5vzTSSz1B7/eejhSADCtrLY= From: Wilco Dijkstra To: "libc-alpha@sourceware.org" CC: nd , DJ Delorie Subject: [PATCH] Small tcache improvements * Date: Fri, 10 May 2019 18:03:54 +0000 Message-ID: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; x-ms-oob-tlc-oobclassifiers: OLM:3276; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED Change the tcache->counts[] entries to uint16_t - this removes the limit set by char and allows a larger tcache. Since 65535 seems too large, limit MAX_TCACHE_COUNT to 1024. bench-malloc-thread with 4 threads is ~15% faster on Cortex-A72. OK for commit? ChangeLog: 2019-05-10 Wilco Dijkstra * malloc/malloc.c (MAX_TCACHE_COUNT): Increase to 1024. (tcache_put): Remove redundant assert. (tcache_get): Remove redundant asserts. (__libc_malloc): Check tcache count is not zero. * manual/tunables.texi (glibc.malloc.tcache_count): Update maximum. diff --git a/malloc/malloc.c b/malloc/malloc.c index b8baaa2706d8d274b04b86e27fc72716753530b0..597406854ddbab8706e487a9ff1e1994a2bb2e83 100644 --- a/malloc/malloc.c +++ b/malloc/malloc.c @@ -321,6 +321,10 @@ __malloc_assert (const char *assertion, const char *file, unsigned int line, /* This is another arbitrary limit, which tunables can change. Each tcache bin will hold at most this number of chunks. */ # define TCACHE_FILL_COUNT 7 + +/* Maximum chunks in tcache bins for tunables. This value must fit the range + of tcache->counts[] entries, else they may overflow. */ +# define MAX_TCACHE_COUNT 1024 #endif @@ -2901,12 +2905,10 @@ typedef struct tcache_entry time), this is for performance reasons. */ typedef struct tcache_perthread_struct { - char counts[TCACHE_MAX_BINS]; + uint16_t counts[TCACHE_MAX_BINS]; tcache_entry *entries[TCACHE_MAX_BINS]; } tcache_perthread_struct; -#define MAX_TCACHE_COUNT 127 /* Maximum value of counts[] entries. */ - static __thread bool tcache_shutting_down = false; static __thread tcache_perthread_struct *tcache = NULL; @@ -2916,7 +2918,6 @@ static __always_inline void tcache_put (mchunkptr chunk, size_t tc_idx) { tcache_entry *e = (tcache_entry *) chunk2mem (chunk); - assert (tc_idx < TCACHE_MAX_BINS); /* Mark this chunk as "in the tcache" so the test in _int_free will detect a double free. */ @@ -2933,8 +2934,6 @@ static __always_inline void * tcache_get (size_t tc_idx) { tcache_entry *e = tcache->entries[tc_idx]; - assert (tc_idx < TCACHE_MAX_BINS); - assert (tcache->counts[tc_idx] > 0); tcache->entries[tc_idx] = e->next; --(tcache->counts[tc_idx]); e->key = NULL; @@ -3046,9 +3045,8 @@ __libc_malloc (size_t bytes) DIAG_PUSH_NEEDS_COMMENT; if (tc_idx < mp_.tcache_bins - /*&& tc_idx < TCACHE_MAX_BINS*/ /* to appease gcc */ && tcache - && tcache->entries[tc_idx] != NULL) + && tcache->counts[tc_idx] > 0) { return tcache_get (tc_idx); } diff --git a/manual/tunables.texi b/manual/tunables.texi index ae638823a21b9cc7aca3684c8e3067cb8cd287e0..cc244512d6893f2abddb9f203ab0ff14838b00e2 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -189,7 +189,7 @@ per-thread cache. The default (and maximum) value is 1032 bytes on @deftp Tunable glibc.malloc.tcache_count The maximum number of chunks of each size to cache. The default is 7. -The upper limit is 127. If set to zero, the per-thread cache is effectively +The upper limit is 1024. If set to zero, the per-thread cache is effectively disabled. The approximate maximum overhead of the per-thread cache is thus equal