From patchwork Mon Nov 30 13:11:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keqian Zhu X-Patchwork-Id: 1408223 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=huawei.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Cl5Hn3w20z9sSf for ; Tue, 1 Dec 2020 00:12:37 +1100 (AEDT) Received: from localhost ([::1]:37174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kjiyp-0004ap-CP for incoming@patchwork.ozlabs.org; Mon, 30 Nov 2020 08:12:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjiy7-0004Z2-7C; Mon, 30 Nov 2020 08:11:53 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:2872) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjixz-0007hu-BK; Mon, 30 Nov 2020 08:11:50 -0500 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4Cl5G217DFz15VhK; Mon, 30 Nov 2020 21:11:06 +0800 (CST) Received: from DESKTOP-5IS4806.china.huawei.com (10.174.187.37) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.487.0; Mon, 30 Nov 2020 21:11:21 +0800 From: Keqian Zhu To: Peter Maydell , Paolo Bonzini , "Dr . David Alan Gilbert" , "Fam Zheng" , Stefan Hajnoczi Subject: [PATCH v2 1/2] ramlist: Make dirty bitmap blocks of ramlist resizable Date: Mon, 30 Nov 2020 21:11:03 +0800 Message-ID: <20201130131104.10600-2-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 In-Reply-To: <20201130131104.10600-1-zhukeqian1@huawei.com> References: <20201130131104.10600-1-zhukeqian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.37] X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.190; envelope-from=zhukeqian1@huawei.com; helo=szxga04-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wanghaibin.wang@huawei.com, qemu-arm@nongnu.org, Keqian Zhu , qemu-devel@nongnu.org, kuhn.chenqun@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When we remove a ramblock, we should decrease the dirty bitmap blocks of ramlist to avoid memory leakage. This patch rebuilds dirty_memory_ extend to support both "extend" and "decrease". Reported-by: Euler Robot Signed-off-by: Keqian Zhu --- softmmu/physmem.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/softmmu/physmem.c b/softmmu/physmem.c index 3027747c03..3e4f29f126 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -1816,17 +1816,19 @@ void qemu_ram_msync(RAMBlock *block, ram_addr_t start, ram_addr_t length) } /* Called with ram_list.mutex held */ -static void dirty_memory_extend(ram_addr_t old_ram_size, +static void dirty_memory_resize(ram_addr_t old_ram_size, ram_addr_t new_ram_size) { ram_addr_t old_num_blocks = DIV_ROUND_UP(old_ram_size, DIRTY_MEMORY_BLOCK_SIZE); ram_addr_t new_num_blocks = DIV_ROUND_UP(new_ram_size, DIRTY_MEMORY_BLOCK_SIZE); + ram_addr_t cpy_num_blocks = MIN(old_num_blocks, new_num_blocks); + bool extend = new_num_blocks > old_num_blocks; int i; - /* Only need to extend if block count increased */ - if (new_num_blocks <= old_num_blocks) { + /* Only need to resize if block count changed */ + if (new_num_blocks == old_num_blocks) { return; } @@ -1839,15 +1841,26 @@ static void dirty_memory_extend(ram_addr_t old_ram_size, new_blocks = g_malloc(sizeof(*new_blocks) + sizeof(new_blocks->blocks[0]) * new_num_blocks); - if (old_num_blocks) { + if (cpy_num_blocks) { memcpy(new_blocks->blocks, old_blocks->blocks, - old_num_blocks * sizeof(old_blocks->blocks[0])); + cpy_num_blocks * sizeof(old_blocks->blocks[0])); } - for (j = old_num_blocks; j < new_num_blocks; j++) { - new_blocks->blocks[j] = bitmap_new(DIRTY_MEMORY_BLOCK_SIZE); + if (extend) { + for (j = cpy_num_blocks; j < new_num_blocks; j++) { + new_blocks->blocks[j] = bitmap_new(DIRTY_MEMORY_BLOCK_SIZE); + } + } else { + for (j = cpy_num_blocks; j < old_num_blocks; j++) { + /* We are safe to free it, for that it is out-of-use */ + g_free(old_blocks->blocks[j]); + } } + if (!new_num_blocks) { + g_free(new_blocks); + new_blocks = NULL; + } qatomic_rcu_set(&ram_list.dirty_memory[i], new_blocks); if (old_blocks) { @@ -1894,7 +1907,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) new_ram_size = MAX(old_ram_size, (new_block->offset + new_block->max_length) >> TARGET_PAGE_BITS); if (new_ram_size > old_ram_size) { - dirty_memory_extend(old_ram_size, new_ram_size); + dirty_memory_resize(old_ram_size, new_ram_size); } /* Keep the list sorted from biggest to smallest block. Unlike QTAILQ, * QLIST (which has an RCU-friendly variant) does not have insertion at From patchwork Mon Nov 30 13:11:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keqian Zhu X-Patchwork-Id: 1408227 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=huawei.com Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Cl5P05ld1z9sSs for ; Tue, 1 Dec 2020 00:17:08 +1100 (AEDT) Received: from localhost ([::1]:44868 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kjj3C-0007oc-NK for incoming@patchwork.ozlabs.org; Mon, 30 Nov 2020 08:17:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjiyA-0004ZA-3b; Mon, 30 Nov 2020 08:11:55 -0500 Received: from szxga04-in.huawei.com ([45.249.212.190]:2873) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjixz-0007hp-GO; Mon, 30 Nov 2020 08:11:52 -0500 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4Cl5G20mDPz15VHM; Mon, 30 Nov 2020 21:11:06 +0800 (CST) Received: from DESKTOP-5IS4806.china.huawei.com (10.174.187.37) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.487.0; Mon, 30 Nov 2020 21:11:22 +0800 From: Keqian Zhu To: Peter Maydell , Paolo Bonzini , "Dr . David Alan Gilbert" , "Fam Zheng" , Stefan Hajnoczi Subject: [PATCH v2 2/2] ramlist: Resize dirty bitmap blocks after remove ramblock Date: Mon, 30 Nov 2020 21:11:04 +0800 Message-ID: <20201130131104.10600-3-zhukeqian1@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 In-Reply-To: <20201130131104.10600-1-zhukeqian1@huawei.com> References: <20201130131104.10600-1-zhukeqian1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.174.187.37] X-CFilter-Loop: Reflected Received-SPF: pass client-ip=45.249.212.190; envelope-from=zhukeqian1@huawei.com; helo=szxga04-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wanghaibin.wang@huawei.com, qemu-arm@nongnu.org, Keqian Zhu , qemu-devel@nongnu.org, kuhn.chenqun@huawei.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Use the new "dirty_memory_resize" interface to reduce dirty bitmap blocks after we remove a ramblock from ramlist. This bug is found by ASAN after executing several qmp commands about object-add/object-del of memory-backend-ram. After applying this patch, the memory leak is not reported anymore. ================================================================= ==qemu-system-aarch64==1720167==ERROR: LeakSanitizer: detected memory leaks Direct leak of 2359296 byte(s) in 9 object(s) allocated from: #0 0xfffeedf3e938 in __interceptor_calloc (/lib64/libasan.so.5+0xee938) #1 0xaaaaf6f1e740 in bitmap_new /qemu/include/qemu/bitmap.h:101 #2 0xaaaaf6f1e81c in dirty_memory_extend ../exec.c:2212 #3 0xaaaaf6f22524 in ram_block_add ../exec.c:2261 #4 0xaaaaf6f22988 in qemu_ram_alloc_internal ../exec.c:2434 #5 0xaaaaf6f8ae70 in memory_region_init_ram_shared_nomigrate ../softmmu/memory.c:1513 #6 0xaaaaf66edee0 in ram_backend_memory_alloc ../backends/hostmem-ram.c:30 #7 0xaaaaf660d03c in host_memory_backend_memory_complete ../backends/hostmem.c:333 #8 0xaaaaf70f6968 in user_creatable_complete ../qom/object_interfaces.c:23 #9 0xaaaaf70f6dac in user_creatable_add_type ../qom/object_interfaces.c:93 #10 0xaaaaf70f7030 in user_creatable_add_dict ../qom/object_interfaces.c:134 #11 0xaaaaf7340174 in do_qmp_dispatch_bh ../qapi/qmp-dispatch.c:110 #12 0xaaaaf732da30 in aio_bh_poll ../util/async.c:164 #13 0xaaaaf735c9a8 in aio_dispatch ../util/aio-posix.c:381 #14 0xaaaaf732d2ec in aio_ctx_dispatch ../util/async.c:306 #15 0xfffeecb029d8 in g_main_context_dispatch (/lib64/libglib-2.0.so.0+0x529d8) #16 0xaaaaf733bb78 in os_host_main_loop_wait ../util/main-loop.c:244 #17 0xaaaaf733beac in main_loop_wait ../util/main-loop.c:520 #18 0xaaaaf70802a4 in qemu_main_loop ../softmmu/vl.c:1677 #19 0xaaaaf655786c in main ../softmmu/main.c:50 #20 0xfffeec043f5c in __libc_start_main (/lib64/libc.so.6+0x23f5c) #21 0xaaaaf656a198 (/qemu/build/qemu-system-aarch64+0x9ba198) SUMMARY: AddressSanitizer: 2359296 byte(s) leaked in 9 allocation(s). Reported-by: Euler Robot Signed-off-by: Keqian Zhu ---- little concern: According to code, my bugfix can solve two problems: 1. Lose reference to dirty bitmap of deleted ramblock, because the reference is covered by dirty bitmap of newly added ramblock. 2. All dirty bitmap is not freed before qemu exit. However, ASAN do not report memory leak for point 2. Any thoughts or explanations? --- softmmu/physmem.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/softmmu/physmem.c b/softmmu/physmem.c index 3e4f29f126..8c5f910677 100644 --- a/softmmu/physmem.c +++ b/softmmu/physmem.c @@ -2132,6 +2132,8 @@ static void reclaim_ramblock(RAMBlock *block) void qemu_ram_free(RAMBlock *block) { + ram_addr_t old_ram_size, new_ram_size; + if (!block) { return; } @@ -2141,12 +2143,18 @@ void qemu_ram_free(RAMBlock *block) } qemu_mutex_lock_ramlist(); + + old_ram_size = last_ram_page(); QLIST_REMOVE_RCU(block, next); + new_ram_size = last_ram_page(); + dirty_memory_resize(old_ram_size, new_ram_size); + ram_list.mru_block = NULL; /* Write list before version */ smp_wmb(); ram_list.version++; call_rcu(block, reclaim_ramblock, rcu); + qemu_mutex_unlock_ramlist(); }