From patchwork Tue Apr 17 08:46:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Smith X-Patchwork-Id: 899131 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40QJkn69q7z9rxs for ; Tue, 17 Apr 2018 18:46:25 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 40QJkn4xnlzF1xW for ; Tue, 17 Apr 2018 18:46:25 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=stewart@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 40QJkg6tWPzF1wB for ; Tue, 17 Apr 2018 18:46:19 +1000 (AEST) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w3H8iNbe032261 for ; Tue, 17 Apr 2018 04:46:17 -0400 Received: from e32.co.us.ibm.com (e32.co.us.ibm.com [32.97.110.150]) by mx0b-001b2d01.pphosted.com with ESMTP id 2hdars8bem-1 (version=TLSv1.2 cipher=AES256-SHA256 bits=256 verify=NOT) for ; Tue, 17 Apr 2018 04:46:16 -0400 Received: from localhost by e32.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 17 Apr 2018 02:46:16 -0600 Received: from b03cxnp07028.gho.boulder.ibm.com (9.17.130.15) by e32.co.us.ibm.com (192.168.1.132) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 17 Apr 2018 02:46:13 -0600 Received: from b03ledav003.gho.boulder.ibm.com (b03ledav003.gho.boulder.ibm.com [9.17.130.234]) by b03cxnp07028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w3H8kDEU13828530 for ; Tue, 17 Apr 2018 01:46:13 -0700 Received: from b03ledav003.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E2D1D6A03D for ; Tue, 17 Apr 2018 02:46:12 -0600 (MDT) Received: from birb.localdomain (unknown [9.185.142.92]) by b03ledav003.gho.boulder.ibm.com (Postfix) with ESMTP id 756266A03C for ; Tue, 17 Apr 2018 02:46:12 -0600 (MDT) Received: by birb.localdomain (Postfix, from userid 1000) id EBD4B4EC649; Tue, 17 Apr 2018 18:46:07 +1000 (AEST) From: Stewart Smith To: skiboot@lists.ozlabs.org Date: Tue, 17 Apr 2018 18:46:07 +1000 X-Mailer: git-send-email 2.14.3 X-TM-AS-GCONF: 00 x-cbid: 18041708-0004-0000-0000-000013F70010 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00008868; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000257; SDB=6.01019133; UDB=6.00519898; IPR=6.00798374; MB=3.00020611; MTD=3.00000008; XFM=3.00000015; UTC=2018-04-17 08:46:14 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18041708-0005-0000-0000-000086EB2877 Message-Id: <20180417084607.18252-1-stewart@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-17_03:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1804170080 Subject: [Skiboot] [PATCH] fast-reboot: parallel memory clearing X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" A real simple hack to split things up into 16GB jobs, and name them as how far we are into zeroing, so we can simply print progress out as XGB cleared as we wait for the jobs to finish. This seems to cut at least ~40% time from memory zeroing on fast-reboot on a 256GB Boston system. Signed-off-by: Stewart Smith --- core/mem_region.c | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 61 insertions(+), 2 deletions(-) diff --git a/core/mem_region.c b/core/mem_region.c index 8ae49bb790fd..a06e725db3ec 100644 --- a/core/mem_region.c +++ b/core/mem_region.c @@ -1206,19 +1206,47 @@ static void mem_clear_range(uint64_t s, uint64_t e) return; } - prlog(PR_NOTICE, "Clearing region %llx-%llx\n", + prlog(PR_DEBUG, "Clearing region %llx-%llx\n", (long long)s, (long long)e); memset((void *)s, 0, e - s); } +static void mem_region_clear_job(void *data) +{ + uint64_t *arg = (uint64_t*)data; + mem_clear_range(arg[0], arg[1]); +} + +#define MEM_REGION_CLEAR_JOB_SIZE (16ULL*(1<<30)) + void mem_region_clear_unused(void) { + int njobs = 0; + struct cpu_job **jobs; struct mem_region *r; + uint64_t *job_args; + uint64_t s,l; + uint64_t total = 0; + char **job_names; + int i; lock(&mem_region_lock); assert(mem_regions_finalised); + list_for_each(®ions, r, list) { + if (!(r->type == REGION_OS)) + continue; + njobs++; + /* One job per 16GB */ + njobs += r->len / MEM_REGION_CLEAR_JOB_SIZE; + } + + jobs = malloc(njobs * sizeof(struct cpu_job*)); + job_args = malloc(2*njobs*sizeof(uint64_t)); + job_names = malloc(njobs*sizeof(char*)); + prlog(PR_NOTICE, "Clearing unused memory:\n"); + i = 0; list_for_each(®ions, r, list) { /* If it's not unused, ignore it. */ if (!(r->type == REGION_OS)) @@ -1226,9 +1254,40 @@ void mem_region_clear_unused(void) assert(r != &skiboot_heap); - mem_clear_range(r->start, r->start + r->len); + s = r->start; + l = r->len; + while(l > MEM_REGION_CLEAR_JOB_SIZE) { + job_args[i*2] = s+l - MEM_REGION_CLEAR_JOB_SIZE; + job_args[i*2+1] = s+l; + l-=MEM_REGION_CLEAR_JOB_SIZE; + job_names[i] = malloc(sizeof(char)*40); + total+=MEM_REGION_CLEAR_JOB_SIZE; + snprintf(job_names[i], 40, "%lldGB cleared", total/(1<<30)); + jobs[i] = cpu_queue_job(NULL, job_names[i], + mem_region_clear_job, + &job_args[i*2]); + i++; + } + job_args[i*2] = s; + job_args[i*2+1] = s+l; + job_names[i] = malloc(sizeof(char)*40); + total+=l; + snprintf(job_names[i], 40, "%lldGB cleared", total/(1<<30)); + jobs[i] = cpu_queue_job(NULL, job_names[i], + mem_region_clear_job, + &job_args[i*2]); + i++; + } + cpu_process_local_jobs(); + for(i=0; i < njobs; i++) { + cpu_wait_job(jobs[i], true); + printf("%s\n",job_names[i]); + free(job_names[i]); } unlock(&mem_region_lock); + free(jobs); + free(job_args); + free(job_names); } static void mem_region_add_dt_reserved_node(struct dt_node *parent,