From patchwork Thu Jun 18 01:15:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 1311714 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49nR5N4gsnz9sXD for ; Thu, 18 Jun 2020 12:41:12 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49nR5N2x82zDqCD for ; Thu, 18 Jun 2020 12:41:12 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=arm.com (client-ip=217.140.110.172; helo=foss.arm.com; envelope-from=anshuman.khandual@arm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lists.ozlabs.org (Postfix) with ESMTP id 49nPCg1tvkzDqnD for ; Thu, 18 Jun 2020 11:16:30 +1000 (AEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9D29E1045; Wed, 17 Jun 2020 18:16:29 -0700 (PDT) Received: from p8cg001049571a15.arm.com (unknown [10.163.80.176]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9F6F43F6CF; Wed, 17 Jun 2020 18:16:23 -0700 (PDT) From: Anshuman Khandual To: linux-mm@kvack.org Subject: [PATCH V3 (RESEND) 2/3] mm/sparsemem: Enable vmem_altmap support in vmemmap_alloc_block_buf() Date: Thu, 18 Jun 2020 06:45:29 +0530 Message-Id: <1592442930-9380-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592442930-9380-1-git-send-email-anshuman.khandual@arm.com> References: <1592442930-9380-1-git-send-email-anshuman.khandual@arm.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: x86@kernel.org, "H. Peter Anvin" , Anshuman Khandual , Peter Zijlstra , Catalin Marinas , Dave Hansen , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Ingo Molnar , Paul Mackerras , Andy Lutomirski , Borislav Petkov , Thomas Gleixner , Will Deacon , Andrew Morton , linux-arm-kernel@lists.infradead.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" There are many instances where vmemap allocation is often switched between regular memory and device memory just based on whether altmap is available or not. vmemmap_alloc_block_buf() is used in various platforms to allocate vmemmap mappings. Lets also enable it to handle altmap based device memory allocation along with existing regular memory allocations. This will help in avoiding the altmap based allocation switch in many places. While here also implement a regular memory allocation fallback mechanism when the first preferred device memory allocation fails. This will ensure preserving the existing semantics on powerpc platform. To summarize there are three different methods to call vmemmap_alloc_block_buf(). (., NULL, false) /* Allocate from system RAM */ (., altmap, false) /* Allocate from altmap without any fallback */ (., altmap, true) /* Allocate from altmap with fallback (system RAM) */ Cc: Catalin Marinas Cc: Will Deacon Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Andrew Morton Cc: x86@kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Tested-by: Jia He Suggested-by: Robin Murphy Signed-off-by: Anshuman Khandual --- arch/arm64/mm/mmu.c | 3 ++- arch/powerpc/mm/init_64.c | 10 +++++----- arch/x86/mm/init_64.c | 6 ++---- include/linux/mm.h | 3 ++- mm/sparse-vmemmap.c | 30 ++++++++++++++++++++++++------ 5 files changed, 35 insertions(+), 17 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0adad8859393..7ca21adb4412 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1100,7 +1100,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, if (pmd_none(READ_ONCE(*pmdp))) { void *p = NULL; - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + p = vmemmap_alloc_block_buf(PMD_SIZE, node, + NULL, false); if (!p) return -ENOMEM; diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index bc73abf0bc25..01e25b56eccb 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -225,12 +225,12 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, * fall back to system memory if the altmap allocation fail. */ if (altmap && !altmap_cross_boundary(altmap, start, page_size)) { - p = altmap_alloc_block_buf(page_size, altmap); - if (!p) - pr_debug("altmap block allocation failed, falling back to system memory"); + p = vmemmap_alloc_block_buf(page_size, node, + altmap, true); + } else { + p = vmemmap_alloc_block_buf(page_size, node, + NULL, false); } - if (!p) - p = vmemmap_alloc_block_buf(page_size, node); if (!p) return -ENOMEM; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 19c0ed3271a3..4ae4f767c004 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -1463,10 +1463,8 @@ static int __meminit vmemmap_populate_hugepages(unsigned long start, if (pmd_none(*pmd)) { void *p; - if (altmap) - p = altmap_alloc_block_buf(PMD_SIZE, altmap); - else - p = vmemmap_alloc_block_buf(PMD_SIZE, node); + p = vmemmap_alloc_block_buf(PMD_SIZE, node, + altmap, false); if (p) { pte_t entry; diff --git a/include/linux/mm.h b/include/linux/mm.h index e40ac543d248..dade7c3f634d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,7 +3015,8 @@ pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, struct vmem_altmap *altmap); void *vmemmap_alloc_block(unsigned long size, int node); struct vmem_altmap; -void *vmemmap_alloc_block_buf(unsigned long size, int node); +void *vmemmap_alloc_block_buf(unsigned long size, int node, + struct vmem_altmap *altmap, bool sysram_fallback); void *altmap_alloc_block_buf(unsigned long size, struct vmem_altmap *altmap); void vmemmap_verify(pte_t *, int, unsigned long, unsigned long); int vmemmap_populate_basepages(unsigned long start, unsigned long end, diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index ceed10dec31e..388121c5bbcb 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -70,10 +70,31 @@ void * __meminit vmemmap_alloc_block(unsigned long size, int node) } /* need to make sure size is all the same during early stage */ -void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node) +void * __meminit vmemmap_alloc_block_buf(unsigned long size, int node, + struct vmem_altmap *altmap, + bool sysram_fallback) { - void *ptr = sparse_buffer_alloc(size); + void *ptr; + /* + * There is no point in asking for sysram fallback + * without an altmap request to begin with. So just + * warn here to catch potential call sites that may + * be violating this. + */ + WARN_ON(!altmap && sysram_fallback); + + if (altmap) { + ptr = altmap_alloc_block_buf(size, altmap); + if (ptr) + return ptr; + pr_debug("altmap block allocation failed\n"); + if (!sysram_fallback) + return NULL; + pr_debug("falling back to system memory\n"); + } + + ptr = sparse_buffer_alloc(size); if (!ptr) ptr = vmemmap_alloc_block(size, node); return ptr; @@ -147,10 +168,7 @@ pte_t * __meminit vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, pte_t entry; void *p; - if (altmap) - p = altmap_alloc_block_buf(PAGE_SIZE, altmap); - else - p = vmemmap_alloc_block_buf(PAGE_SIZE, node); + p = vmemmap_alloc_block_buf(PAGE_SIZE, node, altmap, false); if (!p) return NULL; entry = pfn_pte(__pa(p) >> PAGE_SHIFT, PAGE_KERNEL);