From patchwork Tue Mar 24 10:40:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1260570 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=MhEapYe0; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 48mp4b4Qt8z9sSQ for ; Tue, 24 Mar 2020 21:53:03 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727273AbgCXKww (ORCPT ); Tue, 24 Mar 2020 06:52:52 -0400 Received: from mail.kernel.org ([198.145.29.99]:32968 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726818AbgCXKwv (ORCPT ); Tue, 24 Mar 2020 06:52:51 -0400 Received: from localhost.localdomain (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8ABE1208CA; Tue, 24 Mar 2020 10:52:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585047170; bh=28FajiIpTyOHoigH0+DT2DHz/lzMkgmR1Jc1TPsX4sQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MhEapYe0aCKi6JizZ1IklX+bhS+HoZkqodOna/Qaj6x3D3IrunRq97Hj4wBF8yZjg I2+cT3X7WOnClu4OzWb98T6d70hYsP2fq/8R72vswpKGQY0Z/Ku/NweoCdA21zPAni 0s83c/7g0V7V7QJUSZPEGJ7nw3+a0B5PVd1HEa/Q= From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , "David S. Miller" , Peter Zijlstra , Nick Desaulniers , Matt Fleming , sparclinux@vger.kernel.org Subject: [PATCH 1/4] sparc32: mm: Fix argument checking in __srmmu_get_nocache() Date: Tue, 24 Mar 2020 10:40:02 +0000 Message-Id: <20200324104005.11279-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200324104005.11279-1-will@kernel.org> References: <20200324104005.11279-1-will@kernel.org> MIME-Version: 1.0 Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org The 'size' argument to __srmmu_get_nocache() is a number of bytes not a shift value, so fix up the sanity checking to treat it properly. Cc: "David S. Miller" Cc: Peter Zijlstra Signed-off-by: Will Deacon --- arch/sparc/mm/srmmu.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index f56c3c9a9793..a19863cac0c4 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -175,18 +175,18 @@ pte_t *pte_offset_kernel(pmd_t *dir, unsigned long address) */ static void *__srmmu_get_nocache(int size, int align) { - int offset; + int offset, minsz = 1 << SRMMU_NOCACHE_BITMAP_SHIFT; unsigned long addr; - if (size < SRMMU_NOCACHE_BITMAP_SHIFT) { + if (size < minsz) { printk(KERN_ERR "Size 0x%x too small for nocache request\n", size); - size = SRMMU_NOCACHE_BITMAP_SHIFT; + size = minsz; } - if (size & (SRMMU_NOCACHE_BITMAP_SHIFT - 1)) { - printk(KERN_ERR "Size 0x%x unaligned int nocache request\n", + if (size & (minsz - 1)) { + printk(KERN_ERR "Size 0x%x unaligned in nocache request\n", size); - size += SRMMU_NOCACHE_BITMAP_SHIFT - 1; + size += minsz - 1; } BUG_ON(align > SRMMU_NOCACHE_ALIGN_MAX); From patchwork Tue Mar 24 10:40:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1260567 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=uUc/VB1j; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 48mp4Q3wDsz9sSM for ; Tue, 24 Mar 2020 21:52:54 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727332AbgCXKwy (ORCPT ); Tue, 24 Mar 2020 06:52:54 -0400 Received: from mail.kernel.org ([198.145.29.99]:33020 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726818AbgCXKwy (ORCPT ); Tue, 24 Mar 2020 06:52:54 -0400 Received: from localhost.localdomain (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 61041208D5; Tue, 24 Mar 2020 10:52:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585047172; bh=lQjYhjbRiE6BEEj3PAe1MR/PiYp8oE/mo+RRNJqgots=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uUc/VB1jMj/G1DyCf+3qoHyEZ4568/zF+TYQCgyjQ/xI7scbYT8mz3DT/io2/WJBv 7mcve4IRF0xhrBpOqBfzZHQ7GlZtGYO/PeVZPdAJBGX7GYWizsL6FObiQNGfT5c6at PdLquPhV/nkdURMR7fbvmgr8WDc7ZCtVm1+Gg318= From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , "David S. Miller" , Peter Zijlstra , Nick Desaulniers , Matt Fleming , sparclinux@vger.kernel.org Subject: [PATCH 2/4] sparc32: mm: Restructure sparc32 MMU page-table layout Date: Tue, 24 Mar 2020 10:40:03 +0000 Message-Id: <20200324104005.11279-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200324104005.11279-1-will@kernel.org> References: <20200324104005.11279-1-will@kernel.org> MIME-Version: 1.0 Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org The "SRMMU" supports 4k pages using a fixed three-level walk with a 256-entry PGD and 64-entry PMD/PTE levels. In order to fill a page with a 'pgtable_t', the SRMMU code allocates four native PTE tables into a single PTE allocation and similarly for the PMD level, leading to an array of 16 physical pointers in a 'pmd_t' This breaks the generic code which assumes READ_ONCE(*pmd) will be word sized. In a manner similar to ef22d8abd876 ("m68k: mm: Restructure Motorola MMU page-table layout"), this patch implements the native page-table setup directly. This significantly increases the page-table memory overhead, but will be addresses in a subsequent patch. Cc: "David S. Miller" Cc: Peter Zijlstra Signed-off-by: Will Deacon --- arch/sparc/include/asm/page_32.h | 10 ++--- arch/sparc/include/asm/pgalloc_32.h | 5 ++- arch/sparc/include/asm/pgtable_32.h | 29 +++++++------- arch/sparc/include/asm/pgtsrmmu.h | 36 ++--------------- arch/sparc/include/asm/viking.h | 5 ++- arch/sparc/kernel/head_32.S | 8 ++-- arch/sparc/mm/hypersparc.S | 3 +- arch/sparc/mm/srmmu.c | 60 ++++++++++------------------- arch/sparc/mm/viking.S | 5 ++- 9 files changed, 58 insertions(+), 103 deletions(-) diff --git a/arch/sparc/include/asm/page_32.h b/arch/sparc/include/asm/page_32.h index b76d59edec8c..e76e6c006e10 100644 --- a/arch/sparc/include/asm/page_32.h +++ b/arch/sparc/include/asm/page_32.h @@ -54,7 +54,7 @@ extern struct sparc_phys_banks sp_banks[SPARC_PHYS_BANKS+1]; */ typedef struct { unsigned long pte; } pte_t; typedef struct { unsigned long iopte; } iopte_t; -typedef struct { unsigned long pmdv[16]; } pmd_t; +typedef struct { unsigned long pmd; } pmd_t; typedef struct { unsigned long pgd; } pgd_t; typedef struct { unsigned long ctxd; } ctxd_t; typedef struct { unsigned long pgprot; } pgprot_t; @@ -62,7 +62,7 @@ typedef struct { unsigned long iopgprot; } iopgprot_t; #define pte_val(x) ((x).pte) #define iopte_val(x) ((x).iopte) -#define pmd_val(x) ((x).pmdv[0]) +#define pmd_val(x) ((x).pmd) #define pgd_val(x) ((x).pgd) #define ctxd_val(x) ((x).ctxd) #define pgprot_val(x) ((x).pgprot) @@ -82,7 +82,7 @@ typedef struct { unsigned long iopgprot; } iopgprot_t; */ typedef unsigned long pte_t; typedef unsigned long iopte_t; -typedef struct { unsigned long pmdv[16]; } pmd_t; +typedef unsigned long pmd_t; typedef unsigned long pgd_t; typedef unsigned long ctxd_t; typedef unsigned long pgprot_t; @@ -90,14 +90,14 @@ typedef unsigned long iopgprot_t; #define pte_val(x) (x) #define iopte_val(x) (x) -#define pmd_val(x) ((x).pmdv[0]) +#define pmd_val(x) (x) #define pgd_val(x) (x) #define ctxd_val(x) (x) #define pgprot_val(x) (x) #define iopgprot_val(x) (x) #define __pte(x) (x) -#define __pmd(x) ((pmd_t) { { (x) }, }) +#define __pmd(x) (x) #define __iopte(x) (x) #define __pgd(x) (x) #define __ctxd(x) (x) diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h index eae0c92ec422..99c032424946 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -60,13 +60,14 @@ pgtable_t pte_alloc_one(struct mm_struct *mm); static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { - return srmmu_get_nocache(PTE_SIZE, PTE_SIZE); + return srmmu_get_nocache(SRMMU_PTE_TABLE_SIZE, + SRMMU_PTE_TABLE_SIZE); } static inline void free_pte_fast(pte_t *pte) { - srmmu_free_nocache(pte, PTE_SIZE); + srmmu_free_nocache(pte, SRMMU_PTE_TABLE_SIZE); } #define pte_free_kernel(mm, pte) free_pte_fast(pte) diff --git a/arch/sparc/include/asm/pgtable_32.h b/arch/sparc/include/asm/pgtable_32.h index 6d6f44c0cad9..785dc9a95e70 100644 --- a/arch/sparc/include/asm/pgtable_32.h +++ b/arch/sparc/include/asm/pgtable_32.h @@ -11,6 +11,16 @@ #include +#define PMD_SHIFT 18 +#define PMD_SIZE (1UL << PMD_SHIFT) +#define PMD_MASK (~(PMD_SIZE-1)) +#define PMD_ALIGN(__addr) (((__addr) + ~PMD_MASK) & PMD_MASK) + +#define PGDIR_SHIFT 24 +#define PGDIR_SIZE (1UL << PGDIR_SHIFT) +#define PGDIR_MASK (~(PGDIR_SIZE-1)) +#define PGDIR_ALIGN(__addr) (((__addr) + ~PGDIR_MASK) & PGDIR_MASK) + #ifndef __ASSEMBLY__ #include @@ -34,17 +44,10 @@ unsigned long __init bootmem_init(unsigned long *pages_avail); #define pmd_ERROR(e) __builtin_trap() #define pgd_ERROR(e) __builtin_trap() -#define PMD_SHIFT 22 -#define PMD_SIZE (1UL << PMD_SHIFT) -#define PMD_MASK (~(PMD_SIZE-1)) -#define PMD_ALIGN(__addr) (((__addr) + ~PMD_MASK) & PMD_MASK) -#define PGDIR_SHIFT SRMMU_PGDIR_SHIFT -#define PGDIR_SIZE SRMMU_PGDIR_SIZE -#define PGDIR_MASK SRMMU_PGDIR_MASK -#define PTRS_PER_PTE 1024 -#define PTRS_PER_PMD SRMMU_PTRS_PER_PMD -#define PTRS_PER_PGD SRMMU_PTRS_PER_PGD -#define USER_PTRS_PER_PGD PAGE_OFFSET / SRMMU_PGDIR_SIZE +#define PTRS_PER_PTE 64 +#define PTRS_PER_PMD 64 +#define PTRS_PER_PGD 256 +#define USER_PTRS_PER_PGD PAGE_OFFSET / PGDIR_SIZE #define FIRST_USER_ADDRESS 0UL #define PTE_SIZE (PTRS_PER_PTE*4) @@ -179,9 +182,7 @@ static inline int pmd_none(pmd_t pmd) static inline void pmd_clear(pmd_t *pmdp) { - int i; - for (i = 0; i < PTRS_PER_PTE/SRMMU_REAL_PTRS_PER_PTE; i++) - set_pte((pte_t *)&pmdp->pmdv[i], __pte(0)); + set_pte((pte_t *)&pmd_val(*pmdp), __pte(0)); } static inline int pud_none(pud_t pud) diff --git a/arch/sparc/include/asm/pgtsrmmu.h b/arch/sparc/include/asm/pgtsrmmu.h index 32a508897501..58ea8e8c6ee7 100644 --- a/arch/sparc/include/asm/pgtsrmmu.h +++ b/arch/sparc/include/asm/pgtsrmmu.h @@ -17,39 +17,9 @@ /* Number of contexts is implementation-dependent; 64k is the most we support */ #define SRMMU_MAX_CONTEXTS 65536 -/* PMD_SHIFT determines the size of the area a second-level page table entry can map */ -#define SRMMU_REAL_PMD_SHIFT 18 -#define SRMMU_REAL_PMD_SIZE (1UL << SRMMU_REAL_PMD_SHIFT) -#define SRMMU_REAL_PMD_MASK (~(SRMMU_REAL_PMD_SIZE-1)) -#define SRMMU_REAL_PMD_ALIGN(__addr) (((__addr)+SRMMU_REAL_PMD_SIZE-1)&SRMMU_REAL_PMD_MASK) - -/* PGDIR_SHIFT determines what a third-level page table entry can map */ -#define SRMMU_PGDIR_SHIFT 24 -#define SRMMU_PGDIR_SIZE (1UL << SRMMU_PGDIR_SHIFT) -#define SRMMU_PGDIR_MASK (~(SRMMU_PGDIR_SIZE-1)) -#define SRMMU_PGDIR_ALIGN(addr) (((addr)+SRMMU_PGDIR_SIZE-1)&SRMMU_PGDIR_MASK) - -#define SRMMU_REAL_PTRS_PER_PTE 64 -#define SRMMU_REAL_PTRS_PER_PMD 64 -#define SRMMU_PTRS_PER_PGD 256 - -#define SRMMU_REAL_PTE_TABLE_SIZE (SRMMU_REAL_PTRS_PER_PTE*4) -#define SRMMU_PMD_TABLE_SIZE (SRMMU_REAL_PTRS_PER_PMD*4) -#define SRMMU_PGD_TABLE_SIZE (SRMMU_PTRS_PER_PGD*4) - -/* - * To support pagetables in highmem, Linux introduces APIs which - * return struct page* and generally manipulate page tables when - * they are not mapped into kernel space. Our hardware page tables - * are smaller than pages. We lump hardware tabes into big, page sized - * software tables. - * - * PMD_SHIFT determines the size of the area a second-level page table entry - * can map, and our pmd_t is 16 times larger than normal. The values which - * were once defined here are now generic for 4c and srmmu, so they're - * found in pgtable.h. - */ -#define SRMMU_PTRS_PER_PMD 4 +#define SRMMU_PTE_TABLE_SIZE (PAGE_SIZE) +#define SRMMU_PMD_TABLE_SIZE (PAGE_SIZE) +#define SRMMU_PGD_TABLE_SIZE (PTRS_PER_PGD*4) /* Definition of the values in the ET field of PTD's and PTE's */ #define SRMMU_ET_MASK 0x3 diff --git a/arch/sparc/include/asm/viking.h b/arch/sparc/include/asm/viking.h index 0bbefd184221..08ffc605035f 100644 --- a/arch/sparc/include/asm/viking.h +++ b/arch/sparc/include/asm/viking.h @@ -10,6 +10,7 @@ #include #include +#include #include /* Bits in the SRMMU control register for GNU/Viking modules. @@ -227,7 +228,7 @@ static inline unsigned long viking_hwprobe(unsigned long vaddr) : "=r" (val) : "r" (vaddr | 0x200), "i" (ASI_M_FLUSH_PROBE)); if ((val & SRMMU_ET_MASK) == SRMMU_ET_PTE) { - vaddr &= ~SRMMU_PGDIR_MASK; + vaddr &= ~PGDIR_MASK; vaddr >>= PAGE_SHIFT; return val | (vaddr << 8); } @@ -237,7 +238,7 @@ static inline unsigned long viking_hwprobe(unsigned long vaddr) : "=r" (val) : "r" (vaddr | 0x100), "i" (ASI_M_FLUSH_PROBE)); if ((val & SRMMU_ET_MASK) == SRMMU_ET_PTE) { - vaddr &= ~SRMMU_REAL_PMD_MASK; + vaddr &= ~PMD_MASK; vaddr >>= PAGE_SHIFT; return val | (vaddr << 8); } diff --git a/arch/sparc/kernel/head_32.S b/arch/sparc/kernel/head_32.S index e55f2c075165..be30c8d4cc73 100644 --- a/arch/sparc/kernel/head_32.S +++ b/arch/sparc/kernel/head_32.S @@ -24,7 +24,7 @@ #include #include /* TI_UWINMASK */ #include -#include /* SRMMU_PGDIR_SHIFT */ +#include /* PGDIR_SHIFT */ #include .data @@ -273,7 +273,7 @@ not_a_sun4: lda [%o1] ASI_M_BYPASS, %o2 ! This is the 0x0 16MB pgd /* Calculate to KERNBASE entry. */ - add %o1, KERNBASE >> (SRMMU_PGDIR_SHIFT - 2), %o3 + add %o1, KERNBASE >> (PGDIR_SHIFT - 2), %o3 /* Poke the entry into the calculated address. */ sta %o2, [%o3] ASI_M_BYPASS @@ -317,7 +317,7 @@ srmmu_not_viking: sll %g1, 0x8, %g1 ! make phys addr for l1 tbl lda [%g1] ASI_M_BYPASS, %g2 ! get level1 entry for 0x0 - add %g1, KERNBASE >> (SRMMU_PGDIR_SHIFT - 2), %g3 + add %g1, KERNBASE >> (PGDIR_SHIFT - 2), %g3 sta %g2, [%g3] ASI_M_BYPASS ! place at KERNBASE entry b go_to_highmem nop ! wheee.... @@ -341,7 +341,7 @@ leon_remap: sll %g1, 0x8, %g1 ! make phys addr for l1 tbl lda [%g1] ASI_M_BYPASS, %g2 ! get level1 entry for 0x0 - add %g1, KERNBASE >> (SRMMU_PGDIR_SHIFT - 2), %g3 + add %g1, KERNBASE >> (PGDIR_SHIFT - 2), %g3 sta %g2, [%g3] ASI_M_BYPASS ! place at KERNBASE entry b go_to_highmem nop ! wheee.... diff --git a/arch/sparc/mm/hypersparc.S b/arch/sparc/mm/hypersparc.S index 66885a8dc50a..6c2521e85a42 100644 --- a/arch/sparc/mm/hypersparc.S +++ b/arch/sparc/mm/hypersparc.S @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -293,7 +294,7 @@ hypersparc_flush_tlb_range: cmp %o3, -1 be hypersparc_flush_tlb_range_out #endif - sethi %hi(~((1 << SRMMU_PGDIR_SHIFT) - 1)), %o4 + sethi %hi(~((1 << PGDIR_SHIFT) - 1)), %o4 sta %o3, [%g1] ASI_M_MMUREGS and %o1, %o4, %o1 add %o1, 0x200, %o1 diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index a19863cac0c4..7fa096422bd0 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -136,26 +136,14 @@ static void msi_set_sync(void) void pmd_set(pmd_t *pmdp, pte_t *ptep) { - unsigned long ptp; /* Physical address, shifted right by 4 */ - int i; - - ptp = __nocache_pa(ptep) >> 4; - for (i = 0; i < PTRS_PER_PTE/SRMMU_REAL_PTRS_PER_PTE; i++) { - set_pte((pte_t *)&pmdp->pmdv[i], __pte(SRMMU_ET_PTD | ptp)); - ptp += (SRMMU_REAL_PTRS_PER_PTE * sizeof(pte_t) >> 4); - } + unsigned long ptp = __nocache_pa(ptep) >> 4; + set_pte((pte_t *)&pmd_val(*pmdp), __pte(SRMMU_ET_PTD | ptp)); } void pmd_populate(struct mm_struct *mm, pmd_t *pmdp, struct page *ptep) { - unsigned long ptp; /* Physical address, shifted right by 4 */ - int i; - - ptp = page_to_pfn(ptep) << (PAGE_SHIFT-4); /* watch for overflow */ - for (i = 0; i < PTRS_PER_PTE/SRMMU_REAL_PTRS_PER_PTE; i++) { - set_pte((pte_t *)&pmdp->pmdv[i], __pte(SRMMU_ET_PTD | ptp)); - ptp += (SRMMU_REAL_PTRS_PER_PTE * sizeof(pte_t) >> 4); - } + unsigned long ptp = page_to_pfn(ptep) << (PAGE_SHIFT-4); /* watch for overflow */ + set_pte((pte_t *)&pmd_val(*pmdp), __pte(SRMMU_ET_PTD | ptp)); } /* Find an entry in the third-level page table.. */ @@ -163,7 +151,7 @@ pte_t *pte_offset_kernel(pmd_t *dir, unsigned long address) { void *pte; - pte = __nocache_va((dir->pmdv[0] & SRMMU_PTD_PMASK) << 4); + pte = __nocache_va((pmd_val(*dir) & SRMMU_PTD_PMASK) << 4); return (pte_t *) pte + ((address >> PAGE_SHIFT) & (PTRS_PER_PTE - 1)); } @@ -400,7 +388,7 @@ void pte_free(struct mm_struct *mm, pgtable_t pte) p = page_to_pfn(pte) << PAGE_SHIFT; /* Physical address */ /* free non cached virtual address*/ - srmmu_free_nocache(__nocache_va(p), PTE_SIZE); + srmmu_free_nocache(__nocache_va(p), SRMMU_PTE_TABLE_SIZE); } /* context handling - a dynamically sized pool is used */ @@ -822,13 +810,13 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, what = 0; addr = start - PAGE_SIZE; - if (!(start & ~(SRMMU_REAL_PMD_MASK))) { - if (srmmu_probe(addr + SRMMU_REAL_PMD_SIZE) == probed) + if (!(start & ~(PMD_MASK))) { + if (srmmu_probe(addr + PMD_SIZE) == probed) what = 1; } - if (!(start & ~(SRMMU_PGDIR_MASK))) { - if (srmmu_probe(addr + SRMMU_PGDIR_SIZE) == probed) + if (!(start & ~(PGDIR_MASK))) { + if (srmmu_probe(addr + PGDIR_SIZE) == probed) what = 2; } @@ -837,7 +825,7 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, pudp = pud_offset(p4dp, start); if (what == 2) { *(pgd_t *)__nocache_fix(pgdp) = __pgd(probed); - start += SRMMU_PGDIR_SIZE; + start += PGDIR_SIZE; continue; } if (pud_none(*(pud_t *)__nocache_fix(pudp))) { @@ -849,6 +837,11 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, pud_set(__nocache_fix(pudp), pmdp); } pmdp = pmd_offset(__nocache_fix(pgdp), start); + if (what == 1) { + *(pmd_t *)__nocache_fix(pmdp) = __pmd(probed); + start += PMD_SIZE; + continue; + } if (srmmu_pmd_none(*(pmd_t *)__nocache_fix(pmdp))) { ptep = __srmmu_get_nocache(PTE_SIZE, PTE_SIZE); if (ptep == NULL) @@ -856,19 +849,6 @@ static void __init srmmu_inherit_prom_mappings(unsigned long start, memset(__nocache_fix(ptep), 0, PTE_SIZE); pmd_set(__nocache_fix(pmdp), ptep); } - if (what == 1) { - /* We bend the rule where all 16 PTPs in a pmd_t point - * inside the same PTE page, and we leak a perfectly - * good hardware PTE piece. Alternatives seem worse. - */ - unsigned int x; /* Index of HW PMD in soft cluster */ - unsigned long *val; - x = (start >> PMD_SHIFT) & 15; - val = &pmdp->pmdv[x]; - *(unsigned long *)__nocache_fix(val) = probed; - start += SRMMU_REAL_PMD_SIZE; - continue; - } ptep = pte_offset_kernel(__nocache_fix(pmdp), start); *(pte_t *)__nocache_fix(ptep) = __pte(probed); start += PAGE_SIZE; @@ -890,9 +870,9 @@ static void __init do_large_mapping(unsigned long vaddr, unsigned long phys_base /* Map sp_bank entry SP_ENTRY, starting at virtual address VBASE. */ static unsigned long __init map_spbank(unsigned long vbase, int sp_entry) { - unsigned long pstart = (sp_banks[sp_entry].base_addr & SRMMU_PGDIR_MASK); - unsigned long vstart = (vbase & SRMMU_PGDIR_MASK); - unsigned long vend = SRMMU_PGDIR_ALIGN(vbase + sp_banks[sp_entry].num_bytes); + unsigned long pstart = (sp_banks[sp_entry].base_addr & PGDIR_MASK); + unsigned long vstart = (vbase & PGDIR_MASK); + unsigned long vend = PGDIR_ALIGN(vbase + sp_banks[sp_entry].num_bytes); /* Map "low" memory only */ const unsigned long min_vaddr = PAGE_OFFSET; const unsigned long max_vaddr = PAGE_OFFSET + SRMMU_MAXMEM; @@ -905,7 +885,7 @@ static unsigned long __init map_spbank(unsigned long vbase, int sp_entry) while (vstart < vend) { do_large_mapping(vstart, pstart); - vstart += SRMMU_PGDIR_SIZE; pstart += SRMMU_PGDIR_SIZE; + vstart += PGDIR_SIZE; pstart += PGDIR_SIZE; } return vstart; } diff --git a/arch/sparc/mm/viking.S b/arch/sparc/mm/viking.S index adaef6e7b8cf..48f062de7a7f 100644 --- a/arch/sparc/mm/viking.S +++ b/arch/sparc/mm/viking.S @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -157,7 +158,7 @@ viking_flush_tlb_range: cmp %o3, -1 be 2f #endif - sethi %hi(~((1 << SRMMU_PGDIR_SHIFT) - 1)), %o4 + sethi %hi(~((1 << PGDIR_SHIFT) - 1)), %o4 sta %o3, [%g1] ASI_M_MMUREGS and %o1, %o4, %o1 add %o1, 0x200, %o1 @@ -243,7 +244,7 @@ sun4dsmp_flush_tlb_range: ld [%o0 + VMA_VM_MM], %o0 ld [%o0 + AOFF_mm_context], %o3 lda [%g1] ASI_M_MMUREGS, %g5 - sethi %hi(~((1 << SRMMU_PGDIR_SHIFT) - 1)), %o4 + sethi %hi(~((1 << PGDIR_SHIFT) - 1)), %o4 sta %o3, [%g1] ASI_M_MMUREGS and %o1, %o4, %o1 add %o1, 0x200, %o1 From patchwork Tue Mar 24 10:40:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1260568 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=AMPZKMFs; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 48mp4W5NpMz9sQt for ; Tue, 24 Mar 2020 21:52:59 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727446AbgCXKw4 (ORCPT ); Tue, 24 Mar 2020 06:52:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:33068 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726818AbgCXKwz (ORCPT ); Tue, 24 Mar 2020 06:52:55 -0400 Received: from localhost.localdomain (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5A1D620786; Tue, 24 Mar 2020 10:52:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585047174; bh=HNQfhXx51iW1BFTByuYOqMiuB0J3MOmgoL1IZBf+yd8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AMPZKMFsRBI7UURwUjnbgiAt8skXpRRCOBwwjro67ukTAliBwp6IuV8gNWsTApc30 ftYsUpWep8+YfoLbRs8hPWUDKhyUoHK/5DOUgpaR7hn7MyjWPn8MqT87e4aUuc1W2W oBHcj4JqHd5xpZ/+2Gt79p1XB2Y9GxYyfYTI9tIM= From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , "David S. Miller" , Peter Zijlstra , Nick Desaulniers , Matt Fleming , sparclinux@vger.kernel.org Subject: [PATCH 3/4] sparc32: mm: Change pgtable_t type to pte_t * instead of struct page * Date: Tue, 24 Mar 2020 10:40:04 +0000 Message-Id: <20200324104005.11279-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200324104005.11279-1-will@kernel.org> References: <20200324104005.11279-1-will@kernel.org> MIME-Version: 1.0 Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org Change the 'pgtable_t' type for sparc32 so that it represents the uncached virtual address of the PTE table, rather than the underlying 'struct page'. This allows us to free page table allocations smaller than a page. Cc: "David S. Miller" Cc: Peter Zijlstra Signed-off-by: Will Deacon --- arch/sparc/include/asm/page_32.h | 2 +- arch/sparc/include/asm/pgalloc_32.h | 6 +++--- arch/sparc/include/asm/pgtable_32.h | 11 +++++++++++ arch/sparc/mm/srmmu.c | 29 +++++++++-------------------- 4 files changed, 24 insertions(+), 24 deletions(-) diff --git a/arch/sparc/include/asm/page_32.h b/arch/sparc/include/asm/page_32.h index e76e6c006e10..721fa99abc73 100644 --- a/arch/sparc/include/asm/page_32.h +++ b/arch/sparc/include/asm/page_32.h @@ -106,7 +106,7 @@ typedef unsigned long iopgprot_t; #endif -typedef struct page *pgtable_t; +typedef pte_t *pgtable_t; #define TASK_UNMAPPED_BASE 0x50000000 diff --git a/arch/sparc/include/asm/pgalloc_32.h b/arch/sparc/include/asm/pgalloc_32.h index 99c032424946..b772384871e9 100644 --- a/arch/sparc/include/asm/pgalloc_32.h +++ b/arch/sparc/include/asm/pgalloc_32.h @@ -50,11 +50,11 @@ static inline void free_pmd_fast(pmd_t * pmd) #define pmd_free(mm, pmd) free_pmd_fast(pmd) #define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) -void pmd_populate(struct mm_struct *mm, pmd_t *pmdp, struct page *ptep); -#define pmd_pgtable(pmd) pmd_page(pmd) +#define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) +#define pmd_pgtable(pmd) (pgtable_t)__pmd_page(pmd) void pmd_set(pmd_t *pmdp, pte_t *ptep); -#define pmd_populate_kernel(MM, PMD, PTE) pmd_set(PMD, PTE) +#define pmd_populate_kernel pmd_populate pgtable_t pte_alloc_one(struct mm_struct *mm); diff --git a/arch/sparc/include/asm/pgtable_32.h b/arch/sparc/include/asm/pgtable_32.h index 785dc9a95e70..03f4b84dc8fd 100644 --- a/arch/sparc/include/asm/pgtable_32.h +++ b/arch/sparc/include/asm/pgtable_32.h @@ -135,6 +135,17 @@ static inline struct page *pmd_page(pmd_t pmd) return pfn_to_page((pmd_val(pmd) & SRMMU_PTD_PMASK) >> (PAGE_SHIFT-4)); } +static inline unsigned long __pmd_page(pmd_t pmd) +{ + unsigned long v; + + if (srmmu_device_memory(pmd_val(pmd))) + BUG(); + + v = pmd_val(pmd) & SRMMU_PTD_PMASK; + return (unsigned long)__nocache_va(v << 4); +} + static inline unsigned long pud_page_vaddr(pud_t pud) { if (srmmu_device_memory(pud_val(pud))) { diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index 7fa096422bd0..dfd883b53054 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -140,12 +140,6 @@ void pmd_set(pmd_t *pmdp, pte_t *ptep) set_pte((pte_t *)&pmd_val(*pmdp), __pte(SRMMU_ET_PTD | ptp)); } -void pmd_populate(struct mm_struct *mm, pmd_t *pmdp, struct page *ptep) -{ - unsigned long ptp = page_to_pfn(ptep) << (PAGE_SHIFT-4); /* watch for overflow */ - set_pte((pte_t *)&pmd_val(*pmdp), __pte(SRMMU_ET_PTD | ptp)); -} - /* Find an entry in the third-level page table.. */ pte_t *pte_offset_kernel(pmd_t *dir, unsigned long address) { @@ -364,31 +358,26 @@ pgd_t *get_pgd_fast(void) */ pgtable_t pte_alloc_one(struct mm_struct *mm) { - unsigned long pte; + pte_t *ptep; struct page *page; - if ((pte = (unsigned long)pte_alloc_one_kernel(mm)) == 0) + if ((ptep = pte_alloc_one_kernel(mm)) == 0) return NULL; - page = pfn_to_page(__nocache_pa(pte) >> PAGE_SHIFT); + page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); if (!pgtable_pte_page_ctor(page)) { __free_page(page); return NULL; } - return page; + return ptep; } -void pte_free(struct mm_struct *mm, pgtable_t pte) +void pte_free(struct mm_struct *mm, pgtable_t ptep) { - unsigned long p; - - pgtable_pte_page_dtor(pte); - p = (unsigned long)page_address(pte); /* Cached address (for test) */ - if (p == 0) - BUG(); - p = page_to_pfn(pte) << PAGE_SHIFT; /* Physical address */ + struct page *page; - /* free non cached virtual address*/ - srmmu_free_nocache(__nocache_va(p), SRMMU_PTE_TABLE_SIZE); + page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); + pgtable_pte_page_dtor(page); + srmmu_free_nocache(ptep, SRMMU_PTE_TABLE_SIZE); } /* context handling - a dynamically sized pool is used */ From patchwork Tue Mar 24 10:40:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1260569 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=vger.kernel.org (client-ip=209.132.180.67; helo=vger.kernel.org; envelope-from=sparclinux-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=default header.b=B9iQT6/H; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 48mp4b14sbz9sSN for ; Tue, 24 Mar 2020 21:53:03 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726818AbgCXKw7 (ORCPT ); Tue, 24 Mar 2020 06:52:59 -0400 Received: from mail.kernel.org ([198.145.29.99]:33126 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727444AbgCXKw4 (ORCPT ); Tue, 24 Mar 2020 06:52:56 -0400 Received: from localhost.localdomain (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2CBB1208D6; Tue, 24 Mar 2020 10:52:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1585047176; bh=4AeA8qWdzDTXrdHBQNY6oqVXBpONA5xeP0MnGuOm3ws=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B9iQT6/H/5Xzq0o5fj8/mJPPykRqVvFDZd+Iu3LD/DFJ2124mZU0uBeaVbbb3zEEr PwjG8wArYlccQ+1WL4hCBdLvOYHbwLSHf05qoj+foYATyS+3w6+GlXPr8S3LFfydPd LiWJRA0Jiku5UnMwGZq5U3lutrGCxpLUHeoWu0vg= From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , "David S. Miller" , Peter Zijlstra , Nick Desaulniers , Matt Fleming , sparclinux@vger.kernel.org Subject: [PATCH 4/4] sparc32: mm: Reduce allocation size for PMD and PTE tables Date: Tue, 24 Mar 2020 10:40:05 +0000 Message-Id: <20200324104005.11279-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200324104005.11279-1-will@kernel.org> References: <20200324104005.11279-1-will@kernel.org> MIME-Version: 1.0 Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org Now that the page table allocator can free page table allocations smaller than PAGE_SIZE, reduce the size of the PMD and PTE allocations to avoid needlessly wasting memory. Cc: "David S. Miller" Cc: Peter Zijlstra Signed-off-by: Will Deacon --- arch/sparc/include/asm/pgtsrmmu.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/sparc/include/asm/pgtsrmmu.h b/arch/sparc/include/asm/pgtsrmmu.h index 58ea8e8c6ee7..7708d015712b 100644 --- a/arch/sparc/include/asm/pgtsrmmu.h +++ b/arch/sparc/include/asm/pgtsrmmu.h @@ -17,8 +17,8 @@ /* Number of contexts is implementation-dependent; 64k is the most we support */ #define SRMMU_MAX_CONTEXTS 65536 -#define SRMMU_PTE_TABLE_SIZE (PAGE_SIZE) -#define SRMMU_PMD_TABLE_SIZE (PAGE_SIZE) +#define SRMMU_PTE_TABLE_SIZE (PTRS_PER_PTE*4) +#define SRMMU_PMD_TABLE_SIZE (PTRS_PER_PMD*4) #define SRMMU_PGD_TABLE_SIZE (PTRS_PER_PGD*4) /* Definition of the values in the ET field of PTD's and PTE's */