From patchwork Fri Dec 16 18:35:27 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 706532 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 3tgJxm1fJxz9t0v for ; Sat, 17 Dec 2016 05:39:24 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758180AbcLPSjW (ORCPT ); Fri, 16 Dec 2016 13:39:22 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:44599 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758353AbcLPSgJ (ORCPT ); Fri, 16 Dec 2016 13:36:09 -0500 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id uBGIZxFL029801 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 16 Dec 2016 18:35:59 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserv0021.oracle.com (8.13.8/8.14.4) with ESMTP id uBGIZxPn021701 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 16 Dec 2016 18:35:59 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id uBGIZtx9025341; Fri, 16 Dec 2016 18:35:55 GMT Received: from monkey.oracle.com (/50.188.161.229) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 16 Dec 2016 10:35:54 -0800 From: Mike Kravetz To: sparclinux@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "David S . Miller" , Bob Picco , Nitin Gupta , Vijay Kumar , Julian Calaby , Adam Buchbinder , "Kirill A . Shutemov" , Michal Hocko , Andrew Morton , Mike Kravetz Subject: [RFC PATCH 04/14] sparc64: load shared id into context register 1 Date: Fri, 16 Dec 2016 10:35:27 -0800 Message-Id: <1481913337-9331-5-git-send-email-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1481913337-9331-1-git-send-email-mike.kravetz@oracle.com> References: <1481913337-9331-1-git-send-email-mike.kravetz@oracle.com> X-Source-IP: aserv0021.oracle.com [141.146.126.233] Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org In current code, only context ID register 0 is set and used by the MMU. On sun4v platforms that support MMU shared context, there is an additional context ID register: specifically context register 1. When searching the TLB, the MMU will find a match if the virtual address matches and the ID contained in context register 0 -OR- context register 1 matches. Load the shared context ID into context ID register 1. Care must be taken to load register 1 after register 0, as loading register 0 overwrites both register 0 and 1. Modify code loading register 0 to also load register one if applicable. Signed-off-by: Mike Kravetz --- arch/sparc/include/asm/mmu_context_64.h | 37 +++++++++++++++++-- arch/sparc/include/asm/spitfire.h | 2 ++ arch/sparc/kernel/fpu_traps.S | 63 +++++++++++++++++++++++++++++++++ arch/sparc/kernel/rtrap_64.S | 20 +++++++++++ arch/sparc/kernel/trampoline_64.S | 20 +++++++++++ 5 files changed, 140 insertions(+), 2 deletions(-) diff --git a/arch/sparc/include/asm/mmu_context_64.h b/arch/sparc/include/asm/mmu_context_64.h index acaea6d..84268df 100644 --- a/arch/sparc/include/asm/mmu_context_64.h +++ b/arch/sparc/include/asm/mmu_context_64.h @@ -61,8 +61,11 @@ void smp_tsb_sync(struct mm_struct *mm); #define smp_tsb_sync(__mm) do { } while (0) #endif -/* Set MMU context in the actual hardware. */ -#define load_secondary_context(__mm) \ +/* + * Set MMU context in the actual hardware. Secondary context register + * zero is loaded with task specific context. + */ +#define load_secondary_context_0(__mm) \ __asm__ __volatile__( \ "\n661: stxa %0, [%1] %2\n" \ " .section .sun4v_1insn_patch, \"ax\"\n" \ @@ -74,6 +77,36 @@ void smp_tsb_sync(struct mm_struct *mm); : "r" (CTX_HWBITS((__mm)->context)), \ "r" (SECONDARY_CONTEXT), "i" (ASI_DMMU), "i" (ASI_MMU)) +/* + * Secondary context register one is loaded with shared context if + * it exists for the task. + */ +#define load_secondary_context_1(__mm) \ + __asm__ __volatile__( \ + "\n661: stxa %0, [%1] %2\n" \ + " .section .sun4v_1insn_patch, \"ax\"\n" \ + " .word 661b\n" \ + " stxa %0, [%1] %3\n" \ + " .previous\n" \ + " flush %%g6\n" \ + : /* No outputs */ \ + : "r" (SHARED_CTX_HWBITS((__mm)->context)), \ + "r" (SECONDARY_CONTEXT_R1), "i" (ASI_DMMU), "i" (ASI_MMU)) + +#if defined(CONFIG_SHARED_MMU_CTX) +#define load_secondary_context(__mm) \ + do { \ + load_secondary_context_0(__mm); \ + if ((__mm)->context.shared_ctx) \ + load_secondary_context_1(__mm); \ + } while (0) +#else +#define load_secondary_context(__mm) \ + do { \ + load_secondary_context_0(__mm); \ + } while (0) +#endif + void __flush_tlb_mm(unsigned long, unsigned long); /* Switch the current MM context. */ diff --git a/arch/sparc/include/asm/spitfire.h b/arch/sparc/include/asm/spitfire.h index 1d8321c..1fa4594 100644 --- a/arch/sparc/include/asm/spitfire.h +++ b/arch/sparc/include/asm/spitfire.h @@ -33,6 +33,8 @@ #define DMMU_SFAR 0x0000000000000020 #define VIRT_WATCHPOINT 0x0000000000000038 #define PHYS_WATCHPOINT 0x0000000000000040 +#define PRIMARY_CONTEXT_R1 0x0000000000000108 +#define SECONDARY_CONTEXT_R1 0x0000000000000110 #define SPITFIRE_HIGHEST_LOCKED_TLBENT (64 - 1) #define CHEETAH_HIGHEST_LOCKED_TLBENT (16 - 1) diff --git a/arch/sparc/kernel/fpu_traps.S b/arch/sparc/kernel/fpu_traps.S index 336d275..f85a034 100644 --- a/arch/sparc/kernel/fpu_traps.S +++ b/arch/sparc/kernel/fpu_traps.S @@ -73,6 +73,16 @@ do_fpdis: ldxa [%g3] ASI_MMU, %g5 .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g3 + ldxa [%g3] ASI_MMU, %g4 + .previous + /* Unnecessary on sun4u and pre-Niagara 2 sun4v */ + mov SECONDARY_CONTEXT, %g3 + sethi %hi(sparc64_kern_sec_context), %g2 ldx [%g2 + %lo(sparc64_kern_sec_context)], %g2 @@ -114,6 +124,16 @@ do_fpdis: ldxa [%g3] ASI_MMU, %g5 .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g3 + ldxa [%g3] ASI_MMU, %g4 + .previous + /* Unnecessary on sun4u and pre-Niagara 2 sun4v */ + mov SECONDARY_CONTEXT, %g3 + add %g6, TI_FPREGS, %g1 sethi %hi(sparc64_kern_sec_context), %g2 ldx [%g2 + %lo(sparc64_kern_sec_context)], %g2 @@ -155,6 +175,16 @@ do_fpdis: ldxa [%g3] ASI_MMU, %g5 .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g3 + ldxa [%g3] ASI_MMU, %g4 + .previous + /* Unnecessary on sun4u and pre-Niagara 2 sun4v */ + mov SECONDARY_CONTEXT, %g3 + sethi %hi(sparc64_kern_sec_context), %g2 ldx [%g2 + %lo(sparc64_kern_sec_context)], %g2 @@ -181,11 +211,24 @@ fpdis_exit: stxa %g5, [%g3] ASI_MMU .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g3 + stxa %g4, [%g3] ASI_MMU + .previous + membar #Sync fpdis_exit2: wr %g7, 0, %gsr ldx [%g6 + TI_XFSR], %fsr rdpr %tstate, %g3 +661: nop + .section .sun4v_1insn_patch, "ax" + .word 661b + sethi %hi(TSTATE_PEF), %g4 + .previous or %g3, %g4, %g3 ! anal... wrpr %g3, %tstate wr %g0, FPRS_FEF, %fprs ! clean DU/DL bits @@ -347,6 +390,16 @@ do_fptrap_after_fsr: ldxa [%g3] ASI_MMU, %g5 .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g3 + ldxa [%g3] ASI_MMU, %g4 + .previous + /* Unnecessary on sun4u and pre-Niagara 2 sun4v */ + mov SECONDARY_CONTEXT, %g3 + sethi %hi(sparc64_kern_sec_context), %g2 ldx [%g2 + %lo(sparc64_kern_sec_context)], %g2 @@ -377,7 +430,17 @@ do_fptrap_after_fsr: stxa %g5, [%g1] ASI_MMU .previous +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g1 + stxa %g4, [%g1] ASI_MMU + .previous + membar #Sync + /* Unnecessary on sun4u and pre-Niagara 2 sun4v */ + mov SECONDARY_CONTEXT, %g1 ba,pt %xcc, etrap wr %g0, 0, %fprs .size do_fptrap,.-do_fptrap diff --git a/arch/sparc/kernel/rtrap_64.S b/arch/sparc/kernel/rtrap_64.S index 216948c..d409d84 100644 --- a/arch/sparc/kernel/rtrap_64.S +++ b/arch/sparc/kernel/rtrap_64.S @@ -202,6 +202,7 @@ rt_continue: ldx [%sp + PTREGS_OFF + PT_V9_G1], %g1 brnz,pn %l3, kern_rtt mov PRIMARY_CONTEXT, %l7 + /* Get value from SECONDARY_CONTEXT register */ 661: ldxa [%l7 + %l7] ASI_DMMU, %l0 .section .sun4v_1insn_patch, "ax" .word 661b @@ -212,12 +213,31 @@ rt_continue: ldx [%sp + PTREGS_OFF + PT_V9_G1], %g1 ldx [%l1 + %lo(sparc64_kern_pri_nuc_bits)], %l1 or %l0, %l1, %l0 + /* and, put into PRIMARY_CONTEXT register */ 661: stxa %l0, [%l7] ASI_DMMU .section .sun4v_1insn_patch, "ax" .word 661b stxa %l0, [%l7] ASI_MMU .previous + /* Get value from SECONDARY_CONTEXT_R1 register */ +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %l7 + ldxa [%l7] ASI_MMU, %l0 + .previous + + /* and, put into PRIMARY_CONTEXT_R1 register */ +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov PRIMARY_CONTEXT_R1, %l7 + stxa %l0, [%l7] ASI_MMU + .previous + sethi %hi(KERNBASE), %l7 flush %l7 rdpr %wstate, %l1 diff --git a/arch/sparc/kernel/trampoline_64.S b/arch/sparc/kernel/trampoline_64.S index 88ede1d..7c4ab3b 100644 --- a/arch/sparc/kernel/trampoline_64.S +++ b/arch/sparc/kernel/trampoline_64.S @@ -260,6 +260,16 @@ after_lock_tlb: stxa %g0, [%g7] ASI_MMU .previous + /* Save SECONDARY_CONTEXT_R1, membar should be part of patch */ + membar #Sync +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g7 + ldxa [%g7] ASI_MMU, %g1 + .previous + membar #Sync mov SECONDARY_CONTEXT, %g7 @@ -269,6 +279,16 @@ after_lock_tlb: stxa %g0, [%g7] ASI_MMU .previous + /* Restore SECONDARY_CONTEXT_R1, membar should be part of patch */ + membar #Sync +661: nop + nop + .section .sun4v_2insn_patch, "ax" + .word 661b + mov SECONDARY_CONTEXT_R1, %g7 + stxa %g1, [%g7] ASI_MMU + .previous + membar #Sync /* Everything we do here, until we properly take over the