From patchwork Fri Mar 14 00:00:44 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Scott Wood X-Patchwork-Id: 330154 X-Patchwork-Delegate: scottwood@freescale.com Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from ozlabs.org (localhost [IPv6:::1]) by ozlabs.org (Postfix) with ESMTP id D738D2C0914 for ; Fri, 14 Mar 2014 11:05:22 +1100 (EST) Received: from na01-bl2-obe.outbound.protection.outlook.com (mail-bl2lp0206.outbound.protection.outlook.com [207.46.163.206]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 99C5F2C0114 for ; Fri, 14 Mar 2014 11:01:18 +1100 (EST) Received: from snotra.am.freescale.net (192.88.168.49) by BY2PR03MB395.namprd03.prod.outlook.com (10.141.141.14) with Microsoft SMTP Server (TLS) id 15.0.898.11; Fri, 14 Mar 2014 00:01:13 +0000 From: Scott Wood To: Benjamin Herrenschmidt Subject: [PATCH 05/10] powerpc/booke64: Use SPRG7 for VDSO Date: Thu, 13 Mar 2014 19:00:44 -0500 Message-ID: <1394755249-8856-6-git-send-email-scottwood@freescale.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1394755249-8856-1-git-send-email-scottwood@freescale.com> References: <1394755249-8856-1-git-send-email-scottwood@freescale.com> MIME-Version: 1.0 X-Originating-IP: [192.88.168.49] X-ClientProxiedBy: DM2PR04CA004.namprd04.prod.outlook.com (10.141.96.14) To BY2PR03MB395.namprd03.prod.outlook.com (10.141.141.14) X-Forefront-PRVS: 0150F3F97D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009001)(6009001)(428001)(199002)(189002)(90146001)(56816005)(92726001)(33646001)(76482001)(42186004)(51856001)(92566001)(53806001)(4396001)(50226001)(83072002)(47976001)(46102001)(49866001)(47736001)(63696002)(85852003)(50986001)(97186001)(47776003)(65816001)(97336001)(20776003)(89996001)(80022001)(66066001)(79102001)(54316002)(56776001)(59766001)(95666003)(77982001)(94316002)(86362001)(575784001)(93516002)(95416001)(94946001)(36756003)(76786001)(77096001)(77156001)(76796001)(81342001)(69226001)(62966002)(81542001)(93136001)(50466002)(74876001)(74706001)(551934002)(74502001)(47446002)(31966008)(74662001)(48376002)(74366001)(81686001)(19580405001)(83322001)(19580395003)(80976001)(81816001)(87286001)(87976001)(87266001)(85306002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR03MB395; H:snotra.am.freescale.net; FPR:E721F6FF.E03E9D88.E6F3134E.40C96AB6.20863; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (: freescale.com does not designate permitted sender hosts) X-OriginatorOrg: freescale.com Cc: kvm-ppc@vger.kernel.org, Tiejun Chen , Paul Mackerras , Anton Blanchard , Scott Wood , Mihai Caraman , linuxppc-dev@lists.ozlabs.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Previously SPRG3 was marked for use by both VDSO and critical interrupts (though critical interrupts were not fully implemented). In commit 8b64a9dfb091f1eca8b7e58da82f1e7d1d5fe0ad ("powerpc/booke64: Use SPRG0/3 scratch for bolted TLB miss & crit int"), Mihai Caraman made an attempt to resolve this conflict by restoring the VDSO value early in the critical interrupt, but this has some issues: - It's incompatible with EXCEPTION_COMMON which restores r13 from the by-then-overwritten scratch (this cost me some debugging time). - It forces critical exceptions to be a special case handled differently from even machine check and debug level exceptions. - It didn't occur to me that it was possible to make this work at all (by doing a final "ld r13, PACA_EXCRIT+EX_R13(r13)") until after I made (most of) this patch. :-) It might be worth investigating using a load rather than SPRG on return from all exceptions (except TLB misses where the scratch never leaves the SPRG) -- it could save a few cycles. Until then, let's stick with SPRG for all exceptions. Since we cannot use SPRG4-7 for scratch without corrupting the state of a KVM guest, move VDSO to SPRG7 on book3e. Since neither SPRG4-7 nor critical interrupts exist on book3s, SPRG3 is still used for VDSO there. Signed-off-by: Scott Wood Cc: Mihai Caraman Cc: Anton Blanchard Cc: Paul Mackerras Cc: kvm-ppc@vger.kernel.org --- arch/powerpc/include/asm/exception-64e.h | 5 ++--- arch/powerpc/include/asm/kvm_booke_hv_asm.h | 9 +-------- arch/powerpc/include/asm/paca.h | 2 +- arch/powerpc/include/asm/reg.h | 13 ++++++++++--- arch/powerpc/kernel/asm-offsets.c | 2 +- arch/powerpc/kernel/exceptions-64e.S | 19 ++----------------- arch/powerpc/kernel/vdso.c | 8 ++++---- arch/powerpc/kernel/vdso32/getcpu.S | 2 +- arch/powerpc/kernel/vdso64/getcpu.S | 2 +- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 4 ++-- arch/powerpc/kvm/book3s_interrupts.S | 4 ++-- arch/powerpc/kvm/bookehv_interrupts.S | 10 ++++++---- 12 files changed, 33 insertions(+), 47 deletions(-) diff --git a/arch/powerpc/include/asm/exception-64e.h b/arch/powerpc/include/asm/exception-64e.h index 51fa43e..e73452f 100644 --- a/arch/powerpc/include/asm/exception-64e.h +++ b/arch/powerpc/include/asm/exception-64e.h @@ -46,9 +46,8 @@ #define EX_CR (1 * 8) #define EX_R10 (2 * 8) #define EX_R11 (3 * 8) -#define EX_R13 (4 * 8) -#define EX_R14 (5 * 8) -#define EX_R15 (6 * 8) +#define EX_R14 (4 * 8) +#define EX_R15 (5 * 8) /* * The TLB miss exception uses different slots. diff --git a/arch/powerpc/include/asm/kvm_booke_hv_asm.h b/arch/powerpc/include/asm/kvm_booke_hv_asm.h index 3a79f53..c3e3fd5 100644 --- a/arch/powerpc/include/asm/kvm_booke_hv_asm.h +++ b/arch/powerpc/include/asm/kvm_booke_hv_asm.h @@ -36,20 +36,13 @@ * *(r8 + GPR11) = saved r11 * * 64-bit host - * Expected inputs (GEN/GDBELL/DBG/MC exception types): + * Expected inputs (GEN/GDBELL/DBG/CRIT/MC exception types): * r10 = saved CR * r13 = PACA_POINTER * *(r13 + PACA_EX##type + EX_R10) = saved r10 * *(r13 + PACA_EX##type + EX_R11) = saved r11 * SPRN_SPRG_##type##_SCRATCH = saved r13 * - * Expected inputs (CRIT exception type): - * r10 = saved CR - * r13 = PACA_POINTER - * *(r13 + PACA_EX##type + EX_R10) = saved r10 - * *(r13 + PACA_EX##type + EX_R11) = saved r11 - * *(r13 + PACA_EX##type + EX_R13) = saved r13 - * * Expected inputs (TLB exception type): * r10 = saved CR * r13 = PACA_POINTER diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h index 9c5dbc3..948f01a 100644 --- a/arch/powerpc/include/asm/paca.h +++ b/arch/powerpc/include/asm/paca.h @@ -146,7 +146,7 @@ struct paca_struct { u8 io_sync; /* writel() needs spin_unlock sync */ u8 irq_work_pending; /* IRQ_WORK interrupt while soft-disable */ u8 nap_state_lost; /* NV GPR values lost in power7_idle */ - u64 sprg3; /* Saved user-visible sprg */ + u64 sprg_vdso; /* Saved user-visible sprg */ #ifdef CONFIG_PPC_TRANSACTIONAL_MEM u64 tm_scratch; /* TM scratch area for reclaim */ #endif diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h index 90c06ec..b9ac329 100644 --- a/arch/powerpc/include/asm/reg.h +++ b/arch/powerpc/include/asm/reg.h @@ -577,9 +577,13 @@ #define SPRN_SPRG3 0x113 /* Special Purpose Register General 3 */ #define SPRN_USPRG3 0x103 /* SPRG3 userspace read */ #define SPRN_SPRG4 0x114 /* Special Purpose Register General 4 */ +#define SPRN_USPRG4 0x104 /* SPRG4 userspace read */ #define SPRN_SPRG5 0x115 /* Special Purpose Register General 5 */ +#define SPRN_USPRG5 0x105 /* SPRG5 userspace read */ #define SPRN_SPRG6 0x116 /* Special Purpose Register General 6 */ +#define SPRN_USPRG6 0x106 /* SPRG6 userspace read */ #define SPRN_SPRG7 0x117 /* Special Purpose Register General 7 */ +#define SPRN_USPRG7 0x107 /* SPRG7 userspace read */ #define SPRN_SRR0 0x01A /* Save/Restore Register 0 */ #define SPRN_SRR1 0x01B /* Save/Restore Register 1 */ #define SRR1_ISI_NOPT 0x40000000 /* ISI: Not found in hash */ @@ -879,11 +883,10 @@ * 64-bit embedded * - SPRG0 generic exception scratch * - SPRG2 TLB exception stack - * - SPRG3 critical exception scratch and - * CPU and NUMA node for VDSO getcpu (user visible) + * - SPRG3 critical exception scratch (user visible, sorry!) * - SPRG4 unused (user visible) * - SPRG6 TLB miss scratch (user visible, sorry !) - * - SPRG7 critical exception scratch + * - SPRG7 CPU and NUMA node for VDSO getcpu (user visible) * - SPRG8 machine check exception scratch * - SPRG9 debug exception scratch * @@ -940,6 +943,8 @@ #define SPRN_SPRG_SCRATCH0 SPRN_SPRG2 #define SPRN_SPRG_HPACA SPRN_HSPRG0 #define SPRN_SPRG_HSCRATCH0 SPRN_HSPRG1 +#define SPRN_SPRG_VDSO_READ SPRN_USPRG3 +#define SPRN_SPRG_VDSO_WRITE SPRN_SPRG3 #define GET_PACA(rX) \ BEGIN_FTR_SECTION_NESTED(66); \ @@ -983,6 +988,8 @@ #define SPRN_SPRG_TLB_SCRATCH SPRN_SPRG6 #define SPRN_SPRG_GEN_SCRATCH SPRN_SPRG0 #define SPRN_SPRG_GDBELL_SCRATCH SPRN_SPRG_GEN_SCRATCH +#define SPRN_SPRG_VDSO_READ SPRN_USPRG7 +#define SPRN_SPRG_VDSO_WRITE SPRN_SPRG7 #define SET_PACA(rX) mtspr SPRN_SPRG_PACA,rX #define GET_PACA(rX) mfspr rX,SPRN_SPRG_PACA diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c index b5aacf7..dba8140 100644 --- a/arch/powerpc/kernel/asm-offsets.c +++ b/arch/powerpc/kernel/asm-offsets.c @@ -253,7 +253,7 @@ int main(void) DEFINE(PACA_SYSTEM_TIME, offsetof(struct paca_struct, system_time)); DEFINE(PACA_TRAP_SAVE, offsetof(struct paca_struct, trap_save)); DEFINE(PACA_NAPSTATELOST, offsetof(struct paca_struct, nap_state_lost)); - DEFINE(PACA_SPRG3, offsetof(struct paca_struct, sprg3)); + DEFINE(PACA_SPRG_VDSO, offsetof(struct paca_struct, sprg_vdso)); #endif /* CONFIG_PPC64 */ /* RTAS */ diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S index 41380a4..89e1133 100644 --- a/arch/powerpc/kernel/exceptions-64e.S +++ b/arch/powerpc/kernel/exceptions-64e.S @@ -55,7 +55,6 @@ mfspr r13,SPRN_SPRG_PACA; /* get PACA */ \ std r10,PACA_EX##type+EX_R10(r13); \ std r11,PACA_EX##type+EX_R11(r13); \ - PROLOG_STORE_RESTORE_SCRATCH_##type; \ mfcr r10; /* save CR */ \ mfspr r11,SPRN_##type##_SRR1;/* what are we coming from */ \ DO_KVM intnum,SPRN_##type##_SRR1; /* KVM hook */ \ @@ -116,20 +115,6 @@ #define GDBELL_EXCEPTION_PROLOG(n, intnum, addition) \ EXCEPTION_PROLOG(n, intnum, GDBELL, addition##_GDBELL(n)) -/* - * Store user-visible scratch in PACA exception slots and restore proper value - */ -#define PROLOG_STORE_RESTORE_SCRATCH_GEN -#define PROLOG_STORE_RESTORE_SCRATCH_GDBELL -#define PROLOG_STORE_RESTORE_SCRATCH_DBG -#define PROLOG_STORE_RESTORE_SCRATCH_MC - -#define PROLOG_STORE_RESTORE_SCRATCH_CRIT \ - mfspr r10,SPRN_SPRG_CRIT_SCRATCH; /* get r13 */ \ - std r10,PACA_EXCRIT+EX_R13(r13); \ - ld r11,PACA_SPRG3(r13); \ - mtspr SPRN_SPRG_CRIT_SCRATCH,r11; - /* Variants of the "addition" argument for the prolog */ #define PROLOG_ADDITION_NONE_GEN(n) @@ -529,7 +514,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC) mtcr r10 ld r10,PACA_EXCRIT+EX_R10(r13) /* restore registers */ ld r11,PACA_EXCRIT+EX_R11(r13) - ld r13,PACA_EXCRIT+EX_R13(r13) + mfspr r13,SPRN_SPRG_CRIT_SCRATCH rfci /* Normal debug exception */ @@ -542,7 +527,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ALTIVEC) /* Now we mash up things to make it look like we are coming on a * normal exception */ - ld r15,PACA_EXCRIT+EX_R13(r13) + mfspr r15,SPRN_SPRG_CRIT_SCRATCH mtspr SPRN_SPRG_GEN_SCRATCH,r15 mfspr r14,SPRN_DBSR EXCEPTION_COMMON(0xd00, PACA_EXCRIT, INTS_DISABLE) diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c index 094e45c..ce74c33 100644 --- a/arch/powerpc/kernel/vdso.c +++ b/arch/powerpc/kernel/vdso.c @@ -715,8 +715,8 @@ int vdso_getcpu_init(void) unsigned long cpu, node, val; /* - * SPRG3 contains the CPU in the bottom 16 bits and the NUMA node in - * the next 16 bits. The VDSO uses this to implement getcpu(). + * SPRG_VDSO contains the CPU in the bottom 16 bits and the NUMA node + * in the next 16 bits. The VDSO uses this to implement getcpu(). */ cpu = get_cpu(); WARN_ON_ONCE(cpu > 0xffff); @@ -725,8 +725,8 @@ int vdso_getcpu_init(void) WARN_ON_ONCE(node > 0xffff); val = (cpu & 0xfff) | ((node & 0xffff) << 16); - mtspr(SPRN_SPRG3, val); - get_paca()->sprg3 = val; + mtspr(SPRN_SPRG_VDSO_WRITE, val); + get_paca()->sprg_vdso = val; put_cpu(); diff --git a/arch/powerpc/kernel/vdso32/getcpu.S b/arch/powerpc/kernel/vdso32/getcpu.S index 47afd08..23eb9a9 100644 --- a/arch/powerpc/kernel/vdso32/getcpu.S +++ b/arch/powerpc/kernel/vdso32/getcpu.S @@ -29,7 +29,7 @@ */ V_FUNCTION_BEGIN(__kernel_getcpu) .cfi_startproc - mfspr r5,SPRN_USPRG3 + mfspr r5,SPRN_SPRG_VDSO_READ cmpdi cr0,r3,0 cmpdi cr1,r4,0 clrlwi r6,r5,16 diff --git a/arch/powerpc/kernel/vdso64/getcpu.S b/arch/powerpc/kernel/vdso64/getcpu.S index 47afd08..23eb9a9 100644 --- a/arch/powerpc/kernel/vdso64/getcpu.S +++ b/arch/powerpc/kernel/vdso64/getcpu.S @@ -29,7 +29,7 @@ */ V_FUNCTION_BEGIN(__kernel_getcpu) .cfi_startproc - mfspr r5,SPRN_USPRG3 + mfspr r5,SPRN_SPRG_VDSO_READ cmpdi cr0,r3,0 cmpdi cr1,r4,0 clrlwi r6,r5,16 diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S index e66d4ec..fbfca57 100644 --- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S +++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S @@ -75,8 +75,8 @@ BEGIN_FTR_SECTION END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) /* Restore SPRG3 */ - ld r3,PACA_SPRG3(r13) - mtspr SPRN_SPRG3,r3 + ld r3,PACA_SPRG_VDSO(r13) + mtspr SPRN_SPRG_VDSO_WRITE,r3 /* Reload the host's PMU registers */ ld r3, PACALPPACAPTR(r13) /* is the host using the PMU? */ diff --git a/arch/powerpc/kvm/book3s_interrupts.S b/arch/powerpc/kvm/book3s_interrupts.S index f779450..3533c99 100644 --- a/arch/powerpc/kvm/book3s_interrupts.S +++ b/arch/powerpc/kvm/book3s_interrupts.S @@ -153,8 +153,8 @@ kvm_start_lightweight: * Reload kernel SPRG3 value. * No need to save guest value as usermode can't modify SPRG3. */ - ld r3, PACA_SPRG3(r13) - mtspr SPRN_SPRG3, r3 + ld r3, PACA_SPRG_VDSO(r13) + mtspr SPRN_SPRG_VDSO_WRITE, r3 #endif /* CONFIG_PPC_BOOK3S_64 */ /* R7 = vcpu */ diff --git a/arch/powerpc/kvm/bookehv_interrupts.S b/arch/powerpc/kvm/bookehv_interrupts.S index e4185f6..99635a3 100644 --- a/arch/powerpc/kvm/bookehv_interrupts.S +++ b/arch/powerpc/kvm/bookehv_interrupts.S @@ -229,11 +229,7 @@ stw r10, VCPU_CR(r4) PPC_STL r11, VCPU_GPR(R4)(r4) PPC_STL r5, VCPU_GPR(R5)(r4) - .if \type == EX_CRIT - PPC_LL r5, (\paca_ex + EX_R13)(r13) - .else mfspr r5, \scratch - .endif PPC_STL r6, VCPU_GPR(R6)(r4) PPC_STL r8, VCPU_GPR(R8)(r4) PPC_STL r9, VCPU_GPR(R9)(r4) @@ -435,10 +431,16 @@ _GLOBAL(kvmppc_resume_host) PPC_STL r5, VCPU_LR(r4) mfspr r7, SPRN_SPRG5 stw r3, VCPU_VRSAVE(r4) +#ifdef CONFIG_64BIT + PPC_LL r3, PACA_SPRG_VDSO(r13) +#endif PPC_STD(r6, VCPU_SHARED_SPRG4, r11) mfspr r8, SPRN_SPRG6 PPC_STD(r7, VCPU_SHARED_SPRG5, r11) mfspr r9, SPRN_SPRG7 +#ifdef CONFIG_64BIT + mtspr SPRN_SPRG_VDSO_WRITE, r3 +#endif PPC_STD(r8, VCPU_SHARED_SPRG6, r11) mfxer r3 PPC_STD(r9, VCPU_SHARED_SPRG7, r11)