From patchwork Thu Jun 8 15:36:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 773351 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wk8sD3StRz9s78 for ; Fri, 9 Jun 2017 01:45:52 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="KB/z9ZM4"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 3wk8sD2CN4zDqLV for ; Fri, 9 Jun 2017 01:45:52 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="KB/z9ZM4"; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from mail-pf0-x242.google.com (mail-pf0-x242.google.com [IPv6:2607:f8b0:400e:c00::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 3wk8fQ2mqMzDqQ7 for ; Fri, 9 Jun 2017 01:36:30 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="KB/z9ZM4"; dkim-atps=neutral Received: by mail-pf0-x242.google.com with SMTP id y7so5530721pfd.3 for ; Thu, 08 Jun 2017 08:36:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qm3I6SxsJYC6CSt5CNjvtLbmPUbT7WF0fTULa085isU=; b=KB/z9ZM4yFyoEhEIXGHOqwcdv2FLwy3MT6JStz0JxINU58/tOKwdY5dohn4KmC+bL0 kd8ds/yVMvegV5XvdmTgDL5WQJ0UMYEOhS+y/LQjEZQlzVEf7eVsADfgYv4i4dJM3TbN E6wevtcrim+vl6mu69TO5Wr0aRR/d83bzsFt1CfFo2xl5FtGynjRqori7oYAmUbhM7eQ Fm8vJFzp1TwlT0TlfQVnV6J6Kao5dn7hLOMz7Qg7fvwJDBDIkOmIz25Ug4xUI7oveCv8 4+v6BSiHiv3XivI9vdkiklstDvRPiJXThTnrIyNOaysIwosS38w6Ws6DlWsLAqPHO+R3 q45Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qm3I6SxsJYC6CSt5CNjvtLbmPUbT7WF0fTULa085isU=; b=ah85I1SMiY5ECpdF9OTqygDmHCfJww0Q4yZa3VEdv9lokfM3Eng6kPqP0XJAWf2Jsg eo/5oRdyL98HSF9Wapzt6LAAZ2gJAHoK62C5ups5Ok7pgFQkK7PW2H2Nx7DfR/fELwF5 qd+kc1hDr9NIvAjsB9beArI4eIbqnIH/3hZLvPCbIU33KwIe3+9T7cveck0nznPtIM3y txoG1UDrrAJeVu7rt7WSj/Rlpw4k4LZvU1lpYNrb8UCw26Vcw+5LD9BchKAV4V3VWy9S v0sMJtnlBQmGZzARvX1je3CHtykUukuZL66n7dbv27WSLhUO0vruOXZrIhkG7zEvCKHV NJlw== X-Gm-Message-State: AODbwcA3mgOv1iWiP7JjJyvkcszkKMXHvJWd+7NFFA5BwpchbvBOBni5 viVVRce/e3n7oker X-Received: by 10.99.149.14 with SMTP id p14mr38604163pgd.148.1496936188386; Thu, 08 Jun 2017 08:36:28 -0700 (PDT) Received: from roar.au.ibm.com (14-202-185-133.tpgi.com.au. [14.202.185.133]) by smtp.gmail.com with ESMTPSA id q194sm10360445pfq.56.2017.06.08.08.36.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Jun 2017 08:36:27 -0700 (PDT) From: Nicholas Piggin To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH 4/4] powerpc/64s: context switch avoid cpabort Date: Fri, 9 Jun 2017 01:36:09 +1000 Message-Id: <20170608153609.19217-4-npiggin@gmail.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170608153609.19217-1-npiggin@gmail.com> References: <20170608153609.19217-1-npiggin@gmail.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicholas Piggin Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The ISA v3.0B copy-paste facility only requires cpabort when switching to a process that has foreign real addresses mapped (direct access to accelerators), to clear a potential copy buffer filled by a previous thread. There is no accelerator driver implemented yet, so cpabort can be removed. It can be be re-added when a driver is implemented. POWER9 DD1 requires the copy buffer to always be cleared on context switch, but if accelerators are not in use, then an unpaired copy from a dummy region is sufficient to clear data out of the copy buffer. This increases context switch performance by about 5% on POWER9. Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/ppc-opcode.h | 8 ++++---- arch/powerpc/kernel/entry_64.S | 9 --------- arch/powerpc/kernel/process.c | 27 ++++++++++++++++++++++++++- 3 files changed, 30 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index 3a8d278e7421..3b6bbf5a8683 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -189,8 +189,7 @@ /* sorted alphabetically */ #define PPC_INST_BHRBE 0x7c00025c #define PPC_INST_CLRBHRB 0x7c00035c -#define PPC_INST_COPY 0x7c00060c -#define PPC_INST_COPY_FIRST 0x7c20060c +#define PPC_INST_COPY 0x7c20060c #define PPC_INST_CP_ABORT 0x7c00068c #define PPC_INST_DCBA 0x7c0005ec #define PPC_INST_DCBA_MASK 0xfc0007fe @@ -223,8 +222,7 @@ #define PPC_INST_MSGSNDP 0x7c00011c #define PPC_INST_MTTMR 0x7c0003dc #define PPC_INST_NOP 0x60000000 -#define PPC_INST_PASTE 0x7c00070c -#define PPC_INST_PASTE_LAST 0x7c20070d +#define PPC_INST_PASTE 0x7c20070d #define PPC_INST_POPCNTB 0x7c0000f4 #define PPC_INST_POPCNTB_MASK 0xfc0007fe #define PPC_INST_POPCNTD 0x7c0003f4 @@ -392,6 +390,8 @@ /* Deal with instructions that older assemblers aren't aware of */ #define PPC_CP_ABORT stringify_in_c(.long PPC_INST_CP_ABORT) +#define PPC_COPY(a, b) stringify_in_c(.long PPC_INST_COPY | \ + ___PPC_RA(a) | ___PPC_RB(b)) #define PPC_DCBAL(a, b) stringify_in_c(.long PPC_INST_DCBAL | \ __PPC_RA(a) | __PPC_RB(b)) #define PPC_DCBZL(a, b) stringify_in_c(.long PPC_INST_DCBZL | \ diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S index fb143859cc68..da9486e2fd89 100644 --- a/arch/powerpc/kernel/entry_64.S +++ b/arch/powerpc/kernel/entry_64.S @@ -536,15 +536,6 @@ _GLOBAL(_switch) * which contains larx/stcx, which will clear any reservation * of the task being switched. */ - -BEGIN_FTR_SECTION -/* - * A cp_abort (copy paste abort) here ensures that when context switching, a - * copy from one process can't leak into the paste of another. - */ - PPC_CP_ABORT -END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) - #ifdef CONFIG_PPC_BOOK3S /* Cancel all explict user streams as they will have no use after context * switch and will stop the HW from creating streams itself diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index 45faa9a32a01..6273b5d5baec 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -1137,6 +1137,11 @@ static inline void restore_sprs(struct thread_struct *old_thread, #endif } +#ifdef CONFIG_PPC_BOOK3S_64 +#define CP_SIZE 128 +static const u8 dummy_copy_buffer[CP_SIZE] __attribute__((aligned(CP_SIZE))); +#endif + struct task_struct *__switch_to(struct task_struct *prev, struct task_struct *new) { @@ -1226,8 +1231,28 @@ struct task_struct *__switch_to(struct task_struct *prev, batch->active = 1; } - if (current_thread_info()->task->thread.regs) + if (current_thread_info()->task->thread.regs) { restore_math(current_thread_info()->task->thread.regs); + + /* + * The copy-paste buffer can only store into foreign real + * addresses, so unprivileged processes can not see the + * data or use it in any way unless they have foreign real + * mappings. We don't have a VAS driver that allocates those + * yet, so no cpabort is required. + */ + if (cpu_has_feature(CPU_FTR_POWER9_DD1)) { + /* + * DD1 allows paste into normal system memory, so we + * do an unpaired copy here to clear the buffer and + * prevent a covert channel being set up. + * + * cpabort is not used because it is quite expensive. + */ + asm volatile(PPC_COPY(%0, %1) + : : "r"(dummy_copy_buffer), "r"(0)); + } + } #endif /* CONFIG_PPC_STD_MMU_64 */ return last;