From patchwork Wed Feb 5 20:48:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Grimm X-Patchwork-Id: 1234007 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 48CYZS1Vk3z9sPK for ; Thu, 6 Feb 2020 07:49:04 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 48CYZS0ZN0zDqT5 for ; Thu, 6 Feb 2020 07:49:04 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=grimm@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 48CYZB2ml6zDqTG for ; Thu, 6 Feb 2020 07:48:50 +1100 (AEDT) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 015Ke0H6113250 for ; Wed, 5 Feb 2020 15:48:47 -0500 Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 2xyhmh4uts-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 05 Feb 2020 15:48:47 -0500 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 015KeEpo114269 for ; Wed, 5 Feb 2020 15:48:47 -0500 Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0b-001b2d01.pphosted.com with ESMTP id 2xyhmh4uta-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Feb 2020 15:48:47 -0500 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.0.27/8.16.0.27) with SMTP id 015KjHTv004472; Wed, 5 Feb 2020 20:48:46 GMT Received: from b03cxnp08027.gho.boulder.ibm.com (b03cxnp08027.gho.boulder.ibm.com [9.17.130.19]) by ppma02dal.us.ibm.com with ESMTP id 2xykc9hf32-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 05 Feb 2020 20:48:46 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08027.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 015KmiWm58196452 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Feb 2020 20:48:44 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C272D7805E; Wed, 5 Feb 2020 20:48:44 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4ADE07805C; Wed, 5 Feb 2020 20:48:44 +0000 (GMT) Received: from alain.ibm.com (unknown [9.80.234.88]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Wed, 5 Feb 2020 20:48:44 +0000 (GMT) From: Ryan Grimm To: oohall@gmail.com Date: Wed, 5 Feb 2020 15:48:13 -0500 Message-Id: <20200205204813.17948-2-grimm@linux.ibm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200205204813.17948-1-grimm@linux.ibm.com> References: <20200205204813.17948-1-grimm@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-02-05_06:2020-02-04, 2020-02-05 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=3 spamscore=0 impostorscore=0 mlxlogscore=807 adultscore=0 lowpriorityscore=0 priorityscore=1501 phishscore=0 mlxscore=0 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2001150001 definitions=main-2002050158 Subject: [Skiboot] [PATCH 1/1] Disable protected execution facility X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org, Ryan Grimm Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" This patch disables Protected Execution Faciltiy (PEF). This software procedure is needed for the lab because Cronus will be configured to bring the machine up with PEF on. Hostboot has a similar procedure for running with PEF off. Skiboot can run with PEF on but the kernel cannot; the kernel will take a machine check when trying to write a protected resource, such as the PTCR. So, use this until we have an ultravisor, or if we want to use BML with Cronus without UV = 1. Signed-off-by: Ryan Grimm Tested-by: Alistair Popple --- asm/misc.S | 37 ++++++++++++++++++++++++++++ core/cpu.c | 59 +++++++++++++++++++++++++++++++++++++++++++++ core/init.c | 3 +++ include/cpu.h | 3 +++ include/processor.h | 4 +++ 5 files changed, 106 insertions(+) diff --git a/asm/misc.S b/asm/misc.S index 647f60b2..55a9a99c 100644 --- a/asm/misc.S +++ b/asm/misc.S @@ -255,3 +255,40 @@ enter_p9_pm_state: mtspr SPR_PSSCR,%r3 PPC_INST_STOP b . + +/* Exit UV mode and disable Protected Execution Facility + * For each core, this should be run on all secondary threads first to bring + * them out of UV mode. Then, it is called by the primary thread to disable + * PEF and bring it out of UV mode. All threads will then be running in HV + * mode. The only way to reenable UV mode is with a reboot. + * r3 = 1 if primary thread + * 0 if secondary thread + */ +.global exit_uv_mode +exit_uv_mode: + mfmsr %r4 + LOAD_IMM64(%r5, ~MSR_S) + and %r4,%r4,%r5 + mtspr SPR_USRR1,%r4 + + mfspr %r4,SPR_HSRR1 + and %r4,%r4,%r5 + mtspr SPR_HSRR1,%r3 + + mfspr %r4,SPR_SRR1 + and %r4,%r4,%r5 + mtspr SPR_SRR1,%r4 + + cmpdi %r3,1 + bne 1f + mfspr %r4, SPR_SMFCTRL + LOAD_IMM64(%r5, ~PPC_BIT(0)) + and %r4,%r4,%r5 + mtspr SPR_SMFCTRL,%r4 +1: + isync + + mflr %r4 + mtspr SPR_USRR0,%r4 + + urfid diff --git a/core/cpu.c b/core/cpu.c index d5b7d623..1adf16cc 100644 --- a/core/cpu.c +++ b/core/cpu.c @@ -1644,3 +1644,62 @@ static int64_t opal_nmmu_set_ptcr(uint64_t chip_id, uint64_t ptcr) return rc; } opal_call(OPAL_NMMU_SET_PTCR, opal_nmmu_set_ptcr, 2); + +static void _exit_uv_mode(void *data __unused) +{ + prlog(PR_DEBUG, "Exit uv mode on cpu pir 0x%04x\n", this_cpu()->pir); + /* HW has smfctrl shared between threads but on Mambo it is per-thread */ + if (chip_quirk(QUIRK_MAMBO_CALLOUTS)) + exit_uv_mode(1); + else + exit_uv_mode(cpu_is_thread0(this_cpu())); +} + +void cpu_disable_pef(void) +{ + struct cpu_thread *cpu; + struct cpu_job **jobs; + + if (!(mfmsr() & MSR_S)) { + prlog(PR_DEBUG, "UV mode off on cpu pir 0x%04x\n", this_cpu()->pir); + return; + } + + jobs = zalloc(sizeof(struct cpu_job *) * (cpu_max_pir + 1)); + assert(jobs); + + /* Exit uv mode on all secondary threads before touching + * smfctrl on thread 0 */ + for_each_available_cpu(cpu) { + if (cpu == this_cpu()) + continue; + + if (!cpu_is_thread0(cpu)) + jobs[cpu->pir] = cpu_queue_job(cpu, "exit_uv_mode", + _exit_uv_mode, NULL); + } + + for_each_available_cpu(cpu) + if (jobs[cpu->pir]) { + cpu_wait_job(jobs[cpu->pir], true); + jobs[cpu->pir] = NULL; + } + + /* Exit uv mode and disable smfctrl on primary threads */ + for_each_available_cpu(cpu) { + if (cpu == this_cpu()) + continue; + + if (cpu_is_thread0(cpu)) + jobs[cpu->pir] = cpu_queue_job(cpu, "exit_uv_mode", + _exit_uv_mode, NULL); + } + + for_each_available_cpu(cpu) + if (jobs[cpu->pir]) + cpu_wait_job(jobs[cpu->pir], true); + + free(jobs); + + _exit_uv_mode(NULL); +} diff --git a/core/init.c b/core/init.c index 339462e5..0d993abb 100644 --- a/core/init.c +++ b/core/init.c @@ -1354,6 +1354,9 @@ void __noreturn __nomcount main_cpu_entry(const void *fdt) /* Add the list of interrupts going to OPAL */ add_opal_interrupts(); + /* Disable protected execution facility in BML */ + cpu_disable_pef(); + /* Now release parts of memory nodes we haven't used ourselves... */ mem_region_release_unused(); diff --git a/include/cpu.h b/include/cpu.h index 686310d7..cab63360 100644 --- a/include/cpu.h +++ b/include/cpu.h @@ -309,4 +309,7 @@ int dctl_set_special_wakeup(struct cpu_thread *t); int dctl_clear_special_wakeup(struct cpu_thread *t); int dctl_core_is_gated(struct cpu_thread *t); +extern void exit_uv_mode(int); +void cpu_disable_pef(void); + #endif /* __CPU_H */ diff --git a/include/processor.h b/include/processor.h index a0c2864a..1fdcc02b 100644 --- a/include/processor.h +++ b/include/processor.h @@ -11,6 +11,7 @@ #define MSR_HV PPC_BIT(3) /* Hypervisor mode */ #define MSR_VEC PPC_BIT(38) /* VMX enable */ #define MSR_VSX PPC_BIT(40) /* VSX enable */ +#define MSR_S PPC_BIT(41) /* Secure mode */ #define MSR_EE PPC_BIT(48) /* External Int. Enable */ #define MSR_PR PPC_BIT(49) /* Problem state */ #define MSR_FP PPC_BIT(50) /* Floating Point Enable */ @@ -65,6 +66,9 @@ #define SPR_HMEER 0x151 /* HMER interrupt enable mask */ #define SPR_PCR 0x152 #define SPR_AMOR 0x15d +#define SPR_USRR0 0x1fa /* RW: Ultravisor Save/Restore Register 0 */ +#define SPR_USRR1 0x1fb /* RW: Ultravisor Save/Restore Register 1 */ +#define SPR_SMFCTRL 0x1ff /* RW: Secure Memory Facility Control */ #define SPR_PSSCR 0x357 /* RW: Stop status and control (ISA 3) */ #define SPR_TSCR 0x399 #define SPR_HID0 0x3f0