From patchwork Wed Jul 22 14:55:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333928 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=kcQYSoIJ; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=U3S4gi3Q; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdnj3WRgz9sSn for ; Thu, 23 Jul 2020 00:56:09 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732751AbgGVO4G (ORCPT ); Wed, 22 Jul 2020 10:56:06 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:63228 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730465AbgGVO4F (ORCPT ); Wed, 22 Jul 2020 10:56:05 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEkh9M016190; Wed, 22 Jul 2020 07:55:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=kcQYSoIJb57JAMxTNY/P9ypkRiXs4kNkt2VwJ+EaflWVa/rXQSPYCptlwzM6kdRLsRDo 2xPb3VeceVPI4UKQB1FBGl3J+T33RIQF3sPui4qjLaNIqCNcK+DCqUyUWO2hz9LL4idB z8HjAOKXVD6KI8CVrR6L/8itYtRRgSJ/p1DwdRNnuTdJ5rTBe8+9OQ8+WsLxVOoBNPdd PKpgcReBRvnj85JW08qRbU55oNRbmlBg7SY3LLQmsH5ErvTfosBAJR6nbrxNDYgcUHp/ 7zJGsJ6FGgkOcrkombUSB4aa5F8IW0+T2VmXGNayacVI8fFEltju60oWv6bjhd4DXSXm YA== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens7ep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:55:36 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:55:34 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.170) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:55:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GPEi+5wZxvPKX25j49TConCqDUCaC8AYFsYdPbO4MdT0+yuaD3L8iikXSqbEsPnUealBGwqNFwh0kmS47S8P09psB7bb+Bg/m0uW/dPmdinKZMWPxmhdQEpVtdDSDql+3nWYP5Nt//4ck1GigUrWJXsTRHC4K7cJsF3/SymZxdvlx/c0WRZEKGRdlSDx9BOLFxnBg/m+1JXZ98XD8Vyv9FcrBoZcWiOAsg2oTCZeXcO9oLHmP/AceWsZ3qBsVFaPZl5S/7plVdvvst1d3b7EnSHkpWRQnXdiOr41OuqR+OcK86yXj4fo5+weMS8c9izbP0MxIvew8oOVlWis223N7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=dN4rNuSI3yCLMGsEYONkzcCxZBYgyvYqVat2m6uzrihgJ/t7l4ydm2rfsMMiFWz6AVIJX2Bvh3x2w7kG5PepIdZNCdIBxsjizdpZp9ZdOfq8VOlaYAcmQKyBF3AFdBV7z0YJAqEK3fPLJvFlceTEZIpJOxBVEt+4A3GPQ73Xz/e0cinePuaYrgQbEF2E/5GjZbUnVOIoJWW4OKRlbQ2uFie24QJnTIkt90Y+F4VNwbBulH6/AmnhrO6XcNxBVNR5QlXJo5aYxTiCFHaJmfY+oHKK9GIRNlRp+Fy7OQ2ul8z+NbGNxnHBqdKa7B7Fc0ymHuinwrxwbcgN9mjxfZRxww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=U3S4gi3QCcDFAsrGxxo9tnabt8yR25Fs8svLCq7+WSl/1Ee1fubHzndx3bjtXvG4F26CJjkatgHyVuGmhkckbGFfto3Dv3TSiEC+8k3C9ahNisgizvJVj70vQTJEAy6XVVAFHYc9541b2PY4CZrOXuHhufOqEVl/E/xHK+W/BtI= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:55:33 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:55:33 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH 08/13] task_isolation: arch/arm64: enable task isolation functionality Thread-Topic: [PATCH 08/13] task_isolation: arch/arm64: enable task isolation functionality Thread-Index: AQHWYDgmZidEulvFKU2n/vC9QyCd/A== Date: Wed, 22 Jul 2020 14:55:33 +0000 Message-ID: <470f6068e87a3627111a2615032d7b6bce824451.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dedab54d-a283-42ef-dedb-08d82e4f48f4 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jPHe8/6YSKjjvYJquAT+zK5FYFGjRzAy5Lpics/qowaqhUZoBIehORI57kTQ4qMj5dHSZoMi3kn+RJPxYjYGNI7BKPHD1a65KIptQ+wvvadfPhY+bef3I4mGu6e5IW6jx4F9yFCxZEV7DI2YlL483O5OhKxjvjGKNkURINQIe6p+z6+ylnHDIbNAyIcv105166zZpHJv2wIr35pTM2tXhJD/hFz5sDM9XETxSXmQ/akYbWdbSaBvWelbEcQXxiUZoX7mK2l0HISIrQj56nK6O6mmFpKhO34WT/ZNnUX9eAXWTW+LHJ1PWKk4R19d5xGk x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(30864003)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: GBvucFlf24zgdqJpvXhrb/IFZQZMFRwP8f1DnCwMJLvMQQTlAXLGiTL41HCjHi70K8UQor3oJmzzvUn8v5r4uCiKNainZ/QZp8IyOSvS9M7De+k68+iDUXdGNNN0fJ1uipIyKKwZhqdRkg7J/OLdWaou0CW0lueyCnI1lQ7v0m6DlY/TwvQxNBKtTaT8DwLyqLykJKFpZ9DhI+ZykvrQGeVZv72fs40rNN5qVBq9uKOhgNUysGVK1fn1Emk2jU1gljnpoT99/afI+QkGR6pcCVKy9GKrnoGKMVJ3Ebahxft9US011ebBXIL0gZ6gGn39NQ/e9yatR47yCxFMkg8Pb35kVg0kmaw9VhEjeQWaIlvRUhGqGtFVlLohj0XGlNHBz1hGK81iWsDX9nLEpOM3MtugD1lj3X8kknXGD5qO6f5NZAvlFJkYQwPG/wdeWopSFp+d+7LG4f6X5E7j0rHo7S9+iC3GG7+tvLJurMaZb6c= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dedab54d-a283-42ef-dedb-08d82e4f48f4 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:55:33.2817 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8gS1iM0mhzRBNI15G5gQNTbGDelOksfqrVK8At2d6+ckK2KJNY96jLIFyBNa0HB5Ov/gaIP4FRrcJiFuL/nbSA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Chris Metcalf In do_notify_resume(), call task_isolation_start() for TIF_TASK_ISOLATION tasks. Add _TIF_TASK_ISOLATION to _TIF_WORK_MASK, and define a local NOTIFY_RESUME_LOOP_FLAGS to check in the loop, since we don't clear _TIF_TASK_ISOLATION in the loop. We instrument the smp_send_reschedule() routine so that it checks for isolated tasks and generates a suitable warning if needed. Finally, report on page faults in task-isolation processes in do_page_faults(). Early kernel entry code calls task_isolation_kernel_enter(). In particular: Vectors: el1_sync -> el1_sync_handler() -> task_isolation_kernel_enter() el1_irq -> asm_nmi_enter(), handle_arch_irq() el1_error -> do_serror() el0_sync -> el0_sync_handler() el0_irq -> handle_arch_irq() el0_error -> do_serror() el0_sync_compat -> el0_sync_compat_handler() el0_irq_compat -> handle_arch_irq() el0_error_compat -> do_serror() SDEI entry: __sdei_asm_handler -> __sdei_handler() -> nmi_enter() Functions called from there: asm_nmi_enter() -> nmi_enter() -> task_isolation_kernel_enter() asm_nmi_exit() -> nmi_exit() -> task_isolation_kernel_return() Handlers: do_serror() -> nmi_enter() -> task_isolation_kernel_enter() or task_isolation_kernel_enter() el1_sync_handler() -> task_isolation_kernel_enter() el0_sync_handler() -> task_isolation_kernel_enter() el0_sync_compat_handler() -> task_isolation_kernel_enter() handle_arch_irq() is irqchip-specific, most call handle_domain_irq() or handle_IPI() There is a separate patch for irqchips that do not follow this rule. handle_domain_irq() -> task_isolation_kernel_enter() handle_IPI() -> task_isolation_kernel_enter() nmi_enter() -> task_isolation_kernel_enter() Signed-off-by: Chris Metcalf [abelits@marvell.com: simplified to match kernel 5.6] Signed-off-by: Alex Belits --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/barrier.h | 2 ++ arch/arm64/include/asm/thread_info.h | 5 ++++- arch/arm64/kernel/entry-common.c | 7 +++++++ arch/arm64/kernel/ptrace.c | 16 +++++++++++++++- arch/arm64/kernel/sdei.c | 2 ++ arch/arm64/kernel/signal.c | 13 ++++++++++++- arch/arm64/kernel/smp.c | 9 +++++++++ arch/arm64/mm/fault.c | 5 +++++ 9 files changed, 57 insertions(+), 3 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 66dc41fd49f2..96fefabfa10f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -137,6 +137,7 @@ config ARM64 select HAVE_ARCH_PREL32_RELOCATIONS select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK + select HAVE_ARCH_TASK_ISOLATION select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index fb4c27506ef4..bf4a2adabd5b 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -48,6 +48,8 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +#define instr_sync() isb() + /* * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz * and 0 otherwise. diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 5e784e16ee89..73269bb8a57d 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -67,6 +67,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_TASK_ISOLATION 6 /* task isolation enabled for task */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -86,6 +87,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE) +#define _TIF_TASK_ISOLATION (1 << TIF_TASK_ISOLATION) #define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) #define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) #define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) @@ -99,7 +101,8 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | \ + _TIF_TASK_ISOLATION) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index d3be9dbf5490..8b682aa020ae 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -70,6 +71,8 @@ asmlinkage void notrace el1_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_CUR: case ESR_ELx_EC_IABT_CUR: @@ -231,6 +234,8 @@ asmlinkage void notrace el0_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC64: el0_svc(regs); @@ -300,6 +305,8 @@ asmlinkage void notrace el0_sync_compat_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC32: el0_svc_compat(regs); diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 1e02e98e68dd..5acfc194bdd0 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -1851,7 +1852,11 @@ static void tracehook_report_syscall(struct pt_regs *regs, int syscall_trace_enter(struct pt_regs *regs) { - unsigned long flags = READ_ONCE(current_thread_info()->flags); + unsigned long flags; + + task_isolation_kernel_enter(); + + flags = READ_ONCE(current_thread_info()->flags); if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) { tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER); @@ -1859,6 +1864,15 @@ int syscall_trace_enter(struct pt_regs *regs) return NO_SYSCALL; } + /* + * In task isolation mode, we may prevent the syscall from + * running, and if so we also deliver a signal to the process. + */ + if (test_thread_flag(TIF_TASK_ISOLATION)) { + if (task_isolation_syscall(regs->syscallno) == -1) + return NO_SYSCALL; + } + /* Do the secure computing after ptrace; failures should be fast. */ if (secure_computing() == -1) return NO_SYSCALL; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index dab88260b137..f65b676132f9 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -185,6 +186,7 @@ static __kprobes unsigned long _sdei_handler(struct pt_regs *regs, __uaccess_enable_hw_pan(); err = sdei_event_handler(regs, arg); + task_isolation_interrupt("SDEI handled"); if (err) return SDEI_EV_FAILED; diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 3b4f31f35e45..ece90c5756be 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -907,6 +908,11 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } +#define NOTIFY_RESUME_LOOP_FLAGS \ + (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ + _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ + _TIF_UPROBE | _TIF_FSCHECK) + asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -917,6 +923,8 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, */ trace_hardirqs_off(); + task_isolation_check_run_cleanup(); + do { /* Check valid user FS if needed */ addr_limit_user_check(); @@ -947,7 +955,10 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, local_daif_mask(); thread_flags = READ_ONCE(current_thread_info()->flags); - } while (thread_flags & _TIF_WORK_MASK); + } while (thread_flags & NOTIFY_RESUME_LOOP_FLAGS); + + if (thread_flags & _TIF_TASK_ISOLATION) + task_isolation_start(); } unsigned long __ro_after_init signal_minsigstksz; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index e43a8ff19f0f..c893c8babe76 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -835,6 +836,7 @@ void arch_send_call_function_single_ipi(int cpu) #ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL void arch_send_wakeup_ipi_mask(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "wakeup IPI"); smp_cross_call(mask, IPI_WAKEUP); } #endif @@ -896,11 +898,16 @@ void handle_IPI(int ipinr, struct pt_regs *regs) unsigned int cpu = smp_processor_id(); struct pt_regs *old_regs = set_irq_regs(regs); + task_isolation_kernel_enter(); + if ((unsigned)ipinr < NR_IPI) { trace_ipi_entry_rcuidle(ipi_types[ipinr]); __inc_irq_stat(cpu, ipi_irqs[ipinr]); } + task_isolation_interrupt("IPI type %d (%s)", ipinr, + ipinr < NR_IPI ? ipi_types[ipinr] : "unknown"); + switch (ipinr) { case IPI_RESCHEDULE: scheduler_ipi(); @@ -963,12 +970,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs) void smp_send_reschedule(int cpu) { + task_isolation_remote(cpu, "reschedule IPI"); smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST void tick_broadcast(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "timer IPI"); smp_cross_call(mask, IPI_TIMER); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 8afb238ff335..d01f3cbed87f 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -539,6 +540,10 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, */ if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | VM_FAULT_BADACCESS)))) { + /* No signal was generated, but notify task-isolation tasks. */ + if (user_mode(regs)) + task_isolation_interrupt("page fault at %#lx", addr); + /* * Major/minor page fault accounting is only done * once. If we go through a retry, it is extremely