From patchwork Wed Jul 22 14:47:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333921 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=Xp4AdyKS; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=e09a3zWa; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdcC1h2Dz9sSn for ; Thu, 23 Jul 2020 00:47:55 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732633AbgGVOrw (ORCPT ); Wed, 22 Jul 2020 10:47:52 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:26294 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728405AbgGVOrv (ORCPT ); Wed, 22 Jul 2020 10:47:51 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEkh8m016190; Wed, 22 Jul 2020 07:47:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=p6bTuXQ7WRxMymEyANQpDO/i/w9XN7WepnoSu9OWtzw=; b=Xp4AdyKSekCrbtf9J61nh0nsndUJ3FvUw8wtK8lJdqHtXLjgaxv19G8KJYVCSWQfXSt7 Ic6/aF/29rEGZ1Hma/Q77okVgMbfSZ92Ppf9Lpq3Qvjj24rlcR+IpH50bCTS9CoQFHKD VW4J7nbBYCmmqIN9CV5V337hvnM6VsiZ/cLsxEXEPnP64lzCuH3OE3uxmFUMFrsmB7bF oWpzT9QGwenyhq1Sbd+l0o/UaNhgVzBzRPTWuDdobxlCNcJBEzZij+MvkmusqoIKqFaV +icActqyHLIQMn+o0PYsaJiN09sU5aOKt7PElNGKa4pondBG3ltrNkgrP7G/WMw1/Eo0 yA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens6g6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:47:14 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:47:13 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.174) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:47:13 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=minH2iIryoKS0Dyq4fnC66xXcwHwSeDLMPnQ/7yI2EM0la+ePJAOjYxRLCQHWEKyAtNDKxUCIeCk01Klz++/pOqgj/csNvZYBE6TqK9jMJadcsc27GIOHJWlZ9ew8+0GjFkQ/FtOulpv54wbzTrr8Jm2OEiNSySNfXJ6K5CYLSVpvrQuyWvLh75SP5idzLp1hxOsYjeFmzQCMJ/4xhr+KCLEXj2tqY21weSSfTW65kWWHxMS+H9L7oP56snYsEpv05umDAz+VrWmXVdSf6z0rLn6864/Dv/hJDATQ9ob/nG8Jd3KQl839WM2EosgAlNZeb96PMWaHlRI/l34Chc26Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p6bTuXQ7WRxMymEyANQpDO/i/w9XN7WepnoSu9OWtzw=; b=E08oxM6BDkajx410Ryg/CsfUUoV925GFDxFFeH8vnSPp4KfCPrpBP6FmhlR7I6Z7uoCYL4zFaMOm5RrmgRDFUjvcIJDLVmkbjdP4e85YaD/hSFYh8707Dl7w2/ycpZIJXjGYzXBPfj1vPkpmZA6dCQ7TXZBv+d6xJmPT4El5dh/CVw49PxCYRu1EBrWrVGkU0EX8NQK5z6ZdkyPj66z6BGAqT7L/u2bPp+/8F8N21Qf8e/geMGU9sQc5UGeLxsaGXl7asM9skXGC+nSnugVBeCHBD5lEWPqwWjVdmkjCoQ74fkph1HR6OFmXW6WwCQ9ylJWfi8F/IwBFSMOjsVcNlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p6bTuXQ7WRxMymEyANQpDO/i/w9XN7WepnoSu9OWtzw=; b=e09a3zWa7FifCaD68AlZgDONLej14Is8RBLJT5gBo2fCC1I5eIKCUR7W2cfpbOnLdhiEfTeKd00HY3DfJueU3kFHWjaIFnJZHOCl8q92R8JeURIZrwQSqAnj9HFlj4wSbEflG3Q4ImwU1BnqmwEwXKaws729Fyd5IUgoHaXsY6A= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2091.namprd18.prod.outlook.com (2603:10b6:907:2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.20; Wed, 22 Jul 2020 14:47:11 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:47:11 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Thread-Topic: [PATCH v4 01/13] task_isolation: vmstat: add quiet_vmstat_sync function Thread-Index: AQHWYDb7+u4pW/dIVkegDrd39Y9d0w== Date: Wed, 22 Jul 2020 14:47:11 +0000 Message-ID: References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: cc92bc9e-2e9f-4274-6198-08d82e4e1dfc x-ms-traffictypediagnostic: MW2PR18MB2091: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1186; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: iRCgcY9bBZrb3uXCU7/3qjjkoyJxo6NBMc+5CcH5lib8SezunaMZ/RihX3RjJU5dQ4/4xbvkJUkPFq4LLSdChoMitxZn/nb621mve7HvUwQSjX3Gk0ucr4VWv9NlhPgOpCsUhY+SpsfBlsXe8XMUHc7zRi+iWMiNQN5X5n9fXufugrqGKBWvzKf9TCG98yHpb8J01Wj8nbBUFU2j6tVU0JKs/2lV9fc4v6jKV+QDMSwKP4KAgUQseYu7ixqCOvK2KaN/zMKKOUUot564QGqElFx9voUuw37+moN8FePrPmK/JbF9kXYCTMDsum21B1GqorUT914FWbyZTAGvN/8CXg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(346002)(366004)(396003)(136003)(376002)(8676002)(76116006)(91956017)(110136005)(66946007)(66446008)(64756008)(66556008)(66476007)(316002)(2906002)(6512007)(54906003)(6486002)(7416002)(478600001)(5660300002)(186003)(71200400001)(36756003)(2616005)(86362001)(4326008)(26005)(8936002)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: M606+b3D8umwEBn+jJEZ/QhSWlYeZhoSO0zJjbGhs3X9vr70DpnJemjjkgscJYWv/pVj9+mXz+bzMNdmK5JP4qA2OVdQfmnoqPNUPV+zBi9jrIGJPp62wr7oelUHOF8uhe0u/8sfqH3VDanqoV42lBeTZHmG94ZreLxAdYMyjmv/xro7OOE6PG7R4qfVWrhzcjJOgY0N9lol4wNoeyAFXFgLr01xmVii1zcxJ79NizNrj1ec3Kj/ToXgRaa4wAxaN4zca1rwtwKAbCNd8snOWbU9GeH/kbgpkWHtwIVEwD+o9CvDPCYA08vSUrzfc80uma/ZKrQl7fWMUdps6ORiNT+YqNIXHVClLuJ3BiX2DjnxKhLl7O49ftWc7zh3xA6T41jjkkCYKfqH87lGgppBu3v7ok6ADd317aMgL57wasVfkb60LZcqNyZh2B/O1bh9EUFMA9HKaeescjxkdOL2FLhBhljuVg9vTKaAinoxzyI= Content-ID: <71C0964DB92C2640A8A3A33709BD1768@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: cc92bc9e-2e9f-4274-6198-08d82e4e1dfc X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:47:11.6802 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: EvfeeOPdAM9BpPxupbER8m5ewzrOU64HzuC0EwxrUiqn40lUn9OxeuMvczerSNVaOktWvplGZCG5dHTxhJhA0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2091 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In commit f01f17d3705b ("mm, vmstat: make quiet_vmstat lighter") the quiet_vmstat() function became asynchronous, in the sense that the vmstat work was still scheduled to run on the core when the function returned. For task isolation, we need a synchronous version of the function that guarantees that the vmstat worker will not run on the core on return from the function. Add a quiet_vmstat_sync() function with that semantic. Signed-off-by: Chris Metcalf Signed-off-by: Alex Belits --- include/linux/vmstat.h | 2 ++ mm/vmstat.c | 9 +++++++++ 2 files changed, 11 insertions(+) * differentials of processors that have their worker diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index aa961088c551..ded16dfd21fa 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -272,6 +272,7 @@ extern void __dec_zone_state(struct zone *, enum zone_stat_item); extern void __dec_node_state(struct pglist_data *, enum node_stat_item); void quiet_vmstat(void); +void quiet_vmstat_sync(void); void cpu_vm_stats_fold(int cpu); void refresh_zone_stat_thresholds(void); @@ -374,6 +375,7 @@ static inline void __dec_node_page_state(struct page *page, static inline void refresh_zone_stat_thresholds(void) { } static inline void cpu_vm_stats_fold(int cpu) { } static inline void quiet_vmstat(void) { } +static inline void quiet_vmstat_sync(void) { } static inline void drain_zonestat(struct zone *zone, struct per_cpu_pageset *pset) { } diff --git a/mm/vmstat.c b/mm/vmstat.c index 3fb23a21f6dd..93534f8537ca 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1889,6 +1889,15 @@ void quiet_vmstat(void) refresh_cpu_vm_stats(false); } +/* + * Synchronously quiet vmstat so the work is guaranteed not to run on return. + */ +void quiet_vmstat_sync(void) +{ + cancel_delayed_work_sync(this_cpu_ptr(&vmstat_work)); + refresh_cpu_vm_stats(false); +} + /* * Shepherd worker thread that checks the From patchwork Wed Jul 22 14:48:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333922 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=IPdkZVJy; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=T7rvewSM; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdf60fy8z9sPf for ; Thu, 23 Jul 2020 00:49:34 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732651AbgGVOt1 (ORCPT ); Wed, 22 Jul 2020 10:49:27 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:40066 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729642AbgGVOtZ (ORCPT ); Wed, 22 Jul 2020 10:49:25 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEkokH016196; Wed, 22 Jul 2020 07:48:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=3912sffFv0bNrTJYcPGQ2ATrVX945IvVFvT3Ia9ps+E=; b=IPdkZVJyq/QglTCXxrSXlab/gOO/LQEX/aBZDDeRJvtK1JfoIzt3MADWrbnQ9jz52EQp CVHv8KXSXhIVfdVEKpB5R+DGcqtZh02R6BavoCOq63CseIUNCgqDh23Cd+f4cdlCdVuJ ErYqfv+l710G6OLcQB5AqAMZOPoRTBDKiwJAx6yPB1q5DQG1EfPR3wcnHDeQmkbSSqq6 ZmpkdE832+CvQWgqnxHLTA0lOGEFC/BsquWmeJj+v9rO1RfC5UlH7CwrcizJDkyzSnAt eZ0mXg54okBkx5IgDnLziYa99iwN2uitj5P/QKqhLH6N+FEt6sBewpdzCxPqvYiZLei1 yw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens6p3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:48:58 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:48:57 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.46) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:48:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m32i5eko/qQNOM2jMgw/lmOUqOWptazGsqxjoGl2YoY/knfkztcdpQvjf/FQCdPGcKhc4Mnk383G2/2A+m6Ar0S15NNhDVyLuHPs1LlkkQ5DWIr8ScoXEuCIZmRxR3+lauYEC271Puk531RSM51bhQ07AOQC+Odpql8MD5Z66Zgn3T+ou47mgzOTfblx6uiMGOW5UCjXIlmxmIexnjiNUSsphCkmQ3UnZrAZWvtkTBpuHId1rT4A5K1SSqk2mJ9sl2Ic+yf8Qk5xGbX23+WyY5SEE+TLB4XCzk/EZd7hWkc5kpx7lDYYRQDIAHoOxeYXHjxmnwzAqcsXVQAStJKhbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3912sffFv0bNrTJYcPGQ2ATrVX945IvVFvT3Ia9ps+E=; b=Oa8i1dQru+QP92krqKwO9Jx0OCOxiIReSXXMInYmJ7WwQ4t9+dyENnPWLPduS/b+k4UdJ6dt9ZFEvFNzzb7HLH7IVcfp0h4dAATq/PZ+KZGKOJECYoOEY99h+XSfC8j9wAx0S3ueI5rxuc5FQC6caKxq/Aq78I+oAQUfoSteCUdL040lRGsOdm2I+cdEb/oaprVh+XzT3WAwXwFP05ocgThFaxq6e4UPJ4Mp/F01pnC+gwxjB+aM7I1x3yX5M1+EH/tbwxv42GsD1oN1pit+I1JaJeEOStztLrwU8fHYohyME8K6/KfXs+BS1RNdHisVxJb880mb6adWfpsSDkM2eQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3912sffFv0bNrTJYcPGQ2ATrVX945IvVFvT3Ia9ps+E=; b=T7rvewSMUFpv8PWT5fpb5Bmz6xRM1QPUGVrjgfKZEz4UtT3zvSlAqWTf85FG9yt7spfVZxlWnMR9lSzgFjprNQ5sl83TRT74kp3kzU6inXHHlLzERkl44WXs0EdQ/FY0BMLkTRz3/23hxpVGGm9z555fcKkZZqEANecjAia4/+M= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MWHPR18MB1279.namprd18.prod.outlook.com (2603:10b6:320:2c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.20; Wed, 22 Jul 2020 14:48:55 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:48:55 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 02/13] task_isolation: vmstat: add vmstat_idle function Thread-Topic: [PATCH v4 02/13] task_isolation: vmstat: add vmstat_idle function Thread-Index: AQHWYDc5m+yfBf+oREWWYu7pQOOLOw== Date: Wed, 22 Jul 2020 14:48:55 +0000 Message-ID: <072c0cd559c6ad91898e4982bc32cc8918a3dde0.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f637b885-826a-4f4c-9d97-08d82e4e5c0d x-ms-traffictypediagnostic: MWHPR18MB1279: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: b2zwlKk8oDGdaHMxPtptWq4U3rh1iQKJz/x3mTWUQTC6dM7UEZpcxYHNpdEVwL94Jk0ObYxFcX7oPSp2gLeO2Y0WeubSiXebmTFRcpF75caC95IzTVDrHx84YxFNhdYVAlDWhDeC92dMciRiOynN6P456qtnJ+Kxa3NNR0GEn+FebC7StATxevCeAWmtvH1NQgfK8k+Y2gDW4WLK5LfI/KVSWX/Jvn9gmrf/L6IrAztdXAh4Y+W4NMqSzyhqIdnqe2xdl6sE0Zj9onIWUTEH9i+NaVRBeIiDw0bYIGUG/R9H5DB41S3CHpiHBxwhymWj2VWhPZ39MBv6mScW1X0CDw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(396003)(366004)(39850400004)(376002)(346002)(136003)(54906003)(26005)(316002)(110136005)(2616005)(71200400001)(8936002)(7416002)(86362001)(2906002)(5660300002)(4326008)(64756008)(66556008)(66446008)(6486002)(478600001)(36756003)(6512007)(8676002)(186003)(76116006)(91956017)(66946007)(83380400001)(66476007)(6506007)(53546011);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: ExTWt12KICcHsm166pas+6cBDToHgWWgywTkyiHLCT2RrS3MSdv2uiMQAMSpjgqBZrcUh1Z6UhEk8FLfL5qoATZiomrZf0k15g+ySdxqthkB2zFY8okVbxkoJ6mgNwBLk54PX0caRXNfOUexfrInfEKqM5okmoitSGOwhgdiA4iiFEtJIrCxvgAmgDRsRNT6GpAoI3QOkgmqF5CERcOIg0janbQ3onhPnidC2Hu5Op4LBul3p60GBn6KBRfNwTxmsINZOkcVPtJTfAIm7JVFdH7InxpMPAGhoWP5Esqh/svBl1BDrvHmRCugNfdrRfd/vvqTHeULTiDm2TZfeTvjlyAUDoc3M6hneyAMBnBboGXhLHZYyVLX706aImsELOSk8qGTvWgIrxq8e6FopybdH8nWtyIc5+C4lNq5wTgCtKyK0P4swH9SNEHzAyspv6yje4yjWecb3ZzqKexzlCXCKKegLzmlDtULdvDicf/AdUg= Content-ID: <36C0BDF44ACEAB4DBB21BDD43F5569D5@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: f637b885-826a-4f4c-9d97-08d82e4e5c0d X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:48:55.8398 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: S7Uny6mj+I16UOe4oDlhzaCgE714qQ7hpIobKp6b+nbxQFiOD57n+6NvCKHi17zR/Gc92m3WV0uPKg2K10sRng== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1279 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From 7823be8cd3ba2e66308f334a2e47f60ba7829e0b Mon Sep 17 00:00:00 2001 From: Chris Metcalf Date: Sat, 1 Feb 2020 08:05:45 +0000 Subject: [PATCH 02/13] task_isolation: vmstat: add vmstat_idle function This function checks to see if a vmstat worker is not running, and the vmstat diffs don't require an update. The function is called from the task-isolation code to see if we need to actually do some work to quiet vmstat. Signed-off-by: Chris Metcalf Signed-off-by: Alex Belits --- include/linux/vmstat.h | 2 ++ mm/vmstat.c | 10 ++++++++++ 2 files changed, 12 insertions(+) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index ded16dfd21fa..97bc9ed92036 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -273,6 +273,7 @@ extern void __dec_node_state(struct pglist_data *, enum node_stat_item); void quiet_vmstat(void); void quiet_vmstat_sync(void); +bool vmstat_idle(void); void cpu_vm_stats_fold(int cpu); void refresh_zone_stat_thresholds(void); @@ -376,6 +377,7 @@ static inline void refresh_zone_stat_thresholds(void) { } static inline void cpu_vm_stats_fold(int cpu) { } static inline void quiet_vmstat(void) { } static inline void quiet_vmstat_sync(void) { } +static inline bool vmstat_idle(void) { return true; } static inline void drain_zonestat(struct zone *zone, struct per_cpu_pageset *pset) { } diff --git a/mm/vmstat.c b/mm/vmstat.c index 93534f8537ca..f3693ef0a958 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1898,6 +1898,16 @@ void quiet_vmstat_sync(void) refresh_cpu_vm_stats(false); } +/* + * Report on whether vmstat processing is quiesced on the core currently: + * no vmstat worker running and no vmstat updates to perform. + */ +bool vmstat_idle(void) +{ + return !delayed_work_pending(this_cpu_ptr(&vmstat_work)) && + !need_update(smp_processor_id()); +} + /* * Shepherd worker thread that checks the * differentials of processors that have their worker From patchwork Wed Jul 22 14:49:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333923 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=twWtqgzD; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=h++XFkLc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdgY5Sxmz9sSt for ; Thu, 23 Jul 2020 00:50:49 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732699AbgGVOuk (ORCPT ); Wed, 22 Jul 2020 10:50:40 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:8876 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729642AbgGVOui (ORCPT ); Wed, 22 Jul 2020 10:50:38 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MElg26017001; Wed, 22 Jul 2020 07:49:53 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=m0IUk8iaeUhUl5eO2mz5p024/N6jQIm9hWoauzfqJsY=; b=twWtqgzDgfyyO3TOPI5ukNMatxuqQGD5ml+s+rAOyuq/sTKhFItscvKxye5LD0gewDU+ SxsM2Vdwit3DvWRKEYC6ooS+IEUJjaWCU6bGsVqOMBxLgEjQEQ6k0b1vqOHEeCiAWXlf hmxKdrHrBdzMMu4Ed3W7KibE4cZ7EksepDMTwoF7UI0H5/0Uc0WWoWQ7V9CITziby7F5 TXQaflUljWXKa2JUD1QW68HwmO1TcC7KFunuF5/S6VFip0z2W6OPzxyr5H5f/MZuzo1w Fwgogb0cV9uct38TnOn29fyNxfZjsW3Xgvg7D334RlJp8j6Ye8OdW5nXjfgzLb6ue8ZB dw== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens6sq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:49:52 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:49:51 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.174) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:49:51 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KmWSKykihW+M9B8df4Y2PHojgmacjkWt66FUjfbx/7Yta/UHyotw+pIb1V3ttQ+rMIkXMWULJiknPFcIQoR79Dxa4neAErsaJZWZTsGeQ2BAChXMzA/cPyGGnZgk5IAiOh6C+bgL7hvj5y2JvpbCmUg58hTfuWcGUkhZlnGmx1GYkvV5bHasAh9/mYt8PoT+Ykc+9oHMboijlWnuUmQcCml1z90bznv2G+v0hzMMspnLZle7rXfB+2A60kz/A5kWBhEUF36UN8CAdLVWKVozPE9Obf1mg5VdL1LGjrfnHZ+0M6CHdsmNKl5EpgHe8gfw4ey/k3rE8f7FMz+BFeeUJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0IUk8iaeUhUl5eO2mz5p024/N6jQIm9hWoauzfqJsY=; b=PMrjimCHmZrnqqxOPDOI/lh5ziqzQN1K/QQIgSTmPJsjlEYsrCNoSITaVDcyW9O5FnsP406iygM3oTeN/neLScf2MNfZGXarkS0LgrFA90uhmZMooGyFeubXsrOY5SC+AbkQmkB4hCV2Q/E5MVp8cIxZDXi/qjvWj04+oUPhnOOM8OGRh0NwWeMJszNxBE5Bk8gRHW638zKeVdmKlqn9QzFdAaMIoEqSrLEZLXr0v2yT+PdxEJFdWHAuck4zypRjcYsKmwxWU24h/KvosGFkhQcyuUUa1a9HIbDV8jOa3mezvWMu7e4AnUbywM/lIpk4egaWWTwpOVYdI8RCMZ2v5w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m0IUk8iaeUhUl5eO2mz5p024/N6jQIm9hWoauzfqJsY=; b=h++XFkLcBbk9v3eBd1W5tuyoww1JPneEOBRH+MoKoJ0tCLCVW2NA0c+MFTzSX5gq4es0u1aNE2LCCnNxPV+HErySJetti0X6fiBcZNmUYAv3XOt+DTWahU/65OIMGV7hR5oq7Zc2kMOkFQFMPvoRew1/AukQFhBroCN2E996sCU= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:49:49 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:49:49 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 03/13] task_isolation: userspace hard isolation from kernel Thread-Topic: [PATCH v4 03/13] task_isolation: userspace hard isolation from kernel Thread-Index: AQHWYDdZDqoeqvfF50CzG3l43t/UoQ== Date: Wed, 22 Jul 2020 14:49:49 +0000 Message-ID: References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5305314b-6572-4836-1a1c-08d82e4e7bfb x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: styNdZUpa8KzBHSTTwjAGj/VGgx54Nc4YxCCim68RAYrw/G4bLkUk7iJh3Yojqct3t5oV92dyqJWD+7paoi4bk+VnhoMwaRn+Ol7hUlzBJcnX4KovyhRwBOcWKCKtYybsKS5oHn1kdOoC/ZlVjcG9O8lJ3kS8mebtaPIv0ZQJmyA7aJf0aB3Rm8q0Pbwq+SNENdEodqlMxBTcyQbNVmr1ScG5P4OvJTN+3uF4EWCdMRc/f8C4FX1AcF22xva8qJob5ldfwK951syNpfQhDjqsjWX1+2UHRiirNv+KBQWtpUHAoRfkWO289hpWXQ+TfiCzwqp02e6Chdn3mvNx6/cqYhAFGl3mug/H3JZdGYwn3F59a/pdV1GmIHyzwUu0iHQ x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(346002)(136003)(396003)(376002)(39860400002)(366004)(76116006)(91956017)(83380400001)(86362001)(71200400001)(66556008)(26005)(5660300002)(36756003)(66446008)(7416002)(478600001)(6512007)(8936002)(6506007)(110136005)(6486002)(2906002)(54906003)(66476007)(30864003)(186003)(64756008)(2616005)(8676002)(4326008)(66946007)(316002)(21314003)(579004)(559001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: mV7wDiTIH4W8RhSe2OPepVQqwyvJ7oL4+3Uvp0tzOB2SMc9q7YLpVs3MdQRQrmZm/g5Anbya9hRIFHVfT9D0NmathS5BBrwdAPqGItPJnv1fy9WVWa5BjXIBcBHfzzHxpza2SZIVnMLMUYs5AJk/rSme6Ixq2iKCg6V9cG59TgYleH/HDJoiegLZnKhVVMML5Pqbtd4BRYWcmnpvVdgeX0wzI8QQpkHp5XrmeoXnQ/Ps5XXcMGDaJ4a24YPp5cEWr212PNtSZIEpY2o9CUw7/bS4pX5vbC59f3EoU08a7ZqwVruitGGVq7ug8vX0YaUNJt905X93T8thmjq5HalmSpPAgfwwK49bbb1Es8Klbg/FXKd+kxHTAhVt7J86huJLPeWnJklguJDrSF5eyflV/fdXlqX3odtIPVGDeJ+S9z7cYJd74iPU5AwQ4CxEOZP767GSF3nE4nWX6u/KaflILcD919BCxxgSUfvlTKyMdWk= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5305314b-6572-4836-1a1c-08d82e4e7bfb X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:49:49.4306 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 2xxJ1wCYI7zelQ0/V8DAXIk+lhvjDcCFwY7OglLrA8Hi6kJthcY/EGszF59bhAHUPJh9nq55JPcWan6B3D+kKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The existing nohz_full mode is designed as a "soft" isolation mode that makes tradeoffs to minimize userspace interruptions while still attempting to avoid overheads in the kernel entry/exit path, to provide 100% kernel semantics, etc. However, some applications require a "hard" commitment from the kernel to avoid interruptions, in particular userspace device driver style applications, such as high-speed networking code. This change introduces a framework to allow applications to elect to have the "hard" semantics as needed, specifying prctl(PR_TASK_ISOLATION, PR_TASK_ISOLATION_ENABLE) to do so. The kernel must be built with the new TASK_ISOLATION Kconfig flag to enable this mode, and the kernel booted with an appropriate "isolcpus=nohz,domain,CPULIST" boot argument to enable nohz_full and isolcpus. The "task_isolation" state is then indicated by setting a new task struct field, task_isolation_flag, to the value passed by prctl(), and also setting a TIF_TASK_ISOLATION bit in the thread_info flags. When the kernel is returning to userspace from the prctl() call and sees TIF_TASK_ISOLATION set, it calls the new task_isolation_start() routine to arrange for the task to avoid being interrupted in the future. With interrupts disabled, task_isolation_start() ensures that kernel subsystems that might cause a future interrupt are quiesced. If it doesn't succeed, it adjusts the syscall return value to indicate that fact, and userspace can retry as desired. In addition to stopping the scheduler tick, the code takes any actions that might avoid a future interrupt to the core, such as a worker thread being scheduled that could be quiesced now (e.g. the vmstat worker) or a future IPI to the core to clean up some state that could be cleaned up now (e.g. the mm lru per-cpu cache). Once the task has returned to userspace after issuing the prctl(), if it enters the kernel again via system call, page fault, or any other exception or irq, the kernel will send it a signal to indicate isolation loss. In addition to sending a signal, the code supports a kernel command-line "task_isolation_debug" flag which causes a stack backtrace to be generated whenever a task loses isolation. To allow the state to be entered and exited, the syscall checking test ignores the prctl(PR_TASK_ISOLATION) syscall so that we can clear the bit again later, and ignores exit/exit_group to allow exiting the task without a pointless signal being delivered. The prctl() API allows for specifying a signal number to use instead of the default SIGKILL, to allow for catching the notification signal; for example, in a production environment, it might be helpful to log information to the application logging mechanism before exiting. Or, the signal handler might choose to reset the program counter back to the code segment intended to be run isolated via prctl() to continue execution. In a number of cases we can tell on a remote cpu that we are going to be interrupting the cpu, e.g. via an IPI or a TLB flush. In that case we generate the diagnostic (and optional stack dump) on the remote core to be able to deliver better diagnostics. If the interrupt is not something caught by Linux (e.g. a hypervisor interrupt) we can also request a reschedule IPI to be sent to the remote core so it can be sure to generate a signal to notify the process. Isolation also disables CPU state synchronization mechanisms that are. normally done by IPI. In the future, more synchronization mechanisms, such as TLB flushes, may be disabled for isolated tasks. This requires careful handling of kernel entry from isolated task -- remote synchronization requests must be re-enabled and synchronization procedure triggered, before anything other than low-level kernel entry code is called. Same applies to exiting from kernel to userspace after isolation is enabled -- either the code should not depend on synchronization, or isolation should be broken. For this purpose, per-CPU low-level flags ll_isol_flags are used to indicate isolation state, and task_isolation_kernel_enter() is used to safely clear them early in kernel entry. CPU mask corresponding to isolation bit in ll_isol_flags is visible to userspace as /sys/devices/system/cpu/isolation_running, and can be used for monitoring. Separate patches that follow provide these changes for x86, arm, and arm64 architectures, xen and irqchip drivers. Signed-off-by: Alex Belits --- .../admin-guide/kernel-parameters.txt | 6 + drivers/base/cpu.c | 23 + include/linux/hrtimer.h | 4 + include/linux/isolation.h | 295 ++++++ include/linux/sched.h | 5 + include/linux/tick.h | 3 + include/uapi/linux/prctl.h | 6 + init/Kconfig | 28 + kernel/Makefile | 2 + kernel/isolation.c | 841 ++++++++++++++++++ kernel/signal.c | 2 + kernel/sys.c | 6 + kernel/time/hrtimer.c | 27 + kernel/time/tick-sched.c | 18 + 14 files changed, 1266 insertions(+) create mode 100644 include/linux/isolation.h create mode 100644 kernel/isolation.c diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index fb95fad81c79..321f9d21a99a 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -5015,6 +5015,12 @@ neutralize any effect of /proc/sys/kernel/sysrq. Useful for debugging. + task_isolation_debug [KNL] + In kernels built with CONFIG_TASK_ISOLATION, this + setting will generate console backtraces to + accompany the diagnostics generated about + interrupting tasks running with task isolation. + tcpmhash_entries= [KNL,NET] Set the number of tcp_metrics_hash slots. Default value is 8192 or 16384 depending on total diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c index d2136ab9b14a..733b0e7d598c 100644 --- a/drivers/base/cpu.c +++ b/drivers/base/cpu.c @@ -287,6 +287,26 @@ static ssize_t print_cpus_isolated(struct device *dev, } static DEVICE_ATTR(isolated, 0444, print_cpus_isolated, NULL); +#ifdef CONFIG_TASK_ISOLATION +static ssize_t print_isolation_running(struct device *dev, + struct device_attribute *attr, char *buf) +{ + int n; + cpumask_var_t isolation_running; + + if (!zalloc_cpumask_var(&isolation_running, GFP_KERNEL)) + return -ENOMEM; + + task_isolation_cpumask(isolation_running); + n = sprintf(buf, "%*pbl\n", cpumask_pr_args(isolation_running)); + + free_cpumask_var(isolation_running); + + return n; +} +static DEVICE_ATTR(isolation_running, 0444, print_isolation_running, NULL); +#endif + #ifdef CONFIG_NO_HZ_FULL static ssize_t print_cpus_nohz_full(struct device *dev, struct device_attribute *attr, char *buf) @@ -473,6 +493,9 @@ static struct attribute *cpu_root_attrs[] = { #ifdef CONFIG_NO_HZ_FULL &dev_attr_nohz_full.attr, #endif +#ifdef CONFIG_TASK_ISOLATION + &dev_attr_isolation_running.attr, +#endif #ifdef CONFIG_GENERIC_CPU_AUTOPROBE &dev_attr_modalias.attr, #endif diff --git a/include/linux/hrtimer.h b/include/linux/hrtimer.h index 15c8ac313678..e81252eb4f92 100644 --- a/include/linux/hrtimer.h +++ b/include/linux/hrtimer.h @@ -528,6 +528,10 @@ extern void __init hrtimers_init(void); /* Show pending timers: */ extern void sysrq_timer_list_show(void); +#ifdef CONFIG_TASK_ISOLATION +extern void kick_hrtimer(void); +#endif + int hrtimers_prepare_cpu(unsigned int cpu); #ifdef CONFIG_HOTPLUG_CPU int hrtimers_dead_cpu(unsigned int cpu); diff --git a/include/linux/isolation.h b/include/linux/isolation.h new file mode 100644 index 000000000000..a92ff200db12 --- /dev/null +++ b/include/linux/isolation.h @@ -0,0 +1,295 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Task isolation support + * + * Authors: + * Chris Metcalf + * Alex Belits + * Yuri Norov + */ +#ifndef _LINUX_ISOLATION_H +#define _LINUX_ISOLATION_H + +#include +#include +#include +#include +#include +#include +#include + +struct task_struct; + +#ifdef CONFIG_TASK_ISOLATION + +int task_isolation_message(int cpu, int level, bool supp, const char *fmt, ...); + +#define pr_task_isol_emerg(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_EMERG, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_alert(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_ALERT, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_crit(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_CRIT, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_err(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_ERR, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_warn(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_WARNING, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_notice(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_NOTICE, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_info(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_INFO, false, fmt, ##__VA_ARGS__) +#define pr_task_isol_debug(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_DEBUG, false, fmt, ##__VA_ARGS__) + +#define pr_task_isol_emerg_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_EMERG, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_alert_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_ALERT, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_crit_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_CRIT, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_err_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_ERR, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_warn_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_WARNING, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_notice_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_NOTICE, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_info_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_INFO, true, fmt, ##__VA_ARGS__) +#define pr_task_isol_debug_supp(cpu, fmt, ...) \ + task_isolation_message(cpu, LOGLEVEL_DEBUG, true, fmt, ##__VA_ARGS__) + +#define BIT_LL_TASK_ISOLATION (0) +#define FLAG_LL_TASK_ISOLATION (1 << BIT_LL_TASK_ISOLATION) + +DECLARE_PER_CPU(unsigned long, ll_isol_flags); +extern cpumask_var_t task_isolation_map; + +/** + * task_isolation_request() - prctl hook to request task isolation + * @flags: Flags from PR_TASK_ISOLATION_xxx. + * + * This is called from the generic prctl() code for PR_TASK_ISOLATION. + * + * Return: Returns 0 when task isolation enabled, otherwise a negative + * errno. + */ +extern int task_isolation_request(unsigned int flags); + +/** + * task_isolation_kernel_enter() - clear low-level task isolation flag + * + * This should be called immediately after entering kernel. + */ +static inline void task_isolation_kernel_enter(void) +{ + unsigned long flags; + + /* + * This function runs on a CPU that ran isolated task. + * + * We don't want this CPU running code from the rest of kernel + * until other CPUs know that it is no longer isolated. + * When CPU is running isolated task until this point anything + * that causes an interrupt on this CPU must end up calling this + * before touching the rest of kernel. That is, this function or + * fast_task_isolation_cpu_cleanup() or stop_isolation() calling + * it. If any interrupt, including scheduling timer, arrives, it + * will still end up here early after entering kernel. + * From this point interrupts are disabled until all CPUs will see + * that this CPU is no longer running isolated task. + * + * See also fast_task_isolation_cpu_cleanup(). + */ + smp_rmb(); + if((this_cpu_read(ll_isol_flags) & FLAG_LL_TASK_ISOLATION) == 0) + return; + + local_irq_save(flags); + + /* Clear low-level flags */ + this_cpu_write(ll_isol_flags, 0); + + /* + * If something happened that requires a barrier that would + * otherwise be called from remote CPUs by CPU kick procedure, + * this barrier runs instead of it. After this barrier, CPU + * kick procedure would see the updated ll_isol_flags, so it + * will run its own IPI to trigger a barrier. + */ + smp_mb(); + /* + * Synchronize instructions -- this CPU was not kicked while + * in isolated mode, so it might require synchronization. + * There might be an IPI if kick procedure happened and + * ll_isol_flags was already updated while it assembled a CPU + * mask. However if this did not happen, synchronize everything + * here. + */ + instr_sync(); + local_irq_restore(flags); +} + +extern void task_isolation_cpu_cleanup(void); +/** + * task_isolation_start() - attempt to actually start task isolation + * + * This function should be invoked as the last thing prior to returning to + * user space if TIF_TASK_ISOLATION is set in the thread_info flags. It + * will attempt to quiesce the core and enter task-isolation mode. If it + * fails, it will reset the system call return value to an error code that + * indicates the failure mode. + */ +extern void task_isolation_start(void); + +/** + * is_isolation_cpu() - check if CPU is intended for running isolated tasks. + * @cpu: CPU to check. + */ +static inline bool is_isolation_cpu(int cpu) +{ + return task_isolation_map != NULL && + cpumask_test_cpu(cpu, task_isolation_map); +} + +/** + * task_isolation_on_cpu() - check if the cpu is running isolated task + * @cpu: CPU to check. + */ +static inline int task_isolation_on_cpu(int cpu) +{ + return test_bit(BIT_LL_TASK_ISOLATION, &per_cpu(ll_isol_flags, cpu)); +} + +extern void task_isolation_check_run_cleanup(void); + +/** + * task_isolation_cpumask() - set CPUs currently running isolated tasks + * @mask: Mask to modify. + */ +extern void task_isolation_cpumask(struct cpumask *mask); + +/** + * task_isolation_clear_cpumask() - clear CPUs currently running isolated tasks + * @mask: Mask to modify. + */ +extern void task_isolation_clear_cpumask(struct cpumask *mask); + +/** + * task_isolation_syscall() - report a syscall from an isolated task + * @nr: The syscall number. + * + * This routine should be invoked at syscall entry if TIF_TASK_ISOLATION is + * set in the thread_info flags. It checks for valid syscalls, + * specifically prctl() with PR_TASK_ISOLATION, exit(), and exit_group(). + * For any other syscall it will raise a signal and return failure. + * + * Return: 0 for acceptable syscalls, -1 for all others. + */ +extern int task_isolation_syscall(int nr); + +/** + * _task_isolation_interrupt() - report an interrupt of an isolated task + * @fmt: A format string describing the interrupt + * @...: Format arguments, if any. + * + * This routine should be invoked at any exception or IRQ if + * TIF_TASK_ISOLATION is set in the thread_info flags. It is not necessary + * to invoke it if the exception will generate a signal anyway (e.g. a bad + * page fault), and in that case it is preferable not to invoke it but just + * rely on the standard Linux signal. The macro task_isolation_syscall() + * wraps the TIF_TASK_ISOLATION flag test to simplify the caller code. + */ +extern void _task_isolation_interrupt(const char *fmt, ...); +#define task_isolation_interrupt(fmt, ...) \ + do { \ + if (current_thread_info()->flags & _TIF_TASK_ISOLATION) \ + _task_isolation_interrupt(fmt, ## __VA_ARGS__); \ + } while (0) + +/** + * task_isolation_remote() - report a remote interrupt of an isolated task + * @cpu: The remote cpu that is about to be interrupted. + * @fmt: A format string describing the interrupt + * @...: Format arguments, if any. + * + * This routine should be invoked any time a remote IPI or other type of + * interrupt is being delivered to another cpu. The function will check to + * see if the target core is running a task-isolation task, and generate a + * diagnostic on the console if so; in addition, we tag the task so it + * doesn't generate another diagnostic when the interrupt actually arrives. + * Generating a diagnostic remotely yields a clearer indication of what + * happened then just reporting only when the remote core is interrupted. + * + */ +extern void task_isolation_remote(int cpu, const char *fmt, ...); + +/** + * task_isolation_remote_cpumask() - report interruption of multiple cpus + * @mask: The set of remotes cpus that are about to be interrupted. + * @fmt: A format string describing the interrupt + * @...: Format arguments, if any. + * + * This is the cpumask variant of _task_isolation_remote(). We + * generate a single-line diagnostic message even if multiple remote + * task-isolation cpus are being interrupted. + */ +extern void task_isolation_remote_cpumask(const struct cpumask *mask, + const char *fmt, ...); + +/** + * _task_isolation_signal() - disable task isolation when signal is pending + * @task: The task for which to disable isolation. + * + * This function generates a diagnostic and disables task isolation; it + * should be called if TIF_TASK_ISOLATION is set when notifying a task of a + * pending signal. The task_isolation_interrupt() function normally + * generates a diagnostic for events that just interrupt a task without + * generating a signal; here we need to hook the paths that correspond to + * interrupts that do generate a signal. The macro task_isolation_signal() + * wraps the TIF_TASK_ISOLATION flag test to simplify the caller code. + */ +extern void _task_isolation_signal(struct task_struct *task); +#define task_isolation_signal(task) \ + do { \ + if (task_thread_info(task)->flags & _TIF_TASK_ISOLATION) \ + _task_isolation_signal(task); \ + } while (0) + +/** + * task_isolation_user_exit() - debug all user_exit calls + * + * By default, we don't generate an exception in the low-level user_exit() + * code, because programs lose the ability to disable task isolation: the + * user_exit() hook will cause a signal prior to task_isolation_syscall() + * disabling task isolation. In addition, it means that we lose all the + * diagnostic info otherwise available from task_isolation_interrupt() hooks + * later in the interrupt-handling process. But you may enable it here for + * a special kernel build if you are having undiagnosed userspace jitter. + */ +static inline void task_isolation_user_exit(void) +{ +#ifdef DEBUG_TASK_ISOLATION + task_isolation_interrupt("user_exit"); +#endif +} + +#else /* !CONFIG_TASK_ISOLATION */ +static inline int task_isolation_request(unsigned int flags) { return -EINVAL; } +static inline void task_isolation_kernel_enter(void) {} +static inline void task_isolation_start(void) { } +static inline bool is_isolation_cpu(int cpu) { return 0; } +static inline int task_isolation_on_cpu(int cpu) { return 0; } +static inline void task_isolation_cpumask(struct cpumask *mask) { } +static inline void task_isolation_clear_cpumask(struct cpumask *mask) { } +static inline void task_isolation_cpu_cleanup(void) { } +static inline void task_isolation_check_run_cleanup(void) { } +static inline int task_isolation_syscall(int nr) { return 0; } +static inline void task_isolation_interrupt(const char *fmt, ...) { } +static inline void task_isolation_remote(int cpu, const char *fmt, ...) { } +static inline void task_isolation_remote_cpumask(const struct cpumask *mask, + const char *fmt, ...) { } +static inline void task_isolation_signal(struct task_struct *task) { } +static inline void task_isolation_user_exit(void) { } +#endif + +#endif /* _LINUX_ISOLATION_H */ diff --git a/include/linux/sched.h b/include/linux/sched.h index 683372943093..7fb7bb3fddaa 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1301,6 +1301,11 @@ struct task_struct { unsigned long prev_lowest_stack; #endif +#ifdef CONFIG_TASK_ISOLATION + unsigned short task_isolation_flags; /* prctl */ + unsigned short task_isolation_state; +#endif + #ifdef CONFIG_X86_MCE u64 mce_addr; __u64 mce_ripv : 1, diff --git a/include/linux/tick.h b/include/linux/tick.h index 7340613c7eff..27c7c033d5a8 100644 --- a/include/linux/tick.h +++ b/include/linux/tick.h @@ -268,6 +268,9 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal, extern void tick_nohz_full_kick_cpu(int cpu); extern void __tick_nohz_task_switch(void); extern void __init tick_nohz_full_setup(cpumask_var_t cpumask); +#ifdef CONFIG_TASK_ISOLATION +extern int try_stop_full_tick(void); +#endif #else static inline bool tick_nohz_full_enabled(void) { return false; } static inline bool tick_nohz_full_cpu(int cpu) { return false; } diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 07b4f8131e36..f4848ed2a069 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -238,4 +238,10 @@ struct prctl_mm_map { #define PR_SET_IO_FLUSHER 57 #define PR_GET_IO_FLUSHER 58 +/* Enable task_isolation mode for TASK_ISOLATION kernels. */ +#define PR_TASK_ISOLATION 48 +# define PR_TASK_ISOLATION_ENABLE (1 << 0) +# define PR_TASK_ISOLATION_SET_SIG(sig) (((sig) & 0x7f) << 8) +# define PR_TASK_ISOLATION_GET_SIG(bits) (((bits) >> 8) & 0x7f) + #endif /* _LINUX_PRCTL_H */ diff --git a/init/Kconfig b/init/Kconfig index 0498af567f70..7cbfe3eae3b6 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -620,6 +620,34 @@ config CPU_ISOLATION source "kernel/rcu/Kconfig" +config HAVE_ARCH_TASK_ISOLATION + bool + +config TASK_ISOLATION + bool "Provide hard CPU isolation from the kernel on demand" + depends on NO_HZ_FULL && HAVE_ARCH_TASK_ISOLATION + help + + Allow userspace processes that place themselves on cores with + nohz_full and isolcpus enabled, and run prctl(PR_TASK_ISOLATION), + to "isolate" themselves from the kernel. Prior to returning to + userspace, isolated tasks will arrange that no future kernel + activity will interrupt the task while the task is running in + userspace. Attempting to re-enter the kernel while in this mode + will cause the task to be terminated with a signal; you must + explicitly use prctl() to disable task isolation before resuming + normal use of the kernel. + + This "hard" isolation from the kernel is required for userspace + tasks that are running hard real-time tasks in userspace, such as + a high-speed network driver in userspace. Without this option, but + with NO_HZ_FULL enabled, the kernel will make a best-faith, "soft" + effort to shield a single userspace process from interrupts, but + makes no guarantees. + + You should say "N" unless you are intending to run a + high-performance userspace driver or similar task. + config BUILD_BIN2C bool default n diff --git a/kernel/Makefile b/kernel/Makefile index f3218bc5ec69..546d8d83cc2f 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -130,6 +130,8 @@ KASAN_SANITIZE_stackleak.o := n KCSAN_SANITIZE_stackleak.o := n KCOV_INSTRUMENT_stackleak.o := n +obj-$(CONFIG_TASK_ISOLATION) += isolation.o + $(obj)/configs.o: $(obj)/config_data.gz targets += config_data.gz diff --git a/kernel/isolation.c b/kernel/isolation.c new file mode 100644 index 000000000000..afeff490b76c --- /dev/null +++ b/kernel/isolation.c @@ -0,0 +1,841 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * linux/kernel/isolation.c + * + * Implementation of task isolation. + * + * Authors: + * Chris Metcalf + * Alex Belits + * Yuri Norov + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* + * These values are stored in task_isolation_state. + * Note that STATE_NORMAL + TIF_TASK_ISOLATION means we are still + * returning from sys_prctl() to userspace. + */ +enum { + STATE_NORMAL = 0, /* Not isolated */ + STATE_ISOLATED = 1 /* In userspace, isolated */ +}; + +/* + * Counter for isolation state on a given CPU, increments when entering + * isolation and decrements when exiting isolation (before or after the + * cleanup). Multiple simultaneously running procedures entering or + * exiting isolation are prevented by checking the result of + * incrementing or decrementing this variable. This variable is both + * incremented and decremented by CPU that caused isolation entering or + * exit. + * + * This is necessary because multiple isolation-breaking events may happen + * at once (or one as the result of the other), however isolation exit + * may only happen once to transition from isolated to non-isolated state. + * Therefore, if decrementing this counter results in a value less than 0, + * isolation exit procedure can't be started -- it already happened, or is + * in progress, or isolation is not entered yet. + */ +DEFINE_PER_CPU(atomic_t, isol_counter); + +/* + * Low-level isolation flags. + * Those flags are used by low-level isolation set/clear/check routines. + * Those flags should be set last before return to userspace and cleared + * first upon kernel entry, and synchronized to allow isolation breaking + * detection before touching potentially unsynchronized parts of kernel. + * Isolated task does not receive synchronization events of any kind, so + * at the time of the first entry into kernel it might not be ready to + * run most of the kernel code. However to perform synchronization + * properly, kernel entry code should also enable synchronization events + * at the same time. This presents a problem because more kernel code + * should run to determine the cause of isolation breaking, signals may + * have to be generated, etc. So some flag clearing and synchronization + * should happen in "low-level" entry code but processing of isolation + * breaking should happen in "high-level" code. Low-level isolation flags + * should be set in that low-level code, possibly long before the cause + * of isolation breaking is known. Symmetrically, entering isolation + * should disable synchronization events before returning to userspace + * but after all potentially volatile code is finished. + */ +DEFINE_PER_CPU(unsigned long, ll_isol_flags); + +/* + * Description of the last two tasks that ran isolated on a given CPU. + * This is intended only for messages about isolation breaking. We + * don't want any references to actual task while accessing this from + * CPU that caused isolation breaking -- we know nothing about timing + * and don't want to use locking or RCU. + */ +struct isol_task_desc { + atomic_t curr_index; + atomic_t curr_index_wr; + bool warned[2]; + pid_t pid[2]; + pid_t tgid[2]; + char comm[2][TASK_COMM_LEN]; +}; +static DEFINE_PER_CPU(struct isol_task_desc, isol_task_descs); + +/* + * Counter for isolation exiting procedures (from request to the start of + * cleanup) being attempted at once on a CPU. Normally incrementing of + * this counter is performed from the CPU that caused isolation breaking, + * however decrementing is done from the cleanup procedure, delegated to + * the CPU that is exiting isolation, not from the CPU that caused isolation + * breaking. + * + * If incrementing this counter while starting isolation exit procedure + * results in a value greater than 0, isolation exiting is already in + * progress, and cleanup did not start yet. This means, counter should be + * decremented back, and isolation exit that is already in progress, should + * be allowed to complete. Otherwise, a new isolation exit procedure should + * be started. + */ +DEFINE_PER_CPU(atomic_t, isol_exit_counter); + +/* + * Descriptor for isolation-breaking SMP calls + */ +DEFINE_PER_CPU(call_single_data_t, isol_break_csd); + +cpumask_var_t task_isolation_map; +cpumask_var_t task_isolation_cleanup_map; +static DEFINE_SPINLOCK(task_isolation_cleanup_lock); + +/* We can run on cpus that are isolated from the scheduler and are nohz_full. */ +static int __init task_isolation_init(void) +{ + alloc_bootmem_cpumask_var(&task_isolation_cleanup_map); + if (alloc_cpumask_var(&task_isolation_map, GFP_KERNEL)) + /* + * At this point task isolation should match + * nohz_full. This may change in the future. + */ + cpumask_copy(task_isolation_map, tick_nohz_full_mask); + return 0; +} +core_initcall(task_isolation_init) + +/* Enable stack backtraces of any interrupts of task_isolation cores. */ +static bool task_isolation_debug; +static int __init task_isolation_debug_func(char *str) +{ + task_isolation_debug = true; + return 1; +} +__setup("task_isolation_debug", task_isolation_debug_func); + +/* + * Record name, pid and group pid of the task entering isolation on + * the current CPU. + */ +static void record_curr_isolated_task(void) +{ + int ind; + int cpu = smp_processor_id(); + struct isol_task_desc *desc = &per_cpu(isol_task_descs, cpu); + struct task_struct *task = current; + + /* Finish everything before recording current task */ + smp_mb(); + ind = atomic_inc_return(&desc->curr_index_wr) & 1; + desc->comm[ind][sizeof(task->comm) - 1] = '\0'; + memcpy(desc->comm[ind], task->comm, sizeof(task->comm) - 1); + desc->pid[ind] = task->pid; + desc->tgid[ind] = task->tgid; + desc->warned[ind] = false; + /* Write everything, to be seen by other CPUs */ + smp_mb(); + atomic_inc(&desc->curr_index); + /* Everyone will see the new record from this point */ + smp_mb(); +} + +/* + * Print message prefixed with the description of the current (or + * last) isolated task on a given CPU. Intended for isolation breaking + * messages that include target task for the user's convenience. + * + * Messages produced with this function may have obsolete task + * information if isolated tasks managed to exit, start and enter + * isolation multiple times, or multiple tasks tried to enter + * isolation on the same CPU at once. For those unusual cases it would + * contain a valid description of the cause for isolation breaking and + * target CPU number, just not the correct description of which task + * ended up losing isolation. + */ +int task_isolation_message(int cpu, int level, bool supp, const char *fmt, ...) +{ + struct isol_task_desc *desc; + struct task_struct *task; + va_list args; + char buf_prefix[TASK_COMM_LEN + 20 + 3 * 20]; + char buf[200]; + int curr_cpu, ind_counter, ind_counter_old, ind; + + curr_cpu = get_cpu(); + desc = &per_cpu(isol_task_descs, cpu); + ind_counter = atomic_read(&desc->curr_index); + + if (curr_cpu == cpu) { + /* + * Message is for the current CPU so current + * task_struct should be used instead of cached + * information. + * + * Like in other diagnostic messages, if issued from + * interrupt context, current will be the interrupted + * task. Unlike other diagnostic messages, this is + * always relevant because the message is about + * interrupting a task. + */ + ind = ind_counter & 1; + if (supp && desc->warned[ind]) { + /* + * If supp is true, skip the message if the + * same task was mentioned in the message + * originated on remote CPU, and it did not + * re-enter isolated state since then (warned + * is true). Only local messages following + * remote messages, likely about the same + * isolation breaking event, are skipped to + * avoid duplication. If remote cause is + * immediately followed by a local one before + * isolation is broken, local cause is skipped + * from messages. + */ + put_cpu(); + return 0; + } + task = current; + snprintf(buf_prefix, sizeof(buf_prefix), + "isolation %s/%d/%d (cpu %d)", + task->comm, task->tgid, task->pid, cpu); + put_cpu(); + } else { + /* + * Message is for remote CPU, use cached information. + */ + put_cpu(); + /* + * Make sure, index remained unchanged while data was + * copied. If it changed, data that was copied may be + * inconsistent because two updates in a sequence could + * overwrite the data while it was being read. + */ + do { + /* Make sure we are reading up to date values */ + smp_mb(); + ind = ind_counter & 1; + snprintf(buf_prefix, sizeof(buf_prefix), + "isolation %s/%d/%d (cpu %d)", + desc->comm[ind], desc->tgid[ind], + desc->pid[ind], cpu); + desc->warned[ind] = true; + ind_counter_old = ind_counter; + /* Record the warned flag, then re-read descriptor */ + smp_mb(); + ind_counter = atomic_read(&desc->curr_index); + /* + * If the counter changed, something was updated, so + * repeat everything to get the current data + */ + } while (ind_counter != ind_counter_old); + } + + va_start(args, fmt); + vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + + switch (level) { + case LOGLEVEL_EMERG: + pr_emerg("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_ALERT: + pr_alert("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_CRIT: + pr_crit("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_ERR: + pr_err("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_WARNING: + pr_warn("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_NOTICE: + pr_notice("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_INFO: + pr_info("%s: %s", buf_prefix, buf); + break; + case LOGLEVEL_DEBUG: + pr_debug("%s: %s", buf_prefix, buf); + break; + default: + /* No message without a valid level */ + return 0; + } + return 1; +} + +/* + * Dump stack if need be. This can be helpful even from the final exit + * to usermode code since stack traces sometimes carry information about + * what put you into the kernel, e.g. an interrupt number encoded in + * the initial entry stack frame that is still visible at exit time. + */ +static void debug_dump_stack(void) +{ + if (task_isolation_debug) + dump_stack(); +} + +/* + * Set the flags word but don't try to actually start task isolation yet. + * We will start it when entering user space in task_isolation_start(). + */ +int task_isolation_request(unsigned int flags) +{ + struct task_struct *task = current; + + /* + * The task isolation flags should always be cleared just by + * virtue of having entered the kernel. + */ + WARN_ON_ONCE(test_tsk_thread_flag(task, TIF_TASK_ISOLATION)); + WARN_ON_ONCE(task->task_isolation_flags != 0); + WARN_ON_ONCE(task->task_isolation_state != STATE_NORMAL); + + task->task_isolation_flags = flags; + if (!(task->task_isolation_flags & PR_TASK_ISOLATION_ENABLE)) + return 0; + + /* We are trying to enable task isolation. */ + set_tsk_thread_flag(task, TIF_TASK_ISOLATION); + + /* + * Shut down the vmstat worker so we're not interrupted later. + * We have to try to do this here (with interrupts enabled) since + * we are canceling delayed work and will call flush_work() + * (which enables interrupts) and possibly schedule(). + */ + quiet_vmstat_sync(); + + /* We return 0 here but we may change that in task_isolation_start(). */ + return 0; +} + +/* + * Perform actions that should be done immediately on exit from isolation. + */ +static void fast_task_isolation_cpu_cleanup(void *info) +{ + unsigned long flags; + + /* + * This function runs on a CPU that ran isolated task. + * + * We don't want this CPU running code from the rest of kernel + * until other CPUs know that it is no longer isolated. + * When CPU is running isolated task until this point anything + * that causes an interrupt on this CPU must end up calling this + * or task_isolation_kernel_enter() before touching the rest of + * kernel. That is, task_isolation_kernel_enter(), IPI to this + * function or stop_isolation() calling it. If any interrupt, + * including scheduling timer, arrives before a call to this + * function, it will still end up in task_isolation_kernel_enter() + * early after entering kernel. + * From this point interrupts are disabled until all CPUs will see + * that this CPU is no longer running isolated task. + * + * See also task_isolation_kernel_enter(). + */ + local_irq_save(flags); + atomic_dec(&per_cpu(isol_exit_counter, smp_processor_id())); + smp_mb__after_atomic(); + /* + * At this point breaking isolation from other CPUs is possible again, + * however interrupts won't arrive until local_irq_restore() + */ + + /* + * This task is no longer isolated (and if by any chance this + * is the wrong task, it's already not isolated) + */ + current->task_isolation_flags = 0; + clear_tsk_thread_flag(current, TIF_TASK_ISOLATION); + + /* Run the rest of cleanup later */ + set_tsk_thread_flag(current, TIF_NOTIFY_RESUME); + + /* Clear low-level flags if they are not cleared yet */ + this_cpu_write(ll_isol_flags, 0); + + /* + * If something happened that requires a barrier that would + * otherwise be called from remote CPUs by CPU kick procedure, + * this barrier runs instead of it. After this barrier, CPU + * kick procedure would see the updated ll_isol_flags, so it + * will run its own IPI to trigger a barrier. + */ + smp_mb(); + /* + * Synchronize instructions -- this CPU was not kicked while + * in isolated mode, so it might require synchronization. + * There might be an IPI if kick procedure happened and + * ll_isol_flags was already updated while it assembled a CPU + * mask. However if this did not happen, synchronize everything + * here. + */ + instr_sync(); + local_irq_restore(flags); +} + +/* Disable task isolation for the specified task. */ +static void stop_isolation(struct task_struct *p) +{ + int cpu, this_cpu; + unsigned long flags; + + this_cpu = get_cpu(); + cpu = task_cpu(p); + if (atomic_inc_return(&per_cpu(isol_exit_counter, cpu)) > 1) { + /* Already exiting isolation */ + atomic_dec(&per_cpu(isol_exit_counter, cpu)); + put_cpu(); + return; + } + + if (p == current) { + p->task_isolation_state = STATE_NORMAL; + fast_task_isolation_cpu_cleanup(NULL); + task_isolation_cpu_cleanup(); + if (atomic_dec_return(&per_cpu(isol_counter, cpu)) < 0) { + /* Is not isolated already */ + atomic_inc(&per_cpu(isol_counter, cpu)); + } + put_cpu(); + } else { + if (atomic_dec_return(&per_cpu(isol_counter, cpu)) < 0) { + /* Is not isolated already */ + atomic_inc(&per_cpu(isol_counter, cpu)); + atomic_dec(&per_cpu(isol_exit_counter, cpu)); + put_cpu(); + return; + } + /* + * Schedule "slow" cleanup. This relies on + * TIF_NOTIFY_RESUME being set + */ + spin_lock_irqsave(&task_isolation_cleanup_lock, flags); + cpumask_set_cpu(cpu, task_isolation_cleanup_map); + spin_unlock_irqrestore(&task_isolation_cleanup_lock, flags); + /* + * Setting flags is delegated to the CPU where + * isolated task is running + * isol_exit_counter will be decremented from there as well. + */ + per_cpu(isol_break_csd, cpu).func = + fast_task_isolation_cpu_cleanup; + per_cpu(isol_break_csd, cpu).info = NULL; + per_cpu(isol_break_csd, cpu).flags = 0; + smp_call_function_single_async(cpu, + &per_cpu(isol_break_csd, cpu)); + put_cpu(); + } +} + +/* + * This code runs with interrupts disabled just before the return to + * userspace, after a prctl() has requested enabling task isolation. + * We take whatever steps are needed to avoid being interrupted later: + * drain the lru pages, stop the scheduler tick, etc. More + * functionality may be added here later to avoid other types of + * interrupts from other kernel subsystems. + * + * If we can't enable task isolation, we update the syscall return + * value with an appropriate error. + */ +void task_isolation_start(void) +{ + int error; + unsigned long flags; + + /* + * We should only be called in STATE_NORMAL (isolation disabled), + * on our way out of the kernel from the prctl() that turned it on. + * If we are exiting from the kernel in another state, it means we + * made it back into the kernel without disabling task isolation, + * and we should investigate how (and in any case disable task + * isolation at this point). We are clearly not on the path back + * from the prctl() so we don't touch the syscall return value. + */ + if (WARN_ON_ONCE(current->task_isolation_state != STATE_NORMAL)) { + /* Increment counter, this will allow isolation breaking */ + if (atomic_inc_return(&per_cpu(isol_counter, + smp_processor_id())) > 1) { + atomic_dec(&per_cpu(isol_counter, smp_processor_id())); + } + //atomic_inc(&per_cpu(isol_counter, smp_processor_id())); + stop_isolation(current); + return; + } + + /* + * Must be affinitized to a single core with task isolation possible. + * In principle this could be remotely modified between the prctl() + * and the return to userspace, so we have to check it here. + */ + if (current->nr_cpus_allowed != 1 || + !is_isolation_cpu(smp_processor_id())) { + error = -EINVAL; + goto error; + } + + /* If the vmstat delayed work is not canceled, we have to try again. */ + if (!vmstat_idle()) { + error = -EAGAIN; + goto error; + } + + /* Try to stop the dynamic tick. */ + error = try_stop_full_tick(); + if (error) + goto error; + + /* Drain the pagevecs to avoid unnecessary IPI flushes later. */ + lru_add_drain(); + + /* + * Task is going to be marked as isolated. This disables IPIs + * used for synchronization, so to avoid inconsistency + * don't let anything interrupt us and issue a barrier at the end. + */ + local_irq_save(flags); + + /* Increment counter, this will allow isolation breaking */ + if (atomic_inc_return(&per_cpu(isol_counter, + smp_processor_id())) > 1) { + atomic_dec(&per_cpu(isol_counter, smp_processor_id())); + } + + /* Record isolated task IDs and name */ + record_curr_isolated_task(); + smp_wmb(); + + /* From this point this is recognized as isolated by other CPUs */ + current->task_isolation_state = STATE_ISOLATED; + this_cpu_write(ll_isol_flags, FLAG_LL_TASK_ISOLATION); + smp_mb(); + local_irq_restore(flags); + /* + * If anything interrupts us at this point, it will trigger + * isolation breaking procedure. + */ + return; + +error: + /* Increment counter, this will allow isolation breaking */ + if (atomic_inc_return(&per_cpu(isol_counter, + smp_processor_id())) > 1) { + atomic_dec(&per_cpu(isol_counter, smp_processor_id())); + } + stop_isolation(current); + syscall_set_return_value(current, current_pt_regs(), error, 0); +} + +/* Stop task isolation on the remote task and send it a signal. */ +static void send_isolation_signal(struct task_struct *task) +{ + int flags = task->task_isolation_flags; + kernel_siginfo_t info = { + .si_signo = PR_TASK_ISOLATION_GET_SIG(flags) ?: SIGKILL, + }; + + stop_isolation(task); + send_sig_info(info.si_signo, &info, task); +} + +/* Only a few syscalls are valid once we are in task isolation mode. */ +static bool is_acceptable_syscall(int syscall) +{ + /* No need to incur an isolation signal if we are just exiting. */ + if (syscall == __NR_exit || syscall == __NR_exit_group) + return true; + + /* Check to see if it's the prctl for isolation. */ + if (syscall == __NR_prctl) { + unsigned long arg[SYSCALL_MAX_ARGS]; + + syscall_get_arguments(current, current_pt_regs(), arg); + if (arg[0] == PR_TASK_ISOLATION) + return true; + } + + return false; +} + +/* + * This routine is called from syscall entry, prevents most syscalls + * from executing, and if needed raises a signal to notify the process. + * + * Note that we have to stop isolation before we even print a message + * here, since otherwise we might end up reporting an interrupt due to + * kicking the printk handling code, rather than reporting the true + * cause of interrupt here. + * + * The message is not suppressed by previous remotely triggered + * messages. + */ +int task_isolation_syscall(int syscall) +{ + struct task_struct *task = current; + + if (is_acceptable_syscall(syscall)) { + stop_isolation(task); + return 0; + } + + send_isolation_signal(task); + + pr_task_isol_warn(smp_processor_id(), + "task_isolation lost due to syscall %d\n", + syscall); + debug_dump_stack(); + + syscall_set_return_value(task, current_pt_regs(), -ERESTARTNOINTR, -1); + return -1; +} + +/* + * This routine is called from any exception or irq that doesn't + * otherwise trigger a signal to the user process (e.g. page fault). + * + * Messages will be suppressed if there is already a reported remote + * cause for isolation breaking, so we don't generate multiple + * confusingly similar messages about the same event. + */ +void _task_isolation_interrupt(const char *fmt, ...) +{ + struct task_struct *task = current; + va_list args; + char buf[100]; + + /* RCU should have been enabled prior to this point. */ + RCU_LOCKDEP_WARN(!rcu_is_watching(), "kernel entry without RCU"); + + /* Are we exiting isolation already? */ + if (atomic_read(&per_cpu(isol_exit_counter, smp_processor_id())) != 0) { + task->task_isolation_state = STATE_NORMAL; + return; + } + /* + * Avoid reporting interrupts that happen after we have prctl'ed + * to enable isolation, but before we have returned to userspace. + */ + if (task->task_isolation_state == STATE_NORMAL) + return; + + va_start(args, fmt); + vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + + /* Handle NMIs minimally, since we can't send a signal. */ + if (in_nmi()) { + task_isolation_kernel_enter(); + pr_task_isol_err(smp_processor_id(), + "isolation: in NMI; not delivering signal\n"); + } else { + send_isolation_signal(task); + } + + if (pr_task_isol_warn_supp(smp_processor_id(), + "task_isolation lost due to %s\n", buf)) + debug_dump_stack(); +} + +/* + * Called before we wake up a task that has a signal to process. + * Needs to be done to handle interrupts that trigger signals, which + * we don't catch with task_isolation_interrupt() hooks. + * + * This message is also suppressed if there was already a remotely + * caused message about the same isolation breaking event. + */ +void _task_isolation_signal(struct task_struct *task) +{ + struct isol_task_desc *desc; + int ind, cpu; + bool do_warn = (task->task_isolation_state == STATE_ISOLATED); + + cpu = task_cpu(task); + desc = &per_cpu(isol_task_descs, cpu); + ind = atomic_read(&desc->curr_index) & 1; + if (desc->warned[ind]) + do_warn = false; + + stop_isolation(task); + + if (do_warn) { + pr_warn("isolation: %s/%d/%d (cpu %d): task_isolation lost due to signal\n", + task->comm, task->tgid, task->pid, cpu); + debug_dump_stack(); + } +} + +/* + * Generate a stack backtrace if we are going to interrupt another task + * isolation process. + */ +void task_isolation_remote(int cpu, const char *fmt, ...) +{ + struct task_struct *curr_task; + va_list args; + char buf[200]; + + smp_rmb(); + if (!is_isolation_cpu(cpu) || !task_isolation_on_cpu(cpu)) + return; + + curr_task = current; + + va_start(args, fmt); + vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + if (pr_task_isol_warn(cpu, + "task_isolation lost due to %s by %s/%d/%d on cpu %d\n", + buf, + curr_task->comm, curr_task->tgid, + curr_task->pid, smp_processor_id())) + debug_dump_stack(); +} + +/* + * Generate a stack backtrace if any of the cpus in "mask" are running + * task isolation processes. + */ +void task_isolation_remote_cpumask(const struct cpumask *mask, + const char *fmt, ...) +{ + struct task_struct *curr_task; + cpumask_var_t warn_mask; + va_list args; + char buf[200]; + int cpu, first_cpu; + + if (task_isolation_map == NULL || + !zalloc_cpumask_var(&warn_mask, GFP_KERNEL)) + return; + + first_cpu = -1; + smp_rmb(); + for_each_cpu_and(cpu, mask, task_isolation_map) { + if (task_isolation_on_cpu(cpu)) { + if (first_cpu < 0) + first_cpu = cpu; + else + cpumask_set_cpu(cpu, warn_mask); + } + } + + if (first_cpu < 0) + goto done; + + curr_task = current; + + va_start(args, fmt); + vsnprintf(buf, sizeof(buf), fmt, args); + va_end(args); + + if (cpumask_weight(warn_mask) == 0) + pr_task_isol_warn(first_cpu, + "task_isolation lost due to %s by %s/%d/%d on cpu %d\n", + buf, curr_task->comm, curr_task->tgid, + curr_task->pid, smp_processor_id()); + else + pr_task_isol_warn(first_cpu, + " and cpus %*pbl: task_isolation lost due to %s by %s/%d/%d on cpu %d\n", + cpumask_pr_args(warn_mask), + buf, curr_task->comm, curr_task->tgid, + curr_task->pid, smp_processor_id()); + debug_dump_stack(); + +done: + free_cpumask_var(warn_mask); +} + +/* + * Set CPUs currently running isolated tasks in CPU mask. + */ +void task_isolation_cpumask(struct cpumask *mask) +{ + int cpu; + + if (task_isolation_map == NULL) + return; + + smp_rmb(); + for_each_cpu(cpu, task_isolation_map) + if (task_isolation_on_cpu(cpu)) + cpumask_set_cpu(cpu, mask); +} + +/* + * Clear CPUs currently running isolated tasks in CPU mask. + */ +void task_isolation_clear_cpumask(struct cpumask *mask) +{ + int cpu; + + if (task_isolation_map == NULL) + return; + + smp_rmb(); + for_each_cpu(cpu, task_isolation_map) + if (task_isolation_on_cpu(cpu)) + cpumask_clear_cpu(cpu, mask); +} + +/* + * Cleanup procedure. The call to this procedure may be delayed. + */ +void task_isolation_cpu_cleanup(void) +{ + kick_hrtimer(); +} + +/* + * Check if cleanup is scheduled on the current CPU, and if so, run it. + * Intended to be called from notify_resume() or another such callback + * on the target CPU. + */ +void task_isolation_check_run_cleanup(void) +{ + int cpu; + unsigned long flags; + + spin_lock_irqsave(&task_isolation_cleanup_lock, flags); + + cpu = smp_processor_id(); + + if (cpumask_test_cpu(cpu, task_isolation_cleanup_map)) { + cpumask_clear_cpu(cpu, task_isolation_cleanup_map); + spin_unlock_irqrestore(&task_isolation_cleanup_lock, flags); + task_isolation_cpu_cleanup(); + } else + spin_unlock_irqrestore(&task_isolation_cleanup_lock, flags); +} diff --git a/kernel/signal.c b/kernel/signal.c index ee22ec78fd6d..d01537c32a67 100644 --- a/kernel/signal.c +++ b/kernel/signal.c @@ -46,6 +46,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -758,6 +759,7 @@ static int dequeue_synchronous_signal(kernel_siginfo_t *info) */ void signal_wake_up_state(struct task_struct *t, unsigned int state) { + task_isolation_signal(t); set_tsk_thread_flag(t, TIF_SIGPENDING); /* * TASK_WAKEKILL also means wake it up in the stopped/traced/killable diff --git a/kernel/sys.c b/kernel/sys.c index 00a96746e28a..6538a744c734 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include @@ -2527,6 +2528,11 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, error = (current->flags & PR_IO_FLUSHER) == PR_IO_FLUSHER; break; + case PR_TASK_ISOLATION: + if (arg3 || arg4 || arg5) + return -EINVAL; + error = task_isolation_request(arg2); + break; default: error = -EINVAL; break; diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index d89da1c7e005..f3d1f3d47242 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -721,6 +722,19 @@ static void retrigger_next_event(void *arg) raw_spin_unlock(&base->lock); } +#ifdef CONFIG_TASK_ISOLATION +void kick_hrtimer(void) +{ + unsigned long flags; + + preempt_disable(); + local_irq_save(flags); + retrigger_next_event(NULL); + local_irq_restore(flags); + preempt_enable(); +} +#endif + /* * Switch to high resolution mode */ @@ -868,8 +882,21 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram) void clock_was_set(void) { #ifdef CONFIG_HIGH_RES_TIMERS +#ifdef CONFIG_TASK_ISOLATION + struct cpumask mask; + + cpumask_clear(&mask); + task_isolation_cpumask(&mask); + cpumask_complement(&mask, &mask); + /* + * Retrigger the CPU local events everywhere except CPUs + * running isolated tasks. + */ + on_each_cpu_mask(&mask, retrigger_next_event, NULL, 1); +#else /* Retrigger the CPU local events everywhere */ on_each_cpu(retrigger_next_event, NULL, 1); +#endif #endif timerfd_clock_was_set(); } diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 3e2dc9b8858c..6e4cd8459f05 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -888,6 +888,24 @@ static void tick_nohz_full_update_tick(struct tick_sched *ts) #endif } +#ifdef CONFIG_TASK_ISOLATION +int try_stop_full_tick(void) +{ + int cpu = smp_processor_id(); + struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); + + /* For an unstable clock, we should return a permanent error code. */ + if (atomic_read(&tick_dep_mask) & TICK_DEP_MASK_CLOCK_UNSTABLE) + return -EINVAL; + + if (!can_stop_full_tick(cpu, ts)) + return -EAGAIN; + + tick_nohz_stop_sched_tick(ts, cpu); + return 0; +} +#endif + static bool can_stop_idle_tick(int cpu, struct tick_sched *ts) { /* From patchwork Wed Jul 22 14:51:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333924 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=I4ayta1S; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=qTlaKXDP; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdjb6TGfz9sSy for ; Thu, 23 Jul 2020 00:52:35 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732730AbgGVOwc (ORCPT ); Wed, 22 Jul 2020 10:52:32 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:39462 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729642AbgGVOw3 (ORCPT ); Wed, 22 Jul 2020 10:52:29 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEnMNB012667; Wed, 22 Jul 2020 07:51:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=I4ayta1SZklGiVgd72KpH7+YDgOMADElXrbJZzNfDfEzURG4N/9ylGST9C6XkP0zuKQu ys9BBxqX//isY+c0MFbitf0KZaARmNTeaxnGb+sqrKygmxvrR6c0bC0WSRdruIHaOj3O nK6LuqSAhKTQhr1QdPIBGxSnALyxASvlUWSGep4WHDBmeCZNc3Ii8A5N9IkWU2xqIvXI iXfQ/0dIWrGM8NLpjbFvT0+cO5JqY0sOBmNWdGpmANfz68RZRp51W3ULRWVNLLMnecCW NsjS7yjix1jsHZCK0g4MLdpf0o1axmyOI/BDhR+x+5LUOsiMmwNU5K6T/9cXXHKgU++w 5w== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkravq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:51:08 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:07 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:05 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.176) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:51:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gCyHnXcCoW6KWHxeUvUey/ghlqp10wKzjLxpF+Esw1vt5/IV+RZH2z5ObsdMmq5ALv+bGRqLSIWrrB9etwq/oKLWIghJEB/yC080/lz0mKveKrZAg3hvQHLM1jucI6NI7PHQqrNb3Nb8DYF3If+0oCs75nysQVrg6L74U9KvTS7JosLyhp+tIpfcnDuu7hk0Muo5oTxA8aFlOAWK68sh5xUy0KBHgJz1oy4p/dy4jbEeIkzWNWePKP68ywu8PdeCTXTp1kL4RkCWzM46jvcfWdFGyf1t5zSQVYxMh/SfiYvWmf12P/rpI86FvkXWbmMSpv5RRNpBPB6HksLXaFpRHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=HKj1IrPKPVbyJN6+KTlscqbNWEIwNeBjr6awFbwbg8Vo5EovhzpfWGcFc6GQPI+uirnWmrYbPtmD8oKY7roYLbqRTfrICShFi5fueEMDQ1zHUSC0GXRUShviXlTh71mWhoHcHEIVUiXjlJ9f0WvkS7eRME2WfuHMW051UpPnL/wckMJoqBvNcxtLtiRkUFgPhMteb7EdUHsPY57udQJudKVh/Mn8Jw86vzATZeYRjJ+j4zG1sXULtDSIfJkkOG2+iZeqLewvFBicGocgHWyMKhimrEyL2QsHxjHywTo5Ft3CO0OuFvAA423LiknKngY6OGnGkkrMeyX96Rwgvyjg3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=qTlaKXDPxBNm28mIgmtS8sJ8MhKcJiW8bugYNQ2dOS7nG/3AuKRRou57wL9iCJkHmrxwP13ZbjgBmKrauAPq7vNKidr9RDa2nkrncg0IyRjp7hHTgAIZbCyRICnXJItWu5WIJUSKWntY+Sg6ph1D2yh9qDKBBE8ddJg5hWjAAas= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MWHPR18MB1279.namprd18.prod.outlook.com (2603:10b6:320:2c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.20; Wed, 22 Jul 2020 14:51:04 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:51:04 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 04/13] task_isolation: Add task isolation hooks to arch-independent code Thread-Topic: [PATCH v4 04/13] task_isolation: Add task isolation hooks to arch-independent code Thread-Index: AQHWYDeGpu5YVSQ/OESYEPtA2Bs+XQ== Date: Wed, 22 Jul 2020 14:51:04 +0000 Message-ID: <5f7a1619df6139e53d744b3755838bbe83ffa96c.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d53a6067-3b56-4d84-05d2-08d82e4ea8ae x-ms-traffictypediagnostic: MWHPR18MB1279: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1002; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: EaB1g3rbA0Kci7IxxhFnjB7vrsNaPo3pEDDmP3vMjDT69Ma8HYum96Ic3CfwS0YG08If61TX3iiZPBmaE8DdZlLGnAoj/nxWLDgBFpvRzVlZpPWHO9rUW6MfTbW6QZ27E2BpzdECklZNRmpjsSddjnSh1R0LVAA3srmXVEdyaeKbsuZnOZrB+nbxzTy5cdKZl4s3ygZ1i4eqRUvjU9Qxp5n4iV/qMjYoxoGzs67jpk958wcN2FoUeozTyNolLlKILqCbqk0/QtHCcCu3DyHggzBJK6ZxVk2IOIThPYl+nNZn3/lZ9PzrhU/njJIk1xdw x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(396003)(366004)(39860400002)(376002)(346002)(136003)(54906003)(26005)(316002)(110136005)(2616005)(71200400001)(8936002)(7416002)(86362001)(2906002)(5660300002)(4326008)(64756008)(66556008)(66446008)(6486002)(478600001)(36756003)(6512007)(8676002)(186003)(76116006)(91956017)(66946007)(83380400001)(66476007)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: ZDyeaytwuWkQ/kE9j+lzRTQxbZPYRLjO3syBVEBFFg8c7VrKjS+jqxzOvXZ7ndVr017G/cZZtz8aqFsdeeWeujZu2JKZtxuVyeH1eyoW7+g9jZEGtFPhHooskBTJD/Cd9WHLBBjbmuf2Vwu3kVHvRfIOzQj3kSWPanpTQuE2wUC5D5f22fm2y/9tf3NvL/LybiEh7GfMUvEAVwuO01AlWc6blKEBjynEQZzdAW5SGmNUHkqYzJRhCOSyGBi1GeJ/jqKnHZGsUo8keQiqj3haJkTHUl5KB6mRJkZ64MOn+e9m/DxeE18cXXOyzO091ZcOODW3XKvzKFZTp7CrRm9ToVLqiWd6VdCZbj85ETAC2S1m2C8XW8j6hVqd2qQm44F5kqW/499tbWH0G4x/9pftW/W5wwIz3W978tBEqrXYLobzo7y13oho/jetwjU3PEpzYqnP0Nk93Oe+oBHJC+fPML5ScD1kO/jrHsheAUk6E38= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d53a6067-3b56-4d84-05d2-08d82e4ea8ae X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:51:04.3819 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 48b0tvIcRa83vTx3TbJRkdTtcEyKOVD3MRLfsKDoMmhF66aBDlEBx6H+mnlBqafi0pkqVjgTfOXYI5EkZwQydg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1279 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit adds task isolation hooks as follows: - __handle_domain_irq() and handle_domain_nmi() generate an isolation warning for the local task - irq_work_queue_on() generates an isolation warning for the remote task being interrupted for irq_work (through __smp_call_single_queue()) - generic_exec_single() generates a remote isolation warning for the remote cpu being IPI'd (through __smp_call_single_queue()) - smp_call_function_many() generates a remote isolation warning for the set of remote cpus being IPI'd (through smp_call_function_many_cond()) - on_each_cpu_cond_mask() generates a remote isolation warning for the set of remote cpus being IPI'd (through smp_call_function_many_cond()) - __ttwu_queue_wakelist() generates a remote isolation warning for the remote cpu being IPI'd (through __smp_call_single_queue()) - nmi_enter(), __context_tracking_exit(), __handle_domain_irq(), handle_domain_nmi() and scheduler_ipi() clear low-level flags and synchronize CPUs by calling task_isolation_kernel_enter() Calls to task_isolation_remote() or task_isolation_interrupt() can be placed in the platform-independent code like this when doing so results in fewer lines of code changes, as for example is true of the users of the arch_send_call_function_*() APIs. Or, they can be placed in the per-architecture code when there are many callers, as for example is true of the smp_send_reschedule() call. A further cleanup might be to create an intermediate layer, so that for example smp_send_reschedule() is a single generic function that just calls arch_smp_send_reschedule(), allowing generic code to be called every time smp_send_reschedule() is invoked. But for now, we just update either callers or callees as makes most sense. Calls to task_isolation_kernel_enter() are intended for early kernel entry code. They may be called in platform-independent or platform-specific code. It may be possible to clean up low-level entry code and somehow organize calls to task_isolation_kernel_enter() to avoid multiple per-architecture or driver-specific calls to it. RCU initialization may be a good reference point for those places in kernel (task_isolation_kernel_enter() should precede it), however right now it is not unified between architectures. Signed-off-by: Chris Metcalf [abelits@marvell.com: adapted for kernel 5.8, added low-level flags handling] Signed-off-by: Alex Belits --- include/linux/hardirq.h | 2 ++ include/linux/sched.h | 2 ++ kernel/context_tracking.c | 4 ++++ kernel/irq/irqdesc.c | 13 +++++++++++++ kernel/smp.c | 6 +++++- 5 files changed, 26 insertions(+), 1 deletion(-) diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index 03c9fece7d43..5aab1d0a580e 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -7,6 +7,7 @@ #include #include #include +#include #include extern void synchronize_irq(unsigned int irq); @@ -114,6 +115,7 @@ extern void rcu_nmi_exit(void); #define nmi_enter() \ do { \ arch_nmi_enter(); \ + task_isolation_kernel_enter(); \ printk_nmi_enter(); \ lockdep_off(); \ BUG_ON(in_nmi() == NMI_MASK); \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 7fb7bb3fddaa..cacfa415dc59 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -32,6 +32,7 @@ #include #include #include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -1743,6 +1744,7 @@ extern char *__get_task_comm(char *to, size_t len, struct task_struct *tsk); #ifdef CONFIG_SMP static __always_inline void scheduler_ipi(void) { + task_isolation_kernel_enter(); /* * Fold TIF_NEED_RESCHED into the preempt_count; anybody setting * TIF_NEED_RESCHED remotely (for the first time) will also send diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c index 36a98c48aedc..481a722ddbce 100644 --- a/kernel/context_tracking.c +++ b/kernel/context_tracking.c @@ -21,6 +21,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -148,6 +149,8 @@ void noinstr __context_tracking_exit(enum ctx_state state) if (!context_tracking_recursion_enter()) return; + task_isolation_kernel_enter(); + if (__this_cpu_read(context_tracking.state) == state) { if (__this_cpu_read(context_tracking.active)) { /* @@ -159,6 +162,7 @@ void noinstr __context_tracking_exit(enum ctx_state state) instrumentation_begin(); vtime_user_exit(current); trace_user_exit(0); + task_isolation_user_exit(); instrumentation_end(); } } diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c index 1a7723604399..b351aac7732f 100644 --- a/kernel/irq/irqdesc.c +++ b/kernel/irq/irqdesc.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "internals.h" @@ -669,6 +670,8 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq, unsigned int irq = hwirq; int ret = 0; + task_isolation_kernel_enter(); + irq_enter(); #ifdef CONFIG_IRQ_DOMAIN @@ -676,6 +679,10 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq, irq = irq_find_mapping(domain, hwirq); #endif + task_isolation_interrupt((irq == hwirq) ? + "irq %d (%s)" : "irq %d (%s hwirq %d)", + irq, domain ? domain->name : "", hwirq); + /* * Some hardware gives randomly wrong interrupts. Rather * than crashing, do something sensible. @@ -710,6 +717,8 @@ int handle_domain_nmi(struct irq_domain *domain, unsigned int hwirq, unsigned int irq; int ret = 0; + task_isolation_kernel_enter(); + /* * NMI context needs to be setup earlier in order to deal with tracing. */ @@ -717,6 +726,10 @@ int handle_domain_nmi(struct irq_domain *domain, unsigned int hwirq, irq = irq_find_mapping(domain, hwirq); + task_isolation_interrupt((irq == hwirq) ? + "NMI irq %d (%s)" : "NMI irq %d (%s hwirq %d)", + irq, domain ? domain->name : "", hwirq); + /* * ack_bad_irq is not NMI-safe, just report * an invalid interrupt. diff --git a/kernel/smp.c b/kernel/smp.c index aa17eedff5be..6a6849783948 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "smpboot.h" #include "sched/smp.h" @@ -146,8 +147,10 @@ void __smp_call_single_queue(int cpu, struct llist_node *node) * locking and barrier primitives. Generic code isn't really * equipped to do the right thing... */ - if (llist_add(node, &per_cpu(call_single_queue, cpu))) + if (llist_add(node, &per_cpu(call_single_queue, cpu))) { + task_isolation_remote(cpu, "IPI function"); send_call_function_single_ipi(cpu); + } } /* @@ -545,6 +548,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask, } /* Send a message to all CPUs in the map */ + task_isolation_remote_cpumask(cfd->cpumask_ipi, "IPI function"); arch_send_call_function_ipi_mask(cfd->cpumask_ipi); if (wait) { From patchwork Wed Jul 22 14:51:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333925 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=ODOAX3Vf; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=d9L+C/MS; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdjg2kv9z9sTR for ; Thu, 23 Jul 2020 00:52:39 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732752AbgGVOwg (ORCPT ); Wed, 22 Jul 2020 10:52:36 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:60146 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729642AbgGVOwg (ORCPT ); Wed, 22 Jul 2020 10:52:36 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEkbTw016104; Wed, 22 Jul 2020 07:51:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=RsbROyy9o5twnlyyPcyoUc615/SMC3uVIbiyIp12Dyc=; b=ODOAX3Vf1hr7wktTTss7qEjiVKy0WNTdLe80l5AgSOy3GetviRocF0VLf3bp30jEwfJ6 OQIX10zNQCpMf5iUkP0UiNHhE4hPv/39pqX4CJDYRuTsYAQly2N6BHYwVjIipp/zidcg E/5yEwf53gjPcV9Aqw5wnRzQoEz79oC9T8uITAk4nvj4gkHDN3l+nqLGfhp1TECwVyuM gfZ1fqRySuI0vI0+/d/Yy5+pRev480zX40+ul+UjPJaQ65FU+NCjyNfsgGL4FjsNvYr0 uY+Hrf4PHuvvY7/iPfXQbQIg2ZMsN/MurgNsB9afQKE0ziTSnA4jGr/3YyG8cmuN7IY4 ZA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens71g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:51:58 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:57 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:57 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.171) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:51:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aq9JmuIBHScospIPRBNsAdbx/ABv+xy2J7mOXttr8IFOnsEYJDk5cUnigXbwE6sHEgSZbpOjHijyXTrchxx9B0jKSPzncM/0msOpDs0+Aikn8d490Vxv1eqcXGjRgUbwgqUoKd/SW+zLQ2K2b3SBr+ZGQ74M/jyZJ6H4AGYSO1uYlxOGsctQB/jg4ApGReFKE6KX/uSvxWeAp+PP/Np+k28NVfEPH3HoWqhroyIJkyyiLxZ6TJgBSScQW1rhyNmET7zjmOxkXLio9BDRFw6shd6+OWy2wOQ7Odk9EV26xdkQs+33CSbbBChPMcEpwIYew80SyI0JVH1O5fK7viO1rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RsbROyy9o5twnlyyPcyoUc615/SMC3uVIbiyIp12Dyc=; b=OsA52B3ve9YJO1ybKQB/LELddTeCXQh6fzu50dDJPI5u3t1ajr7KCTLVtNFvvzJnogfu/eRti3FIwp5RsQS6scZa8D10aO7oELhpBtOPcmt3cjlG6IRhxw5f0PNxVxVyTISs/e4u1395T8eecg/ilLUuRPbYECn/hKIxCe0w6BH1lk2ZPYo0SCnc8TzS66RnRiZN7QPlyxNMC6LCaxZUOKEbS5ML6aeMZWSB45z7XrGBW08aN+PBwAFpyo/YDNbeW9RHK4HSjAyCyLKnLCsqSTL0hGzONKHkwetIQXrZmQ4/YBF0/n4Vdf0VHmuo4Ln+qE1CoST7BwKD8k0APvynvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RsbROyy9o5twnlyyPcyoUc615/SMC3uVIbiyIp12Dyc=; b=d9L+C/MSALNOF5jGgBwBWR3vJi3vXnGR/bQ13sjza16iIiOw3tCLGEoy7cE02TlRwD+uPwUwXNW9/XjX7GgwDHs8CYVKhqDSvIUx/cX6Dbxj9MLAw0poNwKwGk+K1fFgl6Wx6hZaccreImkeV9xdDhgxG52P3KKL+kvQK1CJnyM= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:51:56 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:51:55 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 05/13] task_isolation: Add xen-specific hook Thread-Topic: [PATCH v4 05/13] task_isolation: Add xen-specific hook Thread-Index: AQHWYDekSLC//sodZEC38w51R9lIxg== Date: Wed, 22 Jul 2020 14:51:55 +0000 Message-ID: References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7271793c-7189-4ede-83f1-08d82e4ec748 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:275; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: y0w6Wl33ySdDSmee1UhUmOqHM979ESxiJ+iWU3tE9IIgmXFKGeKV4pAn8vz/EuD5+QQnOLMALv8LGwRxSkBmLoOP58r3FWZ0zN3gQ8O64hXF2hkCw94uBTeah/Js5JuZJiq/YlfwdUdI8hON2sBzu6b7U70w2uDrNb5ggr30PL3dxlnEavTfJ/4w8yEzc6q2goF/omjnfyjVxi9jMg0fBOLfC+OJodEg9/81gFzAfvvvIJ/nQ7qUT3qEqGuEEYuTUUYMvwDZlTo8Mk4g7EEFu2Pdzq8cecfC57/9WITVtPhGblliLhX+BBFzwV8AMWthszq03UEiGo+/2uTwK9hW7A== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(4744005)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: CtpYorwSaZrU3TLkn73oTe3fDxZ45Ucq3YfaVU2Uc6tu9ckjle7Gg6dqOuo61R6gCmxCxRsK1DXfyRoCu4+napVFKDrbQBTPljsVlpOQzaU3f103Z+EcupvWWAcQG54+HW2WMgCPZKSBF35ktZgRsC2Zx9D1UZqT7QyR2+COcF8+yyQtNQrO8mub1rzISHWjvtzhF4iDRfrSmC8hsrJCLLmhTYObWtBa9MnLpQi+dAvWWCRgMAIwveAZ37CVPP0ORColSZRbhsQwI1DVzkDfZw3qB02dakQ0VxwNyjNVoJizEcvJcTePgxMHdqs7UcJpMjU6wBa/cg0OYFcw5fS+AzsSzU6HW4AsX+zqni8jAAebnz+o1RMFOka2C582P3jMSrDR9XUUPQKuOlcrHMhdRI28Z6A8sWUhMjb30lvzJwJL163is2NewxEWydhCrIzpqEbp1FmJ+2JAqEBwYnBhHpNoSFmM1hd4yyqjU00tIw8= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7271793c-7189-4ede-83f1-08d82e4ec748 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:51:55.7800 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8hnLOmDlqnd6vx3ULhpH1iGzMTkgonTFak3z5UTgLEF4lxASWkDtshf8ISoDfnzh+O9jWKG7tAcv6DRC/210zQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org xen_evtchn_do_upcall() should call task_isolation_kernel_enter() to indicate that isolation is broken and perform synchronization. Signed-off-by: Alex Belits --- drivers/xen/events/events_base.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 140c7bf33a98..4c16cd58f36b 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -33,6 +33,7 @@ #include #include #include +#include #ifdef CONFIG_X86 #include @@ -1236,6 +1237,8 @@ void xen_evtchn_do_upcall(struct pt_regs *regs) { struct pt_regs *old_regs = set_irq_regs(regs); + task_isolation_kernel_enter(); + irq_enter(); __xen_evtchn_do_upcall(); From patchwork Wed Jul 22 14:53:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333926 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=MjhfU9Nt; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=t12xrwfT; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdkt4qfDz9sSt for ; Thu, 23 Jul 2020 00:53:42 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732702AbgGVOxk (ORCPT ); Wed, 22 Jul 2020 10:53:40 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:33912 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731856AbgGVOxj (ORCPT ); Wed, 22 Jul 2020 10:53:39 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEnMNS012667; Wed, 22 Jul 2020 07:53:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=rlCXkoep0UYBt/xn1VMY0GqZr1LqNLBz/I0H/s5wz4E=; b=MjhfU9Nt2ufW+1bfPMrlXZJGKuncVotU/GzcLWmzZ7Uv4uM6yU+QmPeQcXcYKW5UGGyQ nugjHB+qlZ8ldo2nDj6ej+m2FiLdwhW3JGN1TscRDzTBrvMz/evpyP/0NvglaBgOSUhV HhNe3u3yUPdNnBE81Gl00gfltG4pZmwKhb7wi6rf49LsNTDPpoXtvDeLkR0xDLC9eILT Y6qaO667jdpuU5Paj2wiysbjFYxbOpjtzJwugfVHYHB9e0FndUgYWvdJbZxRsmyeKMXQ 77QiCSl862lNwop8qne9viVEc6j41dotFdDf4JkRU7X4DfziYvGusQfJPdkGuOSfrFO8 YQ== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrb3k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:53:03 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:53:01 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.175) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:53:01 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fk5un9GZrx5RJVEdctPysT0WIHSlBjlHLuVrMrX/ukc3oAq0UyBRXANFYlmTcK7ut+eBk4235J35gEcbV5jg+ElNH4HIZcnqeSbGVSCCYYjyWIA3ell7iBVpfd66fX7ZbmkKdHTtV4yk/YrmFCntw9txhLlAko84Ds2x1AfMaVS2OSvLWJqLr9F2kPlzubPs9ceouU8vPpR7xfkt/vl0jugTjfmvn2Nnf/BgW1OpJBHgBZpq4nn8wxnFLXrbVmV9PmAAYzv4VpdCptB2CBJzdzJhyxMsE0gmim4SQsrHeMCXcbG5DDQjbw4iQJdN7vcelBeYaCeQ2/4iSN1Vl7Rrpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlCXkoep0UYBt/xn1VMY0GqZr1LqNLBz/I0H/s5wz4E=; b=S/vjIW3Wy4zB00bS7ScmOrd92/94o8VnBYP0w4bxIGntKrDuPCfs+ZcgZVlH8XmzULBRsMc2mvT5OxmigSHuAA9AYqqZuK+zcXOFRKl0ZTfSCbafkBiFq1LGTHifkg6oI5CITJojOwKGZKwGxxZpWpcJwC6ac6fYcjB+o8q7U4a4GWkChjJObUvYzfT8XGNKh+X82fGVtRrR1PK9Pa3+w3cCBaBiKCFth+1UC7xNPHrlUYYCUN4djvd9+8ENF5Ku5Y55rAaOvgBpZVqA4jFQGp1/h0cH+iaXdjll98tvYzj8cgabwrMeYQWaC3fXtj4px88/i8O42ysfB+4bNlwzXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rlCXkoep0UYBt/xn1VMY0GqZr1LqNLBz/I0H/s5wz4E=; b=t12xrwfTxsDqP45/4k5EgaU0k3ATC3mqRoEVR5qNqn/AxqdwS5ZmcCaU94jiqKxzEiImfp6b+UrWrZA2mB59DGAK/C2/hUiP67EC45Bc6SaDkqo3JYKxO63Vng6a061x233Sf+fjHRZZ9AK0KSX/uoVIbjE3/TwX/Z4hiHRyG2M= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:53:00 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:53:00 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH 06/13] task_isolation: Add driver-specific hooks Thread-Topic: [PATCH 06/13] task_isolation: Add driver-specific hooks Thread-Index: AQHWYDfLEWm/FvU9QkyARg4LLZWHYw== Date: Wed, 22 Jul 2020 14:53:00 +0000 Message-ID: References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 941caea6-85ae-48c4-c967-08d82e4eedac x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1265; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: e+/lkENi//xP0xCnYw/iMnLdYw/4LHjp/pHNBdvRUrZIfi4Eu8cPrGkcN0MZd3c6ho0KeZHeHveR7V9Z6/WMydBP/teOVeS4Swh+x4XgqNiu+MuSH3ksEfKJ3M7vIIPtZkvjhTQtVRjIOj5PSpYGeEar+awDwVXOc85ytuwb4tPIF6/0wv7vMpf+CdB2N+8tTb3yQGtparWOe0cZeQrSB0Utomq3+Rrg7cq+MrxkXPLI/yENZGSmJDVxUOw/ILDV2zQuAH5X+WH0XL1WcCBCnhYkOVlyWxPMqMnFTIG2yHLGA/psvVK5e3N9PK4SgdcG x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: ierTVGVynEljc2zjIfdxUxy1AnLyVWKAspNPOj6U+tUY+NtiNXpGkBQNen/J02bZPvjDT/7PkL6B3mFmN85pXPNGg/G3LH88VA/ecEksHEOS6t7VCto0M+Bq5Wt5Ke5eiUquQWy/X4dm/LeTGrqO2XNqfT09a9CcDg7nsEsNtIXiusd3XzlwfmEwtriT+LPBoSuVBybD+puZS/M31en1FXlr1uerCXmPN0e6TEE1v6vfcAknMpdIP2ZRNEsgdMaHqiHS/6XKMcWW3KqsO8iKCvZ9ydOT578yiTQcEwGR9g95NEV2qurdgO9Wa5WMIWIAvGjRckqVu4qAbGuMOAMbhUiFbFzz9flHA9znmYe4VtdbOVVvqGmwIpQrDs/TehbksfeJ57wKqsHjxJ7V8+0MilrK6/woLQKPL27MDwbUKxNMS5RtkzqcH7fFxJo3rUZO+r0kH6DKLYwK/ocy1cw/7Ea56d7ev8wNbmGV/vHdgkI= Content-ID: <29DE6F6283742246B93D47862FFA4D3C@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 941caea6-85ae-48c4-c967-08d82e4eedac X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:53:00.1246 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: i6spHPWJA0C7NEFTOGUFpdp3ix0iQwF+RWr2Ma5BSII6XObz4fXOkBST2aIuddXSgebnWAvBPHsoTJ8ZBT2BYQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Some drivers don't call functions that call task_isolation_kernel_enter() in interrupt handlers. Call it directly. Signed-off-by: Alex Belits --- drivers/irqchip/irq-armada-370-xp.c | 6 ++++++ drivers/irqchip/irq-gic-v3.c | 3 +++ drivers/irqchip/irq-gic.c | 3 +++ drivers/s390/cio/cio.c | 3 +++ 4 files changed, 15 insertions(+) diff --git a/drivers/irqchip/irq-armada-370-xp.c b/drivers/irqchip/irq-armada-370-xp.c index c9bdc5221b82..df7f2cce3a54 100644 --- a/drivers/irqchip/irq-armada-370-xp.c +++ b/drivers/irqchip/irq-armada-370-xp.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -473,6 +474,7 @@ static const struct irq_domain_ops armada_370_xp_mpic_irq_ops = { static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained) { u32 msimask, msinr; + int isol_entered = 0; msimask = readl_relaxed(per_cpu_int_base + ARMADA_370_XP_IN_DRBEL_CAUSE_OFFS) @@ -489,6 +491,10 @@ static void armada_370_xp_handle_msi_irq(struct pt_regs *regs, bool is_chained) continue; if (is_chained) { + if (!isol_entered) { + task_isolation_kernel_enter(); + isol_entered = 1; + } irq = irq_find_mapping(armada_370_xp_msi_inner_domain, msinr - PCI_MSI_DOORBELL_START); generic_handle_irq(irq); diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index cc46bc2d634b..be0e0ffa0fb7 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -629,6 +630,8 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs { u32 irqnr; + task_isolation_kernel_enter(); + irqnr = gic_read_iar(); if (gic_supports_nmi() && diff --git a/drivers/irqchip/irq-gic.c b/drivers/irqchip/irq-gic.c index c17fabd6741e..fde547a31566 100644 --- a/drivers/irqchip/irq-gic.c +++ b/drivers/irqchip/irq-gic.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include #include @@ -353,6 +354,8 @@ static void __exception_irq_entry gic_handle_irq(struct pt_regs *regs) struct gic_chip_data *gic = &gic_data[0]; void __iomem *cpu_base = gic_data_cpu_base(gic); + task_isolation_kernel_enter(); + do { irqstat = readl_relaxed(cpu_base + GIC_CPU_INTACK); irqnr = irqstat & GICC_IAR_INT_ID_MASK; diff --git a/drivers/s390/cio/cio.c b/drivers/s390/cio/cio.c index 6d716db2a46a..beab88881b6d 100644 --- a/drivers/s390/cio/cio.c +++ b/drivers/s390/cio/cio.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -584,6 +585,8 @@ void cio_tsch(struct subchannel *sch) struct irb *irb; int irq_context; + task_isolation_kernel_enter(); + irb = this_cpu_ptr(&cio_irb); /* Store interrupt response block to lowcore. */ if (tsch(sch->schid, irb) != 0) From patchwork Wed Jul 22 14:54:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333927 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=SM0tCzIA; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=t9Eu7fGz; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdmR5lvVz9sSn for ; Thu, 23 Jul 2020 00:55:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732644AbgGVOzA (ORCPT ); Wed, 22 Jul 2020 10:55:00 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:38176 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730465AbgGVOy6 (ORCPT ); Wed, 22 Jul 2020 10:54:58 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmkQA012155; Wed, 22 Jul 2020 07:54:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=8IhE7wrUXd1paiBAH3lAOks9OwKS1vDzrpp5nlBKiiU=; b=SM0tCzIAXUKStTBuQJX2EoUmDIrfY0ci/3vGg3bjvGYygaE9zHmzgd7XQPmMGoHkerKY iqikY2VIKzmnUglLY79DxxVlX7xzlSh8GnMzGb/T2kgHtL81TrBlFdX3jEyGlGYY0sXO 1ZG8vh+lbjY16hJcuDqjGQUZiWr3P00dcg1artr8wcaJCEr7H4TfqtDPKzNOu40OUdzY T/+lktyCR0QJb4iv8sN/GI53VBYyK3eKsBcLbiJph05OdUzg7bHBnVSAcVw6UeLMUcbr hGxnV4Hdy64SEo8vQ5FnHWkPzTuAB44p7BSFyTKR6j/dm6Xov+G/GFGzpYIOsxqpc0e3 lg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrb9s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:54:18 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:54:17 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.173) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:54:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Pc2TDJOLIdwJfUExt4fDL6sssjvvJCca0I9PACnI0z/5+w2b8mgr0kCuUDa/cDDwkT/8pyVqCUMF7AHG7RmLXBL1oLDlEAVVItHPMRV61YTP9fZ8B8k+PdSCNY6vfyTBq6C/xdxZ6RxUwYUhpVuXGxNyn5q2kYBJpfIvVGBPCdLH7C3qAUnGb0pavIn5qDoSuJMt9Xr684RC6gAOJUOGIGt8UIiln1mMHFmpslpyy5KFeL0QiEOceonOEwt+YPHq8sDMDnhZXLs8r/yrCwT7FHoiufcuMDIWfkfbhzetAg7/sH6S71kt05MbYwXqUZ3MCQhJyRjwPHV1QEvLkiHHfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8IhE7wrUXd1paiBAH3lAOks9OwKS1vDzrpp5nlBKiiU=; b=DCuy1kiXNi+qXU9nTYScltxGQyoHnNLRwlsscuS4Vm7rf5TrJAz0awQjzSQIaxkSFXC+5mI2fpyMaoskuHBePvpQk2wGL20NmBhVebgi2meVtCAR67S+D1g0rwUMIQelgNni9rKhUchhNlyjVkA8kNfdCs4xp8ZM1Ownlo0BKbpMoyzdyxYX0FlMObTLmxA9mj7TnqI3TtcembbKivD0nnDgRf0kf60XBFDLnxp9omG0CNRIdoAjTjHh8hrevytnLwVcLgESG9U+VwFEW3haEDgLcyF8x0IfYFaIvdp+qWq+m9khpo8a80WyV61suphWVr1AXSCj+J23ytYX6oHUOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8IhE7wrUXd1paiBAH3lAOks9OwKS1vDzrpp5nlBKiiU=; b=t9Eu7fGzlaLQXOSFH1bFLbd6vbL9D1IDY/vJsjUOEpmVZYBq7dElkgVkY/NjmVgIEHywYk5w3gPlbUJSuQzR5G+3SsteqsCdmCSJCBx5tbvnrBvCmQth/5x1VS5oO0fnk+6wTcFOLjX5mlS5T8xJxfH38EU4A62LryEyDvYT+ys= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:54:14 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:54:14 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 07/13] task_isolation: arch/x86: enable task isolation functionality Thread-Topic: [PATCH v4 07/13] task_isolation: arch/x86: enable task isolation functionality Thread-Index: AQHWYDf3tP4J4jNDakOQrklH/k1RhA== Date: Wed, 22 Jul 2020 14:54:14 +0000 Message-ID: References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 52e921dc-811f-40d1-b172-08d82e4f19e0 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: rLGl2UWWLi/PJHknjjC2WOPkoDG74JP/i2pKohav122Ap0W829KD3j8Hig32xfOwjg8PPwWNTdrwOtNJgGCO5cFNkttYkbpZKxeqctE8ZhFK41cPAAFGMjx4PFDilSKIJnMIjUliBzawzy0JoszJ48C1/bAIRbTCdTaFPZDpydhBfHkSg2q4ilXgcF5A35++S4ls1P1Vrw1jKyOXPMdD5DrNsYcFpT1OqaO0dFcho6xA7iCnzdweSb9VTtf0i3bWvHaPX+6kpoiunyTaA7MIjBAtJ7vnEXowUGmQ1nwA/l8QhQJRquQFdzjL9cLKcr15X9rhM8RfviGvt1BE/LvBIg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: C2sOQBfDx7XWfnxVE2h6KzHVOfj6DFLtBXqW5fVp9/26Xi07rYLmWDwyDLs6KgRWUV+xkJhE5EnXd6hTKTSUkiLVmHuisHDjsJgwgPDWMUz5X1YYQxfoOJRG9PsHUKy6LMzn2mmt8IAWB8yb4gw7KW9N9dcDfXaNgSgWPAWy8HLRs5POfTagxw0QEDZO0Q/f4uBhh3FDaCmF61SEkKsCSvDzPLIcLMjNPtVrV5ZOS4RdQWbtAnvkQPG4Bzn6euo+2klzulsSUe9BfXrCKp7AkySCf9dY0Tb1YlHshvtHOR4jiB4clUg+epP5bb7/Mqr4RPy4UaO4hc1hEaaXx82a7MDX3miBDc/VBsNgeWa9tM5Y9lWo6anRIfMPcRRUdIrvOe34aJjIUpUO1xlBo0lFd4o+77hw7Q+ovgxiUXOmzUlJymItAxyPtKquI1GdV+TnwjKfKC6f/xpvbcE+dqpmRS4/pUb5y3WTW8586tVrK8Q= Content-ID: <656247F523334C4E90B4C97A29E836AC@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 52e921dc-811f-40d1-b172-08d82e4f19e0 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:54:14.2646 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: FxK/obFuUAJlpJXhlQEohiiesAb17l9tQZ0WS4SFoGhNJqM4Euxv2atMOatPKgE12gQKNm91ABJqeSoM6I+smA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org In prepare_exit_to_usermode(), run cleanup for tasks exited fromi isolation and call task_isolation_start() for tasks that entered TIF_TASK_ISOLATION. In syscall_trace_enter(), add the necessary support for reporting syscalls for task-isolation processes. Add task_isolation_remote() calls for the kernel exception types that do not result in signals, namely non-signalling page faults. Add task_isolation_kernel_enter() calls to interrupt and syscall entry handlers. This mechanism relies on calls to functions that call task_isolation_kernel_enter() early after entry into kernel. Those functions are: enter_from_user_mode() called from do_syscall_64(), do_int80_syscall_32(), do_fast_syscall_32(), idtentry_enter_user(), idtentry_enter_cond_rcu() idtentry_enter_cond_rcu() called from non-raw IDT macros and other entry points idtentry_enter_user() nmi_enter() xen_call_function_interrupt() xen_call_function_single_interrupt() xen_irq_work_interrupt() Signed-off-by: Chris Metcalf [abelits@marvell.com: adapted for kernel 5.8] Signed-off-by: Alex Belits --- arch/x86/Kconfig | 1 + arch/x86/entry/common.c | 20 +++++++++++++++++++- arch/x86/include/asm/barrier.h | 2 ++ arch/x86/include/asm/thread_info.h | 4 +++- arch/x86/kernel/apic/ipi.c | 2 ++ arch/x86/mm/fault.c | 4 ++++ arch/x86/xen/smp.c | 3 +++ arch/x86/xen/smp_pv.c | 2 ++ 8 files changed, 36 insertions(+), 2 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 883da0abf779..3a80142f85c8 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -149,6 +149,7 @@ config X86 select HAVE_ARCH_COMPAT_MMAP_BASES if MMU && COMPAT select HAVE_ARCH_PREL32_RELOCATIONS select HAVE_ARCH_SECCOMP_FILTER + select HAVE_ARCH_TASK_ISOLATION select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_STACKLEAK select HAVE_ARCH_TRACEHOOK diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index f09288431f28..ab94d90a2bd5 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -26,6 +26,7 @@ #include #include #include +#include #ifdef CONFIG_XEN_PV #include @@ -86,6 +87,7 @@ static noinstr void enter_from_user_mode(void) { enum ctx_state state = ct_state(); + task_isolation_kernel_enter(); lockdep_hardirqs_off(CALLER_ADDR0); user_exit_irqoff(); @@ -97,6 +99,7 @@ static noinstr void enter_from_user_mode(void) #else static __always_inline void enter_from_user_mode(void) { + task_isolation_kernel_enter(); lockdep_hardirqs_off(CALLER_ADDR0); instrumentation_begin(); trace_hardirqs_off_finish(); @@ -161,6 +164,15 @@ static long syscall_trace_enter(struct pt_regs *regs) return -1L; } + /* + * In task isolation mode, we may prevent the syscall from + * running, and if so we also deliver a signal to the process. + */ + if (work & _TIF_TASK_ISOLATION) { + if (task_isolation_syscall(regs->orig_ax) == -1) + return -1L; + work &= ~_TIF_TASK_ISOLATION; + } #ifdef CONFIG_SECCOMP /* * Do seccomp after ptrace, to catch any tracer changes. @@ -263,6 +275,8 @@ static void __prepare_exit_to_usermode(struct pt_regs *regs) lockdep_assert_irqs_disabled(); lockdep_sys_exit(); + task_isolation_check_run_cleanup(); + cached_flags = READ_ONCE(ti->flags); if (unlikely(cached_flags & EXIT_TO_USERMODE_LOOP_FLAGS)) @@ -278,6 +292,9 @@ static void __prepare_exit_to_usermode(struct pt_regs *regs) if (unlikely(cached_flags & _TIF_NEED_FPU_LOAD)) switch_fpu_return(); + if (cached_flags & _TIF_TASK_ISOLATION) + task_isolation_start(); + #ifdef CONFIG_COMPAT /* * Compat syscalls set TS_COMPAT. Make sure we clear it before @@ -597,7 +614,8 @@ bool noinstr idtentry_enter_cond_rcu(struct pt_regs *regs) check_user_regs(regs); enter_from_user_mode(); return false; - } + } else + task_isolation_kernel_enter(); /* * If this entry hit the idle task invoke rcu_irq_enter() whether diff --git a/arch/x86/include/asm/barrier.h b/arch/x86/include/asm/barrier.h index 7f828fe49797..5be6ca0519fc 100644 --- a/arch/x86/include/asm/barrier.h +++ b/arch/x86/include/asm/barrier.h @@ -4,6 +4,7 @@ #include #include +#include /* * Force strict CPU ordering. @@ -53,6 +54,7 @@ static inline unsigned long array_index_mask_nospec(unsigned long index, #define dma_rmb() barrier() #define dma_wmb() barrier() +#define instr_sync() sync_core() #ifdef CONFIG_X86_32 #define __smp_mb() asm volatile("lock; addl $0,-4(%%esp)" ::: "memory", "cc") diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h index 8de8ceccb8bc..6dd1a5cc286d 100644 --- a/arch/x86/include/asm/thread_info.h +++ b/arch/x86/include/asm/thread_info.h @@ -93,6 +93,7 @@ struct thread_info { #define TIF_NOTSC 16 /* TSC is not accessible in userland */ #define TIF_IA32 17 /* IA32 compatibility process */ #define TIF_SLD 18 /* Restore split lock detection on context switch */ +#define TIF_TASK_ISOLATION 19 /* task isolation enabled for task */ #define TIF_MEMDIE 20 /* is terminating due to OOM killer */ #define TIF_POLLING_NRFLAG 21 /* idle is polling for TIF_NEED_RESCHED */ #define TIF_IO_BITMAP 22 /* uses I/O bitmap */ @@ -123,6 +124,7 @@ struct thread_info { #define _TIF_NOTSC (1 << TIF_NOTSC) #define _TIF_IA32 (1 << TIF_IA32) #define _TIF_SLD (1 << TIF_SLD) +#define _TIF_TASK_ISOLATION (1 << TIF_TASK_ISOLATION) #define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG) #define _TIF_IO_BITMAP (1 << TIF_IO_BITMAP) #define _TIF_FORCED_TF (1 << TIF_FORCED_TF) @@ -136,7 +138,7 @@ struct thread_info { /* Work to do before invoking the actual syscall. */ #define _TIF_WORK_SYSCALL_ENTRY \ (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_EMU | _TIF_SYSCALL_AUDIT | \ - _TIF_SECCOMP | _TIF_SYSCALL_TRACEPOINT) + _TIF_SECCOMP | _TIF_SYSCALL_TRACEPOINT | _TIF_TASK_ISOLATION) /* flags to check in __switch_to() */ #define _TIF_WORK_CTXSW_BASE \ diff --git a/arch/x86/kernel/apic/ipi.c b/arch/x86/kernel/apic/ipi.c index 6ca0f91372fd..b4dfaad6a440 100644 --- a/arch/x86/kernel/apic/ipi.c +++ b/arch/x86/kernel/apic/ipi.c @@ -2,6 +2,7 @@ #include #include +#include #include "local.h" @@ -67,6 +68,7 @@ void native_smp_send_reschedule(int cpu) WARN(1, "sched: Unexpected reschedule of offline CPU#%d!\n", cpu); return; } + task_isolation_remote(cpu, "reschedule IPI"); apic->send_IPI(cpu, RESCHEDULE_VECTOR); } diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 1ead568c0101..e16a4f5c7e57 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -18,6 +18,7 @@ #include /* faulthandler_disabled() */ #include /* efi_recover_from_page_fault()*/ #include +#include /* task_isolation_interrupt */ #include /* boot_cpu_has, ... */ #include /* dotraplinkage, ... */ @@ -1332,6 +1333,9 @@ void do_user_addr_fault(struct pt_regs *regs, perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1, regs, address); } + /* No signal was generated, but notify task-isolation tasks. */ + task_isolation_interrupt("page fault at %#lx", address); + check_v8086_mode(regs, address, tsk); } NOKPROBE_SYMBOL(do_user_addr_fault); diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c index 2097fa0ebdb5..9a3a9bae7d06 100644 --- a/arch/x86/xen/smp.c +++ b/arch/x86/xen/smp.c @@ -4,6 +4,7 @@ #include #include #include +#include #include @@ -265,6 +266,7 @@ void xen_send_IPI_allbutself(int vector) static irqreturn_t xen_call_function_interrupt(int irq, void *dev_id) { + task_isolation_kernel_enter(); irq_enter(); generic_smp_call_function_interrupt(); inc_irq_stat(irq_call_count); @@ -275,6 +277,7 @@ static irqreturn_t xen_call_function_interrupt(int irq, void *dev_id) static irqreturn_t xen_call_function_single_interrupt(int irq, void *dev_id) { + task_isolation_kernel_enter(); irq_enter(); generic_smp_call_function_single_interrupt(); inc_irq_stat(irq_call_count); diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c index 171aff1b11f2..d71d3cc36c51 100644 --- a/arch/x86/xen/smp_pv.c +++ b/arch/x86/xen/smp_pv.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -482,6 +483,7 @@ static void xen_pv_stop_other_cpus(int wait) static irqreturn_t xen_irq_work_interrupt(int irq, void *dev_id) { + task_isolation_kernel_enter(); irq_enter(); irq_work_run(); inc_irq_stat(apic_irq_work_irqs); From patchwork Wed Jul 22 14:55:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333928 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=kcQYSoIJ; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=U3S4gi3Q; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdnj3WRgz9sSn for ; Thu, 23 Jul 2020 00:56:09 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732751AbgGVO4G (ORCPT ); Wed, 22 Jul 2020 10:56:06 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:63228 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730465AbgGVO4F (ORCPT ); Wed, 22 Jul 2020 10:56:05 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEkh9M016190; Wed, 22 Jul 2020 07:55:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=kcQYSoIJb57JAMxTNY/P9ypkRiXs4kNkt2VwJ+EaflWVa/rXQSPYCptlwzM6kdRLsRDo 2xPb3VeceVPI4UKQB1FBGl3J+T33RIQF3sPui4qjLaNIqCNcK+DCqUyUWO2hz9LL4idB z8HjAOKXVD6KI8CVrR6L/8itYtRRgSJ/p1DwdRNnuTdJ5rTBe8+9OQ8+WsLxVOoBNPdd PKpgcReBRvnj85JW08qRbU55oNRbmlBg7SY3LLQmsH5ErvTfosBAJR6nbrxNDYgcUHp/ 7zJGsJ6FGgkOcrkombUSB4aa5F8IW0+T2VmXGNayacVI8fFEltju60oWv6bjhd4DXSXm YA== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 32bxens7ep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:55:36 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:55:34 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.170) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:55:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GPEi+5wZxvPKX25j49TConCqDUCaC8AYFsYdPbO4MdT0+yuaD3L8iikXSqbEsPnUealBGwqNFwh0kmS47S8P09psB7bb+Bg/m0uW/dPmdinKZMWPxmhdQEpVtdDSDql+3nWYP5Nt//4ck1GigUrWJXsTRHC4K7cJsF3/SymZxdvlx/c0WRZEKGRdlSDx9BOLFxnBg/m+1JXZ98XD8Vyv9FcrBoZcWiOAsg2oTCZeXcO9oLHmP/AceWsZ3qBsVFaPZl5S/7plVdvvst1d3b7EnSHkpWRQnXdiOr41OuqR+OcK86yXj4fo5+weMS8c9izbP0MxIvew8oOVlWis223N7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=dN4rNuSI3yCLMGsEYONkzcCxZBYgyvYqVat2m6uzrihgJ/t7l4ydm2rfsMMiFWz6AVIJX2Bvh3x2w7kG5PepIdZNCdIBxsjizdpZp9ZdOfq8VOlaYAcmQKyBF3AFdBV7z0YJAqEK3fPLJvFlceTEZIpJOxBVEt+4A3GPQ73Xz/e0cinePuaYrgQbEF2E/5GjZbUnVOIoJWW4OKRlbQ2uFie24QJnTIkt90Y+F4VNwbBulH6/AmnhrO6XcNxBVNR5QlXJo5aYxTiCFHaJmfY+oHKK9GIRNlRp+Fy7OQ2ul8z+NbGNxnHBqdKa7B7Fc0ymHuinwrxwbcgN9mjxfZRxww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5y0pTUZ9XwKzAgNXp/SHP1CqeCLcMcW6A5i+L4kfUdk=; b=U3S4gi3QCcDFAsrGxxo9tnabt8yR25Fs8svLCq7+WSl/1Ee1fubHzndx3bjtXvG4F26CJjkatgHyVuGmhkckbGFfto3Dv3TSiEC+8k3C9ahNisgizvJVj70vQTJEAy6XVVAFHYc9541b2PY4CZrOXuHhufOqEVl/E/xHK+W/BtI= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:55:33 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:55:33 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH 08/13] task_isolation: arch/arm64: enable task isolation functionality Thread-Topic: [PATCH 08/13] task_isolation: arch/arm64: enable task isolation functionality Thread-Index: AQHWYDgmZidEulvFKU2n/vC9QyCd/A== Date: Wed, 22 Jul 2020 14:55:33 +0000 Message-ID: <470f6068e87a3627111a2615032d7b6bce824451.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dedab54d-a283-42ef-dedb-08d82e4f48f4 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jPHe8/6YSKjjvYJquAT+zK5FYFGjRzAy5Lpics/qowaqhUZoBIehORI57kTQ4qMj5dHSZoMi3kn+RJPxYjYGNI7BKPHD1a65KIptQ+wvvadfPhY+bef3I4mGu6e5IW6jx4F9yFCxZEV7DI2YlL483O5OhKxjvjGKNkURINQIe6p+z6+ylnHDIbNAyIcv105166zZpHJv2wIr35pTM2tXhJD/hFz5sDM9XETxSXmQ/akYbWdbSaBvWelbEcQXxiUZoX7mK2l0HISIrQj56nK6O6mmFpKhO34WT/ZNnUX9eAXWTW+LHJ1PWKk4R19d5xGk x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(30864003)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: GBvucFlf24zgdqJpvXhrb/IFZQZMFRwP8f1DnCwMJLvMQQTlAXLGiTL41HCjHi70K8UQor3oJmzzvUn8v5r4uCiKNainZ/QZp8IyOSvS9M7De+k68+iDUXdGNNN0fJ1uipIyKKwZhqdRkg7J/OLdWaou0CW0lueyCnI1lQ7v0m6DlY/TwvQxNBKtTaT8DwLyqLykJKFpZ9DhI+ZykvrQGeVZv72fs40rNN5qVBq9uKOhgNUysGVK1fn1Emk2jU1gljnpoT99/afI+QkGR6pcCVKy9GKrnoGKMVJ3Ebahxft9US011ebBXIL0gZ6gGn39NQ/e9yatR47yCxFMkg8Pb35kVg0kmaw9VhEjeQWaIlvRUhGqGtFVlLohj0XGlNHBz1hGK81iWsDX9nLEpOM3MtugD1lj3X8kknXGD5qO6f5NZAvlFJkYQwPG/wdeWopSFp+d+7LG4f6X5E7j0rHo7S9+iC3GG7+tvLJurMaZb6c= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: dedab54d-a283-42ef-dedb-08d82e4f48f4 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:55:33.2817 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8gS1iM0mhzRBNI15G5gQNTbGDelOksfqrVK8At2d6+ckK2KJNY96jLIFyBNa0HB5Ov/gaIP4FRrcJiFuL/nbSA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_09:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Chris Metcalf In do_notify_resume(), call task_isolation_start() for TIF_TASK_ISOLATION tasks. Add _TIF_TASK_ISOLATION to _TIF_WORK_MASK, and define a local NOTIFY_RESUME_LOOP_FLAGS to check in the loop, since we don't clear _TIF_TASK_ISOLATION in the loop. We instrument the smp_send_reschedule() routine so that it checks for isolated tasks and generates a suitable warning if needed. Finally, report on page faults in task-isolation processes in do_page_faults(). Early kernel entry code calls task_isolation_kernel_enter(). In particular: Vectors: el1_sync -> el1_sync_handler() -> task_isolation_kernel_enter() el1_irq -> asm_nmi_enter(), handle_arch_irq() el1_error -> do_serror() el0_sync -> el0_sync_handler() el0_irq -> handle_arch_irq() el0_error -> do_serror() el0_sync_compat -> el0_sync_compat_handler() el0_irq_compat -> handle_arch_irq() el0_error_compat -> do_serror() SDEI entry: __sdei_asm_handler -> __sdei_handler() -> nmi_enter() Functions called from there: asm_nmi_enter() -> nmi_enter() -> task_isolation_kernel_enter() asm_nmi_exit() -> nmi_exit() -> task_isolation_kernel_return() Handlers: do_serror() -> nmi_enter() -> task_isolation_kernel_enter() or task_isolation_kernel_enter() el1_sync_handler() -> task_isolation_kernel_enter() el0_sync_handler() -> task_isolation_kernel_enter() el0_sync_compat_handler() -> task_isolation_kernel_enter() handle_arch_irq() is irqchip-specific, most call handle_domain_irq() or handle_IPI() There is a separate patch for irqchips that do not follow this rule. handle_domain_irq() -> task_isolation_kernel_enter() handle_IPI() -> task_isolation_kernel_enter() nmi_enter() -> task_isolation_kernel_enter() Signed-off-by: Chris Metcalf [abelits@marvell.com: simplified to match kernel 5.6] Signed-off-by: Alex Belits --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/barrier.h | 2 ++ arch/arm64/include/asm/thread_info.h | 5 ++++- arch/arm64/kernel/entry-common.c | 7 +++++++ arch/arm64/kernel/ptrace.c | 16 +++++++++++++++- arch/arm64/kernel/sdei.c | 2 ++ arch/arm64/kernel/signal.c | 13 ++++++++++++- arch/arm64/kernel/smp.c | 9 +++++++++ arch/arm64/mm/fault.c | 5 +++++ 9 files changed, 57 insertions(+), 3 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 66dc41fd49f2..96fefabfa10f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -137,6 +137,7 @@ config ARM64 select HAVE_ARCH_PREL32_RELOCATIONS select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK + select HAVE_ARCH_TASK_ISOLATION select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_TRACEHOOK select HAVE_ARCH_TRANSPARENT_HUGEPAGE diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index fb4c27506ef4..bf4a2adabd5b 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -48,6 +48,8 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +#define instr_sync() isb() + /* * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz * and 0 otherwise. diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index 5e784e16ee89..73269bb8a57d 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -67,6 +67,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define TIF_FOREIGN_FPSTATE 3 /* CPU's FP state is not current's */ #define TIF_UPROBE 4 /* uprobe breakpoint or singlestep */ #define TIF_FSCHECK 5 /* Check FS is USER_DS on return */ +#define TIF_TASK_ISOLATION 6 /* task isolation enabled for task */ #define TIF_SYSCALL_TRACE 8 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 9 /* syscall auditing */ #define TIF_SYSCALL_TRACEPOINT 10 /* syscall tracepoint for ftrace */ @@ -86,6 +87,7 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_FOREIGN_FPSTATE (1 << TIF_FOREIGN_FPSTATE) +#define _TIF_TASK_ISOLATION (1 << TIF_TASK_ISOLATION) #define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) #define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) #define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) @@ -99,7 +101,8 @@ void arch_release_task_struct(struct task_struct *tsk); #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ - _TIF_UPROBE | _TIF_FSCHECK) + _TIF_UPROBE | _TIF_FSCHECK | \ + _TIF_TASK_ISOLATION) #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index d3be9dbf5490..8b682aa020ae 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -70,6 +71,8 @@ asmlinkage void notrace el1_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_DABT_CUR: case ESR_ELx_EC_IABT_CUR: @@ -231,6 +234,8 @@ asmlinkage void notrace el0_sync_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC64: el0_svc(regs); @@ -300,6 +305,8 @@ asmlinkage void notrace el0_sync_compat_handler(struct pt_regs *regs) { unsigned long esr = read_sysreg(esr_el1); + task_isolation_kernel_enter(); + switch (ESR_ELx_EC(esr)) { case ESR_ELx_EC_SVC32: el0_svc_compat(regs); diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 1e02e98e68dd..5acfc194bdd0 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -1851,7 +1852,11 @@ static void tracehook_report_syscall(struct pt_regs *regs, int syscall_trace_enter(struct pt_regs *regs) { - unsigned long flags = READ_ONCE(current_thread_info()->flags); + unsigned long flags; + + task_isolation_kernel_enter(); + + flags = READ_ONCE(current_thread_info()->flags); if (flags & (_TIF_SYSCALL_EMU | _TIF_SYSCALL_TRACE)) { tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER); @@ -1859,6 +1864,15 @@ int syscall_trace_enter(struct pt_regs *regs) return NO_SYSCALL; } + /* + * In task isolation mode, we may prevent the syscall from + * running, and if so we also deliver a signal to the process. + */ + if (test_thread_flag(TIF_TASK_ISOLATION)) { + if (task_isolation_syscall(regs->syscallno) == -1) + return NO_SYSCALL; + } + /* Do the secure computing after ptrace; failures should be fast. */ if (secure_computing() == -1) return NO_SYSCALL; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index dab88260b137..f65b676132f9 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -185,6 +186,7 @@ static __kprobes unsigned long _sdei_handler(struct pt_regs *regs, __uaccess_enable_hw_pan(); err = sdei_event_handler(regs, arg); + task_isolation_interrupt("SDEI handled"); if (err) return SDEI_EV_FAILED; diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 3b4f31f35e45..ece90c5756be 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include @@ -907,6 +908,11 @@ static void do_signal(struct pt_regs *regs) restore_saved_sigmask(); } +#define NOTIFY_RESUME_LOOP_FLAGS \ + (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ + _TIF_NOTIFY_RESUME | _TIF_FOREIGN_FPSTATE | \ + _TIF_UPROBE | _TIF_FSCHECK) + asmlinkage void do_notify_resume(struct pt_regs *regs, unsigned long thread_flags) { @@ -917,6 +923,8 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, */ trace_hardirqs_off(); + task_isolation_check_run_cleanup(); + do { /* Check valid user FS if needed */ addr_limit_user_check(); @@ -947,7 +955,10 @@ asmlinkage void do_notify_resume(struct pt_regs *regs, local_daif_mask(); thread_flags = READ_ONCE(current_thread_info()->flags); - } while (thread_flags & _TIF_WORK_MASK); + } while (thread_flags & NOTIFY_RESUME_LOOP_FLAGS); + + if (thread_flags & _TIF_TASK_ISOLATION) + task_isolation_start(); } unsigned long __ro_after_init signal_minsigstksz; diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index e43a8ff19f0f..c893c8babe76 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -835,6 +836,7 @@ void arch_send_call_function_single_ipi(int cpu) #ifdef CONFIG_ARM64_ACPI_PARKING_PROTOCOL void arch_send_wakeup_ipi_mask(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "wakeup IPI"); smp_cross_call(mask, IPI_WAKEUP); } #endif @@ -896,11 +898,16 @@ void handle_IPI(int ipinr, struct pt_regs *regs) unsigned int cpu = smp_processor_id(); struct pt_regs *old_regs = set_irq_regs(regs); + task_isolation_kernel_enter(); + if ((unsigned)ipinr < NR_IPI) { trace_ipi_entry_rcuidle(ipi_types[ipinr]); __inc_irq_stat(cpu, ipi_irqs[ipinr]); } + task_isolation_interrupt("IPI type %d (%s)", ipinr, + ipinr < NR_IPI ? ipi_types[ipinr] : "unknown"); + switch (ipinr) { case IPI_RESCHEDULE: scheduler_ipi(); @@ -963,12 +970,14 @@ void handle_IPI(int ipinr, struct pt_regs *regs) void smp_send_reschedule(int cpu) { + task_isolation_remote(cpu, "reschedule IPI"); smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST void tick_broadcast(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "timer IPI"); smp_cross_call(mask, IPI_TIMER); } #endif diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 8afb238ff335..d01f3cbed87f 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -539,6 +540,10 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, */ if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | VM_FAULT_BADACCESS)))) { + /* No signal was generated, but notify task-isolation tasks. */ + if (user_mode(regs)) + task_isolation_interrupt("page fault at %#lx", addr); + /* * Major/minor page fault accounting is only done * once. If we go through a retry, it is extremely From patchwork Wed Jul 22 14:56:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333929 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=oeXnmTgo; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=ugJJVEMl; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdpt2wX1z9sSt for ; Thu, 23 Jul 2020 00:57:10 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732785AbgGVO5H (ORCPT ); Wed, 22 Jul 2020 10:57:07 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:13442 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732143AbgGVO5G (ORCPT ); Wed, 22 Jul 2020 10:57:06 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmkQG012155; Wed, 22 Jul 2020 07:56:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=MjM2XCdIYkku7kyIlS/t7+DCWCHDBdJ+c56XH6d7TYY=; b=oeXnmTgo9bCLG6s0tZSe13m2nzUSfBBrdb9i+N+OhotkI5+qJFNAP56ToQkiykxbj++h ZK+t2veBQOni20E39/qR1eWGf0ya2clrxGEsNiBeHe3F5LKzCpci8VqYh3/nUACl9EK/ OxVzv6U5+E3Iv3vwFHDR+ZUWb6IPUVR9stNl0ZtHNMbEoBJ8E2IqvAU0kzEjKfLHvn+U o9F7cfhvrxJ+4gwERSY90PLvmC/ayDeUnMf+AYoCBoU7yP5NjvqTydI683ProqYtwkaM J9u1tAF0z38L1elimXqr1YzBMenKvIOtUAQ6fjRMDs0VLkGtF1UJBmv2aWiXzjbroL9R FQ== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrbj8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:56:38 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:56:36 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.170) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:56:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bOPDS623+Au7ZQFxvCyDlw+Rphx0y0xLsFvPL5Lqv4gDTbyW70H7aqw/MByxuGPgF9RuTGsSus2hLqm/P9WEG+jOPJ1obd5TmT7nvtCwnqhDQgR7vCiPfWl2w6wGSb7JB1qPp60BUR+wiZUzYdyqNR+mA4fY3F5drrqB9kVbsO+H4YMwC3i58/8/yhHNtBP3kxMTLgD1IuEDLXJbX/hMLKyhWiIuVBpfTCJ+JPJULpOQbPdRPS+EeB3qzDIgyt9tNKiOyw6W34UV7I8fmV7gm3c/8lc/zn+J3mPcmuirEn1clq3o0VEfq2DJ65/d9GcWDEnjPfomIlyMcaL7ygrAhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MjM2XCdIYkku7kyIlS/t7+DCWCHDBdJ+c56XH6d7TYY=; b=BeiD/dFWNXxw61CK8yxspyVmlXbSPgJUzM0mt0UJxJQavZg7D9dtOL/IxP+XPKp8G1UiUCAN6kwMvWJRf2nasNUDjlnO0oVITWYlBHxHlPPHpIgxoMR6yY1NcrDWdHEvxXjTBaUwUOpliObSn83uVaVHSAVvUthJ9Dr6k0oJi2o6STvezX0Dn3V8LxIBFmji9jZCv+bPc9lbjypofNUz7lPUBkr3/sBDxBIHtEQxr6/aD3DV5WaLSprNek3bO5KbvXm6rfrUqtNPKBCmUWzRe8QLrVv2nre7iIzBkgxt0HSFWoAwslYANYjQYU5/RfwmMj/TzaSZK24VH7H4Eap0QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MjM2XCdIYkku7kyIlS/t7+DCWCHDBdJ+c56XH6d7TYY=; b=ugJJVEMlVuZ7jCBCDNeREHujm/XnBe8NfBseBS/m5J4fXRDvMj49Rhw677j9KuaozbcBdo/wLi7p7ARRW+2c63Ba1wxFkEuQqWI+iyB/5XUstrQlmeOOz1hGIeMrG/aTDkZlfHZfrd8Fd/NEQyA9a9Tb3G4O5EdmoJV2t3Np36A= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:56:35 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:56:35 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 09/13] task_isolation: arch/arm: enable task isolation functionality Thread-Topic: [PATCH v4 09/13] task_isolation: arch/arm: enable task isolation functionality Thread-Index: AQHWYDhLQZFUyNR9D06u9QzvmRrPIw== Date: Wed, 22 Jul 2020 14:56:34 +0000 Message-ID: <8b9495dde0dc7dd0f58cc568474179cc2a7c2718.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9a4b4867-134f-41d0-9fa6-08d82e4f6dbe x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vjUmNBrdku6A2MSAMfYdyqRDGlBxp37Ksl55T1wRJJ44OU9z4RjeiGfXaK+Ox6S2x4lO3Ycs83R7X6Iq3WyV55Ivi9cy4D14alvvZ/VmkQ/AAPmn+WhA94paMKAX/nPEgVVEk3c1obvdVvFFrKRk7QhC1tjFMv2jjgjH/tErd95G8UC243IhL5qhe+hwqSNem65+6x4/R9kzmwQKYIlJoP+RfnGLovAaQi/zpOYsA6ogZq7n0xfKKLd6avfyQ0AmEuMv85LznRHMjoerqRJ/F95EeVeKRaYJqmnhmqYgskjRlYjqsSZwShvJlMBeIlwlG6G8uNdhdXeTE/oxcddyYA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(30864003)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: KZYA/OQ23FiJyy6JG1jG/tx2AmD2MWxmehPQURrr52CpZPhGGJPYiI2IBO6SOGHKI4ehYmS1egdrOZ0QuR0tdtm4ASUg6t2WDp9luRsoa7eEnOGWJaH6ibvOhFfn88mKt5ZjaEXlf9cGAh5y/0H53QZLn4TQjnTRPs0C8EU4DNe3NMLlRwyOkbBQY6R5dJBK+9g3WPoHONfhGgkghc5g7bhNOCv6f0bQZ+8ts0SDaEIycCgJi6lxemUzm8FdH1wpF2D0iDmPjpL+19I1rkihjDUBbCAAMqN2WF4KiQH/o6/jQvOXkdgaEqmHGV4W9x/1L2Jd4xAk2G8QhtxxupPK5rM9HzaojNpzZ8cnT2HfPT1lyIqgAxPBkrYs2IoCx+YEPx1RfILGLYun/Un75blpCK1ftKe43zYqtb4Cz8+/zCEg+l4RSl1Vi0t7EyzgvgkLEwf4z8LejPV7N2i0ZbiHFb9bfnicM98oo9Wmbwp3JVM= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a4b4867-134f-41d0-9fa6-08d82e4f6dbe X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:56:34.9869 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DJagwLVxZ/LAq+FUnkV8uRsqnBlK6UELZrFb43aKsNiBdK9utP1/Ej7A+JVab0FAnEZeKHHkEeZX4ocowYPFeQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Francis Giraldeau This patch is a port of the task isolation functionality to the arm 32-bit architecture. The task isolation needs an additional thread flag that requires to change the entry assembly code to accept a bitfield larger than one byte. The constants _TIF_SYSCALL_WORK and _TIF_WORK_MASK are now defined in the literal pool. The rest of the patch is straightforward and reflects what is done on other architectures. To avoid problems with the tst instruction in the v7m build, we renumber TIF_SECCOMP to bit 8 and let TIF_TASK_ISOLATION use bit 7. Early kernel entry relies on task_isolation_kernel_enter(). vector_swi to label __sys_trace -> syscall_trace_enter() when task isolation is enabled, -> task_isolation_kernel_enter() nvic_handle_irq() -> handle_IRQ() -> __handle_domain_irq() -> task_isolation_kernel_enter() __fiq_svc, __fiq_abt __fiq_usr -> handle_fiq_as_nmi() -> uses nmi_enter() / nmi_exit() __irq_svc -> irq_handler __irq_usr -> irq_handler irq_handler -> (handle_arch_irq or (arch_irq_handler_default -> (asm_do_IRQ() -> __handle_domain_irq()) or do_IPI() -> handle_IPI()) asm_do_IRQ() -> __handle_domain_irq() -> task_isolation_kernel_enter() do_IPI() -> handle_IPI() -> task_isolation_kernel_enter() handle_arch_irq for arm-specific controllers calls (handle_IRQ() -> __handle_domain_irq() -> task_isolation_kernel_enter()) or (handle_domain_irq() -> __handle_domain_irq() -> task_isolation_kernel_enter()) Not covered: __dabt_svc -> dabt_helper __dabt_usr -> dabt_helper dabt_helper -> CPU_DABORT_HANDLER (cpu-specific) -> do_DataAbort or PROCESSOR_DABT_FUNC -> _data_abort (cpu-specific) -> do_DataAbort __pabt_svc -> pabt_helper __pabt_usr -> pabt_helper pabt_helper -> CPU_PABORT_HANDLER (cpu-specific) -> do_PrefetchAbort or PROCESSOR_PABT_FUNC -> _prefetch_abort (cpu-specific) -> do_PrefetchAbort Signed-off-by: Francis Giraldeau Signed-off-by: Chris Metcalf [with modifications] [abelits@marvell.com: modified for kernel 5.6, added isolation cleanup] Signed-off-by: Alex Belits --- arch/arm/Kconfig | 1 + arch/arm/include/asm/barrier.h | 2 ++ arch/arm/include/asm/thread_info.h | 10 +++++++--- arch/arm/kernel/entry-common.S | 15 ++++++++++----- arch/arm/kernel/ptrace.c | 12 ++++++++++++ arch/arm/kernel/signal.c | 13 ++++++++++++- arch/arm/kernel/smp.c | 6 ++++++ arch/arm/mm/fault.c | 8 +++++++- 8 files changed, 57 insertions(+), 10 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 2ac74904a3ce..f06d0e0e4fe9 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -67,6 +67,7 @@ config ARM select HAVE_ARCH_KGDB if !CPU_ENDIAN_BE32 && MMU select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_SECCOMP_FILTER if AEABI && !OABI_COMPAT + select HAVE_ARCH_TASK_ISOLATION select HAVE_ARCH_THREAD_STRUCT_WHITELIST select HAVE_ARCH_TRACEHOOK select HAVE_ARM_SMCCC if CPU_V7 diff --git a/arch/arm/include/asm/barrier.h b/arch/arm/include/asm/barrier.h index 83ae97c049d9..3c603df6c290 100644 --- a/arch/arm/include/asm/barrier.h +++ b/arch/arm/include/asm/barrier.h @@ -66,12 +66,14 @@ extern void arm_heavy_mb(void); #define wmb() __arm_heavy_mb(st) #define dma_rmb() dmb(osh) #define dma_wmb() dmb(oshst) +#define instr_sync() isb() #else #define mb() barrier() #define rmb() barrier() #define wmb() barrier() #define dma_rmb() barrier() #define dma_wmb() barrier() +#define instr_sync() barrier() #endif #define __smp_mb() dmb(ish) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index 3609a6980c34..ec0f11e1bb4c 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -139,7 +139,8 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_SYSCALL_TRACE 4 /* syscall trace active */ #define TIF_SYSCALL_AUDIT 5 /* syscall auditing active */ #define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */ -#define TIF_SECCOMP 7 /* seccomp syscall filtering active */ +#define TIF_TASK_ISOLATION 7 /* task isolation enabled for task */ +#define TIF_SECCOMP 8 /* seccomp syscall filtering active */ #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ @@ -152,18 +153,21 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) #define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) #define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) +#define _TIF_TASK_ISOLATION (1 << TIF_TASK_ISOLATION) #define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_USING_IWMMXT (1 << TIF_USING_IWMMXT) /* Checks for any syscall work in entry-common.S */ #define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP) + _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP | \ + _TIF_TASK_ISOLATION) /* * Change these and you break ASM code in entry-common.S */ #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ - _TIF_NOTIFY_RESUME | _TIF_UPROBE) + _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ + _TIF_TASK_ISOLATION) #endif /* __KERNEL__ */ #endif /* __ASM_ARM_THREAD_INFO_H */ diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 271cb8a1eba1..6ceb5cb808a9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -53,7 +53,8 @@ __ret_fast_syscall: cmp r2, #TASK_SIZE blne addr_limit_check_failed ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK + ldr r2, =_TIF_SYSCALL_WORK | _TIF_WORK_MASK + tst r1, r2 bne fast_work_pending @@ -90,7 +91,8 @@ __ret_fast_syscall: cmp r2, #TASK_SIZE blne addr_limit_check_failed ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - tst r1, #_TIF_SYSCALL_WORK | _TIF_WORK_MASK + ldr r2, =_TIF_SYSCALL_WORK | _TIF_WORK_MASK + tst r1, r2 beq no_work_pending UNWIND(.fnend ) ENDPROC(ret_fast_syscall) @@ -98,7 +100,8 @@ ENDPROC(ret_fast_syscall) /* Slower path - fall through to work_pending */ #endif - tst r1, #_TIF_SYSCALL_WORK + ldr r2, =_TIF_SYSCALL_WORK + tst r1, r2 bne __sys_trace_return_nosave slow_work_pending: mov r0, sp @ 'regs' @@ -131,7 +134,8 @@ ENTRY(ret_to_user_from_irq) cmp r2, #TASK_SIZE blne addr_limit_check_failed ldr r1, [tsk, #TI_FLAGS] - tst r1, #_TIF_WORK_MASK + ldr r2, =_TIF_WORK_MASK + tst r1, r2 bne slow_work_pending no_work_pending: asm_trace_hardirqs_on save = 0 @@ -251,7 +255,8 @@ local_restart: ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args - tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? + ldr r11, =_TIF_SYSCALL_WORK @ are we tracing syscalls? + tst r10, r11 bne __sys_trace invoke_syscall tbl, scno, r10, __ret_fast_syscall diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index d0f7c8896c96..7b4fe15ba376 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -24,6 +24,7 @@ #include #include #include +#include #include @@ -917,9 +918,20 @@ asmlinkage int syscall_trace_enter(struct pt_regs *regs, int scno) { current_thread_info()->syscall = scno; + task_isolation_kernel_enter(); + if (test_thread_flag(TIF_SYSCALL_TRACE)) tracehook_report_syscall(regs, PTRACE_SYSCALL_ENTER); + /* + * In task isolation mode, we may prevent the syscall from + * running, and if so we also deliver a signal to the process. + */ + if (test_thread_flag(TIF_TASK_ISOLATION)) { + if (task_isolation_syscall(scno) == -1) + return -1; + } + /* Do seccomp after ptrace; syscall may have changed. */ #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER if (secure_computing() == -1) diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index ab2568996ddb..29ccef8403cd 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -639,6 +640,9 @@ static int do_signal(struct pt_regs *regs, int syscall) return 0; } +#define WORK_PENDING_LOOP_FLAGS (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ + _TIF_NOTIFY_RESUME | _TIF_UPROBE) + asmlinkage int do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) { @@ -648,6 +652,9 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) * Update the trace code with the current status. */ trace_hardirqs_off(); + + task_isolation_check_run_cleanup(); + do { if (likely(thread_flags & _TIF_NEED_RESCHED)) { schedule(); @@ -676,7 +683,11 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) } local_irq_disable(); thread_flags = current_thread_info()->flags; - } while (thread_flags & _TIF_WORK_MASK); + } while (thread_flags & WORK_PENDING_LOOP_FLAGS); + + if (thread_flags & _TIF_TASK_ISOLATION) + task_isolation_start(); + return 0; } diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index 9a6432557871..6a7a4ba02cc4 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -559,6 +560,7 @@ void arch_send_call_function_ipi_mask(const struct cpumask *mask) void arch_send_wakeup_ipi_mask(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "wakeup IPI"); smp_cross_call(mask, IPI_WAKEUP); } @@ -578,6 +580,7 @@ void arch_irq_work_raise(void) #ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST void tick_broadcast(const struct cpumask *mask) { + task_isolation_remote_cpumask(mask, "timer IPI"); smp_cross_call(mask, IPI_TIMER); } #endif @@ -633,6 +636,8 @@ void handle_IPI(int ipinr, struct pt_regs *regs) unsigned int cpu = smp_processor_id(); struct pt_regs *old_regs = set_irq_regs(regs); + task_isolation_kernel_enter(); + if ((unsigned)ipinr < NR_IPI) { trace_ipi_entry_rcuidle(ipi_types[ipinr]); __inc_irq_stat(cpu, ipi_irqs[ipinr]); @@ -701,6 +706,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs) void smp_send_reschedule(int cpu) { + task_isolation_remote(cpu, "reschedule IPI"); smp_cross_call(cpumask_of(cpu), IPI_RESCHEDULE); } diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index c6550eddfce1..23bb383e479e 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include @@ -330,8 +331,13 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) /* * Handle the "normal" case first - VM_FAULT_MAJOR */ - if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | VM_FAULT_BADACCESS)))) + if (likely(!(fault & (VM_FAULT_ERROR | VM_FAULT_BADMAP | + VM_FAULT_BADACCESS)))) { + /* No signal was generated, but notify task-isolation tasks. */ + if (user_mode(regs)) + task_isolation_interrupt("page fault at %#lx", addr); return 0; + } /* * If we are in kernel mode at this point, we From patchwork Wed Jul 22 14:57:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333930 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=EVJa0A16; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=hk9Mbd9c; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdqx442tz9sSy for ; Thu, 23 Jul 2020 00:58:05 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732799AbgGVO6D (ORCPT ); Wed, 22 Jul 2020 10:58:03 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:31122 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730382AbgGVO6C (ORCPT ); Wed, 22 Jul 2020 10:58:02 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEml23012168; Wed, 22 Jul 2020 07:57:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=39S4vVNIusSsi2LuvNZmhBBhMhNTv0T4UrDYfmP6564=; b=EVJa0A16lfuFUlWKLZwXw+n8R881k/ZS22EehTrvAnPM66x6SDId+O0ZbNDXSmi0+T/0 B757ZJwzBlKBCttui7WlyVvARBC7lrNUCEbfRcDKrXist3L4yVRGDhUz2q7XpDdZTbUi LH1FkhAETb7mfolgyHqVT5LNvd5hQZLuHe7l1jSGbd2q70wfiZEa/myKJJg10qCTVZQN R3Z2JyKusEtFW2a4DmD/ai7dXbzGnY5Z6eNeYvqm2BptvjP0nYM8PObbV9CglArzVxu/ n4M0gyQCBnsoRDSQWlbTcNl/4X8dDFwlZSfA70nrjXSoKvDQK5T8qk5cKPxxB+wFkjBj pw== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrbpe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:57:36 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:57:34 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.177) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:57:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Tu4VdwZldBfUismKGfeOqVVBE1uc0BQl4NeXvxy/Aa8/eHi5efGhG7zKLvmEbOrFtTcKR4zT97RBX5jmV7UgRsVUz6U8hN+fnJUW1YN6Lkicuv0+K05S5qTLvfB1WtTIJmYkBF7SVYwaoKHVDevmiKWDa5kCLzNN8qeZ0KkyPf9uPX0GOxCYdFZ0rL6YR88go3H8SRkvpJ0DIwIKJgQumDo+OOeRTN6ih6kDQLQltdlj22i5KKh7/pzBQuP90Vj0yWLAnjKvwzEmAJZga42BqoanQM2Ie8lJ74sMFv0tT08zzjTMXKdqd3GVTiD6W70T272U8u6D67tNaXll0BDwww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=39S4vVNIusSsi2LuvNZmhBBhMhNTv0T4UrDYfmP6564=; b=BDCt/f2IbSIw4jmK1W6+I8fadY3RDlv9TYDWAIRJ7MKjRB+yDmQ6vPnuLItVvQN51GTqat8DwBS6V0ZNIX6TXUNj8yvW/oWoanfmwvNknE0ugHHUauT3iCiL0ZaGz6FEtk9fTEm72C8EFwvddJuRii9gLlAUkkBnna0OHYqo/hRFXjo2IucoEqBMM5j2yYDCBxPIDVcrR3TUUcthZgAiS8ujrUheuiCKhdPwQ098ZlQc0+vUhmiyCnOSlPtviNcPvJc9zcM0aAH2NlIKiFIlQc277pw3e5kw1AFwWXzgWM3se35ql3a3NfhIiOzj/jZtxDli8kZ2XQUCmeQJnuSJug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=39S4vVNIusSsi2LuvNZmhBBhMhNTv0T4UrDYfmP6564=; b=hk9Mbd9ca2K2qL1TZ55BgoIt9eD590LWokURNZkJ27imn4sFtN8H6utPmotL8ape1eojbbKqhpkiH97mbgRhqGgosYrv89KkJ/aICop50lttnoMeIRxO6RRpyFVahb6hofN3IBUTbkSLcyVcuQm85GPFDHh080IoJzZkJMNl87I= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:57:33 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:57:33 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 10/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Thread-Topic: [PATCH v4 10/13] task_isolation: don't interrupt CPUs with tick_nohz_full_kick_cpu() Thread-Index: AQHWYDhuh3VfINlM20KwSBz5WuaF0Q== Date: Wed, 22 Jul 2020 14:57:33 +0000 Message-ID: <5acf7502c071c0d1365ba5e5940e003a7da6521f.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 60530ab6-8501-4f78-a3db-08d82e4f90c8 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: kAcENqRTVC5LJDNWTFi0gJmqH2I+N5X/pfX1V9oBWY4/qyx7ShZZPp5Dc/7ZIfLdcXeCQk2sxHyi7MsmO3DOlAhXRtgug7/EwIVwIxWPB0Or+BHjY0GbOVv3rJXlsFLnQn8iBomnc6kszGA9291xRqBV+SNEN958jtLcVIKkjWWZJ5ir2FVeRSRoJjuZv0Cub7G6iOVGhmPfTH3gh/HrVdfswglAzLCWWGu6W2f4c/H+btRK2P7LqmJu0mR2UNIS27TlGDqVY2eKvJFNJyH/xGOHDXF0YRtKsVu0BuXx9P78Tm8MMjWgwCTFA+tMDvsBAjLJUJYkG+fipYcaboYBnw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(4744005)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: hswFjFfQgQYiuGRCvgk40lPngN77FXIzxt8LURz8O65T+1PLG18Uk5zwZ8uvbF7Ik6iW8iwFxbooCp4hpDjR74dnQdtmSoIMz6cUUZl1EfByxnX/3QQ7eFJtQs0f4LAvSr4rZdSPJ+ZIVIngMoY62TH41L2XdsDRPVV9Beuep4CcH/L5M7uvtLUSLsRisW9rZ8UPjQgtqG/r3u2Dep3pvvPKkpI3fPAr6bI9ytvP7A/L8ognHyFMCtLV0S/s7XQYo2tFWi7wprYtipWgug5/y2oB0qVbLrfhpcaSs3SO7+QA4ZVv6xHKGQQUcuNtHDTKDp6yH6XrS5xHcHSWq9/yjHT+ak+P/Z2KWJWfFCBpdSCggpWm4GxGQ2NKGfJSCmW35Mo45a/+g/KhgQj22OkYQTuIHxj4CH/Jkv95prSKWmiWmlVjrgErrv1Ix8n23tnA9kemjK/Xz7brdP5HqKK0I9Ngn5XmS+yfBqy0bPovHQu4PRnnXF1V/haa7ZHWfARE Content-ID: <3D078BC1A3E6A443A62F85A300A63A00@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60530ab6-8501-4f78-a3db-08d82e4f90c8 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:57:33.8038 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: k1ncmc415f4yR0Orvm+hvx+HAnxoA9diUWrhilJ909VBki5tlOwBAVbVp/a0GwtRS+kh0xjjeGfj2mUUSdno+Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov For nohz_full CPUs the desirable behavior is to receive interrupts generated by tick_nohz_full_kick_cpu(). But for hard isolation it's obviously not desirable because it breaks isolation. This patch adds check for it. Signed-off-by: Yuri Norov [abelits@marvell.com: updated, only exclude CPUs running isolated tasks] Signed-off-by: Alex Belits --- kernel/time/tick-sched.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c index 6e4cd8459f05..2f82a6daf8fc 100644 --- a/kernel/time/tick-sched.c +++ b/kernel/time/tick-sched.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -268,7 +269,8 @@ static void tick_nohz_full_kick(void) */ void tick_nohz_full_kick_cpu(int cpu) { - if (!tick_nohz_full_cpu(cpu)) + smp_rmb(); + if (!tick_nohz_full_cpu(cpu) || task_isolation_on_cpu(cpu)) return; irq_work_queue_on(&per_cpu(nohz_full_kick_work, cpu), cpu); From patchwork Wed Jul 22 14:58:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333931 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=Sj5Yn0q4; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=j7ProWtc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdrx5tYtz9sSt for ; Thu, 23 Jul 2020 00:58:57 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732807AbgGVO6y (ORCPT ); Wed, 22 Jul 2020 10:58:54 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:1568 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732673AbgGVO6y (ORCPT ); Wed, 22 Jul 2020 10:58:54 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmk7C012167; Wed, 22 Jul 2020 07:58:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=8vyugBPOdj7+7vqqpnHMWZbC7+ZwE+5fwk+wZ9JNKXw=; b=Sj5Yn0q4bYnNMYhbmqx9f4hyaLKEJV4Q/tO5SiCLERmPVXc6qXPETJbxNnusXSmzSZhW ds/4uoMKgHV4HPsTVseNNGJg9BehUPZxLBVAQ10X7ruS7kkwDXVJ4wXdTNH3qcDtEChn mXgSaFfWIZtL/XUCvRCDUT0yUxBVCccIqomRmSRYXHGIkrK0ID6VymVsOOKz3RUKkUe1 FtaTnmGRfEdpbUzMr1o0y71JmEda1c7lhDkwjK5p/WoiQ+hTrjsjjSCa5neKUN4Ivsk5 570jScY9HOVMVNGkURuXAjG55A70/vfZzdLJWODzFKsPjM6dfGyUmAmDUqeAfNHNAEES qg== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrbtf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:58:27 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:58:25 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.173) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:58:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=I1shl9hOeOKK/XKcLmraojzrBbZ4/VaxuQTkF6AL2dyMPRT2/mlv3xVu72q6t/5ZXEy3DueY0mWRmu51BMp50e5EcICEwgzudwlGdZVgFq6y9Ivi6fmxUzu32AvLyj2IK6PhV04kzMOCIP9xJ7LR9RzIbimPJAvoamJug3A5Va9KbO1jxhEpvPt2WAyiR/IBF7WtLlRJBM6vzvnL+h1Cc+J10jyvkHIyqrGCgNtoyKd45nT5PaV/7ekopGLnML7xXrLQ1FSc33GTg5g5FbzXbsyieRxj9ySRNp4RQlhXtfL2JNXu4F9ZbobtFaprLGY3w4P0ExAtZlSOjkcq/J0caQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8vyugBPOdj7+7vqqpnHMWZbC7+ZwE+5fwk+wZ9JNKXw=; b=Vb6lVrhFFg8zQNkSSJYZDMhq6AtjIYC6mTMKvXzkZgZppc6shhkmrrn7N9/WVpv57eswjua+B+DaeSvDNWDJIiiZChsoCDkznOHL/PDlquIBmwKahe6neCK28ymxDjEQM8Y1OOzYitKQq5S+z+++x3JzDfFZoA7HLWDWeRFoLJwqhODlZOvHKZ6jXpq7gg173OqI1GTWPbfMFS/cEjBmKRqAoLcr+vsSQZqjY5E1LD38usOm6LJlQ6EyIupTKs3ZYP4hSyvbrefpn+kiXPLi5RGHJDF6tARsgY6x5HOIAuNKl69tuFWymwbi3D2JN3C12vkkPhxz63TShuyeHn8rRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8vyugBPOdj7+7vqqpnHMWZbC7+ZwE+5fwk+wZ9JNKXw=; b=j7ProWtc+naO3FqYHU8C9eJ9fEiQKt+6a+Ff5w4ndyOfiRHRithJPC0tHcDU8+SWr4RUadp6Md9O5O5NxDQnkbQe944GLzk4rbpSLdM80aSXJHcIL8kdfPP5wZL8NYsmzptELra8BWvDay7CiXds/sRO7qaR25IHysEK4P0EY9Q= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:58:24 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:58:24 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 11/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Thread-Topic: [PATCH v4 11/13] task_isolation: net: don't flush backlog on CPUs running isolated tasks Thread-Index: AQHWYDiMpgobFulKPk24EMbX9A+yKQ== Date: Wed, 22 Jul 2020 14:58:24 +0000 Message-ID: <01470cf1f1a2e79e46a87bb5a8a4780a1c3cc740.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 74c4544b-bd8d-4e50-26e4-08d82e4faf14 x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6108; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ltkLcEpv1VSPZTCvJIW5W7QVbfePRWVNMV8TANb3XPuyGQj2+W7pd5QZivs2VnY9DxEtBaYgHg3f9fcmfVH5o0HToMMDPSX/9F17NRnbzh+lDhtP4cr8VSIuX5HQMfUueA157kvnr7Di2ZxYW5LNk/WIYlyoOqXMrW2XiH+UlOyOegr/0bV7E0wWdCGC+utp7msuttRGB2s4nJN2a9vzL6f21VhW1f0TYsjnJDAI7vRnMVOkDd2cGLL1+V79pTs/6rwLXqbEkNEDiQeEw8eqIoNd8RAVAdsLV9vtVGxYswCjgeDiL48Ui6p/h/V5pG5K x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: gYDiOzNkI0/JljNr0ydprnR8hKP/Rm/OgsFIuEEas5Xu+zNYuP4eiXHveilvamrS9VO9ai14A+wQyFcjuN6h00vPy7XslYq1M04vrCzb7R5hAWafLSsWuXx9qoRCs3bEBhvHGBrUvxQ+x84G6GTXNo0qd4RNogOk4dESUlfpA/gNKvjmB6x8S7L+wB2vZuiH2dtElyfvS+pX6+tYCLL5xmlZBsT5NHzTfPbyUrYLxfsliKjcEEt57MJMEyY6wYyAxAnAdDN/8aHuRflVyWWtaktLtxoQ0SRu7lAP1PkfWzGIrJS0dFi25aCWwu77wOih9RP2fwwQ8kolqN6bL4Q0PYz7iWbRJgiqoa8LSHy1f4zL8fpLxYU8AO0XVg6Nq8GblIFilLVXSnQreEff+FAaklcY/regrCQXZ98lTN7+dPRMZg+Dr8vxacEq4vdxhFSzcUF08/hXFLvVzkcR0PA4fCjym1rN3tJHpjaqZrdQ2u2kRC9w1sGxhNyXQfI95WyQ Content-ID: <392AE7C4E756DC44BB5350C22441BE67@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 74c4544b-bd8d-4e50-26e4-08d82e4faf14 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:58:24.6373 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 5aQYoVeUE6y4IndylvkwJ0rzvKg513No9iUPycQ8IAWR+JGajkoAOWxfHpPYd4YdmYkyMW0wlvXraI1CHNgIRg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov If CPU runs isolated task, there's no any backlog on it, and so we don't need to flush it. Currently flush_all_backlogs() enqueues corresponding work on all CPUs including ones that run isolated tasks. It leads to breaking task isolation for nothing. In this patch, backlog flushing is enqueued only on non-isolated CPUs. Signed-off-by: Yuri Norov [abelits@marvell.com: use safe task_isolation_on_cpu() implementation] Signed-off-by: Alex Belits --- net/core/dev.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/net/core/dev.c b/net/core/dev.c index 90b59fc50dc9..83a282f7453d 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -74,6 +74,7 @@ #include #include #include +#include #include #include #include @@ -5624,9 +5625,13 @@ static void flush_all_backlogs(void) get_online_cpus(); - for_each_online_cpu(cpu) + smp_rmb(); + for_each_online_cpu(cpu) { + if (task_isolation_on_cpu(cpu)) + continue; queue_work_on(cpu, system_highpri_wq, per_cpu_ptr(&flush_works, cpu)); + } for_each_online_cpu(cpu) flush_work(per_cpu_ptr(&flush_works, cpu)); From patchwork Wed Jul 22 14:59:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333932 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=eYV0jDsB; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=cBfbVZ97; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdsy5rqQz9sSt for ; Thu, 23 Jul 2020 00:59:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732829AbgGVO7s (ORCPT ); Wed, 22 Jul 2020 10:59:48 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:41176 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730465AbgGVO7q (ORCPT ); Wed, 22 Jul 2020 10:59:46 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmwsv012225; Wed, 22 Jul 2020 07:59:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=eYV0jDsBzQD1G4kxZm1E5pOeTkJ1S8Ad2j7gN9NaW5DlODE0N2O9x3fn2JdCVVLRX59j 0WZLZWn3t+h1yPe0sZK6bxRm7dl5Knihp28FLty81WKbUt9KhOtwBKPdAnZGODR7fprW mp8iNp3GeYOY47y7Y+lGBD8InVRIseLvCni5V72uNlTm7LcroYUL0jjMX+OeUI5Y7/Uf yLSrvqWKNrYtgYWW4sxTk2POoFur7KAnnftb7v6//nFxDAMW3bmNwx9rHL8ABZHxkHCS QAduYybfEWCOVf5WVAIecCqA0hMOuWH+sMN0uwhxGljIgGBkjrch6xpXYW9GAlZvFOMo KA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrbvy-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:59:19 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:59:14 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.36.56) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:59:14 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bBbnrt582VYmuzGJgEvICo8x3wDdb/XXYXEAjzFKtPi/aVt2gZC84jKEisvzahuClypRv1oNA1EppYdW8sWvNNXxvMLKeuY03QR8TpHkTOcml/cntcd3u/ny93JqLf1eVKF5vh3KEIjmFvdflX6RKdmSWciG025ka2QK/TP7GuGCT6ED/IOsDq4fsfdu2V4TuGtC7wjqvCryczxlVmxAERyHRHFPgV5CAdcG8+p1dH0i+7E8Ihy9XyUWSF7PDOS7Nse2sYJrJIITOPdZnVHiS0I6stLiQkwluq5IuHB9GuNOSdNpbABI29vRWg5D2IlrfZujleo0IA/guUZK8Kr+/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=eZ88cXtHsk0vtqbayk6nCmn4pzetG/Lj/Q+UzxXWeZ6qXYocQBvdQtBZjTdT3C1+gc8dWJP8G3ytCBWGPpzv5KQ65ceOmA+P17p/ddW2jTtkHOxVuoMIwGl2fPUT+7fRD5PaOUMYcbjQWkWFofQ6VPRvxZ3EQcUOtgec7x/zb96N0CmlmWO+GcvGghQ73eXl+sQ2MY+Fmuae57FuoDyhg3FZKXPIq6AbT8pob7EzYvliEiO0ve3yMs/FgmRdR9cU4lvFL6zhFN0z/qpS3EdSF/lhDEiaOmGbnXkaoJdu1WdLMoAe7NCaG32TyFerslTGmlXQpSoF5DX7UCpeAvxzdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=cBfbVZ97VkaCNO+e+b8JebOG6R9xvwzne32IGmQ4Tupld/dmYsoOcU2wv3o49ozCk5kbGqgrshyc8spV3xrzSsr4VwBctBBx5huNJoWQMtctHCXJfVDmDn8GYdMdtj7aIjr8LJ69Uq6Bw8eW3d69NhCpOYSX4oqyA5WTI+GUbyE= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:59:13 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:59:13 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Topic: [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Index: AQHWYDipehJFQ80zLEqG2+MBm9Fm7A== Date: Wed, 22 Jul 2020 14:59:13 +0000 Message-ID: <041a4c844b4a0f69aba6df4686e1952dd0f91931.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 586cab22-9cf0-4e35-07d5-08d82e4fcc0e x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: g/rG6B3bl1C8XptqxTm69bNTncIU8+KFD1NqsYEpBnh+lGDeWHJ5HkSgXL8t1DRY8Hq/We4e1CGeiz58g+TPvq12t+tEyZOoPNnY1zq4QCkD77bcuKUsg4LtMXwkituEhDrpJTVFOd1OP+S6PglUb5USnRQLbCpEzPMIcdRzBx7VWuAiD+gQrwpxkbnq6lgZrtXLvLr9cDab0nD+jiKqdnz7K17rv9dyhUND6BVr583hV7cxZRvwOVqgMU/+5EqxIf6w/5mB0JC0tcl8xddNCGdqaW0nR8R4QWpvPea4dXuKi/wL0hgwo00ew6pIWh4HkLW0sCmvB0Txfx139NWehw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: 6x3a8HY0WvuASH2wmVrBcT7rU/tL/KLtA9sNgal6lPTJ6+CVdmFXwcpsn3gFUyVrEcX/YTvq2NUAt1swKCwiF5qCjxVWFDVV85ncBJ9iPH53ebx53KRBZLdgX7u+p3pujBaSnaEKKdXbFi8xb5xnVADedmiIJsvTU7tNezC7vozNuBB10Z3eyVtQBGWg1SuUb+e9m8FwuJewXTZDCczIyJ4xeNrWkqcsRT+a6DMIUgrjYw/2B/gp2aUfDU0w7Hva6WTKGx1Kr81dmQfk07zxYaxAdIYGVMrmj/1SD0QMxkiimv+xH+aNSu8D9zGoheS9hnYxAeW83oXwo8LVAHMTMCu7MUadQcd+XqtsAoBFn/nQBUwUV7xtOL6lLrYQpvN3WmsBwRnkuJAezcPVT/XLWxw+acyarfTEo+j1bnHpir3y+BciVCGDWiwkPFC48Hhwr1lBDH+J2+e+FoHr/8Gwq0DeW52RgMD7VcaZ8uq/c69oWQppmbcyDIOVTnNkjzYg Content-ID: <5763AE40EEE4674CB00A185A60CF3731@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 586cab22-9cf0-4e35-07d5-08d82e4fcc0e X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:59:13.2631 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 38yJa+FR9/bJKRNd1klcRODFvVOX0EeSawGWXD2mWwDNzpi6t1WgrU5M7lw8f/1D0f6zo6bU8lOh8LmDs9+HAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov CPUs running isolated tasks are in userspace, so they don't have to perform ring buffer updates immediately. If ring_buffer_resize() schedules the update on those CPUs, isolation is broken. To prevent that, updates for CPUs running isolated tasks are performed locally, like for offline CPUs. A race condition between this update and isolation breaking is avoided at the cost of disabling per_cpu buffer writing for the time of update when it coincides with isolation breaking. Signed-off-by: Yuri Norov [abelits@marvell.com: updated to prevent race with isolation breaking] Signed-off-by: Alex Belits --- kernel/trace/ring_buffer.c | 63 ++++++++++++++++++++++++++++++++++---- 1 file changed, 57 insertions(+), 6 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 00867ff82412..22d4731f0def 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -1705,6 +1706,38 @@ static void update_pages_handler(struct work_struct *work) complete(&cpu_buffer->update_done); } +static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer, + int cpu) +{ + bool rv = false; + + smp_rmb(); + if (task_isolation_on_cpu(cpu)) { + /* + * CPU is running isolated task. Since it may lose + * isolation and re-enter kernel simultaneously with + * this update, disable recording until it's done. + */ + atomic_inc(&cpu_buffer->record_disabled); + /* Make sure, update is done, and isolation state is current */ + smp_mb(); + if (task_isolation_on_cpu(cpu)) { + /* + * If CPU is still running isolated task, we + * can be sure that breaking isolation will + * happen while recording is disabled, and CPU + * will not touch this buffer until the update + * is done. + */ + rb_update_pages(cpu_buffer); + cpu_buffer->nr_pages_to_update = 0; + rv = true; + } + atomic_dec(&cpu_buffer->record_disabled); + } + return rv; +} + /** * ring_buffer_resize - resize the ring buffer * @buffer: the buffer to resize. @@ -1794,13 +1827,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, if (!cpu_buffer->nr_pages_to_update) continue; - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to + * update ring buffers until they exit + * isolation because they are in + * userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu)) { rb_update_pages(cpu_buffer); cpu_buffer->nr_pages_to_update = 0; } else { - schedule_work_on(cpu, - &cpu_buffer->update_pages_work); + if (!update_if_isolated(cpu_buffer, cpu)) + schedule_work_on(cpu, + &cpu_buffer->update_pages_work); } } @@ -1849,13 +1891,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, get_online_cpus(); - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to update + * ring buffers until they exit isolation because they + * are in userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu_id)) rb_update_pages(cpu_buffer); else { - schedule_work_on(cpu_id, + if (!update_if_isolated(cpu_buffer, cpu_id)) + schedule_work_on(cpu_id, &cpu_buffer->update_pages_work); - wait_for_completion(&cpu_buffer->update_done); + wait_for_completion(&cpu_buffer->update_done); + } } cpu_buffer->nr_pages_to_update = 0; From patchwork Wed Jul 22 14:59:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333933 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=QzpCPRXh; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=aNdc7BCH; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdtj54HDz9sSn for ; Thu, 23 Jul 2020 01:00:29 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732834AbgGVPA0 (ORCPT ); Wed, 22 Jul 2020 11:00:26 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:13780 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730465AbgGVPAZ (ORCPT ); Wed, 22 Jul 2020 11:00:25 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmk7P012167; Wed, 22 Jul 2020 08:00:00 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=N9sM/rax6qcAio4WppyrBpzY7VSKK9ffMiYkuBPlkkM=; b=QzpCPRXhJDiC8rYDx593hgsdnQtJC86vncF/uqUk5Jqp8WpNxvY6T8W5TGWjugcvh2h9 CVzo92TfoX0KwAEjZ71wFT0cVp0eR7WwaJVOcyMCGFzRIHyFl23HziJmZcPjVuDma/bH 6KUurniQy5ATWjU8LWxaxDmhTUEP7Rm3ddpWm67pn97riryqn/1DjRELgYpTqR+IY5Jj EcohMt3heFzLbfC6XwXdv5bKtWZNT2ang1KKN0K434nQ/Grdr5y0e46NAIsNW4CZZj+C if56PPHVEhRF6FnXyWAqpjfnCTQLp7c9imGMZyDBs3LU0e10L8wV/PvavaiFdu/h2DhU rg== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrc16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 08:00:00 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:59:58 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:59:57 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.173) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:59:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ftSkVlWRlLrM8Vucky8Lcl8ahpZaW/CrwrFVWkJyIfktkZ3f57i4huILWPIEa/UtHI10pEtvMjxUItk0oM3mF5zcp1KqZKaNpCmOmWO2JEoQRWdA07vyF80LLj8WPBL0ZpfyOniRFwOGqUfL5RMHOK25TKoZiEFqIYD4DYkYxbRCTBstmLjvspGozadLlOQThsKg5uM61IZZKgxtdD0Gt2IxwGX8FZDrJP0bJ+XfHGJ7ftXEgpaHjB5bbgo4n4KqtK7RlH3gTogsxThszwMkk8AVr7VIGXRjxJR0EUW0hvoEs+DqFgReCS6UgBfY92p5laEZqmDEMN96eY2knqvh1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N9sM/rax6qcAio4WppyrBpzY7VSKK9ffMiYkuBPlkkM=; b=fhjzBUJPskdVX0N0k+YVL3vUzxOsP5B0GoZf0lG3OoDd+BhogS/RnaeE4KdLC+H+IHNzjVpe86We0Ev+r36URyxw5YOTokvoWxCa9cGT8ZiUFKViEB/pJK/jeMz8gLMHzkMo6bpbFBlZesCnQsOHVBfFxWd5QafcVi0DFH8zchYj2lh4gU6zv+Qmc+NeUqbNVz5sxQt553kDrcKK+stJAaU5Ny19StslbXFRW+vUz2qoqMa8LLVK/bLQ9yzPBRdpKwS4JDAbIr9V5FwuUbmPOcn9Af1x3TOyvRgV95DSQNrNXxCP4+j3L9laiZ11mQFUdmI/THf5DTPIJwb6OZtkWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N9sM/rax6qcAio4WppyrBpzY7VSKK9ffMiYkuBPlkkM=; b=aNdc7BCH3bXSCX3R4GojVhj7I8VQiaQDwzHQkGXo6ICK5NhBNlZyEIJOVoQ0jcOKLBDS6ll+qc4jW8MWsV7E2j83MCnwTGheDxXWcWGE6jWeafD6hvxDF63D0qSGYugZxsdwJ+2VUsPDWxowT0qhF7v4I0UQcKQumqZv192Zp+g= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MWHPR18MB1517.namprd18.prod.outlook.com (2603:10b6:300:cd::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.21; Wed, 22 Jul 2020 14:59:55 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:59:55 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH 13/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Thread-Topic: [PATCH 13/13] task_isolation: kick_all_cpus_sync: don't kick isolated cpus Thread-Index: AQHWYDjCVLK1C7iHf0mh+9tcFOIgtg== Date: Wed, 22 Jul 2020 14:59:55 +0000 Message-ID: <5d0088ffd762b266124dfa5da9bcef0d9ddbae96.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1e867496-af1e-4a15-573e-08d82e4fe52a x-ms-traffictypediagnostic: MWHPR18MB1517: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:873; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4ZK13hBho0PjC9DJmS/KBDt/6ebv9zXgnck8/c9Llq5vUeaNi4uDTHQdgsljkxu1GkAbfYcK6Ri/rXOTWDytCj8ve3hdReuTh1Fho4DqkYKmlTVEbwGVUalNDxwMV+sHE9uTCpAm92lzqX05Tsh3LnFlk47k06mH4SS+PA46X9c90bSfQ+EFsnpOaJcvmTgpsIhUPmAGHxpvzJq+lJGiHTc8Er771wU+EDlbNUnpqQx7KUBIpuRAsmhbGwoQVYDWomwKZg3A0Chq/axcFYiYy3D3CXnNc5qX2lNeHynIgcK9kCIH/ORe2T8kUDpjiGbT x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(366004)(136003)(376002)(396003)(39860400002)(346002)(83380400001)(316002)(6512007)(7416002)(478600001)(66476007)(6506007)(6486002)(66446008)(64756008)(66556008)(8936002)(26005)(66946007)(5660300002)(2616005)(36756003)(71200400001)(8676002)(76116006)(91956017)(4326008)(110136005)(54906003)(2906002)(186003)(86362001)(4744005);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: Np0Fwdd8jv1fxTSsx3MwsF69FPbUpsSOKKMrW+n/wr+2jk44eHPHPZACNYS/eHAmXhFCTluXLhW9nEumiVd8/Ae/9snwf0KsjfMcEl5bwOY8TL5k3zEV2rX46xWRcfJ2kLev4OQofpA2WJ6uV6out9vQCaVU7TPmgVUrrQJvU2C6kYlpme6mFRZK8+ogHHEEeyiVa+Pi/pKYo8xEgxlQFko1duhwIyuRpG1+x2kWsjsFeuNu46vmnlCO5fVOx5EG91SITZjsmLiSwkCsw8kU7lpp4i6qzOeq/4cHuyRbt5KFlilzIwdPUXKQuTmMkIsFhmlRVsAc1ATR0N8GnvGelFmqFKDNX/Zkaxp8mm5OV5fQTb3zSAmRWgCpT5gSseJ5hZ39LdXmD0RPS/PXsr+Gi5gBPKJBggyZ1W6PPAmnCdsUT7N0zoRsFxZElbn6kzK1BOhJIqUkUXSoT6f5F/wzmWE5OVhbUXb+Xa0TKV/SDf1cewexPaLXZQrPZqYmR70E Content-ID: <7BED713CFA78E1408C46F0C24DF285EE@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e867496-af1e-4a15-573e-08d82e4fe52a X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:59:55.3607 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rbwq8hvU6DJjX9uMARKpktL7Ahhhm6+UuFQSQHRXsBBM1taU87vvBcP+7YdgrQcsSeVhTKsjdmZ2sB3fW8cq/g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1517 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov Make sure that kick_all_cpus_sync() does not call CPUs that are running isolated tasks. Signed-off-by: Yuri Norov [abelits@marvell.com: use safe task_isolation_cpumask() implementation] Signed-off-by: Alex Belits --- kernel/smp.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/kernel/smp.c b/kernel/smp.c index 6a6849783948..ff0d95db33b3 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -803,9 +803,21 @@ static void do_nothing(void *unused) */ void kick_all_cpus_sync(void) { + struct cpumask mask; + /* Make sure the change is visible before we kick the cpus */ smp_mb(); - smp_call_function(do_nothing, NULL, 1); + + preempt_disable(); +#ifdef CONFIG_TASK_ISOLATION + cpumask_clear(&mask); + task_isolation_cpumask(&mask); + cpumask_complement(&mask, &mask); +#else + cpumask_setall(&mask); +#endif + smp_call_function_many(&mask, do_nothing, NULL, 1); + preempt_enable(); } EXPORT_SYMBOL_GPL(kick_all_cpus_sync);