From patchwork Wed Jul 22 14:51:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333924 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=I4ayta1S; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=qTlaKXDP; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdjb6TGfz9sSy for ; Thu, 23 Jul 2020 00:52:35 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732730AbgGVOwc (ORCPT ); Wed, 22 Jul 2020 10:52:32 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:39462 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729642AbgGVOw3 (ORCPT ); Wed, 22 Jul 2020 10:52:29 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEnMNB012667; Wed, 22 Jul 2020 07:51:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=I4ayta1SZklGiVgd72KpH7+YDgOMADElXrbJZzNfDfEzURG4N/9ylGST9C6XkP0zuKQu ys9BBxqX//isY+c0MFbitf0KZaARmNTeaxnGb+sqrKygmxvrR6c0bC0WSRdruIHaOj3O nK6LuqSAhKTQhr1QdPIBGxSnALyxASvlUWSGep4WHDBmeCZNc3Ii8A5N9IkWU2xqIvXI iXfQ/0dIWrGM8NLpjbFvT0+cO5JqY0sOBmNWdGpmANfz68RZRp51W3ULRWVNLLMnecCW NsjS7yjix1jsHZCK0g4MLdpf0o1axmyOI/BDhR+x+5LUOsiMmwNU5K6T/9cXXHKgU++w 5w== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkravq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:51:08 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:07 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:51:05 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.176) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:51:05 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gCyHnXcCoW6KWHxeUvUey/ghlqp10wKzjLxpF+Esw1vt5/IV+RZH2z5ObsdMmq5ALv+bGRqLSIWrrB9etwq/oKLWIghJEB/yC080/lz0mKveKrZAg3hvQHLM1jucI6NI7PHQqrNb3Nb8DYF3If+0oCs75nysQVrg6L74U9KvTS7JosLyhp+tIpfcnDuu7hk0Muo5oTxA8aFlOAWK68sh5xUy0KBHgJz1oy4p/dy4jbEeIkzWNWePKP68ywu8PdeCTXTp1kL4RkCWzM46jvcfWdFGyf1t5zSQVYxMh/SfiYvWmf12P/rpI86FvkXWbmMSpv5RRNpBPB6HksLXaFpRHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=HKj1IrPKPVbyJN6+KTlscqbNWEIwNeBjr6awFbwbg8Vo5EovhzpfWGcFc6GQPI+uirnWmrYbPtmD8oKY7roYLbqRTfrICShFi5fueEMDQ1zHUSC0GXRUShviXlTh71mWhoHcHEIVUiXjlJ9f0WvkS7eRME2WfuHMW051UpPnL/wckMJoqBvNcxtLtiRkUFgPhMteb7EdUHsPY57udQJudKVh/Mn8Jw86vzATZeYRjJ+j4zG1sXULtDSIfJkkOG2+iZeqLewvFBicGocgHWyMKhimrEyL2QsHxjHywTo5Ft3CO0OuFvAA423LiknKngY6OGnGkkrMeyX96Rwgvyjg3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p7IOApfb9E5rDOR9WAE0Oz3ez6pevSjLvn8UPz8+XrU=; b=qTlaKXDPxBNm28mIgmtS8sJ8MhKcJiW8bugYNQ2dOS7nG/3AuKRRou57wL9iCJkHmrxwP13ZbjgBmKrauAPq7vNKidr9RDa2nkrncg0IyRjp7hHTgAIZbCyRICnXJItWu5WIJUSKWntY+Sg6ph1D2yh9qDKBBE8ddJg5hWjAAas= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MWHPR18MB1279.namprd18.prod.outlook.com (2603:10b6:320:2c::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.20; Wed, 22 Jul 2020 14:51:04 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:51:04 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 04/13] task_isolation: Add task isolation hooks to arch-independent code Thread-Topic: [PATCH v4 04/13] task_isolation: Add task isolation hooks to arch-independent code Thread-Index: AQHWYDeGpu5YVSQ/OESYEPtA2Bs+XQ== Date: Wed, 22 Jul 2020 14:51:04 +0000 Message-ID: <5f7a1619df6139e53d744b3755838bbe83ffa96c.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d53a6067-3b56-4d84-05d2-08d82e4ea8ae x-ms-traffictypediagnostic: MWHPR18MB1279: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:1002; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: EaB1g3rbA0Kci7IxxhFnjB7vrsNaPo3pEDDmP3vMjDT69Ma8HYum96Ic3CfwS0YG08If61TX3iiZPBmaE8DdZlLGnAoj/nxWLDgBFpvRzVlZpPWHO9rUW6MfTbW6QZ27E2BpzdECklZNRmpjsSddjnSh1R0LVAA3srmXVEdyaeKbsuZnOZrB+nbxzTy5cdKZl4s3ygZ1i4eqRUvjU9Qxp5n4iV/qMjYoxoGzs67jpk958wcN2FoUeozTyNolLlKILqCbqk0/QtHCcCu3DyHggzBJK6ZxVk2IOIThPYl+nNZn3/lZ9PzrhU/njJIk1xdw x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(396003)(366004)(39860400002)(376002)(346002)(136003)(54906003)(26005)(316002)(110136005)(2616005)(71200400001)(8936002)(7416002)(86362001)(2906002)(5660300002)(4326008)(64756008)(66556008)(66446008)(6486002)(478600001)(36756003)(6512007)(8676002)(186003)(76116006)(91956017)(66946007)(83380400001)(66476007)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: ZDyeaytwuWkQ/kE9j+lzRTQxbZPYRLjO3syBVEBFFg8c7VrKjS+jqxzOvXZ7ndVr017G/cZZtz8aqFsdeeWeujZu2JKZtxuVyeH1eyoW7+g9jZEGtFPhHooskBTJD/Cd9WHLBBjbmuf2Vwu3kVHvRfIOzQj3kSWPanpTQuE2wUC5D5f22fm2y/9tf3NvL/LybiEh7GfMUvEAVwuO01AlWc6blKEBjynEQZzdAW5SGmNUHkqYzJRhCOSyGBi1GeJ/jqKnHZGsUo8keQiqj3haJkTHUl5KB6mRJkZ64MOn+e9m/DxeE18cXXOyzO091ZcOODW3XKvzKFZTp7CrRm9ToVLqiWd6VdCZbj85ETAC2S1m2C8XW8j6hVqd2qQm44F5kqW/499tbWH0G4x/9pftW/W5wwIz3W978tBEqrXYLobzo7y13oho/jetwjU3PEpzYqnP0Nk93Oe+oBHJC+fPML5ScD1kO/jrHsheAUk6E38= Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d53a6067-3b56-4d84-05d2-08d82e4ea8ae X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:51:04.3819 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 48b0tvIcRa83vTx3TbJRkdTtcEyKOVD3MRLfsKDoMmhF66aBDlEBx6H+mnlBqafi0pkqVjgTfOXYI5EkZwQydg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR18MB1279 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit adds task isolation hooks as follows: - __handle_domain_irq() and handle_domain_nmi() generate an isolation warning for the local task - irq_work_queue_on() generates an isolation warning for the remote task being interrupted for irq_work (through __smp_call_single_queue()) - generic_exec_single() generates a remote isolation warning for the remote cpu being IPI'd (through __smp_call_single_queue()) - smp_call_function_many() generates a remote isolation warning for the set of remote cpus being IPI'd (through smp_call_function_many_cond()) - on_each_cpu_cond_mask() generates a remote isolation warning for the set of remote cpus being IPI'd (through smp_call_function_many_cond()) - __ttwu_queue_wakelist() generates a remote isolation warning for the remote cpu being IPI'd (through __smp_call_single_queue()) - nmi_enter(), __context_tracking_exit(), __handle_domain_irq(), handle_domain_nmi() and scheduler_ipi() clear low-level flags and synchronize CPUs by calling task_isolation_kernel_enter() Calls to task_isolation_remote() or task_isolation_interrupt() can be placed in the platform-independent code like this when doing so results in fewer lines of code changes, as for example is true of the users of the arch_send_call_function_*() APIs. Or, they can be placed in the per-architecture code when there are many callers, as for example is true of the smp_send_reschedule() call. A further cleanup might be to create an intermediate layer, so that for example smp_send_reschedule() is a single generic function that just calls arch_smp_send_reschedule(), allowing generic code to be called every time smp_send_reschedule() is invoked. But for now, we just update either callers or callees as makes most sense. Calls to task_isolation_kernel_enter() are intended for early kernel entry code. They may be called in platform-independent or platform-specific code. It may be possible to clean up low-level entry code and somehow organize calls to task_isolation_kernel_enter() to avoid multiple per-architecture or driver-specific calls to it. RCU initialization may be a good reference point for those places in kernel (task_isolation_kernel_enter() should precede it), however right now it is not unified between architectures. Signed-off-by: Chris Metcalf [abelits@marvell.com: adapted for kernel 5.8, added low-level flags handling] Signed-off-by: Alex Belits --- include/linux/hardirq.h | 2 ++ include/linux/sched.h | 2 ++ kernel/context_tracking.c | 4 ++++ kernel/irq/irqdesc.c | 13 +++++++++++++ kernel/smp.c | 6 +++++- 5 files changed, 26 insertions(+), 1 deletion(-) diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index 03c9fece7d43..5aab1d0a580e 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -7,6 +7,7 @@ #include #include #include +#include #include extern void synchronize_irq(unsigned int irq); @@ -114,6 +115,7 @@ extern void rcu_nmi_exit(void); #define nmi_enter() \ do { \ arch_nmi_enter(); \ + task_isolation_kernel_enter(); \ printk_nmi_enter(); \ lockdep_off(); \ BUG_ON(in_nmi() == NMI_MASK); \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 7fb7bb3fddaa..cacfa415dc59 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -32,6 +32,7 @@ #include #include #include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -1743,6 +1744,7 @@ extern char *__get_task_comm(char *to, size_t len, struct task_struct *tsk); #ifdef CONFIG_SMP static __always_inline void scheduler_ipi(void) { + task_isolation_kernel_enter(); /* * Fold TIF_NEED_RESCHED into the preempt_count; anybody setting * TIF_NEED_RESCHED remotely (for the first time) will also send diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c index 36a98c48aedc..481a722ddbce 100644 --- a/kernel/context_tracking.c +++ b/kernel/context_tracking.c @@ -21,6 +21,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -148,6 +149,8 @@ void noinstr __context_tracking_exit(enum ctx_state state) if (!context_tracking_recursion_enter()) return; + task_isolation_kernel_enter(); + if (__this_cpu_read(context_tracking.state) == state) { if (__this_cpu_read(context_tracking.active)) { /* @@ -159,6 +162,7 @@ void noinstr __context_tracking_exit(enum ctx_state state) instrumentation_begin(); vtime_user_exit(current); trace_user_exit(0); + task_isolation_user_exit(); instrumentation_end(); } } diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c index 1a7723604399..b351aac7732f 100644 --- a/kernel/irq/irqdesc.c +++ b/kernel/irq/irqdesc.c @@ -16,6 +16,7 @@ #include #include #include +#include #include "internals.h" @@ -669,6 +670,8 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq, unsigned int irq = hwirq; int ret = 0; + task_isolation_kernel_enter(); + irq_enter(); #ifdef CONFIG_IRQ_DOMAIN @@ -676,6 +679,10 @@ int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq, irq = irq_find_mapping(domain, hwirq); #endif + task_isolation_interrupt((irq == hwirq) ? + "irq %d (%s)" : "irq %d (%s hwirq %d)", + irq, domain ? domain->name : "", hwirq); + /* * Some hardware gives randomly wrong interrupts. Rather * than crashing, do something sensible. @@ -710,6 +717,8 @@ int handle_domain_nmi(struct irq_domain *domain, unsigned int hwirq, unsigned int irq; int ret = 0; + task_isolation_kernel_enter(); + /* * NMI context needs to be setup earlier in order to deal with tracing. */ @@ -717,6 +726,10 @@ int handle_domain_nmi(struct irq_domain *domain, unsigned int hwirq, irq = irq_find_mapping(domain, hwirq); + task_isolation_interrupt((irq == hwirq) ? + "NMI irq %d (%s)" : "NMI irq %d (%s hwirq %d)", + irq, domain ? domain->name : "", hwirq); + /* * ack_bad_irq is not NMI-safe, just report * an invalid interrupt. diff --git a/kernel/smp.c b/kernel/smp.c index aa17eedff5be..6a6849783948 100644 --- a/kernel/smp.c +++ b/kernel/smp.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "smpboot.h" #include "sched/smp.h" @@ -146,8 +147,10 @@ void __smp_call_single_queue(int cpu, struct llist_node *node) * locking and barrier primitives. Generic code isn't really * equipped to do the right thing... */ - if (llist_add(node, &per_cpu(call_single_queue, cpu))) + if (llist_add(node, &per_cpu(call_single_queue, cpu))) { + task_isolation_remote(cpu, "IPI function"); send_call_function_single_ipi(cpu); + } } /* @@ -545,6 +548,7 @@ static void smp_call_function_many_cond(const struct cpumask *mask, } /* Send a message to all CPUs in the map */ + task_isolation_remote_cpumask(cfd->cpumask_ipi, "IPI function"); arch_send_call_function_ipi_mask(cfd->cpumask_ipi); if (wait) {