From patchwork Wed Jul 22 14:59:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1333932 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0818 header.b=eYV0jDsB; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=cBfbVZ97; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BBdsy5rqQz9sSt for ; Thu, 23 Jul 2020 00:59:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732829AbgGVO7s (ORCPT ); Wed, 22 Jul 2020 10:59:48 -0400 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:41176 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730465AbgGVO7q (ORCPT ); Wed, 22 Jul 2020 10:59:46 -0400 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 06MEmwsv012225; Wed, 22 Jul 2020 07:59:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0818; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=eYV0jDsBzQD1G4kxZm1E5pOeTkJ1S8Ad2j7gN9NaW5DlODE0N2O9x3fn2JdCVVLRX59j 0WZLZWn3t+h1yPe0sZK6bxRm7dl5Knihp28FLty81WKbUt9KhOtwBKPdAnZGODR7fprW mp8iNp3GeYOY47y7Y+lGBD8InVRIseLvCni5V72uNlTm7LcroYUL0jjMX+OeUI5Y7/Uf yLSrvqWKNrYtgYWW4sxTk2POoFur7KAnnftb7v6//nFxDAMW3bmNwx9rHL8ABZHxkHCS QAduYybfEWCOVf5WVAIecCqA0hMOuWH+sMN0uwhxGljIgGBkjrch6xpXYW9GAlZvFOMo KA== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 32c0kkrbvy-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 22 Jul 2020 07:59:19 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 22 Jul 2020 07:59:14 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.36.56) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 22 Jul 2020 07:59:14 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bBbnrt582VYmuzGJgEvICo8x3wDdb/XXYXEAjzFKtPi/aVt2gZC84jKEisvzahuClypRv1oNA1EppYdW8sWvNNXxvMLKeuY03QR8TpHkTOcml/cntcd3u/ny93JqLf1eVKF5vh3KEIjmFvdflX6RKdmSWciG025ka2QK/TP7GuGCT6ED/IOsDq4fsfdu2V4TuGtC7wjqvCryczxlVmxAERyHRHFPgV5CAdcG8+p1dH0i+7E8Ihy9XyUWSF7PDOS7Nse2sYJrJIITOPdZnVHiS0I6stLiQkwluq5IuHB9GuNOSdNpbABI29vRWg5D2IlrfZujleo0IA/guUZK8Kr+/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=eZ88cXtHsk0vtqbayk6nCmn4pzetG/Lj/Q+UzxXWeZ6qXYocQBvdQtBZjTdT3C1+gc8dWJP8G3ytCBWGPpzv5KQ65ceOmA+P17p/ddW2jTtkHOxVuoMIwGl2fPUT+7fRD5PaOUMYcbjQWkWFofQ6VPRvxZ3EQcUOtgec7x/zb96N0CmlmWO+GcvGghQ73eXl+sQ2MY+Fmuae57FuoDyhg3FZKXPIq6AbT8pob7EzYvliEiO0ve3yMs/FgmRdR9cU4lvFL6zhFN0z/qpS3EdSF/lhDEiaOmGbnXkaoJdu1WdLMoAe7NCaG32TyFerslTGmlXQpSoF5DX7UCpeAvxzdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=10AT4bIlylh+LqvjNLjxzqlnqqFrfd3Q9NSD6y1rGNs=; b=cBfbVZ97VkaCNO+e+b8JebOG6R9xvwzne32IGmQ4Tupld/dmYsoOcU2wv3o49ozCk5kbGqgrshyc8spV3xrzSsr4VwBctBBx5huNJoWQMtctHCXJfVDmDn8GYdMdtj7aIjr8LJ69Uq6Bw8eW3d69NhCpOYSX4oqyA5WTI+GUbyE= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW2PR18MB2156.namprd18.prod.outlook.com (2603:10b6:907:a::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.24; Wed, 22 Jul 2020 14:59:13 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::b9a6:a3f2:2263:dc32%4]) with mapi id 15.20.3195.026; Wed, 22 Jul 2020 14:59:13 +0000 From: Alex Belits To: "frederic@kernel.org" , "rostedt@goodmis.org" CC: Prasun Kapoor , "mingo@kernel.org" , "davem@davemloft.net" , "linux-api@vger.kernel.org" , "peterz@infradead.org" , "linux-arch@vger.kernel.org" , "catalin.marinas@arm.com" , "tglx@linutronix.de" , "will@kernel.org" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "netdev@vger.kernel.org" Subject: [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Topic: [PATCH v4 12/13] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Index: AQHWYDipehJFQ80zLEqG2+MBm9Fm7A== Date: Wed, 22 Jul 2020 14:59:13 +0000 Message-ID: <041a4c844b4a0f69aba6df4686e1952dd0f91931.camel@marvell.com> References: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> In-Reply-To: <04be044c1bcd76b7438b7563edc35383417f12c8.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 586cab22-9cf0-4e35-07d5-08d82e4fcc0e x-ms-traffictypediagnostic: MW2PR18MB2156: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: g/rG6B3bl1C8XptqxTm69bNTncIU8+KFD1NqsYEpBnh+lGDeWHJ5HkSgXL8t1DRY8Hq/We4e1CGeiz58g+TPvq12t+tEyZOoPNnY1zq4QCkD77bcuKUsg4LtMXwkituEhDrpJTVFOd1OP+S6PglUb5USnRQLbCpEzPMIcdRzBx7VWuAiD+gQrwpxkbnq6lgZrtXLvLr9cDab0nD+jiKqdnz7K17rv9dyhUND6BVr583hV7cxZRvwOVqgMU/+5EqxIf6w/5mB0JC0tcl8xddNCGdqaW0nR8R4QWpvPea4dXuKi/wL0hgwo00ew6pIWh4HkLW0sCmvB0Txfx139NWehw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(396003)(136003)(346002)(110136005)(6486002)(2906002)(6506007)(54906003)(66476007)(478600001)(6512007)(8936002)(2616005)(8676002)(316002)(4326008)(66946007)(186003)(64756008)(86362001)(83380400001)(76116006)(91956017)(5660300002)(66446008)(36756003)(66556008)(26005)(71200400001)(7416002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: 6x3a8HY0WvuASH2wmVrBcT7rU/tL/KLtA9sNgal6lPTJ6+CVdmFXwcpsn3gFUyVrEcX/YTvq2NUAt1swKCwiF5qCjxVWFDVV85ncBJ9iPH53ebx53KRBZLdgX7u+p3pujBaSnaEKKdXbFi8xb5xnVADedmiIJsvTU7tNezC7vozNuBB10Z3eyVtQBGWg1SuUb+e9m8FwuJewXTZDCczIyJ4xeNrWkqcsRT+a6DMIUgrjYw/2B/gp2aUfDU0w7Hva6WTKGx1Kr81dmQfk07zxYaxAdIYGVMrmj/1SD0QMxkiimv+xH+aNSu8D9zGoheS9hnYxAeW83oXwo8LVAHMTMCu7MUadQcd+XqtsAoBFn/nQBUwUV7xtOL6lLrYQpvN3WmsBwRnkuJAezcPVT/XLWxw+acyarfTEo+j1bnHpir3y+BciVCGDWiwkPFC48Hhwr1lBDH+J2+e+FoHr/8Gwq0DeW52RgMD7VcaZ8uq/c69oWQppmbcyDIOVTnNkjzYg Content-ID: <5763AE40EEE4674CB00A185A60CF3731@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 586cab22-9cf0-4e35-07d5-08d82e4fcc0e X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2020 14:59:13.2631 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 38yJa+FR9/bJKRNd1klcRODFvVOX0EeSawGWXD2mWwDNzpi6t1WgrU5M7lw8f/1D0f6zo6bU8lOh8LmDs9+HAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR18MB2156 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-07-22_08:2020-07-22,2020-07-22 signatures=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov CPUs running isolated tasks are in userspace, so they don't have to perform ring buffer updates immediately. If ring_buffer_resize() schedules the update on those CPUs, isolation is broken. To prevent that, updates for CPUs running isolated tasks are performed locally, like for offline CPUs. A race condition between this update and isolation breaking is avoided at the cost of disabling per_cpu buffer writing for the time of update when it coincides with isolation breaking. Signed-off-by: Yuri Norov [abelits@marvell.com: updated to prevent race with isolation breaking] Signed-off-by: Alex Belits --- kernel/trace/ring_buffer.c | 63 ++++++++++++++++++++++++++++++++++---- 1 file changed, 57 insertions(+), 6 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 00867ff82412..22d4731f0def 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -1705,6 +1706,38 @@ static void update_pages_handler(struct work_struct *work) complete(&cpu_buffer->update_done); } +static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer, + int cpu) +{ + bool rv = false; + + smp_rmb(); + if (task_isolation_on_cpu(cpu)) { + /* + * CPU is running isolated task. Since it may lose + * isolation and re-enter kernel simultaneously with + * this update, disable recording until it's done. + */ + atomic_inc(&cpu_buffer->record_disabled); + /* Make sure, update is done, and isolation state is current */ + smp_mb(); + if (task_isolation_on_cpu(cpu)) { + /* + * If CPU is still running isolated task, we + * can be sure that breaking isolation will + * happen while recording is disabled, and CPU + * will not touch this buffer until the update + * is done. + */ + rb_update_pages(cpu_buffer); + cpu_buffer->nr_pages_to_update = 0; + rv = true; + } + atomic_dec(&cpu_buffer->record_disabled); + } + return rv; +} + /** * ring_buffer_resize - resize the ring buffer * @buffer: the buffer to resize. @@ -1794,13 +1827,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, if (!cpu_buffer->nr_pages_to_update) continue; - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to + * update ring buffers until they exit + * isolation because they are in + * userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu)) { rb_update_pages(cpu_buffer); cpu_buffer->nr_pages_to_update = 0; } else { - schedule_work_on(cpu, - &cpu_buffer->update_pages_work); + if (!update_if_isolated(cpu_buffer, cpu)) + schedule_work_on(cpu, + &cpu_buffer->update_pages_work); } } @@ -1849,13 +1891,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, get_online_cpus(); - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to update + * ring buffers until they exit isolation because they + * are in userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu_id)) rb_update_pages(cpu_buffer); else { - schedule_work_on(cpu_id, + if (!update_if_isolated(cpu_buffer, cpu_id)) + schedule_work_on(cpu_id, &cpu_buffer->update_pages_work); - wait_for_completion(&cpu_buffer->update_done); + wait_for_completion(&cpu_buffer->update_done); + } } cpu_buffer->nr_pages_to_update = 0;