From patchwork Mon Nov 23 17:58:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Belits X-Patchwork-Id: 1404949 Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=marvell.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=marvell.com header.i=@marvell.com header.a=rsa-sha256 header.s=pfpt0220 header.b=HcUYG0LE; dkim=pass (1024-bit key; unprotected) header.d=marvell.onmicrosoft.com header.i=@marvell.onmicrosoft.com header.a=rsa-sha256 header.s=selector1-marvell-onmicrosoft-com header.b=PDnprjLv; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Cfw0S546fz9sTR for ; Tue, 24 Nov 2020 04:59:52 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390588AbgKWR7V (ORCPT ); Mon, 23 Nov 2020 12:59:21 -0500 Received: from mx0b-0016f401.pphosted.com ([67.231.156.173]:46840 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390562AbgKWR7T (ORCPT ); Mon, 23 Nov 2020 12:59:19 -0500 Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 0ANHsitC021505; Mon, 23 Nov 2020 09:58:38 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=pfpt0220; bh=6sfi8us8khBcVaHLEcbGHrXGjcwBkTMGdxvJ+kVM7BE=; b=HcUYG0LEmGg7MdhLhn5gPrfw+jdPyuE7nJfKJgI7WA1q21mO17ZYUCCi6B9J+/Iyuqxn ILolWu21oJpWRX3dCh//BgjtJmPih8LwnVL/xCH+NFXMF/EG+DxSdxx9GjmBfZURsY0z jeje72ijdXZesOSNFY7YZKajlup9W+7R0tVO+cHV7sL995XZfYfgU3lUS9cGq+1aLmdo IJ6AklFV99CYdjqA/3kDiG5TFpvCrtEcyYpNLn1b7I3YzkUYDVQWu5mxSW+QaxMbusMx IaBLuFBAPNklT8KTBqHdtwNzYTAtfYpGDUwodgdYz/BMCIiRYeA2TPMkZxjCyAioLIX2 9g== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 34y39r6dph-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 23 Nov 2020 09:58:38 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 23 Nov 2020 09:58:37 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 23 Nov 2020 09:58:36 -0800 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.173) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 23 Nov 2020 09:58:35 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=STmRXFTjxsFnD8u8HCEy3tmAg0pz6I7ny2jVLBrZESd9bSnFSHtCk2BrpTYI7TrNrf8kwc1aIctsulSrJnktqqfJSDN2/lkJFvWi9riEHWrOuszvP0G5XrfMm49tfiJRbrHOogK+zCj8GnSGwsLNu3HgYAZOYe+nvOQIigBSArzcCRRjhPxPEUkx5XTkf9JQCa8E7ZoNANOn1+UcLN/lFobFZn07RvMbwkrppz3SYSMAQ7UCj/WB6IZaA8qGZ5WE3pWuNLjVjpNaJ27+rH3WB3Lbt64wMWNe6ZaAOkxuSf+1RVr9R3pgIwvT4BvrClzpiQzmo8CQmrPPJlBOus2YTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6sfi8us8khBcVaHLEcbGHrXGjcwBkTMGdxvJ+kVM7BE=; b=n6hP9jhmEdvaJRPYI7gRhy9yicR+YZDoBqzZLQfWmDvVP0cJ99uAXlNPNlVDrEKBrJjKchRQ/XN5aJJEsBFqWapSMuGY0rv9l8j1Q114wS8lG716LZfs6IrIoCuw/0rFeyaiimdI/dUUTJ/zWTUUU92C0SNN5ZW6TmawIoQ4272TB/7zfKScfY7xzKEigDjFIFRzL9QmPuMxIcl7qJHkjHh8t7Eqc0oPfigLNvhhh7A29BmAMaf4ztLB44RA9sy7aL3hzw5kfTC/7Ex6yrI1+8NdA+A7Zcsn1aLXB4iycYQKvFIXI3vxc5fj2PyoXQbzVMpq8eGtAL5EozCNGGFunw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6sfi8us8khBcVaHLEcbGHrXGjcwBkTMGdxvJ+kVM7BE=; b=PDnprjLvAq/9DlCxJb5IcpuQAQzMy9Duzr6UAV+nKKCKXo9F99QkQOwCxpGzmJ6ZTA899bogCszARhtgLrN3ew571xvqposqUE/wEA3tTWt89LC3uTnm3jXS2QF8NIc1iklM4Be3nV4/KbNz3KwmlQLDtW55/EYe6d781INK+4g= Received: from MW2PR18MB2267.namprd18.prod.outlook.com (2603:10b6:907:3::11) by MW3PR18MB3530.namprd18.prod.outlook.com (2603:10b6:303:55::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3589.22; Mon, 23 Nov 2020 17:58:34 +0000 Received: from MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::e17f:37fb:4eb3:c703]) by MW2PR18MB2267.namprd18.prod.outlook.com ([fe80::e17f:37fb:4eb3:c703%4]) with mapi id 15.20.3564.039; Mon, 23 Nov 2020 17:58:34 +0000 From: Alex Belits To: "nitesh@redhat.com" , "frederic@kernel.org" CC: Prasun Kapoor , "linux-api@vger.kernel.org" , "davem@davemloft.net" , "trix@redhat.com" , "mingo@kernel.org" , "catalin.marinas@arm.com" , "rostedt@goodmis.org" , "linux-kernel@vger.kernel.org" , "peterx@redhat.com" , "tglx@linutronix.de" , "linux-arch@vger.kernel.org" , "mtosatti@redhat.com" , "will@kernel.org" , "peterz@infradead.org" , "leon@sidebranch.com" , "linux-arm-kernel@lists.infradead.org" , "pauld@redhat.com" , "netdev@vger.kernel.org" Subject: [PATCH v5 8/9] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Topic: [PATCH v5 8/9] task_isolation: ringbuffer: don't interrupt CPUs running isolated tasks on buffer resize Thread-Index: AQHWwcJCaZAYvgqyU0uLj2p5N5OAJA== Date: Mon, 23 Nov 2020 17:58:34 +0000 Message-ID: References: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> In-Reply-To: <8d887e59ca713726f4fcb25a316e1e932b02823e.camel@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=marvell.com; x-originating-ip: [173.228.7.197] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0ea533cc-7124-4083-3063-08d88fd965a1 x-ms-traffictypediagnostic: MW3PR18MB3530: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ltrLKCokWctmzGqeKU3Dhh8h22z5mBJzfk6nVTXlGS8z3Yek8fIf+hTdLD6vPwzTRj+172kH/Y+TPmp99CpN01T7I2KEweM4bOLYZFBCQrEn6wS4MVLPgqgCTfr3GCmNAyhB+e+WDte5rmhydS9jJeX2mI2SvP6+ceXLtkz+0upo9xID/I69u0PmasiyG+id7AVuwRnC00dyBEZgNNNRVdso38B0gbYy7quueqdXdQexuKdFBFMocfniBGPWvITnx80MLCh1UukjZeHKLf9ZT+G+D4OSZf/iGElLN0n41L1zybZH1EQapyCSW9tLpucvxbZ9t5Gxr4xjHbDMF2w0jw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW2PR18MB2267.namprd18.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(346002)(376002)(39860400002)(136003)(396003)(2906002)(2616005)(71200400001)(5660300002)(6512007)(83380400001)(110136005)(26005)(54906003)(6486002)(186003)(6506007)(36756003)(66946007)(8676002)(316002)(478600001)(8936002)(86362001)(7416002)(66476007)(66556008)(64756008)(66446008)(91956017)(76116006)(4326008);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: ZlO9Y1FX87Fky2QnkKBlk6HdmUqs+cTdJ3gFccetqIaZcOM/DouXCSwrrsrpncpwA0rsfjxgna4MnKaXkoLE4x4t590yCErrIqbP1Bwa4slLzSmId4Zsc2uOtEZdsjlZ5G1VzHgbl44tID8V3oG3g3pWeIxT3346bwrsD8ZVholJyZgWPStMhKJ+8bJL3ZesmGBMBZ3iwrwuUkOgcjttlH16LVKsF5y8de4Hu8iqJ2U+lv5EphDqYDzoFxyDiftZRRHUy1iWVW0rdVjVeXWmAIiV/qS79B2ZyVtJzjn7k5CiSvAnuaEVJmz4QnDopc+/nGprhY4Ygl/UPswgK17PpRIvbv2yyR0WrJzyEckZpFIZKYnjHzYKzd4JBqe7x4QOxOwFLCFBb8uLYkuBLZPT6OzdYkkg5sJ0iebAiNKtrKdYSW2hMey/6brzRbQnAHyUVmMqy+lMOnDquaZIgmNNkYWe4MzHQQcy/3yh1FbcRmTpgcfberkdTof18Ilb+w35CeSn9BbJnrxjxfBeUkmoLKpcnWwZQCNbz2UfnjkSPtGeAL0Ef/NA5OWc3/cXdNMdZPV0n+4pEiB4DueETNyJoqhGK/XeG4tXLVmiFIJdtIbw94S5B0+gxvh91INWUMU5HwKZhzOUqZ9RQd3m4FsxFDZSJhp832WgoTrdz9s8r0zFUflrz0I08EWHExPDd32xDAFvAr6R9PnxdNlratTfJA6b+yFmieoS1f7GwL75d0PfenFjztKdaYROH6XpOgKy346XFexPbHKzCH5D+95ed/9bYViqkxs6nZBg6nPOtFBKaJbMppSqKRsiUlv5nqSmZorQmQe0rF9XEd8OEUY27imoldJkvyfsiNrwN16ua1VdiW8sKsScufLtTI37Kl9gbWvO9id418Twf1njOaSazQ== Content-ID: <79CB53C96CF18B418B712DFEDA3F74B1@namprd18.prod.outlook.com> MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR18MB2267.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0ea533cc-7124-4083-3063-08d88fd965a1 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Nov 2020 17:58:34.6903 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DLwRSqj8L0DPkKrUz7rkl/wZBvMJoC4tkggCKOQfSP8Ut4BW27alTAxLq6ubKqD9U//VvRtazEj7CaXWp1pNmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR18MB3530 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312,18.0.737 definitions=2020-11-23_17:2020-11-23,2020-11-23 signatures=0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Yuri Norov CPUs running isolated tasks are in userspace, so they don't have to perform ring buffer updates immediately. If ring_buffer_resize() schedules the update on those CPUs, isolation is broken. To prevent that, updates for CPUs running isolated tasks are performed locally, like for offline CPUs. A race condition between this update and isolation breaking is avoided at the cost of disabling per_cpu buffer writing for the time of update when it coincides with isolation breaking. Signed-off-by: Yuri Norov [abelits@marvell.com: updated to prevent race with isolation breaking] Signed-off-by: Alex Belits --- kernel/trace/ring_buffer.c | 63 ++++++++++++++++++++++++++++++++++---- 1 file changed, 57 insertions(+), 6 deletions(-) diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index dc83b3fa9fe7..9e4fb3ed2af0 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include @@ -1939,6 +1940,38 @@ static void update_pages_handler(struct work_struct *work) complete(&cpu_buffer->update_done); } +static bool update_if_isolated(struct ring_buffer_per_cpu *cpu_buffer, + int cpu) +{ + bool rv = false; + + smp_rmb(); + if (task_isolation_on_cpu(cpu)) { + /* + * CPU is running isolated task. Since it may lose + * isolation and re-enter kernel simultaneously with + * this update, disable recording until it's done. + */ + atomic_inc(&cpu_buffer->record_disabled); + /* Make sure, update is done, and isolation state is current */ + smp_mb(); + if (task_isolation_on_cpu(cpu)) { + /* + * If CPU is still running isolated task, we + * can be sure that breaking isolation will + * happen while recording is disabled, and CPU + * will not touch this buffer until the update + * is done. + */ + rb_update_pages(cpu_buffer); + cpu_buffer->nr_pages_to_update = 0; + rv = true; + } + atomic_dec(&cpu_buffer->record_disabled); + } + return rv; +} + /** * ring_buffer_resize - resize the ring buffer * @buffer: the buffer to resize. @@ -2028,13 +2061,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, if (!cpu_buffer->nr_pages_to_update) continue; - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to + * update ring buffers until they exit + * isolation because they are in + * userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu)) { rb_update_pages(cpu_buffer); cpu_buffer->nr_pages_to_update = 0; } else { - schedule_work_on(cpu, - &cpu_buffer->update_pages_work); + if (!update_if_isolated(cpu_buffer, cpu)) + schedule_work_on(cpu, + &cpu_buffer->update_pages_work); } } @@ -2083,13 +2125,22 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size, get_online_cpus(); - /* Can't run something on an offline CPU. */ + /* + * Can't run something on an offline CPU. + * + * CPUs running isolated tasks don't have to update + * ring buffers until they exit isolation because they + * are in userspace. Use the procedure that prevents + * race condition with isolation breaking. + */ if (!cpu_online(cpu_id)) rb_update_pages(cpu_buffer); else { - schedule_work_on(cpu_id, + if (!update_if_isolated(cpu_buffer, cpu_id)) + schedule_work_on(cpu_id, &cpu_buffer->update_pages_work); - wait_for_completion(&cpu_buffer->update_done); + wait_for_completion(&cpu_buffer->update_done); + } } cpu_buffer->nr_pages_to_update = 0;