From patchwork Thu Feb 29 11:38:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhihao Cheng X-Patchwork-Id: 1906280 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=v3tf=kg=vger.kernel.org=linux-ext4+bounces-1429-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TlqDv1wYcz1yX7 for ; Thu, 29 Feb 2024 22:46:23 +1100 (AEDT) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4TlqDv1RPrz4wZx for ; Thu, 29 Feb 2024 22:46:23 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4TlqDv1KHfz4wbh; Thu, 29 Feb 2024 22:46:23 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=fail (p=quarantine dis=none) header.from=huawei.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-1429-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4TlqDt3mByz4wZx for ; Thu, 29 Feb 2024 22:46:22 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 59C591F24B74 for ; Thu, 29 Feb 2024 11:46:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C8BFD7CF29; Thu, 29 Feb 2024 11:45:37 +0000 (UTC) X-Original-To: linux-ext4@vger.kernel.org Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7520C7CF17; Thu, 29 Feb 2024 11:45:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.190 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709207137; cv=none; b=pX4pbyvOOHBwZ4ugfJOY3sz9A+aN6c9U+TxlsvH+HvmJ4RXxqXsPBLJW06jovlFOul1HPmDicMDoRJp4TGZLQkgBhTGx+GjvRqkDL+AjShDc3p/isMK6K1o3r1nNFJl49ifELjYkTmP1pL3jmPi3pA8V2pWu1YuMRgNAhEwTTWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709207137; c=relaxed/simple; bh=Kq0TK4bY8t/Mvm9W8afZQDPlQRDXrI1jGg+Gg966bnE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=CNM2OKu2aGWYN1OFmvrLuxpKx/nQpMjIZboz3/Dyp8pLEsq4y/dEzn4rYQrbj4d09WBmjgSa/XQkKQ6ZssDMsExrpJfP2a0sAoXH8AeNlZG51C75OE6hqlDPEdj7utdfD8q442vAWYBIaUGq/MPaDN0U4piVbbE+WglERRqpOfE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=45.249.212.190 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4TlqB83zg7z1xpTm; Thu, 29 Feb 2024 19:44:00 +0800 (CST) Received: from kwepemm600013.china.huawei.com (unknown [7.193.23.68]) by mail.maildlp.com (Postfix) with ESMTPS id CB0B71402CE; Thu, 29 Feb 2024 19:45:31 +0800 (CST) Received: from huawei.com (10.175.104.67) by kwepemm600013.china.huawei.com (7.193.23.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 29 Feb 2024 19:45:30 +0800 From: Zhihao Cheng To: , , , , CC: , , , , Subject: [PATCH RFC 1/2] iomap: Add a IOMAP_DIO_MAY_INLINE_COMP flag Date: Thu, 29 Feb 2024 19:38:48 +0800 Message-ID: <20240229113849.2222577-2-chengzhihao1@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240229113849.2222577-1-chengzhihao1@huawei.com> References: <20240229113849.2222577-1-chengzhihao1@huawei.com> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600013.china.huawei.com (7.193.23.68) It will be more efficient to execute quick endio process(eg. non-sync overwriting case) under irq process rather than starting a worker to do it. Add a flag to control DIO to be finished inline(under irq context), which can be used for non-sync overwriting case. Besides, skip invalidating pages if DIO is finished inline, which will keep the same logic with dio_bio_end_aio in non-sync overwriting case. Signed-off-by: Zhihao Cheng --- fs/iomap/direct-io.c | 10 ++++++++-- include/linux/iomap.h | 6 ++++++ 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index bcd3f8cf5ea4..221715b38ce2 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -110,7 +110,8 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio) * ->end_io() when necessary, otherwise a racing buffer read would cache * zeros from unwritten extents. */ - if (!dio->error && dio->size && (dio->flags & IOMAP_DIO_WRITE)) + if (!dio->error && dio->size && (dio->flags & IOMAP_DIO_WRITE) && + !(dio->flags & IOMAP_DIO_INLINE_COMP)) kiocb_invalidate_post_direct_write(iocb, dio->size); inode_dio_end(file_inode(iocb->ki_filp)); @@ -122,8 +123,10 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio) * If this is a DSYNC write, make sure we push it to stable * storage now that we've written data. */ - if (dio->flags & IOMAP_DIO_NEED_SYNC) + if (dio->flags & IOMAP_DIO_NEED_SYNC) { + WARN_ON_ONCE(dio->flags & IOMAP_DIO_INLINE_COMP); ret = generic_write_sync(iocb, ret); + } if (ret > 0) ret += dio->done_before; } @@ -628,6 +631,9 @@ __iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, */ if (!(iocb->ki_flags & IOCB_SYNC)) dio->flags |= IOMAP_DIO_WRITE_THROUGH; + } else if (dio_flags & IOMAP_DIO_MAY_INLINE_COMP) { + /* writes could complete inline */ + dio->flags |= IOMAP_DIO_INLINE_COMP; } /* diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 96dd0acbba44..f292b10028d0 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -382,6 +382,12 @@ struct iomap_dio_ops { */ #define IOMAP_DIO_PARTIAL (1 << 2) +/* + * DIO will be completed inline unless sync operation is needed after io is + * finished. + */ +#define IOMAP_DIO_MAY_INLINE_COMP (1 << 3) + ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, const struct iomap_ops *ops, const struct iomap_dio_ops *dops, unsigned int dio_flags, void *private, size_t done_before); From patchwork Thu Feb 29 11:38:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhihao Cheng X-Patchwork-Id: 1906279 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=gandalf.ozlabs.org; envelope-from=srs0=r9su=kg=vger.kernel.org=linux-ext4+bounces-1430-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from gandalf.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TlqDt2f9kz1yX7 for ; Thu, 29 Feb 2024 22:46:22 +1100 (AEDT) Received: from gandalf.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4TlqDt27Bkz4wZx for ; Thu, 29 Feb 2024 22:46:22 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4TlqDt242xz4wbh; Thu, 29 Feb 2024 22:46:22 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; dmarc=fail (p=quarantine dis=none) header.from=huawei.com Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-1430-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4TlqDr64XGz4wZx for ; Thu, 29 Feb 2024 22:46:20 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 98E74288F27 for ; Thu, 29 Feb 2024 11:46:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CFA647D090; Thu, 29 Feb 2024 11:45:37 +0000 (UTC) X-Original-To: linux-ext4@vger.kernel.org Received: from szxga07-in.huawei.com (szxga07-in.huawei.com [45.249.212.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 206917C6D2; Thu, 29 Feb 2024 11:45:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.35 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709207137; cv=none; b=FkwW9lJP2IkkEs2JpH/KlataUMMQuMAxX5At28vnfOjXvdTF95uIv5dA4+1l9vUVzU/86h3DApvlNzmy+z5eC181oB5hQLF3nnBNdmcDzMmjwu/09iuPEizAQWJ0TQJDU0AmpW+HCa+p2LcUeY+Pc3Xvf1CVIhrSdK28Nv0c4XY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709207137; c=relaxed/simple; bh=jsKCtJRRKaJUiyg4hQpnl4lXKjzxpQzJM96ZLAswM1U=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nTJk4a7ox/Ysm5+VTRp66E6CKzWiemB1swaS/Comtsy6GX8qN7jhGJNTsTGBDAIBsGvm1ASv7MaG3wyqth0mLE+yKSt3kuPJMJQYDFVF9OnHYQGV0DHhuyhP32TdWHp8ZMvFQmQKP7FKYEhlTv7X6mG9rP6JO6qi7QfUKb0mtsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=45.249.212.35 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.19.88.234]) by szxga07-in.huawei.com (SkyGuard) with ESMTP id 4Tlq9J2Zkdz1Q9Mp; Thu, 29 Feb 2024 19:43:16 +0800 (CST) Received: from kwepemm600013.china.huawei.com (unknown [7.193.23.68]) by mail.maildlp.com (Postfix) with ESMTPS id 9670D140390; Thu, 29 Feb 2024 19:45:32 +0800 (CST) Received: from huawei.com (10.175.104.67) by kwepemm600013.china.huawei.com (7.193.23.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 29 Feb 2024 19:45:31 +0800 From: Zhihao Cheng To: , , , , CC: , , , , Subject: [PATCH RFC 2/2] ext4: Optimize endio process for DIO overwrites Date: Thu, 29 Feb 2024 19:38:49 +0800 Message-ID: <20240229113849.2222577-3-chengzhihao1@huawei.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240229113849.2222577-1-chengzhihao1@huawei.com> References: <20240229113849.2222577-1-chengzhihao1@huawei.com> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600013.china.huawei.com (7.193.23.68) In DIO overwriting case, there is no need to convert unwritten exntents and ext4_handle_inode_extension() can be ignored, which means that endio process can be executed under irq context. Since commit 240930fb7e6b5 ("ext4: dio take shared inode lock when overwriting preallocated blocks") has provided a method to judge whether overwriting is happening, just do nothing in endio process if DIO overwriting happens. This patch enables ext4 processing endio under irq context in DIO overwriting case, which brings a performance improvement in the following fio test on a x86 physical machine with nvme when irq and fio run on the same cpu: Test: fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k -size=2G -numjobs=1 -overwrite=1 -time_based -runtime=60 -group_reporting -filename=/test/test -name=Rand_write_Testing --cpus_allowed=1 before: 953 MiB/s after: 1350 MiB/s, ~41% perf improvement. Suggested-by: Zhang Yi Signed-off-by: Zhihao Cheng --- fs/ext4/file.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 54d6ff22585c..411a05c6b96e 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -503,6 +503,7 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) loff_t offset = iocb->ki_pos; size_t count = iov_iter_count(from); const struct iomap_ops *iomap_ops = &ext4_iomap_ops; + const struct iomap_dio_ops *iomap_dops = &ext4_dio_write_ops; bool extend = false, unwritten = false; bool ilock_shared = true; int dio_flags = 0; @@ -572,9 +573,12 @@ static ssize_t ext4_dio_write_iter(struct kiocb *iocb, struct iov_iter *from) ext4_journal_stop(handle); } - if (ilock_shared && !unwritten) + if (ilock_shared && !unwritten) { iomap_ops = &ext4_iomap_overwrite_ops; - ret = iomap_dio_rw(iocb, from, iomap_ops, &ext4_dio_write_ops, + iomap_dops = NULL; + dio_flags = IOMAP_DIO_MAY_INLINE_COMP; + } + ret = iomap_dio_rw(iocb, from, iomap_ops, iomap_dops, dio_flags, NULL, 0); if (ret == -ENOTBLK) ret = 0;