From patchwork Mon Nov 11 23:37:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009963 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=1XtdPxEX; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=nbir=sg=vger.kernel.org=linux-ext4+bounces-5045-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRCz27wMz1xyG for ; Tue, 12 Nov 2024 10:51:59 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRCw3GMdz4w2R for ; Tue, 12 Nov 2024 10:51:56 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRCw3BxGz4x2J; Tue, 12 Nov 2024 10:51:56 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:40f1:3f00::1" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369113; cv=pass; b=W/5QHJ8iwZS/0qz04IEefEyw1FkX8k4KbVX4HPOZ89uVFen053z2TNDrhj0LGsnOpnhuZiwxhcaBQWEbMBUqeTaUnk+zMHwaSU1gg8Qny2a4whbSn+Jc5fMIFYuNpFxd6DD9FoUcnvodmjSyRJ4AWAO6tt5bHhlfbcS4w4Ohb0IIPLXjw9RM420yO+1D8RblMqWWcfugkrdp8GpA9zJYAVP/YHhSbR6d9ka/Rr/oAX6haTPHmnxaJj17IaBmooOlH0hfa2KeWeQx/Rj+x5VbGmPV6iqQ/WwueF09KUaY3cHds+4V+vQLKwU46RYARPJo5BgIhbiCGCM5ckQxYRKE8w== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369113; c=relaxed/relaxed; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b6Y7HpA+X3ZE5ecl7uPiUaAtlmXszbkYtmZpwrTLWp2pRtKCSYjwiDwbpMBS/2rHh1HiEKJ/2NqWnaeULf6QTPxkQTqSf4cZcpOal4LzqHReGth/5M1CkHh6CbaohYwTBR70m3wimYzJPIY6zokga6pqBy9YnpdEtmy7bspDyELgPQ/CMz53bXQibO7Rn9Ct4f1opM9IG9zTx9XbJjKjRfwfdMSCs8W5tb3Eo0lNSSSdQQv5zveOY9RUJP/q+cXOl3tpuN/lL4d+xtmMYlXo0D8ykIZgnr51EA593Kog3Yr4zHlQ6H8R/Z9VHtmsBHpYYI9KvJU5pjlaxtD/tjhxsw== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=1XtdPxEX; dkim-atps=neutral; spf=pass (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5045-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=1XtdPxEX; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5045-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRCs1V54z4w2R for ; Tue, 12 Nov 2024 10:51:53 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 91E3DB2AF03 for ; Mon, 11 Nov 2024 23:49:13 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C2BC41C1ACB; Mon, 11 Nov 2024 23:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="1XtdPxEX" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFC411BD032 for ; Mon, 11 Nov 2024 23:48:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368930; cv=none; b=SiSpTWy1u9P+Qk7NDTEJQaYIdHvkmYiigYR7k2fstnAOFcpZIBV2fZ6zKB1UL3+ARKh9L5lAHQaEXWUhsJzSL1zG/8RVPPasrQUdQkzIeqlPqSATBaPXf6S3fKIzB4MO7TO0/ehjOeqU+2oktK1Zc7WtiLi2yOTQ5WDi1ZuaZi4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368930; c=relaxed/simple; bh=8sfKwyActgC+UFgrCv8AbW1Vz+K4ykiNjF4V6I33E+c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nfqqlluMGWYMLY0pd8u2v2gVO3Nj2tp4hO9135wp1UigMBKyHCGspgBnCseNy8Pgx63QFHCEI+FhNeTzNTU+xn6bCxQyeMNQMdVsZl1h2f99u4jQ/v+IsDgpxvmFE7ped37hdzJXPnYgZi8N9JAPvH0EO1whVctddr9YnAHw3mE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=1XtdPxEX; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-71e4c2e36daso3638432b3a.0 for ; Mon, 11 Nov 2024 15:48:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368928; x=1731973728; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=1XtdPxEXJTmPsu32YnbH6YQeaTMAo+Ak38XEsETxOrV8Npg7jWx8/6GSBlZkGNTPgG 7DqJMZaJC1NtZLHip/no8SpalHLNny28xCpwp1BgXLGqth0JzVhKgbhw7YN9Bq3yTlQP pyM8K/Yugg/Ii055VtI2mAnFh0JoCX97OdIUn3E1xfnnbPuIBwq6k2PHa2UN8l/EGHM1 VUjlmNAciQxi2SKh0TfDSzWVi9L0P8wt1zx/nduENXrEQfiE0AbHwwL2uP8/P+/pXmew mOozAjra/akldPSVosyR+lmYBuC5GsqK6THkwaNA/mjw6477NUBP/WuuwINJclnK5aaU 0Z2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368928; x=1731973728; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QzX2IvmAROGBANn6HEgih/p/Joy2MXPhit7EzvqeXZU=; b=UKUBRzhhTIM2sfSUE5LK9tbTcDnkV4121bJIus6a2N9V0oRPfZjPk7xekVYqaIA625 LA1+C95Yncb+H9QhFyL24J/xjm2e1idkePUey/KOlqGNMTT2zRHwa0NBHuIUhDqWbC5V C600O1wmLNSp802qrVZvBqKpSUUM9bmMZtZIZcDwaE5gaJy2bpy+i6t4RNEWZu+CbyE/ T5lv3ZiMLggagUg2oT5EsbLCeua6ZaSp4BS1k8LF+UCoN3VzDYg0I1YvbCfqOZ/cqnto vy7GOgXaepWqAm8d0wYgb3IVeAuBy4ImgYOItO8Oxl2vmBDmuvOFxyEM/dEQKEzuXByH zQgQ== X-Forwarded-Encrypted: i=1; AJvYcCVDotZ9vXGmU9sOijvmrFOuaH3OHnUfCziL0Y4bCrixUwHxwhi1ct+8NyKWsUV6rvGU8mAArFssVCPB@vger.kernel.org X-Gm-Message-State: AOJu0YxWk7iLshaBKDHcFv9mSqtjB7xUm3qoKRH6TdXMzhzXi5GLMv0X cpiJLeYbpPxfTSS7C4e54CsBbJJQmudCaD8drY9PO1Qsw2TfRFF/ddUmBBt/Ci4ZVZaxjDL/i9D QlUY= X-Google-Smtp-Source: AGHT+IFPjohwvXk7XeJJhxpW4zvvN4HHe41fbVtZtFFrzITrw1kyHBpDFfhvHRsSHEkN13Wz8BpfRA== X-Received: by 2002:a05:6a20:12ce:b0:1d9:15b2:83e with SMTP id adf61e73a8af0-1dc23322093mr23031517637.7.1731368928327; Mon, 11 Nov 2024 15:48:48 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:47 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 01/16] mm/filemap: change filemap_create_folio() to take a struct kiocb Date: Mon, 11 Nov 2024 16:37:28 -0700 Message-ID: <20241111234842.2024180-2-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Rather than pass in both the file and position directly from the kiocb, just take a struct kiocb instead. While doing so, move the ki_flags checking into filemap_create_folio() as well. In preparation for actually needing the kiocb in the function. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 56fa431c52af..91974308e9bf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2460,15 +2460,17 @@ static int filemap_update_page(struct kiocb *iocb, return error; } -static int filemap_create_folio(struct file *file, - struct address_space *mapping, loff_t pos, - struct folio_batch *fbatch) +static int filemap_create_folio(struct kiocb *iocb, + struct address_space *mapping, struct folio_batch *fbatch) { struct folio *folio; int error; unsigned int min_order = mapping_min_folio_order(mapping); pgoff_t index; + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; @@ -2487,7 +2489,7 @@ static int filemap_create_folio(struct file *file, * well to keep locking rules simple. */ filemap_invalidate_lock_shared(mapping); - index = (pos >> (PAGE_SHIFT + min_order)) << min_order; + index = (iocb->ki_pos >> (PAGE_SHIFT + min_order)) << min_order; error = filemap_add_folio(mapping, folio, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); if (error == -EEXIST) @@ -2495,7 +2497,8 @@ static int filemap_create_folio(struct file *file, if (error) goto error; - error = filemap_read_folio(file, mapping->a_ops->read_folio, folio); + error = filemap_read_folio(iocb->ki_filp, mapping->a_ops->read_folio, + folio); if (error) goto error; @@ -2551,9 +2554,7 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); } if (!folio_batch_count(fbatch)) { - if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) - return -EAGAIN; - err = filemap_create_folio(filp, mapping, iocb->ki_pos, fbatch); + err = filemap_create_folio(iocb, mapping, fbatch); if (err == AOP_TRUNCATED_PAGE) goto retry; return err; From patchwork Mon Nov 11 23:37:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009959 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WQNrJj/G; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=chhu=sg=vger.kernel.org=linux-ext4+bounces-5046-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnR9G2F1bz1xwF for ; Tue, 12 Nov 2024 10:49:38 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnR9C4hWFz4wbp for ; Tue, 12 Nov 2024 10:49:35 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnR9C4Zhdz4x3J; Tue, 12 Nov 2024 10:49:35 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=139.178.88.99 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731368972; cv=pass; b=heawaUuRfElsU8t+kA1zJ58vCcgTdajQsutefXzkXBRFLPi3XUbPoy5SaDU/XAQXt8CAeaIvQht0BUX4xu6Z77efzDVuMsxvY7hWJA50/2Jl15Cz++zX1GkV0fcyU0QcSFIC2fK3bQeLYmiuIGvaN+5nQutR3cVcK43R8mHFk4ZX09ZZYc/1WOv+hW49Rb1khdeNRR8473ro3kJ5Yc7fMTysH8wBlBURMGPszKqIFdqVU3GDEj5fVEzXadh/TJTbjnj+9fMope1gXiZZWwjFdBkwfoPoxapNZqMwhIjTfuMLR7vHVh4cLPwkJXrhApXBG7pUMypHuoAT0IiumGm/ag== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731368972; c=relaxed/relaxed; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=v1mFDW+01Ne95McLgvNu3yb44Xm4wuo8XmA3t2j7E8rrYE46YmqiK9gnkvugo4+C58ueyfYcNbg0Sg62cduNqpP2xJOxq3NCpVQFEL7jLHpR1wBMkPOzr1uA4edPfSQMjC9Sale4ICKWMz7+ZEg/pdAzFxRtBIpBNv/NpfHzoaWI5d43c3fVIpSN3WLMw/4Tv9rQFjMdOheLBzSPUO6zv6MRWZBAWa4BXKZ4PlgUPGRVnMznOVniCAzPnq40lOPDwuGyJ317qLp2HmnDbphs0DFrKiNjWgYRYpszTeprwYAqIecJvspumXh8/c4Q0VEN+/JWhmGZHWs24vftwoui+g== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WQNrJj/G; dkim-atps=neutral; spf=pass (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5046-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WQNrJj/G; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5046-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnR980MS7z4wbp for ; Tue, 12 Nov 2024 10:49:32 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 48DD4284EF5 for ; Mon, 11 Nov 2024 23:49:33 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3F8551C304F; Mon, 11 Nov 2024 23:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="WQNrJj/G" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8567D1BF33F for ; Mon, 11 Nov 2024 23:48:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368932; cv=none; b=DLEIOPkSGMRgB0Nw5rYHzVmL40p05QZ+979z94zBmNunditXppDBVY1/2+iaAwjZ0RwhJCivOMBpwVCpzCNAeehnqZ8M0aOhXo8ARsTYAvW+2ZCD0mBcIk0G5vuOmID2mxaZiCDDineTHncVNAsysIB9ER5SQ+C7FRTfv4ggKEc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368932; c=relaxed/simple; bh=wdE7j3SXsvf8mZ3WLeO9bDbqT1xBl9uoI5Ta1MifPnA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZrqRvHr61ZgDMIpfHFhgByZxV7gOeyeTH1OeBp29O9xb4YkGszTuyOUDkhHYtPxXwBNj6ugzIz/coNuYwVlOYIwf+A6PHmFdpNeIwa05K8B2mw+dBjx6ZFGA2g/R7r8a/YCEWMOzIQvL/1ZZsbEz198jQX2dNJqBCro7TSUPZJA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=WQNrJj/G; arc=none smtp.client-ip=209.85.210.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-720e94d36c8so5156773b3a.1 for ; Mon, 11 Nov 2024 15:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368930; x=1731973730; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=WQNrJj/G6/aAoHDvOTSaHAPL/lSazwCCTJ7q/uUf0pSMKY+ehGBWUDahZQHQpLnDS6 H1yNxaaRUpzjIT7CZrfu8s5tfNhRr6Bn5Me+uSbkX7suRGIpGAXfQdHQDOvIbNzHenM3 XXhM+mF1nYdDbXK+HnT0Kfhv3eHczegGCLnBDArv56n399YX0vdS/fPgxeBG/gwP6+NJ eeUjji4l7C+BhQxKjIpKCY3hOkIM8UmpjO/2+Y3+ZhIcJCgc/nViVYpy6qlIfDPVzATd asLPc8SUbP9pvLF+RYiAIUzBzUIHVB+wHIpS9CWj74VEX3ZLOlFN+ho/SfyWGoRUE2Fz TFEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368930; x=1731973730; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G7YIma1TSdS4GVKncfhre22V8DGIHalzmKUoqzgwI0E=; b=OCTtgTfyijoyHLu0hZWBPHEfAjpc5wfeJXuYjIQ3vvJC776fkJPhO//qboHG8Og023 s0TtHMP7YDPCQcVOuA7QL4wKYcxloaoNRl4YXTb+IEBT+aST7VRrKPnlOjTwrGWkidof pSIr9DU/qkYEA3OLI0d2r+rV4p2HMuHiwV2SyQRh4GTxzZAmYucNSP31A8dqtNrwzSSA d//3QhdqbAxWWjyQBhRkuJOaoPJSp9pH2zy9glMIPgfcubIH+kie+/sr70W95GGBErLC iTKh8GCi98qAzjeYUQNo3jTUMSfJlDdtvXqNCu4X2PKJHlka4gaNAVIKYly312wXMEq3 FOIg== X-Forwarded-Encrypted: i=1; AJvYcCUv0LAiQ8mMIopqzk1GBaruLV/JhUYAp3j8J8/VHW1nj5xJDS9YFEx/XH7Zb3RxkXu9TwS9Fyr/2W3T@vger.kernel.org X-Gm-Message-State: AOJu0YyQSuexzPhOtPeg5c88ADR1RF7S9K+ivOwpBm/eops4VU/GQAi7 ZzAZMxYxvgOKex6Pdm2z/yA44CHPS5EYidUVAf7YwDpH7oLKJEDvJwJNe/70hVc= X-Google-Smtp-Source: AGHT+IFIT751yc6PWfScZ6t4wwtYiUFG2/DBhEvJ8IDhssjBxrF1+5xRFKTmZX93Ms0bG2KlEc/8GA== X-Received: by 2002:a05:6a00:1801:b0:71e:427e:e679 with SMTP id d2e1a72fcca58-7241223447dmr25540831b3a.4.1731368929963; Mon, 11 Nov 2024 15:48:49 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:49 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 02/16] mm/readahead: add folio allocation helper Date: Mon, 11 Nov 2024 16:37:29 -0700 Message-ID: <20241111234842.2024180-3-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Just a wrapper around filemap_alloc_folio() for now, but add it in preparation for modifying the folio based on the 'ractl' being passed in. No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/readahead.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index 3dc6c7a128dd..003cfe79880d 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -188,6 +188,12 @@ static void read_pages(struct readahead_control *rac) BUG_ON(readahead_count(rac)); } +static struct folio *ractl_alloc_folio(struct readahead_control *ractl, + gfp_t gfp_mask, unsigned int order) +{ + return filemap_alloc_folio(gfp_mask, order); +} + /** * page_cache_ra_unbounded - Start unchecked readahead. * @ractl: Readahead control. @@ -260,8 +266,8 @@ void page_cache_ra_unbounded(struct readahead_control *ractl, continue; } - folio = filemap_alloc_folio(gfp_mask, - mapping_min_folio_order(mapping)); + folio = ractl_alloc_folio(ractl, gfp_mask, + mapping_min_folio_order(mapping)); if (!folio) break; @@ -431,7 +437,7 @@ static inline int ra_alloc_folio(struct readahead_control *ractl, pgoff_t index, pgoff_t mark, unsigned int order, gfp_t gfp) { int err; - struct folio *folio = filemap_alloc_folio(gfp, order); + struct folio *folio = ractl_alloc_folio(ractl, gfp, order); if (!folio) return -ENOMEM; @@ -753,7 +759,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; @@ -782,7 +788,7 @@ void readahead_expand(struct readahead_control *ractl, if (folio && !xa_is_value(folio)) return; /* Folio apparently present */ - folio = filemap_alloc_folio(gfp_mask, min_order); + folio = ractl_alloc_folio(ractl, gfp_mask, min_order); if (!folio) return; From patchwork Mon Nov 11 23:37:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009962 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=XyRfUh6B; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=c4rc=sg=vger.kernel.org=linux-ext4+bounces-5047-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRCk6mQHz1xyG for ; Tue, 12 Nov 2024 10:51:46 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRCh265xz4w2R for ; Tue, 12 Nov 2024 10:51:44 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRCh23pnz4x11; Tue, 12 Nov 2024 10:51:44 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:40f1:3f00::1" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369101; cv=pass; b=WoTN4BpqKGKqo2IxnDqTSYRHSxDGqefZmbRgmdMZ2QtnAoGd8uxiNmBF4qlysfhIEzADMkUk3I8okduaRBfMsZa8M4wSmalyhRU1C6fggYaI4u0ZzxB21/MqAwarIULKbHTxt0rWCiDRDAyvVPAJcCULcOACSLSofoNnN882nYvN3zagXF+pW17777TBCLFxnDKdKYBloAw9eeHj/G4ONYTSnP2VU8n4epWaYQhlECNvYrlC6cil6+hCH6IS5rL7nVVJ4Nmiq75Luj1XSVDjcF7NJPdzTYWKnvCchyNnzJ8Be/UCJfbz4eWp6bYIOzY+5eawuK7buljbRD3Mjmx6fQ== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369101; c=relaxed/relaxed; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TU/90Gyavq4TrS6oka8IBrte1IEFVrqI+1gLKiiclIZlaEZddhK572LEpY6wTR9OHDtjYGLWtit0hl3uFKNXnQMS+ed6iLjbXSZCVCgIPF3LpQg/gOtcl+9Qs1FTdgArYz/oWDSmMaB9PypjSf1GrKWA652sToh8O0qbUvgu6MHGG8rpCrkUmWYv3KaDoNq4axK30izJDjVWTeZPkSQkSn9KDlj5k/yw4RKNOganZnJwfQcGo+9uOiw5mI+Pj1bp+DyfoBA5mmVGKX+D+R4X/CZtr1IhMzWRywtYyg+b96F99n4Nv8FDSLV0988Kk+OctpP9jTcypVuctcEodRmlJA== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=XyRfUh6B; dkim-atps=neutral; spf=pass (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5047-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=XyRfUh6B; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5047-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRCc74DXz4w2R for ; Tue, 12 Nov 2024 10:51:40 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 3C4E0B2C6FF for ; Mon, 11 Nov 2024 23:50:16 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 268B21C876B; Mon, 11 Nov 2024 23:48:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="XyRfUh6B" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A69F1C245C for ; Mon, 11 Nov 2024 23:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368933; cv=none; b=qchRWQbmVrww3G8Ywrz2/xooCE8cstB8m/xAL/8X1ajw1A0rq0D4qqH9BBy+ECyA99p8taoQu0Z6bavT+30tYfFn5P/x2lH4ooqSKWXu4xLJ3L8YuA+9uGAxJmM+x48sPRDkrSptx7+Cv8YlQTng7Grk8BJzCracDFEgU+uZPnw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368933; c=relaxed/simple; bh=klEANlDo0QcivFe1NIYDNiOFa1lRCa2wf9JjLv/2My8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=C43zazE0Wp7vSnBXFvYxgg6+2YrrhTPebU6R1oQvzWCc8S9cJfkDpKI36WbtwuuNP1YhEY0JVLR/qXhzPeArH2IHlX9AKt7Xf/Wn3sEfTuUdDvPXcA8pqzt1nP4xqH68awY3whonwq4/WTPcAyu3m9gqQ0fLjvQE3PBzRhbgxl4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=XyRfUh6B; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-720d01caa66so4744381b3a.2 for ; Mon, 11 Nov 2024 15:48:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368932; x=1731973732; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=XyRfUh6BYtZ2g14d7GAGy9wrJQ5P9sCjQKhoDrSVSCjvCANbFguvc/B9iUzDuFcGSZ 0y1w5OnNvFsV7kTfG3zQFD9LNW8T1CKqN0UAzcSDyo4LFB/a6uXK7MDRih87Dp6LFIoW L0bjq0y7i6QPyIEpJ2hI/kzFkLJv8rz8l8V7WzWZqC+W9ecB0XKyA0Tj0UR303WAe2bu IXPnq/ZSCQHjNr+zuA7INYMMu7ZQH8x0/DZgB633RbzUNJAuJhQ1nRA3jeq0OtevjsSJ kpmSt6FHnrVV5DpS+nMnvASBxUrrZPwasuVxZgayILUGIUnXHVHYECS0QrLntiGFmcBN H9tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368932; x=1731973732; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9QDQ8u+XOjRX6rdX1KxWEpFxXqQ9HPCp5iTXnKjwIis=; b=l/6oEW0in02AuKRHRz1dO6KAnu12/1ID/4oDVq8X5rPkRBx4ZcSiNel+Z43T1Y+nuh MYJmryZb92QTZGvtUJnsar9R28jI/tF9MTSldpIze2Uz1koGUulcCtU6A4u5qiB93X3X yMdNpJb4MKjTlMw3dtZJTi9zVtGDCGf6seMI717isHoRjkjKbHiZkfnL+JbJWQLXmS4a YVmv/KMMnsHFEaeCfHWeSFbf75c1syg3ehOu54pE6kD7D35HxhqKaYK0dLLdAo4zH7LZ PeqaRN/AXb6KXVTfX0Duiws/qA4/MBaw1QFyUmPiX4Ggq8Z0LEQ67ZNQowdm/uSD5Umj HMSg== X-Forwarded-Encrypted: i=1; AJvYcCUHD6hmYoWB2K/5cwl5EMmEywN3g1o+fq3DsvVX3+U/N5j+TzJTlZgtArMqcbvI/9E/Qfjr++oPQ85X@vger.kernel.org X-Gm-Message-State: AOJu0YxovVuVp/N8lilx72+yOs7hYs5pK4qFbsP4Zs0t0yOACPVLwfyE Pjfn5O4xixmGWxJdM2FOqlQgGr6bB+cokTZNCFNEvZB+PiwDkfgo9paph7JOaH4= X-Google-Smtp-Source: AGHT+IH9/1Olbxn3KTdIUuB7+y6iPc85Xc8pyUcg5cMVAU1yzHvt71d7cxX7HqTkOKWntgPA3yDsaw== X-Received: by 2002:a05:6a00:2e99:b0:71e:4ee1:6d79 with SMTP id d2e1a72fcca58-724132749f9mr20118106b3a.2.1731368931568; Mon, 11 Nov 2024 15:48:51 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:50 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 03/16] mm: add PG_uncached page flag Date: Mon, 11 Nov 2024 16:37:30 -0700 Message-ID: <20241111234842.2024180-4-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Add a page flag that file IO can use to indicate that the IO being done is uncached, as in it should not persist in the page cache after the IO has been completed. Signed-off-by: Jens Axboe --- include/linux/page-flags.h | 5 +++++ include/trace/events/mmflags.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index cc839e4365c1..3c4003495929 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -110,6 +110,7 @@ enum pageflags { PG_reclaim, /* To be reclaimed asap */ PG_swapbacked, /* Page is backed by RAM/swap */ PG_unevictable, /* Page is "unevictable" */ + PG_uncached, /* uncached read/write IO */ #ifdef CONFIG_MMU PG_mlocked, /* Page is vma mlocked */ #endif @@ -562,6 +563,10 @@ PAGEFLAG(Reclaim, reclaim, PF_NO_TAIL) FOLIO_FLAG(readahead, FOLIO_HEAD_PAGE) FOLIO_TEST_CLEAR_FLAG(readahead, FOLIO_HEAD_PAGE) +FOLIO_FLAG(uncached, FOLIO_HEAD_PAGE) + FOLIO_TEST_CLEAR_FLAG(uncached, FOLIO_HEAD_PAGE) + __FOLIO_SET_FLAG(uncached, FOLIO_HEAD_PAGE) + #ifdef CONFIG_HIGHMEM /* * Must use a macro here due to header dependency issues. page_zone() is not diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index bb8a59c6caa2..b60057284102 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -116,7 +116,8 @@ DEF_PAGEFLAG_NAME(head), \ DEF_PAGEFLAG_NAME(reclaim), \ DEF_PAGEFLAG_NAME(swapbacked), \ - DEF_PAGEFLAG_NAME(unevictable) \ + DEF_PAGEFLAG_NAME(unevictable), \ + DEF_PAGEFLAG_NAME(uncached) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_HWPOISON(hwpoison) \ IF_HAVE_PG_IDLE(idle) \ From patchwork Mon Nov 11 23:37:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009960 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=2qS/cp5W; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=y8qt=sg=vger.kernel.org=linux-ext4+bounces-5048-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRBm6qwJz1xyG for ; Tue, 12 Nov 2024 10:50:56 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRBk0Z92z4x2J for ; Tue, 12 Nov 2024 10:50:54 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRBk0WBFz4x3J; Tue, 12 Nov 2024 10:50:54 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=139.178.88.99 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369050; cv=pass; b=dTkE2BLZfEh8w4vueOwrNdjkeTZJx3a0ZrbEiaOvwa3Yy1wEC10SdS3zbziY5wUjIBzNHwh2YNXpuZ2iOqzvYa8g4MqolEq/G9ecl6yH3Jd/iLRd4lUWYqI9kRB2NbYX3GReg0YcwEzIJpFtJyHs8pnUBwa7MjtddOh5pmOU09TIGgxx8lhbKjIRp6pWMo9v+rQlJv1WHBBhgBRkWd0sdPNWa8r289yAnYyF8WSHad+dwEq7bT5EWdO0Y/rblKURVTj7JLDmigaZ49ddGhcky1LtSOpL66kqZI6ZECteoAdwJ14+LNQMIX5KkiP3pij7K3A7lJEuOsZDivm07MSYug== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369050; c=relaxed/relaxed; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VjUlhvQLSaSUOSbIIrkt4fgra1FMW3CS6lDVWzk44Cga/jC8I9Ex/PWTX/h5GWz4lYVL3itbpho1rxxvbB+30iChP+25u+ajPEg61fYQPajRAIyRKbFBW1jhCwq22D2BnlH8fPNdfib0cUYOC+R0mKtlnKLO8TZ2b6gHHL5pHlTOhvifxKORs1nwHYN+pnCwg27MvKfRVwXtw68SOoUJxHj9iEATd1jxkqQxYAikoNQEe6Hex4044AOBZYGY3a3CgBvL6voHk7RuEm3ZlMPlzUj8O9KWvoxAtgS76ihigez9UZEDkI1L7UbjPPnypCFkatZMhKi5O3iP/TL/FQI19g== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=2qS/cp5W; dkim-atps=neutral; spf=pass (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5048-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=2qS/cp5W; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5048-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRBf3N5Hz4x2J for ; Tue, 12 Nov 2024 10:50:50 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id D0DC02856C4 for ; Mon, 11 Nov 2024 23:50:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B93F51C9DFA; Mon, 11 Nov 2024 23:48:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="2qS/cp5W" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E11A61C7B6F for ; Mon, 11 Nov 2024 23:48:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368935; cv=none; b=BH3IeRVfqb3J28zvA7yRzieMuCkmcAlOB18RAGOcly+mjnRyNQTKEX/jL6sFeSDtNWej0lSZZOq6AlX5GqSn4ERzc7akezyhzia1tgsCU+9EUK18YdtTzhny9xLo6ak3Gb8r/mL+i8dp5UjHE6ptF0QFd1/D4GAEbAYVeqvbzA0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368935; c=relaxed/simple; bh=AFLrZl3fH+/hMhwZTjgZ0OWM0ndgzDi220n1zUDQffQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dUxow8xWLiAmgiHg4jkHvDFXvoBZl25KAyNit5MDjHVfDSXdCTMBGUqOER2dneW7Z995HiaTrAt8dT+rNWSSHEDeTpGzULKsX3Zzw3ZgP+Q6VIlMweZWy0ps6BVUmGYcf2UuSfydp5G4JZglNeh3wZQp66VvWD/hJ6LSvyCyKn0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=2qS/cp5W; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-7206304f93aso4561027b3a.0 for ; Mon, 11 Nov 2024 15:48:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368933; x=1731973733; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=2qS/cp5W20QeHg9RA27riSWL5AkZbAWM/PwKqjqs6Ii8ffsglV/wEcmEWXLLzcHOAd eei6lcVWFwRlLkhM5BqTWB61Gr5sv3uGcIc64ZVwA3+7viLF4AdIO14jOZCTrteKdQwb z3rJva6VXJrUi/QfIe9X1l2ErVYL1uiLOvwgPl4wVObJ56fEM1xUFksJ/uwOprAssIGE hb7Dnro93lgaccCPQLBvZxUlORIUQoJgjI8dRui/XGDro8XEPlpToO13zK3J1vMuwvJJ Gnr1/CPO2Ijp6uWNISxE3CO6UYUFij3OmQAGEunVZ4lPIij1q/ZCUfdzUjipCqbPD141 1aHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368933; x=1731973733; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jvfAlDie45/dO/q2Oa4nMUlSYRbUQddnQckYeJvFoiA=; b=HWVXRnkNnPmcnh9o5A5dEfkXM0zhblkNWd2nESraEp2hGvxpUb5xtvlluIMgfg+4K9 yJZLXiLotNpqSDfPGpkKI3GBeDkSCYYSTc4ff131UTsGTI7zYZCX1X35usVKSU2qi/gq n3mN4K4tJUjxCN5auIPZGZ5bubH9rpPaFB+86w1KC6jNQ/OrgcQdUOpsIVPBCQtfrene cIrs0t34Np7Y349mjFMgeJOvcFZ3sm1CKuljUuFfyzas3cxxlp58ob38qaAtlM7QA4Op OHFNZxLT3Rh33X0CE95lDIQcwVE0GJGniSvywfqkVuwNmhS/Pe2v2KeDE9X4hAZY02bF z7Qw== X-Forwarded-Encrypted: i=1; AJvYcCVBWQuCoChxAwDAZEK2KwkJvV50b8gUTjnSb77sEVg1Ncz18IcOh3gFmvwF7X4P+IKnsXSSVl1tHHO4@vger.kernel.org X-Gm-Message-State: AOJu0YxJkY1bw8sogZlg8ISmZc89GqpuYqgt4CKcVTxuMqGoU33J0gfu zqJlW9FCl0lSbVjtQ28Cwz7hhTTHZCKZvJS/aDmkheMzHyscbJq1NrxoU6T8Cmk= X-Google-Smtp-Source: AGHT+IHeHxas0hJ7ji/qpv6NspxjwSxpVOtrX7CmDHrNgbtKnM+jsbZo98ZnZg8WufLJazefZ+DkyQ== X-Received: by 2002:a05:6a00:138b:b0:710:9d5d:f532 with SMTP id d2e1a72fcca58-72413380f6cmr21319143b3a.19.1731368933147; Mon, 11 Nov 2024 15:48:53 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:52 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 04/16] mm/readahead: add readahead_control->uncached member Date: Mon, 11 Nov 2024 16:37:31 -0700 Message-ID: <20241111234842.2024180-5-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org If ractl->uncached is set to true, then folios created are marked as uncached as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 1 + mm/readahead.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 68a5f1ff3301..8afacb7520d4 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1350,6 +1350,7 @@ struct readahead_control { pgoff_t _index; unsigned int _nr_pages; unsigned int _batch_count; + bool uncached; bool _workingset; unsigned long _pflags; }; diff --git a/mm/readahead.c b/mm/readahead.c index 003cfe79880d..8dbeab9bc1f0 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -191,7 +191,13 @@ static void read_pages(struct readahead_control *rac) static struct folio *ractl_alloc_folio(struct readahead_control *ractl, gfp_t gfp_mask, unsigned int order) { - return filemap_alloc_folio(gfp_mask, order); + struct folio *folio; + + folio = filemap_alloc_folio(gfp_mask, order); + if (folio && ractl->uncached) + __folio_set_uncached(folio); + + return folio; } /** From patchwork Mon Nov 11 23:37:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009961 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=OF0WUr0e; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=dlks=sg=vger.kernel.org=linux-ext4+bounces-5049-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRCS4WFCz1xyG for ; Tue, 12 Nov 2024 10:51:32 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRCP5Yjhz4w2R for ; Tue, 12 Nov 2024 10:51:29 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRCP5Wwyz4x11; Tue, 12 Nov 2024 10:51:29 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:4601:e00::3" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369086; cv=pass; b=eUSk3b173uFN8K5EUWMYo+6a7WL1Y4IgBHq5UVBn1TkDq0doqEZIlLePKkRCS6fi1GlMofOUjyol2RfCN3yTWdRQQcgvHIcCWDsnt2Wlim95gvWHXq2NDq+8VWD5dbANP3ts55Ikuunz+IZEqL5D8DyoAP6S9QtJ70W/E3SgIjspjZZh+73FTfIbudhc7+njMO8u6hjeiiI+nJyh37YgABLoKMnqpYnFtabWFxMrpBr2ccOo2hJy2r30/+Glqv82T1IIfAdPLprmicUI4bPbWBnHsvbYOFmyKIH/wwlHSZG8HCKjiBUB4+ylGlzT6UER08+CKQQD/w31z/hE3N0CcQ== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369086; c=relaxed/relaxed; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eEftsQuap6TLiFyZG6SV/0hQpPcsaSuE37uf+cgLiiLSVgACo7RO0oBQmj634pX37Mj3M0vOT0ZoWLpObgQZuRl3X6Ly8bjkJ7N/wreYlN+3v6vffD2P61DwCSNgwmq0Iw+NA59SLCUJKZgjmQicRABg6TDAMfU060he2VuCAU9rt8LTEBu3EgeOBeBr4+Oob8CWAZ3Nodiz/ihHg+5BuFtQSKZeA1Cuqpkh4qd0UAeJpc/ucbfi1TichP7Z1DUFOsa3MTnki71YCRK0hq/jJgLPdWiBMD1tvvHW0FzOcuFOzRxDNvRWbPpO+kjcVY3S4yHD2N0SufgKUgZllathfw== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=OF0WUr0e; dkim-atps=neutral; spf=pass (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5049-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=OF0WUr0e; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5049-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRCL0Hy3z4w2R for ; Tue, 12 Nov 2024 10:51:26 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 8A8E31F23A8E for ; Mon, 11 Nov 2024 23:51:26 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3C0A81C9B6F; Mon, 11 Nov 2024 23:48:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="OF0WUr0e" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 601FC1C9DD5 for ; Mon, 11 Nov 2024 23:48:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368936; cv=none; b=lFe95paOr8X0QPzoRceTEOxDewezQ9FQATDSaCHhAbY94dAv010mWJQJ1GR/LQ6y11bcU20SaviqZLoDfTeMcfOwRfZBz/Q466SHQx4yDB/Dmd1StjWy4s43mxDYUz4/uluZYJ6slYHVmf88tfYx6dqz2Vh0Ly9Z5GIHjH1FAJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368936; c=relaxed/simple; bh=HNesIF4+NCKJRgMS8RJOzPdBmK2CQSC6p10zaqTN28c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n1hJtKL6kXKyN+XN3MEAOO8ayPoobJa3Ouo9tf/8UYmAtnos9mbfaNHP3/VfU0UHsgcXgWEO7sOTEkGWWscPCgd39PwRK//k9WyusXTcJsSPM9lj058C1uXdiijaJ9Nyb8D4UNqFP0DzvmuUXQuyQI2l3aRHLf2MUNifEmOAd5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=OF0WUr0e; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-720be27db27so4124104b3a.2 for ; Mon, 11 Nov 2024 15:48:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368935; x=1731973735; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=OF0WUr0e79AhMimpy85I61HjDhF5jReL4TeivNjN63K7r49rrhSv7C6PCMZY9s0O3m sLe8wGHHnzUhI7aCOzrlNV/u7656/fPSZM9IXjsZu386cjzZppTSuX3jZxFFhcNeBequ IOforrBU/9NM2a6r8qfZeumikX1uIs0sRNcxiBBcn5X8g6QJJlCMCMx+yr8oj9MBmg1s 8JkNJHdepngU/Kl7g4gqt60K2PlqJua1d01OgSKE1APkdCy6rjaKAnM9PW60BL83fFS3 7XD9h1iDmcXobE4n0Xy4+P85IofXytwjL5Vcg1VRFIwp2MQf0WjHLASbw50JezW06b2T BGgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368935; x=1731973735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=573A7VIJaN7Phef7EPro6/84rHuXaF2Ar2GUBUbETSc=; b=bEM86swaSmBVOgfLbpalP+P2Lf+h3VA0DMlMZC1kpHbUO5tBiEtnhTo4EOnxZ2mzX7 frPARVLy7R/Sr95uU0xzq9G+w2fTvtbYGNRZO8u0hfxZAFnzBvsSbsn8dxbIhJXbr7uP +2Vwr6BkFiTKpGwkJJ82OHYd0sVZrKSJoojIEn/1Mtky7v8/5XhdBw2LyVNYk8Fz+2d6 YKMuZF+YHzM/VclDXzleaTqThC8apYKLEAv1VV1mK0vUOEs4DsUGOitGUQhgFzgxgxSr dkgIgjnRO/cUyzs1J8UoZikxygEN8CSp8l6USWPCzBSbz64QdQ5mmirKS4LMUBrRrRF/ 3IOg== X-Forwarded-Encrypted: i=1; AJvYcCWnmoc8kf2gpW/mRstiv0EbbpW/G+vqpdXIe9KH0d2QshwVPFm3ejYvSZaO65sFpHjT4fFR/QqvQXil@vger.kernel.org X-Gm-Message-State: AOJu0YzayGTZe/FlMCCGorAMVsm2pHN7plJS6CjVrb3kq+/rtVyrF3Oh jf8uNgyNbGQdnx1QEsp7PAZAd9px1jznkHgb27qIO4TtcyNUywNNqUXvaPcUVSo= X-Google-Smtp-Source: AGHT+IFFv9yEg9ETjAil4BLRBKswDm5bw1d1rwvQrtFRdIgpywcgln1Dsm8EO2zN3a/ZuXT+tu0A8A== X-Received: by 2002:a05:6a00:460e:b0:71e:75c0:2545 with SMTP id d2e1a72fcca58-7244a5d480fmr1035562b3a.25.1731368934821; Mon, 11 Nov 2024 15:48:54 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:54 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 05/16] mm/filemap: use page_cache_sync_ra() to kick off read-ahead Date: Mon, 11 Nov 2024 16:37:32 -0700 Message-ID: <20241111234842.2024180-6-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Rather than use the page_cache_sync_readahead() helper, define our own ractl and use page_cache_sync_ra() directly. In preparation for needing to modify ractl inside filemap_get_pages(). No functional changes in this patch. Signed-off-by: Jens Axboe --- mm/filemap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 91974308e9bf..02d9cb585195 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2528,7 +2528,6 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; - struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; struct folio *folio; @@ -2543,12 +2542,13 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, filemap_get_read_batch(mapping, index, last_index - 1, fbatch); if (!folio_batch_count(fbatch)) { + DEFINE_READAHEAD(ractl, filp, &filp->f_ra, mapping, index); + if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); - page_cache_sync_readahead(mapping, ra, filp, index, - last_index - index); + page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); filemap_get_read_batch(mapping, index, last_index - 1, fbatch); From patchwork Mon Nov 11 23:37:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009964 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=R4aiaU9j; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=o/zf=sg=vger.kernel.org=linux-ext4+bounces-5050-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRD144Rbz1xyG for ; Tue, 12 Nov 2024 10:52:01 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRCy6Wb2z4x11 for ; Tue, 12 Nov 2024 10:51:58 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRCy6TYSz4w2R; Tue, 12 Nov 2024 10:51:58 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:45e3:2400::1" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369115; cv=pass; b=gOnRKeGMMcbf7rgsgMt3KMBe9lXJyhRsXllpT2Lw24Ad6UJ2I1qqHB6wkLJAsoOJAzTBnLRdLAz4CiM3s7fC3kkxiGB0W7AxTNsPARPsAMwiPf8op92tkZ1MVnqnhN7HHONd0qWiarISVmTwhY7P0H5VO19l00/4KBNOBq5pPi7QsVwapsb3mp+2bEZOMbw1oiNCAnkQXrlDUT94Va/JaHCXEz6r57krqh6gxda0rLDBwlpOXmk5QJa5YjUdJeTdRwgeve2Ixb9S/ZIZK1Mfd5nP7nBwQ7eE9TFfC5P5Muqxxe+fCDK/TlvZZ4IooiaTHRUQpZm8gU3Z6mM7L5hPvw== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369115; c=relaxed/relaxed; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZqYgBPIRN1WSubB15TZ/12L7rCq8WTWTH3O1eHaqKIbqqEAdeDGK7TbhjSoA0T502O+xwaURcqhkRXUUuQXpH/B9wVEDq791TtAIH9UgH/jfP6ad/B/kuxI1d2j6vVVJKkqdO550xE0vz3gsVl2h48IiNlyPPOIgh9CRPDp5tcklz7D38J11vR9QdrNuICWTS/ZEaZLS8dT/KyMfJcMceNALIgBpM5k8qqLxA23mN8yXS+BxT/urqFRwkz+5ByQ/x/DHRSaUH0zGTZ0ilgF9XdMUIc5rty9zU0bIctjcqQllxgNSe3LaYxFKtTvd0bHH3nEnlwfRv52tz3yoVj90+w== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=R4aiaU9j; dkim-atps=neutral; spf=pass (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5050-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=R4aiaU9j; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5050-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRCv2rnYz4x11 for ; Tue, 12 Nov 2024 10:51:55 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id BE6E32855B2 for ; Mon, 11 Nov 2024 23:51:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B93171CB33D; Mon, 11 Nov 2024 23:48:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="R4aiaU9j" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBD451BD9EB for ; Mon, 11 Nov 2024 23:48:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368938; cv=none; b=Yrfzp94sLXC/NwDj1CA0RltFKDDcdgzHgKtDpPIonX+AZzWGdhUD3fyuWrRGl0SIptfydxHXTuL1z5sc5blk0QyquTRh/e8l7FHlVjul7nRxaciuZjbqB6ZeNxErSo53mCFGC4uoq70V3wxBayU/cmUkuSHrRr+/khbO8fkQjSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368938; c=relaxed/simple; bh=9Ma4XXKwKrbfCmk8++0FMznuxKmPjzqiFxHZUZ9JUvs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p6WkUhzF/v5N/pEei0rFgvhZZMRvmiNLh0/U2oR8d0EUswOFD0Caj3I4EQAJsB5l0S5u/AQJMpkPPzqkOiffn4HAdbHsZwQJ3PB+FSonkG8stiBmVdPh1wLpUHlAhPqX6a1VSo0c58lq+ktLdlFJLqbkMkpnEQS8poh+EhjOsmM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=R4aiaU9j; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-7240d93fffdso4129833b3a.2 for ; Mon, 11 Nov 2024 15:48:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368936; x=1731973736; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; b=R4aiaU9jMdoXAa5LmgNMxyvFc/Uw4BX0xojgO8xFzGRanykKRWYDUDxa5zSd2FP+da 2/d03PDDT7mnhyzNolWF8mXaubonK+n7vPkCmjhw5Q5nTBMgqWqKJpLSFITxoFofw7Km 5bkWpUBbRX0UnN4AcqdLVnOCqscDu88xVBX/kYbwyfpp+nGwVyp3DuORQwedYFsxuAxf tJ76CARYeBEC+/KVXnkWi8pMKzdVicNjxFhmwbo6OPJgCa3gHhMLpQ87VKaS+rEuHMhB lToEni2gTX8sH3PRBY2+gSwr/RdQhyxPOogh4+g1caIeORHHruxmpnlqR3kSbFAW5Coy wRcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368936; x=1731973736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CL+fLnGK0BaMmMKm57rghb0D3nDN7TOwfaAxP5ktg80=; b=A4WGBKvnR80FPGBNbPkX6WMZvoGG+LbVp+nbZO1esN2t2pw8qM/a0paGalbhtMpwJs S+4aT1+H4b5BsRPiOQelMq/ukrI6pO08hhS1IeL/qAVnYCYPOYW7v7QJPfBZkyQbWCaD emss7vU9ERI2cfmUcKEL7Ca8KA/xYka0txfpcvBoCEFZ+IypaPDnQ1j7PG5VyVaVraJ5 EBglHd83rKwe0elHI/FP3q0ryLt8Sl/ATrY+GBs4FQm/qJhtNpLrdh1yaOzNVwC7XhaA 4nuAAymr6qIpiDHvCXYMQVm6qc3mI49pR7oBpAFiOe0JKbI9Eo2mj5zFxyx8rk+zzFqP QYeg== X-Forwarded-Encrypted: i=1; AJvYcCVovkUBNHlk0Y848Guj6NxFRoUrYjAIPaQC5hGWjSpse8z0sRrMdbmlcP1O9QEDPlMi0aqhbZALSlSV@vger.kernel.org X-Gm-Message-State: AOJu0YxlWkdUG9iOJ1FihsMRfq7+vJmPzcC8GAIXkK1tjD7K2zzCXLaV J694Rf3ObE1aQ45lTuZym+IMpOX4hpm3Pt40TmmpguJi8IhEEAtbj1iajIZcj1s= X-Google-Smtp-Source: AGHT+IGkT/vleDHQ8M3Hw8Ha2+EPGBpXcHS4Q/PdXfOEbEN6KEGRSFgIDbBBEJ7VgYq4tyZ3FLhGBg== X-Received: by 2002:a05:6a00:3cd1:b0:71e:64fe:965f with SMTP id d2e1a72fcca58-72413350e9cmr19827356b3a.20.1731368936369; Mon, 11 Nov 2024 15:48:56 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:55 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 06/16] mm/truncate: add folio_unmap_invalidate() helper Date: Mon, 11 Nov 2024 16:37:33 -0700 Message-ID: <20241111234842.2024180-7-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Add a folio_unmap_invalidate() helper, which unmaps and invalidates a given folio. The caller must already have locked the folio. Use this new helper in invalidate_inode_pages2_range(), rather than duplicate the code there. In preparation for using this elsewhere as well, have it take a gfp_t mask rather than assume GFP_KERNEL is the right choice. This bubbles back to invalidate_complete_folio2() as well. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/truncate.c | 33 ++++++++++++++++++++------------- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 8afacb7520d4..d55bf995bd9e 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -34,6 +34,8 @@ int kiocb_invalidate_pages(struct kiocb *iocb, size_t count); void kiocb_invalidate_post_direct_write(struct kiocb *iocb, size_t count); int filemap_invalidate_pages(struct address_space *mapping, loff_t pos, loff_t end, bool nowait); +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp); int write_inode_now(struct inode *, int sync); int filemap_fdatawrite(struct address_space *); diff --git a/mm/truncate.c b/mm/truncate.c index 0668cd340a46..5663c3f1d548 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -547,12 +547,12 @@ EXPORT_SYMBOL(invalidate_mapping_pages); * sitting in the folio_add_lru() caches. */ static int invalidate_complete_folio2(struct address_space *mapping, - struct folio *folio) + struct folio *folio, gfp_t gfp_mask) { if (folio->mapping != mapping) return 0; - if (!filemap_release_folio(folio, GFP_KERNEL)) + if (!filemap_release_folio(folio, gfp_mask)) return 0; spin_lock(&mapping->host->i_lock); @@ -584,6 +584,23 @@ static int folio_launder(struct address_space *mapping, struct folio *folio) return mapping->a_ops->launder_folio(folio); } +int folio_unmap_invalidate(struct address_space *mapping, struct folio *folio, + gfp_t gfp) +{ + int ret; + + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + + if (folio_mapped(folio)) + unmap_mapping_folio(folio); + BUG_ON(folio_mapped(folio)); + + ret = folio_launder(mapping, folio); + if (!ret && !invalidate_complete_folio2(mapping, folio, gfp)) + return -EBUSY; + return ret; +} + /** * invalidate_inode_pages2_range - remove range of pages from an address_space * @mapping: the address_space @@ -641,18 +658,8 @@ int invalidate_inode_pages2_range(struct address_space *mapping, folio_unlock(folio); continue; } - VM_BUG_ON_FOLIO(!folio_contains(folio, indices[i]), folio); folio_wait_writeback(folio); - - if (folio_mapped(folio)) - unmap_mapping_folio(folio); - BUG_ON(folio_mapped(folio)); - - ret2 = folio_launder(mapping, folio); - if (ret2 == 0) { - if (!invalidate_complete_folio2(mapping, folio)) - ret2 = -EBUSY; - } + ret2 = folio_unmap_invalidate(mapping, folio, GFP_KERNEL); if (ret2 < 0) ret = ret2; folio_unlock(folio); From patchwork Mon Nov 11 23:37:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009967 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=Dfkh6fxG; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=idkh=sg=vger.kernel.org=linux-ext4+bounces-5051-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRH543Rbz1xyG for ; Tue, 12 Nov 2024 10:54:41 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRH26TVVz4x11 for ; Tue, 12 Nov 2024 10:54:38 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRH26QK8z4x2J; Tue, 12 Nov 2024 10:54:38 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:40f1:3f00::1" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369275; cv=pass; b=fnKme2t5yjliVsKnjoj1UoTWRYboDWhaZi/MalJKeSPENCNvpABFd5nX6csVqbrWhQ7hJb2orl36G+2bYb/FGfKSqwwhjQSE/Qw9hLpJ9urGLXpRXMk+yyFPcO3hYeUE5kiJl6cGAr/StFqRYkHkLKe74LWnbwWR7EbKeohCkzWnjqxoFzT0QgG8ZMWP1yQZdcLzGDS9KWyADN6PXyc83UD8MvCZyT+2DOS1UVB0XnAAM0OFZ+vRjEc/zSmQg1FgLHUshR0Hcav2f4hiJyc2OA7En4kLAIQYv3Bb17R//myQHOuxE3RW/KhA0ROll5W86aRgkfOikFFi4fIfDJ1dmQ== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369275; c=relaxed/relaxed; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WgWnyyTd3I2yeknioICqWr/KRKkNfh+C8n37LPa/EASuGXEfLX74ixEfRo1QSYBEl8cgP7bk+/s5XWanBpGYYp7HlnAo8nujlWjgU6NVl1A6cpNVU5oW5bss6MrSw+BQr9+JdklT2T3vgBndjb7Y0eGwkQTpXwqgSzyf+NEybPFV797iB8umDTRN28RGsUv5RLO53cRI8ZrOqslBBVinjLwvebZiFDWh8XG0STTWocH1sD21LUnz9JySOjboiR7JbZjIZD3hVJ5JhbR6esEuykSC8FCR9W/e/FkESkjgDyklaxXmekADNQHO0HnRkwNNHvCiOKoEjyegTFEJ7EZ6Rg== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=Dfkh6fxG; dkim-atps=neutral; spf=pass (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5051-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=Dfkh6fxG; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:40f1:3f00::1; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5051-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [IPv6:2604:1380:40f1:3f00::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRGz2QDnz4x11 for ; Tue, 12 Nov 2024 10:54:35 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 70745B2DDCB for ; Mon, 11 Nov 2024 23:52:41 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D45291CBEBB; Mon, 11 Nov 2024 23:49:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="Dfkh6fxG" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0CB01CB50D for ; Mon, 11 Nov 2024 23:48:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368940; cv=none; b=T6jh9tKs8Ex37QMouEyUG5GqKwmFn8q+732WI3i5Ei2RZl9mjUq/+wypCaOfp2YSTqhMeYFbbRQmbJvS/DfX24BQ+ge39W8WcR80hZ9QvsjSR4Ih0hs1VFR/ISHxReYzyZaJtbgUO3tV68p64oOl2p52+LqwWBSceXFQNS6Rj8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368940; c=relaxed/simple; bh=QqsnT+PFB0stp1XOLxW6GR03t+Q+Rw9k/OtfDS+hMd8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mo5PlNAbpebFEjk90fUxV8H0IG+br7xmHG3VdPqEbT+zoNPxXe2BgpXT1XZAYUNaAcEPnzdGzOg62oMI2MH46/VQfCVU3Ies7X8m+I9prfi+hXmJxsMOon+0X/OaPdWOJZEMRSRxcrIw58kzxAw8LPeVDuGMbLfdhXtD7wPjw/s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=Dfkh6fxG; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-720c2db824eso5462416b3a.0 for ; Mon, 11 Nov 2024 15:48:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368938; x=1731973738; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; b=Dfkh6fxGx/uVYKbNvNxD5qGeWmbLclfHCDK9vy3OBrUwLZL2sg8OzxsrvVQN3tncGO rjLuKd6yYqW4TDr+a/0SShuTclv1PK3sSpdNs3L/dJ6+lL87HL0J97KXyfqEJ11xtPBA phSh6vZnHaIK/dGPOR7k/sDn7xR35h10zXtTleG370oUEy5wiiSkp/x1coBRDl8HoMJd JeWc86qXzqDxne+NxbeSvVGWGYV16n4UtbQa9ogJz1kMKlIub2awx+Vo9R2HL6JRFiYM cYW6JBuCHn3SYOPcF0A0n/RQVEX4KjYKSjUjlEg8p8WpEP5eJ138+1Cr3lIeMLLAr1pe VVbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368938; x=1731973738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BpaeGG8sCxZf4JRh/p7NGjZl2gbHvBRPjf0TUvSvRrE=; b=wT3OMyciBrHmFpF0naBlRYbNYhpNEWJZNWctvyBCAviHS+yH21OT660yW4ySXNYdDW taBJ+VOuTqFs7/97tz8sc8vlVRZg9Gjg8Soiq9NV/Kx/ExR+/HUzaHK4xUsLf34afidO TY/1w5/4mQmzBhsCsfjMuv+d2DFCex+J52TdfwKKcdSebm51pcfbyr19Yn9npXXlFFvn DF5YXVY1yey6tRCs4nMsHTMgdDCwxI3fI6usgfcQeDKEila5PRBUhMnOzcQVKPcZK8f3 QS9rjKGamm+XeQABmr29rKGN1O+Rk1LJc3H6yFq1SVvNYDEY6X0UN964OWZoWdc0iPdP EowA== X-Forwarded-Encrypted: i=1; AJvYcCUWfnwiKdRmkaOtHPKa8v2PfyxiIcQ914QfTSvhyB96UR5mP3UFne/rkqYP4q9AXln8lB7Mja4g9HW2@vger.kernel.org X-Gm-Message-State: AOJu0YzuP/5VmOXBGGK7JxNyRhV8yfucSXBaBMnmnMWvIewcuWaLbkHq JXWStiAW6iUijmoA5podO+PazFpfTGYncO9t8rIkN7F41043zN/LSvSFO3ahb5k= X-Google-Smtp-Source: AGHT+IGZeOeX13AG9u0SljALCGlqZd1HzXcprLCQtKHVA2LyzBnUVzw9lDFogBpMHW34whNWJjJhLg== X-Received: by 2002:a05:6a00:3a28:b0:71e:7846:8463 with SMTP id d2e1a72fcca58-7241334a3e2mr19790618b3a.19.1731368938117; Mon, 11 Nov 2024 15:48:58 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:57 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 07/16] fs: add RWF_UNCACHED iocb and FOP_UNCACHED file_operations flag Date: Mon, 11 Nov 2024 16:37:34 -0700 Message-ID: <20241111234842.2024180-8-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org If a file system supports uncached buffered IO, it may set FOP_UNCACHED and enable RWF_UNCACHED. If RWF_UNCACHED is attempted without the file system supporting it, it'll get errored with -EOPNOTSUPP. Signed-off-by: Jens Axboe --- include/linux/fs.h | 10 +++++++++- include/uapi/linux/fs.h | 6 +++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index 3559446279c1..5abc53991cd0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -320,6 +320,7 @@ struct readahead_control; #define IOCB_NOWAIT (__force int) RWF_NOWAIT #define IOCB_APPEND (__force int) RWF_APPEND #define IOCB_ATOMIC (__force int) RWF_ATOMIC +#define IOCB_UNCACHED (__force int) RWF_UNCACHED /* non-RWF related bits - start at 16 */ #define IOCB_EVENTFD (1 << 16) @@ -354,7 +355,8 @@ struct readahead_control; { IOCB_SYNC, "SYNC" }, \ { IOCB_NOWAIT, "NOWAIT" }, \ { IOCB_APPEND, "APPEND" }, \ - { IOCB_ATOMIC, "ATOMIC"}, \ + { IOCB_ATOMIC, "ATOMIC" }, \ + { IOCB_UNCACHED, "UNCACHED" }, \ { IOCB_EVENTFD, "EVENTFD"}, \ { IOCB_DIRECT, "DIRECT" }, \ { IOCB_WRITE, "WRITE" }, \ @@ -2116,6 +2118,8 @@ struct file_operations { #define FOP_HUGE_PAGES ((__force fop_flags_t)(1 << 4)) /* Treat loff_t as unsigned (e.g., /dev/mem) */ #define FOP_UNSIGNED_OFFSET ((__force fop_flags_t)(1 << 5)) +/* File system supports uncached read/write buffered IO */ +#define FOP_UNCACHED ((__force fop_flags_t)(1 << 6)) /* Wrap a directory iterator that needs exclusive inode access */ int wrap_directory_iterator(struct file *, struct dir_context *, @@ -3532,6 +3536,10 @@ static inline int kiocb_set_rw_flags(struct kiocb *ki, rwf_t flags, if (!(ki->ki_filp->f_mode & FMODE_CAN_ATOMIC_WRITE)) return -EOPNOTSUPP; } + if (flags & RWF_UNCACHED) { + if (!(ki->ki_filp->f_op->fop_flags & FOP_UNCACHED)) + return -EOPNOTSUPP; + } kiocb_flags |= (__force int) (flags & RWF_SUPPORTED); if (flags & RWF_SYNC) kiocb_flags |= IOCB_DSYNC; diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index 753971770733..dc77cd8ae1a3 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -332,9 +332,13 @@ typedef int __bitwise __kernel_rwf_t; /* Atomic Write */ #define RWF_ATOMIC ((__force __kernel_rwf_t)0x00000040) +/* buffered IO that drops the cache after reading or writing data */ +#define RWF_UNCACHED ((__force __kernel_rwf_t)0x00000080) + /* mask of flags supported by the kernel */ #define RWF_SUPPORTED (RWF_HIPRI | RWF_DSYNC | RWF_SYNC | RWF_NOWAIT |\ - RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC) + RWF_APPEND | RWF_NOAPPEND | RWF_ATOMIC |\ + RWF_UNCACHED) #define PROCFS_IOCTL_MAGIC 'f' From patchwork Mon Nov 11 23:37:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009965 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=tOp4jZDx; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=615d=sg=vger.kernel.org=linux-ext4+bounces-5052-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRFS3tJgz1xyG for ; Tue, 12 Nov 2024 10:53:16 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRFP6F9tz4w2R for ; Tue, 12 Nov 2024 10:53:13 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRFP6B89z4x11; Tue, 12 Nov 2024 10:53:13 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=139.178.88.99 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369190; cv=pass; b=amLQViBmASl0gbEr2ePFPLU61O8Ql4ZQd26NL3/aqKO2nafNz2oYUdkkyAopQI7c/PaRFO4OhidF4VKvV2QfdrNohlZJRES2ZbNHPSXj6rcTXU1o9XrQwWSoVrI5CtPnCI7Ds/RQ4ui43kG+ED/Y9/dK9oxMr960PF8nTYptT/jy7Un7bZiTMYHHpuurtFIFNbxwqI+NyHhbxTDCS8Q0APdUpRaMc/UxAtztqfDECyBDc3WBa3GWYi8LtyNFdoNn5ioxDd3x18rcWrF9sf0oa8ztsAT98l1TeMeetfJJuFmMAPH/EzS2JFW7zX8Jid0qHUAnIpu7m6qJznByC5zmJw== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369190; c=relaxed/relaxed; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AAcm6ksrK+5VKchSaz+xFKS54vpz8mwLA856V3mt2mMwB1USck5m2MAZSP8mvN8aglB3t5urtBkdy5UFJN0GZdDQ+BWBjRQn4ixv0OHchfv7HIw1Ojhy4IEqau1eFJ36jDKF/Sh0dVIf24KlX8YDVTiTucR4Yn0fJN1nYCkl+8WDbmUT5pbN92HWUUunpTX8NYo3X8X+yd3vxi0XNVbUzWzfbaJdDwUn42u5YIwKZHFOUOCrti8YDHbcdM/2sQBfDpuwZ/7WtM7GvRXmSrAvCD1fWJRMq5ulM3DQyGGq2oeGEZDpZVN0kRvOW0x7NaTaMfLTCJnk4KUBSCTRuTxHuw== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=tOp4jZDx; dkim-atps=neutral; spf=pass (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5052-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=tOp4jZDx; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=139.178.88.99; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5052-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [139.178.88.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRFL1zrSz4w2R for ; Tue, 12 Nov 2024 10:53:10 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 9631D280EBF for ; Mon, 11 Nov 2024 23:53:11 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5E5601CC8AC; Mon, 11 Nov 2024 23:49:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="tOp4jZDx" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71D901CBE9B for ; Mon, 11 Nov 2024 23:49:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368942; cv=none; b=Pb/2P23aE7ChYLoFtIRhbd5bnabUelHxHv0HjdY38m8XzHmjTLc3Am8oVKaYUWXi2M/ARC60aZuD5X3j5jK0V4S6TdppEhn8eRG6q+hnifWDi3B1cvCTvCIztP9lMUTWyXJ6X2g1lvSYF3E4Chny1VcSQ6FwQs8NG/jwx5bODAg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368942; c=relaxed/simple; bh=8kwh8LC4vBsecO/5vBRProS5/OW+UQX7WE2AOBiJLS4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CjmNwsL7qSoW/O91NYSPmNb6aEX3p24u0QgQNuke2t91Nng/TyvvL1APmVjnYzi9fjqP2yPl29lk1B5pZ/1HC6/vSZLcimopwE9FsehiGCWWV61ELfEnL2k1RN9fiVUsiOI37hdK+voycxuRoKm1Y66XBKxOFD9i7RfZR+ce5v0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=tOp4jZDx; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-72041ff06a0so4025366b3a.2 for ; Mon, 11 Nov 2024 15:49:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368940; x=1731973740; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; b=tOp4jZDxaeUTDvkObhwW5GNiOlknWEyM+kvQ+QJS66nzG4Jja09usUHfB/NOpcS/XJ rYo0jcdLD5Gqn/PVbuJxz8U7c3IUyPHb41RQeQIF2rnZWxLgskMUfbEvOV2P1xg7lJIR mZWHDfWK/UYuj7R0I9LHDBeJHThXWDZHbhILKkuK16nuvEZXVw7xBtLtQ9fmZ8CkuBNM 876cFqOAqrdy/aYLxzOY1GMNhMU14Uy4h/3fJvqdQD3WUkQkWekLC+InYU4j5pcMAxlg 8H4WskZPeu/oz+1+/3XmFPFv8dPTnDQwaiDznEb7lj9s9GdizAK8sF+C9PIexMSAI6JI YGjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368940; x=1731973740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNT2xlrRbXw1IZWpIgVp1lVCBQhpFw800CH7QKwlGio=; b=waMIMEOsKdkm117d0ZvZSNtzHsd9IQPlw2si2KwDpor4XtoiEm22j1nytpPpjj5/UZ BuH2Oq1rv53neuF9sNVpXqhsLgc1wM5i2zoK6UsmHvUYyNKMgIWEQT+W+jn0HJj2Qwv9 oCQCCoeabSZJX1BmylObbJXq1I5/Uferh1DjlBgYosC8U8ZiKBiiMo/mSF+xBsxzn99a G/PUrj6rPe3Ny4BZqHsV3JiT904UsD8mcgPuTJQh9i4AjKjAsWyRSmB3Lfam2MCCF7dA PM2WCGX4Z5NpLzJSdeRgDRr+gBk3aWgBAzGt41hXn8+u0C6+GrIWp++kzpNYRSwCUzyb 8kIg== X-Forwarded-Encrypted: i=1; AJvYcCUs+NiAHsUDvxg8uMZW7FfazyHovdFHybf4h7OaDGhL63Jq/7t4FQ4ifVYg1j+/cmmjqSFkaxrMcO9H@vger.kernel.org X-Gm-Message-State: AOJu0Yxondd8AGktcz05bQX9eZgZBeYZWY895c2UHswFIORLuE1U8tK2 oHw4XQR5bHc2OEC/B+lCk4j5xuht2YdAwqiJPhwwOPAeEsTVmLJ1dwMXSmg2BQg= X-Google-Smtp-Source: AGHT+IErnoZ29qXFIn5gwmYcaGsN1FZzl0JZhBUfjT+VcEVWLi3eokc1bOzqdTtbaNvb8n8ixmr53A== X-Received: by 2002:aa7:888c:0:b0:71e:cf8:d6f1 with SMTP id d2e1a72fcca58-724132c4d71mr19840990b3a.14.1731368939764; Mon, 11 Nov 2024 15:48:59 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:48:58 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 08/16] mm/filemap: add read support for RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:35 -0700 Message-ID: <20241111234842.2024180-9-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Add RWF_UNCACHED as a read operation flag, which means that any data read wil be removed from the page cache upon completion. Uses the page cache to synchronize, and simply prunes folios that were instantiated when the operation completes. While it would be possible to use private pages for this, using the page cache as synchronization is handy for a variety of reasons: 1) No special truncate magic is needed 2) Async buffered reads need some place to serialize, using the page cache is a lot easier than writing extra code for this 3) The pruning cost is pretty reasonable and the code to support this is much simpler as a result. You can think of uncached buffered IO as being the much more attractive cousing of O_DIRECT - it has none of the restrictions of O_DIRECT. Yes, it will copy the data, but unlike regular buffered IO, it doesn't run into the unpredictability of the page cache in terms of reclaim. As an example, on a test box with 32 drives, reading them with buffered IO looks as follows: Reading bs 65536, uncached 0 1s: 145945MB/sec 2s: 158067MB/sec 3s: 157007MB/sec 4s: 148622MB/sec 5s: 118824MB/sec 6s: 70494MB/sec 7s: 41754MB/sec 8s: 90811MB/sec 9s: 92204MB/sec 10s: 95178MB/sec 11s: 95488MB/sec 12s: 95552MB/sec 13s: 96275MB/sec where it's quite easy to see where the page cache filled up, and performance went from good to erratic, and finally settles at a much lower rate. Looking at top while this is ongoing, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7535 root 20 0 267004 0 0 S 3199 0.0 8:40.65 uncached 3326 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd4 3327 root 20 0 0 0 0 R 100.0 0.0 0:17.22 kswapd5 3328 root 20 0 0 0 0 R 100.0 0.0 0:13.29 kswapd6 3332 root 20 0 0 0 0 R 100.0 0.0 0:11.11 kswapd10 3339 root 20 0 0 0 0 R 100.0 0.0 0:16.25 kswapd17 3348 root 20 0 0 0 0 R 100.0 0.0 0:16.40 kswapd26 3343 root 20 0 0 0 0 R 100.0 0.0 0:16.30 kswapd21 3344 root 20 0 0 0 0 R 100.0 0.0 0:11.92 kswapd22 3349 root 20 0 0 0 0 R 100.0 0.0 0:16.28 kswapd27 3352 root 20 0 0 0 0 R 99.7 0.0 0:11.89 kswapd30 3353 root 20 0 0 0 0 R 96.7 0.0 0:16.04 kswapd31 3329 root 20 0 0 0 0 R 96.4 0.0 0:11.41 kswapd7 3345 root 20 0 0 0 0 R 96.4 0.0 0:13.40 kswapd23 3330 root 20 0 0 0 0 S 91.1 0.0 0:08.28 kswapd8 3350 root 20 0 0 0 0 S 86.8 0.0 0:11.13 kswapd28 3325 root 20 0 0 0 0 S 76.3 0.0 0:07.43 kswapd3 3341 root 20 0 0 0 0 S 74.7 0.0 0:08.85 kswapd19 3334 root 20 0 0 0 0 S 71.7 0.0 0:10.04 kswapd12 3351 root 20 0 0 0 0 R 60.5 0.0 0:09.59 kswapd29 3323 root 20 0 0 0 0 R 57.6 0.0 0:11.50 kswapd1 [...] which is just showing a partial list of the 32 kswapd threads that are running mostly full tilt, burning ~28 full CPU cores. If the same test case is run with RWF_UNCACHED set for the buffered read, the output looks as follows: Reading bs 65536, uncached 0 1s: 153144MB/sec 2s: 156760MB/sec 3s: 158110MB/sec 4s: 158009MB/sec 5s: 158043MB/sec 6s: 157638MB/sec 7s: 157999MB/sec 8s: 158024MB/sec 9s: 157764MB/sec 10s: 157477MB/sec 11s: 157417MB/sec 12s: 157455MB/sec 13s: 157233MB/sec 14s: 156692MB/sec which is just chugging along at ~155GB/sec of read performance. Looking at top, we see: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7961 root 20 0 267004 0 0 S 3180 0.0 5:37.95 uncached 8024 axboe 20 0 14292 4096 0 R 1.0 0.0 0:00.13 top where just the test app is using CPU, no reclaim is taking place outside of the main thread. Not only is performance 65% better, it's also using half the CPU to do it. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++-- mm/swap.c | 2 ++ 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 02d9cb585195..3d0614ea5f59 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2474,6 +2474,8 @@ static int filemap_create_folio(struct kiocb *iocb, folio = filemap_alloc_folio(mapping_gfp_mask(mapping), min_order); if (!folio) return -ENOMEM; + if (iocb->ki_flags & IOCB_UNCACHED) + __folio_set_uncached(folio); /* * Protect against truncate / hole punch. Grabbing invalidate_lock @@ -2519,6 +2521,8 @@ static int filemap_readahead(struct kiocb *iocb, struct file *file, if (iocb->ki_flags & IOCB_NOIO) return -EAGAIN; + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_async_ra(&ractl, folio, last_index - folio->index); return 0; } @@ -2548,6 +2552,8 @@ static int filemap_get_pages(struct kiocb *iocb, size_t count, return -EAGAIN; if (iocb->ki_flags & IOCB_NOWAIT) flags = memalloc_noio_save(); + if (iocb->ki_flags & IOCB_UNCACHED) + ractl.uncached = 1; page_cache_sync_ra(&ractl, last_index - index); if (iocb->ki_flags & IOCB_NOWAIT) memalloc_noio_restore(flags); @@ -2595,6 +2601,20 @@ static inline bool pos_same_folio(loff_t pos1, loff_t pos2, struct folio *folio) return (pos1 >> shift == pos2 >> shift); } +static void filemap_uncached_read(struct address_space *mapping, + struct folio *folio) +{ + if (!folio_test_uncached(folio)) + return; + if (folio_test_writeback(folio)) + return; + if (folio_test_clear_uncached(folio)) { + folio_lock(folio); + folio_unmap_invalidate(mapping, folio, 0); + folio_unlock(folio); + } +} + /** * filemap_read - Read data from the page cache. * @iocb: The iocb to read. @@ -2706,8 +2726,12 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, } } put_folios: - for (i = 0; i < folio_batch_count(&fbatch); i++) - folio_put(fbatch.folios[i]); + for (i = 0; i < folio_batch_count(&fbatch); i++) { + struct folio *folio = fbatch.folios[i]; + + filemap_uncached_read(mapping, folio); + folio_put(folio); + } folio_batch_init(&fbatch); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); diff --git a/mm/swap.c b/mm/swap.c index b8e3259ea2c4..542f298d3dcd 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -472,6 +472,8 @@ static void folio_inc_refs(struct folio *folio) */ void folio_mark_accessed(struct folio *folio) { + if (folio_test_uncached(folio)) + return; if (lru_gen_enabled()) { folio_inc_refs(folio); return; From patchwork Mon Nov 11 23:37:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009966 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=YTNRgjZ/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=jq5u=sg=vger.kernel.org=linux-ext4+bounces-5053-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRGS2Pkmz1xyG for ; Tue, 12 Nov 2024 10:54:08 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRGP4shxz4w2R for ; Tue, 12 Nov 2024 10:54:05 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRGP4qCnz4x11; Tue, 12 Nov 2024 10:54:05 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=147.75.80.249 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369242; cv=pass; b=ratEzOHPNk5L6knrqZGUcVRs4nZUBf+cnkw3HZqNj6r1EjtDnetfkv6rD+YU9V7n5HnA2k40rZ2nNzYXzOaM/XMaLY5mV1fPlwcf02yq4hm5+GLgN2tbsxG7KQT6ry0Zaz7xrOcBddHinD4k3wN+rsLnsvg06n0kxwNfKvXJXlEoMBtmKZDWgi0o/CShn9a9wCB7uVCorJqqogSbLxIWjcuIyquHelatdXwEWfRcCmR35UaQ9F89fwTcijEK7ECLT5r6+9RAsalCiPy5wml1JkhXMxVM7w3RBmYJi8+3FM6YTP3P0VJs02UnGn72t/Yd9bpI4IN+vybBZpCwdFm3Hw== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369242; c=relaxed/relaxed; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TD8yGqfL/DitNSDNEwy8uMp+lHm8QJu/MVm5SrnVyX01b24quBZmHEAslPBVRcURKvyy1U67Le6EIcCP6qH0Gar0OGIpm7YLGE8tfYY4TXqDGNdpfcTuFG6l8jUk7iwH8zggdHwv3tc51kkGblxa2F3MGCSRvCmEM0YaQoGi8PkeHieRB04HLwXtnu6s2uAMSTtB27I8AmlSFzg1VuTwKO3KD9sVxSJnYbbbTSyRIfPRlQJsexL325fpXDTPX0CryRRA23VtWzLg8Fit6cK2BXKx3QO9fuDyHnKbNWn2zq4Y0iBGNKJw6UBrYlTMuT0NWgzwFYebMzgFuyYDUjRp0A== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=YTNRgjZ/; dkim-atps=neutral; spf=pass (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5053-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=YTNRgjZ/; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5053-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [147.75.80.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRGK6Sr2z4w2R for ; Tue, 12 Nov 2024 10:54:01 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 390201F226BE for ; Mon, 11 Nov 2024 23:54:02 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B998C1CC88A; Mon, 11 Nov 2024 23:49:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="YTNRgjZ/" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBB4C1CC888 for ; Mon, 11 Nov 2024 23:49:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368944; cv=none; b=RfYgTdup5FqREogwKVsLjWTORgqIZ0RwfvcZ+3Oe6o32j59rsmFdYUhPGt8S6YW3/nNmkXlG1LXqjC0iYbwCjNMlvFvK7rg26mxQBhXIhJO9Mx08jTX6QSa3KuW5SLleTD7bPE3TD8pqv8Medivl+yBWVDYJfwcSJs3IyUqEEoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368944; c=relaxed/simple; bh=MmJ6uQz4zS+zNMk4WrnYAN8q8MiBRG7hHBRj4kFfBpY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GBRwKHRXfOC6rZ+TYTMrel2BUAHIhqm8mOJ14st0YNNTgfesZxtGqPqq4ULddhQZY72DaEJngDntguw64wZlDatSPtpWwgUa4L/259Ir/4bm8HctNbz8dUQoaOF++duBJLy1j/vsdn0Pgmbf2H0YGNfMLhY2JUz3s6YegTpIdSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=YTNRgjZ/; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-71e592d7f6eso3759705b3a.3 for ; Mon, 11 Nov 2024 15:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368941; x=1731973741; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=YTNRgjZ/MujhL1xey9f4SDncEWX7de/DW5gFOJgg4nNqo8XNnKioHjJtWjDEDKVrBT +OTMRvJBUxmHACUDseiSgz2Dt3mRgyFr2zaziavJ4bP+8RMrgxwXC/0CHD25DZMJfILu lOUn5A0jpHBfCc0aGLijfpiFvavcMiorqdi0Qjj28n73+8p9j7oOQoVc49EPBCAs20PM eLjl+84GPAk8pawLiXuOUe8k47oW0EM1dZK//nSqFMZeU59vdwpAV4EcFQS9eFIrGBPm KEfwaiuuhyd0wlLJv+/n0Ay+v0LDQghf8eL1pEmWs//NHHOlx5I8F5HyGeugU1lKSLHM xcuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368941; x=1731973741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C5BmZ07D5DYJBYmt5fURgbXo4UgCrxSfGObLVz7UJ+Y=; b=Lf3gazRkvg61aV+rw6yRauz5wIvzr7pCeTMDOYuvAFzEbSwPO+RykAu1mIjl4v0xlA DqKJPKI/fdyrqSKqgMccmqkfzFwUhMZiwI73iyZ4672BI0j7pZFxfIe/TVn/9tQZY2df T+7NMHfkxnGqoYEH75yjAY1xtbsF2bjCbYD0JZXABavQKe2bvGUp+dGScDvQZEKKavoZ KI/4nWXORO2b93Tne1y1L1fLFigWod2jQnLdr0NXwt70qL4nvKaEU1AAx+dv/bHAVIPb nPdh7zAQOKyGkxNu8jXRyV9AB+i6113EOrPggrsF5900Kl5lxgF+4XapeGXldmjeQpB4 HnmQ== X-Forwarded-Encrypted: i=1; AJvYcCVAEB9W/qB2xXQ36tVKiYVDOlNlbgBTrbgQDQ27qc+BxK06PV1MIA4d7Q1SxXY1gK84TejZ250nDWit@vger.kernel.org X-Gm-Message-State: AOJu0YxRswIEUNrgHdVPRe7XgGCmojzZ6r74VQIcq36OwJagg5/Klz/j P7u3n4a1ATDzCsQK8oXMfVmD/Kw6hHdS6UbirA/WKkbrSWr3EyfV7vGtDWhkxNc= X-Google-Smtp-Source: AGHT+IHVAuYVOMfDSGlPxbnuC87wShBLb7OXW58ajqvAinndw3wlptYvFjbRCeInYbnH7Qo2aVFY4A== X-Received: by 2002:a05:6a21:9101:b0:1db:e3c7:9974 with SMTP id adf61e73a8af0-1dc228ef1damr21176002637.12.1731368941302; Mon, 11 Nov 2024 15:49:01 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:00 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 09/16] mm/filemap: drop uncached pages when writeback completes Date: Mon, 11 Nov 2024 16:37:36 -0700 Message-ID: <20241111234842.2024180-10-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 3d0614ea5f59..40debe742abe 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1600,6 +1600,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_UNCACHED writeback, but + * can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1610,6 +1631,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1631,9 +1654,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (folio_test_uncached(folio) && folio_test_clear_uncached(folio)) + folio_uncached = true; if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback); From patchwork Mon Nov 11 23:37:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009978 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=z93zQai8; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=uabo=sh=vger.kernel.org=linux-ext4+bounces-5054-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRw26TsXz1xyG for ; Tue, 12 Nov 2024 11:23:14 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRvy3cz3z4x11 for ; Tue, 12 Nov 2024 11:23:10 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRvy3P2qz4x2J; Tue, 12 Nov 2024 11:23:10 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=147.75.48.161 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731370987; cv=pass; b=wejK28DxvmR3JcwkmE/TuAo1Vd3kie2HlOLzB0LpkvRs+Haqw8xnf71ZeSI4stvg+wQWa9yhjxJx9dYcD529FEYpUxOAy7eWAPoSTxvQDKLW6yCEMjMtg9C2Vxm18QQpE+CqU08eQY3cLuI7GJ4Wr5J0c1uUzqG1vrPEN4+BDbrNqJYevGkgDZmwYcCNOxtM+kxY03aK025FihqrYdIpe/hf/PEYB61x+sBbV3sLKwJs5VsKalIqaI1uCk3jIbv+ejymoioiuOR6s4ETbyEWkZJpYE8amwpMTL9r7d89YrRuVM4UXy7RxP9V822kESarqh76UqOwWllO0Sqg2xWaMw== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731370987; c=relaxed/relaxed; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aOXz8qlo0VS9gfaUDpAVTzZ7pv9VYQzFhUHZfM0G3GD+UsBiFrJGzFQ+845iuc1TuEk2xeMpw7/isd9cDMxN0i8hcINOcG9GY/KRJp+ZE7GaeWiOCTL8Zw0OEOmq8WCHfpqoEuK3DxP5hkvRepaLrPCg+WyYSJhPtEn1w4oqBv6RcoP8PluTEdCx9pQBsvWM060UfWfndp59obNLo0/fp96+6levlryu6wJ0BfKnapDDMUshR2mMv8E9ItR3IDddLV2mJY2aNlQclUQ1RG5S2QmVAJbmeXrmr0qO1MlBT9II4HUaX94M8ZD6LuTTmlwtz+skHqNs4uqcc0XhyVplQw== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=z93zQai8; dkim-atps=neutral; spf=pass (client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5054-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=z93zQai8; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5054-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [147.75.48.161]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRvv1X1Yz4x11 for ; Tue, 12 Nov 2024 11:23:07 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 004EDB2C8B5 for ; Mon, 11 Nov 2024 23:54:22 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9F3AE1CDFBD; Mon, 11 Nov 2024 23:49:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="z93zQai8" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B27AF1CBE9F for ; Mon, 11 Nov 2024 23:49:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368945; cv=none; b=uT6BLuUqOd/8nV40rTcJsQSdJgOPzrua/EsLnaLlUETOZq8/nFDyDtkj7N92aH3PmXdQmWYSCCj7aWSjk4rXm8AZgYa1nC0eJphwYsnKkHaQrjm56NEGKWSUlgs9V+MnR4e2xDGwruHhcDzIK9cC90kENXydf4WZxPGMI+ON+00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368945; c=relaxed/simple; bh=0ZmHSWGFJPGRrfbaLjT1JKVVHDI+Z6ojoSVY2VwoI0U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Peoc5CKveP0zHuWUqhGUuFcZoTKfOW6fhY+c9G8ZPnuIErstCm2JJw0GMs0IRZomYfZ06iDPC7nKJaI/fwBUktjfgV8GHRdxjRyraVqdMepqmVwZHd+TI3TgpkWFdVlPpyEGaSxkMATEGH29fu+LN3leoORiG44RNiZ24izuZuo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=z93zQai8; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-723f37dd76cso5169064b3a.0 for ; Mon, 11 Nov 2024 15:49:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368943; x=1731973743; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; b=z93zQai8EpIDgD0jRAEwLB46HhfxNTDRich3xQOHrqHEJG5auElXfawUzoyo49tXVl SdcnVtshzSvFh/Xsz7tm7NXASeQuHaCfD2xrlIGRSvHx9PCnpda0OJxaH7rnsSPkPNie agc3KKChj1DU+YZs3+6rvxEw6n/Y41cLyUPlal5eoI4Ci+u2J4eOaYuuslRPPd2w9xVC YC8DH9rN1UwT8CIGTVmM/1KJwWytLIo+n92UC22ppiJYn7Ja40/a7UX82i9ve7kc5YhM zwSZbZX8U0ztvWOHyeYGyd79SesUdvtq+VlkvmU2nbd/DGPxN/XWVoN+1zWewLvEtEOv HOEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368943; x=1731973743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=34CEBvu7gjOMz8fnHdZ3R+2IdL/8ao7S7M5vWTfYjmM=; b=B0OXhz2HCA3eH6VjfcwpQiSOvXNo6Lus81QmNpq+N7kCPB6Rteh7Dc8mpm+LUbGp17 bOww3V8l/4j55zGEDF8SKK73p7JsJLO9olPZP1evjHh2xNXpZfHAnDOuZT+OU0nkv9pv kcpgXVNHd0LajH6RuCovDTet0b+dJkPQ7AnWsDn1O60ClQmyd1ghiHbKvoLWYPR3FUF4 0yOjRCH58N4XNVBSdRrtlluBwTB/TGEAzUDK96NGmqfdXu9V0ZbvlCHA4HoLvzktjlF7 kd4W9n1geg8joQwB+yKBBpMaEpkqvhT8/YO+PfBzyWNEkeFDurdq7lGKhV6/8n0uhfzR QeFg== X-Forwarded-Encrypted: i=1; AJvYcCWTBhJ2Poa9mgGU62OME1FbJydT1EBrWJG4qFF6qXMG70SU6k15aZSFJSzyGmgPz5ZUfczsp8L4GTbi@vger.kernel.org X-Gm-Message-State: AOJu0YyQp2DO5B2j1apJOaIdKIBQraZBK0mtN74+cvQ36/Mk1Bfk6W+K nvm5K6zQWngPfjLr6KErmbMaeePfRS9g5RGImuXyrqsDqo3A1dLgX9PvMOrKWIWgNbKxdRiaRaQ jDho= X-Google-Smtp-Source: AGHT+IH/Zn0FG6bgzME+vU3LW7llTjZ8LDSnQYNIFX21AVqsAPSSvTFEg9AexjHSBcXEhmlzou7m5w== X-Received: by 2002:a05:6a20:7292:b0:1db:e82f:2a63 with SMTP id adf61e73a8af0-1dc228c6973mr20790006637.3.1731368942803; Mon, 11 Nov 2024 15:49:02 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:02 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 10/16] mm/filemap: make buffered writes work with RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:37 -0700 Message-ID: <20241111234842.2024180-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org If RWF_UNCACHED is set for a write, mark new folios being written with uncached. This is done by passing in the fact that it's an uncached write through the folio pointer. We can only get there when IOCB_UNCACHED was allowed, which can only happen if the file system opts in. Opting in means they need to check for the LSB in the folio pointer to know if it's an uncached write or not. If it is, then FGP_UNCACHED should be used if creating new folios is necessary. Uncached writes will drop any folios they create upon writeback completion, but leave folios that may exist in that range alone. Since ->write_begin() doesn't currently take any flags, and to avoid needing to change the callback kernel wide, use the foliop being passed in to ->write_begin() to signal if this is an uncached write or not. File systems can then use that to mark newly created folios as uncached. Add a helper, generic_uncached_write(), that generic_file_write_iter() calls upon successful completion of an uncached write. This provides similar benefits to using RWF_UNCACHED with reads. Testing buffered writes on 32 files: writing bs 65536, uncached 0 1s: 196035MB/sec 2s: 132308MB/sec 3s: 132438MB/sec 4s: 116528MB/sec 5s: 103898MB/sec 6s: 108893MB/sec 7s: 99678MB/sec 8s: 106545MB/sec 9s: 106826MB/sec 10s: 101544MB/sec 11s: 111044MB/sec 12s: 124257MB/sec 13s: 116031MB/sec 14s: 114540MB/sec 15s: 115011MB/sec 16s: 115260MB/sec 17s: 116068MB/sec 18s: 116096MB/sec where it's quite obvious where the page cache filled, and performance dropped from to about half of where it started, settling in at around 115GB/sec. Meanwhile, 32 kswapds were running full steam trying to reclaim pages. Running the same test with uncached buffered writes: writing bs 65536, uncached 1 1s: 198974MB/sec 2s: 189618MB/sec 3s: 193601MB/sec 4s: 188582MB/sec 5s: 193487MB/sec 6s: 188341MB/sec 7s: 194325MB/sec 8s: 188114MB/sec 9s: 192740MB/sec 10s: 189206MB/sec 11s: 193442MB/sec 12s: 189659MB/sec 13s: 191732MB/sec 14s: 190701MB/sec 15s: 191789MB/sec 16s: 191259MB/sec 17s: 190613MB/sec 18s: 191951MB/sec and the behavior is fully predictable, performing the same throughout even after the page cache would otherwise have fully filled with dirty data. It's also about 65% faster, and using half the CPU of the system compared to the normal buffered write. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 29 +++++++++++++++++++++++++++++ mm/filemap.c | 17 +++++++++++++++-- 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d55bf995bd9e..d35280744aa1 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -14,6 +14,7 @@ #include #include #include /* for in_interrupt() */ +#include #include struct folio_batch; @@ -70,6 +71,34 @@ static inline int filemap_write_and_wait(struct address_space *mapping) return filemap_write_and_wait_range(mapping, 0, LLONG_MAX); } +/* + * generic_uncached_write - start uncached writeback + * @iocb: the iocb that was written + * @written: the amount of bytes written + * + * When writeback has been handled by write_iter, this helper should be called + * if the file system supports uncached writes. If %IOCB_UNCACHED is set, it + * will kick off writeback for the specified range. + */ +static inline void generic_uncached_write(struct kiocb *iocb, ssize_t written) +{ + if (iocb->ki_flags & IOCB_UNCACHED) { + struct address_space *mapping = iocb->ki_filp->f_mapping; + + /* kick off uncached writeback */ + __filemap_fdatawrite_range(mapping, iocb->ki_pos, + iocb->ki_pos + written, WB_SYNC_NONE); + } +} + +/* + * Value passed in to ->write_begin() if IOCB_UNCACHED is set for the write, + * and the ->write_begin() handler on a file system supporting FOP_UNCACHED + * must check for this and pass FGP_UNCACHED for folio creation. + */ +#define foliop_uncached ((struct folio *) 0xfee1c001) +#define foliop_is_uncached(foliop) (*(foliop) == foliop_uncached) + /** * filemap_set_wb_err - set a writeback error on an address_space * @mapping: mapping in which to set writeback error diff --git a/mm/filemap.c b/mm/filemap.c index 40debe742abe..0d312de4e20c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -430,6 +430,7 @@ int __filemap_fdatawrite_range(struct address_space *mapping, loff_t start, return filemap_fdatawrite_wbc(mapping, &wbc); } +EXPORT_SYMBOL_GPL(__filemap_fdatawrite_range); static inline int __filemap_fdatawrite(struct address_space *mapping, int sync_mode) @@ -4076,7 +4077,7 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) ssize_t written = 0; do { - struct folio *folio; + struct folio *folio = NULL; size_t offset; /* Offset into folio */ size_t bytes; /* Bytes to write to folio */ size_t copied; /* Bytes copied from user */ @@ -4104,6 +4105,16 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i) break; } + /* + * If IOCB_UNCACHED is set here, we now the file system + * supports it. And hence it'll know to check folip for being + * set to this magic value. If so, it's an uncached write. + * Whenever ->write_begin() changes prototypes again, this + * can go away and just pass iocb or iocb flags. + */ + if (iocb->ki_flags & IOCB_UNCACHED) + folio = foliop_uncached; + status = a_ops->write_begin(file, mapping, pos, bytes, &folio, &fsdata); if (unlikely(status < 0)) @@ -4234,8 +4245,10 @@ ssize_t generic_file_write_iter(struct kiocb *iocb, struct iov_iter *from) ret = __generic_file_write_iter(iocb, from); inode_unlock(inode); - if (ret > 0) + if (ret > 0) { + generic_uncached_write(iocb, ret); ret = generic_write_sync(iocb, ret); + } return ret; } EXPORT_SYMBOL(generic_file_write_iter); From patchwork Mon Nov 11 23:37:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009968 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=DiY4ayqz; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=io7q=sg=vger.kernel.org=linux-ext4+bounces-5055-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRHP4f9lz1xyG for ; Tue, 12 Nov 2024 10:54:57 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRHM01N5z4w2L for ; Tue, 12 Nov 2024 10:54:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRHL7566z4w2R; Tue, 12 Nov 2024 10:54:54 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:4601:e00::3" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369291; cv=pass; b=e390xY8RnVreyPOZnXwxirvoWJs38BXuJCs7Wg6yg3p1VAGCS/sWImipWS9X0ASCeOemakUwWTrEKujUmeRQm2DfxSY4FT+AEK8JshGJFby8OS07IVXetH5YKg2B0V9Ly7u7RPbWA5/jQvARYmYLsphyWkd4eBIE7URuuw9pA2QDiu8qkbx68zBjawxd/mcJvUavPNqK8auzQ7PChugBoGM2njrOXbEYbYgEcm7dYw200tslw2fGmfnqiTCHugNhL7gNU5p3JD/aZZGdxDNCxTCAbOJIyTvyKEvdv/xPYhW1I03eXU2O2fZlVv9ZN+Ok6sWRijxk1Rxa55aKEbstDw== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369291; c=relaxed/relaxed; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TcSPDu2Ncy+2iYiw518ezwwpQ3eaMFW+6edWsjpAx3yJZar8e6cUrk2TIob+02j2zxRy8VsFyp6gldEBqAhAaL6Bi7kE8vXeizkjWB9AvkMU4PvBvDt9yFzdb6VXDUQd21JMZkmsmezrOR5owN558bH79nmHIUH9jR+bEmWBquJayClb2BFx3MlSxZt1AcpUvMlWuDb6kK6CNxW5drXdj7N1OPCJf4tClGRemRlFWVZCKhR2H/JCCUyqmLlAJqvUlyh4BFV0NUDFg2FBjWX1Uus9eUUWiUoFerxx5mT8vpv7MA6wzj8Kkajbe1EFLAYVMnT1Z6Fl6lMp0A8b2Ocqgw== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=DiY4ayqz; dkim-atps=neutral; spf=pass (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5055-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=DiY4ayqz; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5055-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRHH1bSXz4w2L for ; Tue, 12 Nov 2024 10:54:51 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id AF80C1F2222D for ; Mon, 11 Nov 2024 23:54:51 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 669AF1CDA1E; Mon, 11 Nov 2024 23:49:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="DiY4ayqz" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4240C1CDA2D for ; Mon, 11 Nov 2024 23:49:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368947; cv=none; b=EaY/eSWQEaSeYMZcDqcw775vxstfBK0W4Ta0RVg5JPxFyJLGDo4PntQ9nHrEueKisnwOWmwZrYb2iYkT0VyzYFYMwVp93+/KuDKGYMx/37alKlr5HYCnH6r8stoz/s/N1ixNE/RDwm9dZAyYG0Hn+vWjRp4VP3mk6SKpvnG0eqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368947; c=relaxed/simple; bh=g/haD3K2EU2Y4QS833/TY88VMz3RgG7lve6et2tjW0k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YNewXC5ppk0/3fCIoCeAqsdPJ2k13z9jhDTBo1TrJgKv/PRU49wWzsrcojql2/t0Ni85dRF6YfQFaN+NorBbPhtlGbYcHpAOmWtMbscIydLnCSqIDGAWISZ3SyICWSbuBBgKGKBlSOCbNdZ+BC60idY1i7dSmNKn4AohyCSGR4c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=DiY4ayqz; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-71ec997ad06so4154536b3a.3 for ; Mon, 11 Nov 2024 15:49:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368945; x=1731973745; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; b=DiY4ayqzzDxIFKJRkM05e6VspCONzC/qGzPdoWEEbQfWMJMgiLpMZD3swUijZxqMAE IUWhkquDG2fMenZuNEbnc5+vNuxgqeD4jnb8WMIJtxWgzd5gfYmZs16g1lV4iKVXWrpJ KvWLo+QQRV7Bf55axkLLjyJ5MyS5bRNb8h0WxNG2r63B8ejocaaSTK8oaxTb94Js+u04 umkCd6PEWkI58RYrfgXO4Bbhobj3XCjIsF2c2KhLiRRh0yz6bxaZ6CzevDMTKKsIggxg 6qYDUIv4aM/5tjGXcexy4XjGVEpG7mlil7fwaF8d89jGv+3Xmy1l3Cmjf4lYJnthgzQ2 6TrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368945; x=1731973745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MuWHSESvZxScxt1gtQJTtWY+NoQXodLd0kknM0wzZSI=; b=jE1OrSTiX68qUb/bq9EieqYmTxTuqKahn64T1paGC0gdgFIpvxDtjSTWj7zPnhqrPu P4zmSJ/21QrIAfEc9m/yPFfej7aIsCDhI0Z85BLWuKnW8RAGGYMKr5telTF7Zzk302+o BmBxPDdL2jxrvG5DoHx3Izy63meXf4to3BDYRGeHDHu1kJ/d11D8Og/RplWtYQMDjLZo fKjw8e7gITtJcq/PlMYtO5eDpOWMW1BTW1ahcN+GzzfKdG0eiPdnDbT/VHSRBDkdk3R6 jwUL8AcI6vDg9Hqmmw4wFoCo9cqwWgk3kVjQVWckJQdNWhzblqiCCj5ij/gMxLkG4mCQ zWLw== X-Forwarded-Encrypted: i=1; AJvYcCWy7R4bgumrvzNF+T9ySR7zxZ3+fwzZbAP6Vh++0YdyqtMNVbwOrwZ0w5sFTc1GcDqlN8aR5f2hqw9n@vger.kernel.org X-Gm-Message-State: AOJu0YzxJpzdzoQ7DLRkLy0jMQvrpfvEdEnfX9TYRpmn+DFjBtHB8H9c rWrVv6jRIfvjt50w7PdV2B3qTayaPYP71nmYAnDYZkaikOej9ce+opSXNad8dhU= X-Google-Smtp-Source: AGHT+IGY8U2EPa5TOjOCgJt8zYCC4on7T5T+FZCdrixkNsVQpkNRkQsZMbpHQUFVB4Pf9fNlDncpCQ== X-Received: by 2002:a05:6a00:22d2:b0:71e:755c:6dad with SMTP id d2e1a72fcca58-7244a4fdea0mr1056066b3a.5.1731368944673; Mon, 11 Nov 2024 15:49:04 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:04 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 11/16] mm: add FGP_UNCACHED folio creation flag Date: Mon, 11 Nov 2024 16:37:38 -0700 Message-ID: <20241111234842.2024180-12-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Callers can pass this in for uncached folio creation, in which case if a folio is newly created it gets marked as uncached. If a folio exists for this index and lookup succeeds, then it will not get marked as uncached. If an !uncached lookup finds a cached folio, clear the flag. For that case, there are competeting uncached and cached users of the folio, and it should not get pruned. Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 2 ++ mm/filemap.c | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index d35280744aa1..0b298e81fcae 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -741,6 +741,7 @@ pgoff_t page_cache_prev_miss(struct address_space *mapping, * * %FGP_NOFS - __GFP_FS will get cleared in gfp. * * %FGP_NOWAIT - Don't block on the folio lock. * * %FGP_STABLE - Wait for the folio to be stable (finished writeback) + * * %FGP_UNCACHED - Uncached buffered IO * * %FGP_WRITEBEGIN - The flags to use in a filesystem write_begin() * implementation. */ @@ -754,6 +755,7 @@ typedef unsigned int __bitwise fgf_t; #define FGP_NOWAIT ((__force fgf_t)0x00000020) #define FGP_FOR_MMAP ((__force fgf_t)0x00000040) #define FGP_STABLE ((__force fgf_t)0x00000080) +#define FGP_UNCACHED ((__force fgf_t)0x00000100) #define FGF_GET_ORDER(fgf) (((__force unsigned)fgf) >> 26) /* top 6 bits */ #define FGP_WRITEBEGIN (FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE) diff --git a/mm/filemap.c b/mm/filemap.c index 0d312de4e20c..0949f0f340f5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1985,6 +1985,8 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, /* Init accessed so avoid atomic mark_page_accessed later */ if (fgp_flags & FGP_ACCESSED) __folio_set_referenced(folio); + if (fgp_flags & FGP_UNCACHED) + __folio_set_uncached(folio); err = filemap_add_folio(mapping, folio, index, gfp); if (!err) @@ -2007,6 +2009,9 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, if (!folio) return ERR_PTR(-ENOENT); + /* not an uncached lookup, clear uncached if set */ + if (folio_test_uncached(folio) && !(fgp_flags & FGP_UNCACHED)) + folio_clear_uncached(folio); return folio; } EXPORT_SYMBOL(__filemap_get_folio); From patchwork Mon Nov 11 23:37:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009969 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=vE3j3An1; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=kvft=sg=vger.kernel.org=linux-ext4+bounces-5056-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRJ64K5Jz1xwV for ; Tue, 12 Nov 2024 10:55:34 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRJ337mBz4w2L for ; Tue, 12 Nov 2024 10:55:31 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRJ332f6z4wbR; Tue, 12 Nov 2024 10:55:31 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=147.75.80.249 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369328; cv=pass; b=G5+I5raksV+wUBNfRENOjcSuyO5GcTvQUnGPj9Tit9om/nYDey2fX8OOf6McfgkBTMnQAdY/XQ7M33Zf7wk4kIjg5PQo8eOUTeWUVzorZmDFHRwCY/YGMrLAz/Bdmsuy87OOcjF3kssxsLZ7QLAAzaYrNcEG538Op7vOw8f/ZzrENkvwyIfJ7oM1Hkyzk2mN2Fy/HBV6s0xKSEibuPj6OAvfqa7KXJ2uG3JKwb0UXhfgMDOKCdxIXfypC5d2hrpK+OXCZBJI+2yV/EW2PivNlflnYMDvyK3Pqsq25QZwl95D7BRs/W1L+3j+1vPZSC0VvHGWJLPHBClOqvIJ1+KROQ== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369328; c=relaxed/relaxed; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LToVPECKJFE4j0Plw5cLn8QSggGK0PGTfftoa70XSMSnVHe95CxupFPSrNJy7MqxD+rNxJ1GfSv/QtrZcjHnXBcngOhBhz5fiI66Yyl/z2DhcdIFuyZxiyh2DTtYOS/oF/N5SeO0eG9KZHKQgwtViVVx1VFLsmFsC0Yd1jCvmXxMEiWcp732rAmCZDUpiya7DvUGyFWhPlQErAcKe9j0HRglfm2k3UZRp5Yflewi7CTCyzIseCFbggQZIodC+WAim4YXcn3LgsDitW1eaf6/jrSmRLWVKuoI4JEJFlAkkOWkuPJa1FayavB3jksM5+VTpssGUYypBAkhp6L1/wu1HQ== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=vE3j3An1; dkim-atps=neutral; spf=pass (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5056-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=vE3j3An1; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5056-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [147.75.80.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRHz3ldyz4w2L for ; Tue, 12 Nov 2024 10:55:27 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 98D991F22418 for ; Mon, 11 Nov 2024 23:55:27 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D2AF1CF2A2; Mon, 11 Nov 2024 23:49:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="vE3j3An1" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA7E71CEAB4 for ; Mon, 11 Nov 2024 23:49:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368948; cv=none; b=Lhc3utxKaP8DWK1vnc+QqrtwtzeqtD2YtwwNq+sQpdk2kdFO1o5oRQjcqvJgfgMPZJdwQtIPNzb8nitX0aCgZ4OBIgP2w6ZaReDkm58xkjVPfG4atddeRGTPh6zVm7nKfU7fG0eIsR8QWoxGHX6uzRWgytaawlCJ2VRPRQhVRHo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368948; c=relaxed/simple; bh=6ZSmJO+bG/O0VvG/m0Wnv2BJ9smDd+VmRMf5KfhKfrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l40o1DVaCQC0I/spcN7YBLixIY1CLik5h9TN5dfETaSxBP8wilX5RPfmsFmYqfpYIstxpTJ6Je3txAxc8BklXQW6JR9aAiA8QUybRyXphIP3MtHRHjuH681GCKfMLXocPUPpX2J1nz89kzzlzLVcxHai0PmkkKdagEHXkqx3y38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=vE3j3An1; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-720be27db74so3759049b3a.1 for ; Mon, 11 Nov 2024 15:49:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368946; x=1731973746; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; b=vE3j3An1o1a7juNKlP82+dNAyPgLPKFuMomn/kw1a1d465mo9/z/0nXwn5MgU3Q9xZ jcZoDXoAOfWm+7jdxIVAnELsZSgLhGQCuTAphBN/IVn8Q7C4dcaogpcy880gowwi0McJ 1iTGHXmqZON4Ul7EhVdfYEsA3UWunG4MZJiumZrzxzisvXHY7MmUvsV3FAXCUehpV3oO n2p0/Gysf2w/LMm9Gd3IxkgUc553CO7pbVJPepDNq1Np2OX3bQF9mUBc/o7ZjTrXNINr HvFw0WYsG9OpQz0tl2OL4qL+gACl7zUJEeDYYR+iDW+Yij8YmYeDpO7RYsXw7IG0DCrh eWxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368946; x=1731973746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TkS4Adcluut9E9TBSf0Q5Dr6BdVnPYdslAPkTGEhwiY=; b=dRtuWFbUt6kkdohUapBFd8myk0egDs4fPXxkT6uNVDDu/9KR7b5N7iW8H7TBUW9H94 z/kOxD8UihcYUBpWNmu1HEASJaKIkKreEJCwj9AhOcMGEcNQqwZrbvd3E3OH1dxbz346 Vb1VX6cYUqxKy4jdT1mlwf9BeROsWzwBcLAu3myOAyiT1SFlWEbyn4/JeMt4j3syXh0l d+t2hGBzM03zqdJG+Gr97y+WrdRvj6v+8vxB06xmgE8QV4R0EJJkimzG89JGRDa9tctt Zkh0DI+LaOaOt+6K+JJMzdI+dTTq8R9QOFCd61+O0vaiDvSX9C9OjepoIGbCo9cjh3JH bptw== X-Forwarded-Encrypted: i=1; AJvYcCWBN2OWWu2JJPPjQC1hCBZaLeinbo9DcpQ2ZkpyPRsbasj6catCsGTaZ0a5yM9ivzxFcLo7jD8GObBd@vger.kernel.org X-Gm-Message-State: AOJu0YzMoLy3bt+mXM7SxPvrtwlF6MZFm4I0doy1Uu+Vw59SyDKROLYm gUv3wWsKQwPzB8BEAYKJWpZfb9EXjtwDJBm5BJB5/L47ef2mugZr39iGPE6SzfI= X-Google-Smtp-Source: AGHT+IH28qnmC+d/sJZ3EdxK+NK8za6V+wXT31dNckiLvxFVwcmETmxIGu/BBIMvr9bfEpwgK4E2eA== X-Received: by 2002:a05:6a00:a1a:b0:71e:71ba:9056 with SMTP id d2e1a72fcca58-7241407b632mr20781928b3a.10.1731368946274; Mon, 11 Nov 2024 15:49:06 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:05 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 12/16] ext4: add RWF_UNCACHED write support Date: Mon, 11 Nov 2024 16:37:39 -0700 Message-ID: <20241111234842.2024180-13-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org IOCB_UNCACHED IO needs to prune writeback regions on IO completion, and hence need the worker punt that ext4 also does for unwritten extents. Add an io_end flag to manage that. If foliop is set to foliop_uncached in ext4_write_begin(), then set FGP_UNCACHED so that __filemap_get_folio() will mark newly created folios as uncached. That in turn will make writeback completion drop these ranges from the page cache. Now that ext4 supports both uncached reads and writes, add the fop_flag FOP_UNCACHED to enable it. Signed-off-by: Jens Axboe --- fs/ext4/ext4.h | 1 + fs/ext4/file.c | 2 +- fs/ext4/inline.c | 7 ++++++- fs/ext4/inode.c | 18 ++++++++++++++++-- fs/ext4/page-io.c | 28 ++++++++++++++++------------ 5 files changed, 40 insertions(+), 16 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 44b0d418143c..60dc9ffae076 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -279,6 +279,7 @@ struct ext4_system_blocks { * Flags for ext4_io_end->flags */ #define EXT4_IO_END_UNWRITTEN 0x0001 +#define EXT4_IO_UNCACHED 0x0002 struct ext4_io_end_vec { struct list_head list; /* list of io_end_vec */ diff --git a/fs/ext4/file.c b/fs/ext4/file.c index f14aed14b9cf..0ef39d738598 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -944,7 +944,7 @@ const struct file_operations ext4_file_operations = { .splice_write = iter_file_splice_write, .fallocate = ext4_fallocate, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_DIO_PARALLEL_WRITE, + FOP_DIO_PARALLEL_WRITE | FOP_UNCACHED, }; const struct inode_operations ext4_file_inode_operations = { diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index 3536ca7e4fcc..4089d0744164 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -667,6 +667,7 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, handle_t *handle; struct folio *folio; struct ext4_iloc iloc; + fgf_t fgp_flags; if (pos + len > ext4_get_max_inline_size(inode)) goto convert; @@ -702,7 +703,11 @@ int ext4_try_to_write_inline_data(struct address_space *mapping, if (ret) goto out; - folio = __filemap_get_folio(mapping, 0, FGP_WRITEBEGIN | FGP_NOFS, + fgp_flags = FGP_WRITEBEGIN | FGP_NOFS; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; + + folio = __filemap_get_folio(mapping, 0, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) { ret = PTR_ERR(folio); diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 54bdd4884fe6..afae3ab64c9e 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -1138,6 +1138,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, int ret, needed_blocks; handle_t *handle; int retries = 0; + fgf_t fgp_flags; struct folio *folio; pgoff_t index; unsigned from, to; @@ -1164,6 +1165,15 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, return 0; } + /* + * Set FGP_WRITEBEGIN, and FGP_UNCACHED if foliop contains + * foliop_uncached. That's how generic_perform_write() informs us + * that this is an uncached write. + */ + fgp_flags = FGP_WRITEBEGIN; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; + /* * __filemap_get_folio() can take a long time if the * system is thrashing due to memory pressure, or if the folio @@ -1172,7 +1182,7 @@ static int ext4_write_begin(struct file *file, struct address_space *mapping, * the folio (if needed) without using GFP_NOFS. */ retry_grab: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); @@ -2903,6 +2913,7 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, struct folio *folio; pgoff_t index; struct inode *inode = mapping->host; + fgf_t fgp_flags; if (unlikely(ext4_forced_shutdown(inode->i_sb))) return -EIO; @@ -2926,8 +2937,11 @@ static int ext4_da_write_begin(struct file *file, struct address_space *mapping, return 0; } + fgp_flags = FGP_WRITEBEGIN; + if (*foliop == foliop_uncached) + fgp_flags |= FGP_UNCACHED; retry: - folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN, + folio = __filemap_get_folio(mapping, index, fgp_flags, mapping_gfp_mask(mapping)); if (IS_ERR(folio)) return PTR_ERR(folio); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index ad5543866d21..10447c3c4ff1 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -226,8 +226,6 @@ static void ext4_add_complete_io(ext4_io_end_t *io_end) unsigned long flags; /* Only reserved conversions from writeback should enter here */ - WARN_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); - WARN_ON(!io_end->handle && sbi->s_journal); spin_lock_irqsave(&ei->i_completed_io_lock, flags); wq = sbi->rsv_conversion_wq; if (list_empty(&ei->i_rsv_conversion_list)) @@ -252,7 +250,7 @@ static int ext4_do_flush_completed_IO(struct inode *inode, while (!list_empty(&unwritten)) { io_end = list_entry(unwritten.next, ext4_io_end_t, list); - BUG_ON(!(io_end->flag & EXT4_IO_END_UNWRITTEN)); + BUG_ON(!(io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED))); list_del_init(&io_end->list); err = ext4_end_io_end(io_end); @@ -287,14 +285,15 @@ ext4_io_end_t *ext4_init_io_end(struct inode *inode, gfp_t flags) void ext4_put_io_end_defer(ext4_io_end_t *io_end) { - if (refcount_dec_and_test(&io_end->count)) { - if (!(io_end->flag & EXT4_IO_END_UNWRITTEN) || - list_empty(&io_end->list_vec)) { - ext4_release_io_end(io_end); - return; - } - ext4_add_complete_io(io_end); + if (!refcount_dec_and_test(&io_end->count)) + return; + if ((!(io_end->flag & EXT4_IO_END_UNWRITTEN) || + list_empty(&io_end->list_vec)) && + !(io_end->flag & EXT4_IO_UNCACHED)) { + ext4_release_io_end(io_end); + return; } + ext4_add_complete_io(io_end); } int ext4_put_io_end(ext4_io_end_t *io_end) @@ -348,7 +347,7 @@ static void ext4_end_bio(struct bio *bio) blk_status_to_errno(bio->bi_status)); } - if (io_end->flag & EXT4_IO_END_UNWRITTEN) { + if (io_end->flag & (EXT4_IO_END_UNWRITTEN|EXT4_IO_UNCACHED)) { /* * Link bio into list hanging from io_end. We have to do it * atomically as bio completions can be racing against each @@ -417,8 +416,13 @@ static void io_submit_add_bh(struct ext4_io_submit *io, submit_and_retry: ext4_io_submit(io); } - if (io->io_bio == NULL) + if (io->io_bio == NULL) { io_submit_init_bio(io, bh); + if (folio_test_uncached(folio)) { + ext4_io_end_t *io_end = io->io_bio->bi_private; + io_end->flag |= EXT4_IO_UNCACHED; + } + } if (!bio_add_folio(io->io_bio, io_folio, bh->b_size, bh_offset(bh))) goto submit_and_retry; wbc_account_cgroup_owner(io->io_wbc, &folio->page, bh->b_size); From patchwork Mon Nov 11 23:37:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009972 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WGbnwRAz; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=ejb5=sg=vger.kernel.org=linux-ext4+bounces-5058-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRKm0Wvsz1xwV for ; Tue, 12 Nov 2024 10:57:00 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRKj30nwz4x11 for ; Tue, 12 Nov 2024 10:56:57 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRKj2xpjz4wbR; Tue, 12 Nov 2024 10:56:57 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:45e3:2400::1" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369414; cv=pass; b=nMwa6khWP0T66BIvhS10uenHayckjGK2n1qR8j+R7v8M5+hsZ0y035rAgfSGr75CvCFGUiT4gajs3H5W8iRVEOjt6/sBhEAhkJV8gkUKZEX6zmZFtkbrgmYNb/mjzRSla7J2HnOnIyKIQGPQ2lcYi9gN9BO0zu6uidJoLUTuBa5VcFu03vpuTsNN19Z0cmSWYwESC9yakmUxsotXfJ7D16t5psMwuhP/+F5Frg+hoEZGIv58VaK4mvXAvHRgG2uh971xNUMSEpdcjMk/Wm/6sa2Mb4UpHfO2QSuKYJbn4R22N4AVIqqWzQMSMwU866SAOND7sbSmScKpZ+TuTYWOPQ== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369414; c=relaxed/relaxed; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aTbXWXSJCWfmEf0VB+TcEga/rtyYpAKn4qjbirwjyyOaxWVloCv71nLFovY9XSrx0uOTH6ZqNciRbj1RFYKy05N1qM+nG5MkxUvvT5/URGRPa06VBx8uIAp2+0iamWQCyPU5Ma2ZmEpSpR3vFJ5n5Vr+WLLjIxJ8ffryFG0Tm1sHL1wG9qx7BlUhb6ksrlCBz5d1JN9l8Ub0Apbh8ycWCtXD8HcAtznTqZ2jHZBWJ9kQ8K8MbPJBq61PwB763rwJSqP0NLv+ef6Jz4t69SGvJxMaG1Ug7foJTUMXmCMso6zIVWeMF8rJIouzGJc/46q1megE8slYfEMWCWd13qcx6g== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WGbnwRAz; dkim-atps=neutral; spf=pass (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5058-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=WGbnwRAz; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:45e3:2400::1; helo=sv.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5058-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org [IPv6:2604:1380:45e3:2400::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRKd5Zy1z4x11 for ; Tue, 12 Nov 2024 10:56:53 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 0C68F280FF2 for ; Mon, 11 Nov 2024 23:56:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D0931D3624; Mon, 11 Nov 2024 23:49:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="WGbnwRAz" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C86851CEEAB for ; Mon, 11 Nov 2024 23:49:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368953; cv=none; b=OKZe9EDasHqbkzZn6QsYOkIMKjJarDtdA7h9ft1YZYc2WUz3t0JdpKRMsAie0SOncUHBCUiI05qbxwTv5yr5R8/L0GJ7lz8w3iZnu7iAXfrny+otN04YSlLm21v/xOaSz+mrfvwgxluoq2nc6Oa6I/BzriffeKI1MdSZJwYiYWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368953; c=relaxed/simple; bh=3H9STUASulXCHklYsN+md87lfFFsAKUuDScGACrWthk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YYKVM0GlvL44fed3JSNjXTuzSXt+HNnY92lEDmSzsJ13qH0RTSPuLXPvqXmSEqUcpz1Z80ozsOXdLOZ7sxuOC23WFYTRu5dJ0XfiouPsOFE1Ow4WzbcMzZL+cHxHNdqeh1wkCT86Sz44bvPUSFKdftIvPDMfRrrfLPpjqHfyIz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=WGbnwRAz; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7240fa50694so3694855b3a.1 for ; Mon, 11 Nov 2024 15:49:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368948; x=1731973748; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; b=WGbnwRAzgJm3+3+5ZZh9FBShx2KIOn2NTTzBoHktZzJOHXl3t4id/cPDN1e+rUQiBU 14xIFWE3au5w4p74aS4ofWKaRtXVkO3/qJOYvxOa9DK7lXgKPVqOTvnvOTroUzTU4R9o yTc5LgquDoTMlNMq4sQNj0PoldwIuA1HDXJzVc2Eb0wxrOFOYmxWQabSPpcwgVVGVnSV YJocXfKgGi3Dok4YSDVx9AKXGsx5IcrltOEtwn40KK5746IONvoFHNFPePG6PdTEa/HY UweAPD7sA3aOe4BXT8VwEh0PWrjF22at1dimvtsVB04Wljk1rtpH9AplC5L3p2qoZqWw jHgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368948; x=1731973748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QHHnA1NsQjr1van9cnJ+PQRWRXhY+a6ETIPA3kXFfjk=; b=wFjZ1hACeAhZigIl/fVfqtROT8tdD6Fx7wxlkXlbwtTPsHq63y8lrEA3Q6I909tTnf TUqACJcgnEy7SKo+purKhkzfr6+9zr8KpkOEHrif4lZsjytf5mv4WKLy8c/Ze4RUwxBl 1PwffvXepv6wZq2GfeV/uuY67eLhWfweVr3ZDltXtq1dfOu/JVxhfFIK6pU8Ki1rgliH OvIZDNVJZqYRlE6pVNePNYYBw7VHOEmp9CipP8fLqT6JzFfbBcB98PFrQarMD2ucAP+n 7DyoqApfjxZpSPHjcCNOJ3QQkHBIp9cLluoOEdhWeWxH5uMMOsABT/7t2HhDDLyh1qpk vuHQ== X-Forwarded-Encrypted: i=1; AJvYcCWj9KkR8OmfjrjVPP/2wz4P1tvvf5hLBZMOHtUenMMNccEGHAZdr4ahVzfCnYMfF5Lad3kyHagLZJWB@vger.kernel.org X-Gm-Message-State: AOJu0YzG5fZBE5i+sOZd0gJ7+NPzFiwonzaOMIbnbc1rsEd6oXaIoBCb CvibTmBpVvu0YUKYhZt7gdWkKi2mbpHXcpKVz3j7zwmEVsap6hc84xDSiyLIgio= X-Google-Smtp-Source: AGHT+IE+DUrjoGziw6EjyU9hF9MosU8fZ1F2/8qqkRJb8RqY6XTyhSKojrzpZHSNP9ce3wMHzfWwwA== X-Received: by 2002:a05:6a00:181d:b0:720:aa27:2e45 with SMTP id d2e1a72fcca58-724132c535bmr19020120b3a.14.1731368948118; Mon, 11 Nov 2024 15:49:08 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:07 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 13/16] iomap: make buffered writes work with RWF_UNCACHED Date: Mon, 11 Nov 2024 16:37:40 -0700 Message-ID: <20241111234842.2024180-14-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Add iomap buffered write support for RWF_UNCACHED. If RWF_UNCACHED is set for a write, mark the folios being written with drop_writeback. Then writeback completion will drop the pages. The write_iter handler simply kicks off writeback for the pages, and writeback completion will take care of the rest. This still needs the user of the iomap buffered write helpers to call iocb_uncached_write() upon successful issue of the writes. Signed-off-by: Jens Axboe --- fs/iomap/buffered-io.c | 15 +++++++++++++-- include/linux/iomap.h | 4 +++- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index ef0b68bccbb6..2f2a5db04a68 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -603,6 +603,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) if (iter->flags & IOMAP_NOWAIT) fgp |= FGP_NOWAIT; + if (iter->flags & IOMAP_UNCACHED) + fgp |= FGP_UNCACHED; fgp |= fgf_set_order(len); return __filemap_get_folio(iter->inode->i_mapping, pos >> PAGE_SHIFT, @@ -1023,8 +1025,9 @@ ssize_t iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, const struct iomap_ops *ops, void *private) { + struct address_space *mapping = iocb->ki_filp->f_mapping; struct iomap_iter iter = { - .inode = iocb->ki_filp->f_mapping->host, + .inode = mapping->host, .pos = iocb->ki_pos, .len = iov_iter_count(i), .flags = IOMAP_WRITE, @@ -1034,9 +1037,14 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, if (iocb->ki_flags & IOCB_NOWAIT) iter.flags |= IOMAP_NOWAIT; + if (iocb->ki_flags & IOCB_UNCACHED) + iter.flags |= IOMAP_UNCACHED; - while ((ret = iomap_iter(&iter, ops)) > 0) + while ((ret = iomap_iter(&iter, ops)) > 0) { + if (iocb->ki_flags & IOCB_UNCACHED) + iter.iomap.flags |= IOMAP_F_UNCACHED; iter.processed = iomap_write_iter(&iter, i); + } if (unlikely(iter.pos == iocb->ki_pos)) return ret; @@ -1770,6 +1778,9 @@ static int iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, size_t poff = offset_in_folio(folio, pos); int error; + if (folio_test_uncached(folio)) + wpc->iomap.flags |= IOMAP_F_UNCACHED; + if (!wpc->ioend || !iomap_can_add_to_ioend(wpc, pos)) { new_ioend: error = iomap_submit_ioend(wpc, 0); diff --git a/include/linux/iomap.h b/include/linux/iomap.h index f61407e3b121..2efc72df19a2 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -64,6 +64,7 @@ struct vm_fault; #define IOMAP_F_BUFFER_HEAD 0 #endif /* CONFIG_BUFFER_HEAD */ #define IOMAP_F_XATTR (1U << 5) +#define IOMAP_F_UNCACHED (1U << 6) /* * Flags set by the core iomap code during operations: @@ -173,8 +174,9 @@ struct iomap_folio_ops { #define IOMAP_NOWAIT (1 << 5) /* do not block */ #define IOMAP_OVERWRITE_ONLY (1 << 6) /* only pure overwrites allowed */ #define IOMAP_UNSHARE (1 << 7) /* unshare_file_range */ +#define IOMAP_UNCACHED (1 << 8) /* uncached IO */ #ifdef CONFIG_FS_DAX -#define IOMAP_DAX (1 << 8) /* DAX mapping */ +#define IOMAP_DAX (1 << 9) /* DAX mapping */ #else #define IOMAP_DAX 0 #endif /* CONFIG_FS_DAX */ From patchwork Mon Nov 11 23:37:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009970 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=MQOeFTKf; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=150.107.74.76; helo=mail.ozlabs.org; envelope-from=srs0=exs3=sg=vger.kernel.org=linux-ext4+bounces-5057-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (gandalf.ozlabs.org [150.107.74.76]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRKH47K4z1xwV for ; Tue, 12 Nov 2024 10:56:35 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRKD5bG0z4wbR for ; Tue, 12 Nov 2024 10:56:32 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRKD5W4mz4x11; Tue, 12 Nov 2024 10:56:32 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=147.75.80.249 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369389; cv=pass; b=rpC0WnwGhnLoR+q+R+EpsOJhHMyVDuYnf81jWoaFs8cPUAZBV9+2mOq9iVHyt0Yt2QljZhgh25Fh8Hbm0mD1khxr9aJl6vC30lHVbnU7X77he0LzC0cZNJDQCKt0X+LvZm0SR99ajnN6+zxjkROoxBGHKR7i1t3KfX/2y9E/iZMCziHv41odvyewSUjQC7NjEhQwrDmKN/5eRSehpaFSJ8Nrou2im3Uz8aGZrdMJJ/clOLSgPXhCA/bM/wEZ645jLkcQ0nYqy099orxwQOJk9uyXrQDzzt3sQM1RPXpFK7A1Cv+9s6mHM7U2kDH1roV0MQ9JN1oM5e8ySm1MD5Z6Fg== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369389; c=relaxed/relaxed; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rdn8up8KpJsU5UmeLGgJRR3tzGQ3jXnHsOcRYZQ2+sGnXAk5nEVshhrmEovbkM43T44gwXs1j5VjvM+lyK3U01yuOmNPNDLCez0pIy6fNND7ApsNFkBcR4x50WeTTph2MsAhFqp267y4Xv0Qn5SaE2tayt1r1Y2j94jf75hOZttqxxreFq+b1R0RaQP8EcoOemUMLDg10bNF23zTeT05cJ57yAWXhCnHqpSqkabgkOUH1iJaiqWCl1xsbwpDgTjYrILMzjGC/7yv92xev0Esd0dI9ApvfV3agQAh4ACQHWSyWYOYgCDb2ArNtvsKil47TchwkJnI69sZuYAcYm9vMg== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=MQOeFTKf; dkim-atps=neutral; spf=pass (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5057-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=MQOeFTKf; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.80.249; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5057-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [147.75.80.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRK86hZsz4wbR for ; Tue, 12 Nov 2024 10:56:28 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 4196C1F20EDE for ; Mon, 11 Nov 2024 23:56:29 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0246A1D14E3; Mon, 11 Nov 2024 23:49:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="MQOeFTKf" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D1201D079C for ; Mon, 11 Nov 2024 23:49:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368951; cv=none; b=DJ1jpiysFM9etVhIeiEVVcUWRSTx8VKoCt344qKlpnfnEqTYJV7GLYGdJILaWW0vpcuIN7XckTnhsXf/1eqtUGoy5C7VOZDdi9drcm33bDcWKHxhpPSjEOL933qIdWf4TLKUaVjGDVq5eekjoSZd6CiM/NZrH3U5AT+uE8of+fE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368951; c=relaxed/simple; bh=JN6ziuaz3h3w93syhui0sSuROUWYBUO3x3xlg3f4CsA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CAeaZUWAIJcW7fONIAVTgjN/sV5YVDzRP+jcLeVUH6yOj4Bk3iBh3nZjoBLQUeSPjrjEQnYjP5OQLR/d9Oz+7MuFvNnupRK5VK6o76YSjeyFSVEA3qhoXDo8v5VkA3TMiw8PTcP+7+OVveb8T9RoVM+ftuGJg4/s8Nj357876RM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=MQOeFTKf; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-720d5ada03cso5128742b3a.1 for ; Mon, 11 Nov 2024 15:49:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368950; x=1731973750; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=MQOeFTKfgntx/fL0KS1w5ehtnzLw2J47Mgw4aCD7qt3FE0GsZguoRF9UzEZDgQNRgw aFV52MGH6xiQKxHGYiA1kQ5dpioB9Eb2OR4Hcr1snqxBl+9BCvPqY53HaGpaY5Dv+Kxt 5uSsPI702LUrz9n1i44B0UVSu0D/HdbH9sI6vGKOckND5vcI7xCVX4UyWAKHjqOvqbeV H6z2/+sDDjg6Ql5St4JCD+3NCRqUcBKc2ATPKwnEKp4Z874xUa6nZGAqIBzYYA1Ms9TM 2EC1Jigln0/b4vOYbtqVWAsGezZdc3xX9GQP/UGaLCY75z7A1NSNfrTGydecQF7a/u2V zVLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368950; x=1731973750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LL88Qd2tMlsmBvcuEyGfQPoq7EBSjG4mmD1oB3CTmL4=; b=KrlRUms2d5b7uxVrSAejC/Ulzfd2tOir0e57ZfSXzIzdEJ5rRyXtrFOPSRmcHMRHvi 58wSpDs/qaV+/gw4TA8G7LkIFCKdBxuxV3xWnM7/w+SlbOFqeL8kJsdowgo4suMb5dwZ P4hoZC0U9fj7TB0QP2l/5cm3Npv1pfVStHSXfDZ5eD91qqMHxAQMdXP12wxbttKXpv/l 68Sxs1y2RqvCrb+p42fx1ghwnvD+D0SRYbRzqqqzlUaUQuNRTISADKMXN9Zf1mfzBZfc CIrR98VMsF5pHm/Zh/9Oi3J1/ZJ/B+nE0NIFFDSdfESrpYCP5fBY1F7AASqvFJc9lO3y 90fQ== X-Forwarded-Encrypted: i=1; AJvYcCXXDsKAz2+GAcgmrhMFrqc2u421X/wiHAynZUYgNf9WZ3vGuqZZhKzt6jyB579JxSJbmCGmm7ej72sV@vger.kernel.org X-Gm-Message-State: AOJu0YzTJIftcUG98eWmv5V+3rD70yGEIC95GGaPOZF6e7pMXTk+vbwY QqDFvNsFdR7ddEJ+ZhEhPSvJzw4Q+pJgttgWl4qBIj6dZcMiPn3yk1bEv/zbmrY= X-Google-Smtp-Source: AGHT+IGxdsMu1dfGOYWXi1R28aWFhjsN5Q3I34Dwpy3CjKRXcDtvBr7LZlUVTVK3RyQbj+sGtC8u7g== X-Received: by 2002:a05:6a00:3a29:b0:71e:680d:5e94 with SMTP id d2e1a72fcca58-724133510d7mr19932677b3a.19.1731368949731; Mon, 11 Nov 2024 15:49:09 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:09 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 14/16] xfs: punt uncached write completions to the completion wq Date: Mon, 11 Nov 2024 16:37:41 -0700 Message-ID: <20241111234842.2024180-15-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org They need non-irq context guaranteed, to be able to prune ranges from the page cache. Treat them like unwritten extents and punt them to the completion workqueue. Signed-off-by: Jens Axboe --- fs/xfs/xfs_aops.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 559a3a577097..c86fc2b8f344 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -416,9 +416,12 @@ xfs_prepare_ioend( memalloc_nofs_restore(nofs_flag); - /* send ioends that might require a transaction to the completion wq */ + /* + * Send ioends that might require a transaction or need blocking + * context to the completion wq + */ if (xfs_ioend_is_append(ioend) || ioend->io_type == IOMAP_UNWRITTEN || - (ioend->io_flags & IOMAP_F_SHARED)) + (ioend->io_flags & (IOMAP_F_SHARED|IOMAP_F_UNCACHED))) ioend->io_bio.bi_end_io = xfs_end_bio; return status; } From patchwork Mon Nov 11 23:37:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009977 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=s3fWExQA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=xhso=sh=vger.kernel.org=linux-ext4+bounces-5059-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRkV2BMDz1xwV for ; Tue, 12 Nov 2024 11:14:58 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRkR3chqz4w2R for ; Tue, 12 Nov 2024 11:14:55 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRkR3SYYz4x11; Tue, 12 Nov 2024 11:14:55 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip=147.75.48.161 arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731370491; cv=pass; b=AUysrsumbf8efpJhIz3AoJec4Fi8QXeOtjB5E50Lh0Pu+DpqXnKkHjowBTucNFD/WE4Ra2TPeOm9f949rcCr2jmrog3bnqm3j5yaR90DPbpkU4Z6+UBSp7hBMpoVsTLj2WMi4VjCN+aXLbFhjfFJIAYIU6Ml7lHcgsuwsf6X/Dreh3jz5CkGea+ruDQS3YY6U6TbYCz8j1y4q7QFvjgwekzS432zcvwy0OsTMdnaRA3dFHWafpmizwsdJSOV17pihF1yPU5J4P+4vKpx7pb6RMBWf2B5grjRd37MZJyvCAo6DvF2lgYcbaex1wTZGmjFfkOlffJxXm8Vg0427SdRsA== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731370491; c=relaxed/relaxed; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OnSD3vaqYB3gpNUoXv1LJKushBjH62bih0Gn7Ce1Z8hUUsB5j2dfcI7fJ9cEngjnD49QzyLYeKn40VQS6y8ptSz0ISJv4u16rfrtdB0ZsNst+B4fKzaP2TW3OexmDW463NrZ4wrTN9w7N2l1LU5BtheMzej0fHjDXNFxnO4oNQ06s0Cryjk0ooqKQ+KW+jGo9+GjvkmuF05riOGmb7zlKzDuGN47wnjwImCGOVrKBP5ZhRxvXaBhI3J6iB1aThAnE3Hy2xc5OvAIGEjtAcM/iYPENRJ7goOBAQ3CqHSKGKE5WL/hoYp7EyCa4OlWFFvhlcmbQxnkpxaDKyZPntx2SA== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=s3fWExQA; dkim-atps=neutral; spf=pass (client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5059-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=s3fWExQA; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=147.75.48.161; helo=sy.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5059-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from sy.mirrors.kernel.org (sy.mirrors.kernel.org [147.75.48.161]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRkM2GqJz4w2R for ; Tue, 12 Nov 2024 11:14:51 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sy.mirrors.kernel.org (Postfix) with ESMTPS id 46FF4B30CAC for ; Mon, 11 Nov 2024 23:57:12 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CFA0A1D967E; Mon, 11 Nov 2024 23:49:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="s3fWExQA" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F8161D1726 for ; Mon, 11 Nov 2024 23:49:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368953; cv=none; b=mOtjZelY18MgszCt2fXnm7NOh1jQNpvml9D1piWlo1HkWuvcra5+3LJXg1PKdFtBWPo+iojzFK74PhcfpcR6PsljNjDZfGx+pSh11IXvTa69yeqt/cl6VUbUtWNbjvqd/FRc5+A+IIabRm1jcUmbqqwTRsxjgMt5Yp2bM1pqT/8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368953; c=relaxed/simple; bh=cP6gr8wADb4EASrEnj9VSH3qpQrBlalS1qET7KiSVew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LLfH371EyF4/hrceVUxSfKbpR9eeJcZGi+GBrdnnuiLNFCloMGuBuLS6dmZKjIMTkJJbyBc2IIsqeXHrNv6J/fvrNAtwkLIZ55PSDuYgX2xhmXGhAf/Lpn93kuwTZuHId7ppSsKWuPGGzBJJbXg9ANLB3czRPPLxorXPNdaJxpg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=s3fWExQA; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-720d01caa66so4744616b3a.2 for ; Mon, 11 Nov 2024 15:49:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368952; x=1731973752; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; b=s3fWExQAqJNrjrhXBcABkuMixXrWdIudo4LplxBPh5u2i3FIjQz8RSwmrUEgHJwJZq 4nbroCZ69uk0s+dMKoqZuQA55Q5dTQoVv2sORNjhWPfC/aSmYILESoMsUEEb8Csh+yKz 24tJ1Cgq7JAqc5GibDIG0D5vvDg2IQ/lkgMaALDkkKdTBtUlskK/8Nj7xgcWo/hfFMJb lyavRh1R1g/HoSPd37GSLf9ifexOvMNPCvg0YrR2Va5+2oULo5MjCxjric8+vwXL4mFH T7mi/ZhHCZGKMmuF3nrFGuz8GPZkC76x1KAiPgeBGpMVdgciw11mo/kH0lOSTZs8rjYu ukoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368952; x=1731973752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ieWCO7ZvnQOESu0rOvKqnHS1oC/GZ0fSczrbrfzlBys=; b=CQIDUaH8VVC4mYqFka41BgATkh/v4N/71nt/S0a4tT4opHmJH8+O2Lc3mMw41m8Mlq esRuH7NpfoPGpr5n3+qpI9EdDDRM3RlPNxM2jago5CSnKLq8MU+0DnDaSNX0ayePlfTJ oh/XagO26lk/YgSM/6NyhTgi1h8VIOQdpZOWndTZ/gNMCj7m/+x+IwxvZvciK5KEjwNy 0Y9KXgSgSZOr2M5WaSqxqzesYjqd0JC1dOwynIM4FANLkz+/EOFrmLEOZvr7heWnotzg 65jA5raPNnN5DJDoRvyRJmRnDofhdbTIGBfZo+CbvdgrSG4NhMEB4WjrS3ww5S0YjjzE pUWg== X-Forwarded-Encrypted: i=1; AJvYcCXG96ImJTbhXFWjMa9V0a4UiZLzJhY4OQBpQ/qzYJ+S0Cz/xyDFDyq3ao+K+gAVRLZGXBUCDvsyzr6m@vger.kernel.org X-Gm-Message-State: AOJu0YyTXRezIumBmVC+3JID6/OX33qIhKtfKlG+hr73ZTu3f56OKi+a Bjy3Xw3ftRrN68L/gZVbDXoqS2vm4DXiviKmtavJUSxFD5xESTFnMbnpDP+CHSRLhhXEZH05FI9 aaqk= X-Google-Smtp-Source: AGHT+IEuKUU8LmiDGdVSE8HEQD2DSqpmQWqCoCseRpXFHWRMzDS4n8tdQLpPQA0yFxWxU+7dmSvSgA== X-Received: by 2002:a05:6a00:a1d:b0:71e:693c:107c with SMTP id d2e1a72fcca58-724132c15a3mr18685493b3a.11.1731368951711; Mon, 11 Nov 2024 15:49:11 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:10 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 15/16] xfs: flag as supporting FOP_UNCACHED Date: Mon, 11 Nov 2024 16:37:42 -0700 Message-ID: <20241111234842.2024180-16-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-0.8 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org Read side was already fully supported, for the write side all that's needed now is calling generic_uncached_write() when uncached writes have been submitted. With that, enable the use of RWF_UNCACHED with XFS by flagging support with FOP_UNCACHED. Signed-off-by: Jens Axboe --- fs/xfs/xfs_file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index b19916b11fd5..1a7f46e13464 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -825,6 +825,7 @@ xfs_file_buffered_write( if (ret > 0) { XFS_STATS_ADD(ip->i_mount, xs_write_bytes, ret); + generic_uncached_write(iocb, ret); /* Handle various SYNC-type writes */ ret = generic_write_sync(iocb, ret); } @@ -1595,7 +1596,8 @@ const struct file_operations xfs_file_operations = { .fadvise = xfs_file_fadvise, .remap_file_range = xfs_file_remap_range, .fop_flags = FOP_MMAP_SYNC | FOP_BUFFER_RASYNC | - FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE, + FOP_BUFFER_WASYNC | FOP_DIO_PARALLEL_WRITE | + FOP_UNCACHED, }; const struct file_operations xfs_dir_file_operations = { From patchwork Mon Nov 11 23:37:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 2009974 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=IdKYygrt; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.org (client-ip=2404:9400:2221:ea00::3; helo=mail.ozlabs.org; envelope-from=srs0=qu2m=sg=vger.kernel.org=linux-ext4+bounces-5060-patchwork-incoming=ozlabs.org@ozlabs.org; receiver=patchwork.ozlabs.org) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XnRLz5gLJz1xwV for ; Tue, 12 Nov 2024 10:58:03 +1100 (AEDT) Received: from mail.ozlabs.org (mail.ozlabs.org [IPv6:2404:9400:2221:ea00::3]) by gandalf.ozlabs.org (Postfix) with ESMTP id 4XnRLx0z4Yz4wbR for ; Tue, 12 Nov 2024 10:58:01 +1100 (AEDT) Received: by gandalf.ozlabs.org (Postfix) id 4XnRLx0wJBz4x11; Tue, 12 Nov 2024 10:58:01 +1100 (AEDT) Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: gandalf.ozlabs.org; arc=pass smtp.remote-ip="2604:1380:4601:e00::3" arc.chain=subspace.kernel.org ARC-Seal: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369477; cv=pass; b=QdraofZSaf09jxLgkBMhVO+RGplnRE8OqQykm2ys8P9Q4ZKMSRet8ykJSj4hUIb7+tgrWnPL5gIwxAUm8M8peipGGJR1+iVCjZSuNiWI3BIfbQGA2SvYpQaEy9KL1Y6bEmuOOo3TuPdxqJIKmKaTFAbXqMxNIfx+Zn8rJ0UngsvWOJ1re64AwHlxWmXrq2yf1jKiZzpQBppYD5mrRaxy9qH0ES8lE7pPRsNRivHmiw8yYnKAS1FMInHfHvPZc54kHhG3wrL5pHcpKjtX+rsX1JfblGI0F4il6V5vYLQn1ltjyg6+WC48XnMkfjbNIvIHp8tYJ917ggi/0yhFtVkKyg== ARC-Message-Signature: i=2; a=rsa-sha256; d=ozlabs.org; s=201707; t=1731369477; c=relaxed/relaxed; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TMze4Z6jhmh158MxTYiHBQQ2j7Sfk4DsSV+ISQE9frtb8Y6lmODCK93cJJN/qrMa0wt9G13dXOgrq3gNQYFS0oVCBFPQwWQolzcrruccsGm3mugr4Z9ORQpLz9botuQp6OjcB3hdPdQaLkbw1en4pHj1GaJfX9OpxS6rrPyW0KWbku28YwmejnasbjfDcRCbhjXlzl8h2U4zPPFz0zC21YG8lWMUN2RmOaC3d9/9iKDHQu2cG5LS86OW1aOaQJY3wvNPOUKtps0YpHEt1HVG1A+Ujxb4nRqCh9pTSi2Dzce3RTjGH9Cw3S99IM7LIbPji5HZpOB3kFTCzXk/qM/GPQ== ARC-Authentication-Results: i=2; gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=IdKYygrt; dkim-atps=neutral; spf=pass (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5060-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) smtp.mailfrom=vger.kernel.org Authentication-Results: gandalf.ozlabs.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: gandalf.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.a=rsa-sha256 header.s=20230601 header.b=IdKYygrt; dkim-atps=neutral Authentication-Results: gandalf.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2604:1380:4601:e00::3; helo=am.mirrors.kernel.org; envelope-from=linux-ext4+bounces-5060-patchwork-incoming=ozlabs.org@vger.kernel.org; receiver=ozlabs.org) Received: from am.mirrors.kernel.org (am.mirrors.kernel.org [IPv6:2604:1380:4601:e00::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by gandalf.ozlabs.org (Postfix) with ESMTPS id 4XnRLr4Wptz4wbR for ; Tue, 12 Nov 2024 10:57:56 +1100 (AEDT) Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id ED6E11F2155A for ; Mon, 11 Nov 2024 23:57:56 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4620E1E1C0F; Mon, 11 Nov 2024 23:49:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b="IdKYygrt" X-Original-To: linux-ext4@vger.kernel.org Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 656CE1E0DE2 for ; Mon, 11 Nov 2024 23:49:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368956; cv=none; b=hXiBMVYPL/hzLNAOPQaI+QRhQbwG3KRkipx1ZPIWfthqA+WqOvFWoa5jqdM1B9BL8LPhsBMs4wZu+xU2RjM7RVT+pIq8cnuE0/6XWYuIM2xTiXkpzQgdfEal70eJz+uwt+G+uJOd0mHwpqhNwZx0Lx0oS/uq5zFYrR24obL1MMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731368956; c=relaxed/simple; bh=IMtVN6mYVCOPkikFXXrJuZnZt8ZpG9Lqn7xujyAZ3eg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bUdxoNPZzbAS6UAx/NMW/eRxZWHv7QZk3Ldw0jtgQaR5F/QJJe0S+nvsUXO+eyNSZ6AWLuwz8CvpuLlkAes4IStpPu/NYe0c6rLX9JTV5R/FKhSzzJEqycoFSJqbuCRD/MiQ4NFmFgZ6GRwMAt+bQwVmDDPo6wetzC3D/hFCWX4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk; spf=pass smtp.mailfrom=kernel.dk; dkim=pass (2048-bit key) header.d=kernel-dk.20230601.gappssmtp.com header.i=@kernel-dk.20230601.gappssmtp.com header.b=IdKYygrt; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.dk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kernel.dk Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-720d14c8dbfso4843576b3a.0 for ; Mon, 11 Nov 2024 15:49:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1731368954; x=1731973754; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; b=IdKYygrtPZRQ4q4zua1tosyRBJ+adpZs/6Ecrnytyi89gBbGOTfaTD5JWVti36KeIX SosDNw0sDX0m66mZQQkD89wPGeDqTvjhTOgH2wH3Mu3+uhuotA9UM34uVKSEUWz67B2O 2n55gmyeZ6wn2eofRfC7WHwDY3014gJzTgBr5M8Ety7CHq5W6d2BWGfz+93tDB6OZQ/N cZ8ST2MSpPFlitU6RBLe4TJXlsDlIHG/rKieoe8MyUg+fIirzxi3hvwS412w0FneHKHP TkAjdZURq8Neqq7wpqsvgac9dUvLClIFXWWwzcc/GoTX4HmwCCPIWikX2shYuIssDAfs wIyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731368954; x=1731973754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F8VuOfX6p9OA5iVa4jUFPNHPqu8A2RaR+lAgXUtkcVc=; b=n8GigQtCQ/SxagrNwRX5UuctElNcm4LoxCJHIstW0V+FbvsToMoRU0zdFHrDlQkSAs dxx8PpS7j4r+JT1AW1q7RUu5GkI+rUv0tiI2wLihR78RNElPNOgD0m9LHoupqPdHTl4o jiuMjeQKVlmL6zqoNhO+RkkdhqhlmtoJjzT92COpsv3zEFvmXrwourRdvvt+77BR/8nH MF4Z9E35horPH3wfaY+OQQPOEznOWwc4NNeqLEOic2HHG5K7Oj5+XQ3HgKDYrx4/J+uR x12uSCdzU3tf71nL5n4tJyH4B+6AeDsJrhkXQFl79ocf4vT8Goc9OhGP6fKRVBQIvOIF OR8g== X-Forwarded-Encrypted: i=1; AJvYcCXAkBe24NP59Vf9t0yasKT3uHyISqCkAXInSqa/DEXkGe/RJmMzu/c0WH2Bie0MmEN7E/VrKIIZDlpE@vger.kernel.org X-Gm-Message-State: AOJu0Yy2lCFKGaiyIh5R5xFZL1KIZUnWB3Aixpylo0NGsopMdC+rZUmq /mrizw5d6GFBBWryZC+B7hspxiFWHnW7vg1lfR7hysUKgttuNVVbrJc7h0vpRVso1+0IMEfIIzR pmtg= X-Google-Smtp-Source: AGHT+IF2+wW3TLckV7VSNmFRA6TqTX3hVCOrji7S4MicSJHEM7Y626gshLEQCdd+Y+7SrufulOQgng== X-Received: by 2002:a05:6a00:2354:b0:71e:60fc:ad11 with SMTP id d2e1a72fcca58-72413354230mr19707325b3a.16.1731368953536; Mon, 11 Nov 2024 15:49:13 -0800 (PST) Received: from localhost.localdomain ([198.8.77.157]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-724078a7ee9sm10046057b3a.64.2024.11.11.15.49.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Nov 2024 15:49:12 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, Jens Axboe Subject: [PATCH 16/16] btrfs: add support for uncached writes Date: Mon, 11 Nov 2024 16:37:43 -0700 Message-ID: <20241111234842.2024180-17-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241111234842.2024180-1-axboe@kernel.dk> References: <20241111234842.2024180-1-axboe@kernel.dk> Precedence: bulk X-Mailing-List: linux-ext4@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Status: No, score=-3.1 required=5.0 tests=ARC_SIGNED,ARC_VALID, DKIM_SIGNED,DKIM_VALID,DMARC_MISSING,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=disabled version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on gandalf.ozlabs.org The read side is already covered as btrfs uses the generic filemap helpers. For writes, just pass in FGP_UNCACHED if uncached IO is being done, then the folios created should be marked appropriately. For IO completion, ensure that writing back folios that are uncached gets punted to one of the btrfs workers, as task context is needed for that. Add an 'uncached_io' member to struct btrfs_bio to manage that. Outside of that, call generic_uncached_write() upon successful completion of a buffered write. With that, add FOP_UNCACHED to the btrfs file_operations fop_flags structure, enabling use of RWF_UNCACHED. Signed-off-by: Jens Axboe --- fs/btrfs/bio.c | 4 ++-- fs/btrfs/bio.h | 2 ++ fs/btrfs/extent_io.c | 8 +++++++- fs/btrfs/file.c | 10 +++++++--- 4 files changed, 18 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/bio.c b/fs/btrfs/bio.c index 7e0f9600b80c..253e1a656934 100644 --- a/fs/btrfs/bio.c +++ b/fs/btrfs/bio.c @@ -334,7 +334,7 @@ static void btrfs_end_bio_work(struct work_struct *work) struct btrfs_bio *bbio = container_of(work, struct btrfs_bio, end_io_work); /* Metadata reads are checked and repaired by the submitter. */ - if (is_data_bbio(bbio)) + if (bio_op(&bbio->bio) == REQ_OP_READ && is_data_bbio(bbio)) btrfs_check_read_bio(bbio, bbio->bio.bi_private); else btrfs_bio_end_io(bbio, bbio->bio.bi_status); @@ -351,7 +351,7 @@ static void btrfs_simple_end_io(struct bio *bio) if (bio->bi_status) btrfs_log_dev_io_error(bio, dev); - if (bio_op(bio) == REQ_OP_READ) { + if (bio_op(bio) == REQ_OP_READ || bbio->uncached_io) { INIT_WORK(&bbio->end_io_work, btrfs_end_bio_work); queue_work(btrfs_end_io_wq(fs_info, bio), &bbio->end_io_work); } else { diff --git a/fs/btrfs/bio.h b/fs/btrfs/bio.h index e2fe16074ad6..39b98326c98f 100644 --- a/fs/btrfs/bio.h +++ b/fs/btrfs/bio.h @@ -82,6 +82,8 @@ struct btrfs_bio { /* Save the first error status of split bio. */ blk_status_t status; + bool uncached_io; + /* * This member must come last, bio_alloc_bioset will allocate enough * bytes for entire btrfs_bio but relies on bio being last. diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 872cca54cc6c..b97b21178ed7 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -760,8 +760,11 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, ASSERT(bio_ctrl->end_io_func); if (bio_ctrl->bbio && - !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) + !btrfs_bio_is_contig(bio_ctrl, folio, disk_bytenr, pg_offset)) { + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; submit_one_bio(bio_ctrl); + } do { u32 len = size; @@ -779,6 +782,9 @@ static void submit_extent_folio(struct btrfs_bio_ctrl *bio_ctrl, len = bio_ctrl->len_to_oe_boundary; } + if (folio_test_uncached(folio)) + bio_ctrl->bbio->uncached_io = true; + if (!bio_add_folio(&bio_ctrl->bbio->bio, folio, len, pg_offset)) { /* bio full: move on to a new one */ submit_one_bio(bio_ctrl); diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 4fb521d91b06..a27d194a28e0 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -919,7 +919,7 @@ static gfp_t get_prepare_gfp_flags(struct inode *inode, bool nowait) static noinline int prepare_pages(struct inode *inode, struct page **pages, size_t num_pages, loff_t pos, size_t write_bytes, bool force_uptodate, - bool nowait) + bool nowait, bool uncached) { int i; unsigned long index = pos >> PAGE_SHIFT; @@ -928,6 +928,8 @@ static noinline int prepare_pages(struct inode *inode, struct page **pages, int ret = 0; int faili; + if (uncached) + fgp_flags |= FGP_UNCACHED; for (i = 0; i < num_pages; i++) { again: pages[i] = pagecache_get_page(inode->i_mapping, index + i, @@ -1323,7 +1325,8 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *i) * contents of pages from loop to loop */ ret = prepare_pages(inode, pages, num_pages, - pos, write_bytes, force_page_uptodate, false); + pos, write_bytes, force_page_uptodate, + false, iocb->ki_flags & IOCB_UNCACHED); if (ret) { btrfs_delalloc_release_extents(BTRFS_I(inode), reserve_bytes); @@ -1512,6 +1515,7 @@ ssize_t btrfs_do_write_iter(struct kiocb *iocb, struct iov_iter *from, btrfs_set_inode_last_sub_trans(inode); if (num_sync > 0) { + generic_uncached_write(iocb, num_sync); num_sync = generic_write_sync(iocb, num_sync); if (num_sync < 0) num_written = num_sync; @@ -3802,7 +3806,7 @@ const struct file_operations btrfs_file_operations = { .compat_ioctl = btrfs_compat_ioctl, #endif .remap_file_range = btrfs_remap_file_range, - .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC, + .fop_flags = FOP_BUFFER_RASYNC | FOP_BUFFER_WASYNC | FOP_UNCACHED, }; int btrfs_fdatawrite_range(struct btrfs_inode *inode, loff_t start, loff_t end)