From patchwork Tue Sep 24 06:09:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988761 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=fmlrgneh; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1539-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTxZ0jqtz1xsN for ; Tue, 24 Sep 2024 16:10:42 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTxT28Mxz2y72; Tue, 24 Sep 2024 16:10:37 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::635" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158237; cv=none; b=XmTsPaJ0x4RZWOfbxzIUkOu8sHxomuDoOQqdYj461yi8l7A9NEJVO9vdwB7EzAulKRFy1p74JDxulteoDGjMZtCxb3NkpI3mjP5herO+1VB9i1RpgYjtTJpeyWyaNHm/ov8LFeYEvqXNqqS6XTbpRwFqYds6yCKq1gDS/xSEc+XOVvxT8oUt5KNKYFWFnHQcYG2iwBE0UJ9pURftRp8d4ibMgDjMgfIQo/wQ1bX5qC9KMUBNAgalxiwa8oBG3cTeHV9KZBU8/CVwDiv5iP3ZCtXaZ9cujbLELfviKpIHbL8wnskpije73Ig6qYiFWXR0sd0LZqC7jMsgqlHPIsNukQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158237; c=relaxed/relaxed; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Yi2AJpMJhqpNTEZJsFagOzxKG57O9Kny3VG1lMEiKPaJzVQPpLsOq6L2L/gVlj7T6jLGVgfqcrvSA8DcRxumU3TP5tLK6RVFCk3kRwJX9RAwDk5GBu8RnEkZzgh/I9nZiKNszsGobywkPrpk8BN2NpyzQIScGKtTQkQ+veS0DDO9chcBRJS6aQAsd/FSkNckQa+/1wdY4zVE88r11KUhCC7yw3cFsoD30X1liosf+YgCXJiIkUJBUQBQRhlheFCnfpTbhp5RXU1zw7lo445452827mgMbpgH4KH5ApkKksIejD0gyfvhudW1womh5rFAC8eBi+AtE+liIgPLutIZ7w== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=fmlrgneh; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::635; helo=mail-pl1-x635.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=fmlrgneh; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::635; helo=mail-pl1-x635.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTxS6rq0z2xHr for ; Tue, 24 Sep 2024 16:10:36 +1000 (AEST) Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-206b9455460so38966205ad.0 for ; Mon, 23 Sep 2024 23:10:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158234; x=1727763034; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; b=fmlrgnehPlpgx6MLI5b42z+8r5uXdx8yG+udyAsTV84PirYP68dhppvEPE3HBnFzdh hkjCZ8l8S6WNa/9C/UKrZvJIw1mdZNlKDs1SzQUYyurHaO1in1N66PZXzWhC9YIcQpNg Ckl7/5XpSz/aiMGZnngI6+EaWPpDO8cxsMPvaNsWnh/VGvJFLr7hBoe4D6wklNQAKMZ7 gNSJhbUzRpCmXIqisOygdyMXeRFTulJwLDhE9Lf9Q62S2Z3k55cyNhpN+JgmLpnm0EiE WFDL0wv0At4ypmKTycDKi/zJmH6hVJR/qLI0E9Uon3VKe1A5MdgQeGiAl8e3q9OOgvy1 3fpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158234; x=1727763034; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wFV2Gg1HPn9r0uTPIEIMJioMCNlgr9zWJ7S9CBl14ro=; b=HfAM08HM4EDeZJY+snBQKPF08r5XcMYdFq/WQUoG7XpscIkjG52cGt9KVY89S8pCEb KsLR27SIsp+xLzqiZqzv5zXOuqHZZJd6TrpqKBbJef0PoPW2pYssNw0WW3dAD/03xCNO a8YuoqIG4KRMuLgGccFaIf+riDhc5mKI+Fhjnszg5gZZJsHdPf0fh3KBbfDeNu/L48yZ lbCQkOny60n2bDb5XHl8xv+n8dQl4xQWNtWQhtb3AEfo/CFibQZP8jvvZTpNLrRzW+6v OSraiodQsJIMIOewSumiP//lA5DCLRCyDghosZHIcUrXHj048cbO3oDpZrcBEpW/npD3 LkGA== X-Forwarded-Encrypted: i=1; AJvYcCXSPShXcyRnA804RFRC61+rOyG3UAdWmyXQ7p98/RLsWEpMB+GUT4fYGBHYo/UYh5BWebDB6hVRjyEGmMM=@lists.ozlabs.org X-Gm-Message-State: AOJu0YygilyxHecmdAL05rsTn+f1QCMCqALuWdhS/xy9qORdzpPoCt95 wt2RJzkosLveOSpfVcn4Nh1weGXInOyAK6l/is0aGM2ru+3j8Vlqk5+pcl3pA3I= X-Google-Smtp-Source: AGHT+IFurxEbSfxWnkgUvSmQpfQ7iOb3of4fLdIGxM8iu8OEamvgIzhTlXEYzcUx/o7HLvoMu2wctg== X-Received: by 2002:a17:902:ce8c:b0:207:6d2:1aa5 with SMTP id d9443c01a7336-20aed0b299cmr27872445ad.13.1727158234439; Mon, 23 Sep 2024 23:10:34 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:33 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 01/13] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Tue, 24 Sep 2024 14:09:53 +0800 Message-Id: <1a4fea06f8cada72553a8d8992a92e9c09f2c9d4.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. It is applicable for may-write cases where any modification operations to the page table may happen after the corresponding spinlock is held afterwards. But the users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will be read-only/read-write protected. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 7 +++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 50 ++++++++++++++++++++++ 3 files changed, 62 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index 546a9406859ad..9a4550cd830c9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3017,6 +3017,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..262b7065a5a2e 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,33 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + VM_WARN_ON_ONCE(!pmdvalp); + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + *pmdvalp = pmdval; + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +383,29 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_ro_nolock() provides the correct spinlock + * pointer for the page table that it returns. Even after grabbing the spinlock, + * we might be looking either at a page table that is still mapped or one that + * was unmapped and is about to get freed. But for R/O access this is sufficient. + * So it is only applicable for read-only cases where any modification operations + * to the page table are not allowed even if the corresponding spinlock is held + * afterwards. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the pdmval. + * It is applicable for may-write cases where any modification operations to the + * page table may happen after the corresponding spinlock is held afterwards. + * But the users should make sure the page table is stable like checking pte_same() + * or checking pmd_same() by using the output pmdval before performing the write + * operations. + * + * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will + * be read-only/read-write protected. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Tue Sep 24 06:09:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988762 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=HUj4kdfJ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1540-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTxm67Y1z1xsN for ; Tue, 24 Sep 2024 16:10:52 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTxb2Lwcz2xHr; Tue, 24 Sep 2024 16:10:43 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::62b" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158243; cv=none; b=V7UQV5y5QqcMq2JRN+L1i+KDJtb+wjcX72lhhLv829CIjFO62UACQoFmBGWch+en12Cy623vuB67SbRPsxe8HrL1xa39BYxnMBtbVOwu8p57ZVxOYY3DS2jsi8129wtq77w67sLnhn2iNvZOGLaMfPg3o0eIUQkvqonH42cJ9vtJUoLoWaR7rkmPT9NtLAhoY3YKunicm7R7aT8sgMbgPLKjbKerwx5m4sDphKzXsLknUKaNW2t1cKBk5lVlGbOVWaSAW+qrCjRWDrU4vl2Yr7qBxuo3KtL6AhsaAU46uWJnxQQvbCQTyY1F1LOQgbiZ/d5gcgFzwQwLqPlgXK9hJA== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158243; c=relaxed/relaxed; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UyXu40RIIchObBsgwI0KlYqUyXT+f5wh0s94a9iaqAVGOhpqcdHGiL/Amj3lWFDJRK/G21wfArwETYxoCvAfkttdswFpH4b7cuqTcaFWXTaBfyQbL89Ix/3FmuUgJolwElP/XlNgvMAUXiP78UGxT+I7PhilH/EkC3WToRVQQBNBDMuSwwwnCLo3jBWiNxaXRu9JdlV+SmKuhkgUsMWrl2dlAFXUMAHv9nFRSThUyislsKHEiQvRs2Ttrrc3x/jDzHrsIE+1qN60NxyeSHrO7voQv63/I+qFbYYFXN0xtS3HZ8LXI8ZSODb2bzv1dbr43Xk9ZJheckgwwWpgKaOfMA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=HUj4kdfJ; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::62b; helo=mail-pl1-x62b.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=HUj4kdfJ; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::62b; helo=mail-pl1-x62b.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTxb04GXz2yDY for ; Tue, 24 Sep 2024 16:10:42 +1000 (AEST) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-2068acc8a4fso47159375ad.1 for ; Mon, 23 Sep 2024 23:10:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158240; x=1727763040; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=HUj4kdfJXSzxjeoN9dUT88i0H1T0pzAh9hGSE77z1qHsp9ADNK7+ApaWex4PbHQd2s MzgPsmvFdJzZKxCmWNNkNIupatif2VSROjPEUoGzGlPp2Yl4lQxunry/JbGSBG23PJEc pGaO0mdWWZVLuONAY0YNzoqmxz5UzDujG+1O1ZzMXm2aGZh3ggqvD6hnfgLgXKi9li2G q+DH2WPvY9NCxuPz3JORQ3uNDPDBZZlonfSP/ZYGs0YgX2g/DHp0xfQHxZbuCtx75UBI gUyIgEpfVmQvatiaOAdTFjIoystyaYWDHMfJHZ6SESCcDcLrp6opoa4bn7PtM9OnrJrt d4dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158240; x=1727763040; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=DZ3BnBgqk0PTIwirkZi5QVKvzTxt+J08o+wUFUhATjXDa4fBV83MZxRIwFlrbYouBX dwZWzZY7eOrcBI9nwzs86Lx5E6wL/jEeY72FGdkdM/ydF++9qMPCFusvp0Su7VbRzXT3 vLB4eo+m2V+l+KEuFodJvm5BW2JGpiSgE9MDrNBr8kGvuSIhHhwV20UOWAr0ouLBmpk1 Nw9GdpLfzf1ze0i3nltV16prMf8S7pZAR3awKydu2RgsFL4vl/+Ia5qP0cwnrgaSMnAS R7EiD4fkiKmy9G23alkdYSNgdre/sRybyJ5wYGhKE4AzLKDCg2cd4gxmAa8PB9wFvXYR NtUQ== X-Forwarded-Encrypted: i=1; AJvYcCVe57tyGm1FEC/2cY4XirK2dCJVlyNgBQNC5/PvvAVtfRCJJDymwSCymioTg5WoyF5pL22CIhCyJL/GRTQ=@lists.ozlabs.org X-Gm-Message-State: AOJu0YxAMWdTdUFSLDixILExQ/hI0XNIhr9hjdrh5zIXU8WiImpynKYE d+03FG0Lb0SV6zVeOuQG51dZIIs3M9cCEB2kQYvESh2Z3eRFtXFim8RYufECb7M= X-Google-Smtp-Source: AGHT+IElcy6/dcdtPcpdDMoiIIDu5/E2AQfQDwpdj/dfRLSSVL2tlA+KvKWV7BmaFFYnOD/6ZCb7Sg== X-Received: by 2002:a17:903:2311:b0:206:a935:2f8 with SMTP id d9443c01a7336-208d832a910mr203885105ad.2.1727158240521; Mon, 23 Sep 2024 23:10:40 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:40 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 02/13] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:54 +0800 Message-Id: <597a93841525dbd58fbebcfdc547f2080694835b.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Tue Sep 24 06:09:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988763 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=QgK+qKP6; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1541-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTxr1RxTz1xsN for ; Tue, 24 Sep 2024 16:10:56 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTxk2GLGz2yF6; Tue, 24 Sep 2024 16:10:50 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::636" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158250; cv=none; b=TZbYO7C2rWPekLVDqgsoDO5PkAUw2esO5a4m+sg3U7y5iFZrm6xWmj75icji3zzcEF8Bn2dVkJlKnjleJrCU4Pr5SVTIeu0ybHqr9qJnh3UJ3FndFQyYskjQmsr7/Tz2hViD6CWPT8xO5o4J9WJaEP+UVZHmJSVExZaE6lCyG0GMWtiG4Ju+A+XK3YdAEfGC1/46OXXTuwW6c0JzshJaM2/XRLhxE9GmBhf5wRqhE+Es17CcLWR+9HXe5GZIgL1QFi+McK2As8Qoe9JhRuIoT3uJp2dwKK8wZuQ9N/DJMRiWiIZle/WZKMyc7FA2alAywcFwEDu2jlwe0HGoZnmMCQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158250; c=relaxed/relaxed; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R8Ti5B5b9/3Jfr/rgwq34Pqp4ipCe8qcb2RJk0OynEM0VS9TEueGvc/+Xl8YdIV6EdeSE5KbyBKWfZLGcQ8hlXRwr5oSbSp+RDuOoUV72nwox1VBWau208PEiFBZVUhJkTTa6Y4UdrtOHSOQRzgHj8iyjklLWlyz/H8WCupo1+qPprF6203qx3dKESrstDGAxu+bTeLXUwxcjQdNCUAlBUp69xO4z/1kckYCwXIQHmnxG5eaNrbvPXWbQh4o/oywuTga+xCCQK/acJdJUCUTJozQj8EWpkFRdG43f31BEfYF/CbjbszSGXqNfHcvfF3F/+yHyEMV5zwjsOAt9yrnrw== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=QgK+qKP6; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=QgK+qKP6; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTxj6x2Nz2yDY for ; Tue, 24 Sep 2024 16:10:49 +1000 (AEST) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-206e614953aso49436355ad.1 for ; Mon, 23 Sep 2024 23:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158247; x=1727763047; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=QgK+qKP6Fg7/MXGar4jye1t+yw9K97i0Uldztrqz0SoXT0SxLkotLJRoPxUz8HNMM4 PxaSEyDgIxcfGnxkGUXdo+LJxSJCPlcaYZTj0reYOZJI500r6e6vb2wSBSqfHpXsVYQA 8t8GUJyeQqWktt2AfaW+5VqeNs220rp31EbKPFj450pECqybpYUlaFBDVBUdmRJDm5jw BNQJogXEe1/qlqGVL4/R/cmLfrCWeeU0J6Yai18WzepfKH9VwdekWvP5ChG75Ss+RJjd 23tIST94fAXuAdFB7Yx1jcF401W4uVnYZQmRU/zzS1OMO7tO/ChbYkwuM9D7fS3ObGrX xCTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158247; x=1727763047; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=fvSRgsXyIC588Yn6uojabQKctojEyQ/CPzZMLkzAZ0gGKsHsw7whqk1PGR5B8AzIQ7 Si6zylUCGc5vi6D/hnS9Y9HaD/TMVtYbIs2MaT0/NyLbo1KN9pqFKvxnxRFk9fCzXRni Di6y96o45gy8aUhgbsIus3UUHoWm2Y4MizYETcPiCvXaDuU09f070knA/WbXVhUultOP uxCjicz5ZRNxPyB/sW6s8a1xNMOQiTm1IU9xmgnNLSHA86uo4q0OtE8AOmIfbS39NV+H aKNX/QWqReGfdEJ19DLTPo+GT80GIUiCTNZYqgsHu6DeHwJYvp3WaxjBpHGTFLTFuk03 8HjQ== X-Forwarded-Encrypted: i=1; AJvYcCWgjz1lAipHhsf+he/njGnSaeyGSpDjG3ggBHZPA97fUKmU8Gd6591rG+wBe15dOry2NQxM+6UHB+pE+PU=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzpRH6fJiKkGDjlqacanuYlZS37BVL+72S+mY8pRZj3JtcHwnzS TbOO/COdcfV2Xm54yKrze/9iNa4LSNJrzrHw5c/0x2ja/ceLpJ6xJXz4pOkgC7k= X-Google-Smtp-Source: AGHT+IGm87aqttqV2Aef0OLuq60VY3Mp4w41J7dgMXq2Y+1L52786e2U6HUOab2ZHlW+VZaRsZgruQ== X-Received: by 2002:a17:903:22c5:b0:205:6f2d:adf7 with SMTP id d9443c01a7336-208d83a8387mr177369015ad.21.1727158246651; Mon, 23 Sep 2024 23:10:46 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:46 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 03/13] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:55 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3e46ca45e13dc..6c6ff8722550f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3271,8 +3271,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Tue Sep 24 06:09:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988764 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AIdU9bR/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1542-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTxz492Cz1xsN for ; Tue, 24 Sep 2024 16:11:03 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTxr3xQkz2yLJ; Tue, 24 Sep 2024 16:10:56 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::62a" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158256; cv=none; b=X0wpqzFta2tIkNDmQ71M7cv43tD1vOz94gLDHMzG7ObMwEMuol7dtsPS7RSp2Pvup0PSN6ksnl0vYsqYtToP4JzxI8HpGCYWW6CpTFSEEEjB+lloUy1IZLZz8K5AS4ock+mmM/OsIV4vkoV9xfgsx+mwObpeS/Q24/wbzXEoiV8wq3zNR82PkZ6bf6PprVfUSZccUAfZdI6W4BS+2V/RWuHKMYTAmqaFo+sOLCEkmnLlxS6Mj/D3dyA0pH5d8tLOuqPaDbUQs8jI+3XDvhhlxgklfpC/ukHBPK0URqlby0TEF/seqTBkOuSjxQVBCG3UAK+w74ubYjc9ZceZRM5HKw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158256; c=relaxed/relaxed; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ilek6hB8gcBusT6oee/4f72rhL90eMp22fDOvbX0xgAxHFd9clUvyEirwNMXGWHewDZ4HsfUqMnKMtcPOMaVfLIPc6FrZd1f58cHtnPgPf/Umgdcvsms+SzTWZSlSOu/y3Pp0/7iWzCIMnKb6IuaSo4wzCQgBgW/DF++TH9INqo3vdXApgUFkzox4wRIHVl54LqGZ+TEs0JzYIihwoop+5DeeEtTHV4a+1GKbU5BcLoZWvsBcPuwoKaJrmeyUPkZ56DC/APhg80GWI3W02kUScSnl24D4qFasB8UbiNyxE68HXoHFo/kixkfayhH7A8CQ60ugj7xQx/w6fVPNIUodQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AIdU9bR/; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::62a; helo=mail-pl1-x62a.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AIdU9bR/; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::62a; helo=mail-pl1-x62a.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTxr1mRvz2yDY for ; Tue, 24 Sep 2024 16:10:56 +1000 (AEST) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-2059112f0a7so45869425ad.3 for ; Mon, 23 Sep 2024 23:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158253; x=1727763053; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=AIdU9bR/LeyKmAJGdSdlcaJvLQYB0SIyK1E7FpQzSyRCSFnt3YV6Ro1WPg99R6u5PE gyz9tFBLi9OvX1LSC2Oz7CvszGY9xdElgba6VbBAfyr9HbqiBVb7TYSmVq+Rgm19LorP JD4qpOG2yD0ALbd6iXKaUWShJfzxh0J9xLpdrdgb+fbd9p9kd72ra+2dIXPGUVZZUafB h6b5Wpgw80REpO4JZgrU8+WR29ywTAN8Xwq1r6V4Os6f6bWI+j8iw19+LYu7MUuQKPN2 uMGpb+Ql7c+Ks45ZXzluGY/EJbZf8XzcsJ/cdoKhKF4qy1rnep9FwwrAyI9fodsdM1Jh 8OXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158253; x=1727763053; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=O2W6GiWTevaBtbX3SRrnI3evUCgrPN6KcafkgeUZSzzEb5oVgrG/xuTtsnkvMa8V1M nnVlNv5lVJq/dbl3MG/4mEWtrHuQkApOLhQwJG8TJ9vnykIfRQjVWJydkTdqlP02QpI0 dfHv/eDDTAOVbY8wRDkrVselON8LDta8hthzuV6gEkenXmSohq67oN5qCEgOF98f/se4 T+fwSJ7Avifnitc1ulXu0wH42isqHH18dRkqJmTZK+TB7if0INMWsKSNLhOI4Op+Fsfu 0zZrK+vgoa6tVAW77t6OI6EJ9odcXwiDGAR/n9QP7ePAkGfQ1R6TUAcPP8peYbErPU0i DixQ== X-Forwarded-Encrypted: i=1; AJvYcCW2Ajn8CskeiUrKKScDWViUeW10rkcp4H5k9xNKB4yvFH00TPy7eF+PLiFPKcDIAiSETHAoX5dy15QWp00=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzhklLffvU7swA+O28BwdwwUeEqKAvp/o0g74mbONP5hNnDEhOx KzFI78quRJfmffHazv1s/ak+w5EYUPRKbc2/6pTz+MgtydnOvsW0eoOQzoMSnuk= X-Google-Smtp-Source: AGHT+IEXIad82sT4qTFz/mgMdJh1108B7x8i2K/1dZJvW9IHButimBDIW5JZoOsyX1eaQQUsC7JSYg== X-Received: by 2002:a17:902:c94c:b0:205:4a37:b2ac with SMTP id d9443c01a7336-208d83c8698mr211905735ad.34.1727158253643; Mon, 23 Sep 2024 23:10:53 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:53 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 04/13] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Tue, 24 Sep 2024 14:09:56 +0800 Message-Id: <69e1c4483c2d679d3e2c8446264e5878e5b52a97.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff6..6498721d4783a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Tue Sep 24 06:09:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988765 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=BEhsSh1S; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1543-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTy628JPz1xsN for ; Tue, 24 Sep 2024 16:11:10 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTy23ZQ1z2xwH; Tue, 24 Sep 2024 16:11:06 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::635" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158266; cv=none; b=KbCdOHW6je51gXbBUGC0Y04ppXqYXRKDv/5ag/ZmE0A8yDkZCUjGoSTZDseY/rNdlXrezvfKXvtYMtjJyTmAay0xCmTcKlQiu4OCj4ORIuPM/K6zgyhDUgMNYenIsfG/SePvd1lRn7fzJW+CgcmfEbXNIvx660E+NHvinXNdkgAt7FvYr3cEkzR5YsdOhJ4vvcpIBcq3GLGq7daFa1L1y/YkTsAoaHIRkxJJMXEiz4eeDfxcFBF5OjM/P9dEGKa+Z79zWPuyUqlQG9y5SVUEkkSswW6AAxzOG63e61+eVISACiGjelR9Gl6/TarrxaOT1Lf+PtS1sYYUztjnekewMw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158266; c=relaxed/relaxed; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BNaYxSQiUW0HD1dxTWY2ro2Xy85mhjiIoWQhSoKpwNE8/0pVvh7QVd5Lb3Pag1ee7MDMIHf2/UlbRks4cgYaPRWDpUK8ki0bJQIHwd2cOLuY+5k3g47Dga4Vu3g9awkELT2alMrb47cY6GsvkaQchQKPYZi6v/r0yn4KeQ1PtriDxDABPUqVtmoDRUFUqccw4eLnl+sZfYjHl2lLzf9GnpXLCrBOIZp3EYTY0fYQfvZX+B4ena7Ne+fBsbGbwzED6V/y3OFIbaZF6g0OW2qyTsgODbGzfl6Uk7GMBsUeY467B1hfb1cSll+oRKKyE3JCCI4dPTHTCcYOV3V0ll8uUA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=BEhsSh1S; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::635; helo=mail-pl1-x635.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=BEhsSh1S; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::635; helo=mail-pl1-x635.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTy21FYPz2xMQ for ; Tue, 24 Sep 2024 16:11:06 +1000 (AEST) Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-20536dcc6e9so32524095ad.2 for ; Mon, 23 Sep 2024 23:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158264; x=1727763064; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; b=BEhsSh1StkuPnncnF0iSNvzXTPizpZ6MwlKU4SwmM3k5v2dJjAV0iELf1PGD3uII9D Ed/CKAB6zuG9MCGCGPbveR0FY56fNHuZQcigNIuptEwgFUEb0+v/rEhZPR/SKrGNm+th 2xLFNQX+CLvR4wr36RsbpH5D1Nk+SXgB3xY9N46fOu6R++6u+Sw3eG1VOju8Hj7Xjppw tRh6ZdEcuVEnjU6aCsyG9LqI0E1c3hYqr7MmihrdDvFeZ6kOBuw7+khpb5Sw5MgxFCFl l6eTWd1XPPMe5ggQXyCzA6BOcv0eviZSHmm2OEInyNtVPlRnLvYJIEg1LGtojWT0lCap fFvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158264; x=1727763064; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WqTOjUnPHL3XRjrt8I2PlSfGhzwzG7n3dPRatvbgECo=; b=aYPYrl3XVvDj+nX77LVQ0M72nCgNxY+GrGMbt9e1E9o69/ehpI83msHxdHjYsVv2Qq dhlRr2iorGQ7G58+E6GOIQcFHxdgXsBGAebBs1RaD096dBkpnBTEBLT70g5eBao7A486 jJNDssFLrmGpUpl+5SVOg/Cu438z46K5z7/DXBuY3gCT48GJbsPVn/w8ccvy/7fHjL9r bnFaZzyE3BHdvUSEt1vzU5dvo7NuZlJHD4JYBFV7COKryzEERs32IiBpUOJiZEstF8s2 0fv8M8MrlSB7MaS1OapFG3N+0JWJeaBezbLXG1jgQ5ALBmI2Q7sh7CCOR3pyZTchB+fn Brxg== X-Forwarded-Encrypted: i=1; AJvYcCW5n9BFvawI7d4BvI5eClW52mDcQtBom0D68N4RtjdodBv/hmFqH/OV80BC20G6yZCp9n9I87M2xDk0VY4=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yzz47aZ64s7emkKmeEAqTJg+L7h9PXytEUlsyiO6wb73ZcYzPZf Dix5IrZS7+rNJSFXx/xGdEPXnzwI4DcFgG4QM7I0Z2agpTkfIlhtqHf66BYmm/I= X-Google-Smtp-Source: AGHT+IGIKI1reF9Jfo+dWimmN1cyjgiGD8qn8mcbfIJc2tOglwlNFzTXQQXYEG1VSSXByHdLorgo7Q== X-Received: by 2002:a17:903:944:b0:205:4e15:54c8 with SMTP id d9443c01a7336-208d8440f7amr165611365ad.61.1727158259613; Mon, 23 Sep 2024 23:10:59 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.10.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:10:59 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 05/13] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:57 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In do_adjust_pte(), we may modify the pte entry. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. All callers of update_mmu_cache_range() hold the vmf->ptl, so we can determined whether split PTE locks is being used by doing the following, just as we do elsewhere in the kernel. ptl != vmf->ptl And then we can delete the do_pte_lock() and do_pte_unlock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- Hi David and Muchun, I did not remove your Acked-by and Reviewed-by tags since there is no functional change. arch/arm/mm/fault-armv.c | 53 +++++++++++++++++----------------------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..2bec87c3327d2 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -61,32 +61,8 @@ static int do_adjust_pte(struct vm_area_struct *vma, unsigned long address, return ret; } -#if defined(CONFIG_SPLIT_PTE_PTLOCKS) -/* - * If we are using split PTE locks, then we need to take the page - * lock here. Otherwise we are using shared mm->page_table_lock - * which is already locked, thus cannot take it. - */ -static inline void do_pte_lock(spinlock_t *ptl) -{ - /* - * Use nested version here to indicate that we are already - * holding one similar spinlock. - */ - spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); -} - -static inline void do_pte_unlock(spinlock_t *ptl) -{ - spin_unlock(ptl); -} -#else /* !defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void do_pte_lock(spinlock_t *ptl) {} -static inline void do_pte_unlock(spinlock_t *ptl) {} -#endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ - static int adjust_pte(struct vm_area_struct *vma, unsigned long address, - unsigned long pfn) + unsigned long pfn, struct vm_fault *vmf) { spinlock_t *ptl; pgd_t *pgd; @@ -94,6 +70,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,20 +89,33 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; - do_pte_lock(ptl); + /* + * If we are using split PTE locks, then we need to take the page + * lock here. Otherwise we are using shared mm->page_table_lock + * which is already locked, thus cannot take it. + */ + if (ptl != vmf->ptl) { + spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + goto again; + } + } ret = do_adjust_pte(vma, address, pfn, pte); - do_pte_unlock(ptl); + if (ptl != vmf->ptl) + spin_unlock(ptl); pte_unmap(pte); return ret; @@ -133,7 +123,8 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, static void make_coherent(struct address_space *mapping, struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned long pfn) + unsigned long addr, pte_t *ptep, unsigned long pfn, + struct vm_fault *vmf) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *mpnt; @@ -160,7 +151,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma, if (!(mpnt->vm_flags & VM_MAYSHARE)) continue; offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT; - aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn); + aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn, vmf); } flush_dcache_mmap_unlock(mapping); if (aliases) @@ -203,7 +194,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma, __flush_dcache_folio(mapping, folio); if (mapping) { if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); + make_coherent(mapping, vma, addr, ptep, pfn, vmf); else if (vma->vm_flags & VM_EXEC) __flush_icache_all(); } From patchwork Tue Sep 24 06:09:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988766 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=F6jLDhfZ; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1544-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyD53Hdz1xsN for ; Tue, 24 Sep 2024 16:11:16 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTy441yNz2yPq; Tue, 24 Sep 2024 16:11:08 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::62c" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158268; cv=none; b=EVfsnOIWf6G6WZv/+8HSuCd+C+5lsPmZnbTIZsAp286Apt0GppMoHKLjo1wxukf/WF4zs7uffiJF5eWuoeQWcqgVlgSonPZWWUTDyrsdrNclGYSmnLCB+wSYY99EBdbF07AvnXBBZZigxTn9wmrcEdu7KDO4YsiQ9WPQokU3HxQ32hWq7WRaPQ3ItOLZ0YbJF9aMjsiE1JJBr3YcPGM48RNCF/d4dSLw+MemVtRz2lq+docuwSk3JIsNICzBmICzJS7G7d1aYwoG6ptsD60lqfssKGzu7jyo+SJmVl9B/6UGxvrJEtE++5zvj1+Qh7xJxAH3SkuhGJty+uo3iXxswQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158268; c=relaxed/relaxed; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=izZFv6ljEM4szTPU8XN7seETYpbnualztXbmJrjtGqqtkkxyWpfKMZAQEKmTgBKwLgu5eRLZBdACuQV2bitc3a+1TWuyKHgrShtSoIpL+jtYBYcxV/ZZagM4vfCVrr4rm3bAZ3owfCCY8cpDf55spv0Df3aPgb5xgBDnJKN9JUAI4zMVAcrwaZeZJdXK3fn33COLVIr1X6iRIoZX8z/yFYma89hivGZ7nsH813MgaQ+kXsxXWfeqXJ3InGHaaxd4k4c9UaoA+DEIrOU2jVkuhJC9YsylsRY/B1oiVB2vr6VVm7bEtFyrLGGQAgIluuvHMvUJR3DGgIoeQBTbKawU2g== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=F6jLDhfZ; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::62c; helo=mail-pl1-x62c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=F6jLDhfZ; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::62c; helo=mail-pl1-x62c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTy42P1Rz2xMQ for ; Tue, 24 Sep 2024 16:11:08 +1000 (AEST) Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-2054feabfc3so45793725ad.1 for ; Mon, 23 Sep 2024 23:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158266; x=1727763066; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=F6jLDhfZs25Nx5GvKLylE5bfWMO9nVh0wtMJKZt0ocBa9T7tJMpIIRy+KpsS4AVXLG PCrUOUO+g2JBWz18qC5voixBR9CKYL/1Ge+hqH3xXOS/7TmlWdKeKyNGx4KcnpveIB+a 6B1Yi4IcGXfy3+Y9he72wHpV9aO0vSo802GduS70mW5gngr+xax+Ab0REBBqme8E7eXE daAN7Fcty6OqH3tsgq7N8CkYtQb2DArUi+ALPY35f0gc2xVzWmSo4tWihBny/qkf8SXd fEYsG/sONvl0cQObMdyLzj0/PIgxkgDvBrAgTX87P5FyTlOfXRWyx+ebZweiCB1UrpTt GoZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158266; x=1727763066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=Gwj4IGiq4291mAD7eQfy9ahJRbWPW3Bxiz+6ZMHmsP44CHRp7W6sSlwtEkHUMcAkkM E/7J83zJhkJeBQ6BPQP3VDdxUeVvGsOHyeTrce8VHRYcJksQJ1+UIAwZrmxzxaO0ptH4 2GQnH1FU4R8Tp0IyXU+459A5ZC9/xRx/wa7cUD/b27uIUpVFT2UPxYOsleRBTHLxo0Mh QCYBMEO51zS1rYId/ER80oB+LOTTuEWJPkbmFKjU2SnRnAI+pNvvcyIjne1sCTf9f+LA zaEyaMYApzXaa0mXEaupl0Q7O10eBUSTC2X/2FKP5vSH9qlQrS0e1vRtnhRFUAuVW7Jk UXPA== X-Forwarded-Encrypted: i=1; AJvYcCWAZeNcnwXx4pzUR0WfOfxGt0sikpa6CjG8WmnRr8gPWJ7Rkf5vKbrlnVuMGj37UQsQbBSTvYeF6GL0q/g=@lists.ozlabs.org X-Gm-Message-State: AOJu0YwtrE15NhZlI8Thdb/upelZ31Y7367utiM4uCNzmxUCa+Zhhrri ayRctS7Kgc04VT0zzpg6/lh4+TAD5Z4xkt8gjUhMYvhQkFZwv3nKPeXLk0tjxqM= X-Google-Smtp-Source: AGHT+IE0gKFIx3/KW4QPIJwQt51ZMLGla2X2YU2KRRy7vcX0w9t/15+gAAip1FE70+J7SP+04M4CVg== X-Received: by 2002:a17:902:d511:b0:205:7574:3b87 with SMTP id d9443c01a7336-208d980bb34mr182109485ad.15.1727158265764; Mon, 23 Sep 2024 23:11:05 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:05 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 06/13] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:58 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a245d28787034..6432b636d1ba7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5750,14 +5750,24 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte may be + * modified, but since we will use pte_same() to detect the + * change of the !pte_none() entry, there is no need to recheck + * the pmdval. Here we chooes to pass a dummy variable instead + * of NULL, which helps new user think about why this place is + * special. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Tue Sep 24 06:09:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988767 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=U70qopK2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1545-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyM4g0nz1xsN for ; Tue, 24 Sep 2024 16:11:23 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyB0q2wz2yS0; Tue, 24 Sep 2024 16:11:14 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::636" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158274; cv=none; b=DXX9MYDUUTAsaRRQne5KVibsJpaYFD1FRfK1e6t3LR7G6Dot93HOdlkOjrp6F8cdVlWQ5qnc4/wpKnsoCxZEC9U4J7I5PWLaUczVZIb0Q2NM4V4PiW0mObAdTKhAk21iG5mLu43nptwBEqFwVyi3LvHQaX3MDRpa8sfruXz4UcDhNeammCvEgDcA3S+onYcOy+PpUgDXCP1DoWWvVy77MlwU8yE/k2J2/Rw8rjDifJRvCC9FSgnWGLiQQfK6fqXHoc77jwh04kBNQUwdTIG5/DcbGigsUvVUaKa/wa7UTrDjFg88DWRJEDv7a6pZwsSRXNHx4bzqwvwE3mUQXqx1vw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158274; c=relaxed/relaxed; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YdEH+4PoyvkQDT+wDLcs0fkGu0MrDRRFtU53FmGS0f0/UrUQqY2+Nj/Dh0A31X1nMW6Vk9NlV7dRlgsMCwBcmxJsO/0daC9eI0H4Kkjb2N0vNLvJpcA0pdXn/J9V8wExkikQnsYi/d4NXM4OWq59I+B9b75rLvXsU2jdWEslu8HjZDL9fPuTm1hFO2cxQ4DMhnxEvq841sohKYCXNR0mDOFr62dUq8DRYgQGDNKGNbOuFrc+Ma9nZcMGEfcRFeFL0mmKTfg3ZrzRF7kvJyY2IusOm5E2pztfbtOBceKnsuug85h6BHLBXa94zvs3jfXbzV5ABaJ4xBLvf/AyGf3ZyA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=U70qopK2; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=U70qopK2; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTy95kBsz2xMQ for ; Tue, 24 Sep 2024 16:11:13 +1000 (AEST) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-206e614953aso49438475ad.1 for ; Mon, 23 Sep 2024 23:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158272; x=1727763072; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; b=U70qopK2jz040D576+8wlMjBrnGImjO8EsWB44PkcsYgyFIyGUpWtoH7PTBU1V4jN7 HZX6SNs/miDdmj6YWgWPEsbMZhjVhTlrTfX7xqA2jCWTC2up8R6p2DZVLmVCGtfm4zWi nJPlVsb5IU/vFFy0EiFWaZaKL9J5serhL+9m3jzDZ6dReUkZdYVYg99/omEYi3IMrv7O /zRdRSFXXFCon99tFHWn6g2TQ6ZZFqleVMtg5cIpX8CKj/IxAMIMk74b7WmuedudIEhN veUJC+LilcR+Bxnxo58fUFAvBaVmg+g+HDqT36QnIL367hNVWNW3+B4hI5KLLv8fVB46 3RtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158272; x=1727763072; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dtqkBA88kwvMRDkdpXIuTPjK06Yw44S9bnA3B3TRVKU=; b=mas/IjSuXjtVhV6L5RHZc/GPUEkxsMaJmT2e2juKKCMKUTsqCIKuuAM1DqX77sJ/Px srbaLCqIT/BzP8t7S3hxbZFnjw2kxvZAlG8Sn8arAgP4BwOzsEQI/GIfWA+JXy64a0kg X9ny8ZjzD31nPhpLAzmhisxt41auW6kfyjbjX8r+dHT8UyGjom0G4wjcrEmrpSqu6KkI 5i+QojuDKwvTm/GU14H/J2R9OhuNl51LTIAZ3RUFAlUiEC9qlfLi8vzDGblc4sTUH2Qg cnItRaJc8OOU2WMVISDdgO1AvCoPoUL4HqepHitls0UURwZjuJDoo7exnrGmOpYoM/Bp JGCA== X-Forwarded-Encrypted: i=1; AJvYcCVQpAClRFwYll7VY5Vsgz/dhAYlmiUquDIWhVLVuK9haxE3q8mx4nW+RrarzQAixevuTzUD7vzgVzJ8cNI=@lists.ozlabs.org X-Gm-Message-State: AOJu0YxmHOahs8lLPdKqo7VRUgmUTxEaoF7hr3hNSd1bWpXb0iQP1+f2 1Zd5j4hO0V+CLdGEfMOCZJ4TZtyDXquwgkQowtKA9fWdQGssYg7czgtSdU8TBhA= X-Google-Smtp-Source: AGHT+IGXhl7eW49ih0T9YK4dpESD/v76/2vHb7m+lE9WqCGFaYPlARnoD/c7DbHxV22Za7qr6kwfpA== X-Received: by 2002:a17:902:f54f:b0:205:755c:dde6 with SMTP id d9443c01a7336-208d83cc8b6mr212103045ad.30.1727158272107; Mon, 23 Sep 2024 23:11:12 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:11 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 07/13] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:09:59 +0800 Message-Id: <07d975c50fe09c246e087303b39998430b1a66bd.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. Signed-off-by: Qi Zheng --- mm/khugepaged.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6498721d4783a..8ab79c13d077f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1605,7 +1605,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1613,6 +1613,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1645,7 +1648,6 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, nr_ptes++; } - pte_unmap(start_pte); if (!pml) spin_unlock(ptl); @@ -1658,13 +1660,19 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); - if (ptl != pml) + if (ptl != pml) { spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + spin_unlock(pml); + goto abort; + } + } } pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd); pmdp_get_lockless_sync(); if (ptl != pml) spin_unlock(ptl); + pte_unmap(start_pte); spin_unlock(pml); mmu_notifier_invalidate_range_end(&range); From patchwork Tue Sep 24 06:10:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988768 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=f2fMbpsV; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1546-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyV1YBTz1xsN for ; Tue, 24 Sep 2024 16:11:30 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyJ3J4hz2xfR; Tue, 24 Sep 2024 16:11:20 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::62a" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158280; cv=none; b=hCCXDYE2t7NwoyeU/PMxSHqip1EG8AHUSlINmiNbja1dzJboHwEH+uLslUpjqcEK5PQAtDw/qMMjExx2138O7npzqEzhogAAjiRAUrl5zkkownCaTkFZGCHzJdS4bwk7/Hep9UEx70KRNy4L0TVuU1mjWM6z7osWABY5xOTXOLFhSspYUhSuR9vHE5mqRMiSl7hsJVUg6vvfm9s5h6Ue6w1xHBxbadmmb9SIxNcZR1wydckcZtnPr2dpo68fGuXUotBrfbWdwQs/LvAx0Zw1nRSsCH0swRanoktBmFt+CNOn7UMw6EiFOTNEwWNAnsPuq7DJGLa/IQRqcU+w2Nnszw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158280; c=relaxed/relaxed; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=W63W8O2hQp2rlsUlhLSbDx6bKp2gDpWqH5BvZuhmhZJqCZRjhLwIn4P+g+akfr/Vt4t5S+lg9Sbvi+SPVAtV8ITIOgmGwdhnb3qAk7kektrUwVvO+RBxbC7sGlIccHKX1a1VWbK3ECKE6caaVloHgs1b0i1bElQ6vGr07NSCeHt6mHEEAKMVq1uVUK8qBRdJThyYNRPdnDl/H6JCNeL1aou/miI2+HlHl0iB3ttEUmeLS0F0U0ZeFA0oGJ6W5VakFZigHKh2s8zuqQW/n1lSe210ZnMneQ0HsHjbgfeczCH9KiqTQx3ODBz8IIuQb7YwrFzTz5AU7MGbtgAy+hfJlg== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=f2fMbpsV; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::62a; helo=mail-pl1-x62a.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=f2fMbpsV; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::62a; helo=mail-pl1-x62a.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTyJ0y9Yz2xMQ for ; Tue, 24 Sep 2024 16:11:20 +1000 (AEST) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-208e0a021cfso25485835ad.0 for ; Mon, 23 Sep 2024 23:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158278; x=1727763078; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; b=f2fMbpsVKLLbVWcMBM6hTZ1AzvGQaRne0CDdd4QOQlHiBd/wESq7x3ObhbXmgVbICv YeIKRwmI9zuVnPuLkqL2ze3C00ymuGvfEwsR6MyNMdjQlMuDFAvD0bq7YrVIpkWgYC/z K9N0fRmXu8PYt7Ni4inusRUeugY9ka1KZb7kRmFdv7vSyVjIT2ourrRwr8Sa4hxdcIAp mc+ncVAdhrK4ldF+i8ZilU6mrCBZ8Muf4mZOoQ7+fVOeaO2r2smxuDpRZhVnGfbeFRJB 9NZMI3c4Cq4eUt3Q92sLyeTKWsZHMPiJEvnIbaZmLqLoga4sPbVCxNie+k5SXyoaxePa zrhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158278; x=1727763078; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a5exoCzehntR4bd+ahks+h4WFwEAjFaMLu5nKDHss+4=; b=wCIsFOPUQ9Hq0stQV/CS5jlqjXNWRcHAZZ1uf0IzLcaCMCQySxLhlMf/0QvqC1c9FX 44nkTUktjwLRavvjEsckwm6zTYZ4uvdw/32KCZPA817F6CGwJCBECUIlXe77LBZAjMtL YyXEcMNgGlBbbxYFXPPuIQNOsgYwdK4Ptc865Krgf22xXmZ5bCQWcY4PWdZpm5OG4J2J Vj7jsfvr1fYGvpencX85f6mXOldaBXvlk4zBIoHC+hjW4NyfzKXVvv57czPXfvaC8nr4 h9eBlo7aXY/mtQTCWuyfushqGxmsXNFKVnsPN5pP5LzbJIRuYVMnXWMWvqCuKRVDKoVB y9Ig== X-Forwarded-Encrypted: i=1; AJvYcCUukIFMWPA1z97M/JP9P/ejwFIMpvWj0oFtH5ZEmT50gbW6sNJej3ELXFH7ph/PojMr271ZI79GSGrI2KU=@lists.ozlabs.org X-Gm-Message-State: AOJu0Ywthdxwexncdf6L7pPQjoKv2DjiIErO8ZiazZgMQPMwVKsZ/t49 R5ratHLA3ohXBvWHDBJiPlOFhzvbMcLTthBo56FoL/XdBQw7A1oVwZqoWedahRY= X-Google-Smtp-Source: AGHT+IHw+OqtuQ5LCBY4YmqC9EJn7stUTlaTSc5oItaKMOqNO6mqJktvnwlzFLYt5xYGl9LymCl5gw== X-Received: by 2002:a17:902:d50f:b0:207:1709:380 with SMTP id d9443c01a7336-208d838ede3mr189786005ad.27.1727158278506; Mon, 23 Sep 2024 23:11:18 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:17 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 08/13] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:00 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we already hold the exclusive mmap_lock, and the copy_pte_range() and retract_page_tables() are using vma->anon_vma to be exclusive, so the PTE page is stable, there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/memory.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6432b636d1ba7..c19cf14e1c565 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1086,6 +1086,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t dummy_pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1111,7 +1112,15 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * We already hold the exclusive mmap_lock, the copy_pte_range() and + * retract_page_tables() are using vma->anon_vma to be exclusive, so + * the PTE page is stable, and there is no need to get pmdval and do + * pmd_same() check. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &dummy_pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Tue Sep 24 06:10:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988769 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=R52qkbps; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1547-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyc40VTz1xsN for ; Tue, 24 Sep 2024 16:11:36 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyQ1n4wz2xnW; Tue, 24 Sep 2024 16:11:26 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::634" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158286; cv=none; b=U04KMNqijiqsDh08UP2btjgldfzNZgyDqg79G3O3v7x3qYBO8xUULlVF4XIyx7g73lUCvimuFcT6mJSk5s34qXK8cRmxGB5sWg54oFfM5rHau+rTEPG9Iu4/cu5+X91oVvUUXVL+7fz07v+moaOuC3NmabrqTvWk58r1pS84maq/c6lS9/bMTNesOqpPTM4ZWRxn+JskbYrCcoXtq3CDZ2yBFlFZJ9pZE/NL5LWc1iSg+Y2AJ/tVRoi+/RUg56091cw5Q7mOclwmYpT0JdoCSSo3n3T7J7rFyP8iqQwTbbqZeXN4VH0asrkZtAQwK0WMmFaUSs1nprvWzCuKcoAAmQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158286; c=relaxed/relaxed; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DZCoi2eQKd9xXCgCfqtekJLqG8wSf9ydBFJlc3pMP89OIC2lYj0PV2jgyMMP+FFXFFxhjoUhYUXAno4j6d7s52NJtqqgeD6EZsYYA6RxieHlPoV86KRy6a4FckK5SVPFVqSNse0/be8HhZXr5Kjyu4owoMP0StTm6xwtWI/9SdMkdZK7VURIdH91WJUzy61QpHo8vJthcIb5W3gfRNX/s/W+1No/uDsVuUbmyUMf1qXbKA4Yy19oYJJ7jNnxaYxIDSwWdyx46d1xqFdJe59CuzTO16Yta3CWShk0BigJIw571L1WAPiQlD52gFARPRKR1ydwK2NonQMwN5WvR3tkaw== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=R52qkbps; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::634; helo=mail-pl1-x634.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=R52qkbps; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::634; helo=mail-pl1-x634.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTyP6cZ9z2xMQ for ; Tue, 24 Sep 2024 16:11:25 +1000 (AEST) Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-2053f6b8201so44771745ad.2 for ; Mon, 23 Sep 2024 23:11:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158284; x=1727763084; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; b=R52qkbpsqEOxvHxY/dGb5vvPjNWqGzJv811yhLGl9y+UvVzSvgfMqy350t+KsmUQuo EvxqIaF4FLlRFNFD6qjRxmH9c5+XXWWhVkLo+PV5M1NPzDRbxNeqw2eG32fRdckg/APt qf6ZLybmNpZyuN8pcY4jbo9j10E42kjxH7tV6wcVeGif5KqemHI4N7+7+nmYDmukrSdK Ryk7gzn9oJiQLQHsg+QluLtyyGXAg1m+uU5fGEaRzpwSu4SxOY51747QuyXY/0lU5zE3 4Gmd/Z8c9WIgmJXCtodN5kmx4D6upYPMDV6CCuURctm0x7FaC1BobsyIzucFaIO7Eoab QHTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158284; x=1727763084; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JYDx6olGBZytK5syFvtWNqesQWJmZ/VcuSiPjSyD9NM=; b=J6M+L8lWB9zTX7vPAXfd6zFzeVgecPN+h7CAAalnJPD40+FJf2RhVcHv9fAP00ybuN RfEgsQh5wKPuPRbf1BgVH6QiwyMYJeK9GlUD/kDd0BrHlYQSiZL/KE+IDn4pcKZeV710 CysmscxMjJLCPBhudqD02p4i9HPMPcQZOy9CW51kHgMolV/oot4TTr8wELIXut3QNXf3 u/l+HV3EvRTAI0pGKk9l8A2uxobkPTAboY5ARZbsz0lnjpxnDAsoQKTiOofPq078faC6 6owoqwi/4rplGeFnqomXa6rheSs1vgN/pe2ik/sRIpNH2eJdjx+zobwhNmlnr1nXObqk r2JQ== X-Forwarded-Encrypted: i=1; AJvYcCVa+tRB2nxKRII9ymNOrBPUYl3wY+ydAdjpDUC5LReTi3Ge3a4i5iDtAWILdGdvraw5Ne/SQuLb2h8yq1U=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yx08q4Rrxb1wbvBcLxJiUh63z5ES5/mmQ+cVY3JZerTZRD3XOOU 671Cof20V+OpCjXEJMfMAMEbQOwqbymbxWpjxovp7bkcS8uaVk74CY7zyaF0Yek= X-Google-Smtp-Source: AGHT+IEhn+iGj1vkzgTAalYoZbG5psMaFd6djM60OU+3J7JbD8pdYTlvGGhg/wgUQ8cNcQFEVSSWnw== X-Received: by 2002:a17:902:d483:b0:206:aac4:b844 with SMTP id d9443c01a7336-208d97e3f78mr215574535ad.6.1727158284398; Mon, 23 Sep 2024 23:11:24 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:24 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 09/13] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:01 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Now new_pte is none, so hpage_collapse_scan_file() path can not find this by traversing file->f_mapping, so there is no concurrency with retract_page_tables(). In addition, we already hold the exclusive mmap_lock, so this new_pte page is stable, so there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..9dffd4a5b4d18 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t dummy_pmdval; int err = 0; /* @@ -175,7 +176,15 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Now new_pte is none, so hpage_collapse_scan_file() path can not find + * this by traversing file->f_mapping, so there is no concurrency with + * retract_page_tables(). In addition, we already hold the exclusive + * mmap_lock, so this new_pte page is stable, so there is no need to get + * pmdval and do pmd_same() check. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Tue Sep 24 06:10:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988770 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AtsJjTF/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1548-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyk6PXYz1xsN for ; Tue, 24 Sep 2024 16:11:42 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyX4cfJz2xMQ; Tue, 24 Sep 2024 16:11:32 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::634" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158292; cv=none; b=cPJGSB/6PR1qNgFvrKoocZHFZGKZU//8cdTWDHObS4XraF+UOCSh/OvciWp1l1wf23sy1nLXr9dy50YstZin4vf3ZYPZKFxzx70MBM2ugmRu2AtvfOpC1wr8OFSoSaGffQsIDwaQEre6mT4qL0rCLSyJWUidw0rq6PHfm7c42u99WXslOWnMw6BcwOiw9Zc7EA0c/OlpIxCkfgnLPJBj4MNs2CMta2NEcCchntN27MsmRrMY+fMWGPQ9c7pWQG2vtWxWLTLXTYN6Wkwd4ik1F7lzysFff2/6Kgvn6rAxop3FBAp/FoQeZEkAfNFquBYhE8N1zyYxbm1u16U3YDgvVw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158292; c=relaxed/relaxed; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LymOT32MKVx88tqvD8mBkFUsxd4kyaZD10zqLbsHSY7X8Rr8hhGo/h5zIq7TyRwrGzgZDw/IzTTjYENBXO4etg29DtoNIlol9kwzu/S/x8gKrdyFQeiRP8WcOaxJmInY65xc7TX4nBQOZwpbx2AJLhRzm0v5OTKjydqMVx+1Z4MBkRejSMOv4g8FWeV9wvxpwI5Ird2/ksM9eh+vqcMqm3pHind8B42sgB0sWsxGOYFzOCJumRfhCzaC2KZLZXnOmE47RH49vOr0HB207g80XOAnCyc9m6GRslnu5alb++e/UcfZQmYwnNCZlcbsj5Dkp9rJfExcPwi4TztCq59+6g== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AtsJjTF/; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::634; helo=mail-pl1-x634.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=AtsJjTF/; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::634; helo=mail-pl1-x634.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTyX2NfCz2yDw for ; Tue, 24 Sep 2024 16:11:32 +1000 (AEST) Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-2055136b612so61987345ad.0 for ; Mon, 23 Sep 2024 23:11:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158290; x=1727763090; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; b=AtsJjTF//wsVFv21yfOKUcJBMEJken34UQKUDTM0hYS17YgoavwY70RWw77nrKXO3a XObMRpKqaYjU9j0E8O0rgbVA9wJd0urSxPzHFxCkiM4cuVpno8+qmTTRuLjBJuuIW8VT 1RGpIUc6+b8rHc4c7IJksCZBUna65VWrvIlNIRGC89i5+ZO21Nicd8hXy+MMwl6yR8fF MqKeksVCVGL0bd45YSskaFnwC60eJpcaUHz1sLaarwS6lBIRbkiQEBLNN31KlC0PXEyD cU0xYO5S0PAPt+7iEpO6yXvEBIzjIiZSMSrWCxE3t0/2o1kFFmK2k1tZghRVzjrpF6ei docQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158290; x=1727763090; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FxMRr8GFLSu76sONoxW9925XRbgCGcQv9BKiL/bcjLY=; b=xQPmWAwiQc+nZ4CaFdQbVrCtXBn4WX3YXyz8z47IUKXBRXMS2XcNb0ik32VG8eEZbI 0DZ9XBLq7gyi73DyaGn0tFhdr4/WUmKDQqQyYHLIdyvcQagJkyorpBJqieazlmRV95cD l2LWa5+FAiWQGKSo8GbIoKMaD1wo3ZEVy+oZhOTze4SbnSg/YdY/aKNAeS1j3K0+Q9mO K3ugN56YZ8E8y7Kd+nmI7HV6YzFI5C58sso3YQhWoUpTrT5EpaoonyGM2DdpGvd8b3je 3aBBnBeJ+FNb7kWtXuWnYNpyC9UhVWgXPNTyZlaQHfKg/BVbPP8ilJ/pphXxKJbGG1F9 5BeQ== X-Forwarded-Encrypted: i=1; AJvYcCVxZkLzKt+ctJZy0981cBC3GneHtBW/pIxeUkCoglgsib5OTTe9WkGc6AbnIawiIbj6Aa+njPZ7AnV4gz0=@lists.ozlabs.org X-Gm-Message-State: AOJu0YxotIA09j9nYKVwN8xQs+hIumxRVyA1IRH5Jup6KObszleUZcxJ 5BUnaqNPZQKzqRD1pUCmzPvk5bqA/f3CnGnhx/dIkVBVLHfFFjTomuNm4gAdBSI= X-Google-Smtp-Source: AGHT+IGuU8VFNPNElLsd/e1qu4NWf2AelC2dd7GFKvCMDCyRlgITelf6+XB9J1CmvT70UykrnIRqmQ== X-Received: by 2002:a17:902:f542:b0:201:f70a:7492 with SMTP id d9443c01a7336-208d98603b2mr202721885ad.53.1727158290456; Mon, 23 Sep 2024 23:11:30 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:30 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 10/13] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:02 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng --- mm/page_vma_mapped.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..6410f29b37c1b 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,9 +13,11 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; + pmd_t pmdval; if (pvmw->flags & PVMW_SYNC) { /* Use the stricter lookup */ @@ -25,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,10 +35,11 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, &pmdval, ptlp); if (!pvmw->pte) return false; + *pmdvalp = pmdval; ptent = ptep_get(pvmw->pte); @@ -67,8 +71,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } else if (!pte_present(ptent)) { return false; } + spin_lock(*ptlp); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, *ptlp); + goto again; + } pvmw->ptl = *ptlp; - spin_lock(pvmw->ptl); + return true; } @@ -278,7 +287,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -307,6 +316,12 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) if (!pvmw->ptl) { pvmw->ptl = ptl; spin_lock(pvmw->ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, pvmw->ptl); + pvmw->ptl = NULL; + pvmw->pte = NULL; + goto restart; + } } goto this_pte; } while (pvmw->address < end); From patchwork Tue Sep 24 06:10:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988771 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DoBB7Hm5; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1549-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTys3Lxzz1xsN for ; Tue, 24 Sep 2024 16:11:49 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyg1scNz2yWr; Tue, 24 Sep 2024 16:11:39 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::632" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158299; cv=none; b=nGsuHN4OeduTd2q549UU8wI126bMWCTES94cL+QXGUlw8b7ghVNI4ANxFsZlqbRx9EMkZM4qA/HOcWoo5o2iolSQ/jqGk1F20les05R1xW0x0eqRBfpMVxrIplHV/EVA/UnGb4tEYBEZI9N7b8WDyw4bW5fIButCCjrdMhM0OVHLhvZT4RlZ/FlYNpzxZGf7cNDVC14ksu/jzbCKUBHZNYQFHPVulqN+AfCyNyAbUGGhF4poEkn9g5joeOygTANnuhfJfv/mGLZKB8skizuT90wlpU1U0WCuiwbG4PqZlIZfdtelkFg/B6d8B/Sx1lK38ryMfjfNUoG6EqcySIXP+Q== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158299; c=relaxed/relaxed; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IiyMMWuu4GmTAdjYBu12XPRy2vKZD6gokCeRdF06jJ5K1d78Du9B9LPbigbmpUq89/qtRYz19gnHPK1eziPfVjOVhE8SydY6DlrdsCeQyTIbO+a6BDENlVKpCjU/89kvKBdCaORuh+2jEam1w0RfgLH6J5SU+hb69Luc57frTkHzsDAg7gOLks7ThL9zYIk+CecWBGMQ8+wRQdypu5QxuJFVZtH7EE8gMTz9oZPWkZvabOeNwlD2bV7cKGr4s43IKTrEqDVabawy14OPr2rlLvyuqxvHUxA9W5YNWMkI46/Ym3utlpKOOU3w+B7K4kl4hPC5X2HOTxtrmyQWJDR7dQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DoBB7Hm5; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DoBB7Hm5; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTyg0GlQz2yQj for ; Tue, 24 Sep 2024 16:11:38 +1000 (AEST) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-20551eeba95so44279745ad.2 for ; Mon, 23 Sep 2024 23:11:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158297; x=1727763097; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=DoBB7Hm5mPKq4GsOE8UFJ/v7MtaTmOxo26f/mfdXqxMez+JSMcsgqf3/5OXj1zorvw c4nyUAbdL+FkuF89d+BFLxLbvif+XZGA0Oec18N0GsOicCP4IofJYNTZ5BzDXPDQeYGb C2iRcfXbuQn7Ht8iPO557nQM6fFWsJYB52hUjLQgBHXCHFrPGrB1weH64gjLHGf+58bH TMvdDnDX6t5VH+UDerRANuk9CfIlvqlniZnsC4KvDojBNtSdH8tSbeP6iSn59jB0V98j AJWP8pRrCfXhRlUL8TcB/a+qlikV4PD7IrlGTc84EqpsqU06a3Prj/DTGEU72HIVftEr zUbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158297; x=1727763097; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=dBBW0g1KVJzbeYxMc1Y+wOwxLs+fX4CGgDLgnmzXQAgHEpNddusE4ZrqWBCpIHSNxz SVn0mvZGJ6KYqEKWnLk/9lxyh8JacGh4gYouKkvkIquPUciFUWAMyldUAF60TgFV+58M ISXuOqZgE4frikkIWNcmwtOSFdESIUMihNSAwa2rTHCXOk6/bQ/TVOJoqXnZTiwDgGA6 jMqQ/oeezxJRZe7UHRwnv0W5J1Z60P/NkHuOotmnrGvHzDu1DtYDufiZy8+/9/5bE8w9 Guq0ZmopMB7LnJoAtkX3K9rC9bdpnmvGRM1V4ynmeCjgednVCiw01AwhpIsZ6MjAW+8i Smxg== X-Forwarded-Encrypted: i=1; AJvYcCWb4RXbvRFhTxbVNgkOFPo84mfEm7sl60e1rRV+zZCnqcF7bpOJLhPNYwfcsYTVKOsTRidipUlO4c0af/A=@lists.ozlabs.org X-Gm-Message-State: AOJu0YwTQo9HvT14CTMoR7JyqaxZwHZAVO/yBLVY13QqywZjpuC0yvZp mnwyXxae/rhpxgMo7PNwBfSBELecHt+Z+VqmeBhdfovmgt8o5UUlhG4m+KbB+Gg= X-Google-Smtp-Source: AGHT+IFBI5Ned/vbTuuRRzdnv79MFASKoi0/m2fLpf4am0H6y8GNq4qW/RgfzVT9no9zcYiBkg/Xzw== X-Received: by 2002:a17:902:ea07:b0:20a:f013:ddb5 with SMTP id d9443c01a7336-20af013e29dmr19152265ad.59.1727158296610; Mon, 23 Sep 2024 23:11:36 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:36 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 11/13] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:03 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will use pte_same() to detect the change of the pte entry, there is no need to get pmdval, so just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ce13c40626472..48b87c62fc3dd 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Tue Sep 24 06:10:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988772 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=ZpstK8c8; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1550-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTyz6SRHz1xsN for ; Tue, 24 Sep 2024 16:11:55 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTym3xrlz2yY1; Tue, 24 Sep 2024 16:11:44 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::632" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158304; cv=none; b=oyo2AAnMHKZZ8wbOwQ8eUIk8GJXAHjHz00OAJ1+dlN8BMFiI5Zq3Cd0QqVwnkzyUewMItNXCjQHVLGGrYENmH84Du1ln2BdOiFSpGe/6UhAB09JbE4nZJAXjtgKjALsgvaMYwFFjPBFoLY846vcFGvE8fkdvTLtnukTaWwS5umdAgNaGp7AtzBhMl4AwAgcF2HmrJNCJS6z1UHM+VUPWU17t1tYFDjsCc41wdIcBZx3TuyJho53F8SVqZSXuSZKlOJWPJ4MRfdTIfMoyOh0swmeQzRAjbhAOv6+3u5rh0h3zKY7dqFf9w1SWoio3+HsY93WyjZRXnSTXmed7zNwUNg== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158304; c=relaxed/relaxed; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KBYxVe5qqL7PJIFJQhYId161hvJBB3Zk38Zp7NwYFctLLnR9Vakmwn6eoQQeFduBCbCz+VObbJOBUUxCClzAHgUjTzptd7hVmLGcL5FsrQFdgfB9DuIOj9d7odS0s/aH6IMb8F/k5w16wEzv5chNX3BfhdcqBPdhDDXcHZqPWQUWnlKsvNbKNYHyfqB+1YifEOYzaxC5dRXPOx6tu5Oes91WLhgLDEmtEUZJY0sm1S2QO2bdvQYItKKNVYnSWZh3rwfYKTAcJBHFs7QHhcmvbaK1BL3EEpIMYFhxac/icNkhazyGMEnFKYXCLyit/sQqBQAH8aVLw5UPkBt/tPZAXA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=ZpstK8c8; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=ZpstK8c8; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::632; helo=mail-pl1-x632.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTym1mSjz2yQj for ; Tue, 24 Sep 2024 16:11:44 +1000 (AEST) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-206bd1c6ccdso48966145ad.3 for ; Mon, 23 Sep 2024 23:11:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158303; x=1727763103; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; b=ZpstK8c8P1Hir9OaEloQZhBnV9LoAZbogF6kO0nSqFCm0AaH6Jnu9Zh00ldIhPmLtH XBQAhewSrJKukhM+IjMz0uthvWF1xtSkC/LxfKMtM4Rqww8p0DurKc0a5jKc1DlbJKYZ lVTtSeUUWQWKhHBYvu/SUztJhNUHZ5mOQL317J3HMJD40l9Qg0iuhTIAKmbPXsjtqJ++ sGqZ+za7kfwbDU5k1z8vAD0BJId/H6mt8heoET/P4xa0uUY7qxKym4QKcrmAgF1X6FT3 AwHHDRWmIKiZOC9HVSbI26EwMaDSSk2su4doRapz0dr+6P7eEbsrRcozULgMMrwyTTX6 T+1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158303; x=1727763103; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fwq8DT6JeNdavhckGZI91GgKWXC+kbG6stmMQ1HgBAg=; b=QbGGeEVZ0mRqdVUCO/oYwghpEEcHSQgfJHqWTSprbUDK0nl4qPy71KwV745QPtRCfL KDMrhHSzZobOyAIMBsxZtqqOOrlI3hQ9mu3HZLsbC0a/8OWVKcDKzICQztUinhzVFkD+ RDPr5379ZKQbcFebcRns5MZYCd3sZdyWRzp/cytFsHbQ5NO878ja41uny95EIdbBUhak GnZrFjrDlotsdjdRBZiXLYGg0PtS3VnLkdMeXGGB/hWmwOA49QRcleNGdIfpWYX/oHwh tftikJ2Qvt/j3fx55bXNOXYKRSQ3l6IVwpByHy6jXyy9X1YPYyGE7iYJ3B7tIzRL0sAd WPTA== X-Forwarded-Encrypted: i=1; AJvYcCWJpU/ZAi6EH97Yb3SPhs795+yFueOlpLV6CtbE8YKKL94L+whG/QJfcWUr1a1AjIHDhcd+dvbey26c2WE=@lists.ozlabs.org X-Gm-Message-State: AOJu0YxIV6YS9e7OdRe+F65y1cTW3jkBoRcYpIj7w1r5iqNJH4XJ8dmo /gpNX5+bGnL0x5/uKKMX0UDYfNhrgT/2tXe1ctjOvBSnw7pZSRDnIIMNflFM7d8= X-Google-Smtp-Source: AGHT+IFl5AsK8a8BNnydzmhf2ct6wAj0Fi3O6gM7MY+2JS5/M4Off4jZQYnq6Yk2iDK5ETk+W1TbBw== X-Received: by 2002:a17:902:ec87:b0:206:b915:58e with SMTP id d9443c01a7336-208d83af7b0mr191498925ad.22.1727158302727; Mon, 23 Sep 2024 23:11:42 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:42 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 12/13] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Tue, 24 Sep 2024 14:10:04 +0800 Message-Id: <6efef5316d7468d13ed5f344452a0b872481972b.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 749cdc110c745..bdca94e663bc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3375,8 +3375,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3384,6 +3386,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Tue Sep 24 06:10:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1988773 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=LchjqvWa; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1551-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XCTz65bGnz1xsN for ; Tue, 24 Sep 2024 16:12:02 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XCTyt5qpmz2yQj; Tue, 24 Sep 2024 16:11:50 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::636" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158310; cv=none; b=iS5Z5a69aA7FtOIBeFYKd3Nsvc2DX6sOXUgl9B6PhxdohBCks7aEvmnHePJ7FAnU6Iy/M5WJAhoQimePE4eMgHRvlJ/gvWM2aV9Xq0r+ax08mUqGZSQ3EO3TtGyhvVP4LXJz5U9NWIi2p7s3p+uP9n1EAsR6CqQs6l0Z8dEK1pp8+NWO+A9zfLnB4x5gOXSfi7ixIoYz7Gxj/3ISk5eqQHslGZDWK5PUoi8Skace4t17VEPFK9sUg9ymgbDJaNDD+4S6PVReyWZIonjWkytAEKb6KxfLdUkp/7+LSOG+zae2Fu+1df3bYtoBrdubJD/D5KpfgkAmIA1OVTT/7bJzww== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727158310; c=relaxed/relaxed; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OzD0/5zeaR2BcmO4kutbjuHqHfdKo6CXe9G7IkHd7511dGOXtyarZU34KJ/hfN/Xr/xDteFh7zzHBgVz6Ya02EWyFJyjGGtxGmceo6QMi79KjXnu219RSFgfQitGmZfWURzhhH9pTULZjzEv2qNb2B7zK9KkMdQ0RaJHlFbR7WWeSBBVXjJeg0XJuA77ViVfAWaCeZci+7BPonnn9M7mUpV2SjjxciBE+Ynaoz0snsuNCmKJsNStACLQnPvnQ5vwA0bfDhR55IAXj/vPwZPyxU1fyY2xlCIUUPasRNaHSmdxaiD2EQFbVMiAVN8nOZaqYj7QKmmjPZl5KHrkFVWTig== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=LchjqvWa; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=LchjqvWa; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XCTyt3XZgz2y8X for ; Tue, 24 Sep 2024 16:11:50 +1000 (AEST) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-206bd1c6ccdso48966775ad.3 for ; Mon, 23 Sep 2024 23:11:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727158309; x=1727763109; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; b=LchjqvWaWM/STtAKIx3Z3Aicx0muPkG0OAIAGajUjIxtCTzrls5WcSJUEDpiQ4LLtl 3VGTIh5N82bh2H4thVMyhrBysIqlhd6Q41SB//r16wXr2ce1sR1hYMBHWlAJJYgPwjc5 YEuvVkX4YYcRP6ht7s1IqPwWTqAb7gdat3cAuSXc/2xy5us00vkXQ7DqIyuB0LIdRfqA 51OW0WICQHVX7t1dfz1Pd+RRB8G7jtGK/cPVj03YK38vwdlvgXTJzFGolVxtZAdRIwXH tCNgnlJB+I5geK9pMybXqatLFh+sXY25I8Rg7Le/Nm2aOvkRrHszcbP7jg89WMSX9YXK mIdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727158309; x=1727763109; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e0T9lvI1pcCPfr6v8oRbivQAnqkZ5MJutHJ+/bNj+/g=; b=wNj8bQBkuAse4vhl452Nfa8HdasjT5uK1upUYM4Y8R+hzrQa+C0jI+PXoyd5p/pHtJ pKjyaAnL2VOeKhUOK7pLiGtDp/xZ02XYUjkruKIdrUaxnnGGn7kKP/DNxsVTOelh6bMf n40+QVQwZQlHcbQNRIQOR/2quVngtpAf0q+uGn7NkEWiV8ngkIC4s0vAalX8Bjf0OaUW ZjHkrRk+0A1Cmvj0VFBTpVFG4MNc+TbQu5Xt58e8iaLUCsgqVjl9RLvGktPmWYfYfDeW 0ijQPSS5TOPEeDG7kIEBsm8hWEgN/aMPyzHIIdlvRBpoOOzV2TvHjRBTO+j0EAh7mTHr PAdw== X-Forwarded-Encrypted: i=1; AJvYcCWUpOMwRe2sLxpPPBMcg4p9QqmQipdYUYjXuFdPflThjMFGqsbNg5dZDk6QR+j2YtksiLEk5YqNmDOyA6A=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzJpQSV5slnSR97hDIfbT0IncrlCgU2eTg6ot/LdAh8JRC/u2mn cZe+0G0s07WanFkKsRGYVb2EIyfbqy4W6rZGsRTLUnTxL/8Q9xl32s1D5VCOixA= X-Google-Smtp-Source: AGHT+IGkJQOlzg3g752NhI9RquMqodCfvTZ7obYXRPzPmN4sDO5P6Za1vWoZJJ9nTCdoHXRNF+rvtA== X-Received: by 2002:a17:902:ec8a:b0:202:311c:1a59 with SMTP id d9443c01a7336-208d83b6ce8mr195135205ad.27.1727158308785; Mon, 23 Sep 2024 23:11:48 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([63.216.146.178]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20af17229c9sm4344885ad.85.2024.09.23.23.11.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2024 23:11:48 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v4 13/13] mm: pgtable: remove pte_offset_map_nolock() Date: Tue, 24 Sep 2024 14:10:05 +0800 Message-Id: <8eb7fcecf9ed8268980d0bd040c0a4f349cbca8f.1727148662.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 9a4550cd830c9..e2a4502ab019b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,8 +3015,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index 262b7065a5a2e..c68aa655b7872 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -374,15 +362,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); * but when successful, it also outputs a pointer to the spinlock in ptlp - as * pte_offset_map_lock() does, but in this case without locking it. This helps