From patchwork Thu Sep 26 06:46:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989630 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=gXAsE39G; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1605-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkfd3wD9z1xt6 for ; Thu, 26 Sep 2024 16:47:05 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkfW41Fmz2yVM; Thu, 26 Sep 2024 16:46:59 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::732" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333219; cv=none; b=F5nt5kRqS2m5X0o25w8TvgWwDWRLBVvWeCuSL0RDb8YygwTAnxg5To3hU5RsQcKl7+0apz3nd4f5f0ZwjhsvGqU4RznFzplo1dbvW4EJ8CJdNrXNIfxu4Qh41TPE/g7IK/CzOe2QE16CpgBaLIJq4mZs2mvB8akXH64QKKVfSG2jYV++XV5RlPhNEBuzFlT8bmERc9vQnUlUP3KPj1HEABHX8V2w4/EDs2SEEL5O4DlzRmgbCUVjaCHhdHiwvdg5drmAsO9/6lwdmAoXmrupKQg+14XqXbronEGhAxzUU/gROy4x//QZmng/ZS9rb+uOcfuZGMkH6Ai7dEZNTtHXoA== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333219; c=relaxed/relaxed; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=l+tAuu3FiPM/vXx7EiYSAcIdB5cId+vDdo+G+2GyYCARR/gNYD+zO6IT9lD6dg5TiS+s05CeIS3+HRJKfjYqt/mrTOOaAjf/LcLEWD5EONIwc2qeYmNcCPlSsS1gBFRJXWEzAe2I4+0qMWqUEtcgNQOmiU6zpq9pxwuxpLJY2t7WINu9dNTAsZrMATO9tyotNoxAx69shn97ItUMTsZGDYCtpu/MROTdMEPQD4drdbWpe/Q9wjYHfdQF2swdcMI5OWRpQR8TBVHMwKftR1BA6N5lISwY4fK+RRC1VX9WGi2fi3dHN9eayv4a5gkWMp2g1DC+LiJixo72t09fEx18VQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=gXAsE39G; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::732; helo=mail-qk1-x732.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=gXAsE39G; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::732; helo=mail-qk1-x732.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkfW1CvFz2yGL for ; Thu, 26 Sep 2024 16:46:59 +1000 (AEST) Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-7acd7d9dbefso65413985a.3 for ; Wed, 25 Sep 2024 23:46:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333216; x=1727938016; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; b=gXAsE39GROAUw0VnmImtgD3VQeTynvT9qxeI69S/HPeh9Cjrulo2uBFVtFA3L5FIPC tR2JA7ILpaslF2yIfHEZy26zZiqJYHS4pnQ7eJlCBS167fGNuytZaj+pwcJWrzvUv7OP VxB5Y4Ltig8cJ/i5Jxkt3pFIkpmsPzcjBm/JtNl8Xcu2kEXRKT0hwbrfCJhzQNYhGO/q xeXTfzb8Pnc5CuNLCEeqzM70bY54UUrAYD4bBXkqwoWsgel8xG6oMXhcRBI5r0oqok82 cI6DVKgHjKio5nwqzuCa12CEGRgaHK283NvDB65h29pPpuaXcxFZ+ly8FnQ6FuhnZjA5 Ou7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333216; x=1727938016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VEMasL/thV9Tj1Tfta4uIol3BYRuzPr3NckO1OF+52k=; b=n7/ra0y2IdXcRQGRyWn4E7MKDgqSPUiwPJGDKXdxLGJESJdMPfqHaFtf0TDfTyrOta VQUpVjXsWaz9gqdttmISvAF/GEAFs8Apl2dJugZLV73vkEJioc9mBwbZ5laXrWzy5Mxm KsVsEZk/3iuWH/P0NZC4sgtIRS6VzPaDJ5z6cRHEayyX0fMcJNghWyEBHdEn4W3v5EIY HlSMeLq8EEBPOe/drCEpm44bXKErnmKJSWgdWiGYnh3qjCRW7qdC8Cg1MaU4sv/NG1v3 8fpPJO7QuYutIM/Su6J/Z00wJkH2Q8P+gi+SBrkQdeps3d8UHg9ikVfWjgjv5S9PE2T2 JUJQ== X-Forwarded-Encrypted: i=1; AJvYcCWyMuyN90SKOWbyP5cFo++5kZ5KKAeXaCGDFc+ITBArcdl+BMrfC0osn6kJzmuGoaYSfy4/+Y9DF8idHyI=@lists.ozlabs.org X-Gm-Message-State: AOJu0YwoS9233G98sc/OLERzEitp535Xs6OdNklYcDPL7bz7ETfyj07L gPGFHjvz3OsP4J1Z1X8abAGdxcjFRgw3jf9IfGrMlwaVhiXbN2g/1Pc9cTQvVsw= X-Google-Smtp-Source: AGHT+IFdBHDg/pcI8fYkiDBKrKz5/bcsNyjF/Vh4ia+kT/ICQCu69/2acL+Eh3ZQbIhYSZdsv978Jw== X-Received: by 2002:a05:622a:20b:b0:458:4b6b:ff16 with SMTP id d75a77b69052e-45b5def4888mr86143491cf.32.1727333215740; Wed, 25 Sep 2024 23:46:55 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.46.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:46:55 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 01/13] mm: pgtable: introduce pte_offset_map_{ro|rw}_nolock() Date: Thu, 26 Sep 2024 14:46:14 +0800 Message-Id: <5aeecfa131600a454b1f3a038a1a54282ca3b856.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 Currently, the usage of pte_offset_map_nolock() can be divided into the following two cases: 1) After acquiring PTL, only read-only operations are performed on the PTE page. In this case, the RCU lock in pte_offset_map_nolock() will ensure that the PTE page will not be freed, and there is no need to worry about whether the pmd entry is modified. 2) After acquiring PTL, the pte or pmd entries may be modified. At this time, we need to ensure that the pmd entry has not been modified concurrently. To more clearing distinguish between these two cases, this commit introduces two new helper functions to replace pte_offset_map_nolock(). For 1), just rename it to pte_offset_map_ro_nolock(). For 2), in addition to changing the name to pte_offset_map_rw_nolock(), it also outputs the pmdval when successful. It is applicable for may-write cases where any modification operations to the page table may happen after the corresponding spinlock is held afterwards. But the users should make sure the page table is stable like checking pte_same() or checking pmd_same() by using the output pmdval before performing the write operations. Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will be read-only/read-write protected. Subsequent commits will convert pte_offset_map_nolock() into the above two functions one by one, and finally completely delete it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 7 ++++ include/linux/mm.h | 5 +++ mm/pgtable-generic.c | 48 ++++++++++++++++++++++ 3 files changed, 60 insertions(+) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index e4f6972eb6c04..08d0e706a32db 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -19,6 +19,13 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_ro_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken), or returns NULL if no PTE table; + - pte_offset_map_rw_nolock() + maps PTE, returns pointer to PTE with pointer to its PTE table + lock (not taken) and the value of its pmd entry, or returns NULL + if no PTE table; - pte_offset_map() maps PTE, returns pointer to PTE, or returns NULL if no PTE table; - pte_unmap() diff --git a/include/linux/mm.h b/include/linux/mm.h index e9077ab169723..46828b9a74f2c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3017,6 +3017,11 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp); +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp); #define pte_unmap_unlock(pte, ptl) do { \ spin_unlock(ptl); \ diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index a78a4adf711ac..daa08b91ab6b2 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -317,6 +317,31 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, return pte; } +pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, spinlock_t **ptlp) +{ + pmd_t pmdval; + pte_t *pte; + + pte = __pte_offset_map(pmd, addr, &pmdval); + if (likely(pte)) + *ptlp = pte_lockptr(mm, &pmdval); + return pte; +} + +pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, + unsigned long addr, pmd_t *pmdvalp, + spinlock_t **ptlp) +{ + pte_t *pte; + + VM_WARN_ON_ONCE(!pmdvalp); + pte = __pte_offset_map(pmd, addr, pmdvalp); + if (likely(pte)) + *ptlp = pte_lockptr(mm, pmdvalp); + return pte; +} + /* * pte_offset_map_lock(mm, pmd, addr, ptlp), and its internal implementation * __pte_offset_map_lock() below, is usually called with the pmd pointer for @@ -356,6 +381,29 @@ pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, * recheck *pmd once the lock is taken; in practice, no callsite needs that - * either the mmap_lock for write, or pte_same() check on contents, is enough. * + * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); + * but when successful, it also outputs a pointer to the spinlock in ptlp - as + * pte_offset_map_lock() does, but in this case without locking it. This helps + * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time + * act on a changed *pmd: pte_offset_map_ro_nolock() provides the correct spinlock + * pointer for the page table that it returns. Even after grabbing the spinlock, + * we might be looking either at a page table that is still mapped or one that + * was unmapped and is about to get freed. But for R/O access this is sufficient. + * So it is only applicable for read-only cases where any modification operations + * to the page table are not allowed even if the corresponding spinlock is held + * afterwards. + * + * pte_offset_map_rw_nolock(mm, pmd, addr, pmdvalp, ptlp), above, is like + * pte_offset_map_ro_nolock(); but when successful, it also outputs the pdmval. + * It is applicable for may-write cases where any modification operations to the + * page table may happen after the corresponding spinlock is held afterwards. + * But the users should make sure the page table is stable like checking pte_same() + * or checking pmd_same() by using the output pmdval before performing the write + * operations. + * + * Note: "RO" / "RW" expresses the intended semantics, not that the *kmap* will + * be read-only/read-write protected. + * * Note that free_pgtables(), used after unmapping detached vmas, or when * exiting the whole mm, does not take page table lock before freeing a page * table, and may not use RCU at all: "outsiders" like khugepaged should avoid From patchwork Thu Sep 26 06:46:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989631 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e4BGEqGB; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1606-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkfj4qpnz1xt6 for ; Thu, 26 Sep 2024 16:47:09 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkff6h5Kz2yVb; Thu, 26 Sep 2024 16:47:06 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::836" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333226; cv=none; b=fsqjbsTBU588/3lTyO0tu88xAJgwfsgiPcGUYXht+yWEXoQJ3/dAbySuxDV6IsM2jK+3EPmsUYqBn4wi1hDDkplOqqBM6Jtf22rkkwQbskhMKUNdZ91laRpp3wvFkT3xGP30XxQdNXYabMw76LWK72EL/iMKOEhsM5LY2jO+J9jwZntVzrUhHAkivrXdf+mM65kxUdhXAIFSL5yfuc9faPNS6JUABbgv4dK+ZLj/NGjjUMYPnY2B3IhdUMZcDA/fAKCAKtSsQriAwItIOgdSb2fSs7sL7yfzmNuuv8LFYb6b/DdtLKYos4MF8GthhCRVr7Z6Zh7pE1rEGJGHJsL6IA== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333226; c=relaxed/relaxed; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YP7FbR0ACqWCDz7MQ3VWA1ZyOxBmBjJN05uyLPtsJnwADP9g6xPqf9aXDXG8uIpYMTqhsQAE0Ee+pj3UCdZpHhP5ipXqATnbY99XQj0zSGtrFO5Spaj0I9tb507oVjD0T7eyCZUY5C8JZf0wTU8TAbxqff0bDQUSt0CiWLSYFY9/mvOa7b5vLabnbtBBDIy7QAaD04BQW7aaHeWomcfZa33Eig7wrEM/IJGLak1SlwUVGyBJcCxvpL6sNGF13I4EmthAO7R+ZHwzeAWlTUTLFi7KTB870DTyOwgl+QY1jhlJq35wrzNO9feKLqAWdiOAQRCb0EkCQAusiPy7+IEunA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e4BGEqGB; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::836; helo=mail-qt1-x836.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e4BGEqGB; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::836; helo=mail-qt1-x836.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkff3bnKz2yGL for ; Thu, 26 Sep 2024 16:47:06 +1000 (AEST) Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-4583068795eso4781491cf.1 for ; Wed, 25 Sep 2024 23:47:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333223; x=1727938023; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=e4BGEqGBD9qmba2OgcAyhE5xxM/ZHF7ZRpBNi0WM2kXiaGMkhbKPTdbT1TAIXgMkVD g1O2nvprAli1CPB+fTXADySdNAgLsF61KXobqH/RuayQBfjz3wezbFtRMMHi1qbyi6ox 7TB86fCDa7+0wOwv5T5bGOYrfq00785yxjh2Qmgq7veHdE6ZXg7N6P06hNzIHmgd8A6k bPXRQwJ+93Ky0fguy6mrBf1rJ4v/VAD7X+qrzJwBn4ihwz7/B/zDlcpnR/S3687Syg4n IKgehF9v1xYK+zRevUwfF1am9o+Z6uzXkGkufZeuirYItp62G2hg4/iGAwJxL3Fdna6R WJ2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333223; x=1727938023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7WBwKEY9MWquOlo01BPdpzLz4TZdRIsQgPATlcDwMyk=; b=X00mESReNV4JODlZ281hb9+FX152fOv4qknd4lcY2VN0QsqNAC/Fb8Yd3f+0kw1pyU cVq/lqzlOt714tlVJDEQHYHMtWNpPgU2EuIf09d6+eVobVtHDulLqnII2nwiPajUf5Vh 4WNNKTY2FP8gWV/YhxeLp0lyeOYzdUhOpUXepTvtZyCNfaIcUjs2oW4hE1uc2Hzew82/ /2qszMVNFzJhrrDAytWgqdGcgU+B6CWCgVeWRBaBwRLGlo2yJG8Nq64vLUmD92W4VUAi w5dEZODx6Z1m6vAxTK7MWeXYXfx0x5pXJ9TwPfLepaytl90sZ3GjktO1lCYBKOVeLShn TXPQ== X-Forwarded-Encrypted: i=1; AJvYcCXs72Oc7Z9QSY2io9orlm3hOlLpisTxpgoMIkM2VOwtTsJjN8teivf7aBcdXt544oPeiugsTyO/MD1rw9c=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzpbAaDhp4GKw5HliQBikOAxZitR1u/rHPsS69EaUuNCvnppRwa gAEZa/QWhbJ9DntMrmAm0pxFDF6PobUx0jzr36xJgYqs0VHR/chrzCKZCciX2xo= X-Google-Smtp-Source: AGHT+IHKcdRFC9q2GD6595fUPe4NgxH7f7bGHM/1jVFUOYjtanMZU2hJPsIx01XdKDMweQg9NQ43+w== X-Received: by 2002:a05:622a:105:b0:458:4c86:4564 with SMTP id d75a77b69052e-45b5e0952b0mr82475131cf.37.1727333223424; Wed, 25 Sep 2024 23:47:03 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:03 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 02/13] powerpc: assert_pte_locked() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:15 +0800 Message-Id: <42559e042eb6fc3129a40f710d671712030646b4.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In assert_pte_locked(), we just get the ptl and assert if it was already held, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/powerpc/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index 7316396e452d8..61df5aed79894 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -398,7 +398,7 @@ void assert_pte_locked(struct mm_struct *mm, unsigned long addr) */ if (pmd_none(*pmd)) return; - pte = pte_offset_map_nolock(mm, pmd, addr, &ptl); + pte = pte_offset_map_ro_nolock(mm, pmd, addr, &ptl); BUG_ON(!pte); assert_spin_locked(ptl); pte_unmap(pte); From patchwork Thu Sep 26 06:46:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989632 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=VgMptUwF; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1607-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkfr2Lwrz1xt6 for ; Thu, 26 Sep 2024 16:47:16 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkfn3NH7z2yWK; Thu, 26 Sep 2024 16:47:13 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::836" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333233; cv=none; b=Lkwif6jVwIfisxhPONSx0lOVDSF7v52x8mRYwTm/5TF053kEXd/cqukUuHfkMSbuYg6QmsGNh08WC3I9kfytc67oBHVTistj3LtsKKVHgP15cWr+HoXN9WSwkK1iACUgCA19c3DMQ0Jwg0a5QLi/zv3BmRtEoFqGW0Gu9sOKB1tejHzX29WtrR3c/WoDMzSTLj56hvuMYKa4airZV1ITncQMmc2ceC7Itjkx9GTRiEwCC68Co/AUlKeUGNS4NKlxJ1HYClwyFFezpU2l1DT6pilTKx6lY9TvVDLVXCawZUrV2Yvc6kfhBLaV/frK+R3qghp9ytQabNEV9GdXZyVUHg== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333233; c=relaxed/relaxed; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g3gh6h5oMc7zoDEdFTkFpSZ4dmlDveEFEKQhUWpCOYyKHD6AEDZk0XXwZhPDObLSKaZfCRNlnbFzmObqU79Z5bvr9Ki+vAfO75+AeLd3ljJAdoDibZ4O0ScMXm6nbxBICoCh+bXgAJPZvafcDTQADHVmb9nZJTYVmF4Xt6D/FIfWyCMJjz6PKd1Irv+3ttpb8XTMk7A8BGIXP27DWcOKw941XiaN0ko7iJ+bAADbGF2x1CR6Bmw2ZJVcqBtZbpl8OTWAnylYWwyrUG5RfTFsaOVFn/6GYxHarCU/AfY0VGRlRDL6Jq6CDtLkPQdbQbpoViltcy1E50Ld7n/6yvaxjA== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=VgMptUwF; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::836; helo=mail-qt1-x836.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=VgMptUwF; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::836; helo=mail-qt1-x836.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkfn0Fxcz2yV8 for ; Thu, 26 Sep 2024 16:47:13 +1000 (AEST) Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-45821eb62daso3474231cf.3 for ; Wed, 25 Sep 2024 23:47:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333231; x=1727938031; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=VgMptUwFmKybnc6oQnxp7E28eVHSX13DLsf/NA1F5Zk5JbzdrWeLFTu+1ywNZXOotW SGYqbyR53loqDe0iYLpExG+tznNA1qPDgdRGHyr8ILACICXqS/VhtomaxynrxUaJZa49 ZjfUgGWtF2qSIE85ojYDFdMaVSxUILKePqNub+obMpqlOPIMTMUyh1JzaF/HhDj6l2js ACaqcQs5xYz2Ua0NdYsIilgQj7k4m+qz7j3sEsZE+x3kY6ZRkLnToCLxOnYYtVeuKzpM VHxPUQRAOC6gq2SIvuV+rnARTiEDDuLSxafBsG/ol9caPnC4l8lWGrYUxSj+67yaHbKE h0PQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333231; x=1727938031; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BmENsM1LVZMrc0W501TyzYIERT8c75Mhan1AJ9vs73Q=; b=POMjy4c6VneOWrNbjIX7mniSGaO6pFRYG5BMS7Gi3DdcJAQEaSaiCiAGDvKiOjO+tz PiI9Fd6kpI0nlrAMtCI2EiHQ1bzGHF1iMM5ebloGC5hrJf7zh7ZJNnFZ9IzLYE2Y/tFO Uu+6m6XxgYWn2GzqwFaFZI2sA/HEjxMSkS01w5fWrAW4b/W9a9grJus6xFR85rJ42BxI JIBN0aL8fqQcmz8Fp0crYf02K82xipzUPik2mUEkONYSHqiWr3Po84n1qkn45F3rYTwh oWHk8T58M7NYO8D+zYkgcrdIgE8iBOFCqp3lNiEppjw/UgbdkI8pOQWv2H2p2P6WK/Z+ W9qQ== X-Forwarded-Encrypted: i=1; AJvYcCWXA1N+ScaYHt7phDVYdPcKGbTz44bz9cbaXlZUR6um9SzIxhdfRzbfl0j5MeFZ7MFAUcXKe3aIdyIB4QM=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzIm6VIT2Qkg+0Jol8SYsGdQO4I4BaptQmUFnoQs+DMHQlKmyqm d0oHbZs7V9FoTHz4x2Qu/jGDrxZ7UhOiYm+y55vuxcsrQRydwfGcEjNsr68Oip8= X-Google-Smtp-Source: AGHT+IF3jq8XAx4d8MvdG//ydygdKBfb8RG21eWCHGRQrOC65QRa0E/gKmvca15+75cy4x5ki2/uwA== X-Received: by 2002:ac8:5914:0:b0:458:fb8:9dc3 with SMTP id d75a77b69052e-45b5def1dadmr61649591cf.25.1727333230912; Wed, 25 Sep 2024 23:47:10 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:10 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 03/13] mm: filemap: filemap_fault_recheck_pte_none() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:16 +0800 Message-Id: <9f7cbbaa772385ced1b8931b67a8b9d246c9b82d.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In filemap_fault_recheck_pte_none(), we just do pte_none() check, so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/filemap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3e46ca45e13dc..6c6ff8722550f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3271,8 +3271,8 @@ static vm_fault_t filemap_fault_recheck_pte_none(struct vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return 0; - ptep = pte_offset_map_nolock(vma->vm_mm, vmf->pmd, vmf->address, - &vmf->ptl); + ptep = pte_offset_map_ro_nolock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); if (unlikely(!ptep)) return VM_FAULT_NOPAGE; From patchwork Thu Sep 26 06:46:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989633 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=bn+JDORk; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1608-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkfz1mmPz1xt6 for ; Thu, 26 Sep 2024 16:47:23 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkfy5gw1z2xHW; Thu, 26 Sep 2024 16:47:22 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::e2c" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333242; cv=none; b=ngCMTT1HIH5yHMvfrWeGL/jarHA8XMA0b4N9AxIJQicZRAfFMIxLse21Hbvq6JkMH9+i/YuDquslW4yAzSaRoidRGTthpzeAInKT10YI5wybuDYWAq7oUivv9cjbWldLbcLP38nLgiJMvHmHKo07OzvU4R/0NWJhUceIeYv1ssGPsaMt9nBRMEZBtBGu4MQVTNOp4vMTnpLgu8PPZuVA2cgMRlMzgV75kDAfTJmcy2XXW2Thsc3OT5z89/sr5ahGIz+rYFDA0AFhqcXeoFGlQ4fvtmndgoEeVRSCUxj45G23vfhYJnTovAOH8rn41O9Kv606Culrfas5LEuaJcwh6g== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333242; c=relaxed/relaxed; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=A6fJxMG00sJHdDi7Kot6CfZNy6YNLPd6qgskZVdQ1i8DE6/fZGt+FF5e810Adh3HGPxUVSdH+EhpNZs52/nCSBn75/LPnbyFSjBPXMC2lwJI+QkpUyxZw42IxZqDIuFRYaq82tP+KDWkWkL4WpB6ZnVDX7NB7SlMN7hfKIjsYThhicbPiEM4V2e6t574F4OZmEfmGDRjUBCNiA1xeQzEvqNeD+8O8rTuOiEVHQuHGLE13ElCpWrr2HpZLhwwmZnwYYAjcXYSn1eSeA+cEZGNIKxuO632jn0juToqm0UVlfV29zNr70cS8+7+CS5jZqF5BLxcA+gRdozrvZzn54AzkQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=bn+JDORk; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::e2c; helo=mail-vs1-xe2c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=bn+JDORk; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::e2c; helo=mail-vs1-xe2c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-vs1-xe2c.google.com (mail-vs1-xe2c.google.com [IPv6:2607:f8b0:4864:20::e2c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkfy2JkHz2xF0 for ; Thu, 26 Sep 2024 16:47:22 +1000 (AEST) Received: by mail-vs1-xe2c.google.com with SMTP id ada2fe7eead31-49bc7387371so271700137.2 for ; Wed, 25 Sep 2024 23:47:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333239; x=1727938039; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=bn+JDORkxpf+6Zt1oj/0cNwHsq826pqG1y+74PH3TVn76aM1fmZUGjbeJ2Zk7o/7W4 bBrIbd5/kmI04hnUXlfJS+rOSTUnwTaoyTs+WFCHVJxM+UFo6sI+BbvC97xRwzCoApjd cFpVRX2G4kOU9UJbiwoaAfBSC27iHB/cwadlWuQ6qK5OOsAJoXXZ7iUYyJAtKGMbYg7g 9DB2ndk08fMEgkdubmD1omm8yycIT9skQXHhRhamq+LQNfz2QchDdbHhcEocF1zExrsM g8duqSbThUPsY9UNxKV0kzzVg7rYBCowJx8WWUIEx/gxi4dZoAx0+0b3sGr4PJ0rwmEh Zk9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333239; x=1727938039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C6e2rRMn75kFV6Fv258+yl1EB6KpOV48BOzHpMuHQ/Y=; b=HevgL7CiRepraSEGOCGwB5Q/QP3/8F4o9hRnxu1RpzrNB2NBWEGetzU4MdueYGZqwe LXu0/6whBErY4elgTpTcv5wGKR6dK8i/6x80x1xGcKf1qzI6la57l3MVDQiEL4r2DlxO lhqwMxs0aEXlMVpIH84CnwnG4vALP/NubY6q0IRczJ7b7AMysBCud6mku4GurOPN2Vxe 45Uumf65oaifMQpnoKITcEN+DuV5QHfR/r02zsiC+sdJaSnm4cHNWjMGEHDwkyCn14GT jWdaCrRS8+aE0e9LfxHLxPVbdYvb2sC7qfXrU7/6tHzYr+HO5qBvgmTEXU4HjYiJ2VVg xe3A== X-Forwarded-Encrypted: i=1; AJvYcCXl/EfW8nACvraWqoxrPOKvG99xESXBGucOKOkBEx2MHHrkNxPsgu5kklNs4d/gxXqWMBVtzaUYUsXavKo=@lists.ozlabs.org X-Gm-Message-State: AOJu0YyjJqYWmqZXoE+C7H63jJJB2pl6ZAOKmzadi+G4WIiJKisvN4qr FsLvLNt//Io/vxuW0v5tFai6GAWqr2zZ3RXoREIv1wJn2CX+Kfn7Yn4oPav/yWQ= X-Google-Smtp-Source: AGHT+IHnBlRBHwKCxjYpHBiGYKwBIoiWGnfAohABSGODP2aU4XU9V+T3qIbiD3kwEf9HDK6sMKBjlg== X-Received: by 2002:a05:6102:6c3:b0:4a0:1cdb:3eb4 with SMTP id ada2fe7eead31-4a15dd1a11amr7011565137.20.1727333238630; Wed, 25 Sep 2024 23:47:18 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:18 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 04/13] mm: khugepaged: __collapse_huge_page_swapin() use pte_offset_map_ro_nolock() Date: Thu, 26 Sep 2024 14:46:17 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In __collapse_huge_page_swapin(), we just use the ptl for pte_same() check in do_swap_page(). In other places, we directly use pte_offset_map_lock(), so convert it to using pte_offset_map_ro_nolock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/khugepaged.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f9c39898eaff6..6498721d4783a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1011,7 +1011,11 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, }; if (!pte++) { - pte = pte_offset_map_nolock(mm, pmd, address, &ptl); + /* + * Here the ptl is only used to check pte_same() in + * do_swap_page(), so readonly version is enough. + */ + pte = pte_offset_map_ro_nolock(mm, pmd, address, &ptl); if (!pte) { mmap_read_unlock(mm); result = SCAN_PMD_NULL; From patchwork Thu Sep 26 06:46:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989634 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TCmBHdQv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1609-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkg60Mlpz1xt6 for ; Thu, 26 Sep 2024 16:47:30 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkg53GmCz2yR9; Thu, 26 Sep 2024 16:47:29 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::829" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333249; cv=none; b=Vv3ZDf07LTtDthQqMGEXpkcfOANfo154FilkTAyDlXDlta6zhejGtv7RqYiAMEFca071nVkTM2ZNq+7JE+aadavbnzlRcBoMMqPVdBo5xFEgCe8mEcFsd751pTdiA+jwBSAlNcqQTPcI8IRLwAf0WnbN7p65HoRNIYLGdAOP065mbIYxFl+SQCce9ACU2RBUk90cqqTwUvMouANnnPf+0G3BF/2ME1gXt137qG1zuWjwPkVDLWtQM03Y9pmCLZbBcgIDjRHZDVl32hKaQh3MyX1mjArl0QfLsYFdVS/YHFytRQ8Cds8WVmdhX8B9E7IplP32sXA10qdvZLGeBwK7Pw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333249; c=relaxed/relaxed; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hSl6Ug0VLar0+W3uwNMGmDw4DNpxTOP4fSTqX+WCESup+hIHyyMQUo626GZR0dFpwydm+TAgYdsBuLDD3YNRYoi7O58l9iTPOG8U4ZOc7S0oueNb/KIkhAGOG0/9kmVFR9Qo/mn7NjkfdyizejleuKNE1SUgGV0ZOvG2mSKtWCt8+mJZXgdgu8pMzI7aBh4DYqbMTbM/5lTkZW7lKTE4GosTqJBvR6VSSuO7UhBSSNRI/z1WUJLWk1X8qftgA6PI+MZ1aKl+MtH8u4WIh1TUFnNWaCVMSeJ9FjeECLGC6xdlg+du0TUSigwLM6WA0hRCSK2o/kCvWpS9Kqpc1EG2aQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TCmBHdQv; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TCmBHdQv; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkg51JCwz2xF0 for ; Thu, 26 Sep 2024 16:47:28 +1000 (AEST) Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-458362e898aso4395541cf.0 for ; Wed, 25 Sep 2024 23:47:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333246; x=1727938046; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; b=TCmBHdQvdGjhxnXYaryLaJ314VZGjyp++WMHxvAnrRzC0QdqkIwNh3MB8MIaadXrBG FAfnrB4UwPXOYcQtNm25hIJLiydPTEe/ADAhO+zjNMwkc2EDI3ar2cBfS7KW9F07g34x xA6/4jZK5+J3lLgmryXmwnRU8WqWdySWWgiP2RzsvpgHmKH4c7BchgB6meh58ZGELUE3 sku2U6KkaKi2SYABY8zIPZpQRzZeNqjDFd2qWQxcqnW90fMtbF8LR39yM74FZ5fbEH1M x40a07iUI33b4iMEOwWa6dB9/lzgXdGvRTTlGJha8VPrquNXfT4sg4tDsNbXmzT7H47w jDjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333246; x=1727938046; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VXy51M36oXFNN3kWVEF6T1v76BlvE2cEswNH4MTPbjU=; b=eq8VpuNS/QDC8CO+A8ga/JuAaO3iCT/fgdi7WXT/m0ha54AIVVj96h9X3n0r6hQ4EX dT3MbyabckFccFEo6Pqo6yyrWEN5yIMKEMwY+QpiJ9lyke3/iWc8XnPRe28SR3Y2Wk4+ o4yBz6KadYuHMthxCiups6fCdw6SwZzb4TDnpu9Gm4JE+tehUOifpu924WALsKXF48Ib U27BrxXKiq+EKp3Ej4OBzjKGyJZCg0Sn78Su0he0npWwwvYM7Be1BtcsmkQlimN26tHr OSw0+OqWaRV1aMSPAn8xNcPeAxXoTT0fDK2Yfn5LsDm1C8HKPOjTAx+/JidzIfOV9zCL BJag== X-Forwarded-Encrypted: i=1; AJvYcCWM2t1gXBqbN96Tn6IvVvI26bF4liIjeJyHgtwLlNQXyxqZqAUuOSTinekWVQRfvspu2Jb8vXWCPFlaIWs=@lists.ozlabs.org X-Gm-Message-State: AOJu0YznyfF+W51sW7tKviVidfSM0VeSBbWgeu4WmKE+uRKJV6nwYOFz zq+cRuQBJh7LHjXryPeImZvD+CuFC6qHob1019vq8/H5oOIPgWZxIVDEj3xjWM4= X-Google-Smtp-Source: AGHT+IGInGARMP6O0lOtN8I02J26ctBRW9WAU/Ifxw0RvFOYg2EOnEie2juwEDAOYLUuf+d4v6O7qQ== X-Received: by 2002:ac8:5885:0:b0:456:94fe:dcdf with SMTP id d75a77b69052e-45b5def4674mr66807991cf.35.1727333246109; Wed, 25 Sep 2024 23:47:26 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:25 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 05/13] arm: adjust_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:18 +0800 Message-Id: <0eaf6b69aeb2fe35092a633fed12537efe645303.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In do_adjust_pte(), we may modify the pte entry. The corresponding pmd entry may have been modified concurrently. Therefore, in order to ensure the stability if pmd entry, use pte_offset_map_rw_nolock() to replace pte_offset_map_nolock(), and do pmd_same() check after holding the PTL. All callers of update_mmu_cache_range() hold the vmf->ptl, so we can determined whether split PTE locks is being used by doing the following, just as we do elsewhere in the kernel. ptl != vmf->ptl And then we can delete the do_pte_lock() and do_pte_unlock(). Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- arch/arm/mm/fault-armv.c | 53 +++++++++++++++++----------------------- 1 file changed, 22 insertions(+), 31 deletions(-) diff --git a/arch/arm/mm/fault-armv.c b/arch/arm/mm/fault-armv.c index 831793cd6ff94..2bec87c3327d2 100644 --- a/arch/arm/mm/fault-armv.c +++ b/arch/arm/mm/fault-armv.c @@ -61,32 +61,8 @@ static int do_adjust_pte(struct vm_area_struct *vma, unsigned long address, return ret; } -#if defined(CONFIG_SPLIT_PTE_PTLOCKS) -/* - * If we are using split PTE locks, then we need to take the page - * lock here. Otherwise we are using shared mm->page_table_lock - * which is already locked, thus cannot take it. - */ -static inline void do_pte_lock(spinlock_t *ptl) -{ - /* - * Use nested version here to indicate that we are already - * holding one similar spinlock. - */ - spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); -} - -static inline void do_pte_unlock(spinlock_t *ptl) -{ - spin_unlock(ptl); -} -#else /* !defined(CONFIG_SPLIT_PTE_PTLOCKS) */ -static inline void do_pte_lock(spinlock_t *ptl) {} -static inline void do_pte_unlock(spinlock_t *ptl) {} -#endif /* defined(CONFIG_SPLIT_PTE_PTLOCKS) */ - static int adjust_pte(struct vm_area_struct *vma, unsigned long address, - unsigned long pfn) + unsigned long pfn, struct vm_fault *vmf) { spinlock_t *ptl; pgd_t *pgd; @@ -94,6 +70,7 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, pud_t *pud; pmd_t *pmd; pte_t *pte; + pmd_t pmdval; int ret; pgd = pgd_offset(vma->vm_mm, address); @@ -112,20 +89,33 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, if (pmd_none_or_clear_bad(pmd)) return 0; +again: /* * This is called while another page table is mapped, so we * must use the nested version. This also means we need to * open-code the spin-locking. */ - pte = pte_offset_map_nolock(vma->vm_mm, pmd, address, &ptl); + pte = pte_offset_map_rw_nolock(vma->vm_mm, pmd, address, &pmdval, &ptl); if (!pte) return 0; - do_pte_lock(ptl); + /* + * If we are using split PTE locks, then we need to take the page + * lock here. Otherwise we are using shared mm->page_table_lock + * which is already locked, thus cannot take it. + */ + if (ptl != vmf->ptl) { + spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + goto again; + } + } ret = do_adjust_pte(vma, address, pfn, pte); - do_pte_unlock(ptl); + if (ptl != vmf->ptl) + spin_unlock(ptl); pte_unmap(pte); return ret; @@ -133,7 +123,8 @@ static int adjust_pte(struct vm_area_struct *vma, unsigned long address, static void make_coherent(struct address_space *mapping, struct vm_area_struct *vma, - unsigned long addr, pte_t *ptep, unsigned long pfn) + unsigned long addr, pte_t *ptep, unsigned long pfn, + struct vm_fault *vmf) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *mpnt; @@ -160,7 +151,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma, if (!(mpnt->vm_flags & VM_MAYSHARE)) continue; offset = (pgoff - mpnt->vm_pgoff) << PAGE_SHIFT; - aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn); + aliases += adjust_pte(mpnt, mpnt->vm_start + offset, pfn, vmf); } flush_dcache_mmap_unlock(mapping); if (aliases) @@ -203,7 +194,7 @@ void update_mmu_cache_range(struct vm_fault *vmf, struct vm_area_struct *vma, __flush_dcache_folio(mapping, folio); if (mapping) { if (cache_is_vivt()) - make_coherent(mapping, vma, addr, ptep, pfn); + make_coherent(mapping, vma, addr, ptep, pfn, vmf); else if (vma->vm_flags & VM_EXEC) __flush_icache_all(); } From patchwork Thu Sep 26 06:46:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989635 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e/chfbs0; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1610-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgF5FKJz1xt6 for ; Thu, 26 Sep 2024 16:47:37 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgF4N7Kz2yYd; Thu, 26 Sep 2024 16:47:37 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::82f" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333257; cv=none; b=O0Gf7jBIKGPh+s0vo/Fib0ati2XShvsBYFU4nJDYtIE7dO3OZBXUVfm/eqRJrMX4iCGsSDxNFoTrqysPU0PyCy6GAdDRT1qTTz1pijVfpkGmRVHQanqKwudE7TVa219/DvzRo/wYGZQYpV2gtjWyFqguR8Mbtk3Xrg8QdyuUiK7vLZXMojiXWGC/AswCIE1fs868PkXBHcYNPYEOi8KddHCoeoRVRNg9TrN30Tsj30H6SNYRerYU4NLDtpHviDBIHBUueIoU1cWG86llN8j0txbMnopxZ2HMdax0naBOenSrJfwtehoCV3/ADqWytyzui8jweOSqWqiuzILsNWoKZw== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333257; c=relaxed/relaxed; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mY5WkhZpRLVae5kG0AEvK1JqkFNriz57e+UtjhbVsltj+YSQYXi8M1z+zm7clgFJaueT52i7lQXIYCXCaYbmmFfMh6D5zML1L/T/BOvggI5Hbt5wq3JBG4xnuCR6pze7o5yZDxQfRM721DP5ZtgplpYxYDi8DVh7REPCaPCAXKhUBS3hxVQIpqCP8j4U/gnoANCk5+4rYGMwORI7a4AAwv/Pus4qm277mu5/nsIi8E/2JcEjCuTI0yaGkHBUWZEkgI4FcsWN2THYqajZpIcDUPUuZaSWXIooi+lt8Rmq1C/py0QD/hHWvn+yUDtmpba7DCjSTyIS5GTJTun8ADTXlw== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e/chfbs0; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::82f; helo=mail-qt1-x82f.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=e/chfbs0; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::82f; helo=mail-qt1-x82f.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgF1DP0z2xF0 for ; Thu, 26 Sep 2024 16:47:37 +1000 (AEST) Received: by mail-qt1-x82f.google.com with SMTP id d75a77b69052e-45aeef3ce92so3444561cf.2 for ; Wed, 25 Sep 2024 23:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333254; x=1727938054; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=e/chfbs0daHyhFDDZPYteCthf/3Sse9reJeZ46PMamz6l/uTGNlLkdagJQQfbcgBye +vnSGZM/Ac5eunu+raAf9bOp4pFiqAMu9aWucg+5K76t0TAlN1gN6WXXkbk3Rf60ox41 Si3uAirNkVIazpCCewyECm6d+FUvLXCKA5opwYJGf7HBn3p7gmnASDJ5Jt+tNPtXHhfC YPJcUOiZehpLVUXcKT0CHkpaNdXl0qvNJLJNO8kQXUQxFdTXkptMORURfXr8XR3AECHM ZJmnbIbPFRbtpS6c6FQYFpZP1ztzFp9C7cyQr0cg6MTD1t1udRmrZokYKWwkHgktQuy6 5fog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333254; x=1727938054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=88oX8R3xcxvOkzUvzbp2hva7xyyimZKCI3msRz7gwSQ=; b=lQuCSjYBk2mWm2aLwCOuzwgOOYepvvkZ7k5QiiIk5ouC5ckobcAZHIM2yZom/amDXn G/teM88IzWSqhQzx2CKNwbgx6JPxHxzLfUCQWauZKub978ACYdE2QE/n2zYb9Ev/Rohb s//hE9Ldz+RUTcf0LO5RU4lIw9A7oYkO5wdS8goGtQ8NBpdoMpT90HreAS3A1fyZK/Sz oPDTPacj2p7QJEr6d7JTKuX0dUxaMrd6Qrjk1mGQWbJa/WbuHjINrE0mYNRgrmmM77dq iHMhbm7VApoT4+kjIOQ2GzuntWwZ8O654vBdk6EisUIEEcWpA6CLkIBX1uHdbL2Xbwhr Vb5g== X-Forwarded-Encrypted: i=1; AJvYcCUgIOQMwmSJvwbEDH8PiZ65eMVF64wkf6g9bXtEK1k7IivJiC1czgAF5qubFTlR2NjC/NndmS27O3xLg/o=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yzsy95M5zx5sXVt4Ya8Ee42c6lot0OsVz8zIKCLvCi7U1BO16SP YW/KBhiS6dcms6TNd39yvlJITVqCzK8TWRd6ZWMKz3xR6I89rQ+aBjq+4l4nAEc= X-Google-Smtp-Source: AGHT+IEkNfjkqcOSOf8WFqyh7ycpXwLGwDHOuzbRwWfPgE4JYY85w8LbZC4JktZjOBGqNKfXWcA2mg== X-Received: by 2002:ac8:5a43:0:b0:458:3824:a083 with SMTP id d75a77b69052e-45b5def43dcmr69885101cf.32.1727333254117; Wed, 25 Sep 2024 23:47:34 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:33 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 06/13] mm: handle_pte_fault() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:19 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In handle_pte_fault(), we may modify the vmf->pte after acquiring the vmf->ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will do the pte_same() check, so there is no need to get pmdval to do pmd_same() check, just pass a dummy variable to it. Signed-off-by: Qi Zheng Acked-by: David Hildenbrand Reviewed-by: Muchun Song --- mm/memory.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index a245d28787034..6432b636d1ba7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5750,14 +5750,24 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) vmf->pte = NULL; vmf->flags &= ~FAULT_FLAG_ORIG_PTE_VALID; } else { + pmd_t dummy_pmdval; + /* * A regular pmd is established and it can't morph into a huge * pmd by anon khugepaged, since that takes mmap_lock in write * mode; but shmem or file collapse to THP could still morph * it into a huge pmd: just retry later if so. + * + * Use the maywrite version to indicate that vmf->pte may be + * modified, but since we will use pte_same() to detect the + * change of the !pte_none() entry, there is no need to recheck + * the pmdval. Here we chooes to pass a dummy variable instead + * of NULL, which helps new user think about why this place is + * special. */ - vmf->pte = pte_offset_map_nolock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &vmf->ptl); + vmf->pte = pte_offset_map_rw_nolock(vmf->vma->vm_mm, vmf->pmd, + vmf->address, &dummy_pmdval, + &vmf->ptl); if (unlikely(!vmf->pte)) return 0; vmf->orig_pte = ptep_get_lockless(vmf->pte); From patchwork Thu Sep 26 06:46:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989636 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DIvJIVgv; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1611-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgP2S35z1xt6 for ; Thu, 26 Sep 2024 16:47:45 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgP1KsFz2yV3; Thu, 26 Sep 2024 16:47:45 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::833" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333265; cv=none; b=lrgOrAu5vNG7Yt288WAuZJZLrFqvkEy6U7gQVQUnBavdzNA4Oqv/CmKVKETAMupF0jnRzXZQaFE2VCY36tyXVxfqU4+rd8bjN2EZNbgbyU8iAm8EtNJOQ15et497HcbDv9G8d0Eau2gUbkHqA4Vv1zNPchqi/nSTkgPh1Oh/byEFQ4iKYd+Pr5hQVPTAfrTb+LTlzYEIX1J++RFaXY0f8U+DjP1buV8DCnExTUYg/hFlV/2kSHUMIOG06nxcssYhA1lDz4rBh7YxNcwssCGx9kIszAbBA2EFq5ppcAz9Oyg+kyaRtDoZp5H28GGSdp5JUPPtHenu6ChUwX+5WOrzaA== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333265; c=relaxed/relaxed; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OgG3AsN+09pLKu64CPVSPnekOP1aSv7ZLDxCosbrekiXjRllZJLza6kyCa0mi26lu0RPXX09wbK2TbEJ2GqCNtpe+gpOcbFN4+S8nzZwJ+lneAxwwlm6t/+kjkPFIp7SHRligcKiopc1gX8IABzAJtaGoTKYcKZZzh9Ndxe5Ms/dI2sftLdxKS49ob0AXy8Gg0weVI4s9Knn2RB4xFvJbnoz4voNn+BxpZ4dpirVTfFsgpQVi18/HU0/tvmzAG0HxDKtYSamsafn+8ZucH/jEK6/LhiXcwVSTAjY9vvRJmND3fmyV4YYlcQC0GdW1/gHKoDbs8fmGj01c+ZEKaVL/A== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DIvJIVgv; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::833; helo=mail-qt1-x833.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DIvJIVgv; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::833; helo=mail-qt1-x833.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgN5FQ6z2yTy for ; Thu, 26 Sep 2024 16:47:44 +1000 (AEST) Received: by mail-qt1-x833.google.com with SMTP id d75a77b69052e-45816db2939so4046211cf.3 for ; Wed, 25 Sep 2024 23:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333262; x=1727938062; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; b=DIvJIVgvmrwDx49Ozbx1Npq3FNMFhRlnQ/X4EvfYYXSwRt2iukFh3zP2C2+dAlF4ym 0mAqa3WpUiTR8MaI97yaDvEDxAHmhmPI6ZIKjf34FJ04KYe0+0nV3WRpDs+Dnr/RUkvN /CgmCcX1jO5Su8OlmR/AnBL53RoLvqowFrnxQhy87MKG8BhX9rzuqwILgh1mKCvOG3Fl ldClPZkQfQSIctJilIzyUCymEPy8k5cBbF5ObA0N0Lo6wUT9KEn5sjxTEG48FMMuWMeK Y67pq34aggItW+QUh3/4v7FR53bEVc/w9toDvdc6FTYqzq1LxAQq73UQ3VuOdYd4RvS2 CcUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333262; x=1727938062; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ffdIJrLHJ9HLo1zmy1fROVnOG/sDLfcmik2lSMoQorU=; b=peeGmqOLzLRtnNf4eRzABSWwxTTgz2l0OZrcoQ5z9y1G2r51+x58wfMeYpK1C4Zn9o M7e8vN0UYw2l1/wkXKM2vzQUn5nl0TPoxlpiKeLAKUAmYjT0VZX77jf3NcbDjBEfhW4v IPNRA47uMUEIx9gkdutu+yeFqIWiyQXvtS1r4ChtrF51e4/XIprWxW4vG+KrwE+4c5Sa /2th9IuhqM4coVqRhqbNlab7ceol2KxiNXxjyId8ClmKrJJ52BRYxt/Vyhvwq8r0jofM VnfwTUjmNI2bvP0O8q5kboeJFgxiGsHLk+nT0pLHtyQsMFWR9nc2N2kQHI3TGU5lIsk8 OmEQ== X-Forwarded-Encrypted: i=1; AJvYcCUTX19rDT077ti5Ss2A57sWkSYmdnmqUvrnH+bzIm8UTtl0yawLyIlVCDKCQHkkbQEXav4JKdvjmATvaUo=@lists.ozlabs.org X-Gm-Message-State: AOJu0YwUfIuVdSu0B0p8/IrNdCNCXgjfbVBU+lO9dM9s2qDgknAmCYRz NV4/bWhprBqYwb8iz31krFRcXO6853roeJRgcBtz0TOf/af0GUtyW4bgzPTyKzU= X-Google-Smtp-Source: AGHT+IFPPE2WMfZ0ye6LGa3/R8zXQ/6KjVtS14uw6772ox/YEy7cH2DCZDAwEci91jHXPd4jVDRFUg== X-Received: by 2002:a05:622a:1493:b0:457:c747:cf3c with SMTP id d75a77b69052e-45b5e0534ebmr99370671cf.57.1727333261763; Wed, 25 Sep 2024 23:47:41 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:41 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 07/13] mm: khugepaged: collapse_pte_mapped_thp() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:20 +0800 Message-Id: <055e42db68da00ac8ecab94bd2633c7cd965eb1c.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In collapse_pte_mapped_thp(), we may modify the pte and pmd entry after acquring the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the PTL held. So we should get pgt_pmd and do pmd_same() check after the ptl held. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/khugepaged.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6498721d4783a..854577f39957d 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1605,7 +1605,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, if (userfaultfd_armed(vma) && !(vma->vm_flags & VM_SHARED)) pml = pmd_lock(mm, pmd); - start_pte = pte_offset_map_nolock(mm, pmd, haddr, &ptl); + start_pte = pte_offset_map_rw_nolock(mm, pmd, haddr, &pgt_pmd, &ptl); if (!start_pte) /* mmap_lock + page lock should prevent this */ goto abort; if (!pml) @@ -1613,6 +1613,9 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, else if (ptl != pml) spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) + goto abort; + /* step 2: clear page table and adjust rmap */ for (i = 0, addr = haddr, pte = start_pte; i < HPAGE_PMD_NR; i++, addr += PAGE_SIZE, pte++) { @@ -1645,7 +1648,6 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, nr_ptes++; } - pte_unmap(start_pte); if (!pml) spin_unlock(ptl); @@ -1658,14 +1660,19 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, /* step 4: remove empty page table */ if (!pml) { pml = pmd_lock(mm, pmd); - if (ptl != pml) + if (ptl != pml) { spin_lock_nested(ptl, SINGLE_DEPTH_NESTING); + if (unlikely(!pmd_same(pgt_pmd, pmdp_get_lockless(pmd)))) { + flush_tlb_mm(mm); + goto unlock; + } + } } pgt_pmd = pmdp_collapse_flush(vma, haddr, pmd); pmdp_get_lockless_sync(); + pte_unmap_unlock(start_pte, ptl); if (ptl != pml) - spin_unlock(ptl); - spin_unlock(pml); + spin_unlock(pml); mmu_notifier_invalidate_range_end(&range); @@ -1685,6 +1692,7 @@ int collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr, folio_ref_sub(folio, nr_ptes); add_mm_counter(mm, mm_counter_file(folio), -nr_ptes); } +unlock: if (start_pte) pte_unmap_unlock(start_pte, ptl); if (pml && pml != ptl) From patchwork Thu Sep 26 06:46:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989637 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=Q7YlhNPE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1612-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgW6ySfz1xt6 for ; Thu, 26 Sep 2024 16:47:51 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgW5j3Qz2yY9; Thu, 26 Sep 2024 16:47:51 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::732" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333271; cv=none; b=LoQbMCbvOPs3gsFYASFutXNHMj9LtgrhIP3Y0Vus0vzkTEhNd83sOnZjFETZ5wynOgb7f5d0ssaHb9yacmp61vM+kZkl9XeE3k9J5yU4B4+9mspbmvwef2OoxJVO88R1aLjAS1AVO03NCmmyUrK1xfgzSQ4Xb7xvWhVnfzkV5qoUi+s33hFhtbG9YKK4LIjsGnaT7fXcraww5w7P9unoku9Yk4Aezd4SWlv4NlFOUEYr8LtGSFunRamYKlIDv0VQvejzittIH4+kbMTqItQjbznQDXx+i1KN1NSFpJj5EccOP9fFgoZ0YGSgg/HZ+xgWqsnByn7903hVqb8ceNdf8w== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333271; c=relaxed/relaxed; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iyGdKZPXrdzNeqo+aRMmC+9ERrehNhtX39YBKBalxvF8RjZ2Dan7OQo3PC0AGHdMTy5oiMrrFR6G/YsjBuamuMFYO8V81t7DaBujfEddQyThtiPLREFB4EmmtsEpcV3bA+RXoCTckd+vpvwunIgNhHIu2NAFCG97TkWHQvo6CZq05/F9+iSiQASTEU8KkrjY9pFu1KLjpGNs7PmtlzZNYTfDt4jSmAXWcQ2y6//AwZz2Ta19PNuYnsT0PkvK4ci6XMYiZoOYlFY9IZhCu7BTPZ8sXAybt8CtGcEI+rWu887QdbqtofyUrsZOmYO9eYGPMCyeucdWY92Lpxk5gJDK8w== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=Q7YlhNPE; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::732; helo=mail-qk1-x732.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=Q7YlhNPE; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::732; helo=mail-qk1-x732.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgW3D3qz2yTy for ; Thu, 26 Sep 2024 16:47:51 +1000 (AEST) Received: by mail-qk1-x732.google.com with SMTP id af79cd13be357-7a7f8b0b7d0so34240585a.2 for ; Wed, 25 Sep 2024 23:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333269; x=1727938069; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; b=Q7YlhNPEG5A1yHkifPQ3PwmKEV6g8tysVZiF1NJAQHSGpFboJQAWiICW8RVFNa5REX pn9ngOY+5apzYEpuf8o7G72pvsTvFpTNRYfzPRsMF1OqN7afjot15ipSKxo2ZpdFCmW4 7W92KND5oqblwiCOTixV7BXETmIj7DhX93lpudq26vcxsR+3/+6gXwlryr5sZzNbmWiT MO3hyJGLXLz//e31naCN5JUPC7eHRPhvrai9pWtfcezUZpPXMh8nrz7OBO55N6Yi5o9w Ync1y1Z9ER2/ZfY4pCo2VXzrfKdlCilQcZyY0ev+3Lb7mE/Q8sET41cDFwj8Q7cIQ9rT rRKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333269; x=1727938069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3DwHdqVJkGu7HChE1YzlDZaDZvpQCVqvAB2HVgACkHs=; b=mwi0OCV3L6HA1YQnpJYad9a+E9Z2PvczJcbx0j0TJaIWneI2DBJeeG30gbXmiqBEyh x4OmbgDSxb/Asr/vVeO98NUSozI3CFE/EYnVuznXW5Ez4Bb5Z1gWDD1XouvS2eEs1e5O vNTR1mpVlxYmHkvuRtHlA76i1oDa0UPTyfBq5iHlmHWDXgG9L8hlhwunbFKEUWwYekBb QpsB9Lh97QkS8ZrfQxHh2Eu8iQv3gMRmE3iunvtI3dItNWOIqVGqnmjZTv+zRVMhkE3W JvhL7I8TmSqrnypuE/bvMxLvmpfmF/hC8xFuFDlLMi8sGve3M+Y15WOYqvelZocLRTrf eUiw== X-Forwarded-Encrypted: i=1; AJvYcCU3te531BpGFvjm1cOJ7S6+fyE/Au3aNE3k+hBfo5Akpr2//041w2TlRR0dDNjJEV82Q1jh+nAn1ZIXJIA=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yw+F80ks5SGVqzzuTgymc45iG7SyP65lHLXRmvlxklsPGVz4sqp a+qCHIWK0Qv2Y6NoN9fRB2LCaJ4nsIt2drzVLfP2+zclklv1V7+kPenLz0sFm+o= X-Google-Smtp-Source: AGHT+IGf6yhFgM7V9dSkmNMD310kmtT4NpPr6tml7YSLsMbfjuBNhYOZWImI0+hdwHwJZd31DH/Ogg== X-Received: by 2002:a05:620a:4009:b0:7a7:f18a:e46f with SMTP id af79cd13be357-7ace744e486mr712325485a.43.1727333269291; Wed, 25 Sep 2024 23:47:49 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:48 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 08/13] mm: copy_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:21 +0800 Message-Id: <9166f6fad806efbca72e318ab6f0f8af458056a9.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In copy_pte_range(), we may modify the src_pte entry after holding the src_ptl, so convert it to using pte_offset_map_rw_nolock(). Since we already hold the exclusive mmap_lock, and the copy_pte_range() and retract_page_tables() are using vma->anon_vma to be exclusive, so the PTE page is stable, there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/memory.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/memory.c b/mm/memory.c index 6432b636d1ba7..c19cf14e1c565 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1086,6 +1086,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, struct mm_struct *src_mm = src_vma->vm_mm; pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; + pmd_t dummy_pmdval; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret = 0; @@ -1111,7 +1112,15 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, ret = -ENOMEM; goto out; } - src_pte = pte_offset_map_nolock(src_mm, src_pmd, addr, &src_ptl); + + /* + * We already hold the exclusive mmap_lock, the copy_pte_range() and + * retract_page_tables() are using vma->anon_vma to be exclusive, so + * the PTE page is stable, and there is no need to get pmdval and do + * pmd_same() check. + */ + src_pte = pte_offset_map_rw_nolock(src_mm, src_pmd, addr, &dummy_pmdval, + &src_ptl); if (!src_pte) { pte_unmap_unlock(dst_pte, dst_ptl); /* ret == 0 */ From patchwork Thu Sep 26 06:46:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989638 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=G5th1lKo; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1613-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgh3gK1z1xt6 for ; Thu, 26 Sep 2024 16:48:00 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgh2FNFz2xjY; Thu, 26 Sep 2024 16:48:00 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::82c" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333280; cv=none; b=UMJjwL/oxFhIhTl1nPCY7SKYZLI+EEIEPPABLHTVolmbIijJ4KEuAfEzi4C6uu6mUClcDdbdGYkv8FCRBamxqAb29KOUpJw/8uz8AflhpXlHKQTnFLwh7l0YMgkNkgF1iTw8Jq3+3PCNUSTZ2frpzEbuaMRWjFs2hpRwF0WzJXinXLM7niOBJSQS9PmKZdlcznQKs7AyL/ZIyiTGnBNYGiA+6hI51TdgQEv9yh2WL0vU3MUfuSZ10C13GDGUoxVC2uxOzgcneAzFcPBoNZFPFklL5Kwmb9WXIbFYbImGpdRt/yGMGAzWxE8xCzpMXv/qkW0Dyycjw7dLWi8LqssgxA== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333280; c=relaxed/relaxed; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=C4CFERbpF9NlrjMD+rdQ1scXO4SuuF3fHuPRN5cl1ouAsbr3sYrDVRnCg62xGOkNb2Uh90mCLu9h495qeMEubjLczpZ0jOXs4LUlmNRDqwwKGq4wEZQg09/bC+Y+btjcCT+7J0f5cJwjsY3M1m4d2MBlQgcgW2i64uH6AX6F/YUUajcsXbkr07obNVqRZFXUWiEMRKwu9PlJk012XZ+Dop2L+xiQs7tsEMLU8w6MXtfgIpUC1NqbTHTFT0m/tnJ1WF3OhTXUlSs1JrMVGVuC6UNW8VFnXes3Rf/jeCg0R9LPCEYjtn8FPMsCTopFGGBEGCUXKbv3PNOtbhcz/4qE1g== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=G5th1lKo; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::82c; helo=mail-qt1-x82c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=G5th1lKo; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::82c; helo=mail-qt1-x82c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x82c.google.com (mail-qt1-x82c.google.com [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgg6w0Mz2yTy for ; Thu, 26 Sep 2024 16:47:59 +1000 (AEST) Received: by mail-qt1-x82c.google.com with SMTP id d75a77b69052e-4581ec0e00eso4039221cf.3 for ; Wed, 25 Sep 2024 23:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333277; x=1727938077; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; b=G5th1lKoDQXjQ2mMTMJWS9Wq+rLWCH31TkkzRhd0AfLMCEKFXp8/J0iJyFMPCVmRbs zqqIHMdDtqQMHxiYi/mHKzVXQfjCrbQWkvBV1gAauh2SPd0TMn+WcW5r7nhjHp6TpIDS h6iVmMriG+8pdEbm0mUqP0/o7fFxw7o9fDLqPyLLusmm5m/eD8MTaTuA9zUEJf8g8X1d Ndm8zCAqjCpYLKp4TO9GPS2zYGEgl5i/gWBcwHIgKmZmcyZfgCIQZCRpYXO33EB3/Y8N xKBjxBKKa/ajGGqVKYOcgQBREhTqDSHg4e1pfRQliUphTQ35XeB2Gmi9pgmg2+LtFCae Vn6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333277; x=1727938077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xHtLGGTaDvd+j7nlO+D4nlmKtDmH5wZnqdyAY1Z6B8s=; b=Gwr3Zo7oCDI5qFLUpPuA8EBAJZpSbdLiquoKDA7ntrgjzWF2k/pNFByQ09XF1kvU5g jfvvfs0TUE1QDobSgOKDGggj9HiegS5Jpm+hPs0kzB17TPzNjuGIgurl7884ekzd0/zP tGHPQwo8aT3DojHlJs0cNskBRP8zYTSYlor0eOp/gK7Ky8GMbzcVflO0BK2Lq7CzQyh5 8nyMe8TbKSH1eZTKX13K8WhqSfkJlXoHLHp1PHt4755ggrhNz6TPe0/PxnC+HPDXlcG0 7E7UyGVewZsiFwypTXf8k9u5VBu2e8G3IeZCpx0hrdm4NFkdBjJQiJzndPILsW52PHUh 6bCw== X-Forwarded-Encrypted: i=1; AJvYcCU7tZoevEh54ERkdsIM/TSZ0zyQA+x2b2j0RjZWShKNo1ZGHSFGH7XJvy6HHJrxu1Gc5TFJLix6mWjRR2M=@lists.ozlabs.org X-Gm-Message-State: AOJu0YyjA9CEUnSsJgx/bl2hwTkCeYqkxT4sBloomMy7GsMVJFzRMR3M 9FK5EFn0bOyrl/AeGCkr/NnmzHQL6bPtOVWW125+DWIA72+K5uxNhocrKjI0W9I= X-Google-Smtp-Source: AGHT+IGbo2MSjNAvLC3c2ApJAOpwKD09S+YUwBzMfKcY13i9YN/E+PXjNs0XRR1eOHqqnOUdrqUWyg== X-Received: by 2002:ac8:7d42:0:b0:458:2ce1:26f8 with SMTP id d75a77b69052e-45b5e045ea9mr77074431cf.60.1727333276776; Wed, 25 Sep 2024 23:47:56 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:47:56 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 09/13] mm: mremap: move_ptes() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:22 +0800 Message-Id: <9d582a09dbcf12e562ac5fe0ba05e9248a58f5e0.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In move_ptes(), we may modify the new_pte after acquiring the new_ptl, so convert it to using pte_offset_map_rw_nolock(). Now new_pte is none, so hpage_collapse_scan_file() path can not find this by traversing file->f_mapping, so there is no concurrency with retract_page_tables(). In addition, we already hold the exclusive mmap_lock, so this new_pte page is stable, so there is no need to get pmdval and do pmd_same() check. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/mremap.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/mm/mremap.c b/mm/mremap.c index 24712f8dbb6b5..9dffd4a5b4d18 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -143,6 +143,7 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, spinlock_t *old_ptl, *new_ptl; bool force_flush = false; unsigned long len = old_end - old_addr; + pmd_t dummy_pmdval; int err = 0; /* @@ -175,7 +176,15 @@ static int move_ptes(struct vm_area_struct *vma, pmd_t *old_pmd, err = -EAGAIN; goto out; } - new_pte = pte_offset_map_nolock(mm, new_pmd, new_addr, &new_ptl); + /* + * Now new_pte is none, so hpage_collapse_scan_file() path can not find + * this by traversing file->f_mapping, so there is no concurrency with + * retract_page_tables(). In addition, we already hold the exclusive + * mmap_lock, so this new_pte page is stable, so there is no need to get + * pmdval and do pmd_same() check. + */ + new_pte = pte_offset_map_rw_nolock(mm, new_pmd, new_addr, &dummy_pmdval, + &new_ptl); if (!new_pte) { pte_unmap_unlock(old_pte, old_ptl); err = -EAGAIN; From patchwork Thu Sep 26 06:46:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989639 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=h9G5dQ98; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1614-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgp6wx7z1xt6 for ; Thu, 26 Sep 2024 16:48:06 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgp5lshz2yTy; Thu, 26 Sep 2024 16:48:06 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::833" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333286; cv=none; b=CC+hwA+CiOdtMY2oOQ6bg4djcaZh4Awanq/P9XADEDEJnCR/vwDLF2nYlIcABr1JXVK8L6W42vLm2X21s2UGD7Gs1JSUk5P4V35IhuZmrO8nHsiwZRp6luDuUlQIzKmMgZtJs/Mr25crK7LZiuBQpedVgn/SVKWPZ+ysCkWzJojnGjAClZNnVJ7NPEjsO9b8afsXuS0BKC8S/x/TbJa9UhwVTH+AtrnYVos8SHkt5X6unJ5VSGdk6Xu3d1OT/Ec2EaEy1PhqYXwbMOETLSIPd8XPFQWFFUgsn7rG6C/4pbAL8mIs95tMNHQ3Vu86ZgGDLbg/UbeAJXzwChDLx45p8g== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333286; c=relaxed/relaxed; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Hj1CSLbPmktmAzMvL65AbUAb/dFmYAFKj20k9YdSTfFyLp9D3UoFLQKYiHw94ZKJPo60i7+dMJTiJGYyhzZxAq74WisTwQrO4V2X45pYoZS94tBof5EFFDwx18qaza3o2lXLq52VpFzFvZyRMj1y/fEIABSmBtAvtmS7nvFL5+s51AIArqVPw8WHTPd1azgX0ssgBOnGiqASqg2nb/ZwiF8Cmujn/jnrpYDfOKMaot9Mvnw0srhNiPK417yextiyvyC/CJx2YCitoCo0bMfu8R3g/jvwDjhhgJPJlcG6Fx3dNsBl/NjMXpFuKZnS4ccPTsSVORqlvk8XMNoMWMjc9g== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=h9G5dQ98; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::833; helo=mail-qt1-x833.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=h9G5dQ98; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::833; helo=mail-qt1-x833.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgp2ZFyz2yVP for ; Thu, 26 Sep 2024 16:48:06 +1000 (AEST) Received: by mail-qt1-x833.google.com with SMTP id d75a77b69052e-4582a0b438aso4510401cf.0 for ; Wed, 25 Sep 2024 23:48:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333284; x=1727938084; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; b=h9G5dQ98DcIuL4nU0dZVq3beiw110v4EYwUOoWmnZaLRhbU8J4X/xPc5PbdlHPXVF/ FZkesPvv1senobbAqJb1+mPN4CWtC8l/pAa870VagQCwL8C803Q2DdsaxJWCFunEOm9B ODq2bxS6LZC4J/9gzf2C+ryxM/Y+oGDoejLZgEXDunDp9YC35mwMF50d22H6z+DKuk0J VjKYtARZHnOMnwLuSXRJdJ8YOZ4JZlGTrPmVHpn00E2h7q+8EvAuGmYjsxSwv5u5D4oS T6duigg4yTnFs8X2kruSGMGUx2uytDGsM5KmDFKRwdoHvOGPGCc5oma3Xp17tGMBc7Ot CW+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333284; x=1727938084; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RsEgu/iD0T6JQDFamAgFHiL1Lm0nKVk+wAfYQhZRThE=; b=tRruZFO2U6ZcU4NpSfBNec67FM74vclt/TuT7a7iwSbNYx0dlCAnQgA7Pdd8DUNKAO 0ip/N0wBVM15WSgycc9ZEn9MOPRrrOL5OYY1GOB5DS1nhzmZ8r9+pNd5ARIgHN99yHrS lUhp6TM0zCUgYap/VmqBIATgkLsj9DGWjX5+JMmwvz2uPAuaNpCa/IeUjSLUqcHn1tLH deOKFgZXKtm9N8LTO2UxyGgzoiDOTQ5v9D1sHWLP7mP3WBApywuGycYDnTK2wYNUR9oO sKvohSsSERj7igHl/p2wtzlHvbJv/84xIk/HeXLPwntQkiOm5MGEctQ9P2Yg/o89fBi9 QyCw== X-Forwarded-Encrypted: i=1; AJvYcCUCl1FMINw0e1g1lueSMTA0EkJ2wlHFn8meIuX59gf01/Ux5a+/rnkQ1AUuNs0Mk033CgH93pXFwlQH9W8=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yy/nE97WZbodg43sYgDrlLdqhCQq/KYrY/kniJa9z96EEpLW73K RjkdeLMR2LMlQzJ3dGI/Jzhl9vgk2P1iooDuFxP/pqNDWVNK42bz/6iR8EC8y2k= X-Google-Smtp-Source: AGHT+IGAlBKd7aKCt6XxIcAXM/Sjs0v2+e+7/droKiEQYGe4LhoKPN6bWAepHInU8bkNuwlluGjQYA== X-Received: by 2002:ac8:7d4c:0:b0:458:4aec:2749 with SMTP id d75a77b69052e-45b5e045e56mr87889061cf.57.1727333284339; Wed, 25 Sep 2024 23:48:04 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:03 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 10/13] mm: page_vma_mapped_walk: map_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:23 +0800 Message-Id: <2620a48f34c9f19864ab0169cdbf253d31a8fcaa.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In the caller of map_pte(), we may modify the pvmw->pte after acquiring the pvmw->ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the pvmw->ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pvmw->pmd. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/page_vma_mapped.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index ae5cc42aa2087..ab1671e71cb2d 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -13,7 +13,8 @@ static inline bool not_found(struct page_vma_mapped_walk *pvmw) return false; } -static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) +static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, + spinlock_t **ptlp) { pte_t ptent; @@ -25,6 +26,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) return !!pvmw->pte; } +again: /* * It is important to return the ptl corresponding to pte, * in case *pvmw->pmd changes underneath us; so we need to @@ -32,8 +34,8 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) * proceeds to loop over next ptes, and finds a match later. * Though, in most cases, page lock already protects this. */ - pvmw->pte = pte_offset_map_nolock(pvmw->vma->vm_mm, pvmw->pmd, - pvmw->address, ptlp); + pvmw->pte = pte_offset_map_rw_nolock(pvmw->vma->vm_mm, pvmw->pmd, + pvmw->address, pmdvalp, ptlp); if (!pvmw->pte) return false; @@ -67,8 +69,13 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, spinlock_t **ptlp) } else if (!pte_present(ptent)) { return false; } + spin_lock(*ptlp); + if (unlikely(!pmd_same(*pmdvalp, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, *ptlp); + goto again; + } pvmw->ptl = *ptlp; - spin_lock(pvmw->ptl); + return true; } @@ -278,7 +285,7 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) step_forward(pvmw, PMD_SIZE); continue; } - if (!map_pte(pvmw, &ptl)) { + if (!map_pte(pvmw, &pmde, &ptl)) { if (!pvmw->pte) goto restart; goto next_pte; @@ -305,8 +312,13 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *pvmw) } while (pte_none(ptep_get(pvmw->pte))); if (!pvmw->ptl) { + spin_lock(ptl); + if (unlikely(!pmd_same(pmde, pmdp_get_lockless(pvmw->pmd)))) { + pte_unmap_unlock(pvmw->pte, ptl); + pvmw->pte = NULL; + goto restart; + } pvmw->ptl = ptl; - spin_lock(pvmw->ptl); } goto this_pte; } while (pvmw->address < end); From patchwork Thu Sep 26 06:46:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989640 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DcKq+tsl; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1615-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkgz5XrQz1xt6 for ; Thu, 26 Sep 2024 16:48:15 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkgz4QmQz2yk8; Thu, 26 Sep 2024 16:48:15 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::92c" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333295; cv=none; b=jtdK18nABriTYuGSWRZ1rFntp5chhZe9e1TqAkxVm/U6c8TuXXGECFDLMv9V41FhwMA7fi0E4OgyKCuXHq4VJVtloavtF2dq9x7eDkX+8fbbgiTsteWn+Nytm+KNVhzNtKp9E7Dzj0Y76M9qzj7Z+xd8J21RYWWtTQnbhSbE8O/fV475mr0ogg+/5Rrna0fJlonMUI8i9cGz/WvqUNgnPEkWNIxWaKUObq4gxkCoJ4KMQCRKxoX74xfRlxuZK4mMvgvd5bRQy4V5BN0JmiMMEbWS3X1KkQuv4dkeba7u3rYw7xGZEd0VMc4l+cqg4EkpMBpNfEoPVIh+kfF2dpbqlQ== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333295; c=relaxed/relaxed; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=h7TcmiCFXjnakKEU6fd86lnSs0ftzxnM9hUhe7Bkc0MqcneUBXLKqiP6hROcoiTWCr3wHM++Og7hQaUa9+w7zU4pipmRgI2qbH9//t5XlqY10sgUsct44rLcfmf2X1cJzkIEgbOIdjUlNK3B5kcdNXkP4GyaOQS94IWkKu6MGyEuIphbQhkJ5aMmsq9Er3md3ciriIgo7G3UPl1yK1wgwbEYyQaAgqWHt0e+j9M0CKjr0N59VV41HPgdqcPO0hvsr+ZIivCAaWN38W5AutDZJuAjvNC5M6QT2rdPG+ajaI71KSNt+NhTj+D3bYTE1n/O77QLV3nhiI5A9bGJF3O+sQ== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DcKq+tsl; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::92c; helo=mail-ua1-x92c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=DcKq+tsl; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::92c; helo=mail-ua1-x92c.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-ua1-x92c.google.com (mail-ua1-x92c.google.com [IPv6:2607:f8b0:4864:20::92c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkgz13d6z2yVP for ; Thu, 26 Sep 2024 16:48:14 +1000 (AEST) Received: by mail-ua1-x92c.google.com with SMTP id a1e0cc1a2514c-84e842c1673so249576241.3 for ; Wed, 25 Sep 2024 23:48:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333292; x=1727938092; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=DcKq+tsl8/Mxa6zgGoDSvr5DwaPu668nWn1Ms/Hp2CIWpRd0S/mSpxKqZahOBX0pTH mf3G1QpGVICNXXjRE9Ww/tsjTEhsMCPX9TLdVmo/wneGs2D8HCkgDMUFVk0KoRa30+bo 0uM8YSmX3KZw1y3DY4dM25qghXvq1PwXufxUZ9BboeKvUtrwyUF5S23A2TyNIjGw3Qcp bsAkA48lmRqdLZrcTPCNIw0TAAabIeXC+AZQFxZ5OG3FHrJRGnT4ixpg3OnJBW1jIHuS RZ/X2eSr2MpuT5Z8LIWSDlmeQBr6J6mkgJX+K8p/McT8TIODu3vnz+4fwEwLlTRKayik QHyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333292; x=1727938092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RnMCKwBqnYl3LJbu0M9HQTu7wcymlNTkAYqiZlfWQ7I=; b=Qo3AkLXM5n2Nx8ZHAVZdX5BDuebQeMp7PH1wEuB5EQqqMQY1fhfbgWmGfytesd366J QCZUzjmuwerJZgsl445pOZhz9GlomVS3GEITXEf7MvOG4n9tsLdqoPB/QXd8C12+dMbF QD2HyLHaObXCyxGc1ng2PF+TAA2rg6Fg0R6pfBXAD41hYHQQG3ZXYnPkwCcjBt0scVnN vgvfQ5x2jZAbd3FZLPD1S9O+jVYnVEIa4V/1RVBLz/SbniQlPFaE59XauqrfOBgN+7pu 3RXVtySP0kXjSy6sWCY9Rc2XuDxHCYeFzpECHw5NJjp4NUHDPLBP6eX1Nxg2zbRxqRqA 2HDw== X-Forwarded-Encrypted: i=1; AJvYcCWTqBBWBVB0QU8vImBn+RLFibkN1N/P0J7DnnL/BoiJLTHxaJU3WNVyDSNmf7AHLGFwgqiUXZ/qvo+0ebU=@lists.ozlabs.org X-Gm-Message-State: AOJu0YzRYJq/gjPdVUGr8+N9BuM126YcGfALch7dRe9CQj7KPsI8ubC3 eep1qaREoYI5ENCHaZ5F7iugXVrlsrW5/RgqFt9bTVV/fmso1nIyTFKFjAjjMEg= X-Google-Smtp-Source: AGHT+IEjiufoSG+9f7J9DudEHL9YuIq0UaJx7YZ52pOk+qHYx73DXmea6YZQnzcRnEEhuuY+i+vKHw== X-Received: by 2002:a05:6102:e06:b0:493:b055:bf56 with SMTP id ada2fe7eead31-4a15dc39965mr6802341137.2.1727333291940; Wed, 25 Sep 2024 23:48:11 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:11 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 11/13] mm: userfaultfd: move_pages_pte() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:24 +0800 Message-Id: <1530e8fdbfc72eacf3b095babe139ce3d715600a.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In move_pages_pte(), we may modify the dst_pte and src_pte after acquiring the ptl, so convert it to using pte_offset_map_rw_nolock(). But since we will use pte_same() to detect the change of the pte entry, there is no need to get pmdval, so just pass a dummy variable to it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/userfaultfd.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index ce13c40626472..48b87c62fc3dd 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1135,7 +1135,7 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, spinlock_t *src_ptl, *dst_ptl; pte_t *src_pte = NULL; pte_t *dst_pte = NULL; - + pmd_t dummy_pmdval; struct folio *src_folio = NULL; struct anon_vma *src_anon_vma = NULL; struct mmu_notifier_range range; @@ -1146,7 +1146,14 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, src_addr, src_addr + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); retry: - dst_pte = pte_offset_map_nolock(mm, dst_pmd, dst_addr, &dst_ptl); + /* + * Use the maywrite version to indicate that dst_pte will be modified, + * but since we will use pte_same() to detect the change of the pte + * entry, there is no need to get pmdval, so just pass a dummy variable + * to it. + */ + dst_pte = pte_offset_map_rw_nolock(mm, dst_pmd, dst_addr, &dummy_pmdval, + &dst_ptl); /* Retry if a huge pmd materialized from under us */ if (unlikely(!dst_pte)) { @@ -1154,7 +1161,9 @@ static int move_pages_pte(struct mm_struct *mm, pmd_t *dst_pmd, pmd_t *src_pmd, goto out; } - src_pte = pte_offset_map_nolock(mm, src_pmd, src_addr, &src_ptl); + /* same as dst_pte */ + src_pte = pte_offset_map_rw_nolock(mm, src_pmd, src_addr, &dummy_pmdval, + &src_ptl); /* * We held the mmap_lock for reading so MADV_DONTNEED From patchwork Thu Sep 26 06:46:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989641 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TlJZs3uO; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1616-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkh62Nqkz1xt6 for ; Thu, 26 Sep 2024 16:48:22 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkh61FSSz2yYq; Thu, 26 Sep 2024 16:48:22 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::829" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333302; cv=none; b=ITSIVsUioGgWquH77b78nxMY+xgSyTIlH/RFRXz9r5fFgAYp5t2dXHh4fm0iV3aj24cuXNkk+sQB9qeIChproM9UrD1+aCe+nNvDxcdsB2shysncin7TuTXHW6C8FiEAirVqRWhTjkcVrcdlyqjuBtRuzIxx8UcJD6Tq9DVSTWNWEzRSopKkHuJspk095yeXAPbeSeDUct0czkqTfJ8dnlYoN+JQ8R5NblWNZk9Zncav0hfML86hvvohLP2zFhyPRydwix82/TFk93Q9beowFXQ3lF+DUhew7SgokruMCh4R55Eqp+RuNb4JYdeApGP9duMpxM9KIidOuC8fqn6u6A== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333302; c=relaxed/relaxed; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gDNgw0BKmzZTyYRPaomJhKeYo1y46SRbRyjjnCbsbaOLmTKsQlvABaPiygEQijGBcv3zqm6wRe7kKm1zQy+ITlJOfwcWEqwffp1G/wvZLoAafvAW/kJSyyWdw+90QKUrw8c/TEE5THnJdN0Y8IQGNSqMTatoBY9uIVJ1z47RJVc3WndFUxat9rQHj0O8CL8bo2TwE8V6tfscdcNmK+c8V8L3VDqeJCBeLgpWFo48Pck61wxE/Bc7/DkzbzO4SK7foe6B8x7nEH76sfOsMpR4S8zHO7O+P43UsOwE2xAkRufkHHhb77sWXw3S3tfmoDyh/3w9nMsc0wY1WnkjFwvrsg== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TlJZs3uO; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=TlJZs3uO; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkh55HNjz2yVP for ; Thu, 26 Sep 2024 16:48:21 +1000 (AEST) Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-458362e898aso4398981cf.0 for ; Wed, 25 Sep 2024 23:48:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333300; x=1727938100; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; b=TlJZs3uOtidYuz+0C8UwsRVPx6mNZfv5q3raUWr3/QlEGPUAJYF/1Sui/AIt0l9Rsc IbKu7l8ar11MUd/pnOJIFYYR7Qc2RBGE4OucVH2CTsMa0Mpu14hMOt233THnqVjLz974 ssAWug5qlS/t/kPqef59SLcg7LtKth58u9f35gZYCou0I+TraKPB6OpYQDCkj+iPWZZ4 Okp+zE+MEJ9DyVrhGWZGJR5FXSIBJgJDpmaq1Z20c7vkd36pBm9aLPa8jC4DiPAZN7aU XftxxxKiYNYtsD092/8PkJaaQusm05U7jblt+VQk1+1rT6oIAZODyyU/YynuNZHmzRlj rmiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333300; x=1727938100; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wKWf8ZYo9tlbP3964sEqp7HhdDy32FNbDdGL8iSt+Ok=; b=SAu7glzEPxWKDqHllEiJ3lB6EUKUNL9YaKaEjryQgxxyibr22Hbg6H3FBi2gib4rNL aNjWptvPhh4hS0kKe1zxFGyAn5Xd8vVnEiqAOlJQFd6zHrSFngcCAUpLTQB8pkW6S093 3IN590tm/to+veGGfg/yWiBC1p8+OgdJ25m4cY1JxBWPvUXCuWDfmlz0B5KpMqWozYZh ElPGNYzq90QrbDKL5YAMPupoGdC5+20iL1HpjJlUEENkOF38Mc/KNt8K29xDVXIqUAA9 LxEM8eNa2XvehySvIxhwAbLHyLo2UYSBDMyhD4Ea0Kl+D8byEp/Ad9DbUx80afqXOrSt JAWQ== X-Forwarded-Encrypted: i=1; AJvYcCWN1FtG9Shqyo3G2TgGqv/6INzkUZryntLG6ob4g9uTN7TO8h/yxER4sn+k59FEfPX5Td/HlZskNJAYicc=@lists.ozlabs.org X-Gm-Message-State: AOJu0Yy4w0KPkeMdtx5a6aNeHbOvY3uHXmdz8HVq8ftJMiCbKp9YfVwL w4IEnydrbDaTVFt4cXzPasIgjxxHvkRlaFLb9zCbNKk+YKVmj/GHCwiBl1drq6I= X-Google-Smtp-Source: AGHT+IH0PykjARYOlhRdYD8Ei00ntSXkZvlp+w/DvL/wK29HFMzX/PK1qWz6iq3HMQhndU3PBLr5iA== X-Received: by 2002:a05:622a:51:b0:458:3e20:65d3 with SMTP id d75a77b69052e-45b5e06627amr81705351cf.60.1727333299714; Wed, 25 Sep 2024 23:48:19 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:19 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 12/13] mm: multi-gen LRU: walk_pte_range() use pte_offset_map_rw_nolock() Date: Thu, 26 Sep 2024 14:46:25 +0800 Message-Id: <7e9c194a5efacc9609cfd31abb9c7df88b53b530.1727332572.git.zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 In walk_pte_range(), we may modify the pte entry after holding the ptl, so convert it to using pte_offset_map_rw_nolock(). At this time, the pte_same() check is not performed after the ptl held, so we should get pmdval and do pmd_same() check to ensure the stability of pmd entry. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- mm/vmscan.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 749cdc110c745..bdca94e663bc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3375,8 +3375,10 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, struct pglist_data *pgdat = lruvec_pgdat(walk->lruvec); DEFINE_MAX_SEQ(walk->lruvec); int old_gen, new_gen = lru_gen_from_seq(max_seq); + pmd_t pmdval; - pte = pte_offset_map_nolock(args->mm, pmd, start & PMD_MASK, &ptl); + pte = pte_offset_map_rw_nolock(args->mm, pmd, start & PMD_MASK, &pmdval, + &ptl); if (!pte) return false; if (!spin_trylock(ptl)) { @@ -3384,6 +3386,11 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, return false; } + if (unlikely(!pmd_same(pmdval, pmdp_get_lockless(pmd)))) { + pte_unmap_unlock(pte, ptl); + return false; + } + arch_enter_lazy_mmu_mode(); restart: for (i = pte_index(start), addr = start; addr != end; i++, addr += PAGE_SIZE) { From patchwork Thu Sep 26 06:46:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 1989642 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=iMWKA+ge; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev+bounces-1617-incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=patchwork.ozlabs.org) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XDkhF6qLGz1xt6 for ; Thu, 26 Sep 2024 16:48:29 +1000 (AEST) Received: from boromir.ozlabs.org (localhost [127.0.0.1]) by lists.ozlabs.org (Postfix) with ESMTP id 4XDkhF5fVDz2yVV; Thu, 26 Sep 2024 16:48:29 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; arc=none smtp.remote-ip="2607:f8b0:4864:20::829" ARC-Seal: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333309; cv=none; b=cw0yQJ5oYNexvsRViZGypWpQvXFHIJg9twk+g96p9BOTMC1NRDx+V4HqTwN+BT4rEA2h20t+rrIa4YKABHQbpXHv7D76QDnHjVWaZATGLHNnSTOzf5vyxnecsufkWiv9mOEJaviJnZFaTlKbJiW7gaaz92ANfQz1zbXVnxqohYzsHqEkF7Zt1RZXMrSLmtdeuM3gzquRLtUtsLcMeEDD8reaR3XZmlXVvfPs47KpUOB/zqL4WkkK5NH3Jyo4JxhN5MwsuJcLVpwg8576nHcImdJTSLxIhzAlJrMNxYfqYc4V6033p/BOU/5SXN/Ac1zkz/gPHhuCKhaoxA0Vb6IIcg== ARC-Message-Signature: i=1; a=rsa-sha256; d=lists.ozlabs.org; s=201707; t=1727333309; c=relaxed/relaxed; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bdOJ6s4W0Qvj4e7k3X9U1PZZswo4bC+GuJqnAiwWDIV9BkbPgmbPIU4VmVk8t3Iqs27c88hZyb5WDTbRhyFnQONrrkI6jaoIZgX5JcwLcnh1pjFPxW6UcbzcqowJN0cs3b5q1MWrVfT2MAgjkryaN5uG2yz605DjKgis1/YMgLGDerqv1niD5OHxkmO3Ipl1ZYudfVm6k9HaJ9bHacs8DSfhXSMuTBT3IX7vo1y7c0C8kuCsnhtQGU0S675UjIwlRcpHOdf7PCXsDtwjOINTaoACVUmtQ/xsXJVF42q/6MUYkYHCHzD2d+wOH17LtJYfZpUK59woy2wKwg+p8TUldw== ARC-Authentication-Results: i=1; lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=iMWKA+ge; dkim-atps=neutral; spf=pass (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) smtp.mailfrom=bytedance.com Authentication-Results: lists.ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=bytedance.com header.i=@bytedance.com header.a=rsa-sha256 header.s=google header.b=iMWKA+ge; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=bytedance.com (client-ip=2607:f8b0:4864:20::829; helo=mail-qt1-x829.google.com; envelope-from=zhengqi.arch@bytedance.com; receiver=lists.ozlabs.org) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4XDkhF2SLVz2yVP for ; Thu, 26 Sep 2024 16:48:29 +1000 (AEST) Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-4585e25f42bso4782261cf.2 for ; Wed, 25 Sep 2024 23:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1727333307; x=1727938107; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; b=iMWKA+geurTeDk5ddTenVfFCO4w/btjar5fuHE4BFKuu/wddXNpknP/QDc+0yo9IqL XHPjuSVG9Qbam80nHZ6It+dKYmN8QcmEdY072/7HwJ8ro0HsBTlLagWDnPg86uOnRIb7 qcJstgs82tpFCEqjfFcuFFhituDwbAzFonmiS4DCSoS3hbJsoUG32KGJmaCcT3fcR/+n Ar4k97/3hh8BoESPAP46ONZsOzhlpj+alznk2O8Oz/kCDkdIjb1H0urgN1QaS65k2R7n NwNNbAl5aqj/+g/tvaDiKYO43JRSd/+t1FTrhEcRKjhZst8JuJSJBwX9zVtpqXw5Q4eT UojA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727333307; x=1727938107; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XK9xLAd1aeWWLl9nHV9gcamA18VQvE5iTgorpb3Ql3Q=; b=hghly2QXxvPAa38ZwAaNhlxYJP29qyeKg1wV2WtsTcnQTjAIMzHUbhRLoYzsUnifBW FZsjbwqMY+XxRjQ9Wi2FiKca9bfjmWRLarc3ZCh6ZpR/YkghpW70cIoBh6I1/fV4OGcR gZR6wWm4dyK5LxNL5KUX+asoTbFD2UED5aHoVHQKYf5Ruquv43RCb9yoP0XPvdUxjqz6 DILcijl09/T4dxbGsOLUFkXZMo3i7404GWJKVbFshzbjQk9T2tZxMMKJsQfeHQ/yEj0U JWs4katvAndtlLkbGVztxLPMb+XiEAYTYDxc8GWyNMyGtUBWEkALfeXk5f52FfVqvNCb mwsg== X-Forwarded-Encrypted: i=1; AJvYcCVDi84cDlgW03dw5oqUFhccQU2iSX28OaB7JlGINZUjA8Vmub01ZpPlpJ/uhoUKKA8n7QfegXc2RXARmQo=@lists.ozlabs.org X-Gm-Message-State: AOJu0YyQuKoZe5Uj6kHkQAPYWVHXKNGJnzd8JsVh3Kt0zau3AntiRdBY vxLAUCU/pUmBWKqxtpWkZQERaTbUtp9WBZhYFgZr8XIAxUdI9OmE6aSOteXaKzc= X-Google-Smtp-Source: AGHT+IHnYRThLjQtuqQ25g++f96or6jGXpOGKBOHjBYtOEzyfC2Zv3m6NDV/6g7eYEpePNjEvQY05w== X-Received: by 2002:a05:622a:2996:b0:458:37c6:46fa with SMTP id d75a77b69052e-45b5def2036mr66535021cf.28.1727333307294; Wed, 25 Sep 2024 23:48:27 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45b5257ff1esm23024611cf.38.2024.09.25.23.48.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Sep 2024 23:48:26 -0700 (PDT) From: Qi Zheng To: david@redhat.com, hughd@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, akpm@linux-foundation.org, rppt@kernel.org, vishal.moola@gmail.com, peterx@redhat.com, ryan.roberts@arm.com, christophe.leroy2@cs-soprasteria.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, Qi Zheng Subject: [PATCH v5 13/13] mm: pgtable: remove pte_offset_map_nolock() Date: Thu, 26 Sep 2024 14:46:26 +0800 Message-Id: X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: References: X-Mailing-List: linuxppc-dev@lists.ozlabs.org List-Id: List-Help: List-Post: List-Subscribe: , , List-Unsubscribe: MIME-Version: 1.0 Now no users are using the pte_offset_map_nolock(), remove it. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Acked-by: David Hildenbrand --- Documentation/mm/split_page_table_lock.rst | 3 --- include/linux/mm.h | 2 -- mm/pgtable-generic.c | 21 --------------------- 3 files changed, 26 deletions(-) diff --git a/Documentation/mm/split_page_table_lock.rst b/Documentation/mm/split_page_table_lock.rst index 08d0e706a32db..581446d4a4eba 100644 --- a/Documentation/mm/split_page_table_lock.rst +++ b/Documentation/mm/split_page_table_lock.rst @@ -16,9 +16,6 @@ There are helpers to lock/unlock a table and other accessor functions: - pte_offset_map_lock() maps PTE and takes PTE table lock, returns pointer to PTE with pointer to its PTE table lock, or returns NULL if no PTE table; - - pte_offset_map_nolock() - maps PTE, returns pointer to PTE with pointer to its PTE table - lock (not taken), or returns NULL if no PTE table; - pte_offset_map_ro_nolock() maps PTE, returns pointer to PTE with pointer to its PTE table lock (not taken), or returns NULL if no PTE table; diff --git a/include/linux/mm.h b/include/linux/mm.h index 46828b9a74f2c..60a55b28cf679 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,8 +3015,6 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, return pte; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp); pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, diff --git a/mm/pgtable-generic.c b/mm/pgtable-generic.c index daa08b91ab6b2..5297dcc38c37a 100644 --- a/mm/pgtable-generic.c +++ b/mm/pgtable-generic.c @@ -305,18 +305,6 @@ pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp) return NULL; } -pte_t *pte_offset_map_nolock(struct mm_struct *mm, pmd_t *pmd, - unsigned long addr, spinlock_t **ptlp) -{ - pmd_t pmdval; - pte_t *pte; - - pte = __pte_offset_map(pmd, addr, &pmdval); - if (likely(pte)) - *ptlp = pte_lockptr(mm, &pmdval); - return pte; -} - pte_t *pte_offset_map_ro_nolock(struct mm_struct *mm, pmd_t *pmd, unsigned long addr, spinlock_t **ptlp) { @@ -372,15 +360,6 @@ pte_t *pte_offset_map_rw_nolock(struct mm_struct *mm, pmd_t *pmd, * and disconnected table. Until pte_unmap(pte) unmaps and rcu_read_unlock()s * afterwards. * - * pte_offset_map_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); - * but when successful, it also outputs a pointer to the spinlock in ptlp - as - * pte_offset_map_lock() does, but in this case without locking it. This helps - * the caller to avoid a later pte_lockptr(mm, *pmd), which might by that time - * act on a changed *pmd: pte_offset_map_nolock() provides the correct spinlock - * pointer for the page table that it returns. In principle, the caller should - * recheck *pmd once the lock is taken; in practice, no callsite needs that - - * either the mmap_lock for write, or pte_same() check on contents, is enough. - * * pte_offset_map_ro_nolock(mm, pmd, addr, ptlp), above, is like pte_offset_map(); * but when successful, it also outputs a pointer to the spinlock in ptlp - as * pte_offset_map_lock() does, but in this case without locking it. This helps