From patchwork Tue Sep 10 01:42:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koichiro Den X-Patchwork-Id: 1982884 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X2mft1cV8z1y1C for ; Tue, 10 Sep 2024 11:42:46 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1snptu-00080R-Hp; Tue, 10 Sep 2024 01:42:38 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1snpts-0007zf-VI for kernel-team@lists.ubuntu.com; Tue, 10 Sep 2024 01:42:36 +0000 Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 9F0913F5BA for ; Tue, 10 Sep 2024 01:42:35 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id d9443c01a7336-2052e7836a0so54386325ad.0 for ; Mon, 09 Sep 2024 18:42:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725932554; x=1726537354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M3TVgB1+Q3VIwf8PMy+7GBF3i2mDp68STn20zeFWo0U=; b=WcjeLQ4Llb76yR/k25wyQAaw1niDka4DkUoqfwbfnHjrYS/qssORFYPR7ShbU5cS6g jtywZdvDvH16aP2o6gCtQGSEY7AlMtnxTZoGjUCXCs1cmLZKTJg3w6Vjt7fQfcbdNYmS vLK3TrZKECiXoVKiSHxVuExUPTWevDstV95uS6/xDq8dvhvCKqNQ+nU+2cokSvTWBv/o scrWw8M5gY/Azs3wIsDafJd6nrHI4f+bUXsBGs0F3pgtqn+U4QXq66GT8TxY6ucxIHir QJ7K8PAaEU5d+9ml11SwC+k+tpYL7UuWmsyR/XPm8J7+xPiaX1Vm1my1x7lmRD5j52ZI 3gww== X-Gm-Message-State: AOJu0Yw0Y/1l8xk9TQN+1Q9xVTz6t2vCzcUXWpX5/4fc31qidwfU7t4u tW15YRcguBgBnflXIgd257SdyyUSzqRa+CzpZsdejPvL7MQIj+gYKRqefS/G0Mlx/dy2Nhqm9Vh TEzWDIv5d+nlrP5TtODzlag/WChZ1W3zdXV8eZa3NKqP9Cx2wfoOvk1w1JcN+BWXor86Kmaq4gP 1JDLwsHd/UaA== X-Received: by 2002:a17:902:f681:b0:205:79b4:c5b1 with SMTP id d9443c01a7336-206f04f1b6amr112433915ad.16.1725932554074; Mon, 09 Sep 2024 18:42:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE//xcAOz9x/aTZ9P90k/eanvxVRx9/C853x4YmFXrDrEShadcNofmGXFRHdZZupV+b9zzQ8w== X-Received: by 2002:a17:902:f681:b0:205:79b4:c5b1 with SMTP id d9443c01a7336-206f04f1b6amr112433655ad.16.1725932553558; Mon, 09 Sep 2024 18:42:33 -0700 (PDT) Received: from localhost.localdomain ([240f:74:7be:1:959:86f5:52c1:2a4c]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20710e33a09sm39413335ad.107.2024.09.09.18.42.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Sep 2024 18:42:33 -0700 (PDT) From: Koichiro Den To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 1/2] net-zerocopy: Refactor frag-is-remappable test. Date: Tue, 10 Sep 2024 10:42:03 +0900 Message-ID: <20240910014210.1052063-2-koichiro.den@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240910014210.1052063-1-koichiro.den@canonical.com> References: <20240910014210.1052063-1-koichiro.den@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Arjun Roy Refactor frag-is-remappable test for tcp receive zerocopy. This is part of a patch set that introduces short-circuited hybrid copies for small receive operations, which results in roughly 33% fewer syscalls for small RPC scenarios. Signed-off-by: Arjun Roy Signed-off-by: Eric Dumazet Signed-off-by: Soheil Hassas Yeganeh Signed-off-by: Jakub Kicinski (backported from commit 98917cf0d6eda01e8c3c34d35398d46b247b6fd3) [koichiroden: Adjusted context due to missing commit 18fb76ed5386 ("net-zerocopy: Copy straggler unaligned data for TCP Rx. zerocopy.")] CVE-2024-26640 Signed-off-by: Koichiro Den --- net/ipv4/tcp.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 54399256a438..7510e1937734 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1753,6 +1753,26 @@ int tcp_mmap(struct file *file, struct socket *sock, } EXPORT_SYMBOL(tcp_mmap); +static bool can_map_frag(const skb_frag_t *frag) +{ + return skb_frag_size(frag) == PAGE_SIZE && !skb_frag_off(frag); +} + +static int find_next_mappable_frag(const skb_frag_t *frag, + int remaining_in_skb) +{ + int offset = 0; + + if (likely(can_map_frag(frag))) + return 0; + + while (offset < remaining_in_skb && !can_map_frag(frag)) { + offset += skb_frag_size(frag); + ++frag; + } + return offset; +} + static int tcp_zerocopy_receive(struct sock *sk, struct tcp_zerocopy_receive *zc) { @@ -1795,6 +1815,8 @@ static int tcp_zerocopy_receive(struct sock *sk, } ret = 0; while (length + PAGE_SIZE <= zc->length) { + int mappable_offset; + if (zc->recv_skip_hint < PAGE_SIZE) { if (skb) { skb = skb->next; @@ -1815,15 +1837,11 @@ static int tcp_zerocopy_receive(struct sock *sk, frags++; } } - if (skb_frag_size(frags) != PAGE_SIZE || skb_frag_off(frags)) { - int remaining = zc->recv_skip_hint; - while (remaining && (skb_frag_size(frags) != PAGE_SIZE || - skb_frag_off(frags))) { - remaining -= skb_frag_size(frags); - frags++; - } - zc->recv_skip_hint -= remaining; + mappable_offset = find_next_mappable_frag(frags, + zc->recv_skip_hint); + if (mappable_offset) { + zc->recv_skip_hint = mappable_offset; break; } ret = vm_insert_page(vma, address + length, From patchwork Tue Sep 10 01:42:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koichiro Den X-Patchwork-Id: 1982885 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X2mft3vXJz1y1v for ; Tue, 10 Sep 2024 11:42:46 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1snptu-00080b-O9; Tue, 10 Sep 2024 01:42:38 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1snptt-000809-95 for kernel-team@lists.ubuntu.com; Tue, 10 Sep 2024 01:42:37 +0000 Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 0E6A23F2F1 for ; Tue, 10 Sep 2024 01:42:37 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id d9443c01a7336-2053f49d0c9so64972795ad.1 for ; Mon, 09 Sep 2024 18:42:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725932555; x=1726537355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=J/f4ky01YuILS9VAeJbLwIdNT3rhVXf0vSz7pPkHauk=; b=R1fWUBL4NO/I1TohmSeYvF0z5bTeio8C0QFzeOLUWErn36WE5jdMjV7n7XRCdG5dib Ie0OvROC2NNVfMOwm0WSKg/ftTKGnTbgPtmR1842a9VCpBOdmxRg3mOVt5qOYWP2fSsP AtkV3XtxKCm8LoR8j3m+vAj/zYR8GKNxyAKwlHVYdEtJALQCYpQnni0gDsNCLEu78KBq jUQHR3FT4M6vL1IWpUKwTHNg3xYwfwwDcZnNZyhep0pS1Pq+hNUAvrkSnzFvOC0Go8cV PldtgThcDhUtJVkiFS/jp/GrOz0loOC9q/SoBZ2+mIelNgYQ3LaU2AtrJCAp0RrYSLv6 IVEA== X-Gm-Message-State: AOJu0YxEOO2ITlnpGejRCYk84e1jMcBQCW4+mG7VA9upwk9mOFQHLaHU pMuoDTghD3AHUaoxqXjnET3o9uHUpyo3Wwx10wWxYqm/4h7+ffwdjqo55XZmP+ssGJ7j6n/EDzS 7GRpXyY4zN6bU4kAe3nPL/cwdtKKhxHh19SAU2RDZu1bPbJtOUB3nvLhD3RBfL/4IJ0uQsiRQrh l9HQOLfSZ54A== X-Received: by 2002:a17:902:d2ce:b0:206:cfb3:9372 with SMTP id d9443c01a7336-2074392230amr26617255ad.11.1725932555252; Mon, 09 Sep 2024 18:42:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IElRwMuYhIKo0RY2vfB2d7bfsFHOb+PSWR6uXEODRriTW6gkwXlkNemKSKwms8fiuasFpMenw== X-Received: by 2002:a17:902:d2ce:b0:206:cfb3:9372 with SMTP id d9443c01a7336-2074392230amr26616885ad.11.1725932554686; Mon, 09 Sep 2024 18:42:34 -0700 (PDT) Received: from localhost.localdomain ([240f:74:7be:1:959:86f5:52c1:2a4c]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20710e33a09sm39413335ad.107.2024.09.09.18.42.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Sep 2024 18:42:34 -0700 (PDT) From: Koichiro Den To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 2/2] tcp: add sanity checks to rx zerocopy Date: Tue, 10 Sep 2024 10:42:04 +0900 Message-ID: <20240910014210.1052063-3-koichiro.den@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240910014210.1052063-1-koichiro.den@canonical.com> References: <20240910014210.1052063-1-koichiro.den@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Eric Dumazet TCP rx zerocopy intent is to map pages initially allocated from NIC drivers, not pages owned by a fs. This patch adds to can_map_frag() these additional checks: - Page must not be a compound one. - page->mapping must be NULL. This fixes the panic reported by ZhangPeng. syzbot was able to loopback packets built with sendfile(), mapping pages owned by an ext4 file to TCP rx zerocopy. r3 = socket$inet_tcp(0x2, 0x1, 0x0) mmap(&(0x7f0000ff9000/0x4000)=nil, 0x4000, 0x0, 0x12, r3, 0x0) r4 = socket$inet_tcp(0x2, 0x1, 0x0) bind$inet(r4, &(0x7f0000000000)={0x2, 0x4e24, @multicast1}, 0x10) connect$inet(r4, &(0x7f00000006c0)={0x2, 0x4e24, @empty}, 0x10) r5 = openat$dir(0xffffffffffffff9c, &(0x7f00000000c0)='./file0\x00', 0x181e42, 0x0) fallocate(r5, 0x0, 0x0, 0x85b8) sendfile(r4, r5, 0x0, 0x8ba0) getsockopt$inet_tcp_TCP_ZEROCOPY_RECEIVE(r4, 0x6, 0x23, &(0x7f00000001c0)={&(0x7f0000ffb000/0x3000)=nil, 0x3000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, &(0x7f0000000440)=0x40) r6 = openat$dir(0xffffffffffffff9c, &(0x7f00000000c0)='./file0\x00', 0x181e42, 0x0) Fixes: 93ab6cc69162 ("tcp: implement mmap() for zero copy receive") Link: https://lore.kernel.org/netdev/5106a58e-04da-372a-b836-9d3d0bd2507b@huawei.com/T/ Reported-and-bisected-by: ZhangPeng Signed-off-by: Eric Dumazet Cc: Arjun Roy Cc: Matthew Wilcox Cc: linux-mm@vger.kernel.org Cc: Andrew Morton Cc: linux-fsdevel@vger.kernel.org Signed-off-by: David S. Miller (cherry picked from commit 577e4432f3ac810049cb7e6b71f4d96ec7c6e894) CVE-2024-26640 Signed-off-by: Koichiro Den --- net/ipv4/tcp.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 7510e1937734..2ca02dc695b2 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1755,7 +1755,17 @@ EXPORT_SYMBOL(tcp_mmap); static bool can_map_frag(const skb_frag_t *frag) { - return skb_frag_size(frag) == PAGE_SIZE && !skb_frag_off(frag); + struct page *page; + + if (skb_frag_size(frag) != PAGE_SIZE || skb_frag_off(frag)) + return false; + + page = skb_frag_page(frag); + + if (PageCompound(page) || page->mapping) + return false; + + return true; } static int find_next_mappable_frag(const skb_frag_t *frag,