From patchwork Tue Sep 10 01:42:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koichiro Den X-Patchwork-Id: 1982884 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4X2mft1cV8z1y1C for ; Tue, 10 Sep 2024 11:42:46 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1snptu-00080R-Hp; Tue, 10 Sep 2024 01:42:38 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1snpts-0007zf-VI for kernel-team@lists.ubuntu.com; Tue, 10 Sep 2024 01:42:36 +0000 Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 9F0913F5BA for ; Tue, 10 Sep 2024 01:42:35 +0000 (UTC) Received: by mail-pl1-f200.google.com with SMTP id d9443c01a7336-2052e7836a0so54386325ad.0 for ; Mon, 09 Sep 2024 18:42:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725932554; x=1726537354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M3TVgB1+Q3VIwf8PMy+7GBF3i2mDp68STn20zeFWo0U=; b=WcjeLQ4Llb76yR/k25wyQAaw1niDka4DkUoqfwbfnHjrYS/qssORFYPR7ShbU5cS6g jtywZdvDvH16aP2o6gCtQGSEY7AlMtnxTZoGjUCXCs1cmLZKTJg3w6Vjt7fQfcbdNYmS vLK3TrZKECiXoVKiSHxVuExUPTWevDstV95uS6/xDq8dvhvCKqNQ+nU+2cokSvTWBv/o scrWw8M5gY/Azs3wIsDafJd6nrHI4f+bUXsBGs0F3pgtqn+U4QXq66GT8TxY6ucxIHir QJ7K8PAaEU5d+9ml11SwC+k+tpYL7UuWmsyR/XPm8J7+xPiaX1Vm1my1x7lmRD5j52ZI 3gww== X-Gm-Message-State: AOJu0Yw0Y/1l8xk9TQN+1Q9xVTz6t2vCzcUXWpX5/4fc31qidwfU7t4u tW15YRcguBgBnflXIgd257SdyyUSzqRa+CzpZsdejPvL7MQIj+gYKRqefS/G0Mlx/dy2Nhqm9Vh TEzWDIv5d+nlrP5TtODzlag/WChZ1W3zdXV8eZa3NKqP9Cx2wfoOvk1w1JcN+BWXor86Kmaq4gP 1JDLwsHd/UaA== X-Received: by 2002:a17:902:f681:b0:205:79b4:c5b1 with SMTP id d9443c01a7336-206f04f1b6amr112433915ad.16.1725932554074; Mon, 09 Sep 2024 18:42:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE//xcAOz9x/aTZ9P90k/eanvxVRx9/C853x4YmFXrDrEShadcNofmGXFRHdZZupV+b9zzQ8w== X-Received: by 2002:a17:902:f681:b0:205:79b4:c5b1 with SMTP id d9443c01a7336-206f04f1b6amr112433655ad.16.1725932553558; Mon, 09 Sep 2024 18:42:33 -0700 (PDT) Received: from localhost.localdomain ([240f:74:7be:1:959:86f5:52c1:2a4c]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20710e33a09sm39413335ad.107.2024.09.09.18.42.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Sep 2024 18:42:33 -0700 (PDT) From: Koichiro Den To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 1/2] net-zerocopy: Refactor frag-is-remappable test. Date: Tue, 10 Sep 2024 10:42:03 +0900 Message-ID: <20240910014210.1052063-2-koichiro.den@canonical.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240910014210.1052063-1-koichiro.den@canonical.com> References: <20240910014210.1052063-1-koichiro.den@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Arjun Roy Refactor frag-is-remappable test for tcp receive zerocopy. This is part of a patch set that introduces short-circuited hybrid copies for small receive operations, which results in roughly 33% fewer syscalls for small RPC scenarios. Signed-off-by: Arjun Roy Signed-off-by: Eric Dumazet Signed-off-by: Soheil Hassas Yeganeh Signed-off-by: Jakub Kicinski (backported from commit 98917cf0d6eda01e8c3c34d35398d46b247b6fd3) [koichiroden: Adjusted context due to missing commit 18fb76ed5386 ("net-zerocopy: Copy straggler unaligned data for TCP Rx. zerocopy.")] CVE-2024-26640 Signed-off-by: Koichiro Den --- net/ipv4/tcp.c | 34 ++++++++++++++++++++++++++-------- 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 54399256a438..7510e1937734 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1753,6 +1753,26 @@ int tcp_mmap(struct file *file, struct socket *sock, } EXPORT_SYMBOL(tcp_mmap); +static bool can_map_frag(const skb_frag_t *frag) +{ + return skb_frag_size(frag) == PAGE_SIZE && !skb_frag_off(frag); +} + +static int find_next_mappable_frag(const skb_frag_t *frag, + int remaining_in_skb) +{ + int offset = 0; + + if (likely(can_map_frag(frag))) + return 0; + + while (offset < remaining_in_skb && !can_map_frag(frag)) { + offset += skb_frag_size(frag); + ++frag; + } + return offset; +} + static int tcp_zerocopy_receive(struct sock *sk, struct tcp_zerocopy_receive *zc) { @@ -1795,6 +1815,8 @@ static int tcp_zerocopy_receive(struct sock *sk, } ret = 0; while (length + PAGE_SIZE <= zc->length) { + int mappable_offset; + if (zc->recv_skip_hint < PAGE_SIZE) { if (skb) { skb = skb->next; @@ -1815,15 +1837,11 @@ static int tcp_zerocopy_receive(struct sock *sk, frags++; } } - if (skb_frag_size(frags) != PAGE_SIZE || skb_frag_off(frags)) { - int remaining = zc->recv_skip_hint; - while (remaining && (skb_frag_size(frags) != PAGE_SIZE || - skb_frag_off(frags))) { - remaining -= skb_frag_size(frags); - frags++; - } - zc->recv_skip_hint -= remaining; + mappable_offset = find_next_mappable_frag(frags, + zc->recv_skip_hint); + if (mappable_offset) { + zc->recv_skip_hint = mappable_offset; break; } ret = vm_insert_page(vma, address + length,