From patchwork Fri Jul 5 20:44:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Pattrick X-Patchwork-Id: 1957445 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TPziSsln; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WG59C6snhz1xr5 for ; Sat, 6 Jul 2024 06:44:31 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3DAAB80BCB; Fri, 5 Jul 2024 20:44:29 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id gLHEpJsJwBDF; Fri, 5 Jul 2024 20:44:26 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E76AB80BA2 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TPziSsln Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id E76AB80BA2; Fri, 5 Jul 2024 20:44:25 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id AF152C0A98; Fri, 5 Jul 2024 20:44:25 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 62DDCC0A96 for ; Fri, 5 Jul 2024 20:44:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5EEB3406E2 for ; Fri, 5 Jul 2024 20:44:24 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id LNHU-_0PkYmM for ; Fri, 5 Jul 2024 20:44:23 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=mkp@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 546B1406C7 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 546B1406C7 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=TPziSsln Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 546B1406C7 for ; Fri, 5 Jul 2024 20:44:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1720212261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=KmqVF75vqYbogVrK8cRT6AONTT5Aphxopw6rYa1JEgI=; b=TPziSslnNjrOoHyTfzyvbmGk3uoJgEQLOUl0QxHDuSRCxBLFW+wSp/iWLy3sJXQ6jBqN3q dN5RUMvnXuLgHnCtXDCCLW/Ne68q5iyLYS3T5iRbfjBGmxNUYjnuUslY4T1BWyqqVkXzPR 8IGS6aHUI7WDFHxY7/ipsOFba/oLefc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-646-GbMs4EEdMs2IsBbW8AlarA-1; Fri, 05 Jul 2024 16:44:20 -0400 X-MC-Unique: GbMs4EEdMs2IsBbW8AlarA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A9FC81955F3B for ; Fri, 5 Jul 2024 20:44:19 +0000 (UTC) Received: from mpattric.remote.csb (unknown [10.22.18.114]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 22F99195605F; Fri, 5 Jul 2024 20:44:17 +0000 (UTC) From: Mike Pattrick To: dev@openvswitch.org Date: Fri, 5 Jul 2024 16:44:11 -0400 Message-Id: <20240705204413.1105558-1-mkp@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 1/3] userspace: Adjust segment size on encapsulation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: david.marchand@redhat.com Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When prepending a tunnel header to a packet marked for segmentation, we need to adjust the segment size. Failure to do so can result in packets that are larger than the intended MTU post segmentation. Fixes: 084c8087292c ("userspace: Support VXLAN and GENEVE TSO.") Signed-off-by: Mike Pattrick Reviewed-by: David Marchand --- lib/netdev-native-tnl.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/lib/netdev-native-tnl.c b/lib/netdev-native-tnl.c index 0f9f07f44..da0bb6fda 100644 --- a/lib/netdev-native-tnl.c +++ b/lib/netdev-native-tnl.c @@ -161,6 +161,17 @@ netdev_tnl_push_ip_header(struct dp_packet *packet, const void *header, *ip_tot_size = dp_packet_size(packet) - sizeof (struct eth_header); memcpy(eth, header, size); + + /* The prepended header may cause TSO marked packets to exceed the intended + * MTU on segmentation. */ + if (dp_packet_hwol_is_tso(packet)) { + uint16_t tso_segsz = dp_packet_get_tso_segsz(packet); + if (tso_segsz > size) { + tso_segsz -= size; + dp_packet_set_tso_segsz(packet, tso_segsz); + } + } + /* The encapsulated packet has type Ethernet. Adjust dp_packet. */ packet->packet_type = htonl(PT_ETH); dp_packet_reset_offsets(packet); From patchwork Fri Jul 5 20:44:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Pattrick X-Patchwork-Id: 1957444 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FcYSSZUs; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WG59C55mzz1xqb for ; Sat, 6 Jul 2024 06:44:31 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 7BDFA41517; Fri, 5 Jul 2024 20:44:29 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id eBnZmlDGbNPJ; Fri, 5 Jul 2024 20:44:27 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 625CD414C9 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FcYSSZUs Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 625CD414C9; Fri, 5 Jul 2024 20:44:27 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id EF6B3C0A99; Fri, 5 Jul 2024 20:44:26 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id A0337C0A96 for ; Fri, 5 Jul 2024 20:44:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 841A560805 for ; Fri, 5 Jul 2024 20:44:25 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id dv_PH7T0kkcw for ; Fri, 5 Jul 2024 20:44:24 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=mkp@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org F4121605AF Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org F4121605AF Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FcYSSZUs Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id F4121605AF for ; Fri, 5 Jul 2024 20:44:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1720212263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nk5CgQbTZYn4R6awV4b+YLdqjm5ALQj3iJICatjzPIw=; b=FcYSSZUsJA9KRDmmwUyPrxC9y1n85W0KNRrTEVL56p0QjgXQzEfN+fEiG2RfnrHuOzA0G5 Z+3zi+0XKAL5qTYw3W+OjYoz29DBwISXhhug9wfYQObOGfjDahY4nVH0juEFwIvz/GxUb1 BtkUn5c1K68fxkEDEaxjxDSIkV03TTM= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-A8gnjGgaNmytvlrEzyWOmQ-1; Fri, 05 Jul 2024 16:44:21 -0400 X-MC-Unique: A8gnjGgaNmytvlrEzyWOmQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CC82719560A1 for ; Fri, 5 Jul 2024 20:44:20 +0000 (UTC) Received: from mpattric.remote.csb (unknown [10.22.18.114]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D3586195605A; Fri, 5 Jul 2024 20:44:19 +0000 (UTC) From: Mike Pattrick To: dev@openvswitch.org Date: Fri, 5 Jul 2024 16:44:12 -0400 Message-Id: <20240705204413.1105558-2-mkp@redhat.com> In-Reply-To: <20240705204413.1105558-1-mkp@redhat.com> References: <20240705204413.1105558-1-mkp@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 2/3] Userspace: Software fallback for UDP encapsulated TCP segmentation. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: david.marchand@redhat.com Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When sending packets that are flagged as requiring segmentation to an interface that does not support this feature, send the packet to the TSO software fallback instead of dropping it. Signed-off-by: Mike Pattrick --- v2: - Fixed udp tunnel length - Added test that UDP headers are correct - Split inner and outer ip_id into different counters - Set tunnel flags in reset_tcp_seg v3: - Changed logic of netdev_send() to account for NICs that support tunnel offload but not checksum offload - Adjusted udp tunnel header length during software tso v4: - Moved a bugfix into its own patch - Fixed an indentation issue - Changed the scope of ip_hdr Signed-off-by: Mike Pattrick --- lib/dp-packet-gso.c | 90 ++++++++++++++++++++++++++++++++--------- lib/dp-packet.h | 34 ++++++++++++++++ lib/netdev.c | 44 ++++++++++---------- tests/system-traffic.at | 87 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 214 insertions(+), 41 deletions(-) diff --git a/lib/dp-packet-gso.c b/lib/dp-packet-gso.c index 847685ad9..04ebb19da 100644 --- a/lib/dp-packet-gso.c +++ b/lib/dp-packet-gso.c @@ -47,6 +47,8 @@ dp_packet_gso_seg_new(const struct dp_packet *p, size_t hdr_len, seg->l2_5_ofs = p->l2_5_ofs; seg->l3_ofs = p->l3_ofs; seg->l4_ofs = p->l4_ofs; + seg->inner_l3_ofs = p->inner_l3_ofs; + seg->inner_l4_ofs = p->inner_l4_ofs; /* The protocol headers remain the same, so preserve hash and mark. */ *dp_packet_rss_ptr(seg) = *dp_packet_rss_ptr(p); @@ -71,7 +73,12 @@ dp_packet_gso_nr_segs(struct dp_packet *p) const char *data_tail; const char *data_pos; - data_pos = dp_packet_get_tcp_payload(p); + if (dp_packet_hwol_is_tunnel_vxlan(p) || + dp_packet_hwol_is_tunnel_geneve(p)) { + data_pos = dp_packet_get_inner_tcp_payload(p); + } else { + data_pos = dp_packet_get_tcp_payload(p); + } data_tail = (char *) dp_packet_tail(p) - dp_packet_l2_pad_size(p); return DIV_ROUND_UP(data_tail - data_pos, segsz); @@ -89,14 +96,16 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch **batches) static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); struct dp_packet_batch *curr_batch = *batches; struct tcp_header *tcp_hdr; - struct ip_header *ip_hdr; + uint16_t inner_ip_id = 0; + uint16_t outer_ip_id = 0; struct dp_packet *seg; uint16_t tcp_offset; uint16_t tso_segsz; uint32_t tcp_seq; - uint16_t ip_id; + bool outer_ipv4; int hdr_len; int seg_len; + bool tnl; tso_segsz = dp_packet_get_tso_segsz(p); if (!tso_segsz) { @@ -105,20 +114,37 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch **batches) return false; } - tcp_hdr = dp_packet_l4(p); + if (dp_packet_hwol_is_tunnel_vxlan(p) || + dp_packet_hwol_is_tunnel_geneve(p)) { + outer_ipv4 = dp_packet_hwol_is_outer_ipv4(p); + tcp_hdr = dp_packet_inner_l4(p); + tnl = true; + + if (outer_ipv4) { + outer_ip_id = ntohs(((struct ip_header *) dp_packet_l3(p))->ip_id); + } + if (dp_packet_hwol_is_ipv4(p)) { + struct ip_header *ip_hdr = dp_packet_inner_l3(p); + inner_ip_id = ntohs(ip_hdr->ip_id); + } + } else { + outer_ipv4 = dp_packet_hwol_is_ipv4(p); + tcp_hdr = dp_packet_l4(p); + tnl = false; + + if (outer_ipv4) { + struct ip_header *ip_hdr = dp_packet_l3(p); + outer_ip_id = ntohs(ip_hdr->ip_id); + } + } + tcp_offset = TCP_OFFSET(tcp_hdr->tcp_ctl); tcp_seq = ntohl(get_16aligned_be32(&tcp_hdr->tcp_seq)); - hdr_len = ((char *) dp_packet_l4(p) - (char *) dp_packet_eth(p)) + hdr_len = ((char *) tcp_hdr - (char *) dp_packet_eth(p)) + tcp_offset * 4; - ip_id = 0; - if (dp_packet_hwol_is_ipv4(p)) { - ip_hdr = dp_packet_l3(p); - ip_id = ntohs(ip_hdr->ip_id); - } - const char *data_tail = (char *) dp_packet_tail(p) - dp_packet_l2_pad_size(p); - const char *data_pos = dp_packet_get_tcp_payload(p); + const char *data_pos = (char *) tcp_hdr + tcp_offset * 4; int n_segs = dp_packet_gso_nr_segs(p); for (int i = 0; i < n_segs; i++) { @@ -130,14 +156,36 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch **batches) seg = dp_packet_gso_seg_new(p, hdr_len, data_pos, seg_len); data_pos += seg_len; + if (tnl) { + /* Update tunnel UDP header length. */ + struct udp_header *tnl_hdr; + + tnl_hdr = dp_packet_l4(seg); + tnl_hdr->udp_len = htons(dp_packet_l4_size(seg)); + + /* Update tunnel inner L3 header. */ + if (dp_packet_hwol_is_ipv4(seg)) { + struct ip_header *ip_hdr = dp_packet_inner_l3(seg); + ip_hdr->ip_tot_len = htons(dp_packet_inner_l3_size(seg)); + ip_hdr->ip_id = htons(inner_ip_id); + ip_hdr->ip_csum = 0; + inner_ip_id++; + } else { + struct ovs_16aligned_ip6_hdr *ip6_hdr; + + ip6_hdr = dp_packet_inner_l3(seg); + ip6_hdr->ip6_ctlun.ip6_un1.ip6_un1_plen + = htons(dp_packet_inner_l3_size(seg) - sizeof *ip6_hdr); + } + } + /* Update L3 header. */ - if (dp_packet_hwol_is_ipv4(seg)) { - ip_hdr = dp_packet_l3(seg); - ip_hdr->ip_tot_len = htons(sizeof *ip_hdr + - dp_packet_l4_size(seg)); - ip_hdr->ip_id = htons(ip_id); + if (outer_ipv4) { + struct ip_header *ip_hdr = dp_packet_l3(seg); + ip_hdr->ip_tot_len = htons(dp_packet_l3_size(seg)); + ip_hdr->ip_id = htons(outer_ip_id); ip_hdr->ip_csum = 0; - ip_id++; + outer_ip_id++; } else { struct ovs_16aligned_ip6_hdr *ip6_hdr = dp_packet_l3(seg); @@ -146,7 +194,11 @@ dp_packet_gso(struct dp_packet *p, struct dp_packet_batch **batches) } /* Update L4 header. */ - tcp_hdr = dp_packet_l4(seg); + if (tnl) { + tcp_hdr = dp_packet_inner_l4(seg); + } else { + tcp_hdr = dp_packet_l4(seg); + } put_16aligned_be32(&tcp_hdr->tcp_seq, htonl(tcp_seq)); tcp_seq += seg_len; if (OVS_LIKELY(i < (n_segs - 1))) { diff --git a/lib/dp-packet.h b/lib/dp-packet.h index a75b1c5cd..af1ff088f 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -529,6 +529,16 @@ dp_packet_inner_l3(const struct dp_packet *b) : NULL; } +static inline size_t +dp_packet_inner_l3_size(const struct dp_packet *b) +{ + return OVS_LIKELY(b->inner_l3_ofs != UINT16_MAX) + ? (const char *) dp_packet_tail(b) + - (const char *) dp_packet_inner_l3(b) + - dp_packet_l2_pad_size(b) + : 0; +} + static inline void * dp_packet_inner_l4(const struct dp_packet *b) { @@ -563,6 +573,22 @@ dp_packet_get_tcp_payload(const struct dp_packet *b) return NULL; } +static inline const void * +dp_packet_get_inner_tcp_payload(const struct dp_packet *b) +{ + size_t l4_size = dp_packet_inner_l4_size(b); + + if (OVS_LIKELY(l4_size >= TCP_HEADER_LEN)) { + struct tcp_header *tcp = dp_packet_inner_l4(b); + int tcp_len = TCP_OFFSET(tcp->tcp_ctl) * 4; + + if (OVS_LIKELY(tcp_len >= TCP_HEADER_LEN && tcp_len <= l4_size)) { + return (const char *) tcp + tcp_len; + } + } + return NULL; +} + static inline uint32_t dp_packet_get_tcp_payload_length(const struct dp_packet *pkt) { @@ -1320,6 +1346,14 @@ dp_packet_hwol_reset_tcp_seg(struct dp_packet *p) ol_flags |= DP_PACKET_OL_TX_IP_CKSUM; } + if (ol_flags & (DP_PACKET_OL_TX_TUNNEL_VXLAN | + DP_PACKET_OL_TX_TUNNEL_GENEVE)) { + if (ol_flags & DP_PACKET_OL_TX_OUTER_IPV4) { + ol_flags |= DP_PACKET_OL_TX_OUTER_IP_CKSUM; + } + ol_flags |= DP_PACKET_OL_TX_OUTER_UDP_CKSUM; + } + *dp_packet_ol_flags_ptr(p) = ol_flags; } diff --git a/lib/netdev.c b/lib/netdev.c index f2d921ed6..866dbf3b7 100644 --- a/lib/netdev.c +++ b/lib/netdev.c @@ -69,8 +69,6 @@ COVERAGE_DEFINE(netdev_received); COVERAGE_DEFINE(netdev_sent); COVERAGE_DEFINE(netdev_add_router); COVERAGE_DEFINE(netdev_get_stats); -COVERAGE_DEFINE(netdev_vxlan_tso_drops); -COVERAGE_DEFINE(netdev_geneve_tso_drops); COVERAGE_DEFINE(netdev_push_header_drops); COVERAGE_DEFINE(netdev_soft_seg_good); COVERAGE_DEFINE(netdev_soft_seg_drops); @@ -910,28 +908,30 @@ netdev_send(struct netdev *netdev, int qid, struct dp_packet_batch *batch, struct dp_packet *packet; int error; - if (userspace_tso_enabled() && - !(netdev_flags & NETDEV_TX_OFFLOAD_TCP_TSO)) { - DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { - if (dp_packet_hwol_is_tso(packet)) { - if (dp_packet_hwol_is_tunnel_vxlan(packet) - && !(netdev_flags & NETDEV_TX_VXLAN_TNL_TSO)) { - VLOG_WARN_RL(&rl, "%s: No VXLAN TSO support", - netdev_get_name(netdev)); - COVERAGE_INC(netdev_vxlan_tso_drops); - dp_packet_delete_batch(batch, true); - return false; + if (userspace_tso_enabled()) { + if (!(netdev_flags & NETDEV_TX_OFFLOAD_TCP_TSO)) { + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + if (dp_packet_hwol_is_tso(packet)) { + return netdev_send_tso(netdev, qid, batch, concurrent_txq); } - - if (dp_packet_hwol_is_tunnel_geneve(packet) - && !(netdev_flags & NETDEV_TX_GENEVE_TNL_TSO)) { - VLOG_WARN_RL(&rl, "%s: No GENEVE TSO support", - netdev_get_name(netdev)); - COVERAGE_INC(netdev_geneve_tso_drops); - dp_packet_delete_batch(batch, true); - return false; + } + } else if (!(netdev_flags & NETDEV_TX_OFFLOAD_OUTER_UDP_CKSUM)) { + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + if (dp_packet_hwol_is_tso(packet) && + (dp_packet_hwol_is_tunnel_vxlan(packet) || + dp_packet_hwol_is_tunnel_geneve(packet)) && + dp_packet_hwol_is_outer_udp_cksum(packet)) { + return netdev_send_tso(netdev, qid, batch, concurrent_txq); + } + } + } else if (!(netdev_flags & (NETDEV_TX_VXLAN_TNL_TSO | + NETDEV_TX_GENEVE_TNL_TSO))) { + DP_PACKET_BATCH_FOR_EACH (i, packet, batch) { + if (dp_packet_hwol_is_tso(packet) && + (dp_packet_hwol_is_tunnel_vxlan(packet) || + dp_packet_hwol_is_tunnel_geneve(packet))) { + return netdev_send_tso(netdev, qid, batch, concurrent_txq); } - return netdev_send_tso(netdev, qid, batch, concurrent_txq); } } } diff --git a/tests/system-traffic.at b/tests/system-traffic.at index 3f1a15445..dd30b6bf5 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -351,6 +351,93 @@ OVS_WAIT_UNTIL([diff -q payload.bin udp_data]) OVS_TRAFFIC_VSWITCHD_STOP AT_CLEANUP +AT_SETUP([datapath - tcp over vxlan tunnel with software fallback]) +AT_SKIP_IF([test $HAVE_NC = no]) +AT_SKIP_IF([test $HAVE_TCPDUMP = no]) +OVS_CHECK_VXLAN() + +dnl This test is only valid with tso. If the kernel segments the packets, the +dnl packet lengths in the final test will be different. +m4_ifndef([CHECK_SYSTEM_TSO], [AT_SKIP_IF(:)]) + +OVS_TRAFFIC_VSWITCHD_START() +ADD_BR([br-underlay]) + +AT_CHECK([ovs-ofctl add-flow br0 "actions=normal"]) +AT_CHECK([ovs-ofctl add-flow br-underlay "actions=normal"]) + +ADD_NAMESPACES(at_ns0) + +dnl Set up underlay link from host into the namespace using veth pair. +ADD_VETH(p0, at_ns0, br-underlay, "172.31.1.1/24") +AT_CHECK([ip addr add dev br-underlay "172.31.1.100/24"]) +AT_CHECK([ip link set dev br-underlay up]) + +dnl Test the case where only one side has all checksum and tso offload disabled. +AT_CHECK([ethtool -K ovs-p0 tso off], [0], [ignore], [ignore]) +AT_CHECK([ethtool -K ovs-p0 sg off], [0], [ignore], [ignore]) + +dnl Reinitialize. +AT_CHECK([ovs-vsctl del-port ovs-p0]) +AT_CHECK([ovs-vsctl add-port br-underlay ovs-p0]) + +dnl Set up tunnel endpoints on OVS outside the namespace and with a native +dnl linux device inside the namespace. +ADD_OVS_TUNNEL([vxlan], [br0], [at_vxlan0], [172.31.1.1], [10.1.1.100/24]) +ADD_NATIVE_TUNNEL([vxlan], [at_vxlan1], [at_ns0], [172.31.1.100], [10.1.1.1/24], + [id 0 dstport 4789]) + +dnl First, check the underlay. +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -W 2 172.31.1.100 | FORMAT_PING], [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + +dnl Check that the tunnel is up. +NS_CHECK_EXEC([at_ns0], [ping -q -c 3 -i 0.3 -W 2 10.1.1.100 | FORMAT_PING], [0], [dnl +3 packets transmitted, 3 received, 0% packet loss, time 0ms +]) + +dnl Start tcpdump to capture the encapsulated packets. +OVS_DAEMONIZE([tcpdump -i ovs-p0 -w p0.pcap], [tcpdump.pid]) +sleep 1 + +dnl Initialize the listener before it is needed. +NETNS_DAEMONIZE([at_ns0], [nc -l 10.1.1.1 1234 > data2], [nc.pid]) + +dnl Verify that ncat is ready. +OVS_WAIT_UNTIL([NS_EXEC([at_ns0], [netstat -ln | grep :1234])]) + +dnl Large TCP transfer aimed towards ovs-p0, which has TSO disabled. +AT_CHECK([dd if=/dev/urandom of=payload.bin bs=60000 count=1 2> /dev/null]) +AT_CHECK([nc $NC_EOF_OPT 10.1.1.1 1234 < payload.bin]) + +dnl Wait until transfer completes before checking. +OVS_WAIT_WHILE([kill -0 $(cat nc.pid)]) +AT_CHECK([diff -q payload.bin data2], [0]) + +dnl Stop OVS and tcpdump and verify the results. +OVS_TRAFFIC_VSWITCHD_STOP + +AT_CHECK([kill -15 $(cat tcpdump.pid)]) + +OVS_WAIT_WHILE([kill -0 $(cat tcpdump.pid)]) + +ovs-pcap p0.pcap + +dnl The exact number of packets sent will vary, but we check that the largest segments have the correct +dnl lengths and certain other fields. +AT_CHECK([test $(ovs-pcap p0.pcap | grep -Ec dnl +"^.{24}0800"dnl Ethernet +"450005aa....4000..11....ac1f0164ac1f0101"dnl IP(len=1450, DF, UDP, 172.31.1.100->172.31.1.1) +"....12b505960000"dnl UDP(len=1430, dport=4789) +"0800000000000000"dnl VXLAN(gpid=0, vni=0) +".{24}0800"dnl Ethernet +"45000578....4000..06....0a0101640a010101"dnl IP(len=1400, DF, TCP, 10.1.1.100->10.1.1.1) +"....04d2............................0000"dnl TCP(dport=1234 +) -ge 20]) + +AT_CLEANUP + AT_SETUP([datapath - ping vlan over vxlan tunnel]) OVS_CHECK_TUNNEL_TSO() OVS_CHECK_VXLAN() From patchwork Fri Jul 5 20:44:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Pattrick X-Patchwork-Id: 1957446 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Nt+DWaPU; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WG59H0f6vz1xqb for ; Sat, 6 Jul 2024 06:44:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id ECC1D80C0E; Fri, 5 Jul 2024 20:44:30 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id mgHLfUigprkY; Fri, 5 Jul 2024 20:44:29 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2228780C57 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Nt+DWaPU Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2228780C57; Fri, 5 Jul 2024 20:44:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E91E1C0A98; Fri, 5 Jul 2024 20:44:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6A875C0A96 for ; Fri, 5 Jul 2024 20:44:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 51B5A414BF for ; Fri, 5 Jul 2024 20:44:27 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id xCYbCXVioS9Z for ; Fri, 5 Jul 2024 20:44:26 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=mkp@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 691F8404FC Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 691F8404FC Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Nt+DWaPU Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 691F8404FC for ; Fri, 5 Jul 2024 20:44:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1720212265; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AMldGrpx+cxsspemmYzQdUKzhpo4vWdukXzzBpIYGi0=; b=Nt+DWaPUO5Xgds/ZnFYeTkKPJ3rj8BkjIRb66dOOKtRmPLZQifg6gcKoetFttIJKyI6A0l p09zAzzJM5QOcxVoj7MGJR4iq5QOnW3JKvzN6n1kKoDuKJCiBonDkcpFJh5uyw026FLa2Y k/LjSpT0xcKdGN6cS+TOKBvr8F769ck= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-524-T1KYne0ZMzmbHiGkRmcOXA-1; Fri, 05 Jul 2024 16:44:23 -0400 X-MC-Unique: T1KYne0ZMzmbHiGkRmcOXA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0F3881956088 for ; Fri, 5 Jul 2024 20:44:23 +0000 (UTC) Received: from mpattric.remote.csb (unknown [10.22.18.114]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 28086195605A; Fri, 5 Jul 2024 20:44:20 +0000 (UTC) From: Mike Pattrick To: dev@openvswitch.org Date: Fri, 5 Jul 2024 16:44:13 -0400 Message-Id: <20240705204413.1105558-3-mkp@redhat.com> In-Reply-To: <20240705204413.1105558-1-mkp@redhat.com> References: <20240705204413.1105558-1-mkp@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v4 3/3] netdev-dpdk: Re-enable VXLAN/Geneve offload for Intel cards. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: david.marchand@redhat.com Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Previously the OVS support for checksum/TSO offloading didn't work well with some network cards that supported VXLAN/Geneve tunnel TSO but not outer UDP checksums. Now support for this configuration is improved and we no longer need to disable the VXLAN/Geneve TSO flags from intel hardware support flags. The modification to outer UDP offload is still required pending a future DPDK release. Suggested-by: David Marchand Signed-off-by: Mike Pattrick Reviewed-by: David Marchand --- lib/netdev-dpdk.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 02cef6e45..95a78241d 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1363,8 +1363,6 @@ dpdk_eth_dev_init(struct netdev_dpdk *dev) "net/ice, net/i40e or net/iavf port.", netdev_get_name(&dev->up)); info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM; - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO; - info.tx_offload_capa &= ~RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO; } if (info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM) {