From patchwork Wed Jan 29 23:23:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: William Tu X-Patchwork-Id: 1231169 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=hzUtHePh; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 487KL40sy1z9s1x for ; Thu, 30 Jan 2020 10:23:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id F2EF2883AD; Wed, 29 Jan 2020 23:23:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GKXERDX+ZsXc; Wed, 29 Jan 2020 23:23:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 05CC787B2E; Wed, 29 Jan 2020 23:23:37 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DC47AC0881; Wed, 29 Jan 2020 23:23:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3D9D5C0171 for ; Wed, 29 Jan 2020 23:23:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 26C1286403 for ; Wed, 29 Jan 2020 23:23:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pXPaRHvYP-SQ for ; Wed, 29 Jan 2020 23:23:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 84A9A863F2 for ; Wed, 29 Jan 2020 23:23:32 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id l24so619732pgk.2 for ; Wed, 29 Jan 2020 15:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=Q+NXuKdHu/zu1GnGVzn1sS+ZScn0cM8Lqz73upe69cY=; b=hzUtHePhRrJ6PqRyXKvzHmtx2yjN6vENW7f8yJbvKU4kE3oWjwspuU1WK0AKL2/yLB qXflAmLw97CSt7UgeD3qXf6OYNY9Woc/jx0Jxj/ae8zvmTItWcTFSsGi5Fz/kLTJ2cLZ 1bADb1JUpH7BDRlJ8FO4QRscrIJUBIktC97FIXLF6koXcLvX2epc+MF0h9Yjc3vidJiF JcgoLvzyCRupNONuBCIcjNWsQ4E6fQJaVczR9+z8yHDegoYXVwKD9zYU0lZ/RETKCVLE q/iZhiGyZHMtUxkGP5RpzXCcTLYKsDE2zAPbrMEpy/KSQv+YOWiDKPVf+tNRO3hfhTfV Rp2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=Q+NXuKdHu/zu1GnGVzn1sS+ZScn0cM8Lqz73upe69cY=; b=T7oLUtf1hMpwdNzhkggmFeK0rAbCEQrmt/1pEOB5Zwps0PAxfCyCKHEbquTIEMqYHp Ohvm/6p4n8hGj+dBItjdlatPqOqkYLxHCaR1qRLA3zCAYy8yBWlAJf+Tm8uOmWpSlQaw alRB4mn9+etUUKHoWlujT0jZXAzFHJJRIOh0b0EP14IceUySx8uICUmuoLQrmSmjRKJW nzsKjwSeH202W5Z7jcWuNpx18/HU+HrGif93aYwXAynSQp4P0TY8Ow48xfWoNmTABsPa CITxS0239x7NDpFFSkNVY9bHpjROW+fIXs84wYJq7jiVw8lX5kAV8Wovn6ThXFrX9HlU S83A== X-Gm-Message-State: APjAAAXzoUl/WXAe60R7cO6DqRvImXuK/BB5WoJiMgduQjrUYDcRIZtb 6/KXYR8u9V4CEZ/+5dtTxiBhrdYrr+A= X-Google-Smtp-Source: APXvYqw933eUpc1NSpnRIkmfg5+NC4BKdTqDCQeZEaPgWuauPIKKJ8A+gJDEk47t1LzPn2p38+aAlg== X-Received: by 2002:aa7:9a86:: with SMTP id w6mr2065908pfi.59.1580340211779; Wed, 29 Jan 2020 15:23:31 -0800 (PST) Received: from sc9-mailhost2.vmware.com ([66.170.99.1]) by smtp.gmail.com with ESMTPSA id x132sm3798460pfc.148.2020.01.29.15.23.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 29 Jan 2020 15:23:31 -0800 (PST) From: William Tu To: dev@openvswitch.org Date: Wed, 29 Jan 2020 15:23:04 -0800 Message-Id: <1580340184-53551-1-git-send-email-u9012063@gmail.com> X-Mailer: git-send-email 2.7.4 Cc: fbl@sysclose.org, i.maximets@ovn.org, txfh2007@aliyun.com Subject: [ovs-dev] [PATCH] userspace: Enable TSO support for non-DPDK. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" For some cases, we want to use userspace datapath but not with DPDK library, ex: using AF_XDP. The patch enables non-DPDK support for TSO (TCP Segmentation Offload). I measured the performance using: iperf3 -c (ns0) -> veth peer -> OVS -> veth peer -> iperf3 -s (ns1) And got around 6Gbps, similar to that with DPDK. Tested-at: https://travis-ci.org/williamtu/ovs-travis/builds/643599592 Signed-off-by: William Tu --- lib/dp-packet.h | 95 ++++++++++++++++++++++++++++++++--------------------- lib/userspace-tso.c | 9 ++--- 2 files changed, 62 insertions(+), 42 deletions(-) diff --git a/lib/dp-packet.h b/lib/dp-packet.h index 69ae5dfac7f0..d0a964b87fb8 100644 --- a/lib/dp-packet.h +++ b/lib/dp-packet.h @@ -53,7 +53,25 @@ enum OVS_PACKED_ENUM dp_packet_source { enum dp_packet_offload_mask { DP_PACKET_OL_RSS_HASH_MASK = 0x1, /* Is the 'rss_hash' valid? */ DP_PACKET_OL_FLOW_MARK_MASK = 0x2, /* Is the 'flow_mark' valid? */ + DP_PACKET_OL_RX_L4_CKSUM_BAD = 1 << 3, + DP_PACKET_OL_RX_IP_CKSUM_BAD = 1 << 4, + DP_PACKET_OL_RX_L4_CKSUM_GOOD = 1 << 5, + DP_PACKET_OL_RX_IP_CKSUM_GOOD = 1 << 6, + DP_PACKET_OL_TX_TCP_SEG = 1 << 7, + DP_PACKET_OL_TX_IPV4 = 1 << 8, + DP_PACKET_OL_TX_IPV6 = 1 << 9, + DP_PACKET_OL_TX_TCP_CKSUM = 1 << 10, + DP_PACKET_OL_TX_UDP_CKSUM = 1 << 11, + DP_PACKET_OL_TX_SCTP_CKSUM = 1 << 12, }; + +#define DP_PACKET_OL_TX_L4_MASK (DP_PACKET_OL_TX_TCP_CKSUM | \ + DP_PACKET_OL_TX_UDP_CKSUM | \ + DP_PACKET_OL_TX_SCTP_CKSUM) +#define DP_PACKET_OL_RX_IP_CKSUM_MASK (DP_PACKET_OL_RX_IP_CKSUM_GOOD | \ + DP_PACKET_OL_RX_IP_CKSUM_BAD) +#define DP_PACKET_OL_RX_L4_CKSUM_MASK (DP_PACKET_OL_RX_L4_CKSUM_GOOD | \ + DP_PACKET_OL_RX_L4_CKSUM_BAD) #else /* DPDK mbuf ol_flags that are not really an offload flags. These are mostly * related to mbuf memory layout and OVS should not touch/clear them. */ @@ -737,82 +755,79 @@ dp_packet_set_allocated(struct dp_packet *b, uint16_t s) b->allocated_ = s; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_is_tso(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_is_tso(const struct dp_packet *b) { - return false; + return !!(b->ol_flags & DP_PACKET_OL_TX_TCP_SEG); } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_is_ipv4(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_is_ipv4(const struct dp_packet *b) { - return false; + return !!(b->ol_flags & DP_PACKET_OL_TX_IPV4); } -/* There are no implementation when not DPDK enabled datapath. */ static inline uint64_t -dp_packet_hwol_l4_mask(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_mask(const struct dp_packet *b) { - return 0; + return b->ol_flags & DP_PACKET_OL_TX_L4_MASK; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_tcp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_tcp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_TCP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_udp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_udp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_UDP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline bool -dp_packet_hwol_l4_is_sctp(const struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_l4_is_sctp(const struct dp_packet *b) { - return false; + return (b->ol_flags & DP_PACKET_OL_TX_L4_MASK) == + DP_PACKET_OL_TX_SCTP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tx_ipv4(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tx_ipv4(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_IPV4; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tx_ipv6(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tx_ipv6(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_IPV6; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_tcp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_tcp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_TCP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_udp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_udp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_UDP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_csum_sctp(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_csum_sctp(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_SCTP_CKSUM; } -/* There are no implementation when not DPDK enabled datapath. */ static inline void -dp_packet_hwol_set_tcp_seg(struct dp_packet *b OVS_UNUSED) +dp_packet_hwol_set_tcp_seg(struct dp_packet *b) { + b->ol_flags |= DP_PACKET_OL_TX_TCP_SEG; } /* Returns the RSS hash of the packet 'p'. Note that the returned value is @@ -843,27 +858,31 @@ dp_packet_reset_offload(struct dp_packet *p) } static inline bool -dp_packet_ip_checksum_valid(const struct dp_packet *p OVS_UNUSED) +dp_packet_ip_checksum_valid(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_IP_CKSUM_MASK) == + DP_PACKET_OL_RX_IP_CKSUM_GOOD; } static inline bool -dp_packet_ip_checksum_bad(const struct dp_packet *p OVS_UNUSED) +dp_packet_ip_checksum_bad(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_IP_CKSUM_MASK) == + DP_PACKET_OL_RX_IP_CKSUM_BAD; } static inline bool -dp_packet_l4_checksum_valid(const struct dp_packet *p OVS_UNUSED) +dp_packet_l4_checksum_valid(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_L4_CKSUM_MASK) == + DP_PACKET_OL_RX_L4_CKSUM_GOOD; } static inline bool -dp_packet_l4_checksum_bad(const struct dp_packet *p OVS_UNUSED) +dp_packet_l4_checksum_bad(const struct dp_packet *p) { - return false; + return (p->ol_flags & DP_PACKET_OL_RX_L4_CKSUM_MASK) == + DP_PACKET_OL_RX_L4_CKSUM_BAD; } static inline bool diff --git a/lib/userspace-tso.c b/lib/userspace-tso.c index 6a4a0149b7f5..a1a5e1de714f 100644 --- a/lib/userspace-tso.c +++ b/lib/userspace-tso.c @@ -35,12 +35,13 @@ userspace_tso_init(const struct smap *ovs_other_config) if (ovsthread_once_start(&once)) { #ifdef DPDK_NETDEV - VLOG_INFO("Userspace TCP Segmentation Offloading support enabled"); - userspace_tso = true; + VLOG_INFO("Userspace TCP Segmentation Offloading support enabled" + " by DPDK"); #else - VLOG_WARN("Userspace TCP Segmentation Offloading can not be enabled" - "since OVS is built without DPDK support."); + VLOG_INFO("Userspace TCP Segmentation Offloading support enabled" + " by native OVS"); #endif + userspace_tso = true; ovsthread_once_done(&once); } }