From patchwork Fri Dec 8 10:53:46 2017
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Mark Kavanagh
X-Patchwork-Id: 846147
X-Patchwork-Delegate: ian.stokes@intel.com
Return-Path:
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Authentication-Results: ozlabs.org;
spf=pass (mailfrom) smtp.mailfrom=openvswitch.org
(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;
envelope-from=ovs-dev-bounces@openvswitch.org;
receiver=)
Received: from mail.linuxfoundation.org (mail.linuxfoundation.org
[140.211.169.12])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (No client certificate requested)
by ozlabs.org (Postfix) with ESMTPS id 3ytTkf4PCZz9s83
for ;
Fri, 8 Dec 2017 21:54:34 +1100 (AEDT)
Received: from mail.linux-foundation.org (localhost [127.0.0.1])
by mail.linuxfoundation.org (Postfix) with ESMTP id 17F63BE8;
Fri, 8 Dec 2017 10:54:00 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@mail.linuxfoundation.org
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 86628BCF
for ; Fri, 8 Dec 2017 10:53:58 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 933811A4
for ; Fri, 8 Dec 2017 10:53:57 +0000 (UTC)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
08 Dec 2017 02:53:57 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="182445149"
Received: from silpixa00380299.ir.intel.com ([10.237.222.17])
by orsmga005.jf.intel.com with ESMTP; 08 Dec 2017 02:53:54 -0800
From: Mark Kavanagh
To: dev@openvswitch.org
Date: Fri, 8 Dec 2017 10:53:46 +0000
Message-Id: <1512730427-34577-2-git-send-email-mark.b.kavanagh@intel.com>
X-Mailer: git-send-email 1.9.3
In-Reply-To: <1512730427-34577-1-git-send-email-mark.b.kavanagh@intel.com>
References: <1512730427-34577-1-git-send-email-mark.b.kavanagh@intel.com>
X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI,
T_RP_MATCHES_RCVD autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: maxime.coquelin@redhat.com, i.maximets@samsung.com
Subject: [ovs-dev] [PATCH V6 1/2] netdev-dpdk: DPDK v17.11 upgrade
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
MIME-Version: 1.0
Sender: ovs-dev-bounces@openvswitch.org
Errors-To: ovs-dev-bounces@openvswitch.org
This commit adds support for DPDK v17.11:
- minor updates to accomodate DPDK API changes
- update references to DPDK version in Documentation
- update DPDK version in travis' linux-build script
- document DPDK v17.11 virtio driver bug
Signed-off-by: Mark Kavanagh
Acked-by: Maxime Coquelin
Acked-by: Ciara Loftus
Acked-by: Jan Scheurich
Tested-by: Jan Scheurich
Tested-by: Guoshuai Li
Acked-by: Kevin Traynor
---
.travis/linux-build.sh | 2 +-
Documentation/faq/releases.rst | 1 +
Documentation/intro/install/dpdk.rst | 10 +++++-----
Documentation/topics/dpdk/ring.rst | 2 +-
Documentation/topics/dpdk/vhost-user.rst | 32 +++++++++++++++++++++++++++-----
NEWS | 2 ++
lib/netdev-dpdk.c | 5 +++--
7 files changed, 40 insertions(+), 14 deletions(-)
diff --git a/.travis/linux-build.sh b/.travis/linux-build.sh
index 4d6459f..ed28ee4 100755
--- a/.travis/linux-build.sh
+++ b/.travis/linux-build.sh
@@ -81,7 +81,7 @@ fi
if [ "$DPDK" ]; then
if [ -z "$DPDK_VER" ]; then
- DPDK_VER="17.05.2"
+ DPDK_VER="17.11"
fi
install_dpdk $DPDK_VER
if [ "$CC" = "clang" ]; then
diff --git a/Documentation/faq/releases.rst b/Documentation/faq/releases.rst
index d903b06..62a1957 100644
--- a/Documentation/faq/releases.rst
+++ b/Documentation/faq/releases.rst
@@ -164,6 +164,7 @@ Q: What DPDK version does each Open vSwitch release work with?
2.6.x 16.07.2
2.7.x 16.11.3
2.8.x 17.05.2
+ 2.9.x 17.11
============ =======
Q: I get an error like this when I configure Open vSwitch:
diff --git a/Documentation/intro/install/dpdk.rst b/Documentation/intro/install/dpdk.rst
index bb69ae5..3fecb5c 100644
--- a/Documentation/intro/install/dpdk.rst
+++ b/Documentation/intro/install/dpdk.rst
@@ -40,7 +40,7 @@ Build requirements
In addition to the requirements described in :doc:`general`, building Open
vSwitch with DPDK will require the following:
-- DPDK 17.05.2
+- DPDK 17.11
- A `DPDK supported NIC`_
@@ -69,9 +69,9 @@ Install DPDK
#. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``::
$ cd /usr/src/
- $ wget http://fast.dpdk.org/rel/dpdk-17.05.2.tar.xz
- $ tar xf dpdk-17.05.2.tar.xz
- $ export DPDK_DIR=/usr/src/dpdk-stable-17.05.2
+ $ wget http://fast.dpdk.org/rel/dpdk-17.11.tar.xz
+ $ tar xf dpdk-17.11.tar.xz
+ $ export DPDK_DIR=/usr/src/dpdk-17.11
$ cd $DPDK_DIR
#. (Optional) Configure DPDK as a shared library
@@ -583,7 +583,7 @@ Limitations
The latest list of validated firmware versions can be found in the `DPDK
release notes`_.
-.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_17_05.html
+.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_17_11.html
Reporting Bugs
--------------
diff --git a/Documentation/topics/dpdk/ring.rst b/Documentation/topics/dpdk/ring.rst
index ad9d7a5..8d0ede8 100644
--- a/Documentation/topics/dpdk/ring.rst
+++ b/Documentation/topics/dpdk/ring.rst
@@ -77,4 +77,4 @@ DPDK. However, this functionality was removed because:
- :doc:`vhost-user interfaces ` are the defacto DPDK-based path to
guests
-.. _DPDK documentation: https://dpdk.readthedocs.io/en/v17.05/prog_guide/ring_lib.html
+.. _DPDK documentation: https://dpdk.readthedocs.io/en/v17.11/prog_guide/ring_lib.html
diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst
index 74ac06e..a43affa 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -292,9 +292,9 @@ To begin, instantiate a guest as described in :ref:`dpdk-vhost-user` or
DPDK sources to VM and build DPDK::
$ cd /root/dpdk/
- $ wget http://fast.dpdk.org/rel/dpdk-17.05.2.tar.xz
- $ tar xf dpdk-17.05.2.tar.xz
- $ export DPDK_DIR=/root/dpdk/dpdk-stable-17.05.2
+ $ wget http://fast.dpdk.org/rel/dpdk-17.11.tar.xz
+ $ tar xf dpdk-17.11.tar.xz
+ $ export DPDK_DIR=/root/dpdk/dpdk-17.11
$ export DPDK_TARGET=x86_64-native-linuxapp-gcc
$ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET
$ cd $DPDK_DIR
@@ -327,6 +327,28 @@ Finally, start the application::
# TODO
+.. important::
+
+ DPDK v17.11 virtio PMD contains a bug in the vectorized Rx function that
+ affects testpmd/DPDK guest applications. As such, guest DPDK applications
+ should use a non-vectorized Rx function.
+
+The DPDK v17.11 virtio net driver contains a bug that prevents guest DPDK
+applications from receiving packets when the vectorized Rx function is used.
+This only occurs when guest-bound traffic is live before a DPDK application is
+started within the guest, and where two or more forwarding cores are used. As
+such, it is not recommended for guests which execute DPDK applications to use
+the virtio vectorized Rx function. A simple method of ensuring that a non-
+vectorized Rx function is used is to enable mergeable buffers for the guest,
+with the following QEMU command line option::
+
+ mrg_rxbuf=on
+
+Additional details regarding the virtio driver bug are available on the
+`DPDK mailing list`_.
+
+.. _DPDK mailing list: http://dpdk.org/ml/archives/dev/2017-December/082801.html
+
.. _dpdk-vhost-user-xml:
Sample XML
@@ -387,7 +409,7 @@ Sample XML
-
+
@@ -395,7 +417,7 @@ Sample XML
-
+
diff --git a/NEWS b/NEWS
index 427c8f8..d4a1c9a 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,8 @@ Post-v2.8.0
* ovn-ctl: New commands run_nb_ovsdb and run_sb_ovsdb.
- Linux kernel 4.13
* Add support for compiling OVS with the latest Linux 4.13 kernel
+ - DPDK:
+ * Add support for DPDK v17.11
v2.8.0 - 31 Aug 2017
--------------------
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index faff842..f552444 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -26,6 +26,7 @@
#include
#include
+#include
#include
#include
#include
@@ -140,8 +141,8 @@ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 20);
#define DPDK_ETH_PORT_ID_INVALID RTE_MAX_ETHPORTS
-/* DPDK library uses uint8_t for port_id. */
-typedef uint8_t dpdk_port_t;
+/* DPDK library uses uint16_t for port_id. */
+typedef uint16_t dpdk_port_t;
#define VHOST_ENQ_RETRY_NUM 8
#define IF_NAME_SZ (PATH_MAX > IFNAMSIZ ? PATH_MAX : IFNAMSIZ)
From patchwork Fri Dec 8 10:53:47 2017
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
X-Patchwork-Submitter: Mark Kavanagh
X-Patchwork-Id: 846150
X-Patchwork-Delegate: ian.stokes@intel.com
Return-Path:
X-Original-To: incoming@patchwork.ozlabs.org
Delivered-To: patchwork-incoming@bilbo.ozlabs.org
Authentication-Results: ozlabs.org;
spf=pass (mailfrom) smtp.mailfrom=openvswitch.org
(client-ip=140.211.169.12; helo=mail.linuxfoundation.org;
envelope-from=ovs-dev-bounces@openvswitch.org;
receiver=)
Received: from mail.linuxfoundation.org (mail.linuxfoundation.org
[140.211.169.12])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (No client certificate requested)
by ozlabs.org (Postfix) with ESMTPS id 3ytTlR6bbdz9s74
for ;
Fri, 8 Dec 2017 21:55:15 +1100 (AEDT)
Received: from mail.linux-foundation.org (localhost [127.0.0.1])
by mail.linuxfoundation.org (Postfix) with ESMTP id ED987BF2;
Fri, 8 Dec 2017 10:54:01 +0000 (UTC)
X-Original-To: dev@openvswitch.org
Delivered-To: ovs-dev@mail.linuxfoundation.org
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 1AAC5BE6
for ; Fri, 8 Dec 2017 10:54:01 +0000 (UTC)
X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6D5EE189
for ; Fri, 8 Dec 2017 10:54:00 +0000 (UTC)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga005.jf.intel.com ([10.7.209.41])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
08 Dec 2017 02:54:00 -0800
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.45,377,1508828400"; d="scan'208";a="182445155"
Received: from silpixa00380299.ir.intel.com ([10.237.222.17])
by orsmga005.jf.intel.com with ESMTP; 08 Dec 2017 02:53:57 -0800
From: Mark Kavanagh
To: dev@openvswitch.org
Date: Fri, 8 Dec 2017 10:53:47 +0000
Message-Id: <1512730427-34577-3-git-send-email-mark.b.kavanagh@intel.com>
X-Mailer: git-send-email 1.9.3
In-Reply-To: <1512730427-34577-1-git-send-email-mark.b.kavanagh@intel.com>
References: <1512730427-34577-1-git-send-email-mark.b.kavanagh@intel.com>
X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI,
T_RP_MATCHES_RCVD autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: maxime.coquelin@redhat.com, i.maximets@samsung.com
Subject: [ovs-dev] [PATCH V6 2/2] netdev-dpdk: vHost IOMMU support
X-BeenThere: ovs-dev@openvswitch.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id:
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
MIME-Version: 1.0
Sender: ovs-dev-bounces@openvswitch.org
Errors-To: ovs-dev-bounces@openvswitch.org
DPDK v17.11 introduces support for the vHost IOMMU feature.
This is a security feature, which restricts the vhost memory
that a virtio device may access.
This feature also enables the vhost REPLY_ACK protocol, the
implementation of which is known to work in newer versions of
QEMU (i.e. v2.10.0), but is buggy in older versions (v2.7.0 -
v2.9.0, inclusive). As such, the feature is disabled by default
in (and should remain so), for the aforementioned older QEMU
verions. Starting with QEMU v2.9.1, vhost-iommu-support can
safely be enabled, even without having an IOMMU device, with
no performance penalty.
This patch adds a new global config option, vhost-iommu-support,
that controls enablement of the vhost IOMMU feature:
ovs-vsctl set Open_vSwitch . other_config:vhost-iommu-support=true
This value defaults to false; to enable IOMMU support, this field
should be set to true when setting other global parameters on init
(such as "dpdk-socket-mem", for example). Changing the value at
runtime is not supported, and requires restarting the vswitch daemon.
Signed-off-by: Mark Kavanagh
Acked-by: Kevin Traynor
---
Documentation/topics/dpdk/vhost-user.rst | 28 ++++++++++++++++++++++++++++
NEWS | 1 +
lib/dpdk-stub.c | 6 ++++++
lib/dpdk.c | 12 ++++++++++++
lib/dpdk.h | 3 +++
lib/netdev-dpdk.c | 14 ++++++++++----
vswitchd/vswitch.xml | 15 +++++++++++++++
7 files changed, 75 insertions(+), 4 deletions(-)
diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst
index a43affa..33d98a4 100644
--- a/Documentation/topics/dpdk/vhost-user.rst
+++ b/Documentation/topics/dpdk/vhost-user.rst
@@ -273,6 +273,34 @@ One benefit of using this mode is the ability for vHost ports to 'reconnect' in
event of the switch crashing or being brought down. Once it is brought back up,
the vHost ports will reconnect automatically and normal service will resume.
+vhost-user-client IOMMU Support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+vhost IOMMU is a feature which restricts the vhost memory that a virtio device
+can access, and as such is useful in deployments in which security is a
+concern.
+
+IOMMU support may be enabled via a global config value,
+```vhost-iommu-support```. Setting this to true enables vhost IOMMU support for
+all vhost ports when/where available::
+
+ $ ovs-vsctl set Open_vSwitch . other_config:vhost-iommu-support=true
+
+The default value is false.
+
+.. important::
+
+ Changing this value requires restarting the daemon.
+
+.. important::
+
+ Enabling the IOMMU feature also enables the vhost user reply-ack protocol;
+ this is known to work on QEMU v2.10.0, but is buggy on older versions
+ (2.7.0 - 2.9.0, inclusive). Consequently, the IOMMU feaure is disabled by
+ default (and should remain so if using the aforementioned versions of
+ QEMU). Starting with QEMU v2.9.1, vhost-iommu-support can safely be
+ enabled, even without having an IOMMU device, with no performance penalty.
+
.. _dpdk-testpmd:
DPDK in the Guest
diff --git a/NEWS b/NEWS
index d4a1c9a..99c47d8 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,7 @@ Post-v2.8.0
* Add support for compiling OVS with the latest Linux 4.13 kernel
- DPDK:
* Add support for DPDK v17.11
+ * Add support for vHost IOMMU
v2.8.0 - 31 Aug 2017
--------------------
diff --git a/lib/dpdk-stub.c b/lib/dpdk-stub.c
index daef729..3602180 100644
--- a/lib/dpdk-stub.c
+++ b/lib/dpdk-stub.c
@@ -48,3 +48,9 @@ dpdk_get_vhost_sock_dir(void)
{
return NULL;
}
+
+bool
+dpdk_vhost_iommu_enabled(void)
+{
+ return false;
+}
diff --git a/lib/dpdk.c b/lib/dpdk.c
index 8da6c32..6710d10 100644
--- a/lib/dpdk.c
+++ b/lib/dpdk.c
@@ -41,6 +41,7 @@ VLOG_DEFINE_THIS_MODULE(dpdk);
static FILE *log_stream = NULL; /* Stream for DPDK log redirection */
static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets */
+static bool vhost_iommu_enabled = false; /* Status of vHost IOMMU support */
static int
process_vhost_flags(char *flag, const char *default_val, int size,
@@ -345,6 +346,11 @@ dpdk_init__(const struct smap *ovs_other_config)
vhost_sock_dir = sock_dir_subcomponent;
}
+ vhost_iommu_enabled = smap_get_bool(ovs_other_config,
+ "vhost-iommu-support", false);
+ VLOG_INFO("IOMMU support for vhost-user-client %s.",
+ vhost_iommu_enabled ? "enabled" : "disabled");
+
argv = grow_argv(&argv, 0, 1);
argc = 1;
argv[0] = xstrdup(ovs_get_program_name());
@@ -482,6 +488,12 @@ dpdk_get_vhost_sock_dir(void)
return vhost_sock_dir;
}
+bool
+dpdk_vhost_iommu_enabled(void)
+{
+ return vhost_iommu_enabled;
+}
+
void
dpdk_set_lcore_id(unsigned cpu)
{
diff --git a/lib/dpdk.h b/lib/dpdk.h
index 673a1f1..dc58d96 100644
--- a/lib/dpdk.h
+++ b/lib/dpdk.h
@@ -17,6 +17,8 @@
#ifndef DPDK_H
#define DPDK_H
+#include
+
#ifdef DPDK_NETDEV
#include
@@ -35,5 +37,6 @@ struct smap;
void dpdk_init(const struct smap *ovs_other_config);
void dpdk_set_lcore_id(unsigned cpu);
const char *dpdk_get_vhost_sock_dir(void);
+bool dpdk_vhost_iommu_enabled(void);
#endif /* dpdk.h */
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index f552444..9715c39 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -3253,6 +3253,7 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
{
struct netdev_dpdk *dev = netdev_dpdk_cast(netdev);
int err;
+ uint64_t vhost_flags = 0;
ovs_mutex_lock(&dev->mutex);
@@ -3263,16 +3264,21 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev)
*/
if (!(dev->vhost_driver_flags & RTE_VHOST_USER_CLIENT)
&& strlen(dev->vhost_id)) {
- /* Register client-mode device */
- err = rte_vhost_driver_register(dev->vhost_id,
- RTE_VHOST_USER_CLIENT);
+ /* Register client-mode device. */
+ vhost_flags |= RTE_VHOST_USER_CLIENT;
+
+ /* Enable IOMMU support, if explicitly requested. */
+ if (dpdk_vhost_iommu_enabled()) {
+ vhost_flags |= RTE_VHOST_USER_IOMMU_SUPPORT;
+ }
+ err = rte_vhost_driver_register(dev->vhost_id, vhost_flags);
if (err) {
VLOG_ERR("vhost-user device setup failure for device %s\n",
dev->vhost_id);
goto unlock;
} else {
/* Configuration successful */
- dev->vhost_driver_flags |= RTE_VHOST_USER_CLIENT;
+ dev->vhost_driver_flags |= vhost_flags;
VLOG_INFO("vHost User device '%s' created in 'client' mode, "
"using client socket '%s'",
dev->up.name, dev->vhost_id);
diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml
index c145e1a..4c317d0 100644
--- a/vswitchd/vswitch.xml
+++ b/vswitchd/vswitch.xml
@@ -344,6 +344,21 @@
+
+
+ vHost IOMMU is a security feature, which restricts the vhost memory
+ that a virtio device may access. vHost IOMMU support is disabled by
+ default, due to a bug in QEMU implementations of the vhost REPLY_ACK
+ protocol, (on which vHost IOMMU relies) prior to v2.9.1. Setting this
+ value to true enables vHost IOMMU support for vHost User
+ Client ports in OvS-DPDK, starting from DPDK v17.11.
+
+
+ Changing this value requires restarting the daemon.
+