From patchwork Thu Dec 7 17:45:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Kavanagh X-Patchwork-Id: 845726 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=openvswitch.org (client-ip=140.211.169.12; helo=mail.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3yt2wn4wT2z9s7B for ; Fri, 8 Dec 2017 04:46:49 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 5C629D25; Thu, 7 Dec 2017 17:45:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 37859D1F for ; Thu, 7 Dec 2017 17:45:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 83DD5F6 for ; Thu, 7 Dec 2017 17:45:51 +0000 (UTC) Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Dec 2017 09:45:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,373,1508828400"; d="scan'208";a="893153" Received: from silpixa00380299.ir.intel.com ([10.237.222.17]) by orsmga007.jf.intel.com with ESMTP; 07 Dec 2017 09:45:48 -0800 From: Mark Kavanagh To: dev@openvswitch.org Date: Thu, 7 Dec 2017 17:45:37 +0000 Message-Id: <1512668737-15021-3-git-send-email-mark.b.kavanagh@intel.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1512668737-15021-1-git-send-email-mark.b.kavanagh@intel.com> References: <1512668737-15021-1-git-send-email-mark.b.kavanagh@intel.com> X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: tiwei.bie@intel.com, maxime.coquelin@redhat.com, i.maximets@samsung.com Subject: [ovs-dev] [PATCH V5 2/2] netdev-dpdk: vHost IOMMU support X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org DPDK v17.11 introduces support for the vHost IOMMU feature. This is a security feature, which restricts the vhost memory that a virtio device may access. This feature also enables the vhost REPLY_ACK protocol, the implementation of which is known to work in newer versions of QEMU (i.e. v2.10.0), but is buggy in older versions (v2.7.0 - v2.9.0, inclusive). As such, the feature is disabled by default in (and should remain so), for the aforementioned older QEMU verions. Starting with QEMU v2.9.1, vhost-iommu-support can safely be enabled, even without having an IOMMU device, with no performance penalty. This patch adds a new global config option, vhost-iommu-support, that controls enablement of the vhost IOMMU feature: ovs-vsctl set Open_vSwitch . other_config:vhost-iommu-support=true This value defaults to false; to enable IOMMU support, this field should be set to true when setting other global parameters on init (such as "dpdk-socket-mem", for example). Changing the value at runtime is not supported, and requires restarting the vswitch daemon. Signed-off-by: Mark Kavanagh --- Documentation/topics/dpdk/vhost-user.rst | 28 ++++++++++++++++++++++++++++ NEWS | 1 + lib/dpdk-stub.c | 6 ++++++ lib/dpdk.c | 12 ++++++++++++ lib/dpdk.h | 3 +++ lib/netdev-dpdk.c | 14 ++++++++++---- vswitchd/vswitch.xml | 15 +++++++++++++++ 7 files changed, 75 insertions(+), 4 deletions(-) diff --git a/Documentation/topics/dpdk/vhost-user.rst b/Documentation/topics/dpdk/vhost-user.rst index 1a5de24..8b1c671 100644 --- a/Documentation/topics/dpdk/vhost-user.rst +++ b/Documentation/topics/dpdk/vhost-user.rst @@ -273,6 +273,34 @@ One benefit of using this mode is the ability for vHost ports to 'reconnect' in event of the switch crashing or being brought down. Once it is brought back up, the vHost ports will reconnect automatically and normal service will resume. +vhost-user-client IOMMU Support +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +vhost IOMMU is a feature which restricts the vhost memory that a virtio device +can access, and as such is useful in deployments in which security is a +concern. + +IOMMU support may be enabled via a global config value, +```vhost-iommu-support```. Setting this to true enables vhost IOMMU support for +all vhost ports when/where available:: + + $ ovs-vsctl set Open_vSwitch . other_config:vhost-iommu-support=true + +The default value is false. + +.. important:: + + Changing this value requires restarting the daemon. + +.. important:: + + Enabling the IOMMU feature also enables the vhost user reply-ack protocol; + this is known to work on QEMU v2.10.0, but is buggy on older versions + (2.7.0 - 2.9.0, inclusive). Consequently, the IOMMU feaure is disabled by + default (and should remain so if using the aforementioned versions of + QEMU). Starting with QEMU v2.9.1, vhost-iommu-support can safely be + enabled, even without having an IOMMU device, with no performance penalty. + .. _dpdk-testpmd: DPDK in the Guest diff --git a/NEWS b/NEWS index d4a1c9a..99c47d8 100644 --- a/NEWS +++ b/NEWS @@ -15,6 +15,7 @@ Post-v2.8.0 * Add support for compiling OVS with the latest Linux 4.13 kernel - DPDK: * Add support for DPDK v17.11 + * Add support for vHost IOMMU v2.8.0 - 31 Aug 2017 -------------------- diff --git a/lib/dpdk-stub.c b/lib/dpdk-stub.c index daef729..3602180 100644 --- a/lib/dpdk-stub.c +++ b/lib/dpdk-stub.c @@ -48,3 +48,9 @@ dpdk_get_vhost_sock_dir(void) { return NULL; } + +bool +dpdk_vhost_iommu_enabled(void) +{ + return false; +} diff --git a/lib/dpdk.c b/lib/dpdk.c index 8da6c32..6710d10 100644 --- a/lib/dpdk.c +++ b/lib/dpdk.c @@ -41,6 +41,7 @@ VLOG_DEFINE_THIS_MODULE(dpdk); static FILE *log_stream = NULL; /* Stream for DPDK log redirection */ static char *vhost_sock_dir = NULL; /* Location of vhost-user sockets */ +static bool vhost_iommu_enabled = false; /* Status of vHost IOMMU support */ static int process_vhost_flags(char *flag, const char *default_val, int size, @@ -345,6 +346,11 @@ dpdk_init__(const struct smap *ovs_other_config) vhost_sock_dir = sock_dir_subcomponent; } + vhost_iommu_enabled = smap_get_bool(ovs_other_config, + "vhost-iommu-support", false); + VLOG_INFO("IOMMU support for vhost-user-client %s.", + vhost_iommu_enabled ? "enabled" : "disabled"); + argv = grow_argv(&argv, 0, 1); argc = 1; argv[0] = xstrdup(ovs_get_program_name()); @@ -482,6 +488,12 @@ dpdk_get_vhost_sock_dir(void) return vhost_sock_dir; } +bool +dpdk_vhost_iommu_enabled(void) +{ + return vhost_iommu_enabled; +} + void dpdk_set_lcore_id(unsigned cpu) { diff --git a/lib/dpdk.h b/lib/dpdk.h index 673a1f1..dc58d96 100644 --- a/lib/dpdk.h +++ b/lib/dpdk.h @@ -17,6 +17,8 @@ #ifndef DPDK_H #define DPDK_H +#include + #ifdef DPDK_NETDEV #include @@ -35,5 +37,6 @@ struct smap; void dpdk_init(const struct smap *ovs_other_config); void dpdk_set_lcore_id(unsigned cpu); const char *dpdk_get_vhost_sock_dir(void); +bool dpdk_vhost_iommu_enabled(void); #endif /* dpdk.h */ diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index f552444..9715c39 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -3253,6 +3253,7 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) { struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); int err; + uint64_t vhost_flags = 0; ovs_mutex_lock(&dev->mutex); @@ -3263,16 +3264,21 @@ netdev_dpdk_vhost_client_reconfigure(struct netdev *netdev) */ if (!(dev->vhost_driver_flags & RTE_VHOST_USER_CLIENT) && strlen(dev->vhost_id)) { - /* Register client-mode device */ - err = rte_vhost_driver_register(dev->vhost_id, - RTE_VHOST_USER_CLIENT); + /* Register client-mode device. */ + vhost_flags |= RTE_VHOST_USER_CLIENT; + + /* Enable IOMMU support, if explicitly requested. */ + if (dpdk_vhost_iommu_enabled()) { + vhost_flags |= RTE_VHOST_USER_IOMMU_SUPPORT; + } + err = rte_vhost_driver_register(dev->vhost_id, vhost_flags); if (err) { VLOG_ERR("vhost-user device setup failure for device %s\n", dev->vhost_id); goto unlock; } else { /* Configuration successful */ - dev->vhost_driver_flags |= RTE_VHOST_USER_CLIENT; + dev->vhost_driver_flags |= vhost_flags; VLOG_INFO("vHost User device '%s' created in 'client' mode, " "using client socket '%s'", dev->up.name, dev->vhost_id); diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index c145e1a..4c317d0 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -344,6 +344,21 @@

+ +

+ vHost IOMMU is a security feature, which restricts the vhost memory + that a virtio device may access. vHost IOMMU support is disabled by + default, due to a bug in QEMU implementations of the vhost REPLY_ACK + protocol, (on which vHost IOMMU relies) prior to v2.9.1. Setting this + value to true enables vHost IOMMU support for vHost User + Client ports in OvS-DPDK, starting from DPDK v17.11. +

+

+ Changing this value requires restarting the daemon. +

+ +