From patchwork Sat Dec 17 22:24:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 706733 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3th1y21MlZz9vFF for ; Sun, 18 Dec 2016 09:27:06 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b="h5QC1oqi"; dkim-atps=neutral Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id C0956B2B; Sat, 17 Dec 2016 22:24:29 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D87C595D for ; Sat, 17 Dec 2016 22:24:28 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from caracal.maple.relay.mailchannels.net (caracal.maple.relay.mailchannels.net [23.83.214.30]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id CEE4914B for ; Sat, 17 Dec 2016 22:24:27 +0000 (UTC) X-Sender-Id: mxroute|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 4C533121084 for ; Sat, 17 Dec 2016 22:24:26 +0000 (UTC) Received: from one.mxroute.com (ip-10-220-3-24.us-west-2.compute.internal [10.220.3.24]) by relay.mailchannels.net (Postfix) with ESMTPA id C43A21202E4 for ; Sat, 17 Dec 2016 22:24:25 +0000 (UTC) X-Sender-Id: mxroute|x-authuser|stephen@that.guru Received: from one.mxroute.com ([UNAVAILABLE]. [10.34.134.237]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.7.8); Sat, 17 Dec 2016 22:24:26 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: mxroute|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: mxroute X-MC-Loop-Signature: 1482013466127:3882368388 X-MC-Ingress-Time: 1482013466126 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=DgS19xR3AGC45JapNFN93szHrzM2U1FJTGpAhemneAw=; b=h5QC1oqi8lXkbc2xRZUYRvMDvQ 52vDbuh7my4U/dbsyeqpuJop3iUc7neNWG4btPFPJ0lxCAx1w6ZN+ZxJk+0MLW6xhbDmgGlT4Ljv5 LBdmHmYQVpca7pxvPxsjRDx1LjiVMAYAw3Xcg86y1hyf49MX5Xk5ZP6/5F/pMcN05dAsOZyAKUaGU 1cg9JcfLJBRfbTMe8JbqHlM5KZ5RditNeYkjZRkPjcOJQDFB8BxLUKKUp3GYtxtBGHGfsazdbam6k TGEgGGcpb6JVf0+8G/eP4z8LKUqxfzQS4TQiBEbjGkHMSAwkiovuBiBVXGTKlSHWUw5g92YDgHKal 7baD0ZQA==; From: Stephen Finucane To: dev@openvswitch.org Date: Sat, 17 Dec 2016 22:24:08 +0000 Message-Id: <20161217222408.22059-5-stephen@that.guru> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161217222408.22059-1-stephen@that.guru> References: <20161217222408.22059-1-stephen@that.guru> X-AuthUser: stephen@that.guru X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_NONE,T_DKIM_INVALID autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH 5/5] doc: Add DPDK to userspace tunneling guide X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org Instructions were provided in the userspace tunneling cookbook. Signed-off-by: Stephen Finucane --- Documentation/automake.mk | 2 +- Documentation/howto/index.rst | 2 +- Documentation/howto/native-tunneling.rst | 176 -------------------- Documentation/howto/userspace-tunneling.rst | 247 ++++++++++++++++++++++++++++ 4 files changed, 249 insertions(+), 178 deletions(-) delete mode 100644 Documentation/howto/native-tunneling.rst create mode 100644 Documentation/howto/userspace-tunneling.rst diff --git a/Documentation/automake.mk b/Documentation/automake.mk index 5ea74f0..02b5ff7 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -38,7 +38,6 @@ EXTRA_DIST += \ Documentation/howto/libvirt.rst \ Documentation/howto/selinux.rst \ Documentation/howto/ssl.rst \ - Documentation/howto/native-tunneling.rst \ Documentation/howto/lisp.rst \ Documentation/howto/openstack-containers.rst \ Documentation/howto/qos.png \ @@ -47,6 +46,7 @@ EXTRA_DIST += \ Documentation/howto/sflow.rst \ Documentation/howto/tunneling.png \ Documentation/howto/tunneling.rst \ + Documentation/howto/userspace-tunneling.rst \ Documentation/howto/vlan.png \ Documentation/howto/vlan.rst \ Documentation/howto/vtep.rst \ diff --git a/Documentation/howto/index.rst b/Documentation/howto/index.rst index 5933911..3bf2af4 100644 --- a/Documentation/howto/index.rst +++ b/Documentation/howto/index.rst @@ -39,7 +39,7 @@ topics covered herein, refer to :doc:`/topics/index`. ssl lisp tunneling - native-tunneling + userspace-tunneling vlan qos vtep diff --git a/Documentation/howto/native-tunneling.rst b/Documentation/howto/native-tunneling.rst deleted file mode 100644 index 78eee5b..0000000 --- a/Documentation/howto/native-tunneling.rst +++ /dev/null @@ -1,176 +0,0 @@ -.. - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - Convention for heading levels in Open vSwitch documentation: - - ======= Heading 0 (reserved for the title in a document) - ------- Heading 1 - ~~~~~~~ Heading 2 - +++++++ Heading 3 - ''''''' Heading 4 - - Avoid deeper levels because they do not render well. - -========================================== -Native Tunneling in Open vSwitch Userspace -========================================== - -Open vSwitch supports tunneling in userspace. Tunneling is implemented in -a platform-independent way. - -Setup ------ - -Setup physical bridges for all physical interfaces. Create integration bridge. -Add VXLAN port to int-bridge. Assign IP address to physical bridge where -VXLAN traffic is expected. - -Example -------- - -Connect to VXLAN tunnel endpoint logical IP: ``192.168.1.2`` and -``192.168.1.1``. - -Configure OVS bridges as follows. - -1. Let's assume ``172.168.1.2/24`` network is reachable via ``eth1``. Create - physical bridge ``br-eth1``. Assign IP address (``172.168.1.1/24``) to - ``br-eth1``. Add ``eth1`` to ``br-eth1``. - -2. Check ovs cached routes using appctl command. - - :: - - $ ovs-appctl ovs/route/show - - Add tunnel route if not present in OVS route table. - - :: - - $ ovs-appctl ovs/route/add 172.168.1.1/24 br-eth1 - -3. Add integration bridge ``int-br`` and add tunnel port using standard syntax. - - :: - - $ ovs-vsctl add-port int-br vxlan0 \ - -- set interface vxlan0 type=vxlan options:remote_ip=172.168.1.2 - -4. Assign IP address to ``int-br``. - -The final topology should looks like so: - -:: - - Diagram - - 192.168.1.1/24 - +--------------+ - | int-br | 192.168.1.2/24 - +--------------+ +--------------+ - | vxlan0 | | vxlan0 | - +--------------+ +--------------+ - | | - | | - | | - 172.168.1.1/24 | - +--------------+ | - | br-eth1 | 172.168.1.2/24 - +--------------+ +---------------+ - | eth1 |----------------------------------| eth1 | - +--------------+ +---------------+ - - Host A with OVS. Remote host. - -With this setup, ping to VXLAN target device (``192.168.1.2``) should work. - -Tunneling-related Commands --------------------------- - -Tunnel routing table -~~~~~~~~~~~~~~~~~~~~ - -To add route: - -:: - - $ ovs-appctl ovs/route/add / - -To see all routes configured: - -:: - - $ ovs-appctl ovs/route/show - -To delete route: - -:: - - $ ovs-appctl ovs/route/del / - -To look up and display the route for a destination: - -:: - - $ ovs-appctl ovs/route/lookup - -ARP -~~~ - -To see arp cache content: - -:: - - $ ovs-appctl tnl/arp/show - -To flush arp cache: - -:: - - $ ovs-appctl tnl/arp/flush - -Ports -~~~~~ - -To check tunnel ports listening in ovs-vswitchd: - -:: - - $ ovs-appctl tnl/ports/show - -To set range for VxLan UDP source port: - -:: - - $ ovs-appctl tnl/egress_port_range - -To show current range: - -:: - - $ ovs-appctl tnl/egress_port_range - -Datapath -~~~~~~~~ - -To check datapath ports: - -:: - - $ ovs-appctl dpif/show - -To check datapath flows: - -:: - - $ ovs-appctl dpif/dump-flows diff --git a/Documentation/howto/userspace-tunneling.rst b/Documentation/howto/userspace-tunneling.rst new file mode 100644 index 0000000..6f73c4c --- /dev/null +++ b/Documentation/howto/userspace-tunneling.rst @@ -0,0 +1,247 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +======================================== +Connecting VMs Using Tunnels (Userspace) +======================================== + +This document describes how to use Open vSwitch to allow VMs on two different +hosts to communicate over VXLAN tunnels. Unlike :doc:`tunneling`, this +configuration works entirely in userspace. + +.. note:: + + This guide covers the steps required to configure VXLAN tunneling. The same + approach can be used for any of the other tunneling protocols supported by + Open vSwitch. + +.. TODO(stephenfin): Convert this to a (prettier) PNG with same styling as the + rest of the document + +:: + + +--------------+ + | vm0 | 192.168.1.1/24 + +--------------+ + (vm_port0) + | + | + | + +--------------+ + | br-int | 192.168.1.2/24 + +--------------+ +--------------+ + | vxlan0 | | vxlan0 | + +--------------+ +--------------+ + | | + | | + | | + 172.168.1.1/24 | + +--------------+ | + | br-phy | 172.168.1.2/24 + +--------------+ +---------------+ + | dpdk0/eth1 |----------------------------------| eth1 | + +--------------+ +---------------+ + Host A with OVS. Remote host. + +Setup +----- + +This guide assumes the environment is configured as described below. + +Two Physical Hosts +~~~~~~~~~~~~~~~~~~ + +The environment assumes the use of two hosts, named `host1` and `host2`. We +only detail the configuration of `host1` but a similar configuration can be +used for `host2`. Both hosts should be configured with Open vSwitch (with or +without the DPDK datapath), QEMU/KVM and suitable VM images. Open vSwitch +should be running before proceeding. + +Configuration Steps +------------------- + +Perform the folowing configuration on `host1`: + +#. Create a ``br-int`` bridge:: + + $ ovs-vsctl --may-exist add-br br-int \ + -- set Bridge br-int datapath_type=netdev \ + -- br-set-external-id br-int bridge-id br-int \ + -- set bridge br-int fail-mode=standalone + +#. Add a port to this bridge. If using tap ports, first boot a VM and then add + the port to the bridge:: + + $ ovs-vsctl add-port br0 tap0 + + If using DPDK vhost-user ports, add the port and then boot the VM + accordingly, using ``vm_port0`` as the interface name:: + + $ ovs-vsctl add-port br-int vm_port0 \ + -- set Interface vm_port0 type=dpdkvhostuser + +#. Configure the IP address of the VM interface *in the VM itself*:: + + $ ip addr add 192.168.1.1/24 dev eth0 + +#. On `host1`, add a port for the VXLAN tunnel:: + + $ ovs-vsctl add-port br-int vxlan0 \ + -- set interface vxlan0 type=vxlan options:remote_ip=172.168.1.2 + + .. note:: + + ``172.168.1.2`` is the remote tunnel end point address. On the remote + host this will be ``172.168.1.1`` + +#. Create a ``br-phy`` bridge:: + + $ ovs-vsctl --may-exist add-br br-phy \ + -- set Bridge br-phy datapath_type=netdev \ + -- br-set-external-id br-phy bridge-id br-phy \ + -- set bridge br-phy fail-mode=standalone \ + other_config:hwaddr= + + .. note:: + + This additional bridge is required when running Open vSwitch in userspace + rather than kernel-based Open vSwitch. The purpose of this bridge is to + allow use of the kernel network stack for routing and ARP resolution. + The datapath needs to look-up the routing table and ARP table to prepare + the tunnel header and transmit data to the output port. + + .. note:: + + ``eth1`` is used rather than ``eth0``. This is to ensure network + connectivity is retained. + +#. Attach ``eth1``/``dpdk0`` to the ``br-phy`` bridge. + + If the physical port ``eth1`` is operating as a kernel network interface, + run:: + + $ ovs-vsctl --timeout 10 add-port br-phy eth1 + $ ip addr add 172.168.1.1/24 dev br-phy + $ ip link set br-phy up + $ ip addr flush dev eth1 2>/dev/null + $ ip link set eth1 up + $ iptables -F + + If instead the interface is a DPDK interface and bound to the ``igb_uio`` or + ``vfio`` driver, run:: + + $ ovs-vsctl --timeout 10 add-port br-phy dpdk0 \ + -- set Interface dpdk0 type=dpdk + $ ip addr add 172.168.1.1/24 dev br-phy + $ ip link set br-phy up + $ iptables -F + + The commands are different as DPDK interfaces are not managed by the kernel, + thus, the port details are not visible to any ``ip`` commands. + + .. important:: + + Attempting to use the kernel network commands for a DPDK interface will + result in a loss of connectivity through ``eth1``. Refer to + :doc:`/faq/configuration` for more details. + +Once complete, check the cached routes using ovs-appctl command:: + + $ ovs-appctl ovs/route/show + +If the tunnel route is missing, adding it now:: + + $ ovs-appctl ovs/route/add 172.168.1.1/24 br-eth1 + +Repeat these steps if necessary for `host2`, but using ``192.168.1.1`` and +``172.168.1.2`` for the VM and tunnel interface IP addresses, respectively. + +Testing +------- + +With this setup, ping to VXLAN target device (``192.168.1.2``) should work. +Traffic will be VXLAN encapsulated and sent over the ``eth1``/``dpdk0`` +interface. + +Tunneling-related Commands +-------------------------- + +Tunnel routing table +~~~~~~~~~~~~~~~~~~~~ + +To add route:: + + $ ovs-appctl ovs/route/add / + +To see all routes configured:: + + $ ovs-appctl ovs/route/show + +To delete route:: + + $ ovs-appctl ovs/route/del / + +To look up and display the route for a destination:: + + $ ovs-appctl ovs/route/lookup + +ARP +~~~ + +To see arp cache content:: + + $ ovs-appctl tnl/arp/show + +To flush arp cache:: + + $ ovs-appctl tnl/arp/flush + +To set a specific arp entry:: + + $ ovs-appctl tnl/arp/set + +Ports +~~~~~ + +To check tunnel ports listening in ovs-vswitchd:: + + $ ovs-appctl tnl/ports/show + +To set range for VxLan UDP source port:: + + $ ovs-appctl tnl/egress_port_range + +To show current range:: + + $ ovs-appctl tnl/egress_port_range + +Datapath +~~~~~~~~ + +To check datapath ports:: + + $ ovs-appctl dpif/show + +To check datapath flows:: + + $ ovs-appctl dpif/dump-flows