From patchwork Sat Oct 1 19:01:31 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stephen Finucane X-Patchwork-Id: 677411 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from archives.nicira.com (archives.nicira.com [96.126.127.54]) by ozlabs.org (Postfix) with ESMTP id 3smd385Vylz9s3s for ; Sun, 2 Oct 2016 06:02:12 +1100 (AEDT) Authentication-Results: ozlabs.org; dkim=fail reason="key not found in DNS" (0-bit key; unprotected) header.d=that.guru header.i=@that.guru header.b=TOzJWHAN; dkim-atps=neutral Received: from archives.nicira.com (localhost [127.0.0.1]) by archives.nicira.com (Postfix) with ESMTP id 3719B108BD; Sat, 1 Oct 2016 12:01:59 -0700 (PDT) X-Original-To: dev@openvswitch.org Delivered-To: dev@openvswitch.org Received: from mx3v3.cudamail.com (mx3.cudamail.com [64.34.241.5]) by archives.nicira.com (Postfix) with ESMTPS id 315A9108BC for ; Sat, 1 Oct 2016 12:01:57 -0700 (PDT) Received: from bar6.cudamail.com (localhost [127.0.0.1]) by mx3v3.cudamail.com (Postfix) with ESMTPS id BE045162964 for ; Sat, 1 Oct 2016 13:01:56 -0600 (MDT) X-ASG-Debug-ID: 1475348509-0b323746cf125730001-byXFYA Received: from mx1-pf2.cudamail.com ([192.168.24.2]) by bar6.cudamail.com with ESMTP id DUSLFkZjqGaZcLsE (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 01 Oct 2016 13:01:49 -0600 (MDT) X-Barracuda-Envelope-From: stephen@that.guru X-Barracuda-RBL-Trusted-Forwarder: 192.168.24.2 Received: from unknown (HELO buffalo.birch.relay.mailchannels.net) (23.83.209.24) by mx1-pf2.cudamail.com with ESMTPS (DHE-RSA-AES256-SHA encrypted); 1 Oct 2016 19:01:48 -0000 Received-SPF: none (mx1-pf2.cudamail.com: domain at that.guru does not designate permitted sender hosts) X-Barracuda-Apparent-Source-IP: 23.83.209.24 X-Barracuda-RBL-IP: 23.83.209.24 X-Sender-Id: mxroute|x-authuser|stephen@that.guru Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 6B369E0426 for ; Sat, 1 Oct 2016 19:01:47 +0000 (UTC) Received: from one.mxroute.com (ip-10-229-10-199.us-west-2.compute.internal [10.229.10.199]) by relay.mailchannels.net (Postfix) with ESMTPA id 67E82E011A for ; Sat, 1 Oct 2016 19:01:46 +0000 (UTC) X-Sender-Id: mxroute|x-authuser|stephen@that.guru Received: from one.mxroute.com ([UNAVAILABLE]. [10.16.27.41]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.7.8); Sat, 01 Oct 2016 19:01:47 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: mxroute|x-authuser|stephen@that.guru X-MailChannels-Auth-Id: mxroute X-MC-Loop-Signature: 1475348507008:487525889 X-MC-Ingress-Time: 1475348507007 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=that.guru; s=default; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=BEhsfkgebhFobLW2t9GrS+lu9hd/GsDIv9kVNELx5vI=; b=TOzJWHANiArz0v+hQrUGYv1PBp beiR/i6HCRixDbS6E4Elt04a3k7JOq8OL/3loyJDpB8y9WLJDi4XAvq6mtuh5KUWh0zhGEjiCIGwT tlwY2TyQHvTRlViblF5VmV/eVxm8SOT4ubYM2vnQWkCEdgSr9xlic0CqTSzMvhG8MNdH6qeyK16OS WdLw/oxXbOQ2WDdIZRTEgy3bihitc1XjaKVf2v0cSjloQe3xgZBqCEqqPBkxNi8K075DqU5AtHNt9 vbYk2IpypqvJHNvJAGOODo4jqsSaCwayPhM2SfjrQcwZsEqFpC6h13S86H5i11s7sJWtJeT29j/Cn sEJfxHcA==; X-CudaMail-Envelope-Sender: stephen@that.guru From: Stephen Finucane To: dev@openvswitch.org X-CudaMail-MID: CM-E2-930019690 X-CudaMail-DTE: 100116 X-CudaMail-Originating-IP: 23.83.209.24 Date: Sat, 1 Oct 2016 20:01:31 +0100 X-ASG-Orig-Subj: [##CM-E2-930019690##][RFC 2/5] doc: Add a 'install-guide' section Message-Id: <1475348494-23944-3-git-send-email-stephen@that.guru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1475348494-23944-1-git-send-email-stephen@that.guru> References: <1475348494-23944-1-git-send-email-stephen@that.guru> X-AuthUser: stephen@that.guru X-GBUdb-Analysis: 0, 23.83.209.24, Ugly c=0.308711 p=-0.3 Source Normal X-MessageSniffer-Rules: 0-0-0-32767-c X-Barracuda-Connect: UNKNOWN[192.168.24.2] X-Barracuda-Start-Time: 1475348509 X-Barracuda-Encrypted: DHE-RSA-AES256-SHA X-Barracuda-URL: https://web.cudamail.com:443/cgi-mod/mark.cgi X-Barracuda-BRTS-Status: 1 X-Virus-Scanned: by bsmtpd at cudamail.com X-Barracuda-Spam-Score: 1.10 X-Barracuda-Spam-Status: No, SCORE=1.10 using global scores of TAG_LEVEL=3.5 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=4.0 tests=BSF_SC0_MV0713, BSF_SC5_MJ1963, DKIM_SIGNED, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.33386 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC0_MV0713 Custom rule MV0713 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Subject: [ovs-dev] [RFC 2/5] doc: Add a 'install-guide' section X-BeenThere: dev@openvswitch.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: dev-bounces@openvswitch.org Sender: "dev" Add reStructuredText versions of three installation guides: the general installation guide, the basic DPDK guide and the Docker guide. To achive this, the docs were first passed through Pandoc. For example: $ pandoc --from=markdown --to=rst INSTALL.XenServer.md \ --output doc/source/install/xenserver.rst Once done, the documents were manually inspected to fix up the (many) broken links to other INSTALL docs, correct the mistakes from the conversion (damn GitHub-style Markdown) and generally prettify the docs. Finally, the docs were listed on the index page, i.e. the root of the documentation. Signed-off-by: Stephen Finucane --- Documentation/index.rst | 8 +- Documentation/install-guide/dpdk.rst | 594 ++++++++++++++++++++++ Documentation/install-guide/general.rst | 870 ++++++++++++++++++++++++++++++++ Documentation/install-guide/index.rst | 34 ++ 4 files changed, 1503 insertions(+), 3 deletions(-) create mode 100644 Documentation/install-guide/dpdk.rst create mode 100644 Documentation/install-guide/general.rst create mode 100644 Documentation/install-guide/index.rst diff --git a/Documentation/index.rst b/Documentation/index.rst index 61a3b4c..e6b9c44 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -25,10 +25,12 @@ Welcome to Open vSwitch's documentation ======================================= -Contents: +.. include:: install-guide/index.rst -.. toctree:: - :maxdepth: 2 +Bugs +---- + +Report problems to bugs@openvswitch.org. Indices and tables ------------------ diff --git a/Documentation/install-guide/dpdk.rst b/Documentation/install-guide/dpdk.rst new file mode 100644 index 0000000..50e2872 --- /dev/null +++ b/Documentation/install-guide/dpdk.rst @@ -0,0 +1,594 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +====================== +Open vSwitch with DPDK +====================== + +This document describes how to build and install Open vSwitch using a DPDK +datapath. Open vSwitch can use the DPDK library to operate entirely in +userspace. + +.. warning:: + The DPDK support of Open vSwitch is considered 'experimental'. + +Build requirements +------------------ + +In addition to the requirements described in :ref:`general-build-reqs`, +building Open vSwitch with DPDK will require the following: + +- DPDK 16.07 + +- A `DPDK supported NIC`_ + + Only required when physical ports are in use + +- A suitable kernel + + On Linux Distros running kernel version >= 3.0, only `IOMMU` needs to enabled + via the grub cmdline, assuming you are using :ref:`vfio `. For + older kernels, ensure the kernel is built with ``UIO``, ``HUGETLBFS``, + ``PROC_PAGE_MONITOR``, ``HPET``, ``HPET_MMAP`` support. If these are not + present, it will be necessary to upgrade your kernel or build a custom kernel + with these flags enabled. + +Detailed system requirements can be found at `DPDK requirements`_, while more +detailed install information can be found in the `advanced installation guide +<../../INSTALL.DPDK-advanced.md>`__. + +.. _DPDK supported NIC: http://dpdk.org/doc/nics +.. _DPDK requirements: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html + +Installing +---------- + +DPDK +~~~~ + +1. Download the `DPDK sources`_, extract the file and set ``DPDK_DIR``:: + + $ cd /usr/src/ + $ wget http://dpdk.org/browse/dpdk/snapshot/dpdk-16.07.zip + $ unzip dpdk-16.07.zip + $ export DPDK_DIR=/usr/src/dpdk-16.07 + $ cd $DPDK_DIR + +2. Configure and install DPDK + + Build and install the DPDK library:: + + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ make install T=$DPDK_TARGET DESTDIR=install + + If IVSHMEM support is required, use a different target:: + + $ export DPDK_TARGET=x86_64-ivshmem-linuxapp-gcc + +.. _DPDK sources: http://dpdk.org/browse/dpdk/refs/ + +Install OVS +~~~~~~~~~~~ + +OVS can be installed using different methods. For OVS to use DPDK datapath, it +has to be configured with DPDK support (``--with-dpdk``). + +.. note:: + This section focuses on generic recipe that suits most cases. For + distribution specific instructions, refer to one of the more relevant guides. + +.. _OVS sources: http://openvswitch.org/releases/ + +1. Ensure the standard OVS requirements, described in :ref:`general-build-reqs` + and :ref:`general-install-reqs`, are installed. + +2. Bootstrap, if required, as described in :ref:`general-bootstrapping`. + +3. Configure the package using the ``--with-dpdk`` flag. + + $ ./configure --with-dpdk=$DPDK_BUILD + + where ``DPDK_BUILD`` is the path to the built DPDK library. This can be + skipped if DPDK library is installed in its default location. + + .. note:: + While ``--with-dpdk`` is required, you can pass any other configuration + option described in :ref:`general-configuring`. + +4. Build and install OVS, as described in :ref:`general-building`. + +Additional information can be found in :doc:`general`. + +Setup +----- + +Setup Hugepages +~~~~~~~~~~~~~~~ + +Allocate a number of 2M Huge pages: + +- For persistent allocation of huge pages, write to hugepages.conf file + in `/etc/sysctl.d`:: + + $ echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf + +- For run-time allocation of huge pages, use the ``sysctl`` utility:: + + $ sysctl -w vm.nr_hugepages=N # where N = No. of 2M huge pages + +To verify hugepage configuration:: + + $ grep HugePages_ /proc/meminfo + +Mount the hugepages, if not already mounted by default:: + + $ mount -t hugetlbfs none /dev/hugepages`` + +.. _dpdk-vfio: + +Setup DPDK devices using VFIO +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +VFIO is prefered to the UIO driver when using recent versions of DPDK. VFIO +support required support from both the kernel and BIOS. For the former, kernel +version > 3.6 must be used. For the latter, you must enable VT-d in the BIOS +and ensure this is configured via grub. To ensure VT-d is enabled via the BIOS, +run:: + + $ dmesg | grep -e DMAR -e IOMMU + +If VT-d is not enabled in the BIOS, enable it now. + +To ensure VT-d is enabled in the kernel, run:: + + $ cat /proc/cmdline | grep iommu=pt + $ cat /proc/cmdline | grep intel_iommu=on + +If VT-d is not enabled in the kernel, enable it now. + +Once VT-d is correctly configured, load the required modules and bind the NIC +to the VFIO driver:: + + $ modprobe vfio-pci + $ /usr/bin/chmod a+x /dev/vfio + $ /usr/bin/chmod 0666 /dev/vfio/* + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1 + $ $DPDK_DIR/tools/dpdk-devbind.py --status + +Setup OVS +~~~~~~~~~ + +Open vSwitch should be started as described in :ref:`general-starting` with the +exception of ovs-vswitchd, which requires some special configuration to enable +DPDK functionality. DPDK configuration arguments can be passed to ovs-vswitchd +via the ``other_config`` column of the ``Open_vSwitch`` table. At a minimum, +the ``dpdk-init`` option must be set to ``true``. For example:: + + $ export DB_SOCK=/usr/local/var/run/openvswitch/db.sock + $ ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true + $ ovs-vswitchd unix:$DB_SOCK --pidfile --detach + +There are many other configuration options, the most important of which are +listed below. Defaults will be provided for all values not explicitly set. + +``dpdk-init`` + Specifies whether OVS should initialize and support DPDK ports. This is a + boolean, and defaults to false. + +``dpdk-lcore-mask`` + Specifies the CPU cores on which dpdk lcore threads should be spawned and + expects hex string (eg '0x123'). + +``dpdk-socket-mem`` + Comma separated list of memory to pre-allocate from hugepages on specific + sockets. + +``dpdk-hugepage-dir`` + Directory where hugetlbfs is mounted + +``vhost-sock-dir`` + Option to set the path to the vhost-user unix socket files. + +If allocating more than one GB hugepage (as for IVSHMEM), you can configure the +amount of memory used from any given NUMA nodes. For example, to use 1GB from +NUMA node 0, run:: + + $ ovs-vsctl --no-wait set Open_vSwitch . \ + other_config:dpdk-socket-mem="1024,0" + +Similarly, if you wish to better scale the workloads across cores, then +multiple pmd threads can be created and pinned to CPU cores by explicity +specifying ``pmd-cpu-mask``. For example, to spawn two pmd threads and pin +them to cores 1,2, run:: + + $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=6 + +For details on using ivshmem with DPDK, refer to `the advanced installation +guide <../../INSTALL.DPDK-ADVANCED.md>`__. + +Refer to ovs-vswitchd.conf.db(5) for additional information on configuration +options. + +.. note:: + Changing any of these options requires restarting the ovs-vswitchd + application + +Validating +---------- + +Creating bridges and ports +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can now use ovs-vsctl to set up bridges and other Open vSwitch features. +Bridges should be created with a ``datapath_type=netdev``:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + +Now you can add DPDK devices. OVS expects DPDK device names to start with +``dpdk`` and end with a portid. ovs-vswitchd should print the number of dpdk +devices found in the log file:: + + $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk + $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk + +After the DPDK ports get added to switch, a polling thread continuously polls +DPDK devices and consumes 100% of the core, as can be checked from 'top' and +'ps' cmds:: + + $ top -H + $ ps -eLo pid,psr,comm | grep pmd + +Creating bonds of DPDK interfaces is slightly different to creating bonds of +system interfaces. For DPDK, the interface type must be explicitly set. For +example: + + $ ovs-vsctl add-bond br0 dpdkbond dpdk0 dpdk1 \ + -- set Interface dpdk0 type=dpdk \ + -- set Interface dpdk1 type=dpdk + +To stop ovs-vswitchd & delete bridge, run:: + + $ ovs-appctl -t ovs-vswitchd exit + $ ovs-appctl -t ovsdb-server exit + $ ovs-vsctl del-br br0 + +PMD thread statistics +~~~~~~~~~~~~~~~~~~~~~ + +To show current stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-show + +To clear previous stats:: + + $ ovs-appctl dpif-netdev/pmd-stats-clear + +Port/rxq assigment to PMD threads +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To show port/rxq assignment:: + + $ ovs-appctl dpif-netdev/pmd-rxq-show + +To change default rxq assignment to pmd threads, rxqs may be manually pinned to +desired cores using:: + + $ ovs-vsctl set Interface \ + other_config:pmd-rxq-affinity= + +where: + +- ```` ::= ``NULL`` | ```` +- ```` ::= ```` | + ```` , ```` +- ```` ::= ```` : ```` + +For example:: + + $ ovs-vsctl set interface dpdk0 options:n_rxq=4 \ + other_config:pmd-rxq-affinity="0:3,1:7,3:8" + +This will ensure: + +- Queue #0 pinned to core 3 +- Queue #1 pinned to core 7 +- Queue #2 not pinned +- Queue #3 pinned to core 8 + +After that PMD threads on cores where RX queues was pinned will become +``isolated``. This means that this thread will poll only pinned RX queues. + +.. warning:: + If there are no ``non-isolated`` PMD threads, ``non-pinned`` RX queues will + not be polled. Also, if provided ``core_id`` is not available (ex. this + ``core_id`` not in ``pmd-cpu-mask``), RX queue will not be polled by any PMD + thread. + +.. _dpdk-guest-setup: + +DPDK in the VM +-------------- + +DPDK 'testpmd' application can be run in the Guest VM for high speed packet +forwarding between vhostuser ports. DPDK and testpmd application has to be +compiled on the guest VM. Below are the steps for setting up the testpmd +application in the VM. More information on the vhostuser ports can be found in +the `advanced install guide <../../INSTALL.DPDK-ADVANCED.md>`__. + +.. note:: + Support for DPDK in the guest requires QEMU >= 2.2.0. + +To being, instantiate the guest:: + + $ export VM_NAME=Centos-vm export GUEST_MEM=3072M + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + + $ qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ + -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ + -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off \ + +Download the DPDK sourcs to VM and build DPDK:: + + $ cd /root/dpdk/ + $ wget http://dpdk.org/browse/dpdk/snapshot/dpdk-16.07.zip + $ unzip dpdk-16.07.zip + $ export DPDK_DIR=/root/dpdk/dpdk-16.07 + $ export DPDK_TARGET=x86_64-native-linuxapp-gcc + $ export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET + $ cd $DPDK_DIR + $ make install T=$DPDK_TARGET DESTDIR=install + +Build the test-pmd application:: + + $ cd app/test-pmd + $ export RTE_SDK=$DPDK_DIR + $ export RTE_TARGET=$DPDK_TARGET + $ make + +Setup huge pages and DPDK devices using UIO:: + + $ sysctl vm.nr_hugepages=1024 + $ mkdir -p /dev/hugepages + $ mount -t hugetlbfs hugetlbfs /dev/hugepages # only if not already mounted + $ modprobe uio + $ insmod $DPDK_BUILD/kmod/igb_uio.ko + $ $DPDK_DIR/tools/dpdk-devbind.py --status + $ $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0 + +.. note:: + + vhost ports pci ids can be retrieved using:: + + lspci | grep Ethernet + +Testing +------- + +Below are few testcases and the list of steps to be followed. Before beginning, +ensure a userspace bridge has been created and two DPDK ports added:: + + $ ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev + $ ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk + $ ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk + +PHY-PHY +~~~~~~~ + +Add test flows to forward packets betwen DPDK port 0 and port 1:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows between port 1 (dpdk0) to port 2 (dpdk1) + $ ovs-ofctl add-flow br0 in_port=1,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:1 + +Transmit traffic into either port. You should see it returned via the other. + +PHY-VM-PHY (vhost loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add two ``dpdkvhostuser`` ports to bridge ``br0``:: + + $ ovs-vsctl add-port br0 dpdkvhostuser0 \ + -- set Interface dpdkvhostuser0 type=dpdkvhostuser + $ ovs-vsctl add-port br0 dpdkvhostuser1 \ + -- set Interface dpdkvhostuser1 type=dpdkvhostuser + +Add test flows to forward packets betwen DPDK devices and VM ports:: + + # Clear current flows + $ ovs-ofctl del-flows br0 + + # Add flows + $ ovs-ofctl add-flow br0 in_port=1,action=output:3 + $ ovs-ofctl add-flow br0 in_port=3,action=output:1 + $ ovs-ofctl add-flow br0 in_port=4,action=output:2 + $ ovs-ofctl add-flow br0 in_port=2,action=output:4 + + # Dump flows + $ ovs-ofctl dump-flows br0 + +Create a VM using the following configuration: + ++----------------------+--------+-----------------+ +| configuration | values | comments | ++----------------------+--------+-----------------+ +| qemu version | 2.2.0 | n/a | +| qemu thread affinity | core 5 | taskset 0x20 | +| memory | 4GB | n/a | +| cores | 2 | n/a | +| Qcow2 image | CentOS7| n/a | +| mrg_rxbuf | off | n/a | ++----------------------+--------+-----------------+ + +You can do this directly with QEMU via the ``qemu-system-x86_64`` application:: + + $ export VM_NAME=vhost-vm + $ export GUEST_MEM=3072M + $ export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2 + $ export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch + + $ taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm \ + -m $GUEST_MEM -drive file=$QCOW2_IMAGE --nographic -snapshot \ + -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 \ + -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on \ + -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 \ + -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off \ + -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 \ + -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ + -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off + +Alternatively, you can configure the guest using libvirt. Below is an XML +configuration for a 'demovm' guest that can be instantiated using `virsh`:: + + + demovm + 4a9b3f53-fa2a-47f3-a757-dd87720d9d1d + 4194304 + 4194304 + + + + + + 2 + + 4096 + + + + + + hvm + + + + + + + + + + + + + + destroy + restart + destroy + + /usr/bin/qemu-kvm + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +Once the guest is configured and booted, configure DPDK packet forwarding +within the guest. To accomplish this, DPDK and testpmd application have to +be first compiled on the VM as described in :ref:`dpdk-guest-setup`. Once +compiled, run the ``test-pmd`` application:: + + $ cd $DPDK_DIR/app/test-pmd; + $ ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- \ + --burst=64 -i --txqflags=0xf00 --disable-hw-vlan + $ set fwd mac retry + $ start + +When you finish testing, bind the vNICs back to kernel. + + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0 + $ $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0 + +.. note:: + Appropriate PCI IDs to be passed in above example. The PCI IDs can be + retrieved like so:: + + $ $DPDK_DIR/tools/dpdk-devbind.py --status + +.. note:: + More information on the dpdkvhostuser ports can be found in the `advanced + installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. + +PHY-VM-PHY (IVSHMEM loopback) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Refer to the `advanced installation guide <../../INSTALL.DPDK-ADVANCED.md>`__. + +Limitations +------------ + +- Currently DPDK ports does not use HW offload functionality. +- Network Interface Firmware requirements: Each release of DPDK is + validated against a specific firmware version for a supported Network + Interface. New firmware versions introduce bug fixes, performance + improvements and new functionality that DPDK leverages. The validated + firmware versions are available as part of the release notes for + DPDK. It is recommended that users update Network Interface firmware + to match what has been validated for the DPDK release. + + The latest list of validated firmware versions can be found in the `DPDK + release notes`_. + +.. _DPDK release notes: http://dpdk.org/doc/guides/rel_notes/release_16.07.html diff --git a/Documentation/install-guide/general.rst b/Documentation/install-guide/general.rst new file mode 100644 index 0000000..e1866f8 --- /dev/null +++ b/Documentation/install-guide/general.rst @@ -0,0 +1,870 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +========================================= +Open vSwitch on Linux, FreeBSD and NetBSD +========================================= + +This document describes how to build and install Open vSwitch on a generic +Linux, FreeBSD, or NetBSD host. For specifics around installation on a specific +platform, refer to one of these installation guides: + +- `Debian <../../INSTALL.Debian.md>`__ +- `Fedora <../../INSTALL.Fedora.md>`__ +- `RHEL <../../INSTALL.RHEL.md>`__ +- `XenServer <../../INSTALL.XenServer.md>`__ +- `NetBSD <../../INSTALL.NetBSD.md>`__ +- `Windows <../../INSTALL.Windows.md>`__ +- `DPDK <../../INSTALL.DPDK.md>`__ + +.. _general-build-reqs: + +Build Requirements +------------------ + +To compile the userspace programs in the Open vSwitch distribution, you will +need the following software: + +- GNU make + +- A C compiler, such as: + + - GCC 4.x. + + - Clang. Clang 3.4 and later provide useful static semantic analysis and + thread-safety checks. For Ubuntu, there are nightly built packages + available on clang's website. + + - MSVC 2013. See the `Windows installation guide <../../INSTALL.Windows>`__ + for additional Windows build instructions. + + While OVS may be compatible with other compilers, optimal support for atomic + operations may be missing, making OVS very slow (see ``lib/ovs-atomic.h``). + +- libssl, from OpenSSL, is optional but recommended if you plan to connect the + Open vSwitch to an OpenFlow controller. libssl is required to establish + confidentiality and authenticity in the connections from an Open vSwitch to + an OpenFlow controller. If libssl is installed, then Open vSwitch will + automatically build with support for it. + +- libcap-ng, written by Steve Grubb, is optional but recommended. It is + required to run OVS daemons as a non-root user with dropped root privileges. + If libcap-ng is installed, then Open vSwitch will automatically build with + support for it. + +- Python 2.7. You must also have the Python ``six`` library. + +On Linux, you may choose to compile the kernel module that comes with the Open +vSwitch distribution or to use the kernel module built into the Linux kernel +(version 3.3 or later). See the `FAQ <../../FAQ.md>`__ question "What features +are not available in the Open vSwitch kernel datapath that ships as part of the +upstream Linux kernel?" for more information on this trade-off. You may also +use the userspace-only implementation, at some cost in features and performance +(see the `userspace installation guide <../../INSTALL.userspace.md>`__ for +details). + +To compile the kernel module on Linux, you must also install the +following: + +- A supported Linux kernel version. Refer to the `README <../../README.md>`__ + for a list of supported versions. + + For optional support of ingress policing, you must enable kernel + configuration options ``NET_CLS_BASIC``, ``NET_SCH_INGRESS``, and + ``NET_ACT_POLICE``, either built-in or as modules. ``NET_CLS_POLICE`` is + obsolete and not needed.) + + On kernels before 3.11, the ``ip_gre`` module, for GRE tunnels over IP + (``NET_IPGRE``), must not be loaded or compiled in. + + To configure HTB or HFSC quality of service with Open vSwitch, you must + enable the respective configuration options. + + To use Open vSwitch support for TAP devices, you must enable ``CONFIG_TUN``. + +- To build a kernel module, you need the same version of GCC that was used to + build that kernel. + +- A kernel build directory corresponding to the Linux kernel image the module + is to run on. Under Debian and Ubuntu, for example, each linux-image package + containing a kernel binary has a corresponding linux-headers package with + the required build infrastructure. + +If you are working from a Git tree or snapshot (instead of from a distribution +tarball), or if you modify the Open vSwitch build system or the database +schema, you will also need the following software: + +- Autoconf version 2.63 or later. + +- Automake version 1.10 or later. + +- libtool version 2.4 or later. (Older versions might work too.) + +To run the unit tests, you also need: + +- Perl. Version 5.10.1 is known to work. Earlier versions should also + work. + +The datapath tests for userspace and Linux datapaths also rely upon: + +- pyftpdlib. Version 1.2.0 is known to work. Earlier versions should + also work. + +- GNU wget. Version 1.16 is known to work. Earlier versions should also + work. + +The ovs-vswitchd.conf.db(5) manpage will include an E-R diagram, in formats +other than plain text, only if you have the following: + +- dot from graphviz (http://www.graphviz.org/). + +- Perl. Version 5.10.1 is known to work. Earlier versions should also + work. + +If you are going to extensively modify Open vSwitch, consider installing the +following to obtain better warnings: + +- "sparse" version 0.4.4 or later + (http://www.kernel.org/pub/software/devel/sparse/dist/). + +- GNU make. + +- clang, version 3.4 or later + +- flake8, version 2.X, along with the hacking flake8 plugin (for Python code). + The automatic flake8 check that runs against Python code has some warnings + enabled that come from the "hacking" flake8 plugin. If it's not installed, + the warnings just won't occur until it's run on a system with "hacking" + installed. Note that there are problems with flake8 3.0 and the "hacking" + plugin. To ensure you get flake8 2.X, you can use:: + + $ pip install 'flake8<3.0' + +You may find the ovs-dev script found in ``utilities/ovs-dev.py`` useful. + +.. _general-install-reqs: + +Installation Requirements +------------------------- + +The machine you build Open vSwitch on may not be the one you run it on. To +simply install and run Open vSwitch you require the following software: + +- libc compatible with the libc used for build. + +- libssl compatible with the libssl used for build, if OpenSSL was used + for the build. + +- On Linux, the same kernel version configured as part of the build. + +- For optional support of ingress policing on Linux, the "tc" program + from iproute2 (part of all major distributions and available at + http://www.linux-foundation.org/en/Net:Iproute2). + +- Python 2.7. You must also have the Python six library. + +On Linux you should ensure that ``/dev/urandom`` exists. To support TAP +devices, you must also ensure that ``/dev/net/tun`` exists. + +.. _general-bootstrapping: + +Bootstrapping +------------- + +This step is not needed if you have downloaded a released tarball. If +you pulled the sources directly from an Open vSwitch Git tree or got a +Git tree snapshot, then run boot.sh in the top source directory to build +the "configure" script:: + + $ ./boot.sh + +.. _general-configuring: + +Configuring +----------- + +Configure the package by running the configure script. You can usually +invoke configure without any arguments. For example:: + + $ ./configure + +By default all files are installed under ``/usr/local``. Open vSwitch also +expects to find its database in ``/usr/local/etc/openvswitch`` by default. If +you want to install all files into, e.g., ``/usr`` and ``/var`` instead of +``/usr/local`` and ``/usr/local/var`` and expect to use ``/etc/openvswitch`` as +the default database directory, add options as shown here:: + + $ ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc + +.. note:: + + Open vSwitch installed with packages like .rpm (e.g. via ``yum install`` or + ``rpm -ivh``) and .deb (e.g. via ``apt-get install`` or ``dpkg -i``) use the + above configure options. + +By default, static libraries are built and linked against. If you want to use +shared libraries instead:: + + $ ./configure --enable-shared + +To use a specific C compiler for compiling Open vSwitch user programs, also +specify it on the configure command line, like so:: + + $ ./configure CC=gcc-4.2 + +To use 'clang' compiler:: + + $ ./configure CC=clang + +To supply special flags to the C compiler, specify them as ``CFLAGS`` on the +configure command line. If you want the default CFLAGS, which include ``-g`` to +build debug symbols and ``-O2`` to enable optimizations, you must include them +yourself. For example, to build with the default CFLAGS plus ``-mssse3``, you +might run configure as follows:: + + $ ./configure CFLAGS="-g -O2 -mssse3" + +For efficient hash computation special flags can be passed to leverage built-in +intrinsics. For example on X86_64 with SSE4.2 instruction set support, CRC32 +intrinsics can be used by passing ``-msse4.2``:: + + $ ./configure CFLAGS="-g -O2 -msse4.2"` + +If you are on a different processor and don't know what flags to choose, it is +recommended to use ``-march=native`` settings:: + + $ ./configure CFLAGS="-g -O2 -march=native" + +With this, GCC will detect the processor and automatically set appropriate +flags for it. This should not be used if you are compiling OVS outside the +target machine. + +.. note:: + CFLAGS are not applied when building the Linux kernel module. Custom CFLAGS + for the kernel module are supplied using the ``EXTRA_CFLAGS`` variable when + running make. For example:: + + $ make EXTRA_CFLAGS="-Wno-error=date-time" + +To build the Linux kernel module, so that you can run the kernel-based switch, +pass the location of the kernel build directory on ``--with-linux``. For +example, to build for a running instance of Linux:: + + $ ./configure --with-linux=/lib/modules/$(uname -r)/build + +.. note:: + If ``--with-linux`` requests building for an unsupported version of Linux, + then ``configure`` will fail with an error message. Refer to the `FAQ + <../../FAQ.md>`__ for advice in that case. + +If you wish to build the kernel module for an architecture other than the +architecture of the machine used for the build, you may specify the kernel +architecture string using the KARCH variable when invoking the configure +script. For example, to build for MIPS with Linux:: + + $ ./configure --with-linux=/path/to/linux KARCH=mips + +If you plan to do much Open vSwitch development, you might want to add +``--enable-Werror``, which adds the ``-Werror`` option to the compiler command +line, turning warnings into errors. That makes it impossible to miss warnings +generated by the build. For example:: + + $ ./configure --enable-Werror + +To build with gcov code coverage support, add ``--enable-coverage``:: + + $ ./configure --enable-coverage + +The configure script accepts a number of other options and honors additional +environment variables. For a full list, invoke configure with the ``--help`` +option:: + + $ ./configure --help + +You can also run configure from a separate build directory. This is helpful if +you want to build Open vSwitch in more than one way from a single source +directory, e.g. to try out both GCC and Clang builds, or to build kernel +modules for more than one Linux version. For example:: + + $ mkdir _gcc && (cd _gcc && ../configure CC=gcc) + $ mkdir _clang && (cd _clang && ../configure CC=clang) + +Under certains loads the ovsdb-server and other components perform better when +using the jemalloc memory allocator, instead of the glibc memory allocator. If +you wish to link with jemalloc add it to LIBS:: + + $ ./configure LIBS=-ljemalloc + +.. _general-building: + +Building +-------- + +1. Run GNU make in the build directory, e.g.:: + + $ make + + or if GNU make is installed as "gmake":: + + $ gmake + + If you used a separate build directory, run make or gmake from that + directory, e.g.:: + + $ make -C _gcc + $ make -C _clang + + For improved warnings if you installed ``sparse`` (see "Prerequisites"), add + ``C=1`` to the command line. + + .. note:: + Some versions of Clang and ccache are not completely compatible. If you + see unusual warnings when you use both together, consider disabling + ccache. + +2. Consider running the testsuite. Refer to :ref:`general-testing` for + instructions. + +3. Run ``make install`` to install the executables and manpages into the + running system, by default under ``/usr/local``:: + + $ make install + +5. If you built kernel modules, you may install them, e.g.:: + + $ make modules_install + + It is possible that you already had a Open vSwitch kernel module installed + on your machine that came from upstream Linux (in a different directory). To + make sure that you load the Open vSwitch kernel module you built from this + repository, you should create a ``depmod.d`` file that prefers your newly + installed kernel modules over the kernel modules from upstream Linux. The + following snippet of code achieves the same:: + + $ config_file="/etc/depmod.d/openvswitch.conf" + $ for module in datapath/linux/*.ko; do + modname="$(basename ${module})" + echo "override ${modname%.ko} * extra" >> "$config_file" + echo "override ${modname%.ko} * weak-updates" >> "$config_file" + done + $ depmod -a + + Finally, load the kernel modules that you need. e.g.:: + + $ /sbin/modprobe openvswitch + + To verify that the modules have been loaded, run ``/sbin/lsmod`` and check + that openvswitch is listed:: + + $ /sbin/lsmod | grep openvswitch + + .. note:: + If the ``modprobe`` operation fails, look at the last few kernel log + messages (e.g. with ``dmesg | tail``). Generally, issues like this occur + when Open vSwitch is built for a kernel different from the one into which + you are trying to load it. Run ``modinfo`` on ``openvswitch.ko`` and on a + module built for the running kernel, e.g.:: + + $ /sbin/modinfo openvswitch.ko + $ /sbin/modinfo /lib/modules/$(uname -r)/kernel/net/bridge/bridge.ko + + Compare the "vermagic" lines output by the two commands. If they differ, + then Open vSwitch was built for the wrong kernel. + + If you decide to report a bug or ask a question related to module loading, + include the output from the ``dmesg`` and ``modinfo`` commands mentioned + above. + +.. _general-starting: + +Starting +-------- + +Before starting ovs-vswitchd itself, you need to start its configuration +database, ovsdb-server. Each machine on which Open vSwitch is installed should +run its own copy of ovsdb-server. Before ovsdb-server itself can be started, +configure a database that it can use:: + + $ mkdir -p /usr/local/etc/openvswitch + $ ovsdb-tool create /usr/local/etc/openvswitch/conf.db \ + vswitchd/vswitch.ovsschema + +Configure ovsdb-server to use database created above, to listen on a Unix +domain socket, to connect to any managers specified in the database itself, and +to use the SSL configuration in the database:: + + $ mkdir -p /usr/local/var/run/openvswitch + $ ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \ + --remote=db:Open_vSwitch,Open_vSwitch,manager_options \ + --private-key=db:Open_vSwitch,SSL,private_key \ + --certificate=db:Open_vSwitch,SSL,certificate \ + --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \ + --pidfile --detach + +.. note:: + If you built Open vSwitch without SSL support, then omit ``--private-key``, + ``--certificate``, and ``--bootstrap-ca-cert``.) + +Initialize the database using ovs-vsctl. This is only necessary the first time +after you create the database with ovsdb-tool, though running it at any time is +harmless:: + + $ ovs-vsctl --no-wait init + +Start the main Open vSwitch daemon, telling it to connect to the same Unix +domain socket:: + + $ ovs-vswitchd --pidfile --detach + +Validating +---------- + +At this point you can use ovs-vsctl to set up bridges and other Open vSwitch +features. For example, to create a bridge named ``br0`` and add ports ``eth0`` +and ``vif1.0`` to it:: + + $ ovs-vsctl add-br br0 + $ ovs-vsctl add-port br0 eth0 + $ ovs-vsctl add-port br0 vif1.0 + +Refer to ovs-vsctl(8) for more details. + +Upgrading +--------- + +When you upgrade Open vSwitch from one version to another you should also +upgrade the database schema: + +1. Stop the Open vSwitch daemons, e.g.:: + + $ kill `cd /usr/local/var/run/openvswitch && cat ovsdb-server.pid ovs-vswitchd.pid` + +2. Install the new Open vSwitch release by using the same configure options as + was used for installing the previous version. If you do not use the same + configure options, you can end up with two different versions of Open + vSwitch executables installed in different locations. + +3. Upgrade the database, in one of the following two ways: + + - If there is no important data in your database, then you may delete the + database file and recreate it with ovsdb-tool, following the instructions + under "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD". + + - If you want to preserve the contents of your database, back it up first, + then use ``ovsdb-tool convert`` to upgrade it, e.g.:: + + $ ovsdb-tool convert /usr/local/etc/openvswitch/conf.db \ + vswitchd/vswitch.ovsschema + +4. Start the Open vSwitch daemons as described under :ref:`general-starting` + above. + +Hot Upgrading +------------- + +Upgrading Open vSwitch from one version to the next version with minimum +disruption of traffic going through the system that is using that Open vSwitch +needs some considerations: + +1. If the upgrade only involves upgrading the userspace utilities and daemons + of Open vSwitch, make sure that the new userspace version is compatible with + the previously loaded kernel module. + +2. An upgrade of userspace daemons means that they have to be restarted. + Restarting the daemons means that the OpenFlow flows in the ovs-vswitchd + daemon will be lost. One way to restore the flows is to let the controller + re-populate it. Another way is to save the previous flows using a utility + like ovs-ofctl and then re-add them after the restart. Restoring the old + flows is accurate only if the new Open vSwitch interfaces retain the old + 'ofport' values. + +3. When the new userspace daemons get restarted, they automatically flush the + old flows setup in the kernel. This can be expensive if there are hundreds + of new flows that are entering the kernel but userspace daemons are busy + setting up new userspace flows from either the controller or an utility like + ovs-ofctl. Open vSwitch database provides an option to solve this problem + through the ``other_config:flow-restore-wait`` column of the + ``Open_vSwitch`` table. Refer to the ovs-vswitchd.conf.db(5) manpage for + details. + +4. If the upgrade also involves upgrading the kernel module, the old kernel + module needs to be unloaded and the new kernel module should be loaded. This + means that the kernel network devices belonging to Open vSwitch is recreated + and the kernel flows are lost. The downtime of the traffic can be reduced if + the userspace daemons are restarted immediately and the userspace flows are + restored as soon as possible. + +The ovs-ctl utility's ``restart`` function only restarts the userspace daemons, +makes sure that the 'ofport' values remain consistent across restarts, restores +userspace flows using the ovs-ofctl utility and also uses the +``other_config:flow-restore-wait`` column to keep the traffic downtime to the +minimum. The ovs-ctl utility's ``force-reload-kmod`` function does all of the +above, but also replaces the old kernel module with the new one. Open vSwitch +startup scripts for Debian, XenServer and RHEL use ovs-ctl's functions and it +is recommended that these functions be used for other software platforms too. + +.. _general-testing: + +Testing +------- + +This section describe Open vSwitch's built-in support for various test +suites. You must bootstrap, configure and build Open vSwitch (steps are +in "Building and Installing Open vSwitch for Linux, FreeBSD or NetBSD" +above) before you run the tests described here. You do not need to +install Open vSwitch or to build or load the kernel module to run these +test suites. You do not need supervisor privilege to run these test +suites. + +Unit Tests +~~~~~~~~~~ + +Open vSwitch includes a suite of self-tests. Before you submit patches +upstream, we advise that you run the tests and ensure that they pass. If you +add new features to Open vSwitch, then adding tests for those features will +ensure your features don't break as developers modify other areas of Open +vSwitch. + +To run all the unit tests in Open vSwitch, one at a time, run:: + + $ make check + +This takes under 5 minutes on a modern desktop system. + +To run all the unit tests in Open vSwitch in parallel, run:: + + $ make check TESTSUITEFLAGS=-j8 + +You can run up to eight threads. This takes under a minute on a modern 4-core +desktop system. + +To see a list of all the available tests, run: + + $ make check TESTSUITEFLAGS=--list + +To run only a subset of tests, e.g. test 123 and tests 477 through 484, run:: + + $ make check TESTSUITEFLAGS='123 477-484' + +Tests do not have inter-dependencies, so you may run any subset. + +To run tests matching a keyword, e.g. ``ovsdb``, run:: + + $ make check TESTSUITEFLAGS='-k ovsdb' + +To see a complete list of test options, run:: + + $ make check TESTSUITEFLAGS=--help + +The results of a testing run are reported in ``tests/testsuite.log``. Report +report test failures as bugs and include the ``testsuite.log`` in your report. + +.. note:: + Sometimes a few tests may fail on some runs but not others. This is usually a + bug in the testsuite, not a bug in Open vSwitch itself. If you find that a + test fails intermittently, please report it, since the developers may not + have noticed. You can make the testsuite automatically rerun tests that fail, + by adding ``RECHECK=yes`` to the ``make`` command line, e.g.:: + + $ make check TESTSUITEFLAGS=-j8 RECHECK=yes + +Coverage +++++++++ + +If the build was configured with ``--enable-coverage`` and the ``lcov`` utility +is installed, you can run the testsuite and generate a code coverage report by +using the ``check-lcoc`` target:: + + $ make check-lcov + +All the same options are avaiable via TESTSUITEFLAGS. For example:: + + $ make check-lcov TESTSUITEFLAGS=-j8 -k ovn + +Profiling ++++++++++ + +If you have ``valgrind`` installed, you can run the testsuite under +valgrind by using the ``check-valgrind`` target:: + + $ make check-valgrind + +When you do this, the "valgrind" results for test ```` are reported in files +named ``tests/testsuite.dir//valgrind.*``. + +All the same options are available via TESTSUITEFLAGS. + +.. hint:: + You may find that the valgrind results are easier to interpret if you put + ``-q`` in ``~/.valgrindrc``, since that reduces the amount of output. + +.. _general-oftest: + +OFTest +~~~~~~ + +OFTest is an OpenFlow protocol testing suite. Open vSwitch includes a Makefile +target to run OFTest with Open vSwitch in "dummy mode". In this mode of +testing, no packets travel across physical or virtual networks. Instead, Unix +domain sockets stand in as simulated networks. This simulation is imperfect, +but it is much easier to set up, does not require extra physical or virtual +hardware, and does not require supervisor privileges. + +To run OFTest with Open vSwitch, first read and follow the instructions under +:ref:`general-testing` above. Second, obtain a copy of OFTest and install its +prerequisites. You need a copy of OFTest that includes commit 406614846c5 (make +ovs-dummy platform work again). This commit was merged into the OFTest +repository on Feb 1, 2013, so any copy of OFTest more recent than that should +work. Testing OVS in dummy mode does not require root privilege, so you may +ignore that requirement. + +Optionally, add the top-level OFTest directory (containing the ``oft`` program) +to your ``$PATH``. This slightly simplifies running OFTest later. + +To run OFTest in dummy mode, run the following command from your Open vSwitch +build directory:: + + $ make check-oftest OFT= + +where ```` is the absolute path to the ``oft`` program in OFTest. +If you added "oft" to your $PATH, you may omit the OFT variable +assignment + +By default, ``check-oftest`` passes ``oft`` just enough options to enable dummy +mode. You can use ``OFTFLAGS`` to pass additional options. For example, to run +just the ``basic.Echo`` test instead of all tests (the default) and enable +verbose logging, run:: + + $ make check-oftest OFT= OFTFLAGS='--verbose -T basic.Echo' + +If you use OFTest that does not include commit 4d1f3eb2c792 (oft: change +default port to 6653), merged into the OFTest repository in October 2013, then +you need to add an option to use the IETF-assigned controller port:: + + $ make check-oftest OFT= OFTFLAGS='--port=6653' + +Interpret OFTest results cautiously. Open vSwitch can fail a given test in +OFTest for many reasons, including bugs in Open vSwitch, bugs in OFTest, bugs +in the "dummy mode" integration, and differing interpretations of the OpenFlow +standard and other standards. + +.. note:: + Open vSwitch has not been validated against OFTest. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and OFTest in your bug report, plus any other information + needed to reproduce the problem. + +Ryu +~~~ + +Ryu is an OpenFlow controller written in Python that includes an extensive +OpenFlow testsuite. Open vSwitch includes a Makefile target to run Ryu in +"dummy mode". See :ref:`general-oftest` above for an explanation of dummy mode. + +To run Ryu tests with Open vSwitch, first read and follow the instructions +under :ref:`general-testing` above. Second, obtain a copy of Ryu, install its +prerequisites, and build it. You do not need to install Ryu (some of the tests +do not get installed, so it does not help). + +To run Ryu tests, run the following command from your Open vSwitch build +directory:: + + $ make check-ryu RYUDIR=`` + +where ```` is the absolute path to the root of the Ryu source +distribution. The default ```` is ``$srcdir/../ryu`` +where ``$srcdir`` is your Open vSwitch source directory. If this is correct, +omit ``RYUDIR`` + +.. note:: + Open vSwitch has not been validated against Ryu. Report test failures that + you believe to represent bugs in Open vSwitch. Include the precise versions + of Open vSwitch and Ryu in your bug report, plus any other information + needed to reproduce the problem. + +Datapath testing +~~~~~~~~~~~~~~~~ + +Open vSwitch includes a suite of tests specifically for datapath functionality, +which can be run against the userspace or kernel datapaths. If you are +developing datapath features, it is recommended that you use these tests and +build upon them to verify your implementation. + +The datapath tests make some assumptions about the environment. They must be +run under root privileges on a Linux system with support for network +namespaces. For ease of use, the OVS source tree includes a vagrant box to +invoke these tests. Running the tests inside Vagrant provides kernel isolation, +protecting your development host from kernel panics or configuration conflicts +in the testsuite. If you wish to run the tests without using the vagrant box, +there are further instructions below. + +Vagrant ++++++++ + +.. important:: + + Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor + +.. note:: + You must :ref:`bootstrap ` and + :ref:`configure` the sources before you run the steps + described here. + +A Vagrantfile is provided allowing to compile and provision the source tree as +found locally in a virtual machine using the following command:: + + $ vagrant up + +This will bring up a Fedora 23 VM by default. If you wish to use a different +box or a vagrant backend not supported by the default box, the ``Vagrantfile`` +can be modified to use a different box as base. + +The VM can be reprovisioned at any time:: + + $ vagrant provision + +OVS out-of-tree compilation environment can be set up with:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,build_ovs + +This will set up an out-of-tree build environment inside the VM in +``/root/build``. The source code can be found in ``/vagrant``. + +To recompile and reinstall OVS in the VM using RPM:: + + $ ./boot.sh + $ vagrant provision --provision-with configure_ovs,install_rpm + +Two provisioners are included to run system tests with the OVS kernel module or +with a userspace datapath. This tests are different from the self-tests +mentioned above. To run them:: + + $ ./boot.sh + $ vagrant provision --provision-with \ + configure_ovs,test_ovs_kmod,test_ovs_system_userspace + +The results of the testsuite reside in the VM root user's home directory:: + + $ vagrant ssh + $ sudo -s + $ cd /root/build + $ ls tests/system* + +Native +++++++ + +The datapath testsuite as invoked by Vagrant above may also be run manually on +a Linux system with root privileges. These tests may take several minutes to +complete, and cannot be run in parallel. + +Userspace datapath +''''''''''''''''''' + +To invoke the datapath testsuite with the userspace datapath, run:: + + $ make check-system-userspace + +The results of the testsuite are in ``tests/system-userspace-traffic.dir``. + +Kernel datapath +''''''''''''''' + +Make targets are also provided for testing the Linux kernel module. Note that +these tests operate by inserting modules into the running Linux kernel, so if +the tests are able to trigger a bug in the OVS kernel module or in the upstream +kernel then the kernel may panic. + +To run the testsuite against the kernel module which is currently installed on +your system, run:: + + $ make check-kernel + +To install the kernel module from the current build directory and run the +testsuite against that kernel module:: + + $ make check-kmod + +The results of the testsuite are in ``tests/system-kmod-traffic.dir``. + +Continuous Integration with Travis-CI +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A .travis.yml file is provided to automatically build Open vSwitch with various +build configurations and run the testsuite using travis-ci. Builds will be +performed with gcc, sparse and clang with the -Werror compiler flag included, +therefore the build will fail if a new warning has been introduced. + +The CI build is triggered via git push (regardless of the specific branch) or +pull request against any Open vSwitch GitHub repository that is linked to +travis-ci. + +Instructions to setup travis-ci for your GitHub repository: + +1. Go to http://travis-ci.org/ and sign in using your GitHub ID. +2. Go to the "Repositories" tab and enable the ovs repository. You may disable + builds for pushes or pull requests. +3. In order to avoid forks sending build failures to the upstream mailing list, + the notification email recipient is encrypted. If you want to receive email + notification for build failures, replace the the encrypted string: + + 1. Install the travis-ci CLI (Requires ruby >=2.0): gem install travis + 2. In your Open vSwitch repository: travis encrypt mylist@mydomain.org + 3. Add/replace the notifications section in .travis.yml and fill in the + secure string as returned by travis encrypt:: + + notifications: + email: + recipients: + - secure: "....." + + .. note:: + You may remove/omit the notifications section to fall back to default + notification behaviour which is to send an email directly to the author and + committer of the failing commit. Note that the email is only sent if the + author/committer have commit rights for the particular GitHub repository. + +4. Pushing a commit to the repository which breaks the build or the + testsuite will now trigger a email sent to mylist@mydomain.org + +Static Code Analysis +~~~~~~~~~~~~~~~~~~~~ + +Static Analysis is a method of debugging Software by examining code rather than +actually executing it. This can be done through 'scan-build' commandline +utility which internally uses clang (or) gcc to compile the code and also +invokes a static analyzer to do the code analysis. At the end of the build, the +reports are aggregated in to a common folder and can later be analyzed using +'scan-view'. + +Open vSwitch includes a Makefile target to trigger static code analysis:: + + $ ./boot.sh + $ ./configure CC=clang # clang + # or + $ ./configure CC=gcc CFLAGS="-std=gnu99" # gcc + $ make clang-analyze + +You should invoke scan-view to view analysis results. The last line of output +from ``clang-analyze`` will list the command (containing results directory) +that you should invoke to view the results on a browser. diff --git a/Documentation/install-guide/index.rst b/Documentation/install-guide/index.rst new file mode 100644 index 0000000..e9d1926 --- /dev/null +++ b/Documentation/install-guide/index.rst @@ -0,0 +1,34 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in Open vSwitch documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +Installation Guides +------------------- + +If you are looking to install Open vSwitch, the below guides should provide +some insight into the many configurations that Open vSwitch offers. + +.. toctree:: + :maxdepth: 1 + + /install-guide/general + /install-guide/dpdk