[ovs-dev,1/6] INSTALL.md: Document system-traffic tests.

Signed-off-by: Joe Stringer <joe@ovn.org>
---
 INSTALL.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 76 insertions(+), 12 deletions(-)

On Fri, May 20, 2016 at 11:49 AM, Joe Stringer <joe@ovn.org> wrote:

> Signed-off-by: Joe Stringer <joe@ovn.org>
> ---
>
Thanks for working on this!


>  INSTALL.md | 88
> +++++++++++++++++++++++++++++++++++++++++++++++++++++---------
>  1 file changed, 76 insertions(+), 12 deletions(-)
>
> diff --git a/INSTALL.md b/INSTALL.md
> index 5b37786d973d..82f5188eacb8 100644
> --- a/INSTALL.md
> +++ b/INSTALL.md
> @@ -101,6 +101,14 @@ To run the unit tests, you also need:
>    - Perl.  Version 5.10.1 is known to work.  Earlier versions should
>      also work.
>
> +The "system" tests for userspace and Linux datapaths also rely upon:
>
Should we just stick with "Datapath testing"

> +
> +  - pyftpdlib. Version 1.2.0 is known to work. Earlier versions should
> +    also work.
> +
> +  - GNU wget. Version 1.16 is known to work. Earlier versions should
> +    also work.
> +
>  The ovs-vswitchd.conf.db(5) manpage will include an E-R diagram, in
>  formats other than plain text, only if you have the following:
>
> @@ -593,24 +601,43 @@ test failures that you believe to represent bugs in
> Open vSwitch.
>  Include the precise versions of Open vSwitch and Ryu in your bug
>  report, plus any other information needed to reproduce the problem.
>
> -Vagrant
> --------
> +Datapath traffic testing
>
How about drop "traffic" from the title. More datapath features can be
tested using
this framework without sending any traffic through, right?


> +------------------------
> +
> +Open vSwitch also includes a suite of tests specifically for datapath
> +functionality, which can be run against the userspace or kernel datapaths.
> +If you are developing datapath features, it is recommended that you use
> +these tests and build upon them to verify your implementation.
>
> -Requires: Vagrant (version 1.7.0 or later) and a compatible hypervisor
> +The datapath tests make some assumptions about the environment. They
> +must be run under root privileges on a Linux system with support for
> +network namespaces. These tests are recommended to be run within an
> +isolated environment that is separate from your development system.
> +
> +For ease of use, a vagrant box is provided to invoke these tests. This
> +also provides kernel isolation, protecting your development host from
> +kernel panics or configuration conflicts in the testsuite. If you wish
> +to run the tests without using the vagrant box, there are further
> +instructions below.
> +
> +### Vagrant
> +
> +*Requires Vagrant (version 1.7.0 or later) and a compatible hypervisor*
>
>  You must bootstrap and configure the sources (steps are in "Building
>  and Installing Open vSwitch for Linux, FreeBSD or NetBSD" above) before
>  you run the steps described here.
>
>  A Vagrantfile is provided allowing to compile and provision the source
> -tree as found locally in a virtual machine using the following commands:
> +tree as found locally in a virtual machine using the following command:
>
>         vagrant up
> -       vagrant ssh
>
> -This will bring up w Fedora 20 VM by default, alternatively the
> -`Vagrantfile` can be modified to use a different distribution box as
> -base. Also, the VM can be reprovisioned at any time:
> +This will bring up a Fedora 20 VM by default. If you wish to use a
> +different box or a vagrant backend not supported by the default box,
> +the `Vagrantfile` can be modified to use a different box as base.
> +
> +The VM can be reprovisioned at any time:
>
>         vagrant provision
>
> @@ -619,11 +646,10 @@ OVS out-of-tree compilation environment can be set
> up with:
>         ./boot.sh
>         vagrant provision --provision-with configure_ovs,build_ovs
>
> -This will set up an out-of-tree build environment in /home/vagrant/build.
> -The source code can be found in /vagrant.  Out-of-tree build is preferred
> -to work around limitations of the sync file systems.
> +This will set up an out-of-tree build environment inside the VM in
> +/root/build. The source code can be found in /vagrant.


> -To recompile and reinstall OVS using RPM:
> +To recompile and reinstall OVS in the VM using RPM:
>
>         ./boot.sh
>         vagrant provision --provision-with configure_ovs,install_rpm
> @@ -635,6 +661,44 @@ the self-tests mentioned above.  To run them:
>         ./boot.sh
>         vagrant provision --provision-with
> configure_ovs,test_ovs_kmod,test_ovs_system_userspace
>
> +The results of the testsuite reside in the VM root user's home directory:
> +
> +        vagrant ssh
> +        sudo -s
> +        cd /root/build
> +        ls tests/system*
> +
> +### Native
> +
> +The datapath testsuite as invoked by Vagrant above may also be run
> +manually on a Linux system with root privileges. If some tests are
> +being skipped, consider updating your iproute2 package and Linux kernel
> +version.

This is not clear to me.


> These tests may take several minutes to complete, and cannot be
> +run in parallel.

+
> +To invoke the system testsuite with the userspace datapath:
> +
> +        make check-system-userspace
> +
> +Make targets are also provided for testing the Linux kernel module.
> +Note that these tests operate by inserting modules into the running
> +Linux kernel, so if the tests are able to trigger a bug in the OVS
> +kernel module or in the upstream kernel then the kernel may panic.
> +
> +To run the testsuite against the kernel module which is currently
> +installed on your system:
> +
> +        make check-kernel
> +
> +To install the kernel module from the current build directory and
> +run the testsuite against that kernel module:
> +
> +        make check-kmod
> +
> +The testsuite logs will be placed in tests/system-userspace-traffic.log
> +for the userspace target, or tests/system-kmod-traffic.log for the
> +kernel targets.
>

May be it will be clearer to separately discuss kernel datapath  and user
space
datapath?

+
>  Continuous Integration with Travis-CI
>  -------------------------------------
>
> --
> 2.8.2
>
>

Thanks for the review,

On 23 May 2016 at 12:49, Andy Zhou <azhou@ovn.org> wrote:
>> @@ -101,6 +101,14 @@ To run the unit tests, you also need:
>>    - Perl.  Version 5.10.1 is known to work.  Earlier versions should
>>      also work.
>>
>> +The "system" tests for userspace and Linux datapaths also rely upon:
>
> Should we just stick with "Datapath testing"

Sure, I'll update this.

>> -Vagrant
>> --------
>> +Datapath traffic testing
>
> How about drop "traffic" from the title. More datapath features can be
> tested using
> this framework without sending any traffic through, right?

True, I'll drop it.

>> +### Native
>> +
>> +The datapath testsuite as invoked by Vagrant above may also be run
>> +manually on a Linux system with root privileges. If some tests are
>> +being skipped, consider updating your iproute2 package and Linux kernel
>> +version.
>
> This is not clear to me.

Which part? The second sentence is a bit of a tangent and could be dropped.


>>
>> These tests may take several minutes to complete, and cannot be
>> +run in parallel.
>>
>> +
>> +To invoke the system testsuite with the userspace datapath:
>> +
>> +        make check-system-userspace
>> +
>> +Make targets are also provided for testing the Linux kernel module.
>> +Note that these tests operate by inserting modules into the running
>> +Linux kernel, so if the tests are able to trigger a bug in the OVS
>> +kernel module or in the upstream kernel then the kernel may panic.
>> +
>> +To run the testsuite against the kernel module which is currently
>> +installed on your system:
>> +
>> +        make check-kernel
>> +
>> +To install the kernel module from the current build directory and
>> +run the testsuite against that kernel module:
>> +
>> +        make check-kmod
>> +
>> +The testsuite logs will be placed in tests/system-userspace-traffic.log
>> +for the userspace target, or tests/system-kmod-traffic.log for the
>> +kernel targets.
>
>
> May be it will be clearer to separately discuss kernel datapath  and user
> space
> datapath?

Sure, I can split this out.

>> +
>>  Continuous Integration with Travis-CI
>>  -------------------------------------
>>
>> --
>> 2.8.2
>>
>

Thanks for writing documentation!

On Fri, May 20, 2016 at 11:49:54AM -0700, Joe Stringer wrote:
> --- a/INSTALL.md
> +++ b/INSTALL.md
> @@ -101,6 +101,14 @@ To run the unit tests, you also need:
>    - Perl.  Version 5.10.1 is known to work.  Earlier versions should
>      also work.
>  
> +The "system" tests for userspace and Linux datapaths also rely upon:

By "system" tests here, do you mean what is called the "datapath
traffic" tests later on?  If so, then it would be nice to use the same
name in both places.

(Looking at Andy's comments, I see I'm duplicating this one.  Great
minds think alike I guess.)

> +Datapath traffic testing
> +------------------------
> +
> +Open vSwitch also includes a suite of tests specifically for datapath
> +functionality, which can be run against the userspace or kernel datapaths.
> +If you are developing datapath features, it is recommended that you use
> +these tests and build upon them to verify your implementation.
>  
> -Requires: Vagrant (version 1.7.0 or later) and a compatible hypervisor
> +The datapath tests make some assumptions about the environment. They
> +must be run under root privileges on a Linux system with support for
> +network namespaces. These tests are recommended to be run within an
> +isolated environment that is separate from your development system.

A couple of uses of passive voice above.  I guess it's clear that the
author of the documentation is the one doing the recommending though.

However, I think that a change to active voice in the next sentence
would enhance clarity; otherwise it isn't clear where the vagrant box
is.  How about: "For ease of use, the OVS source tree includes a vagrant
box to invoke these tests."

> +For ease of use, a vagrant box is provided to invoke these tests. This
> +also provides kernel isolation, protecting your development host from
> +kernel panics or configuration conflicts in the testsuite. If you wish
> +to run the tests without using the vagrant box, there are further
> +instructions below.

Thanks again!

Acked-by: Ben Pfaff <blp@ovn.org>

On 2 June 2016 at 15:01, Ben Pfaff <blp@ovn.org> wrote:
>> +Datapath traffic testing
>> +------------------------
>> +
>> +Open vSwitch also includes a suite of tests specifically for datapath
>> +functionality, which can be run against the userspace or kernel datapaths.
>> +If you are developing datapath features, it is recommended that you use
>> +these tests and build upon them to verify your implementation.
>>
>> -Requires: Vagrant (version 1.7.0 or later) and a compatible hypervisor
>> +The datapath tests make some assumptions about the environment. They
>> +must be run under root privileges on a Linux system with support for
>> +network namespaces. These tests are recommended to be run within an
>> +isolated environment that is separate from your development system.
>
> A couple of uses of passive voice above.  I guess it's clear that the
> author of the documentation is the one doing the recommending though.
>
> However, I think that a change to active voice in the next sentence
> would enhance clarity; otherwise it isn't clear where the vagrant box
> is.  How about: "For ease of use, the OVS source tree includes a vagrant
> box to invoke these tests."
>
>> +For ease of use, a vagrant box is provided to invoke these tests. This
>> +also provides kernel isolation, protecting your development host from
>> +kernel panics or configuration conflicts in the testsuite. If you wish
>> +to run the tests without using the vagrant box, there are further
>> +instructions below.
>
> Thanks again!
>
> Acked-by: Ben Pfaff <blp@ovn.org>

Thanks for the suggestions. I took another pass at the wording,
applied the feedback, and pushed to master.