[ovs-dev,v1,00/13] Introduce local sampling with NXAST_SAMPLE action.

(Was: Add psample support to NXAST_SAMPLE action)

This is the userspace counterpart of the work done in the kernel
which has recently been merged in net-next [1]. There, a new
datapath action is added, called "psample".

From the PoV of ovs-vswitchd, this new action is used to implement
"local sampling". Local sampling (or lsample for short) is configured
in a similar way as current per-flow IPFIX sampling, i.e: using the
Flow_Sample_Collector_Set table and the NXAST_SAMPLE action.

However, instead of sending the sample to an external IPFIX collector
though the network, the sample is emitted using the new action and
made available to locally running sample collector.

The specific way emit_sample sends the sample (and the way the local
collector shall collect it) is datapath-specific.
Currently, currently only the Linux kernel datapath implements it using
the psample netlink multicast group.

~~ Configuration ~~
Local sampling is configured via a new column in the
Flow_Sample_Collector_Set (FSCS) table called "local_group_id".
Configuring this value is orthogonal to also associating the FSCS
entry to an entry in the IPFIX table.

Once that entry in the OVSDB is configured, NXAST_SAMPLE actions coming
from the controller will be translated into the following odp action:

   sample(sample={P}%, actions(emit_sample(group={G},cookie={C})))

Where:
    P: Is the sampling probability from NXAST_SAMPLE
    G: Is the group id in the FSCS entry whose "id" matches the one in
        the NXAST_SAMPLE.
    C: Is a 64bit cookie result of concatenating the obs_domain and
    obs_point from the NXAST_SAMPLE in network order, i.e:
        "htonl(obs_domain) << 32 | htonl(obs_point)"
Notes:
    - The parent sample action might be omitted if the probability is
      100% and there is no IPFIX sampling that requires the use of a
      meter.

~~ Dpif-lsample ~~
Internally, a new object called "dpif-lsample" is introduced to track
the configured local sampling exporters and track statistics based on
odp flow stats (using xcache).
It exposes the list of configured exporters and their statistics on a
new unixctl command called "lsample/show".

~~ Drop monitoring ~~
A common use-case for this action can be to sample drops. However,
adding sample actions to drops makes the existing drop statistics
disappear. In order to fix this, patches 11 and 12 make use of explicit
drop actions to ensure statistics still report drops even if sampled.

~~ Extended OpenFlow sample action ~~
Given the series aims at making sampling production ready, conntrack
integration must be considered. A common use-case for state-full
pipelines is to calculate the observation metadata at connection
establishment, store it in ct_label and then use it for packets of
established connections. However, this forces OVN to create a big number
of OFP Flows (one per distinct cookie). Patch 13 solves this by allowing
controllers to specify the obs_domain and point ids from another OFP
field.

~~ Testing ~~
The series includes an test utility program than can be executed by
running "tests/ovstest test-psample". This utility listens
to packets multicasted by the psample module and prints them (also
printing the obs_domain and obs_point ids).

~~ HW Offload ~~
tc offload is not being introduced in this series as existing sample
or userspace actions are not currently offloadable. Also some
improvements need to be implemented in tc for it to be feasible.

~~ DPDK datapath ~~
By naming the action "psample" it was intentionally restricted to the
Linux datapath only. A follow up task would be spawned to think of a
good way of implementing local-sampling in the userspace datapath.

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=aae0b82b46cb5004bdf82a000c004d69a0885c33

---
* rfc_v3 -> v1
- Add extra test to patch 13/13.
- Rebased.
- Removed RFC tag as kernel patches landed on net-next.

* rfc_v2 -> rfc_v3
- Added patches 11, 12 and 13 fixing drop monitoring and extending the
  OpenFlow action to support field-specifiers.

Adrian Moreno (13):
  ofproto-dpif: Allow forcing dp features.
  odp-util: Add support OVS_ACTION_ATTR_PSAMPLE.
  ofproto_dpif: Check for psample support.
  ofproto: Add ofproto-dpif-lsample.
  vswitchd: Add local sampling to vswitchd schema.
  ofproto-dpif-xlate: Use psample for local sample.
  ofproto-dpif-xlate-cache: Add lsample to xcache.
  ofproto-dpif-lsample: Show stats via unixctl.
  tests: Add test-psample testing utility.
  tests: Test local sampling.
  ofproto: xlate: Make flow-sampled drops explicit.
  ofproto: xlate: Make bridge-sampled drops explicit.
  ofp-actions: Load data from fields in sample action.

 NEWS                               |   6 +
 include/linux/automake.mk          |   1 +
 include/linux/openvswitch.h        |  28 +++
 include/linux/psample.h            |  68 ++++++
 include/openvswitch/ofp-actions.h  |   8 +-
 lib/dpif-netdev.c                  |   1 +
 lib/dpif.c                         |   8 +
 lib/dpif.h                         |   1 +
 lib/odp-execute.c                  |  25 +-
 lib/odp-util.c                     |  93 ++++++++
 lib/odp-util.h                     |   3 +
 lib/ofp-actions.c                  | 249 ++++++++++++++++++--
 ofproto/automake.mk                |   2 +
 ofproto/ofproto-dpif-ipfix.c       |   1 +
 ofproto/ofproto-dpif-lsample.c     | 332 ++++++++++++++++++++++++++
 ofproto/ofproto-dpif-lsample.h     |  46 ++++
 ofproto/ofproto-dpif-sflow.c       |   1 +
 ofproto/ofproto-dpif-xlate-cache.c |  11 +-
 ofproto/ofproto-dpif-xlate-cache.h |   6 +
 ofproto/ofproto-dpif-xlate.c       | 323 +++++++++++++++++++-------
 ofproto/ofproto-dpif-xlate.h       |   5 +-
 ofproto/ofproto-dpif.c             | 122 +++++++++-
 ofproto/ofproto-dpif.h             |   8 +-
 ofproto/ofproto-provider.h         |   9 +
 ofproto/ofproto.c                  |  12 +
 ofproto/ofproto.h                  |   8 +
 python/ovs/flow/odp.py             |   8 +
 python/ovs/flow/ofp.py             |   8 +-
 python/ovs/flow/ofp_act.py         |   4 +-
 tests/automake.mk                  |   3 +-
 tests/drop-stats.at                | 109 +++++++++
 tests/odp.at                       |  16 ++
 tests/ofp-actions.at               |   5 +
 tests/ofproto-dpif.at              | 235 ++++++++++++++++++-
 tests/system-common-macros.at      |   4 +
 tests/system-traffic.at            | 359 +++++++++++++++++++++++++++++
 tests/test-psample.c               | 284 +++++++++++++++++++++++
 vswitchd/bridge.c                  |  78 ++++++-
 vswitchd/vswitch.ovsschema         |   9 +-
 vswitchd/vswitch.xml               |  40 +++-
 40 files changed, 2398 insertions(+), 141 deletions(-)
 create mode 100644 include/linux/psample.h
 create mode 100644 ofproto/ofproto-dpif-lsample.c
 create mode 100644 ofproto/ofproto-dpif-lsample.h
 create mode 100644 tests/test-psample.c