@@ -28,6 +28,8 @@ DOC_SOURCE = \
Documentation/topics/ovn-news-2.8.rst \
Documentation/topics/role-based-access-control.rst \
Documentation/topics/debugging-ddlog.rst \
+ Documentation/topics/vif-plug-providers/index.rst \
+ Documentation/topics/vif-plug-providers/vif-plug-providers.rst \
Documentation/howto/index.rst \
Documentation/howto/docker.rst \
Documentation/howto/firewalld.rst \
@@ -41,6 +41,7 @@ OVN
high-availability
role-based-access-control
ovn-news-2.8
+ vif-plug-providers/index
testing
.. list-table::
new file mode 100644
@@ -0,0 +1,32 @@
+..
+ Licensed under the Apache License, Version 2.0 (the "License"); you may
+ not use this file except in compliance with the License. You may obtain
+ a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ License for the specific language governing permissions and limitations
+ under the License.
+
+ Convention for heading levels in OVN documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ~~~~~~~ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
+==================
+VIF Plug Providers
+==================
+
+
+.. toctree::
+ :maxdepth: 2
+
+ vif-plug-providers
new file mode 100644
@@ -0,0 +1,209 @@
+..
+ Licensed under the Apache License, Version 2.0 (the "License"); you may
+ not use this file except in compliance with the License. You may obtain
+ a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ License for the specific language governing permissions and limitations
+ under the License.
+
+ Convention for heading levels in OVN documentation:
+
+ ======= Heading 0 (reserved for the title in a document)
+ ------- Heading 1
+ ~~~~~~~ Heading 2
+ +++++++ Heading 3
+ ''''''' Heading 4
+
+ Avoid deeper levels because they do not render well.
+
+==================
+VIF Plug Providers
+==================
+
+Traditionally it has been the CMSes responsibility to create VIFs as part of
+instance life cycle, and subsequently manage plug/unplug operations on the
+integration bridge following the conventions described in the
+`Open vSwitch Integration Guide`_ for mapping of VIFs to OVN logical port.
+
+With the advent of NICs connected to multiple distinct CPUs we can have a
+topology where the instance runs on one host and Open vSwitch and OVN runs on
+a different host, the smartnic control plane CPU. The host facing interfaces
+will be visible to Open vSwitch and OVN as representor ports.
+
+The actions necessary for plugging and unplugging the representor port in
+Open vSwitch running on the smartnic control plane CPU would be the same for
+every CMS.
+
+Instead of every CMS having to develop their own version of an agent to do
+the plugging, we provide a pluggable infrastructure in OVN that allows the
+`ovn-controller` to perform the plugging on CMS direction.
+
+Hardware or platform specific details for initialization and lookup of
+representor ports is provided by an plugging provider library hosted inside or
+outside the core OVN repository, and linked at OVN build time.
+
+Life Cycle of an OVN plugged VIF
+--------------------------------
+
+1. CMS creates a record in the OVN Northbound Logical_Switch_Port table with
+ the options column containing the `vif-plug-type` key with a value
+ corresponding to the `const char *type` provided by the VIF plug provider
+ implementation as well as a `requested-chassis` key with a value pointing at
+ the name or hostname of the chassis it wants the VIF plugged on. Additional
+ VIF plug provider specific key/value pairs must be provided for successful
+ lookup.
+
+2. `ovn-northd` looks up the name or hostname provided in the
+ `requested-chassis` option and fills the OVN Southbound Port_Binding
+ requested_chassis column, it also copies relevant options over to the
+ Port_Binding record.
+
+3. `ovn-controller` monitors Southbound Port_Binding entries with a
+ requested_chassis column pointing at its chassis UUID. When it encounters
+ an entry with option `vif-plug-type` and it has registered a VIF plug
+ provider matching that type, it will act on it even if no local binding
+ exists yet.
+
+4. It will fill the `struct vif_plug_port_ctx_in` as defined in
+ `lib/vif-plug.h` with `op_type` set to 'PLUG_OP_CREATE' and make a call to
+ the VIF plug providers `vif_plug_port_prepare` callback function. VIF plug
+ provider performs lookup and fills the `struct vif_plug_port_ctx_out` as
+ defined in `lib/vif-plug.h`.
+
+5. `ovn-controller` creates a port and interface record in the local OVSDB
+ using the details provided by the VIF plug provider and also adds
+ `external-ids:iface-id` with value matching the logical port name and
+ `external-ids:ovn-plugged` with value matching the logical port
+ `vif-plug-type`. When the port creation is done a call will first be made
+ to the VIF plug providers `vif_plug_port_finish` function and then to the
+ `vif_plug_port_ctx_destroy` function to free any memory allocated by the VIF
+ plug implementation.
+
+6. The Open vSwitch vswitchd will assign a ofport to the newly created
+ interface and on the next `ovn-controller` main loop iteration flows will be
+ installed.
+
+7. On each main loop iteration the `ovn-controller` will in addition to normal
+ flow processing make a call to the VIF plug provider again similar to the
+ first creation in case anything needs updating for the interface record.
+
+8. The port will be unplugged when an event occurs which would make the
+ `ovn-controller` release a logical port, for example the Logical_Switch_Port
+ and Port_Binding entry disappearing from the database or its
+ `requested_chassis` column being pointed to a different chassis.
+
+
+The VIF plug provider interface
+-------------------------------
+
+The interface between internals of OVN and a VIF plug provider is a set of
+callbacks as defined by the `struct vif_plug_class` in
+`lib/vif-plug-provider.h`.
+
+It is important to note that these callbacks will be called in the critical
+path of the `ovn-controller` processing loop, so care must be taken to make the
+implementation as efficient as possible, and under no circumstance can any of
+the callback functions make calls that block.
+
+On `ovn-controller` startup, VIF plug providers made available at build time
+will be registered by the identifier provided in the `const char *type`
+pointer, at this time the `init` function pointer will be called if it is
+non-NULL.
+
+> **Note**: apart from the `const char *type` pointer, no attempt will be made
+ to access VIF plug provider data or functions before the call to
+ the `init` has been made.
+
+On `ovn-controller` exit, the VIF plug providers registered in the above
+mentioned procedure will have their `destroy` function pointer called if it is
+non-NULL.
+
+If the VIF plug provider has internal lookup tables that need to be maintained
+they can define a `run` function which will be called as part of the
+`ovn-controller` main loop. If there are any changes encountered the function
+should return 'true' to signal that further processing is necessary, 'false'
+otherwise.
+
+On update of Interface records the `ovn-controller` will pass on a `sset`
+to the `ovsport_update_iface` function containing options the plug
+implementation finds pertinent to maintain for successful operation. This
+`sset` is retrieved by making a call to the plug implementation
+`vif_plug_get_maintained_iface_options` function pointer if it is non-NULL.
+This allows presence of other users of the OVSDB maintaining a different set of
+options on the same set of Interface records without wiping out their changes.
+
+Before creating or updating an existing interface record the VIF plug provider
+`vif_plug_port_prepare` function pointer will be called with valid pointers to
+`struct vif_plug_port_ctx_in` and `struct vif_plug_port_ctx_out` data
+structures. If the VIF plug provider implementation is able to perform lookup
+it should fill the `struct vif_plug_port_ctx_out` data structure and return
+'true'. The `ovn-controller` will then create or update the port/interface
+records and then call `vif_plug_port_finish` when the transactions commits and
+`vif_plug_port_ctx_destroy` to free any allocated memory. If the VIF plug
+provider implementation is unable to perform lookup or prepare the desired
+resource at this time, it should return 'false' which will tell the
+`ovn-controller` to not plug the port, in this case it will not call
+`vif_plug_port_finish` nor `vif_plug_port_ctx_destroy`.
+
+> **Note**: The VIF plug provider implementation should exhaust all
+ non-blocking options to succeed with lookup from within the
+ `vif_plug_port_prepare` handler, including refreshing lookup
+ tables if necessary.
+
+Before removing port and interface records previously plugged by the
+`ovn-controller` as identified by presence of the Interface
+`external-ids:ovn-plugged` key, the `ovn-controller` will look up the
+`vif-plug-type` from `external-ids:ovn-plugged`, fill
+`struct vif_plug_port_ctx_in` with `op_type` set to 'PLUG_OP_REMOVE' and make a
+call to `vif_plug_port_prepare`. After the port and interface has been removed
+a call will be made to `vif_plug_port_finish`. Both calls will be made with
+the pointer to `vif_plug_port_ctx_out` set to 'NULL', and no call will be made
+to `vif_plug_port_ctx_destroy`.
+
+Building with in-tree VIF plug providers
+----------------------------------------
+
+VIF plug providers hosted in the OVN repository live under
+`lib/vif-plug-providers`:
+
+To enable them, provide the `--enable-vif-plug-providers` command line option
+to the configure script when building OVN.
+
+Building with an externally provided VIF plug provider
+------------------------------------------------------
+
+There is also infrastructure in place to support linking OVN with an externally
+built VIF plug provider.
+
+This external VIF plug provider must define a NULL-terminated array of pointers
+to `struct vif_plug_class` data structures named `vif_plug_provider_classes`.
+Example:
+
+.. code-block:: none
+
+ const struct vif_plug_class *vif_plug_provider_classes[] = {
+ &vif_plug_foo,
+ NULL,
+ };
+
+The name of the repository for the external VIF plug provider should be the
+same as the name of the library it produces, and the built library artifact
+should be placed in lib/.libs. Example:
+
+.. code-block:: none
+
+ ovn-vif-foo/
+ ovn-vif-foo/lib/.libs/libovn-vif-foo.la
+
+To enable such a VIF plug provider provide the
+`--with-vif-plug-provider=/path/to/ovn-vif-foo` command line option to the
+configure script when building OVN.
+
+.. LINKS
+.. _Open vSwitch Integration Guide:
+ https://docs.openvswitch.org/en/latest/topics/integration/
@@ -441,3 +441,52 @@ AC_DEFUN([OVN_CHECK_OVS], [
AC_MSG_CHECKING([OVS version])
AC_MSG_RESULT([$OVSVERSION])
])
+
+dnl OVN_CHECK_VIF_PLUG_PROVIDER
+dnl
+dnl Check for external VIF plug provider
+AC_DEFUN([OVN_CHECK_VIF_PLUG_PROVIDER], [
+ AC_ARG_VAR([VIF_PLUG_PROVIDER])
+ AC_ARG_WITH(
+ [vif-plug-provider],
+ [AC_HELP_STRING([--with-vif-plug-provider=/path/to/provider/repository],
+ [Specify path to a configured and built VIF plug provider repository])],
+ [if test "$withval" = yes; then
+ if test -z "$VIF_PLUG_PROVIDER"; then
+ AC_MSG_ERROR([To build with external VIF plug provider, specify the path to a configured and built plug provider repository --with-vif-plug-provider or in \$VIF_PLUG_PROVIDER]),
+ fi
+ VIF_PLUG_PROVIDER="$(realpath $VIF_PLUG_PROVIDER)"
+ else
+ VIF_PLUG_PROVIDER="$(realpath $withval)"
+ fi
+ _vif_plug_provider_name="$(basename $VIF_PLUG_PROVIDER)"
+ if test ! -f "$VIF_PLUG_PROVIDER/lib/.libs/lib${_vif_plug_provider_name}.la"; then
+ AC_MSG_ERROR([$withval is not a configured and built VIF plug provider library repository])
+ fi
+ VIF_PLUG_PROVIDER_LDFLAGS="-L$VIF_PLUG_PROVIDER/lib/.libs -l$_vif_plug_provider_name"
+ ],
+ [VIF_PLUG_PROVIDER=no])
+ AC_MSG_CHECKING([for VIF plug provider])
+ AC_MSG_RESULT([$VIF_PLUG_PROVIDER])
+ AC_SUBST([VIF_PLUG_PROVIDER_LDFLAGS])
+ AM_CONDITIONAL([HAVE_VIF_PLUG_PROVIDER], [test "$VIF_PLUG_PROVIDER" != no])
+ if test "$VIF_PLUG_PROVIDER" != no; then
+ AC_DEFINE([HAVE_VIF_PLUG_PROVIDER], [1],
+ [Build and link with external VIF plug provider])
+ fi
+])
+
+dnl OVN_ENABLE_VIF_PLUG
+dnl
+dnl Enable built-in plug providers
+AC_DEFUN([OVN_ENABLE_VIF_PLUG], [
+ AC_ARG_ENABLE(
+ [vif-plug-providers],
+ [AC_HELP_STRING([--enable-vif-plug-providers], [Enable building of built-in VIF plug providers])],
+ [], [enable_vif_plug=no])
+ AM_CONDITIONAL([ENABLE_VIF_PLUG], [test "$enable_vif_plug" != no])
+ if test "$enable_vif_plug" != no; then
+ AC_DEFINE([ENABLE_VIF_PLUG], [1],
+ [Build built-in VIF plug providers])
+ fi
+])
@@ -172,6 +172,8 @@ OVS_ENABLE_SPARSE
OVS_CHECK_DDLOG([0.47])
OVS_CHECK_PRAGMA_MESSAGE
OVN_CHECK_OVS
+OVN_CHECK_VIF_PLUG_PROVIDER
+OVN_ENABLE_VIF_PLUG
OVS_CTAGS_IDENTIFIERS
AC_SUBST([OVS_CFLAGS])
AC_SUBST([OVS_LDFLAGS])
@@ -37,7 +37,9 @@ controller_ovn_controller_SOURCES = \
controller/local_data.c \
controller/local_data.h \
controller/ovsport.h \
- controller/ovsport.c
+ controller/ovsport.c \
+ controller/vif-plug.h \
+ controller/vif-plug.c
controller_ovn_controller_LDADD = lib/libovn.la $(OVS_LIBDIR)/libopenvswitch.la
man_MANS += controller/ovn-controller.8
new file mode 100644
@@ -0,0 +1,72 @@
+/* Copyright (c) 2021, Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <config.h>
+#include <errno.h>
+
+#include "vif-plug.h"
+#include "vif-plug-provider.h"
+#include "smap.h"
+#include "sset.h"
+#include "tests/ovstest.h"
+
+static void
+test_vif_plug(struct ovs_cmdl_context *ctx OVS_UNUSED)
+{
+ const struct vif_plug_class *vif_plug_class;
+
+ ovs_assert(vif_plug_provider_unregister("dummy") == EINVAL);
+
+ ovs_assert(!vif_plug_provider_register(&vif_plug_dummy_class));
+ vif_plug_class = vif_plug_provider_get("dummy");
+ ovs_assert(vif_plug_provider_register(&vif_plug_dummy_class) == EEXIST);
+
+ ovs_assert(
+ sset_contains(
+ vif_plug_get_maintained_iface_options(vif_plug_class),
+ "plug-dummy-option"));
+
+ struct vif_plug_port_ctx_in ctx_in = {
+ .op_type = PLUG_OP_CREATE,
+ .lport_name = "lsp1",
+ .lport_options = SMAP_INITIALIZER(&ctx_in.lport_options),
+ };
+ struct vif_plug_port_ctx_out ctx_out;
+ vif_plug_port_prepare(vif_plug_class, &ctx_in, &ctx_out);
+ ovs_assert(!strcmp(ctx_out.name, "lsp1"));
+ ovs_assert(!strcmp(ctx_out.type, "internal"));
+ ovs_assert(!strcmp(smap_get(
+ &ctx_out.iface_options, "vif-plug-dummy-option"), "value"));
+
+ vif_plug_port_finish(vif_plug_class, &ctx_in, &ctx_out);
+ vif_plug_port_ctx_destroy(vif_plug_class, &ctx_in, &ctx_out);
+ vif_plug_provider_destroy_all();
+}
+
+static void
+test_vif_plug_main(int argc, char *argv[])
+{
+ set_program_name(argv[0]);
+ static const struct ovs_cmdl_command commands[] = {
+ {"run", NULL, 0, 0, test_vif_plug, OVS_RO},
+ {NULL, NULL, 0, 0, NULL, OVS_RO},
+ };
+ struct ovs_cmdl_context ctx;
+ ctx.argc = argc - 1;
+ ctx.argv = argv + 1;
+ ovs_cmdl_run_command(&ctx, commands);
+}
+
+OVSTEST_REGISTER("test-vif-plug", test_vif_plug_main);
new file mode 100644
@@ -0,0 +1,634 @@
+/*
+ * Copyright (c) 2021 Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <config.h>
+
+/* OVS includes */
+#include "lib/vswitch-idl.h"
+#include "openvswitch/shash.h"
+#include "openvswitch/vlog.h"
+
+/* OVN includes */
+#include "binding.h"
+#include "lib/ovn-sb-idl.h"
+#include "lport.h"
+#include "ovsport.h"
+#include "vif-plug.h"
+#include "vif-plug-provider.h"
+
+VLOG_DEFINE_THIS_MODULE(vif_plug);
+
+#define OVN_PLUGGED_EXT_ID "ovn-plugged"
+#define VIF_PLUG_OPTION_TYPE "vif-plug-type"
+#define VIF_PLUG_OPTION_MTU_REQUEST "vif-plug-mtu-request"
+
+void
+vif_plug_register_ovs_idl(struct ovsdb_idl *ovs_idl)
+{
+ ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_mtu_request);
+}
+
+/* Get the class level 'maintained_iface_options' set. */
+const struct sset *
+vif_plug_get_maintained_iface_options(
+ const struct vif_plug_class *vif_plug_class)
+{
+ return vif_plug_class->vif_plug_get_maintained_iface_options ?
+ vif_plug_class->vif_plug_get_maintained_iface_options() : NULL;
+}
+
+/* Prepare the logical port as identified by 'ctx_in' for port creation, update
+ * or removal as specified by 'ctx_in->op_type'.
+ *
+ * When 'ctx_in->op_type' is PLUG_OP_CREATE the plug implementation must fill
+ * 'ctx_out' with data to apply to the interface record maintained by OVN on
+ * its behalf.
+ *
+ * When 'ctx_in_op_type' is PLUG_OP_REMOVE 'ctx_out' should be set to NULL and
+ * the plug implementation must not attempt to use 'ctx_out'.
+ *
+ * The data in 'ctx_out' is owned by the plug implementation, and a call must
+ * be made to vif_plug_port_ctx_destroy when done with it. */
+bool
+vif_plug_port_prepare(const struct vif_plug_class *vif_plug_class,
+ const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out)
+{
+ return vif_plug_class->vif_plug_port_prepare(ctx_in, ctx_out);
+}
+
+/* Notify the VIF plug implementation that a port creation, update or removal
+ * has been committed to the database. */
+void
+vif_plug_port_finish(const struct vif_plug_class *vif_plug_class,
+ const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out)
+{
+ vif_plug_class->vif_plug_port_finish(ctx_in, ctx_out);
+}
+
+/* Free any data allocated to 'ctx_out' in a prevous call to
+ * vif_plug_port_prepare. */
+void
+vif_plug_port_ctx_destroy(const struct vif_plug_class *vif_plug_class,
+ const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out)
+{
+ vif_plug_class->vif_plug_port_ctx_destroy(ctx_in, ctx_out);
+}
+
+static struct vif_plug_port_ctx *
+build_port_ctx(const struct vif_plug_class *vif_plug,
+ const enum vif_plug_op_type op_type,
+ const struct vif_plug_ctx_in *vif_plug_ctx_in,
+ const struct sbrec_port_binding *pb,
+ const struct ovsrec_interface *iface,
+ const char *iface_id)
+{
+ struct vif_plug_port_ctx *new_ctx = xzalloc(
+ sizeof *new_ctx);
+
+ new_ctx->vif_plug = vif_plug;
+ new_ctx->vif_plug_port_ctx_in.op_type = op_type;
+ new_ctx->vif_plug_port_ctx_in.ovs_table = vif_plug_ctx_in->ovs_table;
+ new_ctx->vif_plug_port_ctx_in.br_int = vif_plug_ctx_in->br_int;
+ new_ctx->vif_plug_port_ctx_in.lport_name = pb ?
+ xstrdup(pb->logical_port) : iface_id ? xstrdup(iface_id) : NULL;
+ /* Prepare vif_plug_port_ctx_in smaps for use.
+ *
+ * Note that smap_init does not allocate memory. Any memory allocated by
+ * putting data into the vif_plug_port_ctx_in smaps will be destroyed by
+ * calls to smap_destroy in destroy_port_ctx */
+ smap_init(&new_ctx->vif_plug_port_ctx_in.lport_options);
+ smap_init(&new_ctx->vif_plug_port_ctx_in.iface_options);
+
+ if (pb) {
+ smap_clone(&new_ctx->vif_plug_port_ctx_in.lport_options,
+ &pb->options);
+ }
+
+ if (iface) {
+ new_ctx->vif_plug_port_ctx_in.iface_name = xstrdup(iface->name);
+ new_ctx->vif_plug_port_ctx_in.iface_type = xstrdup(iface->type);
+ smap_clone(&new_ctx->vif_plug_port_ctx_in.iface_options,
+ &iface->options);
+ }
+
+ /* Prepare vif_plug_port_ctx_out smaps for use.
+ *
+ * Note that smap_init does not allocate memory. Any memory allocated by
+ * putting data into the vif_plug_port_ctx_out smaps is the responsibility
+ * of the VIF plug provider through a call to vif_plug_port_ctx_destroy. */
+ smap_init(&new_ctx->vif_plug_port_ctx_out.iface_options);
+
+ return new_ctx;
+}
+
+static void
+destroy_port_ctx(struct vif_plug_port_ctx *ctx)
+{
+ smap_destroy(&ctx->vif_plug_port_ctx_in.lport_options);
+ smap_destroy(&ctx->vif_plug_port_ctx_in.iface_options);
+ if (ctx->vif_plug_port_ctx_in.lport_name) {
+ free((char *)ctx->vif_plug_port_ctx_in.lport_name);
+ }
+ if (ctx->vif_plug_port_ctx_in.iface_name) {
+ free((char *)ctx->vif_plug_port_ctx_in.iface_name);
+ }
+ if (ctx->vif_plug_port_ctx_in.iface_type) {
+ free((char *)ctx->vif_plug_port_ctx_in.iface_type);
+ }
+ /* Note that data associated with ctx->vif_plug_port_ctx_out must be
+ * destroyed by the plug provider implementation with a call to
+ * vif_plug_port_ctx_destroy prior to calling this function */
+ free(ctx);
+}
+
+/* Our contract with the VIF plug provider is that vif_plug_port_finish
+ * will be called with vif_plug_port_ctx_* objects once the transaction
+ * commits. To handle this we keep track of in-flight deletions
+ * and changes. The tracking data will be cleared after commit at the end of
+ * the ovn-controller main loop. */
+static void
+transact_delete_port(const struct vif_plug_ctx_in *vif_plug_ctx_in,
+ const struct vif_plug_ctx_out *vif_plug_ctx_out,
+ const struct vif_plug_port_ctx *vif_plug_port_ctx,
+ const struct ovsrec_port *port)
+{
+ shash_add(vif_plug_ctx_out->deleted_iface_ids,
+ vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name,
+ vif_plug_port_ctx);
+ ovsport_remove(vif_plug_ctx_in->br_int, port);
+}
+
+static void
+transact_create_port(const struct vif_plug_ctx_in *vif_plug_ctx_in,
+ const struct vif_plug_ctx_out *vif_plug_ctx_out,
+ const struct vif_plug_port_ctx *vif_plug_port_ctx,
+ const struct smap *iface_external_ids,
+ const int64_t mtu_request)
+{
+ shash_add(vif_plug_ctx_out->changed_iface_ids,
+ vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name,
+ vif_plug_port_ctx);
+ ovsport_create(vif_plug_ctx_in->ovs_idl_txn, vif_plug_ctx_in->br_int,
+ vif_plug_port_ctx->vif_plug_port_ctx_out.name,
+ vif_plug_port_ctx->vif_plug_port_ctx_out.type,
+ NULL, iface_external_ids,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out.iface_options,
+ mtu_request);
+}
+
+static void
+transact_update_port(const struct ovsrec_interface *iface_rec,
+ const struct vif_plug_ctx_in *vif_plug_ctx_in OVS_UNUSED,
+ const struct vif_plug_ctx_out *vif_plug_ctx_out,
+ const struct vif_plug_port_ctx *vif_plug_port_ctx,
+ const struct smap *iface_external_ids,
+ const int64_t mtu_request)
+{
+ shash_add(vif_plug_ctx_out->changed_iface_ids,
+ vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name,
+ vif_plug_port_ctx);
+ ovsport_update_iface(
+ iface_rec,
+ vif_plug_port_ctx->vif_plug_port_ctx_out.type,
+ iface_external_ids,
+ NULL,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out.iface_options,
+ vif_plug_get_maintained_iface_options(
+ vif_plug_port_ctx->vif_plug),
+ mtu_request);
+}
+
+
+static bool
+consider_unplug_iface(const struct ovsrec_interface *iface,
+ const struct sbrec_port_binding *pb,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out)
+{
+ const char *vif_plug_type = smap_get(&iface->external_ids,
+ OVN_PLUGGED_EXT_ID);
+ const char *iface_id = smap_get(&iface->external_ids, "iface-id");
+ const struct ovsrec_port *port = ovsport_lookup_by_interface(
+ vif_plug_ctx_in->ovsrec_port_by_interfaces,
+ (struct ovsrec_interface *) iface);
+
+ if (vif_plug_type && iface_id && port) {
+ const struct vif_plug_class *vif_plug;
+ if (!(vif_plug = vif_plug_provider_get(vif_plug_type))) {
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ VLOG_WARN_RL(&rl,
+ "Unable to open VIF plug provider for "
+ "%s %s iface-id %s",
+ VIF_PLUG_OPTION_TYPE, vif_plug_type, iface_id);
+ /* While we are unable to handle this, asking for a recompute
+ * will not change that fact. */
+ return true;
+ }
+ if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int
+ || !vif_plug_ctx_in->ovs_idl_txn)
+ {
+ /* Some of our prerequisites are not available, ask for a
+ * recompute. */
+ return false;
+ }
+
+ /* Our contract with the VIF plug provider is that vif_plug_port_finish
+ * will be called with a vif_plug_port_ctx_in object once the
+ * transaction commits.
+ *
+ * Since this happens asynchronously we need to allocate memory for
+ * and duplicate any database references so that they stay valid.
+ *
+ * The data is freed with a call to destroy_port_ctx after the
+ * transaction completes at the end of the ovn-controller main
+ * loop. */
+ struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx(
+ vif_plug, PLUG_OP_REMOVE, vif_plug_ctx_in, pb, iface, iface_id);
+
+ if (!vif_plug_port_prepare(vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ NULL)) {
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ VLOG_INFO_RL(&rl,
+ "Not unplugging iface %s (iface-id %s) on direction "
+ "from VIF plug provider.",
+ iface->name, iface_id);
+ destroy_port_ctx(vif_plug_port_ctx);
+ return true;
+ }
+ VLOG_INFO("Unplugging port %s from %s for iface-id %s on this "
+ "chassis.",
+ port->name,
+ vif_plug_ctx_in->br_int->name,
+ iface_id);
+
+ /* Add and track delete operation to the transaction */
+ transact_delete_port(vif_plug_ctx_in, vif_plug_ctx_out,
+ vif_plug_port_ctx, port);
+ return true;
+ }
+ return true;
+}
+
+static int64_t
+get_plug_mtu_request(const struct smap *lport_options)
+{
+ return smap_get_int(lport_options, VIF_PLUG_OPTION_MTU_REQUEST, 0);
+}
+
+static bool
+consider_plug_lport_create__(const struct vif_plug_class *vif_plug,
+ const struct smap *iface_external_ids,
+ const struct sbrec_port_binding *pb,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out)
+{
+ if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int
+ || !vif_plug_ctx_in->ovs_idl_txn) {
+ /* Some of our prerequisites are not available, ask for a recompute. */
+ return false;
+ }
+
+ /* Our contract with the VIF plug provider is that vif_plug_port_finish
+ * will be called with vif_plug_port_ctx_in and vif_plug_port_ctx_out
+ * objects once the transaction commits.
+ *
+ * Since this happens asynchronously we need to allocate memory for
+ * and duplicate any database references so that they stay valid.
+ *
+ * The data is freed with a call to destroy_port_ctx after the
+ * transaction completes at the end of the ovn-controller main
+ * loop. */
+ struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx(
+ vif_plug, PLUG_OP_CREATE, vif_plug_ctx_in, pb, NULL, NULL);
+
+ if (!vif_plug_port_prepare(vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out)) {
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ VLOG_INFO_RL(&rl,
+ "Not plugging lport %s on direction from VIF plug "
+ "provider.",
+ pb->logical_port);
+ destroy_port_ctx(vif_plug_port_ctx);
+ return true;
+ }
+
+ VLOG_INFO("Plugging port %s into %s for lport %s on this "
+ "chassis.",
+ vif_plug_port_ctx->vif_plug_port_ctx_out.name,
+ vif_plug_ctx_in->br_int->name,
+ pb->logical_port);
+ transact_create_port(vif_plug_ctx_in, vif_plug_ctx_out,
+ vif_plug_port_ctx,
+ iface_external_ids,
+ get_plug_mtu_request(&pb->options));
+ return true;
+}
+
+static bool
+consider_plug_lport_update__(const struct vif_plug_class *vif_plug,
+ const struct smap *iface_external_ids,
+ const struct sbrec_port_binding *pb,
+ struct local_binding *lbinding,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out)
+{
+ if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int
+ || !vif_plug_ctx_in->ovs_idl_txn) {
+ /* Some of our prerequisites are not available, ask for a recompute. */
+ return false;
+ }
+ /* Our contract with the VIF plug provider is that vif_plug_port_finish
+ * will be called with vif_plug_port_ctx_in and vif_plug_port_ctx_out
+ * objects once the transaction commits.
+ *
+ * Since this happens asynchronously we need to allocate memory for
+ * and duplicate any database references so that they stay valid.
+ *
+ * The data is freed with a call to destroy_port_ctx after the
+ * transaction completes at the end of the ovn-controller main
+ * loop. */
+ struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx(
+ vif_plug, PLUG_OP_CREATE, vif_plug_ctx_in, pb, NULL, NULL);
+
+ if (!vif_plug_port_prepare(vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out)) {
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ VLOG_INFO_RL(&rl,
+ "Not updating lport %s on direction from VIF plug "
+ "provider.",
+ pb->logical_port);
+ destroy_port_ctx(vif_plug_port_ctx);
+ return true;
+ }
+
+ if (strcmp(lbinding->iface->name,
+ vif_plug_port_ctx->vif_plug_port_ctx_out.name)) {
+ VLOG_WARN("Attempt of incompatible change to existing "
+ "port detected, please recreate port: %s",
+ pb->logical_port);
+ vif_plug_port_ctx_destroy(vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out);
+ destroy_port_ctx(vif_plug_port_ctx);
+ return false;
+ }
+ VLOG_DBG("updating iface for: %s", pb->logical_port);
+ transact_update_port(lbinding->iface, vif_plug_ctx_in, vif_plug_ctx_out,
+ vif_plug_port_ctx, iface_external_ids,
+ get_plug_mtu_request(&pb->options));
+
+ return true;
+}
+
+static bool
+consider_plug_lport(const struct sbrec_port_binding *pb,
+ struct local_binding *lbinding,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out)
+{
+ bool ret = true;
+ if (lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb)
+ && pb->requested_chassis == vif_plug_ctx_in->chassis_rec) {
+ const char *vif_plug_type = smap_get(&pb->options,
+ VIF_PLUG_OPTION_TYPE);
+ if (!vif_plug_type) {
+ /* Nothing for us to do and we don't need a recompute. */
+ return true;
+ }
+
+ static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
+ const struct vif_plug_class *vif_plug;
+ if (!(vif_plug = vif_plug_provider_get(vif_plug_type))) {
+ VLOG_WARN_RL(&rl,
+ "Unable to open VIF plug provider for %s: '%s' "
+ "lport %s",
+ VIF_PLUG_OPTION_TYPE,
+ vif_plug_type,
+ pb->logical_port);
+ /* While we are unable to handle this, asking for a recompute will
+ * not change that fact. */
+ return true;
+ }
+ const struct smap iface_external_ids = SMAP_CONST2(
+ &iface_external_ids,
+ OVN_PLUGGED_EXT_ID, vif_plug_type,
+ "iface-id", pb->logical_port);
+ if (lbinding && lbinding->iface) {
+ if (!smap_get(&lbinding->iface->external_ids,
+ OVN_PLUGGED_EXT_ID))
+ {
+ VLOG_WARN_RL(&rl,
+ "CMS requested plugging of lport %s, but a port "
+ "that is not maintained by OVN already exsist "
+ "in local vSwitch: "UUID_FMT,
+ pb->logical_port,
+ UUID_ARGS(&lbinding->iface->header_.uuid));
+ return false;
+ }
+ ret = consider_plug_lport_update__(vif_plug, &iface_external_ids,
+ pb, lbinding, vif_plug_ctx_in,
+ vif_plug_ctx_out);
+ } else {
+ ret = consider_plug_lport_create__(vif_plug, &iface_external_ids,
+ pb, vif_plug_ctx_in,
+ vif_plug_ctx_out);
+ }
+ }
+
+ return ret;
+}
+
+static bool
+vif_plug_iface_touched_this_txn(
+ const struct vif_plug_ctx_out *vif_plug_ctx_out,
+ const char *iface_id)
+{
+ return shash_find(vif_plug_ctx_out->changed_iface_ids, iface_id)
+ || shash_find(vif_plug_ctx_out->deleted_iface_ids, iface_id);
+}
+
+static bool
+vif_plug_handle_lport_vif(const struct sbrec_port_binding *pb,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out,
+ bool can_unplug)
+{
+ if (vif_plug_iface_touched_this_txn(vif_plug_ctx_out, pb->logical_port)) {
+ return true;
+ }
+ bool handled = true;
+ struct local_binding *lbinding = local_binding_find(
+ vif_plug_ctx_in->local_bindings, pb->logical_port);
+
+ if (lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb)) {
+ handled &= consider_plug_lport(pb, lbinding,
+ vif_plug_ctx_in, vif_plug_ctx_out);
+ } else if (can_unplug && lbinding && lbinding->iface) {
+ handled &= consider_unplug_iface(lbinding->iface, pb,
+ vif_plug_ctx_in, vif_plug_ctx_out);
+ }
+ return handled;
+}
+
+static bool
+vif_plug_handle_iface(const struct ovsrec_interface *iface_rec,
+ struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out,
+ bool can_unplug)
+{
+ bool handled = true;
+ const char *vif_plug_type = smap_get(&iface_rec->external_ids,
+ OVN_PLUGGED_EXT_ID);
+ const char *iface_id = smap_get(&iface_rec->external_ids, "iface-id");
+ if (!vif_plug_type || !iface_id
+ || vif_plug_iface_touched_this_txn(vif_plug_ctx_out, iface_id)) {
+ return true;
+ }
+ struct local_binding *lbinding = local_binding_find(
+ vif_plug_ctx_in->local_bindings, iface_id);
+ const struct sbrec_port_binding *pb = lport_lookup_by_name(
+ vif_plug_ctx_in->sbrec_port_binding_by_name, iface_id);
+ if (pb && lbinding
+ && lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb)) {
+ /* Something changed on a interface we have previously plugged,
+ * consider updating it */
+ handled &= consider_plug_lport(pb, lbinding,
+ vif_plug_ctx_in, vif_plug_ctx_out);
+ } else if (can_unplug
+ && (!pb
+ || !lport_can_bind_on_this_chassis(
+ vif_plug_ctx_in->chassis_rec, pb))) {
+ /* No lport for this interface or it is destined for different chassis,
+ * consuder unplugging it */
+ handled &= consider_unplug_iface(iface_rec, pb,
+ vif_plug_ctx_in, vif_plug_ctx_out);
+ }
+ return handled;
+}
+
+/* On initial startup or on IDL reconnect, several rounds of the main loop may
+ * run before data is actually loaded in the IDL. This situation is currently
+ * not reflected in a call to ovsdb_idl_has_ever_connected(). Until we find
+ * the root of this issue we need this counter so that we do not erronously
+ * unplug ports because the data is just not loaded yet.
+ */
+#define VIF_PLUG_PRIME_IDL_COUNT_SEEED 10
+static int vif_plug_prime_idl_count = VIF_PLUG_PRIME_IDL_COUNT_SEEED;
+
+void
+vif_plug_reset_idl_prime_counter(void)
+{
+ vif_plug_prime_idl_count = VIF_PLUG_PRIME_IDL_COUNT_SEEED;
+}
+
+void
+vif_plug_run(struct vif_plug_ctx_in *vif_plug_ctx_in,
+ struct vif_plug_ctx_out *vif_plug_ctx_out)
+{
+ if (vif_plug_prime_idl_count && --vif_plug_prime_idl_count > 0) {
+ VLOG_DBG("vif_plug_run: vif_plug_prime_idl_count=%d, will not unplug "
+ "ports in this iteration.", vif_plug_prime_idl_count);
+ }
+
+ if (!vif_plug_ctx_in->chassis_rec) {
+ return;
+ }
+ const struct ovsrec_interface *iface_rec;
+ OVSREC_INTERFACE_TABLE_FOR_EACH (iface_rec,
+ vif_plug_ctx_in->iface_table) {
+ vif_plug_handle_iface(iface_rec, vif_plug_ctx_in, vif_plug_ctx_out,
+ !vif_plug_prime_idl_count);
+ }
+
+ struct sbrec_port_binding *target =
+ sbrec_port_binding_index_init_row(
+ vif_plug_ctx_in->sbrec_port_binding_by_requested_chassis);
+ sbrec_port_binding_index_set_requested_chassis(
+ target,
+ vif_plug_ctx_in->chassis_rec);
+ const struct sbrec_port_binding *pb;
+ SBREC_PORT_BINDING_FOR_EACH_EQUAL (
+ pb, target,
+ vif_plug_ctx_in->sbrec_port_binding_by_requested_chassis) {
+ enum en_lport_type lport_type = get_lport_type(pb);
+ if (lport_type == LP_VIF) {
+ vif_plug_handle_lport_vif(pb, vif_plug_ctx_in, vif_plug_ctx_out,
+ !vif_plug_prime_idl_count);
+ }
+ }
+ sbrec_port_binding_index_destroy_row(target);
+}
+
+static void
+vif_plug_finish_deleted__(struct shash *deleted_iface_ids, bool txn_success)
+{
+ struct shash_node *node, *next;
+ SHASH_FOR_EACH_SAFE (node, next, deleted_iface_ids) {
+ struct vif_plug_port_ctx *vif_plug_port_ctx = node->data;
+ if (txn_success) {
+ vif_plug_port_finish(vif_plug_port_ctx->vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ NULL);
+ }
+ shash_delete(deleted_iface_ids, node);
+ destroy_port_ctx(vif_plug_port_ctx);
+ }
+}
+
+void
+vif_plug_clear_deleted(struct shash *deleted_iface_ids) {
+ vif_plug_finish_deleted__(deleted_iface_ids, false);
+}
+
+void
+vif_plug_finish_deleted(struct shash *deleted_iface_ids) {
+ vif_plug_finish_deleted__(deleted_iface_ids, true);
+}
+
+static void
+vif_plug_finish_changed__(struct shash *changed_iface_ids, bool txn_success)
+{
+ struct shash_node *node, *next;
+ SHASH_FOR_EACH_SAFE (node, next, changed_iface_ids) {
+ struct vif_plug_port_ctx *vif_plug_port_ctx = node->data;
+ if (txn_success) {
+ vif_plug_port_finish(vif_plug_port_ctx->vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out);
+ }
+ vif_plug_port_ctx_destroy(vif_plug_port_ctx->vif_plug,
+ &vif_plug_port_ctx->vif_plug_port_ctx_in,
+ &vif_plug_port_ctx->vif_plug_port_ctx_out);
+ shash_delete(changed_iface_ids, node);
+ destroy_port_ctx(vif_plug_port_ctx);
+ }
+}
+
+void
+vif_plug_clear_changed(struct shash *deleted_iface_ids) {
+ vif_plug_finish_changed__(deleted_iface_ids, false);
+}
+
+void
+vif_plug_finish_changed(struct shash *deleted_iface_ids) {
+ vif_plug_finish_changed__(deleted_iface_ids, true);
+}
new file mode 100644
@@ -0,0 +1,80 @@
+/*
+ * Copyright (c) 2021 Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef VIF_PLUG_H
+#define VIF_PLUG_H 1
+
+/*
+ * VIF Plug, the controller internal interface to the VIF plug provider
+ * infrastructure.
+ */
+
+#include "openvswitch/shash.h"
+#include "smap.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct vif_plug_ctx_in {
+ struct ovsdb_idl_txn *ovs_idl_txn;
+ struct ovsdb_idl_index *sbrec_port_binding_by_name;
+ struct ovsdb_idl_index *sbrec_port_binding_by_requested_chassis;
+ struct ovsdb_idl_index *ovsrec_port_by_interfaces;
+ const struct ovsrec_open_vswitch_table *ovs_table;
+ const struct ovsrec_bridge *br_int;
+ const struct ovsrec_interface_table *iface_table;
+ const struct sbrec_chassis *chassis_rec;
+ const struct shash *local_bindings;
+};
+
+struct vif_plug_ctx_out {
+ struct shash *deleted_iface_ids;
+ struct shash *changed_iface_ids;
+};
+
+struct vif_plug_class;
+struct vif_plug_port_ctx_out;
+struct vif_plug_port_ctx_in;
+
+const struct sset * vif_plug_get_maintained_iface_options(
+ const struct vif_plug_class *);
+
+bool vif_plug_port_prepare(const struct vif_plug_class *,
+ const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+void vif_plug_port_finish(const struct vif_plug_class *,
+ const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+void vif_plug_port_ctx_destroy(const struct vif_plug_class *,
+ const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+
+struct ovsdb_idl;
+
+void vif_plug_register_ovs_idl(struct ovsdb_idl *ovs_idl);
+void vif_plug_run(struct vif_plug_ctx_in *, struct vif_plug_ctx_out *);
+void vif_plug_clear_changed(struct shash *deleted_iface_ids);
+void vif_plug_finish_changed(struct shash *changed_iface_ids);
+void vif_plug_clear_deleted(struct shash *deleted_iface_ids);
+void vif_plug_finish_deleted(struct shash *changed_iface_ids);
+void vif_plug_reset_idl_prime_counter(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* vif-plug.h */
@@ -4,6 +4,11 @@ lib_libovn_la_LDFLAGS = \
-Wl,--version-script=$(top_builddir)/lib/libovn.sym \
$(OVS_LIBDIR)/libopenvswitch.la \
$(AM_LDFLAGS)
+
+if HAVE_VIF_PLUG_PROVIDER
+lib_libovn_la_LDFLAGS += $(VIF_PLUG_PROVIDER_LDFLAGS)
+endif
+
lib_libovn_la_SOURCES = \
lib/acl-log.c \
lib/acl-log.h \
@@ -33,7 +38,10 @@ lib_libovn_la_SOURCES = \
lib/inc-proc-eng.h \
lib/lb.c \
lib/lb.h \
- lib/stopwatch-names.h
+ lib/stopwatch-names.h \
+ lib/vif-plug-provider.h \
+ lib/vif-plug-provider.c \
+ lib/vif-plug-providers/dummy/vif-plug-dummy.c
nodist_lib_libovn_la_SOURCES = \
lib/ovn-dirs.c \
lib/ovn-nb-idl.c \
new file mode 100644
@@ -0,0 +1,204 @@
+/*
+ * Copyright (c) 2021 Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <config.h>
+#include "vif-plug-provider.h"
+
+#include <errno.h>
+#include <stdint.h>
+#include <string.h>
+
+#include "openvswitch/vlog.h"
+#include "openvswitch/shash.h"
+#include "smap.h"
+#include "sset.h"
+#include "lib/inc-proc-eng.h"
+
+VLOG_DEFINE_THIS_MODULE(vif_plug_provider);
+
+#ifdef ENABLE_VIF_PLUG
+static const struct vif_plug_class *base_vif_plug_classes[] = {
+};
+#endif
+
+static struct shash vif_plug_classes = SHASH_INITIALIZER(&vif_plug_classes);
+
+/* Protects the 'vif_plug_classes' shash. */
+static struct ovs_mutex vif_plug_classes_mutex = OVS_MUTEX_INITIALIZER;
+
+/* Initialize the the VIF plug infrastructure by registering known classes */
+void
+vif_plug_provider_initialize(void)
+{
+ static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER;
+
+ if (ovsthread_once_start(&once)) {
+#ifdef ENABLE_VIF_PLUG
+ /* Register built-in VIF plug provider classes */
+ for (int i = 0; i < ARRAY_SIZE(base_vif_plug_classes); i++) {
+ vif_plug_provider_register(base_vif_plug_classes[i]);
+ }
+#endif
+#ifdef HAVE_VIF_PLUG_PROVIDER
+ /* Register external VIF plug provider classes.
+ *
+ * Note that we cannot use the ARRAY_SIZE macro here as
+ * vif_plug_provider_classes is defined in external code which is not
+ * available at compile time. The convention is to use a
+ * NULL-terminated array instead. */
+ for (const struct vif_plug_class **pp = vif_plug_provider_classes;
+ pp && *pp;
+ pp++)
+ {
+ vif_plug_provider_register(*pp);
+ }
+#endif
+ ovsthread_once_done(&once);
+ }
+}
+
+static int
+vif_plug_provider_register__(const struct vif_plug_class *new_class)
+{
+ struct vif_plug_class *vif_plug_class;
+ int error;
+
+ if (shash_find(&vif_plug_classes, new_class->type)) {
+ VLOG_WARN("attempted to register duplicate VIF plug provider: %s",
+ new_class->type);
+ return EEXIST;
+ }
+
+ error = new_class->init ? new_class->init() : 0;
+ if (error) {
+ VLOG_WARN("failed to initialize %s VIF plug provider class: %s",
+ new_class->type, ovs_strerror(error));
+ return error;
+ }
+
+ vif_plug_class = xmalloc(sizeof *vif_plug_class);
+ memcpy(vif_plug_class, new_class, sizeof *vif_plug_class);
+
+ shash_add(&vif_plug_classes, new_class->type, vif_plug_class);
+
+ return 0;
+}
+
+/* Register the new VIF plug provider referred to in 'new_class' and perform
+ * any class level initialization as specified in its vif_plug_class. */
+int
+vif_plug_provider_register(const struct vif_plug_class *new_class)
+{
+ int error;
+
+ ovs_mutex_lock(&vif_plug_classes_mutex);
+ error = vif_plug_provider_register__(new_class);
+ ovs_mutex_unlock(&vif_plug_classes_mutex);
+
+ return error;
+}
+
+static int
+vif_plug_provider_unregister__(const char *type)
+{
+ int error;
+ struct shash_node *node;
+ struct vif_plug_class *vif_plug_class;
+
+ node = shash_find(&vif_plug_classes, type);
+ if (!node) {
+ return EINVAL;
+ }
+
+ vif_plug_class = node->data;
+ error = vif_plug_class->destroy ? vif_plug_class->destroy() : 0;
+ if (error) {
+ VLOG_WARN("failed to destroy %s VIF plug class: %s",
+ vif_plug_class->type, ovs_strerror(error));
+ return error;
+ }
+
+ shash_delete(&vif_plug_classes, node);
+ free(vif_plug_class);
+
+ return 0;
+}
+
+/* Unregister the VIF plug provider identified by 'type' and perform any class
+ * level de-initialization as specified in its vif_plug_class. */
+int
+vif_plug_provider_unregister(const char *type)
+{
+ int error;
+
+ ovs_mutex_lock(&vif_plug_classes_mutex);
+ error = vif_plug_provider_unregister__(type);
+ ovs_mutex_unlock(&vif_plug_classes_mutex);
+
+ return error;
+}
+
+/* Check whether there are any VIF plug providers registered */
+bool
+vif_plug_provider_has_providers(void)
+{
+ return !shash_is_empty(&vif_plug_classes);
+}
+
+const struct vif_plug_class *
+vif_plug_provider_get(const char *type)
+{
+ struct vif_plug_class *vif_plug_class;
+
+ ovs_mutex_lock(&vif_plug_classes_mutex);
+ vif_plug_class = shash_find_data(&vif_plug_classes, type);
+ ovs_mutex_unlock(&vif_plug_classes_mutex);
+
+ return vif_plug_class;
+}
+
+/* Iterate over VIF plug providers and call their run function.
+ *
+ * Returns 'true' if any of the providers run functions return 'true', 'false'
+ * otherwise.
+ *
+ * A return value of 'true' means that data has changed. */
+bool
+vif_plug_provider_run_all(void)
+{
+ struct shash_node *node, *next;
+ bool changed = false;
+
+ SHASH_FOR_EACH_SAFE (node, next, &vif_plug_classes) {
+ struct vif_plug_class *vif_plug_class = node->data;
+ if (vif_plug_class->run && vif_plug_class->run(vif_plug_class)) {
+ changed = true;
+ }
+ }
+ return changed;
+}
+
+/* De-initialize and unregister the VIF plug provider classes. */
+void
+vif_plug_provider_destroy_all(void)
+{
+ struct shash_node *node, *next;
+
+ SHASH_FOR_EACH_SAFE (node, next, &vif_plug_classes) {
+ struct vif_plug_class *vif_plug_class = node->data;
+ vif_plug_provider_unregister(vif_plug_class->type);
+ }
+}
new file mode 100644
@@ -0,0 +1,163 @@
+/*
+ * Copyright (c) 2021 Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#ifndef VIF_PLUG_PROVIDER_H
+#define VIF_PLUG_PROVIDER_H 1
+
+/* Interface for VIF plug providers.
+ *
+ * A VIF plug provider implementation performs lookup and/or initialization of
+ * ports, typically representor ports, using generic non-blocking hardware
+ * interfaces. This allows the ovn-controller to, upon the CMS's request,
+ * create ports and interfaces in the chassis's Open vSwitch instances (also
+ * known as vif plugging).
+ *
+ * This module contains the infrastructure for registering VIF plug providers
+ * which may be hosted inside or outside the core OVN repository.
+ */
+
+#include <stdbool.h>
+
+#include "smap.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct vif_plug_class;
+struct ovsdb_idl_txn;
+struct ovsrec_bridge;
+
+enum vif_plug_op_type {
+ PLUG_OP_CREATE = 1, /* Port is created or updated */
+ PLUG_OP_REMOVE, /* Port is removed from this chassis */
+};
+
+struct vif_plug_port_ctx_in {
+ /* Operation being performed */
+ enum vif_plug_op_type op_type;
+
+ /* These are provided so that the plug implementation may make decisions
+ * based on environmental factors such as settings in the open-vswitch
+ * table and datapath type settings on the integration bridge. */
+ const struct ovsrec_open_vswitch_table *ovs_table;
+ const struct ovsrec_bridge *br_int;
+
+ /* Name of logical port, can be useful for plugging library to track any
+ * per port resource initialization. */
+ const char *lport_name;
+
+ /* Logical port options, while OVN will forward the contents verbatim from
+ * the Southbound database, the convention is for the plugging library to
+ * only make decisions based on the vif-plug-* options. */
+ struct smap lport_options;
+
+ /* When OVN knows about an existing interface record associated with this
+ * lport, these will be filled in with information about it. */
+ const char *iface_name;
+ const char *iface_type;
+ struct smap iface_options;
+};
+
+struct vif_plug_port_ctx_out {
+ /* The name to use for port and interface record. */
+ char *name;
+
+ /* Type of interface to create. */
+ char *type;
+
+ /* Options to set on the interface record. */
+ struct smap iface_options;
+};
+
+struct vif_plug_port_ctx {
+ const struct vif_plug_class *vif_plug;
+ struct vif_plug_port_ctx_in vif_plug_port_ctx_in;
+ struct vif_plug_port_ctx_out vif_plug_port_ctx_out;
+};
+
+struct vif_plug_class {
+ /* Type of VIF plug provider in this class. */
+ const char *type;
+
+ /* Called when the VIF plug provider is registered, typically at program
+ * startup.
+ *
+ * This function may be set to null if a VIF plug class needs no
+ * initialization at registration time. */
+ int (*init)(void);
+
+ /* Called when the VIF plug provider is unregistered, typically at program
+ * exit.
+ *
+ * This function may be set to null if a plug class needs no
+ * de-initialization at unregister time.*/
+ int (*destroy)(void);
+
+ /* Performs periodic work needed by VIF plug provider, if any is necessary.
+ * Returns 'true; if anything (i.e. lookup tables) changed, 'false'
+ * otherwise. */
+ bool (*run)(struct vif_plug_class *);
+
+ /* Retrieve Interface options this VIF plug provider will maintain. This
+ * sset is used to know which items to remove when maintaining the database
+ * record. */
+ const struct sset * (*vif_plug_get_maintained_iface_options)(void);
+
+ /* Pass vif_plug_port_ctx_in to VIF plug provider implementation to prepare
+ * for port creation/update.
+ *
+ * The VIF plug provider implemantation can perform lookup or any per port
+ * initialization and should fill vif_plug_port_ctx_out with data required
+ * for port/interface creation. The VIF plug implementation should return
+ * 'true' if it wants the caller to create/update a port/interface, 'false'
+ * otherwise.
+ *
+ * Data in the vif_plug_port_ctx_out struct is owned by the VIF plug
+ * provider, and a call must be made to the vif_plug_port_ctx_destroy
+ * callback to free up any allocations when done with port creation/update.
+ */
+ bool (*vif_plug_port_prepare)(const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+
+ /* Notify VIF plug provider that port update is committed to OVSDB. */
+ void (*vif_plug_port_finish)(const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+
+ /* Free any allocations made by the vif_plug_port_prepare callback. */
+ void (*vif_plug_port_ctx_destroy)(const struct vif_plug_port_ctx_in *,
+ struct vif_plug_port_ctx_out *);
+};
+
+extern const struct vif_plug_class vif_plug_dummy_class;
+#ifdef HAVE_VIF_PLUG_PROVIDER
+extern const struct vif_plug_class *vif_plug_provider_classes[];
+#endif
+
+void vif_plug_provider_initialize(void);
+int vif_plug_provider_register(const struct vif_plug_class *);
+int vif_plug_provider_unregister(const char *type);
+bool vif_plug_provider_has_providers(void);
+const struct vif_plug_class * vif_plug_provider_get(const char *);
+bool vif_plug_provider_run_all(void);
+void vif_plug_provider_destroy_all(void);
+void vif_plug_dummy_enable(void);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* vif-plug-provider.h */
new file mode 100644
@@ -0,0 +1,120 @@
+/*
+ * Copyright (c) 2021 Canonical
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <config.h>
+#include "lib/vif-plug-provider.h"
+
+#include <stdint.h>
+
+#include "openvswitch/vlog.h"
+#include "smap.h"
+#include "sset.h"
+
+#ifndef IFNAMSIZ
+#define IFNAMSIZ 16
+#endif
+
+VLOG_DEFINE_THIS_MODULE(vif_plug_dummy);
+
+static struct sset vif_plug_dummy_maintained_iface_options;
+
+static int
+vif_plug_dummy_init(void)
+{
+ sset_init(&vif_plug_dummy_maintained_iface_options);
+ sset_add(&vif_plug_dummy_maintained_iface_options,
+ "vif-plug-dummy-option");
+
+ return 0;
+}
+
+static int
+vif_plug_dummy_destroy(void)
+{
+ sset_destroy(&vif_plug_dummy_maintained_iface_options);
+
+ return 0;
+}
+
+static const struct sset*
+vif_plug_dummy_get_maintained_iface_options(void)
+{
+ return &vif_plug_dummy_maintained_iface_options;
+}
+
+static bool
+vif_plug_dummy_run(struct vif_plug_class *plug)
+{
+ VLOG_DBG("vif_plug_dummy_run(%p)", plug);
+
+ return false;
+}
+
+static bool
+vif_plug_dummy_port_prepare(const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out)
+{
+ VLOG_DBG("vif_plug_dummy_port_prepare: %s", ctx_in->lport_name);
+
+ if (ctx_in->op_type == PLUG_OP_CREATE) {
+ size_t lport_name_len = strlen(ctx_in->lport_name);
+ ctx_out->name = xzalloc(IFNAMSIZ);
+ memcpy(ctx_out->name, ctx_in->lport_name,
+ (lport_name_len < IFNAMSIZ) ? lport_name_len : IFNAMSIZ - 1);
+ ctx_out->type = xstrdup("internal");
+ smap_init(&ctx_out->iface_options);
+ smap_add(&ctx_out->iface_options, "vif-plug-dummy-option", "value");
+ }
+
+ return true;
+}
+
+static void
+vif_plug_dummy_port_finish(const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out OVS_UNUSED)
+{
+ VLOG_DBG("vif_plug_dummy_port_finish: %s", ctx_in->lport_name);
+}
+
+static void
+vif_plug_dummy_port_ctx_destroy(const struct vif_plug_port_ctx_in *ctx_in,
+ struct vif_plug_port_ctx_out *ctx_out)
+{
+ VLOG_DBG("vif_plug_dummy_port_ctx_destroy: %s", ctx_in->lport_name);
+ ovs_assert(ctx_in->op_type == PLUG_OP_CREATE);
+ free(ctx_out->name);
+ free(ctx_out->type);
+ smap_destroy(&ctx_out->iface_options);
+}
+
+const struct vif_plug_class vif_plug_dummy_class = {
+ .type = "dummy",
+ .init = vif_plug_dummy_init,
+ .destroy = vif_plug_dummy_destroy,
+ .vif_plug_get_maintained_iface_options =
+ vif_plug_dummy_get_maintained_iface_options,
+ .run = vif_plug_dummy_run,
+ .vif_plug_port_prepare = vif_plug_dummy_port_prepare,
+ .vif_plug_port_finish = vif_plug_dummy_port_finish,
+ .vif_plug_port_ctx_destroy = vif_plug_dummy_port_ctx_destroy,
+};
+
+void
+vif_plug_dummy_enable(void)
+{
+ vif_plug_provider_register(&vif_plug_dummy_class);
+}
+
@@ -67,8 +67,9 @@
<li>
One or more (usually many) <dfn>hypervisors</dfn>. Hypervisors must run
Open vSwitch and implement the interface described in
- <code>Documentation/topics/integration.rst</code> in the OVN source tree.
- Any hypervisor platform supported by Open vSwitch is acceptable.
+ <code>Documentation/topics/integration.rst</code> in the Open vSwitch
+ source tree. Any hypervisor platform supported by Open vSwitch is
+ acceptable.
</li>
<li>
@@ -318,11 +319,19 @@
<li>
On a hypervisor, any VIFs that are to be attached to logical networks.
- The hypervisor itself, or the integration between Open vSwitch and the
- hypervisor (described in
- <code>Documentation/topics/integration.rst</code>) takes care of this.
- (This is not part of OVN or new to OVN; this is pre-existing integration
- work that has already been done on hypervisors that support OVS.)
+ For instances connected through software emulated ports such as TUN/TAP
+ or VETH pairs, the hypervisor itself will normally create ports and plug
+ them into the integration bridge. For instances connected through
+ representor ports, typically used with hardware offload, the
+ <code>ovn-controller</code> may on CMS direction consult a VIF plug
+ provider for representor port lookup and plug them into the integration
+ bridge (please refer to
+ <code>Documentation/topics/vif-plug-providers/vif-plug-providers.rst
+ </code> for more information). In both cases the conventions described
+ in <code>Documentation/topics/integration.rst</code> in the Open vSwitch
+ source tree is followed to ensure mapping between OVN logical port and
+ VIF. (This is pre-existing integration work that has already been done
+ on hypervisors that support OVS.)
</li>
<li>
@@ -921,12 +930,12 @@
Eventually, a user powers on the VM that owns the VIF. On the hypervisor
where the VM is powered on, the integration between the hypervisor and
Open vSwitch (described in
- <code>Documentation/topics/integration.rst</code>) adds the VIF to the OVN
- integration bridge and stores <var>vif-id</var> in
- <code>external_ids</code>:<code>iface-id</code> to indicate that the
- interface is an instantiation of the new VIF. (None of this code is new
- in OVN; this is pre-existing integration work that has already been done
- on hypervisors that support OVS.)
+ <code>Documentation/topics/integration.rst</code> in the Open vSwitch
+ source tree) adds the VIF to the OVN integration bridge and stores
+ <var>vif-id</var> in <code>external_ids</code>:<code>iface-id</code> to
+ indicate that the interface is an instantiation of the new VIF. (None of
+ this code is new in OVN; this is pre-existing integration work that has
+ already been done on hypervisors that support OVS.)
</li>
<li>
@@ -38,7 +38,8 @@ TESTSUITE_AT = \
tests/ovn-ipam.at \
tests/ovn-features.at \
tests/ovn-lflow-cache.at \
- tests/ovn-ipsec.at
+ tests/ovn-ipsec.at \
+ tests/ovn-vif-plug.at
SYSTEM_KMOD_TESTSUITE_AT = \
tests/system-common-macros.at \
@@ -243,13 +244,23 @@ tests_ovstest_SOURCES = \
tests/test-ovn.c \
controller/test-lflow-cache.c \
controller/test-ofctrl-seqno.c \
+ controller/test-vif-plug.c \
lib/test-ovn-features.c \
northd/test-ipam.c
tests_ovstest_LDADD = $(OVS_LIBDIR)/daemon.lo \
$(OVS_LIBDIR)/libopenvswitch.la lib/libovn.la \
+ controller/binding.$(OBJEXT) \
+ controller/encaps.$(OBJEXT) \
+ controller/ha-chassis.$(OBJEXT) \
+ controller/if-status.$(OBJEXT) \
controller/lflow-cache.$(OBJEXT) \
+ controller/local_data.$(OBJEXT) \
+ controller/lport.$(OBJEXT) \
controller/ofctrl-seqno.$(OBJEXT) \
+ controller/ovsport.$(OBJEXT) \
+ controller/patch.$(OBJEXT) \
+ controller/vif-plug.$(OBJEXT) \
northd/ipam.$(OBJEXT)
# Python tests.
new file mode 100644
@@ -0,0 +1,8 @@
+#
+# Unit tests for the lib/vif-plug-provider.c and controller/vif-plug.c modules.
+#
+AT_BANNER([OVN unit tests - vif-plug])
+
+AT_SETUP([unit test -- plugging infrastructure tests])
+AT_CHECK([ovstest test-plug run], [0], [])
+AT_CLEANUP
New lib/vif-plug-provider module contains the infrastructure for registering VIF plug provider classes which may be hosted inside or outside the core OVN repository. New controller/vif-plug module adds internal interface for interacting with the VIF plug providers. Extend build system to allow building of built-in VIF plug providers and linking an externally built VIF plug provider. Signed-off-by: Frode Nordahl <frode.nordahl@canonical.com> --- Documentation/automake.mk | 2 + Documentation/topics/index.rst | 1 + .../topics/vif-plug-providers/index.rst | 32 + .../vif-plug-providers/vif-plug-providers.rst | 209 ++++++ acinclude.m4 | 49 ++ configure.ac | 2 + controller/automake.mk | 4 +- controller/test-vif-plug.c | 72 ++ controller/vif-plug.c | 634 ++++++++++++++++++ controller/vif-plug.h | 80 +++ lib/automake.mk | 10 +- lib/vif-plug-provider.c | 204 ++++++ lib/vif-plug-provider.h | 163 +++++ lib/vif-plug-providers/dummy/vif-plug-dummy.c | 120 ++++ ovn-architecture.7.xml | 35 +- tests/automake.mk | 13 +- tests/ovn-vif-plug.at | 8 + 17 files changed, 1622 insertions(+), 16 deletions(-) create mode 100644 Documentation/topics/vif-plug-providers/index.rst create mode 100644 Documentation/topics/vif-plug-providers/vif-plug-providers.rst create mode 100644 controller/test-vif-plug.c create mode 100644 controller/vif-plug.c create mode 100644 controller/vif-plug.h create mode 100644 lib/vif-plug-provider.c create mode 100644 lib/vif-plug-provider.h create mode 100644 lib/vif-plug-providers/dummy/vif-plug-dummy.c create mode 100644 tests/ovn-vif-plug.at