From patchwork Fri Nov 5 14:00:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1551399 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=b0UuFlIx; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hm2GH4Dvsz9sXS for ; Sat, 6 Nov 2021 01:00:39 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id CA8368284B; Fri, 5 Nov 2021 14:00:36 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ytfR3clCNwAp; Fri, 5 Nov 2021 14:00:27 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4B6C2827C6; Fri, 5 Nov 2021 14:00:26 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 31BF9C0044; Fri, 5 Nov 2021 14:00:23 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A981C000E for ; Fri, 5 Nov 2021 14:00:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 0A8D040775 for ; Fri, 5 Nov 2021 14:00:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XjxWiH5divcR for ; Fri, 5 Nov 2021 14:00:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by smtp4.osuosl.org (Postfix) with ESMTPS id 611454076C for ; Fri, 5 Nov 2021 14:00:17 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 851CC3F1D4; Fri, 5 Nov 2021 14:00:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636120815; bh=PmstvvPxbpfC52KLC8CjEXK5JYGUPp5GBb/m660uNRY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b0UuFlIxQrHRcfHU6RlG0EqiEJdTzT+bfMF+X0MbVMLRQsOEj8GIdHbbpmoIZv1go QjwNeZf4IKh9JQIJuDBBEGD5L0RCbnY4JpRtPzcw37lpNYqhcnnRwIXvrHSFzErjnj HFQMyHBqn8UZp3uNl5JmAtXPCcy3y4BlEGEEkpbjeiLVtYZw+zZAEC8qFZTwm8b8G8 dJl6Wb6xMt3RRBk7BsIHa6z5rPYarLxAt5AQ0xr8g67/6ktmRU6Hq+BLMLnPBLDbzl RseWuqegCvlQw3XNp0KurXcgg8GZAvBChVV1/ioWZmidgLg7M7oxz4vnckPzaQmcOK kmiUeTwBLI8Nw== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 5 Nov 2021 15:00:11 +0100 Message-Id: <20211105140014.3522766-2-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211105140014.3522766-1-frode.nordahl@canonical.com> References: <20211105140014.3522766-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v10 1/4] lib: Add infrastructure for VIF plug providers. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" New lib/vif-plug-provider module contains the infrastructure for registering VIF plug provider classes which may be hosted inside or outside the core OVN repository. New controller/vif-plug module adds internal interface for interacting with the VIF plug providers. Extend build system to allow building of built-in VIF plug providers and linking an externally built VIF plug provider. Signed-off-by: Frode Nordahl --- Documentation/automake.mk | 2 + Documentation/topics/index.rst | 1 + .../topics/vif-plug-providers/index.rst | 32 + .../vif-plug-providers/vif-plug-providers.rst | 209 ++++++ acinclude.m4 | 49 ++ configure.ac | 2 + controller/automake.mk | 4 +- controller/test-vif-plug.c | 72 ++ controller/vif-plug.c | 634 ++++++++++++++++++ controller/vif-plug.h | 80 +++ lib/automake.mk | 10 +- lib/vif-plug-provider.c | 204 ++++++ lib/vif-plug-provider.h | 163 +++++ lib/vif-plug-providers/dummy/vif-plug-dummy.c | 120 ++++ ovn-architecture.7.xml | 35 +- tests/automake.mk | 13 +- tests/ovn-vif-plug.at | 8 + 17 files changed, 1622 insertions(+), 16 deletions(-) create mode 100644 Documentation/topics/vif-plug-providers/index.rst create mode 100644 Documentation/topics/vif-plug-providers/vif-plug-providers.rst create mode 100644 controller/test-vif-plug.c create mode 100644 controller/vif-plug.c create mode 100644 controller/vif-plug.h create mode 100644 lib/vif-plug-provider.c create mode 100644 lib/vif-plug-provider.h create mode 100644 lib/vif-plug-providers/dummy/vif-plug-dummy.c create mode 100644 tests/ovn-vif-plug.at diff --git a/Documentation/automake.mk b/Documentation/automake.mk index b3fd3d62b..704c80671 100644 --- a/Documentation/automake.mk +++ b/Documentation/automake.mk @@ -28,6 +28,8 @@ DOC_SOURCE = \ Documentation/topics/ovn-news-2.8.rst \ Documentation/topics/role-based-access-control.rst \ Documentation/topics/debugging-ddlog.rst \ + Documentation/topics/vif-plug-providers/index.rst \ + Documentation/topics/vif-plug-providers/vif-plug-providers.rst \ Documentation/howto/index.rst \ Documentation/howto/docker.rst \ Documentation/howto/firewalld.rst \ diff --git a/Documentation/topics/index.rst b/Documentation/topics/index.rst index d58d5618b..e9e49c742 100644 --- a/Documentation/topics/index.rst +++ b/Documentation/topics/index.rst @@ -41,6 +41,7 @@ OVN high-availability role-based-access-control ovn-news-2.8 + vif-plug-providers/index testing .. list-table:: diff --git a/Documentation/topics/vif-plug-providers/index.rst b/Documentation/topics/vif-plug-providers/index.rst new file mode 100644 index 000000000..b7552ac4c --- /dev/null +++ b/Documentation/topics/vif-plug-providers/index.rst @@ -0,0 +1,32 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================== +VIF Plug Providers +================== + + +.. toctree:: + :maxdepth: 2 + + vif-plug-providers diff --git a/Documentation/topics/vif-plug-providers/vif-plug-providers.rst b/Documentation/topics/vif-plug-providers/vif-plug-providers.rst new file mode 100644 index 000000000..77ecf7e0f --- /dev/null +++ b/Documentation/topics/vif-plug-providers/vif-plug-providers.rst @@ -0,0 +1,209 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + Convention for heading levels in OVN documentation: + + ======= Heading 0 (reserved for the title in a document) + ------- Heading 1 + ~~~~~~~ Heading 2 + +++++++ Heading 3 + ''''''' Heading 4 + + Avoid deeper levels because they do not render well. + +================== +VIF Plug Providers +================== + +Traditionally it has been the CMSes responsibility to create VIFs as part of +instance life cycle, and subsequently manage plug/unplug operations on the +integration bridge following the conventions described in the +`Open vSwitch Integration Guide`_ for mapping of VIFs to OVN logical port. + +With the advent of NICs connected to multiple distinct CPUs we can have a +topology where the instance runs on one host and Open vSwitch and OVN runs on +a different host, the smartnic control plane CPU. The host facing interfaces +will be visible to Open vSwitch and OVN as representor ports. + +The actions necessary for plugging and unplugging the representor port in +Open vSwitch running on the smartnic control plane CPU would be the same for +every CMS. + +Instead of every CMS having to develop their own version of an agent to do +the plugging, we provide a pluggable infrastructure in OVN that allows the +`ovn-controller` to perform the plugging on CMS direction. + +Hardware or platform specific details for initialization and lookup of +representor ports is provided by an plugging provider library hosted inside or +outside the core OVN repository, and linked at OVN build time. + +Life Cycle of an OVN plugged VIF +-------------------------------- + +1. CMS creates a record in the OVN Northbound Logical_Switch_Port table with + the options column containing the `vif-plug-type` key with a value + corresponding to the `const char *type` provided by the VIF plug provider + implementation as well as a `requested-chassis` key with a value pointing at + the name or hostname of the chassis it wants the VIF plugged on. Additional + VIF plug provider specific key/value pairs must be provided for successful + lookup. + +2. `ovn-northd` looks up the name or hostname provided in the + `requested-chassis` option and fills the OVN Southbound Port_Binding + requested_chassis column, it also copies relevant options over to the + Port_Binding record. + +3. `ovn-controller` monitors Southbound Port_Binding entries with a + requested_chassis column pointing at its chassis UUID. When it encounters + an entry with option `vif-plug-type` and it has registered a VIF plug + provider matching that type, it will act on it even if no local binding + exists yet. + +4. It will fill the `struct vif_plug_port_ctx_in` as defined in + `lib/vif-plug.h` with `op_type` set to 'PLUG_OP_CREATE' and make a call to + the VIF plug providers `vif_plug_port_prepare` callback function. VIF plug + provider performs lookup and fills the `struct vif_plug_port_ctx_out` as + defined in `lib/vif-plug.h`. + +5. `ovn-controller` creates a port and interface record in the local OVSDB + using the details provided by the VIF plug provider and also adds + `external-ids:iface-id` with value matching the logical port name and + `external-ids:ovn-plugged` with value matching the logical port + `vif-plug-type`. When the port creation is done a call will first be made + to the VIF plug providers `vif_plug_port_finish` function and then to the + `vif_plug_port_ctx_destroy` function to free any memory allocated by the VIF + plug implementation. + +6. The Open vSwitch vswitchd will assign a ofport to the newly created + interface and on the next `ovn-controller` main loop iteration flows will be + installed. + +7. On each main loop iteration the `ovn-controller` will in addition to normal + flow processing make a call to the VIF plug provider again similar to the + first creation in case anything needs updating for the interface record. + +8. The port will be unplugged when an event occurs which would make the + `ovn-controller` release a logical port, for example the Logical_Switch_Port + and Port_Binding entry disappearing from the database or its + `requested_chassis` column being pointed to a different chassis. + + +The VIF plug provider interface +------------------------------- + +The interface between internals of OVN and a VIF plug provider is a set of +callbacks as defined by the `struct vif_plug_class` in +`lib/vif-plug-provider.h`. + +It is important to note that these callbacks will be called in the critical +path of the `ovn-controller` processing loop, so care must be taken to make the +implementation as efficient as possible, and under no circumstance can any of +the callback functions make calls that block. + +On `ovn-controller` startup, VIF plug providers made available at build time +will be registered by the identifier provided in the `const char *type` +pointer, at this time the `init` function pointer will be called if it is +non-NULL. + +> **Note**: apart from the `const char *type` pointer, no attempt will be made + to access VIF plug provider data or functions before the call to + the `init` has been made. + +On `ovn-controller` exit, the VIF plug providers registered in the above +mentioned procedure will have their `destroy` function pointer called if it is +non-NULL. + +If the VIF plug provider has internal lookup tables that need to be maintained +they can define a `run` function which will be called as part of the +`ovn-controller` main loop. If there are any changes encountered the function +should return 'true' to signal that further processing is necessary, 'false' +otherwise. + +On update of Interface records the `ovn-controller` will pass on a `sset` +to the `ovsport_update_iface` function containing options the plug +implementation finds pertinent to maintain for successful operation. This +`sset` is retrieved by making a call to the plug implementation +`vif_plug_get_maintained_iface_options` function pointer if it is non-NULL. +This allows presence of other users of the OVSDB maintaining a different set of +options on the same set of Interface records without wiping out their changes. + +Before creating or updating an existing interface record the VIF plug provider +`vif_plug_port_prepare` function pointer will be called with valid pointers to +`struct vif_plug_port_ctx_in` and `struct vif_plug_port_ctx_out` data +structures. If the VIF plug provider implementation is able to perform lookup +it should fill the `struct vif_plug_port_ctx_out` data structure and return +'true'. The `ovn-controller` will then create or update the port/interface +records and then call `vif_plug_port_finish` when the transactions commits and +`vif_plug_port_ctx_destroy` to free any allocated memory. If the VIF plug +provider implementation is unable to perform lookup or prepare the desired +resource at this time, it should return 'false' which will tell the +`ovn-controller` to not plug the port, in this case it will not call +`vif_plug_port_finish` nor `vif_plug_port_ctx_destroy`. + +> **Note**: The VIF plug provider implementation should exhaust all + non-blocking options to succeed with lookup from within the + `vif_plug_port_prepare` handler, including refreshing lookup + tables if necessary. + +Before removing port and interface records previously plugged by the +`ovn-controller` as identified by presence of the Interface +`external-ids:ovn-plugged` key, the `ovn-controller` will look up the +`vif-plug-type` from `external-ids:ovn-plugged`, fill +`struct vif_plug_port_ctx_in` with `op_type` set to 'PLUG_OP_REMOVE' and make a +call to `vif_plug_port_prepare`. After the port and interface has been removed +a call will be made to `vif_plug_port_finish`. Both calls will be made with +the pointer to `vif_plug_port_ctx_out` set to 'NULL', and no call will be made +to `vif_plug_port_ctx_destroy`. + +Building with in-tree VIF plug providers +---------------------------------------- + +VIF plug providers hosted in the OVN repository live under +`lib/vif-plug-providers`: + +To enable them, provide the `--enable-vif-plug-providers` command line option +to the configure script when building OVN. + +Building with an externally provided VIF plug provider +------------------------------------------------------ + +There is also infrastructure in place to support linking OVN with an externally +built VIF plug provider. + +This external VIF plug provider must define a NULL-terminated array of pointers +to `struct vif_plug_class` data structures named `vif_plug_provider_classes`. +Example: + +.. code-block:: none + + const struct vif_plug_class *vif_plug_provider_classes[] = { + &vif_plug_foo, + NULL, + }; + +The name of the repository for the external VIF plug provider should be the +same as the name of the library it produces, and the built library artifact +should be placed in lib/.libs. Example: + +.. code-block:: none + + ovn-vif-foo/ + ovn-vif-foo/lib/.libs/libovn-vif-foo.la + +To enable such a VIF plug provider provide the +`--with-vif-plug-provider=/path/to/ovn-vif-foo` command line option to the +configure script when building OVN. + +.. LINKS +.. _Open vSwitch Integration Guide: + https://docs.openvswitch.org/en/latest/topics/integration/ diff --git a/acinclude.m4 b/acinclude.m4 index e7f829520..3e4a54086 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -441,3 +441,52 @@ AC_DEFUN([OVN_CHECK_OVS], [ AC_MSG_CHECKING([OVS version]) AC_MSG_RESULT([$OVSVERSION]) ]) + +dnl OVN_CHECK_VIF_PLUG_PROVIDER +dnl +dnl Check for external VIF plug provider +AC_DEFUN([OVN_CHECK_VIF_PLUG_PROVIDER], [ + AC_ARG_VAR([VIF_PLUG_PROVIDER]) + AC_ARG_WITH( + [vif-plug-provider], + [AC_HELP_STRING([--with-vif-plug-provider=/path/to/provider/repository], + [Specify path to a configured and built VIF plug provider repository])], + [if test "$withval" = yes; then + if test -z "$VIF_PLUG_PROVIDER"; then + AC_MSG_ERROR([To build with external VIF plug provider, specify the path to a configured and built plug provider repository --with-vif-plug-provider or in \$VIF_PLUG_PROVIDER]), + fi + VIF_PLUG_PROVIDER="$(realpath $VIF_PLUG_PROVIDER)" + else + VIF_PLUG_PROVIDER="$(realpath $withval)" + fi + _vif_plug_provider_name="$(basename $VIF_PLUG_PROVIDER)" + if test ! -f "$VIF_PLUG_PROVIDER/lib/.libs/lib${_vif_plug_provider_name}.la"; then + AC_MSG_ERROR([$withval is not a configured and built VIF plug provider library repository]) + fi + VIF_PLUG_PROVIDER_LDFLAGS="-L$VIF_PLUG_PROVIDER/lib/.libs -l$_vif_plug_provider_name" + ], + [VIF_PLUG_PROVIDER=no]) + AC_MSG_CHECKING([for VIF plug provider]) + AC_MSG_RESULT([$VIF_PLUG_PROVIDER]) + AC_SUBST([VIF_PLUG_PROVIDER_LDFLAGS]) + AM_CONDITIONAL([HAVE_VIF_PLUG_PROVIDER], [test "$VIF_PLUG_PROVIDER" != no]) + if test "$VIF_PLUG_PROVIDER" != no; then + AC_DEFINE([HAVE_VIF_PLUG_PROVIDER], [1], + [Build and link with external VIF plug provider]) + fi +]) + +dnl OVN_ENABLE_VIF_PLUG +dnl +dnl Enable built-in plug providers +AC_DEFUN([OVN_ENABLE_VIF_PLUG], [ + AC_ARG_ENABLE( + [vif-plug-providers], + [AC_HELP_STRING([--enable-vif-plug-providers], [Enable building of built-in VIF plug providers])], + [], [enable_vif_plug=no]) + AM_CONDITIONAL([ENABLE_VIF_PLUG], [test "$enable_vif_plug" != no]) + if test "$enable_vif_plug" != no; then + AC_DEFINE([ENABLE_VIF_PLUG], [1], + [Build built-in VIF plug providers]) + fi +]) diff --git a/configure.ac b/configure.ac index d1b9b4d55..5a3a5987b 100644 --- a/configure.ac +++ b/configure.ac @@ -172,6 +172,8 @@ OVS_ENABLE_SPARSE OVS_CHECK_DDLOG([0.47]) OVS_CHECK_PRAGMA_MESSAGE OVN_CHECK_OVS +OVN_CHECK_VIF_PLUG_PROVIDER +OVN_ENABLE_VIF_PLUG OVS_CTAGS_IDENTIFIERS AC_SUBST([OVS_CFLAGS]) AC_SUBST([OVS_LDFLAGS]) diff --git a/controller/automake.mk b/controller/automake.mk index ad2d68af2..9f9b49fe0 100644 --- a/controller/automake.mk +++ b/controller/automake.mk @@ -37,7 +37,9 @@ controller_ovn_controller_SOURCES = \ controller/local_data.c \ controller/local_data.h \ controller/ovsport.h \ - controller/ovsport.c + controller/ovsport.c \ + controller/vif-plug.h \ + controller/vif-plug.c controller_ovn_controller_LDADD = lib/libovn.la $(OVS_LIBDIR)/libopenvswitch.la man_MANS += controller/ovn-controller.8 diff --git a/controller/test-vif-plug.c b/controller/test-vif-plug.c new file mode 100644 index 000000000..01ff37d8f --- /dev/null +++ b/controller/test-vif-plug.c @@ -0,0 +1,72 @@ +/* Copyright (c) 2021, Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include + +#include "vif-plug.h" +#include "vif-plug-provider.h" +#include "smap.h" +#include "sset.h" +#include "tests/ovstest.h" + +static void +test_vif_plug(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + const struct vif_plug_class *vif_plug_class; + + ovs_assert(vif_plug_provider_unregister("dummy") == EINVAL); + + ovs_assert(!vif_plug_provider_register(&vif_plug_dummy_class)); + vif_plug_class = vif_plug_provider_get("dummy"); + ovs_assert(vif_plug_provider_register(&vif_plug_dummy_class) == EEXIST); + + ovs_assert( + sset_contains( + vif_plug_get_maintained_iface_options(vif_plug_class), + "plug-dummy-option")); + + struct vif_plug_port_ctx_in ctx_in = { + .op_type = PLUG_OP_CREATE, + .lport_name = "lsp1", + .lport_options = SMAP_INITIALIZER(&ctx_in.lport_options), + }; + struct vif_plug_port_ctx_out ctx_out; + vif_plug_port_prepare(vif_plug_class, &ctx_in, &ctx_out); + ovs_assert(!strcmp(ctx_out.name, "lsp1")); + ovs_assert(!strcmp(ctx_out.type, "internal")); + ovs_assert(!strcmp(smap_get( + &ctx_out.iface_options, "vif-plug-dummy-option"), "value")); + + vif_plug_port_finish(vif_plug_class, &ctx_in, &ctx_out); + vif_plug_port_ctx_destroy(vif_plug_class, &ctx_in, &ctx_out); + vif_plug_provider_destroy_all(); +} + +static void +test_vif_plug_main(int argc, char *argv[]) +{ + set_program_name(argv[0]); + static const struct ovs_cmdl_command commands[] = { + {"run", NULL, 0, 0, test_vif_plug, OVS_RO}, + {NULL, NULL, 0, 0, NULL, OVS_RO}, + }; + struct ovs_cmdl_context ctx; + ctx.argc = argc - 1; + ctx.argv = argv + 1; + ovs_cmdl_run_command(&ctx, commands); +} + +OVSTEST_REGISTER("test-vif-plug", test_vif_plug_main); diff --git a/controller/vif-plug.c b/controller/vif-plug.c new file mode 100644 index 000000000..1015a4c13 --- /dev/null +++ b/controller/vif-plug.c @@ -0,0 +1,634 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +/* OVS includes */ +#include "lib/vswitch-idl.h" +#include "openvswitch/shash.h" +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "binding.h" +#include "lib/ovn-sb-idl.h" +#include "lport.h" +#include "ovsport.h" +#include "vif-plug.h" +#include "vif-plug-provider.h" + +VLOG_DEFINE_THIS_MODULE(vif_plug); + +#define OVN_PLUGGED_EXT_ID "ovn-plugged" +#define VIF_PLUG_OPTION_TYPE "vif-plug-type" +#define VIF_PLUG_OPTION_MTU_REQUEST "vif-plug-mtu-request" + +void +vif_plug_register_ovs_idl(struct ovsdb_idl *ovs_idl) +{ + ovsdb_idl_track_add_column(ovs_idl, &ovsrec_interface_col_mtu_request); +} + +/* Get the class level 'maintained_iface_options' set. */ +const struct sset * +vif_plug_get_maintained_iface_options( + const struct vif_plug_class *vif_plug_class) +{ + return vif_plug_class->vif_plug_get_maintained_iface_options ? + vif_plug_class->vif_plug_get_maintained_iface_options() : NULL; +} + +/* Prepare the logical port as identified by 'ctx_in' for port creation, update + * or removal as specified by 'ctx_in->op_type'. + * + * When 'ctx_in->op_type' is PLUG_OP_CREATE the plug implementation must fill + * 'ctx_out' with data to apply to the interface record maintained by OVN on + * its behalf. + * + * When 'ctx_in_op_type' is PLUG_OP_REMOVE 'ctx_out' should be set to NULL and + * the plug implementation must not attempt to use 'ctx_out'. + * + * The data in 'ctx_out' is owned by the plug implementation, and a call must + * be made to vif_plug_port_ctx_destroy when done with it. */ +bool +vif_plug_port_prepare(const struct vif_plug_class *vif_plug_class, + const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out) +{ + return vif_plug_class->vif_plug_port_prepare(ctx_in, ctx_out); +} + +/* Notify the VIF plug implementation that a port creation, update or removal + * has been committed to the database. */ +void +vif_plug_port_finish(const struct vif_plug_class *vif_plug_class, + const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out) +{ + vif_plug_class->vif_plug_port_finish(ctx_in, ctx_out); +} + +/* Free any data allocated to 'ctx_out' in a prevous call to + * vif_plug_port_prepare. */ +void +vif_plug_port_ctx_destroy(const struct vif_plug_class *vif_plug_class, + const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out) +{ + vif_plug_class->vif_plug_port_ctx_destroy(ctx_in, ctx_out); +} + +static struct vif_plug_port_ctx * +build_port_ctx(const struct vif_plug_class *vif_plug, + const enum vif_plug_op_type op_type, + const struct vif_plug_ctx_in *vif_plug_ctx_in, + const struct sbrec_port_binding *pb, + const struct ovsrec_interface *iface, + const char *iface_id) +{ + struct vif_plug_port_ctx *new_ctx = xzalloc( + sizeof *new_ctx); + + new_ctx->vif_plug = vif_plug; + new_ctx->vif_plug_port_ctx_in.op_type = op_type; + new_ctx->vif_plug_port_ctx_in.ovs_table = vif_plug_ctx_in->ovs_table; + new_ctx->vif_plug_port_ctx_in.br_int = vif_plug_ctx_in->br_int; + new_ctx->vif_plug_port_ctx_in.lport_name = pb ? + xstrdup(pb->logical_port) : iface_id ? xstrdup(iface_id) : NULL; + /* Prepare vif_plug_port_ctx_in smaps for use. + * + * Note that smap_init does not allocate memory. Any memory allocated by + * putting data into the vif_plug_port_ctx_in smaps will be destroyed by + * calls to smap_destroy in destroy_port_ctx */ + smap_init(&new_ctx->vif_plug_port_ctx_in.lport_options); + smap_init(&new_ctx->vif_plug_port_ctx_in.iface_options); + + if (pb) { + smap_clone(&new_ctx->vif_plug_port_ctx_in.lport_options, + &pb->options); + } + + if (iface) { + new_ctx->vif_plug_port_ctx_in.iface_name = xstrdup(iface->name); + new_ctx->vif_plug_port_ctx_in.iface_type = xstrdup(iface->type); + smap_clone(&new_ctx->vif_plug_port_ctx_in.iface_options, + &iface->options); + } + + /* Prepare vif_plug_port_ctx_out smaps for use. + * + * Note that smap_init does not allocate memory. Any memory allocated by + * putting data into the vif_plug_port_ctx_out smaps is the responsibility + * of the VIF plug provider through a call to vif_plug_port_ctx_destroy. */ + smap_init(&new_ctx->vif_plug_port_ctx_out.iface_options); + + return new_ctx; +} + +static void +destroy_port_ctx(struct vif_plug_port_ctx *ctx) +{ + smap_destroy(&ctx->vif_plug_port_ctx_in.lport_options); + smap_destroy(&ctx->vif_plug_port_ctx_in.iface_options); + if (ctx->vif_plug_port_ctx_in.lport_name) { + free((char *)ctx->vif_plug_port_ctx_in.lport_name); + } + if (ctx->vif_plug_port_ctx_in.iface_name) { + free((char *)ctx->vif_plug_port_ctx_in.iface_name); + } + if (ctx->vif_plug_port_ctx_in.iface_type) { + free((char *)ctx->vif_plug_port_ctx_in.iface_type); + } + /* Note that data associated with ctx->vif_plug_port_ctx_out must be + * destroyed by the plug provider implementation with a call to + * vif_plug_port_ctx_destroy prior to calling this function */ + free(ctx); +} + +/* Our contract with the VIF plug provider is that vif_plug_port_finish + * will be called with vif_plug_port_ctx_* objects once the transaction + * commits. To handle this we keep track of in-flight deletions + * and changes. The tracking data will be cleared after commit at the end of + * the ovn-controller main loop. */ +static void +transact_delete_port(const struct vif_plug_ctx_in *vif_plug_ctx_in, + const struct vif_plug_ctx_out *vif_plug_ctx_out, + const struct vif_plug_port_ctx *vif_plug_port_ctx, + const struct ovsrec_port *port) +{ + shash_add(vif_plug_ctx_out->deleted_iface_ids, + vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name, + vif_plug_port_ctx); + ovsport_remove(vif_plug_ctx_in->br_int, port); +} + +static void +transact_create_port(const struct vif_plug_ctx_in *vif_plug_ctx_in, + const struct vif_plug_ctx_out *vif_plug_ctx_out, + const struct vif_plug_port_ctx *vif_plug_port_ctx, + const struct smap *iface_external_ids, + const int64_t mtu_request) +{ + shash_add(vif_plug_ctx_out->changed_iface_ids, + vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name, + vif_plug_port_ctx); + ovsport_create(vif_plug_ctx_in->ovs_idl_txn, vif_plug_ctx_in->br_int, + vif_plug_port_ctx->vif_plug_port_ctx_out.name, + vif_plug_port_ctx->vif_plug_port_ctx_out.type, + NULL, iface_external_ids, + &vif_plug_port_ctx->vif_plug_port_ctx_out.iface_options, + mtu_request); +} + +static void +transact_update_port(const struct ovsrec_interface *iface_rec, + const struct vif_plug_ctx_in *vif_plug_ctx_in OVS_UNUSED, + const struct vif_plug_ctx_out *vif_plug_ctx_out, + const struct vif_plug_port_ctx *vif_plug_port_ctx, + const struct smap *iface_external_ids, + const int64_t mtu_request) +{ + shash_add(vif_plug_ctx_out->changed_iface_ids, + vif_plug_port_ctx->vif_plug_port_ctx_in.lport_name, + vif_plug_port_ctx); + ovsport_update_iface( + iface_rec, + vif_plug_port_ctx->vif_plug_port_ctx_out.type, + iface_external_ids, + NULL, + &vif_plug_port_ctx->vif_plug_port_ctx_out.iface_options, + vif_plug_get_maintained_iface_options( + vif_plug_port_ctx->vif_plug), + mtu_request); +} + + +static bool +consider_unplug_iface(const struct ovsrec_interface *iface, + const struct sbrec_port_binding *pb, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out) +{ + const char *vif_plug_type = smap_get(&iface->external_ids, + OVN_PLUGGED_EXT_ID); + const char *iface_id = smap_get(&iface->external_ids, "iface-id"); + const struct ovsrec_port *port = ovsport_lookup_by_interface( + vif_plug_ctx_in->ovsrec_port_by_interfaces, + (struct ovsrec_interface *) iface); + + if (vif_plug_type && iface_id && port) { + const struct vif_plug_class *vif_plug; + if (!(vif_plug = vif_plug_provider_get(vif_plug_type))) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, + "Unable to open VIF plug provider for " + "%s %s iface-id %s", + VIF_PLUG_OPTION_TYPE, vif_plug_type, iface_id); + /* While we are unable to handle this, asking for a recompute + * will not change that fact. */ + return true; + } + if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int + || !vif_plug_ctx_in->ovs_idl_txn) + { + /* Some of our prerequisites are not available, ask for a + * recompute. */ + return false; + } + + /* Our contract with the VIF plug provider is that vif_plug_port_finish + * will be called with a vif_plug_port_ctx_in object once the + * transaction commits. + * + * Since this happens asynchronously we need to allocate memory for + * and duplicate any database references so that they stay valid. + * + * The data is freed with a call to destroy_port_ctx after the + * transaction completes at the end of the ovn-controller main + * loop. */ + struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx( + vif_plug, PLUG_OP_REMOVE, vif_plug_ctx_in, pb, iface, iface_id); + + if (!vif_plug_port_prepare(vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + NULL)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, + "Not unplugging iface %s (iface-id %s) on direction " + "from VIF plug provider.", + iface->name, iface_id); + destroy_port_ctx(vif_plug_port_ctx); + return true; + } + VLOG_INFO("Unplugging port %s from %s for iface-id %s on this " + "chassis.", + port->name, + vif_plug_ctx_in->br_int->name, + iface_id); + + /* Add and track delete operation to the transaction */ + transact_delete_port(vif_plug_ctx_in, vif_plug_ctx_out, + vif_plug_port_ctx, port); + return true; + } + return true; +} + +static int64_t +get_plug_mtu_request(const struct smap *lport_options) +{ + return smap_get_int(lport_options, VIF_PLUG_OPTION_MTU_REQUEST, 0); +} + +static bool +consider_plug_lport_create__(const struct vif_plug_class *vif_plug, + const struct smap *iface_external_ids, + const struct sbrec_port_binding *pb, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out) +{ + if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int + || !vif_plug_ctx_in->ovs_idl_txn) { + /* Some of our prerequisites are not available, ask for a recompute. */ + return false; + } + + /* Our contract with the VIF plug provider is that vif_plug_port_finish + * will be called with vif_plug_port_ctx_in and vif_plug_port_ctx_out + * objects once the transaction commits. + * + * Since this happens asynchronously we need to allocate memory for + * and duplicate any database references so that they stay valid. + * + * The data is freed with a call to destroy_port_ctx after the + * transaction completes at the end of the ovn-controller main + * loop. */ + struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx( + vif_plug, PLUG_OP_CREATE, vif_plug_ctx_in, pb, NULL, NULL); + + if (!vif_plug_port_prepare(vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + &vif_plug_port_ctx->vif_plug_port_ctx_out)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, + "Not plugging lport %s on direction from VIF plug " + "provider.", + pb->logical_port); + destroy_port_ctx(vif_plug_port_ctx); + return true; + } + + VLOG_INFO("Plugging port %s into %s for lport %s on this " + "chassis.", + vif_plug_port_ctx->vif_plug_port_ctx_out.name, + vif_plug_ctx_in->br_int->name, + pb->logical_port); + transact_create_port(vif_plug_ctx_in, vif_plug_ctx_out, + vif_plug_port_ctx, + iface_external_ids, + get_plug_mtu_request(&pb->options)); + return true; +} + +static bool +consider_plug_lport_update__(const struct vif_plug_class *vif_plug, + const struct smap *iface_external_ids, + const struct sbrec_port_binding *pb, + struct local_binding *lbinding, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out) +{ + if (!vif_plug_ctx_in->chassis_rec || !vif_plug_ctx_in->br_int + || !vif_plug_ctx_in->ovs_idl_txn) { + /* Some of our prerequisites are not available, ask for a recompute. */ + return false; + } + /* Our contract with the VIF plug provider is that vif_plug_port_finish + * will be called with vif_plug_port_ctx_in and vif_plug_port_ctx_out + * objects once the transaction commits. + * + * Since this happens asynchronously we need to allocate memory for + * and duplicate any database references so that they stay valid. + * + * The data is freed with a call to destroy_port_ctx after the + * transaction completes at the end of the ovn-controller main + * loop. */ + struct vif_plug_port_ctx *vif_plug_port_ctx = build_port_ctx( + vif_plug, PLUG_OP_CREATE, vif_plug_ctx_in, pb, NULL, NULL); + + if (!vif_plug_port_prepare(vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + &vif_plug_port_ctx->vif_plug_port_ctx_out)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, + "Not updating lport %s on direction from VIF plug " + "provider.", + pb->logical_port); + destroy_port_ctx(vif_plug_port_ctx); + return true; + } + + if (strcmp(lbinding->iface->name, + vif_plug_port_ctx->vif_plug_port_ctx_out.name)) { + VLOG_WARN("Attempt of incompatible change to existing " + "port detected, please recreate port: %s", + pb->logical_port); + vif_plug_port_ctx_destroy(vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + &vif_plug_port_ctx->vif_plug_port_ctx_out); + destroy_port_ctx(vif_plug_port_ctx); + return false; + } + VLOG_DBG("updating iface for: %s", pb->logical_port); + transact_update_port(lbinding->iface, vif_plug_ctx_in, vif_plug_ctx_out, + vif_plug_port_ctx, iface_external_ids, + get_plug_mtu_request(&pb->options)); + + return true; +} + +static bool +consider_plug_lport(const struct sbrec_port_binding *pb, + struct local_binding *lbinding, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out) +{ + bool ret = true; + if (lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb) + && pb->requested_chassis == vif_plug_ctx_in->chassis_rec) { + const char *vif_plug_type = smap_get(&pb->options, + VIF_PLUG_OPTION_TYPE); + if (!vif_plug_type) { + /* Nothing for us to do and we don't need a recompute. */ + return true; + } + + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + const struct vif_plug_class *vif_plug; + if (!(vif_plug = vif_plug_provider_get(vif_plug_type))) { + VLOG_WARN_RL(&rl, + "Unable to open VIF plug provider for %s: '%s' " + "lport %s", + VIF_PLUG_OPTION_TYPE, + vif_plug_type, + pb->logical_port); + /* While we are unable to handle this, asking for a recompute will + * not change that fact. */ + return true; + } + const struct smap iface_external_ids = SMAP_CONST2( + &iface_external_ids, + OVN_PLUGGED_EXT_ID, vif_plug_type, + "iface-id", pb->logical_port); + if (lbinding && lbinding->iface) { + if (!smap_get(&lbinding->iface->external_ids, + OVN_PLUGGED_EXT_ID)) + { + VLOG_WARN_RL(&rl, + "CMS requested plugging of lport %s, but a port " + "that is not maintained by OVN already exsist " + "in local vSwitch: "UUID_FMT, + pb->logical_port, + UUID_ARGS(&lbinding->iface->header_.uuid)); + return false; + } + ret = consider_plug_lport_update__(vif_plug, &iface_external_ids, + pb, lbinding, vif_plug_ctx_in, + vif_plug_ctx_out); + } else { + ret = consider_plug_lport_create__(vif_plug, &iface_external_ids, + pb, vif_plug_ctx_in, + vif_plug_ctx_out); + } + } + + return ret; +} + +static bool +vif_plug_iface_touched_this_txn( + const struct vif_plug_ctx_out *vif_plug_ctx_out, + const char *iface_id) +{ + return shash_find(vif_plug_ctx_out->changed_iface_ids, iface_id) + || shash_find(vif_plug_ctx_out->deleted_iface_ids, iface_id); +} + +static bool +vif_plug_handle_lport_vif(const struct sbrec_port_binding *pb, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out, + bool can_unplug) +{ + if (vif_plug_iface_touched_this_txn(vif_plug_ctx_out, pb->logical_port)) { + return true; + } + bool handled = true; + struct local_binding *lbinding = local_binding_find( + vif_plug_ctx_in->local_bindings, pb->logical_port); + + if (lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb)) { + handled &= consider_plug_lport(pb, lbinding, + vif_plug_ctx_in, vif_plug_ctx_out); + } else if (can_unplug && lbinding && lbinding->iface) { + handled &= consider_unplug_iface(lbinding->iface, pb, + vif_plug_ctx_in, vif_plug_ctx_out); + } + return handled; +} + +static bool +vif_plug_handle_iface(const struct ovsrec_interface *iface_rec, + struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out, + bool can_unplug) +{ + bool handled = true; + const char *vif_plug_type = smap_get(&iface_rec->external_ids, + OVN_PLUGGED_EXT_ID); + const char *iface_id = smap_get(&iface_rec->external_ids, "iface-id"); + if (!vif_plug_type || !iface_id + || vif_plug_iface_touched_this_txn(vif_plug_ctx_out, iface_id)) { + return true; + } + struct local_binding *lbinding = local_binding_find( + vif_plug_ctx_in->local_bindings, iface_id); + const struct sbrec_port_binding *pb = lport_lookup_by_name( + vif_plug_ctx_in->sbrec_port_binding_by_name, iface_id); + if (pb && lbinding + && lport_can_bind_on_this_chassis(vif_plug_ctx_in->chassis_rec, pb)) { + /* Something changed on a interface we have previously plugged, + * consider updating it */ + handled &= consider_plug_lport(pb, lbinding, + vif_plug_ctx_in, vif_plug_ctx_out); + } else if (can_unplug + && (!pb + || !lport_can_bind_on_this_chassis( + vif_plug_ctx_in->chassis_rec, pb))) { + /* No lport for this interface or it is destined for different chassis, + * consuder unplugging it */ + handled &= consider_unplug_iface(iface_rec, pb, + vif_plug_ctx_in, vif_plug_ctx_out); + } + return handled; +} + +/* On initial startup or on IDL reconnect, several rounds of the main loop may + * run before data is actually loaded in the IDL. This situation is currently + * not reflected in a call to ovsdb_idl_has_ever_connected(). Until we find + * the root of this issue we need this counter so that we do not erronously + * unplug ports because the data is just not loaded yet. + */ +#define VIF_PLUG_PRIME_IDL_COUNT_SEEED 10 +static int vif_plug_prime_idl_count = VIF_PLUG_PRIME_IDL_COUNT_SEEED; + +void +vif_plug_reset_idl_prime_counter(void) +{ + vif_plug_prime_idl_count = VIF_PLUG_PRIME_IDL_COUNT_SEEED; +} + +void +vif_plug_run(struct vif_plug_ctx_in *vif_plug_ctx_in, + struct vif_plug_ctx_out *vif_plug_ctx_out) +{ + if (vif_plug_prime_idl_count && --vif_plug_prime_idl_count > 0) { + VLOG_DBG("vif_plug_run: vif_plug_prime_idl_count=%d, will not unplug " + "ports in this iteration.", vif_plug_prime_idl_count); + } + + if (!vif_plug_ctx_in->chassis_rec) { + return; + } + const struct ovsrec_interface *iface_rec; + OVSREC_INTERFACE_TABLE_FOR_EACH (iface_rec, + vif_plug_ctx_in->iface_table) { + vif_plug_handle_iface(iface_rec, vif_plug_ctx_in, vif_plug_ctx_out, + !vif_plug_prime_idl_count); + } + + struct sbrec_port_binding *target = + sbrec_port_binding_index_init_row( + vif_plug_ctx_in->sbrec_port_binding_by_requested_chassis); + sbrec_port_binding_index_set_requested_chassis( + target, + vif_plug_ctx_in->chassis_rec); + const struct sbrec_port_binding *pb; + SBREC_PORT_BINDING_FOR_EACH_EQUAL ( + pb, target, + vif_plug_ctx_in->sbrec_port_binding_by_requested_chassis) { + enum en_lport_type lport_type = get_lport_type(pb); + if (lport_type == LP_VIF) { + vif_plug_handle_lport_vif(pb, vif_plug_ctx_in, vif_plug_ctx_out, + !vif_plug_prime_idl_count); + } + } + sbrec_port_binding_index_destroy_row(target); +} + +static void +vif_plug_finish_deleted__(struct shash *deleted_iface_ids, bool txn_success) +{ + struct shash_node *node, *next; + SHASH_FOR_EACH_SAFE (node, next, deleted_iface_ids) { + struct vif_plug_port_ctx *vif_plug_port_ctx = node->data; + if (txn_success) { + vif_plug_port_finish(vif_plug_port_ctx->vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + NULL); + } + shash_delete(deleted_iface_ids, node); + destroy_port_ctx(vif_plug_port_ctx); + } +} + +void +vif_plug_clear_deleted(struct shash *deleted_iface_ids) { + vif_plug_finish_deleted__(deleted_iface_ids, false); +} + +void +vif_plug_finish_deleted(struct shash *deleted_iface_ids) { + vif_plug_finish_deleted__(deleted_iface_ids, true); +} + +static void +vif_plug_finish_changed__(struct shash *changed_iface_ids, bool txn_success) +{ + struct shash_node *node, *next; + SHASH_FOR_EACH_SAFE (node, next, changed_iface_ids) { + struct vif_plug_port_ctx *vif_plug_port_ctx = node->data; + if (txn_success) { + vif_plug_port_finish(vif_plug_port_ctx->vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + &vif_plug_port_ctx->vif_plug_port_ctx_out); + } + vif_plug_port_ctx_destroy(vif_plug_port_ctx->vif_plug, + &vif_plug_port_ctx->vif_plug_port_ctx_in, + &vif_plug_port_ctx->vif_plug_port_ctx_out); + shash_delete(changed_iface_ids, node); + destroy_port_ctx(vif_plug_port_ctx); + } +} + +void +vif_plug_clear_changed(struct shash *deleted_iface_ids) { + vif_plug_finish_changed__(deleted_iface_ids, false); +} + +void +vif_plug_finish_changed(struct shash *deleted_iface_ids) { + vif_plug_finish_changed__(deleted_iface_ids, true); +} diff --git a/controller/vif-plug.h b/controller/vif-plug.h new file mode 100644 index 000000000..76063591b --- /dev/null +++ b/controller/vif-plug.h @@ -0,0 +1,80 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef VIF_PLUG_H +#define VIF_PLUG_H 1 + +/* + * VIF Plug, the controller internal interface to the VIF plug provider + * infrastructure. + */ + +#include "openvswitch/shash.h" +#include "smap.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct vif_plug_ctx_in { + struct ovsdb_idl_txn *ovs_idl_txn; + struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *sbrec_port_binding_by_requested_chassis; + struct ovsdb_idl_index *ovsrec_port_by_interfaces; + const struct ovsrec_open_vswitch_table *ovs_table; + const struct ovsrec_bridge *br_int; + const struct ovsrec_interface_table *iface_table; + const struct sbrec_chassis *chassis_rec; + const struct shash *local_bindings; +}; + +struct vif_plug_ctx_out { + struct shash *deleted_iface_ids; + struct shash *changed_iface_ids; +}; + +struct vif_plug_class; +struct vif_plug_port_ctx_out; +struct vif_plug_port_ctx_in; + +const struct sset * vif_plug_get_maintained_iface_options( + const struct vif_plug_class *); + +bool vif_plug_port_prepare(const struct vif_plug_class *, + const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); +void vif_plug_port_finish(const struct vif_plug_class *, + const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); +void vif_plug_port_ctx_destroy(const struct vif_plug_class *, + const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); + +struct ovsdb_idl; + +void vif_plug_register_ovs_idl(struct ovsdb_idl *ovs_idl); +void vif_plug_run(struct vif_plug_ctx_in *, struct vif_plug_ctx_out *); +void vif_plug_clear_changed(struct shash *deleted_iface_ids); +void vif_plug_finish_changed(struct shash *changed_iface_ids); +void vif_plug_clear_deleted(struct shash *deleted_iface_ids); +void vif_plug_finish_deleted(struct shash *changed_iface_ids); +void vif_plug_reset_idl_prime_counter(void); + +#ifdef __cplusplus +} +#endif + +#endif /* vif-plug.h */ diff --git a/lib/automake.mk b/lib/automake.mk index 9f9f447d5..829aedfc5 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -4,6 +4,11 @@ lib_libovn_la_LDFLAGS = \ -Wl,--version-script=$(top_builddir)/lib/libovn.sym \ $(OVS_LIBDIR)/libopenvswitch.la \ $(AM_LDFLAGS) + +if HAVE_VIF_PLUG_PROVIDER +lib_libovn_la_LDFLAGS += $(VIF_PLUG_PROVIDER_LDFLAGS) +endif + lib_libovn_la_SOURCES = \ lib/acl-log.c \ lib/acl-log.h \ @@ -33,7 +38,10 @@ lib_libovn_la_SOURCES = \ lib/inc-proc-eng.h \ lib/lb.c \ lib/lb.h \ - lib/stopwatch-names.h + lib/stopwatch-names.h \ + lib/vif-plug-provider.h \ + lib/vif-plug-provider.c \ + lib/vif-plug-providers/dummy/vif-plug-dummy.c nodist_lib_libovn_la_SOURCES = \ lib/ovn-dirs.c \ lib/ovn-nb-idl.c \ diff --git a/lib/vif-plug-provider.c b/lib/vif-plug-provider.c new file mode 100644 index 000000000..798e90e26 --- /dev/null +++ b/lib/vif-plug-provider.c @@ -0,0 +1,204 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "vif-plug-provider.h" + +#include +#include +#include + +#include "openvswitch/vlog.h" +#include "openvswitch/shash.h" +#include "smap.h" +#include "sset.h" +#include "lib/inc-proc-eng.h" + +VLOG_DEFINE_THIS_MODULE(vif_plug_provider); + +#ifdef ENABLE_VIF_PLUG +static const struct vif_plug_class *base_vif_plug_classes[] = { +}; +#endif + +static struct shash vif_plug_classes = SHASH_INITIALIZER(&vif_plug_classes); + +/* Protects the 'vif_plug_classes' shash. */ +static struct ovs_mutex vif_plug_classes_mutex = OVS_MUTEX_INITIALIZER; + +/* Initialize the the VIF plug infrastructure by registering known classes */ +void +vif_plug_provider_initialize(void) +{ + static struct ovsthread_once once = OVSTHREAD_ONCE_INITIALIZER; + + if (ovsthread_once_start(&once)) { +#ifdef ENABLE_VIF_PLUG + /* Register built-in VIF plug provider classes */ + for (int i = 0; i < ARRAY_SIZE(base_vif_plug_classes); i++) { + vif_plug_provider_register(base_vif_plug_classes[i]); + } +#endif +#ifdef HAVE_VIF_PLUG_PROVIDER + /* Register external VIF plug provider classes. + * + * Note that we cannot use the ARRAY_SIZE macro here as + * vif_plug_provider_classes is defined in external code which is not + * available at compile time. The convention is to use a + * NULL-terminated array instead. */ + for (const struct vif_plug_class **pp = vif_plug_provider_classes; + pp && *pp; + pp++) + { + vif_plug_provider_register(*pp); + } +#endif + ovsthread_once_done(&once); + } +} + +static int +vif_plug_provider_register__(const struct vif_plug_class *new_class) +{ + struct vif_plug_class *vif_plug_class; + int error; + + if (shash_find(&vif_plug_classes, new_class->type)) { + VLOG_WARN("attempted to register duplicate VIF plug provider: %s", + new_class->type); + return EEXIST; + } + + error = new_class->init ? new_class->init() : 0; + if (error) { + VLOG_WARN("failed to initialize %s VIF plug provider class: %s", + new_class->type, ovs_strerror(error)); + return error; + } + + vif_plug_class = xmalloc(sizeof *vif_plug_class); + memcpy(vif_plug_class, new_class, sizeof *vif_plug_class); + + shash_add(&vif_plug_classes, new_class->type, vif_plug_class); + + return 0; +} + +/* Register the new VIF plug provider referred to in 'new_class' and perform + * any class level initialization as specified in its vif_plug_class. */ +int +vif_plug_provider_register(const struct vif_plug_class *new_class) +{ + int error; + + ovs_mutex_lock(&vif_plug_classes_mutex); + error = vif_plug_provider_register__(new_class); + ovs_mutex_unlock(&vif_plug_classes_mutex); + + return error; +} + +static int +vif_plug_provider_unregister__(const char *type) +{ + int error; + struct shash_node *node; + struct vif_plug_class *vif_plug_class; + + node = shash_find(&vif_plug_classes, type); + if (!node) { + return EINVAL; + } + + vif_plug_class = node->data; + error = vif_plug_class->destroy ? vif_plug_class->destroy() : 0; + if (error) { + VLOG_WARN("failed to destroy %s VIF plug class: %s", + vif_plug_class->type, ovs_strerror(error)); + return error; + } + + shash_delete(&vif_plug_classes, node); + free(vif_plug_class); + + return 0; +} + +/* Unregister the VIF plug provider identified by 'type' and perform any class + * level de-initialization as specified in its vif_plug_class. */ +int +vif_plug_provider_unregister(const char *type) +{ + int error; + + ovs_mutex_lock(&vif_plug_classes_mutex); + error = vif_plug_provider_unregister__(type); + ovs_mutex_unlock(&vif_plug_classes_mutex); + + return error; +} + +/* Check whether there are any VIF plug providers registered */ +bool +vif_plug_provider_has_providers(void) +{ + return !shash_is_empty(&vif_plug_classes); +} + +const struct vif_plug_class * +vif_plug_provider_get(const char *type) +{ + struct vif_plug_class *vif_plug_class; + + ovs_mutex_lock(&vif_plug_classes_mutex); + vif_plug_class = shash_find_data(&vif_plug_classes, type); + ovs_mutex_unlock(&vif_plug_classes_mutex); + + return vif_plug_class; +} + +/* Iterate over VIF plug providers and call their run function. + * + * Returns 'true' if any of the providers run functions return 'true', 'false' + * otherwise. + * + * A return value of 'true' means that data has changed. */ +bool +vif_plug_provider_run_all(void) +{ + struct shash_node *node, *next; + bool changed = false; + + SHASH_FOR_EACH_SAFE (node, next, &vif_plug_classes) { + struct vif_plug_class *vif_plug_class = node->data; + if (vif_plug_class->run && vif_plug_class->run(vif_plug_class)) { + changed = true; + } + } + return changed; +} + +/* De-initialize and unregister the VIF plug provider classes. */ +void +vif_plug_provider_destroy_all(void) +{ + struct shash_node *node, *next; + + SHASH_FOR_EACH_SAFE (node, next, &vif_plug_classes) { + struct vif_plug_class *vif_plug_class = node->data; + vif_plug_provider_unregister(vif_plug_class->type); + } +} diff --git a/lib/vif-plug-provider.h b/lib/vif-plug-provider.h new file mode 100644 index 000000000..c0c74b073 --- /dev/null +++ b/lib/vif-plug-provider.h @@ -0,0 +1,163 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef VIF_PLUG_PROVIDER_H +#define VIF_PLUG_PROVIDER_H 1 + +/* Interface for VIF plug providers. + * + * A VIF plug provider implementation performs lookup and/or initialization of + * ports, typically representor ports, using generic non-blocking hardware + * interfaces. This allows the ovn-controller to, upon the CMS's request, + * create ports and interfaces in the chassis's Open vSwitch instances (also + * known as vif plugging). + * + * This module contains the infrastructure for registering VIF plug providers + * which may be hosted inside or outside the core OVN repository. + */ + +#include + +#include "smap.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct vif_plug_class; +struct ovsdb_idl_txn; +struct ovsrec_bridge; + +enum vif_plug_op_type { + PLUG_OP_CREATE = 1, /* Port is created or updated */ + PLUG_OP_REMOVE, /* Port is removed from this chassis */ +}; + +struct vif_plug_port_ctx_in { + /* Operation being performed */ + enum vif_plug_op_type op_type; + + /* These are provided so that the plug implementation may make decisions + * based on environmental factors such as settings in the open-vswitch + * table and datapath type settings on the integration bridge. */ + const struct ovsrec_open_vswitch_table *ovs_table; + const struct ovsrec_bridge *br_int; + + /* Name of logical port, can be useful for plugging library to track any + * per port resource initialization. */ + const char *lport_name; + + /* Logical port options, while OVN will forward the contents verbatim from + * the Southbound database, the convention is for the plugging library to + * only make decisions based on the vif-plug-* options. */ + struct smap lport_options; + + /* When OVN knows about an existing interface record associated with this + * lport, these will be filled in with information about it. */ + const char *iface_name; + const char *iface_type; + struct smap iface_options; +}; + +struct vif_plug_port_ctx_out { + /* The name to use for port and interface record. */ + char *name; + + /* Type of interface to create. */ + char *type; + + /* Options to set on the interface record. */ + struct smap iface_options; +}; + +struct vif_plug_port_ctx { + const struct vif_plug_class *vif_plug; + struct vif_plug_port_ctx_in vif_plug_port_ctx_in; + struct vif_plug_port_ctx_out vif_plug_port_ctx_out; +}; + +struct vif_plug_class { + /* Type of VIF plug provider in this class. */ + const char *type; + + /* Called when the VIF plug provider is registered, typically at program + * startup. + * + * This function may be set to null if a VIF plug class needs no + * initialization at registration time. */ + int (*init)(void); + + /* Called when the VIF plug provider is unregistered, typically at program + * exit. + * + * This function may be set to null if a plug class needs no + * de-initialization at unregister time.*/ + int (*destroy)(void); + + /* Performs periodic work needed by VIF plug provider, if any is necessary. + * Returns 'true; if anything (i.e. lookup tables) changed, 'false' + * otherwise. */ + bool (*run)(struct vif_plug_class *); + + /* Retrieve Interface options this VIF plug provider will maintain. This + * sset is used to know which items to remove when maintaining the database + * record. */ + const struct sset * (*vif_plug_get_maintained_iface_options)(void); + + /* Pass vif_plug_port_ctx_in to VIF plug provider implementation to prepare + * for port creation/update. + * + * The VIF plug provider implemantation can perform lookup or any per port + * initialization and should fill vif_plug_port_ctx_out with data required + * for port/interface creation. The VIF plug implementation should return + * 'true' if it wants the caller to create/update a port/interface, 'false' + * otherwise. + * + * Data in the vif_plug_port_ctx_out struct is owned by the VIF plug + * provider, and a call must be made to the vif_plug_port_ctx_destroy + * callback to free up any allocations when done with port creation/update. + */ + bool (*vif_plug_port_prepare)(const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); + + /* Notify VIF plug provider that port update is committed to OVSDB. */ + void (*vif_plug_port_finish)(const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); + + /* Free any allocations made by the vif_plug_port_prepare callback. */ + void (*vif_plug_port_ctx_destroy)(const struct vif_plug_port_ctx_in *, + struct vif_plug_port_ctx_out *); +}; + +extern const struct vif_plug_class vif_plug_dummy_class; +#ifdef HAVE_VIF_PLUG_PROVIDER +extern const struct vif_plug_class *vif_plug_provider_classes[]; +#endif + +void vif_plug_provider_initialize(void); +int vif_plug_provider_register(const struct vif_plug_class *); +int vif_plug_provider_unregister(const char *type); +bool vif_plug_provider_has_providers(void); +const struct vif_plug_class * vif_plug_provider_get(const char *); +bool vif_plug_provider_run_all(void); +void vif_plug_provider_destroy_all(void); +void vif_plug_dummy_enable(void); + +#ifdef __cplusplus +} +#endif + +#endif /* vif-plug-provider.h */ diff --git a/lib/vif-plug-providers/dummy/vif-plug-dummy.c b/lib/vif-plug-providers/dummy/vif-plug-dummy.c new file mode 100644 index 000000000..bc8953e98 --- /dev/null +++ b/lib/vif-plug-providers/dummy/vif-plug-dummy.c @@ -0,0 +1,120 @@ +/* + * Copyright (c) 2021 Canonical + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include +#include "lib/vif-plug-provider.h" + +#include + +#include "openvswitch/vlog.h" +#include "smap.h" +#include "sset.h" + +#ifndef IFNAMSIZ +#define IFNAMSIZ 16 +#endif + +VLOG_DEFINE_THIS_MODULE(vif_plug_dummy); + +static struct sset vif_plug_dummy_maintained_iface_options; + +static int +vif_plug_dummy_init(void) +{ + sset_init(&vif_plug_dummy_maintained_iface_options); + sset_add(&vif_plug_dummy_maintained_iface_options, + "vif-plug-dummy-option"); + + return 0; +} + +static int +vif_plug_dummy_destroy(void) +{ + sset_destroy(&vif_plug_dummy_maintained_iface_options); + + return 0; +} + +static const struct sset* +vif_plug_dummy_get_maintained_iface_options(void) +{ + return &vif_plug_dummy_maintained_iface_options; +} + +static bool +vif_plug_dummy_run(struct vif_plug_class *plug) +{ + VLOG_DBG("vif_plug_dummy_run(%p)", plug); + + return false; +} + +static bool +vif_plug_dummy_port_prepare(const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("vif_plug_dummy_port_prepare: %s", ctx_in->lport_name); + + if (ctx_in->op_type == PLUG_OP_CREATE) { + size_t lport_name_len = strlen(ctx_in->lport_name); + ctx_out->name = xzalloc(IFNAMSIZ); + memcpy(ctx_out->name, ctx_in->lport_name, + (lport_name_len < IFNAMSIZ) ? lport_name_len : IFNAMSIZ - 1); + ctx_out->type = xstrdup("internal"); + smap_init(&ctx_out->iface_options); + smap_add(&ctx_out->iface_options, "vif-plug-dummy-option", "value"); + } + + return true; +} + +static void +vif_plug_dummy_port_finish(const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out OVS_UNUSED) +{ + VLOG_DBG("vif_plug_dummy_port_finish: %s", ctx_in->lport_name); +} + +static void +vif_plug_dummy_port_ctx_destroy(const struct vif_plug_port_ctx_in *ctx_in, + struct vif_plug_port_ctx_out *ctx_out) +{ + VLOG_DBG("vif_plug_dummy_port_ctx_destroy: %s", ctx_in->lport_name); + ovs_assert(ctx_in->op_type == PLUG_OP_CREATE); + free(ctx_out->name); + free(ctx_out->type); + smap_destroy(&ctx_out->iface_options); +} + +const struct vif_plug_class vif_plug_dummy_class = { + .type = "dummy", + .init = vif_plug_dummy_init, + .destroy = vif_plug_dummy_destroy, + .vif_plug_get_maintained_iface_options = + vif_plug_dummy_get_maintained_iface_options, + .run = vif_plug_dummy_run, + .vif_plug_port_prepare = vif_plug_dummy_port_prepare, + .vif_plug_port_finish = vif_plug_dummy_port_finish, + .vif_plug_port_ctx_destroy = vif_plug_dummy_port_ctx_destroy, +}; + +void +vif_plug_dummy_enable(void) +{ + vif_plug_provider_register(&vif_plug_dummy_class); +} + diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index a71798f68..ef8d669a2 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -67,8 +67,9 @@
  • One or more (usually many) hypervisors. Hypervisors must run Open vSwitch and implement the interface described in - Documentation/topics/integration.rst in the OVN source tree. - Any hypervisor platform supported by Open vSwitch is acceptable. + Documentation/topics/integration.rst in the Open vSwitch + source tree. Any hypervisor platform supported by Open vSwitch is + acceptable.
  • @@ -318,11 +319,19 @@
  • On a hypervisor, any VIFs that are to be attached to logical networks. - The hypervisor itself, or the integration between Open vSwitch and the - hypervisor (described in - Documentation/topics/integration.rst) takes care of this. - (This is not part of OVN or new to OVN; this is pre-existing integration - work that has already been done on hypervisors that support OVS.) + For instances connected through software emulated ports such as TUN/TAP + or VETH pairs, the hypervisor itself will normally create ports and plug + them into the integration bridge. For instances connected through + representor ports, typically used with hardware offload, the + ovn-controller may on CMS direction consult a VIF plug + provider for representor port lookup and plug them into the integration + bridge (please refer to + Documentation/topics/vif-plug-providers/vif-plug-providers.rst + for more information). In both cases the conventions described + in Documentation/topics/integration.rst in the Open vSwitch + source tree is followed to ensure mapping between OVN logical port and + VIF. (This is pre-existing integration work that has already been done + on hypervisors that support OVS.)
  • @@ -921,12 +930,12 @@ Eventually, a user powers on the VM that owns the VIF. On the hypervisor where the VM is powered on, the integration between the hypervisor and Open vSwitch (described in - Documentation/topics/integration.rst) adds the VIF to the OVN - integration bridge and stores vif-id in - external_ids:iface-id to indicate that the - interface is an instantiation of the new VIF. (None of this code is new - in OVN; this is pre-existing integration work that has already been done - on hypervisors that support OVS.) + Documentation/topics/integration.rst in the Open vSwitch + source tree) adds the VIF to the OVN integration bridge and stores + vif-id in external_ids:iface-id to + indicate that the interface is an instantiation of the new VIF. (None of + this code is new in OVN; this is pre-existing integration work that has + already been done on hypervisors that support OVS.)
  • diff --git a/tests/automake.mk b/tests/automake.mk index c4a7c0a5b..685d78c5b 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -38,7 +38,8 @@ TESTSUITE_AT = \ tests/ovn-ipam.at \ tests/ovn-features.at \ tests/ovn-lflow-cache.at \ - tests/ovn-ipsec.at + tests/ovn-ipsec.at \ + tests/ovn-vif-plug.at SYSTEM_KMOD_TESTSUITE_AT = \ tests/system-common-macros.at \ @@ -243,13 +244,23 @@ tests_ovstest_SOURCES = \ tests/test-ovn.c \ controller/test-lflow-cache.c \ controller/test-ofctrl-seqno.c \ + controller/test-vif-plug.c \ lib/test-ovn-features.c \ northd/test-ipam.c tests_ovstest_LDADD = $(OVS_LIBDIR)/daemon.lo \ $(OVS_LIBDIR)/libopenvswitch.la lib/libovn.la \ + controller/binding.$(OBJEXT) \ + controller/encaps.$(OBJEXT) \ + controller/ha-chassis.$(OBJEXT) \ + controller/if-status.$(OBJEXT) \ controller/lflow-cache.$(OBJEXT) \ + controller/local_data.$(OBJEXT) \ + controller/lport.$(OBJEXT) \ controller/ofctrl-seqno.$(OBJEXT) \ + controller/ovsport.$(OBJEXT) \ + controller/patch.$(OBJEXT) \ + controller/vif-plug.$(OBJEXT) \ northd/ipam.$(OBJEXT) # Python tests. diff --git a/tests/ovn-vif-plug.at b/tests/ovn-vif-plug.at new file mode 100644 index 000000000..86b0b4b84 --- /dev/null +++ b/tests/ovn-vif-plug.at @@ -0,0 +1,8 @@ +# +# Unit tests for the lib/vif-plug-provider.c and controller/vif-plug.c modules. +# +AT_BANNER([OVN unit tests - vif-plug]) + +AT_SETUP([unit test -- plugging infrastructure tests]) +AT_CHECK([ovstest test-plug run], [0], []) +AT_CLEANUP From patchwork Fri Nov 5 14:00:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1551395 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=RHLtlujL; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hm2G12VP5z9sXS for ; Sat, 6 Nov 2021 01:00:25 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 24104615AD; Fri, 5 Nov 2021 14:00:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3pVxFKvF_RRX; Fri, 5 Nov 2021 14:00:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 56DEB6157E; Fri, 5 Nov 2021 14:00:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1A002C000E; Fri, 5 Nov 2021 14:00:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id E253BC000E for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BFF39823E8 for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K_Huhfvg2-Hn for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by smtp1.osuosl.org (Postfix) with ESMTPS id F41068187F for ; Fri, 5 Nov 2021 14:00:17 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 18C433F1E4; Fri, 5 Nov 2021 14:00:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636120816; bh=JffA6Ond0aN4d504zMxQ7jhyaXmSDFRi6BI+kZib9PY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RHLtlujLE/mQIr/mlkRmoKKDMWMwhp5bISJzrLC1Gox1SijwBwaJY2HMwGlfqafKf WD2AmLxjdW48wSWLHaXUjFsY31y8PvtgKUqxVpsD6pDV00XdoQM+UsBHMAMq9SmjeE 1VGC6Evbh8YEfwCbd6BLOhv4rWzU8/6cFYqOztgDkhQUL3+nkyRIwB4ITX8P5N98cu wySYDLhCPUHhjf+y3sb4iGA/QWJahXsEi/hnwXGA7RFWhSNR6cW57aXKqz6kFlPZfB aB/WP4tOkU6xwoX22Cfp9OPdreXpibMLBz3At4EY/bn/67zE8Wp/Hh6UZFZIgn6wuX iTbRjIVqEuz1Q== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 5 Nov 2021 15:00:12 +0100 Message-Id: <20211105140014.3522766-3-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211105140014.3522766-1-frode.nordahl@canonical.com> References: <20211105140014.3522766-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v10 2/4] ovn-controller: Prepare VIF plug provider infrastructure. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Add `enable-dummy-plug` option - To enable testing of the VIF plug provider infrastructure without building OVN with an external VIF plug provider we include a dummy implementation which can be enabled using this command line option. Signed-off-by: Frode Nordahl --- controller/ovn-controller.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 4d99da03a..1cef04735 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -56,6 +56,8 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "patch.h" +#include "vif-plug.h" +#include "vif-plug-provider.h" #include "physical.h" #include "pinctrl.h" #include "openvswitch/poll-loop.h" @@ -3082,11 +3084,13 @@ main(int argc, char *argv[]) patch_init(); pinctrl_init(); lflow_init(); + vif_plug_provider_initialize(); /* Connect to OVS OVSDB instance. */ struct ovsdb_idl_loop ovs_idl_loop = OVSDB_IDL_LOOP_INITIALIZER( ovsdb_idl_create(ovs_remote, &ovsrec_idl_class, false, true)); ctrl_register_ovs_idl(ovs_idl_loop.idl); + ovsdb_idl_get_initial_snapshot(ovs_idl_loop.idl); /* Configure OVN SB database. */ @@ -3879,6 +3883,7 @@ loop_done: pinctrl_destroy(); patch_destroy(); if_status_mgr_destroy(if_mgr); + vif_plug_provider_destroy_all(); ovsdb_idl_loop_destroy(&ovs_idl_loop); ovsdb_idl_loop_destroy(&ovnsb_idl_loop); @@ -3899,6 +3904,7 @@ parse_options(int argc, char *argv[]) VLOG_OPTION_ENUMS, OVN_DAEMON_OPTION_ENUMS, SSL_OPTION_ENUMS, + OPT_ENABLE_DUMMY_VIF_PLUG, }; static struct option long_options[] = { @@ -3909,6 +3915,8 @@ parse_options(int argc, char *argv[]) STREAM_SSL_LONG_OPTIONS, {"peer-ca-cert", required_argument, NULL, OPT_PEER_CA_CERT}, {"bootstrap-ca-cert", required_argument, NULL, OPT_BOOTSTRAP_CA_CERT}, + {"enable-dummy-vif-plug", no_argument, NULL, + OPT_ENABLE_DUMMY_VIF_PLUG}, {NULL, 0, NULL, 0} }; char *short_options = ovs_cmdl_long_options_to_short_options(long_options); @@ -3954,6 +3962,10 @@ parse_options(int argc, char *argv[]) stream_ssl_set_ca_cert_file(optarg, true); break; + case OPT_ENABLE_DUMMY_VIF_PLUG: + vif_plug_dummy_enable(); + break; + case '?': exit(EXIT_FAILURE); From patchwork Fri Nov 5 14:00:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1551398 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=XtuBwId3; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hm2GB0NLXz9sfG for ; Sat, 6 Nov 2021 01:00:34 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 51B148272D; Fri, 5 Nov 2021 14:00:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6XAmF0U20eTW; Fri, 5 Nov 2021 14:00:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id F333482768; Fri, 5 Nov 2021 14:00:24 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6C5C3C003F; Fri, 5 Nov 2021 14:00:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 63950C0019 for ; Fri, 5 Nov 2021 14:00:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 537588187F for ; Fri, 5 Nov 2021 14:00:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XpRc4DfFest6 for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by smtp1.osuosl.org (Postfix) with ESMTPS id 287DD81D5F for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 85F4D3F1E5; Fri, 5 Nov 2021 14:00:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636120816; bh=+91ycczBYRVylPELCU8/Pxa3iYg0RlsRqZ55DvZYfC0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XtuBwId3HBtSpch2RE1Y6WNxH0w68ZwWfDX2LDHutoBl8gBI7rGV1mV94eylL9MWr ZUg3Julwcye7YoNTSNPr6YViCWkqyhpFAzQP7Bhj9QWqUVjysqzPUIiNCS+mIsv5MT JxuECF/qJny54CLdtFR66JuD/NrIikg+pZ/sj9l4fKPbkJa7OwdCgi8QMDI+7BRlo2 RP3ewCTpktuVvC7txD+YXyuft4XjRoHBlaUzR0IMMOAdZqT9QKwivevGBgu9C0cL0Y q+HNGo1tKYUxqouVV10Aal4v4fbEkhn4413dSCh76B3HCvYmbKcBnE74dRcWw/D2eG aruyFNu12cstQ== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 5 Nov 2021 15:00:13 +0100 Message-Id: <20211105140014.3522766-4-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211105140014.3522766-1-frode.nordahl@canonical.com> References: <20211105140014.3522766-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v10 3/4] controller: Consider plugging VIF on CMS request. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When OVN is linked with an appropriate VIF plug provider, CMS can request an OVN controller to plug individual lports into the Open vSwitch instance it manages. The port and interface record will be maintained throughout the lifetime of the lport and it will be removed on release of lport. Add port by interfaces index - To be able to effectively remove ports previously plugged by us we need to look up ports by interface records. Add Port_Binding by requested_chassis index - To be able to effectively iterate over ports destined to our chassis we need to look up Port_Binding records by requested_chassis. Signed-off-by: Frode Nordahl --- controller/ovn-controller.c | 69 +++++++++++++++++++- ovn-nb.xml | 21 +++++++ tests/ovn-macros.at | 2 +- tests/ovn.at | 121 ++++++++++++++++++++++++++++++++++++ 4 files changed, 211 insertions(+), 2 deletions(-) diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 1cef04735..f223e0f09 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -106,6 +106,7 @@ static unixctl_cb_func debug_delay_nb_cfg_report; #define IF_STATUS_MGR_UPDATE_STOPWATCH_NAME "if-status-mgr-update" #define OFCTRL_SEQNO_RUN_STOPWATCH_NAME "ofctrl-seqno-run" #define BFD_RUN_STOPWATCH_NAME "bfd-run" +#define VIF_PLUG_RUN_STOPWATCH_NAME "vif-plug-run" #define OVS_NB_CFG_NAME "ovn-nb-cfg" #define OVS_NB_CFG_TS_NAME "ovn-nb-cfg-ts" @@ -937,6 +938,7 @@ ctrl_register_ovs_idl(struct ovsdb_idl *ovs_idl) binding_register_ovs_idl(ovs_idl); bfd_register_ovs_idl(ovs_idl); physical_register_ovs_idl(ovs_idl); + vif_plug_register_ovs_idl(ovs_idl); } #define SB_NODES \ @@ -3091,6 +3093,10 @@ main(int argc, char *argv[]) ovsdb_idl_create(ovs_remote, &ovsrec_idl_class, false, true)); ctrl_register_ovs_idl(ovs_idl_loop.idl); + struct ovsdb_idl_index *ovsrec_port_by_interfaces + = ovsdb_idl_index_create1(ovs_idl_loop.idl, + &ovsrec_port_col_interfaces); + ovsdb_idl_get_initial_snapshot(ovs_idl_loop.idl); /* Configure OVN SB database. */ @@ -3126,6 +3132,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *sbrec_port_binding_by_type = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_port_binding_col_type); + struct ovsdb_idl_index *sbrec_port_binding_by_requested_chassis + = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, + &sbrec_port_binding_col_requested_chassis); struct ovsdb_idl_index *sbrec_datapath_binding_by_key = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_datapath_binding_col_tunnel_key); @@ -3198,6 +3207,7 @@ main(int argc, char *argv[]) stopwatch_create(IF_STATUS_MGR_UPDATE_STOPWATCH_NAME, SW_MS); stopwatch_create(OFCTRL_SEQNO_RUN_STOPWATCH_NAME, SW_MS); stopwatch_create(BFD_RUN_STOPWATCH_NAME, SW_MS); + stopwatch_create(VIF_PLUG_RUN_STOPWATCH_NAME, SW_MS); /* Define inc-proc-engine nodes. */ ENGINE_NODE_WITH_CLEAR_TRACK_DATA_IS_VALID(ct_zones, "ct_zones"); @@ -3429,6 +3439,11 @@ main(int argc, char *argv[]) }; struct if_status_mgr *if_mgr = ctrl_engine_ctx.if_mgr; + struct shash vif_plug_deleted_iface_ids = + SHASH_INITIALIZER(&vif_plug_deleted_iface_ids); + struct shash vif_plug_changed_iface_ids = + SHASH_INITIALIZER(&vif_plug_changed_iface_ids); + char *ovn_version = ovn_get_internal_version(); VLOG_INFO("OVN internal version is : [%s]", ovn_version); @@ -3485,6 +3500,7 @@ main(int argc, char *argv[]) if (!new_ovnsb_cond_seqno) { VLOG_INFO("OVNSB IDL reconnected, force recompute."); engine_set_force_recompute(true); + vif_plug_reset_idl_prime_counter(); } ovnsb_cond_seqno = new_ovnsb_cond_seqno; } @@ -3639,6 +3655,37 @@ main(int argc, char *argv[]) ovsrec_port_table_get(ovs_idl_loop.idl), br_int, chassis, &runtime_data->local_datapaths); stopwatch_stop(PATCH_RUN_STOPWATCH_NAME, time_msec()); + if (vif_plug_provider_has_providers() && ovs_idl_txn) { + struct vif_plug_ctx_in vif_plug_ctx_in = { + .ovs_idl_txn = ovs_idl_txn, + .sbrec_port_binding_by_name = + sbrec_port_binding_by_name, + .sbrec_port_binding_by_requested_chassis = + sbrec_port_binding_by_requested_chassis, + .ovsrec_port_by_interfaces = + ovsrec_port_by_interfaces, + .ovs_table = ovs_table, + .br_int = br_int, + .iface_table = + ovsrec_interface_table_get( + ovs_idl_loop.idl), + .chassis_rec = chassis, + .local_bindings = + &runtime_data->lbinding_data.bindings, + }; + struct vif_plug_ctx_out vif_plug_ctx_out = { + .deleted_iface_ids = + &vif_plug_deleted_iface_ids, + .changed_iface_ids = + &vif_plug_changed_iface_ids, + }; + stopwatch_start(VIF_PLUG_RUN_STOPWATCH_NAME, + time_msec()); + vif_plug_run(&vif_plug_ctx_in, + &vif_plug_ctx_out); + stopwatch_stop(VIF_PLUG_RUN_STOPWATCH_NAME, + time_msec()); + } stopwatch_start(PINCTRL_RUN_STOPWATCH_NAME, time_msec()); pinctrl_run(ovnsb_idl_txn, @@ -3795,7 +3842,16 @@ main(int argc, char *argv[]) engine_set_force_recompute(true); } - if (ovsdb_idl_loop_commit_and_wait(&ovs_idl_loop) == 1) { + int ovs_txn_status = ovsdb_idl_loop_commit_and_wait(&ovs_idl_loop); + if (!ovs_txn_status) { + /* The transaction failed. */ + vif_plug_clear_deleted( + &vif_plug_deleted_iface_ids); + vif_plug_clear_changed( + &vif_plug_changed_iface_ids); + } else if (ovs_txn_status == 1) { + /* The transaction committed successfully + * (or it did not change anything in the database). */ ct_zones_data = engine_get_data(&en_ct_zones); if (ct_zones_data) { struct shash_node *iter, *iter_next; @@ -3808,6 +3864,15 @@ main(int argc, char *argv[]) } } } + + vif_plug_finish_deleted( + &vif_plug_deleted_iface_ids); + vif_plug_finish_changed( + &vif_plug_changed_iface_ids); + } else if (ovs_txn_status == -1) { + /* The commit is still in progress */ + } else { + OVS_NOT_REACHED(); } ovsdb_idl_track_clear(ovnsb_idl_loop.idl); @@ -3883,6 +3948,8 @@ loop_done: pinctrl_destroy(); patch_destroy(); if_status_mgr_destroy(if_mgr); + shash_destroy(&vif_plug_deleted_iface_ids); + shash_destroy(&vif_plug_changed_iface_ids); vif_plug_provider_destroy_all(); ovsdb_idl_loop_destroy(&ovs_idl_loop); diff --git a/ovn-nb.xml b/ovn-nb.xml index e31578fb6..036ffa64f 100644 --- a/ovn-nb.xml +++ b/ovn-nb.xml @@ -1031,6 +1031,27 @@ DHCP reply.

    + + + + If set, OVN will attempt to perform plugging of this VIF. In order + to get this port plugged by the OVN controller, OVN must be built + with support for VIF plugging. The default behavior is for the CMS + to do the VIF plugging. Each VIF plug provider have their own + options namespaced by name, for example "vif-plug:representor:key". + + Please refer to the VIF plug provider documentation located in + Documentation/topics/vif-plug-providers/ for more information. + + + + Requested MTU for plugged interfaces. When set the OVN controller + will fill the column + of the Open vSwitch database's + table. This in turn will + make OVS vswitchd update the MTU of the linked interface. + + diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index f06f2e68e..21b7d985e 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -327,7 +327,7 @@ ovn_az_attach() { -- --may-exist add-br br-int \ -- set bridge br-int fail-mode=secure other-config:disable-in-band=true \ || return 1 - start_daemon ovn-controller || return 1 + start_daemon ovn-controller --enable-dummy-vif-plug || return 1 } # ovn_attach NETWORK BRIDGE IP [MASKLEN] diff --git a/tests/ovn.at b/tests/ovn.at index 6f6f5c2da..8163ad84a 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -28818,3 +28818,124 @@ AT_CHECK([grep -q "Not claiming" hv1/ovn-controller.log], [1], []) OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-controller - VIF plugging]) +AT_KEYWORDS([vif-plug]) + +ovn_start + +net_add n1 +sim_add hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.1 + +sim_add hv2 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.2 + +check ovn-nbctl ls-add lsw0 + +check ovn-nbctl lsp-add lsw0 lsp1 +check ovn-nbctl lsp-set-addresses lsp1 "f0:00:00:00:00:01 172.16.0.101" +check ovn-nbctl lsp-set-options lsp1 \ + requested-chassis=hv1 \ + vif-plug-type=dummy \ + vif-plug-mtu-request=42 + +check ovn-nbctl lsp-add lsw0 lsp2 +check ovn-nbctl lsp-set-addresses lsp2 "f0:00:00:00:00:02 172.16.0.102" +check ovn-nbctl lsp-set-options lsp2 \ + requested-chassis=hv2 \ + vif-plug-type=dummy \ + vif-plug-mtu-request=42 + +wait_for_ports_up lsp1 lsp2 + +lsp1_uuid=$(ovn-nbctl --bare --columns _uuid find Logical_Switch_Port name=lsp1) +iface1_uuid=$(as hv1 ovs-vsctl --bare --columns _uuid find Interface name=lsp1) + +lsp2_uuid=$(ovn-nbctl --bare --columns _uuid find Logical_Switch_Port name=lsp2) +iface2_uuid=$(as hv2 ovs-vsctl --bare --columns _uuid find Interface name=lsp2) + +# Check that the lport was plugged +AT_CHECK([test xvalue = x$(as hv1 ovs-vsctl get Interface ${iface1_uuid} options:vif-plug-dummy-option)], [0], []) +AT_CHECK([test x42 = x$(as hv1 ovs-vsctl get Interface ${iface1_uuid} mtu_request)], [0], []) + +# Check that updating the lport updates the local iface +check ovn-nbctl --wait=hv lsp-set-options lsp1 \ + requested-chassis=hv1 \ + vif-plug-type=dummy \ + vif-plug-mtu-request=43 +OVS_WAIT_UNTIL([ + test x43 = x$(as hv1 ovs-vsctl get Interface ${iface1_uuid} mtu_request) +]) + +# Check that local modification of iface will trigger ovn-controller to update +# the iface record +check as hv1 ovs-vsctl set interface ${iface1_uuid} mtu_request=44 +OVS_WAIT_UNTIL([ + test x43 = x$(as hv1 ovs-vsctl get Interface ${iface1_uuid} mtu_request) +]) + +# Check that pointing requested-chassis somewhere else will unplug the port +check ovn-nbctl --wait=hv set Logical_Switch_Port lsp1 \ + options:requested-chassis=non-existent-chassis +OVS_WAIT_UNTIL([ + ! as hv1 ovs-vsctl get Interface ${iface1_uuid} _uuid +]) + +# Check that removing an lport will unplug it +AT_CHECK([test x${iface2_uuid} = x$(as hv2 ovs-vsctl get Interface ${iface2_uuid} _uuid)], [0], []) +check ovn-nbctl --wait=hv lsp-del ${lsp2_uuid} +OVS_WAIT_UNTIL([ + ! as hv2 ovs-vsctl get Interface ${iface2_uuid} _uuid +]) + +# Check that port is unplugged when we simulate presence of a port previously +# plugged by us in local OVSDB with no record in SB DB. +check as hv2 ovs-vsctl \ + -- add-port br-int vif1 + +# From one moment it's there... +vif1_uuid=$(as hv2 ovs-vsctl --bare --columns _uuid find Interface name=vif1) +OVS_WAIT_UNTIL([ + as hv2 ovs-vsctl get Interface ${vif1_uuid} _uuid +]) + +# Add the external-ids we expect +check as hv2 ovs-vsctl \ + -- set Interface ${vif1_uuid} \ + external-ids:ovn-plugged=dummy \ + external-ids:iface-id=non-existing-lsp + +# ...to the next moment it's gone. +OVS_WAIT_UNTIL([ + ! as hv2 ovs-vsctl get Interface ${vif1_uuid} _uuid +]) + +# Check that a warning is logged when CMS requests plugging of an interface +# with lbinding already plugged by someone else. +check as hv2 ovs-vsctl \ + -- add-port br-int vif3 \ + -- set Interface vif3 \ + external-ids:iface-id=lsp3 + +check ovn-nbctl lsp-add lsw0 lsp3 +check ovn-nbctl lsp-set-addresses lsp3 "f0:00:00:00:00:03 172.16.0.103" +check ovn-nbctl lsp-set-options lsp3 \ + requested-chassis=hv2 + +wait_for_ports_up lsp3 + +check ovn-nbctl --wait=hv lsp-set-options lsp3 \ + requested-chassis=hv2 \ + vif-plug-type=dummy + +OVS_WAIT_UNTIL([ + grep -q "CMS requested plugging of lport lsp3" hv2/ovn-controller.log +]) + +OVN_CLEANUP([hv1],[hv2]) +AT_CLEANUP +]) From patchwork Fri Nov 5 14:00:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frode Nordahl X-Patchwork-Id: 1551396 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=b3BALp4t; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4Hm2G46h7Zz9sXS for ; Sat, 6 Nov 2021 01:00:28 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CC655404B3; Fri, 5 Nov 2021 14:00:25 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2NSc8h1iH5D0; Fri, 5 Nov 2021 14:00:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id C091640499; Fri, 5 Nov 2021 14:00:23 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C7E2AC0038; Fri, 5 Nov 2021 14:00:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 54F5BC000E for ; Fri, 5 Nov 2021 14:00:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 4F52A4077D for ; Fri, 5 Nov 2021 14:00:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=canonical.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id miC2wR6M62ww for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from smtp-relay-canonical-0.canonical.com (smtp-relay-canonical-0.canonical.com [185.125.188.120]) by smtp4.osuosl.org (Postfix) with ESMTPS id B62BE40775 for ; Fri, 5 Nov 2021 14:00:18 +0000 (UTC) Received: from frode-threadripper.. (1.general.frode.uk.vpn [10.172.193.250]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-canonical-0.canonical.com (Postfix) with ESMTPSA id 070013F1F2; Fri, 5 Nov 2021 14:00:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1636120817; bh=4ckgvsP0hz+JWPd2PW+QcPj23FU3HW0Lva43NfYv2JI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b3BALp4tjDZ9WvVGoDZhUybAI2/eGyMU70dl25+Nvvs+2ZjFtg4CmMeiw1hT7N3V2 rVdBYgikyhJzZqJE5/DDmMZxp27gH1KwqT+lwMdeaXWLZRlLNZHN6PSwyfWs9uMJ12 x+7YQBhaLgvev1EapqoOzqrDwT1Sx5JdzaQnP+/J0GIWrbnSxR2rYDge8FIk3xfxUG Ev4ecbs9FOkGUM9ebzBfrGeAsEjMp+l7zr1jrQX7yZyGh9wLiBN0JpCutEciGB1Xe2 ovPdItqjs0Nfo7dA+0Sjk/FMvQuqebSl4fJNJ5x8b2CtTgWdIwp5MSwKGar+kVInY+ AQcwUK9zZXPjA== From: Frode Nordahl To: dev@openvswitch.org Date: Fri, 5 Nov 2021 15:00:14 +0100 Message-Id: <20211105140014.3522766-5-frode.nordahl@canonical.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211105140014.3522766-1-frode.nordahl@canonical.com> References: <20211105140014.3522766-1-frode.nordahl@canonical.com> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v10 4/4] NEWS: Add note on infrastructure for VIF plug providers. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Signed-off-by: Frode Nordahl --- NEWS | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/NEWS b/NEWS index 5f448e67d..97a833a1b 100644 --- a/NEWS +++ b/NEWS @@ -6,6 +6,12 @@ Post v21.09.0 if not desired. - Added Load_Balancer_Group support, which simplifies large scale configurations of load balancers. + - Introduced infrastructure for VIF plug providers. When OVN is linked with + an appropriate VIF plug provider CMS can request OVN to plug lports. This + is particularly useful in topologies where the ovn-controller process is + running on SmartNIC control plane CPUs. Please refer to + Documentation/topics/vif-plug-providers/vif-plug-providers.rst for more + information. OVN v21.09.0 - 01 Oct 2021 --------------------------