From patchwork Mon Nov 2 14:28:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Michelson X-Patchwork-Id: 1392288 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=UCKy6st3; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CPwJq5gLrz9sVM for ; Tue, 3 Nov 2020 01:28:59 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 5E7D6873D5; Mon, 2 Nov 2020 14:28:58 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TEpNofAH1zGh; Mon, 2 Nov 2020 14:28:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 42B5387222; Mon, 2 Nov 2020 14:28:57 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2F088C1AD8; Mon, 2 Nov 2020 14:28:57 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9C54FC0889 for ; Mon, 2 Nov 2020 14:28:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 84563204E1 for ; Mon, 2 Nov 2020 14:28:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zDRgO6ZwVGXV for ; Mon, 2 Nov 2020 14:28:45 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by silver.osuosl.org (Postfix) with ESMTPS id AFB3E204CA for ; Mon, 2 Nov 2020 14:28:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1604327320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=05n9FGVjm1sn/o0K9axD2REMeOo1NjqdotNpc7jvJIo=; b=UCKy6st3p6g2FYUO4RMCN+BX40aWdu58zIqth2/rksO/CKxGVhlaVXKkeAjkFlFOgqzyO6 crr/vuQSOgJ2eMVcnHa4/4gkmid+joVH/k8ga06CMQDGV3Wey9GnoZ1HP65ZZbffu5amCP IBG0c9tNeWy8zs8Zee4bMuFIF/i++6w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-172-n9x3s55EMly6HuhLQjX0NA-1; Mon, 02 Nov 2020 09:28:30 -0500 X-MC-Unique: n9x3s55EMly6HuhLQjX0NA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id F162E8030D1 for ; Mon, 2 Nov 2020 14:28:29 +0000 (UTC) Received: from monae.redhat.com (ovpn-112-233.rdu2.redhat.com [10.10.112.233]) by smtp.corp.redhat.com (Postfix) with ESMTP id 865F555766 for ; Mon, 2 Nov 2020 14:28:29 +0000 (UTC) From: Mark Michelson To: dev@openvswitch.org Date: Mon, 2 Nov 2020 09:28:26 -0500 Message-Id: <20201102142828.3444732-2-mmichels@redhat.com> In-Reply-To: <20201102142828.3444732-1-mmichels@redhat.com> References: <20201102142828.3444732-1-mmichels@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mmichels@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [RFC PATCH ovn v3 1/3] Add unit test framework. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This creates an API that allows for unit tests to be registered in code by OVN. This change enables the following: * At configure time, "--enable-unit-tests" can be specified to allow for unit tests to be compiled. * This configure-time variable also enables a testsuite variable to be defined, allowing for unit tests to be run from the testsuite. * Testsuite tests are defined using OVS_CONSTRUCTOR, meaning they are automatically registered prior to main() being run in whichever program the unit test is defined in. * Unit tests can be written as standalone programs using the ovstest framework. This works so long as the symbols referenced in the unit test can be found in a header file. Unit tests written in this way don't rely on anything introduced in this commit and have existed in the testsuite via the test-ovn.c file for a while. * Unit tests can also be written within the source of a file and accessed using ovn-appctl. This can be useful for testing private functions within source code. Example: `ovn-appctl -t northd unit-test my-unit-test [args]`. Unit tests written this way require the use of the library introduced in this commit. This commit does not define any unit tests. That is saved for a later commit in this series. Signed-off-by: Mark Michelson --- acinclude.m4 | 12 +++++ configure.ac | 1 + lib/automake.mk | 4 +- lib/unit-test.c | 120 +++++++++++++++++++++++++++++++++++++++++++++++ lib/unit-test.h | 34 ++++++++++++++ tests/atlocal.in | 1 + 6 files changed, 171 insertions(+), 1 deletion(-) create mode 100644 lib/unit-test.c create mode 100644 lib/unit-test.h diff --git a/acinclude.m4 b/acinclude.m4 index a797adc82..8b1888075 100644 --- a/acinclude.m4 +++ b/acinclude.m4 @@ -366,3 +366,15 @@ AC_DEFUN([OVN_CHECK_OVS], [ AC_SUBST(OVSVERSION) AC_MSG_RESULT([OVS version is $OVSVERSION]) ]) + +AC_DEFUN([OVN_ENABLE_UNIT_TESTS], + [AC_ARG_ENABLE( + [unit_tests], + [AC_HELP_STRING([--enable-unit-tests], [Enable unit test framework])], + [], [enable_unit_tests=no]) + AC_CONFIG_COMMANDS_PRE( + [if test "X$enable_unit_tests" = Xyes; then + OVS_CFLAGS="$OVS_CFLAGS -DENABLE_UNIT_TESTS" + fi]) + + AC_SUBST([enable_unit_tests])]) diff --git a/configure.ac b/configure.ac index 0b17f05b9..613e2ba3d 100644 --- a/configure.ac +++ b/configure.ac @@ -166,6 +166,7 @@ OVS_CONDITIONAL_CC_OPTION([-Wno-unused], [HAVE_WNO_UNUSED]) OVS_CONDITIONAL_CC_OPTION([-Wno-unused-parameter], [HAVE_WNO_UNUSED_PARAMETER]) OVS_ENABLE_WERROR OVS_ENABLE_SPARSE +OVN_ENABLE_UNIT_TESTS OVS_CHECK_PRAGMA_MESSAGE OVN_CHECK_OVS diff --git a/lib/automake.mk b/lib/automake.mk index f3e9c8818..3e9c2a697 100644 --- a/lib/automake.mk +++ b/lib/automake.mk @@ -23,7 +23,9 @@ lib_libovn_la_SOURCES = \ lib/ovn-util.h \ lib/logical-fields.c \ lib/inc-proc-eng.c \ - lib/inc-proc-eng.h + lib/inc-proc-eng.h \ + lib/unit-test.c \ + lib/unit-test.h nodist_lib_libovn_la_SOURCES = \ lib/ovn-dirs.c \ lib/ovn-nb-idl.c \ diff --git a/lib/unit-test.c b/lib/unit-test.c new file mode 100644 index 000000000..3a12727c6 --- /dev/null +++ b/lib/unit-test.c @@ -0,0 +1,120 @@ +/* + * Copyright (c) 2020 Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include "openvswitch/shash.h" +#include "openvswitch/dynamic-string.h" +#include "util.h" +#include "unixctl.h" +#include "command-line.h" + +#include "lib/unit-test.h" + +#ifdef ENABLE_UNIT_TESTS +static struct shash unit_tests = SHASH_INITIALIZER(&unit_tests); + +struct unit_test_data { + size_t min_args; + size_t max_args; + ovs_cmdl_handler cb; +}; + +void +register_unit_test(const char *name, size_t min_args, size_t max_args, + ovs_cmdl_handler cb) +{ + struct unit_test_data *test_data = xmalloc(sizeof *test_data); + test_data->min_args = min_args; + test_data->max_args = max_args; + test_data->cb = cb; + shash_add_once(&unit_tests, name, test_data); +} + +void +run_unit_test(struct ovs_cmdl_context *ctx) +{ + struct unit_test_data *test_data = shash_find_data(&unit_tests, + ctx->argv[0]); + if (!test_data) { + struct ds *output = ctx->pvt; + ds_put_format(output, "Unable to find unit test %s\n", ctx->argv[0]); + return; + } + + int test_args = ctx->argc - 1; /* Ignore test name */ + if (test_args < test_data->min_args || + test_args > test_data->max_args) { + struct ds *output = ctx->pvt; + ds_put_format(output, "Invalid number of arguments. " + "Minimum: %"PRIuSIZE". Maxium: %"PRIuSIZE". " + "Got: %d\n", + test_data->min_args, test_data->max_args, + test_args); + } + test_data->cb(ctx); +} + +static void +unixctl_run_unit_test(struct unixctl_conn *conn, int argc, + const char *argv[], void *ignore OVS_UNUSED) +{ + if (argc < 2) { + unixctl_command_reply_error(conn, "No unit test specified"); + return; + } + + struct ds reply = DS_EMPTY_INITIALIZER; + struct ovs_cmdl_context ctx = { + .argc = argc - 1, + .argv = (char **) argv + 1, + .pvt = &reply, + }; + run_unit_test(&ctx); + unixctl_command_reply(conn, ds_cstr(&reply)); + + ds_destroy(&reply); +} + +void +register_unixctl_unit_test(void) +{ + unixctl_command_register("unit-test", "", 1, UINT_MAX, + unixctl_run_unit_test, NULL); +} + +#else /* ENABLE_UNIT_TESTS */ + +void +register_unit_test(const char *name OVS_UNUSED, size_t min_args OVS_UNUSED, + size_t max_args OVS_UNUSED, ovs_cmdl_handler cb OVS_UNUSED) +{ + return; +} + +void +run_unit_test(struct ovs_cmdl_context *ctx OVS_UNUSED) +{ + return; +} + +void +register_unixctl_unit_test(void) +{ + return; +} + +#endif /* ENABLE_UNIT_TESTS */ diff --git a/lib/unit-test.h b/lib/unit-test.h new file mode 100644 index 000000000..8dc4e2cd7 --- /dev/null +++ b/lib/unit-test.h @@ -0,0 +1,34 @@ +/* + * Copyright (c) 2020 Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef OVN_UNIT_TEST_H +#define OVN_UNIT_TEST_H 1 + +#include "openvswitch/compiler.h" +#include "command-line.h" + +void register_unit_test(const char *name, size_t min_args, size_t max_args, + ovs_cmdl_handler cb); + +void run_unit_test(struct ovs_cmdl_context *ctx); + +void register_unixctl_unit_test(void); + +#define UNIT_TEST_DEFINE(NAME, MIN_ARGS, MAX_ARGS, CB) \ + OVS_CONSTRUCTOR(unit_test_##NAME) { \ + register_unit_test(#NAME, MIN_ARGS, MAX_ARGS, CB); \ + } +#endif diff --git a/tests/atlocal.in b/tests/atlocal.in index 26681f02d..ca48ee93d 100644 --- a/tests/atlocal.in +++ b/tests/atlocal.in @@ -3,6 +3,7 @@ HAVE_OPENSSL='@HAVE_OPENSSL@' OPENSSL_SUPPORTS_SNI='@OPENSSL_SUPPORTS_SNI@' HAVE_UNBOUND='@HAVE_UNBOUND@' EGREP='@EGREP@' +ENABLE_UNIT_TESTS='@enable_unit_tests@' if test x"$PYTHON3" = x; then PYTHON3='@PYTHON3@' From patchwork Mon Nov 2 14:28:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Michelson X-Patchwork-Id: 1392286 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=P7kejJXx; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CPwJZ71v5z9sVp for ; Tue, 3 Nov 2020 01:28:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id E15338738D; Mon, 2 Nov 2020 14:28:43 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GC2rJiBMx0XC; Mon, 2 Nov 2020 14:28:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by hemlock.osuosl.org (Postfix) with ESMTP id 9AF2887385; Mon, 2 Nov 2020 14:28:41 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 82908C0889; Mon, 2 Nov 2020 14:28:41 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 22855C1AD7 for ; Mon, 2 Nov 2020 14:28:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id CDDA7204A2 for ; Mon, 2 Nov 2020 14:28:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id a+fFHLlmOnGJ for ; Mon, 2 Nov 2020 14:28:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by silver.osuosl.org (Postfix) with ESMTPS id 3B934203D9 for ; Mon, 2 Nov 2020 14:28:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1604327314; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uPiAxf0WcpIzvqIQ23Im9ByqWBkd3x0CNn5ZKe0qLRg=; b=P7kejJXxJSsgC2R0I13d/GtEm2W0yhyutWWcTifaWYfnQs+Wi7rp+0XfJobOSiPtWylRDM 4w5et/iw3FFYp8Re/vtTOzFN0XerNgV0ZP2WfMkngHQtonXmMDYCFr0NWQIQ8/OC2bcqRR 3D5yKEof4o5qgTfb4ovYcG901dzCZ6w= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-363-hyACecQhO2ajsLlIM3dVHA-1; Mon, 02 Nov 2020 09:28:31 -0500 X-MC-Unique: hyACecQhO2ajsLlIM3dVHA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 995C710A0B81 for ; Mon, 2 Nov 2020 14:28:30 +0000 (UTC) Received: from monae.redhat.com (ovpn-112-233.rdu2.redhat.com [10.10.112.233]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2FA825579F for ; Mon, 2 Nov 2020 14:28:30 +0000 (UTC) From: Mark Michelson To: dev@openvswitch.org Date: Mon, 2 Nov 2020 09:28:27 -0500 Message-Id: <20201102142828.3444732-3-mmichels@redhat.com> In-Reply-To: <20201102142828.3444732-1-mmichels@redhat.com> References: <20201102142828.3444732-1-mmichels@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mmichels@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [RFC PATCH ovn v3 2/3] northd: refactor and split some IPAM functions X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This refactor focuses on three efforts: * Break a large function into smaller functions * Pass more specific data types to functions * Isolate these functions in their own code unit Smaller functions have clearer purposes, have fewer chances of unwanted side effects, and are easier to test. The next commit in this series will add some unit tests that exercise the new functions created in this commit. Note that this is not a full refactor of IPAM. Dynamic address assignment is complicated and currently tightly coupled with datapath and port constructs. A refactor of that section of IPAM is difficult enough that it should have its own patch series separate from this one that focuses mostly on unit tests. Signed-off-by: Mark Michelson --- northd/automake.mk | 5 +- northd/ipam.c | 328 ++++++++++++++++++++++++++++++++++++++++++++ northd/ipam.h | 37 +++++ northd/ovn-northd.c | 285 +++----------------------------------- 4 files changed, 389 insertions(+), 266 deletions(-) create mode 100644 northd/ipam.c create mode 100644 northd/ipam.h diff --git a/northd/automake.mk b/northd/automake.mk index 69657e77e..3340178f5 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -1,6 +1,9 @@ # ovn-northd bin_PROGRAMS += northd/ovn-northd -northd_ovn_northd_SOURCES = northd/ovn-northd.c +northd_ovn_northd_SOURCES = \ + northd/ovn-northd.c \ + northd/ipam.c \ + northd/ipam.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/ipam.c b/northd/ipam.c new file mode 100644 index 000000000..e5383c46f --- /dev/null +++ b/northd/ipam.c @@ -0,0 +1,328 @@ +#include + +#include +#include +#include +#include + +#include "ipam.h" +#include "lib/unit-test.h" +#include "ovn/lex.h" + +#include "smap.h" +#include "packets.h" +#include "bitmap.h" +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(ipam) + +static void +init_ipam_ipv6_prefix(const char *ipv6_prefix, struct ipam_info *info) +{ + info->ipv6_prefix_set = false; + info->ipv6_prefix = in6addr_any; + + if (!ipv6_prefix) { + return; + } + + /* XXX Since we only accept /64 addresses, why do we even bother + * with accepting and trying to analyze a user-provided mask? + */ + if (strchr(ipv6_prefix, '/')) { + /* If a prefix length was specified, it must be 64. */ + struct in6_addr mask; + char *error + = ipv6_parse_masked(ipv6_prefix, + &info->ipv6_prefix, &mask); + if (error) { + static struct vlog_rate_limit rl + = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s: %s", + ipv6_prefix, error); + free(error); + } else { + if (ipv6_count_cidr_bits(&mask) == 64) { + info->ipv6_prefix_set = true; + } else { + static struct vlog_rate_limit rl + = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s: must be /64", + ipv6_prefix); + } + } + } else { + info->ipv6_prefix_set = ipv6_parse( + ipv6_prefix, &info->ipv6_prefix); + if (!info->ipv6_prefix_set) { + static struct vlog_rate_limit rl + = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s", ipv6_prefix); + } + } + + if (info->ipv6_prefix_set) { + /* Make sure nothing past first 64 bits are set */ + struct in6_addr mask = ipv6_create_mask(64); + info->ipv6_prefix = ipv6_addr_bitand(&info->ipv6_prefix, &mask); + } +} + +static void +init_ipam_ipv4(const char *subnet_str, const char *exclude_ip_list, + struct ipam_info *info) +{ + info->start_ipv4 = 0; + info->total_ipv4s = 0; + info->allocated_ipv4s = NULL; + + ovs_be32 subnet, mask; + char *error = ip_parse_masked(subnet_str, &subnet, &mask); + if (error || mask == OVS_BE32_MAX || !ip_is_cidr(mask)) { + static struct vlog_rate_limit rl + = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad 'subnet' %s", subnet_str); + free(error); + return; + } + + info->start_ipv4 = ntohl(subnet & mask) + 1; + info->total_ipv4s = ~ntohl(mask); + info->allocated_ipv4s = + bitmap_allocate(info->total_ipv4s); + + /* Mark first IP as taken */ + bitmap_set1(info->allocated_ipv4s, 0); + + if (!exclude_ip_list) { + return; + } + + struct lexer lexer; + lexer_init(&lexer, exclude_ip_list); + /* exclude_ip_list could be in the format - + * "10.0.0.4 10.0.0.10 10.0.0.20..10.0.0.50 10.0.0.100..10.0.0.110". + */ + lexer_get(&lexer); + while (lexer.token.type != LEX_T_END) { + if (lexer.token.type != LEX_T_INTEGER) { + lexer_syntax_error(&lexer, "expecting address"); + break; + } + uint32_t start = ntohl(lexer.token.value.ipv4); + lexer_get(&lexer); + + uint32_t end = start + 1; + if (lexer_match(&lexer, LEX_T_ELLIPSIS)) { + if (lexer.token.type != LEX_T_INTEGER) { + lexer_syntax_error(&lexer, "expecting address range"); + break; + } + end = ntohl(lexer.token.value.ipv4) + 1; + lexer_get(&lexer); + } + + /* Clamp start...end to fit the subnet. */ + start = MAX(info->start_ipv4, start); + end = MIN(info->start_ipv4 + info->total_ipv4s, end); + if (end > start) { + bitmap_set_multiple(info->allocated_ipv4s, + start - info->start_ipv4, + end - start, 1); + } else { + lexer_error(&lexer, "excluded addresses not in subnet"); + } + } + if (lexer.error) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + /* + * VLOG_WARN_RL(&rl, "logical switch "UUID_FMT": bad exclude_ips (%s)", + * UUID_ARGS(&od->key), lexer.error); + */ + VLOG_WARN_RL(&rl, "logical switch: bad exclude_ips (%s)", lexer.error); + } + lexer_destroy(&lexer); +} + +void +init_ipam_info(struct ipam_info *info, const struct smap *config) +{ + const char *subnet_str = smap_get(config, "subnet"); + const char *ipv6_prefix = smap_get(config, "ipv6_prefix"); + const char *exclude_ips = smap_get(config, "exclude_ips"); + + init_ipam_ipv6_prefix(ipv6_prefix, info); + + if (!subnet_str) { + if (!ipv6_prefix) { + info->mac_only = smap_get_bool(config, "mac_only", false); + } + return; + } + + init_ipam_ipv4(subnet_str, exclude_ips, info); +} + +void +destroy_ipam_info(struct ipam_info *info) +{ + bitmap_free(info->allocated_ipv4s); +} + +bool +ipam_insert_ip(struct ipam_info *info, uint32_t ip) +{ + if (!info->allocated_ipv4s) { + return true; + } + + if (ip >= info->start_ipv4 && + ip < (info->start_ipv4 + info->total_ipv4s)) { + if (bitmap_is_set(info->allocated_ipv4s, + ip - info->start_ipv4)) { + return false; + } + bitmap_set1(info->allocated_ipv4s, + ip - info->start_ipv4); + } + return true; +} + +uint32_t +ipam_get_unused_ip(struct ipam_info *info) +{ + if (!info->allocated_ipv4s) { + return 0; + } + + size_t new_ip_index = bitmap_scan(info->allocated_ipv4s, 0, 0, + info->total_ipv4s - 1); + if (new_ip_index == info->total_ipv4s - 1) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL( &rl, "Subnet address space has been exhausted."); + return 0; + } + + return info->start_ipv4 + new_ip_index; +} + +/* MAC address management (macam) table of "struct eth_addr"s, that holds the + * MAC addresses allocated by the OVN ipam module. */ +static struct hmap macam = HMAP_INITIALIZER(&macam); + +struct macam_node { + struct hmap_node hmap_node; + struct eth_addr mac_addr; /* Allocated MAC address. */ +}; + +#define MAC_ADDR_SPACE 0xffffff +static struct eth_addr mac_prefix; +static char mac_prefix_str[18]; + +void +cleanup_macam(void) +{ + struct macam_node *node; + HMAP_FOR_EACH_POP (node, hmap_node, &macam) { + free(node); + } +} + +const char * +set_mac_prefix(const char *prefix) +{ + mac_prefix = eth_addr_zero; + if (prefix) { + struct eth_addr addr; + + memset(&addr, 0, sizeof addr); + if (ovs_scan(prefix, "%"SCNx8":%"SCNx8":%"SCNx8, + &addr.ea[0], &addr.ea[1], &addr.ea[2])) { + mac_prefix = addr; + } + } + + if (eth_addr_equals(mac_prefix, eth_addr_zero)) { + eth_addr_random(&mac_prefix); + memset(&mac_prefix.ea[3], 0, 3); + } + + snprintf(mac_prefix_str, sizeof(mac_prefix_str), + "%02"PRIx8":%02"PRIx8":%02"PRIx8, + mac_prefix.ea[0], mac_prefix.ea[1], mac_prefix.ea[2]); + + return mac_prefix_str; +} + +struct eth_addr +get_mac_prefix(void) +{ + return mac_prefix; +} + +static bool +ipam_is_duplicate_mac(struct eth_addr *ea, uint64_t mac64, bool warn) +{ + struct macam_node *macam_node; + HMAP_FOR_EACH_WITH_HASH (macam_node, hmap_node, hash_uint64(mac64), + &macam) { + if (eth_addr_equals(*ea, macam_node->mac_addr)) { + if (warn) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Duplicate MAC set: "ETH_ADDR_FMT, + ETH_ADDR_ARGS(macam_node->mac_addr)); + } + return true; + } + } + return false; +} + +void +ipam_insert_mac(struct eth_addr *ea, bool check) +{ + if (!ea) { + return; + } + + uint64_t mac64 = eth_addr_to_uint64(*ea); + uint64_t prefix = eth_addr_to_uint64(mac_prefix); + + /* If the new MAC was not assigned by this address management system or + * check is true and the new MAC is a duplicate, do not insert it into the + * macam hmap. */ + if (((mac64 ^ prefix) >> 24) + || (check && ipam_is_duplicate_mac(ea, mac64, true))) { + return; + } + + struct macam_node *new_macam_node = xmalloc(sizeof *new_macam_node); + new_macam_node->mac_addr = *ea; + hmap_insert(&macam, &new_macam_node->hmap_node, hash_uint64(mac64)); +} + +uint64_t +ipam_get_unused_mac(ovs_be32 ip) +{ + uint32_t mac_addr_suffix, i, base_addr = ntohl(ip) & MAC_ADDR_SPACE; + struct eth_addr mac; + uint64_t mac64; + + for (i = 0; i < MAC_ADDR_SPACE - 1; i++) { + /* The tentative MAC's suffix will be in the interval (1, 0xfffffe). */ + mac_addr_suffix = ((base_addr + i) % (MAC_ADDR_SPACE - 1)) + 1; + mac64 = eth_addr_to_uint64(mac_prefix) | mac_addr_suffix; + eth_addr_from_uint64(mac64, &mac); + if (!ipam_is_duplicate_mac(&mac, mac64, false)) { + break; + } + } + + if (i == MAC_ADDR_SPACE) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "MAC address space exhausted."); + mac64 = 0; + } + + return mac64; +} diff --git a/northd/ipam.h b/northd/ipam.h new file mode 100644 index 000000000..5244cce8f --- /dev/null +++ b/northd/ipam.h @@ -0,0 +1,37 @@ +#ifndef NORTHD_IPAM_H +#define NORTHD_IPAM_H 1 + +#include +#include + +#include "openvswitch/types.h" + +struct ipam_info { + uint32_t start_ipv4; + size_t total_ipv4s; + unsigned long *allocated_ipv4s; /* A bitmap of allocated IPv4s */ + bool ipv6_prefix_set; + struct in6_addr ipv6_prefix; + bool mac_only; +}; + +struct smap; +void init_ipam_info(struct ipam_info *info, const struct smap *config); + +void destroy_ipam_info(struct ipam_info *info); + +bool ipam_insert_ip(struct ipam_info *info, uint32_t ip); + +uint32_t ipam_get_unused_ip(struct ipam_info *info); + +void ipam_insert_mac(struct eth_addr *ea, bool check); + +uint64_t ipam_get_unused_mac(ovs_be32 ip); + +void cleanup_macam(void); + +struct eth_addr get_mac_prefix(void); + +const char *set_mac_prefix(const char *hint); + +#endif /* NORTHD_IPAM_H */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 8800a3d3c..0fb7d0969 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -22,6 +22,7 @@ #include "command-line.h" #include "daemon.h" #include "dirs.h" +#include "ipam.h" #include "openvswitch/dynamic-string.h" #include "fatal-signal.h" #include "hash.h" @@ -79,13 +80,6 @@ static const char *ovnnb_db; static const char *ovnsb_db; static const char *unixctl_path; -#define MAC_ADDR_SPACE 0xffffff - -/* MAC address management (macam) table of "struct eth_addr"s, that holds the - * MAC addresses allocated by the OVN ipam module. */ -static struct hmap macam = HMAP_INITIALIZER(&macam); -static struct eth_addr mac_prefix; - static bool controller_event_en; static bool check_lsp_is_up; @@ -478,15 +472,6 @@ port_has_qos_params(const struct smap *opts) } -struct ipam_info { - uint32_t start_ipv4; - size_t total_ipv4s; - unsigned long *allocated_ipv4s; /* A bitmap of allocated IPv4s */ - bool ipv6_prefix_set; - struct in6_addr ipv6_prefix; - bool mac_only; -}; - /* * Multicast snooping and querier per datapath configuration. */ @@ -792,20 +777,6 @@ struct lrouter_group { struct sset ha_chassis_groups; }; -struct macam_node { - struct hmap_node hmap_node; - struct eth_addr mac_addr; /* Allocated MAC address. */ -}; - -static void -cleanup_macam(struct hmap *macam_) -{ - struct macam_node *node; - HMAP_FOR_EACH_POP (node, hmap_node, macam_) { - free(node); - } -} - static struct ovn_datapath * ovn_datapath_create(struct hmap *datapaths, const struct uuid *key, const struct nbrec_logical_switch *nbs, @@ -837,7 +808,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) * use it. */ hmap_remove(datapaths, &od->key_node); ovn_destroy_tnlids(&od->port_tnlids); - bitmap_free(od->ipam_info.allocated_ipv4s); + destroy_ipam_info(&od->ipam_info); free(od->router_ports); destroy_nat_entries(od); free(od->nat_entries); @@ -901,117 +872,7 @@ init_ipam_info_for_datapath(struct ovn_datapath *od) return; } - const char *subnet_str = smap_get(&od->nbs->other_config, "subnet"); - const char *ipv6_prefix = smap_get(&od->nbs->other_config, "ipv6_prefix"); - - if (ipv6_prefix) { - if (strstr(ipv6_prefix, "/")) { - /* If a prefix length was specified, it must be 64. */ - struct in6_addr mask; - char *error - = ipv6_parse_masked(ipv6_prefix, - &od->ipam_info.ipv6_prefix, &mask); - if (error) { - static struct vlog_rate_limit rl - = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s: %s", - ipv6_prefix, error); - free(error); - } else { - if (ipv6_count_cidr_bits(&mask) == 64) { - od->ipam_info.ipv6_prefix_set = true; - } else { - static struct vlog_rate_limit rl - = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s: must be /64", - ipv6_prefix); - } - } - } else { - od->ipam_info.ipv6_prefix_set = ipv6_parse( - ipv6_prefix, &od->ipam_info.ipv6_prefix); - if (!od->ipam_info.ipv6_prefix_set) { - static struct vlog_rate_limit rl - = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad 'ipv6_prefix' %s", ipv6_prefix); - } - } - } - - if (!subnet_str) { - if (!ipv6_prefix) { - od->ipam_info.mac_only = smap_get_bool(&od->nbs->other_config, - "mac_only", false); - } - return; - } - - ovs_be32 subnet, mask; - char *error = ip_parse_masked(subnet_str, &subnet, &mask); - if (error || mask == OVS_BE32_MAX || !ip_is_cidr(mask)) { - static struct vlog_rate_limit rl - = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad 'subnet' %s", subnet_str); - free(error); - return; - } - - od->ipam_info.start_ipv4 = ntohl(subnet & mask) + 1; - od->ipam_info.total_ipv4s = ~ntohl(mask); - od->ipam_info.allocated_ipv4s = - bitmap_allocate(od->ipam_info.total_ipv4s); - - /* Mark first IP as taken */ - bitmap_set1(od->ipam_info.allocated_ipv4s, 0); - - /* Check if there are any reserver IPs (list) to be excluded from IPAM */ - const char *exclude_ip_list = smap_get(&od->nbs->other_config, - "exclude_ips"); - if (!exclude_ip_list) { - return; - } - - struct lexer lexer; - lexer_init(&lexer, exclude_ip_list); - /* exclude_ip_list could be in the format - - * "10.0.0.4 10.0.0.10 10.0.0.20..10.0.0.50 10.0.0.100..10.0.0.110". - */ - lexer_get(&lexer); - while (lexer.token.type != LEX_T_END) { - if (lexer.token.type != LEX_T_INTEGER) { - lexer_syntax_error(&lexer, "expecting address"); - break; - } - uint32_t start = ntohl(lexer.token.value.ipv4); - lexer_get(&lexer); - - uint32_t end = start + 1; - if (lexer_match(&lexer, LEX_T_ELLIPSIS)) { - if (lexer.token.type != LEX_T_INTEGER) { - lexer_syntax_error(&lexer, "expecting address range"); - break; - } - end = ntohl(lexer.token.value.ipv4) + 1; - lexer_get(&lexer); - } - - /* Clamp start...end to fit the subnet. */ - start = MAX(od->ipam_info.start_ipv4, start); - end = MIN(od->ipam_info.start_ipv4 + od->ipam_info.total_ipv4s, end); - if (end > start) { - bitmap_set_multiple(od->ipam_info.allocated_ipv4s, - start - od->ipam_info.start_ipv4, - end - start, 1); - } else { - lexer_error(&lexer, "excluded addresses not in subnet"); - } - } - if (lexer.error) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "logical switch "UUID_FMT": bad exclude_ips (%s)", - UUID_ARGS(&od->key), lexer.error); - } - lexer_destroy(&lexer); + init_ipam_info(&od->ipam_info, &od->nbs->other_config); } static void @@ -1564,64 +1425,17 @@ lrport_is_enabled(const struct nbrec_logical_router_port *lrport) return !lrport->enabled || *lrport->enabled; } -static bool -ipam_is_duplicate_mac(struct eth_addr *ea, uint64_t mac64, bool warn) -{ - struct macam_node *macam_node; - HMAP_FOR_EACH_WITH_HASH (macam_node, hmap_node, hash_uint64(mac64), - &macam) { - if (eth_addr_equals(*ea, macam_node->mac_addr)) { - if (warn) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Duplicate MAC set: "ETH_ADDR_FMT, - ETH_ADDR_ARGS(macam_node->mac_addr)); - } - return true; - } - } - return false; -} - static void -ipam_insert_mac(struct eth_addr *ea, bool check) +ipam_insert_ip_for_datapath(struct ovn_datapath *od, uint32_t ip) { - if (!ea) { - return; - } - - uint64_t mac64 = eth_addr_to_uint64(*ea); - uint64_t prefix = eth_addr_to_uint64(mac_prefix); - - /* If the new MAC was not assigned by this address management system or - * check is true and the new MAC is a duplicate, do not insert it into the - * macam hmap. */ - if (((mac64 ^ prefix) >> 24) - || (check && ipam_is_duplicate_mac(ea, mac64, true))) { + if (!od) { return; } - struct macam_node *new_macam_node = xmalloc(sizeof *new_macam_node); - new_macam_node->mac_addr = *ea; - hmap_insert(&macam, &new_macam_node->hmap_node, hash_uint64(mac64)); -} - -static void -ipam_insert_ip(struct ovn_datapath *od, uint32_t ip) -{ - if (!od || !od->ipam_info.allocated_ipv4s) { - return; - } - - if (ip >= od->ipam_info.start_ipv4 && - ip < (od->ipam_info.start_ipv4 + od->ipam_info.total_ipv4s)) { - if (bitmap_is_set(od->ipam_info.allocated_ipv4s, - ip - od->ipam_info.start_ipv4)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Duplicate IP set on switch %s: "IP_FMT, - od->nbs->name, IP_ARGS(htonl(ip))); - } - bitmap_set1(od->ipam_info.allocated_ipv4s, - ip - od->ipam_info.start_ipv4); + if (!ipam_insert_ip(&od->ipam_info, ip)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Duplicate IP set on switch %s: "IP_FMT, + od->nbs->name, IP_ARGS(htonl(ip))); } } @@ -1651,7 +1465,7 @@ ipam_insert_lsp_addresses(struct ovn_datapath *od, struct ovn_port *op, for (size_t j = 0; j < laddrs.n_ipv4_addrs; j++) { uint32_t ip = ntohl(laddrs.ipv4_addrs[j].addr); - ipam_insert_ip(od, ip); + ipam_insert_ip_for_datapath(od, ip); } destroy_lport_addresses(&laddrs); @@ -1693,7 +1507,7 @@ ipam_add_port_addresses(struct ovn_datapath *od, struct ovn_port *op) * about a duplicate IP address. */ if (ip != op->peer->od->ipam_info.start_ipv4) { - ipam_insert_ip(op->peer->od, ip); + ipam_insert_ip_for_datapath(op->peer->od, ip); } } @@ -1701,50 +1515,6 @@ ipam_add_port_addresses(struct ovn_datapath *od, struct ovn_port *op) } } -static uint64_t -ipam_get_unused_mac(ovs_be32 ip) -{ - uint32_t mac_addr_suffix, i, base_addr = ntohl(ip) & MAC_ADDR_SPACE; - struct eth_addr mac; - uint64_t mac64; - - for (i = 0; i < MAC_ADDR_SPACE - 1; i++) { - /* The tentative MAC's suffix will be in the interval (1, 0xfffffe). */ - mac_addr_suffix = ((base_addr + i) % (MAC_ADDR_SPACE - 1)) + 1; - mac64 = eth_addr_to_uint64(mac_prefix) | mac_addr_suffix; - eth_addr_from_uint64(mac64, &mac); - if (!ipam_is_duplicate_mac(&mac, mac64, false)) { - break; - } - } - - if (i == MAC_ADDR_SPACE) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "MAC address space exhausted."); - mac64 = 0; - } - - return mac64; -} - -static uint32_t -ipam_get_unused_ip(struct ovn_datapath *od) -{ - if (!od || !od->ipam_info.allocated_ipv4s) { - return 0; - } - - size_t new_ip_index = bitmap_scan(od->ipam_info.allocated_ipv4s, 0, 0, - od->ipam_info.total_ipv4s - 1); - if (new_ip_index == od->ipam_info.total_ipv4s - 1) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL( &rl, "Subnet address space has been exhausted."); - return 0; - } - - return od->ipam_info.start_ipv4 + new_ip_index; -} - enum dynamic_update_type { NONE, /* No change to the address */ REMOVE, /* Address is no longer dynamic */ @@ -1784,7 +1554,7 @@ dynamic_mac_changed(const char *lsp_addresses, } uint64_t mac64 = eth_addr_to_uint64(update->current_addresses.ea); - uint64_t prefix = eth_addr_to_uint64(mac_prefix); + uint64_t prefix = eth_addr_to_uint64(get_mac_prefix()); if ((mac64 ^ prefix) >> 24) { return DYNAMIC; @@ -1946,7 +1716,7 @@ update_unchanged_dynamic_addresses(struct dynamic_address_update *update) ipam_insert_mac(&update->current_addresses.ea, false); } if (update->ipv4 == NONE && update->current_addresses.n_ipv4_addrs) { - ipam_insert_ip(update->op->od, + ipam_insert_ip_for_datapath(update->op->od, ntohl(update->current_addresses.ipv4_addrs[0].addr)); } } @@ -2026,7 +1796,7 @@ update_dynamic_addresses(struct dynamic_address_update *update) ip4 = update->static_ip; break; case DYNAMIC: - ip4 = htonl(ipam_get_unused_ip(update->od)); + ip4 = htonl(ipam_get_unused_ip(&update->od->ipam_info)); VLOG_INFO("Assigned dynamic IPv4 address '"IP_FMT"' to port '%s'", IP_ARGS(ip4), update->op->nbsp->name); } @@ -2075,7 +1845,7 @@ update_dynamic_addresses(struct dynamic_address_update *update) ipam_insert_mac(&mac, true); if (ip4) { - ipam_insert_ip(update->od, ntohl(ip4)); + ipam_insert_ip_for_datapath(update->od, ntohl(ip4)); ds_put_format(&new_addr, " "IP_FMT, IP_ARGS(ip4)); } if (!IN6_ARE_ADDR_EQUAL(&ip6, &in6addr_any)) { @@ -12197,16 +11967,8 @@ ovnnb_db_run(struct northd_context *ctx, sbrec_sb_global_set_options(sb, &nb->options); sb_loop->next_cfg = nb->nb_cfg; - const char *mac_addr_prefix = smap_get(&nb->options, "mac_prefix"); - if (mac_addr_prefix) { - struct eth_addr addr; - - memset(&addr, 0, sizeof addr); - if (ovs_scan(mac_addr_prefix, "%"SCNx8":%"SCNx8":%"SCNx8, - &addr.ea[0], &addr.ea[1], &addr.ea[2])) { - mac_prefix = addr; - } - } + const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, + "mac_prefix")); const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); if (monitor_mac) { @@ -12221,15 +11983,7 @@ ovnnb_db_run(struct northd_context *ctx, struct smap options; smap_clone(&options, &nb->options); - if (!mac_addr_prefix) { - eth_addr_random(&mac_prefix); - memset(&mac_prefix.ea[3], 0, 3); - - smap_add_format(&options, "mac_prefix", - "%02"PRIx8":%02"PRIx8":%02"PRIx8, - mac_prefix.ea[0], mac_prefix.ea[1], - mac_prefix.ea[2]); - } + smap_add(&options, "mac_prefix", mac_addr_prefix); if (!monitor_mac) { eth_addr_random(&svc_monitor_mac_ea); @@ -12296,7 +12050,8 @@ ovnnb_db_run(struct northd_context *ctx, } shash_destroy(&meter_groups); - cleanup_macam(&macam); + /* XXX This is awkward */ + cleanup_macam(); } /* Stores the list of chassis which references an ha_chassis_group. From patchwork Mon Nov 2 14:28:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Michelson X-Patchwork-Id: 1392285 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=whitealder.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=K1MiCmuP; dkim-atps=neutral Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4CPwJY2gcjz9sVT for ; Tue, 3 Nov 2020 01:28:44 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 5CAA7867DE; Mon, 2 Nov 2020 14:28:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R8hHHB8gVrWm; Mon, 2 Nov 2020 14:28:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id 78D5086655; Mon, 2 Nov 2020 14:28:39 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5E381C0889; Mon, 2 Nov 2020 14:28:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 482F0C0051 for ; Mon, 2 Nov 2020 14:28:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 2FA6085BD3 for ; Mon, 2 Nov 2020 14:28:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Zx0xSmWSYfsd for ; Mon, 2 Nov 2020 14:28:36 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by whitealder.osuosl.org (Postfix) with ESMTPS id 5EE84864D8 for ; Mon, 2 Nov 2020 14:28:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1604327314; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AEcrKvHHFBSXcnnuoA5qNMTsCjkEIp88tQjKaEi0pqo=; b=K1MiCmuPodWqSZIez8Y4WPPGyJTxhkPoZd+oToNSwiGTpZuWH3Vdi/HTW6RncXMADPu5zA QLKf2MUxqFQbogUkZPrgIaW634ymMov3wg0aDkLOVD11t+luuUcoe2f9IQ8Qo+K1ZR7KXB LFVO9+HF12xabxCnBwytkYkk+ME/t7c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-302-y5bO8shNMa-k98Y7YB0DcA-1; Mon, 02 Nov 2020 09:28:32 -0500 X-MC-Unique: y5bO8shNMa-k98Y7YB0DcA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 34FF51DE12 for ; Mon, 2 Nov 2020 14:28:31 +0000 (UTC) Received: from monae.redhat.com (ovpn-112-233.rdu2.redhat.com [10.10.112.233]) by smtp.corp.redhat.com (Postfix) with ESMTP id D1E9A55766 for ; Mon, 2 Nov 2020 14:28:30 +0000 (UTC) From: Mark Michelson To: dev@openvswitch.org Date: Mon, 2 Nov 2020 09:28:28 -0500 Message-Id: <20201102142828.3444732-4-mmichels@redhat.com> In-Reply-To: <20201102142828.3444732-1-mmichels@redhat.com> References: <20201102142828.3444732-1-mmichels@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mmichels@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [RFC PATCH ovn v3 3/3] Add ipam unit tests X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" This adds unit tests for IPAM IPv6 initialization, IPv4 initialization, and IPv4 address retrieval. It also adds testsuite tests corresponding to these unit tests. The IPv6 initialization and IPv4 initialization tests make use of the new unit test framework. They use ovn-appctl to get access to internal functions in northd/ipam.c. They require ENABLE_UNIT_TESTS to be defined, otherwise the internal unit test code will not be compiled in. The IPv4 address retrieval test makes use of the pre-existing ovstest utility. Signed-off-by: Mark Michelson --- northd/ipam.c | 56 ++++++++++ northd/ovn-northd.c | 3 + tests/automake.mk | 8 +- tests/ovn-unit-tests.at | 237 ++++++++++++++++++++++++++++++++++++++++ tests/testsuite.at | 1 + 5 files changed, 303 insertions(+), 2 deletions(-) create mode 100644 tests/ovn-unit-tests.at diff --git a/northd/ipam.c b/northd/ipam.c index e5383c46f..fb367700b 100644 --- a/northd/ipam.c +++ b/northd/ipam.c @@ -326,3 +326,59 @@ ipam_get_unused_mac(ovs_be32 ip) return mac64; } + +#ifdef ENABLE_UNIT_TESTS + +static void +test_init_ipam_ipv6_prefix(struct ovs_cmdl_context *ctx) +{ + const char *prefix = ctx->argc > 1 ? ctx->argv[1] : NULL; + struct ipam_info ipam; + struct ds *output = ctx->pvt; + + init_ipam_ipv6_prefix(prefix, &ipam); + ds_put_format(output, "ipv6_prefix_set: %s\n", + ipam.ipv6_prefix_set ? "true" : "false"); + if (ipam.ipv6_prefix_set) { + char ipv6[INET6_ADDRSTRLEN]; + inet_ntop(AF_INET6, &ipam.ipv6_prefix, + ipv6, sizeof ipv6); + ds_put_format(output, "ipv6_prefix: %s\n", ipv6); + } +} + +UNIT_TEST_DEFINE(init_ipam_ipv6_prefix, 0, 1, test_init_ipam_ipv6_prefix); + +static void +test_init_ipam_ipv4(struct ovs_cmdl_context *ctx) +{ + const char *subnet = ctx->argv[1]; + const char *exclude_ips = ctx->argc > 2 ? ctx->argv[2] : NULL; + struct ipam_info info; + + init_ipam_ipv4(subnet, exclude_ips, &info); + struct ds *output = ctx->pvt; + + ds_put_format(output, "start_ipv4: " IP_FMT "\n", + IP_ARGS(htonl(info.start_ipv4))); + ds_put_format(output, "total_ipv4s: %" PRIuSIZE "\n", info.total_ipv4s); + + ds_put_cstr(output, "allocated_ipv4s: "); + if (info.allocated_ipv4s) { + int start = 0; + int end = info.total_ipv4s; + for (size_t bit = bitmap_scan(info.allocated_ipv4s, true, start, end); + bit != end; + bit = bitmap_scan(info.allocated_ipv4s, true, bit + 1, end)) { + ds_put_format(output, IP_FMT " ", + IP_ARGS((htonl(info.start_ipv4 + bit)))); + } + } + ds_chomp(output, ' '); + ds_put_char(output, '\n'); + destroy_ipam_info(&info); +} + +UNIT_TEST_DEFINE(init_ipam_ipv4, 1, 2, test_init_ipam_ipv4); + +#endif /* ENABLE_UNIT_TESTS */ diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index 0fb7d0969..41c5df49e 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -36,6 +36,7 @@ #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "lib/unit-test.h" #include "ovn/actions.h" #include "ovn/logical-fields.h" #include "packets.h" @@ -12742,6 +12743,8 @@ main(int argc, char *argv[]) cluster_state_reset_cmd, &reset_ovnnb_idl_min_index); + register_unixctl_unit_test(); + daemonize_complete(); /* We want to detect (almost) all changes to the ovn-nb db. */ diff --git a/tests/automake.mk b/tests/automake.mk index 26b6d11b4..782af79b9 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -30,7 +30,8 @@ TESTSUITE_AT = \ tests/ovn-controller-vtep.at \ tests/ovn-ic.at \ tests/ovn-macros.at \ - tests/ovn-performance.at + tests/ovn-performance.at \ + tests/ovn-unit-tests.at SYSTEM_KMOD_TESTSUITE_AT = \ tests/system-common-macros.at \ @@ -200,7 +201,10 @@ noinst_PROGRAMS += tests/ovstest tests_ovstest_SOURCES = \ tests/ovstest.c \ tests/ovstest.h \ - tests/test-ovn.c + tests/test-ovn.c \ + northd/test-ipam.c \ + northd/ipam.c \ + northd/ipmam.h tests_ovstest_LDADD = $(OVS_LIBDIR)/daemon.lo \ $(OVS_LIBDIR)/libopenvswitch.la lib/libovn.la diff --git a/tests/ovn-unit-tests.at b/tests/ovn-unit-tests.at new file mode 100644 index 000000000..50fc53745 --- /dev/null +++ b/tests/ovn-unit-tests.at @@ -0,0 +1,237 @@ +AT_BANNER([OVN unit tests]) + +AT_SETUP([ovn -- unit test -- init_ipam_ipv4]) +AT_SKIP_IF([test "$ENABLE_UNIT_TESTS" = no]) +ovn_start + +# Valid subnet, no exclude IPs +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 +]) + +# Valid subnet, single exclude IP +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 192.168.0.3], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 192.168.0.3 +]) + +# Valid subnet, two exclude IPs +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 "192.168.0.3 192.168.0.5"], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 192.168.0.3 192.168.0.5 +]) + +# Valid subnet, range of exclude IPs +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 "192.168.0.3..192.168.0.5"], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 192.168.0.3 192.168.0.4 192.168.0.5 +]) + +# Valid subnet, exclude IP outside of subnet +# Excluded IP should be ignored. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 192.168.0.9], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 +]) + +# Valid subnet, range of exclude IPs starts in subnet but ends outside +# Excluded IPs inside the subnet should be allocated +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 "192.168.0.5..192.168.0.11"], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 192.168.0.5 192.168.0.6 192.168.0.7 +]) + +# Valid subnet, range of exclude IPs starts outside subnet but ends inside +# Excluded IPs inside the subnet should be allocated +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.8/29 "192.168.0.5..192.168.0.11"], [0], [dnl +start_ipv4: 192.168.0.9 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.9 192.168.0.10 192.168.0.11 +]) + +# Valid subnet, range of exclude IPs starts before and ends after the subnet +# Entire subnet should be allocated +# XXX Should excluding every address in a subnet be an invalid configuration? +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.8/29 "192.168.0.5..192.168.0.18"], [0], [dnl +start_ipv4: 192.168.0.9 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.9 192.168.0.10 192.168.0.11 192.168.0.12 192.168.0.13 192.168.0.14 192.168.0.15 +]) + +# Valid subnet, inverted exclude range +# Exclude range should be ignored +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/29 "192.168.0.5..192.168.0.2"], [0], [dnl +start_ipv4: 192.168.0.1 +total_ipv4s: 7 +allocated_ipv4s: 192.168.0.1 +]) + +# XXX At this point, I wanted to insert some tests where I put in invalid +# IP addresses like 400.500.600.700 to ensure that the start_ipv4 was set +# to "0.0.0.0". However, ovs_scan_ip_masked() does no validation of the +# IP address. So long as the given IP address follows the format of +# xxx.xxx.xxx.xxx, it's seen as valid. In the specific case of +# "400.500.600.700", it ends up setting the start_ipv4 to +# "144.244.88.185". This result is probably system-dependent. + +# Invalid subnet: Bad mask +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv4 192.168.0.0/-69], [0], [dnl +start_ipv4: 0.0.0.0 +total_ipv4s: 0 +allocated_ipv4s: +]) + +AT_CLEANUP + +AT_SETUP([ovn -- unit test -- init_ipam_ipv6_prefix]) +AT_SKIP_IF([test "$ENABLE_UNIT_TESTS" = no]) +ovn_start + +# No prefix set +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix], [0], [dnl +ipv6_prefix_set: false +]) + +# Good prefix with no mask +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::], [0], [dnl +ipv6_prefix_set: true +ipv6_prefix: aef0:: +]) + +# Good prefix with good mask +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::/64], [0], [dnl +ipv6_prefix_set: true +ipv6_prefix: aef0:: +]) + +# Bad prefix with no mask +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef20::], [0], [dnl +ipv6_prefix_set: false +]) + +# Good prefix with nonsense mask. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::/900], [0], [dnl +ipv6_prefix_set: false +]) + +# Good prefix with a non-/64 mask. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::/32], [0], [dnl +ipv6_prefix_set: false +]) + +# Bad prefix and a non-/64 mask. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef20::/32], [0], [dnl +ipv6_prefix_set: false +]) + +# Overspecify the IPv6 address. +# We should "round down" to the /64 network address. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::2323], [0], [dnl +ipv6_prefix_set: true +ipv6_prefix: aef0:: +]) + +# Overspecify the IPv6 address, and specify a mask. +# We should "round down" to the /64 network address. +AT_CHECK([ovn-appctl -t northd/ovn-northd unit-test init_ipam_ipv6_prefix aef0::2323/64], [0], [dnl +ipv6_prefix_set: true +ipv6_prefix: aef0:: +]) + +AT_CLEANUP + +AT_SETUP([ovn -- unit test -- ipam_get_unused_ip]) +AT_SKIP_IF([test "$ENABLE_UNIT_TESTS" = no]) +ovn_start + +# Ensure first address returned by IPAM is .2, since .1 is reserved for the +# connected router +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 1], [0], [dnl +192.168.0.2 +]) + +# Ensure that we only grab IPs within the requested subnet +# Ignore stderr so that the warning about address space being +# exhausted does not cause the test to fail +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 6], [0], [dnl +192.168.0.2 +192.168.0.3 +192.168.0.4 +192.168.0.5 +192.168.0.6 +0.0.0.0 +],[ignore]) + +# Set up an exclude IP and ensure it does not get selected +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 4 192.168.0.3], [0], [dnl +192.168.0.2 +192.168.0.4 +192.168.0.5 +192.168.0.6 +]) + +# Set up an exclude IP range and ensure none gets selected +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 2 192.168.0.3..192.168.0.5], [0], [dnl +192.168.0.2 +192.168.0.6 +]) + +# Set up an exclude range from outside the subnet. Ensure it is ignored. +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 5 192.168.1.3..192.168.1.5], [0], [dnl +192.168.0.2 +192.168.0.3 +192.168.0.4 +192.168.0.5 +192.168.0.6 +],[ignore]) + +# Set up an exclude range from outside the subnet. Ensure we cannot assign +# addresses outside the subnet +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 6 192.168.1.3..192.168.1.5], [0], [dnl +192.168.0.2 +192.168.0.3 +192.168.0.4 +192.168.0.5 +192.168.0.6 +0.0.0.0 +],[ignore]) + +# Set up an exclude range that starts before the subnet but ends in the subnet. +# The overlapping part should be excluded +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.8/29 2 192.168.0.2..192.168.0.12], [0], [dnl +192.168.0.13 +192.168.0.14 +],[ignore]) + +# Set up an exclude range that starts in the subnet but ends after the subnet. +# The overlapping part should be excluded. +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.0/29 3 192.168.0.4..192.168.0.9], [0], [dnl +192.168.0.2 +192.168.0.3 +0.0.0.0 +],[ignore]) + +# Set up an exclude range that starts before the subnet and ends after the subnet. +# The entire range should be excluded. +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.8/29 1 192.168.0.2..192.168.0.18], [0], [dnl +0.0.0.0 +],[ignore]) + +# Configure the subnet using a starting IP that is not the network address of the +# subnet. Ensure that we "round it down" to the proper subnet starting point. +AT_CHECK([ovstest test-ipam ipam_get_unused_ip 192.168.0.4/29 5], [0], [dnl +192.168.0.2 +192.168.0.3 +192.168.0.4 +192.168.0.5 +192.168.0.6 +]) + +AT_CLEANUP diff --git a/tests/testsuite.at b/tests/testsuite.at index 1985923d5..5e60bad73 100644 --- a/tests/testsuite.at +++ b/tests/testsuite.at @@ -21,6 +21,7 @@ m4_include([tests/ovsdb-macros.at]) m4_include([tests/ofproto-macros.at]) m4_include([tests/ovn-macros.at]) +m4_include([tests/ovn-unit-tests.at]) m4_include([tests/ovn.at]) m4_include([tests/ovn-performance.at]) m4_include([tests/ovn-northd.at])