From patchwork Wed Aug 2 06:24:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815778 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG25b5mgkz1yYC for ; Wed, 2 Aug 2023 16:25:07 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E046A40376; Wed, 2 Aug 2023 06:25:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E046A40376 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LkRGTcBMa2vj; Wed, 2 Aug 2023 06:25:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id CF6C84012F; Wed, 2 Aug 2023 06:25:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CF6C84012F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B52CC0071; Wed, 2 Aug 2023 06:25:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id DD8E3C0032 for ; Wed, 2 Aug 2023 06:25:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id AB4CD809E9 for ; Wed, 2 Aug 2023 06:25:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org AB4CD809E9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7_3k1BFf5UBX for ; Wed, 2 Aug 2023 06:25:01 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.osuosl.org (Postfix) with ESMTPS id 737C080BD5 for ; Wed, 2 Aug 2023 06:25:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 737C080BD5 Received: by mail.gandi.net (Postfix) with ESMTPSA id DDBD340003; Wed, 2 Aug 2023 06:24:58 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:54:41 +0530 Message-Id: <20230802062441.3638651-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 1/8] northd I-P: Sync SB load balancers in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Similar to the commit [1], a new sub-engine node "sync_to_sb_lb" is added with-in the "sync_to_sb" to sync the SB load balancers. Its main input nodes are "northd" (to access the "lbs" hmap built by this node) and "sb_load_balancer" to access the SB load balancer. "sync_to_sb_lb" doesn't add any handlers and falls back to full recompute all the time. [1] - ccafcc2dc321("northd I-P: Add a new engine node 'en_sync_to_sb' to sync SB tables.") Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 -- northd/en-sync-sb.c | 43 ++++++++++++++++++++++++++++++++++++++++ northd/en-sync-sb.h | 6 ++++++ northd/inc-proc-northd.c | 10 ++++++++-- northd/northd.c | 4 +--- northd/northd.h | 6 +++++- tests/ovn-northd.at | 23 +++++++++++++++++++++ 7 files changed, 86 insertions(+), 8 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 044fa70190..f9f2d04452 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -101,8 +101,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis", node)); input_data->sbrec_fdb_table = EN_OVSDB_GET(engine_get_input("SB_fdb", node)); - input_data->sbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); input_data->sbrec_service_monitor_table = EN_OVSDB_GET(engine_get_input("SB_service_monitor", node)); input_data->sbrec_port_group_table = diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index d7fea981f2..dbe3c63752 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -211,6 +211,49 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, return true; } +/* sync_to_sb_lb engine node functions. + * This engine node syncs the SB load balancers. + */ +void * +en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct sbrec_load_balancer_table *sb_load_balancer_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, + &northd_data->ls_datapaths, &northd_data->lbs); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +{ + +} + +bool +sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *nd = engine_get_input_data("northd", node); + if (nd->change_tracked) { + /* There are only NB LSP related changes and these can be safely + * ignore and returned true. However in case the northd engine + * tracking data includes other changes, we need to do additional + * checks before safely ignoring. */ + return true; + } + return false; +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index bcb9799d24..06d2a57710 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -16,4 +16,10 @@ bool sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *, bool sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *, void *data); + +void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_lb_run(struct engine_node *, void *data); +void en_sync_to_sb_lb_cleanup(void *data); +bool sync_to_sb_lb_northd_handler(struct engine_node *, void *data OVS_UNUSED); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index d328deb222..c0874d5294 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -139,6 +139,7 @@ static ENGINE_NODE(sync_to_sb, "sync_to_sb"); static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); +static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -166,7 +167,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); engine_add_input(&en_northd, &en_sb_service_monitor, NULL); - engine_add_input(&en_northd, &en_sb_load_balancer, NULL); engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); @@ -202,11 +202,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_northd, + sync_to_sb_lb_northd_handler); + engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine only syncs the SB Address_Set table. + * Right now this engine syncs the SB Address_Set table and + * SB Load_Balancer table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index b9605862e4..2390c159c3 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4538,7 +4538,7 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ -static void +void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, struct ovn_datapaths *ls_datapaths, struct hmap *lbs) @@ -17580,8 +17580,6 @@ ovnnb_db_run(struct northd_input *input_data, ovn_update_ipv6_options(&data->lr_ports); ovn_update_ipv6_prefix(&data->lr_ports); - sync_lbs(ovnsb_txn, input_data->sbrec_load_balancer_table, - &data->ls_datapaths, &data->lbs); sync_port_groups(ovnsb_txn, input_data->sbrec_port_group_table, &data->port_groups); sync_meters(ovnsb_txn, input_data->nbrec_meter_table, diff --git a/northd/northd.h b/northd/northd.h index f3e63b1e1a..48c282476a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -16,6 +16,7 @@ #include "ovsdb-idl.h" +#include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" #include "lib/sset.h" @@ -47,7 +48,6 @@ struct northd_input { const struct sbrec_ha_chassis_group_table *sbrec_ha_chassis_group_table; const struct sbrec_chassis_table *sbrec_chassis_table; const struct sbrec_fdb_table *sbrec_fdb_table; - const struct sbrec_load_balancer_table *sbrec_load_balancer_table; const struct sbrec_service_monitor_table *sbrec_service_monitor_table; const struct sbrec_port_group_table *sbrec_port_group_table; const struct sbrec_meter_table *sbrec_meter_table; @@ -354,4 +354,8 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, void run_update_worker_pool(int n_threads); const char *northd_get_svc_monitor_mac(void); + +void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, + struct ovn_datapaths *ls_datapaths, struct hmap *lbs); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 3e06f14c94..0ae0593c73 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9512,6 +9512,29 @@ AT_CHECK([ovn-sbctl lflow-list sw | grep ls_out_pre_lb | grep priority=110 | gre AT_CLEANUP ]) + +# This test checks the compute/recompute stats of sync_to_sb_lb engine node +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([sync_to_sb_lb incremental processing]) +ovn_start + +check ovn-nbctl ls-add sw +check ovn-nbctl lsp-add sw p1 + +check ovn-nbctl --wait=sb sync +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb lsp-set-options p1 foo=bar +sb_lb_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_lb recompute) +AT_CHECK([test x$sb_lb_recomp = x0]) + +check ovn-nbctl --wait=sb lsp-set-type p1 external +sb_lb_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_lb recompute) +AT_CHECK([test x$sb_lb_recomp != x0]) + +AT_CLEANUP +]) + OVN_FOR_EACH_NORTHD_NO_HV([ AT_SETUP([LSP incremental processing]) ovn_start From patchwork Wed Aug 2 06:25:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815779 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG25v2V1Fz1yYC for ; Wed, 2 Aug 2023 16:25:23 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 1F11E61199; Wed, 2 Aug 2023 06:25:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1F11E61199 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rzDdH3IlvC8o; Wed, 2 Aug 2023 06:25:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 936F560F34; Wed, 2 Aug 2023 06:25:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 936F560F34 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 566D1C0071; Wed, 2 Aug 2023 06:25:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7387EC0032 for ; Wed, 2 Aug 2023 06:25:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A0D0181427 for ; Wed, 2 Aug 2023 06:25:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A0D0181427 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N1xCzAB3p1uO for ; Wed, 2 Aug 2023 06:25:11 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::222]) by smtp1.osuosl.org (Postfix) with ESMTPS id 81397813A6 for ; Wed, 2 Aug 2023 06:25:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 81397813A6 Received: by mail.gandi.net (Postfix) with ESMTPSA id 71C7540008; Wed, 2 Aug 2023 06:25:06 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:00 +0530 Message-Id: <20230802062500.3638699-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 2/8] northd: Add a new engine node - lb_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch separates out the 'lbs' and 'lb_groups' from the 'northd' engine node data into a new engine node 'lb_data'. This new node becomes an input to the 'northd' node. This makes handling the NB load balancer and load balancer group changes easier. Prior to this patch, 'struct ovn_northd_lb' and 'struct ovn_lb_group' were maintaing the logical switch and logical router association with a load balancer. This data is now maintained by 'northd' engine node. New structs 'struct ovn_lb_datapaths' and 'struct ovn_lb_group_datapaths' is added for this purpose. Acked-by: Han Zhou Signed-off-by: Numan Siddique --- lib/lb.c | 205 ++++++++++-- lib/lb.h | 86 +++-- northd/automake.mk | 2 + northd/en-lb-data.c | 125 ++++++++ northd/en-lb-data.h | 23 ++ northd/en-lflow.c | 3 +- northd/en-northd.c | 11 +- northd/en-sync-sb.c | 2 +- northd/inc-proc-northd.c | 8 +- northd/northd.c | 669 ++++++++++++++++++++++----------------- northd/northd.h | 15 +- 11 files changed, 781 insertions(+), 368 deletions(-) create mode 100644 northd/en-lb-data.c create mode 100644 northd/en-lb-data.h diff --git a/lib/lb.c b/lib/lb.c index 7afdaed65b..e874680a3f 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -26,6 +26,7 @@ #include "openvswitch/vlog.h" #include "lib/bitmap.h" #include "lib/smap.h" +#include "socket-util.h" VLOG_DEFINE_THIS_MODULE(lb); @@ -431,11 +432,62 @@ void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } +static void +ovn_lb_vip_backends_health_check_init(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + struct ovn_northd_lb_vip *lb_vip_nb) +{ + struct ds key = DS_EMPTY_INITIALIZER; + + for (size_t j = 0; j < lb_vip->n_backends; j++) { + struct ovn_lb_backend *backend = &lb_vip->backends[j]; + ds_clear(&key); + ds_put_format(&key, IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? "%s" : "[%s]", backend->ip_str); + + const char *s = smap_get(&lb->nlb->ip_port_mappings, ds_cstr(&key)); + if (!s) { + continue; + } + + char *svc_mon_src_ip = NULL; + char *port_name = xstrdup(s); + char *p = strstr(port_name, ":"); + if (p) { + *p = 0; + p++; + struct sockaddr_storage svc_mon_src_addr; + if (!inet_parse_address(p, &svc_mon_src_addr)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); + } else { + struct ds src_ip_s = DS_EMPTY_INITIALIZER; + ss_format_address_nobracks(&svc_mon_src_addr, + &src_ip_s); + svc_mon_src_ip = ds_steal_cstr(&src_ip_s); + } + } + + if (svc_mon_src_ip) { + struct ovn_northd_lb_backend *backend_nb = + &lb_vip_nb->backends_nb[j]; + backend_nb->health_check = true; + backend_nb->logical_port = xstrdup(port_name); + backend_nb->svc_mon_src_ip = svc_mon_src_ip; + } + free(port_name); + } + + ds_destroy(&key); +} + static void ovn_northd_lb_vip_destroy(struct ovn_northd_lb_vip *vip) { free(vip->backend_ips); for (size_t i = 0; i < vip->n_backends; i++) { + free(vip->backends_nb[i].logical_port); free(vip->backends_nb[i].svc_mon_src_ip); } free(vip->backends_nb); @@ -555,8 +607,7 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, } struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, - size_t n_ls_datapaths, size_t n_lr_datapaths) +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); @@ -595,9 +646,6 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, } lb->affinity_timeout = affinity_timeout; - lb->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb->nb_lr_map = bitmap_allocate(n_lr_datapaths); - sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -632,6 +680,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, xstrdup(node->value)); } n_vips++; + + if (lb_vip_nb->lb_health_check) { + ovn_lb_vip_backends_health_check_init(lb, lb_vip, lb_vip_nb); + } } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the @@ -684,24 +736,6 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_lr_map, ods[i]->index); - } -} - -void -ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_ls_map, ods[i]->index); - } -} - void ovn_northd_lb_destroy(struct ovn_northd_lb *lb) { @@ -715,20 +749,15 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); - bitmap_free(lb->nb_lr_map); - bitmap_free(lb->nb_ls_map); free(lb); } /* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will - * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' - * logical routers to which this LB Group is applied. Can be filled later - * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ + * and an array of 'struct ovn_northd_lb' objects for its associated + * load balancers. */ struct ovn_lb_group * ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs, size_t max_ls_datapaths, - size_t max_lr_datapaths) + const struct hmap *lbs) { struct ovn_lb_group *lb_group; @@ -736,8 +765,6 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, lb_group->uuid = nbrec_lb_group->header_.uuid; lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); - lb_group->ls = xmalloc(max_ls_datapaths * sizeof *lb_group->ls); - lb_group->lr = xmalloc(max_lr_datapaths * sizeof *lb_group->lr); lb_group->lb_ips = ovn_lb_ip_set_create(); for (size_t i = 0; i < nbrec_lb_group->n_load_balancer; i++) { @@ -758,8 +785,6 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) ovn_lb_ip_set_destroy(lb_group->lb_ips); free(lb_group->lbs); - free(lb_group->ls); - free(lb_group->lr); free(lb_group); } @@ -943,3 +968,113 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) hmap_destroy(tuples); } + +void +build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, + const struct ovn_northd_lb *lb) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, &lb->ips_v4) { + sset_add(&lb_ips->ips_v4, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, &lb->ips_v6) { + sset_add(&lb_ips->ips_v6, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v6_routable, ip_address); + } + } +} + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + } +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/lib/lb.h b/lib/lb.h index 23d8fc9e9b..0339050cba 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -59,7 +59,6 @@ struct ovn_northd_lb { struct hmap_node hmap_node; const struct nbrec_load_balancer *nlb; /* May be NULL. */ - const struct sbrec_load_balancer *slb; /* May be NULL. */ const char *proto; char *selection_fields; struct ovn_lb_vip *vips; @@ -78,14 +77,6 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; - - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; - - struct ovn_dp_group *dpg; }; struct ovn_lb_vip { @@ -129,23 +120,19 @@ struct ovn_northd_lb_vip { }; struct ovn_northd_lb_backend { - struct ovn_port *op; /* Logical port to which the ip belong to. */ bool health_check; + char *logical_port; /* Logical port to which the ip belong to. */ char *svc_mon_src_ip; /* Source IP to use for monitoring. */ - const struct sbrec_service_monitor *sbrec_monitor; }; -struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); +struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *); struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); -void ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); -void ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); + +void build_lrouter_lb_ips(struct ovn_lb_ip_set *, + const struct ovn_northd_lb *); struct ovn_lb_group { struct hmap_node hmap_node; @@ -153,35 +140,70 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; +}; + +struct ovn_lb_group *ovn_lb_group_create( + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); +void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); +struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, + const struct uuid *); + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; - /* Datapaths to which this LB group is applied. */ + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ size_t n_ls; struct ovn_datapath **ls; size_t n_lr; struct ovn_datapath **lr; }; -struct ovn_lb_group *ovn_lb_group_create( - const struct nbrec_load_balancer_group *, - const struct hmap *lbs, - size_t max_ls_datapaths, +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, size_t max_lr_datapaths); -void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); -struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, - const struct uuid *); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); static inline void -ovn_lb_group_add_ls(struct ovn_lb_group *lb_group, size_t n, - struct ovn_datapath **ods) +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) { - memcpy(&lb_group->ls[lb_group->n_ls], ods, n * sizeof *ods); - lb_group->n_ls += n; + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; } static inline void -ovn_lb_group_add_lr(struct ovn_lb_group *lb_group, struct ovn_datapath *lr) +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) { - lb_group->lr[lb_group->n_lr++] = lr; + lbg_dps->lr[lbg_dps->n_lr++] = lr; } struct ovn_controller_lb { diff --git a/northd/automake.mk b/northd/automake.mk index b17f1fdb54..11d44212f8 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -18,6 +18,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-sb.h \ northd/en-sync-from-sb.c \ northd/en-sync-from-sb.h \ + northd/en-lb-data.c \ + northd/en-lb-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c new file mode 100644 index 0000000000..04d8d3e5d7 --- /dev/null +++ b/northd/en-lb-data.c @@ -0,0 +1,125 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lb_data); + +static void lb_data_init(struct ed_type_lb_data *); +static void lb_data_destroy(struct ed_type_lb_data *); +static void build_lbs(const struct nbrec_load_balancer_table *, + const struct nbrec_load_balancer_group_table *, + struct hmap *lbs, struct hmap *lb_groups); + +void * +en_lb_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lb_data *data = xzalloc(sizeof *data); + lb_data_init(data); + return data; +} + +void +en_lb_data_run(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + lb_data_destroy(lb_data); + lb_data_init(lb_data); + + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_lb_data_cleanup(void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + lb_data_destroy(lb_data); +} + +/* static functions. */ +static void +lb_data_init(struct ed_type_lb_data *lb_data) +{ + hmap_init(&lb_data->lbs); + hmap_init(&lb_data->lb_groups); +} + +static void +lb_data_destroy(struct ed_type_lb_data *lb_data) +{ + struct ovn_northd_lb *lb; + HMAP_FOR_EACH_POP (lb, hmap_node, &lb_data->lbs) { + ovn_northd_lb_destroy(lb); + } + hmap_destroy(&lb_data->lbs); + + struct ovn_lb_group *lb_group; + HMAP_FOR_EACH_POP (lb_group, hmap_node, &lb_data->lb_groups) { + ovn_lb_group_destroy(lb_group); + } + hmap_destroy(&lb_data->lb_groups); +} + +static void +build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, + const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group; + struct ovn_northd_lb *lb_nb; + + const struct nbrec_load_balancer *nbrec_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { + lb_nb = ovn_northd_lb_create(nbrec_lb); + hmap_insert(lbs, &lb_nb->hmap_node, + uuid_hash(&nbrec_lb->header_.uuid)); + } + + const struct nbrec_load_balancer_group *nbrec_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, + nbrec_lb_group_table) { + lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + } +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h new file mode 100644 index 0000000000..96fb8c1f8d --- /dev/null +++ b/northd/en-lb-data.h @@ -0,0 +1,23 @@ +#ifndef EN_NORTHD_LB_DATA_H +#define EN_NORTHD_LB_DATA_H 1 + +#include + +#include "openvswitch/hmap.h" + +#include "lib/inc-proc-eng.h" + +/* struct which maintains the data of the engine node lb_data. */ +struct ed_type_lb_data { + /* hmap of load balancers. hmap node is 'struct ovn_northd_lb *' */ + struct hmap lbs; + + /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ + struct hmap lb_groups; +}; + +void *en_lb_data_init(struct engine_node *, struct engine_arg *); +void en_lb_data_run(struct engine_node *, void *data); +void en_lb_data_cleanup(void *data); + +#endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 28ab1c67fb..db1bcbccd6 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -57,7 +57,8 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->port_groups = &northd_data->port_groups; lflow_input->meter_groups = &northd_data->meter_groups; - lflow_input->lbs = &northd_data->lbs; + lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; + lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; lflow_input->features = &northd_data->features; lflow_input->ovn_internal_version_changed = northd_data->ovn_internal_version_changed; diff --git a/northd/en-northd.c b/northd/en-northd.c index f9f2d04452..a0dae65752 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -20,6 +20,7 @@ #include "coverage.h" #include "en-northd.h" +#include "en-lb-data.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "openvswitch/list.h" /* TODO This is needed for ovn-parallel-hmap.h. @@ -70,10 +71,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - input_data->nbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); - input_data->nbrec_load_balancer_group_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data->nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); input_data->nbrec_meter_table = @@ -117,6 +114,11 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); input_data->sbrec_mirror_table = EN_OVSDB_GET(engine_get_input("SB_mirror", node)); + + struct ed_type_lb_data *lb_data = + engine_get_input_data("lb_data", node); + input_data->lbs = &lb_data->lbs; + input_data->lb_groups = &lb_data->lb_groups; } void @@ -130,6 +132,7 @@ en_northd_run(struct engine_node *node, void *data) northd_init(data); northd_get_input_data(node, &input_data); + COVERAGE_INC(northd_run); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); ovnnb_db_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index dbe3c63752..9832fce30a 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -230,7 +230,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lbs); + &northd_data->ls_datapaths, &northd_data->lb_datapaths_map); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index c0874d5294..0a99e87bc2 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -30,6 +30,7 @@ #include "openvswitch/poll-loop.h" #include "openvswitch/vlog.h" #include "inc-proc-northd.h" +#include "en-lb-data.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -140,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -147,8 +149,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, /* Define relationships between nodes where first argument is dependent * on the second argument */ engine_add_input(&en_northd, &en_nb_port_group, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); @@ -178,6 +178,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_load_balancer, NULL); + engine_add_input(&en_lb_data, &en_nb_load_balancer_group, NULL); + engine_add_input(&en_northd, &en_lb_data, NULL); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 2390c159c3..250bc5f89f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3862,16 +3862,15 @@ struct service_monitor_info { static struct service_monitor_info * -create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *monitor_map, - const char *ip, const char *logical_port, - uint16_t service_port, const char *protocol) +get_service_mon(const struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) { uint32_t hash = service_port; hash = hash_string(ip, hash); hash = hash_string(logical_port, hash); - struct service_monitor_info *mon_info; + struct service_monitor_info *mon_info; HMAP_FOR_EACH_WITH_HASH (mon_info, hmap_node, hash, monitor_map) { if (mon_info->sbrec_mon->port == service_port && !strcmp(mon_info->sbrec_mon->ip, ip) && @@ -3881,6 +3880,28 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } } + return NULL; +} + +static struct service_monitor_info * +create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) +{ + struct service_monitor_info *mon_info = + get_service_mon(monitor_map, ip, logical_port, service_port, + protocol); + + if (mon_info) { + return mon_info; + } + + /* get_service_mon() also calculates the hash the same way. */ + uint32_t hash = service_port; + hash = hash_string(ip, hash); + hash = hash_string(logical_port, hash); + struct sbrec_service_monitor *sbrec_mon = sbrec_service_monitor_insert(ovnsb_txn); sbrec_service_monitor_set_ip(sbrec_mon, ip); @@ -3894,7 +3915,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } static void -ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, +ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3911,58 +3933,27 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - struct ovn_port *op = NULL; - char *svc_mon_src_ip = NULL; - - struct ds key = DS_EMPTY_INITIALIZER; - ds_put_format(&key, - IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) - ? "%s" : "[%s]", backend->ip_str); - - const char *s = smap_get(&lb->nlb->ip_port_mappings, - ds_cstr(&key)); - if (s) { - char *port_name = xstrdup(s); - char *p = strstr(port_name, ":"); - if (p) { - *p = 0; - p++; - sset_add(svc_monitor_lsps, port_name); - op = ovn_port_find(ls_ports, port_name); - struct sockaddr_storage svc_mon_src_addr; - if (!inet_parse_address(p, &svc_mon_src_addr)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); - } else { - struct ds src_ip_s = DS_EMPTY_INITIALIZER; - ss_format_address_nobracks(&svc_mon_src_addr, - &src_ip_s); - svc_mon_src_ip = ds_steal_cstr(&src_ip_s); - } - } - free(port_name); + if (!backend_nb->health_check) { + continue; } - ds_destroy(&key); - if (!lb_vip_nb->lb_health_check || !op || !svc_mon_src_ip || - !lsp_is_enabled(op->nbsp)) { - free(svc_mon_src_ip); + sset_add(svc_monitor_lsps, backend_nb->logical_port); + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + + if (!op || !lsp_is_enabled(op->nbsp)) { continue; } - backend_nb->op = op; - backend_nb->svc_mon_src_ip = svc_mon_src_ip; - const char *protocol = lb->nlb->protocol; if (!protocol || !protocol[0]) { protocol = "tcp"; } - backend_nb->health_check = true; + struct service_monitor_info *mon_info = create_or_get_service_mon(ovnsb_txn, monitor_map, backend->ip_str, - backend_nb->op->nbsp->name, + backend_nb->logical_port, backend->port, protocol); ovs_assert(mon_info); @@ -3991,18 +3982,20 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, "offline"); } - backend_nb->sbrec_monitor = mon_info->sbrec_mon; mon_info->required = true; } } } static bool -build_lb_vip_actions(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb_vip *lb_vip_nb, +build_lb_vip_actions(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + const struct ovn_northd_lb_vip *lb_vip_nb, struct ds *action, char *selection_fields, - struct ds *skip_snat_action, struct ds *force_snat_action, - bool ls_dp, const struct chassis_features *features) + struct ds *skip_snat_action, + struct ds *force_snat_action, + bool ls_dp, const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const char *ct_lb_action = features->ct_no_masked_label ? "ct_lb_mark" : "ct_lb"; @@ -4017,10 +4010,27 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, struct ovn_lb_backend *backend = &lb_vip->backends[i]; struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[i]; - if (!backend_nb->health_check || - (backend_nb->health_check && backend_nb->sbrec_monitor && - backend_nb->sbrec_monitor->status && - strcmp(backend_nb->sbrec_monitor->status, "online"))) { + + if (!backend_nb->health_check) { + continue; + } + + const char *protocol = lb->nlb->protocol; + if (!protocol || !protocol[0]) { + protocol = "tcp"; + } + + struct service_monitor_info *mon_info = get_service_mon( + svc_monitor_map, backend->ip_str, backend_nb->logical_port, + backend->port, protocol); + + if (!mon_info) { + continue; + } + + ovs_assert(mon_info->sbrec_mon); + if (mon_info->sbrec_mon->status && + strcmp(mon_info->sbrec_mon->status, "online")) { continue; } @@ -4070,59 +4080,32 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, } static void -build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, - const struct ovn_northd_lb *lb) -{ - const char *ip_address; - - SSET_FOR_EACH (ip_address, &lb->ips_v4) { - sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v4_routable, ip_address); - } - } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { - sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v6_routable, ip_address); - } - } -} - -static void -build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, - const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) +build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb; + struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_group *lb_group; + struct ovn_lb_datapaths *lb_dps; + const struct ovn_northd_lb *lb; - hmap_init(lbs); - hmap_init(lb_groups); + hmap_init(lb_datapaths_map); + hmap_init(lb_group_datapaths_map); - const struct nbrec_load_balancer *nbrec_lb; - NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - hmap_insert(lbs, &lb_nb->hmap_node, - uuid_hash(&nbrec_lb->header_.uuid)); + HMAP_FOR_EACH (lb, hmap_node, lbs) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); } - NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, - nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } - - hmap_insert(lb_groups, &lb_group->hmap_node, + HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { + lb_group_dps = ovn_lb_group_datapaths_create( + lb_group, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(&lb_group->uuid)); } @@ -4135,22 +4118,19 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbs->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_ls(lb, 1, &od); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { nbrec_lb_group = od->nbs->load_balancer_group[i]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_ls(lb_group, 1, &od); - } - } - - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], lb_group->n_ls, - lb_group->ls); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); } } @@ -4172,15 +4152,21 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; nbrec_lb_group = od->nbr->load_balancer_group[idx]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_lr(lb_group, od); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, od); if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_clone(lb_group->lb_ips); + od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); } else { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, lb_group->lbs[j]); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(od->lb_ips, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4192,16 +4178,23 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbr->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_lr(lb, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_lr(lb_group->lbs[j], lb_group->n_lr, - lb_group->lr); + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, + lb_group_dps->ls); + ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, + lb_group_dps->lr); } } } @@ -4210,10 +4203,10 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, - struct hmap *ls_ports, struct hmap *lbs, struct sset *svc_monitor_lsps) + struct hmap *ls_ports, struct hmap *lb_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { - struct hmap monitor_map = HMAP_INITIALIZER(&monitor_map); - const struct sbrec_service_monitor *sbrec_mon; SBREC_SERVICE_MONITOR_TABLE_FOR_EACH (sbrec_mon, sbrec_service_monitor_table) { @@ -4223,24 +4216,23 @@ build_lb_svcs( struct service_monitor_info *mon_info = xzalloc(sizeof *mon_info); mon_info->sbrec_mon = sbrec_mon; mon_info->required = false; - hmap_insert(&monitor_map, &mon_info->hmap_node, hash); + hmap_insert(svc_monitor_map, &mon_info->hmap_node, hash); } - struct ovn_northd_lb *lb; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - ovn_lb_svc_create(ovnsb_txn, lb, &monitor_map, ls_ports, + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, svc_monitor_lsps); } struct service_monitor_info *mon_info; - HMAP_FOR_EACH_POP (mon_info, hmap_node, &monitor_map) { + HMAP_FOR_EACH_SAFE (mon_info, hmap_node, svc_monitor_map) { if (!mon_info->required) { sbrec_service_monitor_delete(mon_info->sbrec_mon); + hmap_remove(svc_monitor_map, &mon_info->hmap_node); + free(mon_info); } - - free(mon_info); } - hmap_destroy(&monitor_map); } static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, @@ -4325,7 +4317,8 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) static void build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { struct ovn_datapath *od; @@ -4335,21 +4328,25 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_northd_lb *lb = - ovn_northd_lb_find(lbs, - &od->nbr->load_balancer[i]->header_.uuid); - build_lrouter_lb_reachable_ips(od, lb); + struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_dps_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); } for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { const struct nbrec_load_balancer_group *nbrec_lb_group = od->nbr->load_balancer_group[i]; - struct ovn_lb_group *lb_group; - - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, lb_group->lbs[j]); + struct ovn_lb_group_datapaths *lb_group_dps; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_dps_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lb_group_dps); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4357,45 +4354,50 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { if (!install_ls_lb_from_router) { return; } - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; size_t index; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; - ovn_northd_lb_add_ls(lb, od->n_ls_peers, od->ls_peers); - } - } - - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t i = 0; i < lb_group->n_lr; i++) { - struct ovn_datapath *od = lb_group->lr[i]; - ovn_lb_group_add_ls(lb_group, od->n_ls_peers, od->ls_peers); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], od->n_ls_peers, - od->ls_peers); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { + for (size_t i = 0; i < lb_group_dps->n_lr; i++) { + struct ovn_datapath *od = lb_group_dps->lr[i]; + ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, + od->ls_peers); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_dps_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); } } } } static void -build_lb_count_dps(struct hmap *lbs, +build_lb_count_dps(struct hmap *lb_dps_map, size_t n_ls_datapaths, size_t n_lr_datapaths) { - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - lb->n_nb_lr = bitmap_count1(lb->nb_lr_map, n_lr_datapaths); - lb->n_nb_ls = bitmap_count1(lb->nb_ls_map, n_ls_datapaths); + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + lb_dps->n_nb_lr = bitmap_count1(lb_dps->nb_lr_map, n_lr_datapaths); + lb_dps->n_nb_ls = bitmap_count1(lb_dps->nb_ls_map, n_ls_datapaths); } } @@ -4408,13 +4410,16 @@ build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, - struct hmap *lbs, struct hmap *lb_groups, struct sset *svc_monitor_lsps) + struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lbs, lb_groups); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lbs, - svc_monitor_lsps); - build_lswitch_lbs_from_lrouter(lr_datapaths, lbs, lb_groups); + build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, + lb_group_dps_map); + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + svc_monitor_lsps, svc_monitor_map); + build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -4535,17 +4540,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, return dpg; } +struct sb_lb { + struct hmap_node hmap_node; + + const struct sbrec_load_balancer *slb; + struct ovn_dp_group *dpg; + struct uuid lb_uuid; +}; + +static struct sb_lb * +find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { + if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, struct hmap *lbs) + struct ovn_datapaths *ls_datapaths, struct hmap *lb_dps_map) { struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); size_t bitmap_len = ods_size(ls_datapaths); - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); /* Delete any stale SB load balancer rows and create datapath * groups for existing ones. */ @@ -4568,28 +4595,32 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * "at-least-once" consistency for clustered database tables that * are not indexed in any way. */ - lb = ovn_northd_lb_find(lbs, &lb_uuid); - if (!lb || !lb->n_nb_ls || !hmapx_add(&existing_lbs, lb)) { + lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); + if (!lb_dps || !lb_dps->n_nb_ls || !hmapx_add(&existing_lbs, lb_dps)) { sbrec_load_balancer_delete(sbrec_lb); continue; } - lb->slb = sbrec_lb; + struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_uuid = lb_uuid; + sb_lb->slb = sbrec_lb; + hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); /* Find or create datapath group for this load balancer. */ - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + sb_lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sb_lb->slb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + bitmap_len, true, + ls_datapaths, NULL); } hmapx_destroy(&existing_lbs); /* Create SB Load balancer records if not present and sync * the SB load balancer columns. */ - HMAP_FOR_EACH (lb, hmap_node, lbs) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { continue; } @@ -4597,37 +4628,44 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * transport port) tuple. */ struct smap options; - smap_clone(&options, &lb->nlb->options); + smap_clone(&options, &lb_dps->lb->nlb->options); smap_replace(&options, "hairpin_orig_tuple", "true"); - if (!lb->slb) { + struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, + &lb_dps->lb->nlb->header_.uuid); + ovs_assert(!sb_lb || (sb_lb->slb && sb_lb->dpg)); + struct ovn_dp_group *lb_dpg = NULL; + if (!sb_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - lb->slb = sbrec_lb; char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb->nlb->header_.uuid)); + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); const struct smap external_ids = SMAP_CONST1(&external_ids, "lb_id", lb_id); sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); + } else { + sbrec_lb = sb_lb->slb; + lb_dpg = sb_lb->dpg; } /* Find or create datapath group for this load balancer. */ - if (!lb->dpg) { - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + if (!lb_dpg) { + lb_dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sbrec_lb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, bitmap_len, + true, ls_datapaths, NULL); } /* Update columns. */ - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, ovn_northd_lb_get_vips(lb)); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapath_group(lb->slb, lb->dpg->dp_group); - sbrec_load_balancer_set_options(lb->slb, &options); + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + sbrec_load_balancer_set_datapath_group(sbrec_lb, lb_dpg->dp_group); + sbrec_load_balancer_set_options(sbrec_lb, &options); /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(lb->slb, NULL, 0); + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); smap_destroy(&options); } @@ -4638,6 +4676,12 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } hmap_destroy(&dp_groups); + struct sb_lb *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { + free(sb_lb); + } + hmap_destroy(&sb_lbs); + /* Datapath_Binding.load_balancers is not used anymore, it's still in the * schema for compatibility reasons. Reset it to empty, just in case. */ @@ -7832,15 +7876,17 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules_pre_stateful(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; ds_clear(action); @@ -7886,7 +7932,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -7932,7 +7978,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8118,14 +8164,16 @@ build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_ls_flows(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) { - if (!lb->affinity_timeout || !lb->n_nb_ls) { + if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; struct ds new_lb_match = DS_EMPTY_INITIALIZER; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(&new_lb_match, @@ -8145,9 +8193,9 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, static char *aff_check = REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"; ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb->nlb->header_); + &lb_dps->lb->nlb->header_); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8235,14 +8283,15 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn), &lb->nlb->header_); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), + S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), + &lb->nlb->header_); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8275,11 +8324,13 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, - struct ds *action, const struct shash *meter_groups) + struct ds *action, const struct shash *meter_groups, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -8300,9 +8351,10 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, /* New connections in Ingress table. */ const char *meter = NULL; - bool reject = build_lb_vip_actions(lb_vip, lb_vip_nb, action, - lb->selection_fields, NULL, - NULL, true, features); + bool reject = build_lb_vip_actions(lb, lb_vip, lb_vip_nb, action, + lb->selection_fields, + NULL, NULL, true, features, + svc_monitor_map); ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); @@ -8313,15 +8365,17 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; if (reject) { size_t index; - dp_non_meter = bitmap_clone(lb->nb_ls_map, ods_size(ls_datapaths)); - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + dp_non_meter = bitmap_clone(lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; meter = copp_meter_get(COPP_REJECT, od->nbs->copp, @@ -8339,7 +8393,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( - lflows, dp_non_meter ? dp_non_meter : lb->nb_ls_map, + lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -9554,7 +9608,8 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, + const struct hmap *ls_ports, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -9569,7 +9624,14 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, for (size_t j = 0; j < lb_vip_nb->n_backends; j++) { struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - if (!backend_nb->op || !backend_nb->svc_mon_src_ip) { + + if (!backend_nb->health_check) { + continue; + } + + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + if (!op || !backend_nb->svc_mon_src_ip) { continue; } @@ -9611,7 +9673,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, svc_monitor_mac); } ovn_lflow_add_with_hint(lflows, - backend_nb->op->od, + op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_); @@ -11336,7 +11398,7 @@ struct lrouter_nat_lb_flows_ctx { struct ds *gw_redir_action; struct ovn_lb_vip *lb_vip; - struct ovn_northd_lb *lb; + const struct ovn_northd_lb *lb; bool reject; int prio; @@ -11468,14 +11530,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, static void build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb *lb, + struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; const char *ip_match = ipv4 ? "ip4" : "ip6"; @@ -11490,9 +11554,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_clear(match); ds_clear(action); - bool reject = build_lb_vip_actions(lb_vip, vips_nb, action, + bool reject = build_lb_vip_actions(lb, lb_vip, vips_nb, action, lb->selection_fields, &skip_snat_act, - &force_snat_act, false, features); + &force_snat_act, false, features, + svc_monitor_map); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows. @@ -11567,7 +11632,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * lflow generation for them. */ size_t index; - BITMAP_FOR_EACH_1 (index, bitmap_len, lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, bitmap_len, lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; @@ -11647,16 +11712,19 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } static void -build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -11666,7 +11734,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } size_t index; - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, @@ -11690,10 +11758,10 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * a higher priority rule for load balancing below also commits the * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ - build_lb_rules_pre_stateful(lflows, lb, features->ct_no_masked_label, + build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, ls_datapaths, match, action); - build_lb_rules(lflows, lb, ls_datapaths, features, match, action, - meter_groups); + build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, + meter_groups, svc_monitor_map); } /* If there are any load balancing rules, we should send the packet to @@ -11705,17 +11773,17 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * defragmentation to match on L4 ports. */ static void -build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, +build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } - for (size_t i = 0; i < lb->n_vips; i++) { - struct ovn_lb_vip *lb_vip = &lb->vips[i]; + for (size_t i = 0; i < lb_dps->lb->n_vips; i++) { + struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i]; bool ipv6 = lb_vip->address_family == AF_INET6; int prio = 100; @@ -11724,36 +11792,41 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); ovn_lflow_add_with_dp_group( - lflows, lb->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, - prio, ds_cstr(match), "ct_dnat;", &lb->nlb->header_); + lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), + S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", + &lb_dps->lb->nlb->header_); } } static void -build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { size_t index; - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; - build_lrouter_nat_flows_for_lb(lb_vip, lb, &lb->vips_nb[i], + build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lflows, match, action, - meter_groups, features); + meter_groups, features, + svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, S_ROUTER_IN_DNAT, @@ -11767,7 +11840,7 @@ build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } if (lb->skip_snat) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, @@ -15484,7 +15557,8 @@ struct lswitch_flow_build_info { struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_dps_map; + const struct hmap *svc_monitor_map; const struct hmap *bfd_connections; const struct chassis_features *features; char *svc_check_match; @@ -15628,7 +15702,7 @@ build_lflows_thread(void *arg) struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; int bnum; @@ -15695,28 +15769,33 @@ build_lflows_thread(void *arg) } } for (bnum = control->id; - bnum <= lsi->lbs->mask; + bnum <= lsi->lb_dps_map->mask; bnum += control->pool->size) { - HMAP_FOR_EACH_IN_PARALLEL (lb, hmap_node, bnum, lsi->lbs) { + HMAP_FOR_EACH_IN_PARALLEL (lb_dps, hmap_node, bnum, + lsi->lb_dps_map) { if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb, lsi->lflows, + build_lswitch_arp_nd_service_monitor(lb_dps->lb, + lsi->ls_ports, + lsi->lflows, &lsi->match, &lsi->actions); - build_lrouter_defrag_flows_for_lb(lb, lsi->lflows, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match); - build_lrouter_flows_for_lb(lb, lsi->lflows, + build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); - build_lswitch_flows_for_lb(lb, lsi->lflows, + build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); } } @@ -15782,7 +15861,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, - const struct hmap *lbs, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, const struct chassis_features *features) { @@ -15809,7 +15889,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].port_groups = port_groups; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; - lsiv[index].lbs = lbs; + lsiv[index].lb_dps_map = lb_dps_map; + lsiv[index].svc_monitor_map = svc_monitor_map; lsiv[index].bfd_connections = bfd_connections; lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; @@ -15832,7 +15913,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } else { struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, @@ -15843,7 +15924,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, - .lbs = lbs, + .lb_dps_map = lb_dps_map, + .svc_monitor_map = svc_monitor_map, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, @@ -15875,17 +15957,19 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); - HMAP_FOR_EACH (lb, hmap_node, lbs) { - build_lswitch_arp_nd_service_monitor(lb, lsi.lflows, - &lsi.actions, + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.lflows, &lsi.actions, &lsi.match); - build_lrouter_defrag_flows_for_lb(lb, lsi.lflows, lsi.lr_datapaths, - &lsi.match); - build_lrouter_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, + lsi.lr_datapaths, &lsi.match); + build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); - build_lswitch_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15985,7 +16069,9 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->port_groups, lflows, &igmp_groups, - input_data->meter_groups, input_data->lbs, + input_data->meter_groups, + input_data->lb_datapaths_map, + input_data->svc_monitor_map, input_data->bfd_connections, input_data->features); @@ -17388,8 +17474,8 @@ northd_init(struct northd_data *data) hmap_init(&data->lr_ports); hmap_init(&data->port_groups); shash_init(&data->meter_groups); - hmap_init(&data->lbs); - hmap_init(&data->lb_groups); + hmap_init(&data->lb_datapaths_map); + hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); data->features = (struct chassis_features) { .ct_no_masked_label = true, @@ -17399,6 +17485,7 @@ northd_init(struct northd_data *data) }; data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); + hmap_init(&data->svc_monitor_map); data->change_tracked = false; ovs_list_init(&data->tracked_ls_changes.updated); } @@ -17406,17 +17493,18 @@ northd_init(struct northd_data *data) void northd_destroy(struct northd_data *data) { - struct ovn_northd_lb *lb; - HMAP_FOR_EACH_POP (lb, hmap_node, &data->lbs) { - ovn_northd_lb_destroy(lb); + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH_POP (lb_dps, hmap_node, &data->lb_datapaths_map) { + ovn_lb_datapaths_destroy(lb_dps); } - hmap_destroy(&data->lbs); + hmap_destroy(&data->lb_datapaths_map); - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH_POP (lb_group, hmap_node, &data->lb_groups) { - ovn_lb_group_destroy(lb_group); + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH_POP (lb_group_dps, hmap_node, + &data->lb_group_datapaths_map) { + ovn_lb_group_datapaths_destroy(lb_group_dps); } - hmap_destroy(&data->lb_groups); + hmap_destroy(&data->lb_group_datapaths_map); struct ovn_port_group *pg; HMAP_FOR_EACH_SAFE (pg, key_node, &data->port_groups) { @@ -17431,6 +17519,12 @@ northd_destroy(struct northd_data *data) } shash_destroy(&data->meter_groups); + struct service_monitor_info *mon_info; + HMAP_FOR_EACH_POP (mon_info, hmap_node, &data->svc_monitor_map) { + free(mon_info); + } + hmap_destroy(&data->svc_monitor_map); + /* XXX Having to explicitly clean up macam here * is a bit strange. We don't explicitly initialize * macam in this module, but this is the logical place @@ -17539,10 +17633,9 @@ ovnnb_db_run(struct northd_input *input_data, input_data->sbrec_chassis_table, &data->ls_datapaths, &data->lr_datapaths, &data->lr_list); - build_lbs(input_data->nbrec_load_balancer_table, - input_data->nbrec_load_balancer_group_table, - &data->ls_datapaths, &data->lr_datapaths, &data->lbs, - &data->lb_groups); + build_lb_datapaths(input_data->lbs, input_data->lb_groups, + &data->ls_datapaths, &data->lr_datapaths, + &data->lb_datapaths_map, &data->lb_group_datapaths_map); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, @@ -17557,9 +17650,11 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, &data->lr_datapaths, &data->ls_ports, - &data->lbs, &data->lb_groups, - &data->svc_monitor_lsps); - build_lb_count_dps(&data->lbs, + &data->lb_datapaths_map, + &data->lb_group_datapaths_map, + &data->svc_monitor_lsps, + &data->svc_monitor_map); + build_lb_count_dps(&data->lb_datapaths_map, ods_size(&data->ls_datapaths), ods_size(&data->lr_datapaths)); build_ipam(&data->ls_datapaths.datapaths, &data->ls_ports); diff --git a/northd/northd.h b/northd/northd.h index 48c282476a..7d92028c7d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -28,9 +28,6 @@ struct northd_input { const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; - const struct nbrec_load_balancer_table *nbrec_load_balancer_table; - const struct nbrec_load_balancer_group_table - *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; @@ -59,6 +56,10 @@ struct northd_input { *sbrec_chassis_template_var_table; const struct sbrec_mirror_table *sbrec_mirror_table; + /* Northd lb data node inputs*/ + const struct hmap *lbs; + const struct hmap *lb_groups; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -110,12 +111,13 @@ struct northd_data { struct hmap lr_ports; struct hmap port_groups; struct shash meter_groups; - struct hmap lbs; - struct hmap lb_groups; + struct hmap lb_datapaths_map; + struct hmap lb_group_datapaths_map; struct ovs_list lr_list; bool ovn_internal_version_changed; struct chassis_features features; struct sset svc_monitor_lsps; + struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; }; @@ -146,9 +148,10 @@ struct lflow_input { const struct hmap *lr_ports; const struct hmap *port_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; const struct chassis_features *features; + const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; }; From patchwork Wed Aug 2 06:25:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815780 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG26J3fbbz1yYC for ; Wed, 2 Aug 2023 16:25:44 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6EA034118A; Wed, 2 Aug 2023 06:25:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6EA034118A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8dJWYBg4cF5y; Wed, 2 Aug 2023 06:25:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3022540635; Wed, 2 Aug 2023 06:25:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3022540635 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 00A3FC0032; Wed, 2 Aug 2023 06:25:37 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 194B6C0032 for ; Wed, 2 Aug 2023 06:25:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 04CFB40A15 for ; Wed, 2 Aug 2023 06:25:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 04CFB40A15 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CCbjC-JnESbD for ; Wed, 2 Aug 2023 06:25:22 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp2.osuosl.org (Postfix) with ESMTPS id 7711440395 for ; Wed, 2 Aug 2023 06:25:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 7711440395 Received: by mail.gandi.net (Postfix) with ESMTPSA id 777571BF206; Wed, 2 Aug 2023 06:25:15 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:09 +0530 Message-Id: <20230802062509.3638732-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 3/8] northd: Add initial I-P for load balancer and load balancer groups X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to load balancers and load balancer groups are handled incrementally in the newly added 'lb_data' engine node. 'lb_data' is input to 'northd' node and the handler - northd_lb_data_handler in 'northd' node handles the changes. If a load balancer or load balancer group is associated to a logical switch or router then 'northd' node falls back to full recompute. Upcoming patch will handle this scenario. Signed-off-by: Numan Siddique --- lib/lb.c | 82 +++++-- lib/lb.h | 9 + northd/en-lb-data.c | 273 ++++++++++++++++++++++- northd/en-lb-data.h | 50 +++++ northd/en-northd.c | 41 ++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 12 +- northd/northd.c | 59 +++++ northd/northd.h | 7 + tests/ovn-northd.at | 457 +++++++++++++++++++++++++++++++++++++++ 10 files changed, 960 insertions(+), 31 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index e874680a3f..6fd67e2218 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -606,13 +606,13 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, return NULL; } -struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +static void +ovn_northd_lb_init(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); bool is_sctp = nullable_string_is_equal(nbrec_lb->protocol, "sctp"); - struct ovn_northd_lb *lb = xzalloc(sizeof *lb); int address_family = !strcmp(smap_get_def(&nbrec_lb->options, "address-family", "ipv4"), "ipv4") @@ -668,6 +668,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) "reject", false); ovn_northd_lb_vip_init(lb_vip_nb, lb_vip, nbrec_lb, node->key, node->value, template); + if (lb_vip_nb->lb_health_check) { + lb->health_checks = true; + } + if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { sset_add(&lb->ips_v4, lb_vip->vip_str); } else { @@ -711,6 +715,13 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) ds_chomp(&sel_fields, ','); lb->selection_fields = ds_steal_cstr(&sel_fields); } +} + +struct ovn_northd_lb * +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +{ + struct ovn_northd_lb *lb = xzalloc(sizeof *lb); + ovn_northd_lb_init(lb, nbrec_lb); return lb; } @@ -736,8 +747,8 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +static void +ovn_northd_lb_cleanup(struct ovn_northd_lb *lb) { for (size_t i = 0; i < lb->n_vips; i++) { ovn_lb_vip_destroy(&lb->vips[i]); @@ -745,24 +756,36 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) } free(lb->vips); free(lb->vips_nb); + lb->vips = NULL; + lb->vips_nb = NULL; smap_destroy(&lb->template_vips); sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); + lb->selection_fields = NULL; + lb->health_checks = false; +} + +void +ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +{ + ovn_northd_lb_cleanup(lb); free(lb); } -/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and an array of 'struct ovn_northd_lb' objects for its associated - * load balancers. */ -struct ovn_lb_group * -ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs) +void +ovn_northd_lb_reinit(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { - struct ovn_lb_group *lb_group; + ovn_northd_lb_cleanup(lb); + ovn_northd_lb_init(lb, nbrec_lb); +} - lb_group = xzalloc(sizeof *lb_group); - lb_group->uuid = nbrec_lb_group->header_.uuid; +static void +ovn_lb_group_init(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); lb_group->lb_ips = ovn_lb_ip_set_create(); @@ -772,10 +795,29 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); } +} +/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record + * and an array of 'struct ovn_northd_lb' objects for its associated + * load balancers. */ +struct ovn_lb_group * +ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + struct ovn_lb_group *lb_group = xzalloc(sizeof *lb_group); + lb_group->uuid = nbrec_lb_group->header_.uuid; + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); return lb_group; } +static void +ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) +{ + ovn_lb_ip_set_destroy(lb_group->lb_ips); + lb_group->lb_ips = NULL; + free(lb_group->lbs); +} + void ovn_lb_group_destroy(struct ovn_lb_group *lb_group) { @@ -783,11 +825,19 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) return; } - ovn_lb_ip_set_destroy(lb_group->lb_ips); - free(lb_group->lbs); + ovn_lb_group_cleanup(lb_group); free(lb_group); } +void +ovn_lb_group_reinit(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + ovn_lb_group_cleanup(lb_group); + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); +} + struct ovn_lb_group * ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *uuid) { diff --git a/lib/lb.h b/lib/lb.h index 0339050cba..74905c73b7 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -77,6 +77,9 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; + + /* Indicates if the load balancer has health checks configured. */ + bool health_checks; }; struct ovn_lb_vip { @@ -130,6 +133,8 @@ struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); +void ovn_northd_lb_reinit(struct ovn_northd_lb *, + const struct nbrec_load_balancer *); void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); @@ -148,6 +153,10 @@ struct ovn_lb_group *ovn_lb_group_create( void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *); +void ovn_lb_group_reinit( + struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); struct ovn_lb_datapaths { struct hmap_node hmap_node; diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 04d8d3e5d7..328c003675 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -19,6 +19,7 @@ #include /* OVS includes */ +#include "include/openvswitch/hmap.h" #include "openvswitch/util.h" #include "openvswitch/vlog.h" @@ -38,7 +39,27 @@ static void lb_data_destroy(struct ed_type_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static struct ovn_lb_group *create_lb_group( + const struct nbrec_load_balancer_group *, struct hmap *lbs, + struct hmap *lb_groups); +static void destroy_tracked_data(struct ed_type_lb_data *); +static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, + struct tracked_lb_data *, + bool health_checks); +static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, + struct tracked_lb_data *, + bool health_checks); +static struct crupdated_lb_group * + add_crupdated_lb_group_to_tracked_data(struct ovn_lb_group *, + struct tracked_lb_data *); +static void add_deleted_lb_group_to_tracked_data( + struct ovn_lb_group *, struct tracked_lb_data *); +/* 'lb_data' engine node manages the NB load balancers and load balancer + * groups. For each NB LB, it creates 'struct ovn_northd_lb' and + * for each NB LB group, it creates 'struct ovn_lb_group' and stores in + * the respective hmaps in it data (ed_type_lb_data). + */ void * en_lb_data_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -60,6 +81,7 @@ en_lb_data_run(struct engine_node *node, void *data) const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); engine_set_node_state(node, EN_UPDATED); } @@ -71,12 +93,155 @@ en_lb_data_cleanup(void *data) lb_data_destroy(lb_data); } +void +en_lb_data_clear_tracked_data(void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + destroy_tracked_data(lb_data); +} + + +/* Handler functions. */ +bool +lb_data_load_balancer_handler(struct engine_node *node, void *data) +{ + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + + const struct nbrec_load_balancer *tracked_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH_TRACKED (tracked_lb, nb_lb_table) { + struct ovn_northd_lb *lb; + if (nbrec_load_balancer_is_new(tracked_lb)) { + /* New load balancer. */ + lb = ovn_northd_lb_create(tracked_lb); + hmap_insert(&lb_data->lbs, &lb->hmap_node, + uuid_hash(&tracked_lb->header_.uuid)); + add_crupdated_lb_to_tracked_data(lb, trk_lb_data, + lb->health_checks); + } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + hmap_remove(&lb_data->lbs, &lb->hmap_node); + add_deleted_lb_to_tracked_data(lb, trk_lb_data, + lb->health_checks); + } else { + /* Load balancer updated. */ + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + bool health_checks = lb->health_checks; + ovn_northd_lb_reinit(lb, tracked_lb); + health_checks |= lb->health_checks; + add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +lb_data_load_balancer_group_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct nbrec_load_balancer_group *tracked_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH_TRACKED (tracked_lb_group, + nb_lbg_table) { + if (nbrec_load_balancer_group_is_new(tracked_lb_group)) { + struct ovn_lb_group *lb_group = + create_lb_group(tracked_lb_group, &lb_data->lbs, + &lb_data->lb_groups); + struct crupdated_lb_group *clbg = + add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); + } + } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); + add_deleted_lb_group_to_tracked_data(lb_group, trk_lb_data); + } else { + + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + + /* Determine the lbs which are added or deleted for this + * lb group and add them to tracked data. + * Eg. If an lb group lbg1 before the update had [lb1, lb2, lb3] + * And in the update, lb2 was removed and lb4 got added, then + * add lb2 and lb4 to the trk_lb_data->crupdated_lbs. */ + struct hmapx pre_update_lbs = HMAPX_INITIALIZER(&pre_update_lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + hmapx_add(&pre_update_lbs, lb_group->lbs[i]); + } + ovn_lb_group_reinit(lb_group, tracked_lb_group, &lb_data->lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + struct crupdated_lb_group *clbg = + add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + struct ovn_northd_lb *lb = lb_group->lbs[i]; + struct hmapx_node *hmapx_node = hmapx_find(&pre_update_lbs, + lb); + if (!hmapx_node) { + hmapx_add(&clbg->assoc_lbs, lb); + } else { + hmapx_delete(&pre_update_lbs, hmapx_node); + } + } + + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &pre_update_lbs) { + struct ovn_northd_lb *lb = hmapx_node->data; + /* Check if the pre updated lb is actually deleted or + * just disassociated from the lb group. If it's just + * disassociated, then set 'has_dissassoc_lbs_from_lb_grops' to + * true. Later if required we can add this 'lb' to an hmapx of + * disassociated_lbs. */ + if (!hmapx_find(&trk_lb_data->deleted_lbs, lb)) { + trk_lb_data->has_dissassoc_lbs_from_lb_grops = true; + } + hmapx_delete(&pre_update_lbs, hmapx_node); + } + hmapx_destroy(&pre_update_lbs); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + /* static functions. */ static void lb_data_init(struct ed_type_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + hmap_init(&trk_lb_data->crupdated_lbs); + hmapx_init(&trk_lb_data->deleted_lbs); + hmap_init(&trk_lb_data->crupdated_lb_groups); + hmapx_init(&trk_lb_data->deleted_lb_groups); } static void @@ -93,6 +258,12 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) ovn_lb_group_destroy(lb_group); } hmap_destroy(&lb_data->lb_groups); + + destroy_tracked_data(lb_data); + hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); + hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); + hmapx_destroy(&lb_data->tracked_lb_data.deleted_lb_groups); + hmap_destroy(&lb_data->tracked_lb_data.crupdated_lb_groups); } static void @@ -100,12 +271,9 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, struct hmap *lbs, struct hmap *lb_groups) { - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb_nb; - const struct nbrec_load_balancer *nbrec_lb; NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - lb_nb = ovn_northd_lb_create(nbrec_lb); + struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb); hmap_insert(lbs, &lb_nb->hmap_node, uuid_hash(&nbrec_lb->header_.uuid)); } @@ -113,13 +281,98 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group *nbrec_lb_group; NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + create_lb_group(nbrec_lb_group, lbs, lb_groups); + } +} - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } +static struct ovn_lb_group * +create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); - hmap_insert(lb_groups, &lb_group->hmap_node, - uuid_hash(&lb_group->uuid)); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + + return lb_group; +} + +static void +destroy_tracked_data(struct ed_type_lb_data *lb_data) +{ + lb_data->tracked = false; + lb_data->tracked_lb_data.has_health_checks = false; + lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; + + struct hmapx_node *node; + HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { + ovn_northd_lb_destroy(node->data); + hmapx_delete(&lb_data->tracked_lb_data.deleted_lbs, node); + } + + HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lb_groups) { + ovn_lb_group_destroy(node->data); + hmapx_delete(&lb_data->tracked_lb_data.deleted_lb_groups, node); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH_POP (clb, hmap_node, + &lb_data->tracked_lb_data.crupdated_lbs) { + free(clb); + } + + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH_POP (crupdated_lbg, hmap_node, + &lb_data->tracked_lb_data.crupdated_lb_groups) { + hmapx_destroy(&crupdated_lbg->assoc_lbs); + free(crupdated_lbg); + } +} + +static void +add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, + struct tracked_lb_data *tracked_lb_data, + bool health_checks) +{ + struct crupdated_lb *clb = xzalloc(sizeof *clb); + clb->lb = lb; + hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); + if (health_checks) { + tracked_lb_data->has_health_checks = true; + } +} + +static void +add_deleted_lb_to_tracked_data(struct ovn_northd_lb *lb, + struct tracked_lb_data *tracked_lb_data, + bool health_checks) +{ + hmapx_add(&tracked_lb_data->deleted_lbs, lb); + if (health_checks) { + tracked_lb_data->has_health_checks = true; + } +} + +static struct crupdated_lb_group * +add_crupdated_lb_group_to_tracked_data(struct ovn_lb_group *lbg, + struct tracked_lb_data *tracked_lb_data) +{ + struct crupdated_lb_group *clbg = xzalloc(sizeof *clbg); + clbg->lbg = lbg; + hmapx_init(&clbg->assoc_lbs); + hmap_insert(&tracked_lb_data->crupdated_lb_groups, &clbg->hmap_node, + uuid_hash(&lbg->uuid)); + return clbg; +} + +static void +add_deleted_lb_group_to_tracked_data(struct ovn_lb_group *lbg, + struct tracked_lb_data *tracked_lb_data) +{ + hmapx_add(&tracked_lb_data->deleted_lb_groups, lbg); } diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index 96fb8c1f8d..2e9a024620 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -4,9 +4,51 @@ #include #include "openvswitch/hmap.h" +#include "include/openvswitch/list.h" +#include "lib/hmapx.h" #include "lib/inc-proc-eng.h" +struct ovn_northd_lb; +struct ovn_lb_group; + +struct crupdated_lb { + struct hmap_node hmap_node; + + struct ovn_northd_lb *lb; +}; + +struct crupdated_lb_group { + struct hmap_node hmap_node; + + struct ovn_lb_group *lbg; + /* hmapx of newly associated lbs to this lb group. + * hmapx node is 'struct ovn_northd_lb *' */ + struct hmapx assoc_lbs; +}; + +struct tracked_lb_data { + /* Both created and updated lbs. hmapx node is 'struct crupdated_lb *'. */ + struct hmap crupdated_lbs; + + /* Deleted lbs. */ + struct hmapx deleted_lbs; + + /* Both created and updated lb_groups. hmap node is + * 'struct crupdated_lb_group'. */ + struct hmap crupdated_lb_groups; + + /* Deleted lb_groups. hmapx node is 'struct ovn_lb_group *'. */ + struct hmapx deleted_lb_groups; + + /* Indicates if any of the tracked lb has health checks enabled. */ + bool has_health_checks; + + /* Indicates if any lb got disassociated from a lb group + * but not deleted. */ + bool has_dissassoc_lbs_from_lb_grops; +}; + /* struct which maintains the data of the engine node lb_data. */ struct ed_type_lb_data { /* hmap of load balancers. hmap node is 'struct ovn_northd_lb *' */ @@ -14,10 +56,18 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; + + /* tracked data*/ + bool tracked; + struct tracked_lb_data tracked_lb_data; }; void *en_lb_data_init(struct engine_node *, struct engine_arg *); void en_lb_data_run(struct engine_node *, void *data); void en_lb_data_cleanup(void *data); +void en_lb_data_clear_tracked_data(void *data); + +bool lb_data_load_balancer_handler(struct engine_node *, void *data); +bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index a0dae65752..d59ef062df 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,47 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_lb_data_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); + + if (!lb_data->tracked) { + return false; + } + + if (lb_data->tracked_lb_data.has_health_checks) { + /* Fall back to recompute since a tracked load balancer + * has health checks configured as I-P is not yet supported + * for such load balancers. */ + return false; + } + + /* Fall back to recompute if any load balancer was dissociated from + * a load balancer group (but not deleted). */ + if (lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops) { + return false; + } + + /* Fall back to recompute if load balancer groups are deleted. */ + if (!hmapx_is_empty(&lb_data->tracked_lb_data.deleted_lb_groups)) { + return false; + } + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 20cc77f108..3c77b64bb2 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,5 +17,6 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); +bool northd_lb_data_handler(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 0a99e87bc2..75d059645f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,13 +141,18 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); -static ENGINE_NODE(lb_data, "lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) { /* Define relationships between nodes where first argument is dependent * on the second argument */ + engine_add_input(&en_lb_data, &en_nb_load_balancer, + lb_data_load_balancer_handler); + engine_add_input(&en_lb_data, &en_nb_load_balancer_group, + lb_data_load_balancer_group_handler); + engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); @@ -175,13 +180,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, northd_nb_nb_global_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); - engine_add_input(&en_lb_data, &en_nb_load_balancer, NULL); - engine_add_input(&en_lb_data, &en_nb_load_balancer_group, NULL); - engine_add_input(&en_northd, &en_lb_data, NULL); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 250bc5f89f..f6d67f8283 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/lb.h" #include "memory.h" #include "northd.h" +#include "en-lb-data.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -5323,6 +5324,64 @@ northd_handle_sb_port_binding_changes( return true; } +/* Handler for lb_data engine changes. For every tracked lb_data + * it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths + * from the lb_datapaths hmap and lb_group_datapaths hmap. */ +bool +northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + struct ovn_northd_lb *lb; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lb_data->deleted_lbs) { + lb = hmapx_node->data; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); + ovn_lb_datapaths_destroy(lb_dps); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (!lb_dps) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + struct ovn_lb_group *lbg; + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH (crupdated_lbg, hmap_node, + &trk_lb_data->crupdated_lb_groups) { + lbg = crupdated_lbg->lbg; + const struct uuid *lb_uuid = &lbg->uuid; + + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + if (!lb_group_dps) { + lb_group_dps = ovn_lb_group_datapaths_create( + lbg, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ diff --git a/northd/northd.h b/northd/northd.h index 7d92028c7d..7d17921fa2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -347,6 +347,13 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); +struct tracked_lb_data; +bool northd_handle_lb_data_changes(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); + void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, const struct sbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 0ae0593c73..1b4081414e 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9711,3 +9711,460 @@ AT_CHECK([grep "lr_in_gw_redirect" R1flows |sed s'/table=../table=??/' |sort], [ AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing]) +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check ovn-nbctl --wait=sb set load_balancer . options:foo=bar +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +AT_CHECK([ovn-nbctl --wait=sb \ + -- --id=@hc create Load_Balancer_Health_Check vip="10.0.0.10\:80" \ + options:failure_count=100 \ + -- add Load_Balancer . health_check @hc | uuidfilt], [0], [<0> +]) +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Any change to load balancer health check should also result in full recompute +# of northd node (but not northd_lb_data node) +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer_health_check . options:foo=bar1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Delete the health check from the load balancer. northd engine node should do a full recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear Load_Balancer . health_check +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl ls-add sw0 +check ovn-nbctl --wait=sb lr-add lr0 +ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Associate lb1 to sw0. There should be a full recompute of northd engine node +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add lb1 to lr0 and then disassociate +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test load balancer group now +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) + +# Add lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear load_balancer_group . load_Balancer +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Add back lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Update lb and this should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer . options:bar=foo +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Modify the backend of the lb1 vip +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Cleanup the vip of lb1. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_Balancer lb1 vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Set the vips of lb1 back +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Add another vip to lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_router lr0 load_balancer_group +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Add back lb group to logical switch and then delete it. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group -- \ + destroy load_balancer_group $lbg1_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Test the scenario where a load balancer is associated to +# a logical switch sw0 and also to a lb group lbg1 and lbg1 +# is also associated to the logical switch sw0 and logical +# router lr1 + +check ovn-nbctl lr-add lr1 +check ovn-nbctl lb-add lb2 20.0.0.20:80 30.0.0.30:8080 +check ovn-nbctl lb-add lb3 30.0.0.20:80 30.0.0.30:8080 +check ovn-nbctl --wait=sb lb-add lb4 40.0.0.20:80 30.0.0.30:8080 + +lb2_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb2) +lb3_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb3) +lb4_uuid=$(fetch_column nb:Load_Balancer _uuid name=lb4) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 +check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 +check_engine_stats lb_data norecompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Deleting lb4 should not result in lflow recompute as it is +# only associated with logical switch sw0. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb4 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Deleting lb2 should result in lflow recompute as it is +# associated with logical router lr1 through lb group. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb2 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb remove load_balancer_group . load_balancer $lb3_uuid +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) From patchwork Wed Aug 2 06:25:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815781 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG26M1VQHz1yYC for ; Wed, 2 Aug 2023 16:25:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E19434192F; Wed, 2 Aug 2023 06:25:44 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E19434192F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id glEqX9bm9uUV; Wed, 2 Aug 2023 06:25:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0904941925; Wed, 2 Aug 2023 06:25:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 0904941925 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 442B2C0032; Wed, 2 Aug 2023 06:25:38 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 56450C008D for ; Wed, 2 Aug 2023 06:25:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 82D5181FAC for ; Wed, 2 Aug 2023 06:25:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 82D5181FAC X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wYUVlNCohAQa for ; Wed, 2 Aug 2023 06:25:28 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::229]) by smtp1.osuosl.org (Postfix) with ESMTPS id D480981F90 for ; Wed, 2 Aug 2023 06:25:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org D480981F90 Received: by mail.gandi.net (Postfix) with ESMTPSA id 82EA0FF804; Wed, 2 Aug 2023 06:25:24 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:19 +0530 Message-Id: <20230802062519.3638746-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 4/8] northd: Refactor the 'northd' node code which handles logical switch changes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This will help in handling other column changes of a logical switch. Acked-by: Han Zhou Signed-off-by: Numan Siddique --- northd/en-lb-data.c | 2 +- northd/northd.c | 310 +++++++++++++++++++++++++------------------- 2 files changed, 181 insertions(+), 131 deletions(-) diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 328c003675..23f2cb1021 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -58,7 +58,7 @@ static void add_deleted_lb_group_to_tracked_data( /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and * for each NB LB group, it creates 'struct ovn_lb_group' and stores in - * the respective hmaps in it data (ed_type_lb_data). + * the respective hmaps in it's data (ed_type_lb_data). */ void * en_lb_data_init(struct engine_node *node OVS_UNUSED, diff --git a/northd/northd.c b/northd/northd.c index f6d67f8283..526ec76b63 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4891,23 +4891,29 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, sset_destroy(&active_ha_chassis_grps); } +static void +destroy_tracked_ls_change(struct ls_change *ls_change) +{ + struct ovn_port *op; + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct ls_change *ls_change; LIST_FOR_EACH_SAFE (ls_change, list_node, &nd->tracked_ls_changes.updated) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); - } + destroy_tracked_ls_change(ls_change); ovs_list_remove(&ls_change->list_node); free(ls_change); } @@ -5024,15 +5030,21 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, return op; } +/* Returns true if the logical switch has changes which can be + * incrementally handled. + * Presently supports i-p for the below changes: + * - logical switch ports. + */ static bool -check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) +ls_changes_can_be_handled( + const struct nbrec_logical_switch *ls) { /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col) && col != NBREC_LOGICAL_SWITCH_COL_PORTS) { - return true; + return false; } } @@ -5041,44 +5053,44 @@ check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) for (size_t i = 0; i < ls->n_acls; i++) { if (nbrec_acl_row_get_seqno(ls->acls[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } if (ls->copp && nbrec_copp_row_get_seqno(ls->copp, OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } for (size_t i = 0; i < ls->n_dns_records; i++) { if (nbrec_dns_row_get_seqno(ls->dns_records[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_forwarding_groups; i++) { if (nbrec_forwarding_group_row_get_seqno(ls->forwarding_groups[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_load_balancer; i++) { if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; + return false; } } - return false; + return true; } static bool @@ -5119,6 +5131,149 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) return false; } +/* Handles logical switch port changes of a changed logical switch. + * Returns false, if any logical port can't be incrementally handled. + */ +static bool +ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od) +{ + bool ls_ports_changed = false; + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_PORTS)) { + + for (size_t i = 0; i < changed_ls->n_ports; i++) { + if (nbrec_logical_switch_port_row_get_seqno( + changed_ls->ports[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_ports_changed = true; + break; + } + } + } else { + ls_ports_changed = true; + } + + if (!ls_ports_changed) { + return true; + } + + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + op->visited = false; + } + + /* Compare the individual ports in the old and new Logical Switches */ + for (size_t j = 0; j < changed_ls->n_ports; ++j) { + struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; + op = ovn_port_find_in_datapath(od, new_nbsp->name); + + if (!op) { + if (!lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, NULL, NULL, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->added_ports, + &op->list); + } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { + /* Existing port updated */ + bool temp = false; + if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || + !op->lsp_can_be_inc_processed || + !lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + const struct sbrec_port_binding *sb = op->sb; + if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { + /* This port is used for svc monitor, which may be impacted + * by this change. Fallback to recompute. */ + goto fail; + } + if (!check_lsp_is_up && + !check_lsp_changes_other_than_up(new_nbsp)) { + /* If the only change is the "up" column while the + * "ignore_lsp_down" is set to true, just ignore this + * change. */ + op->visited = true; + continue; + } + struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); + ovs_list_splice(&lflows, op->lflows.next, &op->lflows); + ovn_port_destroy(&nd->ls_ports, op); + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, sb, &lflows, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + ovs_assert(ovs_list_is_empty(&lflows)); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->updated_ports, &op->list); + } + op->visited = true; + } + + /* Check for deleted ports */ + HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { + if (!op->visited) { + if (!op->lsp_can_be_inc_processed) { + goto fail_clean_deleted; + } + if (sset_contains(&nd->svc_monitor_lsps, op->key)) { + /* This port was used for svc monitor, which may be + * impacted by this deletion. Fallback to recompute. */ + goto fail_clean_deleted; + } + ovs_list_push_back(&ls_change->deleted_ports, + &op->list); + hmap_remove(&nd->ls_ports, &op->key_node); + hmap_remove(&od->ports, &op->dp_node); + sbrec_port_binding_delete(op->sb); + delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, + op->tunnel_key); + } + } + + if (!ovs_list_is_empty(&ls_change->added_ports) || + !ovs_list_is_empty(&ls_change->updated_ports) || + !ovs_list_is_empty(&ls_change->deleted_ports)) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } + + return true; + +fail_clean_deleted: + LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { + ovn_port_destroy_orphan(op); + } + +fail: + free(ls_change); + return false; +} + + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5129,12 +5284,9 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; - struct ovn_port *op; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { - ls_change = NULL; if (nbrec_logical_switch_is_new(changed_ls) || nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; @@ -5150,109 +5302,13 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - /* Now only able to handle lsp changes. */ - if (check_ls_changes_other_than_lsp(changed_ls)) { + /* Check if the ls changes can be handled or not. */ + if (!ls_changes_can_be_handled(changed_ls)) { goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - - HMAP_FOR_EACH (op, dp_node, &od->ports) { - op->visited = false; - } - - /* Compare the individual ports in the old and new Logical Switches */ - for (size_t j = 0; j < changed_ls->n_ports; ++j) { - struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; - op = ovn_port_find_in_datapath(od, new_nbsp->name); - - if (!op) { - if (!lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->added_ports, - &op->list); - } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { - /* Existing port updated */ - bool temp = false; - if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || - !op->lsp_can_be_inc_processed || - !lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - const struct sbrec_port_binding *sb = op->sb; - if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { - /* This port is used for svc monitor, which may be impacted - * by this change. Fallback to recompute. */ - goto fail; - } - if (!check_lsp_is_up && - !check_lsp_changes_other_than_up(new_nbsp)) { - /* If the only change is the "up" column while the - * "ignore_lsp_down" is set to true, just ignore this - * change. */ - op->visited = true; - continue; - } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->updated_ports, &op->list); - } - op->visited = true; - } - - /* Check for deleted ports */ - HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { - if (!op->visited) { - if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; - } - if (sset_contains(&nd->svc_monitor_lsps, op->key)) { - /* This port was used for svc monitor, which may be - * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; - } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); - hmap_remove(&nd->ls_ports, &op->key_node); - hmap_remove(&od->ports, &op->dp_node); - sbrec_port_binding_delete(op->sb); - delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, - op->tunnel_key); - } - } - - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, ni, nd, od)) { + goto fail; } } @@ -5261,13 +5317,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: - free(ls_change); destroy_northd_data_tracked_changes(nd); return false; } From patchwork Wed Aug 2 06:25:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815782 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG26q67xyz1yYC for ; Wed, 2 Aug 2023 16:26:11 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id ED0A46116B; Wed, 2 Aug 2023 06:26:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org ED0A46116B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L_vMd7lWJrAC; Wed, 2 Aug 2023 06:26:04 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5D2D2611CD; Wed, 2 Aug 2023 06:26:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5D2D2611CD Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 30093C007F; Wed, 2 Aug 2023 06:26:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 10285C0032 for ; Wed, 2 Aug 2023 06:26:02 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6E5E441902 for ; Wed, 2 Aug 2023 06:25:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6E5E441902 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xga6uhy7tTjj for ; Wed, 2 Aug 2023 06:25:45 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by smtp4.osuosl.org (Postfix) with ESMTPS id 80E444190E for ; Wed, 2 Aug 2023 06:25:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 80E444190E Received: by mail.gandi.net (Postfix) with ESMTPSA id 0EE15E0008; Wed, 2 Aug 2023 06:25:33 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:26 +0530 Message-Id: <20230802062526.3638767-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 5/8] northd: Handle load balancer changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique 'lb_data' engine node now also handles logical switch changes. Its data maintains ls to lb related information. i.e if a logical switch sw0 has lb1, lb2 and lb3 associated then it stores this info in its data. And when a new load balancer lb4 is associated to it, it stores this information in its tracked data so that 'northd' engine node can handle it accordingly. Tracked data will have information like: changed ls -> {sw0 : {associated_lbs: [lb4]} In the 'northd' engne node, an additional handler for 'lb_data' is added after the 'nbrec_logical_switch' changes. With this patch we now have 2 handlers for 'lb_data' in 'northd' engine node. ---- engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); ---- The first handler 'northd_lb_data_handler_pre_od' is called before the 'northd_nb_logical_switch_handler' handler and it just creates or deletes the lb_datapaths hmap for the tracked lbs. The second handler 'northd_lb_data_handler_post_od' is called after the 'northd_nb_logical_switch_handler' handler and it updates the ovn_lb_datapaths's 'nb_ls_map' bitmap. Eg. If the lb_data has the below tracked data: tracked_data = {'crupdated_lbs': [lb1, lb2], 'deleted_lbs': [lb3], 'crupdated_lb_groups': [lbg1, lbg2], 'crupdated_ls_lbs': [{ls: sw0, assoc_lbs: [lb1], {ls: sw1, assoc_lbs: [lb1, lb2]} then the first handler northd_lb_data_handler_pre_od(), creates the ovn_lb_datapaths object for lb1 and lb2 and deletes lb3 from the ovn_lb_datapaths hmap. Similarly for the created or updated lb groups lbg1 and lbg2. The second handler northd_lb_data_handler_post_od() updates the nb_ls_bitmap of lb1 for sw0 and sw1 and nb_ls_bitmap of lb2 for sw1. This second handler is added mainly so that the actual logical switch handler northd_nb_logical_switch_handler() has done modifying the 'od' struct. Signed-off-by: Numan Siddique --- lib/lb.c | 5 +- northd/en-lb-data.c | 176 +++++++++++++++++++++++++++++++++++++++ northd/en-lb-data.h | 17 ++++ northd/en-lflow.c | 6 ++ northd/en-northd.c | 40 +++++++-- northd/en-northd.h | 3 +- northd/inc-proc-northd.c | 5 +- northd/northd.c | 152 ++++++++++++++++++++++++++------- northd/northd.h | 15 ++-- tests/ovn-northd.at | 56 +++++++++---- 10 files changed, 414 insertions(+), 61 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index 6fd67e2218..e6c9dc2be2 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1088,7 +1088,10 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } } } diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 23f2cb1021..e0c4db1422 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -39,6 +39,14 @@ static void lb_data_destroy(struct ed_type_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static void build_od_lb_map(const struct nbrec_logical_switch_table *, + struct hmap *od_lb_map); +static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); +static void destroy_od_lb_data(struct od_lb_data *od_lb_data); +static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); + static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); @@ -54,6 +62,7 @@ static struct crupdated_lb_group * struct tracked_lb_data *); static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); +static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -80,9 +89,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + engine_set_node_state(node, EN_UPDATED); } @@ -230,18 +243,104 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) return true; } +struct od_lb_data { + struct hmap_node hmap_node; + struct uuid od_uuid; + struct uuidset *lbs; + struct uuidset *lbgrps; +}; + +bool +lb_data_logical_switch_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_switch *nbs; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (nbs, nb_ls_table) { + if (nbrec_logical_switch_is_deleted(nbs)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->ls_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); + } + } else { + if (!is_ls_lbs_changed(nbs)) { + continue; + } + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbs->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->ls_lb_map, + &nbs->header_.uuid); + } + + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + /* static functions. */ static void lb_data_init(struct ed_type_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + hmap_init(&lb_data->ls_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); hmapx_init(&trk_lb_data->deleted_lbs); hmap_init(&trk_lb_data->crupdated_lb_groups); hmapx_init(&trk_lb_data->deleted_lb_groups); + ovs_list_init(&trk_lb_data->crupdated_ls_lbs); } static void @@ -259,6 +358,12 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->lb_groups); + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->ls_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->ls_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -301,12 +406,67 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, return lb_group; } +static void +build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, + struct hmap *od_lb_map) +{ + const struct nbrec_logical_switch *nbrec_ls; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { + if (!nbrec_ls->n_load_balancer) { + continue; + } + + struct od_lb_data *od_lb_data = + create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { + uuidset_insert(od_lb_data->lbs, + &nbrec_ls->load_balancer[i]->header_.uuid); + } + } +} + +static struct od_lb_data * +create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); + od_lb_data->od_uuid = *od_uuid; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + hmap_insert(od_lb_map, &od_lb_data->hmap_node, + uuid_hash(&od_lb_data->od_uuid)); + return od_lb_data; +} + +static struct od_lb_data * +find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_WITH_HASH (od_lb_data, hmap_node, uuid_hash(od_uuid), + od_lb_map) { + if (uuid_equals(&od_lb_data->od_uuid, od_uuid)) { + return od_lb_data; + } + } + + return NULL; +} + +static void +destroy_od_lb_data(struct od_lb_data *od_lb_data) +{ + uuidset_destroy(od_lb_data->lbs); + free(od_lb_data->lbs); + free(od_lb_data); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { lb_data->tracked = false; lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; + lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -331,6 +491,15 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) hmapx_destroy(&crupdated_lbg->assoc_lbs); free(crupdated_lbg); } + + struct crupdated_od_lb_data *codlb; + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_ls_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + + free(codlb); + } } static void @@ -376,3 +545,10 @@ add_deleted_lb_group_to_tracked_data(struct ovn_lb_group *lbg, { hmapx_add(&tracked_lb_data->deleted_lb_groups, lbg); } + +static bool +is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index 2e9a024620..b2f86322a2 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -27,6 +28,13 @@ struct crupdated_lb_group { struct hmapx assoc_lbs; }; +struct crupdated_od_lb_data { + struct ovs_list list_node; + + struct uuid od_uuid; + struct uuidset assoc_lbs; +}; + struct tracked_lb_data { /* Both created and updated lbs. hmapx node is 'struct crupdated_lb *'. */ struct hmap crupdated_lbs; @@ -41,12 +49,19 @@ struct tracked_lb_data { /* Deleted lb_groups. hmapx node is 'struct ovn_lb_group *'. */ struct hmapx deleted_lb_groups; + /* List of logical switch <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_ls_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; /* Indicates if any lb got disassociated from a lb group * but not deleted. */ bool has_dissassoc_lbs_from_lb_grops; + + /* Indicates if a lb was disassociated from a logical switch. */ + bool has_dissassoc_lbs_from_od; }; /* struct which maintains the data of the engine node lb_data. */ @@ -56,6 +71,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; + struct hmap ls_lb_map; /* tracked data*/ bool tracked; @@ -69,5 +85,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); +bool lb_data_logical_switch_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index db1bcbccd6..77e2eff056 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -102,6 +102,12 @@ lflow_northd_handler(struct engine_node *node, if (!northd_data->change_tracked) { return false; } + + /* Fall back to recompute if lb related data has changed. */ + if (northd_data->lb_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; diff --git a/northd/en-northd.c b/northd/en-northd.c index d59ef062df..9d1838a1a4 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -207,7 +207,7 @@ northd_sb_port_binding_handler(struct engine_node *node, } bool -northd_lb_data_handler(struct engine_node *node, void *data) +northd_lb_data_handler_pre_od(struct engine_node *node, void *data) { struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); @@ -230,19 +230,47 @@ northd_lb_data_handler(struct engine_node *node, void *data) /* Fall back to recompute if load balancer groups are deleted. */ if (!hmapx_is_empty(&lb_data->tracked_lb_data.deleted_lb_groups)) { + } + + if (lb_data->tracked_lb_data.has_dissassoc_lbs_from_od) { return false; } struct northd_data *nd = data; - if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, - &nd->ls_datapaths, - &nd->lr_datapaths, - &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +northd_lb_data_handler_post_od(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = + engine_get_input_data("lb_data", node); + + ovs_assert(lb_data->tracked); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lb_datapaths_map)) { return false; } + /* Indicate the depedendant engine nodes that load balancer/group + * related data has changed (including association to logical + * switch/router). */ + nd->lb_changed = true; engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/en-northd.h b/northd/en-northd.h index 3c77b64bb2..5926e7a9d3 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,6 +17,7 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); -bool northd_lb_data_handler(struct engine_node *, void *data); +bool northd_lb_data_handler_pre_od(struct engine_node *, void *data); +bool northd_lb_data_handler_post_od(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 75d059645f..402c94e88c 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -152,6 +152,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_handler); engine_add_input(&en_lb_data, &en_nb_load_balancer_group, lb_data_load_balancer_group_handler); + engine_add_input(&en_lb_data, &en_nb_logical_switch, + lb_data_logical_switch_handler); engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); @@ -180,9 +182,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, northd_nb_nb_global_handler); - engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); diff --git a/northd/northd.c b/northd/northd.c index 526ec76b63..059073da19 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -853,7 +853,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) ovn_ls_port_group_destroy(&od->nb_pgs); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); - free(od); } } @@ -4919,6 +4918,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) } nd->change_tracked = false; + nd->lb_changed = false; } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5034,6 +5034,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * incrementally handled. * Presently supports i-p for the below changes: * - logical switch ports. + * - load balancers. */ static bool ls_changes_can_be_handled( @@ -5042,8 +5043,11 @@ ls_changes_can_be_handled( /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { - if (nbrec_logical_switch_is_updated(ls, col) && - col != NBREC_LOGICAL_SWITCH_COL_PORTS) { + if (nbrec_logical_switch_is_updated(ls, col)) { + if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + continue; + } return false; } } @@ -5072,12 +5076,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer; i++) { - if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5139,7 +5137,9 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct nbrec_logical_switch *changed_ls, const struct northd_input *ni, struct northd_data *nd, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5160,12 +5160,6 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - struct ls_change *ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - struct ovn_port *op; HMAP_FOR_EACH (op, dp_node, &od->ports) { op->visited = false; @@ -5255,10 +5249,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!ovs_list_is_empty(&ls_change->added_ports) || !ovs_list_is_empty(&ls_change->updated_ports) || !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + *updated = true; } return true; @@ -5269,11 +5260,20 @@ fail_clean_deleted: } fail: - free(ls_change); + + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } return false; } - /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5307,14 +5307,33 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, ni, nd, od)) { + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + bool updated = false; + if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, + ni, nd, od, ls_change, + &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); goto fail; } + + if (updated) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } } if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { nd->change_tracked = true; } + return true; fail: @@ -5376,16 +5395,21 @@ northd_handle_sb_port_binding_changes( /* Handler for lb_data engine changes. For every tracked lb_data * it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths - * from the lb_datapaths hmap and lb_group_datapaths hmap. */ + * from the lb_datapaths hmap and lb_group_datapaths hmap. + * + * This handler is called if there are any lb_data changes but before + * processing the logical switch changes. + * */ bool -northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map) +northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; + struct ovn_datapath *od; struct hmapx_node *hmapx_node; HMAPX_FOR_EACH (hmapx_node, &trk_lb_data->deleted_lbs) { lb = hmapx_node->data; @@ -5393,6 +5417,16 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + + /* Re-evaluate 'od->has_lb_vip for od's associated with the + * deleted lb. */ + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + init_lb_for_datapath(od); + } + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); ovn_lb_datapaths_destroy(lb_dps); } @@ -5432,6 +5466,65 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return true; } +/* This handler is called if there are any lb_data changes but afer processing + * the logical switch changes. + * + * For every tracked data it does the following: + * For any changes to a created or updated logical switch due to + * association of a load balancer (eg. ovn-nbctl ls-lb-add sw0 lb1), + * the logical switch datapath is added to the load balancer (represented + * by 'struct ovn_lb_datapaths') by calling ovn_lb_datapaths_add_ls(). + * + * For every created or updated load balancer in the tracked data, + * it gets the associated logical switches and for each switch it + * re-evaluates 'od->has_lb_vip'. Since this handler is called after + * handling any logical switch changes. */ +bool +northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct hmap *lb_datapaths_map) +{ + ovs_assert(!trk_lb_data->has_health_checks); + + struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct ovn_datapath *od; + struct crupdated_od_lb_data *codlb; + + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { + od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + + struct crupdated_lb *clb; + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ @@ -16618,6 +16711,7 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *lflows) { struct ls_change *ls_change; + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, diff --git a/northd/northd.h b/northd/northd.h index 7d17921fa2..0ed7215356 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -120,6 +120,8 @@ struct northd_data { struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; + bool lb_changed; /* Indicates if load balancers changed or association of + * load balancer to logical switch/router changed. */ }; struct lflow_data { @@ -348,11 +350,14 @@ bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); struct tracked_lb_data; -bool northd_handle_lb_data_changes(struct tracked_lb_data *, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map); +bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); +bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct hmap *lb_datapaths_map); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 1b4081414e..a874d29f88 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9821,22 +9821,23 @@ ovn-nbctl lsp-set-type sw0-lr0 router ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute -# Associate lb1 to sw0. There should be a full recompute of northd engine node +# Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9844,7 +9845,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9852,7 +9853,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9860,14 +9861,33 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Associate lb1 to sw0 and also create a port sw0p1. This should result in +# full recompute of lflow engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +# Disassociate lb1 from sw0. There should be a recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9955,7 +9975,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10000,7 +10020,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10052,7 +10072,7 @@ check_engine_stats lflow recompute nocompute # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10093,7 +10113,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10107,15 +10127,15 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10129,7 +10149,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Wed Aug 2 06:25:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815783 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG27C6TQnz1yYC for ; Wed, 2 Aug 2023 16:26:31 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 9358D41B4C; Wed, 2 Aug 2023 06:26:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 9358D41B4C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jTuTn4bKwIXF; Wed, 2 Aug 2023 06:26:28 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id BB20F41B08; Wed, 2 Aug 2023 06:26:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BB20F41B08 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 862BBC0032; Wed, 2 Aug 2023 06:26:26 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 17414C0032 for ; Wed, 2 Aug 2023 06:26:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 0298661159 for ; Wed, 2 Aug 2023 06:25:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 0298661159 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OSLzI1co-mTu for ; Wed, 2 Aug 2023 06:25:50 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp3.osuosl.org (Postfix) with ESMTPS id 4B45E611CD for ; Wed, 2 Aug 2023 06:25:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 4B45E611CD Received: by mail.gandi.net (Postfix) with ESMTPSA id EA2D960005; Wed, 2 Aug 2023 06:25:45 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:41 +0530 Message-Id: <20230802062541.3638784-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 6/8] northd: Handle load balancer group changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique For every a given load balancer group 'A', northd engine data maintains a bitmap of datapaths associated to this lb group. So when lb group 'A' gets associated to a logical switch 's1', the bitmap index of 's1' is set in its bitmap. In order to handle the load balancer group changes incrementally for a logical switch, we need to set and clear the bitmap bits accordingly. And this patch does it. Signed-off-by: Numan Siddique --- northd/en-lb-data.c | 101 ++++++++++++++++++++++++++++++++------------ northd/en-lb-data.h | 4 ++ northd/en-northd.c | 9 +++- northd/northd.c | 64 ++++++++++++++++++++++++---- northd/northd.h | 3 +- tests/ovn-northd.at | 30 +++++++++---- 6 files changed, 164 insertions(+), 47 deletions(-) diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index e0c4db1422..0de8bfc29c 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -63,6 +63,7 @@ static struct crupdated_lb_group * static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); +static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -271,12 +272,15 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) destroy_od_lb_data(od_lb_data); } } else { - if (!is_ls_lbs_changed(nbs)) { + bool ls_lbs_changed = is_ls_lbs_changed(nbs); + bool ls_lbgrps_changed = is_ls_lbgrps_changed(nbs); + if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); struct od_lb_data *od_lb_data = find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); @@ -285,38 +289,65 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) &nbs->header_.uuid); } - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); + if (ls_lbs_changed) { + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + } - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; + if (ls_lbgrps_changed) { + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { + const struct uuid *lbg_uuid = + &nbs->load_balancer_group[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbg_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, + lbg_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); + changed = true; + } } - if (unode) { - uuidset_delete(pre_lb_uuids, unode); + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + changed = true; } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); } - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); } } @@ -412,7 +443,7 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { - if (!nbrec_ls->n_load_balancer) { + if (!nbrec_ls->n_load_balancer && !nbrec_ls->n_load_balancer_group) { continue; } @@ -422,6 +453,10 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, uuidset_insert(od_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } + for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { + uuidset_insert(od_lb_data->lbgrps, + &nbrec_ls->load_balancer_group[i]->header_.uuid); + } } } @@ -431,7 +466,9 @@ create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); od_lb_data->od_uuid = *od_uuid; od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); uuidset_init(od_lb_data->lbs); + uuidset_init(od_lb_data->lbgrps); hmap_insert(od_lb_map, &od_lb_data->hmap_node, uuid_hash(&od_lb_data->od_uuid)); @@ -456,7 +493,9 @@ static void destroy_od_lb_data(struct od_lb_data *od_lb_data) { uuidset_destroy(od_lb_data->lbs); + uuidset_destroy(od_lb_data->lbgrps); free(od_lb_data->lbs); + free(od_lb_data->lbgrps); free(od_lb_data); } @@ -467,6 +506,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; + lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -497,7 +537,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) &lb_data->tracked_lb_data.crupdated_ls_lbs) { ovs_list_remove(&codlb->list_node); uuidset_destroy(&codlb->assoc_lbs); - + uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } } @@ -552,3 +592,10 @@ is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); } + +static bool +is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer_group) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index b2f86322a2..bc09ddb7eb 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -33,6 +33,7 @@ struct crupdated_od_lb_data { struct uuid od_uuid; struct uuidset assoc_lbs; + struct uuidset assoc_lbgrps; }; struct tracked_lb_data { @@ -62,6 +63,9 @@ struct tracked_lb_data { /* Indicates if a lb was disassociated from a logical switch. */ bool has_dissassoc_lbs_from_od; + + /* Indicates if a lb group was disassociated from a logical switch. */ + bool has_dissassoc_lbgrps_from_od; }; /* struct which maintains the data of the engine node lb_data. */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 9d1838a1a4..545971f76f 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -236,6 +236,10 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) return false; } + if (lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od) { + return false; + } + struct northd_data *nd = data; if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, @@ -258,12 +262,15 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) ovs_assert(lb_data->tracked); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od); + ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops); struct northd_data *nd = data; if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, &nd->ls_datapaths, - &nd->lb_datapaths_map)) { + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { return false; } diff --git a/northd/northd.c b/northd/northd.c index 059073da19..8d6fb2ea37 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5035,6 +5035,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * Presently supports i-p for the below changes: * - logical switch ports. * - load balancers. + * - load balancer groups. */ static bool ls_changes_can_be_handled( @@ -5045,7 +5046,8 @@ ls_changes_can_be_handled( for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || - col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP) { continue; } return false; @@ -5076,12 +5078,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer_group; i++) { - if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5277,7 +5273,11 @@ fail: /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. */ + * false. + * + * Note: Changes to load balancer and load balancer groups associated with + * the logical switches are handled separately in the lb_data change handlers. + * */ bool northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct northd_input *ni, @@ -5482,12 +5482,15 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, - struct hmap *lb_datapaths_map) + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { ovs_assert(!trk_lb_data->has_health_checks); + ovs_assert(!trk_lb_data->has_dissassoc_lbs_from_lb_grops); struct ovn_northd_lb *lb; struct ovn_lb_datapaths *lb_dps; + struct ovn_lb_group_datapaths *lbgrp_dps; struct ovn_datapath *od; struct crupdated_od_lb_data *codlb; @@ -5502,6 +5505,22 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_ls(lbgrp_dps, 1, &od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + } + /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } @@ -5522,6 +5541,33 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, } } + struct ovn_lb_group *lbgrp; + struct crupdated_lb_group *crupdated_lbg; + HMAP_FOR_EACH (crupdated_lbg, hmap_node, + &trk_lb_data->crupdated_lb_groups) { + lbgrp = crupdated_lbg->lbg; + const struct uuid *lb_uuid = &lbgrp->uuid; + + lbgrp_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbg->assoc_lbs) { + lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { + od = lbgrp_dps->ls[i]; + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + } + return true; } diff --git a/northd/northd.h b/northd/northd.h index 0ed7215356..a51b42bcd5 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -357,7 +357,8 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct hmap *lb_group_datapaths_map); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, - struct hmap *lb_datapaths_map); + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a874d29f88..7868d321b4 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2141,6 +2141,18 @@ check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl ]) +# Now associate vip again to lb4 and then delete it. +check ovn-nbctl set load_balancer $lb4 vips:"10.0.0.13"="10.0.0.6" +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl + table=1 (ls_out_pre_lb ), priority=100 , match=(ip), action=(reg0[[2]] = 1; next;) +]) + +check ovn-nbctl lb-del $lb4 +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl +]) + AT_CLEANUP ]) @@ -9976,21 +9988,21 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute -# Update lb and this should result in recompute +# Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9998,7 +10010,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10006,7 +10018,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10014,7 +10026,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10073,7 +10085,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10114,7 +10126,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Wed Aug 2 06:25:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815784 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG27M51HTz1yYC for ; Wed, 2 Aug 2023 16:26:39 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A124981F90; Wed, 2 Aug 2023 06:26:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A124981F90 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hDZZP9slkpJ4; Wed, 2 Aug 2023 06:26:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9924C82003; Wed, 2 Aug 2023 06:26:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9924C82003 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 73359C007F; Wed, 2 Aug 2023 06:26:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 197B9C0DD2 for ; Wed, 2 Aug 2023 06:26:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E43A0418E4 for ; Wed, 2 Aug 2023 06:25:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E43A0418E4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7WiRQnXRxQ2B for ; Wed, 2 Aug 2023 06:25:58 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp4.osuosl.org (Postfix) with ESMTPS id 85D8C4194E for ; Wed, 2 Aug 2023 06:25:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 85D8C4194E Received: by mail.gandi.net (Postfix) with ESMTPSA id AA5911BF203; Wed, 2 Aug 2023 06:25:54 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:47 +0530 Message-Id: <20230802062547.3638801-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 7/8] northd: Sync SB Port bindings NAT column in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node 'sync_to_sb_pb' is added within 'sync_to_sb' node to sync NAT column of Port bindings table. This separation is required in order to add load balancer group I-P handling in 'northd' engine node (which is handled in the next commit). 'sync_to_sb_pb' engine node can be later expanded to sync other Port binding columns if required. Signed-off-by: Numan Siddique --- northd/en-sync-sb.c | 31 +++++ northd/en-sync-sb.h | 4 + northd/inc-proc-northd.c | 8 +- northd/northd.c | 243 +++++++++++++++++++++------------------ northd/northd.h | 2 + 5 files changed, 174 insertions(+), 114 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 9832fce30a..552ed56452 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -254,6 +254,37 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } +/* sync_to_sb_pb engine node functions. + * This engine node syncs the SB Port Bindings (partly). + * en_northd engine create the SB Port binding rows and + * updates most of the columns. + * This engine node updates the port binding columns which + * needs to be updated after northd engine is run. + */ + +void * +en_sync_to_sb_pb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_pb_cleanup(void *data OVS_UNUSED) +{ + +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index 06d2a57710..700d3340e4 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -22,4 +22,8 @@ void en_sync_to_sb_lb_run(struct engine_node *, void *data); void en_sync_to_sb_lb_cleanup(void *data); bool sync_to_sb_lb_northd_handler(struct engine_node *, void *data OVS_UNUSED); +void *en_sync_to_sb_pb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_pb_run(struct engine_node *, void *data); +void en_sync_to_sb_pb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 402c94e88c..dc8b880fd8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, @@ -215,13 +216,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_northd, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine syncs the SB Address_Set table and - * SB Load_Balancer table. + * Right now this engine syncs the SB Address_Set table, + * SB Load_Balancer table and (partly) SB Port_Binding table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_pb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index 8d6fb2ea37..907bf0887b 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3514,8 +3514,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, ds_destroy(&s); sbrec_port_binding_set_external_ids(op->sb, &op->nbrp->external_ids); - - sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } else { if (!lsp_is_router(op->nbsp)) { uint32_t queue_id = smap_get_int( @@ -3631,116 +3629,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, } else { sbrec_port_binding_set_options(op->sb, NULL); } - const char *nat_addresses = smap_get(&op->nbsp->options, - "nat-addresses"); - size_t n_nats = 0; - char **nats = NULL; - bool l3dgw_ports = op->peer && op->peer->od && - op->peer->od->n_l3dgw_ports; - if (nat_addresses && !strcmp(nat_addresses, "router")) { - if (op->peer && op->peer->od - && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, - "exclude-lb-vips-from-garp", false); - nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); - } - } else if (nat_addresses && (chassis || l3dgw_ports)) { - struct lport_addresses laddrs; - if (!extract_lsp_addresses(nat_addresses, &laddrs)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); - } else { - destroy_lport_addresses(&laddrs); - n_nats = 1; - nats = xcalloc(1, sizeof *nats); - struct ds nat_addr = DS_EMPTY_INITIALIZER; - ds_put_format(&nat_addr, "%s", nat_addresses); - if (l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&nat_addr, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - nats[0] = xstrdup(ds_cstr(&nat_addr)); - ds_destroy(&nat_addr); - } - } - - /* Add the router mac and IPv4 addresses to - * Port_Binding.nat_addresses so that GARP is sent for these - * IPs by the ovn-controller on which the distributed gateway - * router port resides if: - * - * - op->peer has 'reside-on-redirect-chassis' set and the - * the logical router datapath has distributed router port. - * - * - op->peer is distributed gateway router port. - * - * - op->peer's router is a gateway router and op has a localnet - * port. - * - * Note: Port_Binding.nat_addresses column is also used for - * sending the GARPs for the router port IPs. - * */ - bool add_router_port_garp = false; - if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { - if (is_l3dgw_port(op->peer)) { - add_router_port_garp = true; - } else if (smap_get_bool(&op->peer->nbrp->options, - "reside-on-redirect-chassis", false)) { - if (op->peer->od->n_l3dgw_ports == 1) { - add_router_port_garp = true; - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " - "set on logical router port %s, which " - "is on logical router %s, which has %" - PRIuSIZE" distributed gateway ports. This" - "option can only be used when there is " - "a single distributed gateway port.", - op->peer->key, op->peer->od->nbr->name, - op->peer->od->n_l3dgw_ports); - } - } - } else if (chassis && op->od->n_localnet_ports) { - add_router_port_garp = true; - } - - if (add_router_port_garp) { - struct ds garp_info = DS_EMPTY_INITIALIZER; - ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); - - for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; - i++) { - ds_put_format(&garp_info, " %s", - op->peer->lrp_networks.ipv4_addrs[i].addr_s); - } - - if (op->peer->od->n_l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&garp_info, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - - n_nats++; - nats = xrealloc(nats, (n_nats * sizeof *nats)); - nats[n_nats - 1] = ds_steal_cstr(&garp_info); - ds_destroy(&garp_info); - } - sbrec_port_binding_set_nat_addresses(op->sb, - (const char **) nats, n_nats); - for (size_t i = 0; i < n_nats; i++) { - free(nats[i]); - } - free(nats); } sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name); @@ -4695,6 +4583,137 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } +/* Sync the SB Port bindings which needs to be updated. + * Presently it syncs the nat column of port bindings corresponding to + * the logical switch ports. */ +void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +{ + ovs_assert(ovnsb_idl_txn); + + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, ls_ports) { + if (lsp_is_router(op->nbsp)) { + const char *chassis = NULL; + if (op->peer && op->peer->od && op->peer->od->nbr) { + chassis = smap_get(&op->peer->od->nbr->options, "chassis"); + } + + const char *nat_addresses = smap_get(&op->nbsp->options, + "nat-addresses"); + size_t n_nats = 0; + char **nats = NULL; + bool l3dgw_ports = op->peer && op->peer->od && + op->peer->od->n_l3dgw_ports; + if (nat_addresses && !strcmp(nat_addresses, "router")) { + if (op->peer && op->peer->od + && (chassis || op->peer->od->n_l3dgw_ports)) { + bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + "exclude-lb-vips-from-garp", false); + nats = get_nat_addresses(op->peer, &n_nats, false, + !exclude_lb_vips); + } + } else if (nat_addresses && (chassis || l3dgw_ports)) { + struct lport_addresses laddrs; + if (!extract_lsp_addresses(nat_addresses, &laddrs)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); + } else { + destroy_lport_addresses(&laddrs); + n_nats = 1; + nats = xcalloc(1, sizeof *nats); + struct ds nat_addr = DS_EMPTY_INITIALIZER; + ds_put_format(&nat_addr, "%s", nat_addresses); + if (l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&nat_addr, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + nats[0] = xstrdup(ds_cstr(&nat_addr)); + ds_destroy(&nat_addr); + } + } + + /* Add the router mac and IPv4 addresses to + * Port_Binding.nat_addresses so that GARP is sent for these + * IPs by the ovn-controller on which the distributed gateway + * router port resides if: + * + * - op->peer has 'reside-on-redirect-chassis' set and the + * the logical router datapath has distributed router port. + * + * - op->peer is distributed gateway router port. + * + * - op->peer's router is a gateway router and op has a localnet + * port. + * + * Note: Port_Binding.nat_addresses column is also used for + * sending the GARPs for the router port IPs. + * */ + bool add_router_port_garp = false; + if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { + if (is_l3dgw_port(op->peer)) { + add_router_port_garp = true; + } else if (smap_get_bool(&op->peer->nbrp->options, + "reside-on-redirect-chassis", false)) { + if (op->peer->od->n_l3dgw_ports == 1) { + add_router_port_garp = true; + } else { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " + "set on logical router port %s, which " + "is on logical router %s, which has %" + PRIuSIZE" distributed gateway ports. This" + "option can only be used when there is " + "a single distributed gateway port.", + op->peer->key, op->peer->od->nbr->name, + op->peer->od->n_l3dgw_ports); + } + } + } else if (chassis && op->od->n_localnet_ports) { + add_router_port_garp = true; + } + + if (add_router_port_garp) { + struct ds garp_info = DS_EMPTY_INITIALIZER; + ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); + + for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; + i++) { + ds_put_format(&garp_info, " %s", + op->peer->lrp_networks.ipv4_addrs[i].addr_s); + } + + if (op->peer->od->n_l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&garp_info, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + n_nats++; + nats = xrealloc(nats, (n_nats * sizeof *nats)); + nats[n_nats - 1] = ds_steal_cstr(&garp_info); + ds_destroy(&garp_info); + } + sbrec_port_binding_set_nat_addresses(op->sb, + (const char **) nats, n_nats); + for (size_t i = 0; i < n_nats; i++) { + free(nats[i]); + } + free(nats); + } else { + sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + } + } +} + static bool ovn_port_add_tnlid(struct ovn_port *op, uint32_t tunnel_key) { diff --git a/northd/northd.h b/northd/northd.h index a51b42bcd5..8e39d10f58 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -374,4 +374,6 @@ const char *northd_get_svc_monitor_mac(void); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); + #endif /* NORTHD_H */ From patchwork Wed Aug 2 06:25:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1815785 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RG27y4sKTz1yYC for ; Wed, 2 Aug 2023 16:27:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4FEA140BB3; Wed, 2 Aug 2023 06:27:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4FEA140BB3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HUo773UnNEYO; Wed, 2 Aug 2023 06:27:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id CB24341461; Wed, 2 Aug 2023 06:27:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CB24341461 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id B1AAFC0071; Wed, 2 Aug 2023 06:27:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 977CEC0032 for ; Wed, 2 Aug 2023 06:27:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 65B4E41908 for ; Wed, 2 Aug 2023 06:26:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 65B4E41908 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PASYsspk7rHH for ; Wed, 2 Aug 2023 06:26:14 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp4.osuosl.org (Postfix) with ESMTPS id EA22E41A04 for ; Wed, 2 Aug 2023 06:26:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EA22E41A04 Received: by mail.gandi.net (Postfix) with ESMTPSA id 48B4A1C0004; Wed, 2 Aug 2023 06:26:08 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Wed, 2 Aug 2023 11:55:56 +0530 Message-Id: <20230802062556.3638822-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230802062103.3638403-1-numans@ovn.org> References: <20230802062103.3638403-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v4 8/8] northd: Handle load balancer/group changes for a logical router. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique When a logical router gets updated due to load balancer or load balancer groups changes, it is now incrementally handled first in 'lb_data' engine node similar to how logical switch changes are handled. The tracking data of 'lb_data' is updated similarly so that northd engine handler - northd_handle_lb_data_changes_post_od() handles it. A new handler northd_handle_lr_changes() is added in the 'northd' engine node for logical router changes. This handler returns true if only load balancer or load balancer group columns are changed. It returns false for any other changes. northd_handle_lb_data_changes_post_od() also sets the logical router od's lb_ips accordingly. Below are the scale testing results done with these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling in northd engine node) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.132929 2.157103 3.314847 3.331561 4.378626 1.581889 197.736147 125 0 Namespace.add_ports 0.005217 0.005760 0.006565 0.013348 0.021014 0.006106 0.763214 125 0 WorkerNode.bind_port 0.035205 0.045458 0.052278 0.059804 0.063941 0.045652 11.413122 250 0 WorkerNode.ping_port 0.005075 0.006814 3.088548 3.192577 4.242026 0.726453 181.613284 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 4.377260 6.486962 7.502040 8.322587 8.334701 6.559002 819.875306 125 0 Namespace.add_ports 0.005112 0.005484 0.005953 0.009153 0.011452 0.005662 0.707752 125 0 WorkerNode.bind_port 0.035360 0.042732 0.049152 0.053698 0.056635 0.043215 10.803700 250 0 WorkerNode.ping_port 0.005338 1.599904 7.229649 7.798039 8.206537 3.209860 802.464911 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- Few observations: - The total time taken has come down significantly from 819 seconds to 197. - 99%ile with these patches is 3.3 seconds compared to 8.3 seconds for the main. - 90%file with these patches is 3.3 seconds compared to 7.5 seconds for the main. - CPU utilization of northd during the test with these patches is between 100% to 300% which is almost the same as main. Main difference being that, with these patches the test duration is less and hence overall less CPU utilization. [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml Signed-off-by: Numan Siddique --- lib/lb.c | 46 +++++- lib/lb.h | 9 ++ northd/en-lb-data.c | 327 +++++++++++++++++++++++++++++++-------- northd/en-lb-data.h | 15 ++ northd/en-northd.c | 26 ++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 5 +- northd/northd.c | 242 ++++++++++++++++++++++++++--- northd/northd.h | 3 + tests/ovn-northd.at | 42 ++--- 10 files changed, 604 insertions(+), 112 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index e6c9dc2be2..d0d562b6fb 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, const struct uuid *lb_uuid = &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; } } @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) { ovn_lb_ip_set_destroy(lb_group->lb_ips); lb_group->lb_ips = NULL; + lb_group->has_routable_lb = false; free(lb_group->lbs); } @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) void build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, const struct ovn_northd_lb *lb) +{ + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); +} + +void +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { const char *ip_address; - SSET_FOR_EACH (ip_address, &lb->ips_v4) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v4_routable, ip_address); } } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { + SSET_FOR_EACH (ip_address, lb_ips_v6) { sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v6_routable, ip_address); } } } +void +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lb_ips->ips_v4, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lb_ips->ips_v6, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); + } + } +} + /* lb datapaths functions */ struct ovn_lb_datapaths * ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..b8e3c1e8fb 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); struct ovn_lb_group { struct hmap_node hmap_node; @@ -145,6 +153,7 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; + bool has_routable_lb; }; struct ovn_lb_group *ovn_lb_group_create( diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 0de8bfc29c..80cb959a47 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -40,20 +40,30 @@ static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); static void build_od_lb_map(const struct nbrec_logical_switch_table *, - struct hmap *od_lb_map); + const struct nbrec_logical_router_table *, + struct hmap *ls_lb_map, struct hmap *lr_lb_map); static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); static void destroy_od_lb_data(struct od_lb_data *od_lb_data); static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); +static void handle_od_lb_changes(struct nbrec_load_balancer **, + size_t n_nbrec_lbs, + struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, + size_t n_nbrec_lbs, + struct od_lb_data *, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); static void destroy_tracked_data(struct ed_type_lb_data *); -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, - struct tracked_lb_data *, - bool health_checks); +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( + struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); @@ -64,6 +74,8 @@ static void add_deleted_lb_group_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); const struct nbrec_logical_switch_table *nb_ls_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + const struct nbrec_logical_router_table *nb_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, + &lb_data->lr_lb_map); engine_set_node_state(node, EN_UPDATED); } @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) uuid_hash(&tracked_lb->header_.uuid)); add_crupdated_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) hmap_remove(&lb_data->lbs, &lb->hmap_node); add_deleted_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else { /* Load balancer updated. */ lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); ovs_assert(lb); bool health_checks = lb->health_checks; + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); + sset_swap(&lb->ips_v4, &old_ips_v4); + sset_swap(&lb->ips_v6, &old_ips_v6); ovn_northd_lb_reinit(lb, tracked_lb); health_checks |= lb->health_checks; - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( + lb, trk_lb_data, health_checks); + trk_lb_data->has_routable_lb |= lb->routable; + + /* Determine the inserted and deleted vips and store them in + * the tracked data. */ + const char *vip; + SSET_FOR_EACH (vip, &lb->ips_v4) { + if (!sset_find_and_delete(&old_ips_v4, vip)) { + sset_add(&clb->inserted_vips_v4, vip); + } + } + + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); + + SSET_FOR_EACH (vip, &lb->ips_v6) { + if (!sset_find_and_delete(&old_ips_v6, vip)) { + sset_add(&clb->inserted_vips_v6, vip); + } + } + + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); + + sset_destroy(&old_ips_v4); + sset_destroy(&old_ips_v6); } } @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) for (size_t i = 0; i < lb_group->n_lbs; i++) { hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); } + + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { struct ovn_lb_group *lb_group; lb_group = ovn_lb_group_find(&lb_data->lb_groups, @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) ovs_assert(lb_group); hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); add_deleted_lb_group_to_tracked_data(lb_group, trk_lb_data); + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else { struct ovn_lb_group *lb_group; @@ -209,6 +257,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; + struct crupdated_lb_group *clbg = add_crupdated_lb_group_to_tracked_data(lb_group, trk_lb_data); @@ -277,6 +327,7 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } + changed = true; struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); @@ -290,65 +341,77 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) } if (ls_lbs_changed) { - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); - - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); - - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; - } - - if (unode) { - uuidset_delete(pre_lb_uuids, unode); - } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; - } - - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); + handle_od_lb_changes(nbs->load_balancer, nbs->n_load_balancer, + od_lb_data, lb_data, codlb); } if (ls_lbgrps_changed) { - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); - uuidset_init(od_lb_data->lbgrps); - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { - const struct uuid *lbg_uuid = - &nbs->load_balancer_group[i]->header_.uuid; - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); - - if (!uuidset_find_and_delete(pre_lbgrp_uuids, - lbg_uuid)) { - /* Add this lb group to the tracked data. */ - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); - changed = true; - } - } + handle_od_lbgrp_changes(nbs->load_balancer_group, + nbs->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } - if (!uuidset_is_empty(pre_lbgrp_uuids)) { - trk_lb_data->has_dissassoc_lbgrps_from_od = true; - changed = true; - } + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +bool +lb_data_logical_router_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_router_table *nbrec_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - uuidset_destroy(pre_lbgrp_uuids); - free(pre_lbgrp_uuids); + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_router *nbr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { + if (nbrec_logical_router_is_deleted(nbr)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); } + } else { + bool lr_lbs_changed = is_lr_lbs_changed(nbr); + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); + if (!lr_lbs_changed && !lr_lbgrps_changed) { + continue; + } + changed = true; + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbr->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, + &nbr->header_.uuid); + } + + if (lr_lbs_changed) { + handle_od_lb_changes(nbr->load_balancer, nbr->n_load_balancer, + od_lb_data, lb_data, codlb); + } + + if (lr_lbgrps_changed) { + handle_od_lbgrp_changes(nbr->load_balancer_group, + nbr->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } + + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, &codlb->list_node); } } @@ -365,6 +428,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); hmap_init(&lb_data->ls_lb_map); + hmap_init(&lb_data->lr_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); @@ -372,6 +436,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&trk_lb_data->crupdated_lb_groups); hmapx_init(&trk_lb_data->deleted_lb_groups); ovs_list_init(&trk_lb_data->crupdated_ls_lbs); + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); } static void @@ -395,6 +460,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->ls_lb_map); + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->lr_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -439,7 +509,8 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, static void build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, - struct hmap *od_lb_map) + const struct nbrec_logical_router_table *nbrec_lr_table, + struct hmap *ls_lb_map, struct hmap *lr_lb_map) { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { @@ -447,17 +518,35 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct od_lb_data *od_lb_data = - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + struct od_lb_data *ls_lb_data = + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { - uuidset_insert(od_lb_data->lbs, + uuidset_insert(ls_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { - uuidset_insert(od_lb_data->lbgrps, + uuidset_insert(ls_lb_data->lbgrps, &nbrec_ls->load_balancer_group[i]->header_.uuid); } } + + const struct nbrec_logical_router *nbrec_lr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { + if (!nbrec_lr->n_load_balancer && !nbrec_lr->n_load_balancer_group) { + continue; + } + + struct od_lb_data *lr_lb_data = + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { + uuidset_insert(lr_lb_data->lbs, + &nbrec_lr->load_balancer[i]->header_.uuid); + } + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { + uuidset_insert(lr_lb_data->lbgrps, + &nbrec_lr->load_balancer_group[i]->header_.uuid); + } + } } static struct od_lb_data * @@ -499,6 +588,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) free(od_lb_data); } +static void +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < n_nbrec_lbs; i++) { + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_northd_lb *lb = ovn_northd_lb_find(&lb_data->lbs, + lb_uuid); + ovs_assert(lb); + trk_lb_data->has_routable_lb |= lb->routable; + } + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); +} + +static void +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, + size_t n_nbrec_lbgrps, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_lb_group *lbgrp = + ovn_lb_group_find(&lb_data->lb_groups, lbgrp_uuid); + ovs_assert(lbgrp); + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; + } + } + } + + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + } + + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { @@ -507,6 +674,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; + lb_data->tracked_lb_data.has_routable_lb = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -522,6 +690,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) struct crupdated_lb *clb; HMAP_FOR_EACH_POP (clb, hmap_node, &lb_data->tracked_lb_data.crupdated_lbs) { + sset_destroy(&clb->inserted_vips_v4); + sset_destroy(&clb->inserted_vips_v6); + sset_destroy(&clb->deleted_vips_v4); + sset_destroy(&clb->deleted_vips_v6); free(clb); } @@ -540,9 +712,16 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_lr_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + uuidset_destroy(&codlb->assoc_lbgrps); + free(codlb); + } } -static void +static struct crupdated_lb * add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, struct tracked_lb_data *tracked_lb_data, bool health_checks) @@ -551,9 +730,15 @@ add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, clb->lb = lb; hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, uuid_hash(&lb->nlb->header_.uuid)); + sset_init(&clb->inserted_vips_v4); + sset_init(&clb->inserted_vips_v6); + sset_init(&clb->deleted_vips_v4); + sset_init(&clb->deleted_vips_v6); if (health_checks) { tracked_lb_data->has_health_checks = true; } + + return clb; } static void @@ -599,3 +784,17 @@ is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); } + +static bool +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); +} + +static bool +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer_group) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index bc09ddb7eb..28b7367b51 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/sset.h" #include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -17,8 +18,13 @@ struct crupdated_lb { struct hmap_node hmap_node; struct ovn_northd_lb *lb; + struct sset inserted_vips_v4; + struct sset inserted_vips_v6; + struct sset deleted_vips_v4; + struct sset deleted_vips_v6; }; + struct crupdated_lb_group { struct hmap_node hmap_node; @@ -54,6 +60,10 @@ struct tracked_lb_data { * 'struct crupdated_od_lb_data' */ struct ovs_list crupdated_ls_lbs; + /* List of logical router <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_lr_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; @@ -66,6 +76,9 @@ struct tracked_lb_data { /* Indicates if a lb group was disassociated from a logical switch. */ bool has_dissassoc_lbgrps_from_od; + + /* Indicates if any lb (in the tracked data) has 'routable' flag set. */ + bool has_routable_lb; }; /* struct which maintains the data of the engine node lb_data. */ @@ -76,6 +89,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lb_groups; struct hmap ls_lb_map; + struct hmap lr_lb_map; /* tracked data*/ bool tracked; @@ -90,5 +104,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); bool lb_data_logical_switch_handler(struct engine_node *, void *data); +bool lb_data_logical_router_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 545971f76f..3be0f79e19 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,26 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_lb_data_handler_pre_od(struct engine_node *node, void *data) { @@ -240,6 +260,10 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) return false; } + if (lb_data->tracked_lb_data.has_routable_lb) { + return false; + } + struct northd_data *nd = data; if (!northd_handle_lb_data_changes_pre_od(&lb_data->tracked_lb_data, @@ -264,11 +288,13 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_od); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od); ovs_assert(!lb_data->tracked_lb_data.has_dissassoc_lbs_from_lb_grops); + ovs_assert(!lb_data->tracked_lb_data.has_routable_lb); struct northd_data *nd = data; if (!northd_handle_lb_data_changes_post_od(&lb_data->tracked_lb_data, &nd->ls_datapaths, + &nd->lr_datapaths, &nd->lb_datapaths_map, &nd->lb_group_datapaths_map)) { return false; diff --git a/northd/en-northd.h b/northd/en-northd.h index 5926e7a9d3..84d8673e1b 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_lb_data_handler_pre_od(struct engine_node *, void *data); bool northd_lb_data_handler_post_od(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index dc8b880fd8..9dbc2ec81a 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -155,10 +155,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_group_handler); engine_add_input(&en_lb_data, &en_nb_logical_switch, lb_data_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_logical_router, + lb_data_logical_router_handler); engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_meter, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); @@ -186,6 +187,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_pre_od); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler_post_od); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); diff --git a/northd/northd.c b/northd/northd.c index 907bf0887b..6a0120405f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4127,54 +4127,63 @@ static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr); static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr); + static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) +add_neigh_ips_to_lrouter(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { /* If configured to not reply to any neighbor requests for all VIPs * return early. */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { return; } + const char *ip_address; + /* If configured to reply to neighbor requests for all VIPs force them * all to be considered "reachable". */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); - } else { - sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); - } + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); } + return; } /* Otherwise, a VIP is reachable if there's at least one router * subnet that includes it. */ - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); - struct ovn_port *op; + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { if (lrouter_port_ipv4_reachable(op, vip_ip4)) { sset_add(&od->lb_ips->ips_v4_reachable, - lb->vips[i].vip_str); + ip_address); break; } } - } else { - struct ovn_port *op; + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { + if (lrouter_port_ipv6_reachable(op, &vip)) { sset_add(&od->lb_ips->ips_v6_reachable, - lb->vips[i].vip_str); + ip_address); break; } } @@ -4182,6 +4191,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, } } +static void +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); + } +} + +static void +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + + static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -5360,6 +5397,95 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. + * Note: Changes to load balancer and load balancer groups associated with + * the logical routers are handled separately in the lb_data change + * handlers (northd_handle_lb_data_changes_pre_od and + * northd_handle_lb_data_changes_post_od). + * */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, @@ -5501,6 +5627,7 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map) { @@ -5544,6 +5671,45 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, init_lb_for_datapath(od); } + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + struct crupdated_lb *clb; HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { lb = clb->lb; @@ -5558,6 +5724,29 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(od, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + } } struct ovn_lb_group *lbgrp; @@ -5577,8 +5766,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_uuid = &lb->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + } + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { - od = lbgrp_dps->ls[i]; + od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); /* Re-evaluate 'od->has_lb_vip' */ diff --git a/northd/northd.h b/northd/northd.h index 8e39d10f58..2e6fe758dc 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -335,6 +335,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); @@ -357,6 +359,7 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct hmap *lb_group_datapaths_map); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 7868d321b4..23fdaa9a61 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9907,8 +9907,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add lb1 to lr0 and then disassociate check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9916,7 +9916,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9924,7 +9924,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9932,7 +9932,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -9940,13 +9940,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10038,8 +10038,8 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10047,7 +10047,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10055,7 +10055,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10063,7 +10063,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10071,13 +10071,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10132,8 +10132,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10154,8 +10154,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10168,7 +10168,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10178,7 +10178,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10187,7 +10187,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE