From patchwork Sat Jun 24 06:47:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799367 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4Rx17pGz20Xr for ; Sat, 24 Jun 2023 16:47:56 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 24EDC820D0; Sat, 24 Jun 2023 06:47:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 24EDC820D0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TUQTWxPGniUs; Sat, 24 Jun 2023 06:47:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2BF7F81FAA; Sat, 24 Jun 2023 06:47:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2BF7F81FAA Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 10CD9C0037; Sat, 24 Jun 2023 06:47:53 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8E7D0C0029 for ; Sat, 24 Jun 2023 06:47:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5B4B4403E7 for ; Sat, 24 Jun 2023 06:47:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5B4B4403E7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qs-8Cf_trKW2 for ; Sat, 24 Jun 2023 06:47:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 51705404BB Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by smtp2.osuosl.org (Postfix) with ESMTPS id 51705404BB for ; Sat, 24 Jun 2023 06:47:46 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 58DF924000A; Sat, 24 Jun 2023 06:47:41 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:17:36 +0530 Message-Id: <20230624064736.11961-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 1/8] northd I-P: Sync SB load balancers in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Similar to the commit [1], a new sub-engine node "sync_to_sb_lb" is added with-in the "sync_to_sb" to sync the SB load balancers. Its main input nodes are "northd" (to access the "lbs" hmap built by this node) and "sb_load_balancer" to access the SB load balancer. "sync_to_sb_lb" doesn't add any handlers and falls back to full recompute all the time. [1] - ccafcc2dc321("northd I-P: Add a new engine node 'en_sync_to_sb' to sync SB tables.") Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 -- northd/en-sync-sb.c | 29 +++++++++++++++++++++++++++++ northd/en-sync-sb.h | 5 +++++ northd/inc-proc-northd.c | 9 +++++++-- northd/northd.c | 4 +--- northd/northd.h | 6 +++++- 6 files changed, 47 insertions(+), 8 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 044fa70190..f9f2d04452 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -101,8 +101,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis", node)); input_data->sbrec_fdb_table = EN_OVSDB_GET(engine_get_input("SB_fdb", node)); - input_data->sbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); input_data->sbrec_service_monitor_table = EN_OVSDB_GET(engine_get_input("SB_service_monitor", node)); input_data->sbrec_port_group_table = diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index d7fea981f2..821047581c 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -211,6 +211,35 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, return true; } +/* sync_to_sb_lb engine node functions. + * This engine node syncs the SB load balancers. + */ +void * +en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct sbrec_load_balancer_table *sb_load_balancer_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, + &northd_data->ls_datapaths, &northd_data->lbs); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +{ + +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index bcb9799d24..b3bfb5454f 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -16,4 +16,9 @@ bool sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *, bool sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *, void *data); + +void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_lb_run(struct engine_node *, void *data); +void en_sync_to_sb_lb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 19fc677956..dc4d991c97 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -139,6 +139,7 @@ static ENGINE_NODE(sync_to_sb, "sync_to_sb"); static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); +static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -166,7 +167,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); engine_add_input(&en_northd, &en_sb_service_monitor, NULL); - engine_add_input(&en_northd, &en_sb_load_balancer, NULL); engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); @@ -202,11 +202,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine only syncs the SB Address_Set table. + * Right now this engine syncs the SB Address_Set table and + * SB Load_Balancer table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index a45c8b53ad..7547f64e2d 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4494,7 +4494,7 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ -static void +void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, struct ovn_datapaths *ls_datapaths, struct hmap *lbs) @@ -17407,8 +17407,6 @@ ovnnb_db_run(struct northd_input *input_data, ovn_update_ipv6_options(&data->lr_ports); ovn_update_ipv6_prefix(&data->lr_ports); - sync_lbs(ovnsb_txn, input_data->sbrec_load_balancer_table, - &data->ls_datapaths, &data->lbs); sync_port_groups(ovnsb_txn, input_data->sbrec_port_group_table, &data->port_groups); sync_meters(ovnsb_txn, input_data->nbrec_meter_table, diff --git a/northd/northd.h b/northd/northd.h index f3e63b1e1a..48c282476a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -16,6 +16,7 @@ #include "ovsdb-idl.h" +#include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" #include "lib/sset.h" @@ -47,7 +48,6 @@ struct northd_input { const struct sbrec_ha_chassis_group_table *sbrec_ha_chassis_group_table; const struct sbrec_chassis_table *sbrec_chassis_table; const struct sbrec_fdb_table *sbrec_fdb_table; - const struct sbrec_load_balancer_table *sbrec_load_balancer_table; const struct sbrec_service_monitor_table *sbrec_service_monitor_table; const struct sbrec_port_group_table *sbrec_port_group_table; const struct sbrec_meter_table *sbrec_meter_table; @@ -354,4 +354,8 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, void run_update_worker_pool(int n_threads); const char *northd_get_svc_monitor_mac(void); + +void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, + struct ovn_datapaths *ls_datapaths, struct hmap *lbs); + #endif /* NORTHD_H */ From patchwork Sat Jun 24 06:47:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799368 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4SD10Gkz20Xr for ; Sat, 24 Jun 2023 16:48:12 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C042E82125; Sat, 24 Jun 2023 06:48:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C042E82125 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T87OgNt1d29y; Sat, 24 Jun 2023 06:48:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0786B82109; Sat, 24 Jun 2023 06:48:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0786B82109 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C54A5C0037; Sat, 24 Jun 2023 06:48:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3A01AC007C for ; Sat, 24 Jun 2023 06:48:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A239460E1D for ; Sat, 24 Jun 2023 06:48:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A239460E1D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yu0nesdfrWLL for ; Sat, 24 Jun 2023 06:47:58 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 78A20610BE Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id 78A20610BE for ; Sat, 24 Jun 2023 06:47:57 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 11AE11C000A; Sat, 24 Jun 2023 06:47:53 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:17:49 +0530 Message-Id: <20230624064749.11989-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 2/8] northd: Add a new engine node - northd_lb_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch separates out the 'lbs' and 'lb_groups' from the 'northd' engine node data into a new engine node 'northd_lb_data'. This new node becomes an input to the 'northd' node. This makes handling the NB load balancer and load balancer group changes easier. Signed-off-by: Numan Siddique --- lib/lb.c | 201 +++++++++-- lib/lb.h | 86 +++-- northd/automake.mk | 2 + northd/en-lflow.c | 3 +- northd/en-northd-lb-data.c | 126 +++++++ northd/en-northd-lb-data.h | 19 ++ northd/en-northd.c | 11 +- northd/en-sync-sb.c | 2 +- northd/inc-proc-northd.c | 8 +- northd/northd.c | 673 +++++++++++++++++++++---------------- northd/northd.h | 15 +- 11 files changed, 780 insertions(+), 366 deletions(-) create mode 100644 northd/en-northd-lb-data.c create mode 100644 northd/en-northd-lb-data.h diff --git a/lib/lb.c b/lib/lb.c index 7afdaed65b..429dbf15af 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -26,6 +26,7 @@ #include "openvswitch/vlog.h" #include "lib/bitmap.h" #include "lib/smap.h" +#include "socket-util.h" VLOG_DEFINE_THIS_MODULE(lb); @@ -431,11 +432,62 @@ void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } +static void +ovn_lb_vip_backends_health_check_init(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + struct ovn_northd_lb_vip *lb_vip_nb) +{ + struct ds key = DS_EMPTY_INITIALIZER; + + for (size_t j = 0; j < lb_vip->n_backends; j++) { + struct ovn_lb_backend *backend = &lb_vip->backends[j]; + ds_clear(&key); + ds_put_format(&key, IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? "%s" : "[%s]", backend->ip_str); + + const char *s = smap_get(&lb->nlb->ip_port_mappings, ds_cstr(&key)); + if (!s) { + continue; + } + + char *svc_mon_src_ip = NULL; + char *port_name = xstrdup(s); + char *p = strstr(port_name, ":"); + if (p) { + *p = 0; + p++; + struct sockaddr_storage svc_mon_src_addr; + if (!inet_parse_address(p, &svc_mon_src_addr)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); + } else { + struct ds src_ip_s = DS_EMPTY_INITIALIZER; + ss_format_address_nobracks(&svc_mon_src_addr, + &src_ip_s); + svc_mon_src_ip = ds_steal_cstr(&src_ip_s); + } + } + + if (svc_mon_src_ip) { + struct ovn_northd_lb_backend *backend_nb = + &lb_vip_nb->backends_nb[j]; + backend_nb->health_check = true; + backend_nb->logical_port = xstrdup(port_name); + backend_nb->svc_mon_src_ip = svc_mon_src_ip; + } + free(port_name); + } + + ds_destroy(&key); +} + static void ovn_northd_lb_vip_destroy(struct ovn_northd_lb_vip *vip) { free(vip->backend_ips); for (size_t i = 0; i < vip->n_backends; i++) { + free(vip->backends_nb[i].logical_port); free(vip->backends_nb[i].svc_mon_src_ip); } free(vip->backends_nb); @@ -555,8 +607,7 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, } struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, - size_t n_ls_datapaths, size_t n_lr_datapaths) +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); @@ -595,9 +646,6 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, } lb->affinity_timeout = affinity_timeout; - lb->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb->nb_lr_map = bitmap_allocate(n_lr_datapaths); - sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -631,7 +679,12 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, ovn_lb_vip6_template_format_internal(lb_vip), xstrdup(node->value)); } + n_vips++; + + if (lb_vip_nb->lb_health_check) { + ovn_lb_vip_backends_health_check_init(lb, lb_vip, lb_vip_nb); + } } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the @@ -639,6 +692,7 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, */ lb->n_vips = n_vips; + if (nbrec_lb->n_selection_fields) { char *proto = NULL; if (nbrec_lb->protocol && nbrec_lb->protocol[0]) { @@ -684,24 +738,6 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_lr_map, ods[i]->index); - } -} - -void -ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_ls_map, ods[i]->index); - } -} - void ovn_northd_lb_destroy(struct ovn_northd_lb *lb) { @@ -715,8 +751,6 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); - bitmap_free(lb->nb_lr_map); - bitmap_free(lb->nb_ls_map); free(lb); } @@ -727,8 +761,7 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ struct ovn_lb_group * ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs, size_t max_ls_datapaths, - size_t max_lr_datapaths) + const struct hmap *lbs) { struct ovn_lb_group *lb_group; @@ -736,8 +769,6 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, lb_group->uuid = nbrec_lb_group->header_.uuid; lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); - lb_group->ls = xmalloc(max_ls_datapaths * sizeof *lb_group->ls); - lb_group->lr = xmalloc(max_lr_datapaths * sizeof *lb_group->lr); lb_group->lb_ips = ovn_lb_ip_set_create(); for (size_t i = 0; i < nbrec_lb_group->n_load_balancer; i++) { @@ -758,8 +789,6 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) ovn_lb_ip_set_destroy(lb_group->lb_ips); free(lb_group->lbs); - free(lb_group->ls); - free(lb_group->lr); free(lb_group); } @@ -943,3 +972,113 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) hmap_destroy(tuples); } + +void +build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, + const struct ovn_northd_lb *lb) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, &lb->ips_v4) { + sset_add(&lb_ips->ips_v4, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, &lb->ips_v6) { + sset_add(&lb_ips->ips_v6, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v6_routable, ip_address); + } + } +} + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + } +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/lib/lb.h b/lib/lb.h index 23d8fc9e9b..0339050cba 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -59,7 +59,6 @@ struct ovn_northd_lb { struct hmap_node hmap_node; const struct nbrec_load_balancer *nlb; /* May be NULL. */ - const struct sbrec_load_balancer *slb; /* May be NULL. */ const char *proto; char *selection_fields; struct ovn_lb_vip *vips; @@ -78,14 +77,6 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; - - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; - - struct ovn_dp_group *dpg; }; struct ovn_lb_vip { @@ -129,23 +120,19 @@ struct ovn_northd_lb_vip { }; struct ovn_northd_lb_backend { - struct ovn_port *op; /* Logical port to which the ip belong to. */ bool health_check; + char *logical_port; /* Logical port to which the ip belong to. */ char *svc_mon_src_ip; /* Source IP to use for monitoring. */ - const struct sbrec_service_monitor *sbrec_monitor; }; -struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); +struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *); struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); -void ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); -void ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); + +void build_lrouter_lb_ips(struct ovn_lb_ip_set *, + const struct ovn_northd_lb *); struct ovn_lb_group { struct hmap_node hmap_node; @@ -153,35 +140,70 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; +}; + +struct ovn_lb_group *ovn_lb_group_create( + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); +void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); +struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, + const struct uuid *); + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; - /* Datapaths to which this LB group is applied. */ + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ size_t n_ls; struct ovn_datapath **ls; size_t n_lr; struct ovn_datapath **lr; }; -struct ovn_lb_group *ovn_lb_group_create( - const struct nbrec_load_balancer_group *, - const struct hmap *lbs, - size_t max_ls_datapaths, +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, size_t max_lr_datapaths); -void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); -struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, - const struct uuid *); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); static inline void -ovn_lb_group_add_ls(struct ovn_lb_group *lb_group, size_t n, - struct ovn_datapath **ods) +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) { - memcpy(&lb_group->ls[lb_group->n_ls], ods, n * sizeof *ods); - lb_group->n_ls += n; + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; } static inline void -ovn_lb_group_add_lr(struct ovn_lb_group *lb_group, struct ovn_datapath *lr) +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) { - lb_group->lr[lb_group->n_lr++] = lr; + lbg_dps->lr[lbg_dps->n_lr++] = lr; } struct ovn_controller_lb { diff --git a/northd/automake.mk b/northd/automake.mk index b17f1fdb54..6f60265b73 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -18,6 +18,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-sb.h \ northd/en-sync-from-sb.c \ northd/en-sync-from-sb.h \ + northd/en-northd-lb-data.c \ + northd/en-northd-lb-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 28ab1c67fb..db1bcbccd6 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -57,7 +57,8 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->port_groups = &northd_data->port_groups; lflow_input->meter_groups = &northd_data->meter_groups; - lflow_input->lbs = &northd_data->lbs; + lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; + lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; lflow_input->features = &northd_data->features; lflow_input->ovn_internal_version_changed = northd_data->ovn_internal_version_changed; diff --git a/northd/en-northd-lb-data.c b/northd/en-northd-lb-data.c new file mode 100644 index 0000000000..d46c3c27ed --- /dev/null +++ b/northd/en-northd-lb-data.c @@ -0,0 +1,126 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "openvswitch/util.h" + +#include "en-northd-lb-data.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(en_northd_lb_data); + +static void northd_lb_data_init(struct northd_lb_data *); +static void northd_lb_data_destroy(struct northd_lb_data *); +static void build_lbs(const struct nbrec_load_balancer_table *, + const struct nbrec_load_balancer_group_table *, + struct hmap *lbs, struct hmap *lb_groups); + +void * +en_northd_lb_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct northd_lb_data *data = xzalloc(sizeof *data); + + northd_lb_data_init(data); + + return data; +} + +void +en_northd_lb_data_run(struct engine_node *node, void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); + northd_lb_data_init(lb_data); + + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_northd_lb_data_cleanup(void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); +} + +/* static functions. */ +static void +northd_lb_data_init(struct northd_lb_data *lb_data) +{ + hmap_init(&lb_data->lbs); + hmap_init(&lb_data->lb_groups); +} + +static void +northd_lb_data_destroy(struct northd_lb_data *lb_data) +{ + struct ovn_northd_lb *lb; + HMAP_FOR_EACH_POP (lb, hmap_node, &lb_data->lbs) { + ovn_northd_lb_destroy(lb); + } + hmap_destroy(&lb_data->lbs); + + struct ovn_lb_group *lb_group; + HMAP_FOR_EACH_POP (lb_group, hmap_node, &lb_data->lb_groups) { + ovn_lb_group_destroy(lb_group); + } + hmap_destroy(&lb_data->lb_groups); +} + +static void +build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, + const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group; + struct ovn_northd_lb *lb_nb; + + const struct nbrec_load_balancer *nbrec_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { + lb_nb = ovn_northd_lb_create(nbrec_lb); + hmap_insert(lbs, &lb_nb->hmap_node, + uuid_hash(&nbrec_lb->header_.uuid)); + } + + const struct nbrec_load_balancer_group *nbrec_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, + nbrec_lb_group_table) { + lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + } +} diff --git a/northd/en-northd-lb-data.h b/northd/en-northd-lb-data.h new file mode 100644 index 0000000000..eb297e376d --- /dev/null +++ b/northd/en-northd-lb-data.h @@ -0,0 +1,19 @@ +#ifndef EN_NORTHD_LB_DATA_H +#define EN_NORTHD_LB_DATA_H 1 + +#include + +#include "openvswitch/hmap.h" + +#include "lib/inc-proc-eng.h" + +struct northd_lb_data { + struct hmap lbs; + struct hmap lb_groups; +}; + +void *en_northd_lb_data_init(struct engine_node *, struct engine_arg *); +void en_northd_lb_data_run(struct engine_node *, void *data); +void en_northd_lb_data_cleanup(void *data); + +#endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index f9f2d04452..cc7d838451 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -20,6 +20,7 @@ #include "coverage.h" #include "en-northd.h" +#include "en-northd-lb-data.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "openvswitch/list.h" /* TODO This is needed for ovn-parallel-hmap.h. @@ -70,10 +71,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - input_data->nbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); - input_data->nbrec_load_balancer_group_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data->nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); input_data->nbrec_meter_table = @@ -117,6 +114,11 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); input_data->sbrec_mirror_table = EN_OVSDB_GET(engine_get_input("SB_mirror", node)); + + struct northd_lb_data *lb_data = + engine_get_input_data("northd_lb_data", node); + input_data->lbs = &lb_data->lbs; + input_data->lb_groups = &lb_data->lb_groups; } void @@ -130,6 +132,7 @@ en_northd_run(struct engine_node *node, void *data) northd_init(data); northd_get_input_data(node, &input_data); + COVERAGE_INC(northd_run); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); ovnnb_db_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 821047581c..fda0ca5a68 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -230,7 +230,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lbs); + &northd_data->ls_datapaths, &northd_data->lb_datapaths_map); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index dc4d991c97..51f9609b25 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -35,6 +35,7 @@ #include "en-northd-output.h" #include "en-sync-sb.h" #include "en-sync-from-sb.h" +#include "en-northd-lb-data.h" #include "unixctl.h" #include "util.h" @@ -140,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -147,8 +149,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, /* Define relationships between nodes where first argument is dependent * on the second argument */ engine_add_input(&en_northd, &en_nb_port_group, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); @@ -178,6 +178,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, NULL); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, NULL); + engine_add_input(&en_northd, &en_northd_lb_data, NULL); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 7547f64e2d..9061b37dcb 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3818,14 +3818,11 @@ struct service_monitor_info { static struct service_monitor_info * -create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *monitor_map, - const char *ip, const char *logical_port, - uint16_t service_port, const char *protocol) +get_service_mon(const struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol, + uint32_t hash) { - uint32_t hash = service_port; - hash = hash_string(ip, hash); - hash = hash_string(logical_port, hash); struct service_monitor_info *mon_info; HMAP_FOR_EACH_WITH_HASH (mon_info, hmap_node, hash, monitor_map) { @@ -3837,6 +3834,26 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } } + return NULL; +} + +static struct service_monitor_info * +create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) +{ + uint32_t hash = service_port; + hash = hash_string(ip, hash); + hash = hash_string(logical_port, hash); + struct service_monitor_info *mon_info = + get_service_mon(monitor_map, ip, logical_port, service_port, + protocol, hash); + + if (mon_info) { + return mon_info; + } + struct sbrec_service_monitor *sbrec_mon = sbrec_service_monitor_insert(ovnsb_txn); sbrec_service_monitor_set_ip(sbrec_mon, ip); @@ -3850,7 +3867,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } static void -ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, +ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3867,58 +3885,27 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - struct ovn_port *op = NULL; - char *svc_mon_src_ip = NULL; - - struct ds key = DS_EMPTY_INITIALIZER; - ds_put_format(&key, - IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) - ? "%s" : "[%s]", backend->ip_str); - - const char *s = smap_get(&lb->nlb->ip_port_mappings, - ds_cstr(&key)); - if (s) { - char *port_name = xstrdup(s); - char *p = strstr(port_name, ":"); - if (p) { - *p = 0; - p++; - sset_add(svc_monitor_lsps, port_name); - op = ovn_port_find(ls_ports, port_name); - struct sockaddr_storage svc_mon_src_addr; - if (!inet_parse_address(p, &svc_mon_src_addr)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); - } else { - struct ds src_ip_s = DS_EMPTY_INITIALIZER; - ss_format_address_nobracks(&svc_mon_src_addr, - &src_ip_s); - svc_mon_src_ip = ds_steal_cstr(&src_ip_s); - } - } - free(port_name); + if (!backend_nb->health_check) { + continue; } - ds_destroy(&key); - if (!lb_vip_nb->lb_health_check || !op || !svc_mon_src_ip || - !lsp_is_enabled(op->nbsp)) { - free(svc_mon_src_ip); + sset_add(svc_monitor_lsps, backend_nb->logical_port); + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + + if (!op || !lsp_is_enabled(op->nbsp)) { continue; } - backend_nb->op = op; - backend_nb->svc_mon_src_ip = svc_mon_src_ip; - const char *protocol = lb->nlb->protocol; if (!protocol || !protocol[0]) { protocol = "tcp"; } - backend_nb->health_check = true; + struct service_monitor_info *mon_info = create_or_get_service_mon(ovnsb_txn, monitor_map, backend->ip_str, - backend_nb->op->nbsp->name, + backend_nb->logical_port, backend->port, protocol); ovs_assert(mon_info); @@ -3947,18 +3934,20 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, "offline"); } - backend_nb->sbrec_monitor = mon_info->sbrec_mon; mon_info->required = true; } } } static bool -build_lb_vip_actions(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb_vip *lb_vip_nb, +build_lb_vip_actions(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + const struct ovn_northd_lb_vip *lb_vip_nb, struct ds *action, char *selection_fields, - struct ds *skip_snat_action, struct ds *force_snat_action, - bool ls_dp, const struct chassis_features *features) + struct ds *skip_snat_action, + struct ds *force_snat_action, + bool ls_dp, const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const char *ct_lb_action = features->ct_no_masked_label ? "ct_lb_mark" : "ct_lb"; @@ -3973,10 +3962,31 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, struct ovn_lb_backend *backend = &lb_vip->backends[i]; struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[i]; - if (!backend_nb->health_check || - (backend_nb->health_check && backend_nb->sbrec_monitor && - backend_nb->sbrec_monitor->status && - strcmp(backend_nb->sbrec_monitor->status, "online"))) { + + if (!backend_nb->health_check) { + continue; + } + + const char *protocol = lb->nlb->protocol; + if (!protocol || !protocol[0]) { + protocol = "tcp"; + } + + uint32_t hash = backend->port; + hash = hash_string(backend->ip_str, hash); + hash = hash_string(backend_nb->logical_port, hash); + + struct service_monitor_info *mon_info = get_service_mon( + svc_monitor_map, backend->ip_str, backend_nb->logical_port, + backend->port, protocol, hash); + + if (!mon_info) { + continue; + } + + ovs_assert(mon_info->sbrec_mon); + if (mon_info->sbrec_mon->status && + strcmp(mon_info->sbrec_mon->status, "online")) { continue; } @@ -4026,59 +4036,32 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, } static void -build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, - const struct ovn_northd_lb *lb) -{ - const char *ip_address; - - SSET_FOR_EACH (ip_address, &lb->ips_v4) { - sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v4_routable, ip_address); - } - } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { - sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v6_routable, ip_address); - } - } -} - -static void -build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, - const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) +build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb; + struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_group *lb_group; + struct ovn_lb_datapaths *lb_dps; + const struct ovn_northd_lb *lb; - hmap_init(lbs); - hmap_init(lb_groups); + hmap_init(lb_datapaths_map); + hmap_init(lb_group_datapaths_map); - const struct nbrec_load_balancer *nbrec_lb; - NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - hmap_insert(lbs, &lb_nb->hmap_node, - uuid_hash(&nbrec_lb->header_.uuid)); + HMAP_FOR_EACH (lb, hmap_node, lbs) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); } - NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, - nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } - - hmap_insert(lb_groups, &lb_group->hmap_node, + HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { + lb_group_dps = ovn_lb_group_datapaths_create( + lb_group, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(&lb_group->uuid)); } @@ -4091,22 +4074,19 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbs->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_ls(lb, 1, &od); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { nbrec_lb_group = od->nbs->load_balancer_group[i]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_ls(lb_group, 1, &od); - } - } - - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], lb_group->n_ls, - lb_group->ls); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); } } @@ -4128,15 +4108,21 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; nbrec_lb_group = od->nbr->load_balancer_group[idx]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_lr(lb_group, od); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, od); if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_clone(lb_group->lb_ips); + od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); } else { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, lb_group->lbs[j]); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(od->lb_ips, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4148,16 +4134,23 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbr->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_lr(lb, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_lr(lb_group->lbs[j], lb_group->n_lr, - lb_group->lr); + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, + lb_group_dps->ls); + ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, + lb_group_dps->lr); } } } @@ -4166,10 +4159,10 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, - struct hmap *ls_ports, struct hmap *lbs, struct sset *svc_monitor_lsps) + struct hmap *ls_ports, struct hmap *lb_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { - struct hmap monitor_map = HMAP_INITIALIZER(&monitor_map); - const struct sbrec_service_monitor *sbrec_mon; SBREC_SERVICE_MONITOR_TABLE_FOR_EACH (sbrec_mon, sbrec_service_monitor_table) { @@ -4179,24 +4172,23 @@ build_lb_svcs( struct service_monitor_info *mon_info = xzalloc(sizeof *mon_info); mon_info->sbrec_mon = sbrec_mon; mon_info->required = false; - hmap_insert(&monitor_map, &mon_info->hmap_node, hash); + hmap_insert(svc_monitor_map, &mon_info->hmap_node, hash); } - struct ovn_northd_lb *lb; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - ovn_lb_svc_create(ovnsb_txn, lb, &monitor_map, ls_ports, + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, svc_monitor_lsps); } struct service_monitor_info *mon_info; - HMAP_FOR_EACH_POP (mon_info, hmap_node, &monitor_map) { + HMAP_FOR_EACH_SAFE (mon_info, hmap_node, svc_monitor_map) { if (!mon_info->required) { sbrec_service_monitor_delete(mon_info->sbrec_mon); + hmap_remove(svc_monitor_map, &mon_info->hmap_node); + free(mon_info); } - - free(mon_info); } - hmap_destroy(&monitor_map); } static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, @@ -4281,7 +4273,8 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) static void build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { struct ovn_datapath *od; @@ -4291,21 +4284,25 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_northd_lb *lb = - ovn_northd_lb_find(lbs, - &od->nbr->load_balancer[i]->header_.uuid); - build_lrouter_lb_reachable_ips(od, lb); + struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_dps_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); } for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { const struct nbrec_load_balancer_group *nbrec_lb_group = od->nbr->load_balancer_group[i]; - struct ovn_lb_group *lb_group; - - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, lb_group->lbs[j]); + struct ovn_lb_group_datapaths *lb_group_dps; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_dps_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lb_group_dps); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4313,45 +4310,50 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { if (!install_ls_lb_from_router) { return; } - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; size_t index; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; - ovn_northd_lb_add_ls(lb, od->n_ls_peers, od->ls_peers); - } - } - - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t i = 0; i < lb_group->n_lr; i++) { - struct ovn_datapath *od = lb_group->lr[i]; - ovn_lb_group_add_ls(lb_group, od->n_ls_peers, od->ls_peers); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], od->n_ls_peers, - od->ls_peers); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { + for (size_t i = 0; i < lb_group_dps->n_lr; i++) { + struct ovn_datapath *od = lb_group_dps->lr[i]; + ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, + od->ls_peers); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_dps_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); } } } } static void -build_lb_count_dps(struct hmap *lbs, +build_lb_count_dps(struct hmap *lb_dps_map, size_t n_ls_datapaths, size_t n_lr_datapaths) { - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - lb->n_nb_lr = bitmap_count1(lb->nb_lr_map, n_lr_datapaths); - lb->n_nb_ls = bitmap_count1(lb->nb_ls_map, n_ls_datapaths); + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + lb_dps->n_nb_lr = bitmap_count1(lb_dps->nb_lr_map, n_lr_datapaths); + lb_dps->n_nb_ls = bitmap_count1(lb_dps->nb_ls_map, n_ls_datapaths); } } @@ -4364,13 +4366,16 @@ build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, - struct hmap *lbs, struct hmap *lb_groups, struct sset *svc_monitor_lsps) + struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lbs, lb_groups); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lbs, - svc_monitor_lsps); - build_lswitch_lbs_from_lrouter(lr_datapaths, lbs, lb_groups); + build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, + lb_group_dps_map); + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + svc_monitor_lsps, svc_monitor_map); + build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -4491,17 +4496,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, return dpg; } +struct sb_lb { + struct hmap_node hmap_node; + + const struct sbrec_load_balancer *slb; + struct ovn_dp_group *dpg; + struct uuid lb_uuid; +}; + +static struct sb_lb * +find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { + if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, struct hmap *lbs) + struct ovn_datapaths *ls_datapaths, struct hmap *lb_dps_map) { struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); size_t bitmap_len = ods_size(ls_datapaths); - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); /* Delete any stale SB load balancer rows and create datapath * groups for existing ones. */ @@ -4524,28 +4551,32 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * "at-least-once" consistency for clustered database tables that * are not indexed in any way. */ - lb = ovn_northd_lb_find(lbs, &lb_uuid); - if (!lb || !lb->n_nb_ls || !hmapx_add(&existing_lbs, lb)) { + lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); + if (!lb_dps || !lb_dps->n_nb_ls || !hmapx_add(&existing_lbs, lb_dps)) { sbrec_load_balancer_delete(sbrec_lb); continue; } - lb->slb = sbrec_lb; + struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_uuid = lb_uuid; + sb_lb->slb = sbrec_lb; + hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); /* Find or create datapath group for this load balancer. */ - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + sb_lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sb_lb->slb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + bitmap_len, true, + ls_datapaths, NULL); } hmapx_destroy(&existing_lbs); /* Create SB Load balancer records if not present and sync * the SB load balancer columns. */ - HMAP_FOR_EACH (lb, hmap_node, lbs) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { continue; } @@ -4553,37 +4584,44 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * transport port) tuple. */ struct smap options; - smap_clone(&options, &lb->nlb->options); + smap_clone(&options, &lb_dps->lb->nlb->options); smap_replace(&options, "hairpin_orig_tuple", "true"); - if (!lb->slb) { + struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, + &lb_dps->lb->nlb->header_.uuid); + ovs_assert(!sb_lb || (sb_lb->slb && sb_lb->dpg)); + struct ovn_dp_group *lb_dpg = NULL; + if (!sb_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - lb->slb = sbrec_lb; char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb->nlb->header_.uuid)); + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); const struct smap external_ids = SMAP_CONST1(&external_ids, "lb_id", lb_id); sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); + } else { + sbrec_lb = sb_lb->slb; + lb_dpg = sb_lb->dpg; } /* Find or create datapath group for this load balancer. */ - if (!lb->dpg) { - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + if (!lb_dpg) { + lb_dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sbrec_lb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, bitmap_len, + true, ls_datapaths, NULL); } /* Update columns. */ - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, ovn_northd_lb_get_vips(lb)); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapath_group(lb->slb, lb->dpg->dp_group); - sbrec_load_balancer_set_options(lb->slb, &options); + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + sbrec_load_balancer_set_datapath_group(sbrec_lb, lb_dpg->dp_group); + sbrec_load_balancer_set_options(sbrec_lb, &options); /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(lb->slb, NULL, 0); + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); smap_destroy(&options); } @@ -4594,6 +4632,12 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } hmap_destroy(&dp_groups); + struct sb_lb *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { + free(sb_lb); + } + hmap_destroy(&sb_lbs); + /* Datapath_Binding.load_balancers is not used anymore, it's still in the * schema for compatibility reasons. Reset it to empty, just in case. */ @@ -7741,15 +7785,17 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules_pre_stateful(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; ds_clear(action); @@ -7795,7 +7841,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -7841,7 +7887,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8027,14 +8073,16 @@ build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_ls_flows(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) { - if (!lb->affinity_timeout || !lb->n_nb_ls) { + if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; struct ds new_lb_match = DS_EMPTY_INITIALIZER; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(&new_lb_match, @@ -8054,9 +8102,9 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, static char *aff_check = REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"; ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb->nlb->header_); + &lb_dps->lb->nlb->header_); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8144,14 +8192,15 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn), &lb->nlb->header_); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), + S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), + &lb->nlb->header_); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8184,11 +8233,13 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, - struct ds *action, const struct shash *meter_groups) + struct ds *action, const struct shash *meter_groups, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -8209,9 +8260,10 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, /* New connections in Ingress table. */ const char *meter = NULL; - bool reject = build_lb_vip_actions(lb_vip, lb_vip_nb, action, - lb->selection_fields, NULL, - NULL, true, features); + bool reject = build_lb_vip_actions(lb, lb_vip, lb_vip_nb, action, + lb->selection_fields, + NULL, NULL, true, features, + svc_monitor_map); ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); @@ -8222,15 +8274,17 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; if (reject) { size_t index; - dp_non_meter = bitmap_clone(lb->nb_ls_map, ods_size(ls_datapaths)); - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + dp_non_meter = bitmap_clone(lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; meter = copp_meter_get(COPP_REJECT, od->nbs->copp, @@ -8248,7 +8302,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( - lflows, dp_non_meter ? dp_non_meter : lb->nb_ls_map, + lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -9463,7 +9517,8 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, + const struct hmap *ls_ports, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -9478,7 +9533,14 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, for (size_t j = 0; j < lb_vip_nb->n_backends; j++) { struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - if (!backend_nb->op || !backend_nb->svc_mon_src_ip) { + + if (!backend_nb->health_check) { + continue; + } + + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + if (!op || !backend_nb->svc_mon_src_ip) { continue; } @@ -9520,7 +9582,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, svc_monitor_mac); } ovn_lflow_add_with_hint(lflows, - backend_nb->op->od, + op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_); @@ -11245,7 +11307,7 @@ struct lrouter_nat_lb_flows_ctx { struct ds *gw_redir_action; struct ovn_lb_vip *lb_vip; - struct ovn_northd_lb *lb; + const struct ovn_northd_lb *lb; bool reject; int prio; @@ -11377,14 +11439,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, static void build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb *lb, + struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; const char *ip_match = ipv4 ? "ip4" : "ip6"; @@ -11399,9 +11463,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_clear(match); ds_clear(action); - bool reject = build_lb_vip_actions(lb_vip, vips_nb, action, + bool reject = build_lb_vip_actions(lb, lb_vip, vips_nb, action, lb->selection_fields, &skip_snat_act, - &force_snat_act, false, features); + &force_snat_act, false, features, + svc_monitor_map); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows. @@ -11476,7 +11541,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * lflow generation for them. */ size_t index; - BITMAP_FOR_EACH_1 (index, bitmap_len, lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, bitmap_len, lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; @@ -11556,16 +11621,19 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } static void -build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -11575,7 +11643,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } size_t index; - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, @@ -11599,10 +11667,10 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * a higher priority rule for load balancing below also commits the * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ - build_lb_rules_pre_stateful(lflows, lb, features->ct_no_masked_label, + build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, ls_datapaths, match, action); - build_lb_rules(lflows, lb, ls_datapaths, features, match, action, - meter_groups); + build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, + meter_groups, svc_monitor_map); } /* If there are any load balancing rules, we should send the packet to @@ -11614,17 +11682,17 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * defragmentation to match on L4 ports. */ static void -build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, +build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } - for (size_t i = 0; i < lb->n_vips; i++) { - struct ovn_lb_vip *lb_vip = &lb->vips[i]; + for (size_t i = 0; i < lb_dps->lb->n_vips; i++) { + struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i]; bool ipv6 = lb_vip->address_family == AF_INET6; int prio = 100; @@ -11633,36 +11701,41 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); ovn_lflow_add_with_dp_group( - lflows, lb->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, - prio, ds_cstr(match), "ct_dnat;", &lb->nlb->header_); + lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), + S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", + &lb_dps->lb->nlb->header_); } } static void -build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { size_t index; - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; - build_lrouter_nat_flows_for_lb(lb_vip, lb, &lb->vips_nb[i], + build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lflows, match, action, - meter_groups, features); + meter_groups, features, + svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, S_ROUTER_IN_DNAT, @@ -11676,7 +11749,7 @@ build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } if (lb->skip_snat) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, @@ -15393,7 +15466,8 @@ struct lswitch_flow_build_info { struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_dps_map; + const struct hmap *svc_monitor_map; const struct hmap *bfd_connections; const struct chassis_features *features; char *svc_check_match; @@ -15533,7 +15607,7 @@ build_lflows_thread(void *arg) struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; int bnum; @@ -15600,28 +15674,33 @@ build_lflows_thread(void *arg) } } for (bnum = control->id; - bnum <= lsi->lbs->mask; + bnum <= lsi->lb_dps_map->mask; bnum += control->pool->size) { - HMAP_FOR_EACH_IN_PARALLEL (lb, hmap_node, bnum, lsi->lbs) { + HMAP_FOR_EACH_IN_PARALLEL (lb_dps, hmap_node, bnum, + lsi->lb_dps_map) { if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb, lsi->lflows, + build_lswitch_arp_nd_service_monitor(lb_dps->lb, + lsi->ls_ports, + lsi->lflows, &lsi->match, &lsi->actions); - build_lrouter_defrag_flows_for_lb(lb, lsi->lflows, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match); - build_lrouter_flows_for_lb(lb, lsi->lflows, + build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); - build_lswitch_flows_for_lb(lb, lsi->lflows, + build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); } } @@ -15687,7 +15766,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, - const struct hmap *lbs, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, const struct chassis_features *features) { @@ -15714,7 +15794,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].port_groups = port_groups; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; - lsiv[index].lbs = lbs; + lsiv[index].lb_dps_map = lb_dps_map; + lsiv[index].svc_monitor_map = svc_monitor_map; lsiv[index].bfd_connections = bfd_connections; lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; @@ -15737,7 +15818,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } else { struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, @@ -15748,7 +15829,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, - .lbs = lbs, + .lb_dps_map = lb_dps_map, + .svc_monitor_map = svc_monitor_map, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, @@ -15780,17 +15862,19 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); - HMAP_FOR_EACH (lb, hmap_node, lbs) { - build_lswitch_arp_nd_service_monitor(lb, lsi.lflows, - &lsi.actions, + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.lflows, &lsi.actions, &lsi.match); - build_lrouter_defrag_flows_for_lb(lb, lsi.lflows, lsi.lr_datapaths, - &lsi.match); - build_lrouter_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, + lsi.lr_datapaths, &lsi.match); + build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); - build_lswitch_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15890,7 +15974,9 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->port_groups, lflows, &igmp_groups, - input_data->meter_groups, input_data->lbs, + input_data->meter_groups, + input_data->lb_datapaths_map, + input_data->svc_monitor_map, input_data->bfd_connections, input_data->features); @@ -17215,8 +17301,8 @@ northd_init(struct northd_data *data) hmap_init(&data->lr_ports); hmap_init(&data->port_groups); shash_init(&data->meter_groups); - hmap_init(&data->lbs); - hmap_init(&data->lb_groups); + hmap_init(&data->lb_datapaths_map); + hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); data->features = (struct chassis_features) { .ct_no_masked_label = true, @@ -17226,6 +17312,7 @@ northd_init(struct northd_data *data) }; data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); + hmap_init(&data->svc_monitor_map); data->change_tracked = false; ovs_list_init(&data->tracked_ls_changes.updated); } @@ -17233,17 +17320,18 @@ northd_init(struct northd_data *data) void northd_destroy(struct northd_data *data) { - struct ovn_northd_lb *lb; - HMAP_FOR_EACH_POP (lb, hmap_node, &data->lbs) { - ovn_northd_lb_destroy(lb); + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH_POP (lb_dps, hmap_node, &data->lb_datapaths_map) { + ovn_lb_datapaths_destroy(lb_dps); } - hmap_destroy(&data->lbs); + hmap_destroy(&data->lb_datapaths_map); - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH_POP (lb_group, hmap_node, &data->lb_groups) { - ovn_lb_group_destroy(lb_group); + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH_POP (lb_group_dps, hmap_node, + &data->lb_group_datapaths_map) { + ovn_lb_group_datapaths_destroy(lb_group_dps); } - hmap_destroy(&data->lb_groups); + hmap_destroy(&data->lb_group_datapaths_map); struct ovn_port_group *pg; HMAP_FOR_EACH_SAFE (pg, key_node, &data->port_groups) { @@ -17258,6 +17346,12 @@ northd_destroy(struct northd_data *data) } shash_destroy(&data->meter_groups); + struct service_monitor_info *mon_info; + HMAP_FOR_EACH_POP (mon_info, hmap_node, &data->svc_monitor_map) { + free(mon_info); + } + hmap_destroy(&data->svc_monitor_map); + /* XXX Having to explicitly clean up macam here * is a bit strange. We don't explicitly initialize * macam in this module, but this is the logical place @@ -17366,10 +17460,9 @@ ovnnb_db_run(struct northd_input *input_data, input_data->sbrec_chassis_table, &data->ls_datapaths, &data->lr_datapaths, &data->lr_list); - build_lbs(input_data->nbrec_load_balancer_table, - input_data->nbrec_load_balancer_group_table, - &data->ls_datapaths, &data->lr_datapaths, &data->lbs, - &data->lb_groups); + build_lb_datapaths(input_data->lbs, input_data->lb_groups, + &data->ls_datapaths, &data->lr_datapaths, + &data->lb_datapaths_map, &data->lb_group_datapaths_map); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, @@ -17384,9 +17477,11 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, &data->lr_datapaths, &data->ls_ports, - &data->lbs, &data->lb_groups, - &data->svc_monitor_lsps); - build_lb_count_dps(&data->lbs, + &data->lb_datapaths_map, + &data->lb_group_datapaths_map, + &data->svc_monitor_lsps, + &data->svc_monitor_map); + build_lb_count_dps(&data->lb_datapaths_map, ods_size(&data->ls_datapaths), ods_size(&data->lr_datapaths)); build_ipam(&data->ls_datapaths.datapaths, &data->ls_ports); diff --git a/northd/northd.h b/northd/northd.h index 48c282476a..7d92028c7d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -28,9 +28,6 @@ struct northd_input { const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; - const struct nbrec_load_balancer_table *nbrec_load_balancer_table; - const struct nbrec_load_balancer_group_table - *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; @@ -59,6 +56,10 @@ struct northd_input { *sbrec_chassis_template_var_table; const struct sbrec_mirror_table *sbrec_mirror_table; + /* Northd lb data node inputs*/ + const struct hmap *lbs; + const struct hmap *lb_groups; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -110,12 +111,13 @@ struct northd_data { struct hmap lr_ports; struct hmap port_groups; struct shash meter_groups; - struct hmap lbs; - struct hmap lb_groups; + struct hmap lb_datapaths_map; + struct hmap lb_group_datapaths_map; struct ovs_list lr_list; bool ovn_internal_version_changed; struct chassis_features features; struct sset svc_monitor_lsps; + struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; }; @@ -146,9 +148,10 @@ struct lflow_input { const struct hmap *lr_ports; const struct hmap *port_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; const struct chassis_features *features; + const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; }; From patchwork Sat Jun 24 06:48:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799369 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4SX0xKFz20Xr for ; Sat, 24 Jun 2023 16:48:27 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id D9A734064D; Sat, 24 Jun 2023 06:48:25 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D9A734064D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PwXbLOqo5nx8; Sat, 24 Jun 2023 06:48:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 08B1F4015F; Sat, 24 Jun 2023 06:48:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 08B1F4015F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D3F6FC0037; Sat, 24 Jun 2023 06:48:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 739F7C0029 for ; Sat, 24 Jun 2023 06:48:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A5A574046F for ; Sat, 24 Jun 2023 06:48:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A5A574046F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QfyOPfIsMhbQ for ; Sat, 24 Jun 2023 06:48:12 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6674640620 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6674640620 for ; Sat, 24 Jun 2023 06:48:12 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 17B911BF203; Sat, 24 Jun 2023 06:48:07 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:18:04 +0530 Message-Id: <20230624064804.12060-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 3/8] northd: Add initial I-P for load balancer and load balancer groups X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to load balancers and load balancer groups are handled incrementally in the newly added 'northd_lb_data' engine node. 'northd_lb_data' is input to 'northd' node and the handler - northd_northd_lb_data_handler in 'northd' node handles the changes. If a load balancer or load balancer group is associated to a logical switch or router then 'northd' node falls back to full recompute. Signed-off-by: Numan Siddique --- lib/lb.c | 85 ++++++++++++---- lib/lb.h | 9 ++ northd/en-northd-lb-data.c | 202 +++++++++++++++++++++++++++++++++++-- northd/en-northd-lb-data.h | 37 +++++++ northd/en-northd.c | 24 +++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 13 ++- northd/northd.c | 77 ++++++++++++++ northd/northd.h | 7 ++ tests/ovn-northd.at | 123 ++++++++++++++++++++++ 10 files changed, 545 insertions(+), 33 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index 429dbf15af..a51fe225fa 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -606,13 +606,13 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, return NULL; } -struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +static void +ovn_northd_lb_init(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); bool is_sctp = nullable_string_is_equal(nbrec_lb->protocol, "sctp"); - struct ovn_northd_lb *lb = xzalloc(sizeof *lb); int address_family = !strcmp(smap_get_def(&nbrec_lb->options, "address-family", "ipv4"), "ipv4") @@ -668,6 +668,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) "reject", false); ovn_northd_lb_vip_init(lb_vip_nb, lb_vip, nbrec_lb, node->key, node->value, template); + if (lb_vip_nb->lb_health_check) { + lb->health_checks = true; + } + if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { sset_add(&lb->ips_v4, lb_vip->vip_str); } else { @@ -713,6 +717,13 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) ds_chomp(&sel_fields, ','); lb->selection_fields = ds_steal_cstr(&sel_fields); } +} + +struct ovn_northd_lb * +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +{ + struct ovn_northd_lb *lb = xzalloc(sizeof *lb); + ovn_northd_lb_init(lb, nbrec_lb); return lb; } @@ -738,8 +749,8 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +static void +ovn_northd_lb_cleanup(struct ovn_northd_lb *lb) { for (size_t i = 0; i < lb->n_vips; i++) { ovn_lb_vip_destroy(&lb->vips[i]); @@ -747,26 +758,36 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) } free(lb->vips); free(lb->vips_nb); + lb->vips = NULL; + lb->vips_nb = NULL; smap_destroy(&lb->template_vips); sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); + lb->selection_fields = NULL; + lb->health_checks = false; +} + +void +ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +{ + ovn_northd_lb_cleanup(lb); free(lb); } -/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will - * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' - * logical routers to which this LB Group is applied. Can be filled later - * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ -struct ovn_lb_group * -ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs) +void +ovn_northd_lb_reinit(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { - struct ovn_lb_group *lb_group; + ovn_northd_lb_cleanup(lb); + ovn_northd_lb_init(lb, nbrec_lb); +} - lb_group = xzalloc(sizeof *lb_group); - lb_group->uuid = nbrec_lb_group->header_.uuid; +static void +ovn_lb_group_init(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); lb_group->lb_ips = ovn_lb_ip_set_create(); @@ -776,10 +797,30 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); } +} +/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record + * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will + * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' + * logical routers to which this LB Group is applied. Can be filled later + * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ +struct ovn_lb_group * +ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + struct ovn_lb_group *lb_group = xzalloc(sizeof *lb_group); + lb_group->uuid = nbrec_lb_group->header_.uuid; + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); return lb_group; } +static void +ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) +{ + ovn_lb_ip_set_destroy(lb_group->lb_ips); + free(lb_group->lbs); +} + void ovn_lb_group_destroy(struct ovn_lb_group *lb_group) { @@ -787,11 +828,19 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) return; } - ovn_lb_ip_set_destroy(lb_group->lb_ips); - free(lb_group->lbs); + ovn_lb_group_cleanup(lb_group); free(lb_group); } +void +ovn_lb_group_reinit(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + ovn_lb_group_cleanup(lb_group); + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); +} + struct ovn_lb_group * ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *uuid) { diff --git a/lib/lb.h b/lib/lb.h index 0339050cba..74905c73b7 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -77,6 +77,9 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; + + /* Indicates if the load balancer has health checks configured. */ + bool health_checks; }; struct ovn_lb_vip { @@ -130,6 +133,8 @@ struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); +void ovn_northd_lb_reinit(struct ovn_northd_lb *, + const struct nbrec_load_balancer *); void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); @@ -148,6 +153,10 @@ struct ovn_lb_group *ovn_lb_group_create( void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *); +void ovn_lb_group_reinit( + struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); struct ovn_lb_datapaths { struct hmap_node hmap_node; diff --git a/northd/en-northd-lb-data.c b/northd/en-northd-lb-data.c index d46c3c27ed..e52535b4a9 100644 --- a/northd/en-northd-lb-data.c +++ b/northd/en-northd-lb-data.c @@ -37,6 +37,15 @@ static void northd_lb_data_destroy(struct northd_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static struct ovn_lb_group *create_lb_group( + const struct nbrec_load_balancer_group *, struct hmap *lbs, + struct hmap *lb_groups); +static void destroy_tracked_data(struct northd_lb_data *); +static void add_lb_to_tracked_data(struct ovn_northd_lb *, + struct ovs_list *tracked_list, + bool health_checks); +static void add_lb_group_to_tracked_data(struct ovn_lb_group *, + struct ovs_list *tracked_list); void * en_northd_lb_data_init(struct engine_node *node OVS_UNUSED, @@ -61,6 +70,7 @@ en_northd_lb_data_run(struct engine_node *node, void *data) const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); engine_set_node_state(node, EN_UPDATED); } @@ -72,12 +82,122 @@ en_northd_lb_data_cleanup(void *data) northd_lb_data_destroy(lb_data); } +void +en_northd_lb_data_clear_tracked_data(void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + destroy_tracked_data(lb_data); +} + + +/* Handler functions. */ +bool +northd_lb_data_load_balancer_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + + const struct nbrec_load_balancer *tracked_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH_TRACKED (tracked_lb, nb_lb_table) { + struct ovn_northd_lb *lb; + if (nbrec_load_balancer_is_new(tracked_lb)) { + /* New load balancer. */ + lb = ovn_northd_lb_create(tracked_lb); + hmap_insert(&lb_data->lbs, &lb->hmap_node, + uuid_hash(&tracked_lb->header_.uuid)); + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_updated_lbs.updated, + lb->health_checks); + } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + hmap_remove(&lb_data->lbs, &lb->hmap_node); + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_deleted_lbs.updated, + lb->health_checks); + } else { + /* Load balancer updated. */ + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + bool health_checks = lb->health_checks; + ovn_northd_lb_reinit(lb, tracked_lb); + health_checks |= lb->health_checks; + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_updated_lbs.updated, health_checks); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +northd_lb_data_load_balancer_group_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct nbrec_load_balancer_group *tracked_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH_TRACKED (tracked_lb_group, + nb_lbg_table) { + if (nbrec_load_balancer_group_is_new(tracked_lb_group)) { + struct ovn_lb_group *lb_group = + create_lb_group(tracked_lb_group, &lb_data->lbs, + &lb_data->lb_groups); + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_updated_lb_groups.updated); + } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_deleted_lb_groups.updated); + } else if (nbrec_load_balancer_group_is_updated(tracked_lb_group, + NBREC_LOAD_BALANCER_GROUP_COL_LOAD_BALANCER)) { + + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + ovn_lb_group_reinit(lb_group, tracked_lb_group, &lb_data->lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_updated_lb_groups.updated); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + /* static functions. */ static void northd_lb_data_init(struct northd_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + ovs_list_init(&trk_lb_data->tracked_updated_lbs.updated); + ovs_list_init(&trk_lb_data->tracked_deleted_lbs.updated); + ovs_list_init(&trk_lb_data->tracked_updated_lb_groups.updated); + ovs_list_init(&trk_lb_data->tracked_deleted_lb_groups.updated); } static void @@ -94,6 +214,8 @@ northd_lb_data_destroy(struct northd_lb_data *lb_data) ovn_lb_group_destroy(lb_group); } hmap_destroy(&lb_data->lb_groups); + + destroy_tracked_data(lb_data); } static void @@ -101,12 +223,9 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, struct hmap *lbs, struct hmap *lb_groups) { - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb_nb; - const struct nbrec_load_balancer *nbrec_lb; NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - lb_nb = ovn_northd_lb_create(nbrec_lb); + struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb); hmap_insert(lbs, &lb_nb->hmap_node, uuid_hash(&nbrec_lb->header_.uuid)); } @@ -114,13 +233,76 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group *nbrec_lb_group; NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + create_lb_group(nbrec_lb_group, lbs, lb_groups); + } +} - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } +static struct ovn_lb_group * +create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + + return lb_group; +} + +static void +destroy_tracked_data(struct northd_lb_data *lb_data) +{ + lb_data->tracked = false; + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct tracked_lb *tracked_lb; + LIST_FOR_EACH_SAFE (tracked_lb, list_node, + &trk_lb_data->tracked_updated_lbs.updated) { + ovs_list_remove(&tracked_lb->list_node); + free(tracked_lb); + } + + LIST_FOR_EACH_SAFE (tracked_lb, list_node, + &trk_lb_data->tracked_deleted_lbs.updated) { + ovs_list_remove(&tracked_lb->list_node); + ovn_northd_lb_destroy(tracked_lb->lb); + free(tracked_lb); + } + + struct tracked_lb_group *tracked_lb_group; + LIST_FOR_EACH_SAFE (tracked_lb_group, list_node, + &trk_lb_data->tracked_updated_lb_groups.updated) { + ovs_list_remove(&tracked_lb_group->list_node); + free(tracked_lb_group); + } - hmap_insert(lb_groups, &lb_group->hmap_node, - uuid_hash(&lb_group->uuid)); + LIST_FOR_EACH_SAFE (tracked_lb_group, list_node, + &trk_lb_data->tracked_deleted_lb_groups.updated) { + ovs_list_remove(&tracked_lb_group->list_node); + ovn_lb_group_destroy(tracked_lb_group->lb_group); + free(tracked_lb_group); } } + +static void +add_lb_to_tracked_data(struct ovn_northd_lb *lb, struct ovs_list *tracked_list, + bool health_checks) +{ + struct tracked_lb *t_lb = xzalloc(sizeof *t_lb); + t_lb->lb = lb; + t_lb->health_checks = health_checks; + ovs_list_push_back(tracked_list, &t_lb->list_node); +} + +static void +add_lb_group_to_tracked_data(struct ovn_lb_group *lb_group, + struct ovs_list *tracked_list) +{ + struct tracked_lb_group *t_lb_group = xzalloc(sizeof *t_lb_group); + t_lb_group->lb_group = lb_group; + ovs_list_push_back(tracked_list, &t_lb_group->list_node); +} diff --git a/northd/en-northd-lb-data.h b/northd/en-northd-lb-data.h index eb297e376d..bb07fcf686 100644 --- a/northd/en-northd-lb-data.h +++ b/northd/en-northd-lb-data.h @@ -4,16 +4,53 @@ #include #include "openvswitch/hmap.h" +#include "include/openvswitch/list.h" #include "lib/inc-proc-eng.h" +struct ovn_northd_lb; +struct ovn_lb_group; + +struct tracked_lb { + struct ovs_list list_node; + struct ovn_northd_lb *lb; + bool health_checks; +}; + +struct tracked_lb_group { + struct ovs_list list_node; + struct ovn_lb_group *lb_group; +}; + +struct tracked_lb_changes { + struct ovs_list updated; /* contains list of 'struct tracked_lb ' or + list of 'struct tracked_lb_group' */ +}; + +struct tracked_lb_data { + struct tracked_lb_changes tracked_updated_lbs; + struct tracked_lb_changes tracked_deleted_lbs; + struct tracked_lb_changes tracked_updated_lb_groups; + struct tracked_lb_changes tracked_deleted_lb_groups; +}; + struct northd_lb_data { struct hmap lbs; struct hmap lb_groups; + + /* tracked data*/ + bool tracked; + struct tracked_lb_data tracked_lb_data; }; void *en_northd_lb_data_init(struct engine_node *, struct engine_arg *); void en_northd_lb_data_run(struct engine_node *, void *data); void en_northd_lb_data_cleanup(void *data); +void en_northd_lb_data_clear_tracked_data(void *data); + +bool northd_lb_data_load_balancer_handler(struct engine_node *, + void *data); +bool northd_lb_data_load_balancer_group_handler(struct engine_node *, + void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index cc7d838451..b3c03c54bd 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,30 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_northd_lb_data_handler(struct engine_node *node, void *data) +{ + struct northd_lb_data *lb_data = + engine_get_input_data("northd_lb_data", node); + + if (!lb_data->tracked) { + return false; + } + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 20cc77f108..5674f4390c 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,5 +17,6 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); +bool northd_northd_lb_data_handler(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 51f9609b25..546799b9e1 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,13 +141,18 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); -static ENGINE_NODE(northd_lb_data, "northd_lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) { /* Define relationships between nodes where first argument is dependent * on the second argument */ + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, + northd_lb_data_load_balancer_handler); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, + northd_lb_data_load_balancer_group_handler); + engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); @@ -171,6 +176,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + engine_add_input(&en_northd, &en_northd_lb_data, + northd_northd_lb_data_handler); engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, @@ -178,10 +185,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); - engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, NULL); - engine_add_input(&en_northd, &en_northd_lb_data, NULL); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 9061b37dcb..22f6fabff1 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/lb.h" #include "memory.h" #include "northd.h" +#include "en-northd-lb-data.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -5267,6 +5268,82 @@ northd_handle_sb_port_binding_changes( return true; } +bool +northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) +{ + struct tracked_lb *trk_lb; + + struct ovn_lb_datapaths *lb_dps; + LIST_FOR_EACH (trk_lb, list_node, + &trk_lb_data->tracked_deleted_lbs.updated) { + if (trk_lb->health_checks) { + /* Fall back to recompute if the deleted load balancer has + * health checks configured. */ + return false; + } + + const struct uuid *lb_uuid = + &trk_lb->lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); + ovn_lb_datapaths_destroy(lb_dps); + } + + LIST_FOR_EACH (trk_lb, list_node, + &trk_lb_data->tracked_updated_lbs.updated) { + if (trk_lb->health_checks) { + /* Fall back to recompute if the created/updated load balancer has + * health checks configured. */ + return false; + } + + const struct uuid *lb_uuid = + &trk_lb->lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (!lb_dps) { + lb_dps = ovn_lb_datapaths_create(trk_lb->lb, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + struct tracked_lb_group *trk_lb_group; + LIST_FOR_EACH_SAFE (trk_lb_group, list_node, + &trk_lb_data->tracked_deleted_lb_groups.updated) { + const struct uuid *lb_uuid = &trk_lb_group->lb_group->uuid; + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + ovs_assert(lb_group_dps); + hmap_remove(lb_group_datapaths_map, &lb_group_dps->hmap_node); + ovn_lb_group_datapaths_destroy(lb_group_dps); + } + + LIST_FOR_EACH_SAFE (trk_lb_group, list_node, + &trk_lb_data->tracked_updated_lb_groups.updated) { + const struct uuid *lb_uuid = &trk_lb_group->lb_group->uuid; + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + if (!lb_group_dps) { + lb_group_dps = ovn_lb_group_datapaths_create( + trk_lb_group->lb_group, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ diff --git a/northd/northd.h b/northd/northd.h index 7d92028c7d..7d17921fa2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -347,6 +347,13 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); +struct tracked_lb_data; +bool northd_handle_lb_data_changes(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); + void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, const struct sbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index ada2d2a4aa..9c78ac8eaf 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9608,3 +9608,126 @@ AT_CHECK([grep "lr_in_gw_redirect" R1flows |sed s'/table=../table=??/' |sort], [ AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing]) +ovn_start + +check_engine_stats() { + northd_comp=$1 + lflow_comp=$2 + + AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd_lb_data recompute], [0], [0 +]) + + northd_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) + if [[ "$northd_comp" == "norecompute" ]]; then + check test "$northd_recompute_ct" -eq "0" + else + check test "$northd_recompute_ct" -ne "0" + fi + + lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) + if [[ "$lflow_comp" == "norecompute" ]]; then + check test "$lflow_recompute_ct" -eq "0" + else + check test "$lflow_recompute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb set load_balancer . options:foo=bar +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 +check_engine_stats norecompute recompute + +AT_CHECK([ovn-nbctl --wait=sb \ + -- --id=@hc create Load_Balancer_Health_Check vip="10.0.0.10\:80" \ + options:failure_count=100 \ + -- add Load_Balancer . health_check @hc | uuidfilt], [0], [<0> +]) +check_engine_stats recompute recompute + +# Any change to load balancer health check should also result in full recompute +# of northd node (but not northd_lb_data node) +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer_health_check . options:foo=bar1 +check_engine_stats recompute recompute + +# Delete the health check from the load balancer. northd engine node should do a full recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear Load_Balancer . health_check +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl ls-add sw0 +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats recompute recompute + +# Associate lb1 to sw0. There should be a full recompute of northd engine node +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats recompute recompute + +# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats recompute recompute + +# Test load balancer group now +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats norecompute recompute + +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) + +# Add lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear load_balancer_group . load_Balancer +check_engine_stats norecompute recompute + +# Add back lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats recompute recompute + +# Update lb and this should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer . options:bar=foo +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_router lr0 load_balancer_group +check_engine_stats recompute recompute + +AT_CLEANUP +]) From patchwork Sat Jun 24 06:48:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799370 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4Sr6lvVz20Xr for ; Sat, 24 Jun 2023 16:48:44 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id D365640A9B; Sat, 24 Jun 2023 06:48:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D365640A9B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xwd5iQgsnmAv; Sat, 24 Jun 2023 06:48:41 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 53EA940948; Sat, 24 Jun 2023 06:48:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 53EA940948 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 06554C0037; Sat, 24 Jun 2023 06:48:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 16490C0037 for ; Sat, 24 Jun 2023 06:48:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3A2EF6117D for ; Sat, 24 Jun 2023 06:48:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 3A2EF6117D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8_1GepxJryjK for ; Sat, 24 Jun 2023 06:48:24 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 113316111B Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp3.osuosl.org (Postfix) with ESMTPS id 113316111B for ; Sat, 24 Jun 2023 06:48:23 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 65C3860003; Sat, 24 Jun 2023 06:48:19 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:18:15 +0530 Message-Id: <20230624064815.12090-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 4/8] northd: Refactor the 'northd' node code which handles logical switch changes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This will help in handling other column changes of a logical switch. Signed-off-by: Numan Siddique --- northd/northd.c | 245 ++++++++++++++++++++++++++++-------------------- 1 file changed, 144 insertions(+), 101 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 22f6fabff1..572e8fc849 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4976,8 +4976,14 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, return op; } +/* Returns true if the logical switch has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - logical switch ports. + */ static bool -check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) +check_unsupported_inc_proc_for_ls_changes( + const struct nbrec_logical_switch *ls) { /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; @@ -5071,6 +5077,138 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) return false; } +static bool +ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od) +{ + bool ls_ports_changed = false; + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_PORTS)) { + + for (size_t i = 0; i < changed_ls->n_ports; i++) { + if (nbrec_logical_switch_port_row_get_seqno( + changed_ls->ports[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_ports_changed = true; + break; + } + } + } else { + ls_ports_changed = true; + } + + if (!ls_ports_changed) { + return true; + } + + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + op->visited = false; + } + + /* Compare the individual ports in the old and new Logical Switches */ + for (size_t j = 0; j < changed_ls->n_ports; ++j) { + struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; + op = ovn_port_find_in_datapath(od, new_nbsp->name); + + if (!op) { + if (!lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, NULL, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->added_ports, + &op->list); + } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { + /* Existing port updated */ + bool temp = false; + if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || + !op->lsp_can_be_inc_processed || + !lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + const struct sbrec_port_binding *sb = op->sb; + if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { + /* This port is used for svc monitor, which may be impacted + * by this change. Fallback to recompute. */ + goto fail; + } + if (!check_lsp_is_up && + !check_lsp_changes_other_than_up(new_nbsp)) { + /* If the only change is the "up" column while the + * "ignore_lsp_down" is set to true, just ignore this + * change. */ + op->visited = true; + continue; + } + ovn_port_destroy(&nd->ls_ports, op); + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, sb, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->updated_ports, &op->list); + } + op->visited = true; + } + + /* Check for deleted ports */ + HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { + if (!op->visited) { + if (!op->lsp_can_be_inc_processed) { + goto fail; + } + if (sset_contains(&nd->svc_monitor_lsps, op->key)) { + /* This port was used for svc monitor, which may be + * impacted by this deletion. Fallback to recompute. */ + goto fail; + } + ovs_list_push_back(&ls_change->deleted_ports, + &op->list); + hmap_remove(&nd->ls_ports, &op->key_node); + hmap_remove(&od->ports, &op->dp_node); + sbrec_port_binding_delete(op->sb); + delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, + op->tunnel_key); + } + } + + if (!ovs_list_is_empty(&ls_change->added_ports) || + !ovs_list_is_empty(&ls_change->updated_ports) || + !ovs_list_is_empty(&ls_change->deleted_ports)) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } + + return true; + +fail: + free(ls_change); + return false; +} + + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5081,11 +5219,10 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { - ls_change = NULL; if (nbrec_logical_switch_is_new(changed_ls) || nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; @@ -5102,106 +5239,13 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } /* Now only able to handle lsp changes. */ - if (check_ls_changes_other_than_lsp(changed_ls)) { + if (check_unsupported_inc_proc_for_ls_changes(changed_ls)) { goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - - struct ovn_port *op; - HMAP_FOR_EACH (op, dp_node, &od->ports) { - op->visited = false; - } - - /* Compare the individual ports in the old and new Logical Switches */ - for (size_t j = 0; j < changed_ls->n_ports; ++j) { - struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; - op = ovn_port_find_in_datapath(od, new_nbsp->name); - - if (!op) { - if (!lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->added_ports, - &op->list); - } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { - /* Existing port updated */ - bool temp = false; - if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || - !op->lsp_can_be_inc_processed || - !lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - const struct sbrec_port_binding *sb = op->sb; - if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { - /* This port is used for svc monitor, which may be impacted - * by this change. Fallback to recompute. */ - goto fail; - } - if (!check_lsp_is_up && - !check_lsp_changes_other_than_up(new_nbsp)) { - /* If the only change is the "up" column while the - * "ignore_lsp_down" is set to true, just ignore this - * change. */ - op->visited = true; - continue; - } - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->updated_ports, &op->list); - } - op->visited = true; - } - - /* Check for deleted ports */ - HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { - if (!op->visited) { - if (!op->lsp_can_be_inc_processed) { - goto fail; - } - if (sset_contains(&nd->svc_monitor_lsps, op->key)) { - /* This port was used for svc monitor, which may be - * impacted by this deletion. Fallback to recompute. */ - goto fail; - } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); - hmap_remove(&nd->ls_ports, &op->key_node); - hmap_remove(&od->ports, &op->dp_node); - sbrec_port_binding_delete(op->sb); - delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, - op->tunnel_key); - } - } - - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + if (!ls_check_and_handle_lsp_changes(ovnsb_idl_txn, changed_ls, + ni, nd, od)) { + goto fail; } } @@ -5211,7 +5255,6 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; fail: - free(ls_change); destroy_northd_data_tracked_changes(nd); return false; } From patchwork Sat Jun 24 06:48:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799371 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4TK4sJRz20Xr for ; Sat, 24 Jun 2023 16:49:09 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6DA15610BE; Sat, 24 Jun 2023 06:49:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6DA15610BE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rg5a3e00GIxz; Sat, 24 Jun 2023 06:49:06 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1C02360F63; Sat, 24 Jun 2023 06:49:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1C02360F63 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E6E47C0037; Sat, 24 Jun 2023 06:49:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 71EDDC0029 for ; Sat, 24 Jun 2023 06:49:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A815740A0E for ; Sat, 24 Jun 2023 06:48:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A815740A0E X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GOOkm8tiDh09 for ; Sat, 24 Jun 2023 06:48:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2296E40948 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2296E40948 for ; Sat, 24 Jun 2023 06:48:36 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 2E3961C000C; Sat, 24 Jun 2023 06:48:33 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:18:30 +0530 Message-Id: <20230624064830.12126-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 5/8] northd: Handle load balancer changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Logical switch change handler of the 'northd' engine node now also handles changes to load balancers. Signed-off-by: Numan Siddique --- lib/lb.c | 17 +++- lib/lb.h | 2 + northd/northd.c | 211 +++++++++++++++++++++++++++++++++----------- northd/northd.h | 9 ++ tests/ovn-northd.at | 8 +- 5 files changed, 190 insertions(+), 57 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index a51fe225fa..b2b6473c1d 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1091,7 +1091,22 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } + } +} + +void +ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set0(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls--; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..ac33333a7f 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -179,6 +179,8 @@ void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); struct ovn_lb_group_datapaths { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index 572e8fc849..8309ca52e4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -853,7 +853,8 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) ovn_ls_port_group_destroy(&od->nb_pgs); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); - + free(od->lb_uuids); + free(od->lb_group_uuids); free(od); } } @@ -4036,6 +4037,48 @@ build_lb_vip_actions(const struct ovn_northd_lb *lb, return reject; } +static void +associate_ls_lbs(struct ovn_datapath *ls_od, struct hmap *lb_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + + free(ls_od->lb_uuids); + ls_od->lb_uuids = xcalloc(ls_od->nbs->n_load_balancer, + sizeof *ls_od->lb_uuids); + ls_od->n_lb_uuids = ls_od->nbs->n_load_balancer; + for (size_t i = 0; i < ls_od->nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &ls_od->nbs->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &ls_od); + ls_od->lb_uuids[i] = *lb_uuid; + } +} + +static void +associate_ls_lb_groups(struct ovn_datapath *ls_od, + struct hmap *lb_group_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + struct ovn_lb_group_datapaths *lb_group_dps; + + free(ls_od->lb_group_uuids); + ls_od->lb_group_uuids = xcalloc(ls_od->nbs->n_load_balancer_group, + sizeof *ls_od->lb_group_uuids); + ls_od->n_lb_group_uuids = ls_od->nbs->n_load_balancer_group; + for (size_t i = 0; i < ls_od->nbs->n_load_balancer_group; i++) { + nbrec_lb_group = ls_od->nbs->load_balancer_group[i]; + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &ls_od); + ls_od->lb_group_uuids[i] = *lb_group_uuid; + } +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, @@ -4071,24 +4114,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, if (!od->nbs) { continue; } - - for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &od->nbs->load_balancer[i]->header_.uuid; - lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); - ovs_assert(lb_dps); - ovn_lb_datapaths_add_ls(lb_dps, 1, &od); - } - - for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { - nbrec_lb_group = od->nbs->load_balancer_group[i]; - const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_datapaths_map, - lb_group_uuid); - ovs_assert(lb_group_dps); - ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); - } + associate_ls_lbs(od, lb_datapaths_map); + associate_ls_lb_groups(od, lb_group_datapaths_map); } HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { @@ -4847,23 +4874,29 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, sset_destroy(&active_ha_chassis_grps); } +static void +destroy_tracked_ls_change(struct ls_change *ls_change) +{ + struct ovn_port *op; + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct ls_change *ls_change; LIST_FOR_EACH_SAFE (ls_change, list_node, &nd->tracked_ls_changes.updated) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); - } + destroy_tracked_ls_change(ls_change); ovs_list_remove(&ls_change->list_node); free(ls_change); } @@ -4980,6 +5013,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * incrementally handled. * Presently supports i-p for the below changes: * - logical switch ports. + * - load balancers. */ static bool check_unsupported_inc_proc_for_ls_changes( @@ -4988,8 +5022,11 @@ check_unsupported_inc_proc_for_ls_changes( /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { - if (nbrec_logical_switch_is_updated(ls, col) && - col != NBREC_LOGICAL_SWITCH_COL_PORTS) { + if (nbrec_logical_switch_is_updated(ls, col)) { + if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + continue; + } return true; } } @@ -5018,12 +5055,6 @@ check_unsupported_inc_proc_for_ls_changes( return true; } } - for (size_t i = 0; i < ls->n_load_balancer; i++) { - if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; - } - } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5082,7 +5113,9 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct nbrec_logical_switch *changed_ls, const struct northd_input *ni, struct northd_data *nd, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5103,12 +5136,6 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - struct ls_change *ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - struct ovn_port *op; HMAP_FOR_EACH (op, dp_node, &od->ports) { op->visited = false; @@ -5195,19 +5222,66 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!ovs_list_is_empty(&ls_change->added_ports) || !ovs_list_is_empty(&ls_change->updated_ports) || !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + *updated = true; } return true; fail: - free(ls_change); + + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } return false; } +static bool +ls_check_and_handle_lb_changes(const struct nbrec_logical_switch *changed_ls, + struct hmap *lb_datapaths_map, + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) +{ + + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)) { + for (size_t i = 0; i < changed_ls->n_load_balancer; i++) { + if (nbrec_load_balancer_row_get_seqno(changed_ls->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + break; + } + } + + return true; + } + + struct ovn_lb_datapaths *lb_dps; + for (size_t i = 0; i < od->n_lb_uuids; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &od->lb_uuids[i]); + if (lb_dps) { + ovn_lb_datapaths_remove_ls(lb_dps, 1, &od); + } + } + + associate_ls_lbs(od, lb_datapaths_map); + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set @@ -5238,20 +5312,47 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - /* Now only able to handle lsp changes. */ + /* Now only able to handle lsp, load balancerload and + * load balancer group changes. */ if (check_unsupported_inc_proc_for_ls_changes(changed_ls)) { goto fail; } + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + bool updated = false; if (!ls_check_and_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od)) { + ni, nd, od, ls_change, + &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); goto fail; } + + if (!ls_check_and_handle_lb_changes(changed_ls, + &nd->lb_datapaths_map, od, + ls_change, &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); + goto fail; + } + + if (updated) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } } if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { nd->change_tracked = true; } + return true; fail: @@ -16422,6 +16523,12 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *lflows) { struct ls_change *ls_change; + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { + if (ls_change->lbs_changed) { + return false; + } + } + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { ovs_list_init(&temp_lflow_list); add_lflow_to_temp_list = true; diff --git a/northd/northd.h b/northd/northd.h index 7d17921fa2..3afa4eaff2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -94,6 +94,7 @@ struct ls_change { struct ovs_list added_ports; struct ovs_list deleted_ports; struct ovs_list updated_ports; + bool lbs_changed; }; /* Track what's changed for logical switches. @@ -318,6 +319,14 @@ struct ovn_datapath { /* Map of ovn_port objects belonging to this datapath. * This map doesn't include derived ports. */ struct hmap ports; + + /* LB uuids associated with this datapath. */ + struct uuid *lb_uuids; + size_t n_lb_uuids; + + /* LB group uuids associated with this datapath. */ + struct uuid *lb_group_uuids; + size_t n_lb_group_uuids; }; void ovnnb_db_run(struct northd_input *input_data, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 9c78ac8eaf..f133a89263 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9677,15 +9677,15 @@ check ovn-nbctl ls-add sw0 check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats recompute recompute -# Associate lb1 to sw0. There should be a full recompute of northd engine node +# Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats recompute recompute +check_engine_stats norecompute recompute -# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +# Disassociate lb1 from sw0. There should be a no recompute of northd engine node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 -check_engine_stats recompute recompute +check_engine_stats norecompute recompute # Test load balancer group now check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats From patchwork Sat Jun 24 06:48:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799373 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4Tz3BCrz20Xr for ; Sat, 24 Jun 2023 16:49:43 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A8AC141F10; Sat, 24 Jun 2023 06:49:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A8AC141F10 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1CsQcC-_Q3cZ; Sat, 24 Jun 2023 06:49:38 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id B1D3B41FE3; Sat, 24 Jun 2023 06:49:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B1D3B41FE3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 603BBC007C; Sat, 24 Jun 2023 06:49:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 24613C007C for ; Sat, 24 Jun 2023 06:49:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6AB8E40B42 for ; Sat, 24 Jun 2023 06:48:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6AB8E40B42 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KXo8O_ygEvts for ; Sat, 24 Jun 2023 06:48:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C1CBC40A5B Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp2.osuosl.org (Postfix) with ESMTPS id C1CBC40A5B for ; Sat, 24 Jun 2023 06:48:52 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id B9877FF803; Sat, 24 Jun 2023 06:48:48 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:18:43 +0530 Message-Id: <20230624064843.12161-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 6/8] northd: Handle load balancer group changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Logical switch change handler of the 'northd' engine node now also handles changes to load balancer groups. Signed-off-by: Numan Siddique --- lib/lb.c | 45 +++++++++++++++++++++---- lib/lb.h | 33 +++++++++---------- northd/northd.c | 80 ++++++++++++++++++++++++++++++++++++--------- tests/ovn-northd.at | 6 ++-- 4 files changed, 121 insertions(+), 43 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index b2b6473c1d..a0426cc42e 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1100,7 +1100,7 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) + struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { if (bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { @@ -1112,14 +1112,14 @@ ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_lb_group_datapaths * ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, - size_t max_ls_datapaths, - size_t max_lr_datapaths) + size_t n_ls_datapaths, + size_t n_lr_datapaths) { struct ovn_lb_group_datapaths *lb_group_dps = xzalloc(sizeof *lb_group_dps); lb_group_dps->lb_group = lb_group; - lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); - lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + lb_group_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_group_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); return lb_group_dps; } @@ -1127,8 +1127,8 @@ ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) { - free(lb_group_dps->ls); - free(lb_group_dps->lr); + bitmap_free(lb_group_dps->nb_lr_map); + bitmap_free(lb_group_dps->nb_ls_map); free(lb_group_dps); } @@ -1146,3 +1146,34 @@ ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, } return NULL; } + +void +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lbg_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lbg_dps->nb_ls_map, ods[i]->index); + lbg_dps->n_nb_ls++; + } + } +} + +void +ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *lbg_dps, + size_t n, struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lbg_dps->nb_ls_map, ods[i]->index)) { + bitmap_set0(lbg_dps->nb_ls_map, ods[i]->index); + lbg_dps->n_nb_ls--; + } + } +} + +void +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + bitmap_set1(lbg_dps->nb_lr_map, lr->index); +} diff --git a/lib/lb.h b/lib/lb.h index ac33333a7f..08723e8ef3 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -19,6 +19,7 @@ #include #include +#include "lib/bitmap.h" #include "lib/smap.h" #include "openvswitch/hmap.h" #include "ovn-util.h" @@ -179,6 +180,9 @@ void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_set_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); @@ -188,10 +192,11 @@ struct ovn_lb_group_datapaths { const struct ovn_lb_group *lb_group; /* Datapaths to which 'lb_group' is applied. */ - size_t n_ls; - struct ovn_datapath **ls; - size_t n_lr; - struct ovn_datapath **lr; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; }; struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( @@ -202,21 +207,13 @@ void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( const struct hmap *lb_group_dps, const struct uuid *); -static inline void -ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, - struct ovn_datapath **ods) -{ - memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); - lbg_dps->n_ls += n; -} - -static inline void -ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, - struct ovn_datapath *lr) -{ - lbg_dps->lr[lbg_dps->n_lr++] = lr; -} +void ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *, + size_t n, struct ovn_datapath **); +void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *, + struct ovn_datapath *lr); struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index 8309ca52e4..3183057bd9 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4058,7 +4058,8 @@ associate_ls_lbs(struct ovn_datapath *ls_od, struct hmap *lb_datapaths_map) static void associate_ls_lb_groups(struct ovn_datapath *ls_od, - struct hmap *lb_group_datapaths_map) + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; struct ovn_lb_group_datapaths *lb_group_dps; @@ -4076,6 +4077,15 @@ associate_ls_lb_groups(struct ovn_datapath *ls_od, ovs_assert(lb_group_dps); ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &ls_od); ls_od->lb_group_uuids[i] = *lb_group_uuid; + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &ls_od); + } } } @@ -4115,7 +4125,7 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, continue; } associate_ls_lbs(od, lb_datapaths_map); - associate_ls_lb_groups(od, lb_group_datapaths_map); + associate_ls_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); } HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { @@ -4175,10 +4185,12 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); - ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, - lb_group_dps->ls); - ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, - lb_group_dps->lr); + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_group_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + } } } } @@ -4357,8 +4369,9 @@ build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, struct ovn_lb_group_datapaths *lb_group_dps; HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { - for (size_t i = 0; i < lb_group_dps->n_lr; i++) { - struct ovn_datapath *od = lb_group_dps->lr[i]; + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_group_dps->nb_lr_map) { + struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, od->ls_peers); for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { @@ -5024,7 +5037,8 @@ check_unsupported_inc_proc_for_ls_changes( for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || - col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP) { continue; } return true; @@ -5055,12 +5069,6 @@ check_unsupported_inc_proc_for_ls_changes( return true; } } - for (size_t i = 0; i < ls->n_load_balancer_group; i++) { - if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; - } - } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5282,6 +5290,40 @@ ls_check_and_handle_lb_changes(const struct nbrec_logical_switch *changed_ls, return true; } +static bool +ls_check_and_handle_lb_group_changes( + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map, + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) +{ + struct ovn_lb_group_datapaths *lbg_dps; + for (size_t i = 0; i < od->n_lb_group_uuids; i++) { + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &od->lb_group_uuids[i]); + if (lbg_dps) { + ovn_lb_group_datapaths_remove_ls(lbg_dps, 1, &od); + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (lb_dps) { + ovn_lb_datapaths_remove_ls(lb_dps, 1, &od); + } + } + } + } + + associate_ls_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set @@ -5341,6 +5383,14 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } + if (!ls_check_and_handle_lb_group_changes(&nd->lb_group_datapaths_map, + &nd->lb_datapaths_map, + od, ls_change, &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); + goto fail; + } + if (updated) { ovs_list_push_back(&nd->tracked_ls_changes.updated, &ls_change->list_node); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index f133a89263..da9bc16fd9 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9710,16 +9710,16 @@ check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats recompute recompute +check_engine_stats norecompute recompute # Update lb and this should result in recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid From patchwork Sat Jun 24 06:48:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799372 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4Tr556fz20Xr for ; Sat, 24 Jun 2023 16:49:36 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id CCF4A61197; Sat, 24 Jun 2023 06:49:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org CCF4A61197 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id N1bbYz7IuDjf; Sat, 24 Jun 2023 06:49:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 6224B605D5; Sat, 24 Jun 2023 06:49:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6224B605D5 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2EE2EC008C; Sat, 24 Jun 2023 06:49:32 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id C8CCEC0029 for ; Sat, 24 Jun 2023 06:49:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7D4A381F48 for ; Sat, 24 Jun 2023 06:49:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7D4A381F48 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GK0WLRcNknJx for ; Sat, 24 Jun 2023 06:49:06 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org CC76B81FC2 Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp1.osuosl.org (Postfix) with ESMTPS id CC76B81FC2 for ; Sat, 24 Jun 2023 06:49:05 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 553291BF206; Sat, 24 Jun 2023 06:49:01 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:18:58 +0530 Message-Id: <20230624064858.12198-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 7/8] northd: Sync SB Port bindings NAT column in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node 'sync_to_sb_pb' is added within 'sync_to_sb' node to sync NAT column of Port bindings table. This separation is required in order to add load balancer group I-P handling in 'northd' engine node (which is handled in the next commit). 'sync_to_sb_pb' engine node can be later expanded to sync other Port binding columns if required. Signed-off-by: Numan Siddique --- northd/en-sync-sb.c | 31 +++++ northd/en-sync-sb.h | 4 + northd/inc-proc-northd.c | 8 +- northd/northd.c | 243 +++++++++++++++++++++------------------ northd/northd.h | 2 + 5 files changed, 174 insertions(+), 114 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index fda0ca5a68..7c8be49209 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -240,6 +240,37 @@ en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) } +void * +en_sync_to_sb_pb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_pb_cleanup(void *data OVS_UNUSED) +{ + +} + +/* sync_to_sb_pb engine node functions. + * This engine node syncs the SB Port Bindings (partly) + * en_northd engine create the SB Port binding rows and + * updates most of the columns. + * This engine node updates the port binding columns which + * needs to be updated after northd engine is run. + */ + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index b3bfb5454f..f1008915bd 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -21,4 +21,8 @@ void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); void en_sync_to_sb_lb_run(struct engine_node *, void *data); void en_sync_to_sb_lb_cleanup(void *data); +void *en_sync_to_sb_pb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_pb_run(struct engine_node *, void *data); +void en_sync_to_sb_pb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 546799b9e1..8f509ccdad 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, @@ -212,13 +213,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_lb, &en_northd, NULL); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_northd, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine syncs the SB Address_Set table and - * SB Load_Balancer table. + * Right now this engine syncs the SB Address_Set table, + * SB Load_Balancer table and (partly) SB Port_Binding table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_pb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index 3183057bd9..f8adcad763 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3472,8 +3472,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, ds_destroy(&s); sbrec_port_binding_set_external_ids(op->sb, &op->nbrp->external_ids); - - sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } else { if (!lsp_is_router(op->nbsp)) { uint32_t queue_id = smap_get_int( @@ -3589,116 +3587,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, } else { sbrec_port_binding_set_options(op->sb, NULL); } - const char *nat_addresses = smap_get(&op->nbsp->options, - "nat-addresses"); - size_t n_nats = 0; - char **nats = NULL; - bool l3dgw_ports = op->peer && op->peer->od && - op->peer->od->n_l3dgw_ports; - if (nat_addresses && !strcmp(nat_addresses, "router")) { - if (op->peer && op->peer->od - && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, - "exclude-lb-vips-from-garp", false); - nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); - } - } else if (nat_addresses && (chassis || l3dgw_ports)) { - struct lport_addresses laddrs; - if (!extract_lsp_addresses(nat_addresses, &laddrs)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); - } else { - destroy_lport_addresses(&laddrs); - n_nats = 1; - nats = xcalloc(1, sizeof *nats); - struct ds nat_addr = DS_EMPTY_INITIALIZER; - ds_put_format(&nat_addr, "%s", nat_addresses); - if (l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&nat_addr, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - nats[0] = xstrdup(ds_cstr(&nat_addr)); - ds_destroy(&nat_addr); - } - } - - /* Add the router mac and IPv4 addresses to - * Port_Binding.nat_addresses so that GARP is sent for these - * IPs by the ovn-controller on which the distributed gateway - * router port resides if: - * - * - op->peer has 'reside-on-redirect-chassis' set and the - * the logical router datapath has distributed router port. - * - * - op->peer is distributed gateway router port. - * - * - op->peer's router is a gateway router and op has a localnet - * port. - * - * Note: Port_Binding.nat_addresses column is also used for - * sending the GARPs for the router port IPs. - * */ - bool add_router_port_garp = false; - if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { - if (is_l3dgw_port(op->peer)) { - add_router_port_garp = true; - } else if (smap_get_bool(&op->peer->nbrp->options, - "reside-on-redirect-chassis", false)) { - if (op->peer->od->n_l3dgw_ports == 1) { - add_router_port_garp = true; - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " - "set on logical router port %s, which " - "is on logical router %s, which has %" - PRIuSIZE" distributed gateway ports. This" - "option can only be used when there is " - "a single distributed gateway port.", - op->peer->key, op->peer->od->nbr->name, - op->peer->od->n_l3dgw_ports); - } - } - } else if (chassis && op->od->n_localnet_ports) { - add_router_port_garp = true; - } - - if (add_router_port_garp) { - struct ds garp_info = DS_EMPTY_INITIALIZER; - ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); - - for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; - i++) { - ds_put_format(&garp_info, " %s", - op->peer->lrp_networks.ipv4_addrs[i].addr_s); - } - - if (op->peer->od->n_l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&garp_info, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - - n_nats++; - nats = xrealloc(nats, (n_nats * sizeof *nats)); - nats[n_nats - 1] = ds_steal_cstr(&garp_info); - ds_destroy(&garp_info); - } - sbrec_port_binding_set_nat_addresses(op->sb, - (const char **) nats, n_nats); - for (size_t i = 0; i < n_nats; i++) { - free(nats[i]); - } - free(nats); } sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name); @@ -4692,6 +4580,137 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } +/* Sync the SB Port bindings which needs to be updated. + * Presently it syncs the nat column of port bindings corresponding to + * the logical switch ports. */ +void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +{ + ovs_assert(ovnsb_idl_txn); + + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, ls_ports) { + if (lsp_is_router(op->nbsp)) { + const char *chassis = NULL; + if (op->peer && op->peer->od && op->peer->od->nbr) { + chassis = smap_get(&op->peer->od->nbr->options, "chassis"); + } + + const char *nat_addresses = smap_get(&op->nbsp->options, + "nat-addresses"); + size_t n_nats = 0; + char **nats = NULL; + bool l3dgw_ports = op->peer && op->peer->od && + op->peer->od->n_l3dgw_ports; + if (nat_addresses && !strcmp(nat_addresses, "router")) { + if (op->peer && op->peer->od + && (chassis || op->peer->od->n_l3dgw_ports)) { + bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + "exclude-lb-vips-from-garp", false); + nats = get_nat_addresses(op->peer, &n_nats, false, + !exclude_lb_vips); + } + } else if (nat_addresses && (chassis || l3dgw_ports)) { + struct lport_addresses laddrs; + if (!extract_lsp_addresses(nat_addresses, &laddrs)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); + } else { + destroy_lport_addresses(&laddrs); + n_nats = 1; + nats = xcalloc(1, sizeof *nats); + struct ds nat_addr = DS_EMPTY_INITIALIZER; + ds_put_format(&nat_addr, "%s", nat_addresses); + if (l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&nat_addr, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + nats[0] = xstrdup(ds_cstr(&nat_addr)); + ds_destroy(&nat_addr); + } + } + + /* Add the router mac and IPv4 addresses to + * Port_Binding.nat_addresses so that GARP is sent for these + * IPs by the ovn-controller on which the distributed gateway + * router port resides if: + * + * - op->peer has 'reside-on-redirect-chassis' set and the + * the logical router datapath has distributed router port. + * + * - op->peer is distributed gateway router port. + * + * - op->peer's router is a gateway router and op has a localnet + * port. + * + * Note: Port_Binding.nat_addresses column is also used for + * sending the GARPs for the router port IPs. + * */ + bool add_router_port_garp = false; + if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { + if (is_l3dgw_port(op->peer)) { + add_router_port_garp = true; + } else if (smap_get_bool(&op->peer->nbrp->options, + "reside-on-redirect-chassis", false)) { + if (op->peer->od->n_l3dgw_ports == 1) { + add_router_port_garp = true; + } else { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " + "set on logical router port %s, which " + "is on logical router %s, which has %" + PRIuSIZE" distributed gateway ports. This" + "option can only be used when there is " + "a single distributed gateway port.", + op->peer->key, op->peer->od->nbr->name, + op->peer->od->n_l3dgw_ports); + } + } + } else if (chassis && op->od->n_localnet_ports) { + add_router_port_garp = true; + } + + if (add_router_port_garp) { + struct ds garp_info = DS_EMPTY_INITIALIZER; + ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); + + for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; + i++) { + ds_put_format(&garp_info, " %s", + op->peer->lrp_networks.ipv4_addrs[i].addr_s); + } + + if (op->peer->od->n_l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&garp_info, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + n_nats++; + nats = xrealloc(nats, (n_nats * sizeof *nats)); + nats[n_nats - 1] = ds_steal_cstr(&garp_info); + ds_destroy(&garp_info); + } + sbrec_port_binding_set_nat_addresses(op->sb, + (const char **) nats, n_nats); + for (size_t i = 0; i < n_nats; i++) { + free(nats[i]); + } + free(nats); + } else { + sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + } + } +} + static bool ovn_port_add_tnlid(struct ovn_port *op, uint32_t tunnel_key) { diff --git a/northd/northd.h b/northd/northd.h index 3afa4eaff2..a79cc470c3 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -377,4 +377,6 @@ const char *northd_get_svc_monitor_mac(void); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); + #endif /* NORTHD_H */ From patchwork Sat Jun 24 06:49:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1799374 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qp4VW0SYMz20XS for ; Sat, 24 Jun 2023 16:50:11 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3B8E840A56; Sat, 24 Jun 2023 06:50:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3B8E840A56 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FJBys9p0RmQ0; Sat, 24 Jun 2023 06:50:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id B9C4E4091F; Sat, 24 Jun 2023 06:50:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B9C4E4091F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A05ABC0037; Sat, 24 Jun 2023 06:50:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9D48BC0029 for ; Sat, 24 Jun 2023 06:50:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4896B40391 for ; Sat, 24 Jun 2023 06:49:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4896B40391 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ahxqcp1fIo7K for ; Sat, 24 Jun 2023 06:49:17 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 80B5A408FB Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp2.osuosl.org (Postfix) with ESMTPS id 80B5A408FB for ; Sat, 24 Jun 2023 06:49:16 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 71B0560007; Sat, 24 Jun 2023 06:49:13 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Sat, 24 Jun 2023 12:19:10 +0530 Message-Id: <20230624064910.12227-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230624064703.11905-1-numans@ovn.org> References: <20230624064703.11905-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 8/8] northd: Handle load balancer changes for a logical router. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique When a logical router gets updated due to load balancer or load balancer groups changes, it is now incrementally handled in the 'northd' engine node. Other logical router updates result in the full recompute of 'northd' engine node. lflow engine node still falls back to recompute due to logical router changes. Signed-off-by: Numan Siddique --- lib/lb.c | 32 +++- lib/lb.h | 5 + northd/en-lflow.c | 5 + northd/en-northd.c | 20 ++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 3 +- northd/northd.c | 392 ++++++++++++++++++++++++++++++++++----- northd/northd.h | 5 + tests/ovn-northd.at | 12 +- 9 files changed, 422 insertions(+), 53 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index a0426cc42e..a8b694d0b3 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1082,7 +1082,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } @@ -1110,6 +1113,18 @@ ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, } } +void +ovn_lb_datapaths_remove_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set0(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr--; + } + } +} + struct ovn_lb_group_datapaths * ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, size_t n_ls_datapaths, @@ -1175,5 +1190,18 @@ void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, struct ovn_datapath *lr) { - bitmap_set1(lbg_dps->nb_lr_map, lr->index); + if (!bitmap_is_set(lbg_dps->nb_lr_map, lr->index)) { + bitmap_set1(lbg_dps->nb_lr_map, lr->index); + lbg_dps->n_nb_lr++; + } +} + +void +ovn_lb_group_datapaths_remove_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + if (bitmap_is_set(lbg_dps->nb_lr_map, lr->index)) { + bitmap_set1(lbg_dps->nb_lr_map, lr->index); + lbg_dps->n_nb_lr--; + } } diff --git a/lib/lb.h b/lib/lb.h index 08723e8ef3..91eec0199b 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -185,6 +185,8 @@ void ovn_lb_datapaths_set_ls(struct ovn_lb_datapaths *, size_t n, void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_remove_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); struct ovn_lb_group_datapaths { struct hmap_node hmap_node; @@ -214,6 +216,9 @@ void ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *, void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *, struct ovn_datapath *lr); +void ovn_lb_group_datapaths_remove_lr(struct ovn_lb_group_datapaths *, + struct ovn_datapath *lr); + struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/en-lflow.c b/northd/en-lflow.c index db1bcbccd6..ea51f4e3e0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -102,6 +102,11 @@ lflow_northd_handler(struct engine_node *node, if (!northd_data->change_tracked) { return false; } + + if (northd_data->lrouters_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; diff --git a/northd/en-northd.c b/northd/en-northd.c index b3c03c54bd..a321eff323 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -188,6 +188,26 @@ northd_nb_logical_switch_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_sb_port_binding_handler(struct engine_node *node, void *data) diff --git a/northd/en-northd.h b/northd/en-northd.h index 5674f4390c..739aa5e87f 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_northd_lb_data_handler(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 8f509ccdad..cdbd0a65c3 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -156,7 +156,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_meter, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); @@ -185,6 +184,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); diff --git a/northd/northd.c b/northd/northd.c index f8adcad763..52889bf92f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3977,6 +3977,90 @@ associate_ls_lb_groups(struct ovn_datapath *ls_od, } } +static void +associate_lr_lbs(struct ovn_datapath *lr_od, struct hmap *lb_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + + free(lr_od->lb_uuids); + lr_od->lb_uuids = xcalloc(lr_od->nbr->n_load_balancer, + sizeof *lr_od->lb_uuids); + lr_od->n_lb_uuids = lr_od->nbr->n_load_balancer; + + if (!lr_od->lb_ips) { + lr_od->lb_ips = ovn_lb_ip_set_create(); + } + + for (size_t i = 0; i < lr_od->nbr->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &lr_od->nbr->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &lr_od); + build_lrouter_lb_ips(lr_od->lb_ips, lb_dps->lb); + lr_od->lb_uuids[i] = *lb_uuid; + } +} + +static void +associate_lr_lb_groups(struct ovn_datapath *lr_od, + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + struct ovn_lb_group_datapaths *lb_group_dps; + + free(lr_od->lb_group_uuids); + lr_od->lb_group_uuids = xcalloc(lr_od->nbr->n_load_balancer_group, + sizeof *lr_od->lb_group_uuids); + lr_od->n_lb_group_uuids = lr_od->nbr->n_load_balancer_group; + + /* Checking load balancer groups first, starting from the largest one, + * to more efficiently copy IP sets. */ + size_t largest_group = 0; + + for (size_t i = 1; i < lr_od->nbr->n_load_balancer_group; i++) { + if (lr_od->nbr->load_balancer_group[i]->n_load_balancer > + lr_od->nbr->load_balancer_group[largest_group]->n_load_balancer) { + largest_group = i; + } + } + + for (size_t i = 0; i < lr_od->nbr->n_load_balancer_group; i++) { + size_t idx = (i + largest_group) % lr_od->nbr->n_load_balancer_group; + + nbrec_lb_group = lr_od->nbr->load_balancer_group[idx]; + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, lr_od); + + if (!lr_od->lb_ips) { + lr_od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); + } else { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(lr_od->lb_ips, + lb_group_dps->lb_group->lbs[j]); + } + } + + lr_od->lb_group_uuids[i] = *lb_group_uuid; + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &lr_od); + } + } +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, @@ -3984,7 +4068,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map) { - const struct nbrec_load_balancer_group *nbrec_lb_group; struct ovn_lb_group_datapaths *lb_group_dps; const struct ovn_lb_group *lb_group; struct ovn_lb_datapaths *lb_dps; @@ -4018,53 +4101,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - - /* Checking load balancer groups first, starting from the largest one, - * to more efficiently copy IP sets. */ - size_t largest_group = 0; - - for (size_t i = 1; i < od->nbr->n_load_balancer_group; i++) { - if (od->nbr->load_balancer_group[i]->n_load_balancer > - od->nbr->load_balancer_group[largest_group]->n_load_balancer) { - largest_group = i; - } - } - - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; - - nbrec_lb_group = od->nbr->load_balancer_group[idx]; - const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; - - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_datapaths_map, - lb_group_uuid); - ovs_assert(lb_group_dps); - ovn_lb_group_datapaths_add_lr(lb_group_dps, od); - - if (!od->lb_ips) { - od->lb_ips = - ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); - } else { - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, - lb_group_dps->lb_group->lbs[j]); - } - } - } - - if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_create(); - } - - for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &od->nbr->load_balancer[i]->header_.uuid; - lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); - ovs_assert(lb_dps); - ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - } + associate_lr_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + associate_lr_lbs(od, lb_datapaths_map); } HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { @@ -4934,6 +4972,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) } nd->change_tracked = false; + nd->lrouters_changed = false; } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5429,6 +5468,263 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +static bool +lr_check_and_handle_lb_and_lbgrp_changes( + const struct nbrec_logical_router *changed_lr, + struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map, + struct ovn_datapath *od, bool *updated) +{ + ovs_assert(od->nbr); + bool lbs_changed = true; + if (!nbrec_logical_router_is_updated(changed_lr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER) && + !nbrec_logical_router_is_updated(changed_lr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)) { + lbs_changed = false; + for (size_t i = 0; i < changed_lr->n_load_balancer_group; i++) { + if (nbrec_load_balancer_group_row_get_seqno( + changed_lr->load_balancer_group[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + lbs_changed = true; + break; + } + } + + if (!lbs_changed) { + for (size_t i = 0; i < changed_lr->n_load_balancer; i++) { + if (nbrec_load_balancer_row_get_seqno( + changed_lr->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + lbs_changed = true; + break; + } + } + } + + if (!lbs_changed) { + return true; + } + } + + /* We need to do 2 things to handle the router for load balancer or + * load balancer groups. + * + * 1. Disassociate the logical router datapath from the already associated + * load balancers/load balancer groups. + * 2. Associate the logical router datapath with the updated + * load balancers/groups. + * */ + struct ovn_lb_datapaths *lb_dps; + size_t i, j; + for (i = 0; i < od->n_lb_uuids; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &od->lb_uuids[i]); + if (lb_dps) { + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag set. + * This requires updating the router ports's routable + * addresses. */ + return false; + } + ovn_lb_datapaths_remove_lr(lb_dps, 1, &od); + } + } + + struct ovn_lb_group_datapaths *lbg_dps; + for (i = 0; i < od->n_lb_group_uuids; i++) { + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &od->lb_group_uuids[i]); + if (lbg_dps) { + ovn_lb_group_datapaths_remove_lr(lbg_dps, od); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_group_datapaths_remove_ls(lbg_dps, od->n_ls_peers, + od->ls_peers); + } + } + + for (j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (lb_dps) { + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag + * set. This requires updating the router ports's + * routable addresses. */ + return false; + } + ovn_lb_datapaths_remove_lr(lb_dps, 1, &od); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_datapaths_remove_ls(lb_dps, od->n_ls_peers, + od->ls_peers); + } + } + } + } + + ovn_lb_ip_set_destroy(od->lb_ips); + od->lb_ips = NULL; + associate_lr_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + associate_lr_lbs(od, lb_datapaths_map); + + /* Do the job of build_lrouter_lbs_reachable_ips and + * build_lswitch_lbs_from_lrouter for this lrouter datapath. */ + for (i = 0; i < od->nbr->n_load_balancer; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag set. + * This requires updating the router ports's routable addresses. */ + return false; + } + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + for (i = 0; i < od->nbr->n_load_balancer_group; i++) { + const struct nbrec_load_balancer_group *nbrec_lb_group = + od->nbr->load_balancer_group[i]; + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lbg_dps); + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_group_datapaths_add_ls(lbg_dps, od->n_ls_peers, + od->ls_peers); + } + + for (j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, lbg_dps->lb_group->lbs[j]); + + if (install_ls_lb_from_router && od->n_ls_peers) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag + * set. This requires updating the router ports's + * routable addresses. */ + return false; + } + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + } + + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} + + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + bool updated = false; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + struct ovn_datapath *od = ovn_datapath_find( + &nd->lr_datapaths.datapaths, + &changed_lr->header_.uuid); + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + + if (!lr_check_and_handle_lb_and_lbgrp_changes(changed_lr, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map, + od, &updated)) { + goto fail; + } + } + + if (updated) { + nd->change_tracked = true; + nd->lrouters_changed = true; + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, diff --git a/northd/northd.h b/northd/northd.h index a79cc470c3..765720849a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -119,8 +119,11 @@ struct northd_data { struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; + + /* Tracked data. */ bool change_tracked; struct tracked_ls_changes tracked_ls_changes; + bool lrouters_changed; }; struct lflow_data { @@ -342,6 +345,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index da9bc16fd9..6f6f67510e 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9687,6 +9687,14 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats norecompute recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lr-lb-add lr0 lb1 +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lr-lb-del lr0 lb1 +check_engine_stats norecompute recompute + # Test load balancer group now check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) @@ -9723,11 +9731,11 @@ check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats recompute recompute +check_engine_stats norecompute recompute AT_CLEANUP ])