From patchwork Fri Jul 7 05:52:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804603 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2ch6b13z20b8 for ; Fri, 7 Jul 2023 15:53:08 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id E510640412; Fri, 7 Jul 2023 05:53:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E510640412 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f2Eajs3JlySP; Fri, 7 Jul 2023 05:53:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id D0CD440142; Fri, 7 Jul 2023 05:53:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D0CD440142 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9E7F4C0072; Fri, 7 Jul 2023 05:53:04 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id CD7D5C0032 for ; Fri, 7 Jul 2023 05:53:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A722C418BC for ; Fri, 7 Jul 2023 05:53:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A722C418BC X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Nw3PPseVCUSt for ; Fri, 7 Jul 2023 05:53:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C0FFB418FB Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::223]) by smtp4.osuosl.org (Postfix) with ESMTPS id C0FFB418FB for ; Fri, 7 Jul 2023 05:53:01 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id C815960003; Fri, 7 Jul 2023 05:52:57 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:22:52 +0530 Message-Id: <20230707055252.961543-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 1/8] northd I-P: Sync SB load balancers in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Similar to the commit [1], a new sub-engine node "sync_to_sb_lb" is added with-in the "sync_to_sb" to sync the SB load balancers. Its main input nodes are "northd" (to access the "lbs" hmap built by this node) and "sb_load_balancer" to access the SB load balancer. "sync_to_sb_lb" doesn't add any handlers and falls back to full recompute all the time. [1] - ccafcc2dc321("northd I-P: Add a new engine node 'en_sync_to_sb' to sync SB tables.") Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 -- northd/en-sync-sb.c | 29 +++++++++++++++++++++++++++++ northd/en-sync-sb.h | 5 +++++ northd/inc-proc-northd.c | 9 +++++++-- northd/northd.c | 4 +--- northd/northd.h | 6 +++++- 6 files changed, 47 insertions(+), 8 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 044fa70190..f9f2d04452 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -101,8 +101,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis", node)); input_data->sbrec_fdb_table = EN_OVSDB_GET(engine_get_input("SB_fdb", node)); - input_data->sbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); input_data->sbrec_service_monitor_table = EN_OVSDB_GET(engine_get_input("SB_service_monitor", node)); input_data->sbrec_port_group_table = diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index d7fea981f2..821047581c 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -211,6 +211,35 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, return true; } +/* sync_to_sb_lb engine node functions. + * This engine node syncs the SB load balancers. + */ +void * +en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct sbrec_load_balancer_table *sb_load_balancer_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, + &northd_data->ls_datapaths, &northd_data->lbs); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +{ + +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index bcb9799d24..b3bfb5454f 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -16,4 +16,9 @@ bool sync_to_sb_addr_set_nb_address_set_handler(struct engine_node *, bool sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *, void *data); + +void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_lb_run(struct engine_node *, void *data); +void en_sync_to_sb_lb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index d328deb222..507348b719 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -139,6 +139,7 @@ static ENGINE_NODE(sync_to_sb, "sync_to_sb"); static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); +static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -166,7 +167,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); engine_add_input(&en_northd, &en_sb_service_monitor, NULL); - engine_add_input(&en_northd, &en_sb_load_balancer, NULL); engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); @@ -202,11 +202,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine only syncs the SB Address_Set table. + * Right now this engine syncs the SB Address_Set table and + * SB Load_Balancer table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index b9605862e4..2390c159c3 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4538,7 +4538,7 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ -static void +void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, struct ovn_datapaths *ls_datapaths, struct hmap *lbs) @@ -17580,8 +17580,6 @@ ovnnb_db_run(struct northd_input *input_data, ovn_update_ipv6_options(&data->lr_ports); ovn_update_ipv6_prefix(&data->lr_ports); - sync_lbs(ovnsb_txn, input_data->sbrec_load_balancer_table, - &data->ls_datapaths, &data->lbs); sync_port_groups(ovnsb_txn, input_data->sbrec_port_group_table, &data->port_groups); sync_meters(ovnsb_txn, input_data->nbrec_meter_table, diff --git a/northd/northd.h b/northd/northd.h index f3e63b1e1a..48c282476a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -16,6 +16,7 @@ #include "ovsdb-idl.h" +#include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/ovs-atomic.h" #include "lib/sset.h" @@ -47,7 +48,6 @@ struct northd_input { const struct sbrec_ha_chassis_group_table *sbrec_ha_chassis_group_table; const struct sbrec_chassis_table *sbrec_chassis_table; const struct sbrec_fdb_table *sbrec_fdb_table; - const struct sbrec_load_balancer_table *sbrec_load_balancer_table; const struct sbrec_service_monitor_table *sbrec_service_monitor_table; const struct sbrec_port_group_table *sbrec_port_group_table; const struct sbrec_meter_table *sbrec_meter_table; @@ -354,4 +354,8 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, void run_update_worker_pool(int n_threads); const char *northd_get_svc_monitor_mac(void); + +void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, + struct ovn_datapaths *ls_datapaths, struct hmap *lbs); + #endif /* NORTHD_H */ From patchwork Fri Jul 7 05:53:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804604 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2dS6Jy7z20b8 for ; Fri, 7 Jul 2023 15:53:48 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EA5B240567; Fri, 7 Jul 2023 05:53:46 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EA5B240567 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fz36F_Y67pzG; Fri, 7 Jul 2023 05:53:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 791DD40407; Fri, 7 Jul 2023 05:53:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 791DD40407 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5B606C0072; Fri, 7 Jul 2023 05:53:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 82FF8C0032 for ; Fri, 7 Jul 2023 05:53:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5BEC940484 for ; Fri, 7 Jul 2023 05:53:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5BEC940484 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h4pCjRWWnUFc for ; Fri, 7 Jul 2023 05:53:38 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1904740407 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1904740407 for ; Fri, 7 Jul 2023 05:53:37 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id B1F751C0002; Fri, 7 Jul 2023 05:53:34 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:23:26 +0530 Message-Id: <20230707055326.961596-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 2/8] northd: Add a new engine node - northd_lb_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch separates out the 'lbs' and 'lb_groups' from the 'northd' engine node data into a new engine node 'northd_lb_data'. This new node becomes an input to the 'northd' node. This makes handling the NB load balancer and load balancer group changes easier. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- lib/lb.c | 201 +++++++++-- lib/lb.h | 86 +++-- northd/automake.mk | 2 + northd/en-lflow.c | 3 +- northd/en-northd-lb-data.c | 126 +++++++ northd/en-northd-lb-data.h | 19 ++ northd/en-northd.c | 11 +- northd/en-sync-sb.c | 2 +- northd/inc-proc-northd.c | 8 +- northd/northd.c | 673 +++++++++++++++++++++---------------- northd/northd.h | 15 +- 11 files changed, 780 insertions(+), 366 deletions(-) create mode 100644 northd/en-northd-lb-data.c create mode 100644 northd/en-northd-lb-data.h diff --git a/lib/lb.c b/lib/lb.c index 7afdaed65b..429dbf15af 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -26,6 +26,7 @@ #include "openvswitch/vlog.h" #include "lib/bitmap.h" #include "lib/smap.h" +#include "socket-util.h" VLOG_DEFINE_THIS_MODULE(lb); @@ -431,11 +432,62 @@ void ovn_northd_lb_vip_init(struct ovn_northd_lb_vip *lb_vip_nb, ovn_lb_get_health_check(nbrec_lb, vip_port_str, template); } +static void +ovn_lb_vip_backends_health_check_init(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + struct ovn_northd_lb_vip *lb_vip_nb) +{ + struct ds key = DS_EMPTY_INITIALIZER; + + for (size_t j = 0; j < lb_vip->n_backends; j++) { + struct ovn_lb_backend *backend = &lb_vip->backends[j]; + ds_clear(&key); + ds_put_format(&key, IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) + ? "%s" : "[%s]", backend->ip_str); + + const char *s = smap_get(&lb->nlb->ip_port_mappings, ds_cstr(&key)); + if (!s) { + continue; + } + + char *svc_mon_src_ip = NULL; + char *port_name = xstrdup(s); + char *p = strstr(port_name, ":"); + if (p) { + *p = 0; + p++; + struct sockaddr_storage svc_mon_src_addr; + if (!inet_parse_address(p, &svc_mon_src_addr)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); + } else { + struct ds src_ip_s = DS_EMPTY_INITIALIZER; + ss_format_address_nobracks(&svc_mon_src_addr, + &src_ip_s); + svc_mon_src_ip = ds_steal_cstr(&src_ip_s); + } + } + + if (svc_mon_src_ip) { + struct ovn_northd_lb_backend *backend_nb = + &lb_vip_nb->backends_nb[j]; + backend_nb->health_check = true; + backend_nb->logical_port = xstrdup(port_name); + backend_nb->svc_mon_src_ip = svc_mon_src_ip; + } + free(port_name); + } + + ds_destroy(&key); +} + static void ovn_northd_lb_vip_destroy(struct ovn_northd_lb_vip *vip) { free(vip->backend_ips); for (size_t i = 0; i < vip->n_backends; i++) { + free(vip->backends_nb[i].logical_port); free(vip->backends_nb[i].svc_mon_src_ip); } free(vip->backends_nb); @@ -555,8 +607,7 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, } struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, - size_t n_ls_datapaths, size_t n_lr_datapaths) +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); @@ -595,9 +646,6 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, } lb->affinity_timeout = affinity_timeout; - lb->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb->nb_lr_map = bitmap_allocate(n_lr_datapaths); - sset_init(&lb->ips_v4); sset_init(&lb->ips_v6); struct smap_node *node; @@ -631,7 +679,12 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, ovn_lb_vip6_template_format_internal(lb_vip), xstrdup(node->value)); } + n_vips++; + + if (lb_vip_nb->lb_health_check) { + ovn_lb_vip_backends_health_check_init(lb, lb_vip, lb_vip_nb); + } } /* It's possible that parsing VIPs fails. Update the lb->n_vips to the @@ -639,6 +692,7 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb, */ lb->n_vips = n_vips; + if (nbrec_lb->n_selection_fields) { char *proto = NULL; if (nbrec_lb->protocol && nbrec_lb->protocol[0]) { @@ -684,24 +738,6 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_lr_map, ods[i]->index); - } -} - -void -ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - bitmap_set1(lb->nb_ls_map, ods[i]->index); - } -} - void ovn_northd_lb_destroy(struct ovn_northd_lb *lb) { @@ -715,8 +751,6 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); - bitmap_free(lb->nb_lr_map); - bitmap_free(lb->nb_ls_map); free(lb); } @@ -727,8 +761,7 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ struct ovn_lb_group * ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs, size_t max_ls_datapaths, - size_t max_lr_datapaths) + const struct hmap *lbs) { struct ovn_lb_group *lb_group; @@ -736,8 +769,6 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, lb_group->uuid = nbrec_lb_group->header_.uuid; lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); - lb_group->ls = xmalloc(max_ls_datapaths * sizeof *lb_group->ls); - lb_group->lr = xmalloc(max_lr_datapaths * sizeof *lb_group->lr); lb_group->lb_ips = ovn_lb_ip_set_create(); for (size_t i = 0; i < nbrec_lb_group->n_load_balancer; i++) { @@ -758,8 +789,6 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) ovn_lb_ip_set_destroy(lb_group->lb_ips); free(lb_group->lbs); - free(lb_group->ls); - free(lb_group->lr); free(lb_group); } @@ -943,3 +972,113 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) hmap_destroy(tuples); } + +void +build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, + const struct ovn_northd_lb *lb) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, &lb->ips_v4) { + sset_add(&lb_ips->ips_v4, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, &lb->ips_v6) { + sset_add(&lb_ips->ips_v6, ip_address); + if (lb->routable) { + sset_add(&lb_ips->ips_v6_routable, ip_address); + } + } +} + +/* lb datapaths functions */ +struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + } +} + +void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + } +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} diff --git a/lib/lb.h b/lib/lb.h index 23d8fc9e9b..0339050cba 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -59,7 +59,6 @@ struct ovn_northd_lb { struct hmap_node hmap_node; const struct nbrec_load_balancer *nlb; /* May be NULL. */ - const struct sbrec_load_balancer *slb; /* May be NULL. */ const char *proto; char *selection_fields; struct ovn_lb_vip *vips; @@ -78,14 +77,6 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; - - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; - - struct ovn_dp_group *dpg; }; struct ovn_lb_vip { @@ -129,23 +120,19 @@ struct ovn_northd_lb_vip { }; struct ovn_northd_lb_backend { - struct ovn_port *op; /* Logical port to which the ip belong to. */ bool health_check; + char *logical_port; /* Logical port to which the ip belong to. */ char *svc_mon_src_ip; /* Source IP to use for monitoring. */ - const struct sbrec_service_monitor *sbrec_monitor; }; -struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); +struct ovn_northd_lb *ovn_northd_lb_create(const struct nbrec_load_balancer *); struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); -void ovn_northd_lb_add_lr(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); -void ovn_northd_lb_add_ls(struct ovn_northd_lb *lb, size_t n, - struct ovn_datapath **ods); + +void build_lrouter_lb_ips(struct ovn_lb_ip_set *, + const struct ovn_northd_lb *); struct ovn_lb_group { struct hmap_node hmap_node; @@ -153,35 +140,70 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; +}; + +struct ovn_lb_group *ovn_lb_group_create( + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); +void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); +struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, + const struct uuid *); + +struct ovn_lb_datapaths { + struct hmap_node hmap_node; - /* Datapaths to which this LB group is applied. */ + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, + size_t n_ls_datapaths, + size_t n_lr_datapaths); +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); +void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ size_t n_ls; struct ovn_datapath **ls; size_t n_lr; struct ovn_datapath **lr; }; -struct ovn_lb_group *ovn_lb_group_create( - const struct nbrec_load_balancer_group *, - const struct hmap *lbs, - size_t max_ls_datapaths, +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( + const struct ovn_lb_group *, size_t max_ls_datapaths, size_t max_lr_datapaths); -void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); -struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, - const struct uuid *); + +void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); static inline void -ovn_lb_group_add_ls(struct ovn_lb_group *lb_group, size_t n, - struct ovn_datapath **ods) +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) { - memcpy(&lb_group->ls[lb_group->n_ls], ods, n * sizeof *ods); - lb_group->n_ls += n; + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; } static inline void -ovn_lb_group_add_lr(struct ovn_lb_group *lb_group, struct ovn_datapath *lr) +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) { - lb_group->lr[lb_group->n_lr++] = lr; + lbg_dps->lr[lbg_dps->n_lr++] = lr; } struct ovn_controller_lb { diff --git a/northd/automake.mk b/northd/automake.mk index b17f1fdb54..6f60265b73 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -18,6 +18,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-sb.h \ northd/en-sync-from-sb.c \ northd/en-sync-from-sb.h \ + northd/en-northd-lb-data.c \ + northd/en-northd-lb-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 28ab1c67fb..db1bcbccd6 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -57,7 +57,8 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->port_groups = &northd_data->port_groups; lflow_input->meter_groups = &northd_data->meter_groups; - lflow_input->lbs = &northd_data->lbs; + lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; + lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; lflow_input->features = &northd_data->features; lflow_input->ovn_internal_version_changed = northd_data->ovn_internal_version_changed; diff --git a/northd/en-northd-lb-data.c b/northd/en-northd-lb-data.c new file mode 100644 index 0000000000..d46c3c27ed --- /dev/null +++ b/northd/en-northd-lb-data.c @@ -0,0 +1,126 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +#include "openvswitch/util.h" + +#include "en-northd-lb-data.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "northd.h" + +#include "openvswitch/vlog.h" + +VLOG_DEFINE_THIS_MODULE(en_northd_lb_data); + +static void northd_lb_data_init(struct northd_lb_data *); +static void northd_lb_data_destroy(struct northd_lb_data *); +static void build_lbs(const struct nbrec_load_balancer_table *, + const struct nbrec_load_balancer_group_table *, + struct hmap *lbs, struct hmap *lb_groups); + +void * +en_northd_lb_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct northd_lb_data *data = xzalloc(sizeof *data); + + northd_lb_data_init(data); + + return data; +} + +void +en_northd_lb_data_run(struct engine_node *node, void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); + northd_lb_data_init(lb_data); + + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_northd_lb_data_cleanup(void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + northd_lb_data_destroy(lb_data); +} + +/* static functions. */ +static void +northd_lb_data_init(struct northd_lb_data *lb_data) +{ + hmap_init(&lb_data->lbs); + hmap_init(&lb_data->lb_groups); +} + +static void +northd_lb_data_destroy(struct northd_lb_data *lb_data) +{ + struct ovn_northd_lb *lb; + HMAP_FOR_EACH_POP (lb, hmap_node, &lb_data->lbs) { + ovn_northd_lb_destroy(lb); + } + hmap_destroy(&lb_data->lbs); + + struct ovn_lb_group *lb_group; + HMAP_FOR_EACH_POP (lb_group, hmap_node, &lb_data->lb_groups) { + ovn_lb_group_destroy(lb_group); + } + hmap_destroy(&lb_data->lb_groups); +} + +static void +build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, + const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group; + struct ovn_northd_lb *lb_nb; + + const struct nbrec_load_balancer *nbrec_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { + lb_nb = ovn_northd_lb_create(nbrec_lb); + hmap_insert(lbs, &lb_nb->hmap_node, + uuid_hash(&nbrec_lb->header_.uuid)); + } + + const struct nbrec_load_balancer_group *nbrec_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, + nbrec_lb_group_table) { + lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + } +} diff --git a/northd/en-northd-lb-data.h b/northd/en-northd-lb-data.h new file mode 100644 index 0000000000..eb297e376d --- /dev/null +++ b/northd/en-northd-lb-data.h @@ -0,0 +1,19 @@ +#ifndef EN_NORTHD_LB_DATA_H +#define EN_NORTHD_LB_DATA_H 1 + +#include + +#include "openvswitch/hmap.h" + +#include "lib/inc-proc-eng.h" + +struct northd_lb_data { + struct hmap lbs; + struct hmap lb_groups; +}; + +void *en_northd_lb_data_init(struct engine_node *, struct engine_arg *); +void en_northd_lb_data_run(struct engine_node *, void *data); +void en_northd_lb_data_cleanup(void *data); + +#endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index f9f2d04452..cc7d838451 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -20,6 +20,7 @@ #include "coverage.h" #include "en-northd.h" +#include "en-northd-lb-data.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "openvswitch/list.h" /* TODO This is needed for ovn-parallel-hmap.h. @@ -70,10 +71,6 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); - input_data->nbrec_load_balancer_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); - input_data->nbrec_load_balancer_group_table = - EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); input_data->nbrec_port_group_table = EN_OVSDB_GET(engine_get_input("NB_port_group", node)); input_data->nbrec_meter_table = @@ -117,6 +114,11 @@ northd_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_chassis_template_var", node)); input_data->sbrec_mirror_table = EN_OVSDB_GET(engine_get_input("SB_mirror", node)); + + struct northd_lb_data *lb_data = + engine_get_input_data("northd_lb_data", node); + input_data->lbs = &lb_data->lbs; + input_data->lb_groups = &lb_data->lb_groups; } void @@ -130,6 +132,7 @@ en_northd_run(struct engine_node *node, void *data) northd_init(data); northd_get_input_data(node, &input_data); + COVERAGE_INC(northd_run); stopwatch_start(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); ovnnb_db_run(&input_data, data, eng_ctx->ovnnb_idl_txn, diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 821047581c..fda0ca5a68 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -230,7 +230,7 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lbs); + &northd_data->ls_datapaths, &northd_data->lb_datapaths_map); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 507348b719..b2e884962f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -35,6 +35,7 @@ #include "en-northd-output.h" #include "en-sync-sb.h" #include "en-sync-from-sb.h" +#include "en-northd-lb-data.h" #include "unixctl.h" #include "util.h" @@ -140,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -147,8 +149,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, /* Define relationships between nodes where first argument is dependent * on the second argument */ engine_add_input(&en_northd, &en_nb_port_group, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd, &en_nb_load_balancer_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); @@ -178,6 +178,10 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, NULL); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, NULL); + engine_add_input(&en_northd, &en_northd_lb_data, NULL); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 2390c159c3..890186b29c 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3862,14 +3862,11 @@ struct service_monitor_info { static struct service_monitor_info * -create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *monitor_map, - const char *ip, const char *logical_port, - uint16_t service_port, const char *protocol) +get_service_mon(const struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol, + uint32_t hash) { - uint32_t hash = service_port; - hash = hash_string(ip, hash); - hash = hash_string(logical_port, hash); struct service_monitor_info *mon_info; HMAP_FOR_EACH_WITH_HASH (mon_info, hmap_node, hash, monitor_map) { @@ -3881,6 +3878,26 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } } + return NULL; +} + +static struct service_monitor_info * +create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *monitor_map, + const char *ip, const char *logical_port, + uint16_t service_port, const char *protocol) +{ + uint32_t hash = service_port; + hash = hash_string(ip, hash); + hash = hash_string(logical_port, hash); + struct service_monitor_info *mon_info = + get_service_mon(monitor_map, ip, logical_port, service_port, + protocol, hash); + + if (mon_info) { + return mon_info; + } + struct sbrec_service_monitor *sbrec_mon = sbrec_service_monitor_insert(ovnsb_txn); sbrec_service_monitor_set_ip(sbrec_mon, ip); @@ -3894,7 +3911,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, } static void -ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, +ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_northd_lb *lb, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3911,58 +3929,27 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - struct ovn_port *op = NULL; - char *svc_mon_src_ip = NULL; - - struct ds key = DS_EMPTY_INITIALIZER; - ds_put_format(&key, - IN6_IS_ADDR_V4MAPPED(&lb_vip->vip) - ? "%s" : "[%s]", backend->ip_str); - - const char *s = smap_get(&lb->nlb->ip_port_mappings, - ds_cstr(&key)); - if (s) { - char *port_name = xstrdup(s); - char *p = strstr(port_name, ":"); - if (p) { - *p = 0; - p++; - sset_add(svc_monitor_lsps, port_name); - op = ovn_port_find(ls_ports, port_name); - struct sockaddr_storage svc_mon_src_addr; - if (!inet_parse_address(p, &svc_mon_src_addr)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Invalid svc mon src IP %s", p); - } else { - struct ds src_ip_s = DS_EMPTY_INITIALIZER; - ss_format_address_nobracks(&svc_mon_src_addr, - &src_ip_s); - svc_mon_src_ip = ds_steal_cstr(&src_ip_s); - } - } - free(port_name); + if (!backend_nb->health_check) { + continue; } - ds_destroy(&key); - if (!lb_vip_nb->lb_health_check || !op || !svc_mon_src_ip || - !lsp_is_enabled(op->nbsp)) { - free(svc_mon_src_ip); + sset_add(svc_monitor_lsps, backend_nb->logical_port); + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + + if (!op || !lsp_is_enabled(op->nbsp)) { continue; } - backend_nb->op = op; - backend_nb->svc_mon_src_ip = svc_mon_src_ip; - const char *protocol = lb->nlb->protocol; if (!protocol || !protocol[0]) { protocol = "tcp"; } - backend_nb->health_check = true; + struct service_monitor_info *mon_info = create_or_get_service_mon(ovnsb_txn, monitor_map, backend->ip_str, - backend_nb->op->nbsp->name, + backend_nb->logical_port, backend->port, protocol); ovs_assert(mon_info); @@ -3991,18 +3978,20 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct ovn_northd_lb *lb, "offline"); } - backend_nb->sbrec_monitor = mon_info->sbrec_mon; mon_info->required = true; } } } static bool -build_lb_vip_actions(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb_vip *lb_vip_nb, +build_lb_vip_actions(const struct ovn_northd_lb *lb, + const struct ovn_lb_vip *lb_vip, + const struct ovn_northd_lb_vip *lb_vip_nb, struct ds *action, char *selection_fields, - struct ds *skip_snat_action, struct ds *force_snat_action, - bool ls_dp, const struct chassis_features *features) + struct ds *skip_snat_action, + struct ds *force_snat_action, + bool ls_dp, const struct chassis_features *features, + const struct hmap *svc_monitor_map) { const char *ct_lb_action = features->ct_no_masked_label ? "ct_lb_mark" : "ct_lb"; @@ -4017,10 +4006,31 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, struct ovn_lb_backend *backend = &lb_vip->backends[i]; struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[i]; - if (!backend_nb->health_check || - (backend_nb->health_check && backend_nb->sbrec_monitor && - backend_nb->sbrec_monitor->status && - strcmp(backend_nb->sbrec_monitor->status, "online"))) { + + if (!backend_nb->health_check) { + continue; + } + + const char *protocol = lb->nlb->protocol; + if (!protocol || !protocol[0]) { + protocol = "tcp"; + } + + uint32_t hash = backend->port; + hash = hash_string(backend->ip_str, hash); + hash = hash_string(backend_nb->logical_port, hash); + + struct service_monitor_info *mon_info = get_service_mon( + svc_monitor_map, backend->ip_str, backend_nb->logical_port, + backend->port, protocol, hash); + + if (!mon_info) { + continue; + } + + ovs_assert(mon_info->sbrec_mon); + if (mon_info->sbrec_mon->status && + strcmp(mon_info->sbrec_mon->status, "online")) { continue; } @@ -4070,59 +4080,32 @@ build_lb_vip_actions(struct ovn_lb_vip *lb_vip, } static void -build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, - const struct ovn_northd_lb *lb) -{ - const char *ip_address; - - SSET_FOR_EACH (ip_address, &lb->ips_v4) { - sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v4_routable, ip_address); - } - } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { - sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { - sset_add(&lb_ips->ips_v6_routable, ip_address); - } - } -} - -static void -build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, - const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) +build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb; + struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_group *lb_group; + struct ovn_lb_datapaths *lb_dps; + const struct ovn_northd_lb *lb; - hmap_init(lbs); - hmap_init(lb_groups); + hmap_init(lb_datapaths_map); + hmap_init(lb_group_datapaths_map); - const struct nbrec_load_balancer *nbrec_lb; - NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - hmap_insert(lbs, &lb_nb->hmap_node, - uuid_hash(&nbrec_lb->header_.uuid)); + HMAP_FOR_EACH (lb, hmap_node, lbs) { + lb_dps = ovn_lb_datapaths_create(lb, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(&lb->nlb->header_.uuid)); } - NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, - nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs, - ods_size(ls_datapaths), - ods_size(lr_datapaths)); - - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } - - hmap_insert(lb_groups, &lb_group->hmap_node, + HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { + lb_group_dps = ovn_lb_group_datapaths_create( + lb_group, ods_size(ls_datapaths), ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, uuid_hash(&lb_group->uuid)); } @@ -4135,22 +4118,19 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbs->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_ls(lb, 1, &od); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { nbrec_lb_group = od->nbs->load_balancer_group[i]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_ls(lb_group, 1, &od); - } - } - - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], lb_group->n_ls, - lb_group->ls); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); } } @@ -4172,15 +4152,21 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; nbrec_lb_group = od->nbr->load_balancer_group[idx]; - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - ovn_lb_group_add_lr(lb_group, od); + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, od); if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_clone(lb_group->lb_ips); + od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); } else { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, lb_group->lbs[j]); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(od->lb_ips, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4192,16 +4178,23 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { const struct uuid *lb_uuid = &od->nbr->load_balancer[i]->header_.uuid; - lb = ovn_northd_lb_find(lbs, lb_uuid); - ovn_northd_lb_add_lr(lb, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb); + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_lr(lb_group->lbs[j], lb_group->n_lr, - lb_group->lr); + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, + lb_group_dps->ls); + ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, + lb_group_dps->lr); } } } @@ -4210,10 +4203,10 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, - struct hmap *ls_ports, struct hmap *lbs, struct sset *svc_monitor_lsps) + struct hmap *ls_ports, struct hmap *lb_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { - struct hmap monitor_map = HMAP_INITIALIZER(&monitor_map); - const struct sbrec_service_monitor *sbrec_mon; SBREC_SERVICE_MONITOR_TABLE_FOR_EACH (sbrec_mon, sbrec_service_monitor_table) { @@ -4223,24 +4216,23 @@ build_lb_svcs( struct service_monitor_info *mon_info = xzalloc(sizeof *mon_info); mon_info->sbrec_mon = sbrec_mon; mon_info->required = false; - hmap_insert(&monitor_map, &mon_info->hmap_node, hash); + hmap_insert(svc_monitor_map, &mon_info->hmap_node, hash); } - struct ovn_northd_lb *lb; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - ovn_lb_svc_create(ovnsb_txn, lb, &monitor_map, ls_ports, + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, svc_monitor_lsps); } struct service_monitor_info *mon_info; - HMAP_FOR_EACH_POP (mon_info, hmap_node, &monitor_map) { + HMAP_FOR_EACH_SAFE (mon_info, hmap_node, svc_monitor_map) { if (!mon_info->required) { sbrec_service_monitor_delete(mon_info->sbrec_mon); + hmap_remove(svc_monitor_map, &mon_info->hmap_node); + free(mon_info); } - - free(mon_info); } - hmap_destroy(&monitor_map); } static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, @@ -4325,7 +4317,8 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) static void build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { struct ovn_datapath *od; @@ -4335,21 +4328,25 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_northd_lb *lb = - ovn_northd_lb_find(lbs, - &od->nbr->load_balancer[i]->header_.uuid); - build_lrouter_lb_reachable_ips(od, lb); + struct ovn_lb_datapaths *lb_dps = + ovn_lb_datapaths_find(lb_dps_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); } for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { const struct nbrec_load_balancer_group *nbrec_lb_group = od->nbr->load_balancer_group[i]; - struct ovn_lb_group *lb_group; - - lb_group = ovn_lb_group_find(lb_groups, - &nbrec_lb_group->header_.uuid); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, lb_group->lbs[j]); + struct ovn_lb_group_datapaths *lb_group_dps; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_dps_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lb_group_dps); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, + lb_group_dps->lb_group->lbs[j]); } } } @@ -4357,45 +4354,50 @@ build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, struct hmap *lb_groups) + struct hmap *lb_dps_map, + struct hmap *lb_group_dps_map) { if (!install_ls_lb_from_router) { return; } - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; size_t index; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; - ovn_northd_lb_add_ls(lb, od->n_ls_peers, od->ls_peers); - } - } - - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH (lb_group, hmap_node, lb_groups) { - for (size_t i = 0; i < lb_group->n_lr; i++) { - struct ovn_datapath *od = lb_group->lr[i]; - ovn_lb_group_add_ls(lb_group, od->n_ls_peers, od->ls_peers); - for (size_t j = 0; j < lb_group->n_lbs; j++) { - ovn_northd_lb_add_ls(lb_group->lbs[j], od->n_ls_peers, - od->ls_peers); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { + for (size_t i = 0; i < lb_group_dps->n_lr; i++) { + struct ovn_datapath *od = lb_group_dps->lr[i]; + ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, + od->ls_peers); + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_dps_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); } } } } static void -build_lb_count_dps(struct hmap *lbs, +build_lb_count_dps(struct hmap *lb_dps_map, size_t n_ls_datapaths, size_t n_lr_datapaths) { - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; - HMAP_FOR_EACH (lb, hmap_node, lbs) { - lb->n_nb_lr = bitmap_count1(lb->nb_lr_map, n_lr_datapaths); - lb->n_nb_ls = bitmap_count1(lb->nb_ls_map, n_ls_datapaths); + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + lb_dps->n_nb_lr = bitmap_count1(lb_dps->nb_lr_map, n_lr_datapaths); + lb_dps->n_nb_ls = bitmap_count1(lb_dps->nb_ls_map, n_ls_datapaths); } } @@ -4408,13 +4410,16 @@ build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, - struct hmap *lbs, struct hmap *lb_groups, struct sset *svc_monitor_lsps) + struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, + struct sset *svc_monitor_lsps, + struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lbs, lb_groups); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lbs, - svc_monitor_lsps); - build_lswitch_lbs_from_lrouter(lr_datapaths, lbs, lb_groups); + build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, + lb_group_dps_map); + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + svc_monitor_lsps, svc_monitor_map); + build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -4535,17 +4540,39 @@ ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, return dpg; } +struct sb_lb { + struct hmap_node hmap_node; + + const struct sbrec_load_balancer *slb; + struct ovn_dp_group *dpg; + struct uuid lb_uuid; +}; + +static struct sb_lb * +find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { + if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + /* Syncs relevant load balancers (applied to logical switches) to the * Southbound database. */ void sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, struct hmap *lbs) + struct ovn_datapaths *ls_datapaths, struct hmap *lb_dps_map) { struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); size_t bitmap_len = ods_size(ls_datapaths); - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; + struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); /* Delete any stale SB load balancer rows and create datapath * groups for existing ones. */ @@ -4568,28 +4595,32 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * "at-least-once" consistency for clustered database tables that * are not indexed in any way. */ - lb = ovn_northd_lb_find(lbs, &lb_uuid); - if (!lb || !lb->n_nb_ls || !hmapx_add(&existing_lbs, lb)) { + lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); + if (!lb_dps || !lb_dps->n_nb_ls || !hmapx_add(&existing_lbs, lb_dps)) { sbrec_load_balancer_delete(sbrec_lb); continue; } - lb->slb = sbrec_lb; + struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_uuid = lb_uuid; + sb_lb->slb = sbrec_lb; + hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); /* Find or create datapath group for this load balancer. */ - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + sb_lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sb_lb->slb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + bitmap_len, true, + ls_datapaths, NULL); } hmapx_destroy(&existing_lbs); /* Create SB Load balancer records if not present and sync * the SB load balancer columns. */ - HMAP_FOR_EACH (lb, hmap_node, lbs) { + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { continue; } @@ -4597,37 +4628,44 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, * transport port) tuple. */ struct smap options; - smap_clone(&options, &lb->nlb->options); + smap_clone(&options, &lb_dps->lb->nlb->options); smap_replace(&options, "hairpin_orig_tuple", "true"); - if (!lb->slb) { + struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, + &lb_dps->lb->nlb->header_.uuid); + ovs_assert(!sb_lb || (sb_lb->slb && sb_lb->dpg)); + struct ovn_dp_group *lb_dpg = NULL; + if (!sb_lb) { sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - lb->slb = sbrec_lb; char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb->nlb->header_.uuid)); + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); const struct smap external_ids = SMAP_CONST1(&external_ids, "lb_id", lb_id); sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); free(lb_id); + } else { + sbrec_lb = sb_lb->slb; + lb_dpg = sb_lb->dpg; } /* Find or create datapath group for this load balancer. */ - if (!lb->dpg) { - lb->dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, - lb->slb->datapath_group, - lb->n_nb_ls, lb->nb_ls_map, - bitmap_len, true, - ls_datapaths, NULL); + if (!lb_dpg) { + lb_dpg = ovn_dp_group_get_or_create(ovnsb_txn, &dp_groups, + sbrec_lb->datapath_group, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, bitmap_len, + true, ls_datapaths, NULL); } /* Update columns. */ - sbrec_load_balancer_set_name(lb->slb, lb->nlb->name); - sbrec_load_balancer_set_vips(lb->slb, ovn_northd_lb_get_vips(lb)); - sbrec_load_balancer_set_protocol(lb->slb, lb->nlb->protocol); - sbrec_load_balancer_set_datapath_group(lb->slb, lb->dpg->dp_group); - sbrec_load_balancer_set_options(lb->slb, &options); + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + sbrec_load_balancer_set_datapath_group(sbrec_lb, lb_dpg->dp_group); + sbrec_load_balancer_set_options(sbrec_lb, &options); /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(lb->slb, NULL, 0); + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); smap_destroy(&options); } @@ -4638,6 +4676,12 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } hmap_destroy(&dp_groups); + struct sb_lb *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { + free(sb_lb); + } + hmap_destroy(&sb_lbs); + /* Datapath_Binding.load_balancers is not used anymore, it's still in the * schema for compatibility reasons. Reset it to empty, just in case. */ @@ -7832,15 +7876,17 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules_pre_stateful(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; ds_clear(action); @@ -7886,7 +7932,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, } ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -7932,7 +7978,7 @@ build_lb_rules_pre_stateful(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8118,14 +8164,16 @@ build_lb_affinity_lr_flows(struct hmap *lflows, struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_affinity_ls_flows(struct hmap *lflows, + struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) { - if (!lb->affinity_timeout || !lb->n_nb_ls) { + if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; struct ds new_lb_match = DS_EMPTY_INITIALIZER; if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { ds_put_format(&new_lb_match, @@ -8145,9 +8193,9 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, static char *aff_check = REGBIT_KNOWN_LB_SESSION" = chk_lb_aff(); next;"; ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb->nlb->header_); + &lb_dps->lb->nlb->header_); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -8235,14 +8283,15 @@ build_lb_affinity_ls_flows(struct hmap *lflows, struct ovn_northd_lb *lb, /* Forward to OFTABLE_CHK_LB_AFFINITY table to store flow tuple. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), ds_cstr(&aff_action_learn), &lb->nlb->header_); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( - lflows, lb->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), + S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), + &lb->nlb->header_); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -8275,11 +8324,13 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, +build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, - struct ds *action, const struct shash *meter_groups) + struct ds *action, const struct shash *meter_groups, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -8300,9 +8351,10 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, /* New connections in Ingress table. */ const char *meter = NULL; - bool reject = build_lb_vip_actions(lb_vip, lb_vip_nb, action, - lb->selection_fields, NULL, - NULL, true, features); + bool reject = build_lb_vip_actions(lb, lb_vip, lb_vip_nb, action, + lb->selection_fields, + NULL, NULL, true, features, + svc_monitor_map); ds_put_format(match, "ct.new && %s.dst == %s", ip_match, lb_vip->vip_str); @@ -8313,15 +8365,17 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; if (reject) { size_t index; - dp_non_meter = bitmap_clone(lb->nb_ls_map, ods_size(ls_datapaths)); - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + dp_non_meter = bitmap_clone(lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; meter = copp_meter_get(COPP_REJECT, od->nbs->copp, @@ -8339,7 +8393,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_northd_lb *lb, } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( - lflows, dp_non_meter ? dp_non_meter : lb->nb_ls_map, + lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), &lb->nlb->header_); } @@ -9554,7 +9608,8 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, /* Ingress table 19: ARP/ND responder for service monitor source ip. * (priority 110)*/ static void -build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, +build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, + const struct hmap *ls_ports, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -9569,7 +9624,14 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, for (size_t j = 0; j < lb_vip_nb->n_backends; j++) { struct ovn_northd_lb_backend *backend_nb = &lb_vip_nb->backends_nb[j]; - if (!backend_nb->op || !backend_nb->svc_mon_src_ip) { + + if (!backend_nb->health_check) { + continue; + } + + struct ovn_port *op = ovn_port_find(ls_ports, + backend_nb->logical_port); + if (!op || !backend_nb->svc_mon_src_ip) { continue; } @@ -9611,7 +9673,7 @@ build_lswitch_arp_nd_service_monitor(struct ovn_northd_lb *lb, svc_monitor_mac); } ovn_lflow_add_with_hint(lflows, - backend_nb->op->od, + op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), &lb->nlb->header_); @@ -11336,7 +11398,7 @@ struct lrouter_nat_lb_flows_ctx { struct ds *gw_redir_action; struct ovn_lb_vip *lb_vip; - struct ovn_northd_lb *lb; + const struct ovn_northd_lb *lb; bool reject; int prio; @@ -11468,14 +11530,16 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, static void build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, - struct ovn_northd_lb *lb, + struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + const struct hmap *svc_monitor_map) { + const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; const char *ip_match = ipv4 ? "ip4" : "ip6"; @@ -11490,9 +11554,10 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, ds_clear(match); ds_clear(action); - bool reject = build_lb_vip_actions(lb_vip, vips_nb, action, + bool reject = build_lb_vip_actions(lb, lb_vip, vips_nb, action, lb->selection_fields, &skip_snat_act, - &force_snat_act, false, features); + &force_snat_act, false, features, + svc_monitor_map); /* Higher priority rules are added for load-balancing in DNAT * table. For every match (on a VIP[:port]), we add two flows. @@ -11567,7 +11632,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * lflow generation for them. */ size_t index; - BITMAP_FOR_EACH_1 (index, bitmap_len, lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, bitmap_len, lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; @@ -11647,16 +11712,19 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, } static void -build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { - if (!lb->n_nb_ls) { + if (!lb_dps->n_nb_ls) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; @@ -11666,7 +11734,7 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } size_t index; - BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb->nb_ls_map) { + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { struct ovn_datapath *od = ls_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, @@ -11690,10 +11758,10 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * a higher priority rule for load balancing below also commits the * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ - build_lb_rules_pre_stateful(lflows, lb, features->ct_no_masked_label, + build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, ls_datapaths, match, action); - build_lb_rules(lflows, lb, ls_datapaths, features, match, action, - meter_groups); + build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, + meter_groups, svc_monitor_map); } /* If there are any load balancing rules, we should send the packet to @@ -11705,17 +11773,17 @@ build_lswitch_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, * defragmentation to match on L4 ports. */ static void -build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, +build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } - for (size_t i = 0; i < lb->n_vips; i++) { - struct ovn_lb_vip *lb_vip = &lb->vips[i]; + for (size_t i = 0; i < lb_dps->lb->n_vips; i++) { + struct ovn_lb_vip *lb_vip = &lb_dps->lb->vips[i]; bool ipv6 = lb_vip->address_family == AF_INET6; int prio = 100; @@ -11724,36 +11792,41 @@ build_lrouter_defrag_flows_for_lb(struct ovn_northd_lb *lb, lb_vip->vip_str); ovn_lflow_add_with_dp_group( - lflows, lb->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, - prio, ds_cstr(match), "ct_dnat;", &lb->nlb->header_); + lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), + S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", + &lb_dps->lb->nlb->header_); } } static void -build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, +build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, + struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct chassis_features *features, + const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) { size_t index; - if (!lb->n_nb_lr) { + if (!lb_dps->n_nb_lr) { return; } + const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_lb_vip *lb_vip = &lb->vips[i]; - build_lrouter_nat_flows_for_lb(lb_vip, lb, &lb->vips_nb[i], + build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lflows, match, action, - meter_groups, features); + meter_groups, features, + svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; } - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add_with_hint__(lflows, od, S_ROUTER_IN_DNAT, @@ -11767,7 +11840,7 @@ build_lrouter_flows_for_lb(struct ovn_northd_lb *lb, struct hmap *lflows, } if (lb->skip_snat) { - BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb->nb_lr_map) { + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, @@ -15484,7 +15557,8 @@ struct lswitch_flow_build_info { struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_dps_map; + const struct hmap *svc_monitor_map; const struct hmap *bfd_connections; const struct chassis_features *features; char *svc_check_match; @@ -15628,7 +15702,7 @@ build_lflows_thread(void *arg) struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; int bnum; @@ -15695,28 +15769,33 @@ build_lflows_thread(void *arg) } } for (bnum = control->id; - bnum <= lsi->lbs->mask; + bnum <= lsi->lb_dps_map->mask; bnum += control->pool->size) { - HMAP_FOR_EACH_IN_PARALLEL (lb, hmap_node, bnum, lsi->lbs) { + HMAP_FOR_EACH_IN_PARALLEL (lb_dps, hmap_node, bnum, + lsi->lb_dps_map) { if (stop_parallel_processing()) { return NULL; } - build_lswitch_arp_nd_service_monitor(lb, lsi->lflows, + build_lswitch_arp_nd_service_monitor(lb_dps->lb, + lsi->ls_ports, + lsi->lflows, &lsi->match, &lsi->actions); - build_lrouter_defrag_flows_for_lb(lb, lsi->lflows, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match); - build_lrouter_flows_for_lb(lb, lsi->lflows, + build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); - build_lswitch_flows_for_lb(lb, lsi->lflows, + build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, + lsi->svc_monitor_map, &lsi->match, &lsi->actions); } } @@ -15782,7 +15861,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, - const struct hmap *lbs, + const struct hmap *lb_dps_map, + const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, const struct chassis_features *features) { @@ -15809,7 +15889,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].port_groups = port_groups; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; - lsiv[index].lbs = lbs; + lsiv[index].lb_dps_map = lb_dps_map; + lsiv[index].svc_monitor_map = svc_monitor_map; lsiv[index].bfd_connections = bfd_connections; lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; @@ -15832,7 +15913,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } else { struct ovn_datapath *od; struct ovn_port *op; - struct ovn_northd_lb *lb; + struct ovn_lb_datapaths *lb_dps; struct ovn_igmp_group *igmp_group; struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, @@ -15843,7 +15924,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, - .lbs = lbs, + .lb_dps_map = lb_dps_map, + .svc_monitor_map = svc_monitor_map, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, @@ -15875,17 +15957,19 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); - HMAP_FOR_EACH (lb, hmap_node, lbs) { - build_lswitch_arp_nd_service_monitor(lb, lsi.lflows, - &lsi.actions, + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.lflows, &lsi.actions, &lsi.match); - build_lrouter_defrag_flows_for_lb(lb, lsi.lflows, lsi.lr_datapaths, - &lsi.match); - build_lrouter_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, + lsi.lr_datapaths, &lsi.match); + build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); - build_lswitch_flows_for_lb(lb, lsi.lflows, lsi.meter_groups, + build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, + lsi.svc_monitor_map, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15985,7 +16069,9 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->port_groups, lflows, &igmp_groups, - input_data->meter_groups, input_data->lbs, + input_data->meter_groups, + input_data->lb_datapaths_map, + input_data->svc_monitor_map, input_data->bfd_connections, input_data->features); @@ -17388,8 +17474,8 @@ northd_init(struct northd_data *data) hmap_init(&data->lr_ports); hmap_init(&data->port_groups); shash_init(&data->meter_groups); - hmap_init(&data->lbs); - hmap_init(&data->lb_groups); + hmap_init(&data->lb_datapaths_map); + hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); data->features = (struct chassis_features) { .ct_no_masked_label = true, @@ -17399,6 +17485,7 @@ northd_init(struct northd_data *data) }; data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); + hmap_init(&data->svc_monitor_map); data->change_tracked = false; ovs_list_init(&data->tracked_ls_changes.updated); } @@ -17406,17 +17493,18 @@ northd_init(struct northd_data *data) void northd_destroy(struct northd_data *data) { - struct ovn_northd_lb *lb; - HMAP_FOR_EACH_POP (lb, hmap_node, &data->lbs) { - ovn_northd_lb_destroy(lb); + struct ovn_lb_datapaths *lb_dps; + HMAP_FOR_EACH_POP (lb_dps, hmap_node, &data->lb_datapaths_map) { + ovn_lb_datapaths_destroy(lb_dps); } - hmap_destroy(&data->lbs); + hmap_destroy(&data->lb_datapaths_map); - struct ovn_lb_group *lb_group; - HMAP_FOR_EACH_POP (lb_group, hmap_node, &data->lb_groups) { - ovn_lb_group_destroy(lb_group); + struct ovn_lb_group_datapaths *lb_group_dps; + HMAP_FOR_EACH_POP (lb_group_dps, hmap_node, + &data->lb_group_datapaths_map) { + ovn_lb_group_datapaths_destroy(lb_group_dps); } - hmap_destroy(&data->lb_groups); + hmap_destroy(&data->lb_group_datapaths_map); struct ovn_port_group *pg; HMAP_FOR_EACH_SAFE (pg, key_node, &data->port_groups) { @@ -17431,6 +17519,12 @@ northd_destroy(struct northd_data *data) } shash_destroy(&data->meter_groups); + struct service_monitor_info *mon_info; + HMAP_FOR_EACH_POP (mon_info, hmap_node, &data->svc_monitor_map) { + free(mon_info); + } + hmap_destroy(&data->svc_monitor_map); + /* XXX Having to explicitly clean up macam here * is a bit strange. We don't explicitly initialize * macam in this module, but this is the logical place @@ -17539,10 +17633,9 @@ ovnnb_db_run(struct northd_input *input_data, input_data->sbrec_chassis_table, &data->ls_datapaths, &data->lr_datapaths, &data->lr_list); - build_lbs(input_data->nbrec_load_balancer_table, - input_data->nbrec_load_balancer_group_table, - &data->ls_datapaths, &data->lr_datapaths, &data->lbs, - &data->lb_groups); + build_lb_datapaths(input_data->lbs, input_data->lb_groups, + &data->ls_datapaths, &data->lr_datapaths, + &data->lb_datapaths_map, &data->lb_group_datapaths_map); build_ports(ovnsb_txn, input_data->sbrec_port_binding_table, input_data->sbrec_chassis_table, @@ -17557,9 +17650,11 @@ ovnnb_db_run(struct northd_input *input_data, build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, &data->lr_datapaths, &data->ls_ports, - &data->lbs, &data->lb_groups, - &data->svc_monitor_lsps); - build_lb_count_dps(&data->lbs, + &data->lb_datapaths_map, + &data->lb_group_datapaths_map, + &data->svc_monitor_lsps, + &data->svc_monitor_map); + build_lb_count_dps(&data->lb_datapaths_map, ods_size(&data->ls_datapaths), ods_size(&data->lr_datapaths)); build_ipam(&data->ls_datapaths.datapaths, &data->ls_ports); diff --git a/northd/northd.h b/northd/northd.h index 48c282476a..7d92028c7d 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -28,9 +28,6 @@ struct northd_input { const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; - const struct nbrec_load_balancer_table *nbrec_load_balancer_table; - const struct nbrec_load_balancer_group_table - *nbrec_load_balancer_group_table; const struct nbrec_port_group_table *nbrec_port_group_table; const struct nbrec_meter_table *nbrec_meter_table; const struct nbrec_acl_table *nbrec_acl_table; @@ -59,6 +56,10 @@ struct northd_input { *sbrec_chassis_template_var_table; const struct sbrec_mirror_table *sbrec_mirror_table; + /* Northd lb data node inputs*/ + const struct hmap *lbs; + const struct hmap *lb_groups; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -110,12 +111,13 @@ struct northd_data { struct hmap lr_ports; struct hmap port_groups; struct shash meter_groups; - struct hmap lbs; - struct hmap lb_groups; + struct hmap lb_datapaths_map; + struct hmap lb_group_datapaths_map; struct ovs_list lr_list; bool ovn_internal_version_changed; struct chassis_features features; struct sset svc_monitor_lsps; + struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; }; @@ -146,9 +148,10 @@ struct lflow_input { const struct hmap *lr_ports; const struct hmap *port_groups; const struct shash *meter_groups; - const struct hmap *lbs; + const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; const struct chassis_features *features; + const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; }; From patchwork Fri Jul 7 05:53:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804605 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2dq1Yvyz20b8 for ; Fri, 7 Jul 2023 15:54:07 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id EE28E41955; Fri, 7 Jul 2023 05:54:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org EE28E41955 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dGHTIvNQyelQ; Fri, 7 Jul 2023 05:54:02 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8172C4193B; Fri, 7 Jul 2023 05:54:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8172C4193B Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4E8A4C0072; Fri, 7 Jul 2023 05:54:01 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id C9C5BC0032 for ; Fri, 7 Jul 2023 05:53:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 616CD60590 for ; Fri, 7 Jul 2023 05:53:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 616CD60590 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L9-owLeSaryc for ; Fri, 7 Jul 2023 05:53:54 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2B1806113E Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::224]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2B1806113E for ; Fri, 7 Jul 2023 05:53:53 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id F198FE0008; Fri, 7 Jul 2023 05:53:49 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:23:43 +0530 Message-Id: <20230707055343.961653-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 3/8] northd: Add initial I-P for load balancer and load balancer groups X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to load balancers and load balancer groups are handled incrementally in the newly added 'northd_lb_data' engine node. 'northd_lb_data' is input to 'northd' node and the handler - northd_northd_lb_data_handler in 'northd' node handles the changes. If a load balancer or load balancer group is associated to a logical switch or router then 'northd' node falls back to full recompute. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- lib/lb.c | 85 ++++++++++++---- lib/lb.h | 9 ++ northd/en-northd-lb-data.c | 202 +++++++++++++++++++++++++++++++++++-- northd/en-northd-lb-data.h | 37 +++++++ northd/en-northd.c | 24 +++++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 13 ++- northd/northd.c | 77 ++++++++++++++ northd/northd.h | 7 ++ tests/ovn-northd.at | 123 ++++++++++++++++++++++ 10 files changed, 545 insertions(+), 33 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index 429dbf15af..a51fe225fa 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -606,13 +606,13 @@ ovn_lb_get_health_check(const struct nbrec_load_balancer *nbrec_lb, return NULL; } -struct ovn_northd_lb * -ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +static void +ovn_northd_lb_init(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { bool template = smap_get_bool(&nbrec_lb->options, "template", false); bool is_udp = nullable_string_is_equal(nbrec_lb->protocol, "udp"); bool is_sctp = nullable_string_is_equal(nbrec_lb->protocol, "sctp"); - struct ovn_northd_lb *lb = xzalloc(sizeof *lb); int address_family = !strcmp(smap_get_def(&nbrec_lb->options, "address-family", "ipv4"), "ipv4") @@ -668,6 +668,10 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) "reject", false); ovn_northd_lb_vip_init(lb_vip_nb, lb_vip, nbrec_lb, node->key, node->value, template); + if (lb_vip_nb->lb_health_check) { + lb->health_checks = true; + } + if (IN6_IS_ADDR_V4MAPPED(&lb_vip->vip)) { sset_add(&lb->ips_v4, lb_vip->vip_str); } else { @@ -713,6 +717,13 @@ ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) ds_chomp(&sel_fields, ','); lb->selection_fields = ds_steal_cstr(&sel_fields); } +} + +struct ovn_northd_lb * +ovn_northd_lb_create(const struct nbrec_load_balancer *nbrec_lb) +{ + struct ovn_northd_lb *lb = xzalloc(sizeof *lb); + ovn_northd_lb_init(lb, nbrec_lb); return lb; } @@ -738,8 +749,8 @@ ovn_northd_lb_get_vips(const struct ovn_northd_lb *lb) return &lb->nlb->vips; } -void -ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +static void +ovn_northd_lb_cleanup(struct ovn_northd_lb *lb) { for (size_t i = 0; i < lb->n_vips; i++) { ovn_lb_vip_destroy(&lb->vips[i]); @@ -747,26 +758,36 @@ ovn_northd_lb_destroy(struct ovn_northd_lb *lb) } free(lb->vips); free(lb->vips_nb); + lb->vips = NULL; + lb->vips_nb = NULL; smap_destroy(&lb->template_vips); sset_destroy(&lb->ips_v4); sset_destroy(&lb->ips_v6); free(lb->selection_fields); + lb->selection_fields = NULL; + lb->health_checks = false; +} + +void +ovn_northd_lb_destroy(struct ovn_northd_lb *lb) +{ + ovn_northd_lb_cleanup(lb); free(lb); } -/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record - * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will - * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' - * logical routers to which this LB Group is applied. Can be filled later - * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ -struct ovn_lb_group * -ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, - const struct hmap *lbs) +void +ovn_northd_lb_reinit(struct ovn_northd_lb *lb, + const struct nbrec_load_balancer *nbrec_lb) { - struct ovn_lb_group *lb_group; + ovn_northd_lb_cleanup(lb); + ovn_northd_lb_init(lb, nbrec_lb); +} - lb_group = xzalloc(sizeof *lb_group); - lb_group->uuid = nbrec_lb_group->header_.uuid; +static void +ovn_lb_group_init(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ lb_group->n_lbs = nbrec_lb_group->n_load_balancer; lb_group->lbs = xmalloc(lb_group->n_lbs * sizeof *lb_group->lbs); lb_group->lb_ips = ovn_lb_ip_set_create(); @@ -776,10 +797,30 @@ ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); } +} +/* Constructs a new 'struct ovn_lb_group' object from the Nb LB Group record + * and a hash map of all existing 'struct ovn_northd_lb' objects. Space will + * be allocated for 'max_ls_datapaths' logical switches and 'max_lr_datapaths' + * logical routers to which this LB Group is applied. Can be filled later + * with ovn_lb_group_add_ls() and ovn_lb_group_add_lr() respectively. */ +struct ovn_lb_group * +ovn_lb_group_create(const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + struct ovn_lb_group *lb_group = xzalloc(sizeof *lb_group); + lb_group->uuid = nbrec_lb_group->header_.uuid; + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); return lb_group; } +static void +ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) +{ + ovn_lb_ip_set_destroy(lb_group->lb_ips); + free(lb_group->lbs); +} + void ovn_lb_group_destroy(struct ovn_lb_group *lb_group) { @@ -787,11 +828,19 @@ ovn_lb_group_destroy(struct ovn_lb_group *lb_group) return; } - ovn_lb_ip_set_destroy(lb_group->lb_ips); - free(lb_group->lbs); + ovn_lb_group_cleanup(lb_group); free(lb_group); } +void +ovn_lb_group_reinit(struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *nbrec_lb_group, + const struct hmap *lbs) +{ + ovn_lb_group_cleanup(lb_group); + ovn_lb_group_init(lb_group, nbrec_lb_group, lbs); +} + struct ovn_lb_group * ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *uuid) { diff --git a/lib/lb.h b/lib/lb.h index 0339050cba..74905c73b7 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -77,6 +77,9 @@ struct ovn_northd_lb { struct sset ips_v4; struct sset ips_v6; + + /* Indicates if the load balancer has health checks configured. */ + bool health_checks; }; struct ovn_lb_vip { @@ -130,6 +133,8 @@ struct ovn_northd_lb *ovn_northd_lb_find(const struct hmap *, const struct uuid *); const struct smap *ovn_northd_lb_get_vips(const struct ovn_northd_lb *); void ovn_northd_lb_destroy(struct ovn_northd_lb *); +void ovn_northd_lb_reinit(struct ovn_northd_lb *, + const struct nbrec_load_balancer *); void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); @@ -148,6 +153,10 @@ struct ovn_lb_group *ovn_lb_group_create( void ovn_lb_group_destroy(struct ovn_lb_group *lb_group); struct ovn_lb_group *ovn_lb_group_find(const struct hmap *lb_groups, const struct uuid *); +void ovn_lb_group_reinit( + struct ovn_lb_group *lb_group, + const struct nbrec_load_balancer_group *, + const struct hmap *lbs); struct ovn_lb_datapaths { struct hmap_node hmap_node; diff --git a/northd/en-northd-lb-data.c b/northd/en-northd-lb-data.c index d46c3c27ed..e52535b4a9 100644 --- a/northd/en-northd-lb-data.c +++ b/northd/en-northd-lb-data.c @@ -37,6 +37,15 @@ static void northd_lb_data_destroy(struct northd_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static struct ovn_lb_group *create_lb_group( + const struct nbrec_load_balancer_group *, struct hmap *lbs, + struct hmap *lb_groups); +static void destroy_tracked_data(struct northd_lb_data *); +static void add_lb_to_tracked_data(struct ovn_northd_lb *, + struct ovs_list *tracked_list, + bool health_checks); +static void add_lb_group_to_tracked_data(struct ovn_lb_group *, + struct ovs_list *tracked_list); void * en_northd_lb_data_init(struct engine_node *node OVS_UNUSED, @@ -61,6 +70,7 @@ en_northd_lb_data_run(struct engine_node *node, void *data) const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lb_groups); engine_set_node_state(node, EN_UPDATED); } @@ -72,12 +82,122 @@ en_northd_lb_data_cleanup(void *data) northd_lb_data_destroy(lb_data); } +void +en_northd_lb_data_clear_tracked_data(void *data) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + destroy_tracked_data(lb_data); +} + + +/* Handler functions. */ +bool +northd_lb_data_load_balancer_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + const struct nbrec_load_balancer_table *nb_lb_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); + + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + + const struct nbrec_load_balancer *tracked_lb; + NBREC_LOAD_BALANCER_TABLE_FOR_EACH_TRACKED (tracked_lb, nb_lb_table) { + struct ovn_northd_lb *lb; + if (nbrec_load_balancer_is_new(tracked_lb)) { + /* New load balancer. */ + lb = ovn_northd_lb_create(tracked_lb); + hmap_insert(&lb_data->lbs, &lb->hmap_node, + uuid_hash(&tracked_lb->header_.uuid)); + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_updated_lbs.updated, + lb->health_checks); + } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + hmap_remove(&lb_data->lbs, &lb->hmap_node); + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_deleted_lbs.updated, + lb->health_checks); + } else { + /* Load balancer updated. */ + lb = ovn_northd_lb_find(&lb_data->lbs, + &tracked_lb->header_.uuid); + ovs_assert(lb); + bool health_checks = lb->health_checks; + ovn_northd_lb_reinit(lb, tracked_lb); + health_checks |= lb->health_checks; + add_lb_to_tracked_data( + lb, &trk_lb_data->tracked_updated_lbs.updated, health_checks); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +northd_lb_data_load_balancer_group_handler(struct engine_node *node, + void *data OVS_UNUSED) +{ + struct northd_lb_data *lb_data = (struct northd_lb_data *) data; + const struct nbrec_load_balancer_group_table *nb_lbg_table = + EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + + lb_data->tracked = true; + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct nbrec_load_balancer_group *tracked_lb_group; + NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH_TRACKED (tracked_lb_group, + nb_lbg_table) { + if (nbrec_load_balancer_group_is_new(tracked_lb_group)) { + struct ovn_lb_group *lb_group = + create_lb_group(tracked_lb_group, &lb_data->lbs, + &lb_data->lb_groups); + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_updated_lb_groups.updated); + } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + hmap_remove(&lb_data->lb_groups, &lb_group->hmap_node); + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_deleted_lb_groups.updated); + } else if (nbrec_load_balancer_group_is_updated(tracked_lb_group, + NBREC_LOAD_BALANCER_GROUP_COL_LOAD_BALANCER)) { + + struct ovn_lb_group *lb_group; + lb_group = ovn_lb_group_find(&lb_data->lb_groups, + &tracked_lb_group->header_.uuid); + ovs_assert(lb_group); + ovn_lb_group_reinit(lb_group, tracked_lb_group, &lb_data->lbs); + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + add_lb_group_to_tracked_data( + lb_group, &trk_lb_data->tracked_updated_lb_groups.updated); + } + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + /* static functions. */ static void northd_lb_data_init(struct northd_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lb_groups); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + ovs_list_init(&trk_lb_data->tracked_updated_lbs.updated); + ovs_list_init(&trk_lb_data->tracked_deleted_lbs.updated); + ovs_list_init(&trk_lb_data->tracked_updated_lb_groups.updated); + ovs_list_init(&trk_lb_data->tracked_deleted_lb_groups.updated); } static void @@ -94,6 +214,8 @@ northd_lb_data_destroy(struct northd_lb_data *lb_data) ovn_lb_group_destroy(lb_group); } hmap_destroy(&lb_data->lb_groups); + + destroy_tracked_data(lb_data); } static void @@ -101,12 +223,9 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group_table *nbrec_lb_group_table, struct hmap *lbs, struct hmap *lb_groups) { - struct ovn_lb_group *lb_group; - struct ovn_northd_lb *lb_nb; - const struct nbrec_load_balancer *nbrec_lb; NBREC_LOAD_BALANCER_TABLE_FOR_EACH (nbrec_lb, nbrec_load_balancer_table) { - lb_nb = ovn_northd_lb_create(nbrec_lb); + struct ovn_northd_lb *lb_nb = ovn_northd_lb_create(nbrec_lb); hmap_insert(lbs, &lb_nb->hmap_node, uuid_hash(&nbrec_lb->header_.uuid)); } @@ -114,13 +233,76 @@ build_lbs(const struct nbrec_load_balancer_table *nbrec_load_balancer_table, const struct nbrec_load_balancer_group *nbrec_lb_group; NBREC_LOAD_BALANCER_GROUP_TABLE_FOR_EACH (nbrec_lb_group, nbrec_lb_group_table) { - lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + create_lb_group(nbrec_lb_group, lbs, lb_groups); + } +} - for (size_t i = 0; i < lb_group->n_lbs; i++) { - build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); - } +static struct ovn_lb_group * +create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, + struct hmap *lbs, struct hmap *lb_groups) +{ + struct ovn_lb_group *lb_group = ovn_lb_group_create(nbrec_lb_group, lbs); + + for (size_t i = 0; i < lb_group->n_lbs; i++) { + build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); + } + + hmap_insert(lb_groups, &lb_group->hmap_node, + uuid_hash(&lb_group->uuid)); + + return lb_group; +} + +static void +destroy_tracked_data(struct northd_lb_data *lb_data) +{ + lb_data->tracked = false; + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct tracked_lb *tracked_lb; + LIST_FOR_EACH_SAFE (tracked_lb, list_node, + &trk_lb_data->tracked_updated_lbs.updated) { + ovs_list_remove(&tracked_lb->list_node); + free(tracked_lb); + } + + LIST_FOR_EACH_SAFE (tracked_lb, list_node, + &trk_lb_data->tracked_deleted_lbs.updated) { + ovs_list_remove(&tracked_lb->list_node); + ovn_northd_lb_destroy(tracked_lb->lb); + free(tracked_lb); + } + + struct tracked_lb_group *tracked_lb_group; + LIST_FOR_EACH_SAFE (tracked_lb_group, list_node, + &trk_lb_data->tracked_updated_lb_groups.updated) { + ovs_list_remove(&tracked_lb_group->list_node); + free(tracked_lb_group); + } - hmap_insert(lb_groups, &lb_group->hmap_node, - uuid_hash(&lb_group->uuid)); + LIST_FOR_EACH_SAFE (tracked_lb_group, list_node, + &trk_lb_data->tracked_deleted_lb_groups.updated) { + ovs_list_remove(&tracked_lb_group->list_node); + ovn_lb_group_destroy(tracked_lb_group->lb_group); + free(tracked_lb_group); } } + +static void +add_lb_to_tracked_data(struct ovn_northd_lb *lb, struct ovs_list *tracked_list, + bool health_checks) +{ + struct tracked_lb *t_lb = xzalloc(sizeof *t_lb); + t_lb->lb = lb; + t_lb->health_checks = health_checks; + ovs_list_push_back(tracked_list, &t_lb->list_node); +} + +static void +add_lb_group_to_tracked_data(struct ovn_lb_group *lb_group, + struct ovs_list *tracked_list) +{ + struct tracked_lb_group *t_lb_group = xzalloc(sizeof *t_lb_group); + t_lb_group->lb_group = lb_group; + ovs_list_push_back(tracked_list, &t_lb_group->list_node); +} diff --git a/northd/en-northd-lb-data.h b/northd/en-northd-lb-data.h index eb297e376d..bb07fcf686 100644 --- a/northd/en-northd-lb-data.h +++ b/northd/en-northd-lb-data.h @@ -4,16 +4,53 @@ #include #include "openvswitch/hmap.h" +#include "include/openvswitch/list.h" #include "lib/inc-proc-eng.h" +struct ovn_northd_lb; +struct ovn_lb_group; + +struct tracked_lb { + struct ovs_list list_node; + struct ovn_northd_lb *lb; + bool health_checks; +}; + +struct tracked_lb_group { + struct ovs_list list_node; + struct ovn_lb_group *lb_group; +}; + +struct tracked_lb_changes { + struct ovs_list updated; /* contains list of 'struct tracked_lb ' or + list of 'struct tracked_lb_group' */ +}; + +struct tracked_lb_data { + struct tracked_lb_changes tracked_updated_lbs; + struct tracked_lb_changes tracked_deleted_lbs; + struct tracked_lb_changes tracked_updated_lb_groups; + struct tracked_lb_changes tracked_deleted_lb_groups; +}; + struct northd_lb_data { struct hmap lbs; struct hmap lb_groups; + + /* tracked data*/ + bool tracked; + struct tracked_lb_data tracked_lb_data; }; void *en_northd_lb_data_init(struct engine_node *, struct engine_arg *); void en_northd_lb_data_run(struct engine_node *, void *data); void en_northd_lb_data_cleanup(void *data); +void en_northd_lb_data_clear_tracked_data(void *data); + +bool northd_lb_data_load_balancer_handler(struct engine_node *, + void *data); +bool northd_lb_data_load_balancer_group_handler(struct engine_node *, + void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index cc7d838451..b3c03c54bd 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -206,6 +206,30 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_northd_lb_data_handler(struct engine_node *node, void *data) +{ + struct northd_lb_data *lb_data = + engine_get_input_data("northd_lb_data", node); + + if (!lb_data->tracked) { + return false; + } + + struct northd_data *nd = data; + + if (!northd_handle_lb_data_changes(&lb_data->tracked_lb_data, + &nd->ls_datapaths, + &nd->lr_datapaths, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 20cc77f108..5674f4390c 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -17,5 +17,6 @@ void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); +bool northd_northd_lb_data_handler(struct engine_node *, void *data); #endif /* EN_NORTHD_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index b2e884962f..b444a488db 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,13 +141,18 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); -static ENGINE_NODE(northd_lb_data, "northd_lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) { /* Define relationships between nodes where first argument is dependent * on the second argument */ + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, + northd_lb_data_load_balancer_handler); + engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, + northd_lb_data_load_balancer_group_handler); + engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); engine_add_input(&en_northd, &en_nb_logical_router, NULL); @@ -171,6 +176,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + engine_add_input(&en_northd, &en_northd_lb_data, + northd_northd_lb_data_handler); engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_nb_global, @@ -178,10 +185,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); - engine_add_input(&en_northd_lb_data, &en_nb_load_balancer, NULL); - engine_add_input(&en_northd_lb_data, &en_nb_load_balancer_group, NULL); - engine_add_input(&en_northd, &en_northd_lb_data, NULL); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); diff --git a/northd/northd.c b/northd/northd.c index 890186b29c..f27f0de49b 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/lb.h" #include "memory.h" #include "northd.h" +#include "en-northd-lb-data.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -5323,6 +5324,82 @@ northd_handle_sb_port_binding_changes( return true; } +bool +northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map) +{ + struct tracked_lb *trk_lb; + + struct ovn_lb_datapaths *lb_dps; + LIST_FOR_EACH (trk_lb, list_node, + &trk_lb_data->tracked_deleted_lbs.updated) { + if (trk_lb->health_checks) { + /* Fall back to recompute if the deleted load balancer has + * health checks configured. */ + return false; + } + + const struct uuid *lb_uuid = + &trk_lb->lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); + ovn_lb_datapaths_destroy(lb_dps); + } + + LIST_FOR_EACH (trk_lb, list_node, + &trk_lb_data->tracked_updated_lbs.updated) { + if (trk_lb->health_checks) { + /* Fall back to recompute if the created/updated load balancer has + * health checks configured. */ + return false; + } + + const struct uuid *lb_uuid = + &trk_lb->lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (!lb_dps) { + lb_dps = ovn_lb_datapaths_create(trk_lb->lb, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + struct ovn_lb_group_datapaths *lb_group_dps; + struct tracked_lb_group *trk_lb_group; + LIST_FOR_EACH_SAFE (trk_lb_group, list_node, + &trk_lb_data->tracked_deleted_lb_groups.updated) { + const struct uuid *lb_uuid = &trk_lb_group->lb_group->uuid; + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + ovs_assert(lb_group_dps); + hmap_remove(lb_group_datapaths_map, &lb_group_dps->hmap_node); + ovn_lb_group_datapaths_destroy(lb_group_dps); + } + + LIST_FOR_EACH_SAFE (trk_lb_group, list_node, + &trk_lb_data->tracked_updated_lb_groups.updated) { + const struct uuid *lb_uuid = &trk_lb_group->lb_group->uuid; + lb_group_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_uuid); + if (!lb_group_dps) { + lb_group_dps = ovn_lb_group_datapaths_create( + trk_lb_group->lb_group, ods_size(ls_datapaths), + ods_size(lr_datapaths)); + hmap_insert(lb_group_datapaths_map, &lb_group_dps->hmap_node, + uuid_hash(lb_uuid)); + } + } + + return true; +} + struct multicast_group { const char *name; uint16_t key; /* OVN_MIN_MULTICAST...OVN_MAX_MULTICAST. */ diff --git a/northd/northd.h b/northd/northd.h index 7d92028c7d..7d17921fa2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -347,6 +347,13 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); +struct tracked_lb_data; +bool northd_handle_lb_data_changes(struct tracked_lb_data *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct hmap *lb_datapaths_map, + struct hmap *lb_group_datapaths_map); + void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, const struct sbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index e79d33b2ae..dd0bd8b36a 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9681,3 +9681,126 @@ AT_CHECK([grep "lr_in_gw_redirect" R1flows |sed s'/table=../table=??/' |sort], [ AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing]) +ovn_start + +check_engine_stats() { + northd_comp=$1 + lflow_comp=$2 + + AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd_lb_data recompute], [0], [0 +]) + + northd_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd recompute) + if [[ "$northd_comp" == "norecompute" ]]; then + check test "$northd_recompute_ct" -eq "0" + else + check test "$northd_recompute_ct" -ne "0" + fi + + lflow_recompute_ct=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) + if [[ "$lflow_comp" == "norecompute" ]]; then + check test "$lflow_recompute_ct" -eq "0" + else + check test "$lflow_recompute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb set load_balancer . options:foo=bar +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 +check_engine_stats norecompute recompute + +check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 +check_engine_stats norecompute recompute + +AT_CHECK([ovn-nbctl --wait=sb \ + -- --id=@hc create Load_Balancer_Health_Check vip="10.0.0.10\:80" \ + options:failure_count=100 \ + -- add Load_Balancer . health_check @hc | uuidfilt], [0], [<0> +]) +check_engine_stats recompute recompute + +# Any change to load balancer health check should also result in full recompute +# of northd node (but not northd_lb_data node) +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer_health_check . options:foo=bar1 +check_engine_stats recompute recompute + +# Delete the health check from the load balancer. northd engine node should do a full recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear Load_Balancer . health_check +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl ls-add sw0 +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats recompute recompute + +# Associate lb1 to sw0. There should be a full recompute of northd engine node +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats recompute recompute + +# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats recompute recompute + +# Test load balancer group now +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) +check_engine_stats norecompute recompute + +lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) + +# Add lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear load_balancer_group . load_Balancer +check_engine_stats norecompute recompute + +# Add back lb to the lbg1 group +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid +check_engine_stats recompute recompute + +# Update lb and this should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set load_balancer . options:bar=foo +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_switch sw0 load_balancer_group +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid +check_engine_stats recompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl clear logical_router lr0 load_balancer_group +check_engine_stats recompute recompute + +AT_CLEANUP +]) From patchwork Fri Jul 7 05:54:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804606 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2fD1wxdz20b8 for ; Fri, 7 Jul 2023 15:54:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5D11E40AAE; Fri, 7 Jul 2023 05:54:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5D11E40AAE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CjGfNsZ9HYe1; Fri, 7 Jul 2023 05:54:24 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id B4D6B4118A; Fri, 7 Jul 2023 05:54:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B4D6B4118A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 892E2C0072; Fri, 7 Jul 2023 05:54:21 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id B8FFBC0032 for ; Fri, 7 Jul 2023 05:54:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5B61382083 for ; Fri, 7 Jul 2023 05:54:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5B61382083 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8-A11ntZN1Ar for ; Fri, 7 Jul 2023 05:54:14 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 13A1381F48 Received: from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net [217.70.183.193]) by smtp1.osuosl.org (Postfix) with ESMTPS id 13A1381F48 for ; Fri, 7 Jul 2023 05:54:13 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id E504C240003; Fri, 7 Jul 2023 05:54:09 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:24:03 +0530 Message-Id: <20230707055403.961690-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 4/8] northd: Refactor the 'northd' node code which handles logical switch changes. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This will help in handling other column changes of a logical switch. Signed-off-by: Numan Siddique Acked-by: Han Zhou --- northd/northd.c | 260 ++++++++++++++++++++++++++++-------------------- 1 file changed, 151 insertions(+), 109 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index f27f0de49b..fc98ab7bfd 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5024,8 +5024,14 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, return op; } +/* Returns true if the logical switch has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - logical switch ports. + */ static bool -check_ls_changes_other_than_lsp(const struct nbrec_logical_switch *ls) +check_unsupported_inc_proc_for_ls_changes( + const struct nbrec_logical_switch *ls) { /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; @@ -5119,6 +5125,146 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) return false; } +static bool +ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od) +{ + bool ls_ports_changed = false; + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_PORTS)) { + + for (size_t i = 0; i < changed_ls->n_ports; i++) { + if (nbrec_logical_switch_port_row_get_seqno( + changed_ls->ports[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_ports_changed = true; + break; + } + } + } else { + ls_ports_changed = true; + } + + if (!ls_ports_changed) { + return true; + } + + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + op->visited = false; + } + + /* Compare the individual ports in the old and new Logical Switches */ + for (size_t j = 0; j < changed_ls->n_ports; ++j) { + struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; + op = ovn_port_find_in_datapath(od, new_nbsp->name); + + if (!op) { + if (!lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, NULL, NULL, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->added_ports, + &op->list); + } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { + /* Existing port updated */ + bool temp = false; + if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || + !op->lsp_can_be_inc_processed || + !lsp_can_be_inc_processed(new_nbsp)) { + goto fail; + } + const struct sbrec_port_binding *sb = op->sb; + if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { + /* This port is used for svc monitor, which may be impacted + * by this change. Fallback to recompute. */ + goto fail; + } + if (!check_lsp_is_up && + !check_lsp_changes_other_than_up(new_nbsp)) { + /* If the only change is the "up" column while the + * "ignore_lsp_down" is set to true, just ignore this + * change. */ + op->visited = true; + continue; + } + struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); + ovs_list_splice(&lflows, op->lflows.next, &op->lflows); + ovn_port_destroy(&nd->ls_ports, op); + op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, + new_nbsp->name, new_nbsp, od, sb, &lflows, + ni->sbrec_mirror_table, + ni->sbrec_chassis_table, + ni->sbrec_chassis_by_name, + ni->sbrec_chassis_by_hostname); + ovs_assert(ovs_list_is_empty(&lflows)); + if (!op) { + goto fail; + } + ovs_list_push_back(&ls_change->updated_ports, &op->list); + } + op->visited = true; + } + + /* Check for deleted ports */ + HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { + if (!op->visited) { + if (!op->lsp_can_be_inc_processed) { + goto fail_clean_deleted; + } + if (sset_contains(&nd->svc_monitor_lsps, op->key)) { + /* This port was used for svc monitor, which may be + * impacted by this deletion. Fallback to recompute. */ + goto fail_clean_deleted; + } + ovs_list_push_back(&ls_change->deleted_ports, + &op->list); + hmap_remove(&nd->ls_ports, &op->key_node); + hmap_remove(&od->ports, &op->dp_node); + sbrec_port_binding_delete(op->sb); + delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, + op->tunnel_key); + } + } + + if (!ovs_list_is_empty(&ls_change->added_ports) || + !ovs_list_is_empty(&ls_change->updated_ports) || + !ovs_list_is_empty(&ls_change->deleted_ports)) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } + + return true; + +fail_clean_deleted: + LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { + ovn_port_destroy_orphan(op); + } + +fail: + free(ls_change); + return false; +} + + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, @@ -5129,12 +5275,9 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; - struct ovn_port *op; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { - ls_change = NULL; if (nbrec_logical_switch_is_new(changed_ls) || nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; @@ -5151,108 +5294,13 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } /* Now only able to handle lsp changes. */ - if (check_ls_changes_other_than_lsp(changed_ls)) { + if (check_unsupported_inc_proc_for_ls_changes(changed_ls)) { goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - - HMAP_FOR_EACH (op, dp_node, &od->ports) { - op->visited = false; - } - - /* Compare the individual ports in the old and new Logical Switches */ - for (size_t j = 0; j < changed_ls->n_ports; ++j) { - struct nbrec_logical_switch_port *new_nbsp = changed_ls->ports[j]; - op = ovn_port_find_in_datapath(od, new_nbsp->name); - - if (!op) { - if (!lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->added_ports, - &op->list); - } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { - /* Existing port updated */ - bool temp = false; - if (lsp_is_type_changed(op->sb, new_nbsp, &temp) || - !op->lsp_can_be_inc_processed || - !lsp_can_be_inc_processed(new_nbsp)) { - goto fail; - } - const struct sbrec_port_binding *sb = op->sb; - if (sset_contains(&nd->svc_monitor_lsps, new_nbsp->name)) { - /* This port is used for svc monitor, which may be impacted - * by this change. Fallback to recompute. */ - goto fail; - } - if (!check_lsp_is_up && - !check_lsp_changes_other_than_up(new_nbsp)) { - /* If the only change is the "up" column while the - * "ignore_lsp_down" is set to true, just ignore this - * change. */ - op->visited = true; - continue; - } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, - ni->sbrec_chassis_table, - ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { - goto fail; - } - ovs_list_push_back(&ls_change->updated_ports, &op->list); - } - op->visited = true; - } - - /* Check for deleted ports */ - HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { - if (!op->visited) { - if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; - } - if (sset_contains(&nd->svc_monitor_lsps, op->key)) { - /* This port was used for svc monitor, which may be - * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; - } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); - hmap_remove(&nd->ls_ports, &op->key_node); - hmap_remove(&od->ports, &op->dp_node); - sbrec_port_binding_delete(op->sb); - delete_fdb_entry(ni->sbrec_fdb_by_dp_and_port, od->tunnel_key, - op->tunnel_key); - } - } - - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + if (!ls_check_and_handle_lsp_changes(ovnsb_idl_txn, changed_ls, + ni, nd, od)) { + goto fail; } } @@ -5261,13 +5309,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: - free(ls_change); destroy_northd_data_tracked_changes(nd); return false; } From patchwork Fri Jul 7 05:54:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804607 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2fb3gD9z20b8 for ; Fri, 7 Jul 2023 15:54:47 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B3F374118A; Fri, 7 Jul 2023 05:54:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B3F374118A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iIAty296O_x8; Fri, 7 Jul 2023 05:54:44 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 72C6D4051D; Fri, 7 Jul 2023 05:54:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 72C6D4051D Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4B989C0072; Fri, 7 Jul 2023 05:54:43 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B90AC0032 for ; Fri, 7 Jul 2023 05:54:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 3F6CE40577 for ; Fri, 7 Jul 2023 05:54:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3F6CE40577 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fzjJw66oW3H7 for ; Fri, 7 Jul 2023 05:54:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C647840962 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp2.osuosl.org (Postfix) with ESMTPS id C647840962 for ; Fri, 7 Jul 2023 05:54:24 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id C2C261C0005; Fri, 7 Jul 2023 05:54:21 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:24:16 +0530 Message-Id: <20230707055416.961752-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 5/8] northd: Handle load balancer changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Logical switch change handler of the 'northd' engine node now also handles changes to load balancers. Signed-off-by: Numan Siddique --- lib/lb.c | 17 +++- lib/lb.h | 2 + northd/northd.c | 212 +++++++++++++++++++++++++++++++++----------- northd/northd.h | 9 ++ tests/ovn-northd.at | 8 +- 5 files changed, 191 insertions(+), 57 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index a51fe225fa..b2b6473c1d 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1091,7 +1091,22 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } + } +} + +void +ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set0(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls--; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..ac33333a7f 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -179,6 +179,8 @@ void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); struct ovn_lb_group_datapaths { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index fc98ab7bfd..bf02190f7c 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -853,7 +853,8 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) ovn_ls_port_group_destroy(&od->nb_pgs); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); - + free(od->lb_uuids); + free(od->lb_group_uuids); free(od); } } @@ -4080,6 +4081,48 @@ build_lb_vip_actions(const struct ovn_northd_lb *lb, return reject; } +static void +associate_ls_lbs(struct ovn_datapath *ls_od, struct hmap *lb_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + + free(ls_od->lb_uuids); + ls_od->lb_uuids = xcalloc(ls_od->nbs->n_load_balancer, + sizeof *ls_od->lb_uuids); + ls_od->n_lb_uuids = ls_od->nbs->n_load_balancer; + for (size_t i = 0; i < ls_od->nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &ls_od->nbs->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &ls_od); + ls_od->lb_uuids[i] = *lb_uuid; + } +} + +static void +associate_ls_lb_groups(struct ovn_datapath *ls_od, + struct hmap *lb_group_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + struct ovn_lb_group_datapaths *lb_group_dps; + + free(ls_od->lb_group_uuids); + ls_od->lb_group_uuids = xcalloc(ls_od->nbs->n_load_balancer_group, + sizeof *ls_od->lb_group_uuids); + ls_od->n_lb_group_uuids = ls_od->nbs->n_load_balancer_group; + for (size_t i = 0; i < ls_od->nbs->n_load_balancer_group; i++) { + nbrec_lb_group = ls_od->nbs->load_balancer_group[i]; + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &ls_od); + ls_od->lb_group_uuids[i] = *lb_group_uuid; + } +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, @@ -4115,24 +4158,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, if (!od->nbs) { continue; } - - for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &od->nbs->load_balancer[i]->header_.uuid; - lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); - ovs_assert(lb_dps); - ovn_lb_datapaths_add_ls(lb_dps, 1, &od); - } - - for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { - nbrec_lb_group = od->nbs->load_balancer_group[i]; - const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_datapaths_map, - lb_group_uuid); - ovs_assert(lb_group_dps); - ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &od); - } + associate_ls_lbs(od, lb_datapaths_map); + associate_ls_lb_groups(od, lb_group_datapaths_map); } HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { @@ -4891,23 +4918,29 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, sset_destroy(&active_ha_chassis_grps); } +static void +destroy_tracked_ls_change(struct ls_change *ls_change) +{ + struct ovn_port *op; + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct ls_change *ls_change; LIST_FOR_EACH_SAFE (ls_change, list_node, &nd->tracked_ls_changes.updated) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); - } + destroy_tracked_ls_change(ls_change); ovs_list_remove(&ls_change->list_node); free(ls_change); } @@ -5028,6 +5061,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * incrementally handled. * Presently supports i-p for the below changes: * - logical switch ports. + * - load balancers. */ static bool check_unsupported_inc_proc_for_ls_changes( @@ -5036,8 +5070,11 @@ check_unsupported_inc_proc_for_ls_changes( /* Check if the columns are changed in this row. */ enum nbrec_logical_switch_column_id col; for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { - if (nbrec_logical_switch_is_updated(ls, col) && - col != NBREC_LOGICAL_SWITCH_COL_PORTS) { + if (nbrec_logical_switch_is_updated(ls, col)) { + if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + continue; + } return true; } } @@ -5066,12 +5103,6 @@ check_unsupported_inc_proc_for_ls_changes( return true; } } - for (size_t i = 0; i < ls->n_load_balancer; i++) { - if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; - } - } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5130,7 +5161,9 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct nbrec_logical_switch *changed_ls, const struct northd_input *ni, struct northd_data *nd, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5151,12 +5184,6 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - struct ls_change *ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - struct ovn_port *op; HMAP_FOR_EACH (op, dp_node, &od->ports) { op->visited = false; @@ -5246,10 +5273,7 @@ ls_check_and_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!ovs_list_is_empty(&ls_change->added_ports) || !ovs_list_is_empty(&ls_change->updated_ports) || !ovs_list_is_empty(&ls_change->deleted_ports)) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + *updated = true; } return true; @@ -5260,10 +5284,60 @@ fail_clean_deleted: } fail: - free(ls_change); + + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + ovs_list_remove(&op->list); + } + LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { + ovs_list_remove(&op->list); + ovn_port_destroy_orphan(op); + } return false; } +static bool +ls_check_and_handle_lb_changes(const struct nbrec_logical_switch *changed_ls, + struct hmap *lb_datapaths_map, + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) +{ + + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)) { + for (size_t i = 0; i < changed_ls->n_load_balancer; i++) { + if (nbrec_load_balancer_row_get_seqno(changed_ls->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + break; + } + } + + return true; + } + + struct ovn_lb_datapaths *lb_dps; + for (size_t i = 0; i < od->n_lb_uuids; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &od->lb_uuids[i]); + if (lb_dps) { + ovn_lb_datapaths_remove_ls(lb_dps, 1, &od); + } + } + + associate_ls_lbs(od, lb_datapaths_map); + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} + /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set @@ -5293,20 +5367,47 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - /* Now only able to handle lsp changes. */ + /* Now only able to handle lsp, load balancerload and + * load balancer group changes. */ if (check_unsupported_inc_proc_for_ls_changes(changed_ls)) { goto fail; } + struct ls_change *ls_change = xzalloc(sizeof *ls_change); + ls_change->od = od; + ovs_list_init(&ls_change->added_ports); + ovs_list_init(&ls_change->deleted_ports); + ovs_list_init(&ls_change->updated_ports); + + bool updated = false; if (!ls_check_and_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od)) { + ni, nd, od, ls_change, + &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); + goto fail; + } + + if (!ls_check_and_handle_lb_changes(changed_ls, + &nd->lb_datapaths_map, od, + ls_change, &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); goto fail; } + + if (updated) { + ovs_list_push_back(&nd->tracked_ls_changes.updated, + &ls_change->list_node); + } else { + free(ls_change); + } } if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { nd->change_tracked = true; } + return true; fail: @@ -16628,6 +16729,13 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *lflows) { struct ls_change *ls_change; + + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { + if (ls_change->lbs_changed) { + return false; + } + } + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, diff --git a/northd/northd.h b/northd/northd.h index 7d17921fa2..3afa4eaff2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -94,6 +94,7 @@ struct ls_change { struct ovs_list added_ports; struct ovs_list deleted_ports; struct ovs_list updated_ports; + bool lbs_changed; }; /* Track what's changed for logical switches. @@ -318,6 +319,14 @@ struct ovn_datapath { /* Map of ovn_port objects belonging to this datapath. * This map doesn't include derived ports. */ struct hmap ports; + + /* LB uuids associated with this datapath. */ + struct uuid *lb_uuids; + size_t n_lb_uuids; + + /* LB group uuids associated with this datapath. */ + struct uuid *lb_group_uuids; + size_t n_lb_group_uuids; }; void ovnnb_db_run(struct northd_input *input_data, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index dd0bd8b36a..d9f3917a3e 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9750,15 +9750,15 @@ check ovn-nbctl ls-add sw0 check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats recompute recompute -# Associate lb1 to sw0. There should be a full recompute of northd engine node +# Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats recompute recompute +check_engine_stats norecompute recompute -# Disassociate lb1 from sw0. There should be a full recompute of northd engine node. +# Disassociate lb1 from sw0. There should be a no recompute of northd engine node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 -check_engine_stats recompute recompute +check_engine_stats norecompute recompute # Test load balancer group now check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats From patchwork Fri Jul 7 05:54:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804608 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2g727ZFz20ZQ for ; Fri, 7 Jul 2023 15:55:15 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 62B3F41A60; Fri, 7 Jul 2023 05:55:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 62B3F41A60 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PtR9Tx0N1kTq; Fri, 7 Jul 2023 05:55:11 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id BF0974195C; Fri, 7 Jul 2023 05:55:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org BF0974195C Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6D46DC0DD4; Fri, 7 Jul 2023 05:55:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19826C0032 for ; Fri, 7 Jul 2023 05:55:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id BDB826118C for ; Fri, 7 Jul 2023 05:54:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org BDB826118C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Et7xO-lpblxA for ; Fri, 7 Jul 2023 05:54:39 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 56CC66116B Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id 56CC66116B for ; Fri, 7 Jul 2023 05:54:39 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 780571C0003; Fri, 7 Jul 2023 05:54:36 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:24:30 +0530 Message-Id: <20230707055430.961782-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 6/8] northd: Handle load balancer group changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique For every a given load balancer group 'A', northd engine data maintains a bitmap of datapaths associated to this lb group. So when lb group 'A' gets associated to a logical switch 's1', the bitmap index of 's1' is set in its bitmap. In order to handle the load balancer group changes incrementally for a logical switch, we need to set and clear the bitmap bits accordingly. And this patch does it. Signed-off-by: Numan Siddique --- lib/lb.c | 45 +++++++++++++++--- lib/lb.h | 33 ++++++-------- northd/northd.c | 109 ++++++++++++++++++++++++++++++++++++++------ tests/ovn-northd.at | 6 +-- 4 files changed, 150 insertions(+), 43 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index b2b6473c1d..a0426cc42e 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1100,7 +1100,7 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) + struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { if (bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { @@ -1112,14 +1112,14 @@ ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_lb_group_datapaths * ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, - size_t max_ls_datapaths, - size_t max_lr_datapaths) + size_t n_ls_datapaths, + size_t n_lr_datapaths) { struct ovn_lb_group_datapaths *lb_group_dps = xzalloc(sizeof *lb_group_dps); lb_group_dps->lb_group = lb_group; - lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); - lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + lb_group_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_group_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); return lb_group_dps; } @@ -1127,8 +1127,8 @@ ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) { - free(lb_group_dps->ls); - free(lb_group_dps->lr); + bitmap_free(lb_group_dps->nb_lr_map); + bitmap_free(lb_group_dps->nb_ls_map); free(lb_group_dps); } @@ -1146,3 +1146,34 @@ ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, } return NULL; } + +void +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lbg_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lbg_dps->nb_ls_map, ods[i]->index); + lbg_dps->n_nb_ls++; + } + } +} + +void +ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *lbg_dps, + size_t n, struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lbg_dps->nb_ls_map, ods[i]->index)) { + bitmap_set0(lbg_dps->nb_ls_map, ods[i]->index); + lbg_dps->n_nb_ls--; + } + } +} + +void +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + bitmap_set1(lbg_dps->nb_lr_map, lr->index); +} diff --git a/lib/lb.h b/lib/lb.h index ac33333a7f..08723e8ef3 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -19,6 +19,7 @@ #include #include +#include "lib/bitmap.h" #include "lib/smap.h" #include "openvswitch/hmap.h" #include "ovn-util.h" @@ -179,6 +180,9 @@ void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_set_ls(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); + void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); @@ -188,10 +192,11 @@ struct ovn_lb_group_datapaths { const struct ovn_lb_group *lb_group; /* Datapaths to which 'lb_group' is applied. */ - size_t n_ls; - struct ovn_datapath **ls; - size_t n_lr; - struct ovn_datapath **lr; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; }; struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( @@ -202,21 +207,13 @@ void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( const struct hmap *lb_group_dps, const struct uuid *); -static inline void -ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, - struct ovn_datapath **ods) -{ - memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); - lbg_dps->n_ls += n; -} - -static inline void -ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, - struct ovn_datapath *lr) -{ - lbg_dps->lr[lbg_dps->n_lr++] = lr; -} +void ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *, size_t n, + struct ovn_datapath **); +void ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *, + size_t n, struct ovn_datapath **); +void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *, + struct ovn_datapath *lr); struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index bf02190f7c..20a58afa6b 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4102,7 +4102,8 @@ associate_ls_lbs(struct ovn_datapath *ls_od, struct hmap *lb_datapaths_map) static void associate_ls_lb_groups(struct ovn_datapath *ls_od, - struct hmap *lb_group_datapaths_map) + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map) { const struct nbrec_load_balancer_group *nbrec_lb_group; struct ovn_lb_group_datapaths *lb_group_dps; @@ -4120,6 +4121,15 @@ associate_ls_lb_groups(struct ovn_datapath *ls_od, ovs_assert(lb_group_dps); ovn_lb_group_datapaths_add_ls(lb_group_dps, 1, &ls_od); ls_od->lb_group_uuids[i] = *lb_group_uuid; + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &ls_od); + } } } @@ -4159,7 +4169,7 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, continue; } associate_ls_lbs(od, lb_datapaths_map); - associate_ls_lb_groups(od, lb_group_datapaths_map); + associate_ls_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); } HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { @@ -4219,10 +4229,12 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); - ovn_lb_datapaths_add_ls(lb_dps, lb_group_dps->n_ls, - lb_group_dps->ls); - ovn_lb_datapaths_add_lr(lb_dps, lb_group_dps->n_lr, - lb_group_dps->lr); + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_group_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + } } } } @@ -4401,8 +4413,9 @@ build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, struct ovn_lb_group_datapaths *lb_group_dps; HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_dps_map) { - for (size_t i = 0; i < lb_group_dps->n_lr; i++) { - struct ovn_datapath *od = lb_group_dps->lr[i]; + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_group_dps->nb_lr_map) { + struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lb_group_datapaths_add_ls(lb_group_dps, od->n_ls_peers, od->ls_peers); for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { @@ -5072,7 +5085,8 @@ check_unsupported_inc_proc_for_ls_changes( for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_PORTS || - col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP) { continue; } return true; @@ -5103,12 +5117,6 @@ check_unsupported_inc_proc_for_ls_changes( return true; } } - for (size_t i = 0; i < ls->n_load_balancer_group; i++) { - if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return true; - } - } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5338,6 +5346,68 @@ ls_check_and_handle_lb_changes(const struct nbrec_logical_switch *changed_ls, return true; } +static bool +ls_check_and_handle_lb_group_changes( + const struct nbrec_logical_switch *changed_ls, + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map, + struct ovn_datapath *od, + struct ls_change *ls_change, + bool *updated) +{ + bool lbg_modified = false; + /* Check if lb_groups changed or not */ + if (!nbrec_logical_switch_is_updated(changed_ls, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)) { + /* load_balancer_group column of logical switch hasn't changed, but + * its possible that a load balancer group may have changed (like + * a load balancer added or removed from the group). */ + for (size_t i = 0; i < changed_ls->n_load_balancer_group; i++) { + if (nbrec_load_balancer_group_row_get_seqno( + changed_ls->load_balancer_group[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + lbg_modified = true; + break; + } + } + } else { + /* load_balancer_group column of logical switch has changed. */ + lbg_modified = true; + } + + if (!lbg_modified) { + /* Nothing changed */ + return true; + } + + /* Disassociate load balancer group (and its load balancers) from + * the datapath and rebuild the association again. */ + struct ovn_lb_group_datapaths *lbg_dps; + for (size_t i = 0; i < od->n_lb_group_uuids; i++) { + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &od->lb_group_uuids[i]); + if (lbg_dps) { + ovn_lb_group_datapaths_remove_ls(lbg_dps, 1, &od); + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (lb_dps) { + ovn_lb_datapaths_remove_ls(lb_dps, 1, &od); + } + } + } + } + + associate_ls_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + ls_change->lbs_changed = true; + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set @@ -5396,6 +5466,15 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } + if (!ls_check_and_handle_lb_group_changes(changed_ls, + &nd->lb_group_datapaths_map, + &nd->lb_datapaths_map, + od, ls_change, &updated)) { + destroy_tracked_ls_change(ls_change); + free(ls_change); + goto fail; + } + if (updated) { ovs_list_push_back(&nd->tracked_ls_changes.updated, &ls_change->list_node); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index d9f3917a3e..d3a5851e35 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9783,16 +9783,16 @@ check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats recompute recompute +check_engine_stats norecompute recompute # Update lb and this should result in recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid From patchwork Fri Jul 7 05:54:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804609 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2gF6T5Dz20ZQ for ; Fri, 7 Jul 2023 15:55:21 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id F211141B05; Fri, 7 Jul 2023 05:55:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org F211141B05 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vkO3urhe1pTB; Fri, 7 Jul 2023 05:55:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5C60E41A49; Fri, 7 Jul 2023 05:55:17 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5C60E41A49 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3BE92C0072; Fri, 7 Jul 2023 05:55:17 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BCD6C0072 for ; Fri, 7 Jul 2023 05:55:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 5C81241AE9 for ; Fri, 7 Jul 2023 05:54:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5C81241AE9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aLmp4iQ9WQBw for ; Fri, 7 Jul 2023 05:54:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8530041A04 Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp4.osuosl.org (Postfix) with ESMTPS id 8530041A04 for ; Fri, 7 Jul 2023 05:54:56 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 1FE311C0004; Fri, 7 Jul 2023 05:54:53 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:24:49 +0530 Message-Id: <20230707055449.961815-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 7/8] northd: Sync SB Port bindings NAT column in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node 'sync_to_sb_pb' is added within 'sync_to_sb' node to sync NAT column of Port bindings table. This separation is required in order to add load balancer group I-P handling in 'northd' engine node (which is handled in the next commit). 'sync_to_sb_pb' engine node can be later expanded to sync other Port binding columns if required. Signed-off-by: Numan Siddique --- northd/en-sync-sb.c | 31 +++++ northd/en-sync-sb.h | 4 + northd/inc-proc-northd.c | 8 +- northd/northd.c | 243 +++++++++++++++++++++------------------ northd/northd.h | 2 + 5 files changed, 174 insertions(+), 114 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index fda0ca5a68..7c8be49209 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -240,6 +240,37 @@ en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) } +void * +en_sync_to_sb_pb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_pb_cleanup(void *data OVS_UNUSED) +{ + +} + +/* sync_to_sb_pb engine node functions. + * This engine node syncs the SB Port Bindings (partly) + * en_northd engine create the SB Port binding rows and + * updates most of the columns. + * This engine node updates the port binding columns which + * needs to be updated after northd engine is run. + */ + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index b3bfb5454f..f1008915bd 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -21,4 +21,8 @@ void *en_sync_to_sb_lb_init(struct engine_node *, struct engine_arg *); void en_sync_to_sb_lb_run(struct engine_node *, void *data); void en_sync_to_sb_lb_cleanup(void *data); +void *en_sync_to_sb_pb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_pb_run(struct engine_node *, void *data); +void en_sync_to_sb_pb_cleanup(void *data); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index b444a488db..f5b7998f1f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -141,6 +141,7 @@ static ENGINE_NODE(sync_to_sb_addr_set, "sync_to_sb_addr_set"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(northd_lb_data, "northd_lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, @@ -212,13 +213,16 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_lb, &en_northd, NULL); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_northd, NULL); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. - * Right now this engine syncs the SB Address_Set table and - * SB Load_Balancer table. + * Right now this engine syncs the SB Address_Set table, + * SB Load_Balancer table and (partly) SB Port_Binding table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_pb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index 20a58afa6b..a7761285de 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3516,8 +3516,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, ds_destroy(&s); sbrec_port_binding_set_external_ids(op->sb, &op->nbrp->external_ids); - - sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } else { if (!lsp_is_router(op->nbsp)) { uint32_t queue_id = smap_get_int( @@ -3633,116 +3631,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, } else { sbrec_port_binding_set_options(op->sb, NULL); } - const char *nat_addresses = smap_get(&op->nbsp->options, - "nat-addresses"); - size_t n_nats = 0; - char **nats = NULL; - bool l3dgw_ports = op->peer && op->peer->od && - op->peer->od->n_l3dgw_ports; - if (nat_addresses && !strcmp(nat_addresses, "router")) { - if (op->peer && op->peer->od - && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, - "exclude-lb-vips-from-garp", false); - nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); - } - } else if (nat_addresses && (chassis || l3dgw_ports)) { - struct lport_addresses laddrs; - if (!extract_lsp_addresses(nat_addresses, &laddrs)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); - } else { - destroy_lport_addresses(&laddrs); - n_nats = 1; - nats = xcalloc(1, sizeof *nats); - struct ds nat_addr = DS_EMPTY_INITIALIZER; - ds_put_format(&nat_addr, "%s", nat_addresses); - if (l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&nat_addr, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - nats[0] = xstrdup(ds_cstr(&nat_addr)); - ds_destroy(&nat_addr); - } - } - - /* Add the router mac and IPv4 addresses to - * Port_Binding.nat_addresses so that GARP is sent for these - * IPs by the ovn-controller on which the distributed gateway - * router port resides if: - * - * - op->peer has 'reside-on-redirect-chassis' set and the - * the logical router datapath has distributed router port. - * - * - op->peer is distributed gateway router port. - * - * - op->peer's router is a gateway router and op has a localnet - * port. - * - * Note: Port_Binding.nat_addresses column is also used for - * sending the GARPs for the router port IPs. - * */ - bool add_router_port_garp = false; - if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { - if (is_l3dgw_port(op->peer)) { - add_router_port_garp = true; - } else if (smap_get_bool(&op->peer->nbrp->options, - "reside-on-redirect-chassis", false)) { - if (op->peer->od->n_l3dgw_ports == 1) { - add_router_port_garp = true; - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " - "set on logical router port %s, which " - "is on logical router %s, which has %" - PRIuSIZE" distributed gateway ports. This" - "option can only be used when there is " - "a single distributed gateway port.", - op->peer->key, op->peer->od->nbr->name, - op->peer->od->n_l3dgw_ports); - } - } - } else if (chassis && op->od->n_localnet_ports) { - add_router_port_garp = true; - } - - if (add_router_port_garp) { - struct ds garp_info = DS_EMPTY_INITIALIZER; - ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); - - for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; - i++) { - ds_put_format(&garp_info, " %s", - op->peer->lrp_networks.ipv4_addrs[i].addr_s); - } - - if (op->peer->od->n_l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&garp_info, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - - n_nats++; - nats = xrealloc(nats, (n_nats * sizeof *nats)); - nats[n_nats - 1] = ds_steal_cstr(&garp_info); - ds_destroy(&garp_info); - } - sbrec_port_binding_set_nat_addresses(op->sb, - (const char **) nats, n_nats); - for (size_t i = 0; i < n_nats; i++) { - free(nats[i]); - } - free(nats); } sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name); @@ -4736,6 +4624,137 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } +/* Sync the SB Port bindings which needs to be updated. + * Presently it syncs the nat column of port bindings corresponding to + * the logical switch ports. */ +void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +{ + ovs_assert(ovnsb_idl_txn); + + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, ls_ports) { + if (lsp_is_router(op->nbsp)) { + const char *chassis = NULL; + if (op->peer && op->peer->od && op->peer->od->nbr) { + chassis = smap_get(&op->peer->od->nbr->options, "chassis"); + } + + const char *nat_addresses = smap_get(&op->nbsp->options, + "nat-addresses"); + size_t n_nats = 0; + char **nats = NULL; + bool l3dgw_ports = op->peer && op->peer->od && + op->peer->od->n_l3dgw_ports; + if (nat_addresses && !strcmp(nat_addresses, "router")) { + if (op->peer && op->peer->od + && (chassis || op->peer->od->n_l3dgw_ports)) { + bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + "exclude-lb-vips-from-garp", false); + nats = get_nat_addresses(op->peer, &n_nats, false, + !exclude_lb_vips); + } + } else if (nat_addresses && (chassis || l3dgw_ports)) { + struct lport_addresses laddrs; + if (!extract_lsp_addresses(nat_addresses, &laddrs)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); + } else { + destroy_lport_addresses(&laddrs); + n_nats = 1; + nats = xcalloc(1, sizeof *nats); + struct ds nat_addr = DS_EMPTY_INITIALIZER; + ds_put_format(&nat_addr, "%s", nat_addresses); + if (l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&nat_addr, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + nats[0] = xstrdup(ds_cstr(&nat_addr)); + ds_destroy(&nat_addr); + } + } + + /* Add the router mac and IPv4 addresses to + * Port_Binding.nat_addresses so that GARP is sent for these + * IPs by the ovn-controller on which the distributed gateway + * router port resides if: + * + * - op->peer has 'reside-on-redirect-chassis' set and the + * the logical router datapath has distributed router port. + * + * - op->peer is distributed gateway router port. + * + * - op->peer's router is a gateway router and op has a localnet + * port. + * + * Note: Port_Binding.nat_addresses column is also used for + * sending the GARPs for the router port IPs. + * */ + bool add_router_port_garp = false; + if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { + if (is_l3dgw_port(op->peer)) { + add_router_port_garp = true; + } else if (smap_get_bool(&op->peer->nbrp->options, + "reside-on-redirect-chassis", false)) { + if (op->peer->od->n_l3dgw_ports == 1) { + add_router_port_garp = true; + } else { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " + "set on logical router port %s, which " + "is on logical router %s, which has %" + PRIuSIZE" distributed gateway ports. This" + "option can only be used when there is " + "a single distributed gateway port.", + op->peer->key, op->peer->od->nbr->name, + op->peer->od->n_l3dgw_ports); + } + } + } else if (chassis && op->od->n_localnet_ports) { + add_router_port_garp = true; + } + + if (add_router_port_garp) { + struct ds garp_info = DS_EMPTY_INITIALIZER; + ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); + + for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; + i++) { + ds_put_format(&garp_info, " %s", + op->peer->lrp_networks.ipv4_addrs[i].addr_s); + } + + if (op->peer->od->n_l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&garp_info, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + n_nats++; + nats = xrealloc(nats, (n_nats * sizeof *nats)); + nats[n_nats - 1] = ds_steal_cstr(&garp_info); + ds_destroy(&garp_info); + } + sbrec_port_binding_set_nat_addresses(op->sb, + (const char **) nats, n_nats); + for (size_t i = 0; i < n_nats; i++) { + free(nats[i]); + } + free(nats); + } else { + sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + } + } +} + static bool ovn_port_add_tnlid(struct ovn_port *op, uint32_t tunnel_key) { diff --git a/northd/northd.h b/northd/northd.h index 3afa4eaff2..a79cc470c3 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -377,4 +377,6 @@ const char *northd_get_svc_monitor_mac(void); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); + #endif /* NORTHD_H */ From patchwork Fri Jul 7 05:55:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1804610 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Qy2h15n5Wz20ZQ for ; Fri, 7 Jul 2023 15:56:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C03D482355; Fri, 7 Jul 2023 05:55:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C03D482355 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jc6AkDScA8Go; Fri, 7 Jul 2023 05:55:58 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 45C99820C5; Fri, 7 Jul 2023 05:55:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 45C99820C5 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 20FAFC0072; Fri, 7 Jul 2023 05:55:57 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1D6B2C0032 for ; Fri, 7 Jul 2023 05:55:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id E3BCA6115F for ; Fri, 7 Jul 2023 05:55:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E3BCA6115F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id c8huh4gJ01y3 for ; Fri, 7 Jul 2023 05:55:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 239556117D Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::226]) by smtp3.osuosl.org (Postfix) with ESMTPS id 239556117D for ; Fri, 7 Jul 2023 05:55:10 +0000 (UTC) X-GND-Sasl: numans@ovn.org X-GND-Sasl: numans@ovn.org Received: by mail.gandi.net (Postfix) with ESMTPSA id 0996EC0003; Fri, 7 Jul 2023 05:55:06 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 7 Jul 2023 11:25:02 +0530 Message-Id: <20230707055502.961880-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230707055147.961462-1-numans@ovn.org> References: <20230707055147.961462-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v2 8/8] northd: Handle load balancer changes for a logical router. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique When a logical router gets updated due to load balancer or load balancer groups changes, it is now incrementally handled in the 'northd' engine node. Other logical router updates result in the full recompute of 'northd' engine node. lflow engine node still falls back to recompute due to logical router changes. Signed-off-by: Numan Siddique --- lib/lb.c | 32 +++- lib/lb.h | 5 + northd/en-lflow.c | 5 + northd/en-northd.c | 20 ++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 3 +- northd/northd.c | 392 ++++++++++++++++++++++++++++++++++----- northd/northd.h | 5 + tests/ovn-northd.at | 12 +- 9 files changed, 422 insertions(+), 53 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index a0426cc42e..a8b694d0b3 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1082,7 +1082,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } @@ -1110,6 +1113,18 @@ ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *lb_dps, size_t n, } } +void +ovn_lb_datapaths_remove_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set0(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr--; + } + } +} + struct ovn_lb_group_datapaths * ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, size_t n_ls_datapaths, @@ -1175,5 +1190,18 @@ void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, struct ovn_datapath *lr) { - bitmap_set1(lbg_dps->nb_lr_map, lr->index); + if (!bitmap_is_set(lbg_dps->nb_lr_map, lr->index)) { + bitmap_set1(lbg_dps->nb_lr_map, lr->index); + lbg_dps->n_nb_lr++; + } +} + +void +ovn_lb_group_datapaths_remove_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + if (bitmap_is_set(lbg_dps->nb_lr_map, lr->index)) { + bitmap_set1(lbg_dps->nb_lr_map, lr->index); + lbg_dps->n_nb_lr--; + } } diff --git a/lib/lb.h b/lib/lb.h index 08723e8ef3..91eec0199b 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -185,6 +185,8 @@ void ovn_lb_datapaths_set_ls(struct ovn_lb_datapaths *, size_t n, void ovn_lb_datapaths_remove_ls(struct ovn_lb_datapaths *, size_t n, struct ovn_datapath **); +void ovn_lb_datapaths_remove_lr(struct ovn_lb_datapaths *, size_t n, + struct ovn_datapath **); struct ovn_lb_group_datapaths { struct hmap_node hmap_node; @@ -214,6 +216,9 @@ void ovn_lb_group_datapaths_remove_ls(struct ovn_lb_group_datapaths *, void ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *, struct ovn_datapath *lr); +void ovn_lb_group_datapaths_remove_lr(struct ovn_lb_group_datapaths *, + struct ovn_datapath *lr); + struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/en-lflow.c b/northd/en-lflow.c index db1bcbccd6..ea51f4e3e0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -102,6 +102,11 @@ lflow_northd_handler(struct engine_node *node, if (!northd_data->change_tracked) { return false; } + + if (northd_data->lrouters_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; diff --git a/northd/en-northd.c b/northd/en-northd.c index b3c03c54bd..a321eff323 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -188,6 +188,26 @@ northd_nb_logical_switch_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_sb_port_binding_handler(struct engine_node *node, void *data) diff --git a/northd/en-northd.h b/northd/en-northd.h index 5674f4390c..739aa5e87f 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_northd_lb_data_handler(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index f5b7998f1f..362dd5e87a 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -156,7 +156,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_nb_port_group, NULL); engine_add_input(&en_northd, &en_nb_acl, NULL); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_meter, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); @@ -185,6 +184,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); diff --git a/northd/northd.c b/northd/northd.c index a7761285de..4393adc2ad 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4021,6 +4021,90 @@ associate_ls_lb_groups(struct ovn_datapath *ls_od, } } +static void +associate_lr_lbs(struct ovn_datapath *lr_od, struct hmap *lb_datapaths_map) +{ + struct ovn_lb_datapaths *lb_dps; + + free(lr_od->lb_uuids); + lr_od->lb_uuids = xcalloc(lr_od->nbr->n_load_balancer, + sizeof *lr_od->lb_uuids); + lr_od->n_lb_uuids = lr_od->nbr->n_load_balancer; + + if (!lr_od->lb_ips) { + lr_od->lb_ips = ovn_lb_ip_set_create(); + } + + for (size_t i = 0; i < lr_od->nbr->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &lr_od->nbr->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &lr_od); + build_lrouter_lb_ips(lr_od->lb_ips, lb_dps->lb); + lr_od->lb_uuids[i] = *lb_uuid; + } +} + +static void +associate_lr_lb_groups(struct ovn_datapath *lr_od, + struct hmap *lb_group_datapaths_map, + struct hmap *lb_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + struct ovn_lb_group_datapaths *lb_group_dps; + + free(lr_od->lb_group_uuids); + lr_od->lb_group_uuids = xcalloc(lr_od->nbr->n_load_balancer_group, + sizeof *lr_od->lb_group_uuids); + lr_od->n_lb_group_uuids = lr_od->nbr->n_load_balancer_group; + + /* Checking load balancer groups first, starting from the largest one, + * to more efficiently copy IP sets. */ + size_t largest_group = 0; + + for (size_t i = 1; i < lr_od->nbr->n_load_balancer_group; i++) { + if (lr_od->nbr->load_balancer_group[i]->n_load_balancer > + lr_od->nbr->load_balancer_group[largest_group]->n_load_balancer) { + largest_group = i; + } + } + + for (size_t i = 0; i < lr_od->nbr->n_load_balancer_group; i++) { + size_t idx = (i + largest_group) % lr_od->nbr->n_load_balancer_group; + + nbrec_lb_group = lr_od->nbr->load_balancer_group[idx]; + const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lb_group_datapaths_map, + lb_group_uuid); + ovs_assert(lb_group_dps); + ovn_lb_group_datapaths_add_lr(lb_group_dps, lr_od); + + if (!lr_od->lb_ips) { + lr_od->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); + } else { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(lr_od->lb_ips, + lb_group_dps->lb_group->lbs[j]); + } + } + + lr_od->lb_group_uuids[i] = *lb_group_uuid; + + struct ovn_lb_datapaths *lb_dps; + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lb_group_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &lr_od); + } + } +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, @@ -4028,7 +4112,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map) { - const struct nbrec_load_balancer_group *nbrec_lb_group; struct ovn_lb_group_datapaths *lb_group_dps; const struct ovn_lb_group *lb_group; struct ovn_lb_datapaths *lb_dps; @@ -4062,53 +4145,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - - /* Checking load balancer groups first, starting from the largest one, - * to more efficiently copy IP sets. */ - size_t largest_group = 0; - - for (size_t i = 1; i < od->nbr->n_load_balancer_group; i++) { - if (od->nbr->load_balancer_group[i]->n_load_balancer > - od->nbr->load_balancer_group[largest_group]->n_load_balancer) { - largest_group = i; - } - } - - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; - - nbrec_lb_group = od->nbr->load_balancer_group[idx]; - const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; - - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_datapaths_map, - lb_group_uuid); - ovs_assert(lb_group_dps); - ovn_lb_group_datapaths_add_lr(lb_group_dps, od); - - if (!od->lb_ips) { - od->lb_ips = - ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); - } else { - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, - lb_group_dps->lb_group->lbs[j]); - } - } - } - - if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_create(); - } - - for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &od->nbr->load_balancer[i]->header_.uuid; - lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); - ovs_assert(lb_dps); - ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - } + associate_lr_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + associate_lr_lbs(od, lb_datapaths_map); } HMAP_FOR_EACH (lb_group_dps, hmap_node, lb_group_datapaths_map) { @@ -4978,6 +5016,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) } nd->change_tracked = false; + nd->lrouters_changed = false; } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5513,6 +5552,263 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +static bool +lr_check_and_handle_lb_and_lbgrp_changes( + const struct nbrec_logical_router *changed_lr, + struct hmap *lb_datapaths_map, struct hmap *lb_group_datapaths_map, + struct ovn_datapath *od, bool *updated) +{ + ovs_assert(od->nbr); + bool lbs_changed = true; + if (!nbrec_logical_router_is_updated(changed_lr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER) && + !nbrec_logical_router_is_updated(changed_lr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)) { + lbs_changed = false; + for (size_t i = 0; i < changed_lr->n_load_balancer_group; i++) { + if (nbrec_load_balancer_group_row_get_seqno( + changed_lr->load_balancer_group[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + lbs_changed = true; + break; + } + } + + if (!lbs_changed) { + for (size_t i = 0; i < changed_lr->n_load_balancer; i++) { + if (nbrec_load_balancer_row_get_seqno( + changed_lr->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + lbs_changed = true; + break; + } + } + } + + if (!lbs_changed) { + return true; + } + } + + /* We need to do 2 things to handle the router for load balancer or + * load balancer groups. + * + * 1. Disassociate the logical router datapath from the already associated + * load balancers/load balancer groups. + * 2. Associate the logical router datapath with the updated + * load balancers/groups. + * */ + struct ovn_lb_datapaths *lb_dps; + size_t i, j; + for (i = 0; i < od->n_lb_uuids; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &od->lb_uuids[i]); + if (lb_dps) { + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag set. + * This requires updating the router ports's routable + * addresses. */ + return false; + } + ovn_lb_datapaths_remove_lr(lb_dps, 1, &od); + } + } + + struct ovn_lb_group_datapaths *lbg_dps; + for (i = 0; i < od->n_lb_group_uuids; i++) { + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &od->lb_group_uuids[i]); + if (lbg_dps) { + ovn_lb_group_datapaths_remove_lr(lbg_dps, od); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_group_datapaths_remove_ls(lbg_dps, od->n_ls_peers, + od->ls_peers); + } + } + + for (j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + if (lb_dps) { + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag + * set. This requires updating the router ports's + * routable addresses. */ + return false; + } + ovn_lb_datapaths_remove_lr(lb_dps, 1, &od); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_datapaths_remove_ls(lb_dps, od->n_ls_peers, + od->ls_peers); + } + } + } + } + + ovn_lb_ip_set_destroy(od->lb_ips); + od->lb_ips = NULL; + associate_lr_lb_groups(od, lb_group_datapaths_map, lb_datapaths_map); + associate_lr_lbs(od, lb_datapaths_map); + + /* Do the job of build_lrouter_lbs_reachable_ips and + * build_lswitch_lbs_from_lrouter for this lrouter datapath. */ + for (i = 0; i < od->nbr->n_load_balancer; i++) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, + &od->nbr->load_balancer[i]->header_.uuid); + ovs_assert(lb_dps); + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag set. + * This requires updating the router ports's routable addresses. */ + return false; + } + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + + for (i = 0; i < od->nbr->n_load_balancer_group; i++) { + const struct nbrec_load_balancer_group *nbrec_lb_group = + od->nbr->load_balancer_group[i]; + lbg_dps = ovn_lb_group_datapaths_find(lb_group_datapaths_map, + &nbrec_lb_group->header_.uuid); + ovs_assert(lbg_dps); + if (install_ls_lb_from_router && od->n_ls_peers) { + ovn_lb_group_datapaths_add_ls(lbg_dps, od->n_ls_peers, + od->ls_peers); + } + + for (j = 0; j < lbg_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(od, lbg_dps->lb_group->lbs[j]); + + if (install_ls_lb_from_router && od->n_ls_peers) { + const struct uuid *lb_uuid = + &lbg_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + if (lb_dps->lb->routable) { + /* Can't handler if the load balancer has routable flag + * set. This requires updating the router ports's + * routable addresses. */ + return false; + } + ovn_lb_datapaths_add_ls(lb_dps, od->n_ls_peers, od->ls_peers); + } + } + } + + *updated = true; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + return true; +} + + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + bool updated = false; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + struct ovn_datapath *od = ovn_datapath_find( + &nd->lr_datapaths.datapaths, + &changed_lr->header_.uuid); + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + + if (!lr_check_and_handle_lb_and_lbgrp_changes(changed_lr, + &nd->lb_datapaths_map, + &nd->lb_group_datapaths_map, + od, &updated)) { + goto fail; + } + } + + if (updated) { + nd->change_tracked = true; + nd->lrouters_changed = true; + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, diff --git a/northd/northd.h b/northd/northd.h index a79cc470c3..765720849a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -119,8 +119,11 @@ struct northd_data { struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; + + /* Tracked data. */ bool change_tracked; struct tracked_ls_changes tracked_ls_changes; + bool lrouters_changed; }; struct lflow_data { @@ -342,6 +345,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index d3a5851e35..286b311242 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9760,6 +9760,14 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats norecompute recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lr-lb-add lr0 lb1 +check_engine_stats norecompute recompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lr-lb-del lr0 lb1 +check_engine_stats norecompute recompute + # Test load balancer group now check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) @@ -9796,11 +9804,11 @@ check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats recompute recompute +check_engine_stats norecompute recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats recompute recompute +check_engine_stats norecompute recompute AT_CLEANUP ])