From patchwork Mon Sep 11 16:00:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1832347 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rks0G2Pjfz1ygM for ; Tue, 12 Sep 2023 02:01:34 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 79A6281EBE; Mon, 11 Sep 2023 16:01:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 79A6281EBE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IewtTXRPGU6e; Mon, 11 Sep 2023 16:01:26 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 4667C81EA3; Mon, 11 Sep 2023 16:01:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4667C81EA3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2A83EC0071; Mon, 11 Sep 2023 16:01:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id B9CBCC0032 for ; Mon, 11 Sep 2023 16:01:23 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 948BF60E11 for ; Mon, 11 Sep 2023 16:01:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 948BF60E11 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id C1uqeO6iflQD for ; Mon, 11 Sep 2023 16:01:21 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp3.osuosl.org (Postfix) with ESMTPS id 2D68560767 for ; Mon, 11 Sep 2023 16:01:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2D68560767 Received: by mail.gandi.net (Postfix) with ESMTPSA id 41C61FF817; Mon, 11 Sep 2023 16:00:46 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 11 Sep 2023 12:00:41 -0400 Message-ID: <20230911160041.179438-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911155930.179283-1-numans@ovn.org> References: <20230911155930.179283-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v7 1/4] northd: Handle load balancer changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique 'lb_data' engine node now also handles logical switch changes. Its data maintains ls to lb related information. i.e if a logical switch sw0 has lb1, lb2 and lb3 associated then it stores this info in its data. And when a new load balancer lb4 is associated to it, it stores this information in its tracked data so that 'northd' engine node can handle it accordingly. Tracked data will have information like: changed ls -> {sw0 : {associated_lbs: [lb4]} The first handler 'northd_lb_data_handler_pre_od' is called before the 'northd_nb_logical_switch_handler' handler and it just creates or deletes the lb_datapaths hmap for the tracked lbs. The northd handler 'northd_lb_data_handler' updates the ovn_lb_datapaths's 'nb_ls_map' bitmap accordingly. Eg. If the lb_data has the below tracked data: tracked_data = {'crupdated_lbs': [lb1, lb2], 'deleted_lbs': [lb3], 'crupdated_lb_groups': [lbg1, lbg2], 'crupdated_ls_lbs': [{ls: sw0, assoc_lbs: [lb1], {ls: sw1, assoc_lbs: [lb1, lb2]} The handler northd_lb_data_handler(), creates the ovn_lb_datapaths object for lb1 and lb2 and deletes lb3 from the ovn_lb_datapaths hmap. It does the same for the created or updated lb groups lbg1 and lbg2 in the ovn_lbgrp_datapaths map. It also updates the nb_ls_bitmap of lb1 for sw0 and sw1 and nb_ls_bitmap of lb2 for sw1. Reviewed-by: Ales Musil Acked-by: Mark Michelson Signed-off-by: Numan Siddique --- lib/lb.c | 5 +- northd/en-lb-data.c | 176 +++++++++++++++++++++++++++++++++++++++ northd/en-lb-data.h | 17 ++++ northd/en-lflow.c | 6 ++ northd/en-northd.c | 6 +- northd/inc-proc-northd.c | 2 + northd/northd.c | 83 +++++++++++++++--- northd/northd.h | 4 +- tests/ovn-northd.at | 56 +++++++++---- 9 files changed, 322 insertions(+), 33 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index 6fd67e2218..e6c9dc2be2 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1088,7 +1088,10 @@ ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } } } diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 8acd9c8cb2..02b1bfd7a4 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -39,6 +39,14 @@ static void lb_data_destroy(struct ed_type_lb_data *); static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); +static void build_od_lb_map(const struct nbrec_logical_switch_table *, + struct hmap *od_lb_map); +static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); +static void destroy_od_lb_data(struct od_lb_data *od_lb_data); +static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, + const struct uuid *od_uuid); + static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); @@ -54,6 +62,7 @@ static struct crupdated_lbgrp * struct tracked_lb_data *); static void add_deleted_lbgrp_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); +static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -80,9 +89,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer", node)); const struct nbrec_load_balancer_group_table *nb_lbg_table = EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lbgrps); + build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + engine_set_node_state(node, EN_UPDATED); } @@ -230,18 +243,104 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) return true; } +struct od_lb_data { + struct hmap_node hmap_node; + struct uuid od_uuid; + struct uuidset *lbs; + struct uuidset *lbgrps; +}; + +bool +lb_data_logical_switch_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_switch_table *nb_ls_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_switch *nbs; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (nbs, nb_ls_table) { + if (nbrec_logical_switch_is_deleted(nbs)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->ls_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); + } + } else { + if (!is_ls_lbs_changed(nbs)) { + continue; + } + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbs->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->ls_lb_map, + &nbs->header_.uuid); + } + + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } + + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + /* static functions. */ static void lb_data_init(struct ed_type_lb_data *lb_data) { hmap_init(&lb_data->lbs); hmap_init(&lb_data->lbgrps); + hmap_init(&lb_data->ls_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); hmapx_init(&trk_lb_data->deleted_lbs); hmap_init(&trk_lb_data->crupdated_lbgrps); hmapx_init(&trk_lb_data->deleted_lbgrps); + ovs_list_init(&trk_lb_data->crupdated_ls_lbs); } static void @@ -259,6 +358,12 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->lbgrps); + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->ls_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->ls_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -301,12 +406,67 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, return lb_group; } +static void +build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, + struct hmap *od_lb_map) +{ + const struct nbrec_logical_switch *nbrec_ls; + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { + if (!nbrec_ls->n_load_balancer) { + continue; + } + + struct od_lb_data *od_lb_data = + create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { + uuidset_insert(od_lb_data->lbs, + &nbrec_ls->load_balancer[i]->header_.uuid); + } + } +} + +static struct od_lb_data * +create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); + od_lb_data->od_uuid = *od_uuid; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + hmap_insert(od_lb_map, &od_lb_data->hmap_node, + uuid_hash(&od_lb_data->od_uuid)); + return od_lb_data; +} + +static struct od_lb_data * +find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) +{ + struct od_lb_data *od_lb_data; + HMAP_FOR_EACH_WITH_HASH (od_lb_data, hmap_node, uuid_hash(od_uuid), + od_lb_map) { + if (uuid_equals(&od_lb_data->od_uuid, od_uuid)) { + return od_lb_data; + } + } + + return NULL; +} + +static void +destroy_od_lb_data(struct od_lb_data *od_lb_data) +{ + uuidset_destroy(od_lb_data->lbs); + free(od_lb_data->lbs); + free(od_lb_data); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { lb_data->tracked = false; lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; + lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -331,6 +491,15 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) hmapx_destroy(&crupdated_lbg->assoc_lbs); free(crupdated_lbg); } + + struct crupdated_od_lb_data *codlb; + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_ls_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + + free(codlb); + } } static void @@ -376,3 +545,10 @@ add_deleted_lbgrp_to_tracked_data(struct ovn_lb_group *lbg, { hmapx_add(&tracked_lb_data->deleted_lbgrps, lbg); } + +static bool +is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index afb163dd9f..022b38523b 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -27,6 +28,13 @@ struct crupdated_lbgrp { struct hmapx assoc_lbs; }; +struct crupdated_od_lb_data { + struct ovs_list list_node; + + struct uuid od_uuid; + struct uuidset assoc_lbs; +}; + struct tracked_lb_data { /* Both created and updated lbs. hmapx node is 'struct crupdated_lb *'. */ struct hmap crupdated_lbs; @@ -41,12 +49,19 @@ struct tracked_lb_data { /* Deleted lb_groups. hmapx node is 'struct ovn_lb_group *'. */ struct hmapx deleted_lbgrps; + /* List of logical switch <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_ls_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; /* Indicates if any lb got disassociated from a lb group * but not deleted. */ bool has_dissassoc_lbs_from_lbgrps; + + /* Indicates if a lb was disassociated from a logical switch. */ + bool has_dissassoc_lbs_from_od; }; /* struct which maintains the data of the engine node lb_data. */ @@ -56,6 +71,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lbgrps; + struct hmap ls_lb_map; /* tracked data*/ bool tracked; @@ -69,5 +85,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); +bool lb_data_logical_switch_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index ad8b62c735..2b84fef0ef 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -107,6 +107,12 @@ lflow_northd_handler(struct engine_node *node, if (!northd_data->change_tracked) { return false; } + + /* Fall back to recompute if lb related data has changed. */ + if (northd_data->lb_changed) { + return false; + } + const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; diff --git a/northd/en-northd.c b/northd/en-northd.c index 1704a18834..8487b003f7 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -1,4 +1,4 @@ -/* + /* * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at: @@ -214,6 +214,10 @@ northd_lb_data_handler(struct engine_node *node, void *data) return false; } + /* Indicate the depedendant engine nodes that load balancer/group + * related data has changed (including association to logical + * switch/router). */ + nd->lb_changed = true; engine_set_node_state(node, EN_UPDATED); return true; } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index ccc6687207..303b58d43f 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -155,6 +155,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_handler); engine_add_input(&en_lb_data, &en_nb_load_balancer_group, lb_data_load_balancer_group_handler); + engine_add_input(&en_lb_data, &en_nb_logical_switch, + lb_data_logical_switch_handler); engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); diff --git a/northd/northd.c b/northd/northd.c index e9cb906e2a..f767e0b39f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -73,7 +73,7 @@ static bool install_ls_lb_from_router; /* Use common zone for SNAT and DNAT if this option is set to "true". */ static bool use_common_zone = false; -/* MAC allocated for service monitor usage. Just one mac is allocated +/* MAC allocated for service monitor usage. Just one mac is allocatedg5534 * for this purpose and ovn-controller's on each chassis will make use * of this mac when sending out the packets to monitor the services * defined in Service_Monitor Southbound table. Since these packets @@ -852,7 +852,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); destroy_ports_for_datapath(od); - free(od); } } @@ -4959,6 +4958,7 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) } nd->change_tracked = false; + nd->lb_changed = false; } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5074,6 +5074,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * incrementally handled. * Presently supports i-p for the below changes: * - logical switch ports. + * - load balancers. */ static bool ls_changes_can_be_handled( @@ -5084,7 +5085,8 @@ ls_changes_can_be_handled( for (col = 0; col < NBREC_LOGICAL_SWITCH_N_COLUMNS; col++) { if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_ACLS || - col == NBREC_LOGICAL_SWITCH_COL_PORTS) { + col == NBREC_LOGICAL_SWITCH_COL_PORTS || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { continue; } return false; @@ -5109,12 +5111,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer; i++) { - if (nbrec_load_balancer_row_get_seqno(ls->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_load_balancer_group; i++) { if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5365,6 +5361,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { nd->change_tracked = true; } + return true; fail: @@ -5431,9 +5428,20 @@ northd_handle_sb_port_binding_changes( return true; } -/* Handler for lb_data engine changes. For every tracked lb_data - * it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths - * from the lb_datapaths hmap and lb_group_datapaths hmap. */ +/* Handler for lb_data engine changes. It does the following + * For every tracked 'lb' and 'lb_group' + * - it creates or deletes the ovn_lb_datapaths/ovn_lb_group_datapaths + * from the lb_datapaths hmap and lb_group_datapaths hmap. + * + * - For any changes to a logical switch (in 'trk_lb_data->crupdated_ls_lbs') + * due to association of a load balancer (eg. ovn-nbctl ls-lb-add sw0 lb1), + * the logical switch datapath is added to the load balancer (represented + * by 'struct ovn_lb_datapaths') by calling ovn_lb_datapaths_add_ls(). + * + * - For every 'lb' in the tracked data (trk_lb_data->crupdated_lbs) , + * it gets the associated logical switches and for each switch it + * re-evaluates 'od->has_lb_vip' to reflect any changes to the lb vips. + * */ bool northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, @@ -5459,8 +5467,15 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return false; } + /* Fall back to recompute if any load balancer has been disassociated from + * a logical switch or router. */ + if (trk_lb_data->has_dissassoc_lbs_from_od) { + return false; + } + struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; + struct ovn_datapath *od; struct hmapx_node *hmapx_node; HMAPX_FOR_EACH (hmapx_node, &trk_lb_data->deleted_lbs) { lb = hmapx_node->data; @@ -5468,10 +5483,22 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + + /* Re-evaluate 'od->has_lb_vip for od's associated with the + * deleted lb. */ + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + init_lb_for_datapath(od); + } + hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); ovn_lb_datapaths_destroy(lb_dps); } + /* Create the 'lb_dps' if not already created for each + * 'lb' in the trk_lb_data->crupdated_lbs. */ struct crupdated_lb *clb; HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { lb = clb->lb; @@ -5504,6 +5531,37 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } + struct crupdated_od_lb_data *codlb; + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { + od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), + lb_dps->nb_ls_map) { + od = ls_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + return true; } @@ -16529,6 +16587,7 @@ bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *lflows) { struct ls_change *ls_change; + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, diff --git a/northd/northd.h b/northd/northd.h index 2bb6910945..0d206a4e52 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -115,6 +115,8 @@ struct northd_data { struct hmap svc_monitor_map; bool change_tracked; struct tracked_ls_changes tracked_ls_changes; + bool lb_changed; /* Indicates if load balancers changed or association of + * load balancer to logical switch/router changed. */ }; struct lflow_data { @@ -345,7 +347,7 @@ bool northd_handle_lb_data_changes(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lb_group_datapaths_map); + struct hmap *lbgrp_datapaths_map); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 5c9da811fa..f0f8d1f503 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10398,22 +10398,23 @@ ovn-nbctl lsp-set-type sw0-lr0 router ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute -# Associate lb1 to sw0. There should be a full recompute of northd engine node +# Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10421,7 +10422,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10429,7 +10430,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10437,14 +10438,33 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Associate lb1 to sw0 and also create a port sw0p1. This should not result in +# full recompute of northd, but should rsult in full recompute of lflow node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute + +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +# Disassociate lb1 from sw0. There should be a recompute of northd engine node. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10532,7 +10552,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10577,7 +10597,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10629,7 +10649,7 @@ check_engine_stats lflow recompute nocompute # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10670,7 +10690,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10684,15 +10704,15 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10706,7 +10726,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Mon Sep 11 16:01:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1832348 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rks0s0gLKz1ygM for ; Tue, 12 Sep 2023 02:02:05 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 4CF77417BD; Mon, 11 Sep 2023 16:02:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 4CF77417BD X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TyueXVari9Pq; Mon, 11 Sep 2023 16:02:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 55108404FF; Mon, 11 Sep 2023 16:02:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 55108404FF Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 29806C0072; Mon, 11 Sep 2023 16:02:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C37C0C0032 for ; Mon, 11 Sep 2023 16:01:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 94BC841487 for ; Mon, 11 Sep 2023 16:01:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 94BC841487 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 6mYh1k3O-VEJ for ; Mon, 11 Sep 2023 16:01:57 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp2.osuosl.org (Postfix) with ESMTPS id E0B9741479 for ; Mon, 11 Sep 2023 16:01:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org E0B9741479 Received: by mail.gandi.net (Postfix) with ESMTPSA id D5CDC6001D; Mon, 11 Sep 2023 16:01:22 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 11 Sep 2023 12:01:18 -0400 Message-ID: <20230911160118.179671-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911155930.179283-1-numans@ovn.org> References: <20230911155930.179283-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v7 2/4] northd: Handle load balancer group changes for a logical switch. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique For a given load balancer group 'A', northd engine data maintains a bitmap of datapaths associated to this lb group. So when lb group 'A' gets associated to a logical switch 's1', the bitmap index of 's1' is set in its bitmap. In order to handle the load balancer group changes incrementally for a logical switch, we need to set and clear the bitmap bits accordingly. And this patch does it. Reviewed-by: Ales Musil Acked-by: Mark Michelson Signed-off-by: Numan Siddique --- northd/en-lb-data.c | 102 ++++++++++++++++++++++++++++++++------------ northd/en-lb-data.h | 4 ++ northd/northd.c | 62 +++++++++++++++++++++++---- tests/ovn-northd.at | 30 +++++++++---- 4 files changed, 154 insertions(+), 44 deletions(-) diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 02b1bfd7a4..3604ba8226 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -63,6 +63,7 @@ static struct crupdated_lbgrp * static void add_deleted_lbgrp_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); +static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -271,12 +272,15 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) destroy_od_lb_data(od_lb_data); } } else { - if (!is_ls_lbs_changed(nbs)) { + bool ls_lbs_changed = is_ls_lbs_changed(nbs); + bool ls_lbgrps_changed = is_ls_lbgrps_changed(nbs); + if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); struct od_lb_data *od_lb_data = find_od_lb_data(&lb_data->ls_lb_map, &nbs->header_.uuid); @@ -285,38 +289,66 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) &nbs->header_.uuid); } - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); + if (ls_lbs_changed) { + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < nbs->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbs->load_balancer[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, + lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbs->load_balancer[i], + OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + changed = true; + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + changed = true; + } - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); + } - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; + if (ls_lbgrps_changed) { + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { + const struct uuid *lbg_uuid = + &nbs->load_balancer_group[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbg_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, + lbg_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); + changed = true; + } } - if (unode) { - uuidset_delete(pre_lb_uuids, unode); + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + changed = true; } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); } - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); } } @@ -412,7 +444,7 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { - if (!nbrec_ls->n_load_balancer) { + if (!nbrec_ls->n_load_balancer && !nbrec_ls->n_load_balancer_group) { continue; } @@ -422,6 +454,10 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, uuidset_insert(od_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } + for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { + uuidset_insert(od_lb_data->lbgrps, + &nbrec_ls->load_balancer_group[i]->header_.uuid); + } } } @@ -431,7 +467,9 @@ create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid) struct od_lb_data *od_lb_data = xzalloc(sizeof *od_lb_data); od_lb_data->od_uuid = *od_uuid; od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); uuidset_init(od_lb_data->lbs); + uuidset_init(od_lb_data->lbgrps); hmap_insert(od_lb_map, &od_lb_data->hmap_node, uuid_hash(&od_lb_data->od_uuid)); @@ -456,7 +494,9 @@ static void destroy_od_lb_data(struct od_lb_data *od_lb_data) { uuidset_destroy(od_lb_data->lbs); + uuidset_destroy(od_lb_data->lbgrps); free(od_lb_data->lbs); + free(od_lb_data->lbgrps); free(od_lb_data); } @@ -467,6 +507,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_health_checks = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; + lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -497,7 +538,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) &lb_data->tracked_lb_data.crupdated_ls_lbs) { ovs_list_remove(&codlb->list_node); uuidset_destroy(&codlb->assoc_lbs); - + uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } } @@ -552,3 +593,10 @@ is_ls_lbs_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER)); } + +static bool +is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { + return ((nbrec_logical_switch_is_new(nbs) && nbs->n_load_balancer_group) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index 022b38523b..d802d000d4 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -33,6 +33,7 @@ struct crupdated_od_lb_data { struct uuid od_uuid; struct uuidset assoc_lbs; + struct uuidset assoc_lbgrps; }; struct tracked_lb_data { @@ -62,6 +63,9 @@ struct tracked_lb_data { /* Indicates if a lb was disassociated from a logical switch. */ bool has_dissassoc_lbs_from_od; + + /* Indicates if a lb group was disassociated from a logical switch. */ + bool has_dissassoc_lbgrps_from_od; }; /* struct which maintains the data of the engine node lb_data. */ diff --git a/northd/northd.c b/northd/northd.c index f767e0b39f..c5e1d1c5a4 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5075,6 +5075,7 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, * Presently supports i-p for the below changes: * - logical switch ports. * - load balancers. + * - load balancer groups. */ static bool ls_changes_can_be_handled( @@ -5086,7 +5087,8 @@ ls_changes_can_be_handled( if (nbrec_logical_switch_is_updated(ls, col)) { if (col == NBREC_LOGICAL_SWITCH_COL_ACLS || col == NBREC_LOGICAL_SWITCH_COL_PORTS || - col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER) { + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP) { continue; } return false; @@ -5111,12 +5113,6 @@ ls_changes_can_be_handled( return false; } } - for (size_t i = 0; i < ls->n_load_balancer_group; i++) { - if (nbrec_load_balancer_group_row_get_seqno(ls->load_balancer_group[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < ls->n_qos_rules; i++) { if (nbrec_qos_row_get_seqno(ls->qos_rules[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5304,7 +5300,11 @@ fail: /* Return true if changes are handled incrementally, false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, - * false. */ + * false. + * + * Note: Changes to load balancer and load balancer groups associated with + * the logical switches are handled separately in the lb_data change handlers. + * */ bool northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, const struct northd_input *ni, @@ -5473,6 +5473,12 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return false; } + /* Fall back to recompute if any load balancer group has been disassociated + * from a logical switch or router. */ + if (trk_lb_data->has_dissassoc_lbgrps_from_od) { + return false; + } + struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; struct ovn_datapath *od; @@ -5543,6 +5549,22 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovn_lb_datapaths_add_ls(lb_dps, 1, &od); } + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_ls(lbgrp_dps, 1, &od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + } + } + /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } @@ -5562,6 +5584,30 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } + HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, &trk_lb_data->crupdated_lbgrps) { + lbgrp = crupdated_lbgrp->lbgrp; + const struct uuid *lb_uuid = &lbgrp->uuid; + + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbgrp->assoc_lbs) { + lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { + od = lbgrp_dps->ls[i]; + ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + } + } + return true; } diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index f0f8d1f503..34e10663d0 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2191,6 +2191,18 @@ check ovn-nbctl --wait=sb sync AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl ]) +# Now associate vip again to lb4 and then delete it. +check ovn-nbctl set load_balancer $lb4 vips:"10.0.0.13"="10.0.0.6" +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl + table=1 (ls_out_pre_lb ), priority=100 , match=(ip), action=(reg0[[2]] = 1; next;) +]) + +check ovn-nbctl lb-del $lb4 +check ovn-nbctl --wait=sb sync +AT_CHECK([ovn-sbctl lflow-list | grep "ls_out_pre_lb.*priority=100" | grep reg0 | sort], [0], [dnl +]) + AT_CLEANUP ]) @@ -10553,21 +10565,21 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute -# Update lb and this should result in recompute +# Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10575,7 +10587,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10583,7 +10595,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10591,7 +10603,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10650,7 +10662,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10691,7 +10703,7 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Mon Sep 11 16:01:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1832349 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rks2S38Ffz1ygM for ; Tue, 12 Sep 2023 02:03:28 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 8F38A41493; Mon, 11 Sep 2023 16:03:26 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8F38A41493 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gqrv6pVGxO0V; Mon, 11 Sep 2023 16:03:25 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3D6AA41471; Mon, 11 Sep 2023 16:03:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3D6AA41471 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1485DC0071; Mon, 11 Sep 2023 16:03:24 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id EAEEBC0032 for ; Mon, 11 Sep 2023 16:03:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C54BF40332 for ; Mon, 11 Sep 2023 16:03:22 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C54BF40332 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mOzHjHDiBh_2 for ; Mon, 11 Sep 2023 16:03:21 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp4.osuosl.org (Postfix) with ESMTPS id AFB1A4030A for ; Mon, 11 Sep 2023 16:03:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org AFB1A4030A Received: by mail.gandi.net (Postfix) with ESMTPSA id B918F1BF230; Mon, 11 Sep 2023 16:02:46 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 11 Sep 2023 12:01:54 -0400 Message-ID: <20230911160154.179732-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911155930.179283-1-numans@ovn.org> References: <20230911155930.179283-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v7 3/4] northd: Sync SB Port bindings NAT column in a separate engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node 'sync_to_sb_pb' is added within 'sync_to_sb' node to sync NAT column of Port bindings table. This separation is required in order to add load balancer group I-P handling in 'northd' engine node (which is handled in the next commit). 'sync_to_sb_pb' engine node can be later expanded to sync other Port binding columns if required. Reviewed-by: Ales Musil Acked-by: Mark Michelson Signed-off-by: Numan Siddique Acked-by: Han Zhou --- northd/en-sync-sb.c | 52 ++++++++ northd/en-sync-sb.h | 5 + northd/inc-proc-northd.c | 8 +- northd/northd.c | 274 +++++++++++++++++++++++---------------- northd/northd.h | 3 + tests/ovn-northd.at | 17 ++- 6 files changed, 239 insertions(+), 120 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index f6dbd7b2b6..47ace03417 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -244,6 +244,58 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } +/* sync_to_sb_pb engine node functions. + * This engine node syncs the SB Port Bindings (partly). + * en_northd engine create the SB Port binding rows and + * updates most of the columns. + * This engine node updates the port binding columns which + * needs to be updated after northd engine is run. + */ + +void * +en_sync_to_sb_pb_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + return NULL; +} + +void +en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + struct northd_data *northd_data = engine_get_input_data("northd", node); + + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + engine_set_node_state(node, EN_UPDATED); +} + +void +en_sync_to_sb_pb_cleanup(void *data OVS_UNUSED) +{ + +} + +bool +sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + struct northd_data *nd = engine_get_input_data("northd", node); + if (!nd->change_tracked) { + return false; + } + + if (!sync_pbs_for_northd_ls_changes(&nd->tracked_ls_changes)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + return true; +} + /* static functions. */ static void sync_addr_set(struct ovsdb_idl_txn *ovnsb_txn, const char *name, diff --git a/northd/en-sync-sb.h b/northd/en-sync-sb.h index 06d2a57710..f08565eee1 100644 --- a/northd/en-sync-sb.h +++ b/northd/en-sync-sb.h @@ -22,4 +22,9 @@ void en_sync_to_sb_lb_run(struct engine_node *, void *data); void en_sync_to_sb_lb_cleanup(void *data); bool sync_to_sb_lb_northd_handler(struct engine_node *, void *data OVS_UNUSED); +void *en_sync_to_sb_pb_init(struct engine_node *, struct engine_arg *); +void en_sync_to_sb_pb_run(struct engine_node *, void *data); +void en_sync_to_sb_pb_cleanup(void *data); +bool sync_to_sb_pb_northd_handler(struct engine_node *, void *data OVS_UNUSED); + #endif /* end of EN_SYNC_SB_H */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 303b58d43f..e9e28c4bea 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -144,6 +144,7 @@ static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(port_group, "port_group"); static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); +static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, @@ -228,15 +229,20 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_northd, + sync_to_sb_pb_northd_handler); + /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. * Right now this engine syncs the SB Address_Set table, Port_Group table - * SB Meter/Meter_Band tables and SB Load_Balancer table. + * SB Meter/Meter_Band tables and SB Load_Balancer table and + * (partly) SB Port_Binding table. */ engine_add_input(&en_sync_to_sb, &en_sync_to_sb_addr_set, NULL); engine_add_input(&en_sync_to_sb, &en_port_group, NULL); engine_add_input(&en_sync_to_sb, &en_sync_meters, NULL); engine_add_input(&en_sync_to_sb, &en_sync_to_sb_lb, NULL); + engine_add_input(&en_sync_to_sb, &en_sync_to_sb_pb, NULL); engine_add_input(&en_sync_from_sb, &en_northd, sync_from_sb_northd_handler); diff --git a/northd/northd.c b/northd/northd.c index c5e1d1c5a4..5651ee38c5 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3527,8 +3527,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, ds_destroy(&s); sbrec_port_binding_set_external_ids(op->sb, &op->nbrp->external_ids); - - sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); } else { if (!lsp_is_router(op->nbsp)) { uint32_t queue_id = smap_get_int( @@ -3672,116 +3670,6 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, } else { sbrec_port_binding_set_options(op->sb, NULL); } - const char *nat_addresses = smap_get(&op->nbsp->options, - "nat-addresses"); - size_t n_nats = 0; - char **nats = NULL; - bool l3dgw_ports = op->peer && op->peer->od && - op->peer->od->n_l3dgw_ports; - if (nat_addresses && !strcmp(nat_addresses, "router")) { - if (op->peer && op->peer->od - && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, - "exclude-lb-vips-from-garp", false); - nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); - } - } else if (nat_addresses && (chassis || l3dgw_ports)) { - struct lport_addresses laddrs; - if (!extract_lsp_addresses(nat_addresses, &laddrs)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); - } else { - destroy_lport_addresses(&laddrs); - n_nats = 1; - nats = xcalloc(1, sizeof *nats); - struct ds nat_addr = DS_EMPTY_INITIALIZER; - ds_put_format(&nat_addr, "%s", nat_addresses); - if (l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&nat_addr, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - nats[0] = xstrdup(ds_cstr(&nat_addr)); - ds_destroy(&nat_addr); - } - } - - /* Add the router mac and IPv4 addresses to - * Port_Binding.nat_addresses so that GARP is sent for these - * IPs by the ovn-controller on which the distributed gateway - * router port resides if: - * - * - op->peer has 'reside-on-redirect-chassis' set and the - * the logical router datapath has distributed router port. - * - * - op->peer is distributed gateway router port. - * - * - op->peer's router is a gateway router and op has a localnet - * port. - * - * Note: Port_Binding.nat_addresses column is also used for - * sending the GARPs for the router port IPs. - * */ - bool add_router_port_garp = false; - if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { - if (is_l3dgw_port(op->peer)) { - add_router_port_garp = true; - } else if (smap_get_bool(&op->peer->nbrp->options, - "reside-on-redirect-chassis", false)) { - if (op->peer->od->n_l3dgw_ports == 1) { - add_router_port_garp = true; - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " - "set on logical router port %s, which " - "is on logical router %s, which has %" - PRIuSIZE" distributed gateway ports. This" - "option can only be used when there is " - "a single distributed gateway port.", - op->peer->key, op->peer->od->nbr->name, - op->peer->od->n_l3dgw_ports); - } - } - } else if (chassis && op->od->n_localnet_ports) { - add_router_port_garp = true; - } - - if (add_router_port_garp) { - struct ds garp_info = DS_EMPTY_INITIALIZER; - ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); - - for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; - i++) { - ds_put_format(&garp_info, " %s", - op->peer->lrp_networks.ipv4_addrs[i].addr_s); - } - - if (op->peer->od->n_l3dgw_ports) { - const struct ovn_port *l3dgw_port = ( - is_l3dgw_port(op->peer) - ? op->peer - : op->peer->od->l3dgw_ports[0]); - ds_put_format(&garp_info, " is_chassis_resident(%s)", - l3dgw_port->cr_port->json_key); - } - - n_nats++; - nats = xrealloc(nats, (n_nats * sizeof *nats)); - nats[n_nats - 1] = ds_steal_cstr(&garp_info); - ds_destroy(&garp_info); - } - sbrec_port_binding_set_nat_addresses(op->sb, - (const char **) nats, n_nats); - for (size_t i = 0; i < n_nats; i++) { - free(nats[i]); - } - free(nats); } sbrec_port_binding_set_parent_port(op->sb, op->nbsp->parent_name); @@ -4735,6 +4623,168 @@ sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, } } +/* Syncs the SB port binding for the ovn_port 'op'. Caller should make sure + * that the OVN SB IDL txn is not NULL. Presently it only syncs the nat + * column of port binding corresponding to the 'op->nbsp' */ +static void +sync_pb_for_op(struct ovn_port *op) +{ + if (lsp_is_router(op->nbsp)) { + const char *chassis = NULL; + if (op->peer && op->peer->od && op->peer->od->nbr) { + chassis = smap_get(&op->peer->od->nbr->options, "chassis"); + } + + const char *nat_addresses = smap_get(&op->nbsp->options, + "nat-addresses"); + size_t n_nats = 0; + char **nats = NULL; + bool l3dgw_ports = op->peer && op->peer->od && + op->peer->od->n_l3dgw_ports; + if (nat_addresses && !strcmp(nat_addresses, "router")) { + if (op->peer && op->peer->od + && (chassis || op->peer->od->n_l3dgw_ports)) { + bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + "exclude-lb-vips-from-garp", false); + nats = get_nat_addresses(op->peer, &n_nats, false, + !exclude_lb_vips); + } + } else if (nat_addresses && (chassis || l3dgw_ports)) { + struct lport_addresses laddrs; + if (!extract_lsp_addresses(nat_addresses, &laddrs)) { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "Error extracting nat-addresses."); + } else { + destroy_lport_addresses(&laddrs); + n_nats = 1; + nats = xcalloc(1, sizeof *nats); + struct ds nat_addr = DS_EMPTY_INITIALIZER; + ds_put_format(&nat_addr, "%s", nat_addresses); + if (l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&nat_addr, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + nats[0] = xstrdup(ds_cstr(&nat_addr)); + ds_destroy(&nat_addr); + } + } + + /* Add the router mac and IPv4 addresses to + * Port_Binding.nat_addresses so that GARP is sent for these + * IPs by the ovn-controller on which the distributed gateway + * router port resides if: + * + * - op->peer has 'reside-on-redirect-chassis' set and the + * the logical router datapath has distributed router port. + * + * - op->peer is distributed gateway router port. + * + * - op->peer's router is a gateway router and op has a localnet + * port. + * + * Note: Port_Binding.nat_addresses column is also used for + * sending the GARPs for the router port IPs. + * */ + bool add_router_port_garp = false; + if (op->peer && op->peer->nbrp && op->peer->od->n_l3dgw_ports) { + if (is_l3dgw_port(op->peer)) { + add_router_port_garp = true; + } else if (smap_get_bool(&op->peer->nbrp->options, + "reside-on-redirect-chassis", false)) { + if (op->peer->od->n_l3dgw_ports == 1) { + add_router_port_garp = true; + } else { + static struct vlog_rate_limit rl = + VLOG_RATE_LIMIT_INIT(1, 1); + VLOG_WARN_RL(&rl, "\"reside-on-redirect-chassis\" is " + "set on logical router port %s, which " + "is on logical router %s, which has %" + PRIuSIZE" distributed gateway ports. This" + "option can only be used when there is " + "a single distributed gateway port.", + op->peer->key, op->peer->od->nbr->name, + op->peer->od->n_l3dgw_ports); + } + } + } else if (chassis && op->od->n_localnet_ports) { + add_router_port_garp = true; + } + + if (add_router_port_garp) { + struct ds garp_info = DS_EMPTY_INITIALIZER; + ds_put_format(&garp_info, "%s", op->peer->lrp_networks.ea_s); + + for (size_t i = 0; i < op->peer->lrp_networks.n_ipv4_addrs; + i++) { + ds_put_format(&garp_info, " %s", + op->peer->lrp_networks.ipv4_addrs[i].addr_s); + } + + if (op->peer->od->n_l3dgw_ports) { + const struct ovn_port *l3dgw_port = ( + is_l3dgw_port(op->peer) + ? op->peer + : op->peer->od->l3dgw_ports[0]); + ds_put_format(&garp_info, " is_chassis_resident(%s)", + l3dgw_port->cr_port->json_key); + } + + n_nats++; + nats = xrealloc(nats, (n_nats * sizeof *nats)); + nats[n_nats - 1] = ds_steal_cstr(&garp_info); + ds_destroy(&garp_info); + } + sbrec_port_binding_set_nat_addresses(op->sb, + (const char **) nats, n_nats); + for (size_t i = 0; i < n_nats; i++) { + free(nats[i]); + } + free(nats); + } else { + sbrec_port_binding_set_nat_addresses(op->sb, NULL, 0); + } +} + +/* Sync the SB Port bindings which needs to be updated. + * Presently it syncs the nat column of port bindings corresponding to + * the logical switch ports. */ +void +sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +{ + ovs_assert(ovnsb_idl_txn); + + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, ls_ports) { + sync_pb_for_op(op); + } +} + +/* Sync the SB Port bindings for the added and updated logical switch ports + * of the tracked logical switches (from the northd engine node). */ +bool +sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *ls_changes) +{ + struct ls_change *ls_change; + LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { + struct ovn_port *op; + + LIST_FOR_EACH (op, list, &ls_change->added_ports) { + sync_pb_for_op(op); + } + + LIST_FOR_EACH (op, list, &ls_change->updated_ports) { + sync_pb_for_op(op); + } + } + + return true; +} + static bool ovn_port_add_tnlid(struct ovn_port *op, uint32_t tunnel_key) { diff --git a/northd/northd.h b/northd/northd.h index 0d206a4e52..5d8ac6fea0 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -365,4 +365,7 @@ const struct ovn_datapath *northd_get_datapath_for_port( void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct ovn_datapaths *ls_datapaths, struct hmap *lbs); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); +bool sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 34e10663d0..1f8b264bde 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9997,6 +9997,9 @@ check_recompute_counter() { lflow_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats lflow recompute) AT_CHECK([test x$lflow_recomp = x$2]) + + sync_sb_pb_recomp=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats sync_to_sb_pb recompute) + AT_CHECK([test x$sync_sb_pb_recomp = x$3]) } check ovn-nbctl --wait=hv ls-add ls0 @@ -10013,29 +10016,29 @@ check ovn-nbctl --wait=hv lsp-add ls0 lsp0-0 -- lsp-set-addresses lsp0-0 "unknow ovs-vsctl add-port br-int lsp0-0 -- set interface lsp0-0 external_ids:iface-id=lsp0-0 wait_for_ports_up check ovn-nbctl --wait=hv sync -check_recompute_counter 5 5 +check_recompute_counter 5 5 5 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-add ls0 lsp0-1 -- lsp-set-addresses lsp0-1 "aa:aa:aa:00:00:01 192.168.0.11" ovs-vsctl add-port br-int lsp0-1 -- set interface lsp0-1 external_ids:iface-id=lsp0-1 wait_for_ports_up check ovn-nbctl --wait=hv sync -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-add ls0 lsp0-2 -- lsp-set-addresses lsp0-2 "aa:aa:aa:00:00:02 192.168.0.12" ovs-vsctl add-port br-int lsp0-2 -- set interface lsp0-2 external_ids:iface-id=lsp0-2 wait_for_ports_up check ovn-nbctl --wait=hv sync -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-del lsp0-1 -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=hv lsp-set-addresses lsp0-2 "aa:aa:aa:00:00:88 192.168.0.88" -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 # Delete and re-add a LSP for several times continuously, to ensure # frequent operations do not trigger recompute when there are in-flight @@ -10049,12 +10052,12 @@ for i in $(seq 10); do check ovn-nbctl lsp-del lsp0-2 check ovn-nbctl lsp-add ls0 lsp0-2 -- lsp-set-addresses lsp0-2 "aa:aa:aa:00:00:02 192.168.0.12" done -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 # No change, no recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb sync -check_recompute_counter 0 0 +check_recompute_counter 0 0 0 CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Mon Sep 11 16:03:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1832350 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Rks383VvWz1ygM for ; Tue, 12 Sep 2023 02:04:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 929FC414C0; Mon, 11 Sep 2023 16:04:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 929FC414C0 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KYRABT9lIxmo; Mon, 11 Sep 2023 16:04:00 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 1CB524116F; Mon, 11 Sep 2023 16:03:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 1CB524116F Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E8A28C0072; Mon, 11 Sep 2023 16:03:58 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8FED5C0032 for ; Mon, 11 Sep 2023 16:03:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 69A8D80EA5 for ; Mon, 11 Sep 2023 16:03:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 69A8D80EA5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ha9bU81VYGm6 for ; Mon, 11 Sep 2023 16:03:56 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.osuosl.org (Postfix) with ESMTPS id 7688B80C81 for ; Mon, 11 Sep 2023 16:03:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 7688B80C81 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6F32E60028; Mon, 11 Sep 2023 16:03:22 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 11 Sep 2023 12:03:19 -0400 Message-ID: <20230911160319.179800-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230911155930.179283-1-numans@ovn.org> References: <20230911155930.179283-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v7 4/4] northd: Handle load balancer/group changes for a logical router. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique When a logical router gets updated due to load balancer or load balancer groups changes, it is now incrementally handled first in 'lb_data' engine node similar to how logical switch changes are handled. The tracking data of 'lb_data' is updated similarly so that northd engine handler - northd_handle_lb_data_changes() handles it. A new handler northd_handle_lr_changes() is added in the 'northd' engine node for logical router changes. This handler returns true if only load balancer or load balancer group columns are changed. It returns false for any other changes. northd_handle_lb_data_changes() also sets the logical router od's lb_ips accordingly. Below are the scale testing results done with these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling in northd engine node) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.131363 1.189994 3.213526 4.308134 4.394562 1.385713 173.214153 125 0 Namespace.add_ports 0.005176 0.005611 0.006476 0.019867 0.024206 0.006058 0.757188 125 0 WorkerNode.bind_port 0.033776 0.046343 0.054414 0.061719 0.063613 0.046815 11.703773 250 0 WorkerNode.ping_port 0.005156 0.006959 2.044939 3.655328 4.241496 0.627103 156.775832 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 3.233795 4.364926 5.400982 6.412803 7.409757 4.792270 599.033790 125 0 Namespace.add_ports 0.005230 0.006564 0.007379 0.019060 0.037490 0.007223 0.902930 125 0 WorkerNode.bind_port 0.033864 0.044052 0.049608 0.054849 0.056196 0.044005 11.001231 250 0 WorkerNode.ping_port 0.005334 2.060477 5.222422 6.267332 7.284001 2.323020 580.754964 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- Few observations: - The total time taken has come down significantly from 599 seconds to 173. - 99%ile with these patches is 4.3 seconds compared to 6.4 seconds for the main. - 99%ile with these patches is 3.2 seconds compared to 5.4 seconds for the main. - CPU utilization of northd during the test with these patches is between 100% to 300% which is almost the same as main. Main difference being that, with these patches the test duration is less and hence overall less CPU utilization. [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml Reviewed-by: Ales Musil Acked-by: Mark Michelson Signed-off-by: Numan Siddique --- lib/lb.c | 46 +++++- lib/lb.h | 9 ++ northd/en-lb-data.c | 327 +++++++++++++++++++++++++++++++-------- northd/en-lb-data.h | 14 ++ northd/en-northd.c | 20 +++ northd/en-northd.h | 1 + northd/inc-proc-northd.c | 5 +- northd/northd.c | 245 ++++++++++++++++++++++++++--- northd/northd.h | 2 + tests/ovn-northd.at | 42 ++--- 10 files changed, 598 insertions(+), 113 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index e6c9dc2be2..d0d562b6fb 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -794,6 +794,7 @@ ovn_lb_group_init(struct ovn_lb_group *lb_group, const struct uuid *lb_uuid = &nbrec_lb_group->load_balancer[i]->header_.uuid; lb_group->lbs[i] = ovn_northd_lb_find(lbs, lb_uuid); + lb_group->has_routable_lb |= lb_group->lbs[i]->routable; } } @@ -815,6 +816,7 @@ ovn_lb_group_cleanup(struct ovn_lb_group *lb_group) { ovn_lb_ip_set_destroy(lb_group->lb_ips); lb_group->lb_ips = NULL; + lb_group->has_routable_lb = false; free(lb_group->lbs); } @@ -1022,23 +1024,54 @@ ovn_lb_5tuples_destroy(struct hmap *tuples) void build_lrouter_lb_ips(struct ovn_lb_ip_set *lb_ips, const struct ovn_northd_lb *lb) +{ + add_ips_to_lb_ip_set(lb_ips, lb->routable, &lb->ips_v4, &lb->ips_v6); +} + +void +add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { const char *ip_address; - SSET_FOR_EACH (ip_address, &lb->ips_v4) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { sset_add(&lb_ips->ips_v4, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v4_routable, ip_address); } } - SSET_FOR_EACH (ip_address, &lb->ips_v6) { + SSET_FOR_EACH (ip_address, lb_ips_v6) { sset_add(&lb_ips->ips_v6, ip_address); - if (lb->routable) { + if (is_routable) { sset_add(&lb_ips->ips_v6_routable, ip_address); } } } +void +remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + const char *ip_address; + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lb_ips->ips_v4, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v4_routable, ip_address); + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lb_ips->ips_v6, ip_address); + if (is_routable) { + sset_find_and_delete(&lb_ips->ips_v6_routable, ip_address); + } + } +} + /* lb datapaths functions */ struct ovn_lb_datapaths * ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, @@ -1079,7 +1112,10 @@ ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, struct ovn_datapath **ods) { for (size_t i = 0; i < n; i++) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } } } diff --git a/lib/lb.h b/lib/lb.h index 74905c73b7..b8e3c1e8fb 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -138,6 +138,14 @@ void ovn_northd_lb_reinit(struct ovn_northd_lb *, void build_lrouter_lb_ips(struct ovn_lb_ip_set *, const struct ovn_northd_lb *); +void add_ips_to_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +void remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, + bool is_routable, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); struct ovn_lb_group { struct hmap_node hmap_node; @@ -145,6 +153,7 @@ struct ovn_lb_group { size_t n_lbs; struct ovn_northd_lb **lbs; struct ovn_lb_ip_set *lb_ips; + bool has_routable_lb; }; struct ovn_lb_group *ovn_lb_group_create( diff --git a/northd/en-lb-data.c b/northd/en-lb-data.c index 3604ba8226..7f6b1ffdc4 100644 --- a/northd/en-lb-data.c +++ b/northd/en-lb-data.c @@ -40,20 +40,30 @@ static void build_lbs(const struct nbrec_load_balancer_table *, const struct nbrec_load_balancer_group_table *, struct hmap *lbs, struct hmap *lb_groups); static void build_od_lb_map(const struct nbrec_logical_switch_table *, - struct hmap *od_lb_map); + const struct nbrec_logical_router_table *, + struct hmap *ls_lb_map, struct hmap *lr_lb_map); static struct od_lb_data *find_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); static void destroy_od_lb_data(struct od_lb_data *od_lb_data); static struct od_lb_data *create_od_lb_data(struct hmap *od_lb_map, const struct uuid *od_uuid); +static void handle_od_lb_changes(struct nbrec_load_balancer **, + size_t n_nbrec_lbs, + struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); +static void handle_od_lbgrp_changes(struct nbrec_load_balancer_group **, + size_t n_nbrec_lbs, + struct od_lb_data *, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *); static struct ovn_lb_group *create_lb_group( const struct nbrec_load_balancer_group *, struct hmap *lbs, struct hmap *lb_groups); static void destroy_tracked_data(struct ed_type_lb_data *); -static void add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *, - struct tracked_lb_data *, - bool health_checks); +static struct crupdated_lb *add_crupdated_lb_to_tracked_data( + struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); static void add_deleted_lb_to_tracked_data(struct ovn_northd_lb *, struct tracked_lb_data *, bool health_checks); @@ -64,6 +74,8 @@ static void add_deleted_lbgrp_to_tracked_data( struct ovn_lb_group *, struct tracked_lb_data *); static bool is_ls_lbs_changed(const struct nbrec_logical_switch *nbs); static bool is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs); +static bool is_lr_lbs_changed(const struct nbrec_logical_router *); +static bool is_lr_lbgrps_changed(const struct nbrec_logical_router *); /* 'lb_data' engine node manages the NB load balancers and load balancer * groups. For each NB LB, it creates 'struct ovn_northd_lb' and @@ -92,10 +104,13 @@ en_lb_data_run(struct engine_node *node, void *data) EN_OVSDB_GET(engine_get_input("NB_load_balancer_group", node)); const struct nbrec_logical_switch_table *nb_ls_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); + const struct nbrec_logical_router_table *nb_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); lb_data->tracked = false; build_lbs(nb_lb_table, nb_lbg_table, &lb_data->lbs, &lb_data->lbgrps); - build_od_lb_map(nb_ls_table, &lb_data->ls_lb_map); + build_od_lb_map(nb_ls_table, nb_lr_table, &lb_data->ls_lb_map, + &lb_data->lr_lb_map); engine_set_node_state(node, EN_UPDATED); } @@ -137,6 +152,7 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) uuid_hash(&tracked_lb->header_.uuid)); add_crupdated_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else if (nbrec_load_balancer_is_deleted(tracked_lb)) { lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); @@ -144,15 +160,44 @@ lb_data_load_balancer_handler(struct engine_node *node, void *data) hmap_remove(&lb_data->lbs, &lb->hmap_node); add_deleted_lb_to_tracked_data(lb, trk_lb_data, lb->health_checks); + trk_lb_data->has_routable_lb |= lb->routable; } else { /* Load balancer updated. */ lb = ovn_northd_lb_find(&lb_data->lbs, &tracked_lb->header_.uuid); ovs_assert(lb); bool health_checks = lb->health_checks; + struct sset old_ips_v4 = SSET_INITIALIZER(&old_ips_v4); + struct sset old_ips_v6 = SSET_INITIALIZER(&old_ips_v6); + sset_swap(&lb->ips_v4, &old_ips_v4); + sset_swap(&lb->ips_v6, &old_ips_v6); ovn_northd_lb_reinit(lb, tracked_lb); health_checks |= lb->health_checks; - add_crupdated_lb_to_tracked_data(lb, trk_lb_data, health_checks); + struct crupdated_lb *clb = add_crupdated_lb_to_tracked_data( + lb, trk_lb_data, health_checks); + trk_lb_data->has_routable_lb |= lb->routable; + + /* Determine the inserted and deleted vips and store them in + * the tracked data. */ + const char *vip; + SSET_FOR_EACH (vip, &lb->ips_v4) { + if (!sset_find_and_delete(&old_ips_v4, vip)) { + sset_add(&clb->inserted_vips_v4, vip); + } + } + + sset_swap(&old_ips_v4, &clb->deleted_vips_v4); + + SSET_FOR_EACH (vip, &lb->ips_v6) { + if (!sset_find_and_delete(&old_ips_v6, vip)) { + sset_add(&clb->inserted_vips_v6, vip); + } + } + + sset_swap(&old_ips_v6, &clb->deleted_vips_v6); + + sset_destroy(&old_ips_v4); + sset_destroy(&old_ips_v6); } } @@ -181,6 +226,8 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) for (size_t i = 0; i < lb_group->n_lbs; i++) { hmapx_add(&clbg->assoc_lbs, lb_group->lbs[i]); } + + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else if (nbrec_load_balancer_group_is_deleted(tracked_lb_group)) { struct ovn_lb_group *lb_group; lb_group = ovn_lb_group_find(&lb_data->lbgrps, @@ -188,6 +235,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) ovs_assert(lb_group); hmap_remove(&lb_data->lbgrps, &lb_group->hmap_node); add_deleted_lbgrp_to_tracked_data(lb_group, trk_lb_data); + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; } else { struct ovn_lb_group *lb_group; @@ -209,6 +257,7 @@ lb_data_load_balancer_group_handler(struct engine_node *node, void *data) build_lrouter_lb_ips(lb_group->lb_ips, lb_group->lbs[i]); } + trk_lb_data->has_routable_lb |= lb_group->has_routable_lb; struct crupdated_lbgrp *clbg = add_crupdated_lbgrp_to_tracked_data(lb_group, trk_lb_data); @@ -277,6 +326,7 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) if (!ls_lbs_changed && !ls_lbgrps_changed) { continue; } + changed = true; struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); codlb->od_uuid = nbs->header_.uuid; uuidset_init(&codlb->assoc_lbs); @@ -290,66 +340,77 @@ lb_data_logical_switch_handler(struct engine_node *node, void *data) } if (ls_lbs_changed) { - struct uuidset *pre_lb_uuids = od_lb_data->lbs; - od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); - uuidset_init(od_lb_data->lbs); - - for (size_t i = 0; i < nbs->n_load_balancer; i++) { - const struct uuid *lb_uuid = - &nbs->load_balancer[i]->header_.uuid; - uuidset_insert(od_lb_data->lbs, lb_uuid); - - struct uuidset_node *unode = uuidset_find(pre_lb_uuids, - lb_uuid); - - if (!unode || (nbrec_load_balancer_row_get_seqno( - nbs->load_balancer[i], - OVSDB_IDL_CHANGE_MODIFY) > 0)) { - /* Add this lb to the tracked data. */ - uuidset_insert(&codlb->assoc_lbs, lb_uuid); - changed = true; - } - - if (unode) { - uuidset_delete(pre_lb_uuids, unode); - } - } - if (!uuidset_is_empty(pre_lb_uuids)) { - trk_lb_data->has_dissassoc_lbs_from_od = true; - changed = true; - } - - uuidset_destroy(pre_lb_uuids); - free(pre_lb_uuids); + handle_od_lb_changes(nbs->load_balancer, nbs->n_load_balancer, + od_lb_data, lb_data, codlb); } if (ls_lbgrps_changed) { - struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; - od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); - uuidset_init(od_lb_data->lbgrps); - for (size_t i = 0; i < nbs->n_load_balancer_group; i++) { - const struct uuid *lbg_uuid = - &nbs->load_balancer_group[i]->header_.uuid; - uuidset_insert(od_lb_data->lbgrps, lbg_uuid); - - if (!uuidset_find_and_delete(pre_lbgrp_uuids, - lbg_uuid)) { - /* Add this lb group to the tracked data. */ - uuidset_insert(&codlb->assoc_lbgrps, lbg_uuid); - changed = true; - } - } + handle_od_lbgrp_changes(nbs->load_balancer_group, + nbs->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } - if (!uuidset_is_empty(pre_lbgrp_uuids)) { - trk_lb_data->has_dissassoc_lbgrps_from_od = true; - changed = true; - } + ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + } + } - uuidset_destroy(pre_lbgrp_uuids); - free(pre_lbgrp_uuids); + if (changed) { + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +bool +lb_data_logical_router_handler(struct engine_node *node, void *data) +{ + struct ed_type_lb_data *lb_data = (struct ed_type_lb_data *) data; + const struct nbrec_logical_router_table *nbrec_lr_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)); + + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + lb_data->tracked = true; + + bool changed = false; + const struct nbrec_logical_router *nbr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (nbr, nbrec_lr_table) { + if (nbrec_logical_router_is_deleted(nbr)) { + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (od_lb_data) { + hmap_remove(&lb_data->lr_lb_map, &od_lb_data->hmap_node); + destroy_od_lb_data(od_lb_data); + } + } else { + bool lr_lbs_changed = is_lr_lbs_changed(nbr); + bool lr_lbgrps_changed = is_lr_lbgrps_changed(nbr); + if (!lr_lbs_changed && !lr_lbgrps_changed) { + continue; + } + changed = true; + struct crupdated_od_lb_data *codlb = xzalloc(sizeof *codlb); + codlb->od_uuid = nbr->header_.uuid; + uuidset_init(&codlb->assoc_lbs); + uuidset_init(&codlb->assoc_lbgrps); + + struct od_lb_data *od_lb_data = + find_od_lb_data(&lb_data->lr_lb_map, &nbr->header_.uuid); + if (!od_lb_data) { + od_lb_data = create_od_lb_data(&lb_data->lr_lb_map, + &nbr->header_.uuid); } - ovs_list_insert(&trk_lb_data->crupdated_ls_lbs, &codlb->list_node); + if (lr_lbs_changed) { + handle_od_lb_changes(nbr->load_balancer, nbr->n_load_balancer, + od_lb_data, lb_data, codlb); + } + + if (lr_lbgrps_changed) { + handle_od_lbgrp_changes(nbr->load_balancer_group, + nbr->n_load_balancer_group, + od_lb_data, lb_data, codlb); + } + + ovs_list_insert(&trk_lb_data->crupdated_lr_lbs, &codlb->list_node); } } @@ -366,6 +427,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&lb_data->lbs); hmap_init(&lb_data->lbgrps); hmap_init(&lb_data->ls_lb_map); + hmap_init(&lb_data->lr_lb_map); struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; hmap_init(&trk_lb_data->crupdated_lbs); @@ -373,6 +435,7 @@ lb_data_init(struct ed_type_lb_data *lb_data) hmap_init(&trk_lb_data->crupdated_lbgrps); hmapx_init(&trk_lb_data->deleted_lbgrps); ovs_list_init(&trk_lb_data->crupdated_ls_lbs); + ovs_list_init(&trk_lb_data->crupdated_lr_lbs); } static void @@ -396,6 +459,11 @@ lb_data_destroy(struct ed_type_lb_data *lb_data) } hmap_destroy(&lb_data->ls_lb_map); + HMAP_FOR_EACH_POP (od_lb_data, hmap_node, &lb_data->lr_lb_map) { + destroy_od_lb_data(od_lb_data); + } + hmap_destroy(&lb_data->lr_lb_map); + destroy_tracked_data(lb_data); hmap_destroy(&lb_data->tracked_lb_data.crupdated_lbs); hmapx_destroy(&lb_data->tracked_lb_data.deleted_lbs); @@ -440,7 +508,8 @@ create_lb_group(const struct nbrec_load_balancer_group *nbrec_lb_group, static void build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, - struct hmap *od_lb_map) + const struct nbrec_logical_router_table *nbrec_lr_table, + struct hmap *ls_lb_map, struct hmap *lr_lb_map) { const struct nbrec_logical_switch *nbrec_ls; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbrec_ls, nbrec_ls_table) { @@ -448,17 +517,35 @@ build_od_lb_map(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct od_lb_data *od_lb_data = - create_od_lb_data(od_lb_map, &nbrec_ls->header_.uuid); + struct od_lb_data *ls_lb_data = + create_od_lb_data(ls_lb_map, &nbrec_ls->header_.uuid); for (size_t i = 0; i < nbrec_ls->n_load_balancer; i++) { - uuidset_insert(od_lb_data->lbs, + uuidset_insert(ls_lb_data->lbs, &nbrec_ls->load_balancer[i]->header_.uuid); } for (size_t i = 0; i < nbrec_ls->n_load_balancer_group; i++) { - uuidset_insert(od_lb_data->lbgrps, + uuidset_insert(ls_lb_data->lbgrps, &nbrec_ls->load_balancer_group[i]->header_.uuid); } } + + const struct nbrec_logical_router *nbrec_lr; + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH (nbrec_lr, nbrec_lr_table) { + if (!nbrec_lr->n_load_balancer && !nbrec_lr->n_load_balancer_group) { + continue; + } + + struct od_lb_data *lr_lb_data = + create_od_lb_data(lr_lb_map, &nbrec_lr->header_.uuid); + for (size_t i = 0; i < nbrec_lr->n_load_balancer; i++) { + uuidset_insert(lr_lb_data->lbs, + &nbrec_lr->load_balancer[i]->header_.uuid); + } + for (size_t i = 0; i < nbrec_lr->n_load_balancer_group; i++) { + uuidset_insert(lr_lb_data->lbgrps, + &nbrec_lr->load_balancer_group[i]->header_.uuid); + } + } } static struct od_lb_data * @@ -500,6 +587,84 @@ destroy_od_lb_data(struct od_lb_data *od_lb_data) free(od_lb_data); } +static void +handle_od_lb_changes(struct nbrec_load_balancer **nbrec_lbs, + size_t n_nbrec_lbs, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lb_uuids = od_lb_data->lbs; + od_lb_data->lbs = xzalloc(sizeof *od_lb_data->lbs); + uuidset_init(od_lb_data->lbs); + + for (size_t i = 0; i < n_nbrec_lbs; i++) { + const struct uuid *lb_uuid = &nbrec_lbs[i]->header_.uuid; + uuidset_insert(od_lb_data->lbs, lb_uuid); + + struct uuidset_node *unode = uuidset_find(pre_lb_uuids, lb_uuid); + + if (!unode || (nbrec_load_balancer_row_get_seqno( + nbrec_lbs[i], OVSDB_IDL_CHANGE_MODIFY) > 0)) { + /* Add this lb to the tracked data. */ + uuidset_insert(&codlb->assoc_lbs, lb_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_northd_lb *lb = ovn_northd_lb_find(&lb_data->lbs, + lb_uuid); + ovs_assert(lb); + trk_lb_data->has_routable_lb |= lb->routable; + } + } + + if (unode) { + uuidset_delete(pre_lb_uuids, unode); + } + } + + if (!uuidset_is_empty(pre_lb_uuids)) { + trk_lb_data->has_dissassoc_lbs_from_od = true; + } + + uuidset_destroy(pre_lb_uuids); + free(pre_lb_uuids); +} + +static void +handle_od_lbgrp_changes(struct nbrec_load_balancer_group **nbrec_lbgrps, + size_t n_nbrec_lbgrps, struct od_lb_data *od_lb_data, + struct ed_type_lb_data *lb_data, + struct crupdated_od_lb_data *codlb) +{ + struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + struct uuidset *pre_lbgrp_uuids = od_lb_data->lbgrps; + od_lb_data->lbgrps = xzalloc(sizeof *od_lb_data->lbgrps); + uuidset_init(od_lb_data->lbgrps); + for (size_t i = 0; i < n_nbrec_lbgrps; i++) { + const struct uuid *lbgrp_uuid = &nbrec_lbgrps[i]->header_.uuid; + uuidset_insert(od_lb_data->lbgrps, lbgrp_uuid); + + if (!uuidset_find_and_delete(pre_lbgrp_uuids, lbgrp_uuid)) { + /* Add this lb group to the tracked data. */ + uuidset_insert(&codlb->assoc_lbgrps, lbgrp_uuid); + + if (!trk_lb_data->has_routable_lb) { + struct ovn_lb_group *lbgrp = + ovn_lb_group_find(&lb_data->lbgrps, lbgrp_uuid); + ovs_assert(lbgrp); + trk_lb_data->has_routable_lb |= lbgrp->has_routable_lb; + } + } + } + + if (!uuidset_is_empty(pre_lbgrp_uuids)) { + trk_lb_data->has_dissassoc_lbgrps_from_od = true; + } + + uuidset_destroy(pre_lbgrp_uuids); + free(pre_lbgrp_uuids); +} + static void destroy_tracked_data(struct ed_type_lb_data *lb_data) { @@ -508,6 +673,7 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) lb_data->tracked_lb_data.has_dissassoc_lbs_from_lbgrps = false; lb_data->tracked_lb_data.has_dissassoc_lbs_from_od = false; lb_data->tracked_lb_data.has_dissassoc_lbgrps_from_od = false; + lb_data->tracked_lb_data.has_routable_lb = false; struct hmapx_node *node; HMAPX_FOR_EACH_SAFE (node, &lb_data->tracked_lb_data.deleted_lbs) { @@ -523,6 +689,10 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) struct crupdated_lb *clb; HMAP_FOR_EACH_POP (clb, hmap_node, &lb_data->tracked_lb_data.crupdated_lbs) { + sset_destroy(&clb->inserted_vips_v4); + sset_destroy(&clb->inserted_vips_v6); + sset_destroy(&clb->deleted_vips_v4); + sset_destroy(&clb->deleted_vips_v6); free(clb); } @@ -541,9 +711,16 @@ destroy_tracked_data(struct ed_type_lb_data *lb_data) uuidset_destroy(&codlb->assoc_lbgrps); free(codlb); } + LIST_FOR_EACH_SAFE (codlb, list_node, + &lb_data->tracked_lb_data.crupdated_lr_lbs) { + ovs_list_remove(&codlb->list_node); + uuidset_destroy(&codlb->assoc_lbs); + uuidset_destroy(&codlb->assoc_lbgrps); + free(codlb); + } } -static void +static struct crupdated_lb * add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, struct tracked_lb_data *tracked_lb_data, bool health_checks) @@ -552,9 +729,15 @@ add_crupdated_lb_to_tracked_data(struct ovn_northd_lb *lb, clb->lb = lb; hmap_insert(&tracked_lb_data->crupdated_lbs, &clb->hmap_node, uuid_hash(&lb->nlb->header_.uuid)); + sset_init(&clb->inserted_vips_v4); + sset_init(&clb->inserted_vips_v6); + sset_init(&clb->deleted_vips_v4); + sset_init(&clb->deleted_vips_v6); if (health_checks) { tracked_lb_data->has_health_checks = true; } + + return clb; } static void @@ -600,3 +783,17 @@ is_ls_lbgrps_changed(const struct nbrec_logical_switch *nbs) { || nbrec_logical_switch_is_updated(nbs, NBREC_LOGICAL_SWITCH_COL_LOAD_BALANCER_GROUP)); } + +static bool +is_lr_lbs_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER)); +} + +static bool +is_lr_lbgrps_changed(const struct nbrec_logical_router *nbr) { + return ((nbrec_logical_router_is_new(nbr) && nbr->n_load_balancer_group) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP)); +} diff --git a/northd/en-lb-data.h b/northd/en-lb-data.h index d802d000d4..51c8a9e5c5 100644 --- a/northd/en-lb-data.h +++ b/northd/en-lb-data.h @@ -6,6 +6,7 @@ #include "openvswitch/hmap.h" #include "include/openvswitch/list.h" #include "lib/hmapx.h" +#include "lib/sset.h" #include "lib/uuidset.h" #include "lib/inc-proc-eng.h" @@ -17,6 +18,10 @@ struct crupdated_lb { struct hmap_node hmap_node; struct ovn_northd_lb *lb; + struct sset inserted_vips_v4; + struct sset inserted_vips_v6; + struct sset deleted_vips_v4; + struct sset deleted_vips_v6; }; struct crupdated_lbgrp { @@ -54,6 +59,10 @@ struct tracked_lb_data { * 'struct crupdated_od_lb_data' */ struct ovs_list crupdated_ls_lbs; + /* List of logical router <-> lb changes. List node is + * 'struct crupdated_od_lb_data' */ + struct ovs_list crupdated_lr_lbs; + /* Indicates if any of the tracked lb has health checks enabled. */ bool has_health_checks; @@ -66,6 +75,9 @@ struct tracked_lb_data { /* Indicates if a lb group was disassociated from a logical switch. */ bool has_dissassoc_lbgrps_from_od; + + /* Indicates if any lb (in the tracked data) has 'routable' flag set. */ + bool has_routable_lb; }; /* struct which maintains the data of the engine node lb_data. */ @@ -76,6 +88,7 @@ struct ed_type_lb_data { /* hmap of load balancer groups. hmap node is 'struct ovn_lb_group *' */ struct hmap lbgrps; struct hmap ls_lb_map; + struct hmap lr_lb_map; /* tracked data*/ bool tracked; @@ -90,5 +103,6 @@ void en_lb_data_clear_tracked_data(void *data); bool lb_data_load_balancer_handler(struct engine_node *, void *data); bool lb_data_load_balancer_group_handler(struct engine_node *, void *data); bool lb_data_logical_switch_handler(struct engine_node *, void *data); +bool lb_data_logical_router_handler(struct engine_node *, void *data); #endif /* end of EN_NORTHD_LB_DATA_H */ diff --git a/northd/en-northd.c b/northd/en-northd.c index 8487b003f7..aa0f20f0c2 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -196,6 +196,26 @@ northd_sb_port_binding_handler(struct engine_node *node, return true; } +bool +northd_nb_logical_router_handler(struct engine_node *node, + void *data) +{ + struct northd_data *nd = data; + struct northd_input input_data; + + northd_get_input_data(node, &input_data); + + if (!northd_handle_lr_changes(&input_data, nd)) { + return false; + } + + if (nd->change_tracked) { + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + bool northd_lb_data_handler(struct engine_node *node, void *data) { diff --git a/northd/en-northd.h b/northd/en-northd.h index 3c77b64bb2..5a88871760 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -16,6 +16,7 @@ void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); +bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); bool northd_lb_data_handler(struct engine_node *, void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index e9e28c4bea..8b08171179 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -158,8 +158,9 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, lb_data_load_balancer_group_handler); engine_add_input(&en_lb_data, &en_nb_logical_switch, lb_data_logical_switch_handler); + engine_add_input(&en_lb_data, &en_nb_logical_router, + lb_data_logical_router_handler); - engine_add_input(&en_northd, &en_nb_logical_router, NULL); engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); @@ -184,6 +185,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); + engine_add_input(&en_northd, &en_nb_logical_router, + northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); diff --git a/northd/northd.c b/northd/northd.c index 5651ee38c5..82de6be115 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4167,54 +4167,63 @@ static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr); static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr); + static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) +add_neigh_ips_to_lrouter(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) { /* If configured to not reply to any neighbor requests for all VIPs * return early. */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_NONE) { + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { return; } + const char *ip_address; + /* If configured to reply to neighbor requests for all VIPs force them * all to be considered "reachable". */ - if (lb->neigh_mode == LB_NEIGH_RESPOND_ALL) { - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - sset_add(&od->lb_ips->ips_v4_reachable, lb->vips[i].vip_str); - } else { - sset_add(&od->lb_ips->ips_v6_reachable, lb->vips[i].vip_str); - } + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&od->lb_ips->ips_v6_reachable, ip_address); } + return; } /* Otherwise, a VIP is reachable if there's at least one router * subnet that includes it. */ - ovs_assert(lb->neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - for (size_t i = 0; i < lb->n_vips; i++) { - if (lb->vips[i].address_family == AF_INET) { - ovs_be32 vip_ip4 = in6_addr_get_mapped_ipv4(&lb->vips[i].vip); - struct ovn_port *op; + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { if (lrouter_port_ipv4_reachable(op, vip_ip4)) { sset_add(&od->lb_ips->ips_v4_reachable, - lb->vips[i].vip_str); + ip_address); break; } } - } else { - struct ovn_port *op; + } + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &lb->vips[i].vip)) { + if (lrouter_port_ipv6_reachable(op, &vip)) { sset_add(&od->lb_ips->ips_v6_reachable, - lb->vips[i].vip_str); + ip_address); break; } } @@ -4222,6 +4231,34 @@ build_lrouter_lb_reachable_ips(struct ovn_datapath *od, } } +static void +remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); + } +} + +static void +build_lrouter_lb_reachable_ips(struct ovn_datapath *od, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + + static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -5419,6 +5456,95 @@ fail: return false; } +/* Returns true if the logical router has changes which are not + * incrementally handled. + * Presently supports i-p for the below changes: + * - load balancers and load balancer groups. + */ +static bool +check_unsupported_inc_proc_for_lr_changes( + const struct nbrec_logical_router *lr) +{ + /* Check if the columns are changed in this row. */ + enum nbrec_logical_router_column_id col; + for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { + if (nbrec_logical_router_is_updated(lr, col)) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || + col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + continue; + } + return true; + } + } + + /* Check if the referenced rows are changed. + XXX: Need a better OVSDB IDL interface for this check. */ + for (size_t i = 0; i < lr->n_ports; i++) { + if (nbrec_logical_router_port_row_get_seqno(lr->ports[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + if (lr->copp && nbrec_copp_row_get_seqno(lr->copp, + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + for (size_t i = 0; i < lr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(lr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_policies; i++) { + if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + for (size_t i = 0; i < lr->n_static_routes; i++) { + if (nbrec_logical_router_static_route_row_get_seqno( + lr->static_routes[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + return false; +} + +/* Return true if changes are handled incrementally, false otherwise. + * When there are any changes, try to track what's exactly changed and set + * northd_data->change_tracked accordingly: change tracked - true, otherwise, + * false. + * Note: Changes to load balancer and load balancer groups associated with + * the logical routers are handled separately in the lb_data change + * handlers (northd_handle_lb_data_changes_pre_od and + * northd_handle_lb_data_changes_post_od). + * */ +bool +northd_handle_lr_changes(const struct northd_input *ni, + struct northd_data *nd) +{ + const struct nbrec_logical_router *changed_lr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED (changed_lr, + ni->nbrec_logical_router_table) { + if (nbrec_logical_router_is_new(changed_lr) || + nbrec_logical_router_is_deleted(changed_lr)) { + goto fail; + } + + /* Presently only able to handle load balancer and + * load balancer group changes. */ + if (check_unsupported_inc_proc_for_lr_changes(changed_lr)) { + goto fail; + } + } + + return true; +fail: + destroy_northd_data_tracked_changes(nd); + return false; +} + bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, @@ -5529,6 +5655,10 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, return false; } + if (trk_lb_data->has_routable_lb) { + return false; + } + struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; struct ovn_datapath *od; @@ -5619,6 +5749,45 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, init_lb_for_datapath(od); } + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + ovs_assert(od); + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + &uuidnode->uuid); + ovs_assert(lbgrp_dps); + ovn_lb_group_datapaths_add_lr(lbgrp_dps, od); + + /* Associate all the lbs of the lbgrp to the datapath 'od' */ + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(od, lb_dps->lb); + } + } + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + } + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { lb = clb->lb; const struct uuid *lb_uuid = &lb->nlb->header_.uuid; @@ -5632,6 +5801,29 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), + lb_dps->nb_lr_map) { + od = lr_datapaths->array[index]; + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(od->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(od, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + } } HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, &trk_lb_data->crupdated_lbgrps) { @@ -5648,8 +5840,19 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_uuid = &lb->nlb->header_.uuid; lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + ovn_lb_datapaths_add_lr(lb_dps, 1, &od); + + /* Re-evaluate 'od->has_lb_vip' */ + init_lb_for_datapath(od); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + } + for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { - od = lbgrp_dps->ls[i]; + od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); /* Re-evaluate 'od->has_lb_vip' */ diff --git a/northd/northd.h b/northd/northd.h index 5d8ac6fea0..4d030b10da 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -328,6 +328,8 @@ void ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, bool northd_handle_ls_changes(struct ovsdb_idl_txn *, const struct northd_input *, struct northd_data *); +bool northd_handle_lr_changes(const struct northd_input *, + struct northd_data *); void destroy_northd_data_tracked_changes(struct northd_data *); void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 1f8b264bde..399b2d91cf 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10487,8 +10487,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add lb1 to lr0 and then disassociate check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10496,7 +10496,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10504,7 +10504,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10512,7 +10512,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10520,13 +10520,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10618,8 +10618,8 @@ check_engine_stats lflow recompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10627,7 +10627,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10635,7 +10635,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10643,7 +10643,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10651,13 +10651,13 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_router lr0 load_balancer_group -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute @@ -10712,8 +10712,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10734,8 +10734,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 -check_engine_stats lb_data norecompute nocompute -check_engine_stats northd recompute nocompute +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10748,7 +10748,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 -check_engine_stats lb_data norecompute nocompute +check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10758,7 +10758,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10767,7 +10767,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute -check_engine_stats northd recompute nocompute +check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE