From patchwork Fri Aug 18 08:58:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1822765 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RRwn65P51z1ygW for ; Fri, 18 Aug 2023 19:00:10 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id AEDBA61674; Fri, 18 Aug 2023 09:00:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org AEDBA61674 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VbZI8tID7nQ4; Fri, 18 Aug 2023 09:00:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9411461647; Fri, 18 Aug 2023 09:00:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9411461647 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5E5F2C0039; Fri, 18 Aug 2023 09:00:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0413AC0039 for ; Fri, 18 Aug 2023 08:59:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id A39A961580 for ; Fri, 18 Aug 2023 08:58:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A39A961580 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7UjiO3IrQMdk for ; Fri, 18 Aug 2023 08:58:48 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id 26C9C615E4 for ; Fri, 18 Aug 2023 08:58:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 26C9C615E4 Received: by mail.gandi.net (Postfix) with ESMTPSA id BC7021C0002; Fri, 18 Aug 2023 08:58:43 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 18 Aug 2023 14:28:38 +0530 Message-Id: <20230818085838.1031124-1-numans@ovn.org> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230818085606.1030792-1-numans@ovn.org> References: <20230818085606.1030792-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v6 14/16] northd: Handle load balancer change in lflow engine. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique If a load balancer or load balancer group is changed, then the lflow engine handles these changes incrementally. This is possible because we maintain lb to lflow references (using objdep_mgr) for each lb in the 'struct lb_datapaths'. northd engine adds the changed load balancers in the northd tracked data. And for each tracked lb, lflow engine handler - Deletes the old lflow references and - Rebuilds the lflows. This patch also rebuilds the logical flows for the logical switch and logical router datapaths associated with the changed load balancers. Since we already maintain the logical flows related to load balancers in a separte objtype in the objdep_mgr of each datapath, it becomes easier to rebuild on these lflows. Below are the scale testing results done with all of these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling in both northd and lflow engine nodes) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.135651 1.130527 1.179357 1.201410 2.180203 0.674606 84.325717 125 0 Namespace.add_ports 0.005218 0.005678 0.006457 0.018936 0.020812 0.006182 0.772796 125 0 WorkerNode.bind_port 0.033631 0.043287 0.051171 0.058223 0.062819 0.043839 10.959757 250 0 WorkerNode.ping_port 0.005460 0.006791 1.041434 1.064807 1.069957 0.274352 68.587878 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 4.377260 6.486962 7.502040 8.322587 8.334701 6.559002 819.875306 125 0 Namespace.add_ports 0.005112 0.005484 0.005953 0.009153 0.011452 0.005662 0.707752 125 0 WorkerNode.bind_port 0.035360 0.042732 0.049152 0.053698 0.056635 0.043215 10.803700 250 0 WorkerNode.ping_port 0.005338 1.599904 7.229649 7.798039 8.206537 3.209860 802.464911 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- Signed-off-by: Numan Siddique --- northd/en-lflow.c | 17 +- northd/en-northd.c | 21 ++- northd/en-sync-sb.c | 5 +- northd/northd.c | 450 +++++++++++++++++++++++++++++++++++++++++--- northd/northd.h | 45 ++++- tests/ovn-northd.at | 159 ++++++++++++---- 6 files changed, 608 insertions(+), 89 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index dacaaa549a..56fe564c82 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -105,20 +105,21 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if lb related data has changed. */ - if (northd_data->trk_northd_changes.lb_changed) { - return false; - } - const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->trk_northd_changes, - &lflow_input, lflow_data)) { + if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_ovn_ports, + &lflow_input, lflow_data)) { + return false; + } + + if (!lflow_handle_northd_lb_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_lbs, + &lflow_input, lflow_data)) { return false; } diff --git a/northd/en-northd.c b/northd/en-northd.c index 7a4a1d8629..651c53a6ce 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -270,11 +270,15 @@ northd_lb_data_handler_pre_od(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_northd_changes)) { return false; } - engine_set_node_state(node, EN_UPDATED); + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } return true; } @@ -296,15 +300,16 @@ northd_lb_data_handler_post_od(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_northd_changes)) { return false; } - /* Indicate the depedendant engine nodes that load balancer/group - * related data has changed (including association to logical - * switch/router). */ - nd->trk_northd_changes.lb_changed = true; - engine_set_node_state(node, EN_UPDATED); + if (northd_has_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 552ed56452..3ef3be6af1 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -244,10 +244,11 @@ bool sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (nd->change_tracked) { + if (nd->change_tracked && + northd_has_only_ports_in_tracked_data(&nd->trk_northd_changes)) { /* There are only NB LSP related changes and these can be safely * ignore and returned true. However in case the northd engine - * tracking data includes other changes, we need to do additional + * tracking data includes other ports, we need to do additional * checks before safely ignoring. */ return true; } diff --git a/northd/northd.c b/northd/northd.c index 3aefe42334..2a490e9ab6 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5011,19 +5011,68 @@ destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) } } +static void +destroy_tracked_lb_datapaths(struct tracked_lb_datapaths *trk_lbs) +{ + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_lbs->deleted) { + ovn_lb_datapaths_destroy(hmapx_node->data); + hmapx_delete(&trk_lbs->deleted, hmapx_node); + } + + hmapx_clear(&trk_lbs->crupdated); + + if (trk_lbs->nb_ls_map) { + bitmap_free(trk_lbs->nb_ls_map); + trk_lbs->nb_ls_map = NULL; + } + + trk_lbs->n_nb_ls = 0; + + if (trk_lbs->nb_lr_map) { + bitmap_free(trk_lbs->nb_lr_map); + trk_lbs->nb_lr_map = NULL; + } + + trk_lbs->n_nb_lr = 0; +} + void destroy_northd_data_tracked_changes(struct northd_data *nd) { struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); - + destroy_tracked_lb_datapaths(&trk_changes->trk_lbs); nd->change_tracked = false; - trk_changes->lb_changed = false; +} + +bool northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (trk_nd_changes->trk_lbs.n_nb_ls + || trk_nd_changes->trk_lbs.n_nb_lr + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); +} + +bool northd_has_only_ports_in_tracked_data( + struct northd_tracked_data *trk_nd_changes) +{ + return (!trk_nd_changes->trk_lbs.n_nb_ls + && !trk_nd_changes->trk_lbs.n_nb_lr + && hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + && hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + && (!hmap_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmap_is_empty(&trk_nd_changes->trk_ovn_ports.deleted))); } static void add_op_to_northd_tracked_ports(struct hmap *tracked_ovn_ports, - struct ovn_port *op) + struct ovn_port *op, + uint8_t changes) { struct tracked_ovn_port *trk_op; uint32_t hash = hash_string(op->key, 0); @@ -5038,6 +5087,60 @@ add_op_to_northd_tracked_ports(struct hmap *tracked_ovn_ports, trk_op->op = op; hmap_insert(tracked_ovn_ports, &trk_op->hmap_node, hash); } + + trk_op->changes |= changes; +} + +static void +add_od_to_northd_lb_dps_track_data(struct northd_tracked_data *nd_changes, + struct ovn_datapath *od, + size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct tracked_lb_datapaths *trk_lbs = &nd_changes->trk_lbs; + + if (od->nbs) { + if (!trk_lbs->nb_ls_map) { + trk_lbs->nb_ls_map = bitmap_allocate(n_ls_datapaths); + } + + if (!bitmap_is_set(trk_lbs->nb_ls_map, od->index)) { + bitmap_set1(trk_lbs->nb_ls_map, od->index); + trk_lbs->n_nb_ls++; + + /* Also add the router ports of the logical switch + * to the northd tracked data. */ + for (size_t i = 0; i < od->n_router_ports; i++) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, od->router_ports[i], + EN_TRACKED_OP_LB_CHANGED); + } + } + } else { + ovs_assert(od->nbr); + if (!trk_lbs->nb_lr_map) { + trk_lbs->nb_lr_map = bitmap_allocate(n_lr_datapaths); + } + + if (!bitmap_is_set(trk_lbs->nb_lr_map, od->index)) { + bitmap_set1(trk_lbs->nb_lr_map, od->index); + trk_lbs->n_nb_lr++; + + /* Also add the peer (logical switch port) of logical router ports + * to the northd tracked data. */ + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &od->ports) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, op, + EN_TRACKED_OP_LB_CHANGED); + if (op->peer) { + add_op_to_northd_tracked_ports( + &nd_changes->trk_ovn_ports.updated, op->peer, + EN_TRACKED_OP_LB_CHANGED); + } + } + } + } } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5349,7 +5452,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->created, op); + add_op_to_northd_tracked_ports(&trk_ports->created, op, + EN_TRACKED_OP_LPORT_CHANGED); } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5381,7 +5485,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->updated, op); + add_op_to_northd_tracked_ports(&trk_ports->updated, op, + EN_TRACKED_OP_LPORT_CHANGED); } op->visited = true; } @@ -5397,7 +5502,8 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, * impacted by this deletion. Fallback to recompute. */ goto fail; } - add_op_to_northd_tracked_ports(&trk_ports->deleted, op); + add_op_to_northd_tracked_ports(&trk_ports->deleted, op, + EN_TRACKED_OP_LPORT_CHANGED); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); @@ -5652,7 +5758,8 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { struct ovn_lb_datapaths *lb_dps; struct ovn_northd_lb *lb; @@ -5665,6 +5772,9 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); + /* Add the deleted lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.deleted, lb_dps); + /* Re-evaluate 'od->has_lb_vip for od's associated with the * deleted lb. */ size_t index; @@ -5672,10 +5782,14 @@ northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *trk_lb_data, lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); - ovn_lb_datapaths_destroy(lb_dps); } struct crupdated_lb *clb; @@ -5731,7 +5845,8 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { ovs_assert(!trk_lb_data->has_health_checks); ovs_assert(!trk_lb_data->has_dissassoc_lbs_from_lb_grops); @@ -5751,6 +5866,9 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5766,11 +5884,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { @@ -5786,6 +5912,9 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5805,11 +5934,19 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } struct crupdated_lb *clb; @@ -5825,6 +5962,11 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, od = ls_datapaths->array[index]; /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), @@ -5848,7 +5990,15 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, add_neigh_ips_to_lrouter(od, lb->neigh_mode, &clb->inserted_vips_v4, &clb->inserted_vips_v6); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } struct ovn_lb_group *lbgrp; @@ -5877,6 +6027,11 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + + /* Add the lr datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { @@ -5885,7 +6040,15 @@ northd_handle_lb_data_changes_post_od(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + add_od_to_northd_lb_dps_track_data(nd_changes, od, + ods_size(ls_datapaths), + ods_size(lr_datapaths)); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -17814,6 +17977,45 @@ unlink_lflows(struct ovn_datapath *od, enum objdep_type objdep_type, unlink_objres_lflows(res_node, od, lflow_data, lflowdep_mgr); } +/* Unlinks all the lflows from the lflow dep mgr for all the set datapaths + * for the specified type 'objdep_type' and resource name 'res_name'. */ +static void +unlink_all_ods_objres_lflows(struct objdep_mgr *lflowdep_mgr, + enum objdep_type objdep_type, + const char *res_name, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct hmap *lflows_hash_map) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + lflowdep_mgr, objdep_type, res_name); + + if (!res_node) { + return; + } + + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + struct ovn_lflow *lflow = ovn_lflow_uuid_find(lflows_hash_map, + obj_uuid); + if (!lflow) { + continue; + } + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = n_ls_datapaths; + } else { + n_datapaths = n_lr_datapaths; + } + size_t index; + BITMAP_FOR_EACH_1 (index, n_datapaths, lflow->dpg_bitmap) { + bitmap_set0(lflow->dpg_bitmap, index); + } + } +} + static bool sync_lflows_from_objres(struct ovsdb_idl_txn *ovnsb_txn, struct resource_to_objects_node *res_node, @@ -17914,17 +18116,19 @@ unlink_and_sync_lflows(struct ovsdb_idl_txn *sb_txn, struct ovn_datapath *od, lflow_data, lflowdep_mgr); } -bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct northd_tracked_data *trk_nd_changes, - struct lflow_input *lflow_input, - struct lflow_data *lflow_data) +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *trk_ovn_ports, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) { - struct tracked_ovn_ports *trk_ovn_ports = &trk_nd_changes->trk_ovn_ports; - struct tracked_ovn_port *trk_op; HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->deleted) { struct ovn_port *op = trk_op->op; + /* We don't support lflow handling for deleted logical router + * ports yet. */ + ovs_assert(op->nbsp); + /* unlink old lflows. */ unlink_and_sync_lflows(ovnsb_txn, op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, lflow_input, lflow_data, @@ -17937,19 +18141,53 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->updated) { struct ovn_port *op = trk_op->op; + ovs_assert(trk_op->changes); - /* unlink old lflows. */ - unlink_lflows(op->od, OBJDEP_TYPE_LPORT, op->nbsp->name, - lflow_data, &op->lflow_dep_mgr); + const char *res_name = op->nbsp ? op->nbsp->name : op->nbrp->name; /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflow_data); + if (trk_op->changes & EN_TRACKED_OP_LPORT_CHANGED) { + /* We still don't support EN_TRACKED_OP_LPORT_CHANGED for + * router ports. */ + ovs_assert(op->nbsp); + + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LPORT, res_name, lflow_data, + &op->lflow_dep_mgr); + + /* Generate new lflows. */ + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflow_data); + } + + if (trk_op->changes & EN_TRACKED_OP_LB_CHANGED) { + /* unlink old lflows. */ + unlink_lflows(op->od, OBJDEP_TYPE_LB, res_name, lflow_data, + &op->lflow_dep_mgr); + unlink_lflows(op->od, OBJDEP_TYPE_CT_LB, res_name, lflow_data, + &op->lflow_dep_mgr); + + if (op->nbsp) { + build_lswitch_ip_unicast_lookup_lb_vips(op, lflow_data, + &op->lflow_dep_mgr); + build_ip_routing_flows_for_router_type_lsp( + op, lflow_input->lr_ports, lflow_data); + build_arp_resolve_flows_for_routable_addrs_throug_lsp( + op, lflow_data, lflow_input->lr_ports, &match, &actions, + &op->lflow_dep_mgr); + } else { + build_lrouter_ipv4_ip_input_lb_related(op, lflow_data, &match, + lflow_input->meter_groups); + build_lrouter_force_snat_flows_op(op, lflow_data, &match, + &actions); + } + } + ds_destroy(&match); ds_destroy(&actions); @@ -17957,13 +18195,26 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, * groups. */ /* Sync the new flows to SB. */ - sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, op->nbsp->name, - lflow_input, lflow_data, &op->lflow_dep_mgr); + if (trk_op->changes & EN_TRACKED_OP_LPORT_CHANGED) { + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LPORT, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + } + + if (trk_op->changes & EN_TRACKED_OP_LB_CHANGED) { + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, res_name, + lflow_input, lflow_data, &op->lflow_dep_mgr); + } } HMAP_FOR_EACH (trk_op, hmap_node, &trk_ovn_ports->created) { struct ovn_port *op = trk_op->op; + /* We don't support lflow handling for deleted logical router + * ports yet. */ + ovs_assert(op->nbsp); + const struct sbrec_multicast_group *sbmc_flood = mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, MC_FLOOD, op->od->sb); @@ -18017,6 +18268,151 @@ bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +static void +lflow_update_ls_lb_lflows(struct ovn_datapath *od, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + ovs_assert(od->nbs); + + /* unlink old lflows. */ + unlink_lflows(od, OBJDEP_TYPE_LB, od->nbs->name, lflow_data, + &od->lflow_dep_mgr); + unlink_lflows(od, OBJDEP_TYPE_CT_LB, od->nbs->name, lflow_data, + &od->lflow_dep_mgr); + + /* Build the lflows related to LB and CT LB. */ + build_pre_acls(od, lflow_input->port_groups, lflow_data); + build_pre_lb_lb_related(od, lflow_data); + build_acl_hints(od, lflow_input->features, lflow_data); + build_acls(od, lflow_input->features, lflow_data, lflow_input->port_groups, + lflow_input->meter_groups); + build_lb_hairpin(od, lflow_data); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, od->nbs->name, lflow_input, + lflow_data, &od->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, od->nbs->name, + lflow_input, lflow_data, &od->lflow_dep_mgr); +} + +static void +lflow_update_lr_lb_lflows(struct ovn_datapath *od, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + ovs_assert(od->nbr); + + /* unlink old lflows. */ + unlink_lflows(od, OBJDEP_TYPE_LB, od->nbr->name, lflow_data, + &od->lflow_dep_mgr); + unlink_lflows(od, OBJDEP_TYPE_CT_LB, od->nbr->name, lflow_data, + &od->lflow_dep_mgr); + + struct ds actions = DS_EMPTY_INITIALIZER; + struct ds match = DS_EMPTY_INITIALIZER; + + build_lrouter_nat_flows_lb_related(od, lflow_data, &match, + lflow_input->features); + build_lrouter_nat_flows_ct_lb_related(od, lflow_data); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, od->nbr->name, lflow_input, + lflow_data, &od->lflow_dep_mgr); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_CT_LB, od->nbr->name, + lflow_input, lflow_data, &od->lflow_dep_mgr); +} + +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lb_datapaths *trk_lbs, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + unlink_all_ods_objres_lflows(&lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + &lflow_data->lflows_hash_map); + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, lflow_input, + lflow_data, &lb_dps->lflow_dep_mgr); + objdep_mgr_clear(&lb_dps->lflow_dep_mgr); + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + unlink_all_ods_objres_lflows(&lb_dps->lflow_dep_mgr, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + &lflow_data->lflows_hash_map); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lswitch_arp_nd_service_monitor(lb_dps, lflow_input->ls_ports, + lflow_data, &actions, + &match); + build_lrouter_defrag_flows_for_lb(lb_dps, lflow_data, + lflow_input->lr_datapaths, &match); + build_lrouter_flows_for_lb(lb_dps, lflow_data, + lflow_input->meter_groups, + lflow_input->lr_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + build_lswitch_flows_for_lb(lb_dps, lflow_data, + lflow_input->meter_groups, + lflow_input->ls_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + sync_lflows_for_res(ovnsb_txn, OBJDEP_TYPE_LB, + lb_dps->lb->nlb->name, lflow_input, + lflow_data, &lb_dps->lflow_dep_mgr); + + } + + if (trk_lbs->n_nb_ls && trk_lbs->nb_ls_map) { + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lflow_input->ls_datapaths), + trk_lbs->nb_ls_map) { + struct ovn_datapath *od = lflow_input->ls_datapaths->array[index]; + lflow_update_ls_lb_lflows(od, ovnsb_txn, lflow_input, lflow_data); + } + } + + if (trk_lbs->n_nb_lr && trk_lbs->nb_lr_map) { + size_t index; + BITMAP_FOR_EACH_1 (index, ods_size(lflow_input->lr_datapaths), + trk_lbs->nb_lr_map) { + struct ovn_datapath *od = lflow_input->lr_datapaths->array[index]; + lflow_update_lr_lb_lflows(od, ovnsb_txn, lflow_input, lflow_data); + } + } + + return true; +} + /* Each port group in Port_Group table in OVN_Northbound has a corresponding * entry in Port_Group table in OVN_Southbound. In OVN_Northbound the entries * contains lport uuids, while in OVN_Southbound we store the lport names. @@ -18899,6 +19295,8 @@ northd_init(struct northd_data *data) hmap_init(&data->trk_northd_changes.trk_ovn_ports.created); hmap_init(&data->trk_northd_changes.trk_ovn_ports.updated); hmap_init(&data->trk_northd_changes.trk_ovn_ports.deleted); + hmapx_init(&data->trk_northd_changes.trk_lbs.crupdated); + hmapx_init(&data->trk_northd_changes.trk_lbs.deleted); } void @@ -18955,6 +19353,8 @@ northd_destroy(struct northd_data *data) hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.created); hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.updated); hmap_destroy(&data->trk_northd_changes.trk_ovn_ports.deleted); + hmapx_destroy(&data->trk_northd_changes.trk_lbs.crupdated); + hmapx_destroy(&data->trk_northd_changes.trk_lbs.deleted); } void diff --git a/northd/northd.h b/northd/northd.h index 1634463a67..b836aa737f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -88,10 +88,15 @@ struct ovn_datapaths { struct ovn_datapath **array; }; +#define EN_TRACKED_OP_LPORT_CHANGED 0x01 +#define EN_TRACKED_OP_LB_CHANGED 0x02 + /* Represents a tracked ovn_port. */ struct tracked_ovn_port { struct hmap_node hmap_node; struct ovn_port *op; + /* Indicates what changed. one or any or all of EN_TRACKED_OP_* bit. */ + uint8_t changes; }; struct tracked_ovn_ports { @@ -108,13 +113,30 @@ struct tracked_ovn_ports { struct hmap deleted; }; +struct tracked_lb_datapaths { + /* Tracked created or updated load balancers. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx crupdated; + + /* Tracked deleted lbs. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx deleted; + + /* Tracked logical switches related to the tracked lbs. */ + unsigned long *nb_ls_map; + size_t n_nb_ls; + + /* Tracked logical routers related to the tracked lb. */ + unsigned long *nb_lr_map; + size_t n_nb_lr; +}; + /* Track what's changed in the northd engine node. * Now only tracks ovn_ports (of vif type) - created, updated * and deleted and indicates if load balancers have changed. */ struct northd_tracked_data { struct tracked_ovn_ports trk_ovn_ports; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + struct tracked_lb_datapaths trk_lbs; }; struct northd_data { @@ -363,9 +385,14 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_data *lflow_data); -bool lflow_handle_northd_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct northd_tracked_data *, - struct lflow_input *, struct lflow_data *); +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *, + struct lflow_input *, + struct lflow_data *); +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lb_datapaths *, + struct lflow_input *lflow_input, + struct lflow_data *lflow_data); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); @@ -374,12 +401,16 @@ bool northd_handle_lb_data_changes_pre_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); bool northd_handle_lb_data_changes_post_od(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); +bool northd_has_tracked_data(struct northd_tracked_data *); +bool northd_has_only_ports_in_tracked_data(struct northd_tracked_data *); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a5f64ed5bd..4424a1f64d 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -9834,7 +9834,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9842,17 +9842,17 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9860,7 +9860,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -9901,13 +9901,14 @@ ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Associate lb1 to sw0. There should be no recompute of northd engine node check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -9915,7 +9916,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -9923,7 +9924,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -9931,7 +9932,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -9939,7 +9940,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. @@ -9974,7 +9975,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -9982,7 +9983,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -9990,7 +9991,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -9998,7 +9999,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10006,7 +10007,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10021,7 +10022,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10033,7 +10034,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear load_balancer_group . load_Balancer @@ -10046,7 +10047,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10054,21 +10055,23 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10076,7 +10079,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10084,7 +10087,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10092,7 +10095,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10100,12 +10103,13 @@ check ovn-nbctl clear logical_switch sw0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10113,7 +10117,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10121,7 +10125,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10129,7 +10133,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10137,7 +10141,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10145,13 +10149,15 @@ check ovn-nbctl clear logical_router lr0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl clear logical_switch sw0 load_balancer_group -- \ @@ -10180,40 +10186,42 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10221,7 +10229,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10244,7 +10252,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Deleting lb2 should result in lflow recompute as it is @@ -10253,7 +10261,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10265,3 +10273,76 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) + + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing with stateless ACLs]) +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl acl-add sw0 from-lport 1 1 allow-stateless +check ovn-nbctl --wait=sb acl-add sw0 to-lport 1 1 allow-stateless + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clear the VIPs of lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer . vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +])