From patchwork Tue Oct 24 00:43:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854074 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtbL2HBzz23jn for ; Tue, 24 Oct 2023 11:43:42 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CDA7642C67; Tue, 24 Oct 2023 00:43:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org CDA7642C67 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nvs7BUcC5HyS; Tue, 24 Oct 2023 00:43:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id C1EB242C41; Tue, 24 Oct 2023 00:43:35 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C1EB242C41 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7A3E7C0071; Tue, 24 Oct 2023 00:43:35 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 053B7C0032 for ; Tue, 24 Oct 2023 00:43:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A3B84400DC for ; Tue, 24 Oct 2023 00:43:33 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A3B84400DC X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3mdWXZUqqc0O for ; Tue, 24 Oct 2023 00:43:31 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp2.osuosl.org (Postfix) with ESMTPS id D2BFE4195B for ; Tue, 24 Oct 2023 00:43:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D2BFE4195B Received: by mail.gandi.net (Postfix) with ESMTPSA id 00A31C0003; Tue, 24 Oct 2023 00:43:26 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:43:09 -0400 Message-ID: <20231024004309.4133494-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 01/18] northd: Refactor the northd change tracking. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique northd engine tracking data now has the following tracking data - changed ovn_ports (right now only changed logical switch ports are tracked.) - changed load balancers. This separation becomes easier to add lflow handling for these changes in lflow northd engine handler. This patch doesn't handle the load balancer changes in lflow handler. It will be handled in upcoming commits. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 11 +- northd/en-northd.c | 13 +- northd/en-sync-sb.c | 10 +- northd/northd.c | 446 ++++++++++++++++++++++++-------------------- northd/northd.h | 63 ++++--- tests/ovn-northd.at | 10 +- 6 files changed, 313 insertions(+), 240 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 2b84fef0ef..96d03b7ada 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -108,8 +108,8 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if lb related data has changed. */ - if (northd_data->lb_changed) { + /* Fall back to recompute if load balancers have changed. */ + if (northd_has_lbs_in_tracked_data(&northd_data->trk_northd_changes)) { return false; } @@ -119,13 +119,14 @@ lflow_northd_handler(struct engine_node *node, struct lflow_input lflow_input; lflow_get_input_data(node, &lflow_input); - if (!lflow_handle_northd_ls_changes(eng_ctx->ovnsb_idl_txn, - &northd_data->tracked_ls_changes, - &lflow_input, &lflow_data->lflows)) { + if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_ovn_ports, + &lflow_input, &lflow_data->lflows)) { return false; } engine_set_node_state(node, EN_UPDATED); + return true; } diff --git a/northd/en-northd.c b/northd/en-northd.c index aa0f20f0c2..96c2ce9f69 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -230,15 +230,16 @@ northd_lb_data_handler(struct engine_node *node, void *data) &nd->ls_datapaths, &nd->lr_datapaths, &nd->lb_datapaths_map, - &nd->lb_group_datapaths_map)) { + &nd->lb_group_datapaths_map, + &nd->trk_northd_changes)) { return false; } - /* Indicate the depedendant engine nodes that load balancer/group - * related data has changed (including association to logical - * switch/router). */ - nd->lb_changed = true; - engine_set_node_state(node, EN_UPDATED); + if (northd_has_lbs_in_tracked_data(&nd->trk_northd_changes)) { + nd->change_tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + return true; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 2ec3bf54f8..2540fcfb97 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -236,7 +236,8 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked || nd->lb_changed) { + if (!nd->change_tracked || + northd_has_lbs_in_tracked_data(&nd->trk_northd_changes)) { /* Return false if no tracking data or if lbs changed. */ return false; } @@ -306,11 +307,14 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) } struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked) { + if (!nd->change_tracked || + northd_has_lbs_in_tracked_data(&nd->trk_northd_changes)) { + /* Return false if no tracking data or if lbs changed. */ return false; } - if (!sync_pbs_for_northd_ls_changes(&nd->tracked_ls_changes)) { + if (!sync_pbs_for_northd_changed_ovn_ports( + &nd->trk_northd_changes.trk_ovn_ports)) { return false; } diff --git a/northd/northd.c b/northd/northd.c index f8b046d83e..df22a9c658 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -4894,21 +4894,20 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) } /* Sync the SB Port bindings for the added and updated logical switch ports - * of the tracked logical switches (from the northd engine node). */ + * of the tracked northd engine data. */ bool -sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *ls_changes) +sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) { - struct ls_change *ls_change; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - struct ovn_port *op; - - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - sync_pb_for_op(op); - } + struct hmapx_node *hmapx_node; + struct ovn_port *op; + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { + op = hmapx_node->data; + sync_pb_for_op(op); + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - sync_pb_for_op(op); - } + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { + op = hmapx_node->data; + sync_pb_for_op(op); } return true; @@ -5110,34 +5109,95 @@ build_ports(struct ovsdb_idl_txn *ovnsb_txn, } static void -destroy_tracked_ls_change(struct ls_change *ls_change) +destroy_tracked_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports) { - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - ovs_list_remove(&op->list); - } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - ovs_list_remove(&op->list); + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_ovn_ports->deleted) { + ovn_port_destroy_orphan(hmapx_node->data); + hmapx_delete(&trk_ovn_ports->deleted, hmapx_node); } - LIST_FOR_EACH_SAFE (op, list, &ls_change->deleted_ports) { - ovs_list_remove(&op->list); - ovn_port_destroy_orphan(op); + + hmapx_clear(&trk_ovn_ports->created); + hmapx_clear(&trk_ovn_ports->updated); +} + +static void +destroy_tracked_lbs(struct tracked_lbs *trk_lbs) +{ + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &trk_lbs->deleted) { + ovn_lb_datapaths_destroy(hmapx_node->data); + hmapx_delete(&trk_lbs->deleted, hmapx_node); } + + hmapx_clear(&trk_lbs->crupdated); +} + +static void +add_op_to_northd_tracked_ports(struct hmapx *tracked_ovn_ports, + struct ovn_port *op) +{ + hmapx_add(tracked_ovn_ports, op); } void destroy_northd_data_tracked_changes(struct northd_data *nd) { - struct ls_change *ls_change; - LIST_FOR_EACH_SAFE (ls_change, list_node, - &nd->tracked_ls_changes.updated) { - destroy_tracked_ls_change(ls_change); - ovs_list_remove(&ls_change->list_node); - free(ls_change); - } - + struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; + destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); + destroy_tracked_lbs(&trk_changes->trk_lbs); nd->change_tracked = false; - nd->lb_changed = false; +} + +static void +init_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; + hmapx_init(&trk_changes->trk_ovn_ports.created); + hmapx_init(&trk_changes->trk_ovn_ports.updated); + hmapx_init(&trk_changes->trk_ovn_ports.deleted); + hmapx_init(&trk_changes->trk_lbs.crupdated); + hmapx_init(&trk_changes->trk_lbs.deleted); +} + +static void +destroy_northd_tracked_data(struct northd_data *nd) +{ + struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; + hmapx_destroy(&trk_changes->trk_ovn_ports.created); + hmapx_destroy(&trk_changes->trk_ovn_ports.updated); + hmapx_destroy(&trk_changes->trk_ovn_ports.deleted); + hmapx_destroy(&trk_changes->trk_lbs.crupdated); + hmapx_destroy(&trk_changes->trk_lbs.deleted); +} + + +bool +northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (!hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.deleted) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); +} + +bool +northd_has_only_ports_in_tracked_data( + struct northd_tracked_data *trk_nd_changes) +{ + return (hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + && hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + && (!hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.created) + || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.updated) + || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.deleted))); +} + +bool +northd_has_lbs_in_tracked_data(struct northd_tracked_data *trk_nd_changes) +{ + return (!hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); } /* Check if a changed LSP can be handled incrementally within the I-P engine @@ -5344,12 +5404,12 @@ check_lsp_changes_other_than_up(const struct nbrec_logical_switch_port *nbsp) */ static bool ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, - const struct nbrec_logical_switch *changed_ls, - const struct northd_input *ni, - struct northd_data *nd, - struct ovn_datapath *od, - struct ls_change *ls_change, - bool *updated) + const struct nbrec_logical_switch *changed_ls, + const struct northd_input *ni, + struct northd_data *nd, + struct ovn_datapath *od, + struct tracked_ovn_ports *trk_ports, + bool *updated) { bool ls_ports_changed = false; if (!nbrec_logical_switch_is_updated(changed_ls, @@ -5370,7 +5430,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, return true; } - ls_change->had_only_router_ports = (od->n_router_ports + bool ls_had_only_router_ports = (od->n_router_ports && (od->n_router_ports == hmap_count(&od->ports))); struct ovn_port *op; @@ -5396,8 +5456,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->added_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_ports->created, op); } else if (ls_port_has_changed(op->nbsp, new_nbsp)) { /* Existing port updated */ bool temp = false; @@ -5433,7 +5492,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, if (!op) { goto fail; } - ovs_list_push_back(&ls_change->updated_ports, &op->list); + add_op_to_northd_tracked_ports(&trk_ports->updated, op); } op->visited = true; } @@ -5442,15 +5501,14 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, HMAP_FOR_EACH_SAFE (op, dp_node, &od->ports) { if (!op->visited) { if (!op->lsp_can_be_inc_processed) { - goto fail_clean_deleted; + goto fail; } if (sset_contains(&nd->svc_monitor_lsps, op->key)) { /* This port was used for svc monitor, which may be * impacted by this deletion. Fallback to recompute. */ - goto fail_clean_deleted; + goto fail; } - ovs_list_push_back(&ls_change->deleted_ports, - &op->list); + add_op_to_northd_tracked_ports(&trk_ports->deleted, op); hmap_remove(&nd->ls_ports, &op->key_node); hmap_remove(&od->ports, &op->dp_node); sbrec_port_binding_delete(op->sb); @@ -5459,20 +5517,29 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, } } - if (!ovs_list_is_empty(&ls_change->added_ports) || - !ovs_list_is_empty(&ls_change->updated_ports) || - !ovs_list_is_empty(&ls_change->deleted_ports)) { + bool ls_has_only_router_ports = (od->n_router_ports + && (od->n_router_ports == hmap_count(&od->ports))); + + /* If there were only router ports before the change or if there are only + * router ports after the change, then add the router ports of the + * logical switch to the tracked 'updated' ovn ports. */ + if (ls_had_only_router_ports != ls_has_only_router_ports) { + for (size_t i = 0; i < od->n_router_ports; i++) { + op = od->router_ports[i]; + add_op_to_northd_tracked_ports(&trk_ports->updated, op); + } + } + + if (!hmapx_is_empty(&trk_ports->created) || + !hmapx_is_empty(&trk_ports->updated) || + !hmapx_is_empty(&trk_ports->deleted)) { *updated = true; } return true; -fail_clean_deleted: - LIST_FOR_EACH_POP (op, list, &ls_change->deleted_ports) { - ovn_port_destroy_orphan(op); - } - fail: + destroy_tracked_ovn_ports(trk_ports); return false; } @@ -5490,7 +5557,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, struct northd_data *nd) { const struct nbrec_logical_switch *changed_ls; - struct ls_change *ls_change = NULL; + struct northd_tracked_data *trk_nd_changes = &nd->trk_northd_changes; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (changed_ls, ni->nbrec_logical_switch_table) { @@ -5514,33 +5581,18 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } - ls_change = xzalloc(sizeof *ls_change); - ls_change->od = od; - ovs_list_init(&ls_change->added_ports); - ovs_list_init(&ls_change->deleted_ports); - ovs_list_init(&ls_change->updated_ports); - bool updated = false; if (!ls_handle_lsp_changes(ovnsb_idl_txn, changed_ls, - ni, nd, od, ls_change, + ni, nd, od, &trk_nd_changes->trk_ovn_ports, &updated)) { - destroy_tracked_ls_change(ls_change); - free(ls_change); goto fail; } if (updated) { - ovs_list_push_back(&nd->tracked_ls_changes.updated, - &ls_change->list_node); - } else { - free(ls_change); + nd->change_tracked = true; } } - if (!ovs_list_is_empty(&nd->tracked_ls_changes.updated)) { - nd->change_tracked = true; - } - return true; fail: @@ -5715,7 +5767,8 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map) + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *nd_changes) { if (trk_lb_data->has_health_checks) { /* Fall back to recompute since a tracked load balancer @@ -5778,7 +5831,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); - ovn_lb_datapaths_destroy(lb_dps); + + /* Add the deleted lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.deleted, lb_dps); } /* Create the 'lb_dps' if not already created for each @@ -5795,6 +5850,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, hmap_insert(lb_datapaths_map, &lb_dps->hmap_node, uuid_hash(lb_uuid)); } + + /* Add the updated lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } struct ovn_lb_group_datapaths *lbgrp_dps; @@ -5825,6 +5883,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, &uuidnode->uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5840,6 +5901,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_ls(lb_dps, 1, &od); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5860,6 +5924,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { @@ -5879,6 +5946,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); build_lrouter_lb_reachable_ips(od, lb_dps->lb); + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -5957,6 +6027,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); } + + /* Add the lb to the northd tracked data. */ + hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } } @@ -16996,149 +17069,125 @@ delete_lflow_for_lsp(struct ovn_port *op, bool is_update, return true; } -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *ls_changes, - struct lflow_input *lflow_input, - struct hmap *lflows) +bool +lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *trk_ovn_ports, + struct lflow_input *lflow_input, + struct hmap *lflows) { - struct ls_change *ls_change; + struct hmapx_node *hmapx_node; + struct ovn_port *op; - LIST_FOR_EACH (ls_change, list_node, &ls_changes->updated) { - const struct sbrec_multicast_group *sbmc_flood = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_flood_l2 = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_FLOOD_L2, ls_change->od->sb); - const struct sbrec_multicast_group *sbmc_unknown = - mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, - MC_UNKNOWN, ls_change->od->sb); + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->deleted) { + op = hmapx_node->data; + /* We don't support lflow handling for deleted logical router + * ports yet. */ + ovs_assert(op->nbsp); - struct ovn_port *op; - LIST_FOR_EACH (op, list, &ls_change->deleted_ports) { - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { + if (!delete_lflow_for_lsp(op, false, + lflow_input->sbrec_logical_flow_table, + lflows)) { return false; } - /* No need to update SB multicast groups, thanks to weak - * references. */ - } + /* No need to update SB multicast groups, thanks to weak + * references. */ + } - LIST_FOR_EACH (op, list, &ls_change->updated_ports) { - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { + op = hmapx_node->data; + /* We don't support lflow handling for updated logical router + * ports yet. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* SB port_binding is not deleted, so don't update SB multicast - * groups. */ - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Delete old lflows. */ + if (!delete_lflow_for_lsp(op, true, + lflow_input->sbrec_logical_flow_table, + lflows)) { + return false; } - LIST_FOR_EACH (op, list, &ls_change->added_ports) { - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, - lflow_input->lr_ports, - lflow_input->meter_groups, - &match, &actions, - lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Update SB multicast groups for the new port. */ - if (!sbmc_flood) { - sbmc_flood = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); - if (!sbmc_flood_l2) { - sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_FLOOD_L2, - OVN_MCAST_FLOOD_L2_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); - - if (op->has_unknown) { - if (!sbmc_unknown) { - sbmc_unknown = create_sb_multicast_group(ovnsb_txn, - ls_change->od->sb, MC_UNKNOWN, - OVN_MCAST_UNKNOWN_TUNNEL_KEY); - } - sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, - op->sb); - } + /* SB port_binding is not deleted, so don't update SB multicast + * groups. */ - /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + /* Sync the new flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } + } - bool ls_has_only_router_ports = (ls_change->od->n_router_ports && - (ls_change->od->n_router_ports == - hmap_count(&ls_change->od->ports))); - - if (ls_change->had_only_router_ports != ls_has_only_router_ports) { - /* There are lflows related to router ports that depends on whether - * there are switch ports on the logical switch (see - * build_lswitch_rport_arp_req_flow() for more details). Since this - * dependency changed, we need to regenerate lflows for each router - * port on this logical switch. */ - for (size_t i = 0; i < ls_change->od->n_router_ports; i++) { - op = ls_change->od->router_ports[i]; - - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, "affected router", - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { + op = hmapx_node->data; + /* We don't support lflow handling for created logical router + * ports yet. */ + ovs_assert(op->nbsp); - /* Generate new lflows. */ - struct ds match = DS_EMPTY_INITIALIZER; - struct ds actions = DS_EMPTY_INITIALIZER; - build_lswitch_and_lrouter_iterate_by_lsp(op, - lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->meter_groups, &match, &actions, lflows); - ds_destroy(&match); - ds_destroy(&actions); - - /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + const struct sbrec_multicast_group *sbmc_flood = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD, op->od->sb); + const struct sbrec_multicast_group *sbmc_flood_l2 = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_FLOOD_L2, op->od->sb); + const struct sbrec_multicast_group *sbmc_unknown = + mcast_group_lookup(lflow_input->sbrec_mcast_group_by_name_dp, + MC_UNKNOWN, op->od->sb); + + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, + lflow_input->lr_ports, + lflow_input->meter_groups, + &match, &actions, + lflows); + ds_destroy(&match); + ds_destroy(&actions); + + /* Update SB multicast groups for the new port. */ + if (!sbmc_flood) { + sbmc_flood = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD, OVN_MCAST_FLOOD_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood, op->sb); + + if (!sbmc_flood_l2) { + sbmc_flood_l2 = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_FLOOD_L2, + OVN_MCAST_FLOOD_L2_TUNNEL_KEY); + } + sbrec_multicast_group_update_ports_addvalue(sbmc_flood_l2, op->sb); + + if (op->has_unknown) { + if (!sbmc_unknown) { + sbmc_unknown = create_sb_multicast_group(ovnsb_txn, + op->od->sb, MC_UNKNOWN, + OVN_MCAST_UNKNOWN_TUNNEL_KEY); } + sbrec_multicast_group_update_ports_addvalue(sbmc_unknown, + op->sb); + } + + /* Sync the newly added flows to SB. */ + struct lflow_ref_node *lfrn; + LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { + sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, + lfrn->lflow); } } - return true; + return true; } static bool @@ -17771,7 +17820,7 @@ northd_init(struct northd_data *data) sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); data->change_tracked = false; - ovs_list_init(&data->tracked_ls_changes.updated); + init_northd_tracked_data(data); } void @@ -17811,6 +17860,7 @@ northd_destroy(struct northd_data *data) destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); + destroy_northd_tracked_data(data); } void diff --git a/northd/northd.h b/northd/northd.h index 5be7b5384d..2e4f125902 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -83,22 +83,36 @@ struct ovn_datapaths { struct ovn_datapath **array; }; -/* Track what's changed for a single LS. - * Now only track port changes. */ -struct ls_change { - struct ovs_list list_node; - struct ovn_datapath *od; - struct ovs_list added_ports; - struct ovs_list deleted_ports; - struct ovs_list updated_ports; - bool had_only_router_ports; +struct tracked_ovn_ports { + /* tracked created ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx created; + + /* tracked updated ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx updated; + + /* tracked deleted ports. + * hmapx node data is 'struct ovn_port *' */ + struct hmapx deleted; }; -/* Track what's changed for logical switches. - * Now only track updated ones (added or deleted may be supported in the - * future). */ -struct tracked_ls_changes { - struct ovs_list updated; /* Contains struct ls_change */ +struct tracked_lbs { + /* Tracked created or updated load balancers. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx crupdated; + + /* Tracked deleted lbs. + * hmapx node data is 'struct ovn_lb_datapaths' */ + struct hmapx deleted; +}; + +/* Track what's changed in the northd engine node. + * Now only tracks ovn_ports (of vif type) - created, updated + * and deleted. */ +struct northd_tracked_data { + struct tracked_ovn_ports trk_ovn_ports; + struct tracked_lbs trk_lbs; }; struct northd_data { @@ -114,10 +128,9 @@ struct northd_data { struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; + /* Indicates if northd engine node has changes tracked or not. */ bool change_tracked; - struct tracked_ls_changes tracked_ls_changes; - bool lb_changed; /* Indicates if load balancers changed or association of - * load balancer to logical switch/router changed. */ + struct northd_tracked_data trk_northd_changes; }; struct lflow_data { @@ -338,9 +351,10 @@ void northd_indices_create(struct northd_data *data, void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct hmap *lflows); -bool lflow_handle_northd_ls_changes(struct ovsdb_idl_txn *ovnsb_txn, - struct tracked_ls_changes *, - struct lflow_input *, struct hmap *lflows); +bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_ovn_ports *, + struct lflow_input *, + struct hmap *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports); @@ -349,7 +363,8 @@ bool northd_handle_lb_data_changes(struct tracked_lb_data *, struct ovn_datapaths *ls_datapaths, struct ovn_datapaths *lr_datapaths, struct hmap *lb_datapaths_map, - struct hmap *lbgrp_datapaths_map); + struct hmap *lbgrp_datapaths_map, + struct northd_tracked_data *); void build_bfd_table(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_bfd_table *, @@ -372,6 +387,10 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); -bool sync_pbs_for_northd_ls_changes(struct tracked_ls_changes *); +bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + +bool northd_has_tracked_data(struct northd_tracked_data *); +bool northd_has_only_ports_in_tracked_data(struct northd_tracked_data *); +bool northd_has_lbs_in_tracked_data(struct northd_tracked_data *); #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 196fe01fbd..28c293473c 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10620,10 +10620,9 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute - +check_engine_stats lflow norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lb1_uuid=$(fetch_column nb:Load_Balancer _uuid) @@ -10650,7 +10649,6 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute - CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10798,8 +10796,8 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" From patchwork Tue Oct 24 00:43:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854075 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtbh6Y0mz23jn for ; Tue, 24 Oct 2023 11:44:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id CB63742CA3; Tue, 24 Oct 2023 00:43:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org CB63742CA3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zjBWwGnL5wNm; Tue, 24 Oct 2023 00:43:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id C955142C86; Tue, 24 Oct 2023 00:43:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C955142C86 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 94D95C0071; Tue, 24 Oct 2023 00:43:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 137A8C0032 for ; Tue, 24 Oct 2023 00:43:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8349842C6B for ; Tue, 24 Oct 2023 00:43:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8349842C6B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u13qsdQoWaHB for ; Tue, 24 Oct 2023 00:43:49 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp4.osuosl.org (Postfix) with ESMTPS id F104342C65 for ; Tue, 24 Oct 2023 00:43:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org F104342C65 Received: by mail.gandi.net (Postfix) with ESMTPSA id 53DDF40003; Tue, 24 Oct 2023 00:43:43 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:43:28 -0400 Message-ID: <20231024004328.4133518-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 02/18] northd: Track ovn_datapaths in northd engine track data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique northd engine tracked data now also stores the logical switches and logical routers that got updated due to the changed load balancers. Eg 1. For this command 'ovn-nbctl ls-lb-add sw0 lb1 -- lr-lb-add lr0 lb1', northd engine tracking data will store 'sw0' and 'lr0'. Eg 2. If load balancer lb1 is already associated with 'sw0' and 'lr0' then for this command 'ovn-nbctl set load_balancer vips:10.0.0.10=20.0.0.20', northd engine tracking data will store 'sw0' and 'lr0'. An upcoming commit will make use of this tracked data. Signed-off-by: Numan Siddique --- northd/northd.c | 34 +++++++++++++++++++++++++++++++++- northd/northd.h | 12 ++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/northd/northd.c b/northd/northd.c index df22a9c658..9ce1b2cb5a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -5146,6 +5146,8 @@ destroy_northd_data_tracked_changes(struct northd_data *nd) struct northd_tracked_data *trk_changes = &nd->trk_northd_changes; destroy_tracked_ovn_ports(&trk_changes->trk_ovn_ports); destroy_tracked_lbs(&trk_changes->trk_lbs); + hmapx_clear(&trk_changes->ls_with_changed_lbs.crupdated); + hmapx_clear(&trk_changes->lr_with_changed_lbs.crupdated); nd->change_tracked = false; } @@ -5158,6 +5160,8 @@ init_northd_tracked_data(struct northd_data *nd) hmapx_init(&trk_changes->trk_ovn_ports.deleted); hmapx_init(&trk_changes->trk_lbs.crupdated); hmapx_init(&trk_changes->trk_lbs.deleted); + hmapx_init(&trk_changes->ls_with_changed_lbs.crupdated); + hmapx_init(&trk_changes->lr_with_changed_lbs.crupdated); } static void @@ -5169,6 +5173,8 @@ destroy_northd_tracked_data(struct northd_data *nd) hmapx_destroy(&trk_changes->trk_ovn_ports.deleted); hmapx_destroy(&trk_changes->trk_lbs.crupdated); hmapx_destroy(&trk_changes->trk_lbs.deleted); + hmapx_destroy(&trk_changes->ls_with_changed_lbs.crupdated); + hmapx_destroy(&trk_changes->lr_with_changed_lbs.crupdated); } @@ -5179,7 +5185,10 @@ northd_has_tracked_data(struct northd_tracked_data *trk_nd_changes) || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.updated) || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.deleted) || !hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) - || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted)); + || !hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + || !hmapx_is_empty(&trk_nd_changes->ls_with_changed_lbs.crupdated) + || !hmapx_is_empty(&trk_nd_changes->lr_with_changed_lbs.crupdated) + ); } bool @@ -5188,6 +5197,8 @@ northd_has_only_ports_in_tracked_data( { return (hmapx_is_empty(&trk_nd_changes->trk_lbs.crupdated) && hmapx_is_empty(&trk_nd_changes->trk_lbs.deleted) + && hmapx_is_empty(&trk_nd_changes->ls_with_changed_lbs.crupdated) + && hmapx_is_empty(&trk_nd_changes->lr_with_changed_lbs.crupdated) && (!hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.created) || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.updated) || !hmapx_is_empty(&trk_nd_changes->trk_ovn_ports.deleted))); @@ -5828,6 +5839,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } hmap_remove(lb_datapaths_map, &lb_dps->hmap_node); @@ -5909,6 +5923,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { @@ -5954,6 +5971,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the lr datapath to the northd tracked data. */ + hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { @@ -5968,6 +5988,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, od = ls_datapaths->array[index]; /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), @@ -5991,6 +6014,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, add_neigh_ips_to_lrouter(od, lb->neigh_mode, &clb->inserted_vips_v4, &clb->inserted_vips_v6); + + /* Add the lr datapath to the northd tracked data. */ + hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } } @@ -6018,6 +6044,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Add the lb_ips of lb_dps to the od. */ build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); + + /* Add the lr datapath to the northd tracked data. */ + hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } for (size_t i = 0; i < lbgrp_dps->n_ls; i++) { @@ -6026,6 +6055,9 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); + + /* Add the ls datapath to the northd tracked data. */ + hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } /* Add the lb to the northd tracked data. */ diff --git a/northd/northd.h b/northd/northd.h index 2e4f125902..0478826f0a 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -107,12 +107,24 @@ struct tracked_lbs { struct hmapx deleted; }; +/* Tracked logical switches whose load balancers have changed. */ +struct tracked_lswitches_with_changed_lbs { + struct hmapx crupdated; +}; + +/* Tracked logical routers whose load balancers have changed. */ +struct tracked_lrouters_with_changed_lbs { + struct hmapx crupdated; +}; + /* Track what's changed in the northd engine node. * Now only tracks ovn_ports (of vif type) - created, updated * and deleted. */ struct northd_tracked_data { struct tracked_ovn_ports trk_ovn_ports; struct tracked_lbs trk_lbs; + struct tracked_lswitches_with_changed_lbs ls_with_changed_lbs; + struct tracked_lrouters_with_changed_lbs lr_with_changed_lbs; }; struct northd_data { From patchwork Tue Oct 24 00:43:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854076 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtbt6HtCz23jn for ; Tue, 24 Oct 2023 11:44:10 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1F79583C87; Tue, 24 Oct 2023 00:44:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1F79583C87 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qgFsdgeji1Q4; Tue, 24 Oct 2023 00:44:07 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id D91B683C7E; Tue, 24 Oct 2023 00:44:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org D91B683C7E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id A9495C0071; Tue, 24 Oct 2023 00:44:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5928EC0071 for ; Tue, 24 Oct 2023 00:44:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 04CE483C7F for ; Tue, 24 Oct 2023 00:44:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 04CE483C7F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cxfZ2SfquFDP for ; Tue, 24 Oct 2023 00:44:01 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6893D83C7E for ; Tue, 24 Oct 2023 00:44:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6893D83C7E Received: by mail.gandi.net (Postfix) with ESMTPSA id F368A40007; Tue, 24 Oct 2023 00:43:56 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:43:44 -0400 Message-ID: <20231024004344.4133538-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 03/18] tests: Add a couple of tests in ovn-northd for I-P. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique These tests cover scenarios for load balancers and NATs and check for the 'northd' and 'lflow' engine node recompute and compute stats. Signed-off-by: Numan Siddique --- tests/ovn-northd.at | 274 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 274 insertions(+) diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 28c293473c..699f6cfdce 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10893,3 +10893,277 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Load balancer incremental processing with stateless ACLs]) +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +# Test I-P for load balancers. +# Presently ovn-northd handles I-P for NB LBs in northd_lb_data engine node +# only. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl acl-add sw0 from-lport 1 1 allow-stateless +check ovn-nbctl --wait=sb acl-add sw0 to-lport 1 1 allow-stateless + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clear the VIPs of lb1 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear load_balancer . vips +check_engine_stats lb_data norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lb-del lb1 +check_engine_stats lb_data norecompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +AT_CLEANUP +]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([Logical router incremental processing for NAT]) + +ovn_start + +net_add n1 +sim_add hv1 +as hv1 +ovs-vsctl add-br br-phys +ovn_attach n1 br-phys 192.168.0.11 + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +ovn-sbctl chassis-add gw1 geneve 127.0.0.1 + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 10.0.0.4" + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-add lr0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +# Adding a logical router port should result in recompute +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 +# for northd engine there will be both recompute and compute +# first it will be recompute to handle lr0-sw0 and then a compute +# for the SB port binding change. +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl lsp-add sw0 sw0-lr0 +ovn-nbctl lsp-set-type sw0-lr0 router +ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +ovn-nbctl ls-add public +ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 +ovn-nbctl lsp-add public public-lr0 +ovn-nbctl lsp-set-type public-lr0 router +ovn-nbctl lsp-set-addresses public-lr0 router +ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public + +# localnet port +ovn-nbctl lsp-add public ln-public +ovn-nbctl lsp-set-type ln-public localnet +ovn-nbctl lsp-set-addresses ln-public unknown +ovn-nbctl lsp-set-options ln-public network_name=public + +# schedule the gw router port to a chassis. Change the name of the chassis +ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 + +# Modify a logical router port and it should result in recompute. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg + +# Do checks for NATs. +# Add a NAT. This should not result in recompute of both northd and lflow +# engine nodes. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT options column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . options:foo=bar +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT external_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT logical_ip column +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Update the NAT type +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT . type=snat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a dnat_and_snat NAT with external_mac and logical_port +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create a load balancer and add the lb vip as NAT +check ovn-nbctl lb-add lb1 172.168.0.140 10.0.0.20 +check ovn-nbctl lb-add lb2 172.168.0.150:80 10.0.0.40:8080 +check ovn-nbctl lr-lb-add lr0 lb1 +check ovn-nbctl lr-lb-add lr0 lb2 + +# lflow engine should recompute since the nat ip 172.168.0.140 +# is a lb vip. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Delete the NAT +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb clear logical_router lr0 nat +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Create router Policy +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +OVN_CLEANUP([hv1]) +AT_CLEANUP +]) From patchwork Tue Oct 24 00:43:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854078 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtg868JKz23jl for ; Tue, 24 Oct 2023 11:47:00 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C08CF83C7C; Tue, 24 Oct 2023 00:46:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org C08CF83C7C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id iS13wMOv_EQD; Tue, 24 Oct 2023 00:46:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 35EC483C15; Tue, 24 Oct 2023 00:46:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 35EC483C15 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01754C0071; Tue, 24 Oct 2023 00:46:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 83FAEC0032 for ; Tue, 24 Oct 2023 00:46:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 5DE1A83C76 for ; Tue, 24 Oct 2023 00:46:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5DE1A83C76 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qZ51Ofk1xEEB for ; Tue, 24 Oct 2023 00:46:54 +0000 (UTC) Received: from mslow1.mail.gandi.net (mslow1.mail.gandi.net [217.70.178.240]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9AE1F83C15 for ; Tue, 24 Oct 2023 00:46:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9AE1F83C15 Received: from relay5-d.mail.gandi.net (unknown [217.70.183.197]) by mslow1.mail.gandi.net (Postfix) with ESMTP id 8A70CC4A18 for ; Tue, 24 Oct 2023 00:44:17 +0000 (UTC) Received: by mail.gandi.net (Postfix) with ESMTPSA id 93AA51C0006; Tue, 24 Oct 2023 00:44:11 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:43:57 -0400 Message-ID: <20231024004357.4133574-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 04/18] northd: Move router ports SB PB options sync to sync_to_sb_pb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique It also moves the logical router port IPv6 prefix delegation updates to "sync-from-sb" engine node. Signed-off-by: Numan Siddique --- northd/en-northd.c | 2 +- northd/en-sync-sb.c | 3 +- northd/northd.c | 283 ++++++++++++++++++++++++++------------------ northd/northd.h | 6 +- tests/ovn-northd.at | 31 ++++- 5 files changed, 198 insertions(+), 127 deletions(-) diff --git a/northd/en-northd.c b/northd/en-northd.c index 96c2ce9f69..13e731cad9 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -189,7 +189,7 @@ northd_sb_port_binding_handler(struct engine_node *node, northd_get_input_data(node, &input_data); if (!northd_handle_sb_port_binding_changes( - input_data.sbrec_port_binding_table, &nd->ls_ports)) { + input_data.sbrec_port_binding_table, &nd->ls_ports, &nd->lr_ports)) { return false; } diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 2540fcfb97..a14c609acd 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -288,7 +288,8 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, + &northd_data->lr_ports); engine_set_node_state(node, EN_UPDATED); } diff --git a/northd/northd.c b/northd/northd.c index 9ce1b2cb5a..c9c7045755 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3419,6 +3419,9 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, { sbrec_port_binding_set_datapath(op->sb, op->od->sb); if (op->nbrp) { + /* Note: SB port binding options for router ports are set in + * sync_pbs(). */ + /* If the router is for l3 gateway, it resides on a chassis * and its port type is "l3gateway". */ const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); @@ -3430,15 +3433,11 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, sbrec_port_binding_set_type(op->sb, "patch"); } - struct smap new; - smap_init(&new); if (is_cr_port(op)) { ovs_assert(sbrec_chassis_by_name); ovs_assert(sbrec_chassis_by_hostname); ovs_assert(sbrec_ha_chassis_grp_by_name); ovs_assert(active_ha_chassis_grps); - const char *redirect_type = smap_get(&op->nbrp->options, - "redirect-type"); if (op->nbrp->ha_chassis_group) { if (op->nbrp->n_gateway_chassis) { @@ -3480,49 +3479,8 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn, /* Delete the legacy gateway_chassis from the pb. */ sbrec_port_binding_set_gateway_chassis(op->sb, NULL, 0); } - smap_add(&new, "distributed-port", op->nbrp->name); - - bool always_redirect = - !op->od->has_distributed_nat && - !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); - - if (redirect_type) { - smap_add(&new, "redirect-type", redirect_type); - /* XXX Why can't we enable always-redirect when redirect-type - * is bridged? */ - if (!strcmp(redirect_type, "bridged")) { - always_redirect = false; - } - } - - if (always_redirect) { - smap_add(&new, "always-redirect", "true"); - } - } else { - if (op->peer) { - smap_add(&new, "peer", op->peer->key); - if (op->nbrp->ha_chassis_group || - op->nbrp->n_gateway_chassis) { - char *redirect_name = - ovn_chassis_redirect_name(op->nbrp->name); - smap_add(&new, "chassis-redirect-port", redirect_name); - free(redirect_name); - } - } - if (chassis_name) { - smap_add(&new, "l3gateway-chassis", chassis_name); - } - } - - const char *ipv6_pd_list = smap_get(&op->sb->options, - "ipv6_ra_pd_list"); - if (ipv6_pd_list) { - smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); } - sbrec_port_binding_set_options(op->sb, &new); - smap_destroy(&new); - sbrec_port_binding_set_parent_port(op->sb, NULL); sbrec_port_binding_set_tag(op->sb, NULL, 0); @@ -4752,12 +4710,14 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) return duplicates; } -/* Syncs the SB port binding for the ovn_port 'op'. Caller should make sure - * that the OVN SB IDL txn is not NULL. Presently it only syncs the nat - * column of port binding corresponding to the 'op->nbsp' */ +/* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_op(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op) { + ovs_assert(op->nbsp); + if (lsp_is_router(op->nbsp)) { const char *chassis = NULL; if (op->peer && op->peer->od && op->peer->od->nbr) { @@ -4879,18 +4839,87 @@ sync_pb_for_op(struct ovn_port *op) } } +/* Syncs the SB port binding for the ovn_port 'op' of a logical router port. + * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it + * only sets the port binding options column for the router ports */ +static void +sync_pb_for_lrp(struct ovn_port *op) +{ + ovs_assert(op->nbrp); + + struct smap new; + smap_init(&new); + + const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); + if (is_cr_port(op)) { + smap_add(&new, "distributed-port", op->nbrp->name); + + bool always_redirect = + !op->od->has_distributed_nat && + !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); + + const char *redirect_type = smap_get(&op->nbrp->options, + "redirect-type"); + if (redirect_type) { + smap_add(&new, "redirect-type", redirect_type); + /* XXX Why can't we enable always-redirect when redirect-type + * is bridged? */ + if (!strcmp(redirect_type, "bridged")) { + always_redirect = false; + } + } + + if (always_redirect) { + smap_add(&new, "always-redirect", "true"); + } + } else { + if (op->peer) { + smap_add(&new, "peer", op->peer->key); + if (op->nbrp->ha_chassis_group || + op->nbrp->n_gateway_chassis) { + char *redirect_name = + ovn_chassis_redirect_name(op->nbrp->name); + smap_add(&new, "chassis-redirect-port", redirect_name); + free(redirect_name); + } + } + if (chassis_name) { + smap_add(&new, "l3gateway-chassis", chassis_name); + } + } + + const char *ipv6_pd_list = smap_get(&op->sb->options, + "ipv6_ra_pd_list"); + if (ipv6_pd_list) { + smap_add(&new, "ipv6_ra_pd_list", ipv6_pd_list); + } + + sbrec_port_binding_set_options(op->sb, &new); + smap_destroy(&new); +} + +static void ovn_update_ipv6_options(struct hmap *lr_ports); +static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); + /* Sync the SB Port bindings which needs to be updated. * Presently it syncs the nat column of port bindings corresponding to * the logical switch ports. */ void -sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports) +sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, + struct hmap *lr_ports) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_op(op); + sync_pb_for_lsp(op); + } + + HMAP_FOR_EACH (op, key_node, lr_ports) { + sync_pb_for_lrp(op); } + + ovn_update_ipv6_options(lr_ports); } /* Sync the SB Port bindings for the added and updated logical switch ports @@ -4902,12 +4931,22 @@ sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; - sync_pb_for_op(op); + if (op->nbsp) { + sync_pb_for_lsp(op); + } else { + sync_pb_for_lrp(op); + ovn_update_ipv6_opt_for_op(op); + } } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; - sync_pb_for_op(op); + if (op->nbsp) { + sync_pb_for_lsp(op); + } else { + sync_pb_for_lrp(op); + ovn_update_ipv6_opt_for_op(op); + } } return true; @@ -5703,20 +5742,21 @@ fail: bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *sbrec_port_binding_table, - struct hmap *ls_ports) + struct hmap *ls_ports, struct hmap *lr_ports) { const struct sbrec_port_binding *pb; static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED (pb, sbrec_port_binding_table) { - /* Return false if the 'pb' belongs to a router port. We don't handle - * I-P for router ports yet. */ - if (is_pb_router_type(pb)) { - return false; - } + bool is_router_port = is_pb_router_type(pb); + struct ovn_port *op; - struct ovn_port *op = ovn_port_find(ls_ports, pb->logical_port); - if (op && !op->lsp_can_be_inc_processed) { - return false; + if (is_router_port) { + op = ovn_port_find(lr_ports, pb->logical_port); + } else { + op = ovn_port_find(ls_ports, pb->logical_port); + if (op && !op->lsp_can_be_inc_processed) { + return false; + } } if (sbrec_port_binding_is_new(pb)) { @@ -5725,7 +5765,8 @@ northd_handle_sb_port_binding_changes( * pointer in northd data. Fallback to recompute otherwise. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is created but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } op->sb = pb; @@ -5736,7 +5777,7 @@ northd_handle_sb_port_binding_changes( * sb idl pointers and other unexpected behavior. */ if (op) { VLOG_WARN_RL(&rl, "A port-binding for %s is deleted but the " - "LSP still exists.", pb->logical_port); + "LSP/LRP still exists.", pb->logical_port); return false; } } else { @@ -5746,7 +5787,8 @@ northd_handle_sb_port_binding_changes( * Fallback to recompute for anything unexpected. */ if (!op) { VLOG_WARN_RL(&rl, "A port-binding for %s is updated but the " - "LSP is not found.", pb->logical_port); + "%s is not found.", pb->logical_port, + is_router_port ? "LRP" : "LSP"); return false; } if (op->sb != pb) { @@ -7895,67 +7937,72 @@ static void copy_ra_to_sb(struct ovn_port *op, const char *address_mode); static void -ovn_update_ipv6_options(struct hmap *lr_ports) +ovn_update_ipv6_opt_for_op(struct ovn_port *op) { - struct ovn_port *op; - HMAP_FOR_EACH (op, key_node, lr_ports) { - ovs_assert(op->nbrp); - - if (op->nbrp->peer || !op->peer) { - continue; - } + if (op->nbrp->peer || !op->peer) { + return; + } - if (!op->lrp_networks.n_ipv6_addrs) { - continue; - } + if (!op->lrp_networks.n_ipv6_addrs) { + return; + } - struct smap options; - smap_clone(&options, &op->sb->options); + struct smap options; + smap_clone(&options, &op->sb->options); - /* enable IPv6 prefix delegation */ - bool prefix_delegation = smap_get_bool(&op->nbrp->options, - "prefix_delegation", false); - if (!lrport_is_enabled(op->nbrp)) { - prefix_delegation = false; - } - if (smap_get_bool(&options, "ipv6_prefix_delegation", - false) != prefix_delegation) { - smap_add(&options, "ipv6_prefix_delegation", - prefix_delegation ? "true" : "false"); - } + /* enable IPv6 prefix delegation */ + bool prefix_delegation = smap_get_bool(&op->nbrp->options, + "prefix_delegation", false); + if (!lrport_is_enabled(op->nbrp)) { + prefix_delegation = false; + } + if (smap_get_bool(&options, "ipv6_prefix_delegation", + false) != prefix_delegation) { + smap_add(&options, "ipv6_prefix_delegation", + prefix_delegation ? "true" : "false"); + } - bool ipv6_prefix = smap_get_bool(&op->nbrp->options, + bool ipv6_prefix = smap_get_bool(&op->nbrp->options, "prefix", false); - if (!lrport_is_enabled(op->nbrp)) { - ipv6_prefix = false; - } - if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { - smap_add(&options, "ipv6_prefix", - ipv6_prefix ? "true" : "false"); - } - sbrec_port_binding_set_options(op->sb, &options); + if (!lrport_is_enabled(op->nbrp)) { + ipv6_prefix = false; + } + if (smap_get_bool(&options, "ipv6_prefix", false) != ipv6_prefix) { + smap_add(&options, "ipv6_prefix", + ipv6_prefix ? "true" : "false"); + } + sbrec_port_binding_set_options(op->sb, &options); - smap_destroy(&options); + smap_destroy(&options); - const char *address_mode = smap_get( - &op->nbrp->ipv6_ra_configs, "address_mode"); + const char *address_mode = smap_get( + &op->nbrp->ipv6_ra_configs, "address_mode"); - if (!address_mode) { - continue; - } - if (strcmp(address_mode, "slaac") && - strcmp(address_mode, "dhcpv6_stateful") && - strcmp(address_mode, "dhcpv6_stateless")) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); - VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", - address_mode); - continue; - } + if (!address_mode) { + return; + } + if (strcmp(address_mode, "slaac") && + strcmp(address_mode, "dhcpv6_stateful") && + strcmp(address_mode, "dhcpv6_stateless")) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); + VLOG_WARN_RL(&rl, "Invalid address mode [%s] defined", + address_mode); + return; + } - if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", - false)) { - copy_ra_to_sb(op, address_mode); - } + if (smap_get_bool(&op->nbrp->ipv6_ra_configs, "send_periodic", + false)) { + copy_ra_to_sb(op, address_mode); + } +} + +static void +ovn_update_ipv6_options(struct hmap *lr_ports) +{ + struct ovn_port *op; + HMAP_FOR_EACH (op, key_node, lr_ports) { + ovs_assert(op->nbrp); + ovn_update_ipv6_opt_for_op(op); } } @@ -18027,8 +18074,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->lr_ports); stopwatch_stop(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); stopwatch_start(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - ovn_update_ipv6_options(&data->lr_ports); - ovn_update_ipv6_prefix(&data->lr_ports); sync_mirrors(ovnsb_txn, input_data->nbrec_mirror_table, input_data->sbrec_mirror_table); @@ -18359,6 +18404,8 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, &ha_ref_chassis_map); } shash_destroy(&ha_ref_chassis_map); + + ovn_update_ipv6_prefix(lr_ports); } const char * diff --git a/northd/northd.h b/northd/northd.h index 0478826f0a..3945e84bb8 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -368,7 +368,8 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *, struct hmap *lflows); bool northd_handle_sb_port_binding_changes( - const struct sbrec_port_binding_table *, struct hmap *ls_ports); + const struct sbrec_port_binding_table *, struct hmap *ls_ports, + struct hmap *lr_ports); struct tracked_lb_data; bool northd_handle_lb_data_changes(struct tracked_lb_data *, @@ -398,7 +399,8 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); -void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports); +void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, + struct hmap *lr_ports); bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); bool northd_has_tracked_data(struct northd_tracked_data *); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 699f6cfdce..55a244c8c4 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11018,7 +11018,9 @@ check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Adding a logical router port should result in recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11026,8 +11028,10 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for northd engine there will be both recompute and compute # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl lsp-add sw0 sw0-lr0 ovn-nbctl lsp-set-type sw0-lr0 router @@ -11035,7 +11039,9 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE ovn-nbctl ls-add public ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 @@ -11057,9 +11063,9 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute - -check as northd ovn-appctl -t NORTHD_TYPE vlog/set dbg +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Do checks for NATs. # Add a NAT. This should not result in recompute of both northd and lflow @@ -11068,6 +11074,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -11075,6 +11082,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -11082,6 +11090,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11089,6 +11098,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11096,13 +11106,15 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11111,6 +11123,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11124,31 +11137,35 @@ check ovn-nbctl lr-lb-add lr0 lb2 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute nocompute +check_engine_stats northd recompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11157,12 +11174,16 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP From patchwork Tue Oct 24 00:44:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854077 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtcq0Tz1z23jn for ; Tue, 24 Oct 2023 11:44:58 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 6D421614B6; Tue, 24 Oct 2023 00:44:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 6D421614B6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hEj6jOA54wFI; Tue, 24 Oct 2023 00:44:53 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id B5B126149E; Tue, 24 Oct 2023 00:44:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B5B126149E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9940DC0071; Tue, 24 Oct 2023 00:44:52 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 417F5C0032 for ; Tue, 24 Oct 2023 00:44:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2661C83C72 for ; Tue, 24 Oct 2023 00:44:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2661C83C72 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U8LH-quO5TPi for ; Tue, 24 Oct 2023 00:44:37 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp1.osuosl.org (Postfix) with ESMTPS id E9BDD83C09 for ; Tue, 24 Oct 2023 00:44:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E9BDD83C09 Received: by mail.gandi.net (Postfix) with ESMTPSA id 0028B60002; Tue, 24 Oct 2023 00:44:31 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:44:13 -0400 Message-ID: <20231024004413.4133608-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 05/18] northd: Add a new engine 'lr-nat' to manage lr NAT data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the NAT related data for each logical router which was earlier maintained by the northd engine node in the 'struct ovn_datapath'. Main inputs to this engine node are: - northd - NB logical router A record for each logical router is maintained in the 'lr_nats' hmap table and this record - stores the ovn_nat's Handlers are also added to handle the changes to both these inputs. This engine node becomes an input to 'lflow' node. This essentially decouples the lr NAT data from the northd engine node. Signed-off-by: Numan Siddique --- lib/ovn-util.c | 6 +- lib/ovn-util.h | 2 +- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 5 + northd/en-lr-nat.c | 480 ++++++++++++++++++++++++++++++++++++ northd/en-lr-nat.h | 131 ++++++++++ northd/en-sync-sb.c | 11 +- northd/inc-proc-northd.c | 9 + northd/northd.c | 508 ++++++++++++++------------------------- northd/northd.h | 26 +- tests/ovn-northd.at | 18 ++ 12 files changed, 850 insertions(+), 349 deletions(-) create mode 100644 northd/en-lr-nat.c create mode 100644 northd/en-lr-nat.h diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 33105202f2..05e635a6b4 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -395,7 +395,7 @@ extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, } bool -lport_addresses_is_empty(struct lport_addresses *laddrs) +lport_addresses_is_empty(const struct lport_addresses *laddrs) { return !laddrs->n_ipv4_addrs && !laddrs->n_ipv6_addrs; } @@ -405,6 +405,10 @@ destroy_lport_addresses(struct lport_addresses *laddrs) { free(laddrs->ipv4_addrs); free(laddrs->ipv6_addrs); + laddrs->ipv4_addrs = NULL; + laddrs->ipv6_addrs = NULL; + laddrs->n_ipv4_addrs = 0; + laddrs->n_ipv6_addrs = 0; } /* Returns a string of the IP address of 'laddrs' that overlaps with 'ip_s'. diff --git a/lib/ovn-util.h b/lib/ovn-util.h index bff50dbde9..5805415885 100644 --- a/lib/ovn-util.h +++ b/lib/ovn-util.h @@ -112,7 +112,7 @@ bool extract_sbrec_binding_first_mac(const struct sbrec_port_binding *binding, bool extract_lrp_networks__(char *mac, char **networks, size_t n_networks, struct lport_addresses *laddrs); -bool lport_addresses_is_empty(struct lport_addresses *); +bool lport_addresses_is_empty(const struct lport_addresses *); void destroy_lport_addresses(struct lport_addresses *); const char *find_lport_address(const struct lport_addresses *laddrs, const char *ip_s); diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 3452cc71cf..0a16da211e 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -32,5 +32,6 @@ #define LFLOWS_TO_SB_STOPWATCH_NAME "lflows_to_sb" #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" +#define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #endif diff --git a/northd/automake.mk b/northd/automake.mk index cf622fc3c9..ae367a2a8b 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -24,6 +24,8 @@ northd_ovn_northd_SOURCES = \ northd/en-sync-from-sb.h \ northd/en-lb-data.c \ northd/en-lb-data.h \ + northd/en-lr-nat.c \ + northd/en-lr-nat.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 96d03b7ada..22f398d419 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -19,6 +19,7 @@ #include #include "en-lflow.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-meters.h" @@ -40,6 +41,9 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); lflow_input->sbrec_bfd_table = @@ -61,6 +65,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; + lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-nat.c b/northd/en-lr-nat.c new file mode 100644 index 0000000000..79212e4c49 --- /dev/null +++ b/northd/en-lr-nat.c @@ -0,0 +1,480 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lr-nat.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_nat); + +/* Static function declarations. */ +static void lr_nat_table_init(struct lr_nat_table *); +static void lr_nat_table_clear(struct lr_nat_table *); +static void lr_nat_table_destroy(struct lr_nat_table *); +static void lr_nat_table_build(struct lr_nat_table *, + const struct ovn_datapaths *lr_datapaths); +struct lr_nat_record *lr_nat_table_find_(const struct lr_nat_table *, + const struct nbrec_logical_router *); + +static struct lr_nat_record *lr_nat_record_create( + struct lr_nat_table *, const struct ovn_datapath *); +static void lr_nat_record_init(struct lr_nat_record *); +static void lr_nat_record_reinit(struct lr_nat_record *); +static void lr_nat_record_destroy(struct lr_nat_record *); + +static void lr_nat_entries_init(struct lr_nat_record *); +static void lr_nat_entries_destroy(struct lr_nat_record *); +static void lr_nat_external_ips_init(struct lr_nat_record *); +static void lr_nat_external_ips_destroy(struct lr_nat_record *); +static bool get_force_snat_ip(struct lr_nat_record *, const char *key_type, + struct lport_addresses *); +static struct lr_nat_input lr_nat_get_input_data(struct engine_node *); +static bool is_lr_nats_changed(const struct nbrec_logical_router *); +static bool is_lr_nats_seqno_changed(const struct nbrec_logical_router *nbr); + + +const struct lr_nat_record * +lr_nat_table_find(const struct lr_nat_table *table, + const struct nbrec_logical_router *nbr) +{ + return lr_nat_table_find_(table, nbr); +} + +/* 'lr_nat' engine node manages the NB logical router NAT data. + */ +void * +en_lr_nat_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_nat_data *data = xzalloc(sizeof *data); + lr_nat_table_init(&data->lr_nats); + hmapx_init(&data->tracked_data.crupdated); + hmapx_init(&data->tracked_data.deleted); + return data; +} + +void +en_lr_nat_cleanup(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + lr_nat_table_destroy(&data->lr_nats); + hmapx_destroy(&data->tracked_data.crupdated); + hmapx_destroy(&data->tracked_data.deleted); +} + +void +en_lr_nat_clear_tracked_data(void *data_) +{ + struct ed_type_lr_nat_data *data = (struct ed_type_lr_nat_data *) data_; + + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &data->tracked_data.deleted) { + lr_nat_record_destroy(hmapx_node->data); + hmapx_delete(&data->tracked_data.deleted, hmapx_node); + } + + hmapx_clear(&data->tracked_data.crupdated); + data->tracked = false; +} + +void +en_lr_nat_run(struct engine_node *node, void *data_) +{ + struct lr_nat_input input_data = lr_nat_get_input_data(node); + struct ed_type_lr_nat_data *data = data_; + + stopwatch_start(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + data->tracked = false; + lr_nat_table_clear(&data->lr_nats); + lr_nat_table_build(&data->lr_nats, input_data.lr_datapaths); + + stopwatch_stop(LR_NAT_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +lr_nat_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_data->change_tracked) { + return false; + } + + return true; +} + +bool +lr_nat_logical_router_handler(struct engine_node *node, void *data_) +{ + struct lr_nat_input input_data = lr_nat_get_input_data(node); + struct ed_type_lr_nat_data *data = data_; + const struct nbrec_logical_router *nbr; + + NBREC_LOGICAL_ROUTER_TABLE_FOR_EACH_TRACKED ( + nbr, input_data.nbrec_logical_router_table) { + if (!is_lr_nats_changed(nbr)) { + continue; + } + + struct lr_nat_record *lrnat_rec = lr_nat_table_find_(&data->lr_nats, + nbr); + + if (nbrec_logical_router_is_deleted(nbr)) { + if (lrnat_rec) { + /* Remove the record from the entries. */ + hmap_remove(&data->lr_nats.entries, &lrnat_rec->key_node); + + /* Add the lrnet rec to the tracking data. */ + hmapx_add(&data->tracked_data.deleted, lrnat_rec); + } + } else { + if (!lrnat_rec) { + const struct ovn_datapath *od; + od = ovn_datapath_find(&input_data.lr_datapaths->datapaths, + &nbr->header_.uuid); + ovs_assert(od); + lrnat_rec = lr_nat_record_create(&data->lr_nats, od); + } else { + lr_nat_record_reinit(lrnat_rec); + } + + /* Add the lrnet rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lrnat_rec); + } + } + + if (!hmapx_is_empty(&data->tracked_data.deleted) + || !hmapx_is_empty(&data->tracked_data.crupdated)) { + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + return true; +} + +/* static functions. */ +static void +lr_nat_table_init(struct lr_nat_table *table) +{ + *table = (struct lr_nat_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_nat_table_clear(struct lr_nat_table *table) +{ + struct lr_nat_record *lrnat_rec; + HMAP_FOR_EACH_POP (lrnat_rec, key_node, &table->entries) { + lr_nat_record_destroy(lrnat_rec); + } +} + +static void +lr_nat_table_build(struct lr_nat_table *table, + const struct ovn_datapaths *lr_datapaths) +{ + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { + lr_nat_record_create(table, od); + } +} + +static void +lr_nat_table_destroy(struct lr_nat_table *table) +{ + lr_nat_table_clear(table); + hmap_destroy(&table->entries); +} + +struct lr_nat_record * +lr_nat_table_find_(const struct lr_nat_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_nat_record *lrnat_rec; + + HMAP_FOR_EACH_WITH_HASH (lrnat_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lrnat_rec->od->nbr) { + return lrnat_rec; + } + } + return NULL; +} + +static struct lr_nat_record * +lr_nat_record_create(struct lr_nat_table *table, + const struct ovn_datapath *od) +{ + ovs_assert(od->nbr); + + struct lr_nat_record *lrnat_rec = xzalloc(sizeof *lrnat_rec); + lrnat_rec->od = od; + lr_nat_record_init(lrnat_rec); + + hmap_insert(&table->entries, &lrnat_rec->key_node, + uuid_hash(&od->nbr->header_.uuid)); + + return lrnat_rec; +} + +static void +lr_nat_record_init(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_init(lrnat_rec); + lr_nat_external_ips_init(lrnat_rec); +} + +static void +lr_nat_record_reinit(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + lr_nat_record_init(lrnat_rec); +} + +static void +lr_nat_record_destroy(struct lr_nat_record *lrnat_rec) +{ + lr_nat_entries_destroy(lrnat_rec); + lr_nat_external_ips_destroy(lrnat_rec); + free(lrnat_rec); +} + +static void +lr_nat_external_ips_init(struct lr_nat_record *lrnat_rec) +{ + sset_init(&lrnat_rec->external_ips); + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + sset_add(&lrnat_rec->external_ips, + lrnat_rec->od->nbr->nat[i]->external_ip); + } +} + +static void +lr_nat_external_ips_destroy(struct lr_nat_record *lrnat_rec) +{ + sset_destroy(&lrnat_rec->external_ips); +} + +static void +snat_ip_add(struct lr_nat_record *lrnat_rec, const char *ip, + struct ovn_nat *nat_entry) +{ + struct ovn_snat_ip *snat_ip = shash_find_data(&lrnat_rec->snat_ips, ip); + + if (!snat_ip) { + snat_ip = xzalloc(sizeof *snat_ip); + ovs_list_init(&snat_ip->snat_entries); + shash_add(&lrnat_rec->snat_ips, ip, snat_ip); + } + + if (nat_entry) { + ovs_list_push_back(&snat_ip->snat_entries, + &nat_entry->ext_addr_list_node); + } +} + +static void +lr_nat_entries_init(struct lr_nat_record *lrnat_rec) +{ + shash_init(&lrnat_rec->snat_ips); + sset_init(&lrnat_rec->external_macs); + lrnat_rec->has_distributed_nat = false; + + if (get_force_snat_ip(lrnat_rec, "dnat", + &lrnat_rec->dnat_force_snat_addrs)) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + + /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ + const char *lb_force_snat = + smap_get(&lrnat_rec->od->nbr->options, "lb_force_snat_ip"); + if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") + && smap_get(&lrnat_rec->od->nbr->options, "chassis")) { + + /* Set it to true only if its gateway router and + * options:lb_force_snat_ip=router_ip. */ + lrnat_rec->lb_force_snat_router_ip = true; + } else { + lrnat_rec->lb_force_snat_router_ip = false; + + /* Check if 'lb_force_snat_ip' is configured with a set of + * IP address(es). */ + if (get_force_snat_ip(lrnat_rec, "lb", + &lrnat_rec->lb_force_snat_addrs)) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, + NULL); + } + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { + snat_ip_add(lrnat_rec, + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, + NULL); + } + } + } + + if (!lrnat_rec->od->nbr->n_nat) { + return; + } + + lrnat_rec->nat_entries = + xmalloc(lrnat_rec->od->nbr->n_nat * sizeof *lrnat_rec->nat_entries); + + for (size_t i = 0; i < lrnat_rec->od->nbr->n_nat; i++) { + const struct nbrec_nat *nat = lrnat_rec->od->nbr->nat[i]; + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + + nat_entry->nb = nat; + if (!extract_ip_addresses(nat->external_ip, + &nat_entry->ext_addrs) || + !nat_entry_is_valid(nat_entry)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + + VLOG_WARN_RL(&rl, + "Bad ip address %s in nat configuration " + "for router %s", nat->external_ip, + lrnat_rec->od->nbr->name); + continue; + } + + /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ + if (!strcmp(nat->type, "snat")) { + if (!nat_entry_is_v6(nat_entry)) { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv4_addrs[0].addr_s, + nat_entry); + } else { + snat_ip_add(lrnat_rec, + nat_entry->ext_addrs.ipv6_addrs[0].addr_s, + nat_entry); + } + } else { + if (!strcmp(nat->type, "dnat_and_snat") + && nat->logical_port && nat->external_mac) { + lrnat_rec->has_distributed_nat = true; + } + + if (nat->external_mac) { + sset_add(&lrnat_rec->external_macs, nat->external_mac); + } + } + } + lrnat_rec->n_nat_entries = lrnat_rec->od->nbr->n_nat; +} + +static bool +get_force_snat_ip(struct lr_nat_record *lrnat_rec, const char *key_type, + struct lport_addresses *laddrs) +{ + char *key = xasprintf("%s_force_snat_ip", key_type); + const char *addresses = smap_get(&lrnat_rec->od->nbr->options, key); + free(key); + + if (!addresses) { + return false; + } + + if (!extract_ip_address(addresses, laddrs)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", + addresses, UUID_ARGS(&lrnat_rec->od->nbr->header_.uuid)); + return false; + } + + return true; +} + +static void +lr_nat_entries_destroy(struct lr_nat_record *lrnat_rec) +{ + shash_destroy_free_data(&lrnat_rec->snat_ips); + destroy_lport_addresses(&lrnat_rec->dnat_force_snat_addrs); + destroy_lport_addresses(&lrnat_rec->lb_force_snat_addrs); + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + destroy_lport_addresses(&lrnat_rec->nat_entries[i].ext_addrs); + } + + free(lrnat_rec->nat_entries); + lrnat_rec->nat_entries = NULL; + lrnat_rec->n_nat_entries = 0; + sset_destroy(&lrnat_rec->external_macs); +} + +static struct lr_nat_input +lr_nat_get_input_data(struct engine_node *node) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + return (struct lr_nat_input) { + .nbrec_logical_router_table = + EN_OVSDB_GET(engine_get_input("NB_logical_router", node)), + .lr_datapaths = &northd_data->lr_datapaths, + }; +} + +static bool +is_lr_nats_seqno_changed(const struct nbrec_logical_router *nbr) +{ + for (size_t i = 0; i < nbr->n_nat; i++) { + if (nbrec_nat_row_get_seqno(nbr->nat[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_lr_nats_changed(const struct nbrec_logical_router *nbr) { + return (nbrec_logical_router_is_new(nbr) + || nbrec_logical_router_is_deleted(nbr) + || nbrec_logical_router_is_updated(nbr, + NBREC_LOGICAL_ROUTER_COL_NAT) + || nbrec_logical_router_is_updated( + nbr, NBREC_LOGICAL_ROUTER_COL_OPTIONS) + || is_lr_nats_seqno_changed(nbr)); +} diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h new file mode 100644 index 0000000000..6ed1f5ed52 --- /dev/null +++ b/northd/en-lr-nat.h @@ -0,0 +1,131 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_NAT_H +#define EN_LR_NAT_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +/* Contains a NAT entry with the external addresses pre-parsed. */ +struct ovn_nat { + const struct nbrec_nat *nb; + struct lport_addresses ext_addrs; + struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP + * list of nat entries. Currently + * only used for SNAT. + */ +}; + +/* Stores the list of SNAT entries referencing a unique SNAT IP address. + * The 'snat_entries' list will be empty if the SNAT IP is used only for + * dnat_force_snat_ip or lb_force_snat_ip. + */ +struct ovn_snat_ip { + struct ovs_list snat_entries; +}; + +struct lr_nat_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + + struct ovn_nat *nat_entries; + size_t n_nat_entries; + + bool has_distributed_nat; + + /* Set of nat external ips on the router. */ + struct sset external_ips; + + /* Set of nat external macs on the router. */ + struct sset external_macs; + + /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ + struct shash snat_ips; + + struct lport_addresses dnat_force_snat_addrs; + struct lport_addresses lb_force_snat_addrs; + bool lb_force_snat_router_ip; +}; + +struct lr_nat_tracked_data { + /* Created or updated logical router with NAT data. */ + struct hmapx crupdated; + + /* Deleted logical router with NAT data. */ + struct hmapx deleted; /* Stores 'struct lr_nat_record'. */ +}; + +struct lr_nat_table { + struct hmap entries; /* Stores struct lr_nat_record. */ +}; + +const struct lr_nat_record * lr_nat_table_find(const struct lr_nat_table *, + const struct nbrec_logical_router *); + +/* Incremental processing implementation. */ +struct lr_nat_input { + /* Northbound table references. */ + const struct nbrec_logical_router_table *nbrec_logical_router_table; + + const struct ovn_datapaths *lr_datapaths; +}; + +struct ed_type_lr_nat_data { + struct lr_nat_table lr_nats; + + bool tracked; + struct lr_nat_tracked_data tracked_data; +}; + +void *en_lr_nat_init(struct engine_node *, struct engine_arg *); +void en_lr_nat_cleanup(void *data); +void en_lr_nat_clear_tracked_data(void *data); +void en_lr_nat_run(struct engine_node *, void *data); + +bool lr_nat_logical_router_handler(struct engine_node *, void *data); +bool lr_nat_northd_handler(struct engine_node *, void *data); + +/* Returns true if a 'nat_entry' is valid, i.e.: + * - parsing was successful. + * - the string yielded exactly one IPv4 address or exactly one IPv6 address. + */ +static inline bool +nat_entry_is_valid(const struct ovn_nat *nat_entry) +{ + const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; + + return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || + (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); +} + +static inline bool +nat_entry_is_v6(const struct ovn_nat *nat_entry) +{ + return nat_entry->ext_addrs.n_ipv6_addrs > 0; +} + +#endif /* EN_LR_NAT_H */ \ No newline at end of file diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index a14c609acd..10ade620e7 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -21,6 +21,7 @@ #include "lib/svec.h" #include "openvswitch/util.h" +#include "en-lr-nat.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -287,9 +288,10 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports); + &northd_data->lr_ports, &lr_nat_data->lr_nats); engine_set_node_state(node, EN_UPDATED); } @@ -314,8 +316,11 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + if (!sync_pbs_for_northd_changed_ovn_ports( - &nd->trk_northd_changes.trk_ovn_ports)) { + &nd->trk_northd_changes.trk_ovn_ports, &lr_nat_data->lr_nats)) { return false; } diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 04df0b06c2..2bd66b8808 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -146,6 +147,7 @@ static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -189,6 +191,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, northd_nb_logical_router_handler); engine_add_input(&en_northd, &en_lb_data, northd_lb_data_handler); + engine_add_input(&en_lr_nat, &en_northd, + lr_nat_northd_handler); + engine_add_input(&en_lr_nat, &en_nb_logical_router, + lr_nat_logical_router_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -212,6 +219,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); + engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); @@ -235,6 +243,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); + engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index c9c7045755..674c461561 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -43,6 +43,7 @@ #include "memory.h" #include "northd.h" #include "en-lb-data.h" +#include "en-lr-nat.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -555,184 +556,6 @@ ovn_mcast_group_allocate_key(struct mcast_info *mcast_info) &mcast_info->group_tnlid_hint); } -/* Contains a NAT entry with the external addresses pre-parsed. */ -struct ovn_nat { - const struct nbrec_nat *nb; - struct lport_addresses ext_addrs; - struct ovs_list ext_addr_list_node; /* Linkage in the per-external IP - * list of nat entries. Currently - * only used for SNAT. - */ -}; - -/* Stores the list of SNAT entries referencing a unique SNAT IP address. - * The 'snat_entries' list will be empty if the SNAT IP is used only for - * dnat_force_snat_ip or lb_force_snat_ip. - */ -struct ovn_snat_ip { - struct ovs_list snat_entries; -}; - -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs); - -/* Returns true if a 'nat_entry' is valid, i.e.: - * - parsing was successful. - * - the string yielded exactly one IPv4 address or exactly one IPv6 address. - */ -static bool -nat_entry_is_valid(const struct ovn_nat *nat_entry) -{ - const struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; - - return (ext_addrs->n_ipv4_addrs == 1 && ext_addrs->n_ipv6_addrs == 0) || - (ext_addrs->n_ipv4_addrs == 0 && ext_addrs->n_ipv6_addrs == 1); -} - -static bool -nat_entry_is_v6(const struct ovn_nat *nat_entry) -{ - return nat_entry->ext_addrs.n_ipv6_addrs > 0; -} - -static void -snat_ip_add(struct ovn_datapath *od, const char *ip, struct ovn_nat *nat_entry) -{ - struct ovn_snat_ip *snat_ip = shash_find_data(&od->snat_ips, ip); - - if (!snat_ip) { - snat_ip = xzalloc(sizeof *snat_ip); - ovs_list_init(&snat_ip->snat_entries); - shash_add(&od->snat_ips, ip, snat_ip); - } - - if (nat_entry) { - ovs_list_push_back(&snat_ip->snat_entries, - &nat_entry->ext_addr_list_node); - } -} - -static void -init_nat_entries(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - shash_init(&od->snat_ips); - if (get_force_snat_ip(od, "dnat", &od->dnat_force_snat_addrs)) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - - /* Check if 'lb_force_snat_ip' is configured with 'router_ip'. */ - const char *lb_force_snat = - smap_get(&od->nbr->options, "lb_force_snat_ip"); - if (lb_force_snat && !strcmp(lb_force_snat, "router_ip") - && smap_get(&od->nbr->options, "chassis")) { - /* Set it to true only if its gateway router and - * options:lb_force_snat_ip=router_ip. */ - od->lb_force_snat_router_ip = true; - } else { - od->lb_force_snat_router_ip = false; - - /* Check if 'lb_force_snat_ip' is configured with a set of - * IP address(es). */ - if (get_force_snat_ip(od, "lb", &od->lb_force_snat_addrs)) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, - NULL); - } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { - snat_ip_add(od, od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, - NULL); - } - } - } - - if (!od->nbr->n_nat) { - return; - } - - od->nat_entries = xmalloc(od->nbr->n_nat * sizeof *od->nat_entries); - - for (size_t i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; - struct ovn_nat *nat_entry = &od->nat_entries[i]; - - nat_entry->nb = nat; - if (!extract_ip_addresses(nat->external_ip, - &nat_entry->ext_addrs) || - !nat_entry_is_valid(nat_entry)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - - VLOG_WARN_RL(&rl, - "Bad ip address %s in nat configuration " - "for router %s", nat->external_ip, od->nbr->name); - continue; - } - - /* If this is a SNAT rule add the IP to the set of unique SNAT IPs. */ - if (!strcmp(nat->type, "snat")) { - if (!nat_entry_is_v6(nat_entry)) { - snat_ip_add(od, nat_entry->ext_addrs.ipv4_addrs[0].addr_s, - nat_entry); - } else { - snat_ip_add(od, nat_entry->ext_addrs.ipv6_addrs[0].addr_s, - nat_entry); - } - } - - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac) { - od->has_distributed_nat = true; - } - } - od->n_nat_entries = od->nbr->n_nat; -} - -static void -destroy_nat_entries(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - shash_destroy_free_data(&od->snat_ips); - destroy_lport_addresses(&od->dnat_force_snat_addrs); - destroy_lport_addresses(&od->lb_force_snat_addrs); - - for (size_t i = 0; i < od->n_nat_entries; i++) { - destroy_lport_addresses(&od->nat_entries[i].ext_addrs); - } -} - -static void -init_router_external_ips(struct ovn_datapath *od) -{ - ovs_assert(od->nbr); - - sset_init(&od->external_ips); - for (size_t i = 0; i < od->nbr->n_nat; i++) { - sset_add(&od->external_ips, od->nbr->nat[i]->external_ip); - } -} - -static void -destroy_router_external_ips(struct ovn_datapath *od) -{ - if (!od->nbr) { - return; - } - - sset_destroy(&od->external_ips); -} - static bool lb_has_vip(const struct nbrec_load_balancer *lb) { @@ -853,10 +676,7 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_nat_entries(od); - destroy_router_external_ips(od); destroy_lb_for_datapath(od); - free(od->nat_entries); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -873,8 +693,8 @@ ovn_datapath_get_type(const struct ovn_datapath *od) } static struct ovn_datapath * -ovn_datapath_find(const struct hmap *datapaths, - const struct uuid *uuid) +ovn_datapath_find_(const struct hmap *datapaths, + const struct uuid *uuid) { struct ovn_datapath *od; @@ -886,6 +706,13 @@ ovn_datapath_find(const struct hmap *datapaths, return NULL; } +const struct ovn_datapath * +ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid) +{ + return ovn_datapath_find_(datapaths, uuid); +} + static struct ovn_datapath * ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) { @@ -924,7 +751,7 @@ ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, if (!dps) { return NULL; } - struct ovn_datapath *od = ovn_datapath_find(dps, &key); + struct ovn_datapath *od = ovn_datapath_find_(dps, &key); if (od && (od->sb == sb)) { return od; } @@ -1206,7 +1033,7 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - if (ovn_datapath_find(datapaths, &key)) { + if (ovn_datapath_find_(datapaths, &key)) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_INFO_RL( &rl, "deleting Datapath_Binding "UUID_FMT" with " @@ -1223,8 +1050,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, const struct nbrec_logical_switch *nbs; NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH (nbs, nbrec_ls_table) { - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbs->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbs->header_.uuid); if (od) { od->nbs = nbs; ovs_list_remove(&od->list); @@ -1247,8 +1074,8 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, continue; } - struct ovn_datapath *od = ovn_datapath_find(datapaths, - &nbr->header_.uuid); + struct ovn_datapath *od = ovn_datapath_find_(datapaths, + &nbr->header_.uuid); if (od) { if (!od->nbs) { od->nbr = nbr; @@ -1269,8 +1096,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_nat_entries(od); - init_router_external_ips(od); init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; @@ -4843,7 +4668,7 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op) +sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) { ovs_assert(op->nbrp); @@ -4852,10 +4677,14 @@ sync_pb_for_lrp(struct ovn_port *op) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find(lr_nats, op->od->nbr); + ovs_assert(lrnat_rec); + smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !op->od->has_distributed_nat && + !lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4906,7 +4735,7 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports) + struct hmap *lr_ports, const struct lr_nat_table *lr_nats) { ovs_assert(ovnsb_idl_txn); @@ -4916,7 +4745,7 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op); + sync_pb_for_lrp(op, lr_nats); } ovn_update_ipv6_options(lr_ports); @@ -4925,7 +4754,8 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) +sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports, + const struct lr_nat_table *lr_nats) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -4934,7 +4764,7 @@ sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) if (op->nbsp) { sync_pb_for_lsp(op); } else { - sync_pb_for_lrp(op); + sync_pb_for_lrp(op, lr_nats); ovn_update_ipv6_opt_for_op(op); } } @@ -4944,7 +4774,7 @@ sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *trk_ovn_ports) if (op->nbsp) { sync_pb_for_lsp(op); } else { - sync_pb_for_lrp(op); + sync_pb_for_lrp(op, lr_nats); ovn_update_ipv6_opt_for_op(op); } } @@ -5615,7 +5445,7 @@ northd_handle_ls_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, nbrec_logical_switch_is_deleted(changed_ls)) { goto fail; } - struct ovn_datapath *od = ovn_datapath_find( + struct ovn_datapath *od = ovn_datapath_find_( &nd->ls_datapaths.datapaths, &changed_ls->header_.uuid); if (!od) { @@ -5931,7 +5761,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, struct crupdated_od_lb_data *codlb; LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_ls_lbs) { - od = ovn_datapath_find(&ls_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&ls_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -5971,7 +5801,7 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { - od = ovn_datapath_find(&lr_datapaths->datapaths, &codlb->od_uuid); + od = ovn_datapath_find_(&lr_datapaths->datapaths, &codlb->od_uuid); ovs_assert(od); struct uuidset_node *uuidnode; @@ -9322,31 +9152,15 @@ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows) { - struct sset all_eth_addrs = SSET_INITIALIZER(&all_eth_addrs); struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - sset_add(&all_eth_addrs, op->lrp_networks.ea_s); - - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; - const struct nbrec_nat *nat = nat_entry->nb; - - if (!nat_entry_is_valid(nat_entry)) { - continue; - } - - if (!strcmp(nat->type, "snat")) { - continue; - } - - if (!nat->external_mac) { - continue; - } - sset_add(&all_eth_addrs, nat->external_mac); - } + const struct lr_nat_record *lrnat_rec = lr_nat_table_find( + lr_nats, op->od->nbr); + ovs_assert(lrnat_rec); /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated @@ -9356,8 +9170,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, */ const char *eth_addr; - ds_put_cstr(ð_src, "{"); - SSET_FOR_EACH (eth_addr, &all_eth_addrs) { + ds_put_format(ð_src, "{%s, ", op->lrp_networks.ea_s); + SSET_FOR_EACH (eth_addr, &lrnat_rec->external_macs) { ds_put_format(ð_src, "%s, ", eth_addr); } ds_chomp(ð_src, ' '); @@ -9370,7 +9184,6 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), "outport = \""MC_FLOOD_L2"\"; output;"); - sset_destroy(&all_eth_addrs); ds_destroy(ð_src); ds_destroy(&match); } @@ -9476,6 +9289,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -9520,8 +9334,38 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } + + /* Self originated ARP requests/RARP/ND need to be flooded as usual. + * + * However, if the switch doesn't have any non-router ports we shouldn't + * even try to flood. + * + * Priority: 75. + */ + if (sw_od->n_router_ports != sw_od->nbs->n_ports) { + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + lflows); + } + + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find(lr_nats, op->od->nbr); + + if (!lrnat_rec) { + return; + } + + for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9549,28 +9393,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } } - - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - - /* Self originated ARP requests/RARP/ND need to be flooded as usual. - * - * However, if the switch doesn't have any non-router ports we shouldn't - * even try to flood. - * - * Priority: 75. - */ - if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lflows); - } } static void @@ -10624,6 +10446,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -10638,8 +10461,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, - &op->nbsp->header_); + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, + lflows, &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -12002,27 +11825,6 @@ op_put_v6_networks(struct ds *ds, const struct ovn_port *op) ds_put_cstr(ds, "}"); } -static bool -get_force_snat_ip(struct ovn_datapath *od, const char *key_type, - struct lport_addresses *laddrs) -{ - char *key = xasprintf("%s_force_snat_ip", key_type); - const char *addresses = smap_get(&od->nbr->options, key); - free(key); - - if (!addresses) { - return false; - } - - if (!extract_ip_address(addresses, laddrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "bad ip %s in options of router "UUID_FMT"", - addresses, UUID_ARGS(&od->key)); - return false; - } - - return true; -} enum lrouter_nat_lb_flow_type { LROUTER_NAT_LB_FLOW_NORMAL = 0, @@ -12174,6 +11976,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -12279,10 +12082,13 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; + const struct lr_nat_record *lrnat_rec = + lr_nat_table_find(lr_nats, od->nbr); + ovs_assert(lrnat_rec); if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; - } else if (!lport_addresses_is_empty(&od->lb_force_snat_addrs) || - od->lb_force_snat_router_ip) { + } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) + || lrnat_rec->lb_force_snat_router_ip) { type = LROUTER_NAT_LB_FLOW_FORCE_SNAT; } else { type = LROUTER_NAT_LB_FLOW_NORMAL; @@ -12298,7 +12104,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, bitmap_set1(aff_dp_bitmap[type], index); } - if (sset_contains(&od->external_ips, lb_vip->vip_str)) { + if (sset_contains(&lrnat_rec->external_ips, lb_vip->vip_str)) { /* The load balancer vip is also present in the NAT entries. * So add a high priority lflow to advance the the packet * destined to the vip (and the vip port if defined) @@ -12428,6 +12234,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, + const struct lr_nat_table *lr_nats, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -12443,8 +12250,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lflows, match, action, - meter_groups, features, + lr_datapaths, lr_nats, lflows, match, + action, meter_groups, features, svc_monitor_map); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { @@ -12843,7 +12650,9 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, } static void -build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, +build_lrouter_drop_own_dest(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, + enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) { @@ -12853,7 +12662,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12882,7 +12692,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, enum ovn_stage stage, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&op->od->snat_ips, ip); + bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, + ip); bool router_ip_in_lb_ips = !!sset_find(&op->od->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || @@ -12935,11 +12746,12 @@ build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); - if (!op->peer || !op->od->lb_force_snat_router_ip) { + if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { return; } @@ -13892,8 +13704,8 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows, - struct ds *match, struct ds *actions) + struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13969,8 +13781,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, S_ROUTER_IN_ARP_RESOLVE, 2, true, - lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, + true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -14300,6 +14112,7 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -14335,8 +14148,16 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); - for (int j = 0; j < od->n_nat_entries; j++) { - const struct ovn_nat *nat = &od->nat_entries[j]; + + const struct lr_nat_record *lrnat_rec = lr_nat_table_find( + lr_nats, od->nbr); + + if (!lrnat_rec) { + continue; + } + + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { + const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; if (!lrouter_dnat_and_snat_is_stateless(nat->nb) || (!nat->nb->allowed_ext_ips && !nat->nb->exempted_ext_ips)) { @@ -14774,10 +14595,15 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, + const struct lr_nat_table *lr_nats, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); + if (!od->nbr->n_nat) { + return; + } + /* Priority-90-92 flows handle ARP requests and ND packets. Most are * per logical port but DNAT addresses can be handled per datapath * for non gateway router ports. @@ -14786,8 +14612,12 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - for (int i = 0; i < od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &od->nat_entries[i]; + const struct lr_nat_record *lrnat_rec = lr_nat_table_find( + lr_nats, od->nbr); + ovs_assert(lrnat_rec); + + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14795,8 +14625,8 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } @@ -14805,7 +14635,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14823,6 +14653,7 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, + const struct lr_nat_record *lrnat_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -15059,14 +14890,14 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * also a SNAT IP. Those are dropped later, in stage * "lr_in_arp_resolve", if unSNAT was unsuccessful. * - * If op->od->lb_force_snat_router_ip is true, it means the IP of the + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the * router port is also SNAT IP. * * Priority 60. */ - if (!op->od->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, S_ROUTER_IN_IP_INPUT, 60, false, - lflows); + if (!lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, + false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -15081,8 +14912,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (size_t i = 0; i < op->od->nbr->n_nat; i++) { - struct ovn_nat *nat_entry = &op->od->nat_entries[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -15090,18 +14921,18 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } /* Skip SNAT entries for now, we handle unique SNAT IPs separately - * below. - */ + * below. + */ if (!strcmp(nat_entry->nb->type, "snat")) { continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &op->od->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -15110,9 +14941,9 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), - struct ovn_nat, ext_addr_list_node); + struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups); } } @@ -15221,6 +15052,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, @@ -15244,7 +15076,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, nat->external_ip); if (od->is_gw_router) { - if (!lport_addresses_is_empty(&od->dnat_force_snat_addrs)) { + if (!lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs)) { /* Indicate to the future tables that a DNAT has taken * place and a force SNAT needs to be done in the * Egress SNAT table. */ @@ -15800,6 +15632,7 @@ static void build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15910,14 +15743,18 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); + const struct lr_nat_record *lrnat_rec = lr_nat_table_find(lr_nats, + od->nbr); + ovs_assert(lrnat_rec); bool dnat_force_snat_ip = - !lport_addresses_is_empty(&od->dnat_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->dnat_force_snat_addrs); bool lb_force_snat_ip = - !lport_addresses_is_empty(&od->lb_force_snat_addrs); + !lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs); - for (int i = 0; i < od->nbr->n_nat; i++) { - const struct nbrec_nat *nat = od->nbr->nat[i]; + for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + const struct nbrec_nat *nat = nat_entry->nb; struct eth_addr mac = eth_addr_broadcast; bool is_v6, distributed_nat; ovs_be32 mask; @@ -15955,7 +15792,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, distributed_nat, is_v6, l3dgw_port); } /* S_ROUTER_IN_DNAT */ - build_lrouter_in_dnat_flow(lflows, od, nat, match, actions, + build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, l3dgw_port, stateless); @@ -16164,25 +16001,25 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, /* Handle force SNAT options set in the gateway router. */ if (od->is_gw_router) { if (dnat_force_snat_ip) { - if (od->dnat_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, "dnat"); } - if (od->dnat_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, + lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, "dnat"); } } if (lb_force_snat_ip) { - if (od->lb_force_snat_addrs.n_ipv4_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - od->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); } - if (od->lb_force_snat_addrs.n_ipv6_addrs) { + if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - od->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); } } } @@ -16198,6 +16035,7 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16263,14 +16101,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, + &lsi->match, &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lflows, lsi->meter_groups); + build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, + lsi->meter_groups); build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports, &lsi->match, + lsi->lr_ports,lsi->lr_nats, &lsi->match, &lsi->actions, lsi->meter_groups, lsi->features); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); @@ -16283,6 +16122,7 @@ static void build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, const struct hmap *ls_ports, const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, const struct shash *meter_groups, struct ds *match, struct ds *actions, @@ -16299,7 +16139,7 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); /* Build Logical Router Flows. */ build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); @@ -16317,6 +16157,11 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, struct lswitch_flow_build_info *lsi) { ovs_assert(op->nbrp); + + const struct lr_nat_record *lrnet_rec = lr_nat_table_find(lsi->lr_nats, + op->od->nbr); + ovs_assert(lrnet_rec); + build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16324,7 +16169,7 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); @@ -16332,9 +16177,9 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lsi->lflows, &lsi->match, + build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); } @@ -16394,6 +16239,7 @@ build_lflows_thread(void *arg) } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, lsi->lr_ports, + lsi->lr_nats, lsi->meter_groups, &lsi->match, &lsi->actions, @@ -16432,6 +16278,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, + lsi->lr_nats, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16502,6 +16349,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, + const struct lr_nat_table *lr_nats, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16531,6 +16379,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; + lsiv[index].lr_nats = lr_nats; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16565,6 +16414,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, + .lr_nats = lr_nats, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16592,6 +16442,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, + lsi.lr_nats, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16608,8 +16459,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.features, - lsi.svc_monitor_map, + lsi.lr_datapaths, lsi.lr_nats, + lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, @@ -16712,6 +16563,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, + input_data->lr_nats, lflows, &igmp_groups, input_data->meter_groups, @@ -17191,6 +17043,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); @@ -17228,6 +17081,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, + lflow_input->lr_nats, lflow_input->meter_groups, &match, &actions, lflows); diff --git a/northd/northd.h b/northd/northd.h index 3945e84bb8..8c9d0f308f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -152,6 +152,8 @@ struct lflow_data { void lflow_data_init(struct lflow_data *); void lflow_data_destroy(struct lflow_data *); +struct lr_nat_table; + struct lflow_input { /* Northbound table references */ const struct nbrec_bfd_table *nbrec_bfd_table; @@ -170,6 +172,7 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; + const struct lr_nat_table *lr_nats; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -306,24 +309,9 @@ struct ovn_datapath { struct ovn_port **l3dgw_ports; size_t n_l3dgw_ports; - /* NAT entries configured on the router. */ - struct ovn_nat *nat_entries; - size_t n_nat_entries; - - bool has_distributed_nat; /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Set of nat external ips on the router. */ - struct sset external_ips; - - /* SNAT IPs owned by the router (shash of 'struct ovn_snat_ip'). */ - struct shash snat_ips; - - struct lport_addresses dnat_force_snat_addrs; - struct lport_addresses lb_force_snat_addrs; - bool lb_force_snat_router_ip; - /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; @@ -340,6 +328,9 @@ struct ovn_datapath { struct hmap ports; }; +const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, + const struct uuid *uuid); + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -400,8 +391,9 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports); -bool sync_pbs_for_northd_changed_ovn_ports( struct tracked_ovn_ports *); + struct hmap *lr_ports, const struct lr_nat_table *); +bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *, + const struct lr_nat_table *); bool northd_has_tracked_data(struct northd_tracked_data *); bool northd_has_only_ports_in_tracked_data(struct northd_tracked_data *); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 55a244c8c4..b7f9cb5689 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -11018,6 +11018,7 @@ check ovn-nbctl lsp-add sw0 sw0p1 -- lsp-set-addresses sw0p1 "00:00:20:20:12:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11029,6 +11030,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # first it will be recompute to handle lr0-sw0 and then a compute # for the SB port binding change. check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11039,6 +11041,7 @@ ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11063,6 +11066,7 @@ ovn-nbctl --wait=hv lrp-set-gateway-chassis lr0-public hv1 20 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11073,6 +11077,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11081,6 +11086,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11089,6 +11095,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11097,6 +11104,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11105,6 +11113,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11113,6 +11122,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11122,6 +11132,7 @@ nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11137,6 +11148,7 @@ check ovn-nbctl lr-lb-add lr0 lb2 check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11144,6 +11156,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11151,6 +11164,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11158,6 +11172,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11166,6 +11181,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd recompute compute +check_engine_stats lr_nat recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11174,6 +11190,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11181,6 +11198,7 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute +check_engine_stats lr_nat recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Oct 24 00:44:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854079 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDthl6FVGz23jl for ; Tue, 24 Oct 2023 11:48:23 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1438383C96; Tue, 24 Oct 2023 00:48:21 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1438383C96 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PsEcLg4lqwP0; Tue, 24 Oct 2023 00:48:18 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0A02B8148A; Tue, 24 Oct 2023 00:48:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 0A02B8148A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D2823C0071; Tue, 24 Oct 2023 00:48:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5F9BBC0032 for ; Tue, 24 Oct 2023 00:48:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 39D6241971 for ; Tue, 24 Oct 2023 00:48:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 39D6241971 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q-EOZVCkKNps for ; Tue, 24 Oct 2023 00:48:12 +0000 (UTC) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [217.70.183.195]) by smtp2.osuosl.org (Postfix) with ESMTPS id 3AAB441903 for ; Tue, 24 Oct 2023 00:48:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 3AAB441903 Received: by mail.gandi.net (Postfix) with ESMTPSA id 5054D60002; Tue, 24 Oct 2023 00:48:07 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:44:34 -0400 Message-ID: <20231024004434.4133639-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 06/18] northd: Add a new engine 'lr-lb-nat-data' to manage routers' lb data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and NAT data of a logical router which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - lr-nat node A record for each logical router is maintained in the 'lr_lb_nats' hmap table and this record - stores the lb related data - embeds the 'lr-nat' record. This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-lb-nat-data.c | 635 ++++++++++++++++++++++++++++++++++++ northd/en-lr-lb-nat-data.h | 90 ++++++ northd/en-lr-nat.h | 3 + northd/en-sync-sb.c | 50 +-- northd/inc-proc-northd.c | 13 +- northd/northd.c | 644 ++++++++++++------------------------- northd/northd.h | 143 +++++++- tests/ovn-northd.at | 110 +++++-- 11 files changed, 1194 insertions(+), 501 deletions(-) create mode 100644 northd/en-lr-lb-nat-data.c create mode 100644 northd/en-lr-lb-nat-data.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 0a16da211e..7d85acdaea 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -33,5 +33,6 @@ #define PORT_GROUP_RUN_STOPWATCH_NAME "port_group_run" #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" +#define LR_LB_NAT_DATA_RUN_STOPWATCH_NAME "lr_lb_nat_data" #endif diff --git a/northd/automake.mk b/northd/automake.mk index ae367a2a8b..4116c487df 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -26,6 +26,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lb-data.h \ northd/en-lr-nat.c \ northd/en-lr-nat.h \ + northd/en-lr-lb-nat-data.c \ + northd/en-lr-lb-nat-data.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 22f398d419..9cb0ead3f0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -20,6 +20,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" +#include "en-lr-lb-nat-data.h" #include "en-northd.h" #include "en-meters.h" @@ -43,6 +44,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_nat_data *lr_nat_data = engine_get_input_data("lr_nat", node); + struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -66,6 +69,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_nats = &lr_nat_data->lr_nats; + lflow_input->lr_lbnats = &lr_lb_nat_data->lr_lbnats; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-lb-nat-data.c b/northd/en-lr-lb-nat-data.c new file mode 100644 index 0000000000..1a3271cc25 --- /dev/null +++ b/northd/en-lr-lb-nat-data.c @@ -0,0 +1,635 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-lr-lb-nat-data.h" +#include "en-lr-nat.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_lr_lb_nat_data); + +/* Static function declarations. */ +static void lr_lb_nat_data_table_init(struct lr_lb_nat_data_table *); +static void lr_lb_nat_data_table_clear(struct lr_lb_nat_data_table *); +static void lr_lb_nat_data_table_destroy(struct lr_lb_nat_data_table *); +static struct lr_lb_nat_data_record *lr_lb_nat_data_table_find_( + const struct lr_lb_nat_data_table *, const struct nbrec_logical_router *); +static void lr_lb_nat_data_table_build(struct lr_lb_nat_data_table *, + const struct lr_nat_table *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static struct lr_lb_nat_data_input lr_lb_nat_data_get_input_data( + struct engine_node *); + +static struct lr_lb_nat_data_record *lr_lb_nat_data_record_create( + struct lr_lb_nat_data_table *, const struct lr_nat_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); +static void lr_lb_nat_data_record_destroy(struct lr_lb_nat_data_record *); +static void lr_lb_nat_data_record_init( + struct lr_lb_nat_data_record *, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map); + +static void build_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *, + const struct ovn_northd_lb *); +static void add_neigh_ips_to_lrouter(struct lr_lb_nat_data_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *, + enum lb_neighbor_responder_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6); +static void lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *); + +const struct lr_lb_nat_data_record * +lr_lb_nat_data_table_find( + const struct lr_lb_nat_data_table *table, + const struct nbrec_logical_router *nbr) +{ + return lr_lb_nat_data_table_find_(table, nbr); +} + +/* 'lr_lb_nat_data' engine node manages the NB logical router LB data. + */ +void * +en_lr_lb_nat_data_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_lr_lb_nat_data *data = xzalloc(sizeof *data); + lr_lb_nat_data_table_init(&data->lr_lbnats); + hmapx_init(&data->tracked_data.crupdated); + hmapx_init(&data->tracked_data.deleted); + return data; +} + +void +en_lr_lb_nat_data_cleanup(void *data_) +{ + struct ed_type_lr_lb_nat_data *data = + (struct ed_type_lr_lb_nat_data *) data_; + lr_lb_nat_data_table_destroy(&data->lr_lbnats); + hmapx_destroy(&data->tracked_data.crupdated); + hmapx_destroy(&data->tracked_data.deleted); +} + +void +en_lr_lb_nat_data_clear_tracked_data(void *data_) +{ + struct ed_type_lr_lb_nat_data *data = + (struct ed_type_lr_lb_nat_data *) data_; + + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &data->tracked_data.deleted) { + lr_lb_nat_data_record_destroy(hmapx_node->data); + hmapx_delete(&data->tracked_data.deleted, hmapx_node); + } + + hmapx_clear(&data->tracked_data.crupdated); + data->tracked = false; +} + +void +en_lr_lb_nat_data_run(struct engine_node *node, void *data_) +{ + struct lr_lb_nat_data_input input_data = + lr_lb_nat_data_get_input_data(node); + struct ed_type_lr_lb_nat_data *data = data_; + + stopwatch_start(LR_LB_NAT_DATA_RUN_STOPWATCH_NAME, time_msec()); + + lr_lb_nat_data_table_clear(&data->lr_lbnats); + lr_lb_nat_data_table_build(&data->lr_lbnats, input_data.lr_nats, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + stopwatch_stop(LR_LB_NAT_DATA_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +bool +lr_lb_nat_data_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_data->change_tracked) { + return false; + } + + return true; +} + +bool +lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lb_data *lb_data = engine_get_input_data("lb_data", node); + if (!lb_data->tracked) { + return false; + } + + struct ed_type_lr_lb_nat_data *data = + (struct ed_type_lr_lb_nat_data *) data_; + struct lr_lb_nat_data_input input_data = + lr_lb_nat_data_get_input_data(node); + struct lr_lb_nat_data_record *lr_lbnat_rec; + size_t index; + + const struct tracked_lb_data *trk_lb_data = &lb_data->tracked_lb_data; + const struct ovn_lb_group_datapaths *lbgrp_dps; + const struct crupdated_lbgrp *crupdated_lbgrp; + const struct crupdated_od_lb_data *codlb; + const struct ovn_lb_datapaths *lb_dps; + const struct crupdated_lb *clb; + const struct ovn_northd_lb *lb; + const struct ovn_datapath *od; + + LIST_FOR_EACH (codlb, list_node, &trk_lb_data->crupdated_lr_lbs) { + od = ovn_datapath_find(&input_data.lr_datapaths->datapaths, + &codlb->od_uuid); + ovs_assert(od); + + lr_lbnat_rec = lr_lb_nat_data_table_find_(&data->lr_lbnats, od->nbr); + if (!lr_lbnat_rec) { + const struct lr_nat_record *lrnat_rec = lr_nat_table_find( + input_data.lr_nats, od->nbr); + ovs_assert(lrnat_rec); + + lr_lbnat_rec = lr_lb_nat_data_record_create(&data->lr_lbnats, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + + /* Add the lr_lbnat_rec rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + continue; + } + + struct uuidset_node *uuidnode; + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbs) { + lb_dps = ovn_lb_datapaths_find( + input_data.lb_datapaths_map, &uuidnode->uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_lbnat_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_lbnat_rec, lb_dps->lb); + } + + UUIDSET_FOR_EACH (uuidnode, &codlb->assoc_lbgrps) { + lbgrp_dps = ovn_lb_group_datapaths_find( + input_data.lbgrp_datapaths_map, &uuidnode->uuid); + ovs_assert(lbgrp_dps); + + for (size_t j = 0; j < lbgrp_dps->lb_group->n_lbs; j++) { + const struct uuid *lb_uuid + = &lbgrp_dps->lb_group->lbs[j]->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, + lb_uuid); + ovs_assert(lb_dps); + + /* Add the lb_ips of lb_dps to the od. */ + build_lrouter_lb_ips(lr_lbnat_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_lbnat_rec, lb_dps->lb); + } + } + + /* Add the lr_lbnat_rec rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + } + + HMAP_FOR_EACH (clb, hmap_node, &trk_lb_data->crupdated_lbs) { + lb = clb->lb; + const struct uuid *lb_uuid = &lb->nlb->header_.uuid; + + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + + BITMAP_FOR_EACH_1 (index, ods_size(input_data.lr_datapaths), + lb_dps->nb_lr_map) { + od = input_data.lr_datapaths->array[index]; + + lr_lbnat_rec = lr_lb_nat_data_table_find_(&data->lr_lbnats, + od->nbr); + ovs_assert(lr_lbnat_rec); + + /* Update the od->lb_ips with the deleted and inserted + * vips (if any). */ + remove_ips_from_lb_ip_set(lr_lbnat_rec->lb_ips, lb->routable, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_ips_to_lb_ip_set(lr_lbnat_rec->lb_ips, lb->routable, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + remove_lrouter_lb_reachable_ips(lr_lbnat_rec, lb->neigh_mode, + &clb->deleted_vips_v4, + &clb->deleted_vips_v6); + add_neigh_ips_to_lrouter(lr_lbnat_rec, lb->neigh_mode, + &clb->inserted_vips_v4, + &clb->inserted_vips_v6); + + /* Add the lr_lbnat_rec rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + } + } + + HMAP_FOR_EACH (crupdated_lbgrp, hmap_node, + &trk_lb_data->crupdated_lbgrps) { + const struct uuid *lb_uuid = &crupdated_lbgrp->lbgrp->uuid; + + lbgrp_dps = ovn_lb_group_datapaths_find(input_data.lbgrp_datapaths_map, + lb_uuid); + ovs_assert(lbgrp_dps); + + struct hmapx_node *hnode; + HMAPX_FOR_EACH (hnode, &crupdated_lbgrp->assoc_lbs) { + lb = hnode->data; + lb_uuid = &lb->nlb->header_.uuid; + lb_dps = ovn_lb_datapaths_find(input_data.lb_datapaths_map, + lb_uuid); + ovs_assert(lb_dps); + for (size_t i = 0; i < lbgrp_dps->n_lr; i++) { + od = lbgrp_dps->lr[i]; + lr_lbnat_rec = lr_lb_nat_data_table_find_(&data->lr_lbnats, + od->nbr); + ovs_assert(lr_lbnat_rec); + /* Add the lb_ips of lb_dps to the lr lb data. */ + build_lrouter_lb_ips(lr_lbnat_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_lbnat_rec, lb_dps->lb); + + /* Add the lr_lbnat_rec rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + } + } + } + + if (!hmapx_is_empty(&data->tracked_data.crupdated)) { + struct hmapx_node *hmapx_node; + /* For all the modified lr_lb_nat_data records (re)build the + * vip nats. */ + HMAPX_FOR_EACH (hmapx_node, &data->tracked_data.crupdated) { + lr_lb_nat_data_build_vip_nats(hmapx_node->data); + } + + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +lr_lb_nat_data_lr_nat_handler(struct engine_node *node, void *data_) +{ + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + if (!lr_nat_data->tracked + || !hmapx_is_empty(&lr_nat_data->tracked_data.deleted)) { + return false; + } + + struct ed_type_lr_lb_nat_data *data = + (struct ed_type_lr_lb_nat_data *) data_; + struct lr_lb_nat_data_input input_data = + lr_lb_nat_data_get_input_data(node); + const struct lr_nat_record *lrnat_rec; + struct lr_lb_nat_data_record *lr_lbnat_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &lr_nat_data->tracked_data.crupdated) { + lrnat_rec = hmapx_node->data; + lr_lbnat_rec = lr_lb_nat_data_table_find_(&data->lr_lbnats, + lrnat_rec->od->nbr); + if (!lr_lbnat_rec) { + lr_lbnat_rec = lr_lb_nat_data_record_create(&data->lr_lbnats, + lrnat_rec, + input_data.lb_datapaths_map, + input_data.lbgrp_datapaths_map); + } else { + lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + } + + /* Add the lr_lbnat_rec rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + } + + if (!hmapx_is_empty(&data->tracked_data.crupdated)) { + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* static functions. */ +static void +lr_lb_nat_data_table_init(struct lr_lb_nat_data_table *table) +{ + *table = (struct lr_lb_nat_data_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +lr_lb_nat_data_table_destroy(struct lr_lb_nat_data_table *table) +{ + lr_lb_nat_data_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +lr_lb_nat_data_table_clear(struct lr_lb_nat_data_table *table) +{ + struct lr_lb_nat_data_record *lr_lbnat_rec; + HMAP_FOR_EACH_POP (lr_lbnat_rec, key_node, &table->entries) { + lr_lb_nat_data_record_destroy(lr_lbnat_rec); + } +} + +static void +lr_lb_nat_data_table_build(struct lr_lb_nat_data_table *table, + const struct lr_nat_table *lr_nats, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + const struct lr_nat_record *lrnat_rec; + LR_NAT_TABLE_FOR_EACH (lrnat_rec, lr_nats) { + lr_lb_nat_data_record_create(table, lrnat_rec, lb_datapaths_map, + lbgrp_datapaths_map); + } +} + +static struct lr_lb_nat_data_record * +lr_lb_nat_data_table_find_(const struct lr_lb_nat_data_table *table, + const struct nbrec_logical_router *nbr) +{ + struct lr_lb_nat_data_record *lr_lbnat_rec; + + HMAP_FOR_EACH_WITH_HASH (lr_lbnat_rec, key_node, + uuid_hash(&nbr->header_.uuid), &table->entries) { + if (nbr == lr_lbnat_rec->od->nbr) { + return lr_lbnat_rec; + } + } + return NULL; +} + +static struct lr_lb_nat_data_record * +lr_lb_nat_data_record_create(struct lr_lb_nat_data_table *table, + const struct lr_nat_record *lrnat_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + struct lr_lb_nat_data_record *lr_lbnat_rec = xzalloc(sizeof *lr_lbnat_rec); + lr_lbnat_rec->lrnat_rec = lrnat_rec; + lr_lbnat_rec->od = lrnat_rec->od; + lr_lb_nat_data_record_init(lr_lbnat_rec, lb_datapaths_map, + lbgrp_datapaths_map); + + hmap_insert(&table->entries, &lr_lbnat_rec->key_node, + uuid_hash(&lr_lbnat_rec->od->nbr->header_.uuid)); + + return lr_lbnat_rec; +} + +static void +lr_lb_nat_data_record_destroy(struct lr_lb_nat_data_record *lr_lbnat_rec) +{ + ovn_lb_ip_set_destroy(lr_lbnat_rec->lb_ips); + lr_lbnat_rec->lb_ips = NULL; + sset_destroy(&lr_lbnat_rec->vip_nats); + free(lr_lbnat_rec); +} + +static void +lr_lb_nat_data_record_init(struct lr_lb_nat_data_record *lr_lbnat_rec, + const struct hmap *lb_datapaths_map, + const struct hmap *lbgrp_datapaths_map) +{ + const struct nbrec_load_balancer_group *nbrec_lb_group; + const struct ovn_lb_group_datapaths *lb_group_dps; + const struct ovn_lb_datapaths *lb_dps; + + /* Checking load balancer groups first, starting from the largest one, + * to more efficiently copy IP sets. */ + size_t largest_group = 0; + + const struct nbrec_logical_router *nbr = lr_lbnat_rec->od->nbr; + for (size_t i = 1; i < nbr->n_load_balancer_group; i++) { + if (nbr->load_balancer_group[i]->n_load_balancer > + nbr->load_balancer_group[largest_group]->n_load_balancer) { + largest_group = i; + } + } + + for (size_t i = 0; i < nbr->n_load_balancer_group; i++) { + size_t idx = (i + largest_group) % nbr->n_load_balancer_group; + + nbrec_lb_group = nbr->load_balancer_group[idx]; + const struct uuid *lbgrp_uuid = &nbrec_lb_group->header_.uuid; + + lb_group_dps = + ovn_lb_group_datapaths_find(lbgrp_datapaths_map, + lbgrp_uuid); + ovs_assert(lb_group_dps); + + if (!lr_lbnat_rec->lb_ips) { + lr_lbnat_rec->lb_ips = + ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); + } else { + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_ips(lr_lbnat_rec->lb_ips, + lb_group_dps->lb_group->lbs[j]); + } + } + + for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { + build_lrouter_lb_reachable_ips(lr_lbnat_rec, + lb_group_dps->lb_group->lbs[j]); + } + } + + if (!lr_lbnat_rec->lb_ips) { + lr_lbnat_rec->lb_ips = ovn_lb_ip_set_create(); + } + + for (size_t i = 0; i < nbr->n_load_balancer; i++) { + const struct uuid *lb_uuid = + &nbr->load_balancer[i]->header_.uuid; + lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); + ovs_assert(lb_dps); + build_lrouter_lb_ips(lr_lbnat_rec->lb_ips, lb_dps->lb); + build_lrouter_lb_reachable_ips(lr_lbnat_rec, lb_dps->lb); + } + + sset_init(&lr_lbnat_rec->vip_nats); + + if (!nbr->n_nat) { + lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + } +} + +static struct lr_lb_nat_data_input +lr_lb_nat_data_get_input_data(struct engine_node *node) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_lr_nat_data *lr_nat_data = + engine_get_input_data("lr_nat", node); + + return (struct lr_lb_nat_data_input) { + .lr_datapaths = &northd_data->lr_datapaths, + .lb_datapaths_map = &northd_data->lb_datapaths_map, + .lbgrp_datapaths_map = &northd_data->lb_group_datapaths_map, + .lr_nats = &lr_nat_data->lr_nats, + }; +} + +static void +build_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *lr_lbnat_rec, + const struct ovn_northd_lb *lb) +{ + add_neigh_ips_to_lrouter(lr_lbnat_rec, lb->neigh_mode, &lb->ips_v4, + &lb->ips_v6); +} + +static void +add_neigh_ips_to_lrouter(struct lr_lb_nat_data_record *lr_lbnat_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + /* If configured to not reply to any neighbor requests for all VIPs + * return early. + */ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + + /* If configured to reply to neighbor requests for all VIPs force them + * all to be considered "reachable". + */ + if (neigh_mode == LB_NEIGH_RESPOND_ALL) { + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_add(&lr_lbnat_rec->lb_ips->ips_v4_reachable, ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_add(&lr_lbnat_rec->lb_ips->ips_v6_reachable, ip_address); + } + + return; + } + + /* Otherwise, a VIP is reachable if there's at least one router + * subnet that includes it. + */ + ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); + + SSET_FOR_EACH (ip_address, lb_ips_v4) { + struct ovn_port *op; + ovs_be32 vip_ip4; + if (ip_parse(ip_address, &vip_ip4)) { + HMAP_FOR_EACH (op, dp_node, &lr_lbnat_rec->od->ports) { + if (lrouter_port_ipv4_reachable(op, vip_ip4)) { + sset_add(&lr_lbnat_rec->lb_ips->ips_v4_reachable, + ip_address); + break; + } + } + } + } + + SSET_FOR_EACH (ip_address, lb_ips_v6) { + struct ovn_port *op; + struct in6_addr vip; + if (ipv6_parse(ip_address, &vip)) { + HMAP_FOR_EACH (op, dp_node, &lr_lbnat_rec->od->ports) { + if (lrouter_port_ipv6_reachable(op, &vip)) { + sset_add(&lr_lbnat_rec->lb_ips->ips_v6_reachable, + ip_address); + break; + } + } + } + } +} + +static void +remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *lr_lbnat_rec, + enum lb_neighbor_responder_mode neigh_mode, + const struct sset *lb_ips_v4, + const struct sset *lb_ips_v6) +{ + if (neigh_mode == LB_NEIGH_RESPOND_NONE) { + return; + } + + const char *ip_address; + SSET_FOR_EACH (ip_address, lb_ips_v4) { + sset_find_and_delete(&lr_lbnat_rec->lb_ips->ips_v4_reachable, + ip_address); + } + SSET_FOR_EACH (ip_address, lb_ips_v6) { + sset_find_and_delete(&lr_lbnat_rec->lb_ips->ips_v6_reachable, + ip_address); + } +} + +static void +lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *lr_lbnat_rec) +{ + sset_clear(&lr_lbnat_rec->vip_nats); + const char *external_ip; + SSET_FOR_EACH (external_ip, &lr_lbnat_rec->lrnat_rec->external_ips) { + bool is_vip_nat = false; + if (addr_is_ipv6(external_ip)) { + is_vip_nat = sset_contains(&lr_lbnat_rec->lb_ips->ips_v6, + external_ip); + } else { + is_vip_nat = sset_contains(&lr_lbnat_rec->lb_ips->ips_v4, + external_ip); + } + + if (is_vip_nat) { + sset_add(&lr_lbnat_rec->vip_nats, external_ip); + } + } +} diff --git a/northd/en-lr-lb-nat-data.h b/northd/en-lr-lb-nat-data.h new file mode 100644 index 0000000000..30e76011a8 --- /dev/null +++ b/northd/en-lr-lb-nat-data.h @@ -0,0 +1,90 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LR_LB_NAT_DATA_H +#define EN_LR_LB_NAT_DATA_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" + +struct ovn_datapath; +struct lr_nat_record; + +struct lr_lb_nat_data_record { + struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ + + const struct ovn_datapath *od; + const struct lr_nat_record *lrnat_rec; + + /* Load Balancer vIPs relevant for this datapath. */ + struct ovn_lb_ip_set *lb_ips; + + /* sset of vips which are also part of lr nats. */ + struct sset vip_nats; +}; + +struct lr_lb_nat_data_table { + struct hmap entries; +}; + +#define LR_LB_NAT_DATA_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) + +struct lr_lb_nat_data_tracked_data { + /* Created or updated logical router with LB data. */ + struct hmapx crupdated; /* Stores 'struct lr_lb_nat_data_record'. */ + + /* Deleted logical router with LB data. */ + struct hmapx deleted; /* Stores 'struct lr_lb_nat_data_record'. */ +}; + +struct ed_type_lr_lb_nat_data { + struct lr_lb_nat_data_table lr_lbnats; + + bool tracked; + struct lr_lb_nat_data_tracked_data tracked_data; +}; + +struct lr_lb_nat_data_input { + const struct ovn_datapaths *lr_datapaths; + const struct hmap *lb_datapaths_map; + const struct hmap *lbgrp_datapaths_map; + const struct lr_nat_table *lr_nats; +}; + +void *en_lr_lb_nat_data_init(struct engine_node *, struct engine_arg *); +void en_lr_lb_nat_data_cleanup(void *data); +void en_lr_lb_nat_data_clear_tracked_data(void *data); +void en_lr_lb_nat_data_run(struct engine_node *, void *data); + +bool lr_lb_nat_data_northd_handler(struct engine_node *, void *data); +bool lr_lb_nat_data_lr_nat_handler(struct engine_node *, void *data); +bool lr_lb_nat_data_lb_data_handler(struct engine_node *, void *data); + +const struct lr_lb_nat_data_record *lr_lb_nat_data_table_find( + const struct lr_lb_nat_data_table *, const struct nbrec_logical_router *); + +#endif /* EN_LR_LB_NAT_DATA_H */ \ No newline at end of file diff --git a/northd/en-lr-nat.h b/northd/en-lr-nat.h index 6ed1f5ed52..f264c0cb3a 100644 --- a/northd/en-lr-nat.h +++ b/northd/en-lr-nat.h @@ -83,6 +83,9 @@ struct lr_nat_table { struct hmap entries; /* Stores struct lr_nat_record. */ }; +#define LR_NAT_TABLE_FOR_EACH(LR_NAT_REC, TABLE) \ + HMAP_FOR_EACH (LR_NAT_REC, key_node, &(TABLE)->entries) + const struct lr_nat_record * lr_nat_table_find(const struct lr_nat_table *, const struct nbrec_logical_router *); diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 10ade620e7..7c22949f74 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -22,6 +22,7 @@ #include "openvswitch/util.h" #include "en-lr-nat.h" +#include "en-lr-lb-nat-data.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -41,7 +42,7 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct ovn_datapaths *lr_datapaths); + const struct lr_lb_nat_data_table *); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -87,11 +88,11 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) EN_OVSDB_GET(engine_get_input("SB_address_set", node)); const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); - + const struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &northd_data->lr_datapaths); + &lr_lb_nat_data->lr_lbnats); engine_set_node_state(node, EN_UPDATED); } @@ -288,10 +289,12 @@ en_sync_to_sb_pb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); + struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); + sync_pbs(eng_ctx->ovnsb_idl_txn, &northd_data->ls_ports, - &northd_data->lr_ports, &lr_nat_data->lr_nats); + &northd_data->lr_ports, + &lr_lb_nat_data->lr_lbnats); engine_set_node_state(node, EN_UPDATED); } @@ -316,11 +319,12 @@ sync_to_sb_pb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) return false; } - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); + struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); if (!sync_pbs_for_northd_changed_ovn_ports( - &nd->trk_northd_changes.trk_ovn_ports, &lr_nat_data->lr_nats)) { + &nd->trk_northd_changes.trk_ovn_ports, + &lr_lb_nat_data->lr_lbnats)) { return false; } @@ -366,7 +370,7 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct ovn_datapaths *lr_datapaths) + const struct lr_lb_nat_data_table *lr_lbnats) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -410,16 +414,14 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync router load balancer VIP generated address sets. */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (sset_count(&od->lb_ips->ips_v4_reachable)) { - char *ipv4_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET); + const struct lr_lb_nat_data_record *lrlb_rec; + LR_LB_NAT_DATA_TABLE_FOR_EACH (lrlb_rec, lr_lbnats) { + if (sset_count(&lrlb_rec->lb_ips->ips_v4_reachable)) { + char *ipv4_addrs_name = + lr_lb_address_set_name(lrlb_rec->od->tunnel_key, AF_INET); struct sorted_array ipv4_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v4_reachable); + sorted_array_from_sset(&lrlb_rec->lb_ips->ips_v4_reachable); sync_addr_set(ovnsb_txn, ipv4_addrs_name, &ipv4_addrs_sorted, &sb_address_sets); @@ -427,11 +429,11 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, free(ipv4_addrs_name); } - if (sset_count(&od->lb_ips->ips_v6_reachable)) { - char *ipv6_addrs_name = lr_lb_address_set_name(od->tunnel_key, - AF_INET6); - struct sorted_array ipv6_addrs_sorted = - sorted_array_from_sset(&od->lb_ips->ips_v6_reachable); + if (sset_count(&lrlb_rec->lb_ips->ips_v6_reachable)) { + char *ipv6_addrs_name = + lr_lb_address_set_name(lrlb_rec->od->tunnel_key, AF_INET6); + struct sorted_array ipv6_addrs_sorted = sorted_array_from_sset( + &lrlb_rec->lb_ips->ips_v6_reachable); sync_addr_set(ovnsb_txn, ipv6_addrs_name, &ipv6_addrs_sorted, &sb_address_sets); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 2bd66b8808..369a151fa3 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -31,6 +31,7 @@ #include "openvswitch/vlog.h" #include "inc-proc-northd.h" #include "en-lb-data.h" +#include "en-lr-lb-nat-data.h" #include "en-lr-nat.h" #include "en-northd.h" #include "en-lflow.h" @@ -148,6 +149,7 @@ static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_lb_nat_data, "lr_lb_nat_data"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -196,6 +198,13 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_nat, &en_nb_logical_router, lr_nat_logical_router_handler); + engine_add_input(&en_lr_lb_nat_data, &en_northd, + lr_lb_nat_data_northd_handler); + engine_add_input(&en_lr_lb_nat_data, &en_lr_nat, + lr_lb_nat_data_lr_nat_handler); + engine_add_input(&en_lr_lb_nat_data, &en_lb_data, + lr_lb_nat_data_lb_data_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -220,12 +229,14 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_nat, NULL); + engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_lb_nat_data, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_port_group, &en_nb_port_group, @@ -243,7 +254,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); - engine_add_input(&en_sync_to_sb_pb, &en_lr_nat, NULL); + engine_add_input(&en_sync_to_sb_pb, &en_lr_lb_nat_data, NULL); /* en_sync_to_sb engine node syncs the SB database tables from * the NB database tables. diff --git a/northd/northd.c b/northd/northd.c index 674c461561..30e9278961 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -44,6 +44,7 @@ #include "northd.h" #include "en-lb-data.h" #include "en-lr-nat.h" +#include "en-lr-lb-nat-data.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -617,13 +618,6 @@ init_lb_for_datapath(struct ovn_datapath *od) } } -static void -destroy_lb_for_datapath(struct ovn_datapath *od) -{ - ovn_lb_ip_set_destroy(od->lb_ips); - od->lb_ips = NULL; -} - /* A group of logical router datapaths which are connected - either * directly or indirectly. * Each logical router can belong to only one group. */ @@ -676,7 +670,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) destroy_ipam_info(&od->ipam_info); free(od->router_ports); free(od->ls_peers); - destroy_lb_for_datapath(od); free(od->localnet_ports); free(od->l3dgw_ports); destroy_mcast_info_for_datapath(od); @@ -1179,12 +1172,6 @@ ovn_datapath_assign_requested_tnl_id( } } -static inline size_t -ods_size(const struct ovn_datapaths *datapaths) -{ - return hmap_count(&datapaths->datapaths); -} - static void ods_build_array_index(struct ovn_datapaths *datapaths) { @@ -1317,121 +1304,6 @@ struct lflow_ref_node { struct ovn_lflow *lflow; }; -/* A logical switch port or logical router port. - * - * In steady state, an ovn_port points to a northbound Logical_Switch_Port - * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a - * southbound Port_Binding record (via 'sb'). As the state of the system - * changes, join_logical_ports() may determine that there is a new LSP or LRP - * that has no corresponding Port_Binding record (in which case build_ports()) - * will create the missing Port_Binding) or that a Port_Binding record exists - * that has no coresponding LSP (in which case build_ports() will delete the - * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port - * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. - * - * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but - * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). - */ -struct ovn_port { - /* Port name aka key. - * - * This is ordinarily the same as nbsp->name or nbrp->name and - * sb->logical_port. (A distributed gateway port creates a "derived" - * ovn_port with key "cr-%s" % nbrp->name.) */ - struct hmap_node key_node; /* Index on 'key'. */ - char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ - char *json_key; /* 'key', quoted for use in JSON. */ - - const struct sbrec_port_binding *sb; /* May be NULL. */ - - uint32_t tunnel_key; - - /* Logical switch port data. */ - const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ - - struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ - unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ - unsigned int n_lsp_non_router_addrs; /* Number of elements from the - * beginning of 'lsp_addrs' extracted - * directly from LSP 'addresses'. */ - - struct lport_addresses *ps_addrs; /* Port security addresses. */ - unsigned int n_ps_addrs; - - bool lsp_can_be_inc_processed; /* If it can be incrementally processed when - the port changes. */ - - /* Logical router port data. */ - const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ - - struct lport_addresses lrp_networks; - - struct ovn_port_routable_addresses routables; - - /* Logical port multicast data. */ - struct mcast_port_info mcast_info; - - /* At most one of l3dgw_port and cr_port can be not NULL. */ - - /* This is set to a distributed gateway port if and only if this ovn_port - * is "derived" from it. Otherwise this is set to NULL. The derived - * ovn_port represents the instance of distributed gateway port on the - * gateway chassis.*/ - struct ovn_port *l3dgw_port; - - /* This is set to the "derived" chassis-redirect port of this port if and - * only if this port is a distributed gateway port. Otherwise this is set - * to NULL. */ - struct ovn_port *cr_port; - - bool has_unknown; /* If the addresses have 'unknown' defined. */ - - bool has_bfd; - - /* The port's peer: - * - * - A switch port S of type "router" has a router port R as a peer, - * and R in turn has S has its peer. - * - * - Two connected logical router ports have each other as peer. - * - * - Other kinds of ports have no peer. */ - struct ovn_port *peer; - - struct ovn_datapath *od; - - struct ovs_list list; /* In list of similar records. */ - - struct hmap_node dp_node; /* Node in od->ports. */ - - struct lport_addresses proxy_arp_addrs; - - /* Temporarily used for traversing a list (or hmap) of ports. */ - bool visited; - - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. - * - * This data is initialized and destroyed by the en_northd node, but - * populated and used only by the en_lflow node. Ideally this data should - * be maintained as part of en_lflow's data (struct lflow_data): a hash - * index from ovn_port key to lflows. However, it would be less efficient - * and more complex: - * - * 1. It would require an extra search (using the index) to find the - * lflows. - * - * 2. Building the index needs to be thread-safe, using either a global - * lock which is obviously less efficient, or hash-based lock array which - * is more complex. - * - * Adding the list here is more straightforward. The drawback is that we - * need to keep in mind that this data belongs to en_lflow node, so never - * access it from any other nodes. - */ - struct ovs_list lflows; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1456,16 +1328,21 @@ destroy_routable_addresses(struct ovn_port_routable_addresses *ra) } static char **get_nat_addresses(const struct ovn_port *op, size_t *n, - bool routable_only, bool include_lb_ips); + bool routable_only, bool include_lb_ips, + const struct lr_lb_nat_data_record *); -static void -assign_routable_addresses(struct ovn_port *op) +static struct ovn_port_routable_addresses +get_op_routable_addresses(struct ovn_port *op, + const struct lr_lb_nat_data_record *lr_lbnat_rec) { size_t n; - char **nats = get_nat_addresses(op, &n, true, true); + char **nats = get_nat_addresses(op, &n, true, true, lr_lbnat_rec); if (!nats) { - return; + return (struct ovn_port_routable_addresses) { + .laddrs = NULL, + .n_addrs = 0, + }; } struct lport_addresses *laddrs = xcalloc(n, sizeof(*laddrs)); @@ -1481,9 +1358,15 @@ assign_routable_addresses(struct ovn_port *op) } free(nats); - /* Everything seems to have worked out */ - op->routables.laddrs = laddrs; - op->routables.n_addrs = n_addrs; + if (!n_addrs) { + free(laddrs); + laddrs = NULL; + } + + return (struct ovn_port_routable_addresses) { + .laddrs = laddrs, + .n_addrs = n_addrs, + }; } @@ -1543,8 +1426,6 @@ ovn_port_destroy_orphan(struct ovn_port *port) } free(port->ps_addrs); - destroy_routable_addresses(&port->routables); - destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); free(port->json_key); @@ -2586,9 +2467,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, sizeof *od->l3dgw_ports); } od->l3dgw_ports[od->n_l3dgw_ports++] = op; - - assign_routable_addresses(op); - } + } } } @@ -2685,7 +2564,8 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, * and must free the returned array when it is no longer needed. */ static char ** get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, - bool include_lb_ips) + bool include_lb_ips, + const struct lr_lb_nat_data_record *lr_lbnat_rec) { size_t n_nats = 0; struct eth_addr mac; @@ -2770,23 +2650,25 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, } } - if (include_lb_ips) { + if (include_lb_ips && lr_lbnat_rec) { const char *ip_address; if (routable_only) { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4_routable) { + SSET_FOR_EACH (ip_address, + &lr_lbnat_rec->lb_ips->ips_v4_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6_routable) { + SSET_FOR_EACH (ip_address, + &lr_lbnat_rec->lb_ips->ips_v6_routable) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } } else { - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v4) { + SSET_FOR_EACH (ip_address, &lr_lbnat_rec->lb_ips->ips_v4) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } - SSET_FOR_EACH (ip_address, &op->od->lb_ips->ips_v6) { + SSET_FOR_EACH (ip_address, &lr_lbnat_rec->lb_ips->ips_v6) { ds_put_format(&c_addresses, " %s", ip_address); central_ip_address = true; } @@ -3857,21 +3739,8 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - /* Checking load balancer groups first, starting from the largest one, - * to more efficiently copy IP sets. */ - size_t largest_group = 0; - - for (size_t i = 1; i < od->nbr->n_load_balancer_group; i++) { - if (od->nbr->load_balancer_group[i]->n_load_balancer > - od->nbr->load_balancer_group[largest_group]->n_load_balancer) { - largest_group = i; - } - } - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - size_t idx = (i + largest_group) % od->nbr->n_load_balancer_group; - - nbrec_lb_group = od->nbr->load_balancer_group[idx]; + nbrec_lb_group = od->nbr->load_balancer_group[i]; const struct uuid *lb_group_uuid = &nbrec_lb_group->header_.uuid; lb_group_dps = @@ -3879,20 +3748,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_group_uuid); ovs_assert(lb_group_dps); ovn_lb_group_datapaths_add_lr(lb_group_dps, od); - - if (!od->lb_ips) { - od->lb_ips = - ovn_lb_ip_set_clone(lb_group_dps->lb_group->lb_ips); - } else { - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_ips(od->lb_ips, - lb_group_dps->lb_group->lbs[j]); - } - } - } - - if (!od->lb_ips) { - od->lb_ips = ovn_lb_ip_set_create(); } for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { @@ -3901,7 +3756,6 @@ build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, lb_dps = ovn_lb_datapaths_find(lb_datapaths_map, lb_uuid); ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); } } @@ -3955,102 +3809,6 @@ build_lb_svcs( } } -static bool lrouter_port_ipv4_reachable(const struct ovn_port *op, - ovs_be32 addr); -static bool lrouter_port_ipv6_reachable(const struct ovn_port *op, - const struct in6_addr *addr); - -static void -add_neigh_ips_to_lrouter(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - /* If configured to not reply to any neighbor requests for all VIPs - * return early. - */ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - - /* If configured to reply to neighbor requests for all VIPs force them - * all to be considered "reachable". - */ - if (neigh_mode == LB_NEIGH_RESPOND_ALL) { - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_add(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_add(&od->lb_ips->ips_v6_reachable, ip_address); - } - - return; - } - - /* Otherwise, a VIP is reachable if there's at least one router - * subnet that includes it. - */ - ovs_assert(neigh_mode == LB_NEIGH_RESPOND_REACHABLE); - - SSET_FOR_EACH (ip_address, lb_ips_v4) { - struct ovn_port *op; - ovs_be32 vip_ip4; - if (ip_parse(ip_address, &vip_ip4)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv4_reachable(op, vip_ip4)) { - sset_add(&od->lb_ips->ips_v4_reachable, - ip_address); - break; - } - } - } - } - - SSET_FOR_EACH (ip_address, lb_ips_v6) { - struct ovn_port *op; - struct in6_addr vip; - if (ipv6_parse(ip_address, &vip)) { - HMAP_FOR_EACH (op, dp_node, &od->ports) { - if (lrouter_port_ipv6_reachable(op, &vip)) { - sset_add(&od->lb_ips->ips_v6_reachable, - ip_address); - break; - } - } - } - } -} - -static void -remove_lrouter_lb_reachable_ips(struct ovn_datapath *od, - enum lb_neighbor_responder_mode neigh_mode, - const struct sset *lb_ips_v4, - const struct sset *lb_ips_v6) -{ - if (neigh_mode == LB_NEIGH_RESPOND_NONE) { - return; - } - - const char *ip_address; - SSET_FOR_EACH (ip_address, lb_ips_v4) { - sset_find_and_delete(&od->lb_ips->ips_v4_reachable, ip_address); - } - SSET_FOR_EACH (ip_address, lb_ips_v6) { - sset_find_and_delete(&od->lb_ips->ips_v6_reachable, ip_address); - } -} - -static void -build_lrouter_lb_reachable_ips(struct ovn_datapath *od, - const struct ovn_northd_lb *lb) -{ - add_neigh_ips_to_lrouter(od, lb->neigh_mode, &lb->ips_v4, - &lb->ips_v6); -} - - static void build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) { @@ -4072,43 +3830,6 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) } } -static void -build_lrouter_lbs_reachable_ips(struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct hmap *lb_group_dps_map) -{ - struct ovn_datapath *od; - - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - if (!od->nbr) { - continue; - } - - for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { - struct ovn_lb_datapaths *lb_dps = - ovn_lb_datapaths_find(lb_dps_map, - &od->nbr->load_balancer[i]->header_.uuid); - ovs_assert(lb_dps); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - } - - for (size_t i = 0; i < od->nbr->n_load_balancer_group; i++) { - const struct nbrec_load_balancer_group *nbrec_lb_group = - od->nbr->load_balancer_group[i]; - struct ovn_lb_group_datapaths *lb_group_dps; - - lb_group_dps = - ovn_lb_group_datapaths_find(lb_group_dps_map, - &nbrec_lb_group->header_.uuid); - ovs_assert(lb_group_dps); - for (size_t j = 0; j < lb_group_dps->lb_group->n_lbs; j++) { - build_lrouter_lb_reachable_ips(od, - lb_group_dps->lb_group->lbs[j]); - } - } - } -} - static void build_lswitch_lbs_from_lrouter(struct ovn_datapaths *lr_datapaths, struct hmap *lb_dps_map, @@ -4172,8 +3893,6 @@ build_lb_port_related_data( struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lrouter_lbs_reachable_ips(lr_datapaths, lb_dps_map, - lb_group_dps_map); build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); @@ -4539,7 +4258,8 @@ check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ static void -sync_pb_for_lsp(struct ovn_port *op) +sync_pb_for_lsp(struct ovn_port *op, + const struct lr_lb_nat_data_table *lr_lbnats) { ovs_assert(op->nbsp); @@ -4558,10 +4278,17 @@ sync_pb_for_lsp(struct ovn_port *op) if (nat_addresses && !strcmp(nat_addresses, "router")) { if (op->peer && op->peer->od && (chassis || op->peer->od->n_l3dgw_ports)) { - bool exclude_lb_vips = smap_get_bool(&op->nbsp->options, + bool include_lb_vips = !smap_get_bool(&op->nbsp->options, "exclude-lb-vips-from-garp", false); + + const struct lr_lb_nat_data_record *lr_lbnat_rec = NULL; + + if (include_lb_vips) { + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, + op->peer->od->nbr); + } nats = get_nat_addresses(op->peer, &n_nats, false, - !exclude_lb_vips); + include_lb_vips, lr_lbnat_rec); } } else if (nat_addresses && (chassis || l3dgw_ports)) { struct lport_addresses laddrs; @@ -4668,7 +4395,8 @@ sync_pb_for_lsp(struct ovn_port *op) * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only sets the port binding options column for the router ports */ static void -sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) +sync_pb_for_lrp(struct ovn_port *op, + const struct lr_lb_nat_data_table *lr_lbnats) { ovs_assert(op->nbrp); @@ -4677,14 +4405,14 @@ sync_pb_for_lrp(struct ovn_port *op, const struct lr_nat_table *lr_nats) const char *chassis_name = smap_get(&op->od->nbr->options, "chassis"); if (is_cr_port(op)) { - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find(lr_nats, op->od->nbr); - ovs_assert(lrnat_rec); + const struct lr_lb_nat_data_record *lr_lbnat_rec = + lr_lb_nat_data_table_find(lr_lbnats, op->od->nbr); + ovs_assert(lr_lbnat_rec); smap_add(&new, "distributed-port", op->nbrp->name); bool always_redirect = - !lrnat_rec->has_distributed_nat && + !lr_lbnat_rec->lrnat_rec->has_distributed_nat && !l3dgw_port_has_associated_vtep_lports(op->l3dgw_port); const char *redirect_type = smap_get(&op->nbrp->options, @@ -4735,17 +4463,18 @@ static void ovn_update_ipv6_opt_for_op(struct ovn_port *op); * the logical switch ports. */ void sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *lr_nats) + struct hmap *lr_ports, + const struct lr_lb_nat_data_table *lr_lbnats) { ovs_assert(ovnsb_idl_txn); struct ovn_port *op; HMAP_FOR_EACH (op, key_node, ls_ports) { - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_lbnats); } HMAP_FOR_EACH (op, key_node, lr_ports) { - sync_pb_for_lrp(op, lr_nats); + sync_pb_for_lrp(op, lr_lbnats); } ovn_update_ipv6_options(lr_ports); @@ -4754,17 +4483,18 @@ sync_pbs(struct ovsdb_idl_txn *ovnsb_idl_txn, struct hmap *ls_ports, /* Sync the SB Port bindings for the added and updated logical switch ports * of the tracked northd engine data. */ bool -sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports, - const struct lr_nat_table *lr_nats) +sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *trk_ovn_ports, + const struct lr_lb_nat_data_table *lr_lbnats) { struct hmapx_node *hmapx_node; struct ovn_port *op; HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { op = hmapx_node->data; if (op->nbsp) { - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_lbnats); } else { - sync_pb_for_lrp(op, lr_nats); + sync_pb_for_lrp(op, lr_lbnats); ovn_update_ipv6_opt_for_op(op); } } @@ -4772,9 +4502,9 @@ sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *trk_ovn_ports, HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->updated) { op = hmapx_node->data; if (op->nbsp) { - sync_pb_for_lsp(op); + sync_pb_for_lsp(op, lr_lbnats); } else { - sync_pb_for_lrp(op, lr_nats); + sync_pb_for_lrp(op, lr_lbnats); ovn_update_ipv6_opt_for_op(op); } } @@ -5481,20 +5211,24 @@ fail: } /* Returns true if the logical router has changes which can be - * incrementally handled. + * incrementally handled or the changes can be ignored. * Presently supports i-p for the below changes: * - load balancers and load balancer groups. + * + * Presently below changes are ignored: + * - router NAT changes - as the engine node lr-nat handles it. */ static bool -lr_changes_can_be_handled( +lr_changes_can_be_handled_or_ignored( const struct nbrec_logical_router *lr) { /* Check if the columns are changed in this row. */ enum nbrec_logical_router_column_id col; for (col = 0; col < NBREC_LOGICAL_ROUTER_N_COLUMNS; col++) { if (nbrec_logical_router_is_updated(lr, col)) { - if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER || - col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP) { + if (col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER + || col == NBREC_LOGICAL_ROUTER_COL_LOAD_BALANCER_GROUP + || col == NBREC_LOGICAL_ROUTER_COL_NAT) { continue; } return false; @@ -5513,12 +5247,6 @@ lr_changes_can_be_handled( OVSDB_IDL_CHANGE_MODIFY) > 0) { return false; } - for (size_t i = 0; i < lr->n_nat; i++) { - if (nbrec_nat_row_get_seqno(lr->nat[i], - OVSDB_IDL_CHANGE_MODIFY) > 0) { - return false; - } - } for (size_t i = 0; i < lr->n_policies; i++) { if (nbrec_logical_router_policy_row_get_seqno(lr->policies[i], OVSDB_IDL_CHANGE_MODIFY) > 0) { @@ -5534,14 +5262,15 @@ lr_changes_can_be_handled( return true; } -/* Return true if changes are handled incrementally, false otherwise. +/* Return true if changes are handled incrementally or can be safely + * ignored (because those changes are handled by other engine nodes), + * false otherwise. * When there are any changes, try to track what's exactly changed and set * northd_data->change_tracked accordingly: change tracked - true, otherwise, * false. * Note: Changes to load balancer and load balancer groups associated with * the logical routers are handled separately in the lb_data change - * handlers (northd_handle_lb_data_changes_pre_od and - * northd_handle_lb_data_changes_post_od). + * handler (northd_handle_lb_data_changes). * */ bool northd_handle_lr_changes(const struct northd_input *ni, @@ -5556,9 +5285,11 @@ northd_handle_lr_changes(const struct northd_input *ni, goto fail; } - /* Presently only able to handle load balancer and - * load balancer group changes. */ - if (!lr_changes_can_be_handled(changed_lr)) { + /* Presently + * - only able to handle load balancer and load balancer group + changes. + * - and ignore NAT changes */ + if (!lr_changes_can_be_handled_or_ignored(changed_lr)) { goto fail; } } @@ -5810,10 +5541,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5832,10 +5559,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, ovs_assert(lb_dps); ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - build_lrouter_lb_reachable_ips(od, lb_dps->lb); - /* Add the lb to the northd tracked data. */ hmapx_add(&nd_changes->trk_lbs.crupdated, lb_dps); } @@ -5871,22 +5594,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - /* Update the od->lb_ips with the deleted and inserted - * vips (if any). */ - remove_ips_from_lb_ip_set(od->lb_ips, lb->routable, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_ips_to_lb_ip_set(od->lb_ips, lb->routable, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); - - remove_lrouter_lb_reachable_ips(od, lb->neigh_mode, - &clb->deleted_vips_v4, - &clb->deleted_vips_v6); - add_neigh_ips_to_lrouter(od, lb->neigh_mode, - &clb->inserted_vips_v4, - &clb->inserted_vips_v6); - /* Add the lr datapath to the northd tracked data. */ hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } @@ -5914,9 +5621,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, /* Re-evaluate 'od->has_lb_vip' */ init_lb_for_datapath(od); - /* Add the lb_ips of lb_dps to the od. */ - build_lrouter_lb_ips(od->lb_ips, lb_dps->lb); - /* Add the lr datapath to the northd tracked data. */ hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } @@ -9208,7 +8912,7 @@ arp_nd_ns_match(const char *ips, int addr_family, struct ds *match) /* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) { for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -9224,7 +8928,7 @@ lrouter_port_ipv4_reachable(const struct ovn_port *op, ovs_be32 addr) /* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the * IPs configured on the router port. */ -static bool +bool lrouter_port_ipv6_reachable(const struct ovn_port *op, const struct in6_addr *addr) { @@ -9290,6 +8994,7 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -9305,32 +9010,37 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * router port. * Priority: 80. */ - - const char *ip_addr; - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v4_reachable) { - ovs_be32 ipv4_addr; - - /* Check if the ovn port has a network configured on which we could - * expect ARP requests for the LB VIP. - */ - if (ip_parse(ip_addr, &ipv4_addr) && - lrouter_port_ipv4_reachable(op, ipv4_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + const struct lr_lb_nat_data_record *lr_lbnat_rec = NULL; + if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, op->od->nbr); + ovs_assert(lr_lbnat_rec); + + const char *ip_addr; + SSET_FOR_EACH (ip_addr, &lr_lbnat_rec->lb_ips->ips_v4_reachable) { + ovs_be32 ipv4_addr; + + /* Check if the ovn port has a network configured on which we could + * expect ARP requests for the LB VIP. + */ + if (ip_parse(ip_addr, &ipv4_addr) && + lrouter_port_ipv4_reachable(op, ipv4_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET, sw_op, sw_od, 80, lflows, + stage_hint); + } } - } - SSET_FOR_EACH (ip_addr, &op->od->lb_ips->ips_v6_reachable) { - struct in6_addr ipv6_addr; + SSET_FOR_EACH (ip_addr, &lr_lbnat_rec->lb_ips->ips_v6_reachable) { + struct in6_addr ipv6_addr; - /* Check if the ovn port has a network configured on which we could - * expect NS requests for the LB VIP. - */ - if (ipv6_parse(ip_addr, &ipv6_addr) && - lrouter_port_ipv6_reachable(op, &ipv6_addr)) { - build_lswitch_rport_arp_req_flow( - ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + /* Check if the ovn port has a network configured on which we could + * expect NS requests for the LB VIP. + */ + if (ipv6_parse(ip_addr, &ipv6_addr) && + lrouter_port_ipv6_reachable(op, &ipv6_addr)) { + build_lswitch_rport_arp_req_flow( + ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, + stage_hint); + } } } @@ -9380,13 +9090,15 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!sset_contains(&op->od->lb_ips->ips_v6, nat->external_ip)) { + if (!lr_lbnat_rec || !sset_contains(&lr_lbnat_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!sset_contains(&op->od->lb_ips->ips_v4, nat->external_ip)) { + if (!lr_lbnat_rec || !sset_contains(&lr_lbnat_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -10447,6 +10159,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, static void build_lswitch_ip_unicast_lookup(struct ovn_port *op, const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, struct ds *actions, struct ds *match) @@ -10462,7 +10175,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, */ if (lsp_is_router(op->nbsp)) { build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lflows, &op->nbsp->header_); + lr_lbnats, lflows, + &op->nbsp->header_); } for (size_t i = 0; i < op->nbsp->n_addresses; i++) { @@ -12652,6 +12366,7 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_lb_nat_data_record *lr_lbnat_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, struct hmap *lflows) @@ -12664,8 +12379,9 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v4, ip); + bool router_ip_in_lb_ips = (lr_lbnat_rec && + !!sset_find(&lr_lbnat_rec->lb_ips->ips_v4, + ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12694,8 +12410,9 @@ build_lrouter_drop_own_dest(struct ovn_port *op, bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, ip); - bool router_ip_in_lb_ips = - !!sset_find(&op->od->lb_ips->ips_v6, ip); + bool router_ip_in_lb_ips = (lr_lbnat_rec && + !!sset_find(&lr_lbnat_rec->lb_ips->ips_v6, + ip)); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -13407,7 +13124,8 @@ build_ip_routing_flows_for_lrp( */ static void build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct hmap *lr_ports, struct hmap *lflows) + struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, + const struct hmap *lr_ports, struct hmap *lflows) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13415,7 +13133,8 @@ build_ip_routing_flows_for_router_type_lsp( } struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs) { + if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs + || !op->od->n_router_ports) { return; } @@ -13426,19 +13145,28 @@ build_ip_routing_flows_for_router_type_lsp( continue; } - struct ovn_port_routable_addresses *ra = &router_port->routables; - for (size_t j = 0; j < ra->n_addrs; j++) { - struct lport_addresses *laddrs = &ra->laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + const struct lr_lb_nat_data_record *lr_lbnat_rec = + lr_lb_nat_data_table_find(lr_lbnats, router_port->od->nbr); + + if (router_port->nbrp->ha_chassis_group || + router_port->nbrp->n_gateway_chassis) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_lbnat_rec); + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, peer->od, peer, + peer->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &peer->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED); + } } + destroy_routable_addresses(&ra); } } + } static void @@ -13662,33 +13390,36 @@ build_arp_resolve_flows_for_lrouter( static void routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, - struct ovn_port *peer, struct ds *match, - struct ds *actions) + struct ovn_port *peer, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct ds *match, struct ds *actions) { - struct ovn_port_routable_addresses *ra = &router_port->routables; - if (!ra->n_addrs) { + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(router_port, lr_lbnat_rec); + if (!ra.n_addrs) { return; } - for (size_t i = 0; i < ra->n_addrs; i++) { + for (size_t i = 0; i < ra.n_addrs; i++) { ds_clear(match); ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", peer->json_key); bool first = true; - for (size_t j = 0; j < ra->laddrs[i].n_ipv4_addrs; j++) { + for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { if (!first) { ds_put_cstr(match, ", "); } - ds_put_cstr(match, ra->laddrs[i].ipv4_addrs[j].addr_s); + ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); first = false; } ds_put_cstr(match, "}"); ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra->laddrs[i].ea_s); + ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions)); } + destroy_routable_addresses(&ra); } /* Local router ingress table ARP_RESOLVE: ARP Resolution. @@ -13705,6 +13436,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, const struct lr_nat_record *lrnat_rec, + const struct lr_lb_nat_data_record *lr_lbnat_rec, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13781,8 +13513,8 @@ build_arp_resolve_flows_for_lrp( * * Priority 2. */ - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_ARP_RESOLVE, 2, - true, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_lbnat_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13790,6 +13522,7 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, + const struct lr_lb_nat_data_table *lr_lbnats, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13933,8 +13666,11 @@ build_arp_resolve_flows_for_lsp( if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_lb_nat_data_record *lr_lbnat_rec; + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, + router_port->od->nbr); routable_addresses_to_lflows(lflows, router_port, peer, - match, actions); + lr_lbnat_rec, match, actions); } } } @@ -14654,6 +14390,7 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, const struct lr_nat_record *lrnat_rec, + const struct lr_lb_nat_data_record *lr_lbnat_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14778,7 +14515,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (sset_count(&op->od->lb_ips->ips_v4_reachable)) { + if (lr_lbnat_rec && sset_count(&lr_lbnat_rec->lb_ips->ips_v4_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { ds_put_format(match, "is_chassis_resident(%s)", @@ -14794,7 +14531,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, free(lb_ips_v4_as); } - if (sset_count(&op->od->lb_ips->ips_v6_reachable)) { + if (lr_lbnat_rec && sset_count(&lr_lbnat_rec->lb_ips->ips_v6_reachable)) { ds_clear(match); if (is_l3dgw_port(op)) { @@ -14896,8 +14633,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, * Priority 60. */ if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, S_ROUTER_IN_IP_INPUT, 60, - false, lflows); + build_lrouter_drop_own_dest(op, lrnat_rec, lr_lbnat_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); } /* ARP / ND handling for external IP addresses. * @@ -16036,6 +15773,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_lb_nat_data_table *lr_lbnats; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -16119,14 +15857,15 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, * switch port. */ static void -build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct shash *meter_groups, - struct ds *match, - struct ds *actions, - struct hmap *lflows) +build_lswitch_and_lrouter_iterate_by_lsp( + struct ovn_port *op, const struct hmap *ls_ports, + const struct hmap *lr_ports, + const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) { ovs_assert(op->nbsp); start_collecting_lflows(); @@ -16139,11 +15878,14 @@ build_lswitch_and_lrouter_iterate_by_lsp(struct ovn_port *op, meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lflows, actions, match); + build_lswitch_ip_unicast_lookup(op, lr_nats, lr_lbnats, lflows, actions, + match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_ports, lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); + build_ip_routing_flows_for_router_type_lsp(op, lr_lbnats, lr_ports, + lflows); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_lbnats, + match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -16162,6 +15904,8 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, op->od->nbr); ovs_assert(lrnet_rec); + const struct lr_lb_nat_data_record *lr_lbnat_rec = + lr_lb_nat_data_table_find(lsi->lr_lbnats, op->od->nbr); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -16169,15 +15913,15 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_lbnat_rec, lsi->lflows, + &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, + build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_lbnat_rec, &lsi->match, &lsi->actions, lsi->meter_groups); build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, &lsi->actions); @@ -16240,6 +15984,7 @@ build_lflows_thread(void *arg) build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, lsi->lr_ports, lsi->lr_nats, + lsi->lr_lbnats, lsi->meter_groups, &lsi->match, &lsi->actions, @@ -16350,6 +16095,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16380,6 +16126,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_nats = lr_nats; + lsiv[index].lr_lbnats = lr_lbnats; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16415,6 +16162,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_nats = lr_nats, + .lr_lbnats = lr_lbnats, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16443,6 +16191,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, lsi.lr_nats, + lsi.lr_lbnats, lsi.meter_groups, &lsi.match, &lsi.actions, lsi.lflows); @@ -16564,6 +16313,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_nats, + input_data->lr_lbnats, lflows, &igmp_groups, input_data->meter_groups, @@ -17044,6 +16794,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, lflow_input->lr_nats, + lflow_input->lr_lbnats, lflow_input->meter_groups, &match, &actions, lflows); @@ -17082,6 +16833,7 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, lflow_input->lr_nats, + lflow_input->lr_lbnats, lflow_input->meter_groups, &match, &actions, lflows); diff --git a/northd/northd.h b/northd/northd.h index 8c9d0f308f..ace9720b95 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -173,6 +173,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_nat_table *lr_nats; + const struct lr_lb_nat_data_table *lr_lbnats; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -312,9 +313,6 @@ struct ovn_datapath { /* router datapath has a logical port with redirect-type set to bridged. */ bool redirect_bridged; - /* Load Balancer vIPs relevant for this datapath. */ - struct ovn_lb_ip_set *lb_ips; - struct ovn_port **localnet_ports; size_t n_localnet_ports; @@ -331,6 +329,125 @@ struct ovn_datapath { const struct ovn_datapath *ovn_datapath_find(const struct hmap *datapaths, const struct uuid *uuid); +static inline size_t +ods_size(const struct ovn_datapaths *datapaths) +{ + return hmap_count(&datapaths->datapaths); +} + +/* A logical switch port or logical router port. + * + * In steady state, an ovn_port points to a northbound Logical_Switch_Port + * record (via 'nbsp') *or* a Logical_Router_Port record (via 'nbrp'), and to a + * southbound Port_Binding record (via 'sb'). As the state of the system + * changes, join_logical_ports() may determine that there is a new LSP or LRP + * that has no corresponding Port_Binding record (in which case build_ports()) + * will create the missing Port_Binding) or that a Port_Binding record exists + * that has no coresponding LSP (in which case build_ports() will delete the + * spurious Port_Binding). Thus, after build_ports() runs, any given ovn_port + * will have 'sb' nonnull, and 'nbsp' xor 'nbrp' nonnull. + * + * Ordinarily there is only one ovn_port that points to a given LSP or LRP (but + * distributed gateway ports point a "derived" ovn_port to a duplicate LRP). + */ +struct ovn_port { + /* Port name aka key. + * + * This is ordinarily the same as nbsp->name or nbrp->name and + * sb->logical_port. (A distributed gateway port creates a "derived" + * ovn_port with key "cr-%s" % nbrp->name.) */ + struct hmap_node key_node; /* Index on 'key'. */ + char *key; /* nbsp->name, nbrp->name, sb->logical_port. */ + char *json_key; /* 'key', quoted for use in JSON. */ + + const struct sbrec_port_binding *sb; /* May be NULL. */ + + uint32_t tunnel_key; + + /* Logical switch port data. */ + const struct nbrec_logical_switch_port *nbsp; /* May be NULL. */ + + struct lport_addresses *lsp_addrs; /* Logical switch port addresses. */ + unsigned int n_lsp_addrs; /* Total length of lsp_addrs. */ + unsigned int n_lsp_non_router_addrs; /* Number of elements from the + * beginning of 'lsp_addrs' extracted + * directly from LSP 'addresses'. */ + + struct lport_addresses *ps_addrs; /* Port security addresses. */ + unsigned int n_ps_addrs; + + bool lsp_can_be_inc_processed; /* If it can be incrementally processed when + the port changes. */ + + /* Logical router port data. */ + const struct nbrec_logical_router_port *nbrp; /* May be NULL. */ + + struct lport_addresses lrp_networks; + + /* Logical port multicast data. */ + struct mcast_port_info mcast_info; + + /* At most one of l3dgw_port and cr_port can be not NULL. */ + + /* This is set to a distributed gateway port if and only if this ovn_port + * is "derived" from it. Otherwise this is set to NULL. The derived + * ovn_port represents the instance of distributed gateway port on the + * gateway chassis.*/ + struct ovn_port *l3dgw_port; + + /* This is set to the "derived" chassis-redirect port of this port if and + * only if this port is a distributed gateway port. Otherwise this is set + * to NULL. */ + struct ovn_port *cr_port; + + bool has_unknown; /* If the addresses have 'unknown' defined. */ + + bool has_bfd; + + /* The port's peer: + * + * - A switch port S of type "router" has a router port R as a peer, + * and R in turn has S has its peer. + * + * - Two connected logical router ports have each other as peer. + * + * - Other kinds of ports have no peer. */ + struct ovn_port *peer; + + struct ovn_datapath *od; + + struct ovs_list list; /* In list of similar records. */ + + struct hmap_node dp_node; /* Node in od->ports. */ + + struct lport_addresses proxy_arp_addrs; + + /* Temporarily used for traversing a list (or hmap) of ports. */ + bool visited; + + /* List of struct lflow_ref_node that points to the lflows generated by + * this ovn_port. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Adding the list here is more straightforward. The drawback is that we + * need to keep in mind that this data belongs to en_lflow node, so never + * access it from any other nodes. + */ + struct ovs_list lflows; +}; + void ovnnb_db_run(struct northd_input *input_data, struct northd_data *data, struct ovsdb_idl_txn *ovnnb_txn, @@ -390,13 +507,27 @@ void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, struct chassis_features *chassis_features); bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); +struct lr_lb_nat_data_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, - struct hmap *lr_ports, const struct lr_nat_table *); -bool sync_pbs_for_northd_changed_ovn_ports(struct tracked_ovn_ports *, - const struct lr_nat_table *); + struct hmap *lr_ports, + const struct lr_lb_nat_data_table *); +bool sync_pbs_for_northd_changed_ovn_ports( + struct tracked_ovn_ports *, + const struct lr_lb_nat_data_table *); bool northd_has_tracked_data(struct northd_tracked_data *); bool northd_has_only_ports_in_tracked_data(struct northd_tracked_data *); bool northd_has_lbs_in_tracked_data(struct northd_tracked_data *); +/* Returns 'true' if the IPv4 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); + +/* Returns 'true' if the IPv6 'addr' is on the same subnet with one of the + * IPs configured on the router port. + */ +bool lrouter_port_ipv6_reachable(const struct ovn_port *, + const struct in6_addr *); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index b7f9cb5689..8fc5cd1d60 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10416,18 +10416,21 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:10.0.0.2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10437,6 +10440,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10450,6 +10454,7 @@ AT_CHECK([ovn-nbctl --wait=sb \ ]) check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10467,6 +10472,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear Load_Balancer . health_check check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10481,6 +10487,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10489,6 +10496,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) @@ -10501,6 +10509,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10510,6 +10519,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10519,6 +10529,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10528,6 +10539,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10537,6 +10549,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10547,6 +10560,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10567,6 +10581,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10576,6 +10591,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10585,6 +10601,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10594,6 +10611,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10603,6 +10621,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10632,6 +10651,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10639,6 +10659,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10655,6 +10676,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10671,6 +10693,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10680,6 +10703,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10689,6 +10713,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10698,6 +10723,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10713,6 +10739,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10722,6 +10749,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.100:80"' check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10731,6 +10759,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10740,6 +10769,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10749,6 +10779,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10757,6 +10788,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 load_balancer_group check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10765,6 +10797,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10773,6 +10806,7 @@ check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ destroy load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb compute compute @@ -10796,6 +10830,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute @@ -10803,6 +10838,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute @@ -10810,6 +10846,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10818,6 +10855,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10826,6 +10864,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10834,6 +10873,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10843,6 +10883,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb1 check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10851,6 +10892,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-del sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10859,6 +10901,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10869,6 +10912,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10879,6 +10923,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10887,6 +10932,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb remove load_balancer_group . load_balancer $lb3_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11019,6 +11065,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-add lr0 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11031,6 +11078,7 @@ check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 # for the SB port binding change. check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11042,6 +11090,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11067,6 +11116,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl set logical_router_port lr0-sw0 options:foo=bar check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11076,8 +11126,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # engine nodes. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11085,8 +11135,8 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11094,8 +11144,9 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11103,8 +11154,9 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11112,8 +11164,9 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . type=snat -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11121,8 +11174,9 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0p1 30:54:00:00:00:03 -check_engine_stats northd recompute compute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11131,8 +11185,9 @@ nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11147,32 +11202,36 @@ check ovn-nbctl lr-lb-add lr0 lb2 # is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.140 10.0.0.20 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 -check_engine_stats northd recompute nocompute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11180,8 +11239,9 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE # Delete the NAT check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_router lr0 nat -check_engine_stats northd recompute compute -check_engine_stats lr_nat recompute nocompute +check_engine_stats northd norecompute compute +check_engine_stats lr_nat norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11191,6 +11251,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-add lr0 10 "ip4.src == 10.0.0.3" reroute 172.168.0.101,172.168.0.102 check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11199,6 +11260,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-policy-del lr0 10 "ip4.src == 10.0.0.3" check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Oct 24 00:48:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854080 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtj30Cdbz23jl for ; Tue, 24 Oct 2023 11:48:38 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 96D2F4197F; Tue, 24 Oct 2023 00:48:34 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 96D2F4197F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id E7Z21EzUjdau; Tue, 24 Oct 2023 00:48:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2D15441989; Tue, 24 Oct 2023 00:48:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2D15441989 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id F27E4C008C; Tue, 24 Oct 2023 00:48:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 30913C0071 for ; Tue, 24 Oct 2023 00:48:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 99F2A83C90 for ; Tue, 24 Oct 2023 00:48:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 99F2A83C90 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id H8ldUdmBY9A4 for ; Tue, 24 Oct 2023 00:48:25 +0000 (UTC) Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by smtp1.osuosl.org (Postfix) with ESMTPS id E843B83C94 for ; Tue, 24 Oct 2023 00:48:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E843B83C94 Received: by mail.gandi.net (Postfix) with ESMTPSA id EAD3540003; Tue, 24 Oct 2023 00:48:21 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:48:10 -0400 Message-ID: <20231024004810.4133842-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 07/18] northd: Generate logical router's LB and NAT flows using lr_lbnat_data. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Previous commits added new engine nodes to store logical router's lb and NAT data. Make use of the data stored by these engine nodes to generate logical flows related to router's LBs and NATs. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 3 - northd/en-lr-lb-nat-data.h | 4 + northd/inc-proc-northd.c | 1 - northd/northd.c | 751 ++++++++++++++++++++++++------------- northd/northd.h | 1 - 5 files changed, 495 insertions(+), 265 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 9cb0ead3f0..229f4be1d0 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -42,8 +42,6 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("port_group", node); struct sync_meters_data *sync_meters_data = engine_get_input_data("sync_meters", node); - struct ed_type_lr_nat_data *lr_nat_data = - engine_get_input_data("lr_nat", node); struct ed_type_lr_lb_nat_data *lr_lb_nat_data = engine_get_input_data("lr_lb_nat_data", node); @@ -68,7 +66,6 @@ lflow_get_input_data(struct engine_node *node, lflow_input->ls_ports = &northd_data->ls_ports; lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; - lflow_input->lr_nats = &lr_nat_data->lr_nats; lflow_input->lr_lbnats = &lr_lb_nat_data->lr_lbnats; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; diff --git a/northd/en-lr-lb-nat-data.h b/northd/en-lr-lb-nat-data.h index 30e76011a8..653aa6cf89 100644 --- a/northd/en-lr-lb-nat-data.h +++ b/northd/en-lr-lb-nat-data.h @@ -53,6 +53,10 @@ struct lr_lb_nat_data_table { #define LR_LB_NAT_DATA_TABLE_FOR_EACH(LR_LB_NAT_REC, TABLE) \ HMAP_FOR_EACH (LR_LB_NAT_REC, key_node, &(TABLE)->entries) +#define LR_LB_NAT_DATA_TABLE_FOR_EACH_IN_P(LR_LB_NAT_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LR_LB_NAT_REC, key_node, JOBID, \ + &(TABLE)->entries) + struct lr_lb_nat_data_tracked_data { /* Created or updated logical router with LB data. */ struct hmapx crupdated; /* Stores 'struct lr_lb_nat_data_record'. */ diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 369a151fa3..84627070a8 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -228,7 +228,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); - engine_add_input(&en_lflow, &en_lr_nat, NULL); engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, diff --git a/northd/northd.c b/northd/northd.c index 30e9278961..2fcb5c9f3a 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -8854,18 +8854,14 @@ build_lrouter_groups(struct hmap *lr_ports, struct ovs_list *lr_list) */ static void build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, - uint32_t priority, - struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, - struct hmap *lflows) + uint32_t priority, + const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, + struct hmap *lflows) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; - const struct lr_nat_record *lrnat_rec = lr_nat_table_find( - lr_nats, op->od->nbr); - ovs_assert(lrnat_rec); - /* Self originated ARP requests/RARP/ND need to be flooded to the L2 domain * (except on router ports). Determine that packets are self originated * by also matching on source MAC. Matching on ingress port is not @@ -8952,7 +8948,8 @@ lrouter_port_ipv6_reachable(const struct ovn_port *op, */ static void build_lswitch_rport_arp_req_flow(const char *ips, - int addr_family, struct ovn_port *patch_op, struct ovn_datapath *od, + int addr_family, struct ovn_port *patch_op, + const struct ovn_datapath *od, uint32_t priority, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -8993,8 +8990,6 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, - const struct lr_nat_table *lr_nats, - const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, const struct ovsdb_idl_row *stage_hint) { @@ -9010,11 +9005,48 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * router port. * Priority: 80. */ - const struct lr_lb_nat_data_record *lr_lbnat_rec = NULL; - if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { - lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, op->od->nbr); - ovs_assert(lr_lbnat_rec); + for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, + lflows, stage_hint); + } + for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { + build_lswitch_rport_arp_req_flow( + op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, + lflows, stage_hint); + } +} +/* + * Ingress table 25: Flows that forward ARP/ND requests only to the routers + * that own the addresses. + * Priorities: + * - 80: self originated GARPs that need to follow regular processing. + * - 75: ARP requests to router owned IPs (interface IP/LB/NAT). + */ +static void +build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + const struct ovn_datapath *sw_od, + struct ovn_port *sw_op, + struct hmap *lflows, + const struct ovsdb_idl_row *stage_hint) +{ + if (!op || !op->nbrp) { + return; + } + + if (!lrport_is_enabled(op->nbrp)) { + return; + } + + ovs_assert(op->od == lr_lbnat_rec->od); + + /* Forward ARP requests for owned IP addresses (L3, VIP, NAT) only to this + * router port. + * Priority: 80. + */ + if (op->od->nbr->n_load_balancer || op->od->nbr->n_load_balancer_group) { const char *ip_addr; SSET_FOR_EACH (ip_addr, &lr_lbnat_rec->lb_ips->ips_v4_reachable) { ovs_be32 ipv4_addr; @@ -9044,17 +9076,6 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, } } - for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); - } - for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { - build_lswitch_rport_arp_req_flow( - op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); - } - /* Self originated ARP requests/RARP/ND need to be flooded as usual. * * However, if the switch doesn't have any non-router ports we shouldn't @@ -9063,19 +9084,13 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * Priority: 75. */ if (sw_od->n_router_ports != sw_od->nbs->n_ports) { - build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_nats, + build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, + lr_lbnat_rec->lrnat_rec, lflows); } - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find(lr_nats, op->od->nbr); - - if (!lrnat_rec) { - return; - } - - for (size_t i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (size_t i = 0; i < lr_lbnat_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lr_lbnat_rec->lrnat_rec->nat_entries[i]; const struct nbrec_nat *nat = nat_entry->nb; if (!nat_entry_is_valid(nat_entry)) { @@ -9090,15 +9105,15 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, * expect ARP requests/NS for the DNAT external_ip. */ if (nat_entry_is_v6(nat_entry)) { - if (!lr_lbnat_rec || !sset_contains(&lr_lbnat_rec->lb_ips->ips_v6, - nat->external_ip)) { + if (!sset_contains(&lr_lbnat_rec->lb_ips->ips_v6, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, stage_hint); } } else { - if (!lr_lbnat_rec || !sset_contains(&lr_lbnat_rec->lb_ips->ips_v4, - nat->external_ip)) { + if (!sset_contains(&lr_lbnat_rec->lb_ips->ips_v4, + nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, stage_hint); @@ -10157,12 +10172,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, - const struct lr_nat_table *lr_nats, - const struct lr_lb_nat_data_table *lr_lbnats, - struct hmap *lflows, - struct ds *actions, - struct ds *match) +build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, + struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); if (lsp_is_external(op->nbsp)) { @@ -10174,8 +10185,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, * requests only to the router port that owns the IP address. */ if (lsp_is_router(op->nbsp)) { - build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lr_nats, - lr_lbnats, lflows, + build_lswitch_rport_arp_req_flows(op->peer, op->od, op, lflows, &op->nbsp->header_); } @@ -10272,33 +10282,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); - - /* Add ethernet addresses specified in NAT rules on - * distributed logical routers. */ - if (is_l3dgw_port(op->peer)) { - for (int j = 0; j < op->peer->od->nbr->n_nat; j++) { - const struct nbrec_nat *nat - = op->peer->od->nbr->nat[j]; - if (!strcmp(nat->type, "dnat_and_snat") - && nat->logical_port && nat->external_mac - && eth_addr_from_string(nat->external_mac, &mac)) { - - ds_clear(match); - ds_put_format(match, "eth.dst == "ETH_ADDR_FMT - " && is_chassis_resident(\"%s\")", - ETH_ADDR_ARGS(mac), - nat->logical_port); - - ds_clear(actions); - ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); - } - } - } } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10309,6 +10292,52 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, } } +/* Ingress table 25: Destination lookup, unicast handling (priority 50), */ +static void +build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct hmap *lflows, struct ds *match, + struct ds *actions) +{ + ovs_assert(op->nbsp); + + if (!op->peer || !is_l3dgw_port(op->peer)) { + return; + } + + ovs_assert(op->peer->od == lr_lbnat_rec->od); + + const char *action = lsp_is_enabled(op->nbsp) ? + "outport = %s; output;" : + debug_drop_action(); + struct eth_addr mac; + + /* Add ethernet addresses specified in NAT rules on + * distributed logical routers. */ + for (size_t i = 0; i < lr_lbnat_rec->lrnat_rec->n_nat_entries; i++) { + const struct ovn_nat *nat = &lr_lbnat_rec->lrnat_rec->nat_entries[i]; + + if (!strcmp(nat->nb->type, "dnat_and_snat") + && nat->nb->logical_port && nat->nb->external_mac + && eth_addr_from_string(nat->nb->external_mac, &mac)) { + + ds_clear(match); + ds_put_format(match, "eth.dst == "ETH_ADDR_FMT + " && is_chassis_resident(\"%s\")", + ETH_ADDR_ARGS(mac), + nat->nb->logical_port); + + ds_clear(actions); + ds_put_format(actions, action, op->json_key); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_); + } + } +} + struct bfd_entry { struct hmap_node hmap_node; @@ -11690,7 +11719,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_lb_datapaths *lb_dps, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -11796,9 +11825,11 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_datapath *od = lr_datapaths->array[index]; enum lrouter_nat_lb_flow_type type; - const struct lr_nat_record *lrnat_rec = - lr_nat_table_find(lr_nats, od->nbr); - ovs_assert(lrnat_rec); + const struct lr_lb_nat_data_record *lr_lbnat_rec = + lr_lb_nat_data_table_find(lr_lbnats, od->nbr); + ovs_assert(lr_lbnat_rec); + + const struct lr_nat_record *lrnat_rec = lr_lbnat_rec->lrnat_rec; if (lb->skip_snat) { type = LROUTER_NAT_LB_FLOW_SKIP_SNAT; } else if (!lport_addresses_is_empty(&lrnat_rec->lb_force_snat_addrs) @@ -11948,7 +11979,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct hmap *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, - const struct lr_nat_table *lr_nats, + const struct lr_lb_nat_data_table *lr_lbnats, const struct chassis_features *features, const struct hmap *svc_monitor_map, struct ds *match, struct ds *action) @@ -11964,7 +11995,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip = &lb->vips[i]; build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], - lr_datapaths, lr_nats, lflows, match, + lr_datapaths, lr_lbnats, lflows, match, action, meter_groups, features, svc_monitor_map); @@ -12102,7 +12133,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) * and action says "next" instead of ct*. */ static inline void -lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, +lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct hmap *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) @@ -12166,7 +12197,7 @@ lrouter_nat_add_ext_ip_match(struct ovn_datapath *od, * with the given priority. */ static void -build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, @@ -12215,7 +12246,7 @@ build_lrouter_arp_flow(struct ovn_datapath *od, struct ovn_port *op, * 'sn_ip_address'. */ static void -build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, +build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *action, const char *ip_address, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, @@ -12269,7 +12300,7 @@ build_lrouter_nd_flow(struct ovn_datapath *od, struct ovn_port *op, } static void -build_lrouter_nat_arp_nd_flow(struct ovn_datapath *od, +build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct hmap *lflows, const struct shash *meter_groups) @@ -12365,7 +12396,6 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, static void build_lrouter_drop_own_dest(struct ovn_port *op, - const struct lr_nat_record *lrnat_rec, const struct lr_lb_nat_data_record *lr_lbnat_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, @@ -12377,11 +12407,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { const char *ip = op->lrp_networks.ipv4_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_lbnat_rec && - !!sset_find(&lr_lbnat_rec->lb_ips->ips_v4, - ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_lbnat_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_lbnat_rec->lb_ips->ips_v4, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12408,11 +12437,10 @@ build_lrouter_drop_own_dest(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { const char *ip = op->lrp_networks.ipv6_addrs[i].addr_s; - bool router_ip_in_snat_ips = !!shash_find(&lrnat_rec->snat_ips, - ip); - bool router_ip_in_lb_ips = (lr_lbnat_rec && - !!sset_find(&lr_lbnat_rec->lb_ips->ips_v6, - ip)); + bool router_ip_in_snat_ips = + !!shash_find(&lr_lbnat_rec->lrnat_rec->snat_ips, ip); + bool router_ip_in_lb_ips = + !!sset_find(&lr_lbnat_rec->lb_ips->ips_v6, ip); bool drop_router_ip = (drop_snat_ip == (router_ip_in_snat_ips || router_ip_in_lb_ips)); @@ -12436,7 +12464,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_force_snat_flows(struct hmap *lflows, + const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) { @@ -13435,8 +13464,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, /* This function adds ARP resolve flows related to a LRP. */ static void build_arp_resolve_flows_for_lrp( - struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct ovn_port *op, struct hmap *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -13506,15 +13534,6 @@ build_arp_resolve_flows_for_lrp( &op->nbrp->header_); } } - - /* Drop IP traffic destined to router owned IPs. Part of it is dropped - * in stage "lr_in_ip_input" but traffic that could have been unSNATed - * but didn't match any existing session might still end up here. - * - * Priority 2. - */ - build_lrouter_drop_own_dest(op, lrnat_rec, lr_lbnat_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); } /* This function adds ARP resolve flows related to a LSP. */ @@ -13522,7 +13541,6 @@ static void build_arp_resolve_flows_for_lsp( struct ovn_port *op, struct hmap *lflows, const struct hmap *lr_ports, - const struct lr_lb_nat_data_table *lr_lbnats, struct ds *match, struct ds *actions) { ovs_assert(op->nbsp); @@ -13663,15 +13681,50 @@ build_arp_resolve_flows_for_lsp( ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); } + } + } +} - if (smap_get(&peer->od->nbr->options, "chassis") - || peer->cr_port) { - const struct lr_lb_nat_data_record *lr_lbnat_rec; - lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, - router_port->od->nbr); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_lbnat_rec, match, actions); - } +static void +build_arp_resolve_flows_for_lsp_routable_addresses( + struct ovn_port *op, struct hmap *lflows, + const struct hmap *lr_ports, + const struct lr_lb_nat_data_table *lr_lbnats, + struct ds *match, struct ds *actions) +{ + if (!lsp_is_router(op->nbsp)) { + return; + } + + struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); + if (!peer || !peer->nbrp) { + return; + } + + if (peer->od->nbr && + smap_get_bool(&peer->od->nbr->options, + "dynamic_neigh_routers", false)) { + return; + } + + for (size_t i = 0; i < op->od->n_router_ports; i++) { + struct ovn_port *router_port = + ovn_port_get_peer(lr_ports, op->od->router_ports[i]); + if (!router_port || !router_port->nbrp) { + continue; + } + + /* Skip the router port under consideration. */ + if (router_port == peer) { + continue; + } + + if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { + const struct lr_lb_nat_data_record *lr_lbnat_rec; + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, + router_port->od->nbr); + routable_addresses_to_lflows(lflows, router_port, peer, + lr_lbnat_rec, match, actions); } } } @@ -13848,7 +13901,6 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct hmap *lflows, - const struct lr_nat_table *lr_nats, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13865,7 +13917,6 @@ build_gateway_redirect_flows_for_lrouter( } const struct ovsdb_idl_row *stage_hint = NULL; - bool add_def_flow = true; if (od->l3dgw_ports[i]->nbrp) { stage_hint = &od->l3dgw_ports[i]->nbrp->header_; @@ -13884,14 +13935,33 @@ build_gateway_redirect_flows_for_lrouter( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), stage_hint); + } - const struct lr_nat_record *lrnat_rec = lr_nat_table_find( - lr_nats, od->nbr); + /* Packets are allowed by default. */ + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); +} - if (!lrnat_rec) { +/* Logical router ingress table GW_REDIRECT: Gateway redirect. */ +static void +build_lr_gateway_redirect_flows_for_nats( + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, + struct hmap *lflows, struct ds *match, struct ds *actions) +{ + ovs_assert(od->nbr); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + if (l3dgw_port_has_associated_vtep_lports(od->l3dgw_ports[i])) { + /* Skip adding redirect lflow for vtep-enabled l3dgw ports. + * Traffic from hypervisor to VTEP (ramp) switch should go in + * distributed manner. Only returning routed traffic must go + * through centralized gateway (or ha-chassis-group). + * This assumes that attached logical switch with vtep lport(s) has + * no localnet port(s) for NAT. Otherwise centralized NAT will not + * work. */ continue; } + bool add_def_flow = true; + for (int j = 0; j < lrnat_rec->n_nat_entries; j++) { const struct ovn_nat *nat = &lrnat_rec->nat_entries[j]; @@ -13900,6 +13970,12 @@ build_gateway_redirect_flows_for_lrouter( continue; } + const struct ovsdb_idl_row *stage_hint = NULL; + + if (od->l3dgw_ports[i]->nbrp) { + stage_hint = &od->l3dgw_ports[i]->nbrp->header_; + } + struct ds match_ext = DS_EMPTY_INITIALIZER; struct nbrec_address_set *as = nat->nb->allowed_ext_ips ? nat->nb->allowed_ext_ips : nat->nb->exempted_ext_ips; @@ -13929,9 +14005,6 @@ build_gateway_redirect_flows_for_lrouter( ds_destroy(&match_ext); } } - - /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); } /* Local router ingress table ARP_REQUEST: ARP request. @@ -14330,8 +14403,8 @@ build_ipv6_input_flows_for_lrouter_port( } static void -build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, - const struct lr_nat_table *lr_nats, +build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, + const struct lr_nat_record *lrnat_rec, struct hmap *lflows, const struct shash *meter_groups) { @@ -14348,10 +14421,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, * port to handle the special cases. In case we get the packet * on a regular port, just reply with the port's ETH address. */ - const struct lr_nat_record *lrnat_rec = lr_nat_table_find( - lr_nats, od->nbr); - ovs_assert(lrnat_rec); - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; @@ -14389,8 +14458,6 @@ build_lrouter_arp_nd_for_datapath(struct ovn_datapath *od, static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct hmap *lflows, - const struct lr_nat_record *lrnat_rec, - const struct lr_lb_nat_data_record *lr_lbnat_rec, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14515,39 +14582,6 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_, lflows); } - if (lr_lbnat_rec && sset_count(&lr_lbnat_rec->lb_ips->ips_v4_reachable)) { - ds_clear(match); - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ARP rule for all IPs that are used as VIPs. */ - char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET); - build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, - REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); - free(lb_ips_v4_as); - } - - if (lr_lbnat_rec && sset_count(&lr_lbnat_rec->lb_ips->ips_v6_reachable)) { - ds_clear(match); - - if (is_l3dgw_port(op)) { - ds_put_format(match, "is_chassis_resident(%s)", - op->cr_port->json_key); - } - - /* Create a single ND rule for all IPs that are used as VIPs. */ - char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, - AF_INET6); - build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, - REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); - free(lb_ips_v6_as); - } - if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { /* UDP/TCP/SCTP port unreachable. */ for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -14622,20 +14656,55 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, &op->nbrp->header_); } } +} - /* Drop IP traffic destined to router owned IPs except if the IP is - * also a SNAT IP. Those are dropped later, in stage - * "lr_in_arp_resolve", if unSNAT was unsuccessful. - * - * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the - * router port is also SNAT IP. - * - * Priority 60. - */ - if (!lrnat_rec->lb_force_snat_router_ip) { - build_lrouter_drop_own_dest(op, lrnat_rec, lr_lbnat_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); +/* Logical router ingress table 3: IP Input for IPv4. */ +static void +build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, + struct hmap *lflows, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct ds *match, const struct shash *meter_groups) +{ + ovs_assert(op->nbrp); + /* No ingress packets are accepted on a chassisredirect + * port, so no need to program flows for that port. */ + if (is_cr_port(op)) { + return; } + + if (sset_count(&lr_lbnat_rec->lb_ips->ips_v4_reachable)) { + ds_clear(match); + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ARP rule for all IPs that are used as VIPs. */ + char *lb_ips_v4_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET); + build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, + REG_INPORT_ETH_ADDR, + match, false, 90, NULL, lflows); + free(lb_ips_v4_as); + } + + if (sset_count(&lr_lbnat_rec->lb_ips->ips_v6_reachable)) { + ds_clear(match); + + if (is_l3dgw_port(op)) { + ds_put_format(match, "is_chassis_resident(%s)", + op->cr_port->json_key); + } + + /* Create a single ND rule for all IPs that are used as VIPs. */ + char *lb_ips_v6_as = lr_lb_address_set_ref(op->od->tunnel_key, + AF_INET6); + build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, + REG_INPORT_ETH_ADDR, match, false, 90, + NULL, lflows, meter_groups); + free(lb_ips_v6_as); + } + /* ARP / ND handling for external IP addresses. * * DNAT and SNAT IP addresses are external IP addresses that need ARP @@ -14649,8 +14718,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, return; } - for (int i = 0; i < lrnat_rec->n_nat_entries; i++) { - struct ovn_nat *nat_entry = &lrnat_rec->nat_entries[i]; + for (int i = 0; i < lr_lbnat_rec->lrnat_rec->n_nat_entries; i++) { + struct ovn_nat *nat_entry = &lr_lbnat_rec->lrnat_rec->nat_entries[i]; /* Skip entries we failed to parse. */ if (!nat_entry_is_valid(nat_entry)) { @@ -14669,7 +14738,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Now handle SNAT entries too, one per unique SNAT IP. */ struct shash_node *snat_snode; - SHASH_FOR_EACH (snat_snode, &lrnat_rec->snat_ips) { + SHASH_FOR_EACH (snat_snode, &lr_lbnat_rec->lrnat_rec->snat_ips) { struct ovn_snat_ip *snat_ip = snat_snode->data; if (ovs_list_is_empty(&snat_ip->snat_entries)) { @@ -14685,7 +14754,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, } static void -build_lrouter_in_unsnat_match(struct ovn_datapath *od, +build_lrouter_in_unsnat_match(const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14712,7 +14781,7 @@ build_lrouter_in_unsnat_match(struct ovn_datapath *od, static void build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14734,7 +14803,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, static void build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14767,7 +14836,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_unsnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14788,7 +14858,8 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_in_dnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14859,7 +14930,8 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_undnat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, @@ -14909,7 +14981,8 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_is_dnat_local(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port) @@ -14939,7 +15012,8 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, struct ovn_datapath *od, } static void -build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_match(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, struct ovn_port *l3dgw_port) @@ -14968,7 +15042,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_out_snat_stateless_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15011,7 +15085,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, static void build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, - struct ovn_datapath *od, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15072,7 +15146,8 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_out_snat_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, @@ -15119,9 +15194,10 @@ build_lrouter_out_snat_flow(struct hmap *lflows, struct ovn_datapath *od, static void build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, const struct nbrec_nat *nat, - struct ovn_datapath *od, bool is_v6, - struct ds *match, struct ds *actions, - int mtu, struct ovn_port *l3dgw_port, + const struct ovn_datapath *od, + bool is_v6, struct ds *match, + struct ds *actions, int mtu, + struct ovn_port *l3dgw_port, const struct shash *meter_groups) { ds_clear(match); @@ -15188,7 +15264,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, +build_lrouter_ingress_flow(struct hmap *lflows, + const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, @@ -15238,7 +15315,8 @@ build_lrouter_ingress_flow(struct hmap *lflows, struct ovn_datapath *od, } static int -lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, +lrouter_check_nat_entry(const struct ovn_datapath *od, + const struct nbrec_nat *nat, const struct hmap *lr_ports, ovs_be32 *mask, bool *is_v6, int *cidr_bits, struct eth_addr *mac, bool *distributed, struct ovn_port **nat_l3dgw_port) @@ -15365,15 +15443,8 @@ lrouter_check_nat_entry(struct ovn_datapath *od, const struct nbrec_nat *nat, } /* NAT, Defrag and load balancing. */ -static void -build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, - const struct hmap *ls_ports, - const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - struct ds *match, - struct ds *actions, - const struct shash *meter_groups, - const struct chassis_features *features) +static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, + struct hmap *lflows) { ovs_assert(od->nbr); @@ -15390,6 +15461,23 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + /* Send the IPv6 NS packets to next table. When ovn-controller + * generates IPv6 NS (for the action - nd_ns{}), the injected + * packet would go through conntrack - which is not required. */ + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); +} + +static void +build_lrouter_nat_defrag_and_lb( + const struct lr_lb_nat_data_record *lr_lbnat_rec, struct hmap *lflows, + const struct hmap *ls_ports, const struct hmap *lr_ports, + struct ds *match, struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + const struct ovn_datapath *od = lr_lbnat_rec->od; + ovs_assert(od->nbr); + const char *ct_flag_reg = features->ct_no_masked_label ? "ct_mark" : "ct_label"; @@ -15467,11 +15555,6 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, "ip && ct.new", "ct_commit { } ; next; "); } - /* Send the IPv6 NS packets to next table. When ovn-controller - * generates IPv6 NS (for the action - nd_ns{}), the injected - * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); - /* NAT rules are only valid on Gateway routers and routers with * l3dgw_ports (router has port(s) with gateway chassis * specified). */ @@ -15480,8 +15563,7 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, } struct sset nat_entries = SSET_INITIALIZER(&nat_entries); - const struct lr_nat_record *lrnat_rec = lr_nat_table_find(lr_nats, - od->nbr); + const struct lr_nat_record *lrnat_rec = lr_lbnat_rec->lrnat_rec; ovs_assert(lrnat_rec); bool dnat_force_snat_ip = @@ -15764,7 +15846,127 @@ build_lrouter_nat_defrag_and_lb(struct ovn_datapath *od, struct hmap *lflows, sset_destroy(&nat_entries); } +static void +build_lsp_lflows_for_lbnats(struct ovn_port *lsp, + struct ovn_port *lrp_peer, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + const struct lr_lb_nat_data_table *lr_lbnats, + const struct hmap *lr_ports, + struct hmap *lflows, + struct ds *match, + struct ds *actions) +{ + ovs_assert(lsp->nbsp); + start_collecting_lflows(); + build_lswitch_rport_arp_req_flows_for_lbnats( + lrp_peer, lr_lbnat_rec, lsp->od, lsp, + lflows, &lsp->nbsp->header_); + build_ip_routing_flows_for_router_type_lsp(lsp, lr_lbnats, + lr_ports, lflows); + build_arp_resolve_flows_for_lsp_routable_addresses( + lsp, lflows, lr_ports, lr_lbnats, match, actions); + build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_lbnat_rec, lflows, + match, actions); + link_ovn_port_to_lflows(lsp, &collected_lflows); + end_collecting_lflows(); +} + +static void +build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, + const struct lr_lb_nat_data_table *lr_lbnats, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + struct hmap *lflows) +{ + ovs_assert(op->nbsp); + + if (!lsp_is_router(op->nbsp) || !op->peer) { + return; + } + + const struct lr_lb_nat_data_record *lr_lbnat_rec; + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, op->peer->od->nbr); + ovs_assert(lr_lbnat_rec); + + build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, + lr_lbnats, lr_ports, lflows, + match, actions); +} + +static void +build_lrp_lflows_for_lbnats(struct ovn_port *op, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + const struct shash *meter_groups, + struct ds *match, struct ds *actions, + struct hmap *lflows) +{ + /* Drop IP traffic destined to router owned IPs except if the IP is + * also a SNAT IP. Those are dropped later, in stage + * "lr_in_arp_resolve", if unSNAT was unsuccessful. + * + * If lrnat_rec->lb_force_snat_router_ip is true, it means the IP of the + * router port is also SNAT IP. + * + * Priority 60. + */ + if (!lr_lbnat_rec->lrnat_rec->lb_force_snat_router_ip) { + build_lrouter_drop_own_dest(op, lr_lbnat_rec, + S_ROUTER_IN_IP_INPUT, 60, false, lflows); + } + + /* Drop IP traffic destined to router owned IPs. Part of it is dropped + * in stage "lr_in_ip_input" but traffic that could have been unSNATed + * but didn't match any existing session might still end up here. + * + * Priority 2. + */ + build_lrouter_drop_own_dest(op, lr_lbnat_rec, + S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + + build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_lbnat_rec, + match, meter_groups); + build_lrouter_force_snat_flows_op(op, lr_lbnat_rec->lrnat_rec, lflows, + match, actions); +} + +static void +build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, + const struct lr_lb_nat_data_table *lr_lbnats, + const struct shash *meter_groups, + struct ds *match, + struct ds *actions, + struct hmap *lflows) +{ + ovs_assert(op->nbrp); + const struct lr_lb_nat_data_record *lr_lbnat_rec; + lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, op->od->nbr); + ovs_assert(lr_lbnat_rec); + + build_lrp_lflows_for_lbnats(op, lr_lbnat_rec, meter_groups, match, + actions, lflows); +} + +static void +build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct hmap *lflows, + const struct hmap *ls_ports, + const struct hmap *lr_ports, + struct ds *match, + struct ds *actions, + const struct shash *meter_groups, + const struct chassis_features *features) +{ + build_lrouter_nat_defrag_and_lb(lr_lbnat_rec, lflows, ls_ports, lr_ports, + match, actions, meter_groups, features); + build_lr_gateway_redirect_flows_for_nats(lr_lbnat_rec->od, + lr_lbnat_rec->lrnat_rec, lflows, + match, actions); + build_lrouter_arp_nd_for_datapath(lr_lbnat_rec->od, + lr_lbnat_rec->lrnat_rec, lflows, + meter_groups); +} struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; @@ -15772,7 +15974,6 @@ struct lswitch_flow_build_info { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_lb_nat_data_table *lr_lbnats; struct hmap *lflows; struct hmap *igmp_groups; @@ -15839,17 +16040,13 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, lsi->meter_groups); - build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, lsi->lr_nats, - &lsi->match, &lsi->actions); + build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, + &lsi->actions); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); - build_lrouter_arp_nd_for_datapath(od, lsi->lr_nats, lsi->lflows, - lsi->meter_groups); - build_lrouter_nat_defrag_and_lb(od, lsi->lflows, lsi->ls_ports, - lsi->lr_ports,lsi->lr_nats, &lsi->match, - &lsi->actions, lsi->meter_groups, - lsi->features); + + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); build_lrouter_lb_affinity_default_flows(od, lsi->lflows); } @@ -15860,8 +16057,6 @@ static void build_lswitch_and_lrouter_iterate_by_lsp( struct ovn_port *op, const struct hmap *ls_ports, const struct hmap *lr_ports, - const struct lr_nat_table *lr_nats, - const struct lr_lb_nat_data_table *lr_lbnats, const struct shash *meter_groups, struct ds *match, struct ds *actions, @@ -15878,14 +16073,11 @@ build_lswitch_and_lrouter_iterate_by_lsp( meter_groups, actions, match); build_lswitch_dhcp_options_and_response(op, lflows, meter_groups); build_lswitch_external_port(op, lflows); - build_lswitch_ip_unicast_lookup(op, lr_nats, lr_lbnats, lflows, actions, + build_lswitch_ip_unicast_lookup(op, lflows, actions, match); /* Build Logical Router Flows. */ - build_ip_routing_flows_for_router_type_lsp(op, lr_lbnats, lr_ports, - lflows); - build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, lr_lbnats, - match, actions); + build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); link_ovn_port_to_lflows(op, &collected_lflows); end_collecting_lflows(); @@ -15900,12 +16092,6 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, { ovs_assert(op->nbrp); - const struct lr_nat_record *lrnet_rec = lr_nat_table_find(lsi->lr_nats, - op->od->nbr); - ovs_assert(lrnet_rec); - - const struct lr_lb_nat_data_record *lr_lbnat_rec = - lr_lb_nat_data_table_find(lsi->lr_lbnats, op->od->nbr); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, @@ -15913,7 +16099,7 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ip_routing_flows_for_lrp(op, lsi->lflows); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_arp_resolve_flows_for_lrp(op, lrnet_rec, lr_lbnat_rec, lsi->lflows, + build_arp_resolve_flows_for_lrp(op, lsi->lflows, &lsi->match, &lsi->actions); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions); @@ -15921,22 +16107,20 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_ipv4_ip_input(op, lsi->lflows, lrnet_rec, lr_lbnat_rec, - &lsi->match, &lsi->actions, lsi->meter_groups); - build_lrouter_force_snat_flows_op(op, lrnet_rec, lsi->lflows, &lsi->match, - &lsi->actions); + build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, + lsi->meter_groups); } static void * build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; + const struct lr_lb_nat_data_record *lr_lbnat_rec; struct lswitch_flow_build_info *lsi; - + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; int bnum; while (!stop_parallel_processing()) { @@ -15983,12 +16167,15 @@ build_lflows_thread(void *arg) } build_lswitch_and_lrouter_iterate_by_lsp(op, lsi->ls_ports, lsi->lr_ports, - lsi->lr_nats, - lsi->lr_lbnats, lsi->meter_groups, &lsi->match, &lsi->actions, lsi->lflows); + build_lbnat_lflows_iterate_by_lsp(op, lsi->lr_lbnats, + lsi->lr_ports, + &lsi->match, + &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -16001,6 +16188,11 @@ build_lflows_thread(void *arg) return NULL; } build_lswitch_and_lrouter_iterate_by_lrp(op, lsi); + build_lbnat_lflows_iterate_by_lrp(op, lsi->lr_lbnats, + lsi->meter_groups, + &lsi->match, + &lsi->actions, + lsi->lflows); } } for (bnum = control->id; @@ -16023,7 +16215,7 @@ build_lflows_thread(void *arg) build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, - lsi->lr_nats, + lsi->lr_lbnats, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions); @@ -16035,6 +16227,23 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions); } } + for (bnum = control->id; + bnum <= lsi->lr_lbnats->entries.mask; + bnum += control->pool->size) + { + LR_LB_NAT_DATA_TABLE_FOR_EACH_IN_P (lr_lbnat_rec, bnum, + lsi->lr_lbnats) { + if (stop_parallel_processing()) { + return NULL; + } + build_lr_lbnat_data_flows(lr_lbnat_rec, + lsi->lflows, lsi->ls_ports, + lsi->lr_ports, &lsi->match, + &lsi->actions, + lsi->meter_groups, + lsi->features); + } + } for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16094,7 +16303,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *ls_ports, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, - const struct lr_nat_table *lr_nats, const struct lr_lb_nat_data_table *lr_lbnats, struct hmap *lflows, struct hmap *igmp_groups, @@ -16125,7 +16333,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].ls_ports = ls_ports; lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; - lsiv[index].lr_nats = lr_nats; lsiv[index].lr_lbnats = lr_lbnats; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; @@ -16151,17 +16358,18 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, } free(lsiv); } else { + const struct lr_lb_nat_data_record *lr_lbnat_rec; + struct ovn_igmp_group *igmp_group; + struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; struct ovn_port *op; - struct ovn_lb_datapaths *lb_dps; - struct ovn_igmp_group *igmp_group; + struct lswitch_flow_build_info lsi = { .ls_datapaths = ls_datapaths, .lr_datapaths = lr_datapaths, .ls_ports = ls_ports, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, - .lr_nats = lr_nats, .lr_lbnats = lr_lbnats, .lflows = lflows, .igmp_groups = igmp_groups, @@ -16190,14 +16398,21 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, HMAP_FOR_EACH (op, key_node, ls_ports) { build_lswitch_and_lrouter_iterate_by_lsp(op, lsi.ls_ports, lsi.lr_ports, - lsi.lr_nats, - lsi.lr_lbnats, lsi.meter_groups, - &lsi.match, &lsi.actions, + &lsi.match, + &lsi.actions, lsi.lflows); + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, lsi.lr_ports, + &lsi.match, &lsi.actions, + lsi.lflows); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); + build_lbnat_lflows_iterate_by_lrp(op, lsi.lr_lbnats, + lsi.meter_groups, + &lsi.match, + &lsi.actions, + lsi.lflows); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16208,7 +16423,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, - lsi.lr_datapaths, lsi.lr_nats, + lsi.lr_datapaths, lsi.lr_lbnats, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, @@ -16217,6 +16432,14 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); + + LR_LB_NAT_DATA_TABLE_FOR_EACH (lr_lbnat_rec, lr_lbnats) { + build_lr_lbnat_data_flows(lr_lbnat_rec, lsi.lflows, + lsi.ls_ports, lsi.lr_ports, &lsi.match, + &lsi.actions, lsi.meter_groups, + lsi.features); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16312,7 +16535,6 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->ls_ports, input_data->lr_ports, input_data->ls_port_groups, - input_data->lr_nats, input_data->lr_lbnats, lflows, &igmp_groups, @@ -16793,11 +17015,22 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->lr_lbnats, lflow_input->meter_groups, &match, &actions, lflows); + + if (lsp_is_router(op->nbsp) && op->peer && op->peer->od->nbr) { + const struct lr_lb_nat_data_record *lr_lbnat_rec; + lr_lbnat_rec = lr_lb_nat_data_table_find(lflow_input->lr_lbnats, + op->peer->od->nbr); + ovs_assert(lr_lbnat_rec); + + build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, + lflow_input->lr_lbnats, + lflow_input->lr_ports, + lflows, &match, &actions); + } + ds_destroy(&match); ds_destroy(&actions); @@ -16832,8 +17065,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_and_lrouter_iterate_by_lsp(op, lflow_input->ls_ports, lflow_input->lr_ports, - lflow_input->lr_nats, - lflow_input->lr_lbnats, lflow_input->meter_groups, &match, &actions, lflows); diff --git a/northd/northd.h b/northd/northd.h index ace9720b95..f7a44652f3 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -172,7 +172,6 @@ struct lflow_input { const struct hmap *ls_ports; const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; - const struct lr_nat_table *lr_nats; const struct lr_lb_nat_data_table *lr_lbnats; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; From patchwork Tue Oct 24 00:48:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854081 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtjQ2kY5z23jl for ; Tue, 24 Oct 2023 11:48:58 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C608842CE3; Tue, 24 Oct 2023 00:48:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org C608842CE3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id So35EQsDy3JA; Tue, 24 Oct 2023 00:48:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 444C642CEB; Tue, 24 Oct 2023 00:48:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 444C642CEB Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D7967C008D; Tue, 24 Oct 2023 00:48:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 13B4DC0DD7 for ; Tue, 24 Oct 2023 00:48:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8086642C9A for ; Tue, 24 Oct 2023 00:48:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8086642C9A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RuYqZByhtOaY for ; Tue, 24 Oct 2023 00:48:41 +0000 (UTC) X-Greylist: delayed 338 seconds by postgrey-1.37 at util1.osuosl.org; Tue, 24 Oct 2023 00:48:39 UTC DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 2C20242CDA Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::224]) by smtp4.osuosl.org (Postfix) with ESMTPS id 2C20242CDA for ; Tue, 24 Oct 2023 00:48:38 +0000 (UTC) Received: by mail.gandi.net (Postfix) with ESMTPSA id 7155CE0002; Tue, 24 Oct 2023 00:48:34 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:48:23 -0400 Message-ID: <20231024004823.4133864-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 08/18] northd: Don't commit dhcp response flows in the conntrack. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This is not required. Signed-off-by: Numan Siddique --- northd/northd.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/northd/northd.c b/northd/northd.c index 2fcb5c9f3a..0e4987edea 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -9223,9 +9223,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, &op->nbsp->dhcpv4_options->options, "lease_time"); ovs_assert(server_id && server_mac && lease_time); const char *dhcp_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip4.src == %s && udp && udp.src == 67 " @@ -9308,9 +9306,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ipv6_string_mapped(server_ip, &lla); const char *dhcp6_actions = - (op->od->has_stateful_acl || op->od->has_lb_vip) - ? REGBIT_ACL_VERDICT_ALLOW" = 1; ct_commit; next;" - : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; + REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ds_clear(&match); ds_put_format(&match, "outport == %s && eth.src == %s " "&& ip6.src == %s && udp && udp.src == 547 " From patchwork Tue Oct 24 00:48:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854082 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtjs6BCgz23jl for ; Tue, 24 Oct 2023 11:49:21 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 139F783CC8; Tue, 24 Oct 2023 00:49:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 139F783CC8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9peSrCLyq8uZ; Tue, 24 Oct 2023 00:49:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 889EC83C8E; Tue, 24 Oct 2023 00:49:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 889EC83C8E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C13FCC0E11; Tue, 24 Oct 2023 00:49:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 314B6C0DEB for ; Tue, 24 Oct 2023 00:49:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D0E34614B2 for ; Tue, 24 Oct 2023 00:48:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D0E34614B2 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YSyUFoBu3P6k for ; Tue, 24 Oct 2023 00:48:53 +0000 (UTC) Received: from relay4-d.mail.gandi.net (relay4-d.mail.gandi.net [217.70.183.196]) by smtp3.osuosl.org (Postfix) with ESMTPS id E5F7C614D7 for ; Tue, 24 Oct 2023 00:48:51 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E5F7C614D7 Received: by mail.gandi.net (Postfix) with ESMTPSA id 19B54E0002; Tue, 24 Oct 2023 00:48:48 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:48:36 -0400 Message-ID: <20231024004836.4133885-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 09/18] northd: Add a new node ls_lbacls. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This new engine now maintains the load balancer and ACL data of a logical switch which was earlier part of northd engine node data. The main inputs to this engine are: - northd node - NB logical switch node - Port group node A record for each logical switch is maintained in the 'ls_lbacls' hmap table and this record stores the below data which was earlier part of 'struct ovn_datapath'. - bool has_stateful_acl; - bool has_lb_vip; - bool has_acls; - uint64_t max_acl_tier; This engine node becomes an input to 'lflow' node. Signed-off-by: Numan Siddique --- lib/stopwatch-names.h | 1 + northd/automake.mk | 2 + northd/en-lflow.c | 4 + northd/en-lr-lb-nat-data.c | 8 +- northd/en-lr-lb-nat-data.h | 2 + northd/en-ls-lb-acls.c | 527 +++++++++++++++++++++++++++++++++++++ northd/en-ls-lb-acls.h | 88 +++++++ northd/en-port-group.h | 3 + northd/inc-proc-northd.c | 9 + northd/northd.c | 271 +++++++++---------- northd/northd.h | 7 +- 11 files changed, 776 insertions(+), 146 deletions(-) create mode 100644 northd/en-ls-lb-acls.c create mode 100644 northd/en-ls-lb-acls.h diff --git a/lib/stopwatch-names.h b/lib/stopwatch-names.h index 7d85acdaea..8b0018a593 100644 --- a/lib/stopwatch-names.h +++ b/lib/stopwatch-names.h @@ -34,5 +34,6 @@ #define SYNC_METERS_RUN_STOPWATCH_NAME "sync_meters_run" #define LR_NAT_RUN_STOPWATCH_NAME "lr_nat_run" #define LR_LB_NAT_DATA_RUN_STOPWATCH_NAME "lr_lb_nat_data" +#define LS_LBACLS_RUN_STOPWATCH_NAME "lr_lb_acls" #endif diff --git a/northd/automake.mk b/northd/automake.mk index 4116c487df..4593654726 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -28,6 +28,8 @@ northd_ovn_northd_SOURCES = \ northd/en-lr-nat.h \ northd/en-lr-lb-nat-data.c \ northd/en-lr-lb-nat-data.h \ + northd/en-ls-lb-acls.c \ + northd/en-ls-lb-acls.h \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 229f4be1d0..648a477916 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -21,6 +21,7 @@ #include "en-lflow.h" #include "en-lr-nat.h" #include "en-lr-lb-nat-data.h" +#include "en-ls-lb-acls.h" #include "en-northd.h" #include "en-meters.h" @@ -44,6 +45,8 @@ lflow_get_input_data(struct engine_node *node, engine_get_input_data("sync_meters", node); struct ed_type_lr_lb_nat_data *lr_lb_nat_data = engine_get_input_data("lr_lb_nat_data", node); + struct ed_type_ls_lbacls *ls_lbacls_data = + engine_get_input_data("ls_lbacls", node); lflow_input->nbrec_bfd_table = EN_OVSDB_GET(engine_get_input("NB_bfd", node)); @@ -67,6 +70,7 @@ lflow_get_input_data(struct engine_node *node, lflow_input->lr_ports = &northd_data->lr_ports; lflow_input->ls_port_groups = &pg_data->ls_port_groups; lflow_input->lr_lbnats = &lr_lb_nat_data->lr_lbnats; + lflow_input->ls_lbacls = &ls_lbacls_data->ls_lbacls; lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; diff --git a/northd/en-lr-lb-nat-data.c b/northd/en-lr-lb-nat-data.c index 1a3271cc25..7987799ca2 100644 --- a/northd/en-lr-lb-nat-data.c +++ b/northd/en-lr-lb-nat-data.c @@ -302,9 +302,11 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) if (!hmapx_is_empty(&data->tracked_data.crupdated)) { struct hmapx_node *hmapx_node; /* For all the modified lr_lb_nat_data records (re)build the - * vip nats. */ + * vip nats and re-evaluate 'has_lb_vip'. */ HMAPX_FOR_EACH (hmapx_node, &data->tracked_data.crupdated) { - lr_lb_nat_data_build_vip_nats(hmapx_node->data); + lr_lbnat_rec = hmapx_node->data; + lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + lr_lbnat_rec->has_lb_vip = od_has_lb_vip(lr_lbnat_rec->od); } data->tracked = true; @@ -504,6 +506,8 @@ lr_lb_nat_data_record_init(struct lr_lb_nat_data_record *lr_lbnat_rec, if (!nbr->n_nat) { lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); } + + lr_lbnat_rec->has_lb_vip = od_has_lb_vip(lr_lbnat_rec->od); } static struct lr_lb_nat_data_input diff --git a/northd/en-lr-lb-nat-data.h b/northd/en-lr-lb-nat-data.h index 653aa6cf89..759d055f05 100644 --- a/northd/en-lr-lb-nat-data.h +++ b/northd/en-lr-lb-nat-data.h @@ -39,6 +39,8 @@ struct lr_lb_nat_data_record { const struct ovn_datapath *od; const struct lr_nat_record *lrnat_rec; + bool has_lb_vip; + /* Load Balancer vIPs relevant for this datapath. */ struct ovn_lb_ip_set *lb_ips; diff --git a/northd/en-ls-lb-acls.c b/northd/en-ls-lb-acls.c new file mode 100644 index 0000000000..1ba7ecb3e2 --- /dev/null +++ b/northd/en-ls-lb-acls.c @@ -0,0 +1,527 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "lib/bitmap.h" +#include "lib/socket-util.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "en-lb-data.h" +#include "en-ls-lb-acls.h" +#include "en-port-group.h" +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(en_ls_lbacls); + +/* Static function declarations. */ +static void ls_lbacls_table_init(struct ls_lbacls_table *); +static void ls_lbacls_table_clear(struct ls_lbacls_table *); +static void ls_lbacls_table_destroy(struct ls_lbacls_table *); +static struct ls_lbacls_record *ls_lbacls_table_find_( + const struct ls_lbacls_table *, const struct nbrec_logical_switch *); +static void ls_lbacls_table_build(struct ls_lbacls_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *); + +static struct ls_lbacls_input ls_lbacls_get_input_data( + struct engine_node *); + +static struct ls_lbacls_record *ls_lbacls_record_create( + struct ls_lbacls_table *, + const struct ovn_datapath *, + const struct ls_port_group_table *); +static void ls_lbacls_record_destroy(struct ls_lbacls_record *); +static void ls_lbacls_record_init( + struct ls_lbacls_record *, + const struct ovn_datapath *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static void ls_lbacls_record_reinit( + struct ls_lbacls_record *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static bool ls_has_lb_vip(const struct ovn_datapath *); +static void ls_lbacls_record_set_acl_flags(struct ls_lbacls_record *, + const struct ovn_datapath *, + const struct ls_port_group *, + const struct ls_port_group_table *); +static bool ls_lbacls_record_set_acl_flags_(struct ls_lbacls_record *, + struct nbrec_acl **, + size_t n_acls); +static struct ls_lbacls_input ls_lbacls_get_input_data(struct engine_node *); +static bool is_ls_acls_changed(const struct nbrec_logical_switch *); +static bool is_acls_seqno_changed(struct nbrec_acl **, size_t n_nb_acls); + + +/* public functions. */ +const struct ls_lbacls_record * +ls_lbacls_table_find( + const struct ls_lbacls_table *table, + const struct nbrec_logical_switch *nbs) +{ + return ls_lbacls_table_find_(table, nbs); +} + +void * +en_ls_lbacls_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *arg OVS_UNUSED) +{ + struct ed_type_ls_lbacls *data = xzalloc(sizeof *data); + ls_lbacls_table_init(&data->ls_lbacls); + hmapx_init(&data->tracked_data.crupdated); + hmapx_init(&data->tracked_data.deleted); + return data; +} + +void +en_ls_lbacls_cleanup(void *data_) +{ + struct ed_type_ls_lbacls *data = + (struct ed_type_ls_lbacls *) data_; + ls_lbacls_table_destroy(&data->ls_lbacls); + hmapx_destroy(&data->tracked_data.crupdated); + hmapx_destroy(&data->tracked_data.deleted); +} + +void +en_ls_lbacls_clear_tracked_data(void *data_) +{ + struct ed_type_ls_lbacls *data = + (struct ed_type_ls_lbacls *) data_; + + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH_SAFE (hmapx_node, &data->tracked_data.deleted) { + ls_lbacls_record_destroy(hmapx_node->data); + hmapx_delete(&data->tracked_data.deleted, hmapx_node); + } + + hmapx_clear(&data->tracked_data.crupdated); + data->tracked = false; +} + +void +en_ls_lbacls_run(struct engine_node *node, void *data_) +{ + struct ls_lbacls_input input_data = ls_lbacls_get_input_data(node); + struct ed_type_ls_lbacls *data = data_; + + stopwatch_start(LS_LBACLS_RUN_STOPWATCH_NAME, time_msec()); + + ls_lbacls_table_clear(&data->ls_lbacls); + ls_lbacls_table_build(&data->ls_lbacls, input_data.ls_datapaths, + input_data.ls_port_groups); + + data->tracked = false; + stopwatch_stop(LS_LBACLS_RUN_STOPWATCH_NAME, time_msec()); + engine_set_node_state(node, EN_UPDATED); +} + +/* Handler functions. */ +bool +ls_lbacls_northd_handler(struct engine_node *node, void *data_) +{ + struct northd_data *northd_data = engine_get_input_data("northd", node); + if (!northd_data->change_tracked) { + return false; + } + + struct northd_tracked_data *nd_changes = &northd_data->trk_northd_changes; + struct ls_lbacls_input input_data = ls_lbacls_get_input_data(node); + struct ls_lbacls_record *ls_lbacls_rec; + struct ed_type_ls_lbacls *data = data_; + const struct ovn_datapath *od; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &nd_changes->ls_with_changed_lbs.crupdated) { + od = hmapx_node->data; + + ls_lbacls_rec = ls_lbacls_table_find_(&data->ls_lbacls, od->nbs); + if (!ls_lbacls_rec) { + ls_lbacls_rec = ls_lbacls_record_create(&data->ls_lbacls, od, + input_data.ls_port_groups); + } else { + ls_lbacls_record_reinit(ls_lbacls_rec, NULL, + input_data.ls_port_groups); + } + + /* Add the ls_lbacls_rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, ls_lbacls_rec); + } + + if (!hmapx_is_empty(&data->tracked_data.crupdated)) { + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +ls_lbacls_port_group_handler(struct engine_node *node, void *data_) +{ + struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + if (pg_data->ls_port_groups_sets_changed) { + return false; + } + + /* port_group engine node doesn't provide the tracking data yet. + * Loop through all the ls port groups and update the ls_lbacls_rec. + * This is still better than returning false. */ + struct ls_lbacls_input input_data = ls_lbacls_get_input_data(node); + struct ed_type_ls_lbacls *data = data_; + const struct ls_port_group *ls_pg; + + LS_PORT_GROUP_TABLE_FOR_EACH (ls_pg, input_data.ls_port_groups) { + struct ls_lbacls_record *ls_lbacls_rec = + ls_lbacls_table_find_(&data->ls_lbacls, ls_pg->nbs); + + bool modified = false; + if (!ls_lbacls_rec) { + const struct ovn_datapath *od; + od = ovn_datapath_find(&input_data.ls_datapaths->datapaths, + &ls_pg->nbs->header_.uuid); + ovs_assert(od); + ls_lbacls_rec = ls_lbacls_record_create(&data->ls_lbacls, od, + input_data.ls_port_groups); + modified = true; + } else { + bool had_stateful_acl = ls_lbacls_rec->has_stateful_acl; + uint64_t max_acl_tier = ls_lbacls_rec->max_acl_tier; + bool had_acls = ls_lbacls_rec->has_acls; + + ls_lbacls_record_reinit(ls_lbacls_rec, ls_pg, + input_data.ls_port_groups); + + if ((had_stateful_acl != ls_lbacls_rec->has_stateful_acl) + || (had_acls != ls_lbacls_rec->has_acls) + || max_acl_tier != ls_lbacls_rec->max_acl_tier) { + modified = true; + } + } + + if (modified) { + /* Add the ls_lbacls_rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, ls_lbacls_rec); + } + } + + if (!hmapx_is_empty(&data->tracked_data.crupdated)) { + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +bool +ls_lbacls_logical_switch_handler(struct engine_node *node, void *data_) +{ + struct ls_lbacls_input input_data = ls_lbacls_get_input_data(node); + const struct nbrec_logical_switch *nbs; + struct ed_type_ls_lbacls *data = data_; + + NBREC_LOGICAL_SWITCH_TABLE_FOR_EACH_TRACKED (nbs, + input_data.nbrec_logical_switch_table) { + if (!is_ls_acls_changed(nbs)) { + continue; + } + + struct ls_lbacls_record *ls_lbacls_rec = + ls_lbacls_table_find_(&data->ls_lbacls, nbs); + + if (nbrec_logical_switch_is_deleted(nbs)) { + if (ls_lbacls_rec) { + /* Remove the record from the entries. */ + hmap_remove(&data->ls_lbacls.entries, + &ls_lbacls_rec->key_node); + + /* Add the ls_lbacls_rec to the tracking data. */ + hmapx_add(&data->tracked_data.deleted, ls_lbacls_rec); + } + } else { + if (!ls_lbacls_rec) { + const struct ovn_datapath *od; + od = ovn_datapath_find(&input_data.ls_datapaths->datapaths, + &nbs->header_.uuid); + ovs_assert(od); + ls_lbacls_rec = ls_lbacls_record_create(&data->ls_lbacls, od, + input_data.ls_port_groups); + } else { + ls_lbacls_record_reinit(ls_lbacls_rec, NULL, + input_data.ls_port_groups); + } + + /* Add the ls_lbacls_rec to the tracking data. */ + hmapx_add(&data->tracked_data.crupdated, ls_lbacls_rec); + } + } + + if (!hmapx_is_empty(&data->tracked_data.crupdated) + || !hmapx_is_empty(&data->tracked_data.deleted)) { + data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* static functions. */ +static void +ls_lbacls_table_init(struct ls_lbacls_table *table) +{ + *table = (struct ls_lbacls_table) { + .entries = HMAP_INITIALIZER(&table->entries), + }; +} + +static void +ls_lbacls_table_destroy(struct ls_lbacls_table *table) +{ + ls_lbacls_table_clear(table); + hmap_destroy(&table->entries); +} + +static void +ls_lbacls_table_clear(struct ls_lbacls_table *table) +{ + struct ls_lbacls_record *ls_lbacls_rec; + HMAP_FOR_EACH_POP (ls_lbacls_rec, key_node, &table->entries) { + ls_lbacls_record_destroy(ls_lbacls_rec); + } +} + +static void +ls_lbacls_table_build(struct ls_lbacls_table *table, + const struct ovn_datapaths *ls_datapaths, + const struct ls_port_group_table *ls_pgs) +{ + const struct ovn_datapath *od; + HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { + ls_lbacls_record_create(table, od, ls_pgs); + } +} + +struct ls_lbacls_record * +ls_lbacls_table_find_(const struct ls_lbacls_table *table, + const struct nbrec_logical_switch *nbs) +{ + struct ls_lbacls_record *rec; + + HMAP_FOR_EACH_WITH_HASH (rec, key_node, + uuid_hash(&nbs->header_.uuid), &table->entries) { + if (nbs == rec->od->nbs) { + return rec; + } + } + return NULL; +} + +static struct ls_lbacls_record * +ls_lbacls_record_create(struct ls_lbacls_table *table, + const struct ovn_datapath *od, + const struct ls_port_group_table *ls_pgs) +{ + struct ls_lbacls_record *ls_lbacls_rec = xzalloc(sizeof *ls_lbacls_rec); + ls_lbacls_rec->od = od; + ls_lbacls_record_init(ls_lbacls_rec, od, NULL, ls_pgs); + + hmap_insert(&table->entries, &ls_lbacls_rec->key_node, + uuid_hash(&ls_lbacls_rec->od->nbs->header_.uuid)); + + return ls_lbacls_rec; +} + +static void +ls_lbacls_record_destroy(struct ls_lbacls_record *ls_lbacls_rec) +{ + free(ls_lbacls_rec); +} + +static void +ls_lbacls_record_init(struct ls_lbacls_record *ls_lbacls_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_lbacls_rec->has_lb_vip = ls_has_lb_vip(od); + ls_lbacls_record_set_acl_flags(ls_lbacls_rec, od, ls_pg, ls_pgs); +} + +static void +ls_lbacls_record_reinit(struct ls_lbacls_record *ls_lbacls_rec, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_lbacls_record_init(ls_lbacls_rec, ls_lbacls_rec->od, ls_pg, ls_pgs); +} + +static bool +lb_has_vip(const struct nbrec_load_balancer *lb) +{ + return !smap_is_empty(&lb->vips); +} + +static bool +lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) +{ + for (size_t i = 0; i < lb_group->n_load_balancer; i++) { + if (lb_has_vip(lb_group->load_balancer[i])) { + return true; + } + } + return false; +} + +static bool +ls_has_lb_vip(const struct ovn_datapath *od) +{ + for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { + if (lb_has_vip(od->nbs->load_balancer[i])) { + return true; + } + } + + for (size_t i = 0; i < od->nbs->n_load_balancer_group; i++) { + if (lb_group_has_vip(od->nbs->load_balancer_group[i])) { + return true; + } + } + return false; +} + +static void +ls_lbacls_record_set_acl_flags(struct ls_lbacls_record *ls_lbacls_rec, + const struct ovn_datapath *od, + const struct ls_port_group *ls_pg, + const struct ls_port_group_table *ls_pgs) +{ + ls_lbacls_rec->has_stateful_acl = false; + ls_lbacls_rec->max_acl_tier = 0; + ls_lbacls_rec->has_acls = false; + + if (ls_lbacls_record_set_acl_flags_(ls_lbacls_rec, od->nbs->acls, + od->nbs->n_acls)) { + return; + } + + if (!ls_pg) { + ls_pg = ls_port_group_table_find(ls_pgs, od->nbs); + } + + if (!ls_pg) { + return; + } + + const struct ls_port_group_record *ls_pg_rec; + HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { + if (ls_lbacls_record_set_acl_flags_(ls_lbacls_rec, + ls_pg_rec->nb_pg->acls, + ls_pg_rec->nb_pg->n_acls)) { + return; + } + } +} + +static bool +ls_lbacls_record_set_acl_flags_(struct ls_lbacls_record *ls_lbacls_rec, + struct nbrec_acl **acls, + size_t n_acls) +{ + /* A true return indicates that there are no possible ACL flags + * left to set on ls_lbacls record. A false return indicates that + * further ACLs should be explored in case more flags need to be + * set on ls_lbacls record. + */ + if (!n_acls) { + return false; + } + + ls_lbacls_rec->has_acls = true; + for (size_t i = 0; i < n_acls; i++) { + const struct nbrec_acl *acl = acls[i]; + if (acl->tier > ls_lbacls_rec->max_acl_tier) { + ls_lbacls_rec->max_acl_tier = acl->tier; + } + if (!ls_lbacls_rec->has_stateful_acl + && !strcmp(acl->action, "allow-related")) { + ls_lbacls_rec->has_stateful_acl = true; + } + if (ls_lbacls_rec->has_stateful_acl && + ls_lbacls_rec->max_acl_tier == + nbrec_acl_col_tier.type.value.integer.max) { + return true; + } + } + + return false; +} + +static struct ls_lbacls_input +ls_lbacls_get_input_data(struct engine_node *node) +{ + const struct northd_data *northd_data = + engine_get_input_data("northd", node); + const struct port_group_data *pg_data = + engine_get_input_data("port_group", node); + + return (struct ls_lbacls_input) { + .nbrec_logical_switch_table = + EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)), + .ls_port_groups = &pg_data->ls_port_groups, + .ls_datapaths = &northd_data->ls_datapaths, + }; +} + +static bool +is_acls_seqno_changed(struct nbrec_acl **nb_acls, size_t n_nb_acls) +{ + for (size_t i = 0; i < n_nb_acls; i++) { + if (nbrec_acl_row_get_seqno(nb_acls[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return true; + } + } + + return false; +} + +static bool +is_ls_acls_changed(const struct nbrec_logical_switch *nbs) { + return (nbrec_logical_switch_is_new(nbs) + || nbrec_logical_switch_is_deleted(nbs) + || nbrec_logical_switch_is_updated(nbs, + NBREC_LOGICAL_SWITCH_COL_ACLS) + || is_acls_seqno_changed(nbs->acls, nbs->n_acls)); +} diff --git a/northd/en-ls-lb-acls.h b/northd/en-ls-lb-acls.h new file mode 100644 index 0000000000..ccb75e40e8 --- /dev/null +++ b/northd/en-ls-lb-acls.h @@ -0,0 +1,88 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef EN_LS_LB_ACL_H +#define EN_LS_LB_ACL_H 1 + +#include + +/* OVS includes. */ +#include "lib/hmapx.h" +#include "openvswitch/hmap.h" +#include "sset.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" +#include "lib/lb.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "lib/ovn-util.h" +#include "lib/stopwatch-names.h" + +struct ls_lbacls_record { + struct hmap_node key_node; + + const struct ovn_datapath *od; + bool has_stateful_acl; + bool has_lb_vip; + bool has_acls; + uint64_t max_acl_tier; +}; + +struct ls_lbacls_table { + struct hmap entries; +}; + +#define LS_LBACLS_TABLE_FOR_EACH(LS_LBACLS_REC, TABLE) \ + HMAP_FOR_EACH (LS_LBACLS_REC, key_node, &(TABLE)->entries) + +#define LS_LBACLS_TABLE_FOR_EACH_IN_P(LS_LBACLS_REC, JOBID, TABLE) \ + HMAP_FOR_EACH_IN_PARALLEL (LS_LBACLS_REC, key_node, JOBID, \ + &(TABLE)->entries) + +struct ls_lbacls_tracked_data { + /* Created or updated logical switch with LB and ACL data. */ + struct hmapx crupdated; /* Stores 'struct ls_lbacls_record'. */ + + /* Deleted logical switch with LB and ACL data. */ + struct hmapx deleted; /* Stores 'struct ls_lbacls_record'. */ +}; + +struct ed_type_ls_lbacls { + struct ls_lbacls_table ls_lbacls; + + bool tracked; + struct ls_lbacls_tracked_data tracked_data; +}; + +struct ls_lbacls_input { + const struct nbrec_logical_switch_table *nbrec_logical_switch_table; + const struct ls_port_group_table *ls_port_groups; + const struct ovn_datapaths *ls_datapaths; +}; + +void *en_ls_lbacls_init(struct engine_node *, struct engine_arg *); +void en_ls_lbacls_cleanup(void *data); +void en_ls_lbacls_clear_tracked_data(void *data); +void en_ls_lbacls_run(struct engine_node *, void *data); + +bool ls_lbacls_northd_handler(struct engine_node *, void *data); +bool ls_lbacls_port_group_handler(struct engine_node *, void *data); +bool ls_lbacls_logical_switch_handler(struct engine_node *, void *data); + +const struct ls_lbacls_record *ls_lbacls_table_find( + const struct ls_lbacls_table *, const struct nbrec_logical_switch *); + +#endif /* EN_LS_LB_ACL_H */ diff --git a/northd/en-port-group.h b/northd/en-port-group.h index 3b28a23694..54014062ce 100644 --- a/northd/en-port-group.h +++ b/northd/en-port-group.h @@ -48,6 +48,9 @@ struct ls_port_group_record { struct sset ports; /* Subset of 'nb_pg' ports in this record. */ }; +#define LS_PORT_GROUP_TABLE_FOR_EACH(LS_PG, TABLE) \ + HMAP_FOR_EACH (LS_PG, key_node, &(TABLE)->entries) + void ls_port_group_table_init(struct ls_port_group_table *); void ls_port_group_table_clear(struct ls_port_group_table *); void ls_port_group_table_destroy(struct ls_port_group_table *); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 84627070a8..ab4af92aeb 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -33,6 +33,7 @@ #include "en-lb-data.h" #include "en-lr-lb-nat-data.h" #include "en-lr-nat.h" +#include "en-ls-lb-acls.h" #include "en-northd.h" #include "en-lflow.h" #include "en-northd-output.h" @@ -150,6 +151,7 @@ static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_lb_nat_data, "lr_lb_nat_data"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(ls_lbacls, "ls_lbacls"); void inc_proc_northd_init(struct ovsdb_idl_loop *nb, struct ovsdb_idl_loop *sb) @@ -205,6 +207,12 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lr_lb_nat_data, &en_lb_data, lr_lb_nat_data_lb_data_handler); + engine_add_input(&en_ls_lbacls, &en_northd, ls_lbacls_northd_handler); + engine_add_input(&en_ls_lbacls, &en_port_group, + ls_lbacls_port_group_handler); + engine_add_input(&en_ls_lbacls, &en_nb_logical_switch, + ls_lbacls_logical_switch_handler); + engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); @@ -229,6 +237,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); + engine_add_input(&en_lflow, &en_ls_lbacls, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 0e4987edea..7f0cbad0fa 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -45,6 +45,7 @@ #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-lb-nat-data.h" +#include "en-ls-lb-acls.h" #include "lib/ovn-parallel-hmap.h" #include "ovn/actions.h" #include "ovn/features.h" @@ -575,7 +576,7 @@ lb_group_has_vip(const struct nbrec_load_balancer_group *lb_group) } static bool -ls_has_lb_vip(struct ovn_datapath *od) +ls_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbs->n_load_balancer; i++) { if (lb_has_vip(od->nbs->load_balancer[i])) { @@ -592,7 +593,7 @@ ls_has_lb_vip(struct ovn_datapath *od) } static bool -lr_has_lb_vip(struct ovn_datapath *od) +lr_has_lb_vip(const struct ovn_datapath *od) { for (size_t i = 0; i < od->nbr->n_load_balancer; i++) { if (lb_has_vip(od->nbr->load_balancer[i])) { @@ -608,13 +609,13 @@ lr_has_lb_vip(struct ovn_datapath *od) return false; } -static void -init_lb_for_datapath(struct ovn_datapath *od) +bool +od_has_lb_vip(const struct ovn_datapath *od) { if (od->nbs) { - od->has_lb_vip = ls_has_lb_vip(od); + return ls_has_lb_vip(od); } else { - od->has_lb_vip = lr_has_lb_vip(od); + return lr_has_lb_vip(od); } } @@ -1058,7 +1059,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, init_ipam_info_for_datapath(od); init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); } const struct nbrec_logical_router *nbr; @@ -1089,7 +1089,6 @@ join_datapaths(const struct nbrec_logical_switch_table *nbrec_ls_table, ovs_list_push_back(nb_only, &od->list); } init_mcast_info_for_datapath(od); - init_lb_for_datapath(od); if (smap_get(&od->nbr->options, "chassis")) { od->is_gw_router = true; } @@ -2570,7 +2569,8 @@ get_nat_addresses(const struct ovn_port *op, size_t *n, bool routable_only, size_t n_nats = 0; struct eth_addr mac; if (!op || !op->nbrp || !op->od || !op->od->nbr - || (!op->od->nbr->n_nat && !op->od->has_lb_vip) + || (!op->od->nbr->n_nat && (!lr_lbnat_rec + || !lr_lbnat_rec->has_lb_vip)) || !eth_addr_from_string(op->nbrp->mac, &mac)) { *n = n_nats; return NULL; @@ -3817,7 +3817,7 @@ build_lrouter_lbs_check(const struct ovn_datapaths *lr_datapaths) HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { ovs_assert(od->nbr); - if (od->has_lb_vip && od->n_l3dgw_ports > 1 + if (od_has_lb_vip(od) && od->n_l3dgw_ports > 1 && !smap_get(&od->nbr->options, "chassis")) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); VLOG_WARN_RL(&rl, "Load-balancers are configured on logical " @@ -5441,7 +5441,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - init_lb_for_datapath(od); /* Add the ls datapath to the northd tracked data. */ hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); @@ -5524,9 +5523,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - /* Add the ls datapath to the northd tracked data. */ hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } @@ -5564,9 +5560,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, } } - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - /* Add the lr datapath to the northd tracked data. */ hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } @@ -5581,8 +5574,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(ls_datapaths), lb_dps->nb_ls_map) { od = ls_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); /* Add the ls datapath to the northd tracked data. */ hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); @@ -5591,8 +5582,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, BITMAP_FOR_EACH_1 (index, ods_size(lr_datapaths), lb_dps->nb_lr_map) { od = lr_datapaths->array[index]; - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); /* Add the lr datapath to the northd tracked data. */ hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); @@ -5618,9 +5607,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, od = lbgrp_dps->lr[i]; ovn_lb_datapaths_add_lr(lb_dps, 1, &od); - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - /* Add the lr datapath to the northd tracked data. */ hmapx_add(&nd_changes->lr_with_changed_lbs.crupdated, od); } @@ -5629,9 +5615,6 @@ northd_handle_lb_data_changes(struct tracked_lb_data *trk_lb_data, od = lbgrp_dps->ls[i]; ovn_lb_datapaths_add_ls(lb_dps, 1, &od); - /* Re-evaluate 'od->has_lb_vip' */ - init_lb_for_datapath(od); - /* Add the ls datapath to the northd tracked data. */ hmapx_add(&nd_changes->ls_with_changed_lbs.crupdated, od); } @@ -6573,63 +6556,6 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, return true; } -static bool -od_set_acl_flags(struct ovn_datapath *od, struct nbrec_acl **acls, - size_t n_acls) -{ - /* A true return indicates that there are no possible ACL flags - * left to set on od. A false return indicates that further ACLs - * should be explored in case more flags need to be set on od - */ - if (!n_acls) { - return false; - } - - od->has_acls = true; - for (size_t i = 0; i < n_acls; i++) { - const struct nbrec_acl *acl = acls[i]; - if (acl->tier > od->max_acl_tier) { - od->max_acl_tier = acl->tier; - } - if (!od->has_stateful_acl && !strcmp(acl->action, "allow-related")) { - od->has_stateful_acl = true; - } - if (od->has_stateful_acl && - od->max_acl_tier == nbrec_acl_col_tier.type.value.integer.max) { - return true; - } - } - - return false; -} - -static void -ls_get_acl_flags(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups) -{ - od->has_acls = false; - od->has_stateful_acl = false; - od->max_acl_tier = 0; - - if (od_set_acl_flags(od, od->nbs->acls, od->nbs->n_acls)) { - return; - } - - const struct ls_port_group *ls_pg = - ls_port_group_table_find(ls_port_groups, od->nbs); - if (!ls_pg) { - return; - } - - const struct ls_port_group_record *ls_pg_rec; - HMAP_FOR_EACH (ls_pg_rec, key_node, &ls_pg->nb_pgs) { - if (od_set_acl_flags(od, ls_pg_rec->nb_pg->acls, - ls_pg_rec->nb_pg->n_acls)) { - return; - } - } -} - /* Adds the logical flows in the (in/out) check port sec stage only if * - the lport is disabled or * - lport is of type vtep - to skip the ingress pipeline. @@ -6774,9 +6700,10 @@ build_lswitch_output_port_sec_od(struct ovn_datapath *od, } static void -skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, - enum ovn_stage in_stage, enum ovn_stage out_stage, - uint16_t priority, struct hmap *lflows) +skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, + bool has_stateful_acl, enum ovn_stage in_stage, + enum ovn_stage out_stage, uint16_t priority, + struct hmap *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6789,7 +6716,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, * conntrack state across all chassis. */ const char *ingress_action = "next;"; - const char *egress_action = od->has_stateful_acl + const char *egress_action = has_stateful_acl ? "next;" : "ct_clear; next;"; @@ -6808,7 +6735,7 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct ovn_port *op, } static void -build_stateless_filter(struct ovn_datapath *od, +build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, struct hmap *lflows) { @@ -6829,7 +6756,7 @@ build_stateless_filter(struct ovn_datapath *od, } static void -build_stateless_filters(struct ovn_datapath *od, +build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, struct hmap *lflows) { @@ -6859,9 +6786,7 @@ build_stateless_filters(struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6873,18 +6798,26 @@ build_pre_acls(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.src == $svc_monitor_mac", "next;"); +} + +static void +build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, + const struct ls_port_group_table *ls_port_groups, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_lbacls_rec->od; /* If there are any stateful ACL rules in this datapath, we may * send IP packets for some (allow) filters through the conntrack action, * which handles defragmentation, in order to match L4 headers. */ - if (od->has_stateful_acl) { + if (ls_lbacls_rec->has_stateful_acl) { for (size_t i = 0; i < od->n_router_ports; i++) { - skip_port_from_conntrack(od, od->router_ports[i], + skip_port_from_conntrack(od, od->router_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); } for (size_t i = 0; i < od->n_localnet_ports; i++) { - skip_port_from_conntrack(od, od->localnet_ports[i], + skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, 110, lflows); @@ -6922,7 +6855,7 @@ build_pre_acls(struct ovn_datapath *od, REGBIT_CONNTRACK_DEFRAG" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", REGBIT_CONNTRACK_DEFRAG" = 1; next;"); - } else if (od->has_lb_vip) { + } else if (ls_lbacls_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ build_stateless_filters(od, ls_port_groups, lflows); @@ -7050,30 +6983,40 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + /* Do not send statless flows via conntrack */ + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, + REGBIT_ACL_STATELESS" == 1", "next;"); +} + +static void +build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, + struct hmap *lflows) +{ + const struct ovn_datapath *od = ls_lbacls_rec->od; + for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], + ls_lbacls_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } + /* Localnet ports have no need for going through conntrack, unless * the logical switch has a load balancer. Then, conntrack is necessary * so that traffic arriving via the localnet port can be load * balanced. */ - if (!od->has_lb_vip) { + if (!ls_lbacls_rec->has_lb_vip) { for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], + ls_lbacls_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, 110, lflows); } } - /* Do not sent statless flows via conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); - /* 'REGBIT_CONNTRACK_NAT' is set to let the pre-stateful table send * packet to conntrack for defragmentation and possibly for unNATting. * @@ -7104,7 +7047,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, * ingress pipeline if a load balancer is configured. We can now * add a lflow to drop ct.inv packets. */ - if (od->has_lb_vip) { + if (ls_lbacls_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, @@ -7145,10 +7088,12 @@ build_pre_stateful(struct ovn_datapath *od, } static void -build_acl_hints(struct ovn_datapath *od, +build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, const struct chassis_features *features, struct hmap *lflows) { + const struct ovn_datapath *od = ls_lbacls_rec->od; + /* This stage builds hints for the IN/OUT_ACL stage. Based on various * combinations of ct flags packets may hit only a subset of the logical * flows in the IN/OUT_ACL stage. @@ -7172,13 +7117,13 @@ build_acl_hints(struct ovn_datapath *od, const char *match; /* In any case, advance to the next stage. */ - if (!od->has_acls && !od->has_lb_vip) { + if (!ls_lbacls_rec->has_acls && !ls_lbacls_rec->has_lb_vip) { ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); } else { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); } - if (!od->has_stateful_acl && !od->has_lb_vip) { + if (!ls_lbacls_rec->has_stateful_acl && !ls_lbacls_rec->has_lb_vip) { continue; } @@ -7314,10 +7259,10 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, struct ovn_datapath *od, +consider_acl(struct hmap *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -7354,7 +7299,7 @@ consider_acl(struct hmap *lflows, struct ovn_datapath *od, /* All ACLS will start by matching on their respective tier. */ size_t match_tier_len = 0; ds_clear(match); - if (od->max_acl_tier) { + if (max_acl_tier) { ds_put_format(match, REG_ACL_TIER " == %"PRId64" && ", acl->tier); match_tier_len = match->length; } @@ -7543,12 +7488,15 @@ ovn_update_ipv6_options(struct hmap *lr_ports) #define IPV6_CT_OMIT_MATCH "nd || nd_ra || nd_rs || mldv1 || mldv2" static void -build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, + struct hmap *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, struct ds *actions) { + const struct ovn_datapath *od = ls_lbacls_rec->od; + enum ovn_stage stages [] = { S_SWITCH_IN_ACL_ACTION, S_SWITCH_IN_ACL_AFTER_LB_ACTION, @@ -7559,7 +7507,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ds_put_cstr(actions, REGBIT_ACL_VERDICT_ALLOW " = 0; " REGBIT_ACL_VERDICT_DROP " = 0; " REGBIT_ACL_VERDICT_REJECT " = 0; "); - if (od->max_acl_tier) { + if (ls_lbacls_rec->max_acl_tier) { ds_put_cstr(actions, REG_ACL_TIER " = 0; "); } @@ -7567,7 +7515,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; - if (!od->has_acls) { + if (!ls_lbacls_rec->has_acls) { ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); continue; } @@ -7602,7 +7550,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); struct ds tier_actions = DS_EMPTY_INITIALIZER; - for (size_t j = 0; j < od->max_acl_tier; j++) { + for (size_t j = 0; j < ls_lbacls_rec->max_acl_tier; j++) { ds_clear(match); ds_put_format(match, REG_ACL_TIER " == %"PRIuSIZE, j); ds_clear(&tier_actions); @@ -7618,7 +7566,7 @@ build_acl_action_lflows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7691,15 +7639,19 @@ build_acl_log_related_flows(struct ovn_datapath *od, struct hmap *lflows, } static void -build_acls(struct ovn_datapath *od, const struct chassis_features *features, +build_acls(const struct ls_lbacls_record *ls_lbacls_rec, + const struct chassis_features *features, struct hmap *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { + const struct ovn_datapath *od = ls_lbacls_rec->od; + const char *default_acl_action = default_acl_drop ? debug_implicit_drop_action() : "next;"; - bool has_stateful = od->has_stateful_acl || od->has_lb_vip; + bool has_stateful = (ls_lbacls_rec->has_stateful_acl + || ls_lbacls_rec->has_lb_vip); const char *ct_blocked_match = features->ct_no_masked_label ? "ct_mark.blocked" : "ct_label.blocked"; @@ -7713,8 +7665,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, * * A related rule at priority 1 is added below if there * are any stateful ACLs in this datapath. */ - if (!od->has_acls) { - if (!od->has_lb_vip) { + if (!ls_lbacls_rec->has_acls) { + if (!ls_lbacls_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", @@ -7877,7 +7829,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_lbacls_rec->max_acl_tier, + &match, &actions); } const struct ls_port_group *ls_pg = @@ -7893,7 +7846,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, meter_groups, &match, &actions); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, ls_lbacls_rec->max_acl_tier, + &match, &actions); } } } @@ -7911,7 +7865,7 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, dns_actions); } - if (od->has_acls || od->has_lb_vip) { + if (ls_lbacls_rec->has_acls || ls_lbacls_rec->has_lb_vip) { /* Add a 34000 priority flow to advance the service monitor reply * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, @@ -7925,8 +7879,8 @@ build_acls(struct ovn_datapath *od, const struct chassis_features *features, REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); } - build_acl_action_lflows(od, lflows, default_acl_action, meter_groups, - &match, &actions); + build_acl_action_lflows(ls_lbacls_rec, lflows, default_acl_action, + meter_groups, &match, &actions); ds_destroy(&match); ds_destroy(&actions); @@ -8571,8 +8525,11 @@ build_stateful(struct ovn_datapath *od, } static void -build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, + struct hmap *lflows) { + const struct ovn_datapath *od = ls_lbacls_rec->od; + /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ @@ -8580,7 +8537,7 @@ build_lb_hairpin(struct ovn_datapath *od, struct hmap *lflows) ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); - if (od->has_lb_vip) { + if (ls_lbacls_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. * Set REGBIT_HAIRPIN in the original direction and * REGBIT_HAIRPIN_REPLY in the reply direction. @@ -9413,22 +9370,16 @@ build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, - const struct ls_port_group_table *ls_port_groups, const struct chassis_features *features, struct hmap *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); - ls_get_acl_flags(od, ls_port_groups); - - build_pre_acls(od, ls_port_groups, lflows); + build_pre_acls(od, lflows); build_pre_lb(od, meter_groups, lflows); build_pre_stateful(od, features, lflows); - build_acl_hints(od, features, lflows); - build_acls(od, features, lflows, ls_port_groups, meter_groups); build_qos(od, lflows); build_stateful(od, features, lflows); - build_lb_hairpin(od, lflows); build_vtep_hairpin(od, lflows); } @@ -15486,7 +15437,7 @@ build_lrouter_nat_defrag_and_lb( * a dynamically negotiated FTP data channel), but will allow * related traffic such as an ICMP Port Unreachable through * that's generated from a non-listening UDP port. */ - if (od->has_lb_vip && features->ct_lb_related) { + if (lr_lbnat_rec->has_lb_vip && features->ct_lb_related) { ds_clear(match); ds_put_cstr(match, "ct.rel && !ct.est && !ct.new"); @@ -15511,7 +15462,7 @@ build_lrouter_nat_defrag_and_lb( * Pass the traffic that is already established to the next table with * proper flags set. */ - if (od->has_lb_vip) { + if (lr_lbnat_rec->has_lb_vip) { ds_clear(match); ds_put_format(match, "ct.est && !ct.rel && !ct.new && %s.natted", @@ -15541,7 +15492,7 @@ build_lrouter_nat_defrag_and_lb( * not committed, it would produce ongoing datapath flows with the ct.new * flag set. Some NICs are unable to offload these flows. */ - if (od->is_gw_router && (od->nbr->n_nat || od->has_lb_vip)) { + if (od->is_gw_router && (od->nbr->n_nat || lr_lbnat_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, "nd || nd_rs || nd_ra", "next;"); @@ -15964,6 +15915,22 @@ build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, meter_groups); } +static void +build_ls_lbacls_flows(const struct ls_lbacls_record *ls_lbacls_rec, + const struct ls_port_group_table *ls_pgs, + const struct chassis_features *features, + const struct shash *meter_groups, + struct hmap *lflows) +{ + ovs_assert(ls_lbacls_rec->od); + + build_ls_lbacls_rec_pre_acls(ls_lbacls_rec, ls_pgs, lflows); + build_ls_lbacls_rec_pre_lb(ls_lbacls_rec, lflows); + build_acl_hints(ls_lbacls_rec, features, lflows); + build_acls(ls_lbacls_rec, features, lflows, ls_pgs, meter_groups); + build_lb_hairpin(ls_lbacls_rec, lflows); +} + struct lswitch_flow_build_info { const struct ovn_datapaths *ls_datapaths; const struct ovn_datapaths *lr_datapaths; @@ -15971,6 +15938,7 @@ struct lswitch_flow_build_info { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_lb_nat_data_table *lr_lbnats; + const struct ls_lbacls_table *ls_lbacls; struct hmap *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; @@ -15995,9 +15963,7 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbs); - build_lswitch_lflows_pre_acl_and_acl(od, lsi->ls_port_groups, - lsi->features, - lsi->lflows, + build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, lsi->meter_groups); build_fwd_group_lflows(od, lsi->lflows); @@ -16112,6 +16078,7 @@ build_lflows_thread(void *arg) { struct worker_control *control = (struct worker_control *) arg; const struct lr_lb_nat_data_record *lr_lbnat_rec; + const struct ls_lbacls_record *ls_lbacls_rec; struct lswitch_flow_build_info *lsi; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; @@ -16240,6 +16207,19 @@ build_lflows_thread(void *arg) lsi->features); } } + + for (bnum = control->id; + bnum <= lsi->ls_lbacls->entries.mask; + bnum += control->pool->size) + { + LS_LBACLS_TABLE_FOR_EACH_IN_P (ls_lbacls_rec, bnum, + lsi->ls_lbacls) { + build_ls_lbacls_flows(ls_lbacls_rec, lsi->ls_port_groups, + lsi->features, lsi->meter_groups, + lsi->lflows); + } + } + for (bnum = control->id; bnum <= lsi->igmp_groups->mask; bnum += control->pool->size) @@ -16300,6 +16280,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lr_ports, const struct ls_port_group_table *ls_pgs, const struct lr_lb_nat_data_table *lr_lbnats, + const struct ls_lbacls_table *ls_lbacls, struct hmap *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, @@ -16330,6 +16311,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].lr_ports = lr_ports; lsiv[index].ls_port_groups = ls_pgs; lsiv[index].lr_lbnats = lr_lbnats; + lsiv[index].ls_lbacls = ls_lbacls; lsiv[index].igmp_groups = igmp_groups; lsiv[index].meter_groups = meter_groups; lsiv[index].lb_dps_map = lb_dps_map; @@ -16355,6 +16337,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, free(lsiv); } else { const struct lr_lb_nat_data_record *lr_lbnat_rec; + const struct ls_lbacls_record *ls_lbacls_rec; struct ovn_igmp_group *igmp_group; struct ovn_lb_datapaths *lb_dps; struct ovn_datapath *od; @@ -16367,6 +16350,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .lr_ports = lr_ports, .ls_port_groups = ls_pgs, .lr_lbnats = lr_lbnats, + .ls_lbacls = ls_lbacls, .lflows = lflows, .igmp_groups = igmp_groups, .meter_groups = meter_groups, @@ -16436,6 +16420,12 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.features); } + LS_LBACLS_TABLE_FOR_EACH (ls_lbacls_rec, ls_lbacls) { + build_ls_lbacls_flows(ls_lbacls_rec, lsi.ls_port_groups, + lsi.features, lsi.meter_groups, + lsi.lflows); + } + stopwatch_start(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (igmp_group, hmap_node, igmp_groups) { build_lswitch_ip_mcast_igmp_mld(igmp_group, @@ -16532,6 +16522,7 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lr_ports, input_data->ls_port_groups, input_data->lr_lbnats, + input_data->ls_lbacls, lflows, &igmp_groups, input_data->meter_groups, diff --git a/northd/northd.h b/northd/northd.h index f7a44652f3..495e6e8852 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -173,6 +173,7 @@ struct lflow_input { const struct hmap *lr_ports; const struct ls_port_group_table *ls_port_groups; const struct lr_lb_nat_data_table *lr_lbnats; + const struct ls_lbacls_table *ls_lbacls; const struct shash *meter_groups; const struct hmap *lb_datapaths_map; const struct hmap *bfd_connections; @@ -282,11 +283,7 @@ struct ovn_datapath { struct hmap port_tnlids; uint32_t port_key_hint; - bool has_stateful_acl; - bool has_lb_vip; bool has_unknown; - bool has_acls; - uint64_t max_acl_tier; bool has_vtep_lports; bool has_arp_proxy_port; @@ -334,6 +331,8 @@ ods_size(const struct ovn_datapaths *datapaths) return hmap_count(&datapaths->datapaths); } +bool od_has_lb_vip(const struct ovn_datapath *od); + /* A logical switch port or logical router port. * * In steady state, an ovn_port points to a northbound Logical_Switch_Port From patchwork Tue Oct 24 00:48:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854083 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtkf2lPHz23jl for ; Tue, 24 Oct 2023 11:50:02 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E926381F52; Tue, 24 Oct 2023 00:49:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E926381F52 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QWq-kh6xy-S3; Tue, 24 Oct 2023 00:49:55 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id E72B7812D8; Tue, 24 Oct 2023 00:49:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E72B7812D8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CB5C7C0E8C; Tue, 24 Oct 2023 00:49:51 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8CA34C0E43 for ; Tue, 24 Oct 2023 00:49:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 3A8A683C9B for ; Tue, 24 Oct 2023 00:49:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 3A8A683C9B X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vg7Iuc-6CChA for ; Tue, 24 Oct 2023 00:49:07 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp1.osuosl.org (Postfix) with ESMTPS id 36EEE83CE8 for ; Tue, 24 Oct 2023 00:49:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 36EEE83CE8 Received: by mail.gandi.net (Postfix) with ESMTPSA id A5EAC1C0005; Tue, 24 Oct 2023 00:49:02 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:48:50 -0400 Message-ID: <20231024004850.4133913-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 10/18] northd: Refactor lflow management into a separate module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add() and other related functions/macros are now moved into a separate module - lflow-mgr.c. This module maintains a table 'struct lflow_table' for the logical flows. lflow table maintains a 2 hmap tables to store the logical flows. - 1st hmap table uses the lflow "match" to lookup for the logical flows in the table. - 2nd hmap table uses the logical flow uuid to lookup for the logical flows in the table. It also maintains the logical switch and router dp groups. Previous commits which added lflow incremental processing for the VIF logical ports, stored the references to the logical ports' lflows using 'struct lflow_ref_list'. This patch removes the lflow_ref_list and instead makes uses of objdep mgr APIs (provided by lib/objdep.c). Even though objdep mgr is a bit heavy, it becomes easier to extend the lflow I-P for other resource changes (like load balancers, NAT etc). Usage of objdep mgr is embedded inside 'struct lflow_ref' and it is transparent to northd.c and other modules. 'struct ovn_port' maintains 2 instances of lflow_ref. i,e struct ovn_port { ... ... struct lflow_ref *lflow_ref; struct lflow_ref *lbnat_lflow_ref; }; All the logical flows generated by build_lswitch_and_lrouter_iterate_by_lsp() uses the ovn_port->lflow_ref. All the logical flows generated by build_lsp_lflows_for_lbnats() uses the ovn_port->lbnat_lflow_ref. When handling the ovn_port changes incrementally, the lflows referenced in 'struct ovn_port' are cleared and regenerated and synced to the SB logical flows. eg. lflow_ref_clear_lflows(op->lflow_ref, op->od, lflows); build_lswitch_and_lrouter_iterate_by_lsp(op, ...); lflow_ref_sync_lflows_to_sb(op->lflow_ref, ...); This patch does few more changes which are significant: - Logical flows are now hashed without the logical datapaths. If a logical flow is referenced by just one datapath, we don't rehash it. - The synthetic 'hash' column of sbrec_logical_flow now doesn't use the logical datapath. This means that when ovn-northd is updated/upgraded and has this commit, all the logical flows with 'logical_datapath' column set will get deleted and re-added causing some disruptions. - With the commit [1] which added I-P support for logical port changes, multiple logical flows with same match 'M' and actions 'A' are generated and stored without the dp groups, which was not the case prior to that patch. One example to generate these lflows is: ovn-nbctl lsp-set-addresses sw0p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw1p1 "MAC1 IP1" ovn-nbctl lsp-set-addresses sw2p1 "MAC1 IP1" Now with this patch we go back to the earlier way. i.e one logical flow with logical_dp_groups set. - With this patch any updates to a logical port which doesn't result in new logical flows will not result in deletion and addition of same logical flows. Eg. ovn-nbctl set logical_switch_port sw0p1 external_ids:foo=bar will be a no-op to the SB logical flow table. [1] - 8bbd678("northd: Incremental processing of VIF additions in 'lflow' node.") Signed-off-by: Numan Siddique --- lib/objdep.h | 2 + lib/ovn-util.c | 11 +- northd/automake.mk | 4 +- northd/en-lflow.c | 21 +- northd/en-lflow.h | 6 + northd/inc-proc-northd.c | 4 +- northd/lflow-mgr.c | 1167 ++++++++++++++++++++++++ northd/lflow-mgr.h | 188 ++++ northd/northd.c | 1854 +++++++++----------------------------- northd/northd.h | 210 ++++- northd/ovn-northd.c | 4 + 11 files changed, 2035 insertions(+), 1436 deletions(-) create mode 100644 northd/lflow-mgr.c create mode 100644 northd/lflow-mgr.h diff --git a/lib/objdep.h b/lib/objdep.h index 1ea781947c..cfdc3f2a41 100644 --- a/lib/objdep.h +++ b/lib/objdep.h @@ -27,6 +27,8 @@ enum objdep_type { OBJDEP_TYPE_PORTBINDING, OBJDEP_TYPE_MC_GROUP, OBJDEP_TYPE_TEMPLATE, + OBJDEP_TYPE_LFLOW, + OBJDEP_TYPE_LFLOW_OD, OBJDEP_TYPE_MAX, }; diff --git a/lib/ovn-util.c b/lib/ovn-util.c index 05e635a6b4..29464f24a3 100644 --- a/lib/ovn-util.c +++ b/lib/ovn-util.c @@ -630,13 +630,10 @@ ovn_pipeline_from_name(const char *pipeline) uint32_t sbrec_logical_flow_hash(const struct sbrec_logical_flow *lf) { - const struct sbrec_datapath_binding *ld = lf->logical_datapath; - uint32_t hash = ovn_logical_flow_hash(lf->table_id, - ovn_pipeline_from_name(lf->pipeline), - lf->priority, lf->match, - lf->actions); - - return ld ? ovn_logical_flow_hash_datapath(&ld->header_.uuid, hash) : hash; + return ovn_logical_flow_hash(lf->table_id, + ovn_pipeline_from_name(lf->pipeline), + lf->priority, lf->match, + lf->actions); } uint32_t diff --git a/northd/automake.mk b/northd/automake.mk index 4593654726..9707be7d8e 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -33,7 +33,9 @@ northd_ovn_northd_SOURCES = \ northd/inc-proc-northd.c \ northd/inc-proc-northd.h \ northd/ipam.c \ - northd/ipam.h + northd/ipam.h \ + northd/lflow-mgr.c \ + northd/lflow-mgr.h northd_ovn_northd_LDADD = \ lib/libovn.la \ $(OVSDB_LIBDIR)/libovsdb.la \ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 648a477916..efffee6384 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -24,6 +24,7 @@ #include "en-ls-lb-acls.h" #include "en-northd.h" #include "en-meters.h" +#include "lflow-mgr.h" #include "lib/inc-proc-eng.h" #include "northd.h" @@ -58,6 +59,8 @@ lflow_get_input_data(struct engine_node *node, EN_OVSDB_GET(engine_get_input("SB_multicast_group", node)); lflow_input->sbrec_igmp_group_table = EN_OVSDB_GET(engine_get_input("SB_igmp_group", node)); + lflow_input->sbrec_logical_dp_group_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); lflow_input->sbrec_mcast_group_by_name_dp = engine_ovsdb_node_get_index( @@ -91,8 +94,8 @@ void en_lflow_run(struct engine_node *node, void *data) lflow_input.bfd_connections = &bfd_connections; struct lflow_data *lflow_data = data; - lflow_data_destroy(lflow_data); - lflow_data_init(lflow_data); + lflow_table_clear(lflow_data->lflow_table); + lflow_table_init(lflow_data->lflow_table); stopwatch_start(BUILD_LFLOWS_STOPWATCH_NAME, time_msec()); build_bfd_table(eng_ctx->ovnsb_idl_txn, @@ -100,7 +103,8 @@ void en_lflow_run(struct engine_node *node, void *data) lflow_input.sbrec_bfd_table, lflow_input.lr_ports, &bfd_connections); - build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, &lflow_data->lflows); + build_lflows(eng_ctx->ovnsb_idl_txn, &lflow_input, + lflow_data->lflow_table); bfd_cleanup_connections(lflow_input.nbrec_bfd_table, &bfd_connections); hmap_destroy(&bfd_connections); @@ -131,7 +135,7 @@ lflow_northd_handler(struct engine_node *node, if (!lflow_handle_northd_port_changes(eng_ctx->ovnsb_idl_txn, &northd_data->trk_northd_changes.trk_ovn_ports, - &lflow_input, &lflow_data->lflows)) { + &lflow_input, lflow_data->lflow_table)) { return false; } @@ -161,11 +165,14 @@ void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { struct lflow_data *data = xmalloc(sizeof *data); - lflow_data_init(data); + data->lflow_table = lflow_table_alloc(); + lflow_table_init(data->lflow_table); return data; } -void en_lflow_cleanup(void *data) +void en_lflow_cleanup(void *data_) { - lflow_data_destroy(data); + struct lflow_data *data = data_; + lflow_table_destroy(data->lflow_table); + data->lflow_table = NULL; } diff --git a/northd/en-lflow.h b/northd/en-lflow.h index 5417b2faff..f7325c56b1 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -9,6 +9,12 @@ #include "lib/inc-proc-eng.h" +struct lflow_table; + +struct lflow_data { + struct lflow_table *lflow_table; +}; + void en_lflow_run(struct engine_node *node, void *data); void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index ab4af92aeb..3cedd502cb 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -99,7 +99,8 @@ static unixctl_cb_func chassis_features_list; SB_NODE(bfd, "bfd") \ SB_NODE(fdb, "fdb") \ SB_NODE(static_mac_binding, "static_mac_binding") \ - SB_NODE(chassis_template_var, "chassis_template_var") + SB_NODE(chassis_template_var, "chassis_template_var") \ + SB_NODE(logical_dp_group, "logical_dp_group") enum sb_engine_node { #define SB_NODE(NAME, NAME_STR) SB_##NAME, @@ -234,6 +235,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_logical_flow, NULL); engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); + engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c new file mode 100644 index 0000000000..b435d03061 --- /dev/null +++ b/northd/lflow-mgr.c @@ -0,0 +1,1167 @@ +/* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/thread.h" +#include "lib/bitmap.h" +#include "lib/uuidset.h" +#include "openvswitch/util.h" +#include "openvswitch/vlog.h" +#include "ovs-thread.h" +#include "stopwatch.h" + +/* OVN includes */ +#include "debug.h" +#include "lflow-mgr.h" +#include "lib/objdep.h" +#include "lib/ovn-parallel-hmap.h" +#include "northd.h" + +VLOG_DEFINE_THIS_MODULE(lflow_mgr); + +/* Static function declarations. */ +struct ovn_lflow; + +static void ovn_lflow_init(struct ovn_lflow *, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, char *match, + char *actions, char *io_port, + char *ctrl_meter, char *stage_hint, + const char *where, uint32_t hash); +static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, + const char *ctrl_meter, uint32_t hash); +static struct ovn_lflow * ovn_lflow_uuid_find( + const struct hmap *lflows_hash_map, const struct uuid *lflow_uuid); +static void ovn_lflow_destroy(struct lflow_table *lflow_table, + struct ovn_lflow *lflow); +static char *ovn_lflow_hint(const struct ovsdb_idl_row *row); + +static struct ovn_lflow *do_ovn_lflow_add( + struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, uint32_t hash, + enum ovn_stage stage, uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where); + + +static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, + uint32_t hash); +static void lflow_hash_unlock(struct ovs_mutex *hash_lock); + +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct ovn_dp_group *ovn_dp_group_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); +static struct ovn_dp_group *ovn_dp_group_get( + struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *); +static struct ovn_dp_group *ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, + size_t bitmap_len, + uint32_t hash); +static void inc_ovn_dp_group_ref(struct ovn_dp_group *); +static void dec_ovn_dp_group_ref(struct hmap *dp_groups, + struct ovn_dp_group *); +static void ovn_dp_group_add_with_reference(struct ovn_lflow *, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len); + +static bool is_lflow_and_od_type_match(const struct ovn_datapath *, + struct ovn_lflow *); +static void unlink_objres_lflows(struct resource_to_objects_node *, + const struct ovn_datapath *od, + struct lflow_table *, + struct objdep_mgr *); +static void sync_lflows_from_objres( + struct resource_to_objects_node *, struct lflow_table *, + struct objdep_mgr *, struct ovsdb_idl_txn *, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +static void sync_lflow_to_sb(struct ovn_lflow *, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *); + +extern int parallelization_state; +extern thread_local size_t thread_lflow_counter; + +static bool lflow_hash_lock_initialized = false; +/* The lflow_hash_lock is a mutex array that protects updates to the shared + * lflow table across threads when parallel lflow build and dp-group are both + * enabled. To avoid high contention between threads, a big array of mutexes + * are used instead of just one. This is possible because when parallel build + * is used we only use hmap_insert_fast() to update the hmap, which would not + * touch the bucket array but only the list in a single bucket. We only need to + * make sure that when adding lflows to the same hash bucket, the same lock is + * used, so that no two threads can add to the bucket at the same time. It is + * ok that the same lock is used to protect multiple buckets, so a fixed sized + * mutex array is used instead of 1-1 mapping to the hash buckets. This + * simplies the implementation while effectively reduces lock contention + * because the chance that different threads contending the same lock amongst + * the big number of locks is very low. */ +#define LFLOW_HASH_LOCK_MASK 0xFFFF +static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; + +/* Full thread safety analysis is not possible with hash locks, because + * they are taken conditionally based on the 'parallelization_state' and + * a flow hash. Also, the order in which two hash locks are taken is not + * predictable during the static analysis. + * + * Since the order of taking two locks depends on a random hash, to avoid + * ABBA deadlocks, no two hash locks can be nested. In that sense an array + * of hash locks is similar to a single mutex. + * + * Using a fake mutex to partially simulate thread safety restrictions, as + * if it were actually a single mutex. + * + * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional + * nature of the lock. Unlike other attributes, it applies to the + * implementation and not to the interface. So, we can define a function + * that acquires the lock without analysing the way it does that. + */ +extern struct ovs_mutex fake_hash_mutex; + +struct ovn_lflow { + struct hmap_node hmap_node; + struct hmap_node hash_node; + struct ovs_list list_node; /* For temporary list of lflows. Don't remove + at destroy. */ + + struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ + unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ + enum ovn_stage stage; + uint16_t priority; + char *match; + char *actions; + char *io_port; + char *stage_hint; + char *ctrl_meter; + size_t n_ods; /* Number of datapaths referenced by 'od' and + * 'dpg_bitmap'. */ + struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ + + const char *where; + + struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ + struct uuid lflow_uuid; +}; + +struct lflow_table { + struct hmap match_map; + struct hmap hash_map; + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; + ssize_t max_seen_lflow_size; +}; + +struct lflow_table * +lflow_table_alloc(void) +{ + struct lflow_table *lflow_table = xzalloc(sizeof *lflow_table); + lflow_table->max_seen_lflow_size = 128; + + return lflow_table; +} + +void +lflow_table_init(struct lflow_table *lflow_table) +{ + fast_hmap_size_for(&lflow_table->match_map, + lflow_table->max_seen_lflow_size); + fast_hmap_size_for(&lflow_table->hash_map, + lflow_table->max_seen_lflow_size); + ovn_dp_groups_init(&lflow_table->ls_dp_groups); + ovn_dp_groups_init(&lflow_table->lr_dp_groups); +} + +void +lflow_table_clear(struct lflow_table *lflow_table) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_SAFE (lflow, hmap_node, &lflow_table->match_map) { + ovn_lflow_destroy(lflow_table, lflow); + } + hmap_destroy(&lflow_table->match_map); + hmap_destroy(&lflow_table->hash_map); + + ovn_dp_groups_destroy(&lflow_table->ls_dp_groups); + ovn_dp_groups_destroy(&lflow_table->lr_dp_groups); +} + +void +lflow_table_destroy(struct lflow_table *lflow_table) +{ + lflow_table_clear(lflow_table); + free(lflow_table); +} + +void +lflow_table_expand(struct lflow_table *lflow_table) +{ + hmap_expand(&lflow_table->match_map); + hmap_expand(&lflow_table->hash_map); + + if (hmap_count(&lflow_table->match_map) > + lflow_table->max_seen_lflow_size) { + lflow_table->max_seen_lflow_size = hmap_count(&lflow_table->match_map); + } +} + +void +lflow_table_set_size(struct lflow_table *lflow_table, size_t size) +{ + lflow_table->match_map.n = size; + lflow_table->hash_map.n = size; +} + +void +lflow_table_sync_to_sb(struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sb_flow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); + struct hmap *lflows = &lflow_table->match_map; + struct ovn_lflow *lflow; + + /* Push changes to the Logical_Flow table to database. */ + const struct sbrec_logical_flow *sbflow; + SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, sb_flow_table) { + struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; + struct ovn_datapath *logical_datapath_od = NULL; + size_t i; + + /* Find one valid datapath to get the datapath type. */ + struct sbrec_datapath_binding *dp = sbflow->logical_datapath; + if (dp) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, + &lr_datapaths->datapaths, + dp); + if (logical_datapath_od + && ovn_datapath_is_stale(logical_datapath_od)) { + logical_datapath_od = NULL; + } + } + for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { + logical_datapath_od = ovn_datapath_from_sbrec( + &ls_datapaths->datapaths, + &lr_datapaths->datapaths, + dp_group->datapaths[i]); + if (logical_datapath_od + && !ovn_datapath_is_stale(logical_datapath_od)) { + break; + } + logical_datapath_od = NULL; + } + + if (!logical_datapath_od) { + /* This lflow has no valid logical datapaths. */ + sbrec_logical_flow_delete(sbflow); + continue; + } + + enum ovn_pipeline pipeline + = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; + + lflow = ovn_lflow_find( + lflows, + ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), + pipeline, sbflow->table_id), + sbflow->priority, sbflow->match, sbflow->actions, + sbflow->controller_meter, sbflow->hash); + if (lflow) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sbflow, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } else { + sbrec_logical_flow_delete(sbflow); + } + } + + HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + NULL, dpgrp_table); + + hmap_remove(lflows, &lflow->hmap_node); + hmap_insert(&lflows_temp, &lflow->hmap_node, + hmap_node_hash(&lflow->hmap_node)); + } + hmap_swap(lflows, &lflows_temp); + hmap_destroy(&lflows_temp); +} + +/* lflow ref mgr */ +struct lflow_ref { + char *res_name; + struct objdep_mgr objdep_mgr; + const struct ovn_datapath *od; +}; + +struct lflow_ref * +lflow_ref_alloc(const char *res_name) +{ + struct lflow_ref *lflow_ref = xzalloc(sizeof *lflow_ref); + lflow_ref->res_name = xstrdup(res_name); + objdep_mgr_init(&lflow_ref->objdep_mgr); + + return lflow_ref; +} + +void +lflow_ref_set_od(struct lflow_ref *lflow_ref, const struct ovn_datapath *od) +{ + lflow_ref->od = od; +} + +void +lflow_ref_destroy(struct lflow_ref *lflow_ref) +{ + free(lflow_ref->res_name); + objdep_mgr_destroy(&lflow_ref->objdep_mgr); + free(lflow_ref); +} + +void +lflow_ref_clear_lflows(struct lflow_ref *lflow_ref, + const struct ovn_datapath *od, + struct lflow_table *lflow_table) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + &lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW, lflow_ref->res_name); + + unlink_objres_lflows(res_node, od, lflow_table, &lflow_ref->objdep_mgr); +} + +void +lflow_ref_clear_and_sync_lflows(struct lflow_ref *lflow_ref, + const struct ovn_datapath *od, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + &lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW, lflow_ref->res_name); + + unlink_objres_lflows(res_node, od, lflow_table, &lflow_ref->objdep_mgr); + sync_lflows_from_objres(res_node, lflow_table, &lflow_ref->objdep_mgr, + ovnsb_txn, ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +void +lflow_ref_sync_lflows_to_sb(struct lflow_ref *lflow_ref, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + &lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW, lflow_ref->res_name); + + sync_lflows_from_objres(res_node, lflow_table, &lflow_ref->objdep_mgr, + ovnsb_txn, ls_datapaths, lr_datapaths, + ovn_internal_version_changed, sbflow_table, + dpgrp_table); +} + +void +lflow_table_add_lflow(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, + struct lflow_ref *lflow_ref) + OVS_EXCLUDED(fake_hash_mutex) +{ + struct ovs_mutex *hash_lock; + uint32_t hash; + + ovs_assert(!od || + ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); + + hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), + ovn_stage_get_pipeline(stage), + priority, match, + actions); + + hash_lock = lflow_hash_lock(&lflow_table->match_map, hash); + struct ovn_lflow *lflow = + do_ovn_lflow_add(lflow_table, od, dp_bitmap, + dp_bitmap_len, hash, stage, + priority, match, actions, + io_port, ctrl_meter, stage_hint, where); + + if (lflow_ref) { + objdep_mgr_add(&lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW, + lflow_ref->res_name, &lflow->lflow_uuid); + + if (lflow_ref->od && lflow_ref->od != od) { + char uuid_s[UUID_LEN + 1]; + sprintf(uuid_s, UUID_FMT, UUID_ARGS(&lflow->lflow_uuid)); + + struct uuid u = UUID_ZERO; + u.parts[0] = od->index; + objdep_mgr_add(&lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW_OD, + uuid_s, &u); + } + } + + lflow_hash_unlock(hash_lock); + +} + +void +lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + enum ovn_stage stage, + const char *where, + struct lflow_ref *lflow_ref) +{ + lflow_table_add_lflow(lflow_table, od, NULL, 0, stage, 0, "1", + debug_drop_action(), NULL, NULL, NULL, + where, lflow_ref); +} + +/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it + * doesn't exist, creates a new one and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. */ +struct ovn_dp_group * +ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); + if (dpg) { + return dpg; + } + + return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, + desired_bitmap, bitmap_len, is_switch, + ls_datapaths, lr_datapaths); +} + +void +ovn_dp_groups_destroy(struct hmap *dp_groups) +{ + struct ovn_dp_group *dpg; + HMAP_FOR_EACH_POP (dpg, node, dp_groups) { + bitmap_free(dpg->bitmap); + free(dpg); + } + hmap_destroy(dp_groups); +} + + +void +lflow_hash_lock_init(void) +{ + if (!lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_init(&lflow_hash_locks[i]); + } + lflow_hash_lock_initialized = true; + } +} + +void +lflow_hash_lock_destroy(void) +{ + if (lflow_hash_lock_initialized) { + for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { + ovs_mutex_destroy(&lflow_hash_locks[i]); + } + } + lflow_hash_lock_initialized = false; +} + +/* static functions. */ +static void +ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, + size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, + char *match, char *actions, char *io_port, char *ctrl_meter, + char *stage_hint, const char *where, uint32_t hash) +{ + ovs_list_init(&lflow->list_node); + lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); + lflow->od = od; + lflow->stage = stage; + lflow->priority = priority; + lflow->match = match; + lflow->actions = actions; + lflow->io_port = io_port; + lflow->stage_hint = stage_hint; + lflow->ctrl_meter = ctrl_meter; + lflow->dpg = NULL; + lflow->where = where; + lflow->sb_uuid = UUID_ZERO; + lflow->lflow_uuid = uuid_random(); + lflow->lflow_uuid.parts[0] = hash; +} + +static struct ovs_mutex * +lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash) + OVS_ACQUIRES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + struct ovs_mutex *hash_lock = NULL; + + if (parallelization_state == STATE_USE_PARALLELIZATION) { + hash_lock = + &lflow_hash_locks[hash & lflow_table->mask & LFLOW_HASH_LOCK_MASK]; + ovs_mutex_lock(hash_lock); + } + return hash_lock; +} + +static void +lflow_hash_unlock(struct ovs_mutex *hash_lock) + OVS_RELEASES(fake_hash_mutex) + OVS_NO_THREAD_SAFETY_ANALYSIS +{ + if (hash_lock) { + ovs_mutex_unlock(hash_lock); + } +} + +static bool +ovn_lflow_equal(const struct ovn_lflow *a, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *ctrl_meter) +{ + return (a->stage == stage + && a->priority == priority + && !strcmp(a->match, match) + && !strcmp(a->actions, actions) + && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); +} + +static struct ovn_lflow * +ovn_lflow_find(const struct hmap *lflows, + enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *ctrl_meter, uint32_t hash) +{ + struct ovn_lflow *lflow; + HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { + if (ovn_lflow_equal(lflow, stage, priority, match, actions, + ctrl_meter)) { + return lflow; + } + } + return NULL; +} + +static struct ovn_lflow * +ovn_lflow_uuid_find(const struct hmap *lflows_hash_map, + const struct uuid *lflow_uuid) +{ + uint32_t hash = lflow_uuid->parts[0]; + struct ovn_lflow *lflow; + HMAP_FOR_EACH_WITH_HASH (lflow, hash_node, hash, lflows_hash_map) { + if (uuid_equals(&lflow->lflow_uuid, lflow_uuid)) { + return lflow; + } + } + return NULL; +} + +static char * +ovn_lflow_hint(const struct ovsdb_idl_row *row) +{ + if (!row) { + return NULL; + } + return xasprintf("%08x", row->uuid.parts[0]); +} + +static void +ovn_lflow_destroy(struct lflow_table *lflow_table, struct ovn_lflow *lflow) +{ + if (lflow) { + if (lflow_table) { + hmap_remove(&lflow_table->match_map, &lflow->hmap_node); + hmap_remove(&lflow_table->hash_map, &lflow->hash_node); + } + bitmap_free(lflow->dpg_bitmap); + free(lflow->match); + free(lflow->actions); + free(lflow->io_port); + free(lflow->stage_hint); + free(lflow->ctrl_meter); + free(lflow); + } +} + +static struct ovn_lflow * +do_ovn_lflow_add(struct lflow_table *lflow_table, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, size_t dp_bitmap_len, + uint32_t hash, enum ovn_stage stage, uint16_t priority, + const char *match, const char *actions, + const char *io_port, const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where) + OVS_REQUIRES(fake_hash_mutex) +{ + struct ovn_lflow *old_lflow; + struct ovn_lflow *lflow; + + size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; + ovs_assert(bitmap_len); + + old_lflow = ovn_lflow_find(&lflow_table->match_map, stage, + priority, match, actions, ctrl_meter, hash); + if (old_lflow) { + ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, + bitmap_len); + return old_lflow; + } + + lflow = xmalloc(sizeof *lflow); + /* While adding new logical flows we're not setting single datapath, but + * collecting a group. 'od' will be updated later for all flows with only + * one datapath in a group, so it could be hashed correctly. */ + ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, + xstrdup(match), xstrdup(actions), + io_port ? xstrdup(io_port) : NULL, + nullable_xstrdup(ctrl_meter), + ovn_lflow_hint(stage_hint), where, hash); + + ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); + + if (parallelization_state != STATE_USE_PARALLELIZATION) { + hmap_insert(&lflow_table->match_map, &lflow->hmap_node, hash); + hmap_insert(&lflow_table->hash_map, &lflow->hash_node, hash); + } else { + hmap_insert_fast(&lflow_table->match_map, &lflow->hmap_node, + hash); + hmap_insert_fast(&lflow_table->hash_map, &lflow->hash_node, + hash); + thread_lflow_counter++; + } + + return lflow; +} + +static void +sync_lflow_to_sb(struct ovn_lflow *lflow, + struct ovsdb_idl_txn *ovnsb_txn, + struct lflow_table *lflow_table, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow *sbflow, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table) +{ + struct sbrec_logical_dp_group *sbrec_dp_group = NULL; + struct ovn_dp_group *pre_sync_dpg = lflow->dpg; + struct ovn_datapath **datapaths_array; + struct hmap *dp_groups; + size_t n_datapaths; + bool is_switch; + + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(ls_datapaths); + datapaths_array = ls_datapaths->array; + dp_groups = &lflow_table->ls_dp_groups; + is_switch = true; + } else { + n_datapaths = ods_size(lr_datapaths); + datapaths_array = lr_datapaths->array; + dp_groups = &lflow_table->lr_dp_groups; + is_switch = false; + } + + lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + ovs_assert(lflow->n_ods); + + if (lflow->n_ods == 1) { + /* There is only one datapath, so it should be moved out of the + * group to a single 'od'. */ + size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, + n_datapaths); + + lflow->od = datapaths_array[index]; + lflow->dpg = NULL; + } else { + lflow->od = NULL; + } + + if (!sbflow) { + lflow->sb_uuid = uuid_random(); + sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, + &lflow->sb_uuid); + const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); + uint8_t table = ovn_stage_get_table(lflow->stage); + sbrec_logical_flow_set_pipeline(sbflow, pipeline); + sbrec_logical_flow_set_table_id(sbflow, table); + sbrec_logical_flow_set_priority(sbflow, lflow->priority); + sbrec_logical_flow_set_match(sbflow, lflow->match); + sbrec_logical_flow_set_actions(sbflow, lflow->actions); + if (lflow->io_port) { + struct smap tags = SMAP_INITIALIZER(&tags); + smap_add(&tags, "in_out_port", lflow->io_port); + sbrec_logical_flow_set_tags(sbflow, &tags); + smap_destroy(&tags); + } + sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); + + /* Trim the source locator lflow->where, which looks something like + * "ovn/northd/northd.c:1234", down to just the part following the + * last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + struct smap ids = SMAP_INITIALIZER(&ids); + smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); + smap_add(&ids, "source", where); + if (lflow->stage_hint) { + smap_add(&ids, "stage-hint", lflow->stage_hint); + } + sbrec_logical_flow_set_external_ids(sbflow, &ids); + smap_destroy(&ids); + + } else { + lflow->sb_uuid = sbflow->header_.uuid; + sbrec_dp_group = sbflow->logical_dp_group; + + if (ovn_internal_version_changed) { + const char *stage_name = smap_get_def(&sbflow->external_ids, + "stage-name", ""); + const char *stage_hint = smap_get_def(&sbflow->external_ids, + "stage-hint", ""); + const char *source = smap_get_def(&sbflow->external_ids, + "source", ""); + + if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-name", ovn_stage_to_str(lflow->stage)); + } + if (lflow->stage_hint) { + if (strcmp(stage_hint, lflow->stage_hint)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "stage-hint", lflow->stage_hint); + } + } + if (lflow->where) { + + /* Trim the source locator lflow->where, which looks something + * like "ovn/northd/northd.c:1234", down to just the part + * following the last slash, e.g. "northd.c:1234". */ + const char *slash = strrchr(lflow->where, '/'); +#if _WIN32 + const char *backslash = strrchr(lflow->where, '\\'); + if (!slash || backslash > slash) { + slash = backslash; + } +#endif + const char *where = slash ? slash + 1 : lflow->where; + + if (strcmp(source, where)) { + sbrec_logical_flow_update_external_ids_setkey( + sbflow, "source", where); + } + } + } + } + + if (lflow->od) { + sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); + sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); + } else { + sbrec_logical_flow_set_logical_datapath(sbflow, NULL); + lflow->dpg = ovn_dp_group_get(dp_groups, lflow->n_ods, + lflow->dpg_bitmap, + n_datapaths); + if (lflow->dpg) { + /* Update the dpg's sb dp_group. */ + lflow->dpg->dp_group = sbrec_logical_dp_group_table_get_for_uuid( + sb_dpgrp_table, + &lflow->dpg->dpg_uuid); + ovs_assert(lflow->dpg->dp_group); + } else { + lflow->dpg = ovn_dp_group_create( + ovnsb_txn, dp_groups, sbrec_dp_group, + lflow->n_ods, lflow->dpg_bitmap, + n_datapaths, is_switch, + ls_datapaths, + lr_datapaths); + } + sbrec_logical_flow_set_logical_dp_group(sbflow, + lflow->dpg->dp_group); + } + + if (pre_sync_dpg != lflow->dpg) { + if (lflow->dpg) { + inc_ovn_dp_group_ref(lflow->dpg); + } + if (pre_sync_dpg) { + dec_ovn_dp_group_ref(dp_groups, pre_sync_dpg); + } + } +} + +static struct ovn_dp_group * +ovn_dp_group_find(const struct hmap *dp_groups, + const unsigned long *dpg_bitmap, size_t bitmap_len, + uint32_t hash) +{ + struct ovn_dp_group *dpg; + + HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { + if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { + return dpg; + } + } + return NULL; +} + +static void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + +static struct sbrec_logical_dp_group * +ovn_sb_insert_or_update_logical_dp_group( + struct ovsdb_idl_txn *ovnsb_txn, + struct sbrec_logical_dp_group *dp_group, + const unsigned long *dpg_bitmap, + const struct ovn_datapaths *datapaths) +{ + const struct sbrec_datapath_binding **sb; + size_t n = 0, index; + + sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); + BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { + sb[n++] = datapaths->array[index]->sb; + } + if (!dp_group) { + struct uuid dpg_uuid = uuid_random(); + dp_group = sbrec_logical_dp_group_insert_persist_uuid( + ovnsb_txn, &dpg_uuid); + } + sbrec_logical_dp_group_set_datapaths( + dp_group, (struct sbrec_datapath_binding **) sb, n); + free(sb); + + return dp_group; +} + +static struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. + * If 'sb_group' is provided, function will try to re-use this group by + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ +static struct ovn_dp_group * +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) +{ + struct ovn_dp_group *dpg; + + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); + } + + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); + } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); + + return dpg; +} + +/* Adds an OVN datapath to a datapath group of existing logical flow. + * Version to use when hash bucket locking is NOT required or the corresponding + * hash lock is already taken. */ +static void +ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, + const struct ovn_datapath *od, + const unsigned long *dp_bitmap, + size_t bitmap_len) + OVS_REQUIRES(fake_hash_mutex) +{ + if (od) { + bitmap_set1(lflow_ref->dpg_bitmap, od->index); + } + if (dp_bitmap) { + bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); + } +} + +static bool +is_lflow_and_od_type_match(const struct ovn_datapath *od, + struct ovn_lflow *lflow) +{ + enum ovn_datapath_type type = od->nbs ? DP_SWITCH : DP_ROUTER; + return ovn_stage_to_datapath_type(lflow->stage) == type; +} + +/* Unlinks the lflows stored in the resource to object nodes for the + * datapath 'od' from the lflow dependecy manager. + * It basically clears the datapath id of the 'od' for the lflows + * in the 'res_node'. + */ +static void +unlink_objres_lflows(struct resource_to_objects_node *res_node, + const struct ovn_datapath *od, + struct lflow_table *lflow_table, + struct objdep_mgr *lflowdep_mgr) +{ + if (!res_node) { + return; + } + + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + struct ovn_lflow *lflow = ovn_lflow_uuid_find( + &lflow_table->hash_map, obj_uuid); + if (!lflow) { + continue; + } + + /* Check if the lflow datapath is same the od datapath. */ + if (is_lflow_and_od_type_match(od, lflow)) { + bitmap_set0(lflow->dpg_bitmap, od->index); + } else { + /* The datapath type doesn't match. Which means this lflow was + * added due to a resource in the other type. + * Eg. For every logical switch port whose lswitch is connected + * to a router, an lflow is added in the lr_in_arp_resolve stage. + * Get the datapath index of this router and clear it. + * + * OBJDEP_TYPE_LFLOW_OD type is used to store this lflow object to + * logical router resource linking (logical router index is stored + * in the uuid.parts[0]). + */ + char uuid_s[UUID_LEN + 1]; + sprintf(uuid_s, UUID_FMT, UUID_ARGS(&lflow->lflow_uuid)); + + struct resource_to_objects_node *lflow_od_res = + objdep_mgr_find_objs(lflowdep_mgr, OBJDEP_TYPE_LFLOW_OD, + uuid_s); + if (lflow_od_res) { + struct object_to_resources_list_node *r_node; + RESOURCE_FOR_EACH_OBJ (r_node, lflow_od_res) { + size_t index = r_node->obj_uuid.parts[0]; + bitmap_set0(lflow->dpg_bitmap, index); + } + } + } + } +} + +static void +sync_lflows_from_objres(struct resource_to_objects_node *res_node, + struct lflow_table *lflow_table, + struct objdep_mgr *lflowdep_mgr, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *sbflow_table, + const struct sbrec_logical_dp_group_table *dpgrp_table) +{ + if (!res_node) { + return; + } + + struct uuidset lflow_uuidset = UUIDSET_INITIALIZER(&lflow_uuidset); + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + + struct ovn_lflow *lflow = ovn_lflow_uuid_find( + &lflow_table->hash_map, obj_uuid); + if (!lflow) { + continue; + } + + const struct sbrec_logical_flow *sblflow = + sbrec_logical_flow_table_get_for_uuid(sbflow_table, + &lflow->sb_uuid); + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = ods_size(ls_datapaths); + } else { + n_datapaths = ods_size(lr_datapaths); + } + + size_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); + + if (n_ods) { + sync_lflow_to_sb(lflow, ovnsb_txn, lflow_table, ls_datapaths, + lr_datapaths, ovn_internal_version_changed, + sblflow, dpgrp_table); + } else { + if (sblflow) { + sbrec_logical_flow_delete(sblflow); + ovn_lflow_destroy(lflow_table, lflow); + } else { + VLOG_ERR("SB lflow "UUID_FMT" not found when " + "deleting lflows for resource %s (type %d). " + "This should not happen. Asserting", + UUID_ARGS(&lflow->sb_uuid), + resource_list_node->resource_node->res_name, + resource_list_node->resource_node->type); + ovs_assert(sblflow); + } + uuidset_insert(&lflow_uuidset, obj_uuid); + } + } + + struct uuidset_node *unode; + UUIDSET_FOR_EACH (unode, &lflow_uuidset) { + objdep_mgr_remove_obj(lflowdep_mgr, &unode->uuid); + } + uuidset_destroy(&lflow_uuidset); +} diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h new file mode 100644 index 0000000000..7d15b11432 --- /dev/null +++ b/northd/lflow-mgr.h @@ -0,0 +1,188 @@ + /* + * Copyright (c) 2023, Red Hat, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef LFLOW_MGR_H +#define LFLOW_MGR_H 1 + +#include "include/openvswitch/hmap.h" +#include "include/openvswitch/uuid.h" + +#include "northd.h" + +struct ovsdb_idl_txn; +struct ovn_datapath; +struct ovsdb_idl_row; + +/* lflow map which stores the logical flows. */ +struct lflow_table; +struct lflow_table *lflow_table_alloc(void); +void lflow_table_init(struct lflow_table *); +void lflow_table_clear(struct lflow_table *); +void lflow_table_destroy(struct lflow_table *); +void lflow_table_expand(struct lflow_table *); +void lflow_table_set_size(struct lflow_table *, size_t); +void lflow_table_sync_to_sb(struct lflow_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +void lflow_table_destroy(struct lflow_table *); + +void lflow_hash_lock_init(void); +void lflow_hash_lock_destroy(void); + +/* lflow_mgr manages logical flows for a resource (like logical port + * or datapath). */ +struct lflow_ref; + +/* Allocates an lflow manager. */ +struct lflow_ref *lflow_ref_alloc(const char *res_name); +void lflow_ref_set_od(struct lflow_ref *, const struct ovn_datapath *); +void lflow_ref_destroy(struct lflow_ref *); +void lflow_ref_clear_lflows(struct lflow_ref *, const struct ovn_datapath *, + struct lflow_table *); +void lflow_ref_clear_and_sync_lflows(struct lflow_ref *, + const struct ovn_datapath *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); +void lflow_ref_sync_lflows_to_sb(struct lflow_ref *, + struct lflow_table *lflow_table, + struct ovsdb_idl_txn *ovnsb_txn, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths, + bool ovn_internal_version_changed, + const struct sbrec_logical_flow_table *, + const struct sbrec_logical_dp_group_table *); + + +void lflow_table_add_lflow(struct lflow_table *, const struct ovn_datapath *, + const unsigned long *dp_bitmap, + size_t dp_bitmap_len, enum ovn_stage stage, + uint16_t priority, const char *match, + const char *actions, const char *io_port, + const char *ctrl_meter, + const struct ovsdb_idl_row *stage_hint, + const char *where, struct lflow_ref *); +void lflow_table_add_lflow_default_drop(struct lflow_table *, + const struct ovn_datapath *, + enum ovn_stage stage, + const char *where, + struct lflow_ref *); + +/* Adds a row with the specified contents to the Logical_Flow table. */ +#define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + CTRL_METER, STAGE_HINT, LFLOW_REF)\ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, STAGE_HINT, \ + LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ + STAGE, PRIORITY, MATCH, ACTIONS, \ + STAGE_HINT) \ + lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ + PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ + lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ + OVS_SOURCE_LOCATOR, NULL) + + +/* This macro is similar to ovn_lflow_add_with_hint, except that it requires + * the IN_OUT_PORT argument, which tells the lport name that appears in the + * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is + * not local to the chassis. The critiera of the lport to be added using this + * argument: + * + * - For ingress pipeline, the lport that is used to match "inport". + * - For egress pipeline, the lport that is used to match "outport". + * + * For now, only LS pipelines should use this macro. */ +#define ovn_lflow_add_with_lport_and_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ + MATCH, ACTIONS, IN_OUT_PORT, \ + STAGE_HINT, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ + OVS_SOURCE_LOCATOR, LFLOW_REF) + +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) + +#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, LFLOW_REF) \ + lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ + LFLOW_REF) + +#define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + CTRL_METER) \ + ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ + ACTIONS, NULL, CTRL_METER, NULL) + +struct sbrec_logical_dp_group; + +struct ovn_dp_group { + unsigned long *bitmap; + const struct sbrec_logical_dp_group *dp_group; + struct uuid dpg_uuid; + struct hmap_node node; + size_t refcnt; +}; + +static inline void +ovn_dp_groups_init(struct hmap *dp_groups) +{ + hmap_init(dp_groups); +} + +void ovn_dp_groups_destroy(struct hmap *dp_groups); +struct ovn_dp_group *ovn_dp_group_get_or_create( + struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, const unsigned long *desired_bitmap, + size_t bitmap_len, bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths); + +#endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index 7f0cbad0fa..eac3063a18 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -40,6 +40,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/lb.h" +#include "lflow-mgr.h" #include "memory.h" #include "northd.h" #include "en-lb-data.h" @@ -67,7 +68,7 @@ VLOG_DEFINE_THIS_MODULE(northd); static bool controller_event_en; -static bool lflow_hash_lock_initialized = false; + static bool check_lsp_is_up; @@ -96,116 +97,6 @@ static bool default_acl_drop; #define MAX_OVN_TAGS 4096 -/* Pipeline stages. */ - -/* The two purposes for which ovn-northd uses OVN logical datapaths. */ -enum ovn_datapath_type { - DP_SWITCH, /* OVN logical switch. */ - DP_ROUTER /* OVN logical router. */ -}; - -/* Returns an "enum ovn_stage" built from the arguments. - * - * (It's better to use ovn_stage_build() for type-safety reasons, but inline - * functions can't be used in enums or switch cases.) */ -#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ - (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) - -/* A stage within an OVN logical switch or router. - * - * An "enum ovn_stage" indicates whether the stage is part of a logical switch - * or router, whether the stage is part of the ingress or egress pipeline, and - * the table within that pipeline. The first three components are combined to - * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, - * S_ROUTER_OUT_DELIVERY. */ -enum ovn_stage { -#define PIPELINE_STAGES \ - /* Logical switch ingress stages. */ \ - PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ - PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ - PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ - PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ - PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ - PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ - PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ - PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ - PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ - PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ - "ls_in_acl_after_lb_eval") \ - PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ - "ls_in_acl_after_lb_action") \ - PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ - PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ - PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ - PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ - PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ - PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ - PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ - PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ - \ - /* Logical switch egress stages. */ \ - PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ - PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ - PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ - PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ - PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ - PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ - PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ - \ - /* Logical router ingress stages. */ \ - PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ - PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ - PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ - PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ - PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ - PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ - PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ - PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ - PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ - PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ - PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ - PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ - PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ - \ - /* Logical router egress stages. */ \ - PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ - "lr_out_chk_dnat_local") \ - PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ - PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ - PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ - PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ - PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") - -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - S_##DP_TYPE##_##PIPELINE##_##STAGE \ - = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), - PIPELINE_STAGES -#undef PIPELINE_STAGE -}; /* Due to various hard-coded priorities need to implement ACLs, the * northbound database supports a smaller range of ACL priorities than @@ -390,51 +281,9 @@ enum ovn_stage { #define ROUTE_PRIO_OFFSET_STATIC 1 #define ROUTE_PRIO_OFFSET_CONNECTED 2 -/* Returns an "enum ovn_stage" built from the arguments. */ -static enum ovn_stage -ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, - uint8_t table) -{ - return OVN_STAGE_BUILD(dp_type, pipeline, table); -} - -/* Returns the pipeline to which 'stage' belongs. */ -static enum ovn_pipeline -ovn_stage_get_pipeline(enum ovn_stage stage) -{ - return (stage >> 8) & 1; -} - -/* Returns the pipeline name to which 'stage' belongs. */ -static const char * -ovn_stage_get_pipeline_name(enum ovn_stage stage) -{ - return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; -} - -/* Returns the table to which 'stage' belongs. */ -static uint8_t -ovn_stage_get_table(enum ovn_stage stage) -{ - return stage & 0xff; -} - -/* Returns a string name for 'stage'. */ -static const char * -ovn_stage_to_str(enum ovn_stage stage) -{ - switch (stage) { -#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ - case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; - PIPELINE_STAGES -#undef PIPELINE_STAGE - default: return ""; - } -} - /* Returns the type of the datapath to which a flow with the given 'stage' may * be added. */ -static enum ovn_datapath_type +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage) { switch (stage) { @@ -679,13 +528,6 @@ ovn_datapath_destroy(struct hmap *datapaths, struct ovn_datapath *od) } } -/* Returns 'od''s datapath type. */ -static enum ovn_datapath_type -ovn_datapath_get_type(const struct ovn_datapath *od) -{ - return od->nbs ? DP_SWITCH : DP_ROUTER; -} - static struct ovn_datapath * ovn_datapath_find_(const struct hmap *datapaths, const struct uuid *uuid) @@ -721,13 +563,7 @@ ovn_datapath_find_by_key(struct hmap *datapaths, uint32_t dp_key) return NULL; } -static bool -ovn_datapath_is_stale(const struct ovn_datapath *od) -{ - return !od->nbr && !od->nbs; -} - -static struct ovn_datapath * +struct ovn_datapath * ovn_datapath_from_sbrec(const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, const struct sbrec_datapath_binding *sb) @@ -1290,19 +1126,6 @@ struct ovn_port_routable_addresses { size_t n_addrs; }; -/* A node that maintains link between an object (such as an ovn_port) and - * a lflow. */ -struct lflow_ref_node { - /* This list follows different lflows referenced by the same object. List - * head is, for example, ovn_port->lflows. */ - struct ovs_list lflow_list_node; - /* This list follows different objects that reference the same lflow. List - * head is ovn_lflow->referenced_by. */ - struct ovs_list ref_list_node; - /* The lflow. */ - struct ovn_lflow *lflow; -}; - static bool lsp_can_be_inc_processed(const struct nbrec_logical_switch_port *); static bool @@ -1382,6 +1205,8 @@ ovn_port_set_nb(struct ovn_port *op, init_mcast_port_info(&op->mcast_info, op->nbsp, op->nbrp); } +static bool lsp_is_router(const struct nbrec_logical_switch_port *nbsp); + static struct ovn_port * ovn_port_create(struct hmap *ports, const char *key, const struct nbrec_logical_switch_port *nbsp, @@ -1400,12 +1225,22 @@ ovn_port_create(struct hmap *ports, const char *key, op->l3dgw_port = op->cr_port = NULL; hmap_insert(ports, &op->key_node, hash_string(op->key, 0)); - ovs_list_init(&op->lflows); + op->lflow_ref = lflow_ref_alloc(key); + op->lbnat_lflow_ref = lflow_ref_alloc(key); + return op; } static void -ovn_port_destroy_orphan(struct ovn_port *port) +ovn_port_set_od(struct ovn_port *op, struct ovn_datapath *od) +{ + op->od = od; + lflow_ref_set_od(op->lflow_ref, od); + lflow_ref_set_od(op->lbnat_lflow_ref, od); +} + +static void +ovn_port_cleanup(struct ovn_port *port) { if (port->tunnel_key) { ovs_assert(port->od); @@ -1415,6 +1250,8 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->lsp_addrs[i]); } free(port->lsp_addrs); + port->n_lsp_addrs = 0; + port->lsp_addrs = NULL; if (port->peer) { port->peer->peer = NULL; @@ -1424,18 +1261,22 @@ ovn_port_destroy_orphan(struct ovn_port *port) destroy_lport_addresses(&port->ps_addrs[i]); } free(port->ps_addrs); + port->ps_addrs = NULL; + port->n_ps_addrs = 0; destroy_lport_addresses(&port->lrp_networks); destroy_lport_addresses(&port->proxy_arp_addrs); +} + +static void +ovn_port_destroy_orphan(struct ovn_port *port) +{ + ovn_port_cleanup(port); free(port->json_key); free(port->key); + lflow_ref_destroy(port->lflow_ref); + lflow_ref_destroy(port->lbnat_lflow_ref); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, lflow_list_node, &port->lflows) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } free(port); } @@ -2364,7 +2205,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, parse_lsp_addrs(op); - op->od = od; + ovn_port_set_od(op, od); if (op->has_unknown) { od->has_unknown = true; } @@ -2415,7 +2256,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, } op->lrp_networks = lrp_networks; - op->od = od; + ovn_port_set_od(op, od); hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); @@ -2455,7 +2296,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table, } crp->l3dgw_port = op; op->cr_port = crp; - crp->od = od; + ovn_port_set_od(crp, od); free(redirect_name); /* Add to l3dgw_ports in od, for later use during flow @@ -3898,124 +3739,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } - -struct ovn_dp_group { - unsigned long *bitmap; - struct sbrec_logical_dp_group *dp_group; - struct hmap_node node; -}; - -static struct ovn_dp_group * -ovn_dp_group_find(const struct hmap *dp_groups, - const unsigned long *dpg_bitmap, size_t bitmap_len, - uint32_t hash) -{ - struct ovn_dp_group *dpg; - - HMAP_FOR_EACH_WITH_HASH (dpg, node, hash, dp_groups) { - if (bitmap_equal(dpg->bitmap, dpg_bitmap, bitmap_len)) { - return dpg; - } - } - return NULL; -} - -static struct sbrec_logical_dp_group * -ovn_sb_insert_or_update_logical_dp_group( - struct ovsdb_idl_txn *ovnsb_txn, - struct sbrec_logical_dp_group *dp_group, - const unsigned long *dpg_bitmap, - const struct ovn_datapaths *datapaths) -{ - const struct sbrec_datapath_binding **sb; - size_t n = 0, index; - - sb = xmalloc(bitmap_count1(dpg_bitmap, ods_size(datapaths)) * sizeof *sb); - BITMAP_FOR_EACH_1 (index, ods_size(datapaths), dpg_bitmap) { - sb[n++] = datapaths->array[index]->sb; - } - if (!dp_group) { - dp_group = sbrec_logical_dp_group_insert(ovnsb_txn); - } - sbrec_logical_dp_group_set_datapaths( - dp_group, (struct sbrec_datapath_binding **) sb, n); - free(sb); - - return dp_group; -} - -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ -static struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - uint32_t hash; - - hash = hash_int(desired_n, 0); - dpg = ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); - if (dpg) { - return dpg; - } - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - hmap_insert(dp_groups, &dpg->node, hash); - - return dpg; -} - struct sb_lb { struct hmap_node hmap_node; @@ -4873,28 +4596,20 @@ ovn_port_find_in_datapath(struct ovn_datapath *od, const char *name) return NULL; } -static struct ovn_port * -ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, - const char *key, const struct nbrec_logical_switch_port *nbsp, - struct ovn_datapath *od, const struct sbrec_port_binding *sb, - struct ovs_list *lflows, - const struct sbrec_mirror_table *sbrec_mirror_table, - const struct sbrec_chassis_table *sbrec_chassis_table, - struct ovsdb_idl_index *sbrec_chassis_by_name, - struct ovsdb_idl_index *sbrec_chassis_by_hostname) -{ - struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, - NULL); +static bool +ls_port_init(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + ovn_port_set_od(op, od); parse_lsp_addrs(op); - op->od = od; - hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); - if (lflows) { - ovs_list_splice(&op->lflows, lflows->next, lflows); - } - /* Assign explicitly requested tunnel ids first. */ if (!ovn_port_assign_requested_tnl_id(sbrec_chassis_table, op)) { - return NULL; + return false; } if (sb) { op->sb = sb; @@ -4911,14 +4626,57 @@ ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, } /* Assign new tunnel ids where needed. */ if (!ovn_port_allocate_key(sbrec_chassis_table, ls_ports, op)) { - return NULL; + return false; } ovn_port_update_sbrec(ovnsb_txn, sbrec_chassis_by_name, sbrec_chassis_by_hostname, NULL, sbrec_mirror_table, op, NULL, NULL); + return true; +} + +static struct ovn_port * +ls_port_create(struct ovsdb_idl_txn *ovnsb_txn, struct hmap *ls_ports, + const char *key, const struct nbrec_logical_switch_port *nbsp, + struct ovn_datapath *od, const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + struct ovn_port *op = ovn_port_create(ls_ports, key, nbsp, NULL, + NULL); + hmap_insert(&od->ports, &op->dp_node, hmap_node_hash(&op->key_node)); + if (!ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname)) { + ovn_port_destroy(ls_ports, op); + return NULL; + } + return op; } +static bool +ls_port_reinit(struct ovn_port *op, struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *ls_ports, + const struct nbrec_logical_switch_port *nbsp, + const struct nbrec_logical_router_port *nbrp, + struct ovn_datapath *od, + const struct sbrec_port_binding *sb, + const struct sbrec_mirror_table *sbrec_mirror_table, + const struct sbrec_chassis_table *sbrec_chassis_table, + struct ovsdb_idl_index *sbrec_chassis_by_name, + struct ovsdb_idl_index *sbrec_chassis_by_hostname) +{ + ovn_port_cleanup(op); + op->sb = sb; + ovn_port_set_nb(op, nbsp, nbrp); + op->l3dgw_port = op->cr_port = NULL; + return ls_port_init(op, ovnsb_txn, ls_ports, od, sb, + sbrec_mirror_table, sbrec_chassis_table, + sbrec_chassis_by_name, sbrec_chassis_by_hostname); +} + /* Returns true if the logical switch has changes which can be * incrementally handled. * Presently supports i-p for the below changes: @@ -5058,7 +4816,7 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, goto fail; } op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, NULL, NULL, + new_nbsp->name, new_nbsp, od, NULL, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, @@ -5089,19 +4847,15 @@ ls_handle_lsp_changes(struct ovsdb_idl_txn *ovnsb_idl_txn, op->visited = true; continue; } - struct ovs_list lflows = OVS_LIST_INITIALIZER(&lflows); - ovs_list_splice(&lflows, op->lflows.next, &op->lflows); - ovn_port_destroy(&nd->ls_ports, op); - op = ls_port_create(ovnsb_idl_txn, &nd->ls_ports, - new_nbsp->name, new_nbsp, od, sb, &lflows, - ni->sbrec_mirror_table, + if (!ls_port_reinit(op, ovnsb_idl_txn, &nd->ls_ports, + new_nbsp, NULL, + od, sb, ni->sbrec_mirror_table, ni->sbrec_chassis_table, ni->sbrec_chassis_by_name, - ni->sbrec_chassis_by_hostname); - ovs_assert(ovs_list_is_empty(&lflows)); - if (!op) { + ni->sbrec_chassis_by_hostname)) { goto fail; } + add_op_to_northd_tracked_ports(&trk_ports->updated, op); } op->visited = true; @@ -5979,170 +5733,7 @@ ovn_igmp_group_destroy(struct hmap *igmp_groups, * function of most of the northbound database. */ -struct ovn_lflow { - struct hmap_node hmap_node; - struct ovs_list list_node; /* For temporary list of lflows. Don't remove - at destroy. */ - - struct ovn_datapath *od; /* 'logical_datapath' in SB schema. */ - unsigned long *dpg_bitmap; /* Bitmap of all datapaths by their 'index'.*/ - enum ovn_stage stage; - uint16_t priority; - char *match; - char *actions; - char *io_port; - char *stage_hint; - char *ctrl_meter; - size_t n_ods; /* Number of datapaths referenced by 'od' and - * 'dpg_bitmap'. */ - struct ovn_dp_group *dpg; /* Link to unique Sb datapath group. */ - - struct ovs_list referenced_by; /* List of struct lflow_ref_node. */ - const char *where; - - struct uuid sb_uuid; /* SB DB row uuid, specified by northd. */ -}; - -static void ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow); -static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows, - const struct ovn_datapath *od, - enum ovn_stage stage, - uint16_t priority, const char *match, - const char *actions, - const char *ctrl_meter, uint32_t hash); - -static char * -ovn_lflow_hint(const struct ovsdb_idl_row *row) -{ - if (!row) { - return NULL; - } - return xasprintf("%08x", row->uuid.parts[0]); -} - -static bool -ovn_lflow_equal(const struct ovn_lflow *a, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, const char *match, - const char *actions, const char *ctrl_meter) -{ - return (a->od == od - && a->stage == stage - && a->priority == priority - && !strcmp(a->match, match) - && !strcmp(a->actions, actions) - && nullable_string_is_equal(a->ctrl_meter, ctrl_meter)); -} - -enum { - STATE_NULL, /* parallelization is off */ - STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ - STATE_USE_PARALLELIZATION /* parallelization is on */ -}; -static int parallelization_state = STATE_NULL; - -static void -ovn_lflow_init(struct ovn_lflow *lflow, struct ovn_datapath *od, - size_t dp_bitmap_len, enum ovn_stage stage, uint16_t priority, - char *match, char *actions, char *io_port, char *ctrl_meter, - char *stage_hint, const char *where) -{ - ovs_list_init(&lflow->list_node); - ovs_list_init(&lflow->referenced_by); - lflow->dpg_bitmap = bitmap_allocate(dp_bitmap_len); - lflow->od = od; - lflow->stage = stage; - lflow->priority = priority; - lflow->match = match; - lflow->actions = actions; - lflow->io_port = io_port; - lflow->stage_hint = stage_hint; - lflow->ctrl_meter = ctrl_meter; - lflow->dpg = NULL; - lflow->where = where; - lflow->sb_uuid = UUID_ZERO; -} - -/* The lflow_hash_lock is a mutex array that protects updates to the shared - * lflow table across threads when parallel lflow build and dp-group are both - * enabled. To avoid high contention between threads, a big array of mutexes - * are used instead of just one. This is possible because when parallel build - * is used we only use hmap_insert_fast() to update the hmap, which would not - * touch the bucket array but only the list in a single bucket. We only need to - * make sure that when adding lflows to the same hash bucket, the same lock is - * used, so that no two threads can add to the bucket at the same time. It is - * ok that the same lock is used to protect multiple buckets, so a fixed sized - * mutex array is used instead of 1-1 mapping to the hash buckets. This - * simplies the implementation while effectively reduces lock contention - * because the chance that different threads contending the same lock amongst - * the big number of locks is very low. */ -#define LFLOW_HASH_LOCK_MASK 0xFFFF -static struct ovs_mutex lflow_hash_locks[LFLOW_HASH_LOCK_MASK + 1]; - -static void -lflow_hash_lock_init(void) -{ - if (!lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_init(&lflow_hash_locks[i]); - } - lflow_hash_lock_initialized = true; - } -} - -static void -lflow_hash_lock_destroy(void) -{ - if (lflow_hash_lock_initialized) { - for (size_t i = 0; i < LFLOW_HASH_LOCK_MASK + 1; i++) { - ovs_mutex_destroy(&lflow_hash_locks[i]); - } - } - lflow_hash_lock_initialized = false; -} - -/* Full thread safety analysis is not possible with hash locks, because - * they are taken conditionally based on the 'parallelization_state' and - * a flow hash. Also, the order in which two hash locks are taken is not - * predictable during the static analysis. - * - * Since the order of taking two locks depends on a random hash, to avoid - * ABBA deadlocks, no two hash locks can be nested. In that sense an array - * of hash locks is similar to a single mutex. - * - * Using a fake mutex to partially simulate thread safety restrictions, as - * if it were actually a single mutex. - * - * OVS_NO_THREAD_SAFETY_ANALYSIS below allows us to ignore conditional - * nature of the lock. Unlike other attributes, it applies to the - * implementation and not to the interface. So, we can define a function - * that acquires the lock without analysing the way it does that. - */ -extern struct ovs_mutex fake_hash_mutex; - -static struct ovs_mutex * -lflow_hash_lock(const struct hmap *lflow_map, uint32_t hash) - OVS_ACQUIRES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - struct ovs_mutex *hash_lock = NULL; - - if (parallelization_state == STATE_USE_PARALLELIZATION) { - hash_lock = - &lflow_hash_locks[hash & lflow_map->mask & LFLOW_HASH_LOCK_MASK]; - ovs_mutex_lock(hash_lock); - } - return hash_lock; -} - -static void -lflow_hash_unlock(struct ovs_mutex *hash_lock) - OVS_RELEASES(fake_hash_mutex) - OVS_NO_THREAD_SAFETY_ANALYSIS -{ - if (hash_lock) { - ovs_mutex_unlock(hash_lock); - } -} +int parallelization_state = STATE_NULL; /* This thread-local var is used for parallel lflow building when dp-groups is @@ -6155,240 +5746,7 @@ lflow_hash_unlock(struct ovs_mutex *hash_lock) * threads are collected to fix the lflow hmap's size (by the function * fix_flow_map_size()). * */ -static thread_local size_t thread_lflow_counter = 0; - -/* Adds an OVN datapath to a datapath group of existing logical flow. - * Version to use when hash bucket locking is NOT required or the corresponding - * hash lock is already taken. */ -static void -ovn_dp_group_add_with_reference(struct ovn_lflow *lflow_ref, - const struct ovn_datapath *od, - const unsigned long *dp_bitmap, - size_t bitmap_len) - OVS_REQUIRES(fake_hash_mutex) -{ - if (od) { - bitmap_set1(lflow_ref->dpg_bitmap, od->index); - } - if (dp_bitmap) { - bitmap_or(lflow_ref->dpg_bitmap, dp_bitmap, bitmap_len); - } -} - -/* This global variable collects the lflows generated by do_ovn_lflow_add(). - * start_collecting_lflows() will enable the lflow collection and the calls to - * do_ovn_lflow_add (or the macros ovn_lflow_add_...) will add generated lflows - * to the list. end_collecting_lflows() will disable it. */ -static thread_local struct ovs_list collected_lflows; -static thread_local bool collecting_lflows = false; - -static void -start_collecting_lflows(void) -{ - ovs_assert(!collecting_lflows); - ovs_list_init(&collected_lflows); - collecting_lflows = true; -} - -static void -end_collecting_lflows(void) -{ - ovs_assert(collecting_lflows); - collecting_lflows = false; -} - -/* Adds a row with the specified contents to the Logical_Flow table. - * Version to use when hash bucket locking is NOT required. */ -static void -do_ovn_lflow_add(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - uint32_t hash, enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const struct ovsdb_idl_row *stage_hint, - const char *where, const char *ctrl_meter) - OVS_REQUIRES(fake_hash_mutex) -{ - - struct ovn_lflow *old_lflow; - struct ovn_lflow *lflow; - - size_t bitmap_len = od ? ods_size(od->datapaths) : dp_bitmap_len; - ovs_assert(bitmap_len); - - if (collecting_lflows) { - ovs_assert(od); - ovs_assert(!dp_bitmap); - } else { - old_lflow = ovn_lflow_find(lflow_map, NULL, stage, priority, match, - actions, ctrl_meter, hash); - if (old_lflow) { - ovn_dp_group_add_with_reference(old_lflow, od, dp_bitmap, - bitmap_len); - return; - } - } - - lflow = xmalloc(sizeof *lflow); - /* While adding new logical flows we're not setting single datapath, but - * collecting a group. 'od' will be updated later for all flows with only - * one datapath in a group, so it could be hashed correctly. */ - ovn_lflow_init(lflow, NULL, bitmap_len, stage, priority, - xstrdup(match), xstrdup(actions), - io_port ? xstrdup(io_port) : NULL, - nullable_xstrdup(ctrl_meter), - ovn_lflow_hint(stage_hint), where); - - ovn_dp_group_add_with_reference(lflow, od, dp_bitmap, bitmap_len); - - if (parallelization_state != STATE_USE_PARALLELIZATION) { - hmap_insert(lflow_map, &lflow->hmap_node, hash); - } else { - hmap_insert_fast(lflow_map, &lflow->hmap_node, hash); - thread_lflow_counter++; - } - - if (collecting_lflows) { - ovs_list_insert(&collected_lflows, &lflow->list_node); - } -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -static void -ovn_lflow_add_at(struct hmap *lflow_map, const struct ovn_datapath *od, - const unsigned long *dp_bitmap, size_t dp_bitmap_len, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *io_port, - const char *ctrl_meter, - const struct ovsdb_idl_row *stage_hint, const char *where) - OVS_EXCLUDED(fake_hash_mutex) -{ - struct ovs_mutex *hash_lock; - uint32_t hash; - - ovs_assert(!od || - ovn_stage_to_datapath_type(stage) == ovn_datapath_get_type(od)); - - hash = ovn_logical_flow_hash(ovn_stage_get_table(stage), - ovn_stage_get_pipeline(stage), - priority, match, - actions); - - hash_lock = lflow_hash_lock(lflow_map, hash); - do_ovn_lflow_add(lflow_map, od, dp_bitmap, dp_bitmap_len, hash, stage, - priority, match, actions, io_port, stage_hint, where, - ctrl_meter); - lflow_hash_unlock(hash_lock); -} - -static void -__ovn_lflow_add_default_drop(struct hmap *lflow_map, - struct ovn_datapath *od, - enum ovn_stage stage, - const char *where) -{ - ovn_lflow_add_at(lflow_map, od, NULL, 0, stage, 0, "1", - debug_drop_action(), - NULL, NULL, NULL, where ); -} - -/* Adds a row with the specified contents to the Logical_Flow table. */ -#define ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, CTRL_METER, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_hint(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_with_dp_group(LFLOW_MAP, DP_BITMAP, DP_BITMAP_LEN, \ - STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ - PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE) \ - __ovn_lflow_add_default_drop(LFLOW_MAP, OD, STAGE, OVS_SOURCE_LOCATOR) - - -/* This macro is similar to ovn_lflow_add_with_hint, except that it requires - * the IN_OUT_PORT argument, which tells the lport name that appears in the - * MATCH, which helps ovn-controller to bypass lflows parsing when the lport is - * not local to the chassis. The critiera of the lport to be added using this - * argument: - * - * - For ingress pipeline, the lport that is used to match "inport". - * - For egress pipeline, the lport that is used to match "outport". - * - * For now, only LS pipelines should use this macro. */ -#define ovn_lflow_add_with_lport_and_hint(LFLOW_MAP, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - STAGE_HINT) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - IN_OUT_PORT, NULL, STAGE_HINT, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_add(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - ovn_lflow_add_at(LFLOW_MAP, OD, NULL, 0, STAGE, PRIORITY, MATCH, ACTIONS, \ - NULL, NULL, NULL, OVS_SOURCE_LOCATOR) - -#define ovn_lflow_metered(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ - ovn_lflow_add_with_hint__(LFLOW_MAP, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) - -static struct ovn_lflow * -ovn_lflow_find(const struct hmap *lflows, const struct ovn_datapath *od, - enum ovn_stage stage, uint16_t priority, - const char *match, const char *actions, const char *ctrl_meter, - uint32_t hash) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_WITH_HASH (lflow, hmap_node, hash, lflows) { - if (ovn_lflow_equal(lflow, od, stage, priority, match, actions, - ctrl_meter)) { - return lflow; - } - } - return NULL; -} - -static void -ovn_lflow_destroy(struct hmap *lflows, struct ovn_lflow *lflow) -{ - if (lflow) { - if (lflows) { - hmap_remove(lflows, &lflow->hmap_node); - } - bitmap_free(lflow->dpg_bitmap); - free(lflow->match); - free(lflow->actions); - free(lflow->io_port); - free(lflow->stage_hint); - free(lflow->ctrl_meter); - struct lflow_ref_node *l; - LIST_FOR_EACH_SAFE (l, ref_list_node, &lflow->referenced_by) { - ovs_list_remove(&l->lflow_list_node); - ovs_list_remove(&l->ref_list_node); - free(l); - } - free(lflow); - } -} - -static void -link_ovn_port_to_lflows(struct ovn_port *op, struct ovs_list *lflows) -{ - struct ovn_lflow *f; - LIST_FOR_EACH (f, list_node, lflows) { - struct lflow_ref_node *lfrn = xmalloc(sizeof *lfrn); - lfrn->lflow = f; - ovs_list_insert(&op->lflows, &lfrn->lflow_list_node); - ovs_list_insert(&f->referenced_by, &lfrn->ref_list_node); - } -} +thread_local size_t thread_lflow_counter = 0; static bool build_dhcpv4_action(struct ovn_port *op, ovs_be32 offer_ip, @@ -6565,7 +5923,7 @@ build_dhcpv6_action(struct ovn_port *op, struct in6_addr *offer_ip, * build_lswitch_lflows_admission_control() handles the port security. */ static void -build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, +build_lswitch_port_sec_op(struct ovn_port *op, struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6582,13 +5940,13 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 100, ds_cstr(match), REGBIT_PORT_SEC_DROP" = 1; next;", - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, op->lflow_ref); ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_L2_UNKNOWN, 50, ds_cstr(match), - debug_drop_action(), op->key, &op->nbsp->header_); + debug_drop_action(), op->key, &op->nbsp->header_, op->lflow_ref); return; } @@ -6604,14 +5962,16 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (queue_id) { ds_put_cstr(actions, REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_CHECK_PORT_SEC, 70, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); if (!lsp_is_localnet(op->nbsp) && !op->od->n_localnet_ports) { return; @@ -6626,7 +5986,8 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_OUT_APPLY_PORT_SEC, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); } else if (op->od->n_localnet_ports) { ds_put_format(match, "outport == %s && inport == %s", op->od->localnet_ports[0]->json_key, @@ -6635,14 +5996,15 @@ build_lswitch_port_sec_op(struct ovn_port *op, struct hmap *lflows, S_SWITCH_OUT_APPLY_PORT_SEC, 110, ds_cstr(match), ds_cstr(actions), op->od->localnet_ports[0]->key, - &op->od->localnet_ports[0]->nbsp->header_); + &op->od->localnet_ports[0]->nbsp->header_, + op->lflow_ref); } } } static void build_lswitch_learn_fdb_op( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -6657,7 +6019,8 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_LOOKUP_FDB, 100, ds_cstr(match), ds_cstr(actions), - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, + op->lflow_ref); ds_put_cstr(match, " && "REGBIT_LKUP_FDB" == 0"); ds_clear(actions); @@ -6665,13 +6028,14 @@ build_lswitch_learn_fdb_op( ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_PUT_FDB, 100, ds_cstr(match), ds_cstr(actions), op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); @@ -6685,7 +6049,7 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, @@ -6703,7 +6067,7 @@ static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct hmap *lflows) + struct lflow_table *lflows) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6725,10 +6089,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_); + op->key, &op->nbsp->header_, NULL); free(ingress_match); free(egress_match); @@ -6737,7 +6101,7 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct hmap *lflows) + struct lflow_table *lflows) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6758,7 +6122,7 @@ build_stateless_filter(const struct ovn_datapath *od, static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) + struct lflow_table *lflows) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; @@ -6786,7 +6150,7 @@ build_stateless_filters(const struct ovn_datapath *od, } static void -build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ @@ -6803,7 +6167,7 @@ build_pre_acls(struct ovn_datapath *od, struct hmap *lflows) static void build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, const struct ls_port_group_table *ls_port_groups, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -6922,7 +6286,7 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -6956,7 +6320,7 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { /* Handle IGMP/MLD packets crossing AZs. */ build_interconn_mcast_snoop_flows(od, meter_groups, lflows); @@ -6992,7 +6356,7 @@ build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, static void build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -7058,7 +6422,7 @@ build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ @@ -7090,7 +6454,7 @@ build_pre_stateful(struct ovn_datapath *od, static void build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -7259,7 +6623,7 @@ build_acl_log(struct ds *actions, const struct nbrec_acl *acl, } static void -consider_acl(struct hmap *lflows, const struct ovn_datapath *od, +consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, uint64_t max_acl_tier, struct ds *match, struct ds *actions) @@ -7489,7 +6853,7 @@ ovn_update_ipv6_options(struct hmap *lr_ports) static void build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, - struct hmap *lflows, + struct lflow_table *lflows, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, @@ -7566,7 +6930,8 @@ build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, } static void -build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, +build_acl_log_related_flows(const struct ovn_datapath *od, + struct lflow_table *lflows, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, @@ -7641,7 +7006,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, struct hmap *lflows, static void build_acls(const struct ls_lbacls_record *ls_lbacls_rec, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, const struct shash *meter_groups) { @@ -7887,7 +7252,7 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, } static void -build_qos(struct ovn_datapath *od, struct hmap *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { struct ds action = DS_EMPTY_INITIALIZER; ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); @@ -7948,7 +7313,7 @@ build_qos(struct ovn_datapath *od, struct hmap *lflows) { } static void -build_lb_rules_pre_stateful(struct hmap *lflows, +build_lb_rules_pre_stateful(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, @@ -8050,7 +7415,8 @@ build_lb_rules_pre_stateful(struct hmap *lflows, * */ static void -build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, +build_lb_affinity_lr_flows(struct lflow_table *lflows, + const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, const struct ovn_datapaths *lr_datapaths) @@ -8236,7 +7602,7 @@ build_lb_affinity_lr_flows(struct hmap *lflows, const struct ovn_northd_lb *lb, * */ static void -build_lb_affinity_ls_flows(struct hmap *lflows, +build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, const struct ovn_datapaths *ls_datapaths) @@ -8379,7 +7745,7 @@ build_lb_affinity_ls_flows(struct hmap *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8388,7 +7754,7 @@ build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); @@ -8396,7 +7762,7 @@ build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, } static void -build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, +build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, @@ -8476,7 +7842,7 @@ build_lb_rules(struct hmap *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows) + struct lflow_table *lflows) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -8526,7 +7892,7 @@ build_stateful(struct ovn_datapath *od, static void build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, - struct hmap *lflows) + struct lflow_table *lflows) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -8585,7 +7951,7 @@ build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, } static void -build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -8633,7 +7999,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct hmap *lflows) /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct hmap *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8814,7 +8180,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, uint32_t priority, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds eth_src = DS_EMPTY_INITIALIZER; struct ds match = DS_EMPTY_INITIALIZER; @@ -8838,8 +8205,10 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;"); + ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, + ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8907,8 +8276,9 @@ static void build_lswitch_rport_arp_req_flow(const char *ips, int addr_family, struct ovn_port *patch_op, const struct ovn_datapath *od, - uint32_t priority, struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + uint32_t priority, struct lflow_table *lflows, + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -8922,14 +8292,17 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8947,7 +8320,7 @@ static void build_lswitch_rport_arp_req_flows(struct ovn_port *op, struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovsdb_idl_row *stage_hint) { if (!op || !op->nbrp) { @@ -8965,12 +8338,12 @@ build_lswitch_rport_arp_req_flows(struct ovn_port *op, for (size_t i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv4_addrs[i].addr_s, AF_INET, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } for (size_t i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { build_lswitch_rport_arp_req_flow( op->lrp_networks.ipv6_addrs[i].addr_s, AF_INET6, sw_op, sw_od, 80, - lflows, stage_hint); + lflows, stage_hint, sw_op->lflow_ref); } } @@ -8986,8 +8359,9 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, const struct ovn_datapath *sw_od, struct ovn_port *sw_op, - struct hmap *lflows, - const struct ovsdb_idl_row *stage_hint) + struct lflow_table *lflows, + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { if (!op || !op->nbrp) { return; @@ -9015,7 +8389,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, lrouter_port_ipv4_reachable(op, ipv4_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } SSET_FOR_EACH (ip_addr, &lr_lbnat_rec->lb_ips->ips_v6_reachable) { @@ -9028,7 +8402,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, lrouter_port_ipv6_reachable(op, &ipv6_addr)) { build_lswitch_rport_arp_req_flow( ip_addr, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9043,7 +8417,7 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, if (sw_od->n_router_ports != sw_od->nbs->n_ports) { build_lswitch_rport_arp_req_self_orig_flow(op, 75, sw_od, lr_lbnat_rec->lrnat_rec, - lflows); + lflows, lflow_ref); } for (size_t i = 0; i < lr_lbnat_rec->lrnat_rec->n_nat_entries; i++) { @@ -9066,14 +8440,14 @@ build_lswitch_rport_arp_req_flows_for_lbnats(struct ovn_port *op, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET6, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } else { if (!sset_contains(&lr_lbnat_rec->lb_ips->ips_v4, nat->external_ip)) { build_lswitch_rport_arp_req_flow( nat->external_ip, AF_INET, sw_op, sw_od, 80, lflows, - stage_hint); + stage_hint, lflow_ref); } } } @@ -9084,7 +8458,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9107,7 +8481,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9115,7 +8489,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_clear(&match); /* Allow ip4.src = OFFER_IP and * ip4.dst = {SERVER_IP, 255.255.255.255} for the below @@ -9135,7 +8510,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9143,7 +8518,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV4_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -9162,7 +8538,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -9189,7 +8566,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, - &op->nbsp->dhcpv4_options->header_); + &op->nbsp->dhcpv4_options->header_, + op->lflow_ref); } break; } @@ -9202,7 +8580,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9224,7 +8602,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_hint__(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -9232,7 +8610,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, copp_meter_get(COPP_DHCPV6_OPTS, op->od->nbs->copp, meter_groups), - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -9240,7 +8619,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, op->lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -9272,7 +8651,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, - &op->nbsp->dhcpv6_options->header_); + &op->nbsp->dhcpv6_options->header_, + op->lflow_ref); } break; } @@ -9283,7 +8663,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; @@ -9303,7 +8683,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -9319,7 +8699,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } ds_clear(&match); @@ -9335,7 +8715,8 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9350,7 +8731,7 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { @@ -9371,7 +8752,7 @@ static void build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9387,7 +8768,7 @@ build_lswitch_lflows_pre_acl_and_acl( * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ @@ -9415,7 +8796,7 @@ build_lswitch_lflows_admission_control(struct ovn_datapath *od, static void build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbsp); @@ -9427,14 +8808,14 @@ build_lswitch_arp_nd_responder_skip_local(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, op->lflow_ref); } /* Ingress table 19: ARP/ND responder, reply for known IPs. * (priority 50). */ static void build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct shash *meter_groups, struct ds *actions, @@ -9519,7 +8900,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), ds_cstr(actions), vparent, - &vp->nbsp->header_); + &vp->nbsp->header_, + op->lflow_ref); } free(tokstr); @@ -9563,11 +8945,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9586,7 +8969,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } /* For ND solicitations, we need to listen for both the @@ -9616,15 +9000,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9633,7 +9018,8 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, S_SWITCH_IN_ARP_ND_RSP, 100, ds_cstr(match), "next;", op->key, - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -9679,8 +9065,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9723,15 +9113,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9742,7 +9133,7 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); @@ -9753,7 +9144,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -9829,7 +9220,7 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, * priority 100 flows. */ static void build_lswitch_dhcp_options_and_response(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(op->nbsp); @@ -9884,7 +9275,7 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); @@ -9899,7 +9290,7 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, */ static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbs); @@ -9930,7 +9321,7 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, * binding the external ports. */ static void build_lswitch_external_port(struct ovn_port *op, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); if (!lsp_is_external(op->nbsp)) { @@ -9946,7 +9337,7 @@ build_lswitch_external_port(struct ovn_port *op, * (priority 70 - 100). */ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, const struct shash *meter_groups) { @@ -10039,7 +9430,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, * (priority 90). */ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { @@ -10119,7 +9510,8 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, /* Ingress table 25: Destination lookup, unicast handling (priority 50), */ static void -build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, +build_lswitch_ip_unicast_lookup(struct ovn_port *op, + struct lflow_table *lflows, struct ds *actions, struct ds *match) { ovs_assert(op->nbsp); @@ -10152,10 +9544,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -10170,10 +9564,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -10225,10 +9621,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -10243,8 +9640,8 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, struct hmap *lflows, static void build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct hmap *lflows, struct ds *match, - struct ds *actions) + struct lflow_table *lflows, struct ds *match, + struct ds *actions, struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); @@ -10276,11 +9673,12 @@ build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_hint(lflows, op->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + lflow_ref); } } } @@ -10520,7 +9918,7 @@ get_outport_for_routing_policy_nexthop(struct ovn_datapath *od, } static void -build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, +build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, const struct ovsdb_idl_row *stage_hint) @@ -10585,7 +9983,8 @@ build_routing_policy_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -build_ecmp_routing_policy_flows(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_routing_policy_flows(struct lflow_table *lflows, + struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, uint16_t ecmp_group_id) @@ -10721,7 +10120,7 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) } static void -build_route_table_lflow(struct ovn_datapath *od, struct hmap *lflows, +build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, struct simap *route_tables) { @@ -11132,7 +10531,7 @@ find_static_route_outport(struct ovn_datapath *od, const struct hmap *lr_ports, } static void -add_ecmp_symmetric_reply_flows(struct hmap *lflows, +add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const char *port_ip, @@ -11297,7 +10696,7 @@ add_ecmp_symmetric_reply_flows(struct hmap *lflows, } static void -build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, struct ecmp_groups_node *eg) @@ -11384,12 +10783,12 @@ build_ecmp_route_flow(struct hmap *lflows, struct ovn_datapath *od, } static void -add_route(struct hmap *lflows, struct ovn_datapath *od, +add_route(struct lflow_table *lflows, struct ovn_datapath *od, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, const char *gateway, bool is_src_route, const uint32_t rtb_id, const struct ovsdb_idl_row *stage_hint, bool is_discard_route, - int ofs) + int ofs, struct lflow_ref *lflow_ref) { bool is_ipv4 = strchr(network_s, '.') ? true : false; struct ds match = DS_EMPTY_INITIALIZER; @@ -11432,14 +10831,17 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&match), ds_cstr(&actions), - stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions), stage_hint); + ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -11447,7 +10849,7 @@ add_route(struct hmap *lflows, struct ovn_datapath *od, } static void -build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, +build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct parsed_route *route_) { @@ -11473,7 +10875,7 @@ build_static_route_flow(struct hmap *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs); + route_->is_discard_route, ofs, NULL); free(prefix_s); } @@ -11536,7 +10938,7 @@ struct lrouter_nat_lb_flows_ctx { int prio; - struct hmap *lflows; + struct lflow_table *lflows; const struct shash *meter_groups; }; @@ -11667,7 +11069,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ovn_northd_lb_vip *vips_nb, const struct ovn_datapaths *lr_datapaths, const struct lr_lb_nat_data_table *lr_lbnats, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, @@ -11836,7 +11238,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, static void build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, @@ -11897,7 +11299,7 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, */ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct ovn_datapaths *lr_datapaths, struct ds *match) { @@ -11923,7 +11325,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, static void build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups, const struct ovn_datapaths *lr_datapaths, const struct lr_lb_nat_data_table *lr_lbnats, @@ -12081,7 +11483,7 @@ lrouter_dnat_and_snat_is_stateless(const struct nbrec_nat *nat) */ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, - struct hmap *lflows, struct ds *match, + struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, bool is_v6, bool is_src, int cidr_bits) { @@ -12148,7 +11550,7 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12198,7 +11600,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *sn_ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct hmap *lflows, const struct shash *meter_groups) + struct lflow_table *lflows, + const struct shash *meter_groups) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -12249,7 +11652,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12272,7 +11675,7 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; @@ -12346,7 +11749,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct hmap *lflows) + struct lflow_table *lflows) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -12411,7 +11814,7 @@ build_lrouter_drop_own_dest(struct ovn_port *op, } static void -build_lrouter_force_snat_flows(struct hmap *lflows, +build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, const char *context) @@ -12440,7 +11843,7 @@ build_lrouter_force_snat_flows(struct hmap *lflows, static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12512,7 +11915,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, } static void -build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, +build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, const struct shash *meter_groups) { if (!op->has_bfd) { @@ -12567,7 +11970,7 @@ build_lrouter_bfd_flows(struct hmap *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Logical VLANs not supported. @@ -12611,7 +12014,7 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) * function. */ static void OVS_PRINTF_FORMAT(9, 10) -build_gateway_mtu_flow(struct hmap *lflows, struct ovn_port *op, +build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, @@ -12672,7 +12075,7 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) */ static void build_adm_ctrl_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12726,7 +12129,7 @@ build_adm_ctrl_flows_for_lrouter_port( * lflows for logical routers. */ static void build_neigh_learning_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -12857,7 +12260,7 @@ build_neigh_learning_flows_for_lrouter( * for logical router ports. */ static void build_neigh_learning_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -12919,7 +12322,7 @@ build_neigh_learning_flows_for_lrouter_port( * Adv (RA) options and response. */ static void build_ND_RA_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -13034,7 +12437,8 @@ build_ND_RA_flows_for_lrouter_port( /* Logical router ingress table ND_RA_OPTIONS & ND_RA_RESPONSE: RS * responder, by default goto next. (priority 0). */ static void -build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) +build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); @@ -13045,7 +12449,7 @@ build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, struct hmap *lflows) * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", @@ -13073,21 +12477,23 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct hmap *lflows) + struct ovn_port *op, struct lflow_table *lflows) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { add_route(lflows, op->od, op, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, - &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED); + &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, + NULL); } } @@ -13101,7 +12507,8 @@ build_ip_routing_flows_for_lrp( static void build_ip_routing_flows_for_router_type_lsp( struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct hmap *lflows) + const struct hmap *lr_ports, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); if (!lsp_is_router(op->nbsp)) { @@ -13136,7 +12543,8 @@ build_ip_routing_flows_for_router_type_lsp( laddrs->ipv4_addrs[k].network_s, laddrs->ipv4_addrs[k].plen, NULL, false, 0, &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED); + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); } } destroy_routable_addresses(&ra); @@ -13148,7 +12556,7 @@ build_ip_routing_flows_for_router_type_lsp( static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, - struct hmap *lflows, const struct hmap *lr_ports, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct hmap *bfd_connections) { ovs_assert(od->nbr); @@ -13212,7 +12620,7 @@ build_static_route_flows_for_lrouter( */ static void build_mcast_lookup_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13313,7 +12721,7 @@ build_mcast_lookup_flows_for_lrouter( * advances to the next table for ARP/ND resolution. */ static void build_ingress_policy_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports) { ovs_assert(od->nbr); @@ -13347,7 +12755,7 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to @@ -13365,10 +12773,12 @@ build_arp_resolve_flows_for_lrouter( } static void -routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, +routable_addresses_to_lflows(struct lflow_table *lflows, + struct ovn_port *router_port, struct ovn_port *peer, const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { struct ovn_port_routable_addresses ra = get_op_routable_addresses(router_port, lr_lbnat_rec); @@ -13392,8 +12802,9 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions)); + ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); } destroy_routable_addresses(&ra); } @@ -13412,7 +12823,7 @@ routable_addresses_to_lflows(struct hmap *lflows, struct ovn_port *router_port, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, - struct hmap *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -13486,7 +12897,7 @@ build_arp_resolve_flows_for_lrp( /* This function adds ARP resolve flows related to a LSP. */ static void build_arp_resolve_flows_for_lsp( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions) { @@ -13528,11 +12939,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } @@ -13559,11 +12971,12 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13607,10 +13020,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } if (router_port->lrp_networks.n_ipv6_addrs) { @@ -13623,10 +13037,11 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_hint(lflows, peer->od, + ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_); + &op->nbsp->header_, + op->lflow_ref); } } } @@ -13634,10 +13049,11 @@ build_arp_resolve_flows_for_lsp( static void build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, const struct hmap *lr_ports, const struct lr_lb_nat_data_table *lr_lbnats, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { if (!lsp_is_router(op->nbsp)) { return; @@ -13671,13 +13087,15 @@ build_arp_resolve_flows_for_lsp_routable_addresses( lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, router_port->od->nbr); routable_addresses_to_lflows(lflows, router_port, peer, - lr_lbnat_rec, match, actions); + lr_lbnat_rec, match, actions, + lflow_ref); } } } static void -build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, +build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, + struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, struct ovn_port *outport) @@ -13770,7 +13188,7 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct hmap *lflows, static void build_check_pkt_len_flows_for_lrp(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, @@ -13820,7 +13238,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, * */ static void build_check_pkt_len_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups) @@ -13847,7 +13265,7 @@ build_check_pkt_len_flows_for_lrouter( /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_gateway_redirect_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); @@ -13892,7 +13310,7 @@ build_gateway_redirect_flows_for_lrouter( static void build_lr_gateway_redirect_flows_for_nats( const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13961,7 +13379,7 @@ build_lr_gateway_redirect_flows_for_nats( * and sends an ARP/IPv6 NA request (priority 100). */ static void build_arp_request_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows, + struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14039,7 +13457,7 @@ build_arp_request_flows_for_lrouter( */ static void build_egress_delivery_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions) { ovs_assert(op->nbrp); @@ -14081,7 +13499,7 @@ build_egress_delivery_flows_for_lrouter_port( static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct hmap *lflows) + struct ovn_datapath *od, struct lflow_table *lflows) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -14163,7 +13581,7 @@ build_misc_local_traffic_drop_flows_for_lrouter( static void build_dhcpv6_reply_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match) { ovs_assert(op->nbrp); @@ -14183,7 +13601,7 @@ build_dhcpv6_reply_flows_for_lrouter_port( static void build_ipv6_input_flows_for_lrouter_port( - struct ovn_port *op, struct hmap *lflows, + struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14352,7 +13770,7 @@ build_ipv6_input_flows_for_lrouter_port( static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct shash *meter_groups) { ovs_assert(od->nbr); @@ -14404,7 +13822,7 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, struct ds *actions, const struct shash *meter_groups) { @@ -14608,7 +14026,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, /* Logical router ingress table 3: IP Input for IPv4. */ static void build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, - struct hmap *lflows, + struct lflow_table *lflows, const struct lr_lb_nat_data_record *lr_lbnat_rec, struct ds *match, const struct shash *meter_groups) { @@ -14727,7 +14145,7 @@ build_lrouter_in_unsnat_match(const struct ovn_datapath *od, } static void -build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, +build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -14749,7 +14167,7 @@ build_lrouter_in_unsnat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, +build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, @@ -14783,7 +14201,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_in_unsnat_flow(struct hmap *lflows, +build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, @@ -14805,7 +14223,7 @@ build_lrouter_in_unsnat_flow(struct hmap *lflows, } static void -build_lrouter_in_dnat_flow(struct hmap *lflows, +build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, const struct nbrec_nat *nat, struct ds *match, @@ -14877,7 +14295,7 @@ build_lrouter_in_dnat_flow(struct hmap *lflows, } static void -build_lrouter_out_undnat_flow(struct hmap *lflows, +build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14928,7 +14346,7 @@ build_lrouter_out_undnat_flow(struct hmap *lflows, } static void -build_lrouter_out_is_dnat_local(struct hmap *lflows, +build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -14959,7 +14377,7 @@ build_lrouter_out_is_dnat_local(struct hmap *lflows, } static void -build_lrouter_out_snat_match(struct hmap *lflows, +build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, @@ -14988,7 +14406,7 @@ build_lrouter_out_snat_match(struct hmap *lflows, } static void -build_lrouter_out_snat_stateless_flow(struct hmap *lflows, +build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, @@ -15031,7 +14449,7 @@ build_lrouter_out_snat_stateless_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, +build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, @@ -15093,7 +14511,7 @@ build_lrouter_out_snat_in_czone_flow(struct hmap *lflows, } static void -build_lrouter_out_snat_flow(struct hmap *lflows, +build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, @@ -15139,7 +14557,7 @@ build_lrouter_out_snat_flow(struct hmap *lflows, } static void -build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, +build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, const struct nbrec_nat *nat, const struct ovn_datapath *od, bool is_v6, struct ds *match, @@ -15211,7 +14629,7 @@ build_lrouter_ingress_nat_check_pkt_len(struct hmap *lflows, } static void -build_lrouter_ingress_flow(struct hmap *lflows, +build_lrouter_ingress_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, struct eth_addr mac, @@ -15391,7 +14809,7 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(od->nbr); @@ -15416,7 +14834,8 @@ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, static void build_lrouter_nat_defrag_and_lb( - const struct lr_lb_nat_data_record *lr_lbnat_rec, struct hmap *lflows, + const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, @@ -15799,23 +15218,22 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, const struct lr_lb_nat_data_record *lr_lbnat_rec, const struct lr_lb_nat_data_table *lr_lbnats, const struct hmap *lr_ports, - struct hmap *lflows, + struct lflow_table *lflows, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(lsp->nbsp); - start_collecting_lflows(); build_lswitch_rport_arp_req_flows_for_lbnats( lrp_peer, lr_lbnat_rec, lsp->od, lsp, - lflows, &lsp->nbsp->header_); + lflows, &lsp->nbsp->header_, lflow_ref); build_ip_routing_flows_for_router_type_lsp(lsp, lr_lbnats, - lr_ports, lflows); + lr_ports, lflows, + lflow_ref); build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_lbnats, match, actions); + lsp, lflows, lr_ports, lr_lbnats, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_lbnat_rec, lflows, - match, actions); - link_ovn_port_to_lflows(lsp, &collected_lflows); - end_collecting_lflows(); + match, actions, lflow_ref); } static void @@ -15824,7 +15242,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); @@ -15838,7 +15256,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, lr_lbnats, lr_ports, lflows, - match, actions); + match, actions, op->lbnat_lflow_ref); } static void @@ -15846,7 +15264,7 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage @@ -15883,7 +15301,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbrp); @@ -15897,7 +15315,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, static void build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct hmap *lflows, + struct lflow_table *lflows, const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, @@ -15920,7 +15338,7 @@ build_ls_lbacls_flows(const struct ls_lbacls_record *ls_lbacls_rec, const struct ls_port_group_table *ls_pgs, const struct chassis_features *features, const struct shash *meter_groups, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(ls_lbacls_rec->od); @@ -15939,7 +15357,7 @@ struct lswitch_flow_build_info { const struct ls_port_group_table *ls_port_groups; const struct lr_lb_nat_data_table *lr_lbnats; const struct ls_lbacls_table *ls_lbacls; - struct hmap *lflows; + struct lflow_table *lflows; struct hmap *igmp_groups; const struct shash *meter_groups; const struct hmap *lb_dps_map; @@ -16022,10 +15440,9 @@ build_lswitch_and_lrouter_iterate_by_lsp( const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct hmap *lflows) + struct lflow_table *lflows) { ovs_assert(op->nbsp); - start_collecting_lflows(); /* Build Logical Switch Flows. */ build_lswitch_port_sec_op(op, lflows, actions, match); @@ -16040,9 +15457,6 @@ build_lswitch_and_lrouter_iterate_by_lsp( /* Build Logical Router Flows. */ build_arp_resolve_flows_for_lsp(op, lflows, lr_ports, match, actions); - - link_ovn_port_to_lflows(op, &collected_lflows); - end_collecting_lflows(); } /* Helper function to combine all lflow generation which is iterated by logical @@ -16252,7 +15666,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, /* Do nothing */ } -/* Fixes the hmap size (hmap->n) after parallel building the lflow_map when +/* Fixes the hmap size (hmap->n) after parallel building the lflow_table when * dp-groups is enabled, because in that case all threads are updating the * global lflow hmap. Although the lflow_hash_lock prevents currently inserting * to the same hash bucket, the hmap->n is updated currently by all threads and @@ -16262,7 +15676,7 @@ noop_callback(struct worker_pool *pool OVS_UNUSED, * after the worker threads complete the tasks in each iteration before any * future operations on the lflow map. */ static void -fix_flow_map_size(struct hmap *lflow_map, +fix_flow_table_size(struct lflow_table *lflow_table, struct lswitch_flow_build_info *lsiv, size_t n_lsiv) { @@ -16270,7 +15684,7 @@ fix_flow_map_size(struct hmap *lflow_map, for (size_t i = 0; i < n_lsiv; i++) { total += lsiv[i].thread_lflow_counter; } - lflow_map->n = total; + lflow_table_set_size(lflow_table, total); } static void @@ -16281,7 +15695,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct ls_port_group_table *ls_pgs, const struct lr_lb_nat_data_table *lr_lbnats, const struct ls_lbacls_table *ls_lbacls, - struct hmap *lflows, + struct lflow_table *lflows, struct hmap *igmp_groups, const struct shash *meter_groups, const struct hmap *lb_dps_map, @@ -16328,7 +15742,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, /* Run thread pool. */ run_pool_callback(build_lflows_pool, NULL, NULL, noop_callback); - fix_flow_map_size(lflows, lsiv, build_lflows_pool->size); + fix_flow_table_size(lflows, lsiv, build_lflows_pool->size); for (index = 0; index < build_lflows_pool->size; index++) { ds_destroy(&lsiv[index].match); @@ -16442,24 +15856,6 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, free(svc_check_match); } -static ssize_t max_seen_lflow_size = 128; - -void -lflow_data_init(struct lflow_data *data) -{ - fast_hmap_size_for(&data->lflows, max_seen_lflow_size); -} - -void -lflow_data_destroy(struct lflow_data *data) -{ - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, &data->lflows) { - ovn_lflow_destroy(&data->lflows, lflow); - } - hmap_destroy(&data->lflows); -} - void run_update_worker_pool(int n_threads) { /* If number of threads has been updated (or initially set), @@ -16505,7 +15901,7 @@ create_sb_multicast_group(struct ovsdb_idl_txn *ovnsb_txn, * constructing their contents based on the OVN_NB database. */ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmap mcast_groups; struct hmap igmp_groups; @@ -16536,281 +15932,26 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, } /* Parallel build may result in a suboptimal hash. Resize the - * hash to a correct size before doing lookups */ - - hmap_expand(lflows); - - if (hmap_count(lflows) > max_seen_lflow_size) { - max_seen_lflow_size = hmap_count(lflows); - } - - stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); - /* Collecting all unique datapath groups. */ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct hmap single_dp_lflows; - - /* Single dp_flows will never grow bigger than lflows, - * thus the two hmaps will remain the same size regardless - * of how many elements we remove from lflows and add to - * single_dp_lflows. - * Note - lflows is always sized for at least 128 flows. - */ - fast_hmap_size_for(&single_dp_lflows, max_seen_lflow_size); - - struct ovn_lflow *lflow; - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - struct ovn_datapath **datapaths_array; - size_t n_datapaths; - - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(input_data->ls_datapaths); - datapaths_array = input_data->ls_datapaths->array; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - datapaths_array = input_data->lr_datapaths->array; - } - - lflow->n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - - ovs_assert(lflow->n_ods); - - if (lflow->n_ods == 1) { - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add to single_dp_lflows. */ - hmap_insert_fast(&single_dp_lflows, &lflow->hmap_node, hash); - } - } - - /* Merge multiple and single dp hashes. */ - - fast_hmap_merge(lflows, &single_dp_lflows); - - hmap_destroy(&single_dp_lflows); - - stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); + * lflow map to a correct size before doing lookups */ + lflow_table_expand(lflows); + stopwatch_start(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - - struct hmap lflows_temp = HMAP_INITIALIZER(&lflows_temp); - /* Push changes to the Logical_Flow table to database. */ - const struct sbrec_logical_flow *sbflow; - SBREC_LOGICAL_FLOW_TABLE_FOR_EACH_SAFE (sbflow, - input_data->sbrec_logical_flow_table) { - struct sbrec_logical_dp_group *dp_group = sbflow->logical_dp_group; - struct ovn_datapath *logical_datapath_od = NULL; - size_t i; - - /* Find one valid datapath to get the datapath type. */ - struct sbrec_datapath_binding *dp = sbflow->logical_datapath; - if (dp) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp); - if (logical_datapath_od - && ovn_datapath_is_stale(logical_datapath_od)) { - logical_datapath_od = NULL; - } - } - for (i = 0; dp_group && i < dp_group->n_datapaths; i++) { - logical_datapath_od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - dp_group->datapaths[i]); - if (logical_datapath_od - && !ovn_datapath_is_stale(logical_datapath_od)) { - break; - } - logical_datapath_od = NULL; - } - - if (!logical_datapath_od) { - /* This lflow has no valid logical datapaths. */ - sbrec_logical_flow_delete(sbflow); - continue; - } - - enum ovn_pipeline pipeline - = !strcmp(sbflow->pipeline, "ingress") ? P_IN : P_OUT; - - lflow = ovn_lflow_find( - lflows, dp_group ? NULL : logical_datapath_od, - ovn_stage_build(ovn_datapath_get_type(logical_datapath_od), - pipeline, sbflow->table_id), - sbflow->priority, sbflow->match, sbflow->actions, - sbflow->controller_meter, sbflow->hash); - if (lflow) { - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - lflow->sb_uuid = sbflow->header_.uuid; - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - if (input_data->ovn_internal_version_changed) { - const char *stage_name = smap_get_def(&sbflow->external_ids, - "stage-name", ""); - const char *stage_hint = smap_get_def(&sbflow->external_ids, - "stage-hint", ""); - const char *source = smap_get_def(&sbflow->external_ids, - "source", ""); - - if (strcmp(stage_name, ovn_stage_to_str(lflow->stage))) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-name", ovn_stage_to_str(lflow->stage)); - } - if (lflow->stage_hint) { - if (strcmp(stage_hint, lflow->stage_hint)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "stage-hint", lflow->stage_hint); - } - } - if (lflow->where) { - if (strcmp(source, lflow->where)) { - sbrec_logical_flow_update_external_ids_setkey(sbflow, - "source", lflow->where); - } - } - } - - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, dp_group, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_datapath(sbflow, NULL); - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - /* This lflow updated. Not needed anymore. */ - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } else { - sbrec_logical_flow_delete(sbflow); - } - } - - HMAP_FOR_EACH_SAFE (lflow, hmap_node, lflows) { - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - struct hmap *dp_groups; - size_t n_datapaths; - bool is_switch; - - is_switch = ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH; - if (is_switch) { - n_datapaths = ods_size(input_data->ls_datapaths); - dp_groups = &ls_dp_groups; - } else { - n_datapaths = ods_size(input_data->lr_datapaths); - dp_groups = &lr_dp_groups; - } - - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - if (lflow->od) { - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - } else { - lflow->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, dp_groups, NULL, - lflow->n_ods, lflow->dpg_bitmap, - n_datapaths, is_switch, - input_data->ls_datapaths, - input_data->lr_datapaths); - - sbrec_logical_flow_set_logical_dp_group(sbflow, - lflow->dpg->dp_group); - } - - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); - hmap_remove(lflows, &lflow->hmap_node); - hmap_insert(&lflows_temp, &lflow->hmap_node, - hmap_node_hash(&lflow->hmap_node)); - } - hmap_swap(lflows, &lflows_temp); - hmap_destroy(&lflows_temp); + lflow_table_sync_to_sb(lflows, ovnsb_txn, input_data->ls_datapaths, + input_data->lr_datapaths, + input_data->ovn_internal_version_changed, + input_data->sbrec_logical_flow_table, + input_data->sbrec_logical_dp_group_table); stopwatch_stop(LFLOWS_TO_SB_STOPWATCH_NAME, time_msec()); - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); /* Push changes to the Multicast_Group table to database. */ const struct sbrec_multicast_group *sbmc; SBREC_MULTICAST_GROUP_TABLE_FOR_EACH_SAFE (sbmc, input_data->sbrec_multicast_group_table) { struct ovn_datapath *od = ovn_datapath_from_sbrec( - &input_data->ls_datapaths->datapaths, - &input_data->lr_datapaths->datapaths, - sbmc->datapath); + &input_data->ls_datapaths->datapaths, + &input_data->lr_datapaths->datapaths, + sbmc->datapath); if (!od || ovn_datapath_is_stale(od)) { sbrec_multicast_group_delete(sbmc); @@ -16850,120 +15991,11 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, hmap_destroy(&mcast_groups); } -static void -sync_lsp_lflows_to_sb(struct ovsdb_idl_txn *ovnsb_txn, - struct lflow_input *lflow_input, - struct hmap *lflows, - struct ovn_lflow *lflow) -{ - size_t n_datapaths; - struct ovn_datapath **datapaths_array; - if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { - n_datapaths = ods_size(lflow_input->ls_datapaths); - datapaths_array = lflow_input->ls_datapaths->array; - } else { - n_datapaths = ods_size(lflow_input->lr_datapaths); - datapaths_array = lflow_input->lr_datapaths->array; - } - uint32_t n_ods = bitmap_count1(lflow->dpg_bitmap, n_datapaths); - ovs_assert(n_ods == 1); - /* There is only one datapath, so it should be moved out of the - * group to a single 'od'. */ - size_t index = bitmap_scan(lflow->dpg_bitmap, true, 0, - n_datapaths); - - bitmap_set0(lflow->dpg_bitmap, index); - lflow->od = datapaths_array[index]; - - /* Logical flow should be re-hashed to allow lookups. */ - uint32_t hash = hmap_node_hash(&lflow->hmap_node); - /* Remove from lflows. */ - hmap_remove(lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - /* Add back. */ - hmap_insert(lflows, &lflow->hmap_node, hash); - - /* Sync to SB. */ - const struct sbrec_logical_flow *sbflow; - /* Note: uuid_random acquires a global mutex. If we parallelize the sync to - * SB this may become a bottleneck. */ - lflow->sb_uuid = uuid_random(); - sbflow = sbrec_logical_flow_insert_persist_uuid(ovnsb_txn, - &lflow->sb_uuid); - const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); - uint8_t table = ovn_stage_get_table(lflow->stage); - sbrec_logical_flow_set_logical_datapath(sbflow, lflow->od->sb); - sbrec_logical_flow_set_logical_dp_group(sbflow, NULL); - sbrec_logical_flow_set_pipeline(sbflow, pipeline); - sbrec_logical_flow_set_table_id(sbflow, table); - sbrec_logical_flow_set_priority(sbflow, lflow->priority); - sbrec_logical_flow_set_match(sbflow, lflow->match); - sbrec_logical_flow_set_actions(sbflow, lflow->actions); - if (lflow->io_port) { - struct smap tags = SMAP_INITIALIZER(&tags); - smap_add(&tags, "in_out_port", lflow->io_port); - sbrec_logical_flow_set_tags(sbflow, &tags); - smap_destroy(&tags); - } - sbrec_logical_flow_set_controller_meter(sbflow, lflow->ctrl_meter); - /* Trim the source locator lflow->where, which looks something like - * "ovn/northd/northd.c:1234", down to just the part following the - * last slash, e.g. "northd.c:1234". */ - const char *slash = strrchr(lflow->where, '/'); -#if _WIN32 - const char *backslash = strrchr(lflow->where, '\\'); - if (!slash || backslash > slash) { - slash = backslash; - } -#endif - const char *where = slash ? slash + 1 : lflow->where; - - struct smap ids = SMAP_INITIALIZER(&ids); - smap_add(&ids, "stage-name", ovn_stage_to_str(lflow->stage)); - smap_add(&ids, "source", where); - if (lflow->stage_hint) { - smap_add(&ids, "stage-hint", lflow->stage_hint); - } - sbrec_logical_flow_set_external_ids(sbflow, &ids); - smap_destroy(&ids); -} - -static bool -delete_lflow_for_lsp(struct ovn_port *op, bool is_update, - const struct sbrec_logical_flow_table *sb_lflow_table, - struct hmap *lflows) -{ - struct lflow_ref_node *lfrn; - const char *operation = is_update ? "updated" : "deleted"; - LIST_FOR_EACH_SAFE (lfrn, lflow_list_node, &op->lflows) { - VLOG_DBG("Deleting SB lflow "UUID_FMT" for %s port %s", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - - const struct sbrec_logical_flow *sblflow = - sbrec_logical_flow_table_get_for_uuid(sb_lflow_table, - &lfrn->lflow->sb_uuid); - if (sblflow) { - sbrec_logical_flow_delete(sblflow); - } else { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "SB lflow "UUID_FMT" not found when handling " - "%s port %s. Recompute.", - UUID_ARGS(&lfrn->lflow->sb_uuid), operation, op->key); - return false; - } - - ovn_lflow_destroy(lflows, lfrn->lflow); - } - return true; -} - bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *trk_ovn_ports, struct lflow_input *lflow_input, - struct hmap *lflows) + struct lflow_table *lflows) { struct hmapx_node *hmapx_node; struct ovn_port *op; @@ -16973,12 +16005,13 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, /* We don't support lflow handling for deleted logical router * ports yet. */ ovs_assert(op->nbsp); - - if (!delete_lflow_for_lsp(op, false, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + lflow_ref_clear_and_sync_lflows(op->lflow_ref, op->od, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); /* No need to update SB multicast groups, thanks to weak * references. */ @@ -16990,12 +16023,8 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, * ports yet. */ ovs_assert(op->nbsp); - /* Delete old lflows. */ - if (!delete_lflow_for_lsp(op, true, - lflow_input->sbrec_logical_flow_table, - lflows)) { - return false; - } + /* Clear old lflows. */ + lflow_ref_clear_lflows(op->lflow_ref, op->od, lflows); /* Generate new lflows. */ struct ds match = DS_EMPTY_INITIALIZER; @@ -17011,11 +16040,18 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, lr_lbnat_rec = lr_lb_nat_data_table_find(lflow_input->lr_lbnats, op->peer->od->nbr); ovs_assert(lr_lbnat_rec); - + lflow_ref_clear_lflows(op->lbnat_lflow_ref, op->od, lflows); build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, lflow_input->lr_lbnats, lflow_input->lr_ports, - lflows, &match, &actions); + lflows, &match, &actions, + op->lbnat_lflow_ref); + lflow_ref_sync_lflows_to_sb(op->lbnat_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } ds_destroy(&match); @@ -17025,11 +16061,12 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, * groups. */ /* Sync the new flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + lflow_ref_sync_lflows_to_sb(op->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } HMAPX_FOR_EACH (hmapx_node, &trk_ovn_ports->created) { @@ -17083,11 +16120,12 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, } /* Sync the newly added flows to SB. */ - struct lflow_ref_node *lfrn; - LIST_FOR_EACH (lfrn, lflow_list_node, &op->lflows) { - sync_lsp_lflows_to_sb(ovnsb_txn, lflow_input, lflows, - lfrn->lflow); - } + lflow_ref_sync_lflows_to_sb(op->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); } return true; diff --git a/northd/northd.h b/northd/northd.h index 495e6e8852..38aa4e4a53 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -23,6 +23,7 @@ #include "northd/en-port-group.h" #include "northd/ipam.h" #include "openvswitch/hmap.h" +#include "ovs-thread.h" struct northd_input { /* Northbound table references */ @@ -145,13 +146,6 @@ struct northd_data { struct northd_tracked_data trk_northd_changes; }; -struct lflow_data { - struct hmap lflows; -}; - -void lflow_data_init(struct lflow_data *); -void lflow_data_destroy(struct lflow_data *); - struct lr_nat_table; struct lflow_input { @@ -163,6 +157,7 @@ struct lflow_input { const struct sbrec_logical_flow_table *sbrec_logical_flow_table; const struct sbrec_multicast_group_table *sbrec_multicast_group_table; const struct sbrec_igmp_group_table *sbrec_igmp_group_table; + const struct sbrec_logical_dp_group_table *sbrec_logical_dp_group_table; /* Indexes */ struct ovsdb_idl_index *sbrec_mcast_group_by_name_dp; @@ -182,6 +177,15 @@ struct lflow_input { bool ovn_internal_version_changed; }; +extern int parallelization_state; +enum { + STATE_NULL, /* parallelization is off */ + STATE_INIT_HASH_SIZES, /* parallelization is on; hashes sizing needed */ + STATE_USE_PARALLELIZATION /* parallelization is on */ +}; + +extern thread_local size_t thread_lflow_counter; + /* * Multicast snooping and querier per datapath configuration. */ @@ -331,8 +335,181 @@ ods_size(const struct ovn_datapaths *datapaths) return hmap_count(&datapaths->datapaths); } +struct ovn_datapath *ovn_datapath_from_sbrec( + const struct hmap *ls_datapaths, const struct hmap *lr_datapaths, + const struct sbrec_datapath_binding *); + +static inline bool +ovn_datapath_is_stale(const struct ovn_datapath *od) +{ + return !od->nbr && !od->nbs; +}; + bool od_has_lb_vip(const struct ovn_datapath *od); +/* Pipeline stages. */ + +/* The two purposes for which ovn-northd uses OVN logical datapaths. */ +enum ovn_datapath_type { + DP_SWITCH, /* OVN logical switch. */ + DP_ROUTER /* OVN logical router. */ +}; + +/* Returns an "enum ovn_stage" built from the arguments. + * + * (It's better to use ovn_stage_build() for type-safety reasons, but inline + * functions can't be used in enums or switch cases.) */ +#define OVN_STAGE_BUILD(DP_TYPE, PIPELINE, TABLE) \ + (((DP_TYPE) << 9) | ((PIPELINE) << 8) | (TABLE)) + +/* A stage within an OVN logical switch or router. + * + * An "enum ovn_stage" indicates whether the stage is part of a logical switch + * or router, whether the stage is part of the ingress or egress pipeline, and + * the table within that pipeline. The first three components are combined to + * form the stage's full name, e.g. S_SWITCH_IN_PORT_SEC_L2, + * S_ROUTER_OUT_DELIVERY. */ +enum ovn_stage { +#define PIPELINE_STAGES \ + /* Logical switch ingress stages. */ \ + PIPELINE_STAGE(SWITCH, IN, CHECK_PORT_SEC, 0, "ls_in_check_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, APPLY_PORT_SEC, 1, "ls_in_apply_port_sec") \ + PIPELINE_STAGE(SWITCH, IN, LOOKUP_FDB , 2, "ls_in_lookup_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PUT_FDB, 3, "ls_in_put_fdb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_ACL, 4, "ls_in_pre_acl") \ + PIPELINE_STAGE(SWITCH, IN, PRE_LB, 5, "ls_in_pre_lb") \ + PIPELINE_STAGE(SWITCH, IN, PRE_STATEFUL, 6, "ls_in_pre_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ACL_HINT, 7, "ls_in_acl_hint") \ + PIPELINE_STAGE(SWITCH, IN, ACL_EVAL, 8, "ls_in_acl_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_ACTION, 9, "ls_in_acl_action") \ + PIPELINE_STAGE(SWITCH, IN, QOS_MARK, 10, "ls_in_qos_mark") \ + PIPELINE_STAGE(SWITCH, IN, QOS_METER, 11, "ls_in_qos_meter") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_CHECK, 12, "ls_in_lb_aff_check") \ + PIPELINE_STAGE(SWITCH, IN, LB, 13, "ls_in_lb") \ + PIPELINE_STAGE(SWITCH, IN, LB_AFF_LEARN, 14, "ls_in_lb_aff_learn") \ + PIPELINE_STAGE(SWITCH, IN, PRE_HAIRPIN, 15, "ls_in_pre_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, NAT_HAIRPIN, 16, "ls_in_nat_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, HAIRPIN, 17, "ls_in_hairpin") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_EVAL, 18, \ + "ls_in_acl_after_lb_eval") \ + PIPELINE_STAGE(SWITCH, IN, ACL_AFTER_LB_ACTION, 19, \ + "ls_in_acl_after_lb_action") \ + PIPELINE_STAGE(SWITCH, IN, STATEFUL, 20, "ls_in_stateful") \ + PIPELINE_STAGE(SWITCH, IN, ARP_ND_RSP, 21, "ls_in_arp_rsp") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_OPTIONS, 22, "ls_in_dhcp_options") \ + PIPELINE_STAGE(SWITCH, IN, DHCP_RESPONSE, 23, "ls_in_dhcp_response") \ + PIPELINE_STAGE(SWITCH, IN, DNS_LOOKUP, 24, "ls_in_dns_lookup") \ + PIPELINE_STAGE(SWITCH, IN, DNS_RESPONSE, 25, "ls_in_dns_response") \ + PIPELINE_STAGE(SWITCH, IN, EXTERNAL_PORT, 26, "ls_in_external_port") \ + PIPELINE_STAGE(SWITCH, IN, L2_LKUP, 27, "ls_in_l2_lkup") \ + PIPELINE_STAGE(SWITCH, IN, L2_UNKNOWN, 28, "ls_in_l2_unknown") \ + \ + /* Logical switch egress stages. */ \ + PIPELINE_STAGE(SWITCH, OUT, PRE_ACL, 0, "ls_out_pre_acl") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_LB, 1, "ls_out_pre_lb") \ + PIPELINE_STAGE(SWITCH, OUT, PRE_STATEFUL, 2, "ls_out_pre_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_HINT, 3, "ls_out_acl_hint") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_EVAL, 4, "ls_out_acl_eval") \ + PIPELINE_STAGE(SWITCH, OUT, ACL_ACTION, 5, "ls_out_acl_action") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_MARK, 6, "ls_out_qos_mark") \ + PIPELINE_STAGE(SWITCH, OUT, QOS_METER, 7, "ls_out_qos_meter") \ + PIPELINE_STAGE(SWITCH, OUT, STATEFUL, 8, "ls_out_stateful") \ + PIPELINE_STAGE(SWITCH, OUT, CHECK_PORT_SEC, 9, "ls_out_check_port_sec") \ + PIPELINE_STAGE(SWITCH, OUT, APPLY_PORT_SEC, 10, "ls_out_apply_port_sec") \ + \ + /* Logical router ingress stages. */ \ + PIPELINE_STAGE(ROUTER, IN, ADMISSION, 0, "lr_in_admission") \ + PIPELINE_STAGE(ROUTER, IN, LOOKUP_NEIGHBOR, 1, "lr_in_lookup_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, LEARN_NEIGHBOR, 2, "lr_in_learn_neighbor") \ + PIPELINE_STAGE(ROUTER, IN, IP_INPUT, 3, "lr_in_ip_input") \ + PIPELINE_STAGE(ROUTER, IN, UNSNAT, 4, "lr_in_unsnat") \ + PIPELINE_STAGE(ROUTER, IN, DEFRAG, 5, "lr_in_defrag") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_CHECK, 6, "lr_in_lb_aff_check") \ + PIPELINE_STAGE(ROUTER, IN, DNAT, 7, "lr_in_dnat") \ + PIPELINE_STAGE(ROUTER, IN, LB_AFF_LEARN, 8, "lr_in_lb_aff_learn") \ + PIPELINE_STAGE(ROUTER, IN, ECMP_STATEFUL, 9, "lr_in_ecmp_stateful") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_OPTIONS, 10, "lr_in_nd_ra_options") \ + PIPELINE_STAGE(ROUTER, IN, ND_RA_RESPONSE, 11, "lr_in_nd_ra_response") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_PRE, 12, "lr_in_ip_routing_pre") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 13, "lr_in_ip_routing") \ + PIPELINE_STAGE(ROUTER, IN, IP_ROUTING_ECMP, 14, "lr_in_ip_routing_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, POLICY, 15, "lr_in_policy") \ + PIPELINE_STAGE(ROUTER, IN, POLICY_ECMP, 16, "lr_in_policy_ecmp") \ + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 17, "lr_in_arp_resolve") \ + PIPELINE_STAGE(ROUTER, IN, CHK_PKT_LEN, 18, "lr_in_chk_pkt_len") \ + PIPELINE_STAGE(ROUTER, IN, LARGER_PKTS, 19, "lr_in_larger_pkts") \ + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 20, "lr_in_gw_redirect") \ + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 21, "lr_in_arp_request") \ + \ + /* Logical router egress stages. */ \ + PIPELINE_STAGE(ROUTER, OUT, CHECK_DNAT_LOCAL, 0, \ + "lr_out_chk_dnat_local") \ + PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 1, "lr_out_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_UNDNAT, 2, "lr_out_post_undnat") \ + PIPELINE_STAGE(ROUTER, OUT, SNAT, 3, "lr_out_snat") \ + PIPELINE_STAGE(ROUTER, OUT, POST_SNAT, 4, "lr_out_post_snat") \ + PIPELINE_STAGE(ROUTER, OUT, EGR_LOOP, 5, "lr_out_egr_loop") \ + PIPELINE_STAGE(ROUTER, OUT, DELIVERY, 6, "lr_out_delivery") + +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + S_##DP_TYPE##_##PIPELINE##_##STAGE \ + = OVN_STAGE_BUILD(DP_##DP_TYPE, P_##PIPELINE, TABLE), + PIPELINE_STAGES +#undef PIPELINE_STAGE +}; + +enum ovn_datapath_type ovn_stage_to_datapath_type(enum ovn_stage stage); + + +/* Returns 'od''s datapath type. */ +static inline enum ovn_datapath_type +ovn_datapath_get_type(const struct ovn_datapath *od) +{ + return od->nbs ? DP_SWITCH : DP_ROUTER; +} + +/* Returns an "enum ovn_stage" built from the arguments. */ +static inline enum ovn_stage +ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline pipeline, + uint8_t table) +{ + return OVN_STAGE_BUILD(dp_type, pipeline, table); +} + +/* Returns the pipeline to which 'stage' belongs. */ +static inline enum ovn_pipeline +ovn_stage_get_pipeline(enum ovn_stage stage) +{ + return (stage >> 8) & 1; +} + +/* Returns the pipeline name to which 'stage' belongs. */ +static inline const char * +ovn_stage_get_pipeline_name(enum ovn_stage stage) +{ + return ovn_stage_get_pipeline(stage) == P_IN ? "ingress" : "egress"; +} + +/* Returns the table to which 'stage' belongs. */ +static inline uint8_t +ovn_stage_get_table(enum ovn_stage stage) +{ + return stage & 0xff; +} + +/* Returns a string name for 'stage'. */ +static inline const char * +ovn_stage_to_str(enum ovn_stage stage) +{ + switch (stage) { +#define PIPELINE_STAGE(DP_TYPE, PIPELINE, STAGE, TABLE, NAME) \ + case S_##DP_TYPE##_##PIPELINE##_##STAGE: return NAME; + PIPELINE_STAGES +#undef PIPELINE_STAGE + default: return ""; + } +} + /* A logical switch port or logical router port. * * In steady state, an ovn_port points to a northbound Logical_Switch_Port @@ -423,8 +600,7 @@ struct ovn_port { /* Temporarily used for traversing a list (or hmap) of ports. */ bool visited; - /* List of struct lflow_ref_node that points to the lflows generated by - * this ovn_port. + /* Reference of lflows generated for this ovn_port. * * This data is initialized and destroyed by the en_northd node, but * populated and used only by the en_lflow node. Ideally this data should @@ -442,8 +618,16 @@ struct ovn_port { * Adding the list here is more straightforward. The drawback is that we * need to keep in mind that this data belongs to en_lflow node, so never * access it from any other nodes. + * + * 'lflow_ref' is used to reference generic logical flows generated for + * this ovn_port. + * + * 'lbnat_lflow_ref' is used for logical switch ports of type + * 'patch/router' to referenece logical flows generated fo this ovn_port + * from the 'lr_lb_nat_data_table' record of the peer port's datapath. */ - struct ovs_list lflows; + struct lflow_ref *lflow_ref; + struct lflow_ref *lbnat_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -466,13 +650,15 @@ void northd_destroy(struct northd_data *data); void northd_init(struct northd_data *data); void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); + +struct lflow_table; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, - struct hmap *lflows); + struct lflow_table *); bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, - struct hmap *lflows); + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index b2ec67c067..d0ebdbc1e9 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -848,6 +848,10 @@ main(int argc, char *argv[]) ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, &sbrec_port_group_columns[i]); } + for (size_t i = 0; i < SBREC_LOGICAL_DP_GROUP_N_COLUMNS; i++) { + ovsdb_idl_omit_alert(ovnsb_idl_loop.idl, + &sbrec_logical_dp_group_columns[i]); + } unixctl_command_register("sb-connection-status", "", 0, 0, ovn_conn_show, ovnsb_idl_loop.idl); From patchwork Tue Oct 24 00:49:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854084 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtkl592rz202k for ; Tue, 24 Oct 2023 11:50:07 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CE8F5419C5; Tue, 24 Oct 2023 00:50:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org CE8F5419C5 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X3cS50bZ-mM4; Tue, 24 Oct 2023 00:50:03 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id B9C93419B3; Tue, 24 Oct 2023 00:50:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org B9C93419B3 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7F8EAC0DF5; Tue, 24 Oct 2023 00:50:01 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 122A2C0DF5 for ; Tue, 24 Oct 2023 00:50:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C6CFE4198F for ; Tue, 24 Oct 2023 00:49:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C6CFE4198F X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uyrSRpX29yF0 for ; Tue, 24 Oct 2023 00:49:21 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp2.osuosl.org (Postfix) with ESMTPS id EAD0A41995 for ; Tue, 24 Oct 2023 00:49:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EAD0A41995 Received: by mail.gandi.net (Postfix) with ESMTPSA id 94292C0002; Tue, 24 Oct 2023 00:49:17 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:49:04 -0400 Message-ID: <20231024004904.4133932-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 11/18] northd: Use lflow_ref when adding all logical flows. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique ovn_lflow_add* macros now expect 'lflow_ref' to be passed and it can be NULL too. Signed-off-by: Numan Siddique --- northd/lflow-mgr.h | 38 +- northd/northd.c | 1694 +++++++++++++++++++++++++++----------------- 2 files changed, 1042 insertions(+), 690 deletions(-) diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 7d15b11432..2cbc4af157 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -91,41 +91,27 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, /* Adds a row with the specified contents to the Logical_Flow table. */ #define ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, \ - STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, IN_OUT_PORT, \ - CTRL_METER, STAGE_HINT, LFLOW_REF)\ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, IN_OUT_PORT, CTRL_METER, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, STAGE_HINT) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref_hint(LFLOW_TABLE, OD, STAGE, PRIORITY, \ - MATCH, ACTIONS, STAGE_HINT, \ - LFLOW_REF) \ + ACTIONS, STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) #define ovn_lflow_add_with_dp_group(LFLOW_TABLE, DP_BITMAP, DP_BITMAP_LEN, \ STAGE, PRIORITY, MATCH, ACTIONS, \ - STAGE_HINT) \ + STAGE_HINT, LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, NULL, DP_BITMAP, DP_BITMAP_LEN, STAGE, \ PRIORITY, MATCH, ACTIONS, NULL, NULL, STAGE_HINT, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE) \ +#define ovn_lflow_add_default_drop(LFLOW_TABLE, OD, STAGE, LFLOW_REF) \ lflow_table_add_lflow_default_drop(LFLOW_TABLE, OD, STAGE, \ - OVS_SOURCE_LOCATOR, NULL) + OVS_SOURCE_LOCATOR, LFLOW_REF) /* This macro is similar to ovn_lflow_add_with_hint, except that it requires @@ -145,20 +131,16 @@ void lflow_table_add_lflow_default_drop(struct lflow_table *, ACTIONS, IN_OUT_PORT, NULL, STAGE_HINT, \ OVS_SOURCE_LOCATOR, LFLOW_REF) -#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS) \ - lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, NULL) - -#define ovn_lflow_add_with_lflow_ref(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, LFLOW_REF) \ +#define ovn_lflow_add(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ + LFLOW_REF) \ lflow_table_add_lflow(LFLOW_TABLE, OD, NULL, 0, STAGE, PRIORITY, MATCH, \ ACTIONS, NULL, NULL, NULL, OVS_SOURCE_LOCATOR, \ LFLOW_REF) #define ovn_lflow_metered(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, ACTIONS, \ - CTRL_METER) \ + CTRL_METER, LFLOW_REF) \ ovn_lflow_add_with_hint__(LFLOW_TABLE, OD, STAGE, PRIORITY, MATCH, \ - ACTIONS, NULL, CTRL_METER, NULL) + ACTIONS, NULL, CTRL_METER, NULL, LFLOW_REF) struct sbrec_logical_dp_group; diff --git a/northd/northd.c b/northd/northd.c index eac3063a18..36bc5e71bc 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -6035,13 +6035,16 @@ build_lswitch_learn_fdb_op( static void build_lswitch_learn_fdb_od( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LOOKUP_FDB, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PUT_FDB, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 0, "1", - "outport = get_fdb(eth.dst); next;"); + "outport = get_fdb(eth.dst); next;", lflow_ref); } /* Egress tables 8: Egress port security - IP (priority 0) @@ -6049,25 +6052,30 @@ build_lswitch_learn_fdb_od( * (priority 100). */ static void build_lswitch_output_port_sec_od(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 100, - "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;"); + "eth.mcast", REGBIT_PORT_SEC_DROP" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_CHECK_PORT_SEC, 0, "1", - REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_out_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_APPLY_PORT_SEC, 0, - "1", "output;"); + "1", "output;", lflow_ref); } static void skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, bool has_stateful_acl, enum ovn_stage in_stage, enum ovn_stage out_stage, uint16_t priority, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Can't use ct() for router ports. Consider the following configuration: * lp1(10.0.0.2) on hostA--ls1--lr0--ls2--lp2(10.0.1.2) on hostB, For a @@ -6089,10 +6097,10 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, ovn_lflow_add_with_lport_and_hint(lflows, od, in_stage, priority, ingress_match, ingress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); ovn_lflow_add_with_lport_and_hint(lflows, od, out_stage, priority, egress_match, egress_action, - op->key, &op->nbsp->header_, NULL); + op->key, &op->nbsp->header_, lflow_ref); free(ingress_match); free(egress_match); @@ -6101,7 +6109,8 @@ skip_port_from_conntrack(const struct ovn_datapath *od, struct ovn_port *op, static void build_stateless_filter(const struct ovn_datapath *od, const struct nbrec_acl *acl, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *action = REGBIT_ACL_STATELESS" = 1; next;"; if (!strcmp(acl->direction, "from-lport")) { @@ -6109,25 +6118,28 @@ build_stateless_filter(const struct ovn_datapath *od, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } else { ovn_lflow_add_with_hint(lflows, od, S_SWITCH_OUT_PRE_ACL, acl->priority + OVN_ACL_PRI_OFFSET, acl->match, action, - &acl->header_); + &acl->header_, + lflow_ref); } } static void build_stateless_filters(const struct ovn_datapath *od, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { for (size_t i = 0; i < od->nbs->n_acls; i++) { const struct nbrec_acl *acl = od->nbs->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } @@ -6143,31 +6155,37 @@ build_stateless_filters(const struct ovn_datapath *od, const struct nbrec_acl *acl = ls_pg_rec->nb_pg->acls[i]; if (!strcmp(acl->action, "allow-stateless")) { - build_stateless_filter(od, acl, lflows); + build_stateless_filter(od, acl, lflows, lflow_ref); } } } } static void -build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows) +build_pre_acls(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress Pre-ACL Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", + lflow_ref); } static void build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, const struct ls_port_group_table *ls_port_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -6178,17 +6196,17 @@ build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, for (size_t i = 0; i < od->n_router_ports; i++) { skip_port_from_conntrack(od, od->router_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } for (size_t i = 0; i < od->n_localnet_ports; i++) { skip_port_from_conntrack(od, od->localnet_ports[i], true, S_SWITCH_IN_PRE_ACL, S_SWITCH_OUT_PRE_ACL, - 110, lflows); + 110, lflows, lflow_ref); } /* stateless filters always take precedence over stateful ACLs. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 110). * @@ -6196,16 +6214,18 @@ build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, * unreachable packets. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2 || " - "(udp && udp.src == 546 && udp.dst == 547)", "next;"); + "(udp && udp.src == 546 && udp.dst == 547)", "next;", + lflow_ref); /* Do not send multicast packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 110, "eth.mcast", - "next;"); + "next;", lflow_ref); /* Ingress and Egress Pre-ACL Table (Priority 100). * @@ -6216,13 +6236,15 @@ build_ls_lbacls_rec_pre_acls(const struct ls_lbacls_record *ls_lbacls_rec, * 'REGBIT_CONNTRACK_DEFRAG' is set to let the pre-stateful table send * it to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_ACL, 100, "ip", - REGBIT_CONNTRACK_DEFRAG" = 1; next;"); + REGBIT_CONNTRACK_DEFRAG" = 1; next;", + lflow_ref); } else if (ls_lbacls_rec->has_lb_vip) { /* We'll build stateless filters if there are LB rules so that * the stateless flows are not tracked in pre-lb. */ - build_stateless_filters(od, ls_port_groups, lflows); + build_stateless_filters(od, ls_port_groups, lflows, lflow_ref); } } @@ -6286,7 +6308,8 @@ build_empty_lb_event_flow(struct ovn_lb_vip *lb_vip, static void build_interconn_mcast_snoop_flows(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; if (!mcast_sw_info->enabled @@ -6305,7 +6328,8 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); /* Punt MLD traffic to controller. */ @@ -6313,50 +6337,54 @@ build_interconn_mcast_snoop_flows(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_OUT_PRE_LB, 120, match, "clone { igmp; }; next;", copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); free(match); } } static void build_pre_lb(struct ovn_datapath *od, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, struct lflow_ref *lflow_ref) { /* Handle IGMP/MLD packets crossing AZs. */ - build_interconn_mcast_snoop_flows(od, meter_groups, lflows); + build_interconn_mcast_snoop_flows(od, meter_groups, lflows, lflow_ref); /* Do not send multicast packets to conntrack */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "eth.mcast", "next;", + lflow_ref); /* Do not send ND packets to conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, "nd || nd_rs || nd_ra || mldv1 || mldv2", - "next;"); + "next;", lflow_ref); /* Do not send service monitor packets to conntrack. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - "eth.dst == $svc_monitor_mac", "next;"); + "eth.dst == $svc_monitor_mac", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - "eth.src == $svc_monitor_mac", "next;"); + "eth.src == $svc_monitor_mac", "next;", lflow_ref); /* Allow all packets to go to next tables by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 0, "1", "next;", lflow_ref); /* Do not send statless flows via conntrack */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, 110, - REGBIT_ACL_STATELESS" == 1", "next;"); + REGBIT_ACL_STATELESS" == 1", "next;", lflow_ref); } static void build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -6364,7 +6392,7 @@ build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, skip_port_from_conntrack(od, od->router_ports[i], ls_lbacls_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } /* Localnet ports have no need for going through conntrack, unless @@ -6377,7 +6405,7 @@ build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, skip_port_from_conntrack(od, od->localnet_ports[i], ls_lbacls_rec->has_stateful_acl, S_SWITCH_IN_PRE_LB, S_SWITCH_OUT_PRE_LB, - 110, lflows); + 110, lflows, lflow_ref); } } @@ -6413,21 +6441,26 @@ build_ls_lbacls_rec_pre_lb(const struct ls_lbacls_record *ls_lbacls_rec, */ if (ls_lbacls_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_LB, - 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;"); + 100, "ip", REGBIT_CONNTRACK_NAT" = 1; next;", + lflow_ref); } } static void build_pre_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress and Egress pre-stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 0, "1", "next;", + lflow_ref); /* Note: priority-120 flows are added in build_lb_rules_pre_stateful(). */ @@ -6436,25 +6469,30 @@ build_pre_stateful(struct ovn_datapath *od, : "ct_lb;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 110, - REGBIT_CONNTRACK_NAT" == 1", ct_lb_action); + REGBIT_CONNTRACK_NAT" == 1", ct_lb_action, + lflow_ref); /* If REGBIT_CONNTRACK_DEFRAG is set as 1, then the packets should be * sent to conntrack for tracking and defragmentation. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_PRE_STATEFUL, 100, - REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;"); + REGBIT_CONNTRACK_DEFRAG" == 1", "ct_next;", + lflow_ref); } static void build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -6482,9 +6520,10 @@ build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, /* In any case, advance to the next stage. */ if (!ls_lbacls_rec->has_acls && !ls_lbacls_rec->has_lb_vip) { - ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;"); + ovn_lflow_add(lflows, od, stage, UINT16_MAX, "1", "next;", + lflow_ref); } else { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); } if (!ls_lbacls_rec->has_stateful_acl && !ls_lbacls_rec->has_lb_vip) { @@ -6498,7 +6537,7 @@ build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, ovn_lflow_add(lflows, od, stage, 7, "ct.new && !ct.est", REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction that * are already marked as "blocked" may hit either: @@ -6514,13 +6553,13 @@ build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, ovn_lflow_add(lflows, od, stage, 6, match, REGBIT_ACL_HINT_ALLOW_NEW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Not tracked traffic can either be allowed or dropped. */ ovn_lflow_add(lflows, od, stage, 5, "!ct.trk", REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Already established connections in the "request" direction may hit * either: @@ -6536,20 +6575,20 @@ build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, ovn_lflow_add(lflows, od, stage, 4, match, REGBIT_ACL_HINT_ALLOW " = 1; " REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); /* Not established or established and already blocked connections may * hit drop ACLs. */ ovn_lflow_add(lflows, od, stage, 3, "!ct.est", REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); match = features->ct_no_masked_label ? "ct.est && ct_mark.blocked == 1" : "ct.est && ct_label.blocked == 1"; ovn_lflow_add(lflows, od, stage, 2, match, REGBIT_ACL_HINT_DROP " = 1; " - "next;"); + "next;", lflow_ref); /* Established connections that were previously allowed might hit * drop ACLs in which case the connection must be committed with @@ -6560,7 +6599,7 @@ build_acl_hints(const struct ls_lbacls_record *ls_lbacls_rec, : "ct.est && ct_label.blocked == 0"; ovn_lflow_add(lflows, od, stage, 1, match, REGBIT_ACL_HINT_BLOCK " = 1; " - "next;"); + "next;", lflow_ref); } } @@ -6626,7 +6665,8 @@ static void consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - uint64_t max_acl_tier, struct ds *match, struct ds *actions) + uint64_t max_acl_tier, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { const char *ct_blocked_match = ct_masked_mark ? "ct_mark.blocked" @@ -6675,7 +6715,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_format(match, "(%s)", acl->match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); return; } @@ -6712,7 +6752,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* Match on traffic in the request direction for an established * connection tracking entry that has not been marked for @@ -6734,7 +6774,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } else if (!strcmp(acl->action, "drop") || !strcmp(acl->action, "reject")) { /* The implementation of "drop" differs if stateful ACLs are in @@ -6751,7 +6791,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ds_put_cstr(actions, "next;"); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); /* For an existing connection without ct_mark.blocked set, we've * encountered a policy change. ACLs previously allowed * this connection and we committed the connection tracking @@ -6772,7 +6812,7 @@ consider_acl(struct lflow_table *lflows, const struct ovn_datapath *od, ct_blocked_match); ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } } @@ -6857,7 +6897,8 @@ build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, const char *default_acl_action, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -6880,18 +6921,20 @@ build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, for (size_t i = 0; i < ARRAY_SIZE(stages); i++) { enum ovn_stage stage = stages[i]; if (!ls_lbacls_rec->has_acls) { - ovn_lflow_add(lflows, od, stage, 0, "1", "next;"); + ovn_lflow_add(lflows, od, stage, 0, "1", "next;", lflow_ref); continue; } ds_truncate(actions, verdict_len); ds_put_cstr(actions, "next;"); ovn_lflow_add(lflows, od, stage, 1000, - REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions)); + REGBIT_ACL_VERDICT_ALLOW " == 1", ds_cstr(actions), + lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, debug_implicit_drop_action()); ovn_lflow_add(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_DROP " == 1", - ds_cstr(actions)); + ds_cstr(actions), + lflow_ref); bool ingress = ovn_stage_get_pipeline(stage) == P_IN; ds_truncate(actions, verdict_len); @@ -6907,11 +6950,11 @@ build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, ovn_lflow_metered(lflows, od, stage, 1000, REGBIT_ACL_VERDICT_REJECT " == 1", ds_cstr(actions), copp_meter_get(COPP_REJECT, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); ds_truncate(actions, verdict_len); ds_put_cstr(actions, default_acl_action); - ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions)); + ovn_lflow_add(lflows, od, stage, 0, "1", ds_cstr(actions), lflow_ref); struct ds tier_actions = DS_EMPTY_INITIALIZER; for (size_t j = 0; j < ls_lbacls_rec->max_acl_tier; j++) { @@ -6923,7 +6966,7 @@ build_acl_action_lflows(const struct ls_lbacls_record *ls_lbacls_rec, j + 1, ingress ? "ingress" : "egress", ovn_stage_get_table(stage) - 1); ovn_lflow_add(lflows, od, stage, 500, ds_cstr(match), - ds_cstr(&tier_actions)); + ds_cstr(&tier_actions), lflow_ref); } ds_destroy(&tier_actions); } @@ -6935,7 +6978,8 @@ build_acl_log_related_flows(const struct ovn_datapath *od, const struct nbrec_acl *acl, bool has_stateful, bool ct_masked_mark, const struct shash *meter_groups, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { /* Related and reply traffic are universally allowed by priority * 65532 flows created in build_acls(). If logging is enabled on @@ -6989,7 +7033,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); ds_clear(match); ds_put_format(match, "!ct.est && ct.rel && !ct.new%s && " @@ -7000,7 +7044,7 @@ build_acl_log_related_flows(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, log_related_stage, UINT16_MAX - 2, ds_cstr(match), ds_cstr(actions), - &acl->header_); + &acl->header_, lflow_ref); } static void @@ -7008,7 +7052,8 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, const struct chassis_features *features, struct lflow_table *lflows, const struct ls_port_group_table *ls_port_groups, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; @@ -7033,20 +7078,24 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, if (!ls_lbacls_rec->has_acls) { if (!ls_lbacls_rec->has_lb_vip) { ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } else { - ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 0, "1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, 0, "1", - "next;"); + "next;", lflow_ref); } @@ -7078,19 +7127,21 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, ds_cstr(&match), REGBIT_CONNTRACK_COMMIT" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); const char *next_action = default_acl_drop ? "next;" : REGBIT_CONNTRACK_COMMIT" = 1; next;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 1, "ip && !ct.est", - next_action); + next_action, lflow_ref); /* Ingress and Egress ACL Table (Priority 65532). * @@ -7104,9 +7155,11 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, use_ct_inv_match ? "ct.inv || " : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;"); + ds_cstr(&match), REGBIT_ACL_VERDICT_DROP " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535 - 3). * @@ -7126,10 +7179,12 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, ds_cstr(&match), REGBIT_ACL_HINT_DROP" = 0; " REGBIT_ACL_HINT_BLOCK" = 0; " REGBIT_ACL_HINT_ALLOW_REL" = 1; " - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, ds_cstr(&match), - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress and Egress ACL Table (Priority 65535). * @@ -7158,15 +7213,16 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, use_ct_inv_match ? " && !ct.inv" : "", ct_blocked_match); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_in_acl_action); + ds_cstr(&match), ct_in_acl_action, lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, - ds_cstr(&match), ct_out_acl_action); + ds_cstr(&match), ct_out_acl_action, lflow_ref); /* Reply and related traffic matched by an "allow-related" ACL * should be allowed in the ls_in_acl_after_lb stage too. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, REGBIT_ACL_HINT_ALLOW_REL" == 1", - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); } /* Ingress and Egress ACL Table (Priority 65532). @@ -7178,24 +7234,28 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, * https://bugzilla.kernel.org/show_bug.cgi?id=11797. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_AFTER_LB_EVAL, UINT16_MAX - 3, IPV6_CT_OMIT_MATCH, - REGBIT_ACL_VERDICT_ALLOW " = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW " = 1; next;", + lflow_ref); /* Ingress or Egress ACL Table (Various priorities). */ for (size_t i = 0; i < od->nbs->n_acls; i++) { struct nbrec_acl *acl = od->nbs->acls[i]; build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_lbacls_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } const struct ls_port_group *ls_pg = @@ -7208,11 +7268,12 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, build_acl_log_related_flows(od, lflows, acl, has_stateful, features->ct_no_masked_label, - meter_groups, &match, &actions); + meter_groups, &match, &actions, + lflow_ref); consider_acl(lflows, od, acl, has_stateful, features->ct_no_masked_label, meter_groups, ls_lbacls_rec->max_acl_tier, - &match, &actions); + &match, &actions, lflow_ref); } } } @@ -7227,7 +7288,7 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, : REGBIT_ACL_VERDICT_ALLOW" = 1; next;"; ovn_lflow_add( lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "udp.src == 53", - dns_actions); + dns_actions, lflow_ref); } if (ls_lbacls_rec->has_acls || ls_lbacls_rec->has_lb_vip) { @@ -7235,30 +7296,37 @@ build_acls(const struct ls_lbacls_record *ls_lbacls_rec, * packets to skip applying ingress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ACL_EVAL, 34000, "eth.dst == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); /* Add a 34000 priority flow to advance the service monitor packets * generated by ovn-controller to skip applying egress ACLs. */ ovn_lflow_add(lflows, od, S_SWITCH_OUT_ACL_EVAL, 34000, "eth.src == $svc_monitor_mac", - REGBIT_ACL_VERDICT_ALLOW" = 1; next;"); + REGBIT_ACL_VERDICT_ALLOW" = 1; next;", + lflow_ref); } build_acl_action_lflows(ls_lbacls_rec, lflows, default_acl_action, - meter_groups, &match, &actions); + meter_groups, &match, &actions, lflow_ref); ds_destroy(&match); ds_destroy(&actions); } static void -build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { +build_qos(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds action = DS_EMPTY_INITIALIZER; - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_MARK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_QOS_METER, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_QOS_METER, 0, "1", "next;", + lflow_ref); for (size_t i = 0; i < od->nbs->n_qos_rules; i++) { struct nbrec_qos *qos = od->nbs->qos_rules[i]; @@ -7275,7 +7343,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } @@ -7306,7 +7374,7 @@ build_qos(struct ovn_datapath *od, struct lflow_table *lflows) { ovn_lflow_add_with_hint(lflows, od, stage, qos->priority, qos->match, ds_cstr(&action), - &qos->header_); + &qos->header_, lflow_ref); } } ds_destroy(&action); @@ -7317,7 +7385,8 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, bool ct_lb_mark, const struct ovn_datapaths *ls_datapaths, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_ls) { return; @@ -7371,7 +7440,7 @@ build_lb_rules_pre_stateful(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_PRE_STATEFUL, 120, ds_cstr(match), ds_cstr(action), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } @@ -7419,7 +7488,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, const struct ovn_northd_lb *lb, struct ovn_lb_vip *lb_vip, char *new_lb_match, char *lb_action, const unsigned long *dp_bitmap, - const struct ovn_datapaths *lr_datapaths) + const struct ovn_datapaths *lr_datapaths, + struct lflow_ref *lflow_ref) { if (!lb->affinity_timeout || bitmap_is_all_zeros(dp_bitmap, ods_size(lr_datapaths))) { @@ -7458,7 +7528,8 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_CHECK, - 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_); + 100, new_lb_match, ds_cstr(&aff_check_action), &lb->nlb->header_, + lflow_ref); /* Prepare common part of affinity LB and affinity learn action. */ ds_put_format(&aff_action, "%s = %s; ", reg_vip, lb_vip->vip_str); @@ -7540,12 +7611,14 @@ build_lb_affinity_lr_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, 150, - ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_); + ds_cstr(&aff_match), ds_cstr(&aff_action), &lb->nlb->header_, + lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7605,7 +7678,8 @@ static void build_lb_affinity_ls_flows(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, struct ovn_lb_vip *lb_vip, - const struct ovn_datapaths *ls_datapaths) + const struct ovn_datapaths *ls_datapaths, + struct lflow_ref *lflow_ref) { if (!lb_dps->lb->affinity_timeout || !lb_dps->n_nb_ls) { return; @@ -7633,7 +7707,7 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_CHECK, 100, ds_cstr(&new_lb_match), aff_check, - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, lflow_ref); ds_destroy(&new_lb_match); struct ds aff_action = DS_EMPTY_INITIALIZER; @@ -7723,13 +7797,14 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB_AFF_LEARN, 100, ds_cstr(&aff_match_learn), - ds_cstr(&aff_action_learn), &lb->nlb->header_); + ds_cstr(&aff_action_learn), &lb->nlb->header_, + lflow_ref); /* Use already selected backend within affinity timeslot. */ ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, 150, ds_cstr(&aff_match), ds_cstr(&aff_action), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); ds_truncate(&aff_action, aff_action_len); ds_truncate(&aff_action_learn, aff_action_learn_len); @@ -7745,20 +7820,26 @@ build_lb_affinity_ls_flows(struct lflow_table *lflows, static void build_lswitch_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void build_lrouter_lb_affinity_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_CHECK, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_LB_AFF_LEARN, 0, "1", "next;", + lflow_ref); } static void @@ -7766,7 +7847,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, struct ds *match, struct ds *action, const struct shash *meter_groups, - const struct hmap *svc_monitor_map) + const struct hmap *svc_monitor_map, + struct lflow_ref *lflow_ref) { const struct ovn_northd_lb *lb = lb_dps->lb; for (size_t i = 0; i < lb->n_vips; i++) { @@ -7803,7 +7885,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, priority = 120; } - build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths); + build_lb_affinity_ls_flows(lflows, lb_dps, lb_vip, ls_datapaths, + lflow_ref); unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -7826,14 +7909,16 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_hint__( lflows, od, S_SWITCH_IN_LB, priority, ds_cstr(match), ds_cstr(action), - NULL, meter, &lb->nlb->header_); + NULL, meter, &lb->nlb->header_, + lflow_ref); } } if (!reject || build_non_meter) { ovn_lflow_add_with_dp_group( lflows, dp_non_meter ? dp_non_meter : lb_dps->nb_ls_map, ods_size(ls_datapaths), S_SWITCH_IN_LB, priority, - ds_cstr(match), ds_cstr(action), &lb->nlb->header_); + ds_cstr(match), ds_cstr(action), &lb->nlb->header_, + lflow_ref); } bitmap_free(dp_non_meter); } @@ -7842,7 +7927,8 @@ build_lb_rules(struct lflow_table *lflows, struct ovn_lb_datapaths *lb_dps, static void build_stateful(struct ovn_datapath *od, const struct chassis_features *features, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const char *ct_block_action = features->ct_no_masked_label ? "ct_mark.blocked" @@ -7851,9 +7937,11 @@ build_stateful(struct ovn_datapath *od, /* Ingress LB, Ingress and Egress stateful Table (Priority 0): Packets are * allowed by default. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_LB, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 0, "1", "next;", + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1 and * REGBIT_CONNTRACK_SET_LABEL is set to 1, then the packets should be @@ -7867,11 +7955,13 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 1", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); /* If REGBIT_CONNTRACK_COMMIT is set as 1, then the packets should be * committed to conntrack. We always set ct_mark.blocked to 0 here as @@ -7882,26 +7972,32 @@ build_stateful(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_OUT_STATEFUL, 100, REGBIT_CONNTRACK_COMMIT" == 1 && " REGBIT_ACL_LABEL" == 0", - ds_cstr(&actions)); + ds_cstr(&actions), + lflow_ref); ds_destroy(&actions); } static void build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = ls_lbacls_rec->od; /* Ingress Pre-Hairpin/Nat-Hairpin/Hairpin tabled (Priority 0). * Packets that don't need hairpinning should continue processing. */ - ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_PRE_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 0, "1", "next;", + lflow_ref); if (ls_lbacls_rec->has_lb_vip) { /* Check if the packet needs to be hairpinned. @@ -7913,7 +8009,8 @@ build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, REGBIT_HAIRPIN " = chk_lb_hairpin(); " REGBIT_HAIRPIN_REPLY " = chk_lb_hairpin_reply(); " "next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, snat the src ip with the VIP * for new sessions. */ @@ -7921,7 +8018,8 @@ build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, "ip && ct.new && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat_to_vip; next;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* If packet needs to be hairpinned, for established sessions there * should already be an SNAT conntrack entry. @@ -7930,13 +8028,15 @@ build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, "ip && ct.est && ct.trk" " && "REGBIT_HAIRPIN " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* For the reply of hairpinned traffic, snat the src ip to the VIP. */ ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_NAT_HAIRPIN, 90, "ip && "REGBIT_HAIRPIN_REPLY " == 1", "ct_snat;", - &od->nbs->header_); + &od->nbs->header_, + lflow_ref); /* Ingress Hairpin table. * - Priority 1: Packets that were SNAT-ed for hairpinning should be @@ -7946,12 +8046,13 @@ build_lb_hairpin(const struct ls_lbacls_record *ls_lbacls_rec, lflows, od, S_SWITCH_IN_HAIRPIN, 1, "("REGBIT_HAIRPIN " == 1 || " REGBIT_HAIRPIN_REPLY " == 1)", "eth.dst <-> eth.src; outport = inport; flags.loopback = 1; " - "output;"); + "output;", lflow_ref); } } static void -build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) +build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { if (!od->has_vtep_lports) { /* There is no need in these flows if datapath has no vtep lports. */ @@ -7964,7 +8065,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) char *action = xasprintf("next(pipeline=ingress, table=%d);", ovn_stage_get_table(S_SWITCH_IN_L2_LKUP)); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 1000, - REGBIT_FROM_RAMP" == 1", action); + REGBIT_FROM_RAMP" == 1", action, lflow_ref); free(action); /* Ingress pre-arp flow for traffic from VTEP (ramp) switch. @@ -7981,7 +8082,7 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) REGBIT_FROM_RAMP" == 1 && is_chassis_resident(%s)", op->cr_port->json_key); ovn_lflow_add(lflows, od, S_SWITCH_IN_HAIRPIN, 2000, - ds_cstr(&match), "next;"); + ds_cstr(&match), "next;", lflow_ref); } } @@ -7992,14 +8093,15 @@ build_vtep_hairpin(struct ovn_datapath *od, struct lflow_table *lflows) */ ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 65535, REGBIT_FROM_RAMP" == 1 && (arp || nd_ns)", - "flags.loopback = 1; next;"); + "flags.loopback = 1; next;", lflow_ref); ds_destroy(&match); } /* Build logical flows for the forwarding groups */ static void -build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) +build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!od->nbs->n_forwarding_groups) { @@ -8034,7 +8136,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); /* L2 lookup for the forwarding group's virtual MAC */ ds_clear(&match); @@ -8057,7 +8159,7 @@ build_fwd_group_lflows(struct ovn_datapath *od, struct lflow_table *lflows) ds_put_format(&actions, "fwd_group(%s);", ds_cstr(&group_ports)); ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(&match), ds_cstr(&actions), - &fwd_group->header_); + &fwd_group->header_, lflow_ref); } ds_destroy(&match); @@ -8205,10 +8307,8 @@ build_lswitch_rport_arp_req_self_orig_flow(struct ovn_port *op, ds_put_format(&match, "eth.src == %s && (arp.op == 1 || rarp.op == 3 || nd_ns)", ds_cstr(ð_src)); - ovn_lflow_add_with_lflow_ref(lflows, od, S_SWITCH_IN_L2_LKUP, priority, - ds_cstr(&match), - "outport = \""MC_FLOOD_L2"\"; output;", - lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, priority, ds_cstr(&match), + "outport = \""MC_FLOOD_L2"\"; output;", lflow_ref); ds_destroy(ð_src); ds_destroy(&match); @@ -8292,17 +8392,17 @@ build_lswitch_rport_arp_req_flow(const char *ips, ds_put_format(&actions, "clone {outport = %s; output; }; " "outport = \""MC_FLOOD_L2"\"; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); } else { ds_put_format(&actions, "outport = %s; output;", patch_op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_SWITCH_IN_L2_LKUP, - priority, ds_cstr(&match), - ds_cstr(&actions), - stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP, + priority, ds_cstr(&match), + ds_cstr(&actions), + stage_hint, + lflow_ref); } ds_destroy(&match); @@ -8458,7 +8558,8 @@ build_dhcpv4_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8481,7 +8582,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8490,7 +8591,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_clear(&match); /* Allow ip4.src = OFFER_IP and * ip4.dst = {SERVER_IP, 255.255.255.255} for the below @@ -8510,7 +8611,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8519,7 +8620,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_clear(&match); /* If REGBIT_DHCP_OPTS_RESULT is set, it means the @@ -8539,7 +8640,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); ds_destroy(&ipv4_addr_match); @@ -8567,7 +8668,7 @@ build_dhcpv4_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp_actions, op->key, &op->nbsp->dhcpv4_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8580,7 +8681,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, struct lport_addresses *lsp_addrs, struct ovn_port *inport, bool is_external, const struct shash *meter_groups, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8602,7 +8704,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->json_key); } - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, + ovn_lflow_add_with_hint__(lflows, op->od, S_SWITCH_IN_DHCP_OPTIONS, 100, ds_cstr(&match), ds_cstr(&options_action), @@ -8611,7 +8713,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, op->od->nbs->copp, meter_groups), &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); /* If REGBIT_DHCP_OPTS_RESULT is set to 1, it means the * put_dhcpv6_opts action is successful */ @@ -8619,7 +8721,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_DHCP_RESPONSE, 100, ds_cstr(&match), ds_cstr(&response_action), inport->key, - &op->nbsp->dhcpv6_options->header_, op->lflow_ref); + &op->nbsp->dhcpv6_options->header_, lflow_ref); ds_destroy(&options_action); ds_destroy(&response_action); @@ -8652,7 +8754,7 @@ build_dhcpv6_options_flows(struct ovn_port *op, lflows, op->od, S_SWITCH_OUT_ACL_EVAL, 34000, ds_cstr(&match),dhcp6_actions, op->key, &op->nbsp->dhcpv6_options->header_, - op->lflow_ref); + lflow_ref); } break; } @@ -8663,7 +8765,8 @@ build_dhcpv6_options_flows(struct ovn_port *op, static void build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, const struct ovn_port *port, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; @@ -8683,7 +8786,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } for (size_t l = 0; l < rp->lsp_addrs[k].n_ipv6_addrs; l++) { ds_clear(&match); @@ -8699,7 +8802,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, ovn_lflow_add_with_lport_and_hint( lflows, op->od, S_SWITCH_IN_EXTERNAL_PORT, 100, ds_cstr(&match), debug_drop_action(), port->key, - &op->nbsp->header_, op->lflow_ref); + &op->nbsp->header_, lflow_ref); } ds_clear(&match); @@ -8716,7 +8819,7 @@ build_drop_arp_nd_flows_for_unbound_router_ports(struct ovn_port *op, debug_drop_action(), port->key, &op->nbsp->header_, - op->lflow_ref); + lflow_ref); } } } @@ -8731,19 +8834,22 @@ is_vlan_transparent(const struct ovn_datapath *od) static void build_lswitch_lflows_l2_unknown(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Ingress table 25/26: Destination lookup for unknown MACs. */ if (od->has_unknown) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, "outport == \"none\"", - "outport = \""MC_UNKNOWN "\"; output;"); + "outport = \""MC_UNKNOWN "\"; output;", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 50, - "outport == \"none\"", debug_drop_action()); + "outport == \"none\"", debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_UNKNOWN, 0, "1", - "output;"); + "output;", lflow_ref); } /* Build pre-ACL and ACL tables for both ingress and egress. @@ -8753,42 +8859,49 @@ build_lswitch_lflows_pre_acl_and_acl( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - build_pre_acls(od, lflows); - build_pre_lb(od, meter_groups, lflows); - build_pre_stateful(od, features, lflows); - build_qos(od, lflows); - build_stateful(od, features, lflows); - build_vtep_hairpin(od, lflows); + build_pre_acls(od, lflows, lflow_ref); + build_pre_lb(od, meter_groups, lflows, lflow_ref); + build_pre_stateful(od, features, lflows, lflow_ref); + build_qos(od, lflows, lflow_ref); + build_stateful(od, features, lflows, lflow_ref); + build_vtep_hairpin(od, lflows, lflow_ref); } /* Logical switch ingress table 0: Admission control framework (priority * 100). */ static void build_lswitch_lflows_admission_control(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); /* Logical VLANs not supported. */ if (!is_vlan_transparent(od)) { /* Block logical VLANs. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "vlan.present", debug_drop_action()); + "vlan.present", debug_drop_action(), + lflow_ref); } /* Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 100, - "eth.src[40]", debug_drop_action()); + "eth.src[40]", debug_drop_action(), + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_CHECK_PORT_SEC, 50, "1", - REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;"); + REGBIT_PORT_SEC_DROP" = check_in_port_sec(); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 50, - REGBIT_PORT_SEC_DROP" == 1", debug_drop_action()); + REGBIT_PORT_SEC_DROP" == 1", debug_drop_action(), + lflow_ref); - ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_APPLY_PORT_SEC, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder, skip requests coming from localnet @@ -8945,12 +9058,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, "output;", op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ea_s, op->lsp_addrs[i].ipv4_addrs[j].addr_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to an ARP request from the port that owns * the address (otherwise a DHCP client that ARPs to check @@ -9000,16 +9113,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ipv6_addrs[j].addr_s, op->lsp_addrs[i].ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 50, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 50, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); /* Do not reply to a solicitation from the port that owns * the address (otherwise DAD detection will fail). */ @@ -9065,12 +9178,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, - 30, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, + 30, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } /* Add IPv6 NDP responses. @@ -9113,16 +9226,16 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, lsp_is_router(op->nbsp) ? "nd_na_router" : "nd_na", ea_s, ea_s); - ovn_lflow_add_with_lflow_ref_hint__(lflows, op->od, - S_SWITCH_IN_ARP_ND_RSP, 30, - ds_cstr(match), - ds_cstr(actions), - NULL, - copp_meter_get(COPP_ND_NA, - op->od->nbs->copp, - meter_groups), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint__(lflows, op->od, + S_SWITCH_IN_ARP_ND_RSP, 30, + ds_cstr(match), + ds_cstr(actions), + NULL, + copp_meter_get(COPP_ND_NA, + op->od->nbs->copp, + meter_groups), + &op->nbsp->header_, + op->lflow_ref); ds_destroy(&ip6_dst_match); ds_destroy(&nd_target_match); } @@ -9133,10 +9246,12 @@ build_lswitch_arp_nd_responder_known_ips(struct ovn_port *op, * (priority 0)*/ static void build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_ARP_ND_RSP, 0, "1", "next;", + lflow_ref); } /* Ingress table 19: ARP/ND responder for service monitor source ip. @@ -9146,7 +9261,8 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, struct lflow_table *lflows, struct ds *actions, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { for (size_t i = 0; i < lb->n_vips; i++) { struct ovn_northd_lb_vip *lb_vip_nb = &lb->vips_nb[i]; @@ -9210,7 +9326,8 @@ build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, op->od, S_SWITCH_IN_ARP_ND_RSP, 110, ds_cstr(match), ds_cstr(actions), - &lb->nlb->header_); + &lb->nlb->header_, + lflow_ref); } } } @@ -9250,19 +9367,19 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, build_dhcpv4_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); build_dhcpv6_options_flows( op, &op->lsp_addrs[i], op->od->localnet_ports[j], is_external, - meter_groups, lflows); + meter_groups, lflows, op->lflow_ref); } } else { build_dhcpv4_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); build_dhcpv6_options_flows(op, &op->lsp_addrs[i], op, is_external, meter_groups, - lflows); + lflows, op->lflow_ref); } } } @@ -9275,14 +9392,20 @@ build_lswitch_dhcp_options_and_response(struct ovn_port *op, * (priority 0). */ static void build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DHCP_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_LOOKUP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_SWITCH_IN_EXTERNAL_PORT, 0, "1", "next;", + lflow_ref); } /* Logical switch ingress table 22 and 23: DNS lookup and response @@ -9291,7 +9414,8 @@ build_lswitch_dhcp_and_dns_defaults(struct ovn_datapath *od, static void build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); if (!ls_has_dns_records(od->nbs)) { @@ -9301,18 +9425,18 @@ build_lswitch_dns_lookup_and_response(struct ovn_datapath *od, "udp.dst == 53", REGBIT_DNS_LOOKUP_RESULT" = dns_lookup(); next;", copp_meter_get(COPP_DNS, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); const char *dns_action = "eth.dst <-> eth.src; ip4.src <-> ip4.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; const char *dns_match = "udp.dst == 53 && "REGBIT_DNS_LOOKUP_RESULT; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); dns_action = "eth.dst <-> eth.src; ip6.src <-> ip6.dst; " "udp.dst = udp.src; udp.src = 53; outport = inport; " "flags.loopback = 1; output;"; ovn_lflow_add(lflows, od, S_SWITCH_IN_DNS_RESPONSE, 100, - dns_match, dns_action); + dns_match, dns_action, lflow_ref); } /* Table 24: External port. Drop ARP request for router ips from @@ -9329,7 +9453,8 @@ build_lswitch_external_port(struct ovn_port *op, } for (size_t i = 0; i < op->od->n_localnet_ports; i++) { build_drop_arp_nd_flows_for_unbound_router_ports( - op, op->od->localnet_ports[i], lflows); + op, op->od->localnet_ports[i], lflows, + op->lflow_ref); } } @@ -9339,7 +9464,8 @@ static void build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, struct lflow_table *lflows, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbs); @@ -9347,7 +9473,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)", "handle_svc_check(inport);", copp_meter_get(COPP_SVC_MONITOR, od->nbs->copp, - meter_groups)); + meter_groups), lflow_ref); struct mcast_switch_info *mcast_sw_info = &od->mcast_info.sw; @@ -9358,27 +9484,31 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "igmp", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Punt MLD traffic to controller. */ ovn_lflow_metered(lflows, od, S_SWITCH_IN_L2_LKUP, 100, "mldv1 || mldv2", ds_cstr(actions), copp_meter_get(COPP_IGMP, od->nbs->copp, - meter_groups)); + meter_groups), + lflow_ref); /* Flood all IP multicast traffic destined to 224.0.0.X to all * ports - RFC 4541, section 2.1.2, item 2. */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip4.mcast && ip4.dst == 224.0.0.0/24", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); /* Flood all IPv6 multicast traffic destined to reserved * multicast IPs (RFC 4291, 2.7.1). */ ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 85, "ip6.mcast_flood", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", + lflow_ref); /* Forward uregistered IP multicast to routers with relay enabled * and to any ports configured to flood IP multicast traffic. @@ -9410,7 +9540,7 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 80, "ip4.mcast || ip6.mcast", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } } @@ -9418,11 +9548,12 @@ build_lswitch_destination_lookup_bmcast(struct ovn_datapath *od, "broadcast-arps-to-all-routers", true)) { ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 72, "eth.mcast && (arp.op == 1 || nd_ns)", - "outport = \""MC_FLOOD_L2"\"; output;"); + "outport = \""MC_FLOOD_L2"\"; output;", + lflow_ref); } ovn_lflow_add(lflows, od, S_SWITCH_IN_L2_LKUP, 70, "eth.mcast", - "outport = \""MC_FLOOD"\"; output;"); + "outport = \""MC_FLOOD"\"; output;", lflow_ref); } @@ -9432,7 +9563,8 @@ static void build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, struct lflow_table *lflows, struct ds *actions, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { uint64_t dummy; @@ -9504,7 +9636,7 @@ build_lswitch_ip_mcast_igmp_mld(struct ovn_igmp_group *igmp_group, igmp_group->mcgroup.name); ovn_lflow_add(lflows, igmp_group->datapath, S_SWITCH_IN_L2_LKUP, - 90, ds_cstr(match), ds_cstr(actions)); + 90, ds_cstr(match), ds_cstr(actions), lflow_ref); } } @@ -9544,12 +9676,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "unknown")) { continue; } else if (is_dynamic_lsp_address(op->nbsp->addresses[i])) { @@ -9564,12 +9696,12 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, - 50, ds_cstr(match), - ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, + 50, ds_cstr(match), + ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else if (!strcmp(op->nbsp->addresses[i], "router")) { if (!op->peer || !op->peer->nbrp || !ovs_scan(op->peer->nbrp->mac, @@ -9621,11 +9753,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_SWITCH_IN_L2_LKUP, 50, - ds_cstr(match), ds_cstr(actions), - &op->nbsp->header_, - op->lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_SWITCH_IN_L2_LKUP, 50, + ds_cstr(match), ds_cstr(actions), + &op->nbsp->header_, + op->lflow_ref); } else { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); @@ -9673,7 +9805,7 @@ build_lswitch_ip_unicast_lookup_for_nats(struct ovn_port *op, ds_clear(actions); ds_put_format(actions, action, op->json_key); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, + ovn_lflow_add_with_hint(lflows, op->od, S_SWITCH_IN_L2_LKUP, 50, ds_cstr(match), ds_cstr(actions), @@ -9921,7 +10053,8 @@ static void build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - const struct ovsdb_idl_row *stage_hint) + const struct ovsdb_idl_row *stage_hint, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -9977,7 +10110,8 @@ build_routing_policy_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&match, "%s", rule->match); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, - ds_cstr(&match), ds_cstr(&actions), stage_hint); + ds_cstr(&match), ds_cstr(&actions), stage_hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); } @@ -9987,7 +10121,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, const struct nbrec_logical_router_policy *rule, - uint16_t ecmp_group_id) + uint16_t ecmp_group_id, + struct lflow_ref *lflow_ref) { ovs_assert(rule->n_nexthops > 1); @@ -10059,7 +10194,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ecmp_group_id, i + 1); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY_ECMP, 100, ds_cstr(&match), - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); } ds_clear(&actions); @@ -10077,7 +10213,8 @@ build_ecmp_routing_policy_flows(struct lflow_table *lflows, ds_put_cstr(&actions, ");"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, rule->priority, rule->match, - ds_cstr(&actions), &rule->header_); + ds_cstr(&actions), &rule->header_, + lflow_ref); cleanup: ds_destroy(&match); @@ -10122,7 +10259,8 @@ get_route_table_id(struct simap *route_tables, const char *route_table_name) static void build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, struct nbrec_logical_router_port *lrp, - struct simap *route_tables) + struct simap *route_tables, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -10138,7 +10276,7 @@ build_route_table_lflow(struct ovn_datapath *od, struct lflow_table *lflows, REG_ROUTE_TABLE_ID, rtb_id); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -10537,7 +10675,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, const char *port_ip, struct ovn_port *out_port, const struct parsed_route *route, - struct ds *route_match) + struct ds *route_match, + struct lflow_ref *lflow_ref) { const struct nbrec_logical_router_static_route *st_route = route->route; struct ds base_match = DS_EMPTY_INITIALIZER; @@ -10560,12 +10699,12 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, free(cidr); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(&base_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* And packets that go out over an ECMP route need conntrack */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DEFRAG, 100, ds_cstr(route_match), "ct_next;", - &st_route->header_); + &st_route->header_, lflow_ref); /* Save src eth and inport in ct_label for packets that arrive over * an ECMP route. @@ -10582,7 +10721,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && udp", ds_cstr(&base_match)); @@ -10594,7 +10734,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (ct.new && !ct.est) && sctp", ds_cstr(&base_match)); @@ -10606,7 +10747,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10620,7 +10762,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, @@ -10634,7 +10777,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "%s && (!ct.rpl && ct.est) && sctp", @@ -10647,7 +10791,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, ct_ecmp_reply_port_match, out_port->sb->tunnel_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 100, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Bypass ECMP selection if we already have ct_label information * for where to route the packet. @@ -10666,12 +10811,14 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, port_ip, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, 10300, ds_cstr(&match), ds_cstr(&actions), - &st_route->header_); + &st_route->header_, + lflow_ref); /* Egress reply traffic for symmetric ECMP routes skips router policies. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_POLICY, 65535, ds_cstr(&ecmp_reply), "next;", - &st_route->header_); + &st_route->header_, + lflow_ref); /* Use REG_ECMP_ETH_FULL to pass the eth field from ct_label to eth.dst to * avoid masked access to ct_label. Otherwise it may prevent OVS flow @@ -10687,7 +10834,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, " pop(" REG_ECMP_ETH_FULL "); next;"; ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, ds_cstr(&ecmp_reply), - action, &st_route->header_); + action, &st_route->header_, + lflow_ref); ds_destroy(&base_match); ds_destroy(&match); @@ -10698,7 +10846,8 @@ add_ecmp_symmetric_reply_flows(struct lflow_table *lflows, static void build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, bool ct_masked_mark, const struct hmap *lr_ports, - struct ecmp_groups_node *eg) + struct ecmp_groups_node *eg, + struct lflow_ref *lflow_ref) { bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(&eg->prefix); @@ -10731,7 +10880,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_cstr(&actions, ");"); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, - ds_cstr(&route_match), ds_cstr(&actions)); + ds_cstr(&route_match), ds_cstr(&actions), + lflow_ref); /* Add per member flow */ struct ds match = DS_EMPTY_INITIALIZER; @@ -10754,7 +10904,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->key)) { add_ecmp_symmetric_reply_flows(lflows, od, ct_masked_mark, lrp_addr_s, out_port, - route_, &route_match); + route_, &route_match, + lflow_ref); } ds_clear(&match); ds_put_format(&match, REG_ECMP_GROUP_ID" == %"PRIu16" && " @@ -10774,7 +10925,7 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, out_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 100, ds_cstr(&match), ds_cstr(&actions), - &route->header_); + &route->header_, lflow_ref); } sset_destroy(&visited_ports); ds_destroy(&match); @@ -10831,17 +10982,17 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, ds_put_format(&actions, "ip.ttl--; %s", ds_cstr(&common_actions)); } - ovn_lflow_add_with_lflow_ref_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, - priority, ds_cstr(&match), - ds_cstr(&actions), stage_hint, - lflow_ref); + ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_ROUTING, + priority, ds_cstr(&match), + ds_cstr(&actions), stage_hint, + lflow_ref); if (op && op->has_bfd) { ds_put_format(&match, " && udp.dst == 3784"); - ovn_lflow_add_with_lflow_ref_hint(lflows, op->od, - S_ROUTER_IN_IP_ROUTING, - priority + 1, ds_cstr(&match), - ds_cstr(&common_actions),\ - stage_hint, lflow_ref); + ovn_lflow_add_with_hint(lflows, op->od, + S_ROUTER_IN_IP_ROUTING, + priority + 1, ds_cstr(&match), + ds_cstr(&common_actions),\ + stage_hint, lflow_ref); } ds_destroy(&match); ds_destroy(&common_actions); @@ -10851,7 +11002,8 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od, static void build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, const struct hmap *lr_ports, - const struct parsed_route *route_) + const struct parsed_route *route_, + struct lflow_ref *lflow_ref) { const char *lrp_addr_s = NULL; struct ovn_port *out_port = NULL; @@ -10875,7 +11027,7 @@ build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od, add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port, lrp_addr_s, prefix_s, route_->plen, route->nexthop, route_->is_src_route, route_->route_table_id, &route->header_, - route_->is_discard_route, ofs, NULL); + route_->is_discard_route, ofs, lflow_ref); free(prefix_s); } @@ -10951,7 +11103,8 @@ lrouter_use_common_zone(const struct ovn_datapath *od) static void build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, - struct ovn_datapath *od) + struct ovn_datapath *od, + struct lflow_ref *lflow_ref) { struct ovn_port *dgp = od->l3dgw_ports[0]; @@ -10990,7 +11143,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->new_match, new_match_len); @@ -11009,7 +11163,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, ds_cstr(ctx->undnat_match), ds_cstr(ctx->gw_redir_action), - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); ds_put_format(ctx->undnat_match, ") && (inport == %s || outport == %s)" @@ -11017,7 +11172,8 @@ build_distr_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, dgp->cr_port->json_key); ovn_lflow_add_with_hint(ctx->lflows, od, S_ROUTER_OUT_UNDNAT, 120, ds_cstr(ctx->undnat_match), undnat_action, - &ctx->lb->nlb->header_); + &ctx->lb->nlb->header_, + lflow_ref); ds_truncate(ctx->undnat_match, undnat_match_len); } @@ -11025,7 +11181,8 @@ static void build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, enum lrouter_nat_lb_flow_type type, const struct ovn_datapaths *lr_datapaths, - const unsigned long *dp_bitmap) + const unsigned long *dp_bitmap, + struct lflow_ref *lflow_ref) { unsigned long *dp_non_meter = NULL; bool build_non_meter = false; @@ -11051,14 +11208,14 @@ build_gw_lrouter_nat_flows_for_lb(struct lrouter_nat_lb_flows_ctx *ctx, bitmap_set0(dp_non_meter, index); ovn_lflow_add_with_hint__(ctx->lflows, od, S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), ctx->new_action[type], - NULL, meter, &ctx->lb->nlb->header_); + NULL, meter, &ctx->lb->nlb->header_, lflow_ref); } } if (!ctx->reject || build_non_meter) { ovn_lflow_add_with_dp_group(ctx->lflows, dp_non_meter ? dp_non_meter : dp_bitmap, ods_size(lr_datapaths), S_ROUTER_IN_DNAT, ctx->prio, ds_cstr(ctx->new_match), - ctx->new_action[type], &ctx->lb->nlb->header_); + ctx->new_action[type], &ctx->lb->nlb->header_, lflow_ref); } bitmap_free(dp_non_meter); } @@ -11073,7 +11230,8 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, struct ds *match, struct ds *action, const struct shash *meter_groups, const struct chassis_features *features, - const struct hmap *svc_monitor_map) + const struct hmap *svc_monitor_map, + struct lflow_ref *lflow_ref) { const struct ovn_northd_lb *lb = lb_dps->lb; bool ipv4 = lb_vip->address_family == AF_INET; @@ -11191,7 +11349,7 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, if (!od->n_l3dgw_ports) { bitmap_set1(gw_dp_bitmap[type], index); } else { - build_distr_lrouter_nat_flows_for_lb(&ctx, type, od); + build_distr_lrouter_nat_flows_for_lb(&ctx, type, od, lflow_ref); } if (lb->affinity_timeout) { @@ -11212,16 +11370,17 @@ build_lrouter_nat_flows_for_lb(struct ovn_lb_vip *lb_vip, * S_ROUTER_IN_DNAT stage. */ ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 120, ds_cstr(&unsnat_match), "next;", - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } for (size_t type = 0; type < LROUTER_NAT_LB_FLOW_MAX; type++) { build_gw_lrouter_nat_flows_for_lb(&ctx, type, lr_datapaths, - gw_dp_bitmap[type]); + gw_dp_bitmap[type], + lflow_ref); build_lb_affinity_lr_flows(lflows, lb, lb_vip, ds_cstr(match), aff_action[type], aff_dp_bitmap[type], - lr_datapaths); + lr_datapaths, lflow_ref); } ds_destroy(&unsnat_match); @@ -11243,7 +11402,8 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, const struct ovn_datapaths *ls_datapaths, const struct chassis_features *features, const struct hmap *svc_monitor_map, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_ls) { return; @@ -11269,7 +11429,8 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbs->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, + lflow_ref); } /* Ignore L4 port information in the key because fragmented packets * may not have L4 information. The pre-stateful table will send @@ -11284,9 +11445,9 @@ build_lswitch_flows_for_lb(struct ovn_lb_datapaths *lb_dps, * connection, so it is okay if we do not hit the above match on * REGBIT_CONNTRACK_COMMIT. */ build_lb_rules_pre_stateful(lflows, lb_dps, features->ct_no_masked_label, - ls_datapaths, match, action); + ls_datapaths, match, action, lflow_ref); build_lb_rules(lflows, lb_dps, ls_datapaths, features, match, action, - meter_groups, svc_monitor_map); + meter_groups, svc_monitor_map, lflow_ref); } /* If there are any load balancing rules, we should send the packet to @@ -11301,7 +11462,8 @@ static void build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct lflow_table *lflows, const struct ovn_datapaths *lr_datapaths, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { if (!lb_dps->n_nb_lr) { return; @@ -11319,7 +11481,7 @@ build_lrouter_defrag_flows_for_lb(struct ovn_lb_datapaths *lb_dps, ovn_lflow_add_with_dp_group( lflows, lb_dps->nb_lr_map, ods_size(lr_datapaths), S_ROUTER_IN_DEFRAG, prio, ds_cstr(match), "ct_dnat;", - &lb_dps->lb->nlb->header_); + &lb_dps->lb->nlb->header_, lflow_ref); } } @@ -11331,7 +11493,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, const struct lr_lb_nat_data_table *lr_lbnats, const struct chassis_features *features, const struct hmap *svc_monitor_map, - struct ds *match, struct ds *action) + struct ds *match, struct ds *action, + struct lflow_ref *lflow_ref) { size_t index; @@ -11346,7 +11509,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, build_lrouter_nat_flows_for_lb(lb_vip, lb_dps, &lb->vips_nb[i], lr_datapaths, lr_lbnats, lflows, match, action, meter_groups, features, - svc_monitor_map); + svc_monitor_map, lflow_ref); if (!build_empty_lb_event_flow(lb_vip, lb, match, action)) { continue; @@ -11361,7 +11524,7 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, copp_meter_get(COPP_EVENT_ELB, od->nbr->copp, meter_groups), - &lb->nlb->header_); + &lb->nlb->header_, lflow_ref); } } @@ -11370,7 +11533,8 @@ build_lrouter_flows_for_lb(struct ovn_lb_datapaths *lb_dps, struct ovn_datapath *od = lr_datapaths->array[index]; ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, - "flags.skip_snat_for_lb == 1 && ip", "next;"); + "flags.skip_snat_for_lb == 1 && ip", "next;", + lflow_ref); } } } @@ -11485,7 +11649,8 @@ static inline void lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, const struct nbrec_nat *nat, - bool is_v6, bool is_src, int cidr_bits) + bool is_v6, bool is_src, int cidr_bits, + struct lflow_ref *lflow_ref) { struct nbrec_address_set *allowed_ext_ips = nat->allowed_ext_ips; struct nbrec_address_set *exempted_ext_ips = nat->exempted_ext_ips; @@ -11536,7 +11701,7 @@ lrouter_nat_add_ext_ip_match(const struct ovn_datapath *od, ovn_lflow_add_with_hint(lflows, od, stage, priority, ds_cstr(&match_exempt), "next;", - &nat->header_); + &nat->header_, lflow_ref); ds_destroy(&match_exempt); } } @@ -11550,7 +11715,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, const char *ip_address, const char *eth_addr, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11582,7 +11748,8 @@ build_lrouter_arp_flow(const struct ovn_datapath *od, struct ovn_port *op, } ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11601,7 +11768,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, struct ds *extra_match, bool drop, uint16_t priority, const struct ovsdb_idl_row *hint, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; @@ -11624,7 +11792,8 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, if (drop) { ds_put_cstr(&actions, debug_drop_action()); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, priority, - ds_cstr(&match), ds_cstr(&actions), hint); + ds_cstr(&match), ds_cstr(&actions), hint, + lflow_ref); } else { ds_put_format(&actions, "%s { " @@ -11642,7 +11811,7 @@ build_lrouter_nd_flow(const struct ovn_datapath *od, struct ovn_port *op, ds_cstr(&match), ds_cstr(&actions), NULL, copp_meter_get(COPP_ND_NA, od->nbr->copp, meter_groups), - hint); + hint, lflow_ref); } ds_destroy(&match); @@ -11653,7 +11822,8 @@ static void build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11663,12 +11833,14 @@ build_lrouter_nat_arp_nd_flow(const struct ovn_datapath *od, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(od, NULL, ext_addrs->ipv4_addrs[0].addr_s, REG_INPORT_ETH_ADDR, NULL, false, 90, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } } @@ -11676,7 +11848,8 @@ static void build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, struct ovn_nat *nat_entry, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { struct lport_addresses *ext_addrs = &nat_entry->ext_addrs; const struct nbrec_nat *nat = nat_entry->nb; @@ -11724,21 +11897,25 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op, ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, &match, false, 92, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); build_lrouter_nd_flow(op->od, op, "nd_na", ext_addrs->ipv6_addrs[0].addr_s, ext_addrs->ipv6_addrs[0].sn_addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows, meter_groups); + &nat->header_, lflows, meter_groups, + lflow_ref); } else { build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, &match, false, 92, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); build_lrouter_arp_flow(op->od, op, ext_addrs->ipv4_addrs[0].addr_s, mac_s, NULL, true, 91, - &nat->header_, lflows); + &nat->header_, lflows, + lflow_ref); } ds_destroy(&match); @@ -11749,7 +11926,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, enum ovn_stage stage, uint16_t priority, bool drop_snat_ip, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { struct ds match_ips = DS_EMPTY_INITIALIZER; @@ -11776,7 +11954,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip4.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11806,7 +11985,8 @@ build_lrouter_drop_own_dest(struct ovn_port *op, char *match = xasprintf("ip6.dst == {%s}", ds_cstr(&match_ips)); ovn_lflow_add_with_hint(lflows, op->od, stage, priority, match, debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); free(match); } } @@ -11817,14 +11997,15 @@ static void build_lrouter_force_snat_flows(struct lflow_table *lflows, const struct ovn_datapath *od, const char *ip_version, const char *ip_addr, - const char *context) + const char *context, + struct lflow_ref *lflow_ref) { struct ds match = DS_EMPTY_INITIALIZER; struct ds actions = DS_EMPTY_INITIALIZER; ds_put_format(&match, "ip%s && ip%s.dst == %s", ip_version, ip_version, ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(&match), "ct_snat;"); + ds_cstr(&match), "ct_snat;", lflow_ref); /* Higher priority rules to force SNAT with the IP addresses * configured in the Gateway router. This only takes effect @@ -11834,7 +12015,8 @@ build_lrouter_force_snat_flows(struct lflow_table *lflows, context, ip_version); ds_put_format(&actions, "ct_snat(%s);", ip_addr); ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, - ds_cstr(&match), ds_cstr(&actions)); + ds_cstr(&match), ds_cstr(&actions), + lflow_ref); ds_destroy(&match); ds_destroy(&actions); @@ -11844,7 +12026,8 @@ static void build_lrouter_force_snat_flows_op(struct ovn_port *op, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!op->peer || !lrnat_rec->lb_force_snat_router_ip) { @@ -11858,7 +12041,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip4.dst == %s", op->json_key, op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11870,7 +12053,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv4_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv4_addrs > 1) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11890,7 +12074,7 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(match, "inport == %s && ip6.dst == %s", op->json_key, op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_UNSNAT, 110, - ds_cstr(match), "ct_snat;"); + ds_cstr(match), "ct_snat;", lflow_ref); ds_clear(match); @@ -11902,7 +12086,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, ds_put_format(actions, "ct_snat(%s);", op->lrp_networks.ipv6_addrs[0].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_SNAT, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); if (op->lrp_networks.n_ipv6_addrs > 2) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5); VLOG_WARN_RL(&rl, "Logical router port %s is configured with " @@ -11916,7 +12101,8 @@ build_lrouter_force_snat_flows_op(struct ovn_port *op, static void build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (!op->has_bfd) { return; @@ -11931,7 +12117,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip4.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -11939,7 +12126,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { ds_clear(&ip_list); @@ -11950,7 +12138,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&ip_list)); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 110, ds_cstr(&match), "next; ", - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(&match); ds_put_format(&match, "ip6.dst == %s && udp.dst == 3784", ds_cstr(&ip_list)); @@ -11958,7 +12147,8 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, ds_cstr(&match), "handle_bfd_msg(); ", NULL, copp_meter_get(COPP_BFD, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_destroy(&ip_list); @@ -11970,16 +12160,19 @@ build_lrouter_bfd_flows(struct lflow_table *lflows, struct ovn_port *op, */ static void build_adm_ctrl_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Logical VLANs not supported. * Broadcast/multicast source address is invalid. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 100, - "vlan.present || eth.src[40]", debug_drop_action()); + "vlan.present || eth.src[40]", debug_drop_action(), + lflow_ref); /* Default action for L2 security is to drop. */ - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ADMISSION, + lflow_ref); } static int @@ -12013,11 +12206,12 @@ build_gateway_get_l2_hdr_size(struct ovn_port *op) /* All 'gateway_mtu' and 'gateway_mtu_bypass' flows should be built with this * function. */ -static void OVS_PRINTF_FORMAT(9, 10) +static void OVS_PRINTF_FORMAT(10, 11) build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, enum ovn_stage stage, uint16_t prio_low, uint16_t prio_high, struct ds *match, struct ds *actions, const struct ovsdb_idl_row *hint, + struct lflow_ref *lflow_ref, const char *extra_actions_fmt, ...) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); @@ -12035,7 +12229,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format_valist(actions, extra_actions_fmt, extra_actions_args); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_low, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); if (gw_mtu > 0) { const char *gw_mtu_bypass = smap_get(&op->nbrp->options, @@ -12047,7 +12241,7 @@ build_gateway_mtu_flow(struct lflow_table *lflows, struct ovn_port *op, ds_put_format(match, " && (%s)", gw_mtu_bypass); ovn_lflow_add_with_hint(lflows, op->od, stage, prio_high, ds_cstr(match), ds_cstr(actions), - hint); + hint, lflow_ref); } } va_end(extra_actions_args); @@ -12076,7 +12270,8 @@ consider_l3dgw_port_is_centralized(struct ovn_port *op) static void build_adm_ctrl_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12100,6 +12295,7 @@ build_adm_ctrl_flows_for_lrouter_port( ds_put_format(match, "eth.mcast && inport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); @@ -12120,6 +12316,7 @@ build_adm_ctrl_flows_for_lrouter_port( } build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &op->nbrp->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", op->lrp_networks.ea_s); } @@ -12131,7 +12328,8 @@ static void build_neigh_learning_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12178,7 +12376,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, - "arp.op == 2", ds_cstr(actions)); + "arp.op == 2", ds_cstr(actions), lflow_ref); ds_clear(actions); ds_put_format(actions, REGBIT_LOOKUP_NEIGHBOR_RESULT @@ -12186,7 +12384,7 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" = 1; "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_na", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); if (!learn_from_arp_request) { /* Add flow to skip GARP LLA if we don't know it already. @@ -12203,7 +12401,7 @@ build_neigh_learning_flows_for_lrouter( " = lookup_nd_ip(inport, ip6.src); next;"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, "nd_na && ip6.src == fe80::/10 && ip6.dst == ff00::/8", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); } ds_clear(actions); @@ -12213,12 +12411,13 @@ build_neigh_learning_flows_for_lrouter( REGBIT_LOOKUP_NEIGHBOR_IP_RESULT " = lookup_nd_ip(inport, ip6.src); "); ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, "nd_ns", - ds_cstr(actions)); + ds_cstr(actions), lflow_ref); /* For other packet types, we can skip neighbor learning. * So set REGBIT_LOOKUP_NEIGHBOR_RESULT to 1. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 0, "1", - REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;"); + REGBIT_LOOKUP_NEIGHBOR_RESULT" = 1; next;", + lflow_ref); /* Flows for LEARN_NEIGHBOR. */ /* Skip Neighbor learning if not required. */ @@ -12227,33 +12426,40 @@ build_neigh_learning_flows_for_lrouter( learn_from_arp_request ? "" : " || "REGBIT_LOOKUP_NEIGHBOR_IP_RESULT" == 0"); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 100, - ds_cstr(match), "mac_cache_use; next;"); + ds_cstr(match), "mac_cache_use; next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "arp", "put_arp(inport, arp.spa, arp.sha); next;", copp_meter_get(COPP_ARP, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, - "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;"); + "nd_ns && (ip6.src == 0 || nd.sll == 0)", "next;", + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 95, "nd_na && nd.tll == 0", "put_nd(inport, nd.target, eth.src); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_na", "put_nd(inport, nd.target, nd.tll); next;", copp_meter_get(COPP_ND_NA, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, 90, "nd_ns", "put_nd(inport, ip6.src, nd.sll); next;", copp_meter_get(COPP_ND_NS, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_LEARN_NEIGHBOR, + lflow_ref); } /* Logical router ingress Table 1: Neighbor lookup lflows @@ -12261,7 +12467,8 @@ build_neigh_learning_flows_for_lrouter( static void build_neigh_learning_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -12293,7 +12500,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 110, ds_cstr(match), actions_s, - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } ds_clear(match); ds_put_format(match, @@ -12314,7 +12522,8 @@ build_neigh_learning_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_LOOKUP_NEIGHBOR, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12324,7 +12533,8 @@ static void build_ND_RA_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->nbrp->peer || !op->peer) { @@ -12411,7 +12621,8 @@ build_ND_RA_flows_for_lrouter_port( copp_meter_get(COPP_ND_RA_OPTS, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(actions); ds_clear(match); ds_put_format(match, "inport == %s && ip6.dst == ff02::2 && " @@ -12430,7 +12641,8 @@ build_ND_RA_flows_for_lrouter_port( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ND_RA_RESPONSE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12438,22 +12650,26 @@ build_ND_RA_flows_for_lrouter_port( * responder, by default goto next. (priority 0). */ static void build_ND_RA_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_OPTIONS, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ND_RA_RESPONSE, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table IP_ROUTING_PRE: * by default goto next. (priority 0). */ static void build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_PRE, 0, "1", - REG_ROUTE_TABLE_ID" = 0; next;"); + REG_ROUTE_TABLE_ID" = 0; next;", lflow_ref); } /* Logical router ingress table IP_ROUTING : IP Routing. @@ -12477,7 +12693,8 @@ build_ip_routing_pre_flows_for_lrouter(struct ovn_datapath *od, */ static void build_ip_routing_flows_for_lrp( - struct ovn_port *op, struct lflow_table *lflows) + struct ovn_port *op, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); for (int i = 0; i < op->lrp_networks.n_ipv4_addrs; i++) { @@ -12485,7 +12702,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv4_addrs[i].network_s, op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) { @@ -12493,7 +12710,7 @@ build_ip_routing_flows_for_lrp( op->lrp_networks.ipv6_addrs[i].network_s, op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0, &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED, - NULL); + lflow_ref); } } @@ -12557,13 +12774,17 @@ static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, struct lflow_table *lflows, const struct hmap *lr_ports, - const struct hmap *bfd_connections) + const struct hmap *bfd_connections, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_IP_ROUTING, + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); struct hmap ecmp_groups = HMAP_INITIALIZER(&ecmp_groups); struct hmap unique_routes = HMAP_INITIALIZER(&unique_routes); @@ -12573,7 +12794,7 @@ build_static_route_flows_for_lrouter( for (int i = 0; i < od->nbr->n_ports; i++) { build_route_table_lflow(od, lflows, od->nbr->ports[i], - &route_tables); + &route_tables, lflow_ref); } for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -12603,11 +12824,11 @@ build_static_route_flows_for_lrouter( /* add a flow in IP_ROUTING, and one flow for each member in * IP_ROUTING_ECMP. */ build_ecmp_route_flow(lflows, od, features->ct_no_masked_label, - lr_ports, group); + lr_ports, group, lflow_ref); } const struct unique_routes_node *ur; HMAP_FOR_EACH (ur, hmap_node, &unique_routes) { - build_static_route_flow(lflows, od, lr_ports, ur->route); + build_static_route_flow(lflows, od, lr_ports, ur->route, lflow_ref); } ecmp_groups_destroy(&ecmp_groups); unique_routes_destroy(&unique_routes); @@ -12621,7 +12842,8 @@ build_static_route_flows_for_lrouter( static void build_mcast_lookup_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); @@ -12629,7 +12851,8 @@ build_mcast_lookup_flows_for_lrouter( * i.e., router solicitation and router advertisement. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - "nd_rs || nd_ra", debug_drop_action()); + "nd_rs || nd_ra", debug_drop_action(), + lflow_ref); if (!od->mcast_info.rtr.relay) { return; } @@ -12657,7 +12880,8 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(actions, "outport = \"%s\"; ip.ttl--; next;", igmp_group->mcgroup.name); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10500, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } /* If needed, flood unregistered multicast on statically configured @@ -12676,13 +12900,15 @@ build_mcast_lookup_flows_for_lrouter( ds_put_format(match, "eth.src == %s && igmp", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); ds_clear(match); ds_put_format(match, "eth.src == %s && (mldv1 || mldv2)", op->lrp_networks.ea_s); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10550, - ds_cstr(match), debug_drop_action()); + ds_cstr(match), debug_drop_action(), + lflow_ref); } ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, @@ -12690,23 +12916,27 @@ build_mcast_lookup_flows_for_lrouter( "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10460, "mldv1 || mldv2", "clone { " "outport = \""MC_STATIC"\"; " "next; " - "};"); + "};", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, "ip4.mcast || ip6.mcast", "clone { " "outport = \""MC_STATIC"\"; " "ip.ttl--; " "next; " - "};"); + "};", + lflow_ref); } else { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 10450, - "ip4.mcast || ip6.mcast", debug_drop_action()); + "ip4.mcast || ip6.mcast", debug_drop_action(), + lflow_ref); } } @@ -12722,16 +12952,20 @@ build_mcast_lookup_flows_for_lrouter( static void build_ingress_policy_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - const struct hmap *lr_ports) + const struct hmap *lr_ports, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* This is a catch-all rule. It has the lowest priority (0) * does a match-all("1") and pass-through (next) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY, 0, "1", - REG_ECMP_GROUP_ID" = 0; next;"); + REG_ECMP_GROUP_ID" = 0; next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_POLICY_ECMP, 150, - REG_ECMP_GROUP_ID" == 0", "next;"); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP); + REG_ECMP_GROUP_ID" == 0", "next;", + lflow_ref); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_POLICY_ECMP, + lflow_ref); /* Convert routing policies to flows. */ uint16_t ecmp_group_id = 1; @@ -12743,11 +12977,11 @@ build_ingress_policy_flows_for_lrouter( if (is_ecmp_reroute) { build_ecmp_routing_policy_flows(lflows, od, lr_ports, rule, - ecmp_group_id); + ecmp_group_id, lflow_ref); ecmp_group_id++; } else { build_routing_policy_flow(lflows, od, lr_ports, rule, - &rule->header_); + &rule->header_, lflow_ref); } } } @@ -12755,21 +12989,26 @@ build_ingress_policy_flows_for_lrouter( /* Local router ingress table ARP_RESOLVE: ARP Resolution. */ static void build_arp_resolve_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Multicast packets already have the outport set so just advance to * next table (priority 500). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 500, - "ip4.mcast || ip6.mcast", "next;"); + "ip4.mcast || ip6.mcast", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip4", - "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;"); + "get_arp(outport, " REG_NEXT_HOP_IPV4 "); next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 1, "ip6", - "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;"); + "get_nd(outport, " REG_NEXT_HOP_IPV6 "); next;", + lflow_ref); - ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE); + ovn_lflow_add_default_drop(lflows, od, S_ROUTER_IN_ARP_RESOLVE, + lflow_ref); } static void @@ -12802,9 +13041,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add_with_lflow_ref(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, - 100, ds_cstr(match), ds_cstr(actions), - lflow_ref); + ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, + ds_cstr(match), ds_cstr(actions), lflow_ref); } destroy_routable_addresses(&ra); } @@ -12823,7 +13061,8 @@ routable_addresses_to_lflows(struct lflow_table *lflows, static void build_arp_resolve_flows_for_lrp( struct ovn_port *op, - struct lflow_table *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* This is a logical router port. If next-hop IP address in @@ -12848,7 +13087,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } if (op->lrp_networks.n_ipv6_addrs) { @@ -12864,7 +13104,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -12889,7 +13130,8 @@ build_arp_resolve_flows_for_lrp( ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_ARP_RESOLVE, 50, ds_cstr(match), ds_cstr(actions), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -12939,7 +13181,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -12971,7 +13213,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), @@ -13020,7 +13262,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13037,7 +13279,7 @@ build_arp_resolve_flows_for_lsp( ds_clear(actions); ds_put_format(actions, "eth.dst = %s; next;", router_port->lrp_networks.ea_s); - ovn_lflow_add_with_lflow_ref_hint(lflows, peer->od, + ovn_lflow_add_with_hint(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), &op->nbsp->header_, @@ -13098,7 +13340,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, const struct shash *meter_groups, struct ds *match, struct ds *actions, enum ovn_stage stage, - struct ovn_port *outport) + struct ovn_port *outport, + struct lflow_ref *lflow_ref) { char *outport_match = outport ? xasprintf("outport == %s && ", outport->json_key) @@ -13141,7 +13384,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } char *ip6_src = NULL; @@ -13181,7 +13425,8 @@ build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } free(outport_match); } @@ -13192,7 +13437,8 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, const struct hmap *lr_ports, const struct shash *meter_groups, struct ds *match, - struct ds *actions) + struct ds *actions, + struct lflow_ref *lflow_ref) { int gw_mtu = smap_get_int(&op->nbrp->options, "gateway_mtu", 0); if (gw_mtu <= 0) { @@ -13202,12 +13448,13 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); build_gateway_mtu_flow(lflows, op, S_ROUTER_IN_CHK_PKT_LEN, 50, 55, - match, actions, &op->nbrp->header_, "next;"); + match, actions, &op->nbrp->header_, + lflow_ref, "next;"); /* ingress traffic */ build_icmperr_pkt_big_flows(op, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_IP_INPUT, - NULL); + NULL, lflow_ref); for (size_t i = 0; i < op->od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13219,7 +13466,7 @@ build_check_pkt_len_flows_for_lrp(struct ovn_port *op, /* egress traffic */ build_icmperr_pkt_big_flows(rp, gw_mtu, lflows, meter_groups, match, actions, S_ROUTER_IN_LARGER_PKTS, - op); + op, lflow_ref); } } @@ -13241,15 +13488,16 @@ build_check_pkt_len_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_CHK_PKT_LEN, 0, "1", - "next;"); + "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_LARGER_PKTS, 0, "1", - "next;"); + "next;", lflow_ref); for (size_t i = 0; i < od->nbr->n_ports; i++) { struct ovn_port *rp = ovn_port_find(lr_ports, @@ -13258,7 +13506,7 @@ build_check_pkt_len_flows_for_lrouter( continue; } build_check_pkt_len_flows_for_lrp(rp, lflows, lr_ports, meter_groups, - match, actions); + match, actions, lflow_ref); } } @@ -13266,7 +13514,8 @@ build_check_pkt_len_flows_for_lrouter( static void build_gateway_redirect_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13299,18 +13548,20 @@ build_gateway_redirect_flows_for_lrouter( od->l3dgw_ports[i]->cr_port->json_key); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(match), ds_cstr(actions), - stage_hint); + stage_hint, lflow_ref); } /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", "next;", + lflow_ref); } /* Logical router ingress table GW_REDIRECT: Gateway redirect. */ static void build_lr_gateway_redirect_flows_for_nats( const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, - struct lflow_table *lflows, struct ds *match, struct ds *actions) + struct lflow_table *lflows, struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (size_t i = 0; i < od->n_l3dgw_ports; i++) { @@ -13351,21 +13602,23 @@ build_lr_gateway_redirect_flows_for_nats( if (nat->nb->allowed_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), - ds_cstr(actions), stage_hint); + ds_cstr(actions), stage_hint, + lflow_ref); if (add_def_flow) { ds_clear(&match_ext); ds_put_format(&match_ext, "ip && ip%s.dst == %s", nat_entry_is_v6(nat) ? "6" : "4", nat->nb->external_ip); ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 70, - ds_cstr(&match_ext), debug_drop_action()); + ds_cstr(&match_ext), debug_drop_action(), + lflow_ref); add_def_flow = false; } } else if (nat->nb->exempted_ext_ips) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 75, ds_cstr(&match_ext), debug_drop_action(), - stage_hint); + stage_hint, lflow_ref); } ds_destroy(&match_ext); } @@ -13381,7 +13634,8 @@ static void build_arp_request_flows_for_lrouter( struct ovn_datapath *od, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); for (int i = 0; i < od->nbr->n_static_routes; i++) { @@ -13423,7 +13677,8 @@ build_arp_request_flows_for_lrouter( copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, meter_groups), - &route->header_); + &route->header_, + lflow_ref); } ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, @@ -13436,7 +13691,8 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ARP_RESOLVE, od->nbr->copp, - meter_groups)); + meter_groups), + lflow_ref); ovn_lflow_metered(lflows, od, S_ROUTER_IN_ARP_REQUEST, 100, "eth.dst == 00:00:00:00:00:00 && ip6", "nd_ns { " @@ -13444,8 +13700,10 @@ build_arp_request_flows_for_lrouter( "output; " "};", copp_meter_get(COPP_ND_NS_RESOLVE, od->nbr->copp, - meter_groups)); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;"); + meter_groups), + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_REQUEST, 0, "1", "output;", + lflow_ref); } /* Logical router egress table DELIVERY: Delivery (priority 100-110). @@ -13458,7 +13716,8 @@ build_arp_request_flows_for_lrouter( static void build_egress_delivery_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match, struct ds *actions) + struct ds *match, struct ds *actions, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (!lrport_is_enabled(op->nbrp)) { @@ -13486,20 +13745,23 @@ build_egress_delivery_flows_for_lrouter_port( ds_put_format(actions, "eth.src = %s; output;", op->lrp_networks.ea_s); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 110, - ds_cstr(match), ds_cstr(actions)); + ds_cstr(match), ds_cstr(actions), + lflow_ref); } ds_clear(match); ds_put_format(match, "outport == %s", op->json_key); ovn_lflow_add(lflows, op->od, S_ROUTER_OUT_DELIVERY, 100, - ds_cstr(match), "output;"); + ds_cstr(match), "output;", lflow_ref); - ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY); + ovn_lflow_add_default_drop(lflows, op->od, S_ROUTER_OUT_DELIVERY, + lflow_ref); } static void build_misc_local_traffic_drop_flows_for_lrouter( - struct ovn_datapath *od, struct lflow_table *lflows) + struct ovn_datapath *od, struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Allow IGMP and MLD packets (with TTL = 1) if the router is @@ -13507,9 +13769,11 @@ build_misc_local_traffic_drop_flows_for_lrouter( */ if (od->mcast_info.rtr.flood_static) { ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "igmp && ip.ttl == 1", "next;"); + "igmp && ip.ttl == 1", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 120, - "(mldv1 || mldv2) && ip.ttl == 1", "next;"); + "(mldv1 || mldv2) && ip.ttl == 1", "next;", + lflow_ref); } /* L3 admission control: drop multicast and broadcast source, localhost @@ -13522,7 +13786,8 @@ build_misc_local_traffic_drop_flows_for_lrouter( "ip4.dst == 127.0.0.0/8 || " "ip4.src == 0.0.0.0/8 || " "ip4.dst == 0.0.0.0/8", - debug_drop_action()); + debug_drop_action(), + lflow_ref); /* Drop ARP packets (priority 85). ARP request packets for router's own * IPs are handled with priority-90 flows. @@ -13530,28 +13795,32 @@ build_misc_local_traffic_drop_flows_for_lrouter( * IPs are handled with priority-90 flows. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 85, - "arp || nd", debug_drop_action()); + "arp || nd", debug_drop_action(), + lflow_ref); /* Allow IPv6 multicast traffic that's supposed to reach the * router pipeline (e.g., router solicitations). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 84, "nd_rs || nd_ra", - "next;"); + "next;", lflow_ref); /* Drop other reserved multicast. */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 83, - "ip6.mcast_rsvd", debug_drop_action()); + "ip6.mcast_rsvd", debug_drop_action(), + lflow_ref); /* Allow other multicast if relay enabled (priority 82). */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 82, "ip4.mcast || ip6.mcast", (od->mcast_info.rtr.relay ? "next;" : - debug_drop_action())); + debug_drop_action()), + lflow_ref); /* Drop Ethernet local broadcast. By definition this traffic should * not be forwarded.*/ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 50, - "eth.bcast", debug_drop_action()); + "eth.bcast", debug_drop_action(), + lflow_ref); /* Avoid ICMP time exceeded for multicast, silent drop instead. * See RFC1812 section 5.3.1: @@ -13568,21 +13837,25 @@ build_misc_local_traffic_drop_flows_for_lrouter( * (priority-31 flows will send ICMP time exceeded) */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 32, "ip.ttl == {0, 1} && !ip.later_frag && " - "(ip4.mcast || ip6.mcast)", debug_drop_action()); + "(ip4.mcast || ip6.mcast)", debug_drop_action(), + lflow_ref); /* TTL discard */ ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 30, - "ip.ttl == {0, 1}", debug_drop_action()); + "ip.ttl == {0, 1}", debug_drop_action(), + lflow_ref); /* Pass other traffic not already handled to the next table for * routing. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_INPUT, 0, "1", "next;", + lflow_ref); } static void build_dhcpv6_reply_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, - struct ds *match) + struct ds *match, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (op->l3dgw_port) { @@ -13595,7 +13868,8 @@ build_dhcpv6_reply_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s); ovn_lflow_add(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), - "reg0 = 0; handle_dhcpv6_reply;"); + "reg0 = 0; handle_dhcpv6_reply;", + lflow_ref); } } @@ -13603,7 +13877,8 @@ static void build_ipv6_input_flows_for_lrouter_port( struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); if (is_cr_port(op)) { @@ -13627,7 +13902,7 @@ build_ipv6_input_flows_for_lrouter_port( "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), lrp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* ND reply. These flows reply to ND solicitations for the @@ -13648,7 +13923,8 @@ build_ipv6_input_flows_for_lrouter_port( op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].sn_addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows, meter_groups); + &op->nbrp->header_, lflows, meter_groups, + lflow_ref); } /* UDP/TCP/SCTP port unreachable */ @@ -13668,7 +13944,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13684,7 +13961,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13703,7 +13981,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13722,7 +14001,8 @@ build_ipv6_input_flows_for_lrouter_port( COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } @@ -13762,7 +14042,7 @@ build_ipv6_input_flows_for_lrouter_port( 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP6_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); } @@ -13771,7 +14051,8 @@ static void build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, const struct lr_nat_record *lrnat_rec, struct lflow_table *lflows, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); if (!od->nbr->n_nat) { @@ -13800,7 +14081,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, if (!strcmp(nat_entry->nb->type, "snat")) { continue; } - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -13815,7 +14097,8 @@ build_lrouter_arp_nd_for_datapath(const struct ovn_datapath *od, struct ovn_nat *nat_entry = CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); - build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups); + build_lrouter_nat_arp_nd_flow(od, nat_entry, lflows, meter_groups, + lflow_ref); } } @@ -13824,7 +14107,8 @@ static void build_lrouter_ipv4_ip_input(struct ovn_port *op, struct lflow_table *lflows, struct ds *match, struct ds *actions, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -13842,7 +14126,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, ds_put_cstr(match, " && "REGBIT_EGRESS_LOOPBACK" == 0"); ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 100, ds_cstr(match), debug_drop_action(), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); /* ICMP echo reply. These flows reply to ICMP echo requests * received for the router's IP address. Since packets only @@ -13861,11 +14145,11 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, "next; "; ovn_lflow_add_with_hint(lflows, op->od, S_ROUTER_IN_IP_INPUT, 90, ds_cstr(match), icmp_actions, - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } /* BFD msg handling */ - build_lrouter_bfd_flows(lflows, op, meter_groups); + build_lrouter_bfd_flows(lflows, op, meter_groups, lflow_ref); /* ICMP time exceeded */ struct ds ip_ds = DS_EMPTY_INITIALIZER; @@ -13894,7 +14178,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, 31, ds_cstr(match), ds_cstr(actions), NULL, copp_meter_get(COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, lflow_ref); } ds_destroy(&ip_ds); @@ -13944,7 +14228,7 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, build_lrouter_arp_flow(op->od, op, op->lrp_networks.ipv4_addrs[i].addr_s, REG_INPORT_ETH_ADDR, match, false, 90, - &op->nbrp->header_, lflows); + &op->nbrp->header_, lflows, lflow_ref); } if (!op->od->is_gw_router && !op->od->n_l3dgw_ports) { @@ -13967,7 +14251,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13983,7 +14268,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -13999,7 +14285,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_TCP_RESET, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); ds_clear(match); ds_put_format(match, @@ -14018,7 +14305,8 @@ build_lrouter_ipv4_ip_input(struct ovn_port *op, COPP_ICMP4_ERR, op->od->nbr->copp, meter_groups), - &op->nbrp->header_); + &op->nbrp->header_, + lflow_ref); } } } @@ -14028,7 +14316,8 @@ static void build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, struct lflow_table *lflows, const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct ds *match, const struct shash *meter_groups) + struct ds *match, const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); /* No ingress packets are accepted on a chassisredirect @@ -14049,7 +14338,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, AF_INET); build_lrouter_arp_flow(op->od, op, lb_ips_v4_as, REG_INPORT_ETH_ADDR, - match, false, 90, NULL, lflows); + match, false, 90, NULL, lflows, lflow_ref); free(lb_ips_v4_as); } @@ -14066,7 +14355,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, AF_INET6); build_lrouter_nd_flow(op->od, op, "nd_na", lb_ips_v6_as, NULL, REG_INPORT_ETH_ADDR, match, false, 90, - NULL, lflows, meter_groups); + NULL, lflows, meter_groups, lflow_ref); free(lb_ips_v6_as); } @@ -14098,7 +14387,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, continue; } build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } /* Now handle SNAT entries too, one per unique SNAT IP. */ @@ -14114,7 +14403,7 @@ build_lrouter_ipv4_ip_input_for_lbnats(struct ovn_port *op, CONTAINER_OF(ovs_list_front(&snat_ip->snat_entries), struct ovn_nat, ext_addr_list_node); build_lrouter_port_nat_arp_nd_flow(op, nat_entry, lflows, - meter_groups); + meter_groups, lflow_ref); } } @@ -14150,7 +14439,8 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14163,7 +14453,7 @@ build_lrouter_in_unsnat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14171,7 +14461,8 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14189,7 +14480,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ds_put_cstr(match, " && flags.loopback == 0"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat_in_czone;", - &nat->header_); + &nat->header_, lflow_ref); ds_truncate(match, common_match_len); /* Update common zone match for the hairpin traffic. */ @@ -14197,7 +14488,7 @@ build_lrouter_in_unsnat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14205,7 +14496,8 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { @@ -14219,7 +14511,7 @@ build_lrouter_in_unsnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_UNSNAT, priority, ds_cstr(match), "ct_snat;", - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14229,7 +14521,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Ingress DNAT table: Packets enter the pipeline with destination * IP address that needs to be DNATted from a external IP address @@ -14275,7 +14568,8 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, true, cidr_bits); + is_v6, true, cidr_bits, + lflow_ref); } if (stateless) { @@ -14291,7 +14585,7 @@ build_lrouter_in_dnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_DNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14300,7 +14594,8 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, bool is_v6, - struct ovn_port *l3dgw_port, bool stateless) + struct ovn_port *l3dgw_port, bool stateless, + struct lflow_ref *lflow_ref) { /* Egress UNDNAT table: It is for already established connections' * reverse traffic. i.e., DNAT has already been done in ingress @@ -14342,7 +14637,7 @@ build_lrouter_out_undnat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_UNDNAT, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14350,7 +14645,8 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { /* Note that this only applies for NAT on a distributed router. */ @@ -14373,7 +14669,7 @@ build_lrouter_out_is_dnat_local(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } static void @@ -14381,7 +14677,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, const struct ovn_datapath *od, const struct nbrec_nat *nat, struct ds *match, bool distributed_nat, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { ds_clear(match); @@ -14401,7 +14698,8 @@ build_lrouter_out_snat_match(struct lflow_table *lflows, if (nat->allowed_ext_ips || nat->exempted_ext_ips) { lrouter_nat_add_ext_ip_match(od, lflows, match, nat, - is_v6, false, cidr_bits); + is_v6, false, cidr_bits, + lflow_ref); } } @@ -14412,7 +14710,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14426,7 +14725,7 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14445,7 +14744,8 @@ build_lrouter_out_snat_stateless_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14455,7 +14755,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, - bool is_v6, struct ovn_port *l3dgw_port) + bool is_v6, struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14470,7 +14771,8 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, struct ds zone_actions = DS_EMPTY_INITIALIZER; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); if (od->n_l3dgw_ports) { priority += 128; @@ -14499,13 +14801,15 @@ build_lrouter_out_snat_in_czone_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); ds_put_cstr(match, " && "REGBIT_DST_NAT_IP_LOCAL" == 1"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority + 1, ds_cstr(match), - ds_cstr(&zone_actions), &nat->header_); + ds_cstr(&zone_actions), &nat->header_, + lflow_ref); ds_destroy(&zone_actions); } @@ -14516,7 +14820,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, const struct nbrec_nat *nat, struct ds *match, struct ds *actions, bool distributed_nat, struct eth_addr mac, int cidr_bits, bool is_v6, - struct ovn_port *l3dgw_port) + struct ovn_port *l3dgw_port, + struct lflow_ref *lflow_ref) { if (strcmp(nat->type, "snat") && strcmp(nat->type, "dnat_and_snat")) { return; @@ -14530,7 +14835,7 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, uint16_t priority = cidr_bits + 1; build_lrouter_out_snat_match(lflows, od, nat, match, distributed_nat, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, lflow_ref); if (!od->is_gw_router) { /* Distributed router. */ @@ -14553,7 +14858,8 @@ build_lrouter_out_snat_flow(struct lflow_table *lflows, ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_SNAT, priority, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } static void @@ -14563,7 +14869,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, bool is_v6, struct ds *match, struct ds *actions, int mtu, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { ds_clear(match); ds_put_format(match, "inport == %s && "REGBIT_PKT_LARGER @@ -14597,7 +14904,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP4_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } else { ds_put_format(match, " && ip6 && ip6.dst == %s", nat->external_ip); /* Set icmp6.frag_mtu to gw_mtu */ @@ -14624,7 +14932,8 @@ build_lrouter_ingress_nat_check_pkt_len(struct lflow_table *lflows, COPP_ICMP6_ERR, od->nbr->copp, meter_groups), - &nat->header_); + &nat->header_, + lflow_ref); } } @@ -14635,7 +14944,8 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, struct ds *actions, struct eth_addr mac, bool distributed_nat, bool is_v6, struct ovn_port *l3dgw_port, - const struct shash *meter_groups) + const struct shash *meter_groups, + struct lflow_ref *lflow_ref) { if (od->n_l3dgw_ports && !strcmp(nat->type, "snat")) { ds_clear(match); @@ -14645,7 +14955,7 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, is_v6 ? "ip6.src" : "ip4.src", nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_IP_INPUT, 120, ds_cstr(match), "next;", - &nat->header_); + &nat->header_, lflow_ref); } /* Logical router ingress table 0: * For NAT on a distributed router, add rules allowing @@ -14669,12 +14979,14 @@ build_lrouter_ingress_flow(struct lflow_table *lflows, build_gateway_mtu_flow(lflows, l3dgw_port, S_ROUTER_IN_ADMISSION, 50, 55, match, actions, &nat->header_, + lflow_ref, REG_INPORT_ETH_ADDR " = %s; next;", l3dgw_port->lrp_networks.ea_s); if (gw_mtu) { build_lrouter_ingress_nat_check_pkt_len(lflows, nat, od, is_v6, match, actions, gw_mtu, - l3dgw_port, meter_groups); + l3dgw_port, meter_groups, + lflow_ref); } } } @@ -14809,27 +15121,33 @@ lrouter_check_nat_entry(const struct ovn_datapath *od, /* NAT, Defrag and load balancing. */ static void build_lr_nat_defrag_and_lb_default_flows(struct ovn_datapath *od, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(od->nbr); /* Packets are allowed by default. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DEFRAG, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 0, "1", "next;", lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 0, "1", - REGBIT_DST_NAT_IP_LOCAL" = 0; next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;"); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;"); + REGBIT_DST_NAT_IP_LOCAL" = 0; next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 0, "1", "next;", lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_SNAT, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_EGR_LOOP, 0, "1", "next;", + lflow_ref); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ECMP_STATEFUL, 0, "1", "next;", + lflow_ref); /* Send the IPv6 NS packets to next table. When ovn-controller * generates IPv6 NS (for the action - nd_ns{}), the injected * packet would go through conntrack - which is not required. */ - ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;"); + ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 120, "nd_ns", "next;", + lflow_ref); } static void @@ -14839,7 +15157,8 @@ build_lrouter_nat_defrag_and_lb( const struct hmap *ls_ports, const struct hmap *lr_ports, struct ds *match, struct ds *actions, const struct shash *meter_groups, - const struct chassis_features *features) + const struct chassis_features *features, + struct lflow_ref *lflow_ref) { const struct ovn_datapath *od = lr_lbnat_rec->od; ovs_assert(od->nbr); @@ -14864,16 +15183,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; ct_commit_nat;"); + "flags.skip_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; ct_commit_nat;"); + "flags.force_snat_for_lb = 1; ct_commit_nat;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "ct_commit_nat;"); + "ct_commit_nat;", lflow_ref); } /* Ingress DNAT (Priority 50/70). @@ -14890,16 +15211,18 @@ build_lrouter_nat_defrag_and_lb( ds_put_format(match, " && %s.skip_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.skip_snat_for_lb = 1; next;"); + "flags.skip_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ds_put_format(match, " && %s.force_snat == 1", ct_flag_reg); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 70, ds_cstr(match), - "flags.force_snat_for_lb = 1; next;"); + "flags.force_snat_for_lb = 1; next;", + lflow_ref); ds_truncate(match, match_len); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, ds_cstr(match), - "next;"); + "next;", lflow_ref); } /* If the router has load balancer or DNAT rules, re-circulate every packet @@ -14914,11 +15237,14 @@ build_lrouter_nat_defrag_and_lb( if (od->is_gw_router && (od->nbr->n_nat || lr_lbnat_rec->has_lb_vip)) { /* Do not send ND or ICMP packets to connection tracking. */ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 100, - "nd || nd_rs || nd_ra", "next;"); + "nd || nd_rs || nd_ra", "next;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 50, - "ip", "flags.loopback = 1; ct_dnat;"); + "ip", "flags.loopback = 1; ct_dnat;", + lflow_ref); ovn_lflow_add(lflows, od, S_ROUTER_OUT_POST_UNDNAT, 50, - "ip && ct.new", "ct_commit { } ; next; "); + "ip && ct.new", "ct_commit { } ; next; ", + lflow_ref); } /* NAT rules are only valid on Gateway routers and routers with @@ -14967,19 +15293,20 @@ build_lrouter_nat_defrag_and_lb( if (stateless) { build_lrouter_in_unsnat_stateless_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_in_unsnat_in_czone_flow(lflows, od, nat, match, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else { build_lrouter_in_unsnat_flow(lflows, od, nat, match, - distributed_nat, is_v6, l3dgw_port); + distributed_nat, is_v6, l3dgw_port, + lflow_ref); } /* S_ROUTER_IN_DNAT */ build_lrouter_in_dnat_flow(lflows, od, lrnat_rec, nat, match, actions, distributed_nat, cidr_bits, is_v6, - l3dgw_port, stateless); + l3dgw_port, stateless, lflow_ref); /* ARP resolve for NAT IPs. */ if (od->is_gw_router) { @@ -15002,7 +15329,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 150, ds_cstr(match), debug_drop_action(), - &nat->header_); + &nat->header_, + lflow_ref); /* Now for packets coming from other (downlink) LRPs, allow ARP * resolve for the NAT IP, so that such packets can be * forwarded for E/W NAT. */ @@ -15021,7 +15349,8 @@ build_lrouter_nat_defrag_and_lb( S_ROUTER_IN_ARP_RESOLVE, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); if (od->redirect_bridged && distributed_nat) { ds_clear(match); ds_put_format( @@ -15042,7 +15371,8 @@ build_lrouter_nat_defrag_and_lb( ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 90, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, + lflow_ref); } sset_add(&nat_entries, nat->external_ip); } @@ -15052,13 +15382,13 @@ build_lrouter_nat_defrag_and_lb( /* S_ROUTER_OUT_DNAT_LOCAL */ build_lrouter_out_is_dnat_local(lflows, od, nat, match, actions, distributed_nat, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_OUT_UNDNAT */ build_lrouter_out_undnat_flow(lflows, od, nat, match, actions, distributed_nat, mac, is_v6, l3dgw_port, - stateless); + stateless, lflow_ref); /* S_ROUTER_OUT_SNAT * Egress SNAT table: Packets enter the egress pipeline with * source ip address that needs to be SNATted to a external ip @@ -15067,21 +15397,22 @@ build_lrouter_nat_defrag_and_lb( build_lrouter_out_snat_stateless_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } else if (lrouter_use_common_zone(od)) { build_lrouter_out_snat_in_czone_flow(lflows, od, nat, match, actions, distributed_nat, mac, - cidr_bits, is_v6, l3dgw_port); + cidr_bits, is_v6, l3dgw_port, + lflow_ref); } else { build_lrouter_out_snat_flow(lflows, od, nat, match, actions, distributed_nat, mac, cidr_bits, is_v6, - l3dgw_port); + l3dgw_port, lflow_ref); } /* S_ROUTER_IN_ADMISSION - S_ROUTER_IN_IP_INPUT */ build_lrouter_ingress_flow(lflows, od, nat, match, actions, mac, distributed_nat, is_v6, l3dgw_port, - meter_groups); + meter_groups, lflow_ref); /* Ingress Gateway Redirect Table: For NAT on a distributed * router, add flows that are specific to a NAT rule. These @@ -15108,7 +15439,8 @@ build_lrouter_nat_defrag_and_lb( if (op && op->nbsp && !strcmp(op->nbsp->type, "virtual")) { ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 80, ds_cstr(match), - debug_drop_action(), &nat->header_); + debug_drop_action(), &nat->header_, + lflow_ref); } ds_put_format(match, " && is_chassis_resident(\"%s\")", nat->logical_port); @@ -15118,7 +15450,8 @@ build_lrouter_nat_defrag_and_lb( nat->external_ip); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, ds_cstr(match), - ds_cstr(actions), &nat->header_); + ds_cstr(actions), &nat->header_, + lflow_ref); } /* Egress Loopback table: For NAT on a distributed router. @@ -15159,7 +15492,7 @@ build_lrouter_nat_defrag_and_lb( ovn_stage_get_table(S_ROUTER_IN_ADMISSION)); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100, ds_cstr(match), ds_cstr(actions), - &nat->header_); + &nat->header_, lflow_ref); } } @@ -15179,7 +15512,7 @@ build_lrouter_nat_defrag_and_lb( ds_put_cstr(actions, REGBIT_DST_NAT_IP_LOCAL" = 1; next;"); ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_CHECK_DNAT_LOCAL, 50, ds_cstr(match), ds_cstr(actions), - &od->nbr->header_); + &od->nbr->header_, lflow_ref); } @@ -15189,22 +15522,24 @@ build_lrouter_nat_defrag_and_lb( if (lrnat_rec->dnat_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", lrnat_rec->dnat_force_snat_addrs.ipv4_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } if (lrnat_rec->dnat_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", lrnat_rec->dnat_force_snat_addrs.ipv6_addrs[0].addr_s, - "dnat"); + "dnat", lflow_ref); } } if (lb_force_snat_ip) { if (lrnat_rec->lb_force_snat_addrs.n_ipv4_addrs) { build_lrouter_force_snat_flows(lflows, od, "4", - lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv4_addrs[0].addr_s, "lb", + lflow_ref); } if (lrnat_rec->lb_force_snat_addrs.n_ipv6_addrs) { build_lrouter_force_snat_flows(lflows, od, "6", - lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb"); + lrnat_rec->lb_force_snat_addrs.ipv6_addrs[0].addr_s, "lb", + lflow_ref); } } } @@ -15242,7 +15577,8 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct hmap *lr_ports, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbsp); @@ -15256,7 +15592,7 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, lr_lbnats, lr_ports, lflows, - match, actions, op->lbnat_lflow_ref); + match, actions, lflow_ref); } static void @@ -15264,7 +15600,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, const struct lr_lb_nat_data_record *lr_lbnat_rec, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { /* Drop IP traffic destined to router owned IPs except if the IP is * also a SNAT IP. Those are dropped later, in stage @@ -15277,7 +15614,8 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, */ if (!lr_lbnat_rec->lrnat_rec->lb_force_snat_router_ip) { build_lrouter_drop_own_dest(op, lr_lbnat_rec, - S_ROUTER_IN_IP_INPUT, 60, false, lflows); + S_ROUTER_IN_IP_INPUT, 60, false, + lflows, lflow_ref); } /* Drop IP traffic destined to router owned IPs. Part of it is dropped @@ -15287,12 +15625,14 @@ build_lrp_lflows_for_lbnats(struct ovn_port *op, * Priority 2. */ build_lrouter_drop_own_dest(op, lr_lbnat_rec, - S_ROUTER_IN_ARP_RESOLVE, 2, true, lflows); + S_ROUTER_IN_ARP_RESOLVE, 2, true, + lflows, lflow_ref); build_lrouter_ipv4_ip_input_for_lbnats(op, lflows, lr_lbnat_rec, - match, meter_groups); + match, meter_groups, + lflow_ref); build_lrouter_force_snat_flows_op(op, lr_lbnat_rec->lrnat_rec, lflows, - match, actions); + match, actions, lflow_ref); } static void @@ -15301,7 +15641,8 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, const struct shash *meter_groups, struct ds *match, struct ds *actions, - struct lflow_table *lflows) + struct lflow_table *lflows, + struct lflow_ref *lflow_ref) { ovs_assert(op->nbrp); @@ -15310,7 +15651,7 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, ovs_assert(lr_lbnat_rec); build_lrp_lflows_for_lbnats(op, lr_lbnat_rec, meter_groups, match, - actions, lflows); + actions, lflows, lflow_ref); } static void @@ -15324,13 +15665,16 @@ build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, const struct chassis_features *features) { build_lrouter_nat_defrag_and_lb(lr_lbnat_rec, lflows, ls_ports, lr_ports, - match, actions, meter_groups, features); + match, actions, meter_groups, features, + NULL); build_lr_gateway_redirect_flows_for_nats(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, - match, actions); + match, actions, + NULL); build_lrouter_arp_nd_for_datapath(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, - meter_groups); + meter_groups, + NULL); } static void @@ -15342,11 +15686,14 @@ build_ls_lbacls_flows(const struct ls_lbacls_record *ls_lbacls_rec, { ovs_assert(ls_lbacls_rec->od); - build_ls_lbacls_rec_pre_acls(ls_lbacls_rec, ls_pgs, lflows); - build_ls_lbacls_rec_pre_lb(ls_lbacls_rec, lflows); - build_acl_hints(ls_lbacls_rec, features, lflows); - build_acls(ls_lbacls_rec, features, lflows, ls_pgs, meter_groups); - build_lb_hairpin(ls_lbacls_rec, lflows); + build_ls_lbacls_rec_pre_acls(ls_lbacls_rec, ls_pgs, lflows, + NULL); + build_ls_lbacls_rec_pre_lb(ls_lbacls_rec, lflows, + NULL); + build_acl_hints(ls_lbacls_rec, features, lflows, NULL); + build_acls(ls_lbacls_rec, features, lflows, ls_pgs, meter_groups, + NULL); + build_lb_hairpin(ls_lbacls_rec, lflows, NULL); } struct lswitch_flow_build_info { @@ -15382,19 +15729,20 @@ build_lswitch_and_lrouter_iterate_by_ls(struct ovn_datapath *od, { ovs_assert(od->nbs); build_lswitch_lflows_pre_acl_and_acl(od, lsi->features, lsi->lflows, - lsi->meter_groups); - - build_fwd_group_lflows(od, lsi->lflows); - build_lswitch_lflows_admission_control(od, lsi->lflows); - build_lswitch_learn_fdb_od(od, lsi->lflows); - build_lswitch_arp_nd_responder_default(od, lsi->lflows); - build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups); - build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows); + lsi->meter_groups, NULL); + + build_fwd_group_lflows(od, lsi->lflows, NULL); + build_lswitch_lflows_admission_control(od, lsi->lflows, NULL); + build_lswitch_learn_fdb_od(od, lsi->lflows, NULL); + build_lswitch_arp_nd_responder_default(od, lsi->lflows, NULL); + build_lswitch_dns_lookup_and_response(od, lsi->lflows, lsi->meter_groups, + NULL); + build_lswitch_dhcp_and_dns_defaults(od, lsi->lflows, NULL); build_lswitch_destination_lookup_bmcast(od, lsi->lflows, &lsi->actions, - lsi->meter_groups); - build_lswitch_output_port_sec_od(od, lsi->lflows); - build_lswitch_lb_affinity_default_flows(od, lsi->lflows); - build_lswitch_lflows_l2_unknown(od, lsi->lflows); + lsi->meter_groups, NULL); + build_lswitch_output_port_sec_od(od, lsi->lflows, NULL); + build_lswitch_lb_affinity_default_flows(od, lsi->lflows, NULL); + build_lswitch_lflows_l2_unknown(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by @@ -15405,29 +15753,34 @@ build_lswitch_and_lrouter_iterate_by_lr(struct ovn_datapath *od, struct lswitch_flow_build_info *lsi) { ovs_assert(od->nbr); - build_adm_ctrl_flows_for_lrouter(od, lsi->lflows); + build_adm_ctrl_flows_for_lrouter(od, lsi->lflows, NULL); build_neigh_learning_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_ND_RA_flows_for_lrouter(od, lsi->lflows); - build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, NULL); + build_ND_RA_flows_for_lrouter(od, lsi->lflows, NULL); + build_ip_routing_pre_flows_for_lrouter(od, lsi->lflows, NULL); build_static_route_flows_for_lrouter(od, lsi->features, lsi->lflows, lsi->lr_ports, - lsi->bfd_connections); + lsi->bfd_connections, + NULL); build_mcast_lookup_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); - build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports); - build_arp_resolve_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, NULL); + build_ingress_policy_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, + NULL); + build_arp_resolve_flows_for_lrouter(od, lsi->lflows, NULL); build_check_pkt_len_flows_for_lrouter(od, lsi->lflows, lsi->lr_ports, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, NULL); build_gateway_redirect_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, NULL); build_arp_request_flows_for_lrouter(od, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); - build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows); + &lsi->actions, + lsi->meter_groups, + NULL); + build_misc_local_traffic_drop_flows_for_lrouter(od, lsi->lflows, NULL); - build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows); - build_lrouter_lb_affinity_default_flows(od, lsi->lflows); + build_lr_nat_defrag_and_lb_default_flows(od, lsi->lflows, NULL); + build_lrouter_lb_affinity_default_flows(od, lsi->lflows, NULL); } /* Helper function to combine all lflow generation which is iterated by logical @@ -15469,22 +15822,26 @@ build_lswitch_and_lrouter_iterate_by_lrp(struct ovn_port *op, ovs_assert(op->nbrp); build_adm_ctrl_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, op->lflow_ref); build_neigh_learning_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_ip_routing_flows_for_lrp(op, lsi->lflows); + &lsi->actions, op->lflow_ref); + build_ip_routing_flows_for_lrp(op, lsi->lflows, op->lflow_ref); build_ND_RA_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions, lsi->meter_groups); + &lsi->actions, lsi->meter_groups, + op->lflow_ref); build_arp_resolve_flows_for_lrp(op, lsi->lflows, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, op->lflow_ref); build_egress_delivery_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, - &lsi->actions); - build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match); + &lsi->actions, + op->lflow_ref); + build_dhcpv6_reply_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, + op->lflow_ref); build_ipv6_input_flows_for_lrouter_port(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, + op->lflow_ref); build_lrouter_ipv4_ip_input(op, lsi->lflows, &lsi->match, &lsi->actions, - lsi->meter_groups); + lsi->meter_groups, op->lflow_ref); } static void * @@ -15552,7 +15909,8 @@ build_lflows_thread(void *arg) lsi->lr_ports, &lsi->match, &lsi->actions, - lsi->lflows); + lsi->lflows, + op->lbnat_lflow_ref); } } for (bnum = control->id; @@ -15569,7 +15927,8 @@ build_lflows_thread(void *arg) lsi->meter_groups, &lsi->match, &lsi->actions, - lsi->lflows); + lsi->lflows, + op->lbnat_lflow_ref); } } for (bnum = control->id; @@ -15585,23 +15944,27 @@ build_lflows_thread(void *arg) lsi->ls_ports, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, + NULL); build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, - &lsi->match); + &lsi->match, + NULL); build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, lsi->lr_lbnats, lsi->features, lsi->svc_monitor_map, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, + NULL); build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, lsi->svc_monitor_map, - &lsi->match, &lsi->actions); + &lsi->match, &lsi->actions, + NULL); } } for (bnum = control->id; @@ -15645,7 +16008,7 @@ build_lflows_thread(void *arg) } build_lswitch_ip_mcast_igmp_mld(igmp_group, lsi->lflows, &lsi->match, - &lsi->actions); + &lsi->actions, NULL); } } } @@ -15798,7 +16161,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, lsi.lr_ports, &lsi.match, &lsi.actions, - lsi.lflows); + lsi.lflows, + op->lbnat_lflow_ref); } HMAP_FOR_EACH (op, key_node, lr_ports) { build_lswitch_and_lrouter_iterate_by_lrp(op, &lsi); @@ -15806,24 +16170,29 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsi.meter_groups, &lsi.match, &lsi.actions, - lsi.lflows); + lsi.lflows, + op->lbnat_lflow_ref); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, lsi.lflows, &lsi.actions, - &lsi.match); + &lsi.match, + NULL); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, - lsi.lr_datapaths, &lsi.match); + lsi.lr_datapaths, &lsi.match, + NULL); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.lr_lbnats, lsi.features, lsi.svc_monitor_map, - &lsi.match, &lsi.actions); + &lsi.match, &lsi.actions, + NULL); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, lsi.svc_monitor_map, - &lsi.match, &lsi.actions); + &lsi.match, &lsi.actions, + NULL); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -15845,7 +16214,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_ip_mcast_igmp_mld(igmp_group, lsi.lflows, &lsi.actions, - &lsi.match); + &lsi.match, + NULL); } stopwatch_stop(LFLOWS_IGMP_STOPWATCH_NAME, time_msec()); From patchwork Tue Oct 24 00:49:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854085 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtkw0Y4Bz202k for ; Tue, 24 Oct 2023 11:50:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 31942830D7; Tue, 24 Oct 2023 00:50:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 31942830D7 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1NgBZ7uD1o_z; Tue, 24 Oct 2023 00:50:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 55E90821E2; Tue, 24 Oct 2023 00:50:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 55E90821E2 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8E5F9C0DF5; Tue, 24 Oct 2023 00:50:03 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 398E5C0DF5 for ; Tue, 24 Oct 2023 00:50:01 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 013FD42CE8 for ; Tue, 24 Oct 2023 00:49:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 013FD42CE8 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t6P56_VD0v-q for ; Tue, 24 Oct 2023 00:49:36 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::229]) by smtp4.osuosl.org (Postfix) with ESMTPS id 3EBE142CE9 for ; Tue, 24 Oct 2023 00:49:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 3EBE142CE9 Received: by mail.gandi.net (Postfix) with ESMTPSA id 18CA7FF803; Tue, 24 Oct 2023 00:49:31 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:49:19 -0400 Message-ID: <20231024004919.4133952-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 12/18] northd: Move ovn_lb_datapaths from lib to northd module. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- lib/lb.c | 96 ----------------------------------------- lib/lb.h | 57 ------------------------- northd/northd.c | 111 ++++++++++++++++++++++++++++++++++++++++++++++++ northd/northd.h | 28 ++++++++++++ 4 files changed, 139 insertions(+), 153 deletions(-) diff --git a/lib/lb.c b/lib/lb.c index d0d562b6fb..991f20299d 100644 --- a/lib/lb.c +++ b/lib/lb.c @@ -1071,99 +1071,3 @@ remove_ips_from_lb_ip_set(struct ovn_lb_ip_set *lb_ips, } } } - -/* lb datapaths functions */ -struct ovn_lb_datapaths * -ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, - size_t n_lr_datapaths) -{ - struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); - lb_dps->lb = lb; - lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); - lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); - - return lb_dps; -} - -struct ovn_lb_datapaths * -ovn_lb_datapaths_find(const struct hmap *lb_dps_map, - const struct uuid *lb_uuid) -{ - struct ovn_lb_datapaths *lb_dps; - size_t hash = uuid_hash(lb_uuid); - HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { - if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { - return lb_dps; - } - } - return NULL; -} - -void -ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) -{ - bitmap_free(lb_dps->nb_lr_map); - bitmap_free(lb_dps->nb_ls_map); - free(lb_dps); -} - -void -ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); - lb_dps->n_nb_lr++; - } - } -} - -void -ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, - struct ovn_datapath **ods) -{ - for (size_t i = 0; i < n; i++) { - if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { - bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); - lb_dps->n_nb_ls++; - } - } -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, - size_t max_ls_datapaths, - size_t max_lr_datapaths) -{ - struct ovn_lb_group_datapaths *lb_group_dps = - xzalloc(sizeof *lb_group_dps); - lb_group_dps->lb_group = lb_group; - lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); - lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); - - return lb_group_dps; -} - -void -ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) -{ - free(lb_group_dps->ls); - free(lb_group_dps->lr); - free(lb_group_dps); -} - -struct ovn_lb_group_datapaths * -ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, - const struct uuid *lb_group_uuid) -{ - struct ovn_lb_group_datapaths *lb_group_dps; - size_t hash = uuid_hash(lb_group_uuid); - - HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { - if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { - return lb_group_dps; - } - } - return NULL; -} diff --git a/lib/lb.h b/lib/lb.h index b8e3c1e8fb..90ac39ee5a 100644 --- a/lib/lb.h +++ b/lib/lb.h @@ -167,63 +167,6 @@ void ovn_lb_group_reinit( const struct nbrec_load_balancer_group *, const struct hmap *lbs); -struct ovn_lb_datapaths { - struct hmap_node hmap_node; - - const struct ovn_northd_lb *lb; - size_t n_nb_ls; - unsigned long *nb_ls_map; - - size_t n_nb_lr; - unsigned long *nb_lr_map; -}; - -struct ovn_lb_datapaths *ovn_lb_datapaths_create(const struct ovn_northd_lb *, - size_t n_ls_datapaths, - size_t n_lr_datapaths); -struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, - const struct uuid *); -void ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *); -void ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); -void ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *, size_t n, - struct ovn_datapath **); - -struct ovn_lb_group_datapaths { - struct hmap_node hmap_node; - - const struct ovn_lb_group *lb_group; - - /* Datapaths to which 'lb_group' is applied. */ - size_t n_ls; - struct ovn_datapath **ls; - size_t n_lr; - struct ovn_datapath **lr; -}; - -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_create( - const struct ovn_lb_group *, size_t max_ls_datapaths, - size_t max_lr_datapaths); - -void ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *); -struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( - const struct hmap *lb_group_dps, const struct uuid *); - -static inline void -ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, - struct ovn_datapath **ods) -{ - memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); - lbg_dps->n_ls += n; -} - -static inline void -ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, - struct ovn_datapath *lr) -{ - lbg_dps->lr[lbg_dps->n_lr++] = lr; -} - struct ovn_controller_lb { struct hmap_node hmap_node; diff --git a/northd/northd.c b/northd/northd.c index 36bc5e71bc..7f36b7fb71 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3522,6 +3522,117 @@ build_lb_vip_actions(const struct ovn_northd_lb *lb, return reject; } +/* lb datapaths functions */ +static struct ovn_lb_datapaths * +ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, + size_t n_lr_datapaths) +{ + struct ovn_lb_datapaths *lb_dps = xzalloc(sizeof *lb_dps); + lb_dps->lb = lb; + lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); + lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + + return lb_dps; +} + +static void +ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) +{ + bitmap_free(lb_dps->nb_lr_map); + bitmap_free(lb_dps->nb_ls_map); + free(lb_dps); +} + +static void +ovn_lb_datapaths_add_lr(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_lr_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_lr_map, ods[i]->index); + lb_dps->n_nb_lr++; + } + } +} + +static void +ovn_lb_datapaths_add_ls(struct ovn_lb_datapaths *lb_dps, size_t n, + struct ovn_datapath **ods) +{ + for (size_t i = 0; i < n; i++) { + if (!bitmap_is_set(lb_dps->nb_ls_map, ods[i]->index)) { + bitmap_set1(lb_dps->nb_ls_map, ods[i]->index); + lb_dps->n_nb_ls++; + } + } +} + +static struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_create(const struct ovn_lb_group *lb_group, + size_t max_ls_datapaths, + size_t max_lr_datapaths) +{ + struct ovn_lb_group_datapaths *lb_group_dps = + xzalloc(sizeof *lb_group_dps); + lb_group_dps->lb_group = lb_group; + lb_group_dps->ls = xmalloc(max_ls_datapaths * sizeof *lb_group_dps->ls); + lb_group_dps->lr = xmalloc(max_lr_datapaths * sizeof *lb_group_dps->lr); + + return lb_group_dps; +} + +static void +ovn_lb_group_datapaths_destroy(struct ovn_lb_group_datapaths *lb_group_dps) +{ + free(lb_group_dps->ls); + free(lb_group_dps->lr); + free(lb_group_dps); +} + +static void +ovn_lb_group_datapaths_add_ls(struct ovn_lb_group_datapaths *lbg_dps, size_t n, + struct ovn_datapath **ods) +{ + memcpy(&lbg_dps->ls[lbg_dps->n_ls], ods, n * sizeof *ods); + lbg_dps->n_ls += n; +} + +static void +ovn_lb_group_datapaths_add_lr(struct ovn_lb_group_datapaths *lbg_dps, + struct ovn_datapath *lr) +{ + lbg_dps->lr[lbg_dps->n_lr++] = lr; +} + +struct ovn_lb_datapaths * +ovn_lb_datapaths_find(const struct hmap *lb_dps_map, + const struct uuid *lb_uuid) +{ + struct ovn_lb_datapaths *lb_dps; + size_t hash = uuid_hash(lb_uuid); + HMAP_FOR_EACH_WITH_HASH (lb_dps, hmap_node, hash, lb_dps_map) { + if (uuid_equals(&lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return lb_dps; + } + } + return NULL; +} + +struct ovn_lb_group_datapaths * +ovn_lb_group_datapaths_find(const struct hmap *lb_group_dps_map, + const struct uuid *lb_group_uuid) +{ + struct ovn_lb_group_datapaths *lb_group_dps; + size_t hash = uuid_hash(lb_group_uuid); + + HMAP_FOR_EACH_WITH_HASH (lb_group_dps, hmap_node, hash, lb_group_dps_map) { + if (uuid_equals(&lb_group_dps->lb_group->uuid, lb_group_uuid)) { + return lb_group_dps; + } + } + return NULL; +} + static void build_lb_datapaths(const struct hmap *lbs, const struct hmap *lb_groups, struct ovn_datapaths *ls_datapaths, diff --git a/northd/northd.h b/northd/northd.h index 38aa4e4a53..18225cfd07 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -84,6 +84,29 @@ struct ovn_datapaths { struct ovn_datapath **array; }; +struct ovn_lb_datapaths { + struct hmap_node hmap_node; + + const struct ovn_northd_lb *lb; + size_t n_nb_ls; + unsigned long *nb_ls_map; + + size_t n_nb_lr; + unsigned long *nb_lr_map; +}; + +struct ovn_lb_group_datapaths { + struct hmap_node hmap_node; + + const struct ovn_lb_group *lb_group; + + /* Datapaths to which 'lb_group' is applied. */ + size_t n_ls; + struct ovn_datapath **ls; + size_t n_lr; + struct ovn_datapath **lr; +}; + struct tracked_ovn_ports { /* tracked created ports. * hmapx node data is 'struct ovn_port *' */ @@ -98,6 +121,11 @@ struct tracked_ovn_ports { struct hmapx deleted; }; +struct ovn_lb_datapaths *ovn_lb_datapaths_find(const struct hmap *, + const struct uuid *); +struct ovn_lb_group_datapaths *ovn_lb_group_datapaths_find( + const struct hmap *lb_group_dps, const struct uuid *); + struct tracked_lbs { /* Tracked created or updated load balancers. * hmapx node data is 'struct ovn_lb_datapaths' */ From patchwork Tue Oct 24 00:49:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854086 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtkw4WW4z23jn for ; Tue, 24 Oct 2023 11:50:16 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7D5F561507; Tue, 24 Oct 2023 00:50:14 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7D5F561507 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 04QDsi7_7hGj; Tue, 24 Oct 2023 00:50:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 9C562614E2; Tue, 24 Oct 2023 00:50:07 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9C562614E2 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E2AF5C0E01; Tue, 24 Oct 2023 00:50:06 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 193E8C0E0E for ; Tue, 24 Oct 2023 00:50:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E770081258 for ; Tue, 24 Oct 2023 00:49:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E770081258 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GQAvecYScdt5 for ; Tue, 24 Oct 2023 00:49:49 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5CB4680F89 for ; Tue, 24 Oct 2023 00:49:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5CB4680F89 Received: by mail.gandi.net (Postfix) with ESMTPSA id C51FEFF803; Tue, 24 Oct 2023 00:49:46 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:49:33 -0400 Message-ID: <20231024004933.4133972-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 13/18] northd: Handle lb changes in lflow engine. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Since northd tracked data has the changed lb data, northd engine handler for lflow engine now handles the lb changes incrementally. All the lflows generated for each lb is stored in the ovn_lb_datapaths->lflow_ref and this is used similar to how we handle ovn_port changes. Signed-off-by: Numan Siddique --- northd/en-lflow.c | 11 +++--- northd/lflow-mgr.c | 49 ++++++++++++++++++++++++ northd/lflow-mgr.h | 4 ++ northd/northd.c | 91 +++++++++++++++++++++++++++++++++++++++++---- northd/northd.h | 28 ++++++++++++++ tests/ovn-northd.at | 31 ++++++++++----- 6 files changed, 192 insertions(+), 22 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index efffee6384..4400460482 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -122,11 +122,6 @@ lflow_northd_handler(struct engine_node *node, return false; } - /* Fall back to recompute if load balancers have changed. */ - if (northd_has_lbs_in_tracked_data(&northd_data->trk_northd_changes)) { - return false; - } - const struct engine_context *eng_ctx = engine_get_context(); struct lflow_data *lflow_data = data; @@ -139,6 +134,12 @@ lflow_northd_handler(struct engine_node *node, return false; } + if (!lflow_handle_northd_lb_changes(eng_ctx->ovnsb_idl_txn, + &northd_data->trk_northd_changes.trk_lbs, + &lflow_input, lflow_data->lflow_table)) { + return false; + } + engine_set_node_state(node, EN_UPDATED); return true; diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index b435d03061..30d123a7bc 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -111,6 +111,10 @@ static void unlink_objres_lflows(struct resource_to_objects_node *, const struct ovn_datapath *od, struct lflow_table *, struct objdep_mgr *); +static void unlink_all_dps_objres_lflows(struct resource_to_objects_node *, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct lflow_table *); static void sync_lflows_from_objres( struct resource_to_objects_node *, struct lflow_table *, struct objdep_mgr *, struct ovsdb_idl_txn *, @@ -387,6 +391,19 @@ lflow_ref_clear_lflows(struct lflow_ref *lflow_ref, unlink_objres_lflows(res_node, od, lflow_table, &lflow_ref->objdep_mgr); } +void +lflow_ref_clear_lflows_for_all_dps(struct lflow_ref *lflow_ref, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct lflow_table *lflow_table) +{ + struct resource_to_objects_node *res_node = objdep_mgr_find_objs( + &lflow_ref->objdep_mgr, OBJDEP_TYPE_LFLOW, lflow_ref->res_name); + + unlink_all_dps_objres_lflows(res_node, n_ls_datapaths, n_lr_datapaths, + lflow_table); +} + void lflow_ref_clear_and_sync_lflows(struct lflow_ref *lflow_ref, const struct ovn_datapath *od, @@ -1099,6 +1116,38 @@ unlink_objres_lflows(struct resource_to_objects_node *res_node, } } +static void +unlink_all_dps_objres_lflows(struct resource_to_objects_node *res_node, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct lflow_table *lflow_table) +{ + if (!res_node) { + return; + } + + struct object_to_resources_list_node *resource_list_node; + RESOURCE_FOR_EACH_OBJ (resource_list_node, res_node) { + const struct uuid *obj_uuid = &resource_list_node->obj_uuid; + struct ovn_lflow *lflow = ovn_lflow_uuid_find( + &lflow_table->hash_map, obj_uuid); + if (!lflow) { + continue; + } + + size_t n_datapaths; + if (ovn_stage_to_datapath_type(lflow->stage) == DP_SWITCH) { + n_datapaths = n_ls_datapaths; + } else { + n_datapaths = n_lr_datapaths; + } + size_t index; + BITMAP_FOR_EACH_1 (index, n_datapaths, lflow->dpg_bitmap) { + bitmap_set0(lflow->dpg_bitmap, index); + } + } +} + static void sync_lflows_from_objres(struct resource_to_objects_node *res_node, struct lflow_table *lflow_table, diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 2cbc4af157..08805435c8 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -55,6 +55,10 @@ void lflow_ref_set_od(struct lflow_ref *, const struct ovn_datapath *); void lflow_ref_destroy(struct lflow_ref *); void lflow_ref_clear_lflows(struct lflow_ref *, const struct ovn_datapath *, struct lflow_table *); +void lflow_ref_clear_lflows_for_all_dps(struct lflow_ref *, + size_t n_ls_datapaths, + size_t n_lr_datapaths, + struct lflow_table *); void lflow_ref_clear_and_sync_lflows(struct lflow_ref *, const struct ovn_datapath *, struct lflow_table *lflow_table, diff --git a/northd/northd.c b/northd/northd.c index 7f36b7fb71..1033bc1a8f 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3531,6 +3531,7 @@ ovn_lb_datapaths_create(const struct ovn_northd_lb *lb, size_t n_ls_datapaths, lb_dps->lb = lb; lb_dps->nb_ls_map = bitmap_allocate(n_ls_datapaths); lb_dps->nb_lr_map = bitmap_allocate(n_lr_datapaths); + lb_dps->lflow_ref = lflow_ref_alloc(lb->nlb->name); return lb_dps; } @@ -3540,6 +3541,7 @@ ovn_lb_datapaths_destroy(struct ovn_lb_datapaths *lb_dps) { bitmap_free(lb_dps->nb_lr_map); bitmap_free(lb_dps->nb_ls_map); + lflow_ref_destroy(lb_dps->lflow_ref); free(lb_dps); } @@ -16056,11 +16058,11 @@ build_lflows_thread(void *arg) lsi->lflows, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lsi->lflows, lsi->lr_datapaths, &lsi->match, - NULL); + lb_dps->lflow_ref); build_lrouter_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->lr_datapaths, @@ -16068,14 +16070,14 @@ build_lflows_thread(void *arg) lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); build_lswitch_flows_for_lb(lb_dps, lsi->lflows, lsi->meter_groups, lsi->ls_datapaths, lsi->features, lsi->svc_monitor_map, &lsi->match, &lsi->actions, - NULL); + lb_dps->lflow_ref); } } for (bnum = control->id; @@ -16290,20 +16292,20 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, lsi.lflows, &lsi.actions, &lsi.match, - NULL); + lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lsi.lflows, lsi.lr_datapaths, &lsi.match, - NULL); + lb_dps->lflow_ref); build_lrouter_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.lr_datapaths, lsi.lr_lbnats, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions, - NULL); + lb_dps->lflow_ref); build_lswitch_flows_for_lb(lb_dps, lsi.lflows, lsi.meter_groups, lsi.ls_datapaths, lsi.features, lsi.svc_monitor_map, &lsi.match, &lsi.actions, - NULL); + lb_dps->lflow_ref); } stopwatch_stop(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16612,6 +16614,79 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *trk_lbs, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + lflow_ref_clear_lflows_for_all_dps(lb_dps->lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + lflows); + lflow_ref_sync_lflows_to_sb(lb_dps->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + /* unlink old lflows. */ + lflow_ref_clear_lflows_for_all_dps(lb_dps->lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + lflows); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lswitch_arp_nd_service_monitor(lb_dps->lb, lflow_input->ls_ports, + lflows, &actions, + &match, lb_dps->lflow_ref); + build_lrouter_defrag_flows_for_lb(lb_dps, lflows, + lflow_input->lr_datapaths, &match, + lb_dps->lflow_ref); + build_lrouter_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->lr_datapaths, + lflow_input->lr_lbnats, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions, + lb_dps->lflow_ref); + build_lswitch_flows_for_lb(lb_dps, lflows, + lflow_input->meter_groups, + lflow_input->ls_datapaths, + lflow_input->features, + lflow_input->svc_monitor_map, + &match, &actions, + lb_dps->lflow_ref); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(lb_dps->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 18225cfd07..56613f8412 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -93,6 +93,30 @@ struct ovn_lb_datapaths { size_t n_nb_lr; unsigned long *nb_lr_map; + + /* Reference of lflows generated for this load balancer. + * + * This data is initialized and destroyed by the en_northd node, but + * populated and used only by the en_lflow node. Ideally this data should + * be maintained as part of en_lflow's data (struct lflow_data): a hash + * index from ovn_port key to lflows. However, it would be less efficient + * and more complex: + * + * 1. It would require an extra search (using the index) to find the + * lflows. + * + * 2. Building the index needs to be thread-safe, using either a global + * lock which is obviously less efficient, or hash-based lock array which + * is more complex. + * + * Maintaining the lflow_ref here is more straightforward. The drawback is + * that we need to keep in mind that this data belongs to en_lflow node, + * so never access it from any other nodes. + * + * 'lflow_ref' is used to reference logical flows generated for this + * load balancer. + */ + struct lflow_ref *lflow_ref; }; struct ovn_lb_group_datapaths { @@ -687,6 +711,10 @@ bool lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_ovn_ports *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct tracked_lbs *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 8fc5cd1d60..b276c77221 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10407,9 +10407,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute - CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10417,21 +10416,26 @@ check ovn-nbctl --wait=sb set load_balancer . ip_port_mappings:10.0.0.3=sw0-p1:1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:foo=bar check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- lb-add lb2 20.0.0.10:80 20.0.0.20:80 -- lb-add lb3 30.0.0.10:80 30.0.0.20:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10441,7 +10445,7 @@ check ovn-nbctl --wait=sb -- lb-del lb2 -- lb-del lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10652,8 +10656,9 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer_group . load_Balancer @@ -10668,7 +10673,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10677,6 +10682,7 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute +check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10685,6 +10691,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute @@ -10694,6 +10701,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute +check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10791,6 +10799,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add back lb group to logical switch and then delete it. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10800,6 +10809,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear logical_switch sw0 load_balancer_group -- \ @@ -10833,14 +10843,17 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute nocompute check_engine_stats sync_to_sb_lb norecompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer_group . load_balancer="$lb2_uuid,$lb3_uuid,$lb4_uuid" check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats ls_lbacls norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid From patchwork Tue Oct 24 00:49:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtl66hjHz202k for ; Tue, 24 Oct 2023 11:50:26 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D6163614EE; Tue, 24 Oct 2023 00:50:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D6163614EE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fe1szkNNeC4K; Tue, 24 Oct 2023 00:50:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 01002614F7; Tue, 24 Oct 2023 00:50:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 01002614F7 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 922E5C0DD9; Tue, 24 Oct 2023 00:50:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 495D4C0E0D for ; Tue, 24 Oct 2023 00:50:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D6B9C614DB for ; Tue, 24 Oct 2023 00:50:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D6B9C614DB X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jLgHulbTCV1T for ; Tue, 24 Oct 2023 00:50:04 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by smtp3.osuosl.org (Postfix) with ESMTPS id 18CEC614D1 for ; Tue, 24 Oct 2023 00:50:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 18CEC614D1 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6A76420002; Tue, 24 Oct 2023 00:50:01 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:49:47 -0400 Message-ID: <20231024004947.4133992-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 14/18] northd: Add lr_lb_nat_data handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 30 ++++ northd/en-lflow.h | 1 + northd/en-lr-lb-nat-data.c | 41 ++++- northd/en-lr-lb-nat-data.h | 7 + northd/inc-proc-northd.c | 5 +- northd/northd.c | 356 ++++++++++++++++++++----------------- northd/northd.h | 7 + tests/ovn-northd.at | 48 ++--- 8 files changed, 305 insertions(+), 190 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 4400460482..613351eba2 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -162,6 +162,36 @@ lflow_port_group_handler(struct engine_node *node, void *data OVS_UNUSED) return true; } +bool +lflow_lr_lb_nat_data_handler(struct engine_node *node, void *data) +{ + struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); + + if (!lr_lb_nat_data->tracked + || lr_lb_nat_data->tracked_data.vip_nats_changed + || !hmapx_is_empty(&lr_lb_nat_data->tracked_data.deleted)) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + + struct lflow_input lflow_input; + lflow_get_input_data(node, &lflow_input); + + if (!lflow_handle_lr_lb_nat_data_changes(eng_ctx->ovnsb_idl_txn, + &lr_lb_nat_data->tracked_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index f7325c56b1..adbd8767c9 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -20,5 +20,6 @@ void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); +bool lflow_lr_lb_nat_data_handler(struct engine_node *, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-lr-lb-nat-data.c b/northd/en-lr-lb-nat-data.c index 7987799ca2..dc9a66a0a0 100644 --- a/northd/en-lr-lb-nat-data.c +++ b/northd/en-lr-lb-nat-data.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_lr_lb_nat_data); @@ -77,7 +78,7 @@ static void remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *, enum lb_neighbor_responder_mode, const struct sset *lb_ips_v4, const struct sset *lb_ips_v6); -static void lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *); +static bool lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *); const struct lr_lb_nat_data_record * lr_lb_nat_data_table_find( @@ -123,6 +124,7 @@ en_lr_lb_nat_data_clear_tracked_data(void *data_) } hmapx_clear(&data->tracked_data.crupdated); + data->tracked_data.vip_nats_changed = false; data->tracked = false; } @@ -189,7 +191,6 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) const struct lr_nat_record *lrnat_rec = lr_nat_table_find( input_data.lr_nats, od->nbr); ovs_assert(lrnat_rec); - lr_lbnat_rec = lr_lb_nat_data_record_create(&data->lr_lbnats, lrnat_rec, input_data.lb_datapaths_map, @@ -197,6 +198,10 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) /* Add the lr_lbnat_rec rec to the tracking data. */ hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + + if (!sset_is_empty(&lr_lbnat_rec->vip_nats)) { + data->tracked_data.vip_nats_changed = true; + } continue; } @@ -305,7 +310,9 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) * vip nats and re-evaluate 'has_lb_vip'. */ HMAPX_FOR_EACH (hmapx_node, &data->tracked_data.crupdated) { lr_lbnat_rec = hmapx_node->data; - lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + if (lr_lb_nat_data_build_vip_nats(lr_lbnat_rec)) { + data->tracked_data.vip_nats_changed = true; + } lr_lbnat_rec->has_lb_vip = od_has_lb_vip(lr_lbnat_rec->od); } @@ -344,8 +351,13 @@ lr_lb_nat_data_lr_nat_handler(struct engine_node *node, void *data_) lrnat_rec, input_data.lb_datapaths_map, input_data.lbgrp_datapaths_map); + if (!sset_is_empty(&lr_lbnat_rec->vip_nats)) { + data->tracked_data.vip_nats_changed = true; + } } else { - lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + if (lr_lb_nat_data_build_vip_nats(lr_lbnat_rec)) { + data->tracked_data.vip_nats_changed = true; + } } /* Add the lr_lbnat_rec rec to the tracking data. */ @@ -424,6 +436,7 @@ lr_lb_nat_data_record_create(struct lr_lb_nat_data_table *table, lr_lbnat_rec->od = lrnat_rec->od; lr_lb_nat_data_record_init(lr_lbnat_rec, lb_datapaths_map, lbgrp_datapaths_map); + lr_lbnat_rec->lflow_ref = lflow_ref_alloc(lr_lbnat_rec->od->nbr->name); hmap_insert(&table->entries, &lr_lbnat_rec->key_node, uuid_hash(&lr_lbnat_rec->od->nbr->header_.uuid)); @@ -437,6 +450,7 @@ lr_lb_nat_data_record_destroy(struct lr_lb_nat_data_record *lr_lbnat_rec) ovn_lb_ip_set_destroy(lr_lbnat_rec->lb_ips); lr_lbnat_rec->lb_ips = NULL; sset_destroy(&lr_lbnat_rec->vip_nats); + lflow_ref_destroy(lr_lbnat_rec->lflow_ref); free(lr_lbnat_rec); } @@ -503,7 +517,7 @@ lr_lb_nat_data_record_init(struct lr_lb_nat_data_record *lr_lbnat_rec, sset_init(&lr_lbnat_rec->vip_nats); - if (!nbr->n_nat) { + if (nbr->n_nat) { lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); } @@ -617,10 +631,13 @@ remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *lr_lbnat_rec, } } -static void +/* Builds the vip nats and returns true if the lr_lbnat_rec->vip_nats + * changed, false otherwise. */ +static bool lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *lr_lbnat_rec) { - sset_clear(&lr_lbnat_rec->vip_nats); + struct sset old_vip_nats = SSET_INITIALIZER(&old_vip_nats); + sset_swap(&lr_lbnat_rec->vip_nats, &old_vip_nats); const char *external_ip; SSET_FOR_EACH (external_ip, &lr_lbnat_rec->lrnat_rec->external_ips) { bool is_vip_nat = false; @@ -636,4 +653,14 @@ lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *lr_lbnat_rec) sset_add(&lr_lbnat_rec->vip_nats, external_ip); } } + + bool vip_nats_changed = + sset_count(&lr_lbnat_rec->vip_nats) != sset_count(&old_vip_nats); + if (!vip_nats_changed) { + vip_nats_changed = !sset_equals(&lr_lbnat_rec->vip_nats, + &old_vip_nats); + } + sset_destroy(&old_vip_nats); + + return vip_nats_changed; } diff --git a/northd/en-lr-lb-nat-data.h b/northd/en-lr-lb-nat-data.h index 759d055f05..7acf67b731 100644 --- a/northd/en-lr-lb-nat-data.h +++ b/northd/en-lr-lb-nat-data.h @@ -32,6 +32,7 @@ struct ovn_datapath; struct lr_nat_record; +struct lflow_ref; struct lr_lb_nat_data_record { struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ @@ -46,6 +47,8 @@ struct lr_lb_nat_data_record { /* sset of vips which are also part of lr nats. */ struct sset vip_nats; + + struct lflow_ref *lflow_ref; }; struct lr_lb_nat_data_table { @@ -65,6 +68,10 @@ struct lr_lb_nat_data_tracked_data { /* Deleted logical router with LB data. */ struct hmapx deleted; /* Stores 'struct lr_lb_nat_data_record'. */ + + /* Indicates if any router's NATs changed which were also LB vips + * or vice versa. */ + bool vip_nats_changed; }; struct ed_type_lr_lb_nat_data { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 3cedd502cb..955b5e2ed1 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -236,10 +236,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_ls_lbacls, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); - engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); - engine_add_input(&en_lflow, &en_ls_lbacls, NULL); + engine_add_input(&en_lflow, &en_lr_lb_nat_data, + lflow_lr_lb_nat_data_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 1033bc1a8f..5c12194454 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -1227,7 +1227,7 @@ ovn_port_create(struct hmap *ports, const char *key, op->lflow_ref = lflow_ref_alloc(key); op->lbnat_lflow_ref = lflow_ref_alloc(key); - + op->routable_lflow_ref = lflow_ref_alloc(key); return op; } @@ -1276,6 +1276,7 @@ ovn_port_destroy_orphan(struct ovn_port *port) free(port->key); lflow_ref_destroy(port->lflow_ref); lflow_ref_destroy(port->lbnat_lflow_ref); + lflow_ref_destroy(port->routable_lflow_ref); free(port); } @@ -12827,62 +12828,6 @@ build_ip_routing_flows_for_lrp( } } -/* Logical router ingress table IP_ROUTING : IP Routing. - * - * For the LSP 'op' of type router, if there are logical router ports other - * than the LSP's peer connected to the logical switch, then for routable - * addresses (such as NAT IPs, LB VIPs, etc.) on each of the connected router - * ports, add routes to the LSP's peer router. - */ -static void -build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) -{ - ovs_assert(op->nbsp); - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs - || !op->od->n_router_ports) { - return; - } - - for (int i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = ovn_port_get_peer( - lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp || router_port == peer) { - continue; - } - - const struct lr_lb_nat_data_record *lr_lbnat_rec = - lr_lb_nat_data_table_find(lr_lbnats, router_port->od->nbr); - - if (router_port->nbrp->ha_chassis_group || - router_port->nbrp->n_gateway_chassis) { - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_lbnat_rec); - for (size_t j = 0; j < ra.n_addrs; j++) { - struct lport_addresses *laddrs = &ra.laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED, - lflow_ref); - } - } - destroy_routable_addresses(&ra); - } - } - -} - static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, @@ -13124,42 +13069,6 @@ build_arp_resolve_flows_for_lrouter( lflow_ref); } -static void -routable_addresses_to_lflows(struct lflow_table *lflows, - struct ovn_port *router_port, - struct ovn_port *peer, - const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_lbnat_rec); - if (!ra.n_addrs) { - return; - } - - for (size_t i = 0; i < ra.n_addrs; i++) { - ds_clear(match); - ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", - peer->json_key); - bool first = true; - for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { - if (!first) { - ds_put_cstr(match, ", "); - } - ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); - first = false; - } - ds_put_cstr(match, "}"); - - ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions), lflow_ref); - } - destroy_routable_addresses(&ra); -} - /* Local router ingress table ARP_RESOLVE: ARP Resolution. * * Any unicast packet that reaches this table is an IP packet whose @@ -13402,52 +13311,6 @@ build_arp_resolve_flows_for_lsp( } } -static void -build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct lflow_table *lflows, - const struct hmap *lr_ports, - const struct lr_lb_nat_data_table *lr_lbnats, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp) { - return; - } - - if (peer->od->nbr && - smap_get_bool(&peer->od->nbr->options, - "dynamic_neigh_routers", false)) { - return; - } - - for (size_t i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = - ovn_port_get_peer(lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp) { - continue; - } - - /* Skip the router port under consideration. */ - if (router_port == peer) { - continue; - } - - if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { - const struct lr_lb_nat_data_record *lr_lbnat_rec; - lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, - router_port->od->nbr); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_lbnat_rec, match, actions, - lflow_ref); - } - } -} - static void build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, @@ -15664,8 +15527,6 @@ static void build_lsp_lflows_for_lbnats(struct ovn_port *lsp, struct ovn_port *lrp_peer, const struct lr_lb_nat_data_record *lr_lbnat_rec, - const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct lflow_table *lflows, struct ds *match, struct ds *actions, @@ -15675,19 +15536,101 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, build_lswitch_rport_arp_req_flows_for_lbnats( lrp_peer, lr_lbnat_rec, lsp->od, lsp, lflows, &lsp->nbsp->header_, lflow_ref); - build_ip_routing_flows_for_router_type_lsp(lsp, lr_lbnats, - lr_ports, lflows, - lflow_ref); - build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_lbnats, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_lbnat_rec, lflows, match, actions, lflow_ref); } +/* Logical router ingress table IP_ROUTING : IP Routing. + * + * For the LRP 'lrp's peer's logical switch, if there are logical router + * ports ('peer_lrp's) other than the 'lrp', then for routable addresses + * (such as NAT IPs, LB VIPs, etc.) of the 'lrp' add routes to the + * peer_lrp's datapath. + */ +static void +build_routable_flows_for_router_port( + struct ovn_port *lrp, const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct lflow_table *lflows, + struct ds *match, + struct ds *actions, + struct lflow_ref *lflow_ref) +{ + ovs_assert(lrp->nbrp && lrp->od == lr_lbnat_rec->od); + + struct ovn_port *lsp_peer = lrp->peer; + if (!lsp_peer || !lsp_peer->nbsp) { + return; + } + + struct ovn_datapath *peer_ls = lsp_peer->od; + ovs_assert(peer_ls->nbs); + + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(lrp, lr_lbnat_rec); + + struct ovn_port *router_port; + + for (size_t i = 0; i < peer_ls->n_router_ports; i++) { + router_port = peer_ls->router_ports[i]->peer; + + if (router_port == lrp) { + continue; + } + + if (lrp->nbrp->ha_chassis_group || + lrp->nbrp->n_gateway_chassis) { + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, router_port->od, router_port, + router_port->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &router_port->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); + } + } + } + + bool dynamic_neigh_router = + smap_get_bool(&router_port->od->nbr->options, + "dynamic_neigh_routers", false); + + if (!dynamic_neigh_router && + (router_port->od->is_gw_router || router_port->cr_port)) { + + for (size_t k = 0; k < ra.n_addrs; k++) { + ds_clear(match); + ds_put_format(match, "outport == %s && " + REG_NEXT_HOP_IPV4" == {", + router_port->json_key); + bool first = true; + for (size_t j = 0; j < ra.laddrs[k].n_ipv4_addrs; j++) { + if (!first) { + ds_put_cstr(match, ", "); + } + ds_put_cstr(match, ra.laddrs[k].ipv4_addrs[j].addr_s); + first = false; + } + ds_put_cstr(match, "}"); + + ds_clear(actions); + ds_put_format(actions, "eth.dst = %s; next;", + ra.laddrs[k].ea_s); + ovn_lflow_add(lflows, router_port->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); + } + } + } + + destroy_routable_addresses(&ra); +} + static void build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct ds *match, struct ds *actions, struct lflow_table *lflows, @@ -15704,8 +15647,8 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, ovs_assert(lr_lbnat_rec); build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, - lr_lbnats, lr_ports, lflows, - match, actions, lflow_ref); + lflows,match, actions, + lflow_ref); } static void @@ -15755,7 +15698,8 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, struct ds *match, struct ds *actions, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) + struct lflow_ref *lflow_ref, + struct lflow_ref *routable_lflow_ref) { ovs_assert(op->nbrp); @@ -15765,6 +15709,9 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, build_lrp_lflows_for_lbnats(op, lr_lbnat_rec, meter_groups, match, actions, lflows, lflow_ref); + + build_routable_flows_for_router_port(op, lr_lbnat_rec, lflows, match, + actions, routable_lflow_ref); } static void @@ -15779,15 +15726,15 @@ build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, { build_lrouter_nat_defrag_and_lb(lr_lbnat_rec, lflows, ls_ports, lr_ports, match, actions, meter_groups, features, - NULL); + lr_lbnat_rec->lflow_ref); build_lr_gateway_redirect_flows_for_nats(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, match, actions, - NULL); + lr_lbnat_rec->lflow_ref); build_lrouter_arp_nd_for_datapath(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, meter_groups, - NULL); + lr_lbnat_rec->lflow_ref); } static void @@ -16019,7 +15966,6 @@ build_lflows_thread(void *arg) &lsi->actions, lsi->lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi->lr_lbnats, - lsi->lr_ports, &lsi->match, &lsi->actions, lsi->lflows, @@ -16041,7 +15987,8 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions, lsi->lflows, - op->lbnat_lflow_ref); + op->lbnat_lflow_ref, + op->routable_lflow_ref); } } for (bnum = control->id; @@ -16272,9 +16219,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows); - build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, lsi.lr_ports, - &lsi.match, &lsi.actions, - lsi.lflows, + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, &lsi.match, + &lsi.actions, lsi.lflows, op->lbnat_lflow_ref); } HMAP_FOR_EACH (op, key_node, lr_ports) { @@ -16284,7 +16230,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows, - op->lbnat_lflow_ref); + op->lbnat_lflow_ref, + op->routable_lflow_ref); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16525,8 +16472,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, ovs_assert(lr_lbnat_rec); lflow_ref_clear_lflows(op->lbnat_lflow_ref, op->od, lflows); build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, - lflow_input->lr_lbnats, - lflow_input->lr_ports, lflows, &match, &actions, op->lbnat_lflow_ref); lflow_ref_sync_lflows_to_sb(op->lbnat_lflow_ref, lflows, ovnsb_txn, @@ -16687,6 +16632,95 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct lr_lb_nat_data_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct lr_lb_nat_data_record *lr_lbnat_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + lr_lbnat_rec = hmapx_node->data; + + lflow_ref_clear_lflows(lr_lbnat_rec->lflow_ref, lr_lbnat_rec->od, + lflows); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lr_lbnat_data_flows(lr_lbnat_rec, lflows, lflow_input->ls_ports, + lflow_input->lr_ports, &match, &actions, + lflow_input->meter_groups, + lflow_input->features); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(lr_lbnat_rec->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &lr_lbnat_rec->od->ports) { + lflow_ref_clear_lflows(op->lbnat_lflow_ref, lr_lbnat_rec->od, + lflows); + lflow_ref_clear_lflows_for_all_dps(op->routable_lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + lflows); + + build_lbnat_lflows_iterate_by_lrp(op, lflow_input->lr_lbnats, + lflow_input->meter_groups, + &match, &actions, + lflows, + op->lbnat_lflow_ref, + op->routable_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->lbnat_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + lflow_ref_sync_lflows_to_sb(op->routable_lflow_ref, lflows, + ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + if (op->peer && op->peer->nbsp) { + lflow_ref_clear_lflows(op->peer->lbnat_lflow_ref, op->peer->od, + lflows); + + build_lbnat_lflows_iterate_by_lsp(op->peer, + lflow_input->lr_lbnats, + &match, &actions, + lflows, + op->peer->lbnat_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->peer->lbnat_lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + } + + ds_destroy(&match); + ds_destroy(&actions); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 56613f8412..98d3593dc1 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -680,6 +680,7 @@ struct ovn_port { */ struct lflow_ref *lflow_ref; struct lflow_ref *lbnat_lflow_ref; + struct lflow_ref *routable_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -704,6 +705,8 @@ void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); struct lflow_table; +struct lr_lb_nat_data_tracked_data; + void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_table *); @@ -715,6 +718,10 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_lbs *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *, + struct lr_lb_nat_data_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index b276c77221..3c330a051f 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12,6 +12,7 @@ m4_define([_DUMP_DB_TABLES], [ ovn-sbctl list meter >> $1 ovn-sbctl list meter_band >> $1 ovn-sbctl list port_group >> $1 + ovn-sbctl dump-flows > lflows_$1 ]) # CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10586,7 +10587,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10596,7 +10597,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10606,7 +10607,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10616,7 +10617,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10626,7 +10627,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10634,6 +10635,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10748,7 +10750,7 @@ check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10758,7 +10760,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10768,7 +10770,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10778,7 +10780,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10788,7 +10790,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10869,7 +10871,7 @@ check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10897,7 +10899,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11141,7 +11143,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11150,7 +11152,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11160,7 +11162,7 @@ check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11170,7 +11172,7 @@ check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11180,7 +11182,7 @@ check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11190,7 +11192,7 @@ check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11201,7 +11203,7 @@ check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11222,6 +11224,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute @@ -11231,6 +11235,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute @@ -11240,6 +11246,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.140 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute @@ -11255,7 +11263,7 @@ check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE From patchwork Tue Oct 24 00:50:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854088 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtm91yGVz202k for ; Tue, 24 Oct 2023 11:51:21 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4518641972; Tue, 24 Oct 2023 00:51:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4518641972 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YZy2WuyprBkR; Tue, 24 Oct 2023 00:51:17 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 4C7FE400B8; Tue, 24 Oct 2023 00:51:16 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4C7FE400B8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 21AA8C0071; Tue, 24 Oct 2023 00:51:16 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 18857C0032 for ; Tue, 24 Oct 2023 00:51:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5B1BC419C3 for ; Tue, 24 Oct 2023 00:50:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5B1BC419C3 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7FeWBIsrtjGB for ; Tue, 24 Oct 2023 00:50:20 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::228]) by smtp2.osuosl.org (Postfix) with ESMTPS id 984B341A02 for ; Tue, 24 Oct 2023 00:50:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 984B341A02 Received: by mail.gandi.net (Postfix) with ESMTPSA id A51091BF203; Tue, 24 Oct 2023 00:50:15 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:50:02 -0400 Message-ID: <20231024005002.4134039-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 15/18] northd: Add ls_lbacls handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 30 +++++++++++++++++++++ northd/en-lflow.h | 1 + northd/en-ls-lb-acls.c | 3 +++ northd/en-ls-lb-acls.h | 4 +++ northd/inc-proc-northd.c | 2 +- northd/northd.c | 50 ++++++++++++++++++++++++++++++---- northd/northd.h | 5 ++++ tests/ovn-northd.at | 58 ++++++++++++++++++++++++++-------------- 8 files changed, 127 insertions(+), 26 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 613351eba2..90ebb97657 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -192,6 +192,36 @@ lflow_lr_lb_nat_data_handler(struct engine_node *node, void *data) return true; } +bool +lflow_ls_lbacls_handler(struct engine_node *node, void *data) +{ + struct ed_type_ls_lbacls *ls_lbacls_data = + engine_get_input_data("ls_lbacls", node); + + if (!ls_lbacls_data->tracked || + !hmapx_is_empty(&ls_lbacls_data->tracked_data.deleted)) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + + struct lflow_input lflow_input; + lflow_get_input_data(node, &lflow_input); + + if (!lflow_handle_ls_lbacls_changes(eng_ctx->ovnsb_idl_txn, + &ls_lbacls_data->tracked_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + + engine_set_node_state(node, EN_UPDATED); + + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index adbd8767c9..7f4fe43d55 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -21,5 +21,6 @@ void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); bool lflow_lr_lb_nat_data_handler(struct engine_node *, void *data); +bool lflow_ls_lbacls_handler(struct engine_node *node, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-ls-lb-acls.c b/northd/en-ls-lb-acls.c index 1ba7ecb3e2..643c8caab2 100644 --- a/northd/en-ls-lb-acls.c +++ b/northd/en-ls-lb-acls.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_ls_lbacls); @@ -356,6 +357,7 @@ ls_lbacls_record_create(struct ls_lbacls_table *table, struct ls_lbacls_record *ls_lbacls_rec = xzalloc(sizeof *ls_lbacls_rec); ls_lbacls_rec->od = od; ls_lbacls_record_init(ls_lbacls_rec, od, NULL, ls_pgs); + ls_lbacls_rec->lflow_ref = lflow_ref_alloc(od->nbs->name); hmap_insert(&table->entries, &ls_lbacls_rec->key_node, uuid_hash(&ls_lbacls_rec->od->nbs->header_.uuid)); @@ -366,6 +368,7 @@ ls_lbacls_record_create(struct ls_lbacls_table *table, static void ls_lbacls_record_destroy(struct ls_lbacls_record *ls_lbacls_rec) { + lflow_ref_destroy(ls_lbacls_rec->lflow_ref); free(ls_lbacls_rec); } diff --git a/northd/en-ls-lb-acls.h b/northd/en-ls-lb-acls.h index ccb75e40e8..71126aa4f4 100644 --- a/northd/en-ls-lb-acls.h +++ b/northd/en-ls-lb-acls.h @@ -31,6 +31,8 @@ #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +struct lflow_ref; + struct ls_lbacls_record { struct hmap_node key_node; @@ -39,6 +41,8 @@ struct ls_lbacls_record { bool has_lb_vip; bool has_acls; uint64_t max_acl_tier; + + struct lflow_ref *lflow_ref; }; struct ls_lbacls_table { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 955b5e2ed1..5c3d9faafa 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -236,11 +236,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); - engine_add_input(&en_lflow, &en_ls_lbacls, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_lb_nat_data, lflow_lr_lb_nat_data_handler); + engine_add_input(&en_lflow, &en_ls_lbacls, lflow_ls_lbacls_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 5c12194454..1f87102a05 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -15747,13 +15747,14 @@ build_ls_lbacls_flows(const struct ls_lbacls_record *ls_lbacls_rec, ovs_assert(ls_lbacls_rec->od); build_ls_lbacls_rec_pre_acls(ls_lbacls_rec, ls_pgs, lflows, - NULL); + ls_lbacls_rec->lflow_ref); build_ls_lbacls_rec_pre_lb(ls_lbacls_rec, lflows, - NULL); - build_acl_hints(ls_lbacls_rec, features, lflows, NULL); + ls_lbacls_rec->lflow_ref); + build_acl_hints(ls_lbacls_rec, features, lflows, + ls_lbacls_rec->lflow_ref); build_acls(ls_lbacls_rec, features, lflows, ls_pgs, meter_groups, - NULL); - build_lb_hairpin(ls_lbacls_rec, lflows, NULL); + ls_lbacls_rec->lflow_ref); + build_lb_hairpin(ls_lbacls_rec, lflows, ls_lbacls_rec->lflow_ref); } struct lswitch_flow_build_info { @@ -16721,6 +16722,45 @@ lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_ls_lbacls_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct ls_lbacls_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct ls_lbacls_record *ls_lbacls_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + ls_lbacls_rec = hmapx_node->data; + + lflow_ref_clear_lflows(ls_lbacls_rec->lflow_ref, ls_lbacls_rec->od, + lflows); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_ls_lbacls_flows(ls_lbacls_rec, lflow_input->ls_port_groups, + lflow_input->features, lflow_input->meter_groups, + lflows); + + ds_destroy(&match); + ds_destroy(&actions); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(ls_lbacls_rec->lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 98d3593dc1..3008c7855f 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -706,6 +706,7 @@ void northd_indices_create(struct northd_data *data, struct lflow_table; struct lr_lb_nat_data_tracked_data; +struct ls_lbacls_tracked_data; void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, @@ -722,6 +723,10 @@ bool lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *, struct lr_lb_nat_data_tracked_data *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_ls_lbacls_changes(struct ovsdb_idl_txn *, + struct ls_lbacls_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index 3c330a051f..e083c8925f 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -10502,11 +10502,12 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10515,7 +10516,8 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10525,7 +10527,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10535,7 +10538,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10545,7 +10549,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10555,6 +10560,7 @@ check ovn-nbctl --wait=sb ls-lb-del sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute +check_engine_stats ls_lbacls recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10566,7 +10572,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 -- lsp-add sw0 sw0p1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10685,7 +10692,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Update lb and this should not result in northd recompute @@ -10693,8 +10700,9 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set load_balancer . options:bar=foo check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute +check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute # Modify the backend of the lb1 vip @@ -10704,7 +10712,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10714,7 +10722,8 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10724,7 +10733,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10734,7 +10744,8 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10809,7 +10820,8 @@ check ovn-nbctl --wait=sb add logical_switch sw0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10862,7 +10874,8 @@ check ovn-nbctl --wait=sb set logical_switch sw0 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10880,7 +10893,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10889,7 +10903,8 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb3 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10917,6 +10932,7 @@ check ovn-nbctl --wait=sb lr-lb-del lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute +check_engine_stats ls_lbacls recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10928,7 +10944,8 @@ check ovn-nbctl --wait=sb lb-del lb4 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10939,7 +10956,8 @@ check ovn-nbctl --wait=sb lb-del lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats ls_lbacls norecompute compute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11006,7 +11024,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11014,7 +11032,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats From patchwork Tue Oct 24 00:50:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854092 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtnQ3Ltlz202k for ; Tue, 24 Oct 2023 11:52:26 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 4A58141983; Tue, 24 Oct 2023 00:52:24 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 4A58141983 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mj9ZEyQVT3eP; Tue, 24 Oct 2023 00:52:20 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2AE3941960; Tue, 24 Oct 2023 00:52:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 2AE3941960 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id CDAE7C0DD2; Tue, 24 Oct 2023 00:52:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id B8CC2C0071 for ; Tue, 24 Oct 2023 00:52:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id D9B2741DF4 for ; Tue, 24 Oct 2023 00:50:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org D9B2741DF4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z8qap1y2T7lq for ; Tue, 24 Oct 2023 00:50:38 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp4.osuosl.org (Postfix) with ESMTPS id 861B041822 for ; Tue, 24 Oct 2023 00:50:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 861B041822 Received: by mail.gandi.net (Postfix) with ESMTPSA id B30EDFF803; Tue, 24 Oct 2023 00:50:28 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:50:16 -0400 Message-ID: <20231024005016.4134060-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 16/18] northd: Add I-P for NB_Global and SB_Global. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique A new engine node "global_config" is added which handles the changes to NB_Global an SB_Global tables. It also creates these rows if not present. Without the I-P, any changes to the options column of these tables result in recompute of 'northd' and 'lflow' engine nodes. Signed-off-by: Numan Siddique --- northd/aging.c | 21 +- northd/automake.mk | 2 + northd/en-global-config.c | 588 ++++++++++++++++++++++++++++++++++++++ northd/en-global-config.h | 65 +++++ northd/en-lflow.c | 11 +- northd/en-northd.c | 52 ++-- northd/en-northd.h | 2 +- northd/en-sync-sb.c | 22 +- northd/inc-proc-northd.c | 38 ++- northd/northd.c | 230 +++------------ northd/northd.h | 24 +- tests/ovn-northd.at | 256 +++++++++++++++-- 12 files changed, 1014 insertions(+), 297 deletions(-) create mode 100644 northd/en-global-config.c create mode 100644 northd/en-global-config.h diff --git a/northd/aging.c b/northd/aging.c index f626c72c8c..cf988b39c4 100644 --- a/northd/aging.c +++ b/northd/aging.c @@ -15,6 +15,7 @@ #include +#include "en-global-config.h" #include "lib/inc-proc-eng.h" #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" @@ -100,15 +101,10 @@ aging_context_handle_timestamp(struct aging_context *ctx, int64_t timestamp) static uint32_t get_removal_limit(struct engine_node *node, const char *name) { - const struct nbrec_nb_global_table *nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - if (!nb) { - return 0; - } + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); - return smap_get_uint(&nb->options, name, 0); + return smap_get_uint(&global_config->nb_options, name, 0); } /* MAC binding aging */ @@ -142,11 +138,14 @@ en_mac_binding_aging_run(struct engine_node *node, void *data OVS_UNUSED) { const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + struct aging_waker *waker = engine_get_input_data("mac_binding_aging_waker", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.mac_binding_timestamp || + !global_config->features.mac_binding_timestamp || time_msec() < waker->next_wake_msec) { return; } @@ -271,9 +270,11 @@ en_fdb_aging_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); struct aging_waker *waker = engine_get_input_data("fdb_aging_waker", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); if (!eng_ctx->ovnsb_idl_txn || - !northd_data->features.fdb_timestamp || + !global_config->features.fdb_timestamp || time_msec() < waker->next_wake_msec) { return; } diff --git a/northd/automake.mk b/northd/automake.mk index 9707be7d8e..482270f5dd 100644 --- a/northd/automake.mk +++ b/northd/automake.mk @@ -8,6 +8,8 @@ northd_ovn_northd_SOURCES = \ northd/northd.c \ northd/northd.h \ northd/ovn-northd.c \ + northd/en-global-config.c \ + northd/en-global-config.h \ northd/en-northd.c \ northd/en-northd.h \ northd/en-lflow.c \ diff --git a/northd/en-global-config.c b/northd/en-global-config.c new file mode 100644 index 0000000000..0d218f2ab5 --- /dev/null +++ b/northd/en-global-config.c @@ -0,0 +1,588 @@ +/* + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at: + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include + +#include +#include +#include + +/* OVS includes */ +#include "openvswitch/vlog.h" + +/* OVN includes */ +#include "debug.h" +#include "en-global-config.h" +#include "include/ovn/features.h" +#include "ipam.h" +#include "lib/ovn-nb-idl.h" +#include "lib/ovn-sb-idl.h" +#include "northd.h" + + +VLOG_DEFINE_THIS_MODULE(en_global_config); + +/* static function declarations. */ +static void northd_enable_all_features(struct ed_type_global_config *); +static void build_chassis_features(const struct sbrec_chassis_table *, + struct chassis_features *); +static bool chassis_features_changed(const struct chassis_features *, + const struct chassis_features *); +static bool config_out_of_sync(const struct smap *config, + const struct smap *saved_config, + const char *key, bool must_be_present); +static bool check_nb_options_out_of_sync(const struct nbrec_nb_global *, + struct ed_type_global_config *); +static void update_sb_config_options_to_sbrec(struct ed_type_global_config *, + const struct sbrec_sb_global *); + +void * +en_global_config_init(struct engine_node *node OVS_UNUSED, + struct engine_arg *args OVS_UNUSED) +{ + struct ed_type_global_config *data = xzalloc(sizeof *data); + smap_init(&data->nb_options); + smap_init(&data->sb_options); + northd_enable_all_features(data); + return data; +} + +void +en_global_config_run(struct engine_node *node , void *data) +{ + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnnb_idl_txn || !eng_ctx->ovnsb_idl_txn) { + return; + } + + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + + en_global_config_clear_tracked_data(data); + + struct ed_type_global_config *config_data = data; + + /* Sync ipsec configuration. + * Copy nb_cfg from northbound to southbound database. + * Also set up to update sb_cfg once our southbound transaction commits. */ + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + nb = nbrec_nb_global_insert(eng_ctx->ovnnb_idl_txn); + } + + const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, + "mac_prefix")); + + const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); + if (monitor_mac) { + if (eth_addr_from_string(monitor_mac, + &config_data->svc_monitor_mac_ea)) { + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, + ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + } else { + monitor_mac = NULL; + } + } + + struct smap *options = &config_data->nb_options; + smap_destroy(options); + smap_clone(options, &nb->options); + + smap_replace(options, "mac_prefix", mac_addr_prefix); + + if (!monitor_mac) { + eth_addr_random(&config_data->svc_monitor_mac_ea); + snprintf(config_data->svc_monitor_mac, + sizeof config_data->svc_monitor_mac, ETH_ADDR_FMT, + ETH_ADDR_ARGS(config_data->svc_monitor_mac_ea)); + smap_replace(options, "svc_monitor_mac", + config_data->svc_monitor_mac); + } + + char *max_tunid = xasprintf("%d", + get_ovn_max_dp_key_local(sbrec_chassis_table)); + smap_replace(options, "max_tunid", max_tunid); + free(max_tunid); + + char *ovn_internal_version = ovn_get_internal_version(); + if (strcmp(ovn_internal_version, + smap_get_def(options, "northd_internal_version", ""))) { + smap_replace(options, "northd_internal_version", + ovn_internal_version); + config_data->ovn_internal_version_changed = true; + } else { + config_data->ovn_internal_version_changed = false; + } + + free(ovn_internal_version); + + if (!smap_equal(&nb->options, options)) { + nbrec_nb_global_verify_options(nb); + nbrec_nb_global_set_options(nb, options); + } + + if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { + northd_enable_all_features(config_data); + } else { + build_chassis_features(sbrec_chassis_table, &config_data->features); + } + + init_debug_config(nb); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + sb = sbrec_sb_global_insert(eng_ctx->ovnsb_idl_txn); + } + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + /* Set up SB_Global (depends on chassis features). */ + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); +} + +void en_global_config_cleanup(void *data OVS_UNUSED) +{ + struct ed_type_global_config *config_data = data; + smap_destroy(&config_data->nb_options); + smap_destroy(&config_data->sb_options); + destroy_debug_config(); +} + +void +en_global_config_clear_tracked_data(void *data) +{ + struct ed_type_global_config *config_data = data; + config_data->tracked = false; + config_data->tracked_data.nb_options_changed = false; + config_data->tracked_data.chassis_features_changed = false; +} + +bool +global_config_nb_global_handler(struct engine_node *node, void *data) +{ + const struct nbrec_nb_global_table *nb_global_table = + EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct nbrec_nb_global *nb = + nbrec_nb_global_table_first(nb_global_table); + if (!nb) { + return false; + } + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + /* We are only interested in ipsec and options column. */ + bool changes_relevant = false; + if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC) + || nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS)) { + changes_relevant = true; + } + + if (!changes_relevant) { + return true; + } + + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { + return false; + } + + if (nb->ipsec != sb->ipsec) { + sbrec_sb_global_set_ipsec(sb, nb->ipsec); + } + + struct ed_type_global_config *config_data = data; + config_data->tracked = true; + + if (smap_equal(&nb->options, &config_data->nb_options)) { + return true; + } + + /* Return false if an option is out of sync and requires updating the + * NB config. (Like svc_monitor_mac, max_tunid and mac_prefix). */ + /* Check if svc_monitor_mac has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "svc_monitor_mac", true)) { + return false; + } + + /* Check if max_tunid has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "max_tunid", true)) { + return false; + } + + /* Check if mac_prefix has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_prefix", true)) { + return false; + } + + /* Check if ignore_chassis_features has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_chassis_features", false)) { + return false; + } + + /* Check if northd_internal_version has changed or not. */ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "northd_internal_version", false)) { + return false; + } + + if (check_nb_options_out_of_sync(nb, config_data)) { + config_data->tracked_data.nb_options_changed = true; + } + + smap_destroy(&config_data->nb_options); + smap_clone(&config_data->nb_options, &nb->options); + + update_sb_config_options_to_sbrec(config_data, sb); + + engine_set_node_state(node, EN_UPDATED); + return true; +} + +bool +global_config_sb_global_handler(struct engine_node *node, void *data) +{ + const struct sbrec_sb_global_table *sb_global_table = + EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); + + const struct sbrec_sb_global *sb = + sbrec_sb_global_table_first(sb_global_table); + if (!sb) { + return false; + } + + struct ed_type_global_config *config_data = data; + + if (!smap_equal(&sb->options, &config_data->sb_options)) { + return false; + } + + /* No need to update the engine node. */ + return true; +} + +bool +global_config_sb_chassis_handler(struct engine_node *node, void *data) +{ + struct ed_type_global_config *config_data = data; + + const struct sbrec_chassis_table *sbrec_chassis_table = + EN_OVSDB_GET(engine_get_input("SB_chassis", node)); + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_new(chassis) + || sbrec_chassis_is_deleted(chassis) + || sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_ENCAPS)) { + return false; + } + + for (size_t i = 0; i < chassis->n_encaps; i++) { + if (sbrec_encap_row_get_seqno(chassis->encaps[i], + OVSDB_IDL_CHANGE_MODIFY) > 0) { + return false; + } + } + } + + if (smap_get_bool(&config_data->nb_options, "ignore_chassis_features", + false)) { + return true; + } + + bool reevaluate_chassis_features = false; + + /* Check and evaluate chassis features. */ + SBREC_CHASSIS_TABLE_FOR_EACH_TRACKED (chassis, sbrec_chassis_table) { + if (sbrec_chassis_is_updated(chassis, + SBREC_CHASSIS_COL_OTHER_CONFIG)) { + reevaluate_chassis_features = true; + break; + } + } + + if (!reevaluate_chassis_features) { + return true; + } + + struct chassis_features present_features = config_data->features; + + /* Enable all features before calling build_chassis_features() as + * build_chassis_features() only sets the feature flags to false. */ + northd_enable_all_features(config_data); + build_chassis_features(sbrec_chassis_table, &config_data->features); + + if (chassis_features_changed(&present_features, &config_data->features)) { + config_data->tracked_data.chassis_features_changed = true; + config_data->tracked = true; + engine_set_node_state(node, EN_UPDATED); + } + + return true; +} + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool +node_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.chassis_features_changed + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + +/* static functions. */ +static void +northd_enable_all_features(struct ed_type_global_config *data) +{ + data->features = (struct chassis_features) { + .ct_no_masked_label = true, + .mac_binding_timestamp = true, + .ct_lb_related = true, + .fdb_timestamp = true, + .ls_dpg_column = true, + }; +} + +static void +build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, + struct chassis_features *chassis_features) +{ + const struct sbrec_chassis *chassis; + + SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { + /* Only consider local AZ chassis. Remote ones don't install + * flows generated by the local northd. + */ + if (smap_get_bool(&chassis->other_config, "is-remote", false)) { + continue; + } + + bool ct_no_masked_label = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_NO_MASKED_LABEL, + false); + if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { + chassis_features->ct_no_masked_label = false; + } + + bool mac_binding_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_MAC_BINDING_TIMESTAMP, + false); + if (!mac_binding_timestamp && + chassis_features->mac_binding_timestamp) { + chassis_features->mac_binding_timestamp = false; + } + + bool ct_lb_related = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_CT_LB_RELATED, + false); + if (!ct_lb_related && + chassis_features->ct_lb_related) { + chassis_features->ct_lb_related = false; + } + + bool fdb_timestamp = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_FDB_TIMESTAMP, + false); + if (!fdb_timestamp && + chassis_features->fdb_timestamp) { + chassis_features->fdb_timestamp = false; + } + + bool ls_dpg_column = + smap_get_bool(&chassis->other_config, + OVN_FEATURE_LS_DPG_COLUMN, + false); + if (!ls_dpg_column && + chassis_features->ls_dpg_column) { + chassis_features->ls_dpg_column = false; + } + } +} + +static bool +config_out_of_sync(const struct smap *config, const struct smap *saved_config, + const char *key, bool must_be_present) +{ + const char *value = smap_get(config, key); + if (!value && must_be_present) { + return true; + } + + const char *saved_value = smap_get(saved_config, key); + if (!saved_value && must_be_present) { + return true; + } + + if (!value && !saved_value) { + return false; + } + + if ((!value && saved_value) || (value && !saved_value)) { + return true; + } + + return strcmp(value, saved_value); +} + +static bool +check_nb_options_out_of_sync(const struct nbrec_nb_global *nb, + struct ed_type_global_config *config_data) +{ + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "mac_binding_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "fdb_removal_limit", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "controller_event", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "ignore_lsp_down", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_ct_inv_match", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "default_acl_drop", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_domain_id", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "debug_drop_collector_set", false)) { + init_debug_config(nb); + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "use_common_zone", false)) { + return true; + } + + if (config_out_of_sync(&nb->options, &config_data->nb_options, + "install_ls_lb_from_router", false)) { + return true; + } + + return false; +} + +static void +update_sb_config_options_to_sbrec(struct ed_type_global_config *config_data, + const struct sbrec_sb_global *sb) +{ + struct smap *options = &config_data->sb_options; + + smap_destroy(options); + smap_clone(options, &config_data->nb_options); + + /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only + * if all chassis support it). If not explicitly present in the database + * the default value to be used for this option is 'true'. + */ + if (!config_data->features.ct_no_masked_label) { + smap_replace(options, "lb_hairpin_use_ct_mark", "false"); + } else { + smap_remove(options, "lb_hairpin_use_ct_mark"); + } + + /* Hackaround SB_global.options overwrite by NB_Global.options for + * 'sbctl_probe_interval' option. + */ + const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); + if (sip) { + smap_replace(options, "sbctl_probe_interval", sip); + } + + if (!smap_equal(&sb->options, options)) { + sbrec_sb_global_set_options(sb, options); + } +} + +static bool +chassis_features_changed(const struct chassis_features *present, + const struct chassis_features *updated) +{ + if (present->ct_no_masked_label != updated->ct_no_masked_label) { + return true; + } + + if (present->mac_binding_timestamp != updated->mac_binding_timestamp) { + return true; + } + + if (present->ct_lb_related != updated->ct_lb_related) { + return true; + } + + if (present->fdb_timestamp != updated->fdb_timestamp) { + return true; + } + + if (present->ls_dpg_column != updated->ls_dpg_column) { + return true; + } + + return false; +} diff --git a/northd/en-global-config.h b/northd/en-global-config.h new file mode 100644 index 0000000000..436bc7fa35 --- /dev/null +++ b/northd/en-global-config.h @@ -0,0 +1,65 @@ +#ifndef EN_GLOBAL_CONFIG_H +#define EN_GLOBAL_CONFIG_H 1 + +#include + +/* OVS includes. */ +#include "lib/packets.h" +#include "lib/smap.h" + +/* OVN includes. */ +#include "lib/inc-proc-eng.h" + +struct nbrec_nb_global; +struct sbrec_sb_global; + +struct chassis_features { + bool ct_no_masked_label; + bool mac_binding_timestamp; + bool ct_lb_related; + bool fdb_timestamp; + bool ls_dpg_column; +}; + +struct global_config_tracked_data { + bool nb_options_changed; + bool chassis_features_changed; +}; + +/* struct which maintains the data of the engine node global_config. */ +struct ed_type_global_config { + struct smap nb_options; + struct smap sb_options; + const struct nbrec_nb_global *nb_global; + const struct sbrec_sb_global *sb_global; + + /* MAC allocated for service monitor usage. Just one mac is allocated + * for this purpose and ovn-controller's on each chassis will make use + * of this mac when sending out the packets to monitor the services + * defined in Service_Monitor Southbound table. Since these packets + * are locally handled, having just one mac is good enough. */ + char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; + struct eth_addr svc_monitor_mac_ea; + + struct chassis_features features; + + bool ovn_internal_version_changed; + + bool tracked; + struct global_config_tracked_data tracked_data; +}; + +void *en_global_config_init(struct engine_node *, struct engine_arg *); +void en_global_config_run(struct engine_node *, void *data); +void en_global_config_cleanup(void *data); +void en_global_config_clear_tracked_data(void *data); + +bool global_config_nb_global_handler(struct engine_node *, void *data); +bool global_config_sb_global_handler(struct engine_node *, void *data); +bool global_config_sb_chassis_handler(struct engine_node *, void *data); + +/* generic global config handler for any engine node which has global_config + * has an input node . */ +bool node_global_config_handler(struct engine_node *, void *data); + +#endif /* EN_GLOBAL_CONFIG_H */ diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 90ebb97657..222665006b 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -22,6 +22,7 @@ #include "en-lr-nat.h" #include "en-lr-lb-nat-data.h" #include "en-ls-lb-acls.h" +#include "en-global-config.h" #include "en-northd.h" #include "en-meters.h" #include "lflow-mgr.h" @@ -77,10 +78,14 @@ lflow_get_input_data(struct engine_node *node, lflow_input->meter_groups = &sync_meters_data->meter_groups; lflow_input->lb_datapaths_map = &northd_data->lb_datapaths_map; lflow_input->svc_monitor_map = &northd_data->svc_monitor_map; - lflow_input->features = &northd_data->features; - lflow_input->ovn_internal_version_changed = - northd_data->ovn_internal_version_changed; lflow_input->bfd_connections = NULL; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + lflow_input->features = &global_config->features; + lflow_input->ovn_internal_version_changed = + global_config->ovn_internal_version_changed; + lflow_input->svc_monitor_mac = global_config->svc_monitor_mac; } void en_lflow_run(struct engine_node *node, void *data) diff --git a/northd/en-northd.c b/northd/en-northd.c index 13e731cad9..d4338c6a45 100644 --- a/northd/en-northd.c +++ b/northd/en-northd.c @@ -19,6 +19,7 @@ #include #include "coverage.h" +#include "en-global-config.h" #include "en-northd.h" #include "en-lb-data.h" #include "lib/inc-proc-eng.h" @@ -65,8 +66,6 @@ northd_get_input_data(struct engine_node *node, engine_get_input("SB_fdb", node), "sbrec_fdb_by_dp_and_port"); - input_data->nbrec_nb_global_table = - EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); input_data->nbrec_logical_switch_table = EN_OVSDB_GET(engine_get_input("NB_logical_switch", node)); input_data->nbrec_logical_router_table = @@ -78,8 +77,6 @@ northd_get_input_data(struct engine_node *node, input_data->nbrec_mirror_table = EN_OVSDB_GET(engine_get_input("NB_mirror", node)); - input_data->sbrec_sb_global_table = - EN_OVSDB_GET(engine_get_input("SB_sb_global", node)); input_data->sbrec_datapath_binding_table = EN_OVSDB_GET(engine_get_input("SB_datapath_binding", node)); input_data->sbrec_port_binding_table = @@ -109,6 +106,14 @@ northd_get_input_data(struct engine_node *node, engine_get_input_data("lb_data", node); input_data->lbs = &lb_data->lbs; input_data->lbgrps = &lb_data->lbgrps; + + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + input_data->nb_options = &global_config->nb_options; + input_data->sb_options = &global_config->sb_options; + input_data->svc_monitor_mac = global_config->svc_monitor_mac; + input_data->svc_monitor_mac_ea = global_config->svc_monitor_mac_ea; + input_data->features = &global_config->features; } void @@ -129,31 +134,6 @@ en_northd_run(struct engine_node *node, void *data) eng_ctx->ovnsb_idl_txn); stopwatch_stop(OVNNB_DB_RUN_STOPWATCH_NAME, time_msec()); engine_set_node_state(node, EN_UPDATED); - -} - -bool -northd_nb_nb_global_handler(struct engine_node *node, - void *data OVS_UNUSED) -{ - const struct nbrec_nb_global_table *nb_global_table - = EN_OVSDB_GET(engine_get_input("NB_nb_global", node)); - - const struct nbrec_nb_global *nb = - nbrec_nb_global_table_first(nb_global_table); - - if (!nb) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "NB_Global is updated but has no record."); - return false; - } - - /* We care about the 'options' and 'ipsec' columns only. */ - if (nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_OPTIONS) || - nbrec_nb_global_is_updated(nb, NBREC_NB_GLOBAL_COL_IPSEC)) { - return false; - } - return true; } bool @@ -243,6 +223,20 @@ northd_lb_data_handler(struct engine_node *node, void *data) return true; } +bool +northd_global_config_handler(struct engine_node *node, void *data OVS_UNUSED) +{ + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + + if (!global_config->tracked + || global_config->tracked_data.nb_options_changed) { + return false; + } + + return true; +} + void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) diff --git a/northd/en-northd.h b/northd/en-northd.h index 5a88871760..9b7bda32aa 100644 --- a/northd/en-northd.h +++ b/northd/en-northd.h @@ -14,7 +14,7 @@ void *en_northd_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg); void en_northd_cleanup(void *data); void en_northd_clear_tracked_data(void *data); -bool northd_nb_nb_global_handler(struct engine_node *, void *data OVS_UNUSED); +bool northd_global_config_handler(struct engine_node *, void *data OVS_UNUSED); bool northd_nb_logical_switch_handler(struct engine_node *, void *data); bool northd_nb_logical_router_handler(struct engine_node *, void *data); bool northd_sb_port_binding_handler(struct engine_node *, void *data); diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 7c22949f74..7a5d2f0ccd 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -23,6 +23,7 @@ #include "en-lr-nat.h" #include "en-lr-lb-nat-data.h" +#include "en-global-config.h" #include "en-sync-sb.h" #include "lib/inc-proc-eng.h" #include "lib/lb.h" @@ -42,7 +43,8 @@ static void sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *, const struct nbrec_port_group_table *, const struct sbrec_address_set_table *, - const struct lr_lb_nat_data_table *); + const struct lr_lb_nat_data_table *, + const char *svc_monitor_macp); static const struct sbrec_address_set *sb_address_set_lookup_by_name( struct ovsdb_idl_index *, const char *name); static void update_sb_addr_set(struct sorted_array *, @@ -90,9 +92,12 @@ en_sync_to_sb_addr_set_run(struct engine_node *node, void *data OVS_UNUSED) const struct engine_context *eng_ctx = engine_get_context(); const struct ed_type_lr_lb_nat_data *lr_lb_nat_data = engine_get_input_data("lr_lb_nat_data", node); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); sync_addr_sets(eng_ctx->ovnsb_idl_txn, nb_address_set_table, nb_port_group_table, sb_address_set_table, - &lr_lb_nat_data->lr_lbnats); + &lr_lb_nat_data->lr_lbnats, + global_config->svc_monitor_mac); engine_set_node_state(node, EN_UPDATED); } @@ -218,12 +223,14 @@ en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) { const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); const struct engine_context *eng_ctx = engine_get_context(); struct northd_data *northd_data = engine_get_input_data("northd", node); sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->lb_datapaths_map, &northd_data->features); + &northd_data->lb_datapaths_map, &global_config->features); engine_set_node_state(node, EN_UPDATED); } @@ -370,7 +377,8 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, const struct nbrec_address_set_table *nb_address_set_table, const struct nbrec_port_group_table *nb_port_group_table, const struct sbrec_address_set_table *sb_address_set_table, - const struct lr_lb_nat_data_table *lr_lbnats) + const struct lr_lb_nat_data_table *lr_lbnats, + const char *svc_monitor_macp) { struct shash sb_address_sets = SHASH_INITIALIZER(&sb_address_sets); @@ -381,8 +389,10 @@ sync_addr_sets(struct ovsdb_idl_txn *ovnsb_txn, } /* Service monitor MAC. */ - const char *svc_monitor_macp = northd_get_svc_monitor_mac(); - struct sorted_array svc = sorted_array_create(&svc_monitor_macp, 1, false); + struct sorted_array svc = { + .arr = &svc_monitor_macp, + .n = 1, + }; sync_addr_set(ovnsb_txn, "svc_monitor_mac", &svc, &sb_address_sets); sorted_array_destroy(&svc); diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 5c3d9faafa..99ac1079ec 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -30,6 +30,7 @@ #include "openvswitch/poll-loop.h" #include "openvswitch/vlog.h" #include "inc-proc-northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-lb-nat-data.h" #include "en-lr-nat.h" @@ -149,6 +150,7 @@ static ENGINE_NODE(fdb_aging, "fdb_aging"); static ENGINE_NODE(fdb_aging_waker, "fdb_aging_waker"); static ENGINE_NODE(sync_to_sb_lb, "sync_to_sb_lb"); static ENGINE_NODE(sync_to_sb_pb, "sync_to_sb_pb"); +static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(global_config, "global_config"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lb_data, "lb_data"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_nat, "lr_nat"); static ENGINE_NODE_WITH_CLEAR_TRACK_DATA(lr_lb_nat_data, "lr_lb_nat_data"); @@ -168,11 +170,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lb_data, &en_nb_logical_router, lb_data_logical_router_handler); + engine_add_input(&en_global_config, &en_nb_nb_global, + global_config_nb_global_handler); + engine_add_input(&en_global_config, &en_sb_sb_global, + global_config_sb_global_handler); + engine_add_input(&en_global_config, &en_sb_chassis, + global_config_sb_chassis_handler); + engine_add_input(&en_northd, &en_nb_mirror, NULL); engine_add_input(&en_northd, &en_nb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_nb_chassis_template_var, NULL); - engine_add_input(&en_northd, &en_sb_sb_global, NULL); engine_add_input(&en_northd, &en_sb_chassis, NULL); engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); @@ -185,11 +193,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_fdb, NULL); engine_add_input(&en_northd, &en_sb_static_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_chassis_template_var, NULL); + engine_add_input(&en_northd, &en_global_config, + northd_global_config_handler); engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); - engine_add_input(&en_northd, &en_nb_nb_global, - northd_nb_nb_global_handler); engine_add_input(&en_northd, &en_nb_logical_switch, northd_nb_logical_switch_handler); engine_add_input(&en_northd, &en_nb_logical_router, @@ -214,15 +222,17 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_ls_lbacls, &en_nb_logical_switch, ls_lbacls_logical_switch_handler); - engine_add_input(&en_mac_binding_aging, &en_nb_nb_global, NULL); engine_add_input(&en_mac_binding_aging, &en_sb_mac_binding, NULL); engine_add_input(&en_mac_binding_aging, &en_northd, NULL); engine_add_input(&en_mac_binding_aging, &en_mac_binding_aging_waker, NULL); + engine_add_input(&en_mac_binding_aging, &en_global_config, + node_global_config_handler); - engine_add_input(&en_fdb_aging, &en_nb_nb_global, NULL); engine_add_input(&en_fdb_aging, &en_sb_fdb, NULL); engine_add_input(&en_fdb_aging, &en_northd, NULL); engine_add_input(&en_fdb_aging, &en_fdb_aging_waker, NULL); + engine_add_input(&en_fdb_aging, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_meters, &en_nb_acl, NULL); engine_add_input(&en_sync_meters, &en_nb_meter, NULL); @@ -236,19 +246,23 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_global_config, + node_global_config_handler); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); engine_add_input(&en_lflow, &en_lr_lb_nat_data, lflow_lr_lb_nat_data_handler); engine_add_input(&en_lflow, &en_ls_lbacls, lflow_ls_lbacls_handler); + engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_lr_lb_nat_data, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_port_group, sync_to_sb_addr_set_nb_port_group_handler); - engine_add_input(&en_sync_to_sb_addr_set, &en_northd, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_lr_lb_nat_data, NULL); - engine_add_input(&en_sync_to_sb_addr_set, &en_sb_address_set, NULL); + engine_add_input(&en_sync_to_sb_addr_set, &en_global_config, + node_global_config_handler); engine_add_input(&en_port_group, &en_nb_port_group, port_group_nb_port_group_handler); @@ -258,6 +272,8 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, * table too (because of the explicit dependency in the schema). */ engine_add_input(&en_port_group, &en_northd, engine_noop_handler); + engine_add_input(&en_sync_to_sb_lb, &en_global_config, + node_global_config_handler); engine_add_input(&en_sync_to_sb_lb, &en_northd, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, @@ -362,11 +378,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, "sbrec_fdb_by_dp_and_port", sbrec_fdb_by_dp_and_port); - struct northd_data *northd_data = - engine_get_internal_data(&en_northd); + struct ed_type_global_config *global_config = + engine_get_internal_data(&en_global_config); unixctl_command_register("debug/chassis-features-list", "", 0, 0, chassis_features_list, - &northd_data->features); + &global_config->features); } /* Returns true if the incremental processing ended up updating nodes. */ diff --git a/northd/northd.c b/northd/northd.c index 1f87102a05..b6e24f94d8 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -43,6 +43,7 @@ #include "lflow-mgr.h" #include "memory.h" #include "northd.h" +#include "en-global-config.h" #include "en-lb-data.h" #include "en-lr-nat.h" #include "en-lr-lb-nat-data.h" @@ -77,14 +78,6 @@ static bool install_ls_lb_from_router; /* Use common zone for SNAT and DNAT if this option is set to "true". */ static bool use_common_zone = false; -/* MAC allocated for service monitor usage. Just one mac is allocatedg5534 - * for this purpose and ovn-controller's on each chassis will make use - * of this mac when sending out the packets to monitor the services - * defined in Service_Monitor Southbound table. Since these packets - * all locally handled, having just one mac is good enough. */ -static char svc_monitor_mac[ETH_ADDR_STRLEN + 1]; -static struct eth_addr svc_monitor_mac_ea; - /* If this option is 'true' northd will make use of ct.inv match fields. * Otherwise, it will avoid using it. The default is true. */ static bool use_ct_inv_match = true; @@ -295,66 +288,6 @@ ovn_stage_to_datapath_type(enum ovn_stage stage) } } -static void -build_chassis_features(const struct sbrec_chassis_table *sbrec_chassis_table, - struct chassis_features *chassis_features) -{ - const struct sbrec_chassis *chassis; - - SBREC_CHASSIS_TABLE_FOR_EACH (chassis, sbrec_chassis_table) { - /* Only consider local AZ chassis. Remote ones don't install - * flows generated by the local northd. - */ - if (smap_get_bool(&chassis->other_config, "is-remote", false)) { - continue; - } - - bool ct_no_masked_label = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_NO_MASKED_LABEL, - false); - if (!ct_no_masked_label && chassis_features->ct_no_masked_label) { - chassis_features->ct_no_masked_label = false; - } - - bool mac_binding_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_MAC_BINDING_TIMESTAMP, - false); - if (!mac_binding_timestamp && - chassis_features->mac_binding_timestamp) { - chassis_features->mac_binding_timestamp = false; - } - - bool ct_lb_related = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_CT_LB_RELATED, - false); - if (!ct_lb_related && - chassis_features->ct_lb_related) { - chassis_features->ct_lb_related = false; - } - - bool fdb_timestamp = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_FDB_TIMESTAMP, - false); - if (!fdb_timestamp && - chassis_features->fdb_timestamp) { - chassis_features->fdb_timestamp = false; - } - - bool ls_dpg_column = - smap_get_bool(&chassis->other_config, - OVN_FEATURE_LS_DPG_COLUMN, - false); - if (!ls_dpg_column && - chassis_features->ls_dpg_column) { - chassis_features->ls_dpg_column = false; - } - } -} - static uint32_t allocate_queueid(unsigned long *queue_id_bitmap) { @@ -946,7 +879,7 @@ is_vxlan_mode(const struct sbrec_chassis_table *sbrec_chassis_table) return false; } -static uint32_t +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *sbrec_chassis_table) { if (is_vxlan_mode(sbrec_chassis_table)) { @@ -3362,6 +3295,8 @@ create_or_get_service_mon(struct ovsdb_idl_txn *ovnsb_txn, static void ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, const struct ovn_northd_lb *lb, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *monitor_map, struct hmap *ls_ports, struct sset *svc_monitor_lsps) { @@ -3407,7 +3342,7 @@ ovn_lb_svc_create(struct ovsdb_idl_txn *ovnsb_txn, struct eth_addr ea; if (!mon_info->sbrec_mon->src_mac || !eth_addr_from_string(mon_info->sbrec_mon->src_mac, &ea) || - !eth_addr_equals(ea, svc_monitor_mac_ea)) { + !eth_addr_equals(ea, *svc_monitor_mac_ea)) { sbrec_service_monitor_set_src_mac(mon_info->sbrec_mon, svc_monitor_mac); } @@ -3732,6 +3667,8 @@ static void build_lb_svcs( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct hmap *ls_ports, struct hmap *lb_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) @@ -3750,7 +3687,8 @@ build_lb_svcs( struct ovn_lb_datapaths *lb_dps; HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_map, ls_ports, + ovn_lb_svc_create(ovnsb_txn, lb_dps->lb, svc_monitor_mac, + svc_monitor_mac_ea, svc_monitor_map, ls_ports, svc_monitor_lsps); } @@ -3842,13 +3780,16 @@ static void build_lb_port_related_data( struct ovsdb_idl_txn *ovnsb_txn, const struct sbrec_service_monitor_table *sbrec_service_monitor_table, + const char *svc_monitor_mac, + const struct eth_addr *svc_monitor_mac_ea, struct ovn_datapaths *lr_datapaths, struct hmap *ls_ports, struct hmap *lb_dps_map, struct hmap *lb_group_dps_map, struct sset *svc_monitor_lsps, struct hmap *svc_monitor_map) { build_lrouter_lbs_check(lr_datapaths); - build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, ls_ports, lb_dps_map, + build_lb_svcs(ovnsb_txn, sbrec_service_monitor_table, svc_monitor_mac, + svc_monitor_mac_ea, ls_ports, lb_dps_map, svc_monitor_lsps, svc_monitor_map); build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } @@ -9373,6 +9314,7 @@ build_lswitch_arp_nd_responder_default(struct ovn_datapath *od, static void build_lswitch_arp_nd_service_monitor(const struct ovn_northd_lb *lb, const struct hmap *ls_ports, + const char *svc_monitor_mac, struct lflow_table *lflows, struct ds *actions, struct ds *match, @@ -15776,6 +15718,7 @@ struct lswitch_flow_build_info { struct ds match; struct ds actions; size_t thread_lflow_counter; + const char *svc_monitor_mac; }; /* Helper function to combine all lflow generation which is iterated by @@ -16003,6 +15946,7 @@ build_lflows_thread(void *arg) } build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi->ls_ports, + lsi->svc_monitor_mac, lsi->lflows, &lsi->match, &lsi->actions, @@ -16125,7 +16069,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, const struct hmap *lb_dps_map, const struct hmap *svc_monitor_map, const struct hmap *bfd_connections, - const struct chassis_features *features) + const struct chassis_features *features, + const char *svc_monitor_mac) { char *svc_check_match = xasprintf("eth.dst == %s", svc_monitor_mac); @@ -16158,6 +16103,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, lsiv[index].features = features; lsiv[index].svc_check_match = svc_check_match; lsiv[index].thread_lflow_counter = 0; + lsiv[index].svc_monitor_mac = svc_monitor_mac; ds_init(&lsiv[index].match); ds_init(&lsiv[index].actions); @@ -16197,6 +16143,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, .bfd_connections = bfd_connections, .features = features, .svc_check_match = svc_check_match, + .svc_monitor_mac = svc_monitor_mac, .match = DS_EMPTY_INITIALIZER, .actions = DS_EMPTY_INITIALIZER, }; @@ -16238,6 +16185,7 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { build_lswitch_arp_nd_service_monitor(lb_dps->lb, lsi.ls_ports, + lsi.svc_monitor_mac, lsi.lflows, &lsi.actions, &lsi.match, lb_dps->lflow_ref); @@ -16356,7 +16304,8 @@ void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, input_data->lb_datapaths_map, input_data->svc_monitor_map, input_data->bfd_connections, - input_data->features); + input_data->features, + input_data->svc_monitor_mac); if (parallelization_state == STATE_INIT_HASH_SIZES) { parallelization_state = STATE_USE_PARALLELIZATION; @@ -16597,6 +16546,7 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct ds actions = DS_EMPTY_INITIALIZER; build_lswitch_arp_nd_service_monitor(lb_dps->lb, lflow_input->ls_ports, + lflow_input->svc_monitor_mac, lflows, &actions, &match, lb_dps->lflow_ref); build_lrouter_defrag_flows_for_lb(lb_dps, lflows, @@ -17364,18 +17314,6 @@ destroy_datapaths_and_ports(struct ovn_datapaths *ls_datapaths, ovn_datapaths_destroy(lr_datapaths); } -static void -northd_enable_all_features(struct northd_data *data) -{ - data->features = (struct chassis_features) { - .ct_no_masked_label = true, - .mac_binding_timestamp = true, - .ct_lb_related = true, - .fdb_timestamp = true, - .ls_dpg_column = true, - }; -} - void northd_init(struct northd_data *data) { @@ -17386,8 +17324,6 @@ northd_init(struct northd_data *data) hmap_init(&data->lb_datapaths_map); hmap_init(&data->lb_group_datapaths_map); ovs_list_init(&data->lr_list); - northd_enable_all_features(data); - data->ovn_internal_version_changed = false; sset_init(&data->svc_monitor_lsps); hmap_init(&data->svc_monitor_map); data->change_tracked = false; @@ -17428,7 +17364,6 @@ northd_destroy(struct northd_data *data) destroy_datapaths_and_ports(&data->ls_datapaths, &data->lr_datapaths, &data->ls_ports, &data->lr_ports, &data->lr_list); - destroy_debug_config(); sset_destroy(&data->svc_monitor_lsps); destroy_northd_tracked_data(data); @@ -17445,83 +17380,22 @@ ovnnb_db_run(struct northd_input *input_data, } stopwatch_start(BUILD_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Sync ipsec configuration. - * Copy nb_cfg from northbound to southbound database. - * Also set up to update sb_cfg once our southbound transaction commits. */ - const struct nbrec_nb_global *nb = nbrec_nb_global_table_first( - input_data->nbrec_nb_global_table); - if (!nb) { - nb = nbrec_nb_global_insert(ovnnb_txn); - } - - const char *mac_addr_prefix = set_mac_prefix(smap_get(&nb->options, - "mac_prefix")); - - const char *monitor_mac = smap_get(&nb->options, "svc_monitor_mac"); - if (monitor_mac) { - if (eth_addr_from_string(monitor_mac, &svc_monitor_mac_ea)) { - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - } else { - monitor_mac = NULL; - } - } - - struct smap options; - smap_clone(&options, &nb->options); - - smap_replace(&options, "mac_prefix", mac_addr_prefix); - - if (!monitor_mac) { - eth_addr_random(&svc_monitor_mac_ea); - snprintf(svc_monitor_mac, sizeof svc_monitor_mac, - ETH_ADDR_FMT, ETH_ADDR_ARGS(svc_monitor_mac_ea)); - smap_replace(&options, "svc_monitor_mac", svc_monitor_mac); - } - - char *max_tunid = xasprintf("%d", - get_ovn_max_dp_key_local(input_data->sbrec_chassis_table)); - smap_replace(&options, "max_tunid", max_tunid); - free(max_tunid); - - char *ovn_internal_version = ovn_get_internal_version(); - if (!strcmp(ovn_internal_version, - smap_get_def(&options, "northd_internal_version", ""))) { - data->ovn_internal_version_changed = false; - } else { - smap_replace(&options, "northd_internal_version", - ovn_internal_version); - } - free(ovn_internal_version); - - if (!smap_equal(&nb->options, &options)) { - nbrec_nb_global_verify_options(nb); - nbrec_nb_global_set_options(nb, &options); - } - - use_ct_inv_match = smap_get_bool(&nb->options, + use_ct_inv_match = smap_get_bool(input_data->nb_options, "use_ct_inv_match", true); /* deprecated, use --event instead */ - controller_event_en = smap_get_bool(&nb->options, + controller_event_en = smap_get_bool(input_data->nb_options, "controller_event", false); - check_lsp_is_up = !smap_get_bool(&nb->options, + check_lsp_is_up = !smap_get_bool(input_data->nb_options, "ignore_lsp_down", true); - default_acl_drop = smap_get_bool(&nb->options, "default_acl_drop", false); + default_acl_drop = smap_get_bool(input_data->nb_options, + "default_acl_drop", false); - install_ls_lb_from_router = smap_get_bool(&nb->options, + install_ls_lb_from_router = smap_get_bool(input_data->nb_options, "install_ls_lb_from_router", false); - use_common_zone = smap_get_bool(&nb->options, "use_common_zone", false); - - if (smap_get_bool(&nb->options, "ignore_chassis_features", false)) { - northd_enable_all_features(data); - } else { - build_chassis_features(input_data->sbrec_chassis_table, - &data->features); - } - - init_debug_config(nb); + use_common_zone = smap_get_bool(input_data->nb_options, "use_common_zone", + false); build_datapaths(ovnsb_txn, input_data->nbrec_logical_switch_table, @@ -17546,6 +17420,8 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_ports, &data->lr_ports); build_lb_port_related_data(ovnsb_txn, input_data->sbrec_service_monitor_table, + input_data->svc_monitor_mac, + &input_data->svc_monitor_mac_ea, &data->lr_datapaths, &data->ls_ports, &data->lb_datapaths_map, &data->lb_group_datapaths_map, @@ -17578,38 +17454,6 @@ ovnnb_db_run(struct northd_input *input_data, &data->ls_datapaths.datapaths); stopwatch_stop(CLEAR_LFLOWS_CTX_STOPWATCH_NAME, time_msec()); - /* Set up SB_Global (depends on chassis features). */ - const struct sbrec_sb_global *sb = sbrec_sb_global_table_first( - input_data->sbrec_sb_global_table); - if (!sb) { - sb = sbrec_sb_global_insert(ovnsb_txn); - } - if (nb->ipsec != sb->ipsec) { - sbrec_sb_global_set_ipsec(sb, nb->ipsec); - } - - /* Inform ovn-controllers whether LB flows will use ct_mark (i.e., only - * if all chassis support it). If not explicitly present in the database - * the default value to be used for this option is 'true'. - */ - if (!data->features.ct_no_masked_label) { - smap_replace(&options, "lb_hairpin_use_ct_mark", "false"); - } else { - smap_remove(&options, "lb_hairpin_use_ct_mark"); - } - - /* Hackaround SB_global.options overwrite by NB_Global.options for - * 'sbctl_probe_interval' option. - */ - const char *sip = smap_get(&sb->options, "sbctl_probe_interval"); - if (sip) { - smap_replace(&options, "sbctl_probe_interval", sip); - } - - if (!smap_equal(&sb->options, &options)) { - sbrec_sb_global_set_options(sb, &options); - } - smap_destroy(&options); } /* Stores the set of chassis which references an ha_chassis_group. @@ -17900,12 +17744,6 @@ ovnsb_db_run(struct ovsdb_idl_txn *ovnnb_txn, ovn_update_ipv6_prefix(lr_ports); } -const char * -northd_get_svc_monitor_mac(void) -{ - return svc_monitor_mac; -} - const struct ovn_datapath * northd_get_datapath_for_port(const struct hmap *ls_ports, const char *port_name) diff --git a/northd/northd.h b/northd/northd.h index 3008c7855f..d1f9d95ee7 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -27,7 +27,6 @@ struct northd_input { /* Northbound table references */ - const struct nbrec_nb_global_table *nbrec_nb_global_table; const struct nbrec_logical_switch_table *nbrec_logical_switch_table; const struct nbrec_logical_router_table *nbrec_logical_router_table; const struct nbrec_static_mac_binding_table @@ -37,7 +36,6 @@ struct northd_input { const struct nbrec_mirror_table *nbrec_mirror_table; /* Southbound table references */ - const struct sbrec_sb_global_table *sbrec_sb_global_table; const struct sbrec_datapath_binding_table *sbrec_datapath_binding_table; const struct sbrec_port_binding_table *sbrec_port_binding_table; const struct sbrec_mac_binding_table *sbrec_mac_binding_table; @@ -57,6 +55,13 @@ struct northd_input { const struct hmap *lbs; const struct hmap *lbgrps; + /* Global config data node inputs. */ + const struct smap *nb_options; + const struct smap *sb_options; + const char *svc_monitor_mac; + struct eth_addr svc_monitor_mac_ea; + const struct chassis_features *features; + /* Indexes */ struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_chassis_by_hostname; @@ -66,14 +71,6 @@ struct northd_input { struct ovsdb_idl_index *sbrec_fdb_by_dp_and_port; }; -struct chassis_features { - bool ct_no_masked_label; - bool mac_binding_timestamp; - bool ct_lb_related; - bool fdb_timestamp; - bool ls_dpg_column; -}; - /* A collection of datapaths. E.g. all logical switch datapaths, or all * logical router datapaths. */ struct ovn_datapaths { @@ -189,8 +186,6 @@ struct northd_data { struct hmap lb_datapaths_map; struct hmap lb_group_datapaths_map; struct ovs_list lr_list; - bool ovn_internal_version_changed; - struct chassis_features features; struct sset svc_monitor_lsps; struct hmap svc_monitor_map; /* Indicates if northd engine node has changes tracked or not. */ @@ -227,6 +222,7 @@ struct lflow_input { const struct chassis_features *features; const struct hmap *svc_monitor_map; bool ovn_internal_version_changed; + const char *svc_monitor_mac; }; extern int parallelization_state; @@ -748,8 +744,6 @@ void bfd_cleanup_connections(const struct nbrec_bfd_table *, struct hmap *bfd_map); void run_update_worker_pool(int n_threads); -const char *northd_get_svc_monitor_mac(void); - const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, @@ -782,4 +776,6 @@ bool lrouter_port_ipv4_reachable(const struct ovn_port *, ovs_be32 addr); bool lrouter_port_ipv6_reachable(const struct ovn_port *, const struct in6_addr *); +uint32_t get_ovn_max_dp_key_local(const struct sbrec_chassis_table *); + #endif /* NORTHD_H */ diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index e083c8925f..ba30f4b84c 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -8698,7 +8698,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=0 , match=(1), action=(next;) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=true ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -8717,6 +8717,7 @@ AT_CHECK([grep "ls_in_lb " S1flows | sed 's/table=../table=??/' | sort], [0], [d table=??(ls_in_lb ), priority=120 , match=(ct.new && ip4.dst == 172.16.0.11 && tcp.dst == 8080), action=(reg0[[1]] = 0; ct_lb_mark(backends=10.0.0.2:8080);) ]) + ovn-sbctl get datapath S0 _uuid > dp_uuids ovn-sbctl get datapath S1 _uuid >> dp_uuids lb_dp_group=$(ovn-sbctl --bare --columns ls_datapath_group find Load_Balancer name=lb0) @@ -8725,7 +8726,7 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat dp_uuids | sort) ]) -ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false +check ovn-nbctl --wait=sb set NB_Global . options:install_ls_lb_from_router=false ovn-sbctl dump-flows S0 > S0flows ovn-sbctl dump-flows S1 > S1flows @@ -9083,12 +9084,11 @@ $4 AS_BOX([Create new PG1 and PG2]) check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb -- pg-add pg1 -- pg-add pg2 -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes every time a NB port group is added/deleted. @@ -9121,12 +9121,11 @@ check ovn-nbctl --wait=sb \ check_column "sw1.1" sb:Port_Group ports name="${sw1_key}_pg1" check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9158,12 +9157,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl The port_group node recomputes also every time a port from a new switch @@ -9196,12 +9194,11 @@ check_column "sw2.1 sw2.3" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2 sw1.3" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2 sw2.3" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9239,7 +9236,7 @@ dnl though, therefore "compute: 1". AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We did not change the set of switches a pg is applied to, there should be @@ -9271,12 +9268,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9309,12 +9305,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be @@ -9347,12 +9342,11 @@ check_column "sw2.1" sb:Port_Group ports name="${sw2_key}_pg1" check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" check_column "sw2.2" sb:Port_Group ports name="${sw2_key}_pg2" -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -9387,12 +9381,11 @@ check_column "sw1.2" sb:Port_Group ports name="${sw1_key}_pg2" AT_CHECK([fetch_column sb:Port_Group ports name="${sw2_key}_pg2"], [0], [ ]) -dnl The northd node should not recompute, it should handle nb_global update -dnl though, therefore "compute: 1". +dnl The northd node should not recompute,. AT_CHECK([as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats northd], [0], [dnl Node: northd - recompute: 0 -- compute: 1 +- compute: 0 - abort: 0 ]) dnl We changed the set of switches a pg is applied to, there should be a @@ -11307,3 +11300,212 @@ CHECK_NO_CHANGE_AFTER_RECOMPUTE OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([NB_Global and SB_Global incremental processing]) + +ovn_start + +check_engine_stats() { + node=$1 + recompute=$2 + compute=$3 + + echo "__file__:__line__: Checking engine stats for node $node : recompute - \ +$recompute : compute - $compute" + + node_stat=$(as northd ovn-appctl -t NORTHD_TYPE inc-engine/show-stats $node) + # node_stat will be of this format : + # - Node: lflow - recompute: 3 - compute: 0 - abort: 0 + node_recompute_ct=$(echo $node_stat | cut -d '-' -f2 | cut -d ':' -f2) + node_compute_ct=$(echo $node_stat | cut -d '-' -f3 | cut -d ':' -f2) + + if [[ "$recompute" == "norecompute" ]]; then + # node should not be recomputed + echo "Expecting $node recompute count - $node_recompute_ct to be 0" + check test "$node_recompute_ct" -eq "0" + else + echo "Expecting $node recompute count - $node_recompute_ct not to be 0" + check test "$node_recompute_ct" -ne "0" + fi + + if [[ "$compute" == "nocompute" ]]; then + # node should not be computed + echo "Expecting $node compute count - $node_compute_ct to be 0" + check test "$node_compute_ct" -eq "0" + else + echo "Expecting $node compute count - $node_compute_ct not to be 0" + check test "$node_compute_ct" -ne "0" + fi +} + +check ovn-nbctl ls-add sw0 +check ovn-nbctl lr-add lr0 +check ovn-nbctl lsp-add sw0 sw0-p1 -- lsp-set-addresses sw0-p1 "00:00:20:20:00:03 10.0.0.3" +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:20:20:12:14 10.0.0.1/24 +check ovn-nbctl lsp-add sw0 sw0-lr0 +check ovn-nbctl lsp-set-type sw0-lr0 router +check ovn-nbctl lsp-set-addresses sw0-lr0 router +check ovn-nbctl --wait=sb lsp-set-options sw0-lr0 router-port=lr0-sw0 +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + +# This should not result in recomputes. +check ovn-nbctl --wait=sb set NB_Global . options:foo=bar +check_engine_stats global_config norecompute compute +check_engine_stats northd norecompute compute +check_engine_stats lflow norecompute compute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +# This should result in recomputes. +check ovn-sbctl set SB_Global . options:bar=foo +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute +CHECK_NO_CHANGE_AFTER_RECOMPUTE + +# Clears an nb option and checks that recomputes were triggered +# and the option was added back by ovn-northd or not depending +# on the 'added_back' argument. +clear_nb_option() { + option=$1 + add_back=$2 + echo "clearing the nb option - $option" + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config recompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + + local retval=1 + if [ "$add_back" == "true" ]; then + retval=0 + fi + AT_CHECK([ovn-nbctl get NB_Global . options:$option], [$retval], [ignore], [ignore]) +} + +# Clear svc_monitor_mac and few other options which result in recompute. +# and ovn-northd should update the nb options back. +clear_nb_option svc_monitor_mac true +clear_nb_option max_tunid true +clear_nb_option mac_prefix true +clear_nb_option northd_internal_version true + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-nbctl --wait=sb set NB_Global . options:ignore_chassis_features=true +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +clear_nb_option ignore_chassis_features false + +set_nb_option_lflow_recompute() { + local option=$1 + local value=$2 + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb set NB_Global . options:$option=$value + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +clear_nb_option_lflow_recompute() { + local option=$1 + check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats + check ovn-nbctl --wait=sb remove NB_Global . options $option + check_engine_stats global_config norecompute compute + check_engine_stats northd recompute nocompute + check_engine_stats lflow recompute nocompute + check_engine_stats mac_binding_aging recompute nocompute + CHECK_NO_CHANGE_AFTER_RECOMPUTE +} + +set_nb_option_lflow_recompute debug_drop_domain_id 1 +clear_nb_option_lflow_recompute debug_drop_domain_id + +set_nb_option_lflow_recompute debug_drop_collector_set 1 +clear_nb_option_lflow_recompute debug_drop_collector_set + +set_nb_option_lflow_recompute mac_binding_removal_limit 100 +clear_nb_option_lflow_recompute mac_binding_removal_limit + +set_nb_option_lflow_recompute fdb_removal_limit 100 +clear_nb_option_lflow_recompute fdb_removal_limit + +set_nb_option_lflow_recompute controller_event true +clear_nb_option_lflow_recompute controller_event + +set_nb_option_lflow_recompute ignore_lsp_down true +clear_nb_option_lflow_recompute ignore_lsp_down + +set_nb_option_lflow_recompute use_ct_inv_match true +clear_nb_option_lflow_recompute use_ct_inv_match + +set_nb_option_lflow_recompute default_acl_drop true +clear_nb_option_lflow_recompute default_acl_drop + +set_nb_option_lflow_recompute use_common_zone true +clear_nb_option_lflow_recompute use_common_zone + +set_nb_option_lflow_recompute install_ls_lb_from_router true +clear_nb_option_lflow_recompute install_ls_lb_from_router + +# Now test changes to chassis for feature changes. +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-add ch1 geneve 127.0.0.1 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-add ch2 geneve 127.0.0.2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [16711680 +], []) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl chassis-del ch2 +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set encap . type=vxlan +check ovn-nbctl --wait=sb sync +check_engine_stats global_config recompute compute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CHECK([ovn-nbctl get NB_Global . options:max_tunid | \ +sed s/":"//g | sed s/\"//g], [0], [4095 +], []) + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:foo=bar +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats +check ovn-sbctl set chassis . other_config:ct-no-masked-label=true +check ovn-nbctl --wait=sb sync +check_engine_stats global_config norecompute compute +check_engine_stats mac_binding_aging recompute nocompute +check_engine_stats fdb_aging recompute nocompute +check_engine_stats northd recompute nocompute +check_engine_stats lflow recompute nocompute + +AT_CLEANUP +]) From patchwork Tue Oct 24 00:50:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854093 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtnn3WHLz202k for ; Tue, 24 Oct 2023 11:52:45 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9EBDE614F4; Tue, 24 Oct 2023 00:52:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9EBDE614F4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7bFIIkXlXM4A; Tue, 24 Oct 2023 00:52:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 1B247614F2; Tue, 24 Oct 2023 00:52:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 1B247614F2 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BB61FC0032; Tue, 24 Oct 2023 00:52:39 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id A830EC0032 for ; Tue, 24 Oct 2023 00:52:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 443C341909 for ; Tue, 24 Oct 2023 00:50:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 443C341909 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k4iuxPpF501f for ; Tue, 24 Oct 2023 00:50:54 +0000 (UTC) Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp4.osuosl.org (Postfix) with ESMTPS id E2FB542138 for ; Tue, 24 Oct 2023 00:50:45 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E2FB542138 Received: by mail.gandi.net (Postfix) with ESMTPSA id 576B5FF803; Tue, 24 Oct 2023 00:50:43 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:50:30 -0400 Message-ID: <20231024005030.4134081-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 17/18] northd: Add a noop handler for northd SB mac binding. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/inc-proc-northd.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 99ac1079ec..cdc4927deb 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -185,7 +185,6 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_sb_mirror, NULL); engine_add_input(&en_northd, &en_sb_meter, NULL); engine_add_input(&en_northd, &en_sb_datapath_binding, NULL); - engine_add_input(&en_northd, &en_sb_mac_binding, NULL); engine_add_input(&en_northd, &en_sb_dns, NULL); engine_add_input(&en_northd, &en_sb_ha_chassis_group, NULL); engine_add_input(&en_northd, &en_sb_ip_multicast, NULL); @@ -196,6 +195,14 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_northd, &en_global_config, northd_global_config_handler); + /* northd engine node uses the sb mac binding table to + * cleanup mac binding entries for deleted logical ports + * and datapaths. Any update to to SB mac binding doesn't + * change the northd engine node state or data. Hence + * it is ok to add a noop_handler here. */ + engine_add_input(&en_northd, &en_sb_mac_binding, + engine_noop_handler); + engine_add_input(&en_northd, &en_sb_port_binding, northd_sb_port_binding_handler); engine_add_input(&en_northd, &en_nb_logical_switch, From patchwork Tue Oct 24 00:50:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854094 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtp73wDzz202k for ; Tue, 24 Oct 2023 11:53:03 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 815B241969; Tue, 24 Oct 2023 00:53:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 815B241969 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k6W_4c-GoWku; Tue, 24 Oct 2023 00:52:56 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id C55F241995; Tue, 24 Oct 2023 00:52:55 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C55F241995 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id ACA3EC0071; Tue, 24 Oct 2023 00:52:55 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id AAF69C0032 for ; Tue, 24 Oct 2023 00:52:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A457442D37 for ; Tue, 24 Oct 2023 00:51:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org A457442D37 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A065VSQDqYaE for ; Tue, 24 Oct 2023 00:51:02 +0000 (UTC) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by smtp4.osuosl.org (Postfix) with ESMTPS id 6BA664209D for ; Tue, 24 Oct 2023 00:51:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 6BA664209D Received: by mail.gandi.net (Postfix) with ESMTPSA id DA6C3C0005; Tue, 24 Oct 2023 00:50:58 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:50:44 -0400 Message-ID: <20231024005044.4134115-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 18/18] northd: Add northd change handler for sync_to_sb_lb node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Any changes to northd engine node due to load balancers are now handled in 'sync_to_sb_lb' node to sync the changed load balancers to SB load balancers. The logic to sync the SB load balancers is changed a bit and it now mimics the SB lflow sync. Below are the scale testing results done with all the patches applied in this series using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. The resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.136883 1.129016 1.192001 1.204167 1.212728 0.665017 83.127099 125 0 Namespace.add_ports 0.005216 0.005736 0.007034 0.015486 0.018978 0.006211 0.776373 125 0 WorkerNode.bind_port 0.035030 0.046082 0.052469 0.058293 0.060311 0.045973 11.493259 250 0 WorkerNode.ping_port 0.005057 0.006727 1.047692 1.069253 1.071336 0.266896 66.724094 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main [2] are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.135491 2.223805 3.311270 3.339078 3.345346 1.729172 216.146495 125 0 Namespace.add_ports 0.005380 0.005744 0.006819 0.018773 0.020800 0.006292 0.786532 125 0 WorkerNode.bind_port 0.034179 0.046055 0.053488 0.058801 0.071043 0.046117 11.529311 250 0 WorkerNode.ping_port 0.004956 0.006952 3.086952 3.191743 3.192807 0.791544 197.886026 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- [1] - https://github.com/ovn-org/ovn-heater/blob/main/test-scenarios/ocp-500-density-heavy.yml [2] - 2a12cda890a7("controller, northd: Wait for cleanup before replying to exit") Signed-off-by: Numan Siddique --- northd/en-sync-sb.c | 447 +++++++++++++++++++++++++++++++++++++-- northd/inc-proc-northd.c | 1 + northd/lflow-mgr.c | 196 ++++++----------- northd/lflow-mgr.h | 23 +- northd/northd.c | 238 --------------------- northd/northd.h | 6 - tests/ovn-northd.at | 102 +++++---- 7 files changed, 586 insertions(+), 427 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 7a5d2f0ccd..1cb4a58718 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -30,6 +30,7 @@ #include "lib/ovn-nb-idl.h" #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" +#include "lflow-mgr.h" #include "northd.h" #include "openvswitch/vlog.h" @@ -53,6 +54,38 @@ static void build_port_group_address_set(const struct nbrec_port_group *, struct svec *ipv4_addrs, struct svec *ipv6_addrs); +struct sb_lb_table; +struct sb_lb_record; +static void sb_lb_table_init(struct sb_lb_table *); +static void sb_lb_table_clear(struct sb_lb_table *); +static struct sb_lb_record *sb_lb_table_find(struct hmap *sb_lbs, + const struct uuid *); +static void sb_lb_table_build_and_sync(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static void sync_sb_lb_record(struct sb_lb_record *, + const struct sbrec_load_balancer *, + const struct sbrec_logical_dp_group_table *, + struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static void sync_changed_lbs(struct sb_lb_table *, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *, + const struct sbrec_logical_dp_group_table *, + struct tracked_lbs *, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *); +static bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); + void * en_sync_to_sb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -211,51 +244,103 @@ sync_to_sb_addr_set_nb_port_group_handler(struct engine_node *node, /* sync_to_sb_lb engine node functions. * This engine node syncs the SB load balancers. */ +struct sb_lb_record { + struct hmap_node key_node; /* Index on 'nblb->header_.uuid'. */ + + struct ovn_lb_datapaths *lb_dps; + const struct sbrec_load_balancer *sbrec_lb; + struct ovn_dp_group *ls_dpg; + struct ovn_dp_group *lr_dpg; + struct uuid sb_uuid; +}; + +struct sb_lb_table { + struct hmap entries; /* Stores struct sb_lb_record. */ + struct hmap ls_dp_groups; + struct hmap lr_dp_groups; +}; + +struct ed_type_sync_to_sb_lb_data { + struct sb_lb_table sb_lbs; +}; + void * en_sync_to_sb_lb_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { - return NULL; + struct ed_type_sync_to_sb_lb_data *data = xzalloc(sizeof *data); + sb_lb_table_init(&data->sb_lbs); + + return data; } void -en_sync_to_sb_lb_run(struct engine_node *node, void *data OVS_UNUSED) +en_sync_to_sb_lb_run(struct engine_node *node, void *data_) { + struct northd_data *northd_data = engine_get_input_data("northd", node); const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); struct ed_type_global_config *global_config = engine_get_input_data("global_config", node); + const struct engine_context *eng_ctx = engine_get_context(); - struct northd_data *northd_data = engine_get_input_data("northd", node); + struct ed_type_sync_to_sb_lb_data *data = data_; + + sb_lb_table_clear(&data->sb_lbs); + sb_lb_table_init(&data->sb_lbs); + sb_lb_table_build_and_sync(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, + sb_load_balancer_table, + sb_dpgrp_table, + &northd_data->lb_datapaths_map, + &northd_data->ls_datapaths, + &northd_data->lr_datapaths, + &global_config->features); - sync_lbs(eng_ctx->ovnsb_idl_txn, sb_load_balancer_table, - &northd_data->ls_datapaths, &northd_data->lr_datapaths, - &northd_data->lb_datapaths_map, &global_config->features); engine_set_node_state(node, EN_UPDATED); } void -en_sync_to_sb_lb_cleanup(void *data OVS_UNUSED) +en_sync_to_sb_lb_cleanup(void *data_) { - + struct ed_type_sync_to_sb_lb_data *data = data_; + sb_lb_table_clear(&data->sb_lbs); } bool -sync_to_sb_lb_northd_handler(struct engine_node *node, void *data OVS_UNUSED) +sync_to_sb_lb_northd_handler(struct engine_node *node, void *data_) { struct northd_data *nd = engine_get_input_data("northd", node); - if (!nd->change_tracked || - northd_has_lbs_in_tracked_data(&nd->trk_northd_changes)) { - /* Return false if no tracking data or if lbs changed. */ + if (!nd->change_tracked) { + /* Return false if no tracking data. */ + return false; + } + + if (!northd_has_lbs_in_tracked_data(&nd->trk_northd_changes)) { + return true; + } + + const struct engine_context *eng_ctx = engine_get_context(); + if (!eng_ctx->ovnsb_idl_txn) { return false; } + const struct sbrec_logical_dp_group_table *sb_dpgrp_table = + EN_OVSDB_GET(engine_get_input("SB_logical_dp_group", node)); + const struct sbrec_load_balancer_table *sb_lb_table = + EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_global_config *global_config = + engine_get_input_data("global_config", node); + struct ed_type_sync_to_sb_lb_data *data = data_; + + sync_changed_lbs(&data->sb_lbs, eng_ctx->ovnsb_idl_txn, sb_lb_table, + sb_dpgrp_table, &nd->trk_northd_changes.trk_lbs, + &nd->ls_datapaths, &nd->lr_datapaths, + &global_config->features); - /* There are only NB LSP related changes and these can be safely - * ignore and returned true. However in case the northd engine - * tracking data includes other changes, we need to do additional - * checks before safely ignoring. */ + engine_set_node_state(node, EN_UPDATED); return true; } @@ -530,3 +615,333 @@ sb_address_set_lookup_by_name(struct ovsdb_idl_index *sbrec_addr_set_by_name, return retval; } + +/* static functions related to sync_to_sb_lb */ + +static void +sb_lb_table_init(struct sb_lb_table *sb_lbs) +{ + hmap_init(&sb_lbs->entries); + ovn_dp_groups_init(&sb_lbs->ls_dp_groups); + ovn_dp_groups_init(&sb_lbs->lr_dp_groups); +} + +static void +sb_lb_table_clear(struct sb_lb_table *sb_lbs) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_POP (sb_lb, key_node, &sb_lbs->entries) { + free(sb_lb); + } + hmap_destroy(&sb_lbs->entries); + + ovn_dp_groups_destroy(&sb_lbs->ls_dp_groups); + ovn_dp_groups_destroy(&sb_lbs->lr_dp_groups); +} + +static struct sb_lb_record * +sb_lb_table_find(struct hmap *sb_lbs, const struct uuid *lb_uuid) +{ + struct sb_lb_record *sb_lb; + HMAP_FOR_EACH_WITH_HASH (sb_lb, key_node, uuid_hash(lb_uuid), + sb_lbs) { + if (uuid_equals(&sb_lb->lb_dps->lb->nlb->header_.uuid, lb_uuid)) { + return sb_lb; + } + } + + return NULL; +} + +static void +sb_lb_table_build_and_sync(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct hmap *lb_dps_map, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct hmap tmp_sb_lbs = HMAP_INITIALIZER(&tmp_sb_lbs); + struct ovn_lb_datapaths *lb_dps; + struct sb_lb_record *sb_lb; + + HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { + if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&tmp_sb_lbs, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } + + const struct sbrec_load_balancer *sbrec_lb; + SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, + sb_lb_table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + struct uuid lb_uuid; + if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { + sbrec_load_balancer_delete(sbrec_lb); + continue; + } + + sb_lb = sb_lb_table_find(&tmp_sb_lbs, &lb_uuid); + if (sb_lb) { + sb_lb->sbrec_lb = sbrec_lb; + sync_sb_lb_record(sb_lb, sbrec_lb, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + + hmap_remove(&tmp_sb_lbs, &sb_lb->key_node); + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } else { + sbrec_load_balancer_delete(sbrec_lb); + } + } + + HMAP_FOR_EACH_POP (sb_lb, key_node, &tmp_sb_lbs) { + sync_sb_lb_record(sb_lb, NULL, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); + } + + hmap_destroy(&tmp_sb_lbs); +} + +static void +sync_sb_lb_record(struct sb_lb_record *sb_lb, + const struct sbrec_load_balancer *sbrec_lb, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct sbrec_logical_dp_group *sbrec_ls_dp_group = NULL; + struct sbrec_logical_dp_group *sbrec_lr_dp_group = NULL; + const struct ovn_lb_datapaths *lb_dps; + struct ovn_dp_group *pre_sync_ls_dpg; + struct ovn_dp_group *pre_sync_lr_dpg; + + lb_dps = sb_lb->lb_dps; + pre_sync_ls_dpg = sb_lb->ls_dpg; + pre_sync_lr_dpg = sb_lb->lr_dpg; + + if (!sbrec_lb) { + sb_lb->sb_uuid = uuid_random(); + sbrec_lb = sbrec_load_balancer_insert_persist_uuid(ovnsb_txn, + &sb_lb->sb_uuid); + char *lb_id = xasprintf( + UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); + const struct smap external_ids = + SMAP_CONST1(&external_ids, "lb_id", lb_id); + sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); + free(lb_id); + } else { + sb_lb->sb_uuid = sbrec_lb->header_.uuid; + sbrec_ls_dp_group = + chassis_features->ls_dpg_column + ? sbrec_lb->ls_datapath_group + : sbrec_lb->datapath_group; /* deprecated */ + + sbrec_lr_dp_group = sbrec_lb->lr_datapath_group; + } + + if (lb_dps->n_nb_ls) { + sb_lb->ls_dpg = ovn_dp_group_get(&sb_lbs->ls_dp_groups, + lb_dps->n_nb_ls, + lb_dps->nb_ls_map, + ods_size(ls_datapaths)); + if (sb_lb->ls_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->ls_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->ls_dpg->dpg_uuid); + ovs_assert(sb_lb->ls_dpg->dp_group); + } else { + sb_lb->ls_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->ls_dp_groups, + sbrec_ls_dp_group, + lb_dps->n_nb_ls, lb_dps->nb_ls_map, + ods_size(ls_datapaths), true, + ls_datapaths, + lr_datapaths); + } + + if (chassis_features->ls_dpg_column) { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } else { + /* datapath_group column is deprecated. */ + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, + sb_lb->ls_dpg->dp_group); + + } + } else { + sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); + sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); + } + + + if (lb_dps->n_nb_lr) { + sb_lb->lr_dpg = ovn_dp_group_get(&sb_lbs->lr_dp_groups, + lb_dps->n_nb_lr, + lb_dps->nb_lr_map, + ods_size(lr_datapaths)); + if (sb_lb->lr_dpg) { + /* Update the dpg's sb dp_group. */ + sb_lb->lr_dpg->dp_group = + sbrec_logical_dp_group_table_get_for_uuid(sb_dpgrp_table, + &sb_lb->lr_dpg->dpg_uuid); + ovs_assert(sb_lb->lr_dpg->dp_group); + } else { + sb_lb->lr_dpg = ovn_dp_group_create( + ovnsb_txn, &sb_lbs->lr_dp_groups, + sbrec_lr_dp_group, + lb_dps->n_nb_lr, lb_dps->nb_lr_map, + ods_size(lr_datapaths), false, + ls_datapaths, + lr_datapaths); + } + + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, + sb_lb->lr_dpg->dp_group); + } else { + sbrec_load_balancer_set_lr_datapath_group(sbrec_lb, NULL); + } + + if (pre_sync_ls_dpg != sb_lb->ls_dpg) { + if (sb_lb->ls_dpg) { + inc_ovn_dp_group_ref(sb_lb->ls_dpg); + } + if (pre_sync_ls_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->ls_dp_groups, pre_sync_ls_dpg); + } + } + + if (pre_sync_lr_dpg != sb_lb->lr_dpg) { + if (sb_lb->lr_dpg) { + inc_ovn_dp_group_ref(sb_lb->lr_dpg); + } + if (pre_sync_lr_dpg) { + dec_ovn_dp_group_ref(&sb_lbs->lr_dp_groups, pre_sync_lr_dpg); + } + } + + /* Update columns. */ + sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); + sbrec_load_balancer_set_vips(sbrec_lb, + ovn_northd_lb_get_vips(lb_dps->lb)); + sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); + + /* Store the fact that northd provides the original (destination IP + + * transport port) tuple. + */ + struct smap options; + smap_clone(&options, &lb_dps->lb->nlb->options); + smap_replace(&options, "hairpin_orig_tuple", "true"); + sbrec_load_balancer_set_options(sbrec_lb, &options); + /* Clearing 'datapaths' column, since 'dp_group' is in use. */ + sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); + smap_destroy(&options); +} + +static void +sync_changed_lbs(struct sb_lb_table *sb_lbs, + struct ovsdb_idl_txn *ovnsb_txn, + const struct sbrec_load_balancer_table *sb_lb_table, + const struct sbrec_logical_dp_group_table *sb_dpgrp_table, + struct tracked_lbs *trk_lbs, + struct ovn_datapaths *ls_datapaths, + struct ovn_datapaths *lr_datapaths, + struct chassis_features *chassis_features) +{ + struct ovn_lb_datapaths *lb_dps; + struct hmapx_node *hmapx_node; + struct sb_lb_record *sb_lb; + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->deleted) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + if (sb_lb) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + } + + HMAPX_FOR_EACH (hmapx_node, &trk_lbs->crupdated) { + lb_dps = hmapx_node->data; + + sb_lb = sb_lb_table_find(&sb_lbs->entries, + &lb_dps->lb->nlb->header_.uuid); + + if (!sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + continue; + } + + if (!sb_lb) { + sb_lb = xzalloc(sizeof *sb_lb); + sb_lb->lb_dps = lb_dps; + hmap_insert(&sb_lbs->entries, &sb_lb->key_node, + uuid_hash(&lb_dps->lb->nlb->header_.uuid)); + } else { + sb_lb->sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + } + + if (sb_lb && !lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { + const struct sbrec_load_balancer *sbrec_lb = + sbrec_load_balancer_table_get_for_uuid(sb_lb_table, + &sb_lb->sb_uuid); + if (sbrec_lb) { + sbrec_load_balancer_delete(sbrec_lb); + } + + hmap_remove(&sb_lbs->entries, &sb_lb->key_node); + free(sb_lb); + } + + sync_sb_lb_record(sb_lb, sb_lb->sbrec_lb, sb_dpgrp_table, sb_lbs, + ovnsb_txn, ls_datapaths, lr_datapaths, + chassis_features); + } +} + +static bool +check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) +{ + struct sset existing_nb_lb_uuids = + SSET_INITIALIZER(&existing_nb_lb_uuids); + const struct sbrec_load_balancer *sbrec_lb; + bool duplicates = false; + + SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { + const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); + if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { + duplicates = true; + break; + } + } + + sset_destroy(&existing_nb_lb_uuids); + return duplicates; +} diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index cdc4927deb..6dcbf11d62 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -285,6 +285,7 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, sync_to_sb_lb_northd_handler); engine_add_input(&en_sync_to_sb_lb, &en_sb_load_balancer, sync_to_sb_lb_sb_load_balancer); + engine_add_input(&en_sync_to_sb_lb, &en_sb_logical_dp_group, NULL); engine_add_input(&en_sync_to_sb_pb, &en_northd, sync_to_sb_pb_northd_handler); diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c index 30d123a7bc..7e4dc61f50 100644 --- a/northd/lflow-mgr.c +++ b/northd/lflow-mgr.c @@ -73,21 +73,6 @@ static struct ovs_mutex *lflow_hash_lock(const struct hmap *lflow_table, uint32_t hash); static void lflow_hash_unlock(struct ovs_mutex *hash_lock); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); -static struct ovn_dp_group *ovn_dp_group_create( - struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, - struct sbrec_logical_dp_group *, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths); -static struct ovn_dp_group *ovn_dp_group_get( - struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len); static struct sbrec_logical_dp_group *ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, struct sbrec_logical_dp_group *, @@ -505,31 +490,81 @@ lflow_table_add_lflow_default_drop(struct lflow_table *lflow_table, where, lflow_ref); } -/* Given a desired bitmap, finds a datapath group in 'dp_groups'. If it - * doesn't exist, creates a new one and adds it to 'dp_groups'. +struct ovn_dp_group * +ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len) +{ + uint32_t hash; + + hash = hash_int(desired_n, 0); + return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); +} + +/* Creates a new datapath group and adds it to 'dp_groups'. * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. */ + * either taking it directly, or by modifying, if it's not already in use. + * Caller should first call ovn_dp_group_get() before calling this function. */ struct ovn_dp_group * -ovn_dp_group_get_or_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) +ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, + struct hmap *dp_groups, + struct sbrec_logical_dp_group *sb_group, + size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len, + bool is_switch, + const struct ovn_datapaths *ls_datapaths, + const struct ovn_datapaths *lr_datapaths) { struct ovn_dp_group *dpg; - dpg = ovn_dp_group_get(dp_groups, desired_n, desired_bitmap, bitmap_len); - if (dpg) { - return dpg; + bool update_dp_group = false, can_modify = false; + unsigned long *dpg_bitmap; + size_t i, n = 0; + + dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; + for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { + struct ovn_datapath *datapath_od; + + datapath_od = ovn_datapath_from_sbrec( + ls_datapaths ? &ls_datapaths->datapaths : NULL, + lr_datapaths ? &lr_datapaths->datapaths : NULL, + sb_group->datapaths[i]); + if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { + break; + } + bitmap_set1(dpg_bitmap, datapath_od->index); + n++; + } + if (!sb_group || i != sb_group->n_datapaths) { + /* No group or stale group. Not going to be used. */ + update_dp_group = true; + can_modify = true; + } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { + /* The group in Sb is different. */ + update_dp_group = true; + /* We can modify existing group if it's not already in use. */ + can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, + bitmap_len, hash_int(n, 0)); } - return ovn_dp_group_create(ovnsb_txn, dp_groups, sb_group, desired_n, - desired_bitmap, bitmap_len, is_switch, - ls_datapaths, lr_datapaths); + bitmap_free(dpg_bitmap); + + dpg = xzalloc(sizeof *dpg); + dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); + if (!update_dp_group) { + dpg->dp_group = sb_group; + } else { + dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( + ovnsb_txn, + can_modify ? sb_group : NULL, + desired_bitmap, + is_switch ? ls_datapaths : lr_datapaths); + } + dpg->dpg_uuid = dpg->dp_group->header_.uuid; + hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); + + return dpg; } void @@ -914,24 +949,6 @@ ovn_dp_group_find(const struct hmap *dp_groups, return NULL; } -static void -inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) -{ - dpg->refcnt++; -} - -static void -dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) -{ - dpg->refcnt--; - - if (!dpg->refcnt) { - hmap_remove(dp_groups, &dpg->node); - free(dpg->bitmap); - free(dpg); - } -} - static struct sbrec_logical_dp_group * ovn_sb_insert_or_update_logical_dp_group( struct ovsdb_idl_txn *ovnsb_txn, @@ -958,83 +975,6 @@ ovn_sb_insert_or_update_logical_dp_group( return dp_group; } -static struct ovn_dp_group * -ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len) -{ - uint32_t hash; - - hash = hash_int(desired_n, 0); - return ovn_dp_group_find(dp_groups, desired_bitmap, bitmap_len, hash); -} - -/* Creates a new datapath group and adds it to 'dp_groups'. - * If 'sb_group' is provided, function will try to re-use this group by - * either taking it directly, or by modifying, if it's not already in use. - * Caller should first call ovn_dp_group_get() before calling this function. */ -static struct ovn_dp_group * -ovn_dp_group_create(struct ovsdb_idl_txn *ovnsb_txn, - struct hmap *dp_groups, - struct sbrec_logical_dp_group *sb_group, - size_t desired_n, - const unsigned long *desired_bitmap, - size_t bitmap_len, - bool is_switch, - const struct ovn_datapaths *ls_datapaths, - const struct ovn_datapaths *lr_datapaths) -{ - struct ovn_dp_group *dpg; - - bool update_dp_group = false, can_modify = false; - unsigned long *dpg_bitmap; - size_t i, n = 0; - - dpg_bitmap = sb_group ? bitmap_allocate(bitmap_len) : NULL; - for (i = 0; sb_group && i < sb_group->n_datapaths; i++) { - struct ovn_datapath *datapath_od; - - datapath_od = ovn_datapath_from_sbrec( - ls_datapaths ? &ls_datapaths->datapaths : NULL, - lr_datapaths ? &lr_datapaths->datapaths : NULL, - sb_group->datapaths[i]); - if (!datapath_od || ovn_datapath_is_stale(datapath_od)) { - break; - } - bitmap_set1(dpg_bitmap, datapath_od->index); - n++; - } - if (!sb_group || i != sb_group->n_datapaths) { - /* No group or stale group. Not going to be used. */ - update_dp_group = true; - can_modify = true; - } else if (!bitmap_equal(dpg_bitmap, desired_bitmap, bitmap_len)) { - /* The group in Sb is different. */ - update_dp_group = true; - /* We can modify existing group if it's not already in use. */ - can_modify = !ovn_dp_group_find(dp_groups, dpg_bitmap, - bitmap_len, hash_int(n, 0)); - } - - bitmap_free(dpg_bitmap); - - dpg = xzalloc(sizeof *dpg); - dpg->bitmap = bitmap_clone(desired_bitmap, bitmap_len); - if (!update_dp_group) { - dpg->dp_group = sb_group; - } else { - dpg->dp_group = ovn_sb_insert_or_update_logical_dp_group( - ovnsb_txn, - can_modify ? sb_group : NULL, - desired_bitmap, - is_switch ? ls_datapaths : lr_datapaths); - } - dpg->dpg_uuid = dpg->dp_group->header_.uuid; - hmap_insert(dp_groups, &dpg->node, hash_int(desired_n, 0)); - - return dpg; -} - /* Adds an OVN datapath to a datapath group of existing logical flow. * Version to use when hash bucket locking is NOT required or the corresponding * hash lock is already taken. */ diff --git a/northd/lflow-mgr.h b/northd/lflow-mgr.h index 08805435c8..1e0ac20e70 100644 --- a/northd/lflow-mgr.h +++ b/northd/lflow-mgr.h @@ -163,7 +163,10 @@ ovn_dp_groups_init(struct hmap *dp_groups) } void ovn_dp_groups_destroy(struct hmap *dp_groups); -struct ovn_dp_group *ovn_dp_group_get_or_create( +struct ovn_dp_group *ovn_dp_group_get(struct hmap *dp_groups, size_t desired_n, + const unsigned long *desired_bitmap, + size_t bitmap_len); +struct ovn_dp_group *ovn_dp_group_create( struct ovsdb_idl_txn *ovnsb_txn, struct hmap *dp_groups, struct sbrec_logical_dp_group *sb_group, size_t desired_n, const unsigned long *desired_bitmap, @@ -171,4 +174,22 @@ struct ovn_dp_group *ovn_dp_group_get_or_create( const struct ovn_datapaths *ls_datapaths, const struct ovn_datapaths *lr_datapaths); +static inline void +inc_ovn_dp_group_ref(struct ovn_dp_group *dpg) +{ + dpg->refcnt++; +} + +static inline void +dec_ovn_dp_group_ref(struct hmap *dp_groups, struct ovn_dp_group *dpg) +{ + dpg->refcnt--; + + if (!dpg->refcnt) { + hmap_remove(dp_groups, &dpg->node); + free(dpg->bitmap); + free(dpg); + } +} + #endif /* LFLOW_MGR_H */ \ No newline at end of file diff --git a/northd/northd.c b/northd/northd.c index b6e24f94d8..21c99753fd 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -3794,244 +3794,6 @@ build_lb_port_related_data( build_lswitch_lbs_from_lrouter(lr_datapaths, lb_dps_map, lb_group_dps_map); } -struct sb_lb { - struct hmap_node hmap_node; - - const struct sbrec_load_balancer *slb; - struct ovn_dp_group *dpg; - struct ovn_dp_group *lr_dpg; - struct uuid lb_uuid; -}; - -static struct sb_lb * -find_slb_in_sb_lbs(struct hmap *sb_lbs, const struct uuid *lb_uuid) -{ - struct sb_lb *sb_lb; - HMAP_FOR_EACH_WITH_HASH (sb_lb, hmap_node, uuid_hash(lb_uuid), sb_lbs) { - if (uuid_equals(&sb_lb->lb_uuid, lb_uuid)) { - return sb_lb; - } - } - - return NULL; -} - -/* Syncs relevant load balancers (applied to logical switches) to the - * Southbound database. - */ -void -sync_lbs(struct ovsdb_idl_txn *ovnsb_txn, - const struct sbrec_load_balancer_table *sbrec_load_balancer_table, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lb_dps_map, - struct chassis_features *chassis_features) -{ - struct hmap ls_dp_groups = HMAP_INITIALIZER(&ls_dp_groups); - struct hmap lr_dp_groups = HMAP_INITIALIZER(&lr_dp_groups); - struct ovn_lb_datapaths *lb_dps; - struct hmap sb_lbs = HMAP_INITIALIZER(&sb_lbs); - - /* Delete any stale SB load balancer rows and create datapath - * groups for existing ones. */ - struct hmapx existing_lbs = HMAPX_INITIALIZER(&existing_lbs); - const struct sbrec_load_balancer *sbrec_lb; - SBREC_LOAD_BALANCER_TABLE_FOR_EACH_SAFE (sbrec_lb, - sbrec_load_balancer_table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - struct uuid lb_uuid; - if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - /* Delete any SB load balancer entries that refer to NB load balancers - * that don't exist anymore or are not applied to switches/routers - * anymore. - * - * There is also a special case in which duplicate LBs might be created - * in the SB, e.g., due to the fact that OVSDB only ensures - * "at-least-once" consistency for clustered database tables that - * are not indexed in any way. - */ - lb_dps = ovn_lb_datapaths_find(lb_dps_map, &lb_uuid); - if (!lb_dps || (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) || - !hmapx_add(&existing_lbs, lb_dps)) { - sbrec_load_balancer_delete(sbrec_lb); - continue; - } - - struct sb_lb *sb_lb = xzalloc(sizeof *sb_lb); - sb_lb->lb_uuid = lb_uuid; - sb_lb->slb = sbrec_lb; - hmap_insert(&sb_lbs, &sb_lb->hmap_node, uuid_hash(&lb_uuid)); - - /* Find or create datapath group for this load balancer. */ - if (lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sb_lb->slb->ls_datapath_group - : sb_lb->slb->datapath_group; /* deprecated */ - sb_lb->dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (lb_dps->n_nb_lr) { - sb_lb->lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sb_lb->slb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - } - hmapx_destroy(&existing_lbs); - - /* Create SB Load balancer records if not present and sync - * the SB load balancer columns. */ - HMAP_FOR_EACH (lb_dps, hmap_node, lb_dps_map) { - - if (!lb_dps->n_nb_ls && !lb_dps->n_nb_lr) { - continue; - } - - /* Store the fact that northd provides the original (destination IP + - * transport port) tuple. - */ - struct smap options; - smap_clone(&options, &lb_dps->lb->nlb->options); - smap_replace(&options, "hairpin_orig_tuple", "true"); - - struct sb_lb *sb_lb = find_slb_in_sb_lbs(&sb_lbs, - &lb_dps->lb->nlb->header_.uuid); - ovs_assert(!sb_lb || (sb_lb->slb && (sb_lb->dpg || sb_lb->lr_dpg))); - struct ovn_dp_group *lb_dpg = NULL, *lb_lr_dpg = NULL; - if (!sb_lb) { - sbrec_lb = sbrec_load_balancer_insert(ovnsb_txn); - char *lb_id = xasprintf( - UUID_FMT, UUID_ARGS(&lb_dps->lb->nlb->header_.uuid)); - const struct smap external_ids = - SMAP_CONST1(&external_ids, "lb_id", lb_id); - sbrec_load_balancer_set_external_ids(sbrec_lb, &external_ids); - free(lb_id); - } else { - sbrec_lb = sb_lb->slb; - lb_dpg = sb_lb->dpg; - lb_lr_dpg = sb_lb->lr_dpg; - } - - /* Find or create datapath group for this load balancer. */ - if (!lb_dpg && lb_dps->n_nb_ls) { - struct sbrec_logical_dp_group *ls_datapath_group - = chassis_features->ls_dpg_column - ? sbrec_lb->ls_datapath_group - : sbrec_lb->datapath_group; /* deprecated */ - lb_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &ls_dp_groups, - ls_datapath_group, - lb_dps->n_nb_ls, lb_dps->nb_ls_map, - ods_size(ls_datapaths), true, - ls_datapaths, NULL); - } - if (!lb_lr_dpg && lb_dps->n_nb_lr) { - lb_lr_dpg = ovn_dp_group_get_or_create( - ovnsb_txn, &lr_dp_groups, - sbrec_lb->lr_datapath_group, - lb_dps->n_nb_lr, lb_dps->nb_lr_map, - ods_size(lr_datapaths), false, - NULL, lr_datapaths); - } - - /* Update columns. */ - sbrec_load_balancer_set_name(sbrec_lb, lb_dps->lb->nlb->name); - sbrec_load_balancer_set_vips(sbrec_lb, - ovn_northd_lb_get_vips(lb_dps->lb)); - sbrec_load_balancer_set_protocol(sbrec_lb, lb_dps->lb->nlb->protocol); - - if (chassis_features->ls_dpg_column) { - sbrec_load_balancer_set_ls_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_datapath_group(sbrec_lb, NULL); - } else { - /* datapath_group column is deprecated. */ - sbrec_load_balancer_set_ls_datapath_group(sbrec_lb, NULL); - sbrec_load_balancer_set_datapath_group( - sbrec_lb, lb_dpg ? lb_dpg->dp_group : NULL - ); - } - - sbrec_load_balancer_set_lr_datapath_group( - sbrec_lb, lb_lr_dpg ? lb_lr_dpg->dp_group : NULL - ); - sbrec_load_balancer_set_options(sbrec_lb, &options); - /* Clearing 'datapaths' column, since 'dp_group' is in use. */ - sbrec_load_balancer_set_datapaths(sbrec_lb, NULL, 0); - smap_destroy(&options); - } - - struct ovn_dp_group *dpg; - HMAP_FOR_EACH_POP (dpg, node, &ls_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&ls_dp_groups); - - HMAP_FOR_EACH_POP (dpg, node, &lr_dp_groups) { - bitmap_free(dpg->bitmap); - free(dpg); - } - hmap_destroy(&lr_dp_groups); - - struct sb_lb *sb_lb; - HMAP_FOR_EACH_POP (sb_lb, hmap_node, &sb_lbs) { - free(sb_lb); - } - hmap_destroy(&sb_lbs); - - /* Datapath_Binding.load_balancers is not used anymore, it's still in the - * schema for compatibility reasons. Reset it to empty, just in case. - */ - struct ovn_datapath *od; - HMAP_FOR_EACH (od, key_node, &ls_datapaths->datapaths) { - ovs_assert(od->nbs); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } - HMAP_FOR_EACH (od, key_node, &lr_datapaths->datapaths) { - ovs_assert(od->nbr); - - if (od->sb->n_load_balancers) { - sbrec_datapath_binding_set_load_balancers(od->sb, NULL, 0); - } - } -} - -bool -check_sb_lb_duplicates(const struct sbrec_load_balancer_table *table) -{ - struct sset existing_nb_lb_uuids = - SSET_INITIALIZER(&existing_nb_lb_uuids); - const struct sbrec_load_balancer *sbrec_lb; - bool duplicates = false; - - SBREC_LOAD_BALANCER_TABLE_FOR_EACH (sbrec_lb, table) { - const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); - if (nb_lb_uuid && !sset_add(&existing_nb_lb_uuids, nb_lb_uuid)) { - duplicates = true; - break; - } - } - - sset_destroy(&existing_nb_lb_uuids); - return duplicates; -} - /* Syncs the SB port binding for the ovn_port 'op' of a logical switch port. * Caller should make sure that the OVN SB IDL txn is not NULL. Presently it * only syncs the nat column of port binding corresponding to the 'op->nbsp' */ diff --git a/northd/northd.h b/northd/northd.h index d1f9d95ee7..920f1f43f2 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -746,12 +746,6 @@ void run_update_worker_pool(int n_threads); const struct ovn_datapath *northd_get_datapath_for_port( const struct hmap *ls_ports, const char *port_name); -void sync_lbs(struct ovsdb_idl_txn *, const struct sbrec_load_balancer_table *, - struct ovn_datapaths *ls_datapaths, - struct ovn_datapaths *lr_datapaths, - struct hmap *lbs, - struct chassis_features *chassis_features); -bool check_sb_lb_duplicates(const struct sbrec_load_balancer_table *); struct lr_lb_nat_data_table; void sync_pbs(struct ovsdb_idl_txn *, struct hmap *ls_ports, diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index ba30f4b84c..5180c353b3 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -2930,6 +2930,8 @@ sw1_sb_uuid=$(fetch_column datapath_binding _uuid external_ids:name=sw1) echo "$sw0_sb_uuid" > sw_sb_uuids echo "$sw1_sb_uuid" >> sw_sb_uuids +lb0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lb0) + echo echo "__file__:__line__: Check that SB lb0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -2937,6 +2939,8 @@ AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Gr $(cat sw_sb_uuids | sort) ]) +lbg0_dp_group=$(fetch_column sb:load_balancer ls_datapath_group name=lbg0) + echo echo "__file__:__line__: Check that SB lbg0 has sw0 and sw1 in datapaths column." AT_CHECK_UNQUOTED([ovn-sbctl --bare --columns _uuid,datapaths find Logical_DP_Group dnl @@ -10402,7 +10406,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10411,7 +10415,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10421,7 +10425,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10430,7 +10434,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10440,7 +10444,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10498,7 +10502,7 @@ check_engine_stats lr_lb_nat_data norecompute compute # A LB applied to a switch/router triggers: # - a recompute in the first iteration (handling northd change) # - a compute in the second iteration (handling SB update) -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10511,7 +10515,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10522,7 +10526,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10533,7 +10537,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10544,7 +10548,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Disassociate lb1 from sw0. There should be a full recompute of northd engine node. @@ -10567,7 +10571,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10588,7 +10592,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10598,7 +10602,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10608,7 +10612,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10618,7 +10622,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10628,7 +10632,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10646,6 +10650,7 @@ lbg1_uuid=$(ovn-nbctl --wait=sb create load_balancer_group name=lbg1) check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10659,7 +10664,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10676,7 +10681,7 @@ check ovn-nbctl --wait=sb add load_balancer_group . load_Balancer $lb1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10686,7 +10691,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Update lb and this should not result in northd recompute check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10696,7 +10701,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute # Modify the backend of the lb1 vip check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10706,7 +10711,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10717,7 +10722,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10728,7 +10733,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10739,7 +10744,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10755,7 +10760,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Modify the backend of the lb1 vip @@ -10765,7 +10770,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Cleanup the vip of lb1. @@ -10775,7 +10780,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Set the vips of lb1 back @@ -10785,7 +10790,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Add another vip to lb1 @@ -10795,7 +10800,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10815,7 +10820,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10859,7 +10864,7 @@ check_engine_stats northd norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10869,7 +10874,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10878,7 +10883,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10888,7 +10893,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10898,7 +10903,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10908,7 +10913,7 @@ check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -10939,7 +10944,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Deleting lb2 should result in lflow recompute as it is @@ -10951,7 +10956,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats ls_lbacls norecompute compute check_engine_stats lflow norecompute compute -check_engine_stats sync_to_sb_lb recompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11018,6 +11023,7 @@ check ovn-nbctl --wait=sb ls-lb-add sw0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Clear the VIPs of lb1 @@ -11026,6 +11032,7 @@ check ovn-nbctl --wait=sb clear load_balancer . vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lflow norecompute compute +check_engine_stats sync_to_sb_lb norecompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats @@ -11033,6 +11040,7 @@ check ovn-nbctl --wait=sb lb-del lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute check_engine_stats lflow recompute nocompute +check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE AT_CLEANUP @@ -11093,6 +11101,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11106,6 +11115,7 @@ check_engine_stats northd recompute compute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11118,6 +11128,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11144,6 +11155,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11156,6 +11168,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT options column @@ -11165,6 +11178,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT external_ip column @@ -11175,6 +11189,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT logical_ip column @@ -11185,6 +11200,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Update the NAT type @@ -11195,6 +11211,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a dnat_and_snat NAT with external_mac and logical_port @@ -11205,6 +11222,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE nat2_uuid=$(ovn-nbctl --bare --columns _uuid find nat logical_ip=10.0.0.4) @@ -11216,6 +11234,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create a load balancer and add the lb vip as NAT @@ -11232,6 +11251,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11243,6 +11263,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11254,6 +11275,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11265,6 +11287,7 @@ check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11276,6 +11299,7 @@ check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb norecompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE # Create router Policy @@ -11285,6 +11309,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11294,6 +11319,7 @@ check_engine_stats northd recompute nocompute check_engine_stats lr_nat recompute nocompute check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats sync_to_sb_pb recompute nocompute +check_engine_stats sync_to_sb_lb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE