From patchwork Tue Oct 24 00:49:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1854087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4SDtl66hjHz202k for ; Tue, 24 Oct 2023 11:50:26 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D6163614EE; Tue, 24 Oct 2023 00:50:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D6163614EE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Fe1szkNNeC4K; Tue, 24 Oct 2023 00:50:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 01002614F7; Tue, 24 Oct 2023 00:50:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 01002614F7 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 922E5C0DD9; Tue, 24 Oct 2023 00:50:11 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 495D4C0E0D for ; Tue, 24 Oct 2023 00:50:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D6B9C614DB for ; Tue, 24 Oct 2023 00:50:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D6B9C614DB X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jLgHulbTCV1T for ; Tue, 24 Oct 2023 00:50:04 +0000 (UTC) Received: from relay7-d.mail.gandi.net (relay7-d.mail.gandi.net [217.70.183.200]) by smtp3.osuosl.org (Postfix) with ESMTPS id 18CEC614D1 for ; Tue, 24 Oct 2023 00:50:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 18CEC614D1 Received: by mail.gandi.net (Postfix) with ESMTPSA id 6A76420002; Tue, 24 Oct 2023 00:50:01 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 23 Oct 2023 20:49:47 -0400 Message-ID: <20231024004947.4133992-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231024004244.4133452-1-numans@ovn.org> References: <20231024004244.4133452-1-numans@ovn.org> MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v1 14/18] northd: Add lr_lb_nat_data handler for lflow engine node. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique Signed-off-by: Numan Siddique --- northd/en-lflow.c | 30 ++++ northd/en-lflow.h | 1 + northd/en-lr-lb-nat-data.c | 41 ++++- northd/en-lr-lb-nat-data.h | 7 + northd/inc-proc-northd.c | 5 +- northd/northd.c | 356 ++++++++++++++++++++----------------- northd/northd.h | 7 + tests/ovn-northd.at | 48 ++--- 8 files changed, 305 insertions(+), 190 deletions(-) diff --git a/northd/en-lflow.c b/northd/en-lflow.c index 4400460482..613351eba2 100644 --- a/northd/en-lflow.c +++ b/northd/en-lflow.c @@ -162,6 +162,36 @@ lflow_port_group_handler(struct engine_node *node, void *data OVS_UNUSED) return true; } +bool +lflow_lr_lb_nat_data_handler(struct engine_node *node, void *data) +{ + struct ed_type_lr_lb_nat_data *lr_lb_nat_data = + engine_get_input_data("lr_lb_nat_data", node); + + if (!lr_lb_nat_data->tracked + || lr_lb_nat_data->tracked_data.vip_nats_changed + || !hmapx_is_empty(&lr_lb_nat_data->tracked_data.deleted)) { + return false; + } + + const struct engine_context *eng_ctx = engine_get_context(); + struct lflow_data *lflow_data = data; + + struct lflow_input lflow_input; + lflow_get_input_data(node, &lflow_input); + + if (!lflow_handle_lr_lb_nat_data_changes(eng_ctx->ovnsb_idl_txn, + &lr_lb_nat_data->tracked_data, + &lflow_input, + lflow_data->lflow_table)) { + return false; + } + + engine_set_node_state(node, EN_UPDATED); + + return true; +} + void *en_lflow_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) { diff --git a/northd/en-lflow.h b/northd/en-lflow.h index f7325c56b1..adbd8767c9 100644 --- a/northd/en-lflow.h +++ b/northd/en-lflow.h @@ -20,5 +20,6 @@ void *en_lflow_init(struct engine_node *node, struct engine_arg *arg); void en_lflow_cleanup(void *data); bool lflow_northd_handler(struct engine_node *, void *data); bool lflow_port_group_handler(struct engine_node *, void *data); +bool lflow_lr_lb_nat_data_handler(struct engine_node *, void *data); #endif /* EN_LFLOW_H */ diff --git a/northd/en-lr-lb-nat-data.c b/northd/en-lr-lb-nat-data.c index 7987799ca2..dc9a66a0a0 100644 --- a/northd/en-lr-lb-nat-data.c +++ b/northd/en-lr-lb-nat-data.c @@ -39,6 +39,7 @@ #include "lib/ovn-sb-idl.h" #include "lib/ovn-util.h" #include "lib/stopwatch-names.h" +#include "lflow-mgr.h" #include "northd.h" VLOG_DEFINE_THIS_MODULE(en_lr_lb_nat_data); @@ -77,7 +78,7 @@ static void remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *, enum lb_neighbor_responder_mode, const struct sset *lb_ips_v4, const struct sset *lb_ips_v6); -static void lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *); +static bool lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *); const struct lr_lb_nat_data_record * lr_lb_nat_data_table_find( @@ -123,6 +124,7 @@ en_lr_lb_nat_data_clear_tracked_data(void *data_) } hmapx_clear(&data->tracked_data.crupdated); + data->tracked_data.vip_nats_changed = false; data->tracked = false; } @@ -189,7 +191,6 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) const struct lr_nat_record *lrnat_rec = lr_nat_table_find( input_data.lr_nats, od->nbr); ovs_assert(lrnat_rec); - lr_lbnat_rec = lr_lb_nat_data_record_create(&data->lr_lbnats, lrnat_rec, input_data.lb_datapaths_map, @@ -197,6 +198,10 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) /* Add the lr_lbnat_rec rec to the tracking data. */ hmapx_add(&data->tracked_data.crupdated, lr_lbnat_rec); + + if (!sset_is_empty(&lr_lbnat_rec->vip_nats)) { + data->tracked_data.vip_nats_changed = true; + } continue; } @@ -305,7 +310,9 @@ lr_lb_nat_data_lb_data_handler(struct engine_node *node, void *data_) * vip nats and re-evaluate 'has_lb_vip'. */ HMAPX_FOR_EACH (hmapx_node, &data->tracked_data.crupdated) { lr_lbnat_rec = hmapx_node->data; - lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + if (lr_lb_nat_data_build_vip_nats(lr_lbnat_rec)) { + data->tracked_data.vip_nats_changed = true; + } lr_lbnat_rec->has_lb_vip = od_has_lb_vip(lr_lbnat_rec->od); } @@ -344,8 +351,13 @@ lr_lb_nat_data_lr_nat_handler(struct engine_node *node, void *data_) lrnat_rec, input_data.lb_datapaths_map, input_data.lbgrp_datapaths_map); + if (!sset_is_empty(&lr_lbnat_rec->vip_nats)) { + data->tracked_data.vip_nats_changed = true; + } } else { - lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); + if (lr_lb_nat_data_build_vip_nats(lr_lbnat_rec)) { + data->tracked_data.vip_nats_changed = true; + } } /* Add the lr_lbnat_rec rec to the tracking data. */ @@ -424,6 +436,7 @@ lr_lb_nat_data_record_create(struct lr_lb_nat_data_table *table, lr_lbnat_rec->od = lrnat_rec->od; lr_lb_nat_data_record_init(lr_lbnat_rec, lb_datapaths_map, lbgrp_datapaths_map); + lr_lbnat_rec->lflow_ref = lflow_ref_alloc(lr_lbnat_rec->od->nbr->name); hmap_insert(&table->entries, &lr_lbnat_rec->key_node, uuid_hash(&lr_lbnat_rec->od->nbr->header_.uuid)); @@ -437,6 +450,7 @@ lr_lb_nat_data_record_destroy(struct lr_lb_nat_data_record *lr_lbnat_rec) ovn_lb_ip_set_destroy(lr_lbnat_rec->lb_ips); lr_lbnat_rec->lb_ips = NULL; sset_destroy(&lr_lbnat_rec->vip_nats); + lflow_ref_destroy(lr_lbnat_rec->lflow_ref); free(lr_lbnat_rec); } @@ -503,7 +517,7 @@ lr_lb_nat_data_record_init(struct lr_lb_nat_data_record *lr_lbnat_rec, sset_init(&lr_lbnat_rec->vip_nats); - if (!nbr->n_nat) { + if (nbr->n_nat) { lr_lb_nat_data_build_vip_nats(lr_lbnat_rec); } @@ -617,10 +631,13 @@ remove_lrouter_lb_reachable_ips(struct lr_lb_nat_data_record *lr_lbnat_rec, } } -static void +/* Builds the vip nats and returns true if the lr_lbnat_rec->vip_nats + * changed, false otherwise. */ +static bool lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *lr_lbnat_rec) { - sset_clear(&lr_lbnat_rec->vip_nats); + struct sset old_vip_nats = SSET_INITIALIZER(&old_vip_nats); + sset_swap(&lr_lbnat_rec->vip_nats, &old_vip_nats); const char *external_ip; SSET_FOR_EACH (external_ip, &lr_lbnat_rec->lrnat_rec->external_ips) { bool is_vip_nat = false; @@ -636,4 +653,14 @@ lr_lb_nat_data_build_vip_nats(struct lr_lb_nat_data_record *lr_lbnat_rec) sset_add(&lr_lbnat_rec->vip_nats, external_ip); } } + + bool vip_nats_changed = + sset_count(&lr_lbnat_rec->vip_nats) != sset_count(&old_vip_nats); + if (!vip_nats_changed) { + vip_nats_changed = !sset_equals(&lr_lbnat_rec->vip_nats, + &old_vip_nats); + } + sset_destroy(&old_vip_nats); + + return vip_nats_changed; } diff --git a/northd/en-lr-lb-nat-data.h b/northd/en-lr-lb-nat-data.h index 759d055f05..7acf67b731 100644 --- a/northd/en-lr-lb-nat-data.h +++ b/northd/en-lr-lb-nat-data.h @@ -32,6 +32,7 @@ struct ovn_datapath; struct lr_nat_record; +struct lflow_ref; struct lr_lb_nat_data_record { struct hmap_node key_node; /* Index on 'nbr->header_.uuid'. */ @@ -46,6 +47,8 @@ struct lr_lb_nat_data_record { /* sset of vips which are also part of lr nats. */ struct sset vip_nats; + + struct lflow_ref *lflow_ref; }; struct lr_lb_nat_data_table { @@ -65,6 +68,10 @@ struct lr_lb_nat_data_tracked_data { /* Deleted logical router with LB data. */ struct hmapx deleted; /* Stores 'struct lr_lb_nat_data_record'. */ + + /* Indicates if any router's NATs changed which were also LB vips + * or vice versa. */ + bool vip_nats_changed; }; struct ed_type_lr_lb_nat_data { diff --git a/northd/inc-proc-northd.c b/northd/inc-proc-northd.c index 3cedd502cb..955b5e2ed1 100644 --- a/northd/inc-proc-northd.c +++ b/northd/inc-proc-northd.c @@ -236,10 +236,11 @@ void inc_proc_northd_init(struct ovsdb_idl_loop *nb, engine_add_input(&en_lflow, &en_sb_multicast_group, NULL); engine_add_input(&en_lflow, &en_sb_igmp_group, NULL); engine_add_input(&en_lflow, &en_sb_logical_dp_group, NULL); + engine_add_input(&en_lflow, &en_ls_lbacls, NULL); engine_add_input(&en_lflow, &en_northd, lflow_northd_handler); engine_add_input(&en_lflow, &en_port_group, lflow_port_group_handler); - engine_add_input(&en_lflow, &en_lr_lb_nat_data, NULL); - engine_add_input(&en_lflow, &en_ls_lbacls, NULL); + engine_add_input(&en_lflow, &en_lr_lb_nat_data, + lflow_lr_lb_nat_data_handler); engine_add_input(&en_sync_to_sb_addr_set, &en_nb_address_set, sync_to_sb_addr_set_nb_address_set_handler); diff --git a/northd/northd.c b/northd/northd.c index 1033bc1a8f..5c12194454 100644 --- a/northd/northd.c +++ b/northd/northd.c @@ -1227,7 +1227,7 @@ ovn_port_create(struct hmap *ports, const char *key, op->lflow_ref = lflow_ref_alloc(key); op->lbnat_lflow_ref = lflow_ref_alloc(key); - + op->routable_lflow_ref = lflow_ref_alloc(key); return op; } @@ -1276,6 +1276,7 @@ ovn_port_destroy_orphan(struct ovn_port *port) free(port->key); lflow_ref_destroy(port->lflow_ref); lflow_ref_destroy(port->lbnat_lflow_ref); + lflow_ref_destroy(port->routable_lflow_ref); free(port); } @@ -12827,62 +12828,6 @@ build_ip_routing_flows_for_lrp( } } -/* Logical router ingress table IP_ROUTING : IP Routing. - * - * For the LSP 'op' of type router, if there are logical router ports other - * than the LSP's peer connected to the logical switch, then for routable - * addresses (such as NAT IPs, LB VIPs, etc.) on each of the connected router - * ports, add routes to the LSP's peer router. - */ -static void -build_ip_routing_flows_for_router_type_lsp( - struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) -{ - ovs_assert(op->nbsp); - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp || !peer->lrp_networks.n_ipv4_addrs - || !op->od->n_router_ports) { - return; - } - - for (int i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = ovn_port_get_peer( - lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp || router_port == peer) { - continue; - } - - const struct lr_lb_nat_data_record *lr_lbnat_rec = - lr_lb_nat_data_table_find(lr_lbnats, router_port->od->nbr); - - if (router_port->nbrp->ha_chassis_group || - router_port->nbrp->n_gateway_chassis) { - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_lbnat_rec); - for (size_t j = 0; j < ra.n_addrs; j++) { - struct lport_addresses *laddrs = &ra.laddrs[j]; - for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { - add_route(lflows, peer->od, peer, - peer->lrp_networks.ipv4_addrs[0].addr_s, - laddrs->ipv4_addrs[k].network_s, - laddrs->ipv4_addrs[k].plen, NULL, false, 0, - &peer->nbrp->header_, false, - ROUTE_PRIO_OFFSET_CONNECTED, - lflow_ref); - } - } - destroy_routable_addresses(&ra); - } - } - -} - static void build_static_route_flows_for_lrouter( struct ovn_datapath *od, const struct chassis_features *features, @@ -13124,42 +13069,6 @@ build_arp_resolve_flows_for_lrouter( lflow_ref); } -static void -routable_addresses_to_lflows(struct lflow_table *lflows, - struct ovn_port *router_port, - struct ovn_port *peer, - const struct lr_lb_nat_data_record *lr_lbnat_rec, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - struct ovn_port_routable_addresses ra = - get_op_routable_addresses(router_port, lr_lbnat_rec); - if (!ra.n_addrs) { - return; - } - - for (size_t i = 0; i < ra.n_addrs; i++) { - ds_clear(match); - ds_put_format(match, "outport == %s && "REG_NEXT_HOP_IPV4" == {", - peer->json_key); - bool first = true; - for (size_t j = 0; j < ra.laddrs[i].n_ipv4_addrs; j++) { - if (!first) { - ds_put_cstr(match, ", "); - } - ds_put_cstr(match, ra.laddrs[i].ipv4_addrs[j].addr_s); - first = false; - } - ds_put_cstr(match, "}"); - - ds_clear(actions); - ds_put_format(actions, "eth.dst = %s; next;", ra.laddrs[i].ea_s); - ovn_lflow_add(lflows, peer->od, S_ROUTER_IN_ARP_RESOLVE, 100, - ds_cstr(match), ds_cstr(actions), lflow_ref); - } - destroy_routable_addresses(&ra); -} - /* Local router ingress table ARP_RESOLVE: ARP Resolution. * * Any unicast packet that reaches this table is an IP packet whose @@ -13402,52 +13311,6 @@ build_arp_resolve_flows_for_lsp( } } -static void -build_arp_resolve_flows_for_lsp_routable_addresses( - struct ovn_port *op, struct lflow_table *lflows, - const struct hmap *lr_ports, - const struct lr_lb_nat_data_table *lr_lbnats, - struct ds *match, struct ds *actions, - struct lflow_ref *lflow_ref) -{ - if (!lsp_is_router(op->nbsp)) { - return; - } - - struct ovn_port *peer = ovn_port_get_peer(lr_ports, op); - if (!peer || !peer->nbrp) { - return; - } - - if (peer->od->nbr && - smap_get_bool(&peer->od->nbr->options, - "dynamic_neigh_routers", false)) { - return; - } - - for (size_t i = 0; i < op->od->n_router_ports; i++) { - struct ovn_port *router_port = - ovn_port_get_peer(lr_ports, op->od->router_ports[i]); - if (!router_port || !router_port->nbrp) { - continue; - } - - /* Skip the router port under consideration. */ - if (router_port == peer) { - continue; - } - - if (smap_get(&peer->od->nbr->options, "chassis") || peer->cr_port) { - const struct lr_lb_nat_data_record *lr_lbnat_rec; - lr_lbnat_rec = lr_lb_nat_data_table_find(lr_lbnats, - router_port->od->nbr); - routable_addresses_to_lflows(lflows, router_port, peer, - lr_lbnat_rec, match, actions, - lflow_ref); - } - } -} - static void build_icmperr_pkt_big_flows(struct ovn_port *op, int mtu, struct lflow_table *lflows, @@ -15664,8 +15527,6 @@ static void build_lsp_lflows_for_lbnats(struct ovn_port *lsp, struct ovn_port *lrp_peer, const struct lr_lb_nat_data_record *lr_lbnat_rec, - const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct lflow_table *lflows, struct ds *match, struct ds *actions, @@ -15675,19 +15536,101 @@ build_lsp_lflows_for_lbnats(struct ovn_port *lsp, build_lswitch_rport_arp_req_flows_for_lbnats( lrp_peer, lr_lbnat_rec, lsp->od, lsp, lflows, &lsp->nbsp->header_, lflow_ref); - build_ip_routing_flows_for_router_type_lsp(lsp, lr_lbnats, - lr_ports, lflows, - lflow_ref); - build_arp_resolve_flows_for_lsp_routable_addresses( - lsp, lflows, lr_ports, lr_lbnats, match, actions, lflow_ref); build_lswitch_ip_unicast_lookup_for_nats(lsp, lr_lbnat_rec, lflows, match, actions, lflow_ref); } +/* Logical router ingress table IP_ROUTING : IP Routing. + * + * For the LRP 'lrp's peer's logical switch, if there are logical router + * ports ('peer_lrp's) other than the 'lrp', then for routable addresses + * (such as NAT IPs, LB VIPs, etc.) of the 'lrp' add routes to the + * peer_lrp's datapath. + */ +static void +build_routable_flows_for_router_port( + struct ovn_port *lrp, const struct lr_lb_nat_data_record *lr_lbnat_rec, + struct lflow_table *lflows, + struct ds *match, + struct ds *actions, + struct lflow_ref *lflow_ref) +{ + ovs_assert(lrp->nbrp && lrp->od == lr_lbnat_rec->od); + + struct ovn_port *lsp_peer = lrp->peer; + if (!lsp_peer || !lsp_peer->nbsp) { + return; + } + + struct ovn_datapath *peer_ls = lsp_peer->od; + ovs_assert(peer_ls->nbs); + + struct ovn_port_routable_addresses ra = + get_op_routable_addresses(lrp, lr_lbnat_rec); + + struct ovn_port *router_port; + + for (size_t i = 0; i < peer_ls->n_router_ports; i++) { + router_port = peer_ls->router_ports[i]->peer; + + if (router_port == lrp) { + continue; + } + + if (lrp->nbrp->ha_chassis_group || + lrp->nbrp->n_gateway_chassis) { + for (size_t j = 0; j < ra.n_addrs; j++) { + struct lport_addresses *laddrs = &ra.laddrs[j]; + for (size_t k = 0; k < laddrs->n_ipv4_addrs; k++) { + add_route(lflows, router_port->od, router_port, + router_port->lrp_networks.ipv4_addrs[0].addr_s, + laddrs->ipv4_addrs[k].network_s, + laddrs->ipv4_addrs[k].plen, NULL, false, 0, + &router_port->nbrp->header_, false, + ROUTE_PRIO_OFFSET_CONNECTED, + lflow_ref); + } + } + } + + bool dynamic_neigh_router = + smap_get_bool(&router_port->od->nbr->options, + "dynamic_neigh_routers", false); + + if (!dynamic_neigh_router && + (router_port->od->is_gw_router || router_port->cr_port)) { + + for (size_t k = 0; k < ra.n_addrs; k++) { + ds_clear(match); + ds_put_format(match, "outport == %s && " + REG_NEXT_HOP_IPV4" == {", + router_port->json_key); + bool first = true; + for (size_t j = 0; j < ra.laddrs[k].n_ipv4_addrs; j++) { + if (!first) { + ds_put_cstr(match, ", "); + } + ds_put_cstr(match, ra.laddrs[k].ipv4_addrs[j].addr_s); + first = false; + } + ds_put_cstr(match, "}"); + + ds_clear(actions); + ds_put_format(actions, "eth.dst = %s; next;", + ra.laddrs[k].ea_s); + ovn_lflow_add(lflows, router_port->od, S_ROUTER_IN_ARP_RESOLVE, + 100, ds_cstr(match), ds_cstr(actions), + lflow_ref); + } + } + } + + destroy_routable_addresses(&ra); +} + static void build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, const struct lr_lb_nat_data_table *lr_lbnats, - const struct hmap *lr_ports, struct ds *match, struct ds *actions, struct lflow_table *lflows, @@ -15704,8 +15647,8 @@ build_lbnat_lflows_iterate_by_lsp(struct ovn_port *op, ovs_assert(lr_lbnat_rec); build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, - lr_lbnats, lr_ports, lflows, - match, actions, lflow_ref); + lflows,match, actions, + lflow_ref); } static void @@ -15755,7 +15698,8 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, struct ds *match, struct ds *actions, struct lflow_table *lflows, - struct lflow_ref *lflow_ref) + struct lflow_ref *lflow_ref, + struct lflow_ref *routable_lflow_ref) { ovs_assert(op->nbrp); @@ -15765,6 +15709,9 @@ build_lbnat_lflows_iterate_by_lrp(struct ovn_port *op, build_lrp_lflows_for_lbnats(op, lr_lbnat_rec, meter_groups, match, actions, lflows, lflow_ref); + + build_routable_flows_for_router_port(op, lr_lbnat_rec, lflows, match, + actions, routable_lflow_ref); } static void @@ -15779,15 +15726,15 @@ build_lr_lbnat_data_flows(const struct lr_lb_nat_data_record *lr_lbnat_rec, { build_lrouter_nat_defrag_and_lb(lr_lbnat_rec, lflows, ls_ports, lr_ports, match, actions, meter_groups, features, - NULL); + lr_lbnat_rec->lflow_ref); build_lr_gateway_redirect_flows_for_nats(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, match, actions, - NULL); + lr_lbnat_rec->lflow_ref); build_lrouter_arp_nd_for_datapath(lr_lbnat_rec->od, lr_lbnat_rec->lrnat_rec, lflows, meter_groups, - NULL); + lr_lbnat_rec->lflow_ref); } static void @@ -16019,7 +15966,6 @@ build_lflows_thread(void *arg) &lsi->actions, lsi->lflows); build_lbnat_lflows_iterate_by_lsp(op, lsi->lr_lbnats, - lsi->lr_ports, &lsi->match, &lsi->actions, lsi->lflows, @@ -16041,7 +15987,8 @@ build_lflows_thread(void *arg) &lsi->match, &lsi->actions, lsi->lflows, - op->lbnat_lflow_ref); + op->lbnat_lflow_ref, + op->routable_lflow_ref); } } for (bnum = control->id; @@ -16272,9 +16219,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows); - build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, lsi.lr_ports, - &lsi.match, &lsi.actions, - lsi.lflows, + build_lbnat_lflows_iterate_by_lsp(op, lsi.lr_lbnats, &lsi.match, + &lsi.actions, lsi.lflows, op->lbnat_lflow_ref); } HMAP_FOR_EACH (op, key_node, lr_ports) { @@ -16284,7 +16230,8 @@ build_lswitch_and_lrouter_flows(const struct ovn_datapaths *ls_datapaths, &lsi.match, &lsi.actions, lsi.lflows, - op->lbnat_lflow_ref); + op->lbnat_lflow_ref, + op->routable_lflow_ref); } stopwatch_stop(LFLOWS_PORTS_STOPWATCH_NAME, time_msec()); stopwatch_start(LFLOWS_LBS_STOPWATCH_NAME, time_msec()); @@ -16525,8 +16472,6 @@ lflow_handle_northd_port_changes(struct ovsdb_idl_txn *ovnsb_txn, ovs_assert(lr_lbnat_rec); lflow_ref_clear_lflows(op->lbnat_lflow_ref, op->od, lflows); build_lsp_lflows_for_lbnats(op, op->peer, lr_lbnat_rec, - lflow_input->lr_lbnats, - lflow_input->lr_ports, lflows, &match, &actions, op->lbnat_lflow_ref); lflow_ref_sync_lflows_to_sb(op->lbnat_lflow_ref, lflows, ovnsb_txn, @@ -16687,6 +16632,95 @@ lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, return true; } +bool +lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *ovnsb_txn, + struct lr_lb_nat_data_tracked_data *trk_data, + struct lflow_input *lflow_input, + struct lflow_table *lflows) +{ + struct lr_lb_nat_data_record *lr_lbnat_rec; + struct hmapx_node *hmapx_node; + + HMAPX_FOR_EACH (hmapx_node, &trk_data->crupdated) { + lr_lbnat_rec = hmapx_node->data; + + lflow_ref_clear_lflows(lr_lbnat_rec->lflow_ref, lr_lbnat_rec->od, + lflows); + + /* Generate new lflows. */ + struct ds match = DS_EMPTY_INITIALIZER; + struct ds actions = DS_EMPTY_INITIALIZER; + + build_lr_lbnat_data_flows(lr_lbnat_rec, lflows, lflow_input->ls_ports, + lflow_input->lr_ports, &match, &actions, + lflow_input->meter_groups, + lflow_input->features); + + /* Sync the new flows to SB. */ + lflow_ref_sync_lflows_to_sb(lr_lbnat_rec->lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + struct ovn_port *op; + HMAP_FOR_EACH (op, dp_node, &lr_lbnat_rec->od->ports) { + lflow_ref_clear_lflows(op->lbnat_lflow_ref, lr_lbnat_rec->od, + lflows); + lflow_ref_clear_lflows_for_all_dps(op->routable_lflow_ref, + ods_size(lflow_input->ls_datapaths), + ods_size(lflow_input->lr_datapaths), + lflows); + + build_lbnat_lflows_iterate_by_lrp(op, lflow_input->lr_lbnats, + lflow_input->meter_groups, + &match, &actions, + lflows, + op->lbnat_lflow_ref, + op->routable_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->lbnat_lflow_ref, lflows, ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + lflow_ref_sync_lflows_to_sb(op->routable_lflow_ref, lflows, + ovnsb_txn, lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + + if (op->peer && op->peer->nbsp) { + lflow_ref_clear_lflows(op->peer->lbnat_lflow_ref, op->peer->od, + lflows); + + build_lbnat_lflows_iterate_by_lsp(op->peer, + lflow_input->lr_lbnats, + &match, &actions, + lflows, + op->peer->lbnat_lflow_ref); + + lflow_ref_sync_lflows_to_sb(op->peer->lbnat_lflow_ref, lflows, + ovnsb_txn, + lflow_input->ls_datapaths, + lflow_input->lr_datapaths, + lflow_input->ovn_internal_version_changed, + lflow_input->sbrec_logical_flow_table, + lflow_input->sbrec_logical_dp_group_table); + } + } + + ds_destroy(&match); + ds_destroy(&actions); + } + + return true; +} + static bool mirror_needs_update(const struct nbrec_mirror *nb_mirror, const struct sbrec_mirror *sb_mirror) diff --git a/northd/northd.h b/northd/northd.h index 56613f8412..98d3593dc1 100644 --- a/northd/northd.h +++ b/northd/northd.h @@ -680,6 +680,7 @@ struct ovn_port { */ struct lflow_ref *lflow_ref; struct lflow_ref *lbnat_lflow_ref; + struct lflow_ref *routable_lflow_ref; }; void ovnnb_db_run(struct northd_input *input_data, @@ -704,6 +705,8 @@ void northd_indices_create(struct northd_data *data, struct ovsdb_idl *ovnsb_idl); struct lflow_table; +struct lr_lb_nat_data_tracked_data; + void build_lflows(struct ovsdb_idl_txn *ovnsb_txn, struct lflow_input *input_data, struct lflow_table *); @@ -715,6 +718,10 @@ bool lflow_handle_northd_lb_changes(struct ovsdb_idl_txn *ovnsb_txn, struct tracked_lbs *, struct lflow_input *, struct lflow_table *lflows); +bool lflow_handle_lr_lb_nat_data_changes(struct ovsdb_idl_txn *, + struct lr_lb_nat_data_tracked_data *, + struct lflow_input *, + struct lflow_table *lflows); bool northd_handle_sb_port_binding_changes( const struct sbrec_port_binding_table *, struct hmap *ls_ports, struct hmap *lr_ports); diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index b276c77221..3c330a051f 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12,6 +12,7 @@ m4_define([_DUMP_DB_TABLES], [ ovn-sbctl list meter >> $1 ovn-sbctl list meter_band >> $1 ovn-sbctl list port_group >> $1 + ovn-sbctl dump-flows > lflows_$1 ]) # CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10586,7 +10587,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10596,7 +10597,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10606,7 +10607,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10616,7 +10617,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10626,7 +10627,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10634,6 +10635,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-lb-del lr0 lb1 check_engine_stats lb_data norecompute compute check_engine_stats northd recompute nocompute +check_engine_stats lr_lb_nat_data recompute nocompute check_engine_stats lflow recompute nocompute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10748,7 +10750,7 @@ check ovn-nbctl add logical_router lr0 load_balancer_group $lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10758,7 +10760,7 @@ check ovn-nbctl --wait=sb set load_balancer lb1 vips:'"10.0.0.10:80"'='"10.0.0.1 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10768,7 +10770,7 @@ check ovn-nbctl --wait=sb clear load_Balancer lb1 vips check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10778,7 +10780,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.10:80 10.0.0.3:80 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10788,7 +10790,7 @@ check ovn-nbctl --wait=sb lb-add lb1 10.0.0.20:80 10.0.0.30:8080 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10869,7 +10871,7 @@ check ovn-nbctl --wait=sb set logical_router lr1 load_balancer_group=$lbg1_uuid check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -10897,7 +10899,7 @@ check ovn-nbctl --wait=sb lr-lb-add lr1 lb2 check_engine_stats lb_data norecompute compute check_engine_stats northd norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_lb recompute compute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11141,7 +11143,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11150,7 +11152,7 @@ check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb set NAT . options:foo=bar check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11160,7 +11162,7 @@ check ovn-nbctl --wait=sb set NAT . external_ip=172.168.0.120 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11170,7 +11172,7 @@ check ovn-nbctl --wait=sb set NAT . logical_ip=10.0.0.10 check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11180,7 +11182,7 @@ check ovn-nbctl --wait=sb set NAT . type=snat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11190,7 +11192,7 @@ check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11201,7 +11203,7 @@ check ovn-nbctl --wait=sb set NAT $nat2_uuid external_mac='"30:54:00:00:00:04"' check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE @@ -11222,6 +11224,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-add lr0 dnat_and_snat 172.168.0.150 10.0.0.41 check_engine_stats northd norecompute compute @@ -11231,6 +11235,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.150 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.150 check_engine_stats northd norecompute compute @@ -11240,6 +11246,8 @@ check_engine_stats sync_to_sb_pb recompute nocompute check_engine_stats lflow recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE +# lflow engine should recompute since the deleted nat ip 172.168.0.140 +# is a lb vip. check as northd ovn-appctl -t NORTHD_TYPE inc-engine/clear-stats check ovn-nbctl --wait=sb lr-nat-del lr0 dnat_and_snat 172.168.0.140 check_engine_stats northd norecompute compute @@ -11255,7 +11263,7 @@ check ovn-nbctl --wait=sb clear logical_router lr0 nat check_engine_stats northd norecompute compute check_engine_stats lr_nat norecompute compute check_engine_stats lr_lb_nat_data norecompute compute -check_engine_stats lflow recompute nocompute +check_engine_stats lflow norecompute compute check_engine_stats sync_to_sb_pb recompute nocompute CHECK_NO_CHANGE_AFTER_RECOMPUTE