[ovs-dev,v1,2/2] Add support for overlay provider networks.

Message ID	20240423164344.2644464-1-numans@ovn.org
State	Changes Requested
Headers	show Return-Path: <ovs-dev-bounces@openvswitch.org> Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2001:4b98:dc4:8::227; helo=relay7-d.mail.gandi.net; envelope-from=numans@ovn.org; receiver=<UNKNOWN> DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 78BA181309 From: numans@ovn.org To: dev@openvswitch.org Date: Tue, 23 Apr 2024 12:43:44 -0400 Message-ID: <20240423164344.2644464-1-numans@ovn.org> MIME-Version: 1.0 Subject: [ovs-dev] [PATCH ovn v1 2/2] Add support for overlay provider networks. Precedence: list Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" <ovs-dev-bounces@openvswitch.org>
Series	[ovs-dev,v1,1/2] northd: Don't reparse lport's addresses while adding L2_LKUP flows. \| expand [ovs-dev,v1,1/2] northd: Don't reparse lport's addresses while adding L2_LKUP flows. [ovs-dev,v1,2/2] Add support for overlay provider networks.

Context	Check	Description
ovsrobot/apply-robot	success	apply and check: success
ovsrobot/github-robot-_Build_and_Test	success	github build: passed
ovsrobot/github-robot-_ovn-kubernetes	success	github build: passed

Numan Siddique April 23, 2024, 4:43 p.m. UTC

From: Numan Siddique <numans@ovn.org>

It is expected that a provider network logical switch has a localnet logical
switch port in order to bridge the overlay traffic to the underlay traffic.
There can be some usecases where a overlay logical switch (without
a localnet port) can act as a provider network and presently NAT doesn't
work as expected.  This patch adds this support.  A new config option
"overlay_provider_network" is added to support this feature.
This feature gets enabled for a logical switch 'P' if:
  - The above option is set to true in the Logical_Switch.other_config
    column.
  - The logical switch 'P' doesn't have any localnet ports.
  - The logical router port of a router 'R' connecting to 'P'
    is a gateway router port.
  - And the logical router 'R' has only one gateway router port.

If all the above conditions are met, ovn-northd creates a chassisredirect
port for the logical switch port (of type router) connecting to the
router 'R'.  For example, if the logical port is named as "P-R" and its
peer router port is "R-P", then chassisredirect port cr-P-R is created
along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
This ensures that the routing is centralized on this gateway chassis for
the traffic coming from switch "P" towards the router or vice versa.
This centralization is required in order to support NAT (both SNAT and
DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
not supported and instead the router port mac is used for such traffic.

Reported-at: https://issues.redhat.com/browse/FDP-364
Signed-off-by: Numan Siddique <numans@ovn.org>
---
 controller/physical.c     |   4 +
 northd/northd.c           | 226 +++++++++++++----
 northd/northd.h           |   1 +
 tests/multinode-macros.at |   2 +-
 tests/multinode.at        | 177 +++++++++++++
 tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
 tests/ovn.at              |   8 +-
 7 files changed, 886 insertions(+), 52 deletions(-)

Mark Michelson April 24, 2024, 9:13 p.m. UTC | #1

Hi Numan,

I haven't done a full review of this yet, but I figured I'd give some 
initial feedback from what I had looked at.

At a high level, this is missing documentation in ovn-nb.xml for the new 
"overlay_provider_network" option. There should also be a NEWS entry for 
the new functionality.

See below for a couple more notes.

On 4/23/24 12:43, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> It is expected that a provider network logical switch has a localnet logical
> switch port in order to bridge the overlay traffic to the underlay traffic.
> There can be some usecases where a overlay logical switch (without
> a localnet port) can act as a provider network and presently NAT doesn't
> work as expected.  This patch adds this support.  A new config option
> "overlay_provider_network" is added to support this feature.
> This feature gets enabled for a logical switch 'P' if:
>    - The above option is set to true in the Logical_Switch.other_config
>      column.
>    - The logical switch 'P' doesn't have any localnet ports.
>    - The logical router port of a router 'R' connecting to 'P'
>      is a gateway router port.
>    - And the logical router 'R' has only one gateway router port.
> 
> If all the above conditions are met, ovn-northd creates a chassisredirect
> port for the logical switch port (of type router) connecting to the
> router 'R'.  For example, if the logical port is named as "P-R" and its
> peer router port is "R-P", then chassisredirect port cr-P-R is created
> along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
> This ensures that the routing is centralized on this gateway chassis for
> the traffic coming from switch "P" towards the router or vice versa.
> This centralization is required in order to support NAT (both SNAT and
> DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic.
> 
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>   controller/physical.c     |   4 +
>   northd/northd.c           | 226 +++++++++++++----
>   northd/northd.h           |   1 +
>   tests/multinode-macros.at |   2 +-
>   tests/multinode.at        | 177 +++++++++++++
>   tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
>   tests/ovn.at              |   8 +-
>   7 files changed, 886 insertions(+), 52 deletions(-)
> 
> diff --git a/controller/physical.c b/controller/physical.c
> index 7ee3086940..625e37e8a7 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1587,6 +1587,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                       ct_zones);
>               put_zones_ofpacts(&zone_ids, ofpacts_p);
>   
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>               /* Resubmit to table 41. */
>               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>           }
> diff --git a/northd/northd.c b/northd/northd.c
> index ead54235c8..1942f9f7a1 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2063,6 +2063,35 @@ parse_lsp_addrs(struct ovn_port *op)
>       }
>   }
>   
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_datapath *od = op->od;
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (!(crp && crp->sb && crp->sb->datapath == od->sb)) {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +    }
> +
> +    crp->l3dgw_port = op;
> +    op->cr_port = crp;
> +    crp->od = od;
> +    free(redirect_name);
> +
> +    /* Add to l3dgw_ports in od, for later use during flow creation. */
> +    if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +        od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                     &od->n_allocated_l3dgw_ports,
> +                                     sizeof *od->l3dgw_ports);
> +    }
> +    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +    return crp;
> +}
> +
>   static void
>   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2170,9 +2199,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>           }
>       }
> +
> +    struct hmapx gw_ports = HMAPX_INITIALIZER(&gw_ports);
>       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>           ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>           for (size_t i = 0; i < od->nbr->n_ports; i++) {
>               const struct nbrec_logical_router_port *nbrp
>                   = od->nbr->ports[i];
> @@ -2236,10 +2266,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       redirect_type && !strcasecmp(redirect_type, "bridged");
>               }
>   
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                   const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                  "chassis");
>                   if (gw_chassis) {
> @@ -2248,34 +2275,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                    "gateway port configured on port %s "
>                                    "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                   } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->l3dgw_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&gw_ports, op);
>                   }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>              }
>           }
>       }
> @@ -2332,12 +2334,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                           arp_proxy, op->nbsp->name);
>                   }
>               }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>           } else if (op->nbrp && op->nbrp->peer && !op->l3dgw_port) {
>               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>               if (peer) {
> @@ -2358,6 +2354,65 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>           }
>       }
>   
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports);
> +        ovs_assert(crp);
> +        if (crp && crp->sb && crp->sb->datapath == od->sb) {
> +            ovn_port_set_nb(crp, NULL, op->nbrp);
> +            ovs_list_remove(&crp->list);
> +            ovs_list_push_back(both, &crp->list);
> +        } else {
> +            ovs_list_push_back(nb_only, &crp->list);
> +        }
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the gateway router port's peer if
> +     *  - Gateway router port's router has only one gateway router port and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports.
> +     *  - Its peer's logical switch has the option overlay_provider_network
> +     *    is set to true in the other_config column.
> +     *
> +     * This is required to support NAT via geneve (for the overlay provider
> +     * networks) and the routing coming from this logical switch destined to
> +     * the router port and vice versa is centralized on the gateway chassis.
> +     *
> +     * Future enhancement: Support NAT via geneve if the logical router has
> +     * multiple gateway ports.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> +        op = hmapx_node->data;
> +        if (op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +            && !op->peer->od->n_localnet_ports &&
> +            smap_get_bool(&op->peer->od->nbs->other_config,
> +                          "overlay_provider_network", false)) {
> +            struct ovn_port *crp = create_cr_port(op->peer, ports);
> +            if (crp && crp->sb && crp->sb->datapath == op->peer->od->sb) {
> +                ovn_port_set_nb(crp, op->peer->nbsp, NULL);
> +                ovs_list_remove(&crp->list);
> +                ovs_list_push_back(both, &crp->list);
> +            } else {
> +                ovs_list_push_back(nb_only, &crp->list);
> +            }
> +        }
> +    }
> +    hmapx_destroy(&gw_ports);
> +
>       /* Wait until all ports have been connected to add to IPAM since
>        * it relies on proper peers to be set
>        */
> @@ -3140,16 +3195,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>                * type "l3gateway". */
>               if (chassis) {
>                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->l3dgw_port->peer);
> +                ovs_assert(op->l3dgw_port->peer->cr_port);
> +                ovs_assert(op->l3dgw_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->l3dgw_port->peer->cr_port->sb->ha_chassis_group);
> +
>               } else {
>                   sbrec_port_binding_set_type(op->sb, "patch");
>               }
>   
>               const char *router_port = smap_get(&op->nbsp->options,
>                                                  "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                   struct smap new;
>                   smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                       smap_add(&new, "peer", router_port);
>                   }
>                   if (chassis) {
> @@ -8148,9 +8215,18 @@ build_lswitch_rport_arp_req_flow(
>       struct lflow_ref *lflow_ref)
>   {
>       struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>       struct ds actions = DS_EMPTY_INITIALIZER;
>   
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>   
>       /* Send a the packet to the router pipeline.  If the switch has non-router
>        * ports then flood it there as well.
> @@ -8172,6 +8248,31 @@ build_lswitch_rport_arp_req_flow(
>                                   lflow_ref);
>       }
>   
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                        patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority + 1, ds_cstr(&match),

Why is this flow (and the one below) added with priority + 1? Since the 
match conditions are different from the flows that are installed earlier 
in this function, this can be installed at the same priority, right?

> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority + 1, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>       ds_destroy(&match);
>       ds_destroy(&actions);
>   }
> @@ -9454,7 +9555,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                   struct ds *actions, struct ds *match)
>   {
>       ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {

Is is_cr_port() the correct function to use here? is_cr_port() returns 
true if op->l3dgw_port is non-NULL. In this case, since op represents a 
logical switch port, is_cr_port(op) will always return false. The 
is_l3dgw_port() function returns true if op->cr_port is non-NULL, but 
that is an unintuitive function to call for a logical switch port. I 
think checking for op->cr_port directly would be most appropriate here.

Also, on a separate note, why in the world do is_cr_port() and 
is_l3dgw_port() do seemingly the opposite of what their function names 
imply?

>           return;
>       }
>   
> @@ -9466,8 +9567,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                              "outport = \""MC_UNKNOWN "\"; output;"
>                            : "outport = %s; output;")
>                            : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>   
>       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>           /* For ports connected to logical routers add flows to bypass the
> @@ -9514,14 +9613,35 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>               if (add_chassis_resident_check) {
>                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>               }
> +        } else if (op->cr_port) {
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>           }
>   
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           ovn_lflow_add_with_hint(lflows, op->od,
>                                   S_SWITCH_IN_L2_LKUP, 50,
>                                   ds_cstr(match), ds_cstr(actions),
>                                   &op->nbsp->header_,
>                                   op->lflow_ref);
>       } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>               ds_clear(match);
>               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11619,6 +11739,15 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>           return;
>       }
>   
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the traffic from the peer port to this
> +         * router or from this router to the peer logical switch.
> +         */
> +        return;
> +    }
> +
>       /* Mac address to use when replying to ARP/NS. */
>       const char *mac_s = REG_INPORT_ETH_ADDR;
>       struct eth_addr mac;
> @@ -14852,6 +14981,17 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>       /* For distributed router NAT, determine whether this NAT rule
>        * satisfies the conditions for distributed NAT processing. */
>       *distributed = false;
> +
> +    /* NAT cannnot be distributed if the gateway port's peer
> +     * has a chassisresident port (and the routing is centralized
> +     * on the gateway chassis for the traffic from the peer
> +     * to this router and traffic to the peer.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>           nat->logical_port && nat->external_mac) {
>           if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index 18cad5234a..dad3a77673 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>        * will be NULL. */
>       struct ovn_port **l3dgw_ports;
>       size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>   
>       /* router datapath has a logical port with redirect-type set to bridged. */
>       bool redirect_bridged;
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index c04506a52a..25cfa186ee 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -66,7 +66,7 @@ m_count_rows() {
>   m_check_row_count() {
>       local db=$(parse_db $1) table=$(parse_table $1); shift
>       local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>       echo
>       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>       if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index b959a25506..d549bedd66 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -890,4 +890,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.1.2 |
>   
>   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -c 20 -i 0.5 -s 1300 -M do 172.20.1.2 2>&1 |grep -q "mtu = 1150"])
>   
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# overlay_provider_network=true is not yet set for public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option now.
> +check multinode_nbctl --wait=hv set logical_switch public other_config:overlay_provider_network=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>   AT_CLEANUP
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 3d944a3aef..451e57d61d 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2061,7 +2061,7 @@ match=(inport == "lrp-public" && nd_ns && nd.target == \$${lb_as_v6}), dnl
>   action=(nd_na { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   # Test chassis redirect port.
> @@ -2089,7 +2089,7 @@ action=(xreg0[[0..47]] = 00:00:00:00:01:00; next;)
>   ])
>   
>   # Ingress router port is used for ARP reply/NA in lr_in_ip_input.
> -# xxreg0[0..47] is used unless external_mac is set.
> +# xxreg0[[0..47]] is used unless external_mac is set.
>   # Priority 90 flows (per router).
>   AT_CHECK_UNQUOTED([ovn-sbctl lflow-list | grep -E "lr_in_ip_input.*priority=90" | grep "arp\|nd" | ovn_strip_lflows], [0], [dnl
>     table=??(lr_in_ip_input     ), priority=90   , dnl
> @@ -2164,7 +2164,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   AT_CLEANUP
> @@ -5434,13 +5434,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>   
>   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>   
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>   
>   ovn-sbctl lflow-list ls1 > ls1_lflows
>   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12435,3 +12436,512 @@ check_engine_stats northd recompute nocompute
>   check_engine_stats lflow recompute nocompute
>   
>   AT_CLEANUP
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([dnat])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "overlay_provider_network=true" for public.
> +check ovn-nbctl --wait=sb set logical_switch public other_config:overlay_provider_network=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index e81cd4f45a..8b1d3c846f 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21181,10 +21181,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>       type=router options:router-port=sw0 \
>       -- lsp-set-addresses rp-sw0 router
>   
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>       -- lsp-set-addresses rp-sw1 router
>   
>   ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21196,6 +21196,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>   ovn-nbctl lsp-add sw1 sw1-p0 \
>       -- lsp-set-addresses sw1-p0 unknown
>   
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>

Numan Siddique April 24, 2024, 11:46 p.m. UTC | #2

On Wed, Apr 24, 2024 at 5:14 PM Mark Michelson <mmichels@redhat.com> wrote:
>
> Hi Numan,
>
> I haven't done a full review of this yet, but I figured I'd give some
> initial feedback from what I had looked at.
>
> At a high level, this is missing documentation in ovn-nb.xml for the new
> "overlay_provider_network" option. There should also be a NEWS entry for
> the new functionality.
>

Thanks for the comments.   Yes.  I missed adding them.  I'll address it in v2.

>
>
> See below for a couple more notes.
>
> On 4/23/24 12:43, numans@ovn.org wrote:
> > From: Numan Siddique <numans@ovn.org>
> >
> > It is expected that a provider network logical switch has a localnet logical
> > switch port in order to bridge the overlay traffic to the underlay traffic.
> > There can be some usecases where a overlay logical switch (without
> > a localnet port) can act as a provider network and presently NAT doesn't
> > work as expected.  This patch adds this support.  A new config option
> > "overlay_provider_network" is added to support this feature.
> > This feature gets enabled for a logical switch 'P' if:
> >    - The above option is set to true in the Logical_Switch.other_config
> >      column.
> >    - The logical switch 'P' doesn't have any localnet ports.
> >    - The logical router port of a router 'R' connecting to 'P'
> >      is a gateway router port.
> >    - And the logical router 'R' has only one gateway router port.
> >
> > If all the above conditions are met, ovn-northd creates a chassisredirect
> > port for the logical switch port (of type router) connecting to the
> > router 'R'.  For example, if the logical port is named as "P-R" and its
> > peer router port is "R-P", then chassisredirect port cr-P-R is created
> > along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
> > This ensures that the routing is centralized on this gateway chassis for
> > the traffic coming from switch "P" towards the router or vice versa.
> > This centralization is required in order to support NAT (both SNAT and
> > DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
> > not supported and instead the router port mac is used for such traffic.
> >
> > Reported-at: https://issues.redhat.com/browse/FDP-364
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >   controller/physical.c     |   4 +
> >   northd/northd.c           | 226 +++++++++++++----
> >   northd/northd.h           |   1 +
> >   tests/multinode-macros.at |   2 +-
> >   tests/multinode.at        | 177 +++++++++++++
> >   tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
> >   tests/ovn.at              |   8 +-
> >   7 files changed, 886 insertions(+), 52 deletions(-)
> >
> > diff --git a/controller/physical.c b/controller/physical.c
> > index 7ee3086940..625e37e8a7 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -1587,6 +1587,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                                                       ct_zones);
> >               put_zones_ofpacts(&zone_ids, ofpacts_p);
> >
> > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > +             * go out from the same tunnel inport. */
> > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > +
> >               /* Resubmit to table 41. */
> >               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >           }
> > diff --git a/northd/northd.c b/northd/northd.c
> > index ead54235c8..1942f9f7a1 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -2063,6 +2063,35 @@ parse_lsp_addrs(struct ovn_port *op)
> >       }
> >   }
> >
> > +static struct ovn_port *
> > +create_cr_port(struct ovn_port *op, struct hmap *ports)
> > +{
> > +    char *redirect_name = ovn_chassis_redirect_name(
> > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > +
> > +    struct ovn_datapath *od = op->od;
> > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > +    if (!(crp && crp->sb && crp->sb->datapath == od->sb)) {
> > +        crp = ovn_port_create(ports, redirect_name,
> > +                              op->nbsp, op->nbrp, NULL);
> > +    }
> > +
> > +    crp->l3dgw_port = op;
> > +    op->cr_port = crp;
> > +    crp->od = od;
> > +    free(redirect_name);
> > +
> > +    /* Add to l3dgw_ports in od, for later use during flow creation. */
> > +    if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > +        od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > +                                     &od->n_allocated_l3dgw_ports,
> > +                                     sizeof *od->l3dgw_ports);
> > +    }
> > +    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > +
> > +    return crp;
> > +}
> > +
> >   static void
> >   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > @@ -2170,9 +2199,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> >           }
> >       }
> > +
> > +    struct hmapx gw_ports = HMAPX_INITIALIZER(&gw_ports);
> >       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> >           ovs_assert(od->nbr);
> > -        size_t n_allocated_l3dgw_ports = 0;
> >           for (size_t i = 0; i < od->nbr->n_ports; i++) {
> >               const struct nbrec_logical_router_port *nbrp
> >                   = od->nbr->ports[i];
> > @@ -2236,10 +2266,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                       redirect_type && !strcasecmp(redirect_type, "bridged");
> >               }
> >
> > -            if (op->nbrp->ha_chassis_group ||
> > -                op->nbrp->n_gateway_chassis) {
> > -                /* Additional "derived" ovn_port crp represents the
> > -                 * instance of op on the gateway chassis. */
> > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> >                   const char *gw_chassis = smap_get(&op->od->nbr->options,
> >                                                  "chassis");
> >                   if (gw_chassis) {
> > @@ -2248,34 +2275,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> >                                    "gateway port configured on port %s "
> >                                    "on L3 gateway router", nbrp->name);
> > -                    continue;
> > -                }
> > -
> > -                char *redirect_name =
> > -                    ovn_chassis_redirect_name(nbrp->name);
> > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > -                    ovs_list_remove(&crp->list);
> > -                    ovs_list_push_back(both, &crp->list);
> >                   } else {
> > -                    crp = ovn_port_create(ports, redirect_name,
> > -                                          NULL, nbrp, NULL);
> > -                    ovs_list_push_back(nb_only, &crp->list);
> > -                }
> > -                crp->l3dgw_port = op;
> > -                op->cr_port = crp;
> > -                crp->od = od;
> > -                free(redirect_name);
> > -
> > -                /* Add to l3dgw_ports in od, for later use during flow
> > -                 * creation. */
> > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > -                                                 &n_allocated_l3dgw_ports,
> > -                                                 sizeof *od->l3dgw_ports);
> > +                    hmapx_add(&gw_ports, op);
> >                   }
> > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> >              }
> >           }
> >       }
> > @@ -2332,12 +2334,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                           arp_proxy, op->nbsp->name);
> >                   }
> >               }
> > -
> > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > -                op->enable_router_port_acl = smap_get_bool(
> > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > -            }
> >           } else if (op->nbrp && op->nbrp->peer && !op->l3dgw_port) {
> >               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> >               if (peer) {
> > @@ -2358,6 +2354,65 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >           }
> >       }
> >
> > +    struct hmapx_node *hmapx_node;
> > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > +        op = hmapx_node->data;
> > +        od = op->od;
> > +        ovs_assert(op->nbrp);
> > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > +
> > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > +         * the gateway chassis. */
> > +        struct ovn_port *crp = create_cr_port(op, ports);
> > +        ovs_assert(crp);
> > +        if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > +            ovn_port_set_nb(crp, NULL, op->nbrp);
> > +            ovs_list_remove(&crp->list);
> > +            ovs_list_push_back(both, &crp->list);
> > +        } else {
> > +            ovs_list_push_back(nb_only, &crp->list);
> > +        }
> > +
> > +        if (op->peer && op->peer->nbsp) {
> > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > +            op->peer->enable_router_port_acl = smap_get_bool(
> > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > +        }
> > +    }
> > +
> > +
> > +    /* Create chassisresident port for the gateway router port's peer if
> > +     *  - Gateway router port's router has only one gateway router port and
> > +     *  - Its peer is a logical switch port and
> > +     *  - It's peer's logical switch has no localnet ports.
> > +     *  - Its peer's logical switch has the option overlay_provider_network
> > +     *    is set to true in the other_config column.
> > +     *
> > +     * This is required to support NAT via geneve (for the overlay provider
> > +     * networks) and the routing coming from this logical switch destined to
> > +     * the router port and vice versa is centralized on the gateway chassis.
> > +     *
> > +     * Future enhancement: Support NAT via geneve if the logical router has
> > +     * multiple gateway ports.
> > +     * */
> > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > +        op = hmapx_node->data;
> > +        if (op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > +            && !op->peer->od->n_localnet_ports &&
> > +            smap_get_bool(&op->peer->od->nbs->other_config,
> > +                          "overlay_provider_network", false)) {
> > +            struct ovn_port *crp = create_cr_port(op->peer, ports);
> > +            if (crp && crp->sb && crp->sb->datapath == op->peer->od->sb) {
> > +                ovn_port_set_nb(crp, op->peer->nbsp, NULL);
> > +                ovs_list_remove(&crp->list);
> > +                ovs_list_push_back(both, &crp->list);
> > +            } else {
> > +                ovs_list_push_back(nb_only, &crp->list);
> > +            }
> > +        }
> > +    }
> > +    hmapx_destroy(&gw_ports);
> > +
> >       /* Wait until all ports have been connected to add to IPAM since
> >        * it relies on proper peers to be set
> >        */
> > @@ -3140,16 +3195,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> >                * type "l3gateway". */
> >               if (chassis) {
> >                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> > +            } else if (is_cr_port(op)) {
> > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > +                ovs_assert(op->l3dgw_port->peer);
> > +                ovs_assert(op->l3dgw_port->peer->cr_port);
> > +                ovs_assert(op->l3dgw_port->peer->cr_port->sb);
> > +                sbrec_port_binding_set_ha_chassis_group(
> > +                    op->sb,
> > +                    op->l3dgw_port->peer->cr_port->sb->ha_chassis_group);
> > +
> >               } else {
> >                   sbrec_port_binding_set_type(op->sb, "patch");
> >               }
> >
> >               const char *router_port = smap_get(&op->nbsp->options,
> >                                                  "router-port");
> > -            if (router_port || chassis) {
> > +            if (router_port || chassis || is_cr_port(op)) {
> >                   struct smap new;
> >                   smap_init(&new);
> > -                if (router_port) {
> > +
> > +                if (is_cr_port(op)) {
> > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > +                } else if (router_port) {
> >                       smap_add(&new, "peer", router_port);
> >                   }
> >                   if (chassis) {
> > @@ -8148,9 +8215,18 @@ build_lswitch_rport_arp_req_flow(
> >       struct lflow_ref *lflow_ref)
> >   {
> >       struct ds match   = DS_EMPTY_INITIALIZER;
> > +    struct ds m       = DS_EMPTY_INITIALIZER;
> >       struct ds actions = DS_EMPTY_INITIALIZER;
> >
> > -    arp_nd_ns_match(ips, addr_family, &match);
> > +    arp_nd_ns_match(ips, addr_family, &m);
> > +    ds_clone(&match, &m);
> > +
> > +    bool has_cr_port = patch_op->cr_port;
> > +
> > +    if (has_cr_port) {
> > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                      patch_op->cr_port->json_key);
> > +    }
> >
> >       /* Send a the packet to the router pipeline.  If the switch has non-router
> >        * ports then flood it there as well.
> > @@ -8172,6 +8248,31 @@ build_lswitch_rport_arp_req_flow(
> >                                   lflow_ref);
> >       }
> >
> > +    if (has_cr_port) {
> > +        ds_clear(&match);
> > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > +                      patch_op->cr_port->json_key);
> > +        ds_clear(&actions);
> > +        if (od->n_router_ports != od->nbs->n_ports) {
> > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > +                        patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority + 1, ds_cstr(&match),
>
> Why is this flow (and the one below) added with priority + 1? Since the
> match conditions are different from the flows that are installed earlier
> in this function, this can be installed at the same priority, right?


You're right.  It can be installed at the same prio.  I don't think I had any
particular reason for bumping the prio + 1.  I'll address it in v2.

>
> > +                                    ds_cstr(&actions), stage_hint,
> > +                                    lflow_ref);
> > +        } else {
> > +            ds_put_format(&actions, "outport = %s; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority + 1, ds_cstr(&match),
> > +                                    ds_cstr(&actions),
> > +                                    stage_hint,
> > +                                    lflow_ref);
> > +        }
> > +    }
> > +
> > +    ds_destroy(&m);
> >       ds_destroy(&match);
> >       ds_destroy(&actions);
> >   }
> > @@ -9454,7 +9555,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                                   struct ds *actions, struct ds *match)
> >   {
> >       ovs_assert(op->nbsp);
> > -    if (lsp_is_external(op->nbsp)) {
> > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>
> Is is_cr_port() the correct function to use here? is_cr_port() returns
> true if op->l3dgw_port is non-NULL. In this case, since op represents a
> logical switch port, is_cr_port(op) will always return false. The
> is_l3dgw_port() function returns true if op->cr_port is non-NULL, but
> that is an unintuitive function to call for a logical switch port. I
> think checking for op->cr_port directly would be most appropriate here.


Agree.  I thought about using it here too.  If you see I've used op->cr_port
in other places in this patch.

>
> Also, on a separate note, why in the world do is_cr_port() and
> is_l3dgw_port() do seemingly the opposite of what their function names
> imply?


I had the same doubt.

A gateway port means it has ha_chassis or gateway chassis
configured.  Which means ovn-northd will create a chassisredirect port
and stores this
reference in op->cr_port of  a gateway port.  And when chassisredirect
'ovn_port' is created,
its corresponding gateway port reference is stored in 'op->l3dgw_port'.

So the function is_cr_port(op) checks for op->l3dgw_port.  If it is
set, it means 'op' is a chassisredirect port.
And the function is_l3dgw_port(op) checks for op->cr_port.  If it is
set,  it means 'op' is a gateway router port.

Although it seems a bit confusing,  the functions don't do anything
opposite.  It's just a way to figure out
the type.

Thanks
Numan


>
>
> >           return;
> >       }
> >
> > @@ -9466,8 +9567,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                              "outport = \""MC_UNKNOWN "\"; output;"
> >                            : "outport = %s; output;")
> >                            : debug_drop_action();
> > -    ds_clear(actions);
> > -    ds_put_format(actions, action, op->json_key);
> >
> >       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> >           /* For ports connected to logical routers add flows to bypass the
> > @@ -9514,14 +9613,35 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >               if (add_chassis_resident_check) {
> >                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> >               }
> > +        } else if (op->cr_port) {
> > +            ds_clear(actions);
> > +            ds_put_format(actions, action, op->cr_port->json_key);
> > +
> > +            struct ds m = DS_EMPTY_INITIALIZER;
> > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > +                          op->peer->lrp_networks.ea_s,
> > +                          op->cr_port->json_key);
> > +
> > +            ovn_lflow_add_with_hint(lflows, op->od,
> > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > +                                    ds_cstr(&m), ds_cstr(actions),
> > +                                    &op->nbsp->header_,
> > +                                    op->lflow_ref);
> > +            ds_destroy(&m);
> > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > +                          op->cr_port->json_key);
> >           }
> >
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >           ovn_lflow_add_with_hint(lflows, op->od,
> >                                   S_SWITCH_IN_L2_LKUP, 50,
> >                                   ds_cstr(match), ds_cstr(actions),
> >                                   &op->nbsp->header_,
> >                                   op->lflow_ref);
> >       } else {
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> >               ds_clear(match);
> >               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > @@ -11619,6 +11739,15 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> >           return;
> >       }
> >
> > +    if (op->peer && op->peer->cr_port) {
> > +        /* We don't add the below flows if the router port's peer has
> > +         * a chassisresident port.  That's because routing is centralized on
> > +         * the gateway chassis for the traffic from the peer port to this
> > +         * router or from this router to the peer logical switch.
> > +         */
> > +        return;
> > +    }
> > +
> >       /* Mac address to use when replying to ARP/NS. */
> >       const char *mac_s = REG_INPORT_ETH_ADDR;
> >       struct eth_addr mac;
> > @@ -14852,6 +14981,17 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> >       /* For distributed router NAT, determine whether this NAT rule
> >        * satisfies the conditions for distributed NAT processing. */
> >       *distributed = false;
> > +
> > +    /* NAT cannnot be distributed if the gateway port's peer
> > +     * has a chassisresident port (and the routing is centralized
> > +     * on the gateway chassis for the traffic from the peer
> > +     * to this router and traffic to the peer.)
> > +     */
> > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > +        return 0;
> > +    }
> > +
> >       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> >           nat->logical_port && nat->external_mac) {
> >           if (eth_addr_from_string(nat->external_mac, mac)) {
> > diff --git a/northd/northd.h b/northd/northd.h
> > index 18cad5234a..dad3a77673 100644
> > --- a/northd/northd.h
> > +++ b/northd/northd.h
> > @@ -325,6 +325,7 @@ struct ovn_datapath {
> >        * will be NULL. */
> >       struct ovn_port **l3dgw_ports;
> >       size_t n_l3dgw_ports;
> > +    size_t n_allocated_l3dgw_ports;
> >
> >       /* router datapath has a logical port with redirect-type set to bridged. */
> >       bool redirect_bridged;
> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > index c04506a52a..25cfa186ee 100644
> > --- a/tests/multinode-macros.at
> > +++ b/tests/multinode-macros.at
> > @@ -66,7 +66,7 @@ m_count_rows() {
> >   m_check_row_count() {
> >       local db=$(parse_db $1) table=$(parse_table $1); shift
> >       local count=$1; shift
> > -    local found=$(m_count_rows $c $db:$table "$@")
> > +    local found=$(m_count_rows $db:$table "$@")
> >       echo
> >       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> >       if test "$count" != "$found"; then
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index b959a25506..d549bedd66 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -890,4 +890,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.1.2 |
> >
> >   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -c 20 -i 0.5 -s 1300 -M do 172.20.1.2 2>&1 |grep -q "mtu = 1150"])
> >
> > +# Reset back to geneve tunnels
> > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > +do
> > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > +done
> > +
> > +AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +
> > +m_wait_for_ports_up
> > +
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create the second logical switch with one port
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > +
> > +# Create a logical router and attach both logical switches
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > +
> > +# create exteranl connection for N/S traffic
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > +
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > +
> > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > +check multinode_nbctl lsp-add public public-port1
> > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +# First do basic ping tests before deleting the localnet port - ln-public.
> > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > +# is centralized on ovn-gw-1.
> > +
> > +# This function checks the North-South traffic.
> > +run_ns_traffic() {
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +}
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > +
> > +# cr-port should not be created for public-lr0 since the option
> > +# overlay_provider_network=true is not yet set for public.
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Set the option now.
> > +check multinode_nbctl --wait=hv set logical_switch public other_config:overlay_provider_network=true
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Re-add the localnet port
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > +
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the ln-public port this time.
> > +check multinode_nbctl --wait=hv lsp-del ln-public
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> >   AT_CLEANUP
> > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > index 3d944a3aef..451e57d61d 100644
> > --- a/tests/ovn-northd.at
> > +++ b/tests/ovn-northd.at
> > @@ -2061,7 +2061,7 @@ match=(inport == "lrp-public" && nd_ns && nd.target == \$${lb_as_v6}), dnl
> >   action=(nd_na { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> >   ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >   # Test chassis redirect port.
> > @@ -2089,7 +2089,7 @@ action=(xreg0[[0..47]] = 00:00:00:00:01:00; next;)
> >   ])
> >
> >   # Ingress router port is used for ARP reply/NA in lr_in_ip_input.
> > -# xxreg0[0..47] is used unless external_mac is set.
> > +# xxreg0[[0..47]] is used unless external_mac is set.
> >   # Priority 90 flows (per router).
> >   AT_CHECK_UNQUOTED([ovn-sbctl lflow-list | grep -E "lr_in_ip_input.*priority=90" | grep "arp\|nd" | ovn_strip_lflows], [0], [dnl
> >     table=??(lr_in_ip_input     ), priority=90   , dnl
> > @@ -2164,7 +2164,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> >   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> >   ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >   AT_CLEANUP
> > @@ -5434,13 +5434,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> >
> >   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> >
> > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> >
> >   ovn-sbctl lflow-list ls1 > ls1_lflows
> >   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> >     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> >     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > @@ -12435,3 +12436,512 @@ check_engine_stats northd recompute nocompute
> >   check_engine_stats lflow recompute nocompute
> >
> >   AT_CLEANUP
> > +
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([NAT on a provider network with no localnet ports])
> > +AT_KEYWORDS([dnat])
> > +ovn_start
> > +
> > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lsp-add public pub-p1
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > +
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > +
> > +check ovn-nbctl --wait=sb sync
> > +
> > +check_flows_no_cr_port_for_public_lr0() {
> > +  # check that there is no port binding cr-public-lr0
> > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +check_flows_cr_port_for_public_lr0() {
> > +  # check that there is port binding cr-public-lr0
> > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Remove the localnet port from public logical switch.
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > +
> > +# Check that the lflows are as expected and there is no cr port
> > +# created for "public-lr0"  when public has no localnet port
> > +# since public doesn't have the option "overlay_provider_network=true"
> > +# set.
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +
> > +# Set the option "overlay_provider_network=true" for public.
> > +check ovn-nbctl --wait=sb set logical_switch public other_config:overlay_provider_network=true
> > +
> > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Set the type of ln-public back to localnet
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Delete the localnet port
> > +check ovn-nbctl --wait=sb lsp-del ln-public
> > +
> > +# Check that the lflows are as expected when public has no localnet port.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Create multiple gateway ports.  chassisresident port should not be
> > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > +# logical switch.
> > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > +# check that there is no port binding cr-public-lr0
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CLEANUP
> > +])
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index e81cd4f45a..8b1d3c846f 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -21181,10 +21181,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> >       type=router options:router-port=sw0 \
> >       -- lsp-set-addresses rp-sw0 router
> >
> > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > -    -- lrp-set-gateway-chassis sw1 hv2
> > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> >   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > -    type=router options:router-port=sw1 \
> > +    type=router options:router-port=lr0-sw1 \
> >       -- lsp-set-addresses rp-sw1 router
> >
> >   ovn-nbctl lsp-add sw0 sw0-p0 \
> > @@ -21196,6 +21196,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> >   ovn-nbctl lsp-add sw1 sw1-p0 \
> >       -- lsp-set-addresses sw1-p0 unknown
> >
> > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > +
> >   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> >   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> >
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

Mark Michelson May 8, 2024, 6:12 p.m. UTC | #3

Hi Numan,

If you were waiting no further comments, I've now reviewed the full 
patch and have no comments beyond what I initially gave.

On 4/23/24 12:43, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> It is expected that a provider network logical switch has a localnet logical
> switch port in order to bridge the overlay traffic to the underlay traffic.
> There can be some usecases where a overlay logical switch (without
> a localnet port) can act as a provider network and presently NAT doesn't
> work as expected.  This patch adds this support.  A new config option
> "overlay_provider_network" is added to support this feature.
> This feature gets enabled for a logical switch 'P' if:
>    - The above option is set to true in the Logical_Switch.other_config
>      column.
>    - The logical switch 'P' doesn't have any localnet ports.
>    - The logical router port of a router 'R' connecting to 'P'
>      is a gateway router port.
>    - And the logical router 'R' has only one gateway router port.
> 
> If all the above conditions are met, ovn-northd creates a chassisredirect
> port for the logical switch port (of type router) connecting to the
> router 'R'.  For example, if the logical port is named as "P-R" and its
> peer router port is "R-P", then chassisredirect port cr-P-R is created
> along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
> This ensures that the routing is centralized on this gateway chassis for
> the traffic coming from switch "P" towards the router or vice versa.
> This centralization is required in order to support NAT (both SNAT and
> DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic.
> 
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>   controller/physical.c     |   4 +
>   northd/northd.c           | 226 +++++++++++++----
>   northd/northd.h           |   1 +
>   tests/multinode-macros.at |   2 +-
>   tests/multinode.at        | 177 +++++++++++++
>   tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
>   tests/ovn.at              |   8 +-
>   7 files changed, 886 insertions(+), 52 deletions(-)
> 
> diff --git a/controller/physical.c b/controller/physical.c
> index 7ee3086940..625e37e8a7 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1587,6 +1587,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                       ct_zones);
>               put_zones_ofpacts(&zone_ids, ofpacts_p);
>   
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>               /* Resubmit to table 41. */
>               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>           }
> diff --git a/northd/northd.c b/northd/northd.c
> index ead54235c8..1942f9f7a1 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2063,6 +2063,35 @@ parse_lsp_addrs(struct ovn_port *op)
>       }
>   }
>   
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_datapath *od = op->od;
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (!(crp && crp->sb && crp->sb->datapath == od->sb)) {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +    }
> +
> +    crp->l3dgw_port = op;
> +    op->cr_port = crp;
> +    crp->od = od;
> +    free(redirect_name);
> +
> +    /* Add to l3dgw_ports in od, for later use during flow creation. */
> +    if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +        od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                     &od->n_allocated_l3dgw_ports,
> +                                     sizeof *od->l3dgw_ports);
> +    }
> +    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +    return crp;
> +}
> +
>   static void
>   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2170,9 +2199,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>           }
>       }
> +
> +    struct hmapx gw_ports = HMAPX_INITIALIZER(&gw_ports);
>       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>           ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>           for (size_t i = 0; i < od->nbr->n_ports; i++) {
>               const struct nbrec_logical_router_port *nbrp
>                   = od->nbr->ports[i];
> @@ -2236,10 +2266,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       redirect_type && !strcasecmp(redirect_type, "bridged");
>               }
>   
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                   const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                  "chassis");
>                   if (gw_chassis) {
> @@ -2248,34 +2275,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                    "gateway port configured on port %s "
>                                    "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                   } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->l3dgw_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&gw_ports, op);
>                   }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>              }
>           }
>       }
> @@ -2332,12 +2334,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                           arp_proxy, op->nbsp->name);
>                   }
>               }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>           } else if (op->nbrp && op->nbrp->peer && !op->l3dgw_port) {
>               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>               if (peer) {
> @@ -2358,6 +2354,65 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>           }
>       }
>   
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports);
> +        ovs_assert(crp);
> +        if (crp && crp->sb && crp->sb->datapath == od->sb) {
> +            ovn_port_set_nb(crp, NULL, op->nbrp);
> +            ovs_list_remove(&crp->list);
> +            ovs_list_push_back(both, &crp->list);
> +        } else {
> +            ovs_list_push_back(nb_only, &crp->list);
> +        }
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the gateway router port's peer if
> +     *  - Gateway router port's router has only one gateway router port and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports.
> +     *  - Its peer's logical switch has the option overlay_provider_network
> +     *    is set to true in the other_config column.
> +     *
> +     * This is required to support NAT via geneve (for the overlay provider
> +     * networks) and the routing coming from this logical switch destined to
> +     * the router port and vice versa is centralized on the gateway chassis.
> +     *
> +     * Future enhancement: Support NAT via geneve if the logical router has
> +     * multiple gateway ports.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> +        op = hmapx_node->data;
> +        if (op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +            && !op->peer->od->n_localnet_ports &&
> +            smap_get_bool(&op->peer->od->nbs->other_config,
> +                          "overlay_provider_network", false)) {
> +            struct ovn_port *crp = create_cr_port(op->peer, ports);
> +            if (crp && crp->sb && crp->sb->datapath == op->peer->od->sb) {
> +                ovn_port_set_nb(crp, op->peer->nbsp, NULL);
> +                ovs_list_remove(&crp->list);
> +                ovs_list_push_back(both, &crp->list);
> +            } else {
> +                ovs_list_push_back(nb_only, &crp->list);
> +            }
> +        }
> +    }
> +    hmapx_destroy(&gw_ports);
> +
>       /* Wait until all ports have been connected to add to IPAM since
>        * it relies on proper peers to be set
>        */
> @@ -3140,16 +3195,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>                * type "l3gateway". */
>               if (chassis) {
>                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->l3dgw_port->peer);
> +                ovs_assert(op->l3dgw_port->peer->cr_port);
> +                ovs_assert(op->l3dgw_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->l3dgw_port->peer->cr_port->sb->ha_chassis_group);
> +
>               } else {
>                   sbrec_port_binding_set_type(op->sb, "patch");
>               }
>   
>               const char *router_port = smap_get(&op->nbsp->options,
>                                                  "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                   struct smap new;
>                   smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                       smap_add(&new, "peer", router_port);
>                   }
>                   if (chassis) {
> @@ -8148,9 +8215,18 @@ build_lswitch_rport_arp_req_flow(
>       struct lflow_ref *lflow_ref)
>   {
>       struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>       struct ds actions = DS_EMPTY_INITIALIZER;
>   
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>   
>       /* Send a the packet to the router pipeline.  If the switch has non-router
>        * ports then flood it there as well.
> @@ -8172,6 +8248,31 @@ build_lswitch_rport_arp_req_flow(
>                                   lflow_ref);
>       }
>   
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                        patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority + 1, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority + 1, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>       ds_destroy(&match);
>       ds_destroy(&actions);
>   }
> @@ -9454,7 +9555,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                   struct ds *actions, struct ds *match)
>   {
>       ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>           return;
>       }
>   
> @@ -9466,8 +9567,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                              "outport = \""MC_UNKNOWN "\"; output;"
>                            : "outport = %s; output;")
>                            : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>   
>       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>           /* For ports connected to logical routers add flows to bypass the
> @@ -9514,14 +9613,35 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>               if (add_chassis_resident_check) {
>                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>               }
> +        } else if (op->cr_port) {
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>           }
>   
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           ovn_lflow_add_with_hint(lflows, op->od,
>                                   S_SWITCH_IN_L2_LKUP, 50,
>                                   ds_cstr(match), ds_cstr(actions),
>                                   &op->nbsp->header_,
>                                   op->lflow_ref);
>       } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>               ds_clear(match);
>               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11619,6 +11739,15 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>           return;
>       }
>   
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the traffic from the peer port to this
> +         * router or from this router to the peer logical switch.
> +         */
> +        return;
> +    }
> +
>       /* Mac address to use when replying to ARP/NS. */
>       const char *mac_s = REG_INPORT_ETH_ADDR;
>       struct eth_addr mac;
> @@ -14852,6 +14981,17 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>       /* For distributed router NAT, determine whether this NAT rule
>        * satisfies the conditions for distributed NAT processing. */
>       *distributed = false;
> +
> +    /* NAT cannnot be distributed if the gateway port's peer
> +     * has a chassisresident port (and the routing is centralized
> +     * on the gateway chassis for the traffic from the peer
> +     * to this router and traffic to the peer.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>           nat->logical_port && nat->external_mac) {
>           if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index 18cad5234a..dad3a77673 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>        * will be NULL. */
>       struct ovn_port **l3dgw_ports;
>       size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>   
>       /* router datapath has a logical port with redirect-type set to bridged. */
>       bool redirect_bridged;
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index c04506a52a..25cfa186ee 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -66,7 +66,7 @@ m_count_rows() {
>   m_check_row_count() {
>       local db=$(parse_db $1) table=$(parse_table $1); shift
>       local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>       echo
>       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>       if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index b959a25506..d549bedd66 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -890,4 +890,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.1.2 |
>   
>   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -c 20 -i 0.5 -s 1300 -M do 172.20.1.2 2>&1 |grep -q "mtu = 1150"])
>   
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# overlay_provider_network=true is not yet set for public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option now.
> +check multinode_nbctl --wait=hv set logical_switch public other_config:overlay_provider_network=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>   AT_CLEANUP
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 3d944a3aef..451e57d61d 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2061,7 +2061,7 @@ match=(inport == "lrp-public" && nd_ns && nd.target == \$${lb_as_v6}), dnl
>   action=(nd_na { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   # Test chassis redirect port.
> @@ -2089,7 +2089,7 @@ action=(xreg0[[0..47]] = 00:00:00:00:01:00; next;)
>   ])
>   
>   # Ingress router port is used for ARP reply/NA in lr_in_ip_input.
> -# xxreg0[0..47] is used unless external_mac is set.
> +# xxreg0[[0..47]] is used unless external_mac is set.
>   # Priority 90 flows (per router).
>   AT_CHECK_UNQUOTED([ovn-sbctl lflow-list | grep -E "lr_in_ip_input.*priority=90" | grep "arp\|nd" | ovn_strip_lflows], [0], [dnl
>     table=??(lr_in_ip_input     ), priority=90   , dnl
> @@ -2164,7 +2164,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   AT_CLEANUP
> @@ -5434,13 +5434,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>   
>   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>   
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>   
>   ovn-sbctl lflow-list ls1 > ls1_lflows
>   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12435,3 +12436,512 @@ check_engine_stats northd recompute nocompute
>   check_engine_stats lflow recompute nocompute
>   
>   AT_CLEANUP
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([dnat])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "overlay_provider_network=true" for public.
> +check ovn-nbctl --wait=sb set logical_switch public other_config:overlay_provider_network=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index e81cd4f45a..8b1d3c846f 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21181,10 +21181,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>       type=router options:router-port=sw0 \
>       -- lsp-set-addresses rp-sw0 router
>   
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>       -- lsp-set-addresses rp-sw1 router
>   
>   ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21196,6 +21196,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>   ovn-nbctl lsp-add sw1 sw1-p0 \
>       -- lsp-set-addresses sw1-p0 unknown
>   
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>

Numan Siddique May 8, 2024, 9:33 p.m. UTC | #4

On Wed, May 8, 2024 at 2:12 PM Mark Michelson <mmichels@redhat.com> wrote:
>
> Hi Numan,
>
> If you were waiting no further comments, I've now reviewed the full
> patch and have no comments beyond what I initially gave.

Thanks.  I'll submit v2 with the changes.  Please check out this patch
- https://patchwork.ozlabs.org/project/ovn/patch/20240507215713.902148-1-numans@ovn.org/
which improves documentation for is_cr_port() and is_gw_port()

Thanks
Numan


>
> On 4/23/24 12:43, numans@ovn.org wrote:
> > From: Numan Siddique <numans@ovn.org>
> >
> > It is expected that a provider network logical switch has a localnet logical
> > switch port in order to bridge the overlay traffic to the underlay traffic.
> > There can be some usecases where a overlay logical switch (without
> > a localnet port) can act as a provider network and presently NAT doesn't
> > work as expected.  This patch adds this support.  A new config option
> > "overlay_provider_network" is added to support this feature.
> > This feature gets enabled for a logical switch 'P' if:
> >    - The above option is set to true in the Logical_Switch.other_config
> >      column.
> >    - The logical switch 'P' doesn't have any localnet ports.
> >    - The logical router port of a router 'R' connecting to 'P'
> >      is a gateway router port.
> >    - And the logical router 'R' has only one gateway router port.
> >
> > If all the above conditions are met, ovn-northd creates a chassisredirect
> > port for the logical switch port (of type router) connecting to the
> > router 'R'.  For example, if the logical port is named as "P-R" and its
> > peer router port is "R-P", then chassisredirect port cr-P-R is created
> > along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
> > This ensures that the routing is centralized on this gateway chassis for
> > the traffic coming from switch "P" towards the router or vice versa.
> > This centralization is required in order to support NAT (both SNAT and
> > DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
> > not supported and instead the router port mac is used for such traffic.
> >
> > Reported-at: https://issues.redhat.com/browse/FDP-364
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >   controller/physical.c     |   4 +
> >   northd/northd.c           | 226 +++++++++++++----
> >   northd/northd.h           |   1 +
> >   tests/multinode-macros.at |   2 +-
> >   tests/multinode.at        | 177 +++++++++++++
> >   tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
> >   tests/ovn.at              |   8 +-
> >   7 files changed, 886 insertions(+), 52 deletions(-)
> >
> > diff --git a/controller/physical.c b/controller/physical.c
> > index 7ee3086940..625e37e8a7 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -1587,6 +1587,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                                                       ct_zones);
> >               put_zones_ofpacts(&zone_ids, ofpacts_p);
> >
> > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > +             * go out from the same tunnel inport. */
> > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > +
> >               /* Resubmit to table 41. */
> >               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >           }
> > diff --git a/northd/northd.c b/northd/northd.c
> > index ead54235c8..1942f9f7a1 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -2063,6 +2063,35 @@ parse_lsp_addrs(struct ovn_port *op)
> >       }
> >   }
> >
> > +static struct ovn_port *
> > +create_cr_port(struct ovn_port *op, struct hmap *ports)
> > +{
> > +    char *redirect_name = ovn_chassis_redirect_name(
> > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > +
> > +    struct ovn_datapath *od = op->od;
> > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > +    if (!(crp && crp->sb && crp->sb->datapath == od->sb)) {
> > +        crp = ovn_port_create(ports, redirect_name,
> > +                              op->nbsp, op->nbrp, NULL);
> > +    }
> > +
> > +    crp->l3dgw_port = op;
> > +    op->cr_port = crp;
> > +    crp->od = od;
> > +    free(redirect_name);
> > +
> > +    /* Add to l3dgw_ports in od, for later use during flow creation. */
> > +    if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > +        od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > +                                     &od->n_allocated_l3dgw_ports,
> > +                                     sizeof *od->l3dgw_ports);
> > +    }
> > +    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > +
> > +    return crp;
> > +}
> > +
> >   static void
> >   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > @@ -2170,9 +2199,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> >           }
> >       }
> > +
> > +    struct hmapx gw_ports = HMAPX_INITIALIZER(&gw_ports);
> >       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> >           ovs_assert(od->nbr);
> > -        size_t n_allocated_l3dgw_ports = 0;
> >           for (size_t i = 0; i < od->nbr->n_ports; i++) {
> >               const struct nbrec_logical_router_port *nbrp
> >                   = od->nbr->ports[i];
> > @@ -2236,10 +2266,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                       redirect_type && !strcasecmp(redirect_type, "bridged");
> >               }
> >
> > -            if (op->nbrp->ha_chassis_group ||
> > -                op->nbrp->n_gateway_chassis) {
> > -                /* Additional "derived" ovn_port crp represents the
> > -                 * instance of op on the gateway chassis. */
> > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> >                   const char *gw_chassis = smap_get(&op->od->nbr->options,
> >                                                  "chassis");
> >                   if (gw_chassis) {
> > @@ -2248,34 +2275,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> >                                    "gateway port configured on port %s "
> >                                    "on L3 gateway router", nbrp->name);
> > -                    continue;
> > -                }
> > -
> > -                char *redirect_name =
> > -                    ovn_chassis_redirect_name(nbrp->name);
> > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > -                    ovs_list_remove(&crp->list);
> > -                    ovs_list_push_back(both, &crp->list);
> >                   } else {
> > -                    crp = ovn_port_create(ports, redirect_name,
> > -                                          NULL, nbrp, NULL);
> > -                    ovs_list_push_back(nb_only, &crp->list);
> > -                }
> > -                crp->l3dgw_port = op;
> > -                op->cr_port = crp;
> > -                crp->od = od;
> > -                free(redirect_name);
> > -
> > -                /* Add to l3dgw_ports in od, for later use during flow
> > -                 * creation. */
> > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > -                                                 &n_allocated_l3dgw_ports,
> > -                                                 sizeof *od->l3dgw_ports);
> > +                    hmapx_add(&gw_ports, op);
> >                   }
> > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> >              }
> >           }
> >       }
> > @@ -2332,12 +2334,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                           arp_proxy, op->nbsp->name);
> >                   }
> >               }
> > -
> > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > -                op->enable_router_port_acl = smap_get_bool(
> > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > -            }
> >           } else if (op->nbrp && op->nbrp->peer && !op->l3dgw_port) {
> >               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> >               if (peer) {
> > @@ -2358,6 +2354,65 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >           }
> >       }
> >
> > +    struct hmapx_node *hmapx_node;
> > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > +        op = hmapx_node->data;
> > +        od = op->od;
> > +        ovs_assert(op->nbrp);
> > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > +
> > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > +         * the gateway chassis. */
> > +        struct ovn_port *crp = create_cr_port(op, ports);
> > +        ovs_assert(crp);
> > +        if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > +            ovn_port_set_nb(crp, NULL, op->nbrp);
> > +            ovs_list_remove(&crp->list);
> > +            ovs_list_push_back(both, &crp->list);
> > +        } else {
> > +            ovs_list_push_back(nb_only, &crp->list);
> > +        }
> > +
> > +        if (op->peer && op->peer->nbsp) {
> > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > +            op->peer->enable_router_port_acl = smap_get_bool(
> > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > +        }
> > +    }
> > +
> > +
> > +    /* Create chassisresident port for the gateway router port's peer if
> > +     *  - Gateway router port's router has only one gateway router port and
> > +     *  - Its peer is a logical switch port and
> > +     *  - It's peer's logical switch has no localnet ports.
> > +     *  - Its peer's logical switch has the option overlay_provider_network
> > +     *    is set to true in the other_config column.
> > +     *
> > +     * This is required to support NAT via geneve (for the overlay provider
> > +     * networks) and the routing coming from this logical switch destined to
> > +     * the router port and vice versa is centralized on the gateway chassis.
> > +     *
> > +     * Future enhancement: Support NAT via geneve if the logical router has
> > +     * multiple gateway ports.
> > +     * */
> > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > +        op = hmapx_node->data;
> > +        if (op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > +            && !op->peer->od->n_localnet_ports &&
> > +            smap_get_bool(&op->peer->od->nbs->other_config,
> > +                          "overlay_provider_network", false)) {
> > +            struct ovn_port *crp = create_cr_port(op->peer, ports);
> > +            if (crp && crp->sb && crp->sb->datapath == op->peer->od->sb) {
> > +                ovn_port_set_nb(crp, op->peer->nbsp, NULL);
> > +                ovs_list_remove(&crp->list);
> > +                ovs_list_push_back(both, &crp->list);
> > +            } else {
> > +                ovs_list_push_back(nb_only, &crp->list);
> > +            }
> > +        }
> > +    }
> > +    hmapx_destroy(&gw_ports);
> > +
> >       /* Wait until all ports have been connected to add to IPAM since
> >        * it relies on proper peers to be set
> >        */
> > @@ -3140,16 +3195,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> >                * type "l3gateway". */
> >               if (chassis) {
> >                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> > +            } else if (is_cr_port(op)) {
> > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > +                ovs_assert(op->l3dgw_port->peer);
> > +                ovs_assert(op->l3dgw_port->peer->cr_port);
> > +                ovs_assert(op->l3dgw_port->peer->cr_port->sb);
> > +                sbrec_port_binding_set_ha_chassis_group(
> > +                    op->sb,
> > +                    op->l3dgw_port->peer->cr_port->sb->ha_chassis_group);
> > +
> >               } else {
> >                   sbrec_port_binding_set_type(op->sb, "patch");
> >               }
> >
> >               const char *router_port = smap_get(&op->nbsp->options,
> >                                                  "router-port");
> > -            if (router_port || chassis) {
> > +            if (router_port || chassis || is_cr_port(op)) {
> >                   struct smap new;
> >                   smap_init(&new);
> > -                if (router_port) {
> > +
> > +                if (is_cr_port(op)) {
> > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > +                } else if (router_port) {
> >                       smap_add(&new, "peer", router_port);
> >                   }
> >                   if (chassis) {
> > @@ -8148,9 +8215,18 @@ build_lswitch_rport_arp_req_flow(
> >       struct lflow_ref *lflow_ref)
> >   {
> >       struct ds match   = DS_EMPTY_INITIALIZER;
> > +    struct ds m       = DS_EMPTY_INITIALIZER;
> >       struct ds actions = DS_EMPTY_INITIALIZER;
> >
> > -    arp_nd_ns_match(ips, addr_family, &match);
> > +    arp_nd_ns_match(ips, addr_family, &m);
> > +    ds_clone(&match, &m);
> > +
> > +    bool has_cr_port = patch_op->cr_port;
> > +
> > +    if (has_cr_port) {
> > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                      patch_op->cr_port->json_key);
> > +    }
> >
> >       /* Send a the packet to the router pipeline.  If the switch has non-router
> >        * ports then flood it there as well.
> > @@ -8172,6 +8248,31 @@ build_lswitch_rport_arp_req_flow(
> >                                   lflow_ref);
> >       }
> >
> > +    if (has_cr_port) {
> > +        ds_clear(&match);
> > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > +                      patch_op->cr_port->json_key);
> > +        ds_clear(&actions);
> > +        if (od->n_router_ports != od->nbs->n_ports) {
> > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > +                        patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority + 1, ds_cstr(&match),
> > +                                    ds_cstr(&actions), stage_hint,
> > +                                    lflow_ref);
> > +        } else {
> > +            ds_put_format(&actions, "outport = %s; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority + 1, ds_cstr(&match),
> > +                                    ds_cstr(&actions),
> > +                                    stage_hint,
> > +                                    lflow_ref);
> > +        }
> > +    }
> > +
> > +    ds_destroy(&m);
> >       ds_destroy(&match);
> >       ds_destroy(&actions);
> >   }
> > @@ -9454,7 +9555,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                                   struct ds *actions, struct ds *match)
> >   {
> >       ovs_assert(op->nbsp);
> > -    if (lsp_is_external(op->nbsp)) {
> > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {

> >           return;
> >       }
> >
> > @@ -9466,8 +9567,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                              "outport = \""MC_UNKNOWN "\"; output;"
> >                            : "outport = %s; output;")
> >                            : debug_drop_action();
> > -    ds_clear(actions);
> > -    ds_put_format(actions, action, op->json_key);
> >
> >       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> >           /* For ports connected to logical routers add flows to bypass the
> > @@ -9514,14 +9613,35 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >               if (add_chassis_resident_check) {
> >                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> >               }
> > +        } else if (op->cr_port) {
> > +            ds_clear(actions);
> > +            ds_put_format(actions, action, op->cr_port->json_key);
> > +
> > +            struct ds m = DS_EMPTY_INITIALIZER;
> > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > +                          op->peer->lrp_networks.ea_s,
> > +                          op->cr_port->json_key);
> > +
> > +            ovn_lflow_add_with_hint(lflows, op->od,
> > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > +                                    ds_cstr(&m), ds_cstr(actions),
> > +                                    &op->nbsp->header_,
> > +                                    op->lflow_ref);
> > +            ds_destroy(&m);
> > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > +                          op->cr_port->json_key);
> >           }
> >
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >           ovn_lflow_add_with_hint(lflows, op->od,
> >                                   S_SWITCH_IN_L2_LKUP, 50,
> >                                   ds_cstr(match), ds_cstr(actions),
> >                                   &op->nbsp->header_,
> >                                   op->lflow_ref);
> >       } else {
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> >               ds_clear(match);
> >               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > @@ -11619,6 +11739,15 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> >           return;
> >       }
> >
> > +    if (op->peer && op->peer->cr_port) {
> > +        /* We don't add the below flows if the router port's peer has
> > +         * a chassisresident port.  That's because routing is centralized on
> > +         * the gateway chassis for the traffic from the peer port to this
> > +         * router or from this router to the peer logical switch.
> > +         */
> > +        return;
> > +    }
> > +
> >       /* Mac address to use when replying to ARP/NS. */
> >       const char *mac_s = REG_INPORT_ETH_ADDR;
> >       struct eth_addr mac;
> > @@ -14852,6 +14981,17 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> >       /* For distributed router NAT, determine whether this NAT rule
> >        * satisfies the conditions for distributed NAT processing. */
> >       *distributed = false;
> > +
> > +    /* NAT cannnot be distributed if the gateway port's peer
> > +     * has a chassisresident port (and the routing is centralized
> > +     * on the gateway chassis for the traffic from the peer
> > +     * to this router and traffic to the peer.)
> > +     */
> > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > +        return 0;
> > +    }
> > +
> >       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> >           nat->logical_port && nat->external_mac) {
> >           if (eth_addr_from_string(nat->external_mac, mac)) {
> > diff --git a/northd/northd.h b/northd/northd.h
> > index 18cad5234a..dad3a77673 100644
> > --- a/northd/northd.h
> > +++ b/northd/northd.h
> > @@ -325,6 +325,7 @@ struct ovn_datapath {
> >        * will be NULL. */
> >       struct ovn_port **l3dgw_ports;
> >       size_t n_l3dgw_ports;
> > +    size_t n_allocated_l3dgw_ports;
> >
> >       /* router datapath has a logical port with redirect-type set to bridged. */
> >       bool redirect_bridged;
> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > index c04506a52a..25cfa186ee 100644
> > --- a/tests/multinode-macros.at
> > +++ b/tests/multinode-macros.at
> > @@ -66,7 +66,7 @@ m_count_rows() {
> >   m_check_row_count() {
> >       local db=$(parse_db $1) table=$(parse_table $1); shift
> >       local count=$1; shift
> > -    local found=$(m_count_rows $c $db:$table "$@")
> > +    local found=$(m_count_rows $db:$table "$@")
> >       echo
> >       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> >       if test "$count" != "$found"; then
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index b959a25506..d549bedd66 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -890,4 +890,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.1.2 |
> >
> >   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -c 20 -i 0.5 -s 1300 -M do 172.20.1.2 2>&1 |grep -q "mtu = 1150"])
> >
> > +# Reset back to geneve tunnels
> > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > +do
> > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > +done
> > +
> > +AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +
> > +m_wait_for_ports_up
> > +
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create the second logical switch with one port
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > +
> > +# Create a logical router and attach both logical switches
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > +
> > +# create exteranl connection for N/S traffic
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > +
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > +
> > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > +check multinode_nbctl lsp-add public public-port1
> > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +# First do basic ping tests before deleting the localnet port - ln-public.
> > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > +# is centralized on ovn-gw-1.
> > +
> > +# This function checks the North-South traffic.
> > +run_ns_traffic() {
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +}
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > +
> > +# cr-port should not be created for public-lr0 since the option
> > +# overlay_provider_network=true is not yet set for public.
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Set the option now.
> > +check multinode_nbctl --wait=hv set logical_switch public other_config:overlay_provider_network=true
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Re-add the localnet port
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > +
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the ln-public port this time.
> > +check multinode_nbctl --wait=hv lsp-del ln-public
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> >   AT_CLEANUP
> > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > index 3d944a3aef..451e57d61d 100644
> > --- a/tests/ovn-northd.at
> > +++ b/tests/ovn-northd.at
> > @@ -2061,7 +2061,7 @@ match=(inport == "lrp-public" && nd_ns && nd.target == \$${lb_as_v6}), dnl
> >   action=(nd_na { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> >   ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >   # Test chassis redirect port.
> > @@ -2089,7 +2089,7 @@ action=(xreg0[[0..47]] = 00:00:00:00:01:00; next;)
> >   ])
> >
> >   # Ingress router port is used for ARP reply/NA in lr_in_ip_input.
> > -# xxreg0[0..47] is used unless external_mac is set.
> > +# xxreg0[[0..47]] is used unless external_mac is set.
> >   # Priority 90 flows (per router).
> >   AT_CHECK_UNQUOTED([ovn-sbctl lflow-list | grep -E "lr_in_ip_input.*priority=90" | grep "arp\|nd" | ovn_strip_lflows], [0], [dnl
> >     table=??(lr_in_ip_input     ), priority=90   , dnl
> > @@ -2164,7 +2164,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> >   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> >   ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >   AT_CLEANUP
> > @@ -5434,13 +5434,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> >
> >   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> >
> > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> >
> >   ovn-sbctl lflow-list ls1 > ls1_lflows
> >   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> >     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> >     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> >     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > @@ -12435,3 +12436,512 @@ check_engine_stats northd recompute nocompute
> >   check_engine_stats lflow recompute nocompute
> >
> >   AT_CLEANUP
> > +
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([NAT on a provider network with no localnet ports])
> > +AT_KEYWORDS([dnat])
> > +ovn_start
> > +
> > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lsp-add public pub-p1
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > +
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > +
> > +check ovn-nbctl --wait=sb sync
> > +
> > +check_flows_no_cr_port_for_public_lr0() {
> > +  # check that there is no port binding cr-public-lr0
> > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +check_flows_cr_port_for_public_lr0() {
> > +  # check that there is port binding cr-public-lr0
> > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Remove the localnet port from public logical switch.
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > +
> > +# Check that the lflows are as expected and there is no cr port
> > +# created for "public-lr0"  when public has no localnet port
> > +# since public doesn't have the option "overlay_provider_network=true"
> > +# set.
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +
> > +# Set the option "overlay_provider_network=true" for public.
> > +check ovn-nbctl --wait=sb set logical_switch public other_config:overlay_provider_network=true
> > +
> > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Set the type of ln-public back to localnet
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Delete the localnet port
> > +check ovn-nbctl --wait=sb lsp-del ln-public
> > +
> > +# Check that the lflows are as expected when public has no localnet port.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Create multiple gateway ports.  chassisresident port should not be
> > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > +# logical switch.
> > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > +# check that there is no port binding cr-public-lr0
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CLEANUP
> > +])
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index e81cd4f45a..8b1d3c846f 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -21181,10 +21181,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> >       type=router options:router-port=sw0 \
> >       -- lsp-set-addresses rp-sw0 router
> >
> > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > -    -- lrp-set-gateway-chassis sw1 hv2
> > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> >   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > -    type=router options:router-port=sw1 \
> > +    type=router options:router-port=lr0-sw1 \
> >       -- lsp-set-addresses rp-sw1 router
> >
> >   ovn-nbctl lsp-add sw0 sw0-p0 \
> > @@ -21196,6 +21196,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> >   ovn-nbctl lsp-add sw1 sw1-p0 \
> >       -- lsp-set-addresses sw1-p0 unknown
> >
> > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > +
> >   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> >   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> >
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

Numan Siddique May 8, 2024, 9:37 p.m. UTC | #5

On Wed, Apr 24, 2024 at 7:46 PM Numan Siddique <numans@ovn.org> wrote:
>
> On Wed, Apr 24, 2024 at 5:14 PM Mark Michelson <mmichels@redhat.com> wrote:
> >
> > Hi Numan,
> >
> > I haven't done a full review of this yet, but I figured I'd give some
> > initial feedback from what I had looked at.
> >
> > At a high level, this is missing documentation in ovn-nb.xml for the new
> > "overlay_provider_network" option. There should also be a NEWS entry for
> > the new functionality.
> >
>
> Thanks for the comments.   Yes.  I missed adding them.  I'll address it in v2.
>
> >
> >
> > See below for a couple more notes.
> >
> > On 4/23/24 12:43, numans@ovn.org wrote:
> > > From: Numan Siddique <numans@ovn.org>
> > >
> > > It is expected that a provider network logical switch has a localnet logical
> > > switch port in order to bridge the overlay traffic to the underlay traffic.
> > > There can be some usecases where a overlay logical switch (without
> > > a localnet port) can act as a provider network and presently NAT doesn't
> > > work as expected.  This patch adds this support.  A new config option
> > > "overlay_provider_network" is added to support this feature.
> > > This feature gets enabled for a logical switch 'P' if:
> > >    - The above option is set to true in the Logical_Switch.other_config
> > >      column.
> > >    - The logical switch 'P' doesn't have any localnet ports.
> > >    - The logical router port of a router 'R' connecting to 'P'
> > >      is a gateway router port.
> > >    - And the logical router 'R' has only one gateway router port.
> > >
> > > If all the above conditions are met, ovn-northd creates a chassisredirect
> > > port for the logical switch port (of type router) connecting to the
> > > router 'R'.  For example, if the logical port is named as "P-R" and its
> > > peer router port is "R-P", then chassisredirect port cr-P-R is created
> > > along with cr-R-P.  Gateway chassis binding the cr-R-P also binds cr-P-R.
> > > This ensures that the routing is centralized on this gateway chassis for
> > > the traffic coming from switch "P" towards the router or vice versa.
> > > This centralization is required in order to support NAT (both SNAT and
> > > DNAT).  Distributed NAT (i.e if external_mac and router_port is set) is
> > > not supported and instead the router port mac is used for such traffic.
> > >
> > > Reported-at: https://issues.redhat.com/browse/FDP-364
> > > Signed-off-by: Numan Siddique <numans@ovn.org>
> > > ---
> > >   controller/physical.c     |   4 +
> > >   northd/northd.c           | 226 +++++++++++++----
> > >   northd/northd.h           |   1 +
> > >   tests/multinode-macros.at |   2 +-
> > >   tests/multinode.at        | 177 +++++++++++++
> > >   tests/ovn-northd.at       | 520 +++++++++++++++++++++++++++++++++++++-
> > >   tests/ovn.at              |   8 +-
> > >   7 files changed, 886 insertions(+), 52 deletions(-)
> > >
> > > diff --git a/controller/physical.c b/controller/physical.c
> > > index 7ee3086940..625e37e8a7 100644
> > > --- a/controller/physical.c
> > > +++ b/controller/physical.c
> > > @@ -1587,6 +1587,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > >                                                       ct_zones);
> > >               put_zones_ofpacts(&zone_ids, ofpacts_p);
> > >
> > > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > > +             * go out from the same tunnel inport. */
> > > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > > +
> > >               /* Resubmit to table 41. */
> > >               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> > >           }
> > > diff --git a/northd/northd.c b/northd/northd.c
> > > index ead54235c8..1942f9f7a1 100644
> > > --- a/northd/northd.c
> > > +++ b/northd/northd.c
> > > @@ -2063,6 +2063,35 @@ parse_lsp_addrs(struct ovn_port *op)
> > >       }
> > >   }
> > >
> > > +static struct ovn_port *
> > > +create_cr_port(struct ovn_port *op, struct hmap *ports)
> > > +{
> > > +    char *redirect_name = ovn_chassis_redirect_name(
> > > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > > +
> > > +    struct ovn_datapath *od = op->od;
> > > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > +    if (!(crp && crp->sb && crp->sb->datapath == od->sb)) {
> > > +        crp = ovn_port_create(ports, redirect_name,
> > > +                              op->nbsp, op->nbrp, NULL);
> > > +    }
> > > +
> > > +    crp->l3dgw_port = op;
> > > +    op->cr_port = crp;
> > > +    crp->od = od;
> > > +    free(redirect_name);
> > > +
> > > +    /* Add to l3dgw_ports in od, for later use during flow creation. */
> > > +    if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > > +        od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > +                                     &od->n_allocated_l3dgw_ports,
> > > +                                     sizeof *od->l3dgw_ports);
> > > +    }
> > > +    od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > > +
> > > +    return crp;
> > > +}
> > > +
> > >   static void
> > >   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > > @@ -2170,9 +2199,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> > >           }
> > >       }
> > > +
> > > +    struct hmapx gw_ports = HMAPX_INITIALIZER(&gw_ports);
> > >       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> > >           ovs_assert(od->nbr);
> > > -        size_t n_allocated_l3dgw_ports = 0;
> > >           for (size_t i = 0; i < od->nbr->n_ports; i++) {
> > >               const struct nbrec_logical_router_port *nbrp
> > >                   = od->nbr->ports[i];
> > > @@ -2236,10 +2266,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                       redirect_type && !strcasecmp(redirect_type, "bridged");
> > >               }
> > >
> > > -            if (op->nbrp->ha_chassis_group ||
> > > -                op->nbrp->n_gateway_chassis) {
> > > -                /* Additional "derived" ovn_port crp represents the
> > > -                 * instance of op on the gateway chassis. */
> > > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> > >                   const char *gw_chassis = smap_get(&op->od->nbr->options,
> > >                                                  "chassis");
> > >                   if (gw_chassis) {
> > > @@ -2248,34 +2275,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> > >                                    "gateway port configured on port %s "
> > >                                    "on L3 gateway router", nbrp->name);
> > > -                    continue;
> > > -                }
> > > -
> > > -                char *redirect_name =
> > > -                    ovn_chassis_redirect_name(nbrp->name);
> > > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > > -                    ovs_list_remove(&crp->list);
> > > -                    ovs_list_push_back(both, &crp->list);
> > >                   } else {
> > > -                    crp = ovn_port_create(ports, redirect_name,
> > > -                                          NULL, nbrp, NULL);
> > > -                    ovs_list_push_back(nb_only, &crp->list);
> > > -                }
> > > -                crp->l3dgw_port = op;
> > > -                op->cr_port = crp;
> > > -                crp->od = od;
> > > -                free(redirect_name);
> > > -
> > > -                /* Add to l3dgw_ports in od, for later use during flow
> > > -                 * creation. */
> > > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > -                                                 &n_allocated_l3dgw_ports,
> > > -                                                 sizeof *od->l3dgw_ports);
> > > +                    hmapx_add(&gw_ports, op);
> > >                   }
> > > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > >              }
> > >           }
> > >       }
> > > @@ -2332,12 +2334,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                           arp_proxy, op->nbsp->name);
> > >                   }
> > >               }
> > > -
> > > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > > -                op->enable_router_port_acl = smap_get_bool(
> > > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > > -            }
> > >           } else if (op->nbrp && op->nbrp->peer && !op->l3dgw_port) {
> > >               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> > >               if (peer) {
> > > @@ -2358,6 +2354,65 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >           }
> > >       }
> > >
> > > +    struct hmapx_node *hmapx_node;
> > > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > > +        op = hmapx_node->data;
> > > +        od = op->od;
> > > +        ovs_assert(op->nbrp);
> > > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > > +
> > > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > > +         * the gateway chassis. */
> > > +        struct ovn_port *crp = create_cr_port(op, ports);
> > > +        ovs_assert(crp);
> > > +        if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > > +            ovn_port_set_nb(crp, NULL, op->nbrp);
> > > +            ovs_list_remove(&crp->list);
> > > +            ovs_list_push_back(both, &crp->list);
> > > +        } else {
> > > +            ovs_list_push_back(nb_only, &crp->list);
> > > +        }
> > > +
> > > +        if (op->peer && op->peer->nbsp) {
> > > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > > +            op->peer->enable_router_port_acl = smap_get_bool(
> > > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > > +        }
> > > +    }
> > > +
> > > +
> > > +    /* Create chassisresident port for the gateway router port's peer if
> > > +     *  - Gateway router port's router has only one gateway router port and
> > > +     *  - Its peer is a logical switch port and
> > > +     *  - It's peer's logical switch has no localnet ports.
> > > +     *  - Its peer's logical switch has the option overlay_provider_network
> > > +     *    is set to true in the other_config column.
> > > +     *
> > > +     * This is required to support NAT via geneve (for the overlay provider
> > > +     * networks) and the routing coming from this logical switch destined to
> > > +     * the router port and vice versa is centralized on the gateway chassis.
> > > +     *
> > > +     * Future enhancement: Support NAT via geneve if the logical router has
> > > +     * multiple gateway ports.
> > > +     * */
> > > +    HMAPX_FOR_EACH (hmapx_node, &gw_ports) {
> > > +        op = hmapx_node->data;
> > > +        if (op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > > +            && !op->peer->od->n_localnet_ports &&
> > > +            smap_get_bool(&op->peer->od->nbs->other_config,
> > > +                          "overlay_provider_network", false)) {
> > > +            struct ovn_port *crp = create_cr_port(op->peer, ports);
> > > +            if (crp && crp->sb && crp->sb->datapath == op->peer->od->sb) {
> > > +                ovn_port_set_nb(crp, op->peer->nbsp, NULL);
> > > +                ovs_list_remove(&crp->list);
> > > +                ovs_list_push_back(both, &crp->list);
> > > +            } else {
> > > +                ovs_list_push_back(nb_only, &crp->list);
> > > +            }
> > > +        }
> > > +    }
> > > +    hmapx_destroy(&gw_ports);
> > > +
> > >       /* Wait until all ports have been connected to add to IPAM since
> > >        * it relies on proper peers to be set
> > >        */
> > > @@ -3140,16 +3195,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> > >                * type "l3gateway". */
> > >               if (chassis) {
> > >                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> > > +            } else if (is_cr_port(op)) {
> > > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > > +                ovs_assert(op->l3dgw_port->peer);
> > > +                ovs_assert(op->l3dgw_port->peer->cr_port);
> > > +                ovs_assert(op->l3dgw_port->peer->cr_port->sb);
> > > +                sbrec_port_binding_set_ha_chassis_group(
> > > +                    op->sb,
> > > +                    op->l3dgw_port->peer->cr_port->sb->ha_chassis_group);
> > > +
> > >               } else {
> > >                   sbrec_port_binding_set_type(op->sb, "patch");
> > >               }
> > >
> > >               const char *router_port = smap_get(&op->nbsp->options,
> > >                                                  "router-port");
> > > -            if (router_port || chassis) {
> > > +            if (router_port || chassis || is_cr_port(op)) {
> > >                   struct smap new;
> > >                   smap_init(&new);
> > > -                if (router_port) {
> > > +
> > > +                if (is_cr_port(op)) {
> > > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > > +                } else if (router_port) {
> > >                       smap_add(&new, "peer", router_port);
> > >                   }
> > >                   if (chassis) {
> > > @@ -8148,9 +8215,18 @@ build_lswitch_rport_arp_req_flow(
> > >       struct lflow_ref *lflow_ref)
> > >   {
> > >       struct ds match   = DS_EMPTY_INITIALIZER;
> > > +    struct ds m       = DS_EMPTY_INITIALIZER;
> > >       struct ds actions = DS_EMPTY_INITIALIZER;
> > >
> > > -    arp_nd_ns_match(ips, addr_family, &match);
> > > +    arp_nd_ns_match(ips, addr_family, &m);
> > > +    ds_clone(&match, &m);
> > > +
> > > +    bool has_cr_port = patch_op->cr_port;
> > > +
> > > +    if (has_cr_port) {
> > > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > > +                      patch_op->cr_port->json_key);
> > > +    }
> > >
> > >       /* Send a the packet to the router pipeline.  If the switch has non-router
> > >        * ports then flood it there as well.
> > > @@ -8172,6 +8248,31 @@ build_lswitch_rport_arp_req_flow(
> > >                                   lflow_ref);
> > >       }
> > >
> > > +    if (has_cr_port) {
> > > +        ds_clear(&match);
> > > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > > +                      patch_op->cr_port->json_key);
> > > +        ds_clear(&actions);
> > > +        if (od->n_router_ports != od->nbs->n_ports) {
> > > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > > +                        patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority + 1, ds_cstr(&match),
> >
> > Why is this flow (and the one below) added with priority + 1? Since the
> > match conditions are different from the flows that are installed earlier
> > in this function, this can be installed at the same priority, right?
>
>
> You're right.  It can be installed at the same prio.  I don't think I had any
> particular reason for bumping the prio + 1.  I'll address it in v2.
>
> >
> > > +                                    ds_cstr(&actions), stage_hint,
> > > +                                    lflow_ref);
> > > +        } else {
> > > +            ds_put_format(&actions, "outport = %s; output;",
> > > +                          patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority + 1, ds_cstr(&match),
> > > +                                    ds_cstr(&actions),
> > > +                                    stage_hint,
> > > +                                    lflow_ref);
> > > +        }
> > > +    }
> > > +
> > > +    ds_destroy(&m);
> > >       ds_destroy(&match);
> > >       ds_destroy(&actions);
> > >   }
> > > @@ -9454,7 +9555,7 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >                                   struct ds *actions, struct ds *match)
> > >   {
> > >       ovs_assert(op->nbsp);
> > > -    if (lsp_is_external(op->nbsp)) {
> > > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> >
> > Is is_cr_port() the correct function to use here? is_cr_port() returns
> > true if op->l3dgw_port is non-NULL. In this case, since op represents a
> > logical switch port, is_cr_port(op) will always return false. The
> > is_l3dgw_port() function returns true if op->cr_port is non-NULL, but
> > that is an unintuitive function to call for a logical switch port. I
> > think checking for op->cr_port directly would be most appropriate here.
>
>
> Agree.  I thought about using it here too.  If you see I've used op->cr_port
> in other places in this patch.

On second thought,  we can't actually check for op->cr_port as
op->cr_port wll be NULL
for  a chassis resident port.  This above check is added to skip
adding the unicast lookup flows
for the chassis resident port (which is SB only) of the logical switch
port of type "patch".
Please check this patch which improves the documentation of
is_cr_port() and is_l3dgw_port().
https://patchwork.ozlabs.org/project/ovn/patch/20240507215713.902148-1-numans@ovn.org/

Thanks
Numan


>
> >
> > Also, on a separate note, why in the world do is_cr_port() and
> > is_l3dgw_port() do seemingly the opposite of what their function names
> > imply?
>
>
> I had the same doubt.
>
> A gateway port means it has ha_chassis or gateway chassis
> configured.  Which means ovn-northd will create a chassisredirect port
> and stores this
> reference in op->cr_port of  a gateway port.  And when chassisredirect
> 'ovn_port' is created,
> its corresponding gateway port reference is stored in 'op->l3dgw_port'.
>
> So the function is_cr_port(op) checks for op->l3dgw_port.  If it is
> set, it means 'op' is a chassisredirect port.
> And the function is_l3dgw_port(op) checks for op->cr_port.  If it is
> set,  it means 'op' is a gateway router port.
>
> Although it seems a bit confusing,  the functions don't do anything
> opposite.  It's just a way to figure out
> the type.
>
> Thanks
> Numan
>
>
> >
> >
> > >           return;
> > >       }
> > >
> > > @@ -9466,8 +9567,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >                              "outport = \""MC_UNKNOWN "\"; output;"
> > >                            : "outport = %s; output;")
> > >                            : debug_drop_action();
> > > -    ds_clear(actions);
> > > -    ds_put_format(actions, action, op->json_key);
> > >
> > >       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> > >           /* For ports connected to logical routers add flows to bypass the
> > > @@ -9514,14 +9613,35 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >               if (add_chassis_resident_check) {
> > >                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> > >               }
> > > +        } else if (op->cr_port) {
> > > +            ds_clear(actions);
> > > +            ds_put_format(actions, action, op->cr_port->json_key);
> > > +
> > > +            struct ds m = DS_EMPTY_INITIALIZER;
> > > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > > +                          op->peer->lrp_networks.ea_s,
> > > +                          op->cr_port->json_key);
> > > +
> > > +            ovn_lflow_add_with_hint(lflows, op->od,
> > > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > > +                                    ds_cstr(&m), ds_cstr(actions),
> > > +                                    &op->nbsp->header_,
> > > +                                    op->lflow_ref);
> > > +            ds_destroy(&m);
> > > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > > +                          op->cr_port->json_key);
> > >           }
> > >
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >           ovn_lflow_add_with_hint(lflows, op->od,
> > >                                   S_SWITCH_IN_L2_LKUP, 50,
> > >                                   ds_cstr(match), ds_cstr(actions),
> > >                                   &op->nbsp->header_,
> > >                                   op->lflow_ref);
> > >       } else {
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> > >               ds_clear(match);
> > >               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > > @@ -11619,6 +11739,15 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> > >           return;
> > >       }
> > >
> > > +    if (op->peer && op->peer->cr_port) {
> > > +        /* We don't add the below flows if the router port's peer has
> > > +         * a chassisresident port.  That's because routing is centralized on
> > > +         * the gateway chassis for the traffic from the peer port to this
> > > +         * router or from this router to the peer logical switch.
> > > +         */
> > > +        return;
> > > +    }
> > > +
> > >       /* Mac address to use when replying to ARP/NS. */
> > >       const char *mac_s = REG_INPORT_ETH_ADDR;
> > >       struct eth_addr mac;
> > > @@ -14852,6 +14981,17 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> > >       /* For distributed router NAT, determine whether this NAT rule
> > >        * satisfies the conditions for distributed NAT processing. */
> > >       *distributed = false;
> > > +
> > > +    /* NAT cannnot be distributed if the gateway port's peer
> > > +     * has a chassisresident port (and the routing is centralized
> > > +     * on the gateway chassis for the traffic from the peer
> > > +     * to this router and traffic to the peer.)
> > > +     */
> > > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > > +        return 0;
> > > +    }
> > > +
> > >       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> > >           nat->logical_port && nat->external_mac) {
> > >           if (eth_addr_from_string(nat->external_mac, mac)) {
> > > diff --git a/northd/northd.h b/northd/northd.h
> > > index 18cad5234a..dad3a77673 100644
> > > --- a/northd/northd.h
> > > +++ b/northd/northd.h
> > > @@ -325,6 +325,7 @@ struct ovn_datapath {
> > >        * will be NULL. */
> > >       struct ovn_port **l3dgw_ports;
> > >       size_t n_l3dgw_ports;
> > > +    size_t n_allocated_l3dgw_ports;
> > >
> > >       /* router datapath has a logical port with redirect-type set to bridged. */
> > >       bool redirect_bridged;
> > > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > > index c04506a52a..25cfa186ee 100644
> > > --- a/tests/multinode-macros.at
> > > +++ b/tests/multinode-macros.at
> > > @@ -66,7 +66,7 @@ m_count_rows() {
> > >   m_check_row_count() {
> > >       local db=$(parse_db $1) table=$(parse_table $1); shift
> > >       local count=$1; shift
> > > -    local found=$(m_count_rows $c $db:$table "$@")
> > > +    local found=$(m_count_rows $db:$table "$@")
> > >       echo
> > >       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> > >       if test "$count" != "$found"; then
> > > diff --git a/tests/multinode.at b/tests/multinode.at
> > > index b959a25506..d549bedd66 100644
> > > --- a/tests/multinode.at
> > > +++ b/tests/multinode.at
> > > @@ -890,4 +890,181 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.1.2 |
> > >
> > >   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -c 20 -i 0.5 -s 1300 -M do 172.20.1.2 2>&1 |grep -q "mtu = 1150"])
> > >
> > > +# Reset back to geneve tunnels
> > > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > > +do
> > > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > > +done
> > > +
> > > +AT_CLEANUP
> > > +
> > > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > > +
> > > +# Check that ovn-fake-multinode setup is up and running
> > > +check_fake_multinode_setup
> > > +
> > > +# Delete the multinode NB and OVS resources before starting the test.
> > > +cleanup_multinode_resources
> > > +
> > > +check multinode_nbctl ls-add sw0
> > > +check multinode_nbctl lsp-add sw0 sw0-port1
> > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > > +check multinode_nbctl lsp-add sw0 sw0-port2
> > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > > +
> > > +m_wait_for_ports_up
> > > +
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# Create the second logical switch with one port
> > > +check multinode_nbctl ls-add sw1
> > > +check multinode_nbctl lsp-add sw1 sw1-port1
> > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > > +
> > > +# Create a logical router and attach both logical switches
> > > +check multinode_nbctl lr-add lr0
> > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > > +
> > > +# create exteranl connection for N/S traffic
> > > +check multinode_nbctl ls-add public
> > > +check multinode_nbctl lsp-add public ln-public
> > > +check multinode_nbctl lsp-set-type ln-public localnet
> > > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > > +check multinode_nbctl lsp-add public public-lr0
> > > +check multinode_nbctl lsp-set-type public-lr0 router
> > > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > > +
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > > +
> > > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > > +check multinode_nbctl lsp-add public public-port1
> > > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +
> > > +# First do basic ping tests before deleting the localnet port - ln-public.
> > > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > > +# is centralized on ovn-gw-1.
> > > +
> > > +# This function checks the North-South traffic.
> > > +run_ns_traffic() {
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +}
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > > +
> > > +# cr-port should not be created for public-lr0 since the option
> > > +# overlay_provider_network=true is not yet set for public.
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Set the option now.
> > > +check multinode_nbctl --wait=hv set logical_switch public other_config:overlay_provider_network=true
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Re-add the localnet port
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > > +
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the ln-public port this time.
> > > +check multinode_nbctl --wait=hv lsp-del ln-public
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > >   AT_CLEANUP
> > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > > index 3d944a3aef..451e57d61d 100644
> > > --- a/tests/ovn-northd.at
> > > +++ b/tests/ovn-northd.at
> > > @@ -2061,7 +2061,7 @@ match=(inport == "lrp-public" && nd_ns && nd.target == \$${lb_as_v6}), dnl
> > >   action=(nd_na { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > >   ])
> > >
> > > -# xreg0[0..47] isn't used anywhere else.
> > > +# xreg0[[0..47]] isn't used anywhere else.
> > >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> > >
> > >   # Test chassis redirect port.
> > > @@ -2089,7 +2089,7 @@ action=(xreg0[[0..47]] = 00:00:00:00:01:00; next;)
> > >   ])
> > >
> > >   # Ingress router port is used for ARP reply/NA in lr_in_ip_input.
> > > -# xxreg0[0..47] is used unless external_mac is set.
> > > +# xxreg0[[0..47]] is used unless external_mac is set.
> > >   # Priority 90 flows (per router).
> > >   AT_CHECK_UNQUOTED([ovn-sbctl lflow-list | grep -E "lr_in_ip_input.*priority=90" | grep "arp\|nd" | ovn_strip_lflows], [0], [dnl
> > >     table=??(lr_in_ip_input     ), priority=90   , dnl
> > > @@ -2164,7 +2164,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> > >   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > >   ])
> > >
> > > -# xreg0[0..47] isn't used anywhere else.
> > > +# xreg0[[0..47]] isn't used anywhere else.
> > >   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> > >
> > >   AT_CLEANUP
> > > @@ -5434,13 +5434,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> > >
> > >   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> > >
> > > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> > >
> > >   ovn-sbctl lflow-list ls1 > ls1_lflows
> > >   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> > >     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > >     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> > >     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> > >     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > >     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > @@ -12435,3 +12436,512 @@ check_engine_stats northd recompute nocompute
> > >   check_engine_stats lflow recompute nocompute
> > >
> > >   AT_CLEANUP
> > > +
> > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > +AT_SETUP([NAT on a provider network with no localnet ports])
> > > +AT_KEYWORDS([dnat])
> > > +ovn_start
> > > +
> > > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > > +check ovn-nbctl lsp-add sw0 sw0-port1
> > > +check ovn-nbctl lr-add lr0
> > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > > +check ovn-nbctl ls-add public
> > > +check ovn-nbctl lsp-add public pub-p1
> > > +
> > > +# localnet port
> > > +check ovn-nbctl lsp-add public ln-public
> > > +check ovn-nbctl lsp-set-type ln-public localnet
> > > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > > +
> > > +check ovn-nbctl lsp-add public public-lr0
> > > +check ovn-nbctl lsp-set-type public-lr0 router
> > > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > > +
> > > +check ovn-nbctl --wait=sb sync
> > > +
> > > +check_flows_no_cr_port_for_public_lr0() {
> > > +  # check that there is no port binding cr-public-lr0
> > > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +check_flows_cr_port_for_public_lr0() {
> > > +  # check that there is port binding cr-public-lr0
> > > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=81   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Remove the localnet port from public logical switch.
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > > +
> > > +# Check that the lflows are as expected and there is no cr port
> > > +# created for "public-lr0"  when public has no localnet port
> > > +# since public doesn't have the option "overlay_provider_network=true"
> > > +# set.
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +
> > > +# Set the option "overlay_provider_network=true" for public.
> > > +check ovn-nbctl --wait=sb set logical_switch public other_config:overlay_provider_network=true
> > > +
> > > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Set the type of ln-public back to localnet
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Delete the localnet port
> > > +check ovn-nbctl --wait=sb lsp-del ln-public
> > > +
> > > +# Check that the lflows are as expected when public has no localnet port.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Create multiple gateway ports.  chassisresident port should not be
> > > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > > +# logical switch.
> > > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > > +# check that there is no port binding cr-public-lr0
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CLEANUP
> > > +])
> > > diff --git a/tests/ovn.at b/tests/ovn.at
> > > index e81cd4f45a..8b1d3c846f 100644
> > > --- a/tests/ovn.at
> > > +++ b/tests/ovn.at
> > > @@ -21181,10 +21181,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> > >       type=router options:router-port=sw0 \
> > >       -- lsp-set-addresses rp-sw0 router
> > >
> > > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > > -    -- lrp-set-gateway-chassis sw1 hv2
> > > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> > >   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > > -    type=router options:router-port=sw1 \
> > > +    type=router options:router-port=lr0-sw1 \
> > >       -- lsp-set-addresses rp-sw1 router
> > >
> > >   ovn-nbctl lsp-add sw0 sw0-p0 \
> > > @@ -21196,6 +21196,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> > >   ovn-nbctl lsp-add sw1 sw1-p0 \
> > >       -- lsp-set-addresses sw1-p0 unknown
> > >
> > > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > > +
> > >   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> > >   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> > >
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> >

[ovs-dev,v1,2/2] Add support for overlay provider networks.

Checks

Commit Message

Comments

Patch